Mathias Rousset’s research while affiliated with French National Centre for Scientific Research and other places

This work proposes an adaptive sequential Monte Carlo sampling algorithm for solving inverse Bayesian problems in a context where a (costly) likelihood evaluation can be approximated by a surrogate, constructed from previous evaluations of the true likelihood. A rough error estimation of the obtained surrogates is required. The method is based on an adaptive sequential Monte-Carlo (SMC) simulation that jointly adapts the likelihood approximations and a standard tempering scheme of the target posterior distribution. This algorithm is well-suited to cases where the posterior is concentrated in a rare and unknown region of the prior. It is also suitable for solving low-temperature and rare-event simulation problems. The main contribution is to propose an entropy criteria that associates to the accuracy of the current surrogate a maximum inverse temperature for the likelihood approximation. The latter is used to sample a so-called snapshot, perform an exact likelihood evaluation, and update the surrogate and its error quantification. Some consistency results are presented in an idealized framework of the proposed algorithm. Our numerical experiments use in particular a reduced basis approach to construct approximate parametric solutions of a partially observed solution of an elliptic Partial Differential Equation. They demonstrate the convergence of the algorithm and show a significant cost reduction (close to a factor 10) for comparable accuracy.

Atmospheric motion vectors (AMVs) extracted from satellite imagery are the only wind observations with good global coverage. They are important features for feeding numerical weather prediction (NWP) models. Several Bayesian models have been proposed to estimate AMVs. Although critical for correct assimilation into NWP models, very few methods provide a thorough characterization of the estimation errors. The difficulty of estimating errors stems from the specificity of the posterior distribution, which is both very high dimensional, and highly ill-conditioned due to a singular likelihood, which becomes critical in particular in the case of missing data (unobserved pixels). Motivated by this difficult inverse problem, this work studies the evaluation of the (expected) estimation errors using gradient-based Markov chain Monte Carlo (MCMC) algorithms. The main contribution is to propose a general strategy, called here ‘chilling’, which amounts to sampling a local approximation of the posterior distribution in the neighborhood of a point estimate. From a theoretical point of view, we show that under regularity assumptions, the family of chilled posterior distributions converges in distribution as temperature decreases to an optimal Gaussian approximation at a point estimate given by the maximum a posteriori, also known as the Laplace approximation. Chilled sampling therefore provides access to this approximation generally out of reach in such high-dimensional nonlinear contexts. From an empirical perspective, we evaluate the proposed approach based on some quantitative Bayesian criteria. Our numerical simulations are performed on synthetic and real meteorological data. They reveal that not only the proposed chilling exhibits a significant gain in terms of accuracy of the AMV point estimates and of their associated expected error estimates, but also a substantial acceleration in the convergence speed of the MCMC algorithms.

Deep neural networks are robust against random corruptions of the inputs to some extent. This global sense of safety is not sufficient in critical applications where probabilities of failure must be assessed with accuracy. Some previous works applied known statistical methods from the field of rare event analysis to classification. Yet, they use classifiers as black-box models without taking into account gradient information, readily available for deep learning models via auto-differentiation. We propose a new and highly efficient estimator of probabilities of failure dedicated to neural networks as it leverages the fast computation of gradients of the model through back-propagation.

Diffusion processes with small noise conditioned to reach a target set are considered. The AMS algorithm is a Monte Carlo method that is used to sample such rare events by iteratively simulating clones of the process and selecting trajectories that have reached the highest value of a so-called importance function. In this paper, the large sample size relative variance of the AMS small probability estimator is considered. The main result is a large deviations logarithmic equivalent of the latter in the small noise asymptotics, which is rigorously derived. It is given as a maximisation problem explicit in terms of the quasi-potential cost function associated with the underlying small noise large deviations. Necessary and sufficient geometric conditions ensuring the vanishing of the obtained quantity ('weak' asymptotic efficiency) are provided. Interpretations and practical consequences are discussed.

Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to proposal distributions. Using a reference measure as a reference for cost, we prove under some general conditions that the worst-case optimal proposal is precisely given by the distribution minimizing entropy with respect to the reference within the considered convex class of distributions. The latter conditions are in particular satisfied when the convex class is defined using a push-forward map defining atomless conditional measures. Applications in which the optimal proposal is Gibbsian and can be practically sampled using Monte Carlo methods are discussed.

Atmospheric motion vectors (AMVs) extracted from satellite imagery are the only wind observations with good global coverage. They are important features for feeding numerical weather prediction (NWP) models. Several Bayesian models have been proposed to estimate AMVs. Although critical for correct assimilation into NWP models, very few methods provide a thorough characterization of the estimation errors. The difficulty of estimating errors stems from the specificity of the posterior distribution, which is both very high dimensional, and highly ill-conditioned due to a singular likelihood, which becomes critical in particular in the case of missing data (unobserved pixels). This work studies the evaluation of the expected error of AMVs using gradient-based Markov Chain Monte Carlo (MCMC) algorithms. Our main contribution is to propose a tempering strategy, which amounts to sampling a local approximation of the joint posterior distribution of AMVs and image variables in the neighborhood of a point estimate. In addition, we provide efficient preconditioning with the covariance related to the prior family itself (fractional Brownian motion), with possibly different hyper-parameters. From a theoretical point of view, we show that under regularity assumptions, the family of tempered posterior distributions converges in distribution as temperature decreases to an {optimal} Gaussian approximation at a point estimate given by the Maximum A Posteriori (MAP) log-density. From an empirical perspective, we evaluate the proposed approach based on some quantitative Bayesian evaluation criteria. Our numerical simulations performed on synthetic and real meteorological data reveal a significant gain in terms of accuracy of the AMV point estimates and of their associated expected error estimates, but also a substantial acceleration in the convergence speed of the MCMC algorithms.

This work introduces and studies a new family of velocity jump Markov processes directly amenable to exact simulation with the following two properties: (i) trajectories converge in law, when a time-step parameter vanishes, towards a given Langevin or Hamiltonian dynamics; (ii) the stationary distribution of the process is always exactly given by the product of a Gaussian (for velocities) by any target log-density. The simulation itself, in addition to the computability of the gradient of the log-density, depends on the knowledge of appropriate explicit upper bounds on lower order derivatives of this log-density. The process does not exhibit any velocity reflections (maximum size of jumps can be controlled) and is suitable for the ’factorization method’. We provide rigorous mathematical proofs of the convergence towards Hamiltonian/Langevin dynamics when the time step vanishes, and of the exponentially fast convergence towards the target distribution when a suitable noise on velocities is present. Numerical implementation is detailed and illustrated.

We quantify the robustness of a trained network to input uncertainties with a stochastic simulation inspired by the field of Statistical Reliability Engineering. The robustness assessment is cast as a statistical hypothesis test: the network is deemed as locally robust if the estimated probability of failure is lower than a critical level. The procedure is based on an Importance Splitting simulation generating samples of rare events. We derive theoretical guarantees that are non-asymptotic w.r.t. sample size. Experiments tackling large scale networks outline the efficiency of our method making a low number of calls to the network function.

6th European Cyber Week (6ème semaine européenne de la cybersécurité et de la cyberdéfense)