Kody Law

• PhD Mathematics
• Chair at The University of Manchester

This book provides a systematic treatment of the mathematical underpinnings of work in data assimilation, covering both theoretical and computational approaches. Specifically the authors develop a unified mathematical framework in which a Bayesian formulation of the problem provides the bedrock for the derivation, development and analysis of algori...

In this paper we consider sequential joint state and static parameter estimation given discrete time observations associated to a partially observed stochastic partial differential equation (SPDE). It is assumed that one can only estimate the hidden state using a discretization of the model. In this context, it is known that the multi-index Monte C...

We propose a dimension reduction technique for Bayesian inverse problems with nonlinear forward operators, non-Gaussian priors, and non-Gaussian observation noise. The likelihood function is approximated by a ridge function, i.e., a map which depends non-trivially only on a few linear combinations of the parameters. We build this ridge approximatio...

We develop an importance sampling (IS) type estimator for Bayesian joint inference on the model parameters and latent states of a class of hidden Markov models (HMMs). We are interested in the class of HMMs for which the hidden state dynamics is a diffusion process and noisy observations are obtained at discrete times. We suppose that the diffusion...

This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. It is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the corresponding label is the output; and fin...

Mixtures of experts have become an indispensable tool for flexible modelling in a supervised learning context, allowing not only the mean function but the entire density of the output to change with the inputs. Sparse Gaussian processes (GP) have shown promise as a leading candidate for the experts in such models, and in this article, we propose to...

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalizing constant, and where even the unnormalized target needs to be approximated at finite resolution. Under such an assumption, this work builds upon a recently introduced multi-index sequential Monte Carlo (SMC) ratio estimator, which pro...

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalising constant, and where even the un-normalised target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics...

Finding efficient means of quantitatively describing material microstructure is a critical step towards harnessing data-centric machine learning approaches to understanding and predicting processing–microstructure–property relationships. Current quantitative descriptors of microstructure tend to consider only specific, narrow features such as grain...

Several configurations for the core and pedestal plasma are examined for a predefined tokamak design by implementing multiple heating/current drive (H/CD) sources to achieve an optimum configuration of high fusion power in a noninductive operation while maintaining an ideally magnetohydrodynamic (MHD) stable core plasma using the IPS-FASTRAN framew...

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalizing constant, and where even the unnormalized target needs to be approximated at finite resolution. Under such an assumption, this work builds upon a recently introduced multi-index Sequential Monte Carlo (SMC) ratio estimator, which pro...

Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assig...

Cyber-physical system security presents unique challenges to conventional measurement science and technology. Anomaly detection in software-assisted physical systems, such as those employed in additive manufacturing or in DNA synthesis, is often hampered by the limited available parameter space of the underlying mechanism that is transducing the an...

We analyze the behavior of projected stochastic gradient descent focusing on the case where the optimum is on the boundary of the constraint set and the gradient does not vanish at the optimum. Here iterates may in expectation make progress against the objective at each step. When this and an appropriate moment condition on noise holds, we prove th...

In the last decade, the atomically-focused electron beams utilized in scanning transmission electron microscopes (STEMs) have been shown to induce a broad set of local structural transformations in materials, opening pathways for directing material synthesis and modification atom-by-atom. The mechanisms underlying these transformations remain large...

In this article we consider a Monte-Carlo-based method to filter partially observed diffusions observed at regular and discrete times. Given access only to Euler discretizations of the diffusion process, we present a new procedure which can return online estimates of the filtering distribution with no time-discretization bias and finite variance. O...

Finding efficient means of fingerprinting microstructural information is a critical step towards harnessing data-centric machine learning approaches. A statistical framework is systematically developed for compressed characterisation of a population of images, which includes some classical computer vision methods as special cases. The focus is on m...

In this article we consider Bayesian inference associated to deep neural networks (DNNs) and in particular, trace-class neural network (TNN) priors which were proposed by Sell et al. [39]. Such priors were developed as more robust alternatives to classical architectures in the context of inference problems. For this work we develop multilevel Monte...

This position paper summarizes a recently developed research program focused on inference in the context of data centric science and engineering applications, and forecasts its trajectory forward over the next decade. Often one endeavours in this context to learn complex systems in order to make more informed predictions and high stakes decisions u...

We consider the problem of estimating expectations with respect to a target distribution with an unknown normalizing constant, and where even the unnormalized target needs to be approximated at finite resolution. This setting is ubiquitous across science and engineering applications, for example in the context of Bayesian inference where a physics-...

The method of classical shadows proposed by Huang, Kueng, and Preskill heralds remarkable opportunities for quantum estimation with limited measurements. Yet its relationship to established quantum tomographic approaches, particularly those based on likelihood models, remains unclear. In this article, we investigate classical shadows through the le...

This position paper summarizes a recently developed research program focused on inference in the context of data centric science and engineering applications, and forecasts its trajectory forward over the next decade. Often one endeavours in this context to learn complex systems in order to make more informed predictions and high stakes decisions u...

We consider the problem of estimating a parameter θ∈Θ⊆Rdθ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta \in \Theta \subseteq {\mathbb {R}}^{d_{\theta }}$$\end{d...

Significant progress in many classes of materials could be made with the availability of experimentally-derived large datasets composed of atomic identities and three-dimensional coordinates. Methods for visualizing the local atomic structure, such as atom probe tomography (APT), which routinely generate datasets comprised of millions of atoms, are...

This work considers variational Bayesian inference as an inexpensive and scalable alternative to a fully Bayesian approach in the context of sparsity-promoting priors. In particular, the priors considered arise from scale mixtures of Normal distributions with a generalized inverse Gaussian mixing distribution. This includes the variational Bayesian...

In this article, we consider computing expectations w.r.t. probability measures which are subject to discretization error. Examples include partially observed diffusion processes or inverse problems, where one may have to discretize time and/or space, in order to practically work with the probability of interest. Given access only to these discreti...

Significant progress in many classes of materials could be made with the availability of experimentally-derived large datasets composed of atomic identities and three-dimensional coordinates. Methods for visualizing the local atomic structure, such as atom probe tomography (APT), which routinely generate datasets comprised of millions of atoms, are...

We design and analyse the performance of a multilevel ensemble Kalman filter method (MLEnKF) for filtering settings where the underlying state-space model is an infinite- dimensional spatio-temporal process. We consider underlying models that needs to be simulated by numerical methods, with discretization in both space and time. The mul- tilevel Mo...

Classical shadows enable remarkably efficient estimation of quantum observables, yet their connection to conventional techniques is unclear. In simulated examples we show that Bayesian mean estimation attains lower error on average, whereas classical shadows excel for specific states of interest.

The method of classical shadows heralds unprecedented opportunities for quantum estimation with limited measurements [H.-Y. Huang, R. Kueng, and J. Preskill, Nat. Phys. 16, 1050 (2020)]. Yet its relationship to established quantum tomographic approaches, particularly those based on likelihood models, remains unclear. In this article, we investigate...

Mixtures of experts have become an indispensable tool for flexible modelling in a supervised learning context, and sparse Gaussian processes (GP) have shown promise as a leading candidate for the experts in such models. In the present article, we propose to design the gating network for selecting the experts from such mixtures of sparse GPs using a...

Bayesian inference is a powerful paradigm for quantum state tomography, treating uncertainty in meaningful and informative ways. Yet the numerical challenges associated with sampling from complex probability distributions hampers Bayesian tomography in practical settings. In this article, we introduce an improved, self-contained approach for Bayesi...

The broad incorporation of microscopic methods is yielding a wealth of information on the atomic and mesoscale dynamics of individual atoms, molecules, and particles on surfaces and in open volumes. Analysis of such data necessitates statistical frameworks to convert observed dynamic behaviors to effective properties of materials. Here, we develop...

We consider the problem of estimating a parameter associated to a Bayesian inverse problem. Treating the unknown initial condition as a nuisance parameter, typically one must resort to a numerical approximation of gradient of the log-likelihood and also adopt a discretization of the problem in space and/or time. We develop a new methodology to unbi...

The broad incorporation of microscopic methods is yielding a wealth of information on atomic and mesoscale dynamics of individual atoms, molecules, and particles on surfaces and in open volumes. Analysis of such data necessitates statistical frameworks to convert observed dynamic behaviors to effective properties of materials. Here we develop a met...

Bayesian inference is a powerful paradigm for quantum state tomography, treating uncertainty in meaningful and informative ways. Yet the numerical challenges associated with sampling from complex probability distributions hampers Bayesian tomography in practical settings. In this Article, we introduce an improved, self-contained approach for Bayesi...

In this article we consider a Monte Carlo-based method to filter partially observed diffusions observed at regular and discrete times. Given access only to Euler discretizations of the diffusion process, we present a new procedure which can return online estimates of the filtering distribution with no discretization bias and finite variance. Our ap...

Physics based forward models are the basis on which many experimental diagnostics are interpreted. For some diagnostics, models can be computationally expensive which precludes their use in real time analysis. Reduced models have the potential to capture sufficient physics thereby enabling the desired real time analysis. Using statistical inference...

We develop algorithms for computing expectations of the laws of models associated to stochastic differential equations (SDEs) driven by pure L\'evy processes. We consider filtering such processes and well as pricing of path dependent options. We propose a multilevel particle filter (MLPF) to address the computational issues involved in solving thes...

The article Multilevel particle filters for Lévy-driven stochastic differential equations, written by Ajay Jasra, Kody J. H. Law, Prince Peprah Osei, was originally published electronically on the publisher’s Internet portal (currently SpringerLink) on 20 October 2018 without open access.

This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. It is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the corresponding label is the output; and fin...

The problem of estimating certain distributions over $\{0,1\}^d$ is considered here. The distribution represents a quantum system of $d$ qubits, where there are non-trivial dependencies between the qubits. A maximum entropy approach is adopted to reconstruct the distribution from exact moments or observed empirical moments. The Robbins Monro algori...

Point set registration involves identifying a smooth invertible transformation between corresponding points in two point sets, one of which may be smaller than the other and possibly corrupted by observation noise. This problem is traditionally decomposed into two separate optimization problems: (i) assignment or correspondence, and (ii) identifica...

Point set registration involves identifying a smooth invertible transformation between corresponding points in two point sets, one of which may be smaller than the other and possibly corrupted by observation noise. This problem is traditionally decomposed into two separate optimization problems: (1) assignment or correspondence, and (2) identificat...

Point set registration involves identifying a smooth invertible transformation between corresponding points in two point sets, one of which may be smaller than the other and possibly corrupted by observation noise. This problem is traditionally decomposed into two separate optimization problems: (i) assignment or correspondence, and (ii) identifica...

Bayesian state estimation of a dynamical system from a stream of noisy measurements is important in many geophysical and engineering applications where high dimensionality of the state space, sparse observations, and model error pose key challenges. Here, three computationally feasible, approximate Gaussian data assimilation/filtering algorithms ar...

Leveraging Single Atom Dynamics to Measure the Electron-Beam-Induced Force and Atomic Potentials - Volume 24 Supplement - Ondrej Dyck, Feng Bao, Maxim Ziatdinov, Ali Yousefzadi Nobakht, Seungha Shin, Kody Law, Artem Maksov, Bobby G. Sumpter, Richard Archibald, Stephen Jesse, Sergei V. Kalinin

This paper considers a new approach to using Markov chain Monte Carlo (MCMC) in contexts where one may adopt multilevel (ML) Monte Carlo. The underlying problem is to approximate expectations w.r.t. an underlying probability measure that is associated to a continuum problem, such as a continuous-time stochastic process. It is then assumed that the...

In the last decade, the atomically focused beam of a scanning transmission electron microscope (STEM) was shown to induce a broad set of transformations of material structure, open pathways for probing atomic-scale reactions and atom-by-atom matter assembly. However, the mechanisms of beam-induced transformations remain largely unknown, due to an e...

Spectroscopic measurements of current-voltage curves in scanning probe microscopy is the earliest and one of the most common methods for characterizing local energy-dependent electronic properties, providing insight into superconductive, semiconductor, and memristive behaviors. However, the quasistatic nature of these measurements renders them extr...

We consider using regression to fit a theory-based log-linear ansatz, as well as higher order approximations, for the thermal energy confinement of a Tokamak as a function of device features. We use general linear models based on total order polynomials, as well as deep neural networks. The results indicate that the theory-based model fits the data...

We consider using regression to fit a theory-based log-linear ansatz, as well as higher order approximations, for the thermal energy confinement of a Tokamak as a function of device features. We use general linear models based on total order polynomials, as well as deep neural networks. The results indicate that the theory-based model fits the data...

This work concerns state-space models, in which the state-space is an infinite-dimensional spatial field, and the evolution is in continuous time, hence requiring approximation in space and time. The multilevel Monte Carlo (MLMC) sampling strategy is leveraged in the Monte Carlo step of the ensemble Kalman filter (EnKF), thereby yielding a multilev...

This article reviews the application of advanced Monte Carlo techniques in the context of Multilevel Monte Carlo (MLMC). MLMC is a strategy employed to compute expectations which can be biased in some sense, for instance, by using the discretization of a associated probability law. The MLMC approach works with a hierarchy of biased approximations w...

In this article we consider computing expectations w.r.t.~probability laws associated to a certain class of stochastic systems. In order to achieve such a task, one must not only resort to numerical approximation of the expectation, but also to a biased discretization of the associated probability. We are concerned with the situation for which the...

In this article we develop a new sequential Monte Carlo (SMC) method for multilevel (ML) Monte Carlo estimation. In particular, the method can be used to estimate expectations with respect to a target probability distribution over an infinite-dimensional and non-compact space as given, for example, by a Bayesian inverse problem with Gaussian random...

In this article we consider static Bayesian parameter estimation for partially observed diffusions that are discretely observed. We work under the assumption that one must resort to discretizing the underlying diffusion process, for instance using the Euler-Maruyama method. Given this assumption, we show how one can use Markov chain Monte Carlo (MC...

In this article, we consider the multilevel sequential Monte Carlo (MLSMC) method of Beskos et al. (Stoch. Proc. Appl. [to appear]). This is a technique designed to approximate expectations w.r.t. probability laws associated to a discretization. For instance, in the context of inverse problems, where one discretizes the solution of a partial differ...

In this paper the filtering of partially observed diffusions, with discrete-time observa- tions, is considered. It is assumed that only biased approximations of the diffusion can be obtained for choice of an accuracy parameter indexed by l. A multilevel estimator is proposed consisting of a telescopic sum of increment estimators associated to the s...

This work embeds a multilevel Monte Carlo (MLMC) sampling strategy into the Monte Carlo step of the ensemble Kalman filter (EnKF), thereby yielding a multilevel ensemble Kalman filter (MLEnKF) which has provably superior asymptotic cost to a given accuracy level. The development of MLEnKF for finite-dimensional state-spaces in the work [20] is here...

This paper considers uncertainty quantification for an elliptic nonlocal equation. In particular, it is assumed that the parameters which define the kernel in the nonlocal operator are uncertain and a priori distributed according to a probability measure. It is shown that the induced probability measure on some quantities of interest arising from f...

This article considers the sequential Monte Carlo (SMC) approximation of ratios of normalizing constants associated to posterior distributions which in principle rely on continuum models. Therefore, the Monte Carlo estimation error and the discrete approximation error must be balanced. A multilevel strategy is utilized to substantially reduce the c...

In the context of filtering chaotic dynamical systems it is well-known that partial observations, if sufficiently informative, can be used to control the inherent uncertainty due to chaos. The purpose of this paper is to investigate, both theoretically and numerically, conditions on the observations of chaotic systems under which they can be accura...

In this paper the filtering of partially observed diffusions, with discrete-time observations, is considered. It is assumed that only biased approximations of the diffusion can be obtained, for choice of an accuracy parameter indexed by $l$. A multilevel estimator is proposed, consisting of a telescopic sum of increment estimators associated to the...

This work considers black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm of (Haario etal. 2001) is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algori...

In this article we consider the approximation of expectations w.r.t. probability distributions associated to the solution of partial differential equations (PDEs); this scenario appears routinely in Bayesian inverse problems. In practice, one often has to solve the associated PDE numerically, using, for instance finite element methods and leading t...

This work embeds a multilevel Monte Carlo sampling strategy into the Monte Carlo step of the ensemble Kalman filter (EnKF) in the setting of finite dimensional signal evolution and noisy discrete-time observations. The signal dynamics is assumed to be governed by a stochastic differential equation (SDE), and a hierarchy of time grids is introduced...

In this chapter, we describe various algorithms for the smoothing problem in continuous time. We begin, in Section 7.1, by describing the Kalman–Bucy smoother , which applies in the case of linear dynamics when the initial conditions and the observational noise are Gaussian; the explicit Kalman–Bucy formulas are useful for the building of intuition...

This chapter is dedicated to illustrating the examples, theory, and algorithms, as presented in the previous chapters, through a few short and easy-to-follow MATLAB programs. These programs are provided for two reasons: (i) For some readers, they will form the best route by which to appreciate the details of the examples, theory, and algorithms we...

The purpose of this chapter is to provide a brief overview of the key mathematical ways of thinking that underpin our presentation of the subject of data assimilation . In particular, we touch on the subjects of probability, dynamical systems, probability metrics, and dynamical systems for probability measures, in Sections 1.1, 1.2, 1.3, and 1.4 re...

In this chapter, we describe various algorithms for the filtering problem. Recall from Section 2. 4 that filtering refers to the sequential update of the probability distribution on the state given the data, as data is acquired, and that \(Y _{j} =\{ y_{\ell}\}_{\ell=1}^{j}\) denotes the data accumulated up to time j. The filtering update from time...

In this chapter, we describe various algorithms for determination of the filtering distribution μ t in continuous time. We begin in Section 8.1 with the Kalman–Bucy filter, which provides an exact algorithm for linear problems. Since the filtering distribution is Gaussian in this case, the distribution is entirely characterized by the mean and cova...

This chapter is dedicated to illustrating the examples, theory, and algorithms presented in the preceding three chapters through a few short and easy-to-follow MATLAB programs. We have followed the same principles as in Chapter 9, and again the code may be readily extended to solve problems more complex than those described in Examples 6. 4–6. 8, w...

In this chapter, and in all subsequent chapters, we consider continuous-time signal dynamics and continuous-time data . This takes us into a part of the subject that is potentially rather technical, a fact that can obscure the structure manifest in the continuous-time formulation. In order to avoid technicalities that can obfuscate the derivations,...

In this chapter, we introduce the mathematical framework for discrete-time data assimilation. Section 2.1 describes the mathematical models we use for the underlying signal , which we wish to recover, and for the data , which we use for the recovery.

The formulation of the data-assimilation problem described in the previous chapter is probabilistic, and its computational resolution requires the probing of a posterior probability distribution on signal-given data.

Many Bayesian inference problems require exploring the posterior distribution of high-dimensional parameters that represent the discretization of an underlying function. This work introduces a family of Markov chain Monte Carlo (MCMC) samplers that can adapt to the particular structure of a posterior distribution over functions. Two distinct lines...

The long-time behavior of filters for the partially observed Lorenz '96 model is studied. It is proven that for both discrete-time and continuous-time observations the 3DVAR filter can recover the signal within a neighborhood defined by the size of the observational noise, as long as a sufficiently large proportion of the state vector is observed;...

The ensemble Kalman filter (EnKF) is a method for combining a dynamical model with data in a sequential fashion. Despite its widespread use, there has been little analysis of its theoretical properties. Many of the algorithmic innovations associated with the filter, which are required to make a useable algorithm in practice, are derived in an ad ho...

The proof of convergence of the standard ensemble Kalman filter (EnKF) from Legland etal. (2011) is extended to non-Gaussian state space models. A density-based deterministic approximation of the mean-field limit EnKF (DMFEnKF) is proposed, consisting of a PDE solver and a quadrature rule. Given a certain minimal order of convergence κ between the...

We study conditions under which vortices in a highly oblate harmonically trapped Bose-Einstein condensate (BEC) can be stabilized due to pinning by a blue-detuned Gaussian laser beam, with particular emphasis on the potentially destabilizing effects of laser beam positioning within the BEC. Our approach involves theoretical and numerical exploratio...

Inverse problems lend themselves naturally to a Bayesian formulation, in which the quantity of interest is a posterior distribution of state and/or parameters given some uncertain observations. For the common case in which the forward operator is smoothing, then the inverse problem is ill-posed. Well-posedness is imposed via regularization in the f...

Fluids subjected to suitable forcing will exhibit turbulence, with characteristics strongly affected by the fluid’s physical properties and dimensionality. In this work, we explore two-dimensional (2D) quantum turbulence in an oblate Bose-Einstein condensate confined to an annular trapping potential. Experimentally, we find conditions for which sma...

The 3DVAR filter is prototypical of methods used to combine observed data with a dynamical system, online, in order to improve estimation of the state of the system. Such methods are used for high dimensional data assimilation problems, such as those arising in weather forecasting. To gain understanding of filters in applications such as these, it...