Adrian Wills

• University of Newcastle Australia

This paper presents a method for obtaining distribution grid models based on available measurements from the grid. In particular, we propose a nonlinear approximation of the DistFlow model [1], which includes line losses between nodes and is parameterized by the unknown line parameters, namely the impedance between nodes. Based on measured voltage...

Polycrystals illuminated by high-energy X-rays or neutrons produce diffraction patterns in which the measured diffraction peaks encode the individual single crystal strain states. While state of the art X-ray and neutron diffraction approaches can be used to routinely recover per grain mean strain tensors, less work has been produced on the recover...

Sequential Monte Carlo methods—also known as particle filters—offer approximate solutions to filtering problems for nonlinear state-space systems. These filtering problems are notoriously difficult to solve in general due to a lack of closed-form expressions and challenging expectation integrals. The essential idea behind particle filters is to emp...

This paper considers parameter estimation for nonlinear state-space models, which is an important but challenging problem. We address this challenge by employing a variational inference (VI) approach, which is a principled method that has deep connections to maximum likelihood estimation. This VI approach ultimately provides estimates of the model...

Identification of nonlinear systems is a challenging problem. Physical knowledge of the system can be used in the identification process to significantly improve the predictive performance by restricting the space of possible mappings from the input to the output. Typically, the physical models contain unknown parameters that must be learned from d...

We provide a proof-of-concept for a novel state-space modelling approach for predicting monthly deaths due to political violence. Attention is focused on developing the method and demonstrating the utility of this approach, which provides exciting opportunities to engage with domain experts in developing new and improved state-space models for pred...

This paper presents a method for calculating the smoothed state distribution for Jump Markov Linear Systems. More specifically, the paper details a novel two-filter smoother that provides closed-form expressions for the smoothed hybrid state distribution. This distribution can be expressed as a Gaussian mixture with a known, but exponentially incre...

This article describes a memory efficient method for solving large-scale optimization problems that arise when planning scanning-beam lithography processes. These processes require the identification of an exposure pattern that minimizes the difference between a desired and predicted output image, subject to constraints. The number of free variable...

The modern world contains an immense number of different and interacting systems, from the evolution of weather systems to variations in the stock market, autonomous vehicles interacting with their environment, and the spread of diseases. For society to function, it is essential to understand the behavior of the world so that informed decisions can...

Jump Markov linear systems (JMLS) are a useful model class for capturing abrupt changes in system behaviour that are temporally random, such as when a fault occurs. In many situations, accurate knowledge of the model is not readily available and can be difficult to obtain based on first principles. This paper presents a method for learning paramete...

This paper addresses the problem of computing fixed-interval smoothed state estimates of a linear time varying Gaussian stochastic system. There already exist many algorithms that perform this computation, but all of them impose certain restrictions on system matrices in order for them to be applicable, and the restrictions vary considerably betwee...

This paper considers the problem of determining an optimal control action based on observed data. We formulate the problem assuming that the system can be modelled by a nonlinear state-space model, but where the model parameters, state and future disturbances are not known and are treated as random variables. Central to our formulation is that the...

This paper proposes an improved prediction update for extended target tracking with the random matrix model. A key innovation is to employ a generalised non-central inverse Wishart distribution to model the state transition density of the target extent; resulting in a prediction update that accounts for kinematic state dependent transformations. Mo...

In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (Hessian) information based on...

This paper considers the problem of determining an optimal control action based on observed data. We formulate the problem assuming that the system can be modelled by a nonlinear state-space model, but where the model parameters, state and future disturbances are not known and are treated as random variables. Central to our formulation is that the...

Machine learning practitioners invest significant manual and computational resources in finding suitable learning rates for optimization algorithms. We provide a probabilistic motivation, in terms of Gaussian inference, for popular stochastic first-order methods. As an important special case, it recovers the Polyak step with a general metric. The i...

Machine vision is an important sensing technology used in mobile robotic systems. Advancing the autonomy of such systems requires accurate characterisation of sensor uncertainty. Vision includes intrinsic uncertainty due to the camera sensor and extrinsic uncertainty due to environmental lighting and texture, which propagate through the image proce...

This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable,...

System identification aims to build models of dynamical systems from data. Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset. It is typically observed tha...

This paper is directed towards the problem of learning nonlinear ARX models based on observed input-output data. In particular, our interest is in learning a conditional distribution of the current output based on a finite window of past inputs and outputs. To achieve this, we consider the use of so-called energy-based models, which have been devel...

In this paper, the problem of state estimation, in the context of both filtering and smoothing, for nonlinear state-space models is considered. Due to the nonlinear nature of the models, the state estimation problem is generally intractable as it involves integrals of general nonlinear functions and the filtered and smoothed state distributions lac...

This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable,...

System identification aims to build models of dynamical systems from data. Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset. It is typically observed tha...

This paper considers parameter estimation for nonlinear state-space models, which is an important but challenging problem. We address this challenge by employing a variational inference (VI) approach, which is a principled method that has deep connections to maximum likelihood estimation. This VI approach ultimately provides estimates of the model...

This paper is directed towards the problem of learning nonlinear ARX models based on system input--output data. In particular, our interest is in learning a conditional distribution of the current output based on a finite window of past inputs and outputs. To achieve this, we consider the use of so-called energy-based models, which have been develo...

This paper addresses Bayesian system identification using a Markov Chain Monte Carlo approach. In particular, the Metroplis-Hastings algorithm with a Hamiltonian proposal - known as Hamiltonian Monte Carlo - is reviewed and adapted to linear and nonlinear system identification problems. The paper details how the Hamiltonian proposal can be arranged...

Machine vision is an important sensing technology used in mobile robotic systems. Advancing the autonomy of such systems requires accurate characterisation of sensor uncertainty. Vision includes intrinsic uncertainty due to the camera sensor and extrinsic uncertainty due to environmental lighting and texture, which propagate through the image proce...

Energy resolved neutron transmission techniques can provide high-resolution images of strain within polycrystalline samples allowing the study of residual strain and stress in engineered components. Strain is estimated from such data by analysing features known as Bragg-edges for which several methods exist. It is important for these methods to pro...

Energy resolved neutron transmission techniques can provide high-resolution images of strain within polycrystalline samples allowing the study of residual strain and stress in engineered components. Strain is estimated from the recorded neutron transmission data by analysing features known as Bragg-edges for which several methods exist. It is impor...

This paper presents a method for calculating the smoothed state distribution for Jump Markov Linear Systems. More specifically, the paper details a novel two-filter smoother that provides closed-form expressions for the smoothed hybrid state distribution. This distribution can be expressed as a Gaussian mixture with a known, but exponentially incre...

This paper presents a Bayesian method for identification of jump Markov linear systems that is powered by a Markov chain Monte Carlo method called the Gibbs sampler. Unlike maximum likelihood approaches, this method provides the parameter distributions or the variation of likely system responses, which could be useful for analysing the stability ma...

Jump Markov linear systems (JMLS) are a useful class which can be used to model processes which exhibit random changes in behavior during operation. This paper presents a numerically stable method for learning the parameters of jump Markov linear systems using the expectation-maximisation (EM) approach. The solution provided herein is a determinist...

This paper proposes a new class of state transition models that afford closed-form predictions for the tracking of extended targets. A key innovation is to employ a non-central inverse Wishart distribution to model the state transition density of the target extent. Importantly, this results in a simplified prediction update that is computationally...

Glitches introduce impulse-like disturbances which are not be readily attenuated by low-pass filtering. This article presents a model that describes the behaviour of glitches, and a method for mitigation based on a large-amplitude dither signal. Analytical and experimental results demonstrate that a dither signal with sufficient amplitude can mitig...

Diffraction of high‐energy X‐rays produced at synchrotron sources can provide rapid strain measurements, with high spatial resolution, and good penetrating power. With an uncollimated diffracted beam, through‐thickness averages of strain can be measured using this technique, which poses an associated rich tomography problem. This paper proposes a G...

We consider the problem of maximum likelihood parameter estimation for nonlinear state-space models. This is an important, but challenging problem. This challenge stems from the intractable multidimensional integrals that must be solved in order to compute, and maximise, the likelihood. Here we present a new variational family where variational inf...

We present an approach to designing neural network based models that will explicitly satisfy known linear constraints. To achieve this, the target function is modelled as a linear transformation of an underlying function. This transformation is chosen such that any prediction of the target function is guaranteed to satisfy the constraints and can b...

Recently, a number of reconstruction algorithms have been presented for residual strain tomography from Bragg-edge neutron transmission measurements. In this paper, we examine whether strain tomography can also be achieved using diffraction instruments. We outline the proposed method and develop a suitable reconstruction algorithm. This technique i...

In recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradient-based routines can improve the convergence by incorporating second-order information. In this work we propose a direct least-squares approach conceptually similar to the limited memory quasi-Newton m...

In this paper, a unified identification framework called constrained subspace method for structured state-space models (COSMOS) is presented, where the structure is defined by a user specified linear or polynomial parametrization. The new approach operates directly from the input and output data, which differs from the traditional two-step method t...

This paper presents a proof-of-concept demonstration of triaxial strain tomography from Bragg-edge neutron imaging within a three-dimensional sample. Bragg-edge neutron transmission can provide high-resolution images of the average through thickness strain within a polycrystalline material. This poses an associated rich tomography problem which see...

We present a new method of learning a continuous occupancy field for use in robot navigation. Occupancy grid maps, or variants of, are possibly the most widely used and accepted method of building a map of a robot's environment. Various methods have been developed to learn continuous occupancy maps and have successfully resolved many of the shortco...

Recently, several algorithms for strain tomography from energy-resolved neutron transmission measurements have been proposed. These methods assume that the stress-free lattice spacing d0 is a known constant limiting their application to the study of stresses generated by manufacturing and loading methods that do not alter this parameter. In this pa...

Deep kernel learning refers to a Gaussian process that incorporates neural networks to improve the modelling of complex functions. We present a method that makes this approach feasible for problems where the data consists of line integral measurements of the target function. The performance is illustrated on computed tomography reconstruction examp...

In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (Hessian) information based on...

Several recent methods for tomographic reconstruction of stress and strain fields from Bragg‐edge neutron strain images have been proposed in the literature. This paper presents an extension of a previously demonstrated approach based on Gaussian process regression that enforces equilibrium in the method. This extension incorporates knowledge of bo...

This paper develops and experimentally evaluates a dither-based method for improved generation of arbitrary signals in digital-to-analogue converters that exhibits glitches --- essentially converting the glitches from high-frequency to low-frequency disturbances. One major benefit of this behaviour appears in closed-loop control applications, as th...

This paper presents the first triaxial reconstruction of strain in three-dimensions from Bragg-edge neutron imaging. Bragg-edge neutron transmission can provide high-resolution tomographic images of strain within a polycrystalline material. This poses an associated rich tomography problem which seeks to reconstruct the full triaxial strain field fr...

Recently, a number of reconstruction algorithms have been presented for residual strain tomography from Bragg-edge neutron transmission measurements. In this paper, we examine whether strain tomography can also be achieved from diffraction measurements. We outline the proposed method and develop a suitable reconstruction algorithm. This technique i...

Recently, several algorithms for strain tomography from energy-resolved neutron transmission measurements have been proposed. These methods assume that the strain-free lattice spacing $d_0$ is a known constant limiting their application to the study of stresses generated by manufacturing and loading methods that do not alter this parameter. In this...

Diffraction-based methods have become an invaluable tool for the detailed assessment of residual strain and stress within experimental mechanics. These methods typically measure a component of the average strain within a gauge volume. It is common place to treat these measurements as point measurements and to interpolate and extrapolate their value...

Diffraction of high-energy X-rays produced at synchrotron sources can provide rapid strain measurements, with high spatial resolution, and good penetrating power. With an uncollimated diffracted beam, through thickness averages of strain can be measured using this technique. This poses an associated rich tomography problem. This paper proposes a Ga...

This paper deals with the evaluation of double line integrals of the squared exponential covariance function. We propose a new approach in which the double integral is reduced to a single integral using the error function. This single integral is then computed with efficiently implemented numerical techniques. The performance is compared against ex...

Bragg-edge strain imaging from energy-resolved neutron-transmission measurements poses an interesting tomography problem. The solution to this problem will allow the reconstruction of detailed triaxial stress and strain distributions within polycrystalline solids from sets of Bragg-edge strain images. Work over the last decade has provided some sol...

During recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradient-based routines can improve the convergence by incorporating second order information. In this work we propose a direct least-squares approach conceptually similar to the limited memory quasi-Newt...

Diffraction-based methods have become an invaluable tool for the detailed assessment of residual strain and stress within experimental mechanics. These methods typically measure a component of the average strain within a gauge volume. It is common place to treat these measurements as point measurements and to interpolate and extrapolate their value...

Bragg-edge strain imaging from energy-resolved neutron transmission measurements poses an interesting tomography problem. The solution to this problem will allow the reconstruction of detailed triaxial stress and strain distributions within polycrystalline solids from sets of Bragg-edge strain images. Work over the last decade has provided some sol...

Pseudo-marginal Metropolis-Hastings (pmMH) is a versatile algorithm for sampling from target distributions which are not easy to evaluate point-wise. However, pmMH requires good proposal distributions to sample efficiently from the target, which can be problematic to construct in practice. This is especially a problem for high-dimensional targets w...

Scanning laser lithography is a maskless lithography method for selectively exposing features on a film of photoresist. A set of exposure positions and beam energies are required to optimally reproduce the desired feature pattern. The task of determining the exposure energies is inherently nonlinear due to the photoresist model and the requirement...

We provide a numerically robust and fast method capable of exploiting the local geometry when solving large-scale stochastic optimisation problems. Our key innovation is an auxiliary variable construction coupled with an inverse Hessian approximation computed using a receding history of iterates and gradients. It is the Markov chain nature of the c...

This paper deals with modelling and reconstruction of strain fields, relying upon data generated from neutron Bragg-edge measurements. We propose a probabilistic approach in which the strain field is modelled as a Gaussian process, assigned a covariance structure customised by incorporation of the so-called equilibrium constraints. The computationa...

This paper deals with modelling and reconstruction of strain fields, relying upon data generated from neutron Bragg-edge measurements. We propose a probabilistic approach in which the strain field is modelled as a Gaussian process, assigned a covariance structure customised by incorporation of the so-called equilibrium constraints. The computationa...

This paper considers the problem of computing Bayesian estimates of system parameters and functions of them on the basis of observed system performance data. This is a previously studied issue where stochastic simulation approaches have been examined using the popular Metropolis--Hastings (MH) algorithm. This prior study has identified a recognised...

This paper considers the problem of estimating linear dynamic system models when the observations are corrupted by random disturbances with nonstandard distributions. The paper is particularly motivated by applications where sensor imperfections involve significant contribution of outliers or wrap-around issues resulting in multi-modal distribution...

This paper considers the problem of estimating linear dynamic system models when the observations are corrupted by random disturbances with nonstandard distributions. The paper is particularly motivated by applications where sensor imperfections involve significant contribution of outliers or wrap-around issues resulting in multi-modal distribution...

The computation of Bayesian estimates of system parameters and functions of them on the basis of observed system performance data is a common problem within system identification. This is a previously studied issue where stochastic simulation approaches have been examined using the popular Metropolis--Hastings (MH) algorithm. This prior study has i...

Using Maximum Likelihood (or Prediction Error) methods to identify linear state space model is a prime technique. The likelihood function is a nonconvex function and care must be exercised in the numerical maximization. Here the focus will be on affine parameterizations which allow some special techniques and algorithms. Three approaches to formula...

We consider a modification of the covariance function in Gaussian processes to correctly account for known linear constraints. By modelling the target function as a transformation of an underlying function, the constraints are explicitly incorporated in the model such that they are guaranteed to be fulfilled by any sample drawn or prediction made....

It has recently been shown that many of the existing quasi-Newton algorithms can be formulated as learning algorithms, capable of learning local models of the cost functions. Importantly, this understanding allows us to safely start assembling probabilistic Newton-type algorithms, applicable in situations where we only have access to noisy observat...

Scanning laser lithography is a maskless method for exposing photoresist during semiconductor manufacturing. In this method, the energy of a focused beam is controlled while scanning the beam or substrate. With a positive photoresist material, areas that receive an exposure dosage over the threshold energy are dissolved during development. The surf...

Laser scanning lithography is a maskless method for exposing films of photoresist during semiconductor manufacturing. In this method a focused beam is scanned over a surface with varying intensity to create features in the photoresist. Given the shape of a desired feature, an exposure pattern must be found that approximates this shape in the develo...

This paper addresses the problem of computing fixed interval smoothed state estimates of a linear time varying Gaussian stochastic system. There already exist many algorithms that perform this computation, but all of them impose certain restrictions on system matrices in order for them to be applicable. This paper develops a new forwards–backwards...

A Bayesian filtering algorithm is developed for a class of state-space systems that can be modelled via Gaussian mixtures. In general, the exact solution to this filtering problem involves an exponential growth in the number of mixture terms and this is handled here by utilising a Gaussian mixture reduction step after both the time and measurement...

In 1959, the integrated circuit (IC) was invented simultaneously by Jack Kilby of Texas Instruments and Robert Noyce of Shockley Semiconductor [Ki lby, 2000]. This development has been considered one of mankind's most significant innovations.

Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian. We estimate the gradient and Hessian using Fisher's identity in combination with a smoothing algorithm. We explore two approximations of the...

This paper considers the system architec-ture and design issues for implementation of on-line Model Predictive Control (MPC) in Field Programmable Gate Arrays (FPGAs) and Application Specific Inte-grated Circuits (ASICs). In particular, the computa-tionally itensive tasks of fast matrix QR factorisation, and subsequent sequential quadratic programm...

The article describes a method for estimating the spectrum of a low-level signal corrupted by noise. If dual sensors are available and the additive sensor noise is uncorrelated, the cross power spectrum can recover the power spectrum of signal if the statistical properties do not change with time. When using the Welch method to estimate the cross-p...

We propose an algorithm for optimal input design in nonlinear stochastic dynamic systems. The approach relies on minimizing a function of the covariance of the parameter estimates of the system with respect to the input. The covariance matrix is approximated using a joint likelihood function of hidden states and measurements, and a combination of s...

The article describes a method for estimating the spectrum or RMS value of a low-level signal corrupted by noise. If two identical sensors can be employed simultaneously and the additive noise sources are uncorrelated, the cross power spectrum can recover the power spectrum of the underlying signal. When using the Welch method to estimate the cross...

This paper presents a Matlab-based software package for the estimation of dynamic systems. It has been developed primarily as a platform to support the objective evaluation of novel approaches relative to existing methods within a common software framework. This is designed to streamline comparisons. The work here provides an explanation of the too...

This paper develops and illustrates a new maximum-likelihood based method for the identification of Hammerstein–Wiener model structures. A central aspect is that a very general situation is considered wherein multivariable data, non-invertible Hammerstein and Wiener nonlinearities, and coloured stochastic disturbances both before and after the Wien...

This paper examines the use of a so-called “generalised Hammerstein–Wiener” model structure that is formed as the concatenation of an arbitrary number of Hammerstein systems. The latter are taken here to be memoryless non-linearities followed by linear time invariant dynamics. Hammerstein, Wiener, Hammerstein–Wiener and Wiener–Hammerstein models ar...

This paper examines the problem of estimating the parameters describing system models of quite general nonlinear and multi-variable form. The approach is a computational one in which quantities that are intractable to evaluate exactly are approximated by sample averages from randomized algorithms. The main contribution is to illustrate the viabilit...

This paper considers a Bayesian approach to linear system identification. One motivation is the advantage of the minimum mean square error of the associated conditional mean estimate. A further motivation is the error quantifications afforded by the posterior density which are not reliant on asymptotic in data length derivations. To compute these p...

This paper addresses the implementation of linear model predictive control (MPC) at millisecond range, or faster, sampling rates. This is achieved by designing a custom integrated circuit architecture that is specifically targeted to the MPC problem. As opposed to the more usual approach using a generic serial architecture processor, the design her...

This study investigates the design of a field-programmable gate array-based custom computer architecture solution for implementing model predictive control (MPC). The solution employs a primal logarithmic-barrier interior-point algorithm in order to handle actuator constraints. The solution also incorporates practical aspects of a control algorithm...

This paper considers a Bayesian approach to linear system identification. One motivation is the advantage of the minimum mean square error of the associated conditional mean estimate. A further motivation is the error quantifications afforded by the posterior density which are not reliant on asymptotic in data length derivations. To compute these p...

This paper addresses the implementation of linear model predictive control (MPC) at millisecond range, or faster, sampling rates. This is achieved by designing a custom integrated circuit architecture that is specifically targeted to the MPC problem. As opposed to the more usual approach using a generic serial architecture processor, the design her...

This chapter examines the estimation of multivariable linear models for which the parameters vary in a time-varying manner that depends in an affine fashion on a known or otherwise measured signal. These locally linear models which depend on a measurable operating point are known as linear parameter varying (LPV) models. The contribution here relat...

This paper develops and illustrates methods for the identification of Wiener model structures. These techniques are capable of accommodating the ``blind'' situation where the input excitation to the linear block is not observed. Furthermore, the algorithm developed here can accommodate a nonlinearity which need not be invertible, and may also be mu...

The work here is directed at examining a model predictive control (MPC) implementation that takes advantage of recent advances in the availability of high performance computing platforms at modest cost. The focus here is on the potential for developing custom architecture solutions on field programmable gate array (FPGA) platforms. This is illustra...

This paper is concerned with the parameter estimation of a general class of nonlinear dynamic systems in state-space form. More specifically, a Maximum Likelihood (ML) framework is employed and an Expectation Maximisation (EM) algorithm is derived to compute these ML estimates. The Expectation (E) step involves solving a nonlinear state estimation...

The expectation maximisation (EM) algorithm has proven to be effective for a range of identification problems. Unfortunately, the way in which the EM algorithm has previously been applied has proven unsuitable for the commonly employed innovations form model structure. This paper addresses this problem, and presents a previously unexamined method o...