Dongbin Xiu

• PhD
• Professor at The Ohio State University

We present a new method for solving stochastic differential equations based on Galerkin projections and extensions of Wiener's polynomial chaos. Specifically, we represent the stochastic processes with an optimum trial basis from the Askey family of orthogonal polynomials that reduces the dimensionality of the system and leads to exponential conver...

Recently there has been a growing interest in designing efficient methods for the solution of ordinary/partial differential equations with random inputs. To this end, stochastic Galerkin methods appear to be superior to other nonsampling methods and, in many cases, to several sampling methods. However, when the governing equations take complicated...

We present a numerical framework for approximating unknown governing equations using observation data and deep neural networks (DNN). In particular, we propose to use residual network (ResNet) as the basic building block for equation approximation. We demonstrate that the ResNet block can be considered as a one-step method that is exact in temporal...

We present effective numerical algorithms for learning unknown governing differential equations from measurement data. We employ a set of standard basis functions, e.g., polynomials, to approximate the governing equation with high accuracy. Upon recasting the problem into a function approximation problem, we discuss several important aspects for ac...

Equations, particularly differential equations, are fundamental for understanding natural phenomena and predicting complex dynamics across various scientific and engineering disciplines. However, the governing equations for many complex systems remain unknown due to intricate underlying mechanisms. Recent advancements in machine learning and data s...

We present a multi-fidelity method for uncertainty quantification of parameter estimates in complex systems, leveraging generative models trained to sample the target conditional distribution. In the Bayesian inference setting, traditional parameter estimation methods rely on repeated simulations of potentially expensive forward models to determine...

We present a new deep neural network (DNN) architecture capable of approximating functions up to machine accuracy. Termed the Chebyshev feature neural network (CFNN), the new structure employs Chebyshev functions with learnable frequencies as the first hidden layer, followed by the standard fully connected hidden layers. The learnable frequencies o...

We present a numerical method for learning the dynamics of slow components of unknown multiscale stochastic dynamical systems. While the governing equations of the systems are unknown, bursts of observation data of the slow variables are available. By utilizing the observation data, our proposed method is capable of constructing a generative stocha...

This study introduces a training-free conditional diffusion model for learning unknown stochastic differential equations (SDEs) using data. The proposed approach addresses key challenges in computational efficiency and accuracy for modeling SDEs by utilizing a score-based diffusion model to approximate their stochastic flow map. Unlike the existing...

We present a new Deep Neural Network (DNN) architecture capable of approximating functions up to machine accuracy. Termed Chebyshev Feature Neural Network (CFNN), the new structure employs Chebyshev functions with learnable frequencies as the first hidden layer, followed by the standard fully connected hidden layers. The learnable frequencies of th...

We present a numerical method for learning the dynamics of slow components of unknown multiscale stochastic dynamical systems. While the governing equations of the systems are unknown, bursts of observation data of the slow variables are available. By utilizing the observation data, our proposed method is capable of constructing a generative stocha...

Many application areas rely on models that can be readily simulated but lack a closed-form likelihood, or an accurate approximation under arbitrary parameter values. Existing parameter estimation approaches in this setting are generally approximate. Recent work on using neural network models to reconstruct the mapping from the data space to the par...

We present a numerical method for learning unknown nonautonomous stochastic dynamical system, i.e., stochastic system subject to time dependent excitation or control signals. Our basic assumption is that the governing equations for the stochastic system are unavailable. However, short bursts of input/output (I/O) data consisting of certain known ex...

Non-intrusive least-square-based polynomial chaos expansion (PCE) techniques have attracted increasing attention among researchers for simple yet efficient surrogate constructions. Different sampling approaches, including optimal design of experiments (DoEs), have been developed to facilitate the least-square-based PCE construction by reducing the...

We present a numerical framework for learning unknown stochastic dynamical systems using measurement data. Termed stochastic flow map learning (sFML), the new framework is an extension of flow map learning (FML) that was developed for learning deterministic dynamical systems. For learning stochastic systems, we define a stochastic flow map that is...

Polynomial approximations of functions are widely used in scientific computing. In certain applications, it is often desired to require the polynomial approximation to be non-negative (resp. non-positive), or bounded within a given range, due to constraints posed by the underlying physical problems. Efficient numerical methods are thus needed to en...

We present a numerical approach for modeling unknown dynamical systems using partially observed data, with a focus on biological systems with (relatively) complex dynamical behavior. As an extension of the recently developed deep neural network (DNN) learning methods, our approach is particularly suitable for practical situations when (i) measureme...

This study utilizes an ensemble of feedforward neural network models to analyze large-volume and high-dimensional consumer touchpoints and their impact on purchase decisions. When applied to a proprietary dataset of consumer touchpoints and purchases from a global software service provider, the proposed approach demonstrates better predictive accur...

We present a numerical method to learn an accurate predictive model for an unknown stochastic dynamical system from its trajectory data. The method seeks to approximate the unknown flow map of the underlying system. It employs the idea of autoencoder to identify the unobserved latent random variables. In our approach, we design an encoding function...

We present a numerical method to learn an accurate predictive model for an unknown stochastic dynamical system from its trajectory data. The method seeks to approximate the unknown flow map of the underlying system. It employs the idea of autoencoder to identify the unobserved latent random variables. In our approach, we design an encoding function...

We present a computational technique for modeling the evolution of partial differential equations (PDEs) with incomplete data. It is a significant extension of the recent work of data driven learning of PDEs, in the sense that we consider two forms of partial data: data are observed only on a subset of the domain, and data are observed only on a su...

Flow map learning (FML), in conjunction with deep neural networks (DNNs), has shown promises for data driven modeling of unknown dynamical systems. A remarkable feature of FML is that it is capable of producing accurate predictive models for partially observed systems, even when their exact mathematical models do not exist. In this paper, we presen...

We present a numerical framework for learning unknown stochastic dynamical systems using measurement data. Termed stochastic flow map learning (sFML), the new framework is an extension of flow map learning (FML) that was developed for learning deterministic dynamical systems. For learning stochastic systems, we define a stochastic flow map that is...

We present a computational technique for modeling the evolution of partial differential equations (PDEs) with incomplete data. It is a significant extension of the recent work of data driven learning of PDEs, in the sense that we consider two forms of partial data: data are observed only on a subset of the domain, and data are observed only on a su...

Flow map learning (FML), in conjunction with deep neural networks (DNNs), has shown promise for data driven modeling of unknown dynamical systems. A remarkable feature of FML is that it is capable of producing accurate predictive models for partially observed systems, even when their exact mathematical models do not exist. In this paper, we present...

Recent work has focused on data-driven learning of the evolution of unknown systems via deep neural networks (DNNs), with the goal of conducting long term prediction of the dynamics of the unknown system. In many real-world applications, data from time-dependent systems are often collected on a time scale that is coarser than desired, due to variou...

Recently, a general data driven numerical framework has been developed for learning and modeling of unknown dynamical systems using fully- or partially-observed data. The method utilizes deep neural networks (DNNs) to construct a model for the flow map of the unknown system. Once an accurate DNN approximation of the flow map is constructed, it can...

We present a discontinuity detector constructed by deep neural networks. Using convolutional neural network (CNN) structure, we design a comprehensive set of synthetic training data. The data consist of randomly generated piecewise smooth functions evaluated at equidistance grids, with labels denoting troubled cells where discontinuities are presen...

Recent work has focused on data-driven learning of the evolution of unknown systems via deep neural networks (DNNs), with the goal of conducting long time prediction of the evolution of the unknown system. Training a DNN with low generalization error is a particularly important task in this case as error is accumulated over time. Because of the inh...

We present a data-driven numerical approach for modeling unknown dynamical systems with missing/hidden parameters. The method is based on training a deep neural network (DNN) model for the unknown system using its trajectory data. A key feature is that the unknown dynamical system contains system parameters that are completely hidden, in the sense...

We present a numerical framework for deep neural network (DNN) modeling of unknown time-dependent partial differential equation (PDE) using their trajectory data. Unlike the recent work of [Wu and Xiu, J. Comput. Phys. 2020], where the learning takes place in modal/Fourier space, the current method conducts the learning and modeling in physical spa...

We present a numerical framework for deep neural network (DNN) modeling of unknown time-dependent partial differential equations (PDE) using their trajectory data. Unlike the recent work of [Wu and Xiu, J. Comput. Phys. 2020], where the learning takes place in modal/Fourier space, the current method conducts the learning and modeling in physical sp...

We present a general numerical approach for learning unknown dynamical systems using deep neural networks (DNNs). Our method is built upon recent studies that identified residual network (ResNet) as an effective neural network learning structure. In this paper, we present a generalized ResNet framework and broadly define “residue” as the discrepanc...

Mathematical equations are often used to model biological processes. However, for many systems, determining analytically the underlying equations is highly challenging due to the complexity and unknown factors involved in the biological processes. In this work, we present a numerical procedure to discover dynamical physical laws behind biological d...

We study the problem of identifying unknown processes embedded in time-dependent partial differential equation (PDE) using observational data, with an application to advection-diffusion type PDE. We first conduct theoretical analysis and derive conditions to ensure the solvability of the problem. We then present a set of numerical approaches, inclu...

A novel correction algorithm is proposed for multi-class classification problems with corrupted training data. The algorithm is non-intrusive, in the sense that it post-processes a trained classification model by adding a correction procedure to the model prediction. The correction procedure can be coupled with any approximators, such as logistic r...

We present a numerical framework for recovering unknown non-autonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the non-autonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are...

We present a general numerical approach for constructing governing equations for unknown dynamical systems when only data on a subset of the state variables are available. The unknown equations for these observed variables are thus a reduced system of the complete set of state variables. Reduced systems possess memory integrals, based on the well k...

We study the problem of identifying unknown processes embedded in time-dependent partial differential equation (PDE) using observational data, with an application to advection-diffusion type PDE. We first conduct theoretical analysis and derive conditions to ensure the solvability of the problem. We then present a set of numerical approaches, inclu...

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a bounded interval, as opposed to th...

A novel correction algorithm is proposed for multi-class classification problems with corrupted training data. The algorithm is non-intrusive, in the sense that it post-processes a trained classification model by adding a correction procedure to the model prediction. The correction procedure can be coupled with any approximators, such as logistic r...

We present a general numerical approach for learning unknown dynamical systems using deep neural networks (DNNs). Our method is built upon recent studies that identified the residue network (ResNet) as an effective neural network structure. In this paper, we present a generalized ResNet framework and broadly define residue as the discrepancy betwee...

We present a framework for recovering/approximating unknown time-dependent partial differential equation (PDE) using its solution data. Instead of identifying the terms in the underlying PDE, we seek to approximate the evolution operator of the underlying PDE numerically. The evolution operator of the PDE, defined in infinite-dimensional space, map...

In this work we propose a numerical framework for uncertainty quantification (UQ) for time-dependent problems with neural network surrogates. The new appoach is based on approximating the exact time-integral form of the equation by neural networks, of which the structure is an adaptation of the residual network. The network is trained with data gen...

A non-intrusive bifidelity strategy [1] is applied to the computation of statistics of a quantity of interest (QoI) which depends in a non-smooth way upon the stochastic parameters. The procedure leverages the accuracy of a high-fidelity model and the efficiency of a low-fidelity model, obtained through the use of different levels of numerical reso...

We present an iterative algorithm for approximating an unknown function sequentially using random samples of the function values and gradients. This is an extension of the recently developed sequential approximation (SA) method, which approximates a target function using samples of function values only. The current paper extends the development of...

We present a numerical approach for approximating unknown Hamiltonian systems using observation data. A distinct feature of the proposed method is that it is structure-preserving, in the sense that it enforces conservation of the reconstructed Hamiltonian. This is achieved by directly approximating the underlying unknown Hamiltonian, rather than th...

Wave propagation problems for heterogeneous media are known to have many applications in physics and engineering. Recently, there has been an increasing interest in stochastic effects due to the uncertainty, which may arise from impurities of the media. This work considers a two-dimensional wave equation with random coefficients which may be discon...

We present a numerical framework for approximating unknown governing equations using observation data and deep neural networks (DNN). In particular, we propose to use residual network (ResNet) as the basic building block for equation approximation. We demonstrate that the ResNet block can be considered as a one-step method that is exact in temporal...

We present effective numerical algorithms for locally recovering unknown governing differential equations from measurement data. We employ a set of standard basis functions, e.g., polynomials, to approximate the governing equation with high accuracy. Upon recasting the problem into a function approximation problem, we discuss several important aspe...

Pore-scale simulation is an essential tool to understand complex physical process in many environmental problems. However, structural heterogeneity and data scarcity render the porous medium, and in turn its macroscopic properties, uncertain. Meanwhile, direct numerical simulation of the medium at the fine scale often incurs high computational cost...

We present an explicit construction for feedforward neural network (FNN), which provides a piecewise constant approximation for multivariate functions. The proposed FNN has two hidden layers, where the weights and thresholds are explicitly defined and do not require numerical optimization for training. Unlike most of the existing work on explicit F...

A common challenge in systems biology is quantifying the effects of unknown parameters and estimating parameter values from data. For many systems, this task is computationally intractable due to expensive model evaluations and large numbers of parameters. In this work, we investigate a new method for performing sensitivity analysis and parameter e...

Model 1 parameter correlations. Correlations between the 8 kinetic parameters in the MCMC chain using the polynomial surrogate of Model 1. (EPS)

Model 2 parameter correlations. Correlations between the parameters in the MCMC chain using the 15-dimensional polynomial surrogate of Model 2. (EPS)

Parameter values for model 1. Parameter values are taken from [37]. Values for kGa and kGd were estimated based on least-squares fit to time course and dose-response data. (PDF)

Experimental data. Experimental data for the given time points and α-factor levels from [37], and the resulting data. Output is the fraction of free Gβγ (Gbg/Gt). Data are given as mean ± standard deviation. (PDF)

Parameter ranges for model 1. Ranges for the kinetic parameters used for parameter estimation of all 8 parameters in Model 1 (heterotrimeric G-protein model). (PDF)

Sensitivity coefficients for model 2. Sensitivity coefficients, in order of ascending magnitude, from sensitivity analysis of all 35 parameters in Model 2 using a 5th order surrogate polynomial fit to 5000 sample points. (PDF)

Schematic reaction diagram of the yeast mating signal transduction pathway. Arrows indicate the conversion of protein species from inactive to active form or from cytoplasmic localization to membrane localization (where the protein is active). Solid dots represent reactions catalyzed by the connected proteins. Lines terminating in a vertical bar (i...

Model 2 35-dimensional polynomial error. Error in the 35-dimensional polynomial surrogate function for Model 2 fit using 5000 points, and measured (tested) at 500 uniform random samples. (EPS)

Model 2 15-dimensional polynomial error. Error in the 15-dimensional polynomial surrogate function for Model 2 fit using 6000 points, and measured (tested) via 10-fold cross-validation. (EPS)

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a bounded interval, as opposed to th...

Wave propagation problems for heterogeneous media are known to have many applications in physics and engineering. Recently, there has been an increasing interest in stochastic effects due to the uncertainty, which may arise from impurities of the media. This work considers a two-dimensional wave equation with random coefficients which may be discon...

We present a sequential method for approximating an unknown function sequentially using random noisy samples. Unlike the traditional function approximation methods, the current method constructs the approximation using one sample at a time. This results in a simple numerical implementation using only vector operations and avoids the need to store t...

We present a randomized iterative method for approximating unknown function sequentially on arbitrary point set. The method is based on a recently developed sequential approximation (SA) method, which approximates a target function using one data point at each step and avoids matrix operations. The focus of this paper is on data sets with highly ir...

Stochastic collocation (SC) has become one of the major computational tools for uncertainty quantification. Its primary advantage lies in its ease of implementation. To carry out SC, one needs only a reliable deterministic simulation code that can be run repetitively at different parameter values. And yet, the modern-day SC methods can retain the h...

This paper is concerned with generalized polynomial chaos (gPC) approximation for first-order quasilinear hyperbolic systems with uncertainty. The one-dimensional (1D) hyperbolic system is first symmetrized with the aid of left eigenvector matrix of the Jacobian matrix. Then the gPC stochastic Galerkin method is applied to derive a provably symmetr...

We present an efficient numerical algorithm to approximate the statistical moments of stochastic problems, in the presence of models with different fidelities. The method extends the multi-fidelity approximation method developed in [18,26]. By combining the efficiency of low-fidelity models and the accuracy of high-fidelity models, our method exhib...

We discuss the properties of sparse approximation using ℓ1 - ℓ2 minimization. We present several theoretical estimates regarding its recoverability for both sparse and nonsparse signals. We then apply the method to sparse orthogonal polynomial approximations for stochastic collocation, with a focus on the use of Legendre polynomials. We study the r...

The randomized Kaczmarz (RK) method is a randomized iterative algorithm for solving (overdetermined) linear systems of equations. In this paper, we extend the RK method to function approximation in a bounded domain. We demonstrate that by conducting the approximation randomly one sample at a time the method converges. Convergence analysis is conduc...

We present a numerical method for polynomial approximation of multivariate functions. The method utilizes Gauss quadrature in tensor product form, which is known to be inefficientin high dimensions. Here we demonstrate that by using a new randomized algorithm and taking advantage of the tensor structure of the grids, a highly efficient algorithm ca...

We present a sampling strategy of least squares polynomial regression. The strategy combines two recently developed methods for least squares method: Christoffel least squares algorithm and quasi-optimal sampling. More specifically, our new strategy first choose samples from the pluripotential equilibrium measure and then re-order the samples by th...

We propose a generalized polynomial chaos based stochastic Galerkin methods for scalar hyperbolic balance laws with random geometric source terms or random initial data. This method is well-balanced (WB), in the sense that it captures the stochastic steady state solution with high order accuracy. The framework of the stochastic WB schemes is presen...

In this paper we present a strategy for correcting model deficiency using observational data. We first present the model correction in a general form, involving both external correction and internal correction. The model correction problem is then parameterized and casted into an optimization problem, from which the parameters are determined. More...

This paper is concerned with generalized polynomial chaos (gPC) approximation for a general system of quasilinear hyperbolic conservation laws with uncertainty. The one-dimensional (1D) hyperbolic system is first symmetrized with the aid of left eigenvector matrix of the Jacobian matrix. Stochastic Galerkin method is then applied to derive the equa...

In this paper we present a quasi-optimal sample set for ordinary least squares (OLS) regression. The quasi-optimal set is designed in such a way that, for a given number of samples, it can deliver the regression result as close as possible to the result obtained by a (much) larger set of candidate samples. The quasi-optimal set is determined by max...

Peer review is an integral part of safeguarding the fairness of scientific proposal evaluations, but it is also subject to uncertainty, such as reviewer’s bias and potential discussion under the table, termed "DaZhaoHu" in Chinese. In this paper, we present a mathematical framework to model the peer review process. Through sensitivity analysis, we...

We discuss the problem of constructing an accurate function approximation when data are corrupted by unexpected errors. The unexpected corruption errors are different from the standard observational noise in the sense that they can have much larger magnitude and in most cases are sparse. By focusing on overdetermined case, we prove that the sparse...

We discuss the problem of polynomial approximation of multivariate functions using discrete least squares collocation. The problem stems from uncertainty quantification (UQ), where the independent variables of the functions are random variables with specified probability measure. We propose to construct the least squares approximation on points ran...