No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Application of Dimension Truncation Error Analysis to High-Dimensional Function Approximation in Uncertainty Quantification
ResearchGate has not been able to resolve any citations for this publication.
We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem—and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.
We consider uncertainty quantification for the Poisson problem subject to domain uncertainty. For the stochastic parameterization of the random domain, we use the model recently introduced by Kaarnioja et al. (SIAM J. Numer. Anal., 2020) in which a countably infinite number of independent random variables enter the random field as periodic functions. We develop lattice quasi-Monte Carlo (QMC) cubature rules for computing the expected value of the solution to the Poisson problem subject to domain uncertainty. These QMC rules can be shown to exhibit higher order cubature convergence rates permitted by the periodic setting independently of the stochastic dimension of the problem. In addition, we present a complete error analysis for the problem by taking into account the approximation errors incurred by truncating the input random field to a finite number of terms and discretizing the spatial domain using finite elements. The paper concludes with numerical experiments demonstrating the theoretical error estimates.
This paper deals with the kernel-based approximation of a multivariate periodic function by interpolation at the points of an integration lattice—a setting that, as pointed out by Zeng et al. (Monte Carlo and Quasi-Monte Carlo Methods 2004, Springer, New York, 2006) and Zeng et al. (Constr. Approx. 30: 529–555, 2009), allows fast evaluation by fast Fourier transform, so avoiding the need for a linear solver. The main contribution of the paper is the application to the approximation problem for uncertainty quantification of elliptic partial differential equations, with the diffusion coefficient given by a random field that is periodic in the stochastic variables, in the model proposed recently by Kaarnioja et al. (SIAM J Numer Anal 58(2): 1068–1091, 2020). The paper gives a full error analysis, and full details of the construction of lattices needed to ensure a good (but inevitably not optimal) rate of convergence and an error bound independent of dimension. Numerical experiments support the theory.
Code available: https://github.com/yifanc96/NonlinearPDEs-GPsolver.
We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs, (2) has guaranteed convergence with a path to compute error bounds in the PDE setting, and (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE with a MAP estimator of a Gaussian process given the observation of the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has a quadratic loss and nonlinear constraints, and it is in turn solved with a variant of the Gauss-Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) is found in practice to converge in a small number (two to ten) of iterations in experiments conducted on a range of PDEs. For IPs, while the traditional approach has been to iterate between the identifications of parameters in the PDE and the numerical approximation of its solution, our algorithm tackles both simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework.
We perform a comprehensive numerical study of the effect of approximation-theoretical results for neural networks on practical learning problems in the context of numerical analysis. As the underlying model, we study the machine-learning-based solution of parametric partial differential equations. Here, approximation theory for fully-connected neural networks predicts that the performance of the model should depend only very mildly on the dimension of the parameter space and is determined by the intrinsic dimension of the solution manifold of the parametric partial differential equation. We use various methods to establish comparability between test-cases by minimizing the effect of the choice of test-cases on the optimization and sampling aspects of the learning problem. We find strong support for the hypothesis that approximation-theoretical effects heavily influence the practical behavior of learning problems in numerical analysis. Turning to practically more successful and modern architectures, at the end of this study we derive improved error bounds by focusing on convolutional neural networks.
We consider the forward problem of uncertainty quantification for the generalised Dirichlet eigenvalue problem for a coercive second order partial differential operator with random coefficients, motivated by problems in structural mechanics, photonic crystals and neutron diffusion. The PDE coefficients are assumed to be uniformly bounded random fields, represented as infinite series parametrised by uniformly distributed i.i.d. random variables. The expectation of the fundamental eigenvalue of this problem is computed by (a) truncating the infinite series which define the coefficients; (b) approximating the resulting truncated problem using lowest order conforming finite elements and a sparse matrix eigenvalue solver; and (c) approximating the resulting finite (but high dimensional) integral by a randomly shifted quasi-Monte Carlo lattice rule, with specially chosen generating vector. We prove error estimates for the combined error, which depend on the truncation dimension s, the finite element mesh diameter h, and the number of quasi-Monte Carlo samples N. Under suitable regularity assumptions, our bounds are of the particular form , where is arbitrary and the hidden constant is independent of the truncation dimension, which needs to grow as and . As for the analogous PDE source problem, the conditions under which our error bounds hold depend on a parameter representing the summability of the terms in the series expansions of the coefficients. Although the eigenvalue problem is nonlinear, which means it is generally considered harder than the source problem, in almost all cases () we obtain error bounds that converge at the same rate as the corresponding rate for the source problem. The proof involves a detailed study of the regularity of the fundamental eigenvalue as a function of the random parameters. As a key intermediate result in the analysis, we prove that the spectral gap (between the fundamental and the second eigenvalues) is uniformly positive over all realisations of the random problem.
We analyze combined Quasi-Monte Carlo quadrature and Finite Element approximations in Bayesian estimation of solutions to countably-parametric operator equations with holomorphic dependence on the parameters as considered in [Cl.~Schillings and Ch.~Schwab: Sparsity in Bayesian Inversion of Parametric Operator Equations. Inverse Problems, {\bf 30}, (2014)]. Such problems arise in numerical uncertainty quantification and in Bayesian inversion of operator equations with distributed uncertain inputs, such as uncertain coefficients, uncertain domains or uncertain source terms and boundary data. We show that the parametric Bayesian posterior densities belong to a class of weighted Bochner spaces of functions of countably many variables, with a particular structure of the QMC quadrature weights: up to a (problem-dependent, and possibly large) finite dimension S product weights can be used, and beyond this dimension, weighted spaces with so-called SPOD weights are used to describe the solution regularity. We establish error bounds for higher order Quasi-Monte Carlo quadrature for the Bayesian estimation based on [J.~Dick, Q.T.~LeGia and Ch.~Schwab, Higher order Quasi-Monte Carlo integration for holomorphic, parametric operator equations, Report 2014-23, SAM, ETH Z\"urich]. It implies, in particular, regularity of the parametric solution and of the countably-parametric Bayesian posterior density in SPOD weighted spaces. This, in turn, implies that the Quasi-Monte Carlo quadrature methods in [J. Dick, F.Y.~Kuo, Q.T.~Le Gia, D.~Nuyens, Ch.~Schwab, Higher order QMC Galerkin discretization for parametric operator equations, SINUM (2014)] are applicable to these problem classes, with dimension-independent convergence rates \calO(N^{-1/p}) of N-point HoQMC approximated Bayesian estimates, where depends only on the sparsity class of the uncertain input in the Bayesian estimation.
Deterministic Galerkin approximations of a class of second order elliptic PDEs with random coefficients on a bounded domain
D⊂ℝ
d
are introduced and their convergence rates are estimated. The approximations are based on expansions of the random diffusion
coefficients in L
2(D)-orthogonal bases, and on viewing the coefficients of these expansions as random parameters y=y(ω)=(y
i
(ω)). This yields an equivalent parametric deterministic PDE whose solution u(x,y) is a function of both the space variable x∈D and the in general countably many parametersy.
We establish new regularity theorems describing the smoothness properties of the solution u as a map from y∈U=(−1,1)∞ to V=H10(D)V=H^{1}_{0}(D). These results lead to analytic estimates on the V norms of the coefficients (which are functions of x) in a so-called “generalized polynomial chaos” (gpc) expansion of u.
Convergence estimates of approximations of u by best N-term truncated V valued polynomials in the variable y∈U are established. These estimates are of the form N
−r
, where the rate of convergence r depends only on the decay of the random input expansion. It is shown that r exceeds the benchmark rate 1/2 afforded by Monte Carlo simulations with N “samples” (i.e., deterministic solves) under mild smoothness conditions on the random diffusion coefficients.
A class of fully discrete approximations is obtained by Galerkin approximation from a hierarchic family {Vl}l=0¥ Ì V\{V_{l}\}_{l=0}^{\infty}\subset V of finite element spaces in D of the coefficients in the N-term truncated gpc expansions of u(x,y). In contrast to previous works, the level l of spatial resolution is adapted to the gpc coefficient. New regularity theorems describing the smoothness properties of
the solution u as a map from y∈U=(−1,1)∞ to a smoothness space W⊂V are established leading to analytic estimates on the W norms of the gpc coefficients and on their space discretization error. The space W coincides with H2(D)ÇH10(D)H^{2}(D)\cap H^{1}_{0}(D) in the case where D is a smooth or convex domain.
Our analysis shows that in realistic settings a convergence rate Ndof-sN_{\mathrm{dof}}^{-s} in terms of the total number of degrees of freedom N
dof can be obtained. Here the rate s is determined by both the best N-term approximation rate r and the approximation order of the space discretization inD.
KeywordsStochastic and parametric elliptic equations-Wiener polynomial chaos-Approximation rates-Nonlinear approximation-Sparsity
Mathematics Subject Classification (2000)41A-65N-65C30
Splines are minimum-norm approximations to functions that interpolate the given data, (xi, f(xi)). Examples of multidimensional splines are those based on radial basis functions. This paper considers splines of a similar
form but where the kernels are not necessarily radially symmetric. The designs, {xi}, considered here are node-sets of integration lattices. Choosing the kernels to be periodic facilitates the computation
of the spline and the error analysis. The error of the spline is shown to depend on the smoothness of the function being approximated
and the quality of the lattice design. The quality measure for the lattice design is similar, but not equivalent to, traditional
quality measures for lattice integration rules.
We consider linear systems A(α)x(α) = b(α) depending on possibly many parameters α = (α1,...,αp). Solving these systems simultaneously for a standard discretization of the parameter space would require a computational effort growing exponentially in the number of parameters. We show that this curse of dimensionality can be avoided for sufficiently smooth parameter dependencies. For this purpose, computational methods are developed that benefit from the fact that x(α) can be well approximated by a tensor of low rank. In particular, low-rank tensor variants of short-recurrence Krylov subspace methods are presented. Numerical experiments for deterministic PDEs with parametrized coefficients and stochastic elliptic PDEs demonstrate the effectiveness of our approach.
We present a new method for solving stochastic differential equations based on Galerkin projections and extensions of Wiener's polynomial chaos. Specifically, we represent the stochastic processes with an optimum trial basis from the Askey family of orthogonal polynomials that reduces the dimensionality of the system and leads to exponential convergence of the error. Several continuous and discrete processes are treated, and numerical examples show substantial speed-up compared to Monte Carlo simulations for low dimensional stochastic inputs.
In this paper we consider (hierarchical, Lagrange) reduced basis approximation and a posteriori error estimation for linear functional outputs of affinely parametrized elliptic coercive partial differential equations. The essential ingredients are (primal-dual) Galerkin projection onto a low- dimensional space associated with a smooth ``parametric manifold'' --- dimension reduction; efficient and effective greedy sampling methods for identification of optimal and numerically stable approximations --- rapid convergence; a posteriori error estimation procedures --- rigorous and sharp bounds for the linear-functional outputs of interest; and Offline-Online computational decomposition strategies --- minimum marginal cost for high performance in the real-time/embedded (e.g., parameter-estimation, control) and many-query (e.g., design optimization, multi-model scale) contexts. We present illustrative results for heat conduction and convection- diffusion, inviscid flow, and linear elasticity; outputs include transport rates, added mass, and stress intensity factors.
In recent work it has been established that deep neural networks (DNNs) are capable of approximating solutions to a large class of parabolic partial differential equations without incurring the curse of dimension. However, all this work has been restricted to problems formulated on the whole Euclidean domain. On the other hand, most problems in engineering and in the sciences are formulated on finite domains and subjected to boundary conditions. The present paper considers an important such model problem, namely the Poisson equation on a domain subject to Dirichlet boundary conditions. It is shown that DNNs are capable of representing solutions of that problem without incurring the curse of dimension. The proofs are based on a probabilistic representation of the solution to the Poisson equation as well as a suitable sampling method.
We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. Numerically we demonstrate the effectiveness of the method on a class of parametric elliptic PDE problems, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare our method with existing algorithms from the literature.
We estimate the expressive power of certain deep neural networks (DNNs for short) on a class of countably-parametric, holomorphic maps u: U. on the parameter domain U = [-1, 1].,•. Dimension-independent rates of best n-term truncations of generalized polynomial chaos (gpc for short) approximations depend only on the summability exponent of the sequence of their gpc expansion coefficients. So-called (b,)-holomorphic maps u, with b .,"p for some p (0, 1), are known to allow gpc expansions with coefficient sequences in .,"p. Such maps arise for example as response surfaces of parametric PDEs, with applications in PDE uncertainty quantification (UQ) for many mathematical models in engineering and the sciences. Up to logarithmic terms, we establish the dimension independent approximation rate s = 1/p-1 for these functions in terms of the total number N of units and weights in the DNN. It follows that certain DNN architectures can overcome the curse of dimensionality when expressing possibly countably-parametric, real-valued maps with a certain degree of sparsity in the sequences of their gpc expansion coefficients. We also obtain rates of expressive power of DNNs for countably-parametric maps u: U' V, where V is the Hilbert space H01([0, 1]).
An application of quasi-Monte Carlo methods of significant recent interest in the MCQMC community is the quantification of uncertainties in partial differential equation models. Uncertainty quantification for both forward problems and Bayesian inverse problems leads to high-dimensional integrals that are well-suited for QMC approximation. One of the approximations required in a general formulation as an affine-parametric operator equation is the truncation of the formally infinite-parametric operator to a finite number of dimensions. To date, a numerical study of the available theoretical convergence rates for this error have to the author’s knowledge not been published. We present novel results for a selection of model problems, the computation of which has been enabled by recently developed, higher-order QMC methods based on interlaced polynomial lattice rules. Surprisingly, the observed rates are one order better in the case of integration over the parameters than the commonly cited theory suggests; a proof of this higher rate is included, resulting in a theoretical statement consistent with the observed numerics.
We consider a class of parametric operator equations where the involved parameters could either be of deterministic or stochastic nature. In both cases we focus on scenarios involving a large number of parameters. Typical strategies for addressing the challenges posed by high dimensionality use low-rank approximations of solutions based on a separation of spatial and parametric variables. One such strategy is based on performing sparse best n-term approximations of the solution map in an a priori chosen system of tensor product form in the parametric variables. This approach has been extensively analyzed in the case of tensor product Legendre polynomial bases, for which approximation rates have been established. The objective of this paper is to investigate what can be gained by exploiting further low rank structures, in particular using optimized systems of basis functions obtained by singular value decomposition techniques. On the theoretical side, we show that optimized low-rank expansions can either bring significant or no improvement over sparse polynomial expansions, depending on the type of parametric problem. On the computational side, we analyze an adaptive solver which, at near-optimal computational cost for this type of approximation, exploits low-rank structure as well as sparsity of basis expansions.
In this paper quasi-Monte Carlo (QMC) methods are applied to a class of elliptic partial differential equations (PDEs) with random coefficients, where the random coefficient is parametrized by a countably infinite number of terms in a Karhunen-Loève expansion. Models of this kind appear frequently in numerical models of physical systems, and in uncertainty quantification. The method uses a QMC method to estimate expected values of linear functionals of the exact or approximate solution of the PDE, with the expected value considered as an infinite dimensional integral in the parameter space corresponding to the randomness induced by the random coefficient. The error analysis, arguably the first rigorous application of the modern theory of QMC in weighted spaces, exploits the regularity with respect to both the physical variables (the variables in the physical domain) and the parametric variables (the parameters corresponding to randomness). In the weighted-space theory of QMC methods, "weights," describing the varying difficulty of different subsets of the variables, are introduced in order to make sure that the high-dimensional integration problem is tractable. It turns out that the weights arising from the present analysis are of a nonstandard kind, being of neither product nor order dependent form, but instead a hybrid of the two-we refer to these as "product and order dependent weights," or "POD weights" for short. These POD weights are of a simple enough form to permit a component-by-component construction of a randomly shifted lattice rule that has optimal convergence properties for the given weighted space setting. If the terms in the expansion for the random coefficient have an appropriate decay property, and if we choose POD weights that minimize a certain upper bound on the error, then the solution of the PDE belongs to the joint function space needed for the analysis, and the QMC error (in the sense of a root-mean-square error averaged over shifts) is of order O(N-1+δ) for arbitrary δ > 0, where N denotes the number of sampling points in the parameter space. Moreover, this convergence rate and slower convergence rates under more relaxed conditions are achieved under conditions similar to those found recently by Cohen, De Vore, and Schwab [Found. Comput. Math., 10 (2010), pp. 615- 646] for the same model by best N-term approximation. We analyze the impact of a finite element (FE) discretization on the overall efficiency of the scheme, in terms of accuracy versus overall cost, with results that are comparable to those of the best N-term approximation.
We develop the generic "Multivariate Decomposition Method" (MDM) for weighted
integration of functions of infinitely many variables .
The method works for functions that admit a decomposition
, where runs over all
finite subsets of positive integers, and for each
the function depends only
on . Moreover, we assume that
belongs to a normed space , and that a bound on
is known. We also specialize MDM to
being the -tensor product of an anchored
reproducing kernel Hilbert space, or a particular non-Hilbert space.
Splines are minimum-norm approximations of functions that interpolate the given data (x
l
,f(x
l
)),l=0,…,N−1. This paper considers spline approximations to functions in certain reproducing kernel Hilbert spaces, with special choices of the designs {x
l
,l=0,…,N−1}. The designs considered here are node sets of integration lattices and digital nets, and the domain of the functions to be approximated, f is the d-dimensional unit cube. The worst-case errors (in the L
∞ norm) of the splines are shown to depend on the smoothness or digital smoothness of the functions being approximated. Although the convergence rates may be less than ideal, the algorithms are constructive and they do not suffer from a curse of dimensionality.
Lattice generating vectors
- F Y Kuo
Strong and weak error estimates for elliptic partial differential equations with random coefficients
- J Charrier