No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Existence and Optimality Conditions for Risk-Averse PDE-Constrained Optimization
Abstract
Uncertainty is ubiquitous in virtually all engineering applications, and, for such problems, it is inadequate to simulate the underlying physics without quantifying the uncertainty in unknown or random inputs, boundary and initial conditions, and modeling assumptions. In this work, we introduce a general framework for analyzing risk-averse optimization problems constrained by partial differential equations (PDEs). In particular, we postulate conditions on the random variable objective function as well as the PDE solution that guarantee existence of minimizers. Furthermore, we derive optimality conditions and apply our results to the control of an environmental contaminant. Finally, we introduce a new risk measure, called the conditional entropic risk, that fuses desirable properties from both the conditional value-at-risk and the entropic risk measures. © 2018 National Technology and Engineering Solutions of Sandia, LLC, operator of Sandia National Laboratories for the U.S. Department of Energy.
... Coherent risk measures are popular as numerous desirable properties can be derived from the above conditions (see, e.g., [29] and the references therein). However, it can be shown (see [30,Theorem 1]) that the only coherent risk measures that are Fréchet differentiable are linear ones. ...
... With the uniformly boundedly invertible forward operator B y , our setting fits into the abstract framework of [29] where the authors derive existence and optimality conditions for PDE-constrained optimization under uncertainty. In particular, the forward operator B y , the regularization term α 3 2 z 2 L 2 (V ;I ) and the random variable trackingtype objective function y satisfy the assumptions of [29,Proposition 3.12]. ...
... With the uniformly boundedly invertible forward operator B y , our setting fits into the abstract framework of [29] where the authors derive existence and optimality conditions for PDE-constrained optimization under uncertainty. In particular, the forward operator B y , the regularization term α 3 2 z 2 L 2 (V ;I ) and the random variable trackingtype objective function y satisfy the assumptions of [29,Proposition 3.12]. In order to present the result about the existence and uniqueness of the solution of (3.4), which is based on [29, Proposition 3.12], we recall some definitions from convex analysis (see, e.g., [29] and the references therein): A functional R : ...
We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem—and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.
... On the other end, one is rather interested in absolutely safe decisions and addresses uncertainty via almost sure constraints [13,14]. In between these two extremes, reasonable compromises can be made based on several risk-averse approaches such as conditional value-at-risk (CVaR), value-at-risk (VaR), stochastic dominance, or robust constraints [1,9,22,23,26]. Besides modeling, numerical solution, and structural analysis, there have been efforts in the past few years to derive optimality conditions for problems involving random PDEs, see [23] (without state constraints, meaning additional constraints on the solution of the PDE) or [12,13,14] (with almost sure state constraints). ...
... In between these two extremes, reasonable compromises can be made based on several risk-averse approaches such as conditional value-at-risk (CVaR), value-at-risk (VaR), stochastic dominance, or robust constraints [1,9,22,23,26]. Besides modeling, numerical solution, and structural analysis, there have been efforts in the past few years to derive optimality conditions for problems involving random PDEs, see [23] (without state constraints, meaning additional constraints on the solution of the PDE) or [12,13,14] (with almost sure state constraints). ...
... The technical results of the previous section allows us now to formulate a fully explicit (in terms of the problem data) upper estimate for the subdifferential of the probability function in problem (23) and explicit necessary optimality conditions for that same problem. Combining Corollary 2.9 with Lemmas 2.11 and 2.12, we end up with Theorem 2.13. ...
In this paper, we discuss optimality conditions for optimization problems subject to random state constraints, which are modeled in probabilistic or almost sure form. While the latter can be understood as the limiting case of the former, the derivation of optimality conditions requires substantially different approaches. We apply them to a linear elliptic partial differential equation (PDE) with random inputs. In the probabilistic case, we rely on the spherical-radial decomposition of Gaussian random vectors in order to formulate fully explicit optimality conditions involving a spherical integral. In the almost sure case, we derive optimality conditions and compare them to a model based on robust constraints with respect to the (compact) support of the given distribution.
... The other risk measure considered in this work is the conditional value-at-risk (CVaR) (Rockafellar & Uryasev, 2000. The CVaR at confidence level β ∈ (0, 1), denoted CVaR β , is a well-established decision-making tool in finance (Krokhmal et al., 2002;Shapiro et al., 2009), and is becoming increasingly prominent in engineering (Rockafellar & Royset, 2015;Kouri & Surowiec, 2016;Yang & Gunzburger, 2017;Kouri & Surowiec, 2018;Chaudhuri et al., 2020a,b). In this work, stochastic programs featuring the risk measure R = E are referred to as risk-neutral; meanwhile, those involving the risk measure R = CVaR β are referred to as risk-averse. ...
... In these high-cost scenarios, one wishes to evaluate as few samples as possible. It is well established that the expected risk (1.2) is often unsuitable to predict immediate and long-term performance, manufacturing and maintenance costs, system response, levels of damage and numerous other quantities of interest (Rockafellar & Royset, 2010;Kouri & Surowiec, 2016;Kouri & Shapiro, 2018;Kouri & Surowiec, 2018). Therefore, today's industrial problems are made even more challenging because they typically require a risk-averse formulation (Rockafellar & Royset, 2015;Kodakkal et al., 2022). ...
... Rockafellar & Uryasev (2000 and Section 4. This observation informs the layout of the paper by allowing us to first focus on the case R = E and then deal with the treatment of risk-averse problems in the later sections. A large family of other important risk measures, including the entropic risk and the conditional entropic risk (Kouri & Surowiec, 2018), have a similar reformulation involving E (Rockafellar & Uryasev, 2013;Kouri & Surowiec, 2018), which leads us to conclude that there is little loss of generality in treating (1.1) in this incremental and case-specific way. Likewise, in order to develop our sample size conditions and then analyze the corresponding algorithms, we begin with a convex constraint set C ⊆ R n . ...
We introduce adaptive sampling methods for stochastic programs with deterministic constraints. First, we propose and analyze a variant of the stochastic projected gradient method, where the sample size used to approximate the reduced gradient is determined on-the-fly and updated adaptively. This method is applicable to a broad class of expectation-based risk measures, and leads to a significant reduction in the individual gradient evaluations used to estimate the objective function gradient. Numerical experiments with expected risk minimization and conditional value-at-risk minimization support this conclusion, and demonstrate practical performance and efficacy for both risk-neutral and risk-averse problems. Second, we propose an SQP-type method based on similar adaptive sampling principles. The benefits of this method are demonstrated in a simplified engineering design application, featuring risk-averse shape optimization of a steel shell structure subject to uncertain loading conditions and model uncertainty.
... Optimality conditions with an application to PDE-constrained optimization under uncertainty were first presented in [29]. While the framework in [29] allows for generally nonconvex problems, the case where the feasible set X 2,ad (ω) contains conical constraints is not covered. ...
... Optimality conditions with an application to PDE-constrained optimization under uncertainty were first presented in [29]. While the framework in [29] allows for generally nonconvex problems, the case where the feasible set X 2,ad (ω) contains conical constraints is not covered. Our work is related to a recent paper [19]. ...
... Remark 3.2. The growth condition (3.1) in combination with the Carathéodory property ensures that J 2 : X → L p (Ω) is continuous; see Theorem 3.5 of [29]. Continuity in the case p = ∞ is possible under additional assumptions; see Theorem 3.17 of [3]. ...
We analyze a potentially risk-averse convex stochastic optimization problem, where the control is deterministic and the state is a Banach-valued essentially bounded random variable. We obtain strong forms of necessary and sufficient optimality conditions for problems subject to equality and conical constraints. We propose a Moreau--Yosida regularization for the conical constraint and show consistency of the optimality conditions for the regularized problem as the regularization parameter is taken to infinity.
... [48] and [42] and the references therein. Modeling choices in engineering were explored in [43] and their application to PDE-constrained optimisation was popularised in [31,32,30]. However, these papers typically require S to be Fréchet differentiable, which does not hold in general for solution operators of variational inequalities. ...
... . Now we argue similarly to the corrigendum to [31]. It follows that there exists a ν ∈ ∂R(J (y * τ )) and ...
... An adjoint equation can be obtained, like in [31] and [30], by setting (T τ (u * τ )) * J (y * τ ) = B * p * τ = B * (·)p * τ (·) with p * τ taking the role of the adjoint variable (this allows for the formulation of more explicit optimality conditions). Doing so, we get the following. ...
We consider a risk-averse optimal control problem governed by an elliptic variational inequality (VI) subject to random inputs. By deriving KKT-type optimality conditions for a penalised and smoothed problem and studying convergence of the stationary points with respect to the penalisation parameter, we obtain two forms of stationarity conditions. The lack of regularity with respect to the uncertain parameters and complexities induced by the presence of the risk measure give rise to new challenges unique to the stochastic setting. We also propose a path-following stochastic approximation algorithm using variance reduction techniques and demonstrate the algorithm on a modified benchmark problem.
... If not taken into account, the inherent uncertainty of such problems has the potential to render worthless any solutions obtained using state-of-the-art methods for deterministic problems. The careful analysis of the uncertainty in PDE-constrained optimization is hence indispensable and a growing field of research (see, e.g., [4,5,6,15,26,27,33,34,42,44,45]). ...
... Coherent risk measures are popular as numerous desirable properties can be derived from the above conditions (see, e.g., [26] and the references therein). However, it can be shown (see [27,Theorem 1]) that the only coherent risk measures that are Fréchet differentiable are linear ones. ...
... With the uniformly boundedly invertible forward operator B y , our setting fits into the abstract framework of [26] where the authors derive existence and optimality conditions for PDE-constrained optimization under uncertainty. In particular, the forward operator B y , the regularization term α 3 2 z 2 L 2 (V ′ ;I) and the random variable tracking-type objective function Φ y satisfy the assumptions of [26,Proposition 3.12]. ...
We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem -- and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.
... Optimality conditions with an application to PDE-constrained optimization under uncertainty were first presented in [28]. While the framework in [28] allows for generally nonconvex problems, the case where the feasible set X 2,ad (ω) contains conical constraints is not covered. ...
... Optimality conditions with an application to PDE-constrained optimization under uncertainty were first presented in [28]. While the framework in [28] allows for generally nonconvex problems, the case where the feasible set X 2,ad (ω) contains conical constraints is not covered. Our work is related to a recent paper [19]. ...
... Remark 2.2. The growth condition (6) in combination with the Carathéodory property ensures that J 2 : X → L p (Ω) is continuous; see [28,Theorem 3.5]. Continuity in the case p = ∞ is possible under additional assumptions; see [3,Theorem 3.17]. ...
We analyze a potentially risk-averse convex stochastic optimization problem, where the control is deterministic and the state is a Banach-valued essentially bounded random variable. We obtain strong forms of necessary and sufficient optimality conditions for problems subject to equality and conical constraints. We propose a Moreau--Yosida regularization for the conical constraint and show consistency of the optimality conditions for the regularized problem as the regularization parameter is taken to infinity.
... In [9], the authors also consider a similar risk measure and propose a reduced basis method on the space of controls to significantly reduce the computational effort. A more general class of risk measures (including the CVaR) for OCPs has been considered in [31], where also the corresponding optimality system of PDEs are derived. The subsequent work [29] introduces a trust-region (Newton) conjugate gradient algorithm combined with an adaptive sparse grid collocation as PDE discretization in the stochastic space for the numerical treatment of these more general OCPs. ...
... Existence and uniqueness results for the OCP (4) can be obtained as a particular case of the more general results in, e.g., the work [31]. We state the result in the next Lemma, without proof. ...
... We have investigated also other choices for (τ j , N j ) in the SG method (31). However, among the cases considered, we have found that (τ j , N j ) = ( τ0 j , N ) leads to the best complexity. ...
We consider the numerical approximation of an optimal control problem for an elliptic Partial Differential Equation (PDE) with random coefficients. Specifically, the control function is a deterministic, distributed forcing term that minimizes the expected squared L 2 misfit between the state (i.e. solution to the PDE) and a target function, subject to a regularization for well posedness. For the numerical treatment of this risk-averse Optimal Control Problem (OCP) we consider a Finite Element discretization of the underlying PDE, a Monte Carlo sampling method, and gradient-type iterations to obtain the approximate optimal control. We provide full error and complexity analyses of the proposed numerical schemes. In particular we investigate the complexity of a conjugate gradient method applied to the fully discretized OCP (so called Sample Average Approximation), in which the Finite Element discretization and Monte Carlo sample are chosen in advance and kept fixed over the iterations. This is compared with a Stochastic Gradient method on a fixed or varying Finite Element discretization, in which the expectation in the computation of the steepest descent direction is approximated by Monte Carlo estimators, independent across iterations, with small sample sizes. We show in particular that the second strategy results in an improved computational complexity. The theoretical error estimates and complexity results are confirmed by numerical experiments.
... Now suppose p < \infty , q < \infty , and the growth condition (3.3) holds. Then the desired result follows from Proposition 3.12 in [39]. On the other hand, if p = q = \infty and the uniform boundedness condition (3.4) holds, then for C = sup n \| \sansz n \| L 2 (\Omega ) (which is finite since \sansz n weakly converges), there exists \gamma = \gamma (C) \geq 0 such that | F n (s)| \leq \gamma a.s. ...
... Proof. If 1 \leq p, q < \infty , then Theorem 3.11 in [39] ensures that \sansu \mapsto \rightar f (\cdot , \sansu (\cdot )) is Fr\' echet differentiable. On the other hand, if p = q = \infty , then r = \infty and we can take K to be constant. ...
... Since \sansz \mapsto \rightar \sansS (\sansz ) is a continuous linear mapping and \sansu \mapsto \rightar f (\cdot , \sansu (\cdot )) is Fr\' echet differentiable, \sansz \mapsto \rightar \widehat f (\sansz ) is Fr\' echet differentiable from L 2 (\Omega ) into L p (\Xi , \scrB , P ). The first-order conditions then follow directly from Corollary 3.14 in [39]. ...
In this paper, we introduce and analyze a new class of optimal control problems constrained by elliptic equations with uncertain fractional exponents. We utilize risk measures to formulate the resulting optimization problem. We develop a functional analytic framework, study the existence of solution, and rigorously derive the first-order optimality conditions. Additionally, we employ a sample-based approximation for the uncertain exponent and the finite element method to discretize in space. We prove the rate of convergence for the optimal risk neutral controls when using quadrature approximation for the uncertain exponent and conclude with illustrative examples. © 2021 National Technology \& Engineering Solutions of Sandia, LLC
... Further discretization approaches for expectations are, for example, stochastic collocation [74] and low-rank tensor approximations [23]. Besides risk-neutral PDE-constrained optimization, risk-averse PDE-constrained optimization [3,17,43,44,55], distributionally robust PDE-constrained optimization [41,59], robust PDE-constrained optimization [5,39,51], and PDE-constrained optimization with chance constraints [16,20,21,26,75] provide approaches to decision making under uncertainty with PDEs. ...
... We refer the reader to [42,Theorem 1] and [44,Proposition 3.12] for theorems on the existence of solutions to risk-averse PDE-constrained optimization problems. For some u 0 ∈ U ad with E J (u 0 , ξ) < ∞ and a scalar ρ ∈ (0, ∞), we define the set ...
We apply the sample average approximation (SAA) method to risk-neutral optimization problems governed by nonlinear partial differential equations (PDEs) with random inputs. We analyze the consistency of the SAA optimal values and SAA solutions. Our analysis exploits problem structure in PDE-constrained optimization problems, allowing us to construct deterministic, compact subsets of the feasible set that contain the solutions to the risk-neutral problem and eventually those to the SAA problems. The construction is used to study the consistency using results established in the literature on stochastic programming. The assumptions of our framework are verified on three nonlinear optimization problems under uncertainty.
... In this section we test the OSA Algorithm 2 on a strongly convex problem arising from the optimal control of a linear elliptic PDE with uncertain coefficients. Our test problem is motivated by the example in [21] and has the form ...
... and f is the sum of five Gaussian sources whose locations, widths and magnitudes are random. The explicit form for β is described in [21,Sect. 4] (where it is denoted by δ). The random inputs ξ are uniformly distributed on [−1, 1] 37 . ...
We consider a class of convex risk-neutral PDE-constrained optimization
problems subject to pointwise control and state constraints. Due to the many
challenges associated with almost sure constraints on pointwise evaluations of
the state, we suggest a relaxation via a smooth functional bound with similar
properties to well-known probability constraints. First, we introduce and
analyze the relaxed problem, discuss its asymptotic properties, and derive
formulae for the gradient using the adjoint calculus. We then build on the
theoretical results by extending a recently published online convex
optimization algorithm (OSA) to the infinite-dimensional setting. Similar to
the regret-based analysis of time-varying stochastic optimization problems, we
enhance the method further by allowing for periodic restarts at pre-defined
epochs. Not only does this allow for larger step sizes, it also proves to be an
essential factor in obtaining high-quality solutions in practice. The behavior
of the algorithm is demonstrated in a numerical example involving a linear
advection-diffusion equation with random inputs. In order to judge the
quality of the solution, the results are compared to those arising from a
sample average approximation (SAA). This is done first by comparing the
resulting cumulative distributions of the objectives at the optimal solution as
a function of step numbers and epoch lengths. In addition, we conduct
statistical tests to further analyze the behavior of the online algorithm and
the quality of its solutions. For a sufficiently large number of steps, the
solutions from OSA and SAA lead to random integrands for the objective and
penalty functions that appear to be drawn from similar distributions.
... In particular, we view u(y) as a function of the right-hand side f of (5). For a function Φ : U → R and some θ ∈ (0, ∞), the entropic risk measure [12] is defined by ...
... Due to convexity of R and α 2 > 0, the functional J is strongly convex so that (12) is a well-posed minimization problem [9,12]. ...
We propose a novel a-posteriori error estimation technique where the target quantities of interest are ratios of high-dimensional integrals, as occur e.g. in PDE constrained Bayesian inversion and PDE constrained optimal control subject to an entropic risk measure. We consider in particular parametric, elliptic PDEs with affine-parametric diffusion coefficient, on high-dimensional parameter spaces. We combine our recent a-posteriori Quasi-Monte Carlo (QMC) error analysis, with Finite Element a-posteriori error estimation. The proposed approach yields a computable a-posteriori estimator which is reliable, up to higher order terms. The estimator's reliability is uniform with respect to the PDE discretization, and robust with respect to the parametric dimension of the uncertain PDE input.
... The Carath\' eodory condition in Assumption 2.3 ensures that the state cost function evaluated at the PDE solution, \omega \mapsto \rightar G(S(\xi (\omega ); z), \xi (\omega )), is a random variable, and owing to Theorem 2.2, the integrability condition in Assumption 2.3 follows, for example, if the uniform boundedness condition in Assumption 3.1 of [19] holds. We seek to minimize the objective function J(z) := \scrR (G(S(\xi ; z), \xi )) + \wp (z), (2.3) where \scrR : \scrX \rightar \BbbR is an OCE risk measure. ...
... In addition, \scrR (X) is finite since \BbbE [v(X)] is. The differentiability claim follows from Assumption 2.5 and Theorem 3.11 in [19]. ...
... Optimality conditions are important in algorithmic development and to understand solutions of optimization problems. For a discussion of this subject, we refer to [265,254,147]; see also the subgradient formulas in Subsection 6.4. Algorithms for computing stationary points of nonconvex, nonsmooth risk-minimization problems emerge from [173]. ...
... Dual expressions for measures of risk and related quantities remain a crucial stepping stone in several contexts. The papers [111,76] utilize them in sensitivity analysis and [147] in derivation of optimality condition; see also [254]. Dual expressions are key in developing algorithms, including in the computationally challenging areas of PDE-constrained optimization [146,148,149] and statistical learning [52,163,155,157]. ...
Uncertainty is prevalent in engineering design, statistical learning, and decision making broadly. Due to inherent risk-averseness and ambiguity about assumptions, it is common to address uncertainty by formulating and solving conservative optimization models expressed using measure of risk and related concepts. We survey the rapid development of risk measures over the last quarter century. From its beginning in financial engineering, we recount their spread to nearly all areas of engineering and applied mathematics. Solidly rooted in convex analysis, risk measures furnish a general framework for handling uncertainty with significant computational and theoretical advantages. We describe the key facts, list several concrete algorithms, and provide an extensive list of references for further reading. The survey recalls connections with utility theory and distributionally robust optimization, points to emerging applications areas such as fair machine learning, and defines measures of reliability.
... It is not necessary for our analysis to restrict ourselves to the tracking-type objective in Assumption 2.(ii). We could proceed in a more general manner as suggested in [44] under appropriate convexity, continuity, and growth conditions. This would require further technical assumptions that we believe would detract from the main purpose of the text. ...
... , see e.g., [44,Ex. 3.2]. ...
A class of risk-neutral generalized Nash equilibrium problems is introduced in which the feasible strategy set of each player is subject to a common linear elliptic partial differential equation with random inputs. In addition, each player’s actions are taken from a bounded, closed, and convex set on the individual strategies and a bound constraint on the common state variable. Existence of Nash equilibria and first-order optimality conditions are derived by exploiting higher integrability and regularity of the random field state variables and a specially tailored constraint qualification for GNEPs with the assumed structure. A relaxation scheme based on the Moreau-Yosida approximation of the bound constraint is proposed, which ultimately leads to numerical algorithms for the individual player problems as well as the GNEP as a whole. The relaxation scheme is related to probability constraints and the viability of the proposed numerical algorithms are demonstrated via several examples.
... Mathematics Subject Classification 49J20 · 49J55 · 60F17 · 65C05 · 90C15 · 35R60 1 Introduction PDE-constrained optimization under uncertainty is a rapidly growing field with a number of recent contributions in theory [9,31,32,34], numerical and computational methods [17,18,30,59], and applications [7,8,11,49]. Nevertheless, a number of Dedication from T.M. Surowiec. ...
Monte Carlo approximations for random linear elliptic PDE constrained optimization problems are studied. We use empirical process theory to obtain best possible mean convergence rates O(n-12) for optimal values and solutions, and a central limit theorem for optimal values. The latter allows to determine asymptotically consistent confidence intervals by using resampling techniques. The theoretical results are illustrated with two sets of numerical experiments. The first demonstrates the theoretical convergence rates for optimal values and optimal solutions. This is complemented by a study illustrating the usage of subsampling bootstrap methods for estimating the confidence intervals.
... In this section we consider an instance of risk-averse OCPUU. This class of problems has recently drawn lot of attention since in engineering applications it is important to compute a control that minimizes the quantity of interest even in rare, but often troublesome, scenarios [2,6,43,44]. As a risk-measure [45], we use the Conditional Value-At-Risk (CVaR) of confidence level λ ∈ (0, 1), ...
In this manuscript, we present a collective multigrid algorithm to solve efficiently the large saddle-point systems of equations that typically arise in PDE-constrained optimization under uncertainty, and develop a novel convergence analysis of collective smoothers and collective two-level methods. The multigrid algorithm is based on a collective smoother that at each iteration sweeps over the nodes of the computational mesh, and solves a reduced saddle-point system whose size is proportional to the number N of samples used to discretized the probability space. We show that this reduced system can be solved with optimal O(N) complexity. The multigrid method is tested both as a stationary method and as a preconditioner for GMRES on three problems: a linear-quadratic problem, possibly with a local or a boundary control, for which the multigrid method is used to solve directly the linear optimality system; a nonsmooth problem with box constraints and L1-norm penalization on the control, in which the multigrid scheme is used as an inner solver within a semismooth Newton iteration; a risk-averse problem with the smoothed CVaR risk measure where the multigrid method is called within a preconditioned Newton iteration. In all cases, the multigrid algorithm exhibits excellent performances and robustness with respect to the parameters of interest.
... Theoretical foundations for the well-posedness of OCPs under uncertainty have been established in, e.g, [21,28,22]. These analyses show that the evaluation of the gradient of J requires the computation of the expectation of a properly defined adjoint variable over the probability distribution of the random inputs. ...
This manuscript presents a framework for using multilevel quadrature formulae to compute the solution of optimal control problems constrained by random partial differential equations. Our approach consists in solving a sequence of optimal control problems discretized with different levels of accuracy of the physical and probability discretizations. The final approximation of the control is then obtained in a postprocessing step, by suitably combining the adjoint variables computed on the different levels. We present a convergence analysis for an unconstrained linear quadratic problem, and detail our framework for the specific case of a Multilevel Monte Carlo quadrature formula. Numerical experiments confirm the better computational complexity of our MLMC approach compared to a standard Monte Carlo sample average approximation, even beyond the theoretical assumptions.
... The resulting gradient could be used in a gradient based optimization problem to find a solution. The optimality conditions in a more general setting and for more elaborate risk measures are discussed in [25]. Ideas from several previous works are drawn upon in this paper. ...
This paper considers the problem of optimizing the average tracking error for an elliptic partial differential equation with an uncertain lognormal diffusion coefficient. In particular, the application of the multilevel quasi-Monte Carlo (MLQMC) method to the estimation of the gradient is investigated, with a circulant embedding method used to sample the stochastic field. A novel regularity analysis of the adjoint variable is essential for the MLQMC estimation of the gradient in combination with the samples generated using the circulant embedding method. A rigorous cost and error analysis shows that a randomly shifted quasi-Monte Carlo method leads to a faster rate of decay in the root mean square error of the gradient than the ordinary Monte Carlo method, while considering multiple levels substantially reduces the computational effort. Numerical experiments confirm the improved rate of convergence and show that the MLQMC method outperforms the multilevel Monte Carlo method and single level quasi-Monte Carlo method.
... Infinite-dimensional optimization problems arise in a plethora of research fields such as dynamic programming [56], statistical estimation [26], feedback stabilization of dynamical systems [54], and optimization problems governed by partial differential equations (PDEs) [49]. PDE-constrained optimization is an active research field with a focus on modeling, analyzing and solving complex optimization problems with PDE constraints. ...
We consider stochastic optimization problems with possibly nonsmooth integrands posed in Banach spaces and approximate these stochastic programs via a sample-based approaches. We establish the consistency of approximate Clarke stationary points of the sample-based approximations. Our framework is applied to risk-averse semilinear PDE-constrained optimization using the average value-at-risk and to risk-neutral bilinear PDE-constrained optimization.
... The proof of Proposition 3.2 is quite similar to the proof given in [31] or [8]. Nevertheless, we report the full proof of Proposition 3.2 in Appendix A.2 for the sake of completeness. ...
In this paper, we consider continuous-time stochastic optimal control problems where the cost is evaluated through a coherent risk measure. We provide an explicit gradient descent-ascent algorithm which applies to problems subject to non-linear stochastic differential equations. More specifically, we leverage duality properties of coherent risk measures to relax the problem via a smooth min-max reformulation which induces artificial strong concavity in the max subproblem. We then formulate necessary conditions of optimality for this relaxed problem which we leverage to prove convergence of the gradient descent-ascent algorithm to candidate solutions of the original problem. Finally, we showcase the efficiency of our algorithm through numerical simulations involving trajectory tracking problems and highlight the benefit of favoring risk measures over classical expectation.
... In this section we consider an instance of risk-adverse OCPUU. This class of problems has recently drawn lot of attention since in engineering applications it is important to compute a control that minimizes the quantity of interest even in rare, but often troublesome, scenarios [20,21,1]. As a risk-measure [35], we use the Conditional Value-At-Risk (CVaR) of confidence level λ, λ ∈ (0, 1), ...
We present a multigrid algorithm to solve efficiently the large saddle-point systems of equations that typically arise in PDE-constrained optimization under uncertainty. The algorithm is based on a collective smoother that at each iteration sweeps over the nodes of the computational mesh, and solves a reduced saddle-point system whose size depends on the number N of samples used to discretized the probability space. We show that this reduced system can be solved with optimal O(N) complexity. We test the multigrid method on three problems: a linear-quadratic problem for which the multigrid method is used to solve directly the linear optimality system; a nonsmooth problem with box constraints and -norm penalization on the control, in which the multigrid scheme is used within a semismooth Newton iteration; a risk-adverse problem with the smoothed CVaR risk measure where the multigrid method is called within a preconditioned Newton iteration. In all cases, the multigrid algorithm exhibits very good performances and robustness with respect to all parameters of interest.
... Despite the rapid development of technologies for controlling internal combustion engine pollution, it is still a major problem in this feld to fnd the optimal amount of purifers [23][24][25][26][27]. Aiming at the above problems, this paper proposes a l 1 -norm PDE constrained optimization algorithm. ...
Aiming at the problem of secondary pollution of waters due to the difficulty of controlling the dosage of purifiers in the treatment of internal combustion engine pollution, a partial differential equation (referred to as PDE) constrained optimization algorithm based on l1-norm is proposed. The algorithm first converts the internal combustion engine control model of the scavenger dose into a constrained optimization problem with a l1-penalty term. Secondly, it introduces a dose constraint condition based on PDE and uses the inherent property of Moreau-Yosida regularization to establish a smooth minimization function. Finally, the semismooth Newton method is used to iteratively find the optimal solution. The results of the comparison experiment show that the algorithm in this paper has a great improvement in the results of Newton step number and dose area percentage.
... We assume that (2.2) is well-posed ( see, e.g. [19,20,25], for sufficient conditions on F , e(·, ·), U ,... for well posedness). ...
We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of Optimal Control Problems (OCPs) constrained by random partial differential equations. The method requires to solve the OCP for several low-fidelity spatial grids and quadrature formulae for the objective functional. All the computed solutions are then linearly combined to get a final approximation which, under suitable regularity assumptions, preserves the same accuracy of fine tensor product approximations, while drastically reducing the computational cost. The combination technique involves only tensor product quadrature formulae, thus the discretized OCPs preserve the convexity of the continuous OCP. Hence, the combination technique avoids the inconveniences of Multilevel Monte Carlo and/or sparse grids approaches, but remains suitable for high dimensional problems. The manuscript presents an a-priori procedure to choose the most important mixed differences and an asymptotic complexity analysis, which states that the asymptotic complexity is exclusively determined by the spatial solver. Numerical experiments validate the results.
... Motivated by recent advances in partial differential equation (PDE)-constrained optimization under uncertainty [22,35], scientific machine learning [9,46], nonconvex stochastic programming [40,49,16], and statistical estimation [54,55,41], we provide such consistency results for stochastic optimization problems in which the decision variables z may be taken in an infinite-dimensional space Z. We will consider more general "risk-averse" problems in which the expectation E P is allowed to be replaced by certain classes of convex risk functionals R. ...
Optimal values and solutions of empirical approximations of stochastic optimization problems can be viewed as statistical estimators of their true values. From this perspective, it is important to understand the asymptotic behavior of these estimators as the sample size goes to infinity, which is both of theoretical as well as practical interest. This area of study has a long tradition in stochastic programming. However, the literature is lacking consistency analysis for problems in which the decision variables are taken from an infinite-dimensional space, which arise in optimal control, scientific machine learning, and statistical estimation. By exploiting the typical problem structures found in these applications that give rise to hidden norm compactness properties for solution sets, we prove consistency results for nonconvex risk-averse stochastic optimization problems formulated in infinite-dimensional space. The proof is based on several crucial results from the theory of variational convergence. The theoretical results are demonstrated for several important problem classes arising in the literature.
... We refer the reader to [41,Thm. 1] and [43,Prop. 3.12] for theorems on the existence of solutions to risk-averse PDE-constrained optimization problems. ...
We apply the sample average approximation (SAA) method to risk-neutral optimization problems governed by nonlinear partial differential equations (PDEs) with random inputs. We analyze the consistency of the SAA optimal values and SAA solutions. Our analysis exploits problem structure in PDE-constrained optimization problems, allowing us to construct deterministic, compact subsets of the feasible set that contain the solutions to the risk-neutral problem and eventually those to the SAA problems. The construction is used to study the consistency using results established in the literature on stochastic programming. The assumptions of our framework are verified on three nonlinear optimization problems under uncertainty.
... As a result, we often analyze the "other approximations" in a setting of finite dimensions and this reduces the required assumptions. We omit an analysis of optimality conditions and the assumptions required to ensure convergence of stationary points of approximating problems to those of the actual problem; see for example [27,19] for efforts in this direction. Our results provide the foundation for numerous algorithms, including those involving adaptive refinements. ...
Computational approaches to PDE-constrained optimization under uncertainty may involve finite-dimensional approximations of control and state spaces, sample average approximations of measures of risk and reliability, smooth approximations of nonsmooth functions, penalty approximations of constraints as well as many other kinds of inaccuracies. In this paper, we analyze the performance of controls obtained by an approximation-based algorithm and in the process develop estimates of optimality gaps for general optimization problems defined on metric spaces. Under mild assumptions, we establish that limiting controls have arbitrarily small optimality gaps provided that the inaccuracies in the various approximations vanish. We carry out the analysis for a broad class of problems with multiple expectation, risk, and reliability functions involving PDE solutions and appearing in objective as well as constraint expressions. In particular, we address problems with buffered failure probability constraints approximated via an augmented Lagrangian. We demonstrate the framework on an elliptic PDE with a random coefficient field and a distributed control function.
... So far, research has mostly been limited to the case where the control (in our notation, the first-stage variable x 1 ) has been subject to additional constraints. In this case, optimality conditions have already been established for risk-averse problems in [27,28]. However, additional constraints on the state (here, x 2 ), beyond a uniquely solvable equation, have yet to be investigated thoroughly. ...
... In such OCPs, the objective functional involves suitable statistical measures, often called risk measures [52,Chapter 6.3], of the quantity of interest to be minimized. Examples of risk measures are an expectation, an expectation plus variance, a quantile, or a conditional expectation above a quantile, also called Conditional Value at Risk (CVaR) [29,28]. In this article, we consider a mean-variance quadratic model. ...
The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle-point systems, which are fully coupled across the random realizations. Despite its relevance for numerous engineering problems, the solution of such systems is notoriusly challenging. In this manuscript, we study efficient preconditioners for all-at-once approaches using both an algebraic and an operator preconditioning framework. We show in particular that for values of the regularization parameter not too small, the saddle-point system can be efficiently solved by preconditioning in parallel all the state and adjoint equations. For small values of the regularization parameter, robustness can be recovered by the additional solution of a small linear system, which however couples all realizations. A mean approximation and a Chebyshev semi-iterative method are investigated to solve this reduced system. Our analysis considers a random elliptic partial differential equation whose diffusion coefficient is modeled as an almost surely continuous and positive random field, though not necessarily uniformly bounded and coercive. We further provide estimates on the dependence of the preconditioned system on the variance of the random field. Such estimates involve either the first or second moment of the random variables and , where D is the spatial domain. The theoretical results are confirmed by numerical experiments, and implementation details are further addressed.
... The resulting gradient could be used in a gradient based optimization problem to find a solution. The optimality conditions in a more general setting and for more elaborate risk measures are discussed in [22]. Ideas from several previous works are drawn upon in this paper. ...
This paper considers the problem of optimizing the average tracking error for an elliptic partial differential equation with an uncertain lognormal diffusion coefficient. In particular, the application of the multilevel quasi-Monte Carlo (MLQMC) method to the estimation of the gradient is investigated, with a circulant embedding method used to sample the stochastic field. A novel regularity analysis of the adjoint variable is essential for the MLQMC estimation of the gradient in combination with the samples generated using the CE method. A rigorous cost and error analysis shows that a randomly shifted quasi-Monte Carlo method leads to a faster rate of decay in the root mean square error of the gradient than the ordinary Monte Carlo method, while considering multiple levels substantially reduces the computational effort. Numerical experiments confirm the improved rate of convergence and show that the MLQMC method outperforms the multilevel Monte Carlo method and the single level quasi-Monte Carlo method.
... In a risk-neutral optimization problem (also called a robust formulation; see [14,), one is interested in minimizing the expectation of the cost functional. This is in contrast with a so-called risk-averse formulation; see [11,12,13]. ...
The study of optimal control problems under uncertainty plays an important role in scientific numerical simulations. Nowadays this class of optimization problems is strongly utilized in engineering, biology and finance. In this paper, a stochastic gradient-based method is proposed for the numerical resolution of a nonconvex stochastic optimization problem on a Hilbert space. We show that, under suitable assumptions, strong or weak accumulation points of the iterates produced by the method converge almost surely to stationary points of the original optimization problem. The proof is based on classical results, such as the theorem by Robbins and Siegmund and the theory of stochastic approximation. The novelty of our contribution lies in the convergence analysis extended to some nonconvex infinite dimensional optimization problems. To conclude, the application to an optimal control problem for a class of elliptic semilinear partial differential equations (PDEs) under uncertainty will be addressed in detail.
... PDE-constrained optimization under uncertainty is a rapidly growing field with a number of recent contributions in theory [6,24,25,27], numerical and computational methods [16,17,23,46], and applications [4,5,8,39]. Nevertheless, a number of open questions remain unanswered, even for the ideal setting including a strongly convex objective function, a closed, bounded and convex feasible set, and a linear elliptic PDE with random inputs. ...
Monte Carlo approximations for random linear elliptic PDE constrained optimization problems are studied. We use empirical process theory to obtain best possible mean convergence rates O(1/\sqrt{n}) for optimal values and solutions , and a central limit theorem for optimal values. The latter allows to determine asymptotically consistent confidence intervals by using resampling techniques.
Optimal values and solutions of empirical approximations of stochastic optimization problems can be viewed as statistical estimators of their true values. From this perspective, it is important to understand the asymptotic behavior of these estimators as the sample size goes to infinity. This area of study has a long tradition in stochastic programming. However, the literature is lacking consistency analysis for problems in which the decision variables are taken from an infinite-dimensional space, which arise in optimal control, scientific machine learning, and statistical estimation. By exploiting the typical problem structures found in these applications that give rise to hidden norm compactness properties for solution sets, we prove consistency results for nonconvex risk-averse stochastic optimization problems formulated in infinite-dimensional space. The proof is based on several crucial results from the theory of variational convergence. The theoretical results are demonstrated for several important problem classes arising in the literature.
We analyze risk-neutral multiobjective optimal control problems, governed by Volterra integral equations with random inputs, and subject to expectation-type final/pure state constraints and mixed
pointwise control-state constraints. Moreover, controls are unbounded, and inclusion-form pure state as well as control-state constraints are given by measurable set-valued mappings, whose images are nonempty closed subsets of infinite dimensional spaces. With such constraints, the multiplier corresponding to pure state constraints belongs to the dual space of the Banach space-valued continuous functions. We set up a decomposition result for this dual, and develop a Lagrange multiplier theorem for an abstract multi-criteria optimization model, in which a Robinson-type constraint qualification is applied. Then, making use of the obtained outcomes, exploiting higher integrability and regularity of the random field control/state variables, and employing Banach space-valued integration techniques, we derive Fritz-John
necessary optimality conditions, with integrable functions and (countably additive) measures singular with respect to the Lebesgue measure as multipliers, for local weak Pareto solutions of studied problems.
The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle‐point systems, which are fully coupled across the random realizations. Despite its relevance for numerous engineering problems, the solution of such systems is notoriously challenging. In this manuscript, we study efficient preconditioners for all‐at‐once approaches using both an algebraic and an operator preconditioning framework. We show in particular that for values of the regularization parameter not too small, the saddle‐point system can be efficiently solved by preconditioning in parallel all the state and adjoint equations. For small values of the regularization parameter, robustness can be recovered by the additional solution of a small linear system, which however couples all realizations. A mean approximation and a Chebyshev semi‐iterative method are proposed to solve this reduced system. We consider a random elliptic partial differential equation whose diffusion coefficient κ(x,ω) is modeled as an almost surely continuous and positive random field, though not necessarily uniformly bounded and coercive. We further provide estimates of the dependence of the spectrum of the preconditioned system matrix on the statistical properties of the random field and on the discretization of the probability space. Such estimates involve either the first or second moment of the random variables 1/minx∈D‾κ(x,ω) and maxx∈D‾κ(x,ω), where D D is the spatial domain. The theoretical results are confirmed by numerical experiments, and implementation details are further addressed.
We analyze the tail behavior of solutions to sample average approximations (SAAs) of stochastic programs posed in Hilbert spaces. We require that the integrand be strongly convex with the same convexity parameter for each realization. Combined with a standard condition from the literature on stochastic programming, we establish non-asymptotic exponential tail bounds for the distance between the SAA solutions and the stochastic program’s solution, without assuming compactness of the feasible set. Our assumptions are verified on a class of infinite-dimensional optimization problems governed by affine-linear partial differential equations with random inputs. We present numerical results illustrating our theoretical findings.
We develop a sampling-free approximation scheme for distributionally robust PDEconstrained optimization problems, which are min-max control problems. We define the ambiguity set through moment and entropic constraints. We use second-order Taylor's expansions of the reduced objective function w.r.t. uncertain parameters, allowing us to compute the expected value of the quadratic function explicitly. The objective function of the approximated min-max problem separates into a trust-region problem and a semidefinite program. We construct smoothing functions for the optimal value functions defined by these problems. We prove the existence of optimal solutions for the distributionally robust control problem, and the approximated and smoothed problems, and show that a worst-case distribution exists. For the numerical solution of the approximated problem, we develop a homotopy method that computes a sequence of stationary points of smoothed problems while decreasing smoothing parameters to zero. The adjoint approach is used to compute derivatives of the smoothing functions. Numerical results for two nonlinear optimization problems are presented.
PDE-constrained (generalized) Nash equilibrium problems (GNEPs) are considered in a deterministic setting as well as under uncertainty. This includes a study of deterministic GNEPs with nonlinear and/or multi-valued operator equations as forward problems and PDE-constrained GNEPs with uncertain data. The deterministic nonlinear problems are analyzed using the theory of generalized convexity for set-valued operators, and a variational approximation approach is proposed. The stochastic setting includes a detailed overview of the recently developed theory and algorithms for risk-averse PDE-constrained optimization problems. These new results open the way to a rigorous study of stochastic PDE-constrained GNEPs.
We present a surrogate modeling framework for conservatively estimating measures of risk from limited realizations of an expensive physical experiment or computational simulation. Risk measures combine objective probabilities with the subjective values of a decision maker to quantify anticipated outcomes. Given a set of samples, we construct a surrogate model that produces estimates of risk measures that are always greater than their empirical approximations obtained from the training data. These surrogate models limit over-confidence in reliability and safety assessments and produce estimates of risk measures that converge much faster to the true value than purely sample-based estimates. We first detail the construction of conservative surrogate models that can be tailored to a stakeholder’s risk preferences and then present an approach, based on stochastic orders, for constructing surrogate models that are conservative with respect to families of risk measures. Our surrogate models include biases that permit them to conservatively estimate the target risk measures. We provide theoretical results that show that these biases decay at the same rate as the L2 error in the surrogate model. Numerical demonstrations confirm that risk-adapted surrogate models do indeed overestimate the target risk measures while converging at the expected rate.
We propose a general framework for machine learning based optimization under uncertainty. Our approach replaces the complex forward model by a surrogate, e.g., a neural network, which is learned simultaneously in a one-shot sense when solving the optimal control problem. Our approach relies on a reformulation of the problem as a penalized empirical risk minimization problem for which we provide a consistency analysis in terms of large data and increasing penalty parameter. To solve the resulting problem, we suggest a stochastic gradient method with adaptive control of the penalty parameter and prove convergence under suitable assumptions on the surrogate model. Numerical experiments illustrate the results for linear and nonlinear surrogate models.
We propose a fast and scalable optimization method to solve chance or probabilistic constrained optimization problems governed by partial differential equations (PDEs) with high-dimensional random parameters. To address the critical computational challenges of expensive PDE solution and highdimensional uncertainty, we construct surrogates of the constraint function by Taylor approximation, which relies on efficient computation of the derivatives, low-rank approximation of the Hessian, and a randomized algorithm for eigenvalue decomposition. To tackle the difficulty of the nondifferentiability of the inequality chance constraint, we use a smooth approximation of the discontinuous indicator function involved in the chance constraint, and we apply a penalty method to transform the inequality constrained optimization problem to an unconstrained one. Moreover, we design a gradient-based optimization scheme that gradually increases smoothing and penalty parameters to achieve convergence, for which we present an efficient computation of the gradient of the approximate cost functional by the Taylor approximation. Based on numerical experiments for a problem in optimal groundwater management, we demonstrate the accuracy of the Taylor approximation, its ability to greatly accelerate constraint evaluations, the convergence of the continuation optimization scheme, and the scalability of the proposed method in terms of the number of PDE solves with increasing random parameter dimension from one thousand to hundreds of thousands.
This work is motivated by the need to study the impact of data uncertainties and material imperfections on the solution to optimal control problems constrained by partial differential equations. We consider a pathwise optimal control problem constrained by a diffusion equation with random coefficient together with box constraints for the control. For each realization of the diffusion coefficient we solve an optimal control problem using the variational discretization [M. Hinze, Comput. Optim. Appl., 30 (2005), pp. 45-61]. Our framework allows for lognormal coefficients whose realizations are not uniformly bounded away from zero and infinity. We establish finite element error bounds for the pathwise optimal controls. This analysis is nontrivial due to the limited spatial regularity and the lack of uniform ellipticity and boundedness of the diffusion operator. We apply the error bounds to prove convergence of a multilevel Monte Carlo estimator for the expected value of the pathwise optimal controls. In addition we analyze the computational complexity of the multilevel estimator. We perform numerical experiments in 2D space to confirm the convergence result and the complexity bound.
We consider the numerical simulation of an optimal control problem constrained by the unsteady Stokes-Brinkman equation involving random data. More precisely, we treat the state, the control, the target (or the desired state), as well as the viscosity, as analytic functions depending on uncertain parameters. This allows for a simultaneous generalized polynomial chaos approximation of these random functions in the stochastic Galerkin finite element method discretization of the model. The discrete problem yields a prohibitively high dimensional saddle point system with Kronecker product structure. We develop a new alternating iterative tensor method for an efficient reduction of this system by the low-rank Tensor Train representation. Besides, we propose and analyze a robust Schur complement-based preconditioner for the solution of the saddle-point system. The performance of our approach is illustrated with extensive numerical experiments based on two- and three-dimensional examples, where the full problem size exceeds one billion degrees of freedom. The developed Tensor Train scheme reduces the solution storage by two–three orders of magnitude, depending on discretization parameters.
We discuss the use of stochastic collocation for the solution of optimal control problems which are constrained by stochastic partial differential equations (SPDE). Thereby the constraining, SPDE depends on data which is not deterministic but random. Assuming a deterministic control, randomness within the states of the input data will propagate to the states of the system. For the solution of SPDEs there has recently been an increasing effort in the development of efficient numerical schemes based upon the mathematical concept of generalized polynomial chaos. Modal-based stochastic Galerkin and nodal-based stochastic collocation versions of this methodology exist, both of which rely on a certain level of smoothness of the solution in the random space to yield accelerated convergence rates. In this paper we apply the stochastic collocation method to develop a gradient descent as well as a sequential quadratic program (SQP) for the minimization of objective functions constrained by an SPDE. The stochastic function involves several higher-order moments of the random states of the system as well as classical regularization of the control. In particular we discuss several objective functions of tracking type. Numerical examples are presented to demonstrate the performance of our new stochastic collocation minimization approach.
A strategy for the fast computation of robust controls of PDE models with random-field coefficients is presented. A robust control is defined as the control function that minimizes the expectation value of the objective over all coefficient configurations. A straightforward application of the adjoint method on this problem results in a very large optimality system. In contrast, a fast method is presented where the expectation value of the objective is minimized with respect to a reduced POD basis of the space of controls. Comparison of the POD scheme with the full optimization procedure in the case of elliptic control problems with random reaction terms and with random diffusivity demonstrates the superior computational performance of the POD method.
In reliability engineering focused on the design and optimization of structures, the typical measure of reliability is the probability of failure of the structure or its individual components relative to specific limit states. However, the failure probability has troublesome properties that raise several theoretical, practical, and computational issues. This paper explains the seriousness of these issues in the context of design optimization and goes on to propose a new alternative measure, the buffered failure probability, which offers significant advantages. The buffered failure probability is handled with relative ease in design optimization problems, accounts for the degree of violation of a performance threshold, and is more conservative than the failure probability.
Recently, optimization has become an integral part of the aerody-namic design process chain. Besides standard optimization routines which require some multitude of the computational effort necessary for the simulation only, also fast optimization methods based on one-shot ideas are available, which are only 4 to 10 times as costly as one forward flow simulation computation. However, the full potential of mathematical optimization can only be exploited, if optimal designs can be computed, which are robust with respect to small (or even large) perturbations of the optimization setpoint conditions. That means, the optimal de-signs computed should still be good designs, even if the input parameters for the optimization problem formulation are changed by a non-negligible amount. Thus even more experimental or numerical effort can be saved. In this paper, we aim at an improvement of existing simulation and optimization technology, developed in the German collaborative effort MEGADESIGN, so that numerical uncertainties are identified, quantized and included in the overall optimization procedure, thus making robust design in this sense possible. These investigations are part of the current German research program MUNA.
In this paper we study both market risks and nonmarket risks, without complete markets assumption, and discuss methods of measurement of these risks. We present and justify a set of four desirable properties for measures of risk, and call the measures satisfying these properties “coherent.” We examine the measures of risk provided and the related actions required by SPAN, by the SEC/NASD rules, and by quantile-based methods. We demonstrate the universality of scenario-based methods for providing coherent measures. We offer suggestions concerning the SEC method. We also suggest a method to repair the failure of subadditivity of quantile-based methods.
We study sample approximations of chance constrained problems. In particular, we consider the sample average approximation (SAA) approach and discuss the convergence properties of the resulting problem. We discuss how one can use the SAA method
to obtain good candidate solutions for chance constrained problems. Numerical experiments are performed to correctly tune
the parameters involved in the SAA. In addition, we present a method for constructing statistical lower bounds for the optimal
value of the considered problem and discuss how one should tune the underlying parameters. We apply the SAA to two chance
constrained problems. The first is a linear portfolio selection problem with returns following a multivariate lognormal distribution.
The second is a joint chance constrained version of a simple blending problem.
This work proposes and analyzes an anisotropic sparse grid stochastic collocation method for solving partial differential equations with random coefficients and forcing terms (input data of the model). The method consists of a Galerkin approximation in the space variables and a collocation, in probability space, on sparse tensor product grids utilizing either Clenshaw-Curtis or Gaussian knots. Even in the presence of nonlinearities, the collocation approach leads to the solution of uncoupled deterministic problems, just as in the Monte Carlo method. This work includes a priori and a posteriori procedures to adapt the anisotropy of the sparse grids to each given problem. These procedures seem to be very effective for the problems under study. The proposed method combines the advantages of isotropic sparse collocation with those of anisotropic full tensor product collocation: the first approach is effective for problems depending on random variables which weigh approximately equally in the solution, while the benefits of the latter approach become apparent when solving highly anisotropic problems depending on a relatively small number of random variables, as in the case where input random variables are Karhunen-Loève truncations of “smooth” random fields. This work also provides a rigorous convergence analysis of the fully discrete problem and demonstrates (sub)exponential convergence in the asymptotic regime and algebraic convergence in the preasymptotic regime, with respect to the total number of collocation points. It also shows that the anisotropic approximation breaks the curse of dimensionality for a wide set of problems. Numerical examples illustrate the theoretical results and are used to compare this approach with several others, including the standard Monte Carlo. In particular, for moderately large-dimensional problems, the sparse grid approach with a properly chosen anisotropy seems to be very efficient and superior to all examined methods.
We study approximations of optimization problems with probabilistic constraints in which the original distribution of the underlying random vector is replaced with an empirical distribution obtained from a random sample. We show that such a sample approximation problem with a risk level larger than the required risk level will yield a lower bound to the true optimal value with probability approaching one exponentially fast. This leads to an a priori estimate of the sample size required to have high confidence that the sample approximation will yield a lower bound. We then provide conditions under which solving a sample approximation problem with a risk level smaller than the required risk level will yield feasible solutions to the original problem with high probability. Once again, we obtain a priori estimates on the sample size required to obtain high confidence that the sample approximation problem will yield a feasible solution to the original problem. Finally, we present numerical illustrations of how these results can be used to obtain feasible solutions and optimality bounds for optimization problems with probabilistic constraints.
We consider a chance constrained problem, where one seeks to minimize a convex objective over solutions satisfying, with a given close to one probability, a system of randomly perturbed convex constraints. This problem may happen to be computationally intractable; our goal is to build its computationally tractable approximation, i.e., an efficiently solvable deterministic optimization program with the feasible set contained in the chance constrained problem. We construct a general class of such convex conservative approximations of the corresponding chance constrained problem. Moreover, under the assumptions that the constraints are a. ne in the perturbations and the entries in the perturbation vector are independent-of-each-other random variables, we build a large deviation-type approximation, referred to as "Bernstein approximation," of the chance constrained problem. This approximation is convex and efficiently solvable. We propose a simulation-based scheme for bounding the optimal value in the chance constrained problem and report numerical experiments aimed at comparing the Bernstein and well-known scenario approximation approaches. Finally, we extend our construction to the case of ambiguous chance constrained problems, where the random perturbations are independent with the collection of distributions known to belong to a given convex compact set rather than to be known exactly, while the chance constraint should be satisfied for every distribution given by this set.
We introduce the notion of a convex measure of risk, an extension of the concept of a coherent risk measure defined in Artzner et al. (1999), and we prove a corresponding extension of the representation theorem in terms of probability measures on the underlying space of scenarios. As a case study, we consider convex measures of risk defined in terms of a robust notion of bounded shortfall risk. In the context of a financial market model, it turns out that the representation theorem is closely related to the superhedging duality under convex constraints.
The optimized certainty equivalent (OCE) is a decision theoretic criterion based on a utility function, that was first introduced by the authors in 1986. This paper re-examines this fundamental concept, studies and extends its main properties, and puts it in perspective to recent concepts of risk measures. We show that the negative of the OCE naturally provides a wide family of risk measures that fits the axiomatic formalism of convex risk measures. Duality theory is used to reveal the link between the OCE and the Õ-divergence functional (a generalization of relative entropy), and allows for deriving various variational formulas for risk measures. Within this interpretation of the OCE, we prove that several risk measures recently analyzed and proposed in the literature (e.g., conditional value of risk, bounded shortfall risk) can be derived as special cases of the OCE by using particular utility functions. We further study the relations between the OCE and other certainty equivalents, providing general conditions under which these can be viewed as coherent/convex risk measures. Throughout the paper several examples illustrate the flexibility and adequacy of the OCE for building risk measures.
We consider a nonlinear optimization problem governed by partial differential equations with uncertain parameters. It is addressed by a robust worst case formulation. The resulting optimization problem is of bilevel structure and is difficult to treat numerically. We propose an approximate robust formulation that employs linear and quadratic approximations. To speed up the computation, reduced order models based on proper orthogonal decomposition in combination with a posteriori error estimators are developed. The proposed strategy is then applied to the optimal placement of a permanent magnet in the rotor of a synchronous machine with moving rotor. Numerical results are presented to validate the presented approach.
Low-rank tensor methods provide efficient representations and computations for high-dimensional problems and are able to break the curse of dimensionality when dealing with systems involving multiple parameters. Motivated by optimal control problems with PDEs under uncertainty, we present algorithms for constrained nonlinear optimization problems that use low-rank tensors. They are applied to optimal control of PDEs with uncertain parameters and to parametrized variational inequalities of obstacle type. These methods are tailored to the usage of low-rank tensor arithmetics and allow us to solve huge scale optimization problems. In particular, we consider a semismooth Newton method for an optimal control problem with pointwise control constraints and an interior point algorithm for an obstacle problem, both with uncertainties in the coefficients.
We consider probability functions of parameter-dependent random inequality systems under Gaussian distribution. As a main result, we provide an upper estimate for the Clarke subdifferential of such probability functions without imposing compactness conditions. A constraint qualification ensuring continuous differentiability is formulated. Explicit formulae are derived from the general result in the case of linear random inequality systems. In the case of a constant coefficient matrix, an upper estimate for even the smaller Mordukhovich subdifferential is proven.
We present a method for optimal control of systems governed by partial differential equations (PDEs) with uncertain parameter fields. We consider an objective function that involves the mean and variance of the control objective, leading to a risk-averse optimal control problem. To make the optimal control problem tractable, we invoke a quadratic Taylor series approximation of the control objective with respect to the uncertain parameter field. This enables deriving explicit expressions for the mean and variance of the control objective in terms of its gradients and Hessians with respect to the uncertain parameter. The risk-averse optimal control problem is then formulated as a PDE-constrained optimization problem with constraints given by the forward and adjoint PDEs defining these gradients and Hessians. The expressions for the mean and variance of the control objective under the quadratic approximation involve the trace of the (preconditioned) Hessian, and are thus prohibitive to evaluate. To overcome this difficulty, we employ randomized trace estimators. We illustrate our approach with two specific problems: the control of a semilinear elliptic PDE with an uncertain boundary source term, and the control of a linear elliptic PDE with an uncertain coefficient field. For the latter problem, we derive adjoint-based expressions for efficient computation of the gradient of the risk-averse objective with respect to the controls. Our method ensures that the cost of computing the risk-averse objective and its gradient with respect to the control---measured in the number of PDE solves---is independent of the (discretized) parameter and control dimensions, and depends only on the number of random vectors employed in the trace estimation. Finally, we present a comprehensive numerical study of an optimal control problem for fluid flow in a porous medium with uncertain permeability field.
Uncertainty is inevitable when solving science and engineering application problems. In the face of uncertainty, it is essential to determine robust and risk-averse solutions. In this work, we consider a class of PDE-constrained optimization problems in which the PDE coefficients and inputs may be uncertain. We introduce two approximations for minimizing the conditional value-atrisk (CVaR) for such PDE-constrained optimization problems. These approximations are based on the primal and dual formulations of CVaR. For the primal problem, we introduce a smooth approximation of CVaR in order to utilize derivative-based optimization algorithms and to take advantage of the convergence properties of quadrature-based discretizations. For this smoothed CVaR, we prove differentiability as well as consistency of our approximation. For the dual problem, we regularize the inner maximization problem, rigorously derive optimality conditions, and demonstrate the consistency of our approximation. Furthermore, we propose a fixed-point iteration that takes advantage of the structure of the regularized optimality conditions and provides a means of calculating worst-case probability distributions based on the given probability level. We conclude with numerical results.
In this work, we apply the MG/OPT framework to a multilevel-in-sample-space discretization of optimization problems governed by PDEs with uncertain coefficients. The MG/OPT algorithm is a template for the application of multigrid to deterministic PDE optimization problems. We employ MG/OPT to exploit the hierarchical structure of sparse grids in order to formulate a multilevel stochastic collocation algorithm. The algorithm is provably first-order convergent under standard assumptions on the hierarchy of discretized objective functions as well as on the optimization routines used as pre- and postsmoothers. We present explicit bounds on the total number of PDE solves and an upper bound on the error for one V-cycle of the MG/OPT algorithm applied to a linear quadratic control problem. We provide numerical results that confirm the theoretical bound on the number of PDE solves and show a dramatic reduction in the total number of PDE solves required to solve these optimization problems when compared with stand...
Convex analysis provides the tools to extend results of differential calculus to nonsmooth real valued functions. The purpose of this article is to study those extensions for convex vector valued mappings. We study their continuity properties, develop a subdifferential calculus and a duality theory, similar to the one existing for real valued functions. We conclude with some useful deconvexification results.
This paper improves the trust-region algorithm with adaptive sparse grids introduced in [SIAM J. Sci. Comput., 35 (2013), pp. A1847-A1879] for the solution of optimization problems governed by partial differential equations (PDEs) with uncertain coefficients. The previous algorithm used adaptive sparse-grid discretizations to generate models that are applied in a trust-region framework to generate a trial step. The decision whether to accept this trial step as the new iterate, however, required relatively high-fidelity adaptive discretizations of the objective function. In this paper, we extend the algorithm and convergence theory to allow the use of low-fidelity adaptive sparse-grid models in objective function evaluations. This is accomplished by extending conditions on inexact function evaluations used in previous trust-region frameworks. Our algorithm adaptively builds two separate sparse grids: one to generate optimization models for the step computation and one to approximate the objective function. These adapted sparse grids often contain significantly fewer points than the high-fidelity grids, which leads to a dramatic reduction in the computational cost. This is demonstrated numerically using two examples. Moreover, the numerical results indicate that the new algorithm rapidly identifies the stochastic variables that are relevant to obtaining an accurate optimal solution. When the number of such variables is independent of the dimension of the stochastic space, the algorithm exhibits near dimension-independent behavior.
In this paper we develop and analyze an efficient computational method for solving stochastic optimal control problems constrained by an elliptic partial differential equation (PDE) with random input data. We first prove both existence and uniqueness of the optimal solution. Regularity of the optimal solution in the stochastic space is studied in view of the analysis of stochastic approximation error. For numerical approximation, we employ a finite element method for the discretization of physical variables, and a stochastic collocation method for the discretization of random variables. In order to alleviate the computational effort, we develop a model order reduction strategy based on a weighted reduced basis method. A global error analysis of the numerical approximation is carried out, and several numerical tests are performed to verify our analysis.
This paper is concerned with continuity and differentiability of NEMYTSKIJ operators acting between spaces of summable abstract functions. In a first part, necessary and sufficient conditions for continuity are collected. Then main emphasis is given to sufficient conditions for differentiability in the sense of FRÉCHET and GÂTEAUX. Finally, second order differentiability is briefly discussed.
In this work we deal with a stochastic optimal Robin boundary control problem constrained by an advection-diffusion-reaction elliptic equation with advection-dominated term. We assume that the uncertainty comes from the advection field and consider a stochastic Robin boundary condition as control function. A stochastic saddle point system is formulated and proved to be equivalent to the first order optimality system for the optimal control problem, based on which we provide the existence and uniqueness of the optimal solution as well as some results on stochastic regularity with respect to the random variables. Stabilized finite element approximations in physical space and collocation approximations in stochastic space are applied to discretize the optimality system. A global error estimate in the product of physical space and stochastic space for the numerical approximation is derived. Illustrative numerical experiments are provided.
Recently, optimization has become an integral part of the aerodynamic design process chain. Besides standard optimization routines, which require some multitude of the computational effort necessary for the simulation only, fast optimization methods based on one-shot ideas are also available, which are only 4 to 10 times as costly as one forward flow simulation computation. However, the full potential of mathematical optimization can only be exploited, if optimal designs can be computed, which are robust with respect to small (or even large) perturbations of the optimization set-point conditions. That means the optimal designs computed should still be good designs, even if the input parameters for the optimization problem formulation are changed by a nonnegligible amount. Thus even more experimental or numerical effort can be saved. In this paper, we aim at an improvement of existing simulation and optimization technology, developed in the German collaborative effort MEGADESIGN, so that numerical uncertainties are identified, quantized, and included in the overall optimization procedure, thus making robust design in this sense possible. These investigations are part of the current German research program MUNA.
The numerical solution of optimization problems governed by partial differential equations (PDEs) with random coefficients is computationally challenging because of the large number of deterministic PDE solves required at each optimization iteration. This paper introduces an efficient algorithm for solving such problems based on a combination of adaptive sparse-grid collocation for the discretization of the PDE in the stochastic space and a trust-region framework for optimization and fidelity management of the stochastic discretization. The overall algorithm adapts the collocation points based on the progress of the optimization algorithm and the impact of the random variables on the solution of the optimization problem. It frequently uses few collocation points initially and increases the number of collocation points only as necessary, thereby keeping the number of deterministic PDE solves low while guaranteeing convergence. Currently an error indicator is used to estimate gradient errors due to adaptive stochastic collocation. The algorithm is applied to three examples, and the numerical results demonstrate a significant reduction in the total number of PDE solves required to obtain an optimal solution when compared with a Newton conjugate gradient algorithm applied to a fixed high-fidelity discretization of the optimization problem.
The subject of inverse problems in differential equations is of enormous practical importance, and has also generated substantial mathematical and computational innovation. Typically some form of regularization is required to ameliorate ill-posed behaviour. In this article we review the Bayesian approach to regularization, developing a function space viewpoint on the subject. This approach allows for a full characterization of all possible solutions, and their relative probabilities, whilst simultaneously forcing significant modelling issues to be addressed in a clear and precise fashion. Although expensive to implement, this approach is starting to lie within the range of the available computational resources in many application areas. It also allows for the quantification of uncertainty and risk, something which is increasingly demanded by these applications. Furthermore, the approach is conceptually important for the understanding of simpler, computationally expedient approaches to inverse problems. We demonstrate that, when formulated in a Bayesian fashion, a wide range of inverse problems share a common mathematical framework, and we highlight a theory of well-posedness which stems from this. The well-posedness theory provides the basis for a number of stability and approximation results which we describe. We also review a range of algorithmic approaches which are used when adopting the Bayesian approach to inverse problems. These include Markov chain Monte Carlo methods, filtering and the variational approach.
The main subject of this book is perturbation analysis of continuous optimization problems. In the last two decades considerable progress has been made in that area, and it seems that it is time now to present a synthetic view of many important results that apply to various classes of problems.
Probability functions, depending upon their parameters, are reduced to integrals calculated over sets given by many inequalities. A new general formula for the differentiation of such integrals is proposed. A gradient of the integral is represented as the sum of integrals taken over a volume and over a surface. These results are used to calculate the sensitivities of probability functions, and also for chance-constrained optimization.
Most physical phenomena are significantly affected by uncertainties associated with variations in properties and fluctuations in operating conditions. This has to be reflected also in the design and control of real-application systems.
Recent advances in PDE constrained optimization open the possibility of realistic optimization of such systems in the presence of model and data uncertainties. These emerging techniques require only the knowledge of the probability distribution of the perturbations, which is usually available, and provide optimization solutions that are robust with respect to the stochasticity of the application framework.
In this paper, some of these methodologies are reviewed. The focus is on PDE constrained optimization frameworks where distributed uncertainties are modeled by random fields and the structures in the underlying optimization problems are exploited in the form of multigrid methods and one-shot methods. Applications are presented, including control problems with uncertain coefficients and erodynamic design under geometric uncertainties (© 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)
A new approach to optimizing or hedging a portfolio of financial instruments to reduce risk is presented and tested on applications. It focuses on minimizing Conditional Value-at-Risk (CVaR) rather than minimizing Value-at-Risk (VaR), but portfolios with low CVaR necessarily have low VaR as well. CVaR, also called Mean Excess Loss, Mean Shortfall, or Tail VaR, is anyway considered to be a more consistent measure of risk than VaR. Central to the new approach is a technique for portfolio optimization which calculates VaR and optimizes CVaR simultaneously. This technique is suitable for use by investment companies, brokerage firms, mutual funds, and any business that evaluates risks. It can be combined with analytical or scenario-based methods to optimize portfolios with large numbers of instruments, in which case the calculations often come down to linear programming or nonsmooth programming. The methodology can be applied also to the optimization of percentiles in contexts outside of finance.
We consider nonlinear programming problem (P) with stochastic constraints. The Lagrangean corresponding to such problems has a stochastic part, which in this work is replaced by its certainty equivalent (in the sense of expected utility theory). It is shown that the deterministic surrogate problem (CE-P) thus obtained, contains a penalty function which penalizes violation of the constraints in the mean. The approach is related to several known methods in stochastic programming such as: chance constraints, stochastic goal programming, reliability programming and mean-variance analysis. The dual problem of (CE-P) is studied (for problems with stochastic righthand sides in the constraints) and a comprehensive duality theory is developed by introducing a new certainty equivalent (NCE) concept. Motivation for the NCE and its potential role in Decision Theory are discussed, as well as mean-variance approximations.
Probability functions depending upon parameters are represented as integrals over sets given by inequalities. New derivative formulas for the intergrals over a volume are considered. Derivatives are presented as sums of integrals over a volume and over a surface. Two examples are discussed: probability functions with linear constraints (random right-hand sides), and a dynamical shut-down problem with sensors.
We study a space of coherent risk measures Mφ obtained as certain expansions of coherent elementary basis measures. In this space, the concept of “risk aversion function” φ naturally arises as the spectral representation of each risk measure in a space of functions of confidence level probabilities. We give necessary and sufficient conditions on φ for Mφ to be a coherent measure. We find in this way a simple interpretation of the concept of coherence and a way to map any rational investor's subjective risk aversion onto a coherent measure and vice-versa. We also provide for these measures their discrete versions M(N)φ acting on finite sets of N independent realizations of a r.v. which are not only shown to be coherent measures for any fixed N, but also consistent estimators of Mφ for large N.
The optimal control of problems that are constrained by partial differential
equations with uncertainties and with uncertain controls is addressed. The
Lagrangian that defines the problem is postulated in terms of stochastic
functions, with the control function possibly decomposed into an unknown
deterministic component and a known zero-mean stochastic component. The extra
freedom provided by the stochastic dimension in defining cost functionals is
explored, demonstrating the scope for controlling statistical aspects of the
system response. One-shot stochastic finite element methods are used to find
approximate solutions to control problems. It is shown that applying the
stochastic collocation finite element to the formulated problem leads to a
coupling between stochastic collocation points when a deterministic optimal
control is considered or when moments are included in the cost functional,
thereby obviating the primary advantage of the collocation method over the
stochastic Galerkin method for the considered problem. The application of the
presented methods is demonstrated through a number of numerical examples. The
presented framework is sufficiently general to also consider a class of inverse
problems, and numerical examples of this type are also presented.
- H Attouch
- G Buttazzo
- G Michaille
H. Attouch, G. Buttazzo, and G. Michaille, Variational Analysis in Sobolev and BV Spaces,
MPS/SIAM Ser. Optim. 6, SIAM, Philadelphia, 2006.
- D Calvetti
- E Somersalo
D. Calvetti and E. Somersalo, Introduction to Bayesian Scientific Computing: Ten Lectures on
Subjective Computing, Surveys Tutorials Appl. Math. Sci. 2, Springer, New York, 2007.
- G B Folland
G. B. Folland, Real Analysis. Modern Techniques and Their Applications, 2nd ed., Pure Appl. Math.
(N. Y.), John Wiley & Sons, New York, 1999.
- L Gasinski
- N S Papageorgiou
L. Gasinski and N. S. Papageorgiou, Nonlinear Analysis, Ser. Math. Anal. Appl. 9, Chapman &
Hall/CRC Press, Boca Raton, FL, 2006.
- K Marti
K. Marti, ed., Stochastic Optimization. Numerical Methods and Technical Applications, Lecture Notes
Econom. Math. Syst. 379, Springer, Berlin, 1992.
- R T Rockafellar
R. T. Rockafellar, Convex integral functionals and duality, in Contributions to Nonlinear Functional
Analysis (Proc. Symposium, Math. Res. Center, University of Wisconsin, Madison, 1971), Academic
Press, New York, 1971, pp. 215-236.
- A Shapiro
- D Dentcheva
- A Ruszczynski
A. Shapiro, D. Dentcheva, and A. Ruszczynski, Lectures on Stochastic Programming: Modeling and
Theory, Second Edition, MOS-SIAM Ser. Optim. 16, SIAM, Philadelphia, 2014.
Convex Analysis, reprint of the 1970 original
- R T Rockafellar
R. T. Rockafellar, Convex Analysis, reprint of the 1970 original, Princeton Landmarks in Mathematics,
Princeton University Press, Princeton, NJ, 1997.