No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Getting started with ADOL-C
Abstract
The C++ package ADOL-C described in this paper facilitates the evaluation of first and higher derivatives of vector functions that are defined by computer programs written in C or C++. The numerical values of derivative vectors are obtained free of truncation errors at mostly a small multiple of the run time and a fix small multiple random access memory required by the given function evaluation program. Derivative matrices are obtained by columns, by rows or in sparse format. This tutorial describes the source code modification required for the application of ADOL-C, the most frequently used drivers to evaluate derivatives and some recent developments. @InProceedings{walther:DSP:2009:2084, author = {Andrea Walther}, title = {Getting Started with ADOL-C}, booktitle = {Combinatorial Scientific Computing}, year = {2009}, editor = {Uwe Naumann and Olaf Schenk and Horst D. Simon and Sivan Toledo}, number = {09061}, series = {Dagstuhl Seminar Proceedings}, ISSN = {1862-4405}, publisher = {Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany}, address = {Dagstuhl, Germany}, URL = {http://drops.dagstuhl.de/opus/volltexte/2009/2084}, annote = {Keywords: ADOL-C, algorithmic differentiation of C/C++ programs} }
... and thereby directly obtain the second inequality from (22). The other way to manipulate (24) is to simply drop the right-hand side due to (ω + κ 1 + κ 2 ) > 0. This immediately yields ...
... where we use the Cauchy-Schwarz-Inequality and divide by (ω−ω) x(ω) X which again can be assumed to be non-zero (and positive) without loss of generality. The ensuing estimate then constitutes the first part of (22), completing the proof. ...
... We use automatic differentiation by adol-C in order to establish the second order model and TNNMG to solve update step computation subproblems, cf. [22]. Additionally, the subproblem solver is provided with stopping criteria in the form of our inexactness criteria (20) and (37) with corresponding parameters η k ∈ [0, 1[ for each iteration and globalω max > 0. ...
We consider proximal Newton methods with an inexact computation of update steps. To this end, we introduce two inexactness criteria which characterize sufficient accuracy of these update step and with the aid of these investigate global convergence and local acceleration of our method. The inexactness criteria are designed to be adequate for the Hilbert space framework we find ourselves in while traditional inexactness criteria from smooth Newton or finite dimensional proximal Newton methods appear to be inefficient in this scenario. The performance of the method and its gain in effectiveness in contrast to the exact case are showcased considering a simple model problem in function space.
... Obviously, (28) involves ś m i"1 k i and ś n i"1 i affine function terms in contrast to the first representation (27) Proof. We will consider the representations (27) from which (26) can be directly obtained in the form (28). Firstly, the independent variables x j are linear functions of themselves with gradient a " e j and inhomogeneity α " 0. Then for multiplications by a constant c ą 0 we have to scale all affine functions by c. Secondly, addition requires appending the expansions of the two summands to each other without any computation. ...
... The implementation of the DCA algorithm can be optimized in various ways. Notice that for applying the Simplex method in standard form, one could use for the representation as DC function the max-part in the more economical representation Equation (27) introducingm additional variables, rather than the potentially combinatorial Equation (28) to assemble the constraint matrix. In any case it seems doubtful that solving each sub problem to completion is a good idea, especially as the resulting step in the outer iteration is probably much too small anyhow. ...
... Finally, one should always keep in mind that the task of minimizing a piecewise linear function will most likely occur as an inner problem in the optimization of a piecewise smooth and nonlinear function. As we have shown in [27] the local piecewise linear model problem can be obtained easily by a slight generalization of automatic or algorithmic differentiation, e.g., ADOL-C [28] and Tapenade [29]. as LIKQ in [7] and obviously requires that no more than n switches are active atx. ...
For piecewise linear functions f:Rn↦R we show how their abs-linear representation can be extended to yield simultaneously their decomposition into a convex fˇ and a concave part fˆ , including a pair of generalized gradients gˇ∈Rn∋gˆ . The latter satisfy strict chain rules and can be computed in the reverse mode of algorithmic differentiation, at a small multiple of the cost of evaluating f itself. It is shown how fˇ and fˆ can be expressed as a single maximum and a single minimum of affine functions, respectively. The two subgradients gˇ and −gˆ are then used to drive DCA algorithms, where the (convex) inner problem can be solved in finitely many steps, e.g., by a Simplex variant or the true steepest descent method. Using a reflection technique to update the gradients of the concave part, one can ensure finite convergence to a local minimizer of f, provided the Linear Independence Kink Qualification holds. For piecewise smooth objectives the approach can be used as an inner method for successive piecewise linearization.
... The difference to symbolic differentiation is discretisation. In AD, differentiation takes place at discrete points and no derivation function is calculated (Griewank and Walther, 2008a;Walther and Griewank, 2012). In the last years AD has seen a significant spread in different communities and is now used in machine learning (Baydin et al., 2018) computational fluid dynamics (Carle et al., 1994), robotics (Degrave et al., 2019;Giftthaler et al., 2017), optimisation (Renaud, 1997) and error propagation (Zeier et al., 2012). ...
... For the detection of conflicts, the respective uncertainty has to be propagated. Several works illustrate the implementation of the uncertainty propagation in data-driven models and use AD to calculate the derivatives (Baydin et al., 2018;Walther and Griewank, 2012;Margossian, 2019). ...
Decision-making highly relies on the accuracy and veracity of data.
Therefore, redundant data acquisition and fusion has established but lack the ability to handle conflicting data correctly. Especially digital twins, which complement physical products with mathematical models, and contribute to redundancy. Uncertainty propagates through the digital twin and provides the opportunity to check data for conflicts, to identify affected subsystems and to infer a possible cause. This work presents an approach that combines a digital twin with the ability of uncertainty propagation, conflict detection, processing and visualisation techniques for mastering data-induced conflicts. The capability of this method to identify and isolate faults was examined on a technical system with a multitude of sensors.
... The difference to symbolic differentiation is discretisation. In AD, differentiation takes place at discrete points and no derivation function is calculated (Griewank and Walther, 2008a;Walther and Griewank, 2012). In the last years AD has seen a significant spread in different communities and is now used in machine learning (Baydin et al., 2018) computational fluid dynamics (Carle et al., 1994), robotics (Degrave et al., 2019;Giftthaler et al., 2017), optimisation (Renaud, 1997) and error propagation (Zeier et al., 2012). ...
... For the detection of conflicts, the respective uncertainty has to be propagated. Several works illustrate the implementation of the uncertainty propagation in data-driven models and use AD to calculate the derivatives (Baydin et al., 2018;Walther and Griewank, 2012;Margossian, 2019). ...
... AD methods are often categorized into two main types a) source-code transformation, where the semantics of the code is re-written explicitly to include the computation of the derivatives along with the function [18] or b) Operator-overloading techniques, which depend on redefining elementary operators used in a function or algorithm to propagate additional information during computation. Sophisticated AD tools like CasADi [19] and ADOL-C [20] use operator overloading strategies to build an expression graph of the function to be differentiated. This approach supports both forward-mode and reverse-mode sensitivity propagation. ...
Model-based control for robots has increasingly been dependent on optimization-based methods like Differential Dynamic Programming and iterative LQR (iLQR). These methods can form the basis of Model-Predictive Control (MPC), which is commonly used for controlling legged robots. Computing the partial derivatives of the dynamics is often the most expensive part of these algorithms, regardless of whether analytical methods, Finite Difference, Automatic Differentiation (AD), or Chain-Rule accumulation is used. Since the second-order derivatives of dynamics result in tensor computations, they are often ignored, leading to the use of iLQR, instead of the full second-order DDP method. In this paper, we present analytical methods to compute the second-order derivatives of inverse and forward dynamics for open-chain rigid-body systems with multi-DoF joints and fixed/floating bases. An extensive comparison of accuracy and run-time performance with AD and other methods is provided, including the consideration of code-generation techniques in C/C++ to speed up the computations. For the 36 DoF ATLAS humanoid, the second-order Inverse, and the Forward dynamics derivatives take approx 200 mu s, and approx 2.1 ms respectively, resulting in a 3x speedup over the AD approach.
... Operator-overloading AD tools like ADOL-C [3], CoDiPack [4], the autograd tool [5] used by PyTorch [6], and the internal AD tool of TensorFlow [7], use the tape to store information about the entire real-arithmetic evaluation tree instead of producing code. They can thus avoid parsing and transforming control structures in the implementation of f , and only need to receive a stream of all the real-arithmetic operations performed while evaluating f . ...
Tools for algorithmic differentiation (AD) provide accurate derivatives of computer-implemented functions for use in, e. g., optimization and machine learning (ML). However, they often require the source code of the function to be available in a restricted set of programming languages. As a step towards making AD accessible for code bases with cross-language or closed-source components, we recently presented the forward-mode AD tool Derivgrind. It inserts forward-mode AD logic into the machine code of a compiled program using the Valgrind dynamic binary instrumentation framework. This work extends Derivgrind, adding the capability to record the real-arithmetic evaluation tree, and thus enabling operator overloading style reverse-mode AD for compiled programs. We maintain the high level of correctness reported for Derivgrind's forward mode, failing the same few testcases in an extensive test suite for the same well-understood reasons. Runtime-wise, the recording slows down the execution of a compiled 64-bit benchmark program by a factor of about 180.
... In the following we will dive deeper into the specifics of our implementation of the algorithm: In order to differentiate the smooth part of the composite objective functional and create a second order model of it around some current iterate, we take advantage of the automatic differentiation software package adol-C, cf. [30]. With the second order model at hand we can then consider subproblem (16) which has to be solved in order to obtain a candidate for the update of the current iterate. ...
We develop a globalized Proximal Newton method for composite and possibly non-convex minimization problems in Hilbert spaces. Additionally, we impose less restrictive assumptions on the composite objective functional considering differentiability and convexity than in existing theory. As far as differentiability of the smooth part of the objective function is concerned, we introduce the notion of second order semi-smoothness and discuss why it constitutes an adequate framework for our Proximal Newton method. However, both global convergence as well as local acceleration still pertain to hold in our scenario. Eventually, the convergence properties of our algorithm are displayed by solving a toy model problem in function space.
... That way, no algebraic constraint violation occurs, and the algebraic error introduced by the iterative nature of the solver remains negligible. The implementation is a hand-written C ++ code based on the Dune libraries [12,41], and the gradients and Hesse matrices of the discrete energy functionals are computed by the automatic differentiation software ADOL-C [46]. ...
We numerically benchmark methods for computing harmonic maps into the unit sphere, with particular focus on harmonic maps with singularities. For the discretization we compare two different approaches, both based on Lagrange finite elements. While the first method enforces the unit-length constraint only at the Lagrange nodes, the other one adds a pointwise projection to fulfill the constraint everywhere. For the solution of the resulting algebraic problems we compare a nonconforming gradient flow with a Riemannian trust-region method. Both are energy-decreasing and can be shown to converge globally to a stationary point of the Dirichlet energy. We observe that while the nonconforming and the conforming discretizations both show similar behavior, the second-order trust-region method needs less iterations than the solver based on gradient flow.
... We solve each inner problem until the maximum-norm of the correction drops below 10 −5 . The large but sparse tangent matrices are computed using the ADOL-C algorithmic differentiation system (Walther and Griewank 2012). ...
Deciding whether a given function is quasiconvex is generally a difficult task. Here, we discuss a number of numerical approaches that can be used in the search for a counterexample to the quasiconvexity of a given function W . We will demonstrate these methods using the planar isotropic rank-one convex function $$\begin{aligned} W_\mathrm{magic}^+(F)=\frac{\lambda _\mathrm{max}}{\lambda _\mathrm{min}}-\log \frac{\lambda _\mathrm{max}}{\lambda _\mathrm{min}}+\log \det F=\frac{\lambda _\mathrm{max}}{\lambda _\mathrm{min}}+2\log \lambda _\mathrm{min}\,, \end{aligned}$$ W magic + ( F ) = λ max λ min - log λ max λ min + log det F = λ max λ min + 2 log λ min , where $$\lambda _\mathrm{max}\ge \lambda _\mathrm{min}$$ λ max ≥ λ min are the singular values of F , as our main example. In a previous contribution, we have shown that quasiconvexity of this function would imply quasiconvexity for all rank-one convex isotropic planar energies $$W:{\text {GL}}^+(2)\rightarrow {\mathbb {R}}$$ W : GL + ( 2 ) → R with an additive volumetric-isochoric split of the form $$\begin{aligned} W(F)=W_\mathrm{iso}(F)+W_\mathrm{vol}(\det F)={\widetilde{W}}_\mathrm{iso}\bigg (\frac{F}{\sqrt{\det F}}\bigg )+W_\mathrm{vol}(\det F) \end{aligned}$$ W ( F ) = W iso ( F ) + W vol ( det F ) = W ~ iso ( F det F ) + W vol ( det F ) with a concave volumetric part. This example is therefore of particular interest with regard to Morrey’s open question whether or not rank-one convexity implies quasiconvexity in the planar case.
... Additionally, this subproblem solver is provided with stopping criteria in the form of our inexactness criteria (3.2) and (4.7) with corresponding parameters η k ∈ [0, 1] for each iteration and globalω max > 0. The required derivatives were computed by automatic differentiation, using adol-C, cf. [20]. ...
We consider Proximal Newton methods with an inexact computation of update steps. To this end, we introduce two inexactness criteria which characterize sufficient accuracy of these update step and with the aid of these investigate global convergence and local acceleration of our method. The inexactness criteria are designed to be adequate for the Hilbert space framework we find ourselves in while traditional inexactness criteria from smooth Newton or finite dimensional Proximal Newton methods appear to be inefficient in this scenario. The performance of the method and its gain in effectiveness in contrast to the exact case are showcased considering a simple model problem in function space.
... In the following we will dive deeper into the specifics of our implementation of the algorithm: In order to differentiate the smooth part of the composite objective functional and create a second order model of it around some current iterate, we take advantage of the automatic differentiation software package adol-C, cf. [30]. With the second order model at hand we can then consider subproblem (16) which has to be solved in order to obtain a candidate for the update of the current iterate. ...
We develop a globalized Proximal Newton method for composite and possibly non-convex minimization problems in Hilbert spaces. Additionally, we impose less restrictive assumptions on the composite objective functional considering differentiability and convexity than in existing theory. As far as differentiability of the smooth part of the objective function is concerned, we introduce the notion of second order semi-smoothness and discuss why it constitutes an adequate framework for our Proximal Newton method. However, both global convergence as well as local acceleration still pertain to hold in our scenario. Eventually, the convergence properties of our algorithm are displayed by solving a toy model problem in function space.
... The most straightforward strategy is certainly to substitute any arithmetic operations by calls to an AD library that implements the additional AD arithmetics. Just like the ad-hoc forward AD tool of Fig. 2d, many AD tools make use of polymorphism and operator overloading [75][76][77][78]. This is a feature of many contemporary programming languages, by which the compiler or interpreter automatically dispatches any calls to arithmetic operators or functions to custom implementations if one of the involved variables is of a custom type. ...
The full optimization of the design and operation of instruments whose functioning relies on the interaction of radiation with matter is a super-human task, given the large dimensionality of the space of possible choices for geometry, detection technology, materials, data-acquisition, and information-extraction techniques, and the interdependence of the related parameters. On the other hand, massive potential gains in performance over standard, "experience-driven" layouts are in principle within our reach if an objective function fully aligned with the final goals of the instrument is maximized by means of a systematic search of the configuration space. The stochastic nature of the involved quantum processes make the modeling of these systems an intractable problem from a classical statistics point of view, yet the construction of a fully differentiable pipeline and the use of deep learning techniques may allow the simultaneous optimization of all design parameters. In this document we lay down our plans for the design of a modular and versatile modeling tool for the end-to-end optimization of complex instruments for particle physics experiments as well as industrial and medical applications that share the detection of radiation as their basic ingredient. We consider a selected set of use cases to highlight the specific needs of different applications.
... A general introduction to automatic differentiation can be found in Hoffman, 2015 [48]. More recently, more and more libraries for automatic differentiation have been released, both generic ones (ADOL-C [99], CasADI [2], ad [57]) and some dedicated to more specific problems (the MC Stan math library [89] is used by MC Stan, a Bayesian inference library; pytorch [70], tensorflow [1] and theano [79] are all example of machine-learning libraries that implement an automatic differentiation engine). Automatic differentiation is identified as a tool for optimal design of experiments around 2000 by Bauer et al. [15,16], but there is still not an unifying toolbox proposing automatic differentiation capacities for the ODE models usually found in systems biology. ...
In the last two decades, systems biology has experienced an ever-increasing boom, the momentum of which has also accelerated the rise in popularity of mathematical modeling in biology. Models of increasing complexity are being published daily, but their usability as prediction tools remains limited. Mathematical and computational methods to design the experimental plan beforehand can help to maximize the expected information yield. The theoretical foundations of experimental design go back to the 20th century, but their application to practical use-cases remains a challenge nowadays.During my PhD, I focused on applying optimal experimental design to an enzyme-mediated antibiotic resistance model. To do so, I studied both how to find an experimental plan, and how to assess in silico its quality to validate our design procedure.In this thesis, I present my study of potential methods to design optimal experimental plans in the face of non-identifiability. This study is supported by a shift in paradigm, from trying to identify parameter values, to trying to identify models solely for their prediction power. To demonstrate the practical usability of my approach, I propose a pipeline to validate the quality of the designs, supported by several software developments. In iterative design strategies, I studied the effect of a priori information on the superiority of this optimal experimental design approach compared to expert's designs.
... Another desirable property of these piecewise linearizations is that their representation can be factored and normalised into a specialized block matrix-based representation called abs-linear form (ALF). Hence, ALFs are operator representations of piecewise linear functions that can be realized or implemented as a slight as well as modular update on top of any existing modern matrix or linear algebra software package [Gri+15;Str+14]. Only small changes of existing tools for algorithmic differentiation like ADOL-C [WG12], cycADa [cyc21] or Tapenade [HP13] were required to obtain the local abs-linear model ∆F (c; .) in ALF in a completely automated way. Hence, this ingredient required by SALMIN is available once a code to evaluate F (.) is available. ...
... Many tools have been developed to derive and implement adjoint models automatically. These automatic tools take as input a forward model that users implement in languages such as C [8,37], C++ [6], Fortran [18], Python [27], and Julia [24], and they produce as output the associated discrete adjoint model in a line-by-line fashion, through source-to-source transformations, operator overloading, or a combination of both. While this black-box approach gives the highest degree of automation and (2.2) u n+1 = N (u n ), n = 0, . . . ...
... AD tools that differentiate programs at runtime are often relatively straightforward to develop using, for example, operator overloading in C++ [6,26,30,39,50,59]. Unfortunately, in reverse mode, they generally produce a large tape to store operations and intermediate values for subsequent reverse differentiation, which causes challenges with their memory footprint in real-world applications. ...
Computing derivatives is key to many algorithms in scientific computing and machine learning such as optimization, uncertainty
quantification, and stability analysis. Enzyme is a LLVM compiler
plugin that performs reverse-mode automatic differentiation (AD)
and thus generates high performance gradients of programs in languages including C/C++, Fortran, Julia, and Rust. Prior to this work,
Enzyme and other AD tools were not capable of generating gradients of GPU kernels. Our paper presents a combination of novel
techniques that make Enzyme the first fully automatic reversemode AD tool to generate gradients of GPU kernels. Since unlike
other tools Enzyme performs automatic differentiation within a
general-purpose compiler, we are able to introduce several novel
GPU and AD-specific optimizations. To show the generality and
efficiency of our approach, we compute gradients of five GPU-based
HPC applications, executed on NVIDIA and AMD GPUs. All benchmarks run within an order of magnitude of the original program’s
execution time. Without GPU and AD-specific optimizations, gradients of GPU kernels either fail to run from a lack of resources or
have infeasible overhead. Finally, we demonstrate that increasing
the problem size by either increasing the number of threads or
increasing the work per thread, does not substantially impact the
overhead from differentiation.
... Here, we use AD locally for the automatic setup of solvers and eliminate the inconvenience of hand-computed derivatives. For classical CPU architectures, several mature AD tools are available by now, for example ADOL-C [63], dco/c++ [31] and CoDiPack [47]. Advances in the direction of AD for GPU codes are more recent, examples include dco/map with applications in computational finance [32]. ...
We propose a universal method for the evaluation of generalized standard materials that greatly simplifies the material law implementation process. By means of automatic differentiation and a numerical integration scheme, AutoMat reduces the implementation effort to two potential functions. By moving AutoMat to the GPU, we close the performance gap to conventional evaluation routines and demonstrate in detail that the expression level reverse mode of automatic differentiation as well as its extension to second order derivatives can be applied inside CUDA kernels. We underline the effectiveness and the applicability of AutoMat by integrating it into the FFT-based homogenization scheme of Moulinec and Suquet and discuss the benefits of using AutoMat with respect to runtime and solution accuracy for an elasto-viscoplastic example.
... In comparison to numerical methods AD has the benefit of calculating the exact derivative. In addition, data-driven models in the form of software code can be assigned a derivative with the help of AD [12,109,159]. (iii) Visualisation. Especially in the case of data-induced conflicts, knowledge about possible dependencies between data sources is important. ...
This chapter describes the various approaches to analyse, quantify and evaluate uncertainty along the phases of the product life cycle. It is based on the previous chapters that introduce a consistent classification of uncertainty and a holistic approach to master the uncertainty of technical systems in mechanical engineering. Here, the following topics are presented: the identification of uncertainty by modelling technical processes, the detection and handling of data-induced conflicts, the analysis, quantification and evaluation of model uncertainty as well as the representation and visualisation of uncertainty. The different approaches are discussed and demonstrated on exemplary technical systems.
... -ADOL-C [31,64] -ArborX [44] -ARPACK [45] -Assimp [59] -BLAS and LAPACK [4] -cuSOLVER [19] ...
This paper provides an overview of the new features of the finite element library deal.II, version 9.3.
... Mykhaskiv et al. [59] use the same IDW mesh deformation approach from Xu et al. [58] described in the previous paragraph, but they include OpenCascade, which is an open source CAD kernel, to handle the geometry manipulation and intersection computation. Then they use ADOL-C [60] to differentiate OpenCascade using operator overloading. This allows the inclusion of the CAD engine directly into the optimization framework. ...
Advances in computational power allow the increase in the fidelity level of analysis tools used in conceptual aircraft design and optimization. These tools not only give more accurate assessments of aircraft efficiency, but also provide insights to improve the performance of next-generation aircraft. Aerodynamic shape optimization involves the inclusion of aerodynamic analysis tools in optimization frameworks to maximize the aerodynamic efficiency of an aircraft configuration via modifications of its outer mold line. When using CFD-based aerodynamic shape optimization, generating high-quality structured meshes for complex aircraft configurations becomes challenging, especially near junctions. Furthermore, mesh deformation procedures frequently generate negative volume cells when applied to these structured meshes during optimization. Complex geometries can be accurately modeled using overset meshes, whereby multiple high-quality structured meshes corresponding to different aircraft components overlap to model the complete aircraft configuration. However, from the standpoint of geometry manipulation, most methods operate on the entire geometry rather than on separate components, which diminishes the advantages of overset meshes. Tracking intersections among multiple components is a key challenge in the implementation of component-based geometry manipulation methods. The mesh nodes should also be updated in accordance to the intersection curves. This thesis addresses this issue by introducing of a geometry module that operates on individual components and uses triangulated surfaces to automatically compute intersections during optimization. A modified hyperbolic mesh marching algorithm is used to regenerate the overset meshes near intersections. The reverse-mode automatic differentiation is used to compute partial derivatives across this geometry module, so that it fits into an optimization framework that uses a hybrid adjoint method (ADjoint) to efficiently compute gradients for a large number of design variables. Particularities of the automatic differentiation of the geometry module are detailed in this thesis. By using these automatically updated meshes and the corresponding derivatives, the aerodynamic shape of the DLR-F6 geometry is optimized while allowing changes in the wing-fuselage intersection. Sixteen design variables control the fuselage shape and 128 design variables determine the wing surface. Under transonic flight conditions, the optimization reduces drag by 16 counts (5%) compared with the baseline design. This approach is also used to minimize drag of the PADRI 2017 strut-braced wing benchmark for a fixed lift constraint at transonic flight conditions. The drag of the optimized configuration is 15% lower than the baseline due to reduction of shocks and separation in the wing-strut junction region. This result is an example where high-fidelity modeling is required to quantify the benefits of a new aircraft configuration and address potential issues during the conceptual design. The methodologies developed in this work give additional flexibility for geometry and mesh manipulation tools used in aerodynamic shape optimization frameworks. This extends the applicability of design optimization tools to provide insights to more complex cases involving multiple components, including unconventional aircraft configurations.
... The equations of motion can be differentiated with respect to state variables, control output (generalized forces), time and physical parameters of the robot (see [8] for an overview). These derivatives can be computed with several methods: 1) approximation by finite differences, 2) automatic differentiation [31], i.e. by applying the chain rule formula in an automatic way knowing the derivatives of basic functions (cos, sin or exp), 3) closed-form derivatives of the EOM and 4) recursive and analytical formulations exploiting the structure of the closed-form equations of motion. While the first two methods are generic and numerical in nature, the latter two are analytical in nature and exploit the structure of the EOM. ...
Derivatives of equations of motion (EOM) describing the dynamics of rigid body systems are becoming increasingly relevant for the robotics community and find many applications in design and control of robotic systems. Controlling robots, and multibody systems comprising elastic components in particular, not only requires smooth trajectories but also the time derivatives of the control forces/torques, hence of the EOM. This paper presents the time derivatives of the EOM in closed form up to second-order as an alternative formulation to the existing recursive algorithms for this purpose, which provides a direct insight into the structure of the derivatives. The Lie group formulation for rigid body systems is used giving rise to very compact and easily parameterized equations.
... This requires the numerical evaluation of both the gradient and the Hessian matrix of the circular Raman emission. To calculate the occurring partial derivatives with machine precision we employ algorithmic differentiation (AD) [50][51][52][53]. The CoDiPack library [54] is used for AD as it shows a high performance for the problem considered in this paper. ...
Semiconductor quantum dots embedded in optical cavities are promising on-demand sources of single photons. Here, we theoretically study single photon emission from an optically driven two-photon Raman transition between the biexciton and the ground state of a quantum dot. The advantage of this process is that it allows all-optical control of the properties of the emitted single photon with a laser pulse. However, with the presence of other decay channels and excitation-induced quantum interference, on-demand emission of the single Raman photon is generally difficult to achieve. Here we show that laser pulses with non-trivial shapes can be used to maintain excitation conditions for which with increasing pulse intensities the on-demand regime is reached. To provide a realistic picture of the achievable system performance, we include phonon-mediated processes in the theoretical caluclations. While preserving both high photon purity and indistinguishability, we find that although based on a higher-order emission process, for realistic system parameters on-demand Raman photon emission is indeed achievable with suitably tailored laser pulses.
... In this paper we focus on the basic methodology of AD in a systematic application of the chain rule to implement isogeometric elements. Note that there are a wide range of advanced implementations [41][42][43] that might also be suitable for other applications e.g. gradient based shape optimization of IGA models, as shown in [44]. ...
We present a consistent and efficient approach to the formulation of geometric nonlinear finite elements for isogeometric analysis (IGA) and isogeometric B-Rep analysis (IBRA) based on the adjoint method. IGA elements are computationally expensive, especially for high polynomial degrees. Using the method presented here enables us to reduce this disadvantage and develop a methodical framework for the efficient implementation of IGA elements. The elements are consistently derived from energy functionals. The load vector and stiffness matrix are obtained from the first and second order derivatives of the energy. Starting from the functional, we apply the concept of algorithmic or automatic differentiation to compute the precise derivatives. Here, we compare the direct (forward) and adjoint (reversed) methods. Analysis of the computational graph allows us to optimize the computation and identify recurring modules. It turns out that using the adjoint method leads to a core-congruential formulation, which enables a clean separation between the mechanical behavior and the geometric description. This is particularly useful in CAD-integrated analysis, where mechanical properties are applied to different geometry types. The adjoint method produces the same results but requires significantly fewer operations and fewer intermediate results. Moreover, the number of intermediate results is no longer dependent on the polynomial degree of the NURBS. This is important for implementation efficiency and computation speed.
The procedure can be applied to arbitrary element formulations and coupling conditions based on energy functionals. For demonstration purposes, we present the proposed approach specifically for use with geometrically nonlinear trusses, beams, membranes, shells, and coupling conditions based on the penalty method.
... In the following we will dive deeper into the specifics of our implementation of the algorithm: In order to differentiate the smooth part of the composite objective functional and create a second order model of it around some current iterate, we take advantage of the automatic differentiation software package adol-C, cf. [29]. With the second order model at hand we can then consider subproblem (4.1) which has to be solved in order to obtain a candidate for the update of the current iterate. ...
We develop a globalized Proximal Newton method for composite and possibly non-convex minimization problems in Hilbert spaces. Additionally, we impose less restrictive assumptions on the composite objective functional considering differentiability and convexity than in existing theory. As far as differentiability of the smooth part of the objective function is concerned, we introduce the notion of second order semi-smoothness and discuss why it constitutes an adequate framework for our Proximal Newton method. However, both global convergence as well as local acceleration still pertain to hold in our scenario. Eventually, the convergence properties of our algorithm are displayed by solving a toy model problem in function space.
... Automatic differentiation (AD) [32] algorithms compute derivatives at working precision of any function using at most a small constant factor more arithmetic operations than the original function, solving the issues related to the other strategies. AD libraries are available for C and C++, but they typically rely on involved implementations and require particular type definitions and redefinition (or compatibility layers) of functions not already implemented in terms of basic AD operators [4,7,16,34,41,45,63]. Due to these issues, the project under consideration [20] opted for more inefficient symbolic computation. ...
This report is written as a post-mortem of a project that
has, perhaps, been the author’s most extensive personal
project: creating a complete and comprehensive solution
to one of the most famous programming problem sets
in the modern computer science curriculum Structure
and Interpretation of Computer Programs by Abelson,
G. J. Sussman, and J. Sussman.
It measures exactly:
• How much effort SICP requires (729 hours 19
minutes (over eight months), 292 sessions).
• How many computer languages it involves (6).
• How many pieces of software are required (9).
• How much communication with peers is needed.
It suggests:
• A practical software-supported task management
procedure for solving coursework.
• Several improvements, on the technical side, to any
hard skills teaching process.
• Several improvements, on the social side, to any
kind of teaching process.
The solution is published online (the source code and
pdf file):
• http://gitlab.com/Lockywolf/chibi-sicp (Nikishkin
2020)
This report (and the data in the appendix) can be
applied immediately as:
• A single-point estimate of the SICP problem set
difficulty.
• A class handout aimed at increasing students’ mo-
tivation to study.
• A data source for a study of learning patterns
among adult professionals aiming for continuing
education.
• An “almost ready” protocol for a convenient problem-
set solution procedure, which produces artefacts
that can be later used as a student portfolio.
• An “almost ready”, and “almost convenient” pro-
tocol for measuring time consumption of almost
any problem set expressible in a digital form.
Additionally, a time-tracking data analysis can be re-
produced interactively in the org-mode version of this
report. (See: Appendix: Emacs Lisp code for data ana-
lysis)
... Automatic differentiation (AD) [32] algorithms compute derivatives at working precision of any function using at most a small constant factor more arithmetic operations than the original function, solving the issues related to the other strategies. AD libraries are available for C and C++, but they typically rely on involved implementations and require particular type definitions and redefinition (or compatibility layers) of functions not already implemented in terms of basic AD operators [4,7,16,34,41,45,63]. Due to these issues, the project under consideration [20] opted for more inefficient symbolic computation. ...
This report aggregates the papers presented at the twenty-first annual Scheme and Functional Programming Workshop, hosted on August 28th, 2020, online and co-located with the twenty-fifth International Conference on Functional Programming. The Scheme and Functional Programming Workshop is held every year to provide an opportunity for researchers and practitioners using Scheme and related functional programming languages like Racket, Clojure, and Lisp, to share research findings and discuss the future of the Scheme programming language.
... Diese Typdefinition kann umgeschaltet werden, so das verschiedene AD-Datentypen eingebunden werden können. Dies wurde umgesetzt für die Vorwärts-und Rückwärts-Typen von ADOL-C [137], DCO [121] und CodiPack [122]. ADOL-C dient als technisch reifste Implementierung basierend auf der Technik des Operator-Überladens in C++. ...
Die Arbeit thematisiert die Beschleunigung multikriterieller interdisziplinärer Formoptimierung von Turbomaschinen durch Ableitungsinformation aus adjungierten Strömungssimulationen. Es werden zwei Vorgehensweisen zur Implementierung diskret adjungierter Pendants eines existierenden Strömungssimulationsverfahrens gegenübergestellt: Manuelle Adjungierung unterstützt durch algorithmische Differentiation einzelner Routinen sowie algorithmische Differentiation des gesamten Strömungslösers gefolgt von manueller Beschleunigung und Speicherreduktion des Resultats. Zu beiden Ansätzen werden jeweils Implementierungen auf Basis des gleichen zugrundeliegenden Simulationsverfahrens vorgestellt und diese miteinander verglichen. Es wird gradientenbasiertes Kriging, erweitert um eine geeignete Regularisierungstechnik, eingesetzt. Die Kombination bildet einen robustes gradientenbasiertes Optimierungsverfahren. Diese wird auf die Optimierung eines gegenläufigen Triebwerksfans angewandt.
... In particular, several implementations exist for C++, e.g. [13][14][15]. Majority of implementations of AD fall into one of the two categories of implementation techniques: ...
In mathematics and computer algebra, automatic differentiation (AD) is a set of techniques to evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.), elementary functions (exp, log, sin, cos, etc.) and control flow statements. AD takes source code of a function as input and produces source code of the derived function. By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program. This paper presents AD techniques available in ROOT, supported by Cling, to produce derivatives of arbitrary C/C++ functions through implementing source code transformation and employing the chain rule of differential calculus in both forward mode and reverse mode. We explain its current integration for gradient computation in TFormula. We demonstrate the correctness and performance improvements in ROOT’s fitting algorithms.
Data-flow reversal is at the heart of source-transformation reverse algorithmic differentiation (reverse ST-AD), arguably the most efficient way to obtain gradients of numerical models. However, when the model implementation language uses garbage collection (GC), for instance in Java or Python, the notion of address that is needed for data-flow reversal disappears. Moreover, GC is asynchronous and does not appear explicitly in the source. This paper presents an extension to the model of reverse ST-AD suitable for a language with GC. The approach is validated on a Java implementation of a simple Navier-Stokes solver. Performance is compared with existing AD tools ADOL-C and Tapenade on an equivalent implementation in C and Fortran.
The adjoint reversal of long evolutionary calculations (e.g. loops), where each iteration depends on the output of the previous iteration, is a common occurrence in computational engineering (e.g. computational fluid dynamics (CFD) simulation), physics (e.g. molecular dynamics) and computational finance (e.g. long Monte Carlo paths). For the edge case of a scalar state, the execution, as well as adjoint control flow reversal, are inherently serial operations, as there is no spatial dimension to parallelize. Our proposed method exploits the run time difference between passive function evaluation and augmented forward evaluation, which is inherent to most adjoint AD techniques. For high dimensional states, additional parallelization of the primal computation can and should be exploited at the spatial level. Still, for problem sizes where the parallelization of the primal has reached the barrier of scalability, the proposed method can be used to better utilize available computing resources and improve the efficiency of adjoint reversal.We expect this method to be especially useful for operator-overloading AD tools. However, the concepts are also applicable to source-to-source transformation and handwritten adjoints, or a hybrid of all approaches. For illustration, C++ reference implementations of a low dimensional evolution (lorenz attractor) and a high dimensional evolution (computational fluid dynamics problem in OpenFOAM) are discussed. Both theoretical bounds on the speedup and run time measurements are presented.KeywordsAlgorithmic DifferentiationAdjointsMPI
The full optimization of the design and operation of instruments whose functioning relies on the interaction of radiation with matter is a super-human task, due to the large dimensionality of the space of possible choices for geometry, detection technology, materials, data-acquisition, and information-extraction techniques, and the interdependence of the related parameters. On the other hand, massive potential gains in performance over standard, “experience-driven” layouts are in principle within our reach if an objective function fully aligned with the final goals of the instrument is maximized through a systematic search of the configuration space. The stochastic nature of the involved quantum processes make the modeling of these systems an intractable problem from a classical statistics point of view, yet the construction of a fully differentiable pipeline and the use of deep learning techniques may allow the simultaneous optimization of all design parameters.In this white paper, we lay down our plans for the design of a modular and versatile modeling tool for the end-to-end optimization of complex instruments for particle physics experiments as well as industrial and medical applications that share the detection of radiation as their basic ingredient. We consider a selected set of use cases to highlight the specific needs of different applications.
We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on adjoint variables can be lifted in a fine-grained manner. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. We demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on adjoint variables and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.
Though dynamic operation of chemical processes has been extensively explored theoretically in contexts such as economic model predictive control or even considering the potential for cyberattacks on control systems creating non-standard operating policies, important practical questions remain regarding dynamic operation. In this work, we look at two of these with particular relevance to process safety: (1) evaluating dynamic operating policies with respect to process equipment fidelity and (2) evaluating procedures for determining the parameters of an advanced control law that can promote both dynamic operation as well as safety if appropriately designed. Regarding the first topic, we utilize computational fluid dynamics and finite element analysis simulations to analyze how cyberattacks on control systems could impact a metric for stress in equipment (maximum Von Mises stress) over time. Subsequently, we develop reduced-order models showing how both a process variable and maximum Von Mises stress vary over time in response to temperature variations at the boundary of the equipment, to use in evaluating how advanced control frameworks might impact and consider the stress. We close by investigating options for obtaining parameters of an economic model predictive control design that would need to meet a variety of theoretical requirements for safety guarantees to hold. This provides insights on practical safety aspects of control theory, and also indicates relationships between control and design from a safety perspective that highlight further relationships between design and control under dynamic operation to deepen perspectives from the computational fluid dynamics and finite element analysis discussions.
We present a novel, general construction to abstractly interpret higher-order automatic differentiation (AD). Our construction allows one to instantiate an abstract interpreter for computing derivatives up to a chosen order. Furthermore, since our construction reduces the problem of abstractly reasoning about derivatives to abstractly reasoning about real-valued straight-line programs, it can be instantiated with almost any numerical abstract domain, both relational and non-relational. We formally establish the soundness of this construction.
We implement our technique by instantiating our construction with both the non-relational interval domain and the relational zonotope domain to compute both first and higher-order derivatives. In the latter case, we are the first to apply a relational domain to automatic differentiation for abstracting higher-order derivatives, and hence we are also the first abstract interpretation work to track correlations across not only different variables, but different orders of derivatives.
We evaluate these instantiations on multiple case studies, namely robustly explaining a neural network and more precisely computing a neural network’s Lipschitz constant. For robust interpretation, first and second derivatives computed via zonotope AD are up to 4.76× and 6.98× more precise, respectively, compared to interval AD. For Lipschitz certification, we obtain bounds that are up to 11,850× more precise with zonotopes, compared to the state-of-the-art interval-based tool.
Algorithmic differentiation (AD) is a set of techniques to obtain accurate derivatives of a computer-implemented function in an automatic fashion. State-of-the-art AD tools rely on the source code of the implementation or internal representations of compilers building it. We present the new AD tool Derivgrind, which augments the machine code of compiled programs with forward AD logic. Derivgrind leverages the Valgrind instrumentation framework for a structured access to the machine code, and a shadow memory tool to store dot values. Depending on the application scenario, no access to the source code is required at all, or the access is restricted to the parts defining input and output variables. Derivgrind's versatility comes at the price of scaling the running time by a factor between 60 and 140, measured on a benchmark based on a PDE solver. Results of our extensive test suite indicate that Derivgrind produces correct results on GCC- and Clang-compiled programs, including a Python interpreter, with a small number of exceptions. While we provide a list of scenarios that Derivgrind does not handle correctly, most of them are academic examples or originate from highly optimized math libraries. We will therefore further study the potential of our tool in more complex software projects.
Cybersecurity of control systems is an important issue in next-generation manufacturing that can impact both operational objectives (safety and performance) as well as process designs (via hazard analysis). Cyberattacks differ from faults in that they can be coordinated efforts to exploit system vulnerabilities to create otherwise unlikely hazard scenarios. Because coordination and targeted process manipulation can be characteristics of attacks, some of the tactics previously analyzed in our group from a control system cybersecurity perspective have incorporated randomness to attempt to thwart attacks. The underlying assumption for the generation of this randomness has been that it can be achieved on a classical computer; however, quantum computers can also create random behavior in the results of computations. This work explores how errors in quantum hardware that can create non-deterministic outputs from quantum computers interact with control system cybersecurity. These studies serve as a reminder of the need to incorporate cybersecurity considerations at the process design stage.
Algorithmic derivatives can be useful to quantify uncertainties and optimize parameters using computer simulations. Whether they actually are, depends on how "well-linearizable" the program is. Proton computed tomography (pCT) is a medical imaging technology with the potential to increase the spatial accuracy of the dose delivered in proton-beam radiotherapy. The Bergen pCT collaboration is developing and constructing a digital tracking calorimeter (DTC) to measure the position, direction and energy of protons after they passed through a patient, and a software pipeline to process these data into a pCT image. We revisit the software pipeline from the perspective of algorithmic differentiation (AD). In the early subprocedures, several obstacles such as discrete variables or frequent discontinuities were identified, and are probably tackled best by using surrogate models. The model-based iterative reconstruction (MBIR) subprocedure in the end seems to be AD-ready, and we propose changes in the AD workflow that can reduce the memory consumption in reverse mode.
Fouling in condensers of thermal power stations can reduce their heat load, which increases the backpressure of the steam turbine and cause deleterious effects to the energy output. A complete analysis of the condenser behavior must handle the exchanger as an integrated part of an entire cooling system, which also encompasses hydraulic equipment such as the pumping system. This paper presents a computer simulation analysis of the behavior of a condenser affected by inorganic precipitation fouling in the water side. Instead of using a fixed ‘fouling factor’, a dynamic fouling model is incorporated in the overall thermal-hydraulic calculations in order to simulate fouling resistances overtime during long term condenser operation, and their dependence on operating parameters such as water velocity and temperature. The mathematical model employed shows how the system performance is affected by the reduction of the overall heat transfer coefficient due to the deposit accumulation and also predicts the hydraulic impact of fouling, which increases the hydraulic resistance for the cooling water flow, resulting in different fouling rates in different tube rows and along each tube length. The condenser model is validated through its comparison with operating data of real power plants.
The complex time-dependent heat and electromagnetic energy transfer in a new type of reverse thermal engine is analyzed. The reverse thermal engine consists of a cold body, a Peltier element and an electric circuit containing an inductor with controllable inductance. This system allows cooling a body below the ambient temperature. The inductor acts as an accumulator of magnetic energy, receiving electrical work from the Peltier element during some time intervals and supplying the Peltier element with electrical work during other time intervals. The system is named self-driven reverse thermal engine since one of its subsystems (the inductor) receives and releases work. The new type of engine has features which make it different from classical reverse thermal engines: it cannot operate in steady state and it operates for a finite time interval. Instead of usual indicators of performance such as the coefficient of performance, other performance indicators should be used, such as the minimum cooled body temperature and the interval of time needed to reach a given cooled body temperature. Significant cooling effects do not necessarily need high inductance values. The operation of the new engine has been optimized by using direct optimal control procedures. The open-source program package BOCOP has been used to transform the optimal control problem into a non-linear dynamic problem. The minimum temperature reached by the cooled body depends on its mass. There exists, however, a global minimum temperature, for a specified optimum mass of the cooled body. The minimum temperature decreases by increasing the value of the Seebeck coefficient and by decreasing the thermal conductance. Thermal damped oscillations may arise under special circumstances. This implies a very small difference between the initial temperatures of the cold and cooled bodies and a specific range of variation for the conductance of the Peltier element.
A discrete adjoint method implemented in a coupled pressure-based RANS solver is presented in this paper. The adjoint equations are solved using an adjoint fixed point iteration that inherits the convergence properties of the primal solver. Automatic differentiation is used extensively for the construction of the adjoint fixed point iteration. The concept of Krylov subspace methods was adopted to stabilize the solution procedure. A common linearization technique in collocated pressure-based algorithms is the introduction of a mass flux variable on the cell faces which is kept constant during the inner iterations. This variable is treated as an independent adjoint variable in related publications. We propose a new method that allows to treat the mass fluxes implicitly in order to take full advantage of the preconditioner of the primal solver. The adjoint solver is general and is not restricted by the commonly used frozen turbulence approximation. It can deal with any turbulence model that is supported by the flow solver as well as any boundary condition. This includes mixing planes and mesh interfaces needed for multi stage turbo machinery simulations. Furthermore, there is no restriction on the choice of objective function. The sensitivities of the adjoint solver have been validated with sensitivities obtained with finite differences.
An entirely surface based interpolation method based on radial basis functions (RBF) was developed to deform the surface mesh. We propose the use of discrete geodesics instead of the classical Euclidean distance as the distance measure for the RBF interpolation. As an alternative, a direct deformation method with adjoint consistent smoothing is also described and used in the presented optimization cases.
The developed adjoint solver and deformation routines were used to optimize a turbulent bend with different Reynolds numbers as well as the the rotor blade of an axial turbine.
Safety-critical processes are becoming increasingly automated and connected. While automation can increase efficiency, it brings new challenges associated with guaranteeing safety in the presence of uncertainty especially in the presence of control system cyberattacks. One of the challenges for developing control strategies with guaranteed safety and cybersecurity properties under sufficient conditions is the development of appropriate detection strategies that work with control laws to prevent undetected attacks that have immediate closed-loop stability consequences. Achieving this, in the presence of uncertainty brought about by plant/model mismatch and process dynamics that can change with time, requires a fundamental understanding of the characteristics of attacks that can be detected with reasonable detection mechanisms and characterizing and verifying system safety properties when cyberattacks and changing system behavior cannot be distinguished. Motivated by this, this paper discusses three cyberattack detection strategies for nonlinear processes whose dynamics change with time when these processes are operated under an optimization-based control strategy known as Lyapunov-based economic model predictive control (LEMPC) until the closed-loop state either leaves a characterizable region of state-space or an attack detection threshold related to state estimates or state predictions is exceeded. Following this, the closed-loop state is maintained within a larger region of operation under an updated cyberattack detection strategy for a characterizable time period. A Taylor series-based model is used for making state predictions to allow theoretical guarantees to be explicitly tied to the numerical approximation of the model used within the LEMPC. A process example illustrates the Taylor series-based model concept.
Acid impact is the highly efficient kind of carbonate reservoir stimulation. In contrast to hydraulic fracturing acid impact improves matrix permeability that remains high while production. In this case propagation of reactive flow into matrix brings stimulation that means that the flow factor is crucial for fracture acidizing. This paper is focused on the reactive flow modelling in the existing fracture and oil saturated matrix. Two phase multicomponent Darcy flow which takes into account dissolution kinetics is modelled in two domains (fracture, matrix). Using prescribed porosity permeability relationship the stimulated permeability field is calculated. Acid propagation into matrix is investigated for different injection control regimes. Stimulated productivity is assessed at the final stage of stimulation in the enlarged domain. Obtained results show that the injection time plays a key role for stimulated productivity. Sensitivity analysis to the other parameters is also given.
La volonté de diminuer la consommation en carburant en aéronautique amène à repenser les architectures de distribution et de conversion de puissance embarquées. L’utilisation d’un outil d’optimisation déterministe apparait comme une solution prometteuse pour prendre en compte les compromis intrinsèques aux convertisseurs statiques en pré-dimensionnement, avec pour objectif de minimiser leur masse. Un outil d’optimisation est donc proposé pour apporter des éléments de comparaison entre différentes topologies d’onduleur et entre différentes solutions technologiques.Le développement de l’outil passe par l’élaboration de modèles analytiques pour être compatible avec l’utilisation d’un algorithme à base de gradients. Une analyse harmonique reposant sur des transformées de Fourier permet de reconstituer les ondulations de chaque signal, en regard des normes aéronautiques. La modélisation s’intéresse également à la construction de modèles pour les composants passifs et actifs, permettant en particulier d’évaluer les pertes et donc le rendement du convertisseur. L’intégralité des modèles développés fait l’objet d’une validation expérimentale sur un prototype à 10 kW.L’utilisation d’un algorithme à base de gradient n’étant pas classique en électronique de puissance, des études sont menées pour valider la méthodologie proposée. Ces études montrent l’adéquation du choix d’un tel algorithme avec le besoin de pré-dimensionnement, tout en indiquant ses limites et les perspectives. Enfin, l’outil d’optimisation est utilisé pour comparer différentes solutions architecturales et technologiques sur des cas d’application aéronautiques. Les résultats obtenus sont discutés en regard des méthodes de dimensionnement classiques.
Nonsmooth optimization refers to the general problem of minimizing (or maximizing)
functions that have discontinuous gradients. This Special Issue contains six research articles that collect together the most recent techniques and applications in the area of nonsmooth optimization. These include novel techniques utilizing some decomposable structures in nonsmooth problems—for instance, the difference-of-convex (DC) structure—and interesting important practical problems, like multiple instance learning, hydrothermal unit-commitment problem, and scheduling the disposal of nuclear waste. In the first article, “A Mixed-Integer and Asynchronous Level Decomposition with Application to
the Stochastic Hydrothermal Unit-Commitment Problem” by Bruno Colonetti, Erlon Cristian Finardi and Welington de Oliveira, the authors develop an efficient algorithm for solving uncertain unit-commitment (UC) problems. The efficiency of the algorithm is based on the novel asynchronous level decomposition of the UC problem and the parallelization of the algorithm.
We provide a new approach for the efficient matrix-free application of the transpose of the Jacobian for the spectral element method for the adjoint-based solution of partial differential equation (PDE) constrained optimization. This results in optimizations of nonlinear PDEs using explicit integrators where the integration of the adjoint problem is not more expensive than the forward simulation. Solving PDE constrained optimization problems entails combining expertise from multiple areas, including simulation, computation of derivatives, and optimization. The Portable, Extensible Toolkit for Scientific computation (PETSc) together with its companion package, the Toolkit for Advanced Optimization (TAO), is an integrated numerical software library that contains an algorithmic/software stack for solving linear systems, nonlinear systems, ordinary differential equations, differential algebraic equations, and large-scale optimization problems and, as such, is an ideal tool for performing PDE-constrained optimization. This paper describes an efficient approach in which the software stack provided by PETSc/TAO can be used for large-scale nonlinear time-dependent problems. Time integration can involve a range of high-order methods, both implicit and explicit. The PDE-constrained optimization algorithm used is gradient-based and seamlessly integrated with the simulation of the physical problem.
This work proposes a windowed least-squares (WLS) approach for model reduction of dynamical systems. The proposed approach sequentially minimizes the time-continuous full-order-model residual within a low-dimensional space–time trial subspace over time windows. The approach comprises a generalization of existing model reduction approaches, as particular instances of the methodology recover Galerkin, least-squares Petrov–Galerkin (LSPG), and space–time LSPG projection. In addition, the approach addresses key deficiencies in existing model reduction techniques, e.g., the dependence of LSPG and space–time LSPG projection on the time discretization and the exponential growth in time exhibited by a posteriori error bounds for both Galerkin and LSPG projection. We consider two types of space–time trial subspaces within the proposed approach: one that reduces only the spatial dimension of the full-order model, and one that reduces both the spatial and temporal dimensions of the full-order model. For each type of trial subspace, we consider two different solution techniques: direct (i.e., discretize then optimize) and indirect (i.e., optimize then discretize). Numerical experiments conducted using trial subspaces characterized by spatial dimension reduction demonstrate that the WLS approach can yield more accurate solutions with lower space–time residuals than Galerkin and LSPG projection.
We apply reverse accumulation to obtain automatic gradients and error estimates of functions which include in their computation a convergent iteration of the form y= Φ(y,u), where y and u are vectors.We suggest an implementation approach which allows this to be done by a fairly routine extension of existing reverse accumulation code. We show how to re-use the computational graph for the fixed point constructor Φ so as to set explicit stopping criteria for the iterations, based on the gradient accuracy required.Our construction allows the gradient vector to be obtained to the same order of accuracy as the objective function values (which is in general the best we can hope to achieve), and the same order of computational cost (which does not explicitly depend upon the number of independent variables.) The technique can be applied to functions which contain several iterative constructions, either serially or nested
In its basic form, the reverse mode of computational differentiation yields the gradient of a scalar-valued function at a cost that is a small multiple of the computational work needed to evaluate the function itself. However, the corresponding memory requirement is proportional to the run-time of the evaluation program. Therefore, the practical applicability of the reverse mode in its original formulation is limited despite the availability of ever larger memory systems. This observation leads to the development of checkpointing schedules to reduce the storage requirements. This article presents the function revolve, which generates checkpointing schedules that are provably optimal with regard to a primary and a secondary criterion. This routine is intended to be used as an explicit “controller” for running a time-dependent applications program.
Acyclic and star coloring problems are specialized vertex coloring problems that arise in the efficient computation of Hessians using automatic differentiation or finite differencing, when both sparsity and symmetry are exploited. We present an algorithmic paradigm for finding heuristic solutions for these two NP-hard problems. The underlying common technique is the exploitation of the structure of two-colored induced subgraphs. For a graph G on n vertices and m edges, the time complexity of our star coloring algorithm is O(nd2), where dk, a generalization of vertex degree, denotes the average number of distinct paths of length at most k edges starting at a vertex in G. The time complexity of our acyclic coloring algorithm is larger by a multiplicative factor involving the inverse of Ackermann's function. The space complexity of both algorithms is O(m). To the best of our knowledge, our work is the first practical algorithm for the acyclic coloring problem. For the star coloring problem, our algorithm uses fewer colors and is considerably faster than a previously known O(nd3)-time algorithm. Computational results from experiments on various large-size test graphs demonstrate that the algorithms are fast and produce highly effective solutions. The use of these algorithms in Hessian computation is expected to reduce overall runtime drastically.
Adjoint equations of dierential,equations have seen widespread applications in optimization, inverse problems and uncertainty quantification. A major challenge in solving adjoint equations for time dependent systems has been the need to use the solution of the original system in the adjoint calculation and the associated memory,requirement. In applications where storing the entire solution history is impractical, checkpointing methods have frequently been used. However, traditional checkpointing algorithms such as revolve require a priori knowledge of the number,of time steps, making these methods incompatible with adaptive time stepping. We propose a dynamic,checkpointing algorithm applicable when,the number,of time steps is a priori unknown. Our algorithm maintains a specified number,of checkpoints on the fly as time integration proceeds for an arbitrary number of time steps. The resulting checkpoints at any snapshot during the time integration have the optimal repetition number. The eciency,of our algorithm is demonstrated both analytically and experimentally in solving adjoint equations. This algorithm also has significant advantage in automatic dierentiation,when the length of execution is variable. Key words. adjoint equation, dynamic checkpointing, automatic dierentiation, checkpointing
Graph coloring has been employed since the 1980s to efficiently compute sparse Jacobian and Hessian matrices using either finite differences or automatic differentiation. Several coloring problems occur in this context, depending on whether the matrix is a Jacobian or a Hessian, and on the specifics of the computational techniques employed. We consider eight variant vertex coloring problems here. This article begins with a gentle introduction to the problem of computing a sparse Jacobian, followed by an overview of the historical development of the research area. Then we present a unifying framework for the graph models of the variant matrix estimation problems. The framework is based upon the viewpoint that a partition of a matrix into structurally orthogonal groups of columns corresponds to distance-2 coloring an appropriate graph representation. The unified framework helps integrate earlier work and leads to fresh insights; enables the design of more efficient algorithms for many problems; leads to new algorithms for others; and eases the task of building graph models for new problems. We report computational results on two of the coloring problems to support our claims. Most of the methods for these problems treat a column or a row of a matrix as an atomic entity, and partition the columns or rows (or both). A brief review of methods that do not fit these criteria is provided. We also discuss results in discrete mathematics and theoretical computer science that intersect with the topics considered here.
This article considers the problem of evaluating all pure and mixed partial derivatives of some vector function defined by an evaluation procedure. The natural approach to evaluate derivative tensors might appear to be their recursive calculation in the usual forward mode of Computational Differentiation. However, with the approach presented in this article, much simpler data access patterns and similar or lower computational counts can be achieved through propagating a family of univariate Taylor series of a suitable degree. It is applicable for arbitrary orders of derivatives. Also it is possible to calculate derivatives only in some directions instead of the full derivative tensor. Explicit formulas for all tensor entries as well as estimates for the corresponding computational complexities are given. 1991 Mathematics Subject Classification: Primary 65D05, 65Y20, 68Q40 Keywords: Higher Order Derivatives, Computational Differentiation 1 Introduction Many applications in scientific...
As model based optimization techniques play a more and more important role in the chemical process industries, there is a great demand for ever more efficient and reliable process optimization software. In the first part of this paper, the theoretical aspects of a tailored multiple shooting based solution strategy for dynamic process optimization have been presented (Leineweber, Bauer, Bock & Schlöder, 2002. An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization-part I: theoretical aspects). The current second part describes software aspects of the specific implementation muscod-ii and provides numerical results for several application examples. muscod-ii has been coupled with the dynamic process modeling software gPROMS via the standard equation set object (ESO) interface of CAPE-OPEN. Thereby, an advanced dynamic optimization platform for integrated batch processes has been created, where each process stage is separately modeled in gPROMS, and the multistage dynamic optimization problem is assembled and solved with MUSCOD-II. The code has also been parallelized based on the portable MPI standard. It is shown that the use of directional sensitivities becomes very important for larger problems with many algebraic variables, leading to drastically reduced computing times compared with strategies with complete constraint linearization. In addition, gPROMS ESO models are compared with classical Fortran models in terms of computational performance, and it is found that only a moderate loss of performance occurs if so-called in-process ESOs are employed. Finally, it is demonstrated that a significant speed-up can be obtained through parallel function and gradient evaluations.
Frequently, the computation of derivatives for optimizing time-dependent problems is based on the integration of the adjoint differential equation. For this purpose, the knowledge of the complete forward solution may be required. Similar information is needed in the context of a posteriori error estimation with respect to a given functional. In the area of flow control, especially for three dimensional problems, it is usually impossible to keep track of the full forward solution due to the lack of storage capacities. Further, for many problems, adaptive time-stepping procedures are needed toward efficient integration schemes in time. Therefore, standard optimal offline checkpointing strategies are usually not well suited in that framework. In this paper we present two algorithms for an online checkpointing procedure that determines the checkpoint distribution on the fly. We prove that these approaches yield checkpointing distributions that are either optimal or almost optimal with only a small gap to optimality. Numerical results underline the theoretical results.