Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Information flow of Clad in ROOT
Source publication
In mathematics and computer algebra, automatic differentiation (AD) is a set of techniques to evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, et...
Context in source publication
Similar publications
A new perturbation and continuation method is presented for computing and analyzing stellarator equilibria. The method is formally derived from a series expansion about the equilibrium condition $F \equiv J \times B - \nabla p = 0$, and an efficient algorithm for computing solutions to 2nd and 3rd order perturbations is developed. The method has be...
Composite materials and structures are inherently in-homogeneous across multiple scales. Multi-scale modelling offers opportunities to apprehend the coupling of material behaviour and characteristics from the micro- to meso- and macro-scales. In this paper, a multi-scale finite element method (FE2)is proposed to compute the modal properties of visc...
Accurate estimates of left ventricle elastances based on non-invasive measurements are required for clinical decision-making during treatment of valvular diseases. The present study proposes a parameter discovery approach based on a lumped parameter model of the cardiovascular system in conjunction with optimization and non-invasive, clinical input...
The use of digitally reconstructed radiographs (DRRs) to solve inverse problems such as slice-to-volume registration and 3D reconstruction is well-studied in preoperative settings. In intraoperative imaging, the utility of DRRs is limited by the challenges in generating them in real-time and supporting optimization procedures that rely on repeated...
We present the Differentiable Lensing Lightcone (DLL), a fully differentiable physical model designed for being used as a forward model in Bayesian inference algorithms requiring access to derivatives of lensing observables with respect to cosmological parameters. We extend the public FlowPM N-body code, a particle-mesh N-body solver, simulating le...
Citations
... HEP projects that exploit AD include work on MadJAX [20], ACTS [21] and ROOT [22]. Gradients are used in objective-function optimizations, or in the propagation of uncertainty in different applications. ...
... As expected by the AD theory, gradients produced by reverse mode AD scale better as the time complexity of the computation is independent on the number of inputs. The better performance of Clad AD approach versus the numerical differentiation, via the central finite difference method, has been previously studied [22]. Further performance improvements can be achieved by moving the second order derivatives (the Hessian matrix) to use Clad. ...
Automatic Differentiation (AD) is instrumental for science and industry. It is a tool to evaluate the derivative of a function specified through a computer program. The range of AD application domain spans from Machine Learning to Robotics to High Energy Physics. Computing gradients with the help of AD is guaranteed to be more precise than the numerical alternative and have a low, constant factor more arithmetical operations compared to the original function. Moreover, AD applications to domain problems typically are computationally bound. They are often limited by the computational requirements of high-dimensional parameters and thus can benefit from parallel implementations on graphics processing units (GPUs).
Clad aims to enable differential analysis for C/C++ and CUDA and is a compiler-assisted AD tool available both as a compiler extension and in ROOT. Moreover, Clad works as a plugin extending the Clang compiler; as a plugin extending the interactive interpreter Cling; and as a Jupyter kernel extension based on xeus-cling.
We demonstrate the advantages of parallel gradient computations on GPUs with Clad. We explain how to bring forth a new layer of optimization and a proportional speed up by extending Clad to support CUDA. The gradients of well-behaved C++ functions can be automatically executed on a GPU. The library can be easily integrated into existing frameworks or used interactively. Furthermore, we demonstrate the achieved application performance improvements, including (≈10x) in ROOT histogram fitting and corresponding performance gains from offloading to GPUs.
... In this method, automatic differentiation is used to obtain the derivatives of each order of N (x, t, w, b). Automatic differentiation (AD) allows derivatives of arbitrary order to be calculated automatically to working accuracy by repeated application of the chain rule [45]. In a neural network, forward pass is used to calculate the values of all variables and reverse pass is used to calculate the derivatives. ...
The Fokker–Planck (FP) equation is of significant relevance in stochastic dynamics. To solve the FP equation, this paper proposes a physically guided deep learning-based method. The proposed method uses physical modeling as the constraint for guiding the deep neural networks to learn the solutions of the FP equation from unlabeled data. Four numerical examples were used to verify the accuracy and effectiveness of the proposed method for solving linear FP equations and nonlinear FP equations. Moreover, we examined the robustness of the trained model. Finally, extra observation data was used to replace the boundary conditions and initial conditions in complicated forms. Meanwhile, the effect of missing boundary conditions or initial conditions to the proposed method was further analyzed. The results demonstrated that observation data can be used as promising substitution for the initial and boundary conditions while the proposed method can still obtain accurate numerical solutions in the absence of some conditions.
... which can be differentiated to arbitrary order up to machine precision via application of chain rule [172,214]. While biases are presently inevitable [222], these regression models are in theory constructed without necessarily committing to a designated class of basis functions (e.g. ...
Edge plasma turbulence is critical to the performance of magnetic confinement fusion devices. Towards better understanding edge turbulence in both theory and experiment, a custom-built physics-informed deep learning framework constrained by partial differential equations is developed to accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure. This calculation is not otherwise possible using conventional equilibrium models. With this technique, the first direct quantitative comparisons of turbulent fields between electrostatic two-fluid theory and electromagnetic gyrokinetic modelling are demonstrated with good overall agreement found in magnetized helical plasmas at low normalized pressure. To translate these computational techniques to experimental fusion plasmas, a novel method to translate brightness measurements of HeI line radiation into local plasma fluctuations is demonstrated via a newly created deep learning framework that integrates neutral transport physics and collisional radiative theory for the $3^3 D - 2^3 P$ transition in atomic helium. Using fast camera data on the Alcator C-Mod tokamak, this thesis presents the first 2-dimensional time-dependent experimental measurements of the turbulent electron density, electron temperature, and neutral density in a fusion plasma using a single spectral line. With this experimentally inferred data, initial estimates of the 2-dimensional turbulent electric field consistent with drift-reduced Braginskii theory under the framework of an axisymmetric fusion plasma with purely toroidal field are calculated. The inclusion of atomic helium effects on particle and energy sources are found to strengthen correlations between the electric field and electron pressure while broadening turbulent field amplitudes which impact ${\bf E \times B}$ flows and shearing rates.
... HEP projects that exploit AD include work on MadJAX [20], ACTS [21] and ROOT [22]. Gradients are used in objective-function optimizations, or in the propagation of uncertainty in different applications. ...
... As expected by the AD theory, gradients produced by reverse mode AD scale better as the time complexity of the computation is independent on the number of inputs. The better performance of Clad AD approach versus the numerical differentiation, via central finite difference method, has been previously studied [22]. Further performance improvements can be achieved by moving the second order derivatives (the Hessian matrix) to use Clad. ...
Automatic Differentiation (AD) is instrumental for science and industry. It is a tool to evaluate the derivative of a function specified through a computer program. The range of AD application domain spans from Machine Learning to Robotics to High Energy Physics. Computing gradients with the help of AD is guaranteed to be more precise than the numerical alternative and have a low, constant factor more arithmetical operations compared to the original function. Moreover, AD applications to domain problems typically are computationally bound. They are often limited by the computational requirements of high-dimensional parameters and thus can benefit from parallel implementations on graphics processing units (GPUs). Clad aims to enable differential analysis for C/C++ and CUDA and is a compiler-assisted AD tool available both as a compiler extension and in ROOT. Moreover, Clad works as a plugin extending the Clang compiler; as a plugin extending the interactive interpreter Cling; and as a Jupyter kernel extension based on xeus-cling. We demonstrate the advantages of parallel gradient computations on GPUs with Clad. We explain how to bring forth a new layer of optimization and a proportional speed up by extending Clad to support CUDA. The gradients of well-behaved C++ functions can be automatically executed on a GPU. The library can be easily integrated into existing frameworks or used interactively. Furthermore, we demonstrate the achieved application performance improvements, including (~10x) in ROOT histogram fitting and corresponding performance gains from offloading to GPUs.
... (Laurent Hascoet, 2013) performs the code transformation at compile time in an AOT language, but requires that the user request differentiation outside the language. Clad (Vassilev et al., 2020) is a similar successor system operating on C/C++. Other AD systems that allow the user to request differentiation in the language trace the computation at runtime and differentiate the trace (Abadi et al., 2016;Maclaurin et al., 2015;Bradbury et al., 2020;Paszke et al., 2019) or use dynamic language features to transform the code at runtime (Innes et al., 2019). ...
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic differentiation system and multiple Tensor implementations within a modern ahead-of-time compiled language oriented around mutable value semantics. The resulting platform has been validated through use in over 30 deep learning models and has been employed across data center and mobile applications.
One of the most intensely studied aspects of magnetic confinement fusion is edge plasma turbulence which is critical to reactor performance and operation. Drift-reduced Braginskii two-fluid theory has for decades been widely applied to model boundary plasmas with varying success. Towards better understanding edge turbulence in both theory and experiment, we demonstrate that a physics-informed deep learning framework constrained by partial differential equations can accurately learn turbulent fields consistent with the two-fluid theory from partial observations of electron pressure which is not otherwise possible using conventional equilibrium models. This technique presents a paradigm for the advanced design of plasma diagnostics and validation of magnetized plasma turbulence theories in challenging thermonuclear environments.
