Xavier Warin

Électricité de France (EDF) | EDF · OSIRIS

In this paper, we propose a complete modelling framework to value several batteries in the electricity intraday market at the trading session scale. The model consists of a stochastic model for the 24 mid-prices (one price per delivery hour) combined with a deterministic model for the liquidity costs (representing the cost of going deeper in the or...

We propose a comprehensive framework for policy gradient methods tailored to continuous time reinforcement learning. This is based on the connection between stochastic control problems and randomised problems, enabling applications across various classes of Markovian continuous time control problems, beyond diffusion models, including e.g. regular,...

Rahimi and Recht (2007) introduced the idea of decomposing positive definite shift-invariant kernels by randomly sampling from their spectral distribution. This famous technique, known as Random Fourier Features (RFF), is in principle applicable to any such kernel whose spectral distribution can be identified and simulated. In practice, however, it...

We study deterministic optimal control problems for differential games with finite horizon. We propose new approximations of the strategies in feedback form and show error estimates and a convergence result of the value in some weak sense for one of the formulations. This result applies in particular to neural network approximations. This work foll...

A new Kolmogorov-Arnold network (KAN) is proposed to approximate potentially irregular functions in high dimension. We show that it outperforms multilayer perceptrons in terms of accuracy and converges faster. We also compare it with ReLU-KAN, a recently proposed network: it is more time consuming than ReLU-KAN, but more accurate.

In this paper, we introduce various machine learning solvers for (coupled) forward-backward systems of stochastic differential equations (FBSDEs) driven by a Brownian motion and a Poisson random measure. We provide a rigorous comparison of the different algorithms and demonstrate their effectiveness in various applications, such as cases derived fr...

We study deterministic optimal control problems for differential games with finite horizon. We propose new approximations of the strategies in feedback form, and show error estimates and a convergence result of the value in some weak sense for one of the formulations. This result applies in particular to neural networks approximations. This work fo...

We consider a deterministic optimal control problem, focusing on a finite horizon scenario. Our proposal involves employing deep neural network approximations to capture Bellman’s dynamic programming principle. This also corresponds to solving first-order Hamilton–Jacobi–Bellman (HJB) equations. Our work builds upon the research conducted by Huré e...

We develop a new policy gradient and actor-critic algorithm for solving mean-field control problems within a continuous time reinforcement learning setting. Our approach leverages a gradient-based representation of the value function, employing parametrized randomized policies. The learning for both the actor (policy) and critic (value function) is...

In this paper, we propose a multidimensional statistical model of intraday electricity prices at the scale of the trading session, which allows all products to be simulated simultaneously. This model, based on Poisson measures and inspired by the Common Shock Poisson Model, reproduces the Samuelson effect (intensity and volatility increases as time...

Optimization of storage using neural networks is now commonly achieved by solving a single optimization problem. We first show that this approach allows solving high-dimensional storage problems, but is limited by memory issues. We propose a modification of this algorithm based on the dynamic programming principle and propose neural networks that o...

We study news neural networks to approximate function of distributions in a probability space. Two classes of neural networks based on quantile and moment approximation are proposed to learn these functions and are theoretically supported by universal approximation theorems. By mixing the quantile and moment features in other new networks, we devel...

We present a new neural network to approximate convex functions. This network has the particularity to approximate the function with cuts which is, for example, a necessary feature to approximate Bellman values when solving linear stochastic optimization problems. The network can be easily adapted to partial convexity. We give an universal approxim...

This paper is devoted to the numerical resolution of McKean-Vlasov control problems via the class of mean-field neural networks introduced in our companion paper [25] in order to learn the solution on the Wasserstein space. We propose several algorithms either based on dynamic programming with control learning by policy or value iteration, or backw...

We prove a rate of convergence for the N-particle approximation of a second-order partial differential equation in the space of probability measures, like the Master equation or Bellman equation of mean-field control problem under common noise. The rate is of order 1/N for the pathwise error on the solution v and of order 1/ √ N for the L 2-error o...

We study the machine learning task for models with operators mapping between the Wasserstein space of probability measures and a space of functions, like e.g. in mean-field games/control problems. Two classes of neural networks, based on bin density and on cylindrical approximation, are proposed to learn these so-called mean-field functions, and ar...

We consider a deterministic optimal control problem with a maximum running cost functional, in a finite horizon context, and propose deep neural network approximations for Bellman's dynamic programming principle, corresponding also to some first-order Hamilton-Jacobi-Bellman equations. This work follows the lines of Hur\'e et al. (SIAM J. Numer. An...

We present a new neural network to approximate convex functions. This network has the particularity to approximate the function with cuts and can be easily adapted to partial convexity. We give an universal approximation theorem in the full convex case and give many numerical results proving it efficiency.

Machine learning methods for solving nonlinear partial differential equations (PDEs) are hot topical issues, and different algorithms proposed in the literature show efficient numerical approximation in high dimension. In this paper, we introduce a class of PDEs that are invariant to permutations, and called symmetric PDEs. Such problems are widesp...

In this work, we provide a general mathematical formalism to study the optimal control of an epidemic, such as the COVID-19 pandemic, via incentives to lockdown and testing. In particular, we model the interplay between the government and the population as a principal–agent problem with moral hazard, à la Cvitanić et al. (Finance Stoch 22(1):1–37,...

We propose several algorithms to solve McKean-Vlasov Forward Backward Stochastic Differential Equations (FBSDEs). Our schemes rely on the approximating power of neural networks to estimate the solution or its gradient through minimization problems. As a consequence, we obtain methods able to tackle both mean-field games and mean-field control probl...

Recently proposed numerical algorithms for solving high-dimensional nonlinear partial differential equations (PDEs) based on neural networks have shown their remarkable performance. We review some of them and study their convergence properties. The methods rely on probabilistic representation of PDEs by backward stochastic differential equations (B...

We consider the control of McKean-Vlasov dynamics (or mean-field control) with probabilistic state constraints. We rely on a level-set approach which provides a representation of the constrained problem in terms of an unconstrained one with exact penalization and running maximum or integral cost. The method is then extended to the common noise sett...

We consider the control of McKean-Vlasov dynamics (or mean-field control) with probabilistic state constraints. We rely on a level-set approach which provides a representation of the constrained problem in terms of an unconstrained one with exact penalization and running maximum or integral cost. The method is then extended to the common noise set...

This paper presents machine learning techniques and deep reinforcement learningbased algorithms for the efficient resolution of nonlinear partial differential equations and dynamic optimization problems arising in investment decisions and derivative pricing in financial engineering. We survey recent results in the literature, present new developmen...

After showing the efficiency of feedforward networks to estimate control in high dimension in the global optimization of some storages problems, we develop a modification of an algorithm based on some dynamic programming principle. We show that classical feedforward networks are not effective to estimate Bellman values for reservoir problems and we...

The problem of computing empirical cumulative distribution functions (ECDF) efficiently on large, multivariate datasets, is revisited. Computing an ECDF at one evaluation point requires O(N) operations on a dataset composed of N data points. Therefore, a direct evaluation of ECDFs at N evaluation points requires a quadratic O(N2) operations, which...

Machine learning methods for solving nonlinear partial differential equations (PDEs) are hot topical issues, and different algorithms proposed in the literature show efficient numerical approximation in high dimension. In this paper, we introduce a class of PDEs that are invariant to permutations, and called symmetric PDEs. Such problems are widesp...

We prove a rate of convergence for the $N$-particle approximation of a second-order partial diffe\-rential equation in the space of probability measures, like the Master equation or Bellman equation of mean-field control problem under common noise. The rate is of order $1/N$ for the pathwise error on the solution $v$ and of order $1/\sqrt{N}$ for...

We propose a numerical method for solving high dimensional fully nonlinear partial differential equations (PDEs). Our algorithm estimates simultaneously by backward time induction the solution and its gradient by multi-layer neural networks, while the Hessian is approximated by automatic differentiation of the gradient at previous step. This method...

This paper presents machine learning techniques and deep reinforcement learningbased algorithms for the efficient resolution of nonlinear partial differential equations and dynamic optimization problems arising in investment decisions and derivative pricing in financial engineering. We survey recent results in the literature, present new developmen...

We study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to...

We propose deep neural network algorithms to calculate efficient frontier in some Mean-Variance and Mean-CVaR portfolio optimization problems. We show that we are able to deal with such problems when both the dimension of the state and the dimension of the control are high. Adding some additional constraints, we compare different formulations and s...

We consider the control of the COVID-19 pandemic via incentives, through either stochastic SIS or SIR compartmental models. When the epidemic is ongoing, the population can reduce interactions between individuals in order to decrease the rate of transmission of the disease, and thus limit the epidemic. However, this effort comes at a cost for the p...

Discrete-time hedging produces a residual P&L, namely the tracking error. The major problem is to get valuation/hedging policies minimising this error. We evaluate the risk between trading dates through a function penalising profits and losses asymmetrically. After deriving the asymptotics from a discrete-time risk measurement for a large number of...

We develop multistep machine learning schemes for solving nonlinear partial differential equations (PDEs) in high dimension. The method is based on probabilistic representation of PDEs by backward stochastic differential equations (BSDEs) and its iterated time discretization. In the case of semilinear PDEs, our algorithm estimates simultaneously by...

We propose a numerical method for solving high dimensional fully nonlinear partial differential equations (PDEs). Our algorithm estimates simultaneously by backward time induction the solution and its gradient by multi-layer neural networks, while the Hessian is approximated by automatic differentiation of the gradient at previous step. This metho...

This paper revisits the problem of computing empirical cumulative distribution functions (ECDF) efficiently on large, multivariate datasets. Computing an ECDF at one evaluation point requires O(N) operations on a dataset composed of N data points. Therefore, a direct evaluation of ECDFs at N evaluation points requires a quadratic O(N^2) operations,...

Multi stage stochastic programs arise in many applications from engineering whenever a set of inventories or stocks has to be valued. Such is the case in seasonal storage valuation of a set of cascaded reservoir chains in hydro management. A popular method is Stochastic Dual Dynamic Programming (SDDP), especially when the dimensionality of the prob...

We study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to...

We propose new machine learning schemes for solving high dimensional nonlinear partial differential equations (PDEs). Relying on the classical backward stochastic differential equation (BSDE) representation of PDEs, our algorithms estimate simultaneously the solution and its gradient by deep neural networks. These approximations are performed at e...

We propose several algorithms to solve McKean-Vlasov Forward Backward Stochastic Differential Equations. Our schemes rely on the approximating power of neural networks to estimate the solution or its gradient through minimization problems. As a consequence, we obtain methods able to tackle both mean field games and mean field control problems in hi...

Kernel density estimation and kernel regression are powerful but computationally expensive techniques: a direct evaluation of kernel density estimates at $M$ evaluation points given $N$ input sample points requires a quadratic $\mathcal{O}(MN)$ operations, which is prohibitive for large scale problems. For this reason, approximate methods such as b...

Recent machine learning algorithms dedicated to solving semi-linear PDEs are improved by using different neural network architectures and different parameterizations. These algorithms are compared to a new one that solves a fixed point problem by using deep learning techniques. This new algorithm appears to be competitive in terms of accuracy with...

Power producers are interested in valuing their power plant production. By trading into forward contracts, we propose to reduce the contingency of the associated income considering the fixed costs and using an asymmetric risk criterion. In an asymptotic framework, we provide an optimal hedging strategy through a solution of a nonlinear partial diff...

We propose some machine-learning-based algorithms to solve hedging problems in incomplete markets. Sources of incompleteness cover illiquidity, untradable risk factors, discrete hedging dates and transaction costs. The proposed algorithms resulting strategies are compared to classical stochastic control techniques on several payoffs using a varianc...

We propose new machine learning schemes for solving high dimensional nonlinear partial differential equations (PDEs). Relying on the classical backward stochastic differential equation (BSDE) representation of PDEs, our algorithms estimate simultaneously the solution and its gradient by deep neural networks. These approximations are performed at ea...

Recent machine learning algorithms dedicated to solving semi-linear PDEs are improved by using different neural network architectures and different parameterizations. These algorithms are compared to a new one that solves a fixed point problem by using deep learning techniques. This new algorithm appears to be competitive in terms of accuracy with...

We extend a recently developed method to solve semi-linear PDEs to the case of a degenerated diffusion. Being a pure Monte Carlo method it does not suffer from the so called curse of dimensionality and it can be used to solve problems that were out of reach so far. We give some results of convergence and show numerically that it is effective. Besid...

A new method based on nesting Monte Carlo is developed to solve high-dimensional semi-linear PDEs. Depending on the type of non-linearity, different schemes are proposed and theoretically studied: variance error are given and it is shown that the bias of the schemes can be controlled. The limitation of the method is that the maturity or the Lipschi...

Discrete-time hedging produces a residual risk, i.e., the tracking error. The major problem is to get valuation/hedging policies minimizing this error. We evaluate the risk between trading dates through a function penalizing profits and losses asymmetrically. After deriving the asymptotics within a discrete-time risk measurement for a large number...

We study an islanded microgrid system designed to supply a small village with the power produced by photovoltaic panels, wind turbines and a diesel generator. A battery storage system device is used to shift power from times of high renewable production to times of high demand. We build on the mathematical model introduced in [14] and optimize the...

We study an islanded microgrid system designed to supply a small village with the power produced by photovoltaic panels, wind turbines and a diesel generator. A battery storage system device is used to shift power from times of high renewable production to times of high demand. We introduce a methodology to solve microgrid management problem using...

In Electricity markets, illiquidity, transaction costs and market price characteristics prevent managers to replicate exactly contracts. A residual risk is always present and the hedging strategy depends on a risk criterion chosen. We present an algorithm to hedge a position for a mean variance criterion taking into account the transaction cost and...

In Electricity markets, illiquidity, transaction costs and market price characteristics prevent managers to replicate exactly contracts. A residual risk is always present and the hedging strategy depends on a risk criterion chosen. We present an algorithm to hedge a position for a mean variance criterion taking into account the transaction cost and...

We extend the branching process based numerical algorithm of Bouchard et al. [3], that is dedicated to semilinear PDEs (or BSDEs) with Lipschitz nonlinearity, to the case where the nonlinearity involves the gradient of the solution. As in [3], this requires a localization procedure that uses a priori estimates on the true solution, so as to ensure...

We extend the branching process based numerical algorithm of Bouchard et al. [3], that is dedicated to semilinear PDEs (or BSDEs) with Lipschitz nonlinearity, to the case where the nonlinearity involves the gradient of the solution. As in [3], this requires a localization procedure that uses a priori estimates on the true solution, so as to ensure...

We propose a new numerical scheme for Backward Stochastic Dierential Equations based on branching processes. We approximate an arbitrary (Lipschitz) driver by local polynomials and then use a Picard iteration scheme. Each step of the Picard iteration can be solved by using a representation in terms of branching diusion systems, thus avoiding the ne...

Course at CEMRACS 2017

The STochastic OPTimization library (StOpt) aims at providing tools in C++ for solving some stochastic optimization problems encountered in finance or in the industry. A python binding is available for some C++ objects provided permitting to easily solve an optimization problem by regression. Different methods are available : - dynamic programming...

The branching methods developed are effective methods to solve some semi linear PDEs and are shown numerically to be able to solve some full non linear PDEs. These methods are however restricted to some small coefficients in the PDE and small maturities. This article shows numerically that these methods can be adapted to solve the problems with lon...

We propose an unbiased Monte Carlo method to compute E(g(XT)) where g is a Lipschitz function and X an Ito process. This approach extends the method proposed in [16] to the case where X is solution of a multidimensional stochastic differential equation with varying drift and diffusion coefficients. A variance reduction method relying on interacting...

We present different schemes for the branching method used to solve PDEs

We propose a new numerical scheme for Backward Stochastic Differential Equations based on branching processes. We approximate an arbitrary (Lipschitz) driver by local polynomials and then use a Picard iteration scheme. Each step of the Picard iteration can be solved by using a representation in terms of branching diffusion systems, thus avoiding th...

We numerically study an Asset Liability Management problem linked to the decommissioning of French nuclear power plants. We link the risk aversion of practitioners to an optimization problem. Using different price models we show that the optimal solution is linked to a de-risking management strategy similar to a concave strategy and we propose an e...

This paper examines the dividend and investment policies of a cash constrained firm, assuming a decreasing-returns-to-scale technology and adjustment costs. We extend the literature by allowing the firm to draw on a secured credit line both to hedge against cash-flow shortfalls and to invest/disinvest in a productive asset. We formulate this proble...

We provide a representation result of parabolic semi-linear PD-Es, with polynomial nonlinearity, by branching diffusion processes. We extend the classical representation for KPP equations, introduced by Skorokhod (1964), Watanabe (1965) and McKean (1975), by allowing for polynomial nonlinearity in the pair $(u, Du)$, where $u$ is the solution of th...

We consider a singular control problem with regime switching that arises in problems of optimal investment decisions of cash-constrained firms. The value function is proved to be the unique viscosity solution of the associated Hamilton-Jacobi-Bellman equation. Moreover, we give regularity properties of the value function as well as a description of...

We consider a singular control problem with regime switching that arises in problems of optimal investment decisions of cash-constrained firms. The value function is proved to be the unique viscosity solution of the associated Hamilton-Jacobi-Bellman equation. Moreover, we give regularity properties of the value function as well as a description of...