Moritz Diehl

• University of Freiburg

The efficient computation of parametric solution sensitivities is a key challenge in the integration of learning-enhanced methods with nonlinear model predictive control (MPC), as their availability is crucial for many learning algorithms. While approaches presented in the machine learning community are limited to convex or unconstrained formulatio...

We propose a novel approach for combining model predictive control (MPC) with reinforcement learning (RL) to reduce online computation while achieving high closed-loop tracking performance and constraint satisfaction. This method, called Policy-Enhanced Partial Tightening (PEPT), approximates the optimal value function through a Riccati recursion a...

High-level decision-making for dynamical systems often involves performance and safety specifications that are activated or deactivated depending on conditions related to the system state and commands. Such decision-making problems can be naturally formulated as optimization problems where these conditional activations are regulated by discrete var...

This paper considers Bayesian optimization (BO) for problems with known outer problem structure. In contrast to the classic BO setting, where the objective function itself is unknown and needs to be iteratively estimated from noisy observations, we analyze the case where the objective has a known outer structure - given in terms of a loss function...

In order to solve continuous-time optimal control problems, direct methods transcribe the infinite-dimensional problem to a nonlinear program (NLP) using numerical integration methods. In cases where the integration error can be manipulated by the chosen control trajectory, the transcription might produce spurious local NLP solutions as a by-produc...

Model predictive path integral (MPPI) control has recently received a lot of attention, especially in the robotics and reinforcement learning communities. This letter aims to make the MPPI control framework more accessible to the optimal control community. We present three classes of optimal control problems and their solutions by MPPI. Further, we...

The fields of MPC and RL consider two successful control techniques for Markov decision processes. Both approaches are derived from similar fundamental principles, and both are widely used in practical applications, including robotics, process control, energy systems, and autonomous driving. Despite their similarities, MPC and RL follow distinct pa...

Autonomous surface vessels are a promising building block of the future's transport sector and are investigated by research groups worldwide. This paper presents a comprehensive and systematic overview of the autonomous research vessel Solgenia including the latest investigations and recently presented methods that contributed to the fields of auto...

In this paper, we present an early software integrating Reinforcement Learning (RL) with Model Predictive Control (MPC). Our aim is to make recent theoretical contributions from the literature more accessible to both the RL and MPC communities. We combine standard software tools developed by the RL community, such as Gymnasium, stable-baselines3, o...

Trajectory optimization of highly oscillatory systems can require huge computational efforts if the horizon of the problem is much larger than the duration of a single cycle. To alleviate this effort, stroboscopic averaging methods can be used, which utilize local single‐cycle simulations of the oscillatory dynamics to then approximate the average...

We introduce a modeling framework for manipulation planning based on the formulation of the dynamics as a projected dynamical system. This method uses implicit signed distance functions and their gradients to formulate an equivalent gradient complementarity system. The optimal control problem is then solved via a direct method, discretized using fi...

Computationally efficient nonlinear model predictive control (NMPC) relies on elaborate discrete‐time optimal control problem (OCP) formulations trading off accuracy with respect to the continuous‐time problem and online computational burden. Such formulations, however, are in general not easy to implement within specialized software frameworks tai...

The transition to a carbon-neutral energy system requires massive installation of renewable energy sources and economically feasible energy storage solutions. This study addresses these challenges by optimizing the design and control strategies of an energy system that meets the heat and electricity demands of a community. The proposed system integ...

This paper presents an efficient Mixed-Integer Nonlinear Programming (MINLP) formulation for systems with discrete control inputs under dwell time constraints. By viewing such systems as a switched system, the problem is decomposed into a Sequence Optimization (SO) and a Switching Time Optimization (STO) -- the former providing the sequence of the...

Airborne wind energy (AWE) systems harvest energy from high-altitude winds using tethered wings. The flight trajectory of pumping AWE systems consists of a reel-out phase where the wing flies fast crosswind loops while slowly reeling out the tether, and a reel-in phase where the wing directly flies back to its original starting point. The number of...

In this paper we introduce an open-source software package written in C++ for efficiently finding solutions to quadratic programming problems with linear complementarity constraints. These problems arise in a wide range of applications in engineering and economics, and they are challenging to solve due to their structural violation of standard cons...

Constrained dynamical systems are systems such that, by some means, the state stays within a given set. Two such systems are the (perturbed) Moreau sweeping process and the recently proposed extended Projected Dynamical System (ePDS). We show that under certain conditions solutions to the ePDS correspond to the solutions of a dynamic complementarit...

Incorporating learning-based models, such as Gaussian processes (GPs), into model predictive control (MPC) strategies can significantly improve control performance and online adaptation capabilities for real-world applications. Still, despite recent advances in numerical optimization and real-time GP inference, its widespread application is limited...

In this brief, we propose a novel real-time numerical algorithm for solving nonlinear model predictive control (NMPC) with convex–concave constraints, which arise in various practical applications. Instead of requiring full convergence for each problem at every sampling time, the proposed algorithm, called real-time iteration sequential convex prog...

When optimizing the operation of stationary batteries using optimal control in a time-of-use application, aging costs can have a strong influence on the economic outcome. Thus, it is beneficial to consider aging costs in the controller design. First, we give a detailed explanation of how to include varying stress factors into a lab-based aging mode...

Online planning of collision-free trajectories is a fundamental task for robotics and self-driving car applications. This paper revisits collision avoidance between ellipsoidal objects using differentiable constraints. Two ellipsoids do not overlap if and only if the endpoint of the vector between the center points of the ellipsoids does not lie in...

This paper examines solution methods for mathematical programs with complementarity constraints (MPCC) obtained from the time-discretization of optimal control problems (OCPs) subject to nonsmooth dynamical systems. The MPCC theory and stationarity concepts are reviewed and summarized. The focus is on relaxation-based methods for MPCCs, which solve...

Computationally efficient nonlinear model predictive control relies on elaborate discrete-time optimal control problem (OCP) formulations trading off accuracy with respect to the continuous-time problem and associated computational burden. Such formulations, however, are in general not easy to implement within specialized software frameworks tailor...

System level synthesis enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem for linear time-varying systems. The proposed algorithm iterates between optimizing the contro...

Robot navigation around humans can be a challenging problem since human movements are hard to predict. Stochastic model predictive control (MPC) can account for such uncertainties and approximately bound the probability of a collision to take place. In this paper, to counteract the rapidly growing human motion uncertainty over time, we incorporate...

\Ac{MPC} and \ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic \ac{RL} techniques can be leveraged to improve the performance of \ac{MPC}. The \ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal...

This paper introduces Finite Elements with Switch Detection (FESD), a numerical discretization method for nonsmooth differential equations. We consider the Filippov convexification of these systems and a transformation into dynamic complementarity systems introduced by Stewart (Numer Math 58(1):299–328, 1990). FESD is based on solving nonlinear com...

District heating networks with decentralized heat production are ideally suited to include a high share of renewable energy sources for heat production in urban areas with limited space. A new concept is a prosumer-based district heating network in which some or even all buildings are equipped with decentralized building-level heat storages and hea...

System Level Synthesis (SLS) enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem. The proposed algorithm builds on a recently proposed joint optimization scheme and iter...

We investigate the suboptimality resulting from the application of nominal model predictive control (MPC) to a nonlinear discrete time stochastic system. The suboptimality is defined with respect to the corresponding stochastic optimal control problem (OCP) that minimizes the expected cost of the closed loop system. In this context, nominal MPC cor...

This work considers the problem of optimal lane changing in a structured multi-agent road environment. A novel motion planning algorithm that can capture long-horizon dependencies as well as short-horizon dynamics is presented. Pivotal to our approach is a geometric approximation of the long-horizon combinatorial transition problem which we formula...

This letter examines the question of finding feasible points to discrete-time optimal control problems. The optimization problem of finding a feasible trajectory is transcribed to an unconstrained optimal control problem. An efficient algorithm, called FP-DDP, is proposed that solves the resulting problem using Differential Dynamic Programming pres...

Mixed-integer quadratic programs (MIQPs) are a versatile way of formulating vehicle decision making (DM) and motion planning problems, where the prediction model is a hybrid dynamical system that involves both discrete and continuous decision variables. However, even the most advanced MIQP solvers can hardly account for the challenging requirements...

The Real-Time Iteration (RTI) is an online nonlinear model predictive control algorithm that performs a single Sequential Quadratic Programming (SQP) per sampling time. The algorithm is split into a preparation and a feedback phase, where the latter one performs as little computations as possible solving a single prepared quadratic program. To furt...

Die Dekarbonisierung des Gebäudebestands ist eine der größten Herausforderungen der anstehenden Energiewende. Gebäude sind für 40 Prozent des Endenergieverbrauchs und 36 Prozent der CO2-Emissionen in der EU verantwortlich [1]. Nach den ursprünglichen Regelungen des deutschen Klimaschutzgesetzes muss der Gebäudesektor in den nächsten sieben Jahren 4...

This paper examines solution methods for mathematical programs with comple-mentarity constraints (MPCC) obtained from the time-discretization of optimal control problems (OCPs) subject to nonsmooth dynamical systems. The MPCC theory and stationarity concepts are reviewed and summarized. The focus is on relaxation-based methods for MPCCs, which solv...

This paper introduces a novel time-freezing reformulation and numerical methods for optimal control of complementarity Lagrangian systems (CLS) with state jumps. We cover the difficult case when the system evolves on the boundary of the dynamic’s feasible set after the state jump. In nonsmooth mechanics, this corresponds to inelastic impacts. The m...

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integr...

This paper develops high-accuracy methods for numerically solving optimal control problems subject to nonsmooth differential equations with set-valued step functions. A notable subclass of these systems are Filippov systems. The set-valued step functions are here written as the solution map of a linear program. Using the optimality conditions of th...

This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, th...

VV4RTOS is an activity supported by the European Space Agency aimed at the development and validation of a framework for the verification and validation of spacecraft guidance, navigation, and control (GNC) systems based on embedded optimisation, tailored to handle different layers of abstraction, from guidance and control (G&C) requirements down t...

In this article, we propose a novel time-energy optimal control approach with applications in cooperative eco-driving of connected and automated vehicles (CAVs) in urban traffic networks. Safely approaching and departing signalized intersections requires the satisfaction of both spatial equality constraints determined by intersection locations and...

For the fast approximate solution of Mixed-Integer Non-Linear Programs (MINLPs) arising in the context of Mixed-Integer Optimal Control Problems (MIOCPs) a decomposition algorithm exists that solves a sequence of three comparatively less hard subproblems to determine an approximate MINLP solution. In this work, we propose a problem formulation for...

Dynamic soaring for UAVs is a flight technique that enables continuous, powerless periodic flight patterns in the presence of a wind gradient. However, sufficiently large wind gradients are uncommon over land, while at offshore locations the largest wind gradients are located close to the ocean surface, thereby limiting the scope of practical appli...

The Finite Elements with Switch Detection (FESD) is a high-accuracy method for the numerical simulation and solution of optimal control problems subject to discontinuous ODEs. In this article, we extend the FESD method [35] to the dynamic equations of multiple rigid bodies that exhibit state jumps due to impacts and Coulomb friction. This new metho...

Direct collocation methods are powerful tools to solve trajectory optimization problems in robotics. While their resulting trajectories tend to be dynamically accurate, they may also present large kinematic errors in the case of constrained mechanical systems, i.e., those whose state coordinates are subject to holonomic or nonholonomic constraints,...

This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-functio...

This paper extends the Finite Elements with Switch Detection (FESD) method [Nurkanovi\'c et al., 2022] to optimal control problems with nonsmooth systems involving set-valued step functions. Logical relations and common nonsmooth functions within a dynamical system can be expressed using linear and nonlinear expressions of the components of the ste...

We revisit three classical numerical methods for solving unconstrained optimal control problems - Multiple Shooting (MS), Single Shooting (SS), and Differential Dynamic Programming (DDP) - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or more...

We consider a stage-varying nonlinear model predictive control (NMPC) formulation and provide a stability result for the corresponding closed-loop system under the assumption that cost and constraints are progressively tightening. We illustrate the generality of the stage-varying formulation pointing out various approaches proposed in the literatur...

Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we present a self-supervised learning a...

In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesia...

Flexible robots may overcome the industry's major problems: safe human-robot collaboration and increased load-to-mass ratio. However, oscillations and high dimensional state space complicate the control of flexible robots. This work investigates nonlinear model predictive control (NMPC) of flexible robots -- for simultaneous planning and control --...