Ruggero Carli

• University of Padua

This short paper describes our proposed solution for the third edition of the "AI Olympics with RealAIGym" competition, held at ICRA 2025. We employed Monte-Carlo Probabilistic Inference for Learning Control (MC-PILCO), an MBRL algorithm recognized for its exceptional data efficiency across various low-dimensional robotic tasks, including cart-pole...

Digital twins for power electronics require accurate power losses whose direct measurements are often impractical or impossible in real-world applications. This paper presents a novel hybrid framework that combines physics-based thermal modeling with data-driven techniques to identify and correct power losses accurately using only temperature measu...

Recent advancements in Large Language Models (LLMs) and Visual Language Models (VLMs) have significantly impacted robotics, enabling high-level semantic motion planning applications. Reinforcement Learning (RL), a complementary paradigm, enables agents to autonomously optimize complex behaviors through interaction and reward signals. However, desig...

In this paper, we consider a network of agents that jointly aim to minimise the sum of local functions subject to coupling constraints involving all local variables. To solve this problem, we propose a novel solution based on a primal-dual architecture. The algorithm is derived starting from an alternative definition of the Lagrangian function, and...

Pick-and-place (PnP) operations, featuring object grasping and trajectory planning, are fundamental in industrial robotics applications. Despite many advancements in the field, PnP is limited by workspace constraints, reducing flexibility. Pick-and-throw (PnT) is a promising alternative where the robot throws objects to target locations, leveraging...

In this paper, we propose a novel distributed algorithm for consensus optimization over networks and a robust extension tailored to deal with asynchronous agents and packet losses. Indeed, to robustly achieve dynamic consensus on the solution estimates and the global descent direction, we embed in our algorithms a distributed implementation of the...

Behavior-based Driver Identification is an emerging technology that recognizes drivers based on their unique driving behaviors, offering important applications such as vehicle theft prevention and personalized driving experiences. However, most studies fail to account for the real-world challenges of deploying Deep Learning models within vehicles....

Deep Reinforcement Learning is gaining increasing attention thanks to its capability to learn complex policies in high-dimensional settings. Recent advancements utilize a dual-network architecture to learn optimal policies through the Q-learning algorithm. However, this approach has notable drawbacks, such as an overestimation bias that can disrupt...

This paper proposes a framework to study the convergence of stochastic optimization and learning algorithms. The framework is modeled over the different challenges that these algorithms pose, such as (i) the presence of random additive errors ( e.g. due to stochastic gradients), and (ii) random coordinate updates ( e.g. due to asynchrony in distr...

Continuous control Deep Reinforcement Learning (RL) approaches are known to suffer from estimation biases, leading to suboptimal policies. This paper introduces innovative methods in RL, focus-ing on addressing and exploiting estimation biases in Actor-Critic methods for continuous control tasks, using Deep Double Q-Learning. We design a Bias Explo...

This paper investigates the existence of a separation principle between model identification and control design in the context of model predictive control. First, we elucidate that the separation principle holds asymptotically in the number of data in a Fisherian setting, and universally in a Bayesian setting. Then, by formulating model predictive...

This report describes our proposed solution for the second AI Olympics competition held at IROS 2024. Our solution is based on a recent Model-Based Reinforcement Learning algorithm named MC-PILCO. Besides briefly reviewing the algorithm, we discuss the most critical aspects of the MC-PILCO implementation in the tasks at hand.

As artificial intelligence gains new capabilities, it becomes important to evaluate it on real-world tasks. In particular, the fields of robotics and reinforcement learning (RL) are lacking in standardized benchmarking tasks on real hardware. To facilitate reproducibility and stimulate algorithmic advancements, we held an AI Olympics competition at...

Multi-agent systems are increasingly widespread in a range of application domains, with optimization and learning underpinning many of the tasks that arise in this context. Different approaches have been proposed to enable the cooperative solution of these optimization and learning problems, including first- and second-order methods, and dual (or L...

Learning the inverse dynamics of robots directly from data, adopting a black-box approach, is interesting for several real-world scenarios where limited knowledge about the system is available. In this paper, we propose a black-box model based on Gaussian Process (GP) Regression for the identification of the inverse dynamics of robotic manipulators...

The commonly adopted route to control a dynamic system and make it follow the desired behavior consists of two steps. First, a model of the system is learned from input–output data, a task known as system identification in the engineering literature. Here, an important point is not only to derive a nominal model of the plant but also confidence b...

In this paper, we propose to estimate the forward dynamics equations of mechanical systems by learning a model of the inverse dynamics and estimating individual dynamics components from it. We revisit the classical formulation of rigid body dynamics in order to extrapolate the physical dynamical components, such as inertial and gravitational compon...

In this paper, we propose an inverse-kinematics controller for a class of multi-robot systems in the scenario of sampled communication. The goal is to make a group of robots perform trajectory tracking in a coordinated way when the sampling time of communications is much larger than the sampling time of low-level controllers, disrupting theoretical...

italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective: Effective dosing of anticoagulants aims to prevent blood clot formation while avoiding hemorrhages. This complex task is challenged by several disturbing factors and drug-effect uncertainties, requesting frequent monitoring and adjustment. B...

We propose a Model-Based Reinforcement Learning (MBRL) algorithm named VF-MC-PILCO, specifically designed for application to mechanical systems where velocities cannot be directly measured. This circumstance, if not adequately considered, can compromise the success of MBRL approaches. To cope with this problem, we define a velocity-free state formu...

In this paper we propose a model-based approach to the design of online optimization algorithms, with the goal of improving the tracking of the solution trajectory (trajectories) w.r.t. state-of-the-art methods. We focus first on quadratic problems with a time-varying linear term, and use digital control tools (a robust internal model principle) to...

In this article, we present a model-based reinforcement learning (MBRL) algorithm named Monte Carlo probabilistic inference for learning control (MC-PILCO). This algorithm relies on Gaussian processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the c...

In this paper we propose a model-based approach to the design of online optimization algorithms, with the goal of improving the tracking of the solution trajectory (trajectories) w.r.t. state-of-the-art methods. We focus first on quadratic problems with a time-varying linear term, and use digital control tools to propose a novel online algorithm th...

This paper deals with distributed consensus optimization problems in which a connected network of agents aims at minimizing the sum of local cost functions over a common variable. In particular, we focus on the ADMM-based algorithm proposed by Shi et al. in Shi et al. (2014) that, under the assumption that local cost functions are strongly convex w...

In this paper, we propose a feedback control approach for solving optimal power flow (OPF) problems in power distribution networks (DNs) based on a projected gradient scheme, where the cost is given by the sum of functions of local power injections. This approach does not require detailed knowledge of the grid model and enables real-time tracking o...

This paper proposes a fast and accurate solver for implicit Continuous Set Model Predictive Control for the current control loop of synchronous motor drives with input constraints, allowing for reaching the maximum voltage feasible set. The related control problem requires an iterative solver to find the optimal solution. The real-time certificatio...

In this paper we propose and analyze a distributed algorithm for dynamic average consensus that is derived in the context of online optimization and based on the alternating direction method of multipliers (ADMM). In particular, we are interested in a scenario in which the multi-agent system is subject to the following challenges: (i) asynchronous...

The energy sustainability of multi-access edge computing (MEC) platforms is here addressed by developing Energy-Aware job Scheduling at the Edge (EASE), a computing resource scheduler for edge servers co-powered by renewable energy resources and the power grid. The scenario under study involves the optimal allocation and migration of time-sensitive...

In this work we consider the problem of mobile robots that need to manipulate/transport an object via cables or robotic arms. We consider the scenario where the number of manipulating robots is redundant, i.e. a desired object configuration can be obtained by different configurations of the robots. The objective of this work is to show that communi...

In this paper, we propose an inverse-kinematics controller for a class of multi-robot systems in the scenario of sampled communication. The goal is to make a group of robots perform trajectory tracking in a coordinated way when the sampling time of communications is much larger than the sampling time of low-level controllers, disrupting theoretical...

The energy sustainability of multi-access edge computing (MEC) platforms is addressed in this paper, by developing Energy-Aware job Scheduling at the Edge (EASE), a computing resource scheduler for edge servers co-powered by renewable energy resources and the power grid. The scenario under study involves the optimal allocation and migration of time...

div>This paper proposes an efficient QP solver for electric motors.</div

div>This paper proposes a fast and accurate solver for implicit Continuous Set Model Predictive Control for the current control loop of synchronous motor drives with input constraints, allowing for reaching the maximum voltage feasible set. The related control problem requires an iterative solver to find the optimal solution. The real-time certific...

Efficient management of energy resources is crucial in smart buildings. In this work, model predictive control (MPC) is used to minimize the economic costs of prosumers equipped with production units, energy storage systems, and electric vehicles. To this purpose, the predictive control manages the available energy resources by exploiting future in...

div>This paper proposes an effective solver for implicit Continuous Set Model Predictive Control for the current control loop of synchronous motor drives with input constraints, allowing for reaching the maximum voltage feasible set. The related quadratic programming problem requires an iterative solver to find the optimal solution. The real-time c...

div>This paper proposes an effective solver for implicit Continuous Set Model Predictive Control for the current control loop of synchronous motor drives with input constraints, allowing for reaching the maximum voltage feasible set. The related quadratic programming problem requires an iterative solver to find the optimal solution. The real-time c...

Volterra series approximate a broad range of nonlinear systems. Their identification is challenging due to the curse of dimensionality: the number of model parameters grows exponentially with the complexity of the input–output response. This fact limits the applicability of such models and has stimulated recently much research on regularized soluti...

This paper provides a unified stochastic operator framework to analyze the convergence of iterative optimization algorithms for both static problems and online optimization and learning. In particular, the framework is well suited for algorithms that are implemented in an inexact or stochastic fashion because of (i) stochastic errors emerging in al...

In this paper, we consider the use of black-box Gaussian process (GP) models for trajectory tracking control based on feedback linearization, in the context of mechanical systems. We considered two strategies. The first computes the control input directly by using the GP model, whereas the second computes the input after estimating the individual c...

In this paper, we present a Model-Based Reinforcement Learning algorithm named Monte Carlo Probabilistic Inference for Learning COntrol (MC-PILCO). The algorithm relies on Gaussian Processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the choice of the...

In this paper, we propose a Model-Based Reinforcement Learning (MBRL) algorithm for Partially Measurable Systems (PMS), i.e., systems where the state can not be directly measured, but must be estimated through proper state observers. The proposed algorithm, named Monte Carlo Probabilistic Inference for Learning COntrol for Partially Measurable Syst...

Robotics systems are becoming more and more autonomous and reconfigurable. In this context, the design of algorithms capable of deriving kinematics and dynamics models directly from data could be particularly useful. In this article, we present an algorithm that learns a forward kinematics model of a robot starting from a time series of visual obse...

In this paper, we propose, discuss, and validate an online Nonlinear Model Predictive Control (NMPC) method for multi-rotor aerial systems with arbitrarily positioned and oriented rotors which simultaneously addresses the local ref- erence trajectory planning and tracking problems. This work brings into question some common modeling and control des...

State Estimation (SE) is one of the essential tasks to monitor and control the smart power grid. This paper presents a method to estimate the state variables combining the measurement of power demand at each bus with the data collected from a limited number of Phasor Measurement Units (PMUs). Although PMU data are usually assumed to be perfectly sy...

This paper introduces a dual-regularized ADMM approach to distributed, time-varying optimization. The proposed algorithm is designed in a prediction-correction framework, in which the computing nodes predict the future local costs based on past observations, and exploit this information to solve the time-varying problem more effectively. In order t...

In this work we focus on the problem of minimizing the sum of convex cost functions in a distributed fashion over a peer-to-peer network. In particular, we are interested in the case in which communications between nodes are prone to failures and the agents are not synchronized among themselves. We address the problem proposing a modified version o...

The diffusion of distributed energy resources in distribution networks requires new approaches to exploit the users’ capabilities of providing ancillary services. Of particular interest will be the coordination of microgrids operating as an aggregate of demand and supply units. This work reports a model predictive control (MPC) application in micro...

We propose a unified framework for time-varying convex optimization based on the prediction-correction paradigm, both in the primal and dual spaces. In this framework, a continuously varying optimization problem is sampled at fixed intervals, and each problem is approximately solved with a primal or dual correction step. The solution method is warm...

Volterra series is especially useful for nonlinear system identification, also thanks to its capability to approximate a broad range of input-output maps. However, its identification from a finite set of data is hard, due to the curse of dimensionality. Recent approaches have shown how regularization strategies can be useful for this task. In this...

In this paper, we propose a suboptimal distributed LQR control method, applicable to systems coupled through both physical interconnections and the quadratic cost to be minimized. Thanks to a novel suboptimal but distributed cost-to-go matrix update that enforces block-diagonality, the suboptimal LQR gain matrix is structured, making the overall co...

We consider the problem of PMU-based state estimation combining information coming from ubiquitous power demand time series and only a limited number of PMUs. Conversely to recent literature in which synchrophasor devices are often assumed perfectly synchronized with the Coordinated Universal Time (UTC), we explicitly consider the presence of time-...

In this paper we propose, test, and validate an online Nonlinear Model Predictive Control (NMPC) method applied to multi-rotor aerial systems with arbitrarily positioned and oriented rotors. This work brings into question some common modeling and control design choices that are typically adopted in order to guarantee robustness and reliability but...

We address the problem of multi-agent partition-based convex optimization which arises, for example, in robot localization problems and in regional state estimation in smart grids. More specifically, the global cost function is the sum of locally coupled cost functions that depend only on each agent variables and their neighbors’ variables. Inspire...

In this paper, we introduce a novel data-driven inverse dynamics estimator based on Gaussian Process Regression. Driven by the fact that the inverse dynamics can be described as a polynomial function on a suitable input space, we propose the use of a novel kernel, called $Geometrically\ Inspired\ Polynomial\ Kernel$ (GIP). The resulting estimator...

This work presents the application of model predic-tive control (MPC) for the energy management of smart buildings in microgrids. It is shown that by means of the described MPC formulation the power exchange at the point of connection of the building can be made close to a given power reference, typically available in microgrid contexts, and, there...

We study the solution of a time-varying optimization problem which is observed, i.e., it is known, only intermittently. We propose three approaches based on the prediction-correction scheme for solving this problem by exploiting splitting methods. We present convergence results in mean to a bounded asymptotical error, and showcase them in a numeric...

This paper proposes a technique to control distributed energy resources in low-voltage microgrids aiming at (i) allowing power flow control at the point of connection with the upstream grid, (ii) keeping voltage profiles within the operational limits. The first feature is crucial in smart low-voltage power systems. In fact, it enables both demand-r...

In this work, we propose a feedback-based motion planner for a class of multi-agent manipulation systems with a sparse kinematics structure. In other words, the agents are coupled together only by the transported object. The goal is to steer the load into a desired configuration. We suppose that a global motion planner generates a sequence of desir...

We consider the problem of regulating the voltage profile of a power distribution grid by controlling the reactive power injection of distributed microgenerators. We define a very general class of purely local feedback controllers in which reactive power injection is adjusted based on the local voltage measurements. This class includes most of the...

Volterra series are especially useful for nonlinear system identification, also thanks to their capability to approximate a broad range of input-output maps. However, their identification from a finite set of data is hard, due to the curse of dimensionality. Recent approaches have shown how regularized kernel-based methods can be useful for this ta...

In this paper we introduce a novel data driven inverse dynamics estimator based on Gaussian Process Regression. Driven by the fact that the inverse dynamics can be described as a polynomial function on a suitable input space, we propose the use of a polynomial kernel, based on a set of parameters which is different from the one typically considered...

In this paper we present a novel algorithm to solve the robot kinematic structure identification problem. Given a time series of data, typically obtained processing a set of visual observations, the proposed approach identifies the ordered sequence of links associated to the kinematic chain, the joint type interconnecting each couple of consecutive...

We address the solution of time-varying optimization problems characterized by the sum of a time-varying strongly convex function and a time-invariant nonsmooth convex function. We design an online algorithmic framework based on prediction-correction, which employs splitting methods to solve the sampled instances of the time-varying problem. We des...

We present an algorithm for minimizing the sum of a strongly convex time-varying function with a time-invariant, convex, and nonsmooth function. The proposed algorithm employs the prediction-correction scheme alongside the forward-backward envelope, and we are able to prove the convergence of the solutions to a neighborhood of the optimizer that de...

In this work we focus on the problem of minimizing the sum of convex cost functions in a distributed fashion over a peer-to-peer network. In particular we are interested in the case in which communications between nodes are lossy and the agents are not synchronized among themselves. We address this problem by proposing a modified version of the rel...

We propose a non-linear model predictive scheme for planning fuel efficient maneuvers of small spacecrafts that shall rendezvous space debris. The paper addresses the specific issues of potential limited on-board computational capabilities and low-thrust actuators in the chasing spacecraft, and solves them by using a novel MatLab-based toolbox for...

In this paper, we propose a novel distributed fault detection method to monitor the state of a - possibly large-scale - linear system, partitioned into interconnected subsystems. The approach hinges on the definition of a partition-based distributed Luenberger-like estimator, based on the local model of the subsystems and that takes into account th...

In this work we study the problem of unconstrained convex- optimization in a fully distributed multi-agent setting, which includes asynchronous computation and lossy communication. In particular, we extend a recently proposed algorithm named Newton-Raphson Consensus by integrating it with a broadcast-based average consensus algorithm which is robus...

In this paper we propose a distributed implementation of the relaxed Alternating Direction Method of Multipliers algorithm (R-ADMM) for optimization of a separable convex cost function, whose terms are stored by a set of interacting agents, one for each agent. Specifically the local cost stored by each node is in general a function of both the stat...

In this work we address the problem of distributed optimization of the sum of convex cost functions in the context of multi-agent systems over lossy communication networks. Building upon operator theory, first, we derive an ADMM-like algorithm that we refer to as relaxed ADMM (R-ADMM) via a generalized Peaceman-Rachford Splitting operator on the La...

In this work, we present the equivalent of many theorems available for continuous time systems. In particular, the theory is applied to Averaging Theory and Separation of time scales. In particular the proofs developed for Averaging Theory and Separation of time scales departs from those typically used in continuous time systems that are based on t...

In this paper we consider a novel partitioned framework for distributed optimization in peer-to-peer networks. In several important applications the agents of a network have to solve an optimization problem with two key features: (i) the dimension of the decision variable depends on the network size, and (ii) cost function and constraints have a sp...

Wireless power transfer networks (WPTNs) are composed of dedicated energy transmitters that charge energy receivers via radio frequency waves. A safe-charging WPTN should keep electromagnetic radiation below pre-determined limits meanwhile maximizing the transmitted power. In this article we consider this requirement as an optimization problem: the...

A class of abstract aerial robotic systems is introduced, the laterally bounded force vehicles, in which most of the control authority is expressed along a principal thrust direction, while along the lateral directions a (smaller and possibly null) force may be exploited to achieve full-pose tracking. This class approximates platforms endowed with...

There is a common belief that the ADMM, a popular algorithm employed for distributed convex optimization over graphs, is faster than another distributed algorithm typically referred as the Average Consensus. This belief is based on the observation that the ratio of the number of iterations necessary to achieve a desired error with respect to the op...

We consider the problem of letting a network of mobile agents distributedly track and maintain a formation while using communication schemes that are asynchronous, broadcasts based, and prone to packet losses. To this purpose we revisit and modify an existing distributed optimization algorithm that corresponds to a distributed version of the Newton...