Sadegh Soudjani’s research while affiliated with Max Planck Institute for Software Systems and other places

In this paper, we propose an intent-aware Model Predictive Control (MPC) approach for the remain-well-clear (RWC) functionality of a multi-agent aircraft detect-and-avoid (DAA) system and compare its performance with the standardized Airborne Collision Avoidance System Xu (ACAS Xu). The aircraft system is modeled as a linear system for horizontal maneuvering, with advisories on the rate of turn as the control input. Both deterministic and stochastic time delays are considered to account for the lag between control guidance issuance and the response of the aircraft. The capability of the MPC scheme in producing an optimal control profile over the entire horizon is used to mitigate the impact of the delay. We compare the proposed MPC method with ACAS Xu using various evaluation metrics, including loss of DAA well-clear percentage, near mid-air collision percentage, horizontal miss distance, and additional flight distance across different encounter scenarios. It is shown that the MPC scheme achieves better evaluation metrics than ACAS Xu for both deterministic and stochastic scenarios.

In this paper, we propose a distributionally robust control synthesis for an agent with stochastic dynamics that interacts with other agents under uncertainties and constraints expressed by signal temporal logic (STL). We formulate the control synthesis as a chance-constrained program (CCP) with STL specifications that must be satisfied with high probability under all uncertainty tubes induced by the other agents. To tackle the CCP, we propose two methods based on concentration of measure (CoM) theory and conditional value at risk (CVaR) and compare the required assumptions and resulting optimizations. These approaches convert the CCP into an expectation-constrained program (ECP), which is simpler to solve than the original CCP. To estimate the expectation using a finite set of observed data, we adopt a distributionally robust optimization (DRO) approach. The underlying DRO can be approximated as a robust data-driven optimization that provides a probabilistic under-approximation to the original ECP, where the probability depends on the number of samples. Therefore, under feasibility, the original STL constraints are satisfied with two layers of designed confidence: the confidence of the chance constraint and the confidence of the approximated data-driven optimization, which depends on the number of samples. We then provide details on solving the resulting robust data-driven optimization numerically. Finally, we compare the two proposed approaches through case studies.

Designing multi-agent systems with safety constraints and uncertain dynamics is a challenging problem. This paper studies a stochastic dynamic non-cooperative game with coupling safety chance constraints. The uncertainty is assumed to satisfy a concentration of measure property. Firstly, due to the non-convexity of chance constraints, a convex under-approximation of chance constraints is given using constraints on the expectation. Then, the conditions for the existence of the stochastic generalized Nash equilibrium (SGNE) of the under-approximated game are investigated, and the relation between the $\varepsilon-$SGNE of the original game and the under-approximated one is derived. A sampling-based algorithm is proposed for the SGNE seeking of the under-approximated game that does not require knowing the distribution of the uncertainty nor the analytical computation of expectations. Finally, under some assumptions on the game's pseudo-gradient mapping, the almost sure convergence of the algorithm to SGNE is proven. A numerical study is carried out on demand-side management in microgrids with shared battery to demonstrate the applicability of the proposed scheme.

The automated synthesis of control policies for stochastic dynamical systems presents significant challenges. A standard approach is to construct a finite-state abstraction of the continuous system, typically represented as a Markov decision process (MDP). However, generating abstractions is challenging when (1) the system's dynamics are nonlinear, and/or (2) we do not have complete knowledge of the dynamics. In this work, we introduce a novel data-driven abstraction technique for nonlinear dynamical systems with additive stochastic noise that addresses both of these issues. As a key step, we use samples of the dynamics to learn the enabled actions and transition probabilities of the abstraction. We represent abstractions as MDPs with intervals of transition probabilities, known as interval MDPs (IMDPs). These abstractions enable the synthesis of control policies for the concrete nonlinear system, with probably approximately correct (PAC) guarantees on the probability of satisfying a specified control objective. Through numerical experiments, we illustrate the effectiveness and robustness of our approach in achieving reliable control under uncertainty.

Reinforcement learning (RL) is a promising method to learn optimal control policies for systems with unknown dynamics. In particular, synthesizing controllers for safety-critical systems based on high-level specifications, such as those expressed in temporal languages like linear temporal logic (LTL), presents a significant challenge in control systems research. Current RL-based methods designed for LTL tasks typically offer only asymptotic guarantees, which provide no insight into the transient performance during the learning phase. While running an RL algorithm, it is crucial to assess how close we are to achieving optimal behavior if we stop learning. In this paper, we present the first regret-free online algorithm for learning a controller that addresses the general class of LTL specifications over Markov decision processes (MDPs) with a finite set of states and actions. We begin by proposing a regret-free learning algorithm to solve infinite-horizon reach-avoid problems. For general LTL specifications, we show that the synthesis problem can be reduced to a reach-avoid problem when the graph structure is known. Additionally, we provide an algorithm for learning the graph structure, assuming knowledge of a minimum transition probability, which operates independently of the main regret-free algorithm.

This paper presents a step towards a formal controller design method for autonomous agents based on knowledge awareness to improve decision-making. Our approach is to first create an organized repository of information (a knowledge base) for autonomous agents which can be accessed and then translated into temporal specifications. Secondly, to develop a controller with formal guarantees that meets a combination of mission-specific objective and the specification from the knowledge base, we utilize an abstraction-based controller design (ABCD) approach, capable of managing both nonlinear dynamics and temporal requirements. Unlike the conventional offline ABCD approach, our method dynamically updates the controller whenever the knowledge base prompts changes in the specifications. A three-dimensional nonlinear car model navigating an urban road scenario with traffic signs and obstacles is considered for validation. Results show the effectiveness of the method in guiding the autonomous agents to the target while complying with the knowledge base and the mission-specific objective.