Matthias Althoff

Technical University of Munich | TUM · Chair of Robotics and Embedded Systems

In recent years, different approaches for motion planning of autonomous vehicles have been proposed that can handle complex traffic situations. However, these approaches are rarely compared on the same set of benchmarks. To address this issue, we present the results of a large-scale motion planning competition for autonomous vehicles based on the C...

Reinforcement learning (RL) is not yet competitive for many cyber-physical systems, such as robotics, process automation, and power systems, as training on a system with physical components cannot be accelerated, and simulation models do not exist or suffer from a large simulation-to-reality gap. During the long training time, expensive equipment c...

Motion planners are essential for the safe operation of automated vehicles across various scenarios. However, no motion planning algorithm has achieved perfection in the literature, and improving its performance is often time-consuming and labor-intensive. To tackle the aforementioned issues, we present ${\mathtt {DrPlanner}}$ , the first framewo...

Calculating the inverse kinematics (IK) is fundamental for motion planning in robotics. Compared to numerical or learning-based approaches, analytical IK provides higher efficiency and accuracy. However, existing analytical approaches require manual intervention, are ill-conditioned, or rely on time-consuming symbolic manipulation. In this paper, w...

Reinforcement learning often uses neural networks to solve complex control tasks. However, neural networks are sensitive to input perturbations, which makes their deployment in safety-critical environments challenging. This work lifts recent results from formally verifying neural networks against such disturbances to reinforcement learning in conti...

Adjusting robot behavior to human preferences can require intensive human feedback, preventing quick adaptation to new users and changing circumstances. Moreover, current approaches typically treat user preferences as a reward, which requires a manual balance between task success and user satisfaction. To integrate new user preferences in a zero-sh...

Automated vehicles must comply explicitly with specifications, including traffic-based and handcrafted rules, in order for them to safely and effectively participate in mixed traffic. In addition to driving individually, there are many traffic situations in which cooperation between vehicles maximizes their collective benefits, including preventing...

Hybrid systems are often safety-critical and at the same time difficult to formally verify due to their mixed discrete and continuous behavior. To address this issue, we propose a novel incremental verification algorithm for hybrid systems based on online monitoring techniques and reachability analysis. To this end, we develop a four-valued semanti...

Formal verification techniques play a pivotal role in ensuring the safety of complex cyber-physical systems. To transfer model-based verification results to the real world, we require that the measurements of the target system lie in the set of reachable outputs of the corresponding model, a property we refer to as reachset conformance. This paper...

We introduce new techniques to check whether a zonotope is contained in another zonotope. This fundamental problem in control theory has many applications, such as the verification of invariant sets, formal verification of controllers, and fault detection. Our first method uses a search-based vertex enumeration to quickly and efficiently check cont...

Continuous action spaces in reinforcement learning (RL) are commonly defined as interval sets. While intervals usually reflect the action boundaries for tasks well, they can be challenging for learning because the typically large global action space leads to frequent exploration of irrelevant actions. Yet, little task knowledge can be sufficient to...

Compliance with the rules of the road is crucial for the safe operation of autonomous vehicles. Previous work has shown that one can expedite rule-compliant motion planning by constraining the search space based on the reachable states of the vehicle. We propose an algorithm to overapproximate the states that a vehicle can reach while adhering to a...

Motion planning algorithms should be tested on a large, diverse, and realistic set of scenarios before deploying them in real vehicles. However, existing 3D simulators usually focus on perception and end-to-end learning, lacking specific interfaces for motion planning. We present an interface for the CARLA simulator focusing on motion planning, e.g...

Validating motion planning algorithms for autonomous vehicles on a real system is essential to improve their safety in the real world. Open-source initiatives, such as Autoware, provide a deployable software stack for real vehicles. However, such driving stacks have a high entry barrier, so that integrating new algorithms is tedious. Especially new...

Formal verification of neural networks is a challenging problem due to the complexity and nonlinearity of neural networks. It has been shown that polynomial zonotopes can tightly enclose the output set of a neural network. Unfortunately, the tight enclosure comes with additional complexity in the set representation, thus, rendering subsequent opera...

By monitoring the set of reachable outputs, safety can be verified. However, to compute the reachable set of real-world systems, we require models that are able to produce all possible system behaviors. These kinds of models are called reachset-conformant, and their identification is a promising new research direction. While many existing reachset-...

For safe operation, autonomous vehicles have to obey traffic rules that are set forth in legal documents formulated in natural language. Temporal logic is a suitable concept to formalize such traffic rules. Still, temporal logic rules often result in constraints that are hard to solve using optimization-based motion planners. Reinforcement learning...

Reachability analysis is a formal method that rigorously proves whether a dynamical system can reach certain states. Inner approximations of the exact reachable set contain only states that are definitely reachable and are therefore used to falsify specifications. While the majority of state-of-the-art approaches for nonlinear systems obtain an inn...

To meet regulatory and safety standards, a significant amount of ground-truth data is necessary for data-driven validation of perception algorithms. To overcome the high costs and extensive manual efforts required by existing methods, we propose an automatic offline approach to generate ground-truth data for 3D object poses. Our method solely uses...

Equipping any controller with formal safety guarantees can be achieved by using safety filters. These filters modify the desired control input in the least restrictive way to guarantee safety. However, it is an unresolved issue to construct scalable safety filters without assuming the availability of the disturbance set. We address this issue by pr...

Reachability analysis is a formal method to guarantee safety of dynamical systems under the influence of uncertainties. A substantial bottleneck of all reachability algorithms is the necessity to adequately tune specific algorithm parameters, such as the time step size, which requires expert knowledge. In this work, we solve this issue with a fully...

The robustness of signal temporal logic not only assesses whether a signal adheres to a specification but also provides a measure of how much a formula is fulfilled or violated. The calculation of robustness is based on evaluating the robustness of underlying predicates. However, the robustness of predicates is usually defined in a model-free way,...

Ensuring the safety of reinforcement learning (RL) algorithms is crucial to unlock their potential for many real-world tasks. However, vanilla RL and most safe RL approaches do not guarantee safety. In recent years, several methods have been proposed to provide hard safety guarantees for RL, which is essential for applications where unsafe actions...

Autonomous vehicles benefit from correct maps to participate in traffic safely, but often maps are not verified before their usage. We address this problem by providing an approach to verify and repair maps automatically based on a formalization of map specifications in higher-order logic. Unlike previous work, we provide a collection of map specif...

Robots are used increasingly often in safety-critical scenarios, such as robotic surgery or human–robot interaction. To ensure stringent performance criteria, formal controller synthesis is a promising direction to guarantee that robots behave as desired. However, formally ensured properties only transfer to the real robot when the model is appropr...

Verifying the correct behavior of robots in contact tasks is challenging due to model uncertainties associated with contacts. Standard methods for testing often fall short since all (uncountable many) solutions cannot be obtained. Instead, we propose to formally and efficiently verify robot behaviors in contact tasks using reachability analysis, wh...

We present a novel approach to efficiently compute tight non-convex enclosures of the image through neural networks with ReLU, sigmoid, or hyperbolic tangent activation functions. In particular, we abstract the input-output relation of each neuron by a polynomial approximation, which is evaluated in a set-based manner using polynomial zonotopes. Wh...

Scenarios are a crucial element for developing, testing, and verifying autonomous driving systems. However, open-source scenarios are often formulated using different terminologies. This limits their usage across different applications as many scenario representation formats are not directly compatible with each other. To address this problem, we p...

The formal verification of neural networks is essential for their application in safety-critical environments. However, the set-based verification of neural networks using linear approximations often obtains overly conservative results, while nonlinear approximations quickly become computationally infeasible in deep neural networks. We address this...

We introduce constrained polynomial zonotopes, a novel non-convex set representation that is closed under linear map, Minkowski sum, Cartesian product, convex hull, intersection, union, and quadratic as well as higher-order maps. We show that the computational complexity of the above-mentioned set operations for constrained polynomial zonotopes is...

Formal verification of neural networks is essential before their deployment in safety-critical settings. However, existing methods for formally verifying neural networks are not yet scalable enough to handle practical problems that involve a large number of neurons. In this work, we propose a novel approach to address this challenge: A conservative...

The robustness of signal temporal logic not only assesses whether a signal adheres to a specification but also provides a measure of how much a formula is fulfilled or violated. The calculation of robustness is based on evaluating the robustness of underlying predicates. However, the robustness of predicates is usually defined in a model-free way,...

Criticality measures are essential for autonomous vehicles to capture the complexity of the surrounding environment, trigger emergency maneuvers, and verify safety. However, there is currently no publicly available toolbox that allows researchers to use or evaluate a large number of criticality measures on arbitrary traffic scenarios. To address th...

We propose two distributed set-based observers using strip-based and set-propagation approaches for linear discrete-time dynamical systems with bounded modeling and measurement uncertainties. Both algorithms utilize a set-based diffusion step, which decreases the estimation errors and the size of estimated sets, and can be seen as a lightweight app...

While reinforcement learning produces very promising results for many applications, its main disadvantage is the lack of safety guarantees, which prevents its use in safety-critical systems. In this work, we address this issue by a safety shield for nonlinear continuous systems that solve reach-avoid tasks. Our safety shield prevents applying poten...

Deep Reinforcement Learning (RL) has shown promise in addressing complex robotic challenges. In real-world applications, RL is often accompanied by failsafe controllers as a last resort to avoid catastrophic events. While necessary for safety, these interventions can result in undesirable behaviors, such as abrupt braking or aggressive steering. Th...

Verifying the safety of autonomous vehicles is one of the major challenges towards their deployment on public roads due to the vast number of possible situations that can occur in traffic. Scenario-based testing has been proposed to reduce the number of required tests using catalogs of abstractly defined scenarios. However, when specifying test sce...

The set-based estimation has gained a lot of attention due to its ability to guarantee state enclosures for safety-critical systems. However, collecting measurements from distributed sensors often requires outsourcing the set-based operations to an aggregator node, raising many privacy concerns. To address this problem, we present set-based estimat...

Ensuring robust constraint satisfaction for an infinite time horizon is a challenging, yet crucial task when deploying safety-critical systems. We address this issue by synthesizing robust control invariant sets of perturbed nonlinear sampled-data systems. This task can be encoded as a non-convex program that we approximate by a tailored, computati...

We present a novel, correct-by-construction control approach for disturbed, nonlinear systems with continuous state feedback under state and input constraints. For the first time, we jointly synthesize a feedforward and feedback controller by solving a single non-convex, continuously differentiable approximation of the original synthesis problem, w...

Forecasting the motion of others in shared spaces is a key for intelligent agents to operate safely and smoothly. We present an approach for probabilistic prediction of pedestrian motion incorporating various context cues. Our approach is based on goal-oriented prediction, yielding interpretable results for the predicted pedestrian intention, even...

Vanilla Reinforcement Learning (RL) can efficiently solve complex tasks but does not provide any guarantees on system behavior. To bridge this gap, we propose a three-step safe RL procedure for continuous action spaces that provides probabilistic guarantees with respect to temporal logic specifications. First, our approach probabilistically verifie...

In many reachability algorithms for nonlinear ordinary differential equations (ODEs), the tightness of the computed reachable sets mainly depends on abstraction errors and the choice of the set representation. One has to mitigate the resulting wrapping effects by suitable tuning of internally-used algorithm parameters since there exists no wrapping...

While reinforcement learning produces very promising results for many applications, its main disadvantage is the lack of safety guarantees, which prevents its use in safety-critical systems. In this work, we address this issue by a safety shield for nonlinear continuous systems that solve reach-avoid tasks. Our safety shield prevents applying poten...

Autonomous vessels can increase safety and reduce emissions compared to human-operated vessels. One important task for autonomous vessels is motion planning. Currently, there are no benchmarks for autonomous vessels to compare different motion planning methods. Thus, we introduce composable benchmarks for motion planning on oceans (CommonOcean), wh...

Deep reinforcement learning (RL) has been widely applied to motion planning problems of autonomous vehicles in urban traffic. However, traditional deep RL algorithms cannot ensure safe trajectories throughout training and deployment. We propose a provably safe RL algorithm for urban autonomous driving to address this. We add a novel safety layer to...

Model-based verification uses a model to reason about the correctness of a real system. This requires the model and the system to be conformant, such that verification results on the model can be transfered to the real system. Especially for hybrid systems, which combine discrete and continuous behavior, defining and checking conformance is a diffi...

When planning motions for autonomous vehicles, traffic rules must be obeyed to ensure safety and reject liability claims. However, present solutions do not scale well with the complexity of traffic rules or even consider them. To solve this problem, we propose a scalable approach based on constrained policy optimization to improve traffic rule comp...

In recent years, reachability analysis has gained considerable popularity in motion planning and safeguarding of automated vehicles (AVs). While existing tools for reachability analysis mainly focus on general-purpose algorithms for formal verification of dynamical systems, a toolbox tailored to AV-specific applications is not yet available. In thi...

We present a novel approach to efficiently compute tight non-convex enclosures of the image through neural networks with ReLU, sigmoid, or hyperbolic tangent activation functions. In particular, we abstract the input-output relation of each neuron by a polynomial approximation, which is evaluated in a set-based manner using polynomial zonotopes. Ou...

Intersections are difficult to navigate for both human drivers and autonomous vehicles because several diverse traffic rules must be considered. In addition, current traffic rules are ambiguous and cannot be applied directly by autonomous vehicles. Therefore, national traffic rules must be concretized and formalized so that they are machine-interpr...

Ensuring safety of reinforcement learning (RL) algorithms is crucial for many real-world tasks. However, vanilla RL does not guarantee safety for an agent. In recent years, several methods have been proposed to provide safety guarantees for RL. To the best of our knowledge, there is no comprehensive comparison of these provably safe RL methods. We...

Deep reinforcement learning (RL) has shown promising results in the motion planning of manipulators. However, no method guarantees the safety of highly dynamic obstacles, such as humans, in RL-based manipulator control. This lack of formal safety assurances prevents the application of RL for manipulators in real-world human environments. Therefore,...

Autonomous vehicles must comply with traffic rules. However, most motion planners do not explicitly consider all relevant traffic rules. Once traffic rule violations of an initially-planned trajectory are detected, there is often not enough time to replan the entire trajectory. To solve this problem , we propose to repair the initial trajectory by...

We propose a counterexample-guided inductive synthesis framework for the formal synthesis of closed-form sampled-data controllers for nonlinear systems to meet STL specifications over finite-time trajectories. Rather than stating the STL specification for a single initial condition, we consider an (infinite and bounded) set of initial conditions. C...

Accurate velocity information is often essential to the control of robot manipulators, especially for precise tracking of fast trajectories. However, joint velocities are rarely directly measured and instead estimated to save costs. While many approaches have been proposed for the velocity estimation of robot joints, no comprehensive experimental e...

Ensuring safety is crucial for the successful deployment of autonomous systems, such as self-driving vehicles and robots acting close to humans. While there exist many controllers which optimize certain criteria, such as energy consumption, comfort, or low wear, they are usually not able to guarantee safety at all times for constrained nonlinear sy...

We propose a control approach for fully actuated mechanical systems using interval arithmetic, which guarantees global uniform ultimate boundedness of the tracking error and robust performance despite model uncertainties and input disturbances. Existing robust control methods often require computationally expensive or empirical estimations of bound...

Reinforcement learning (RL) methods have gained popularity in the field of motion planning for autonomous vehicles due to their success in robotics and computer games. However, no existing work enables researchers to conveniently compare different underlying the Markov decision processes (MDPs). To address this issue, we present CommonRoad-RL-an op...

The development of autonomous vehicles requires extensive testing of software modules. Developing a reliable software platform which allows testing on a real vehicle is yet a challenging task. Therefore, open-source software platforms are becoming more important for researchers in the field of autonomous driving. For example, Baidu provides the ope...

Humans must not be harmed when they physically interact with robots. We present a new method guaranteeing impact force limits when humans and robots share a workspace. Formal guarantees are realized by an online verification method, which continuously plans and verifies fail-safe maneuvers through predicting reachable impact forces by considering a...

Maps are essential for testing autonomous driving functions. Several map and scenario formats are available. However, they are usually not compatible with each other, limiting their usability. In this paper, we address this problem using our open-source toolbox that provides map converters from different formats to the well-known CommonRoad format....

Motion planners for autonomous vehicles must obtain feasible trajectories in real-time regardless of the complexity of traffic conditions. Planning approaches that discretize the search space may perform sufficiently in general driving situations, however, they inherently struggle in critical situations with small solution spaces. To address this p...

Guaranteed state estimation computes the set of possible states of dynamical systems given the bounds of model uncertainties, disturbances, and noises. For the first time, we evaluate and compare a broad class of guaranteed state estimators for linear time-invariant systems regarding scalability, accuracy, and real-time applicability. In particular...