Johannes Betz

Technische Universität München | TUM · Department of Mobility Systems Engineering

Whether BMW, VW or Google: Almost all leading automobile and technology companies are researching and developing the multi-stage autonomy of vehicles, which enables a completely self-driven vehicle without a driver in autonomy level 5. Based on his assessment and experience, the driver had previously carried out environmental detection, localizatio...

This paper shows a software stack capable of planning a minimum curvature trajectory for an autonomous race car on the basis of an occupancy grid map and introduces a controller design that allows to follow the trajectory at the handling limits. The minimum curvature path is generated using a quadratic optimisation problem (QP) formulation. The key...

This paper presents a detailed description of the software architecture that is used in the autonomous Roborace vehicles by the TUM-Team. The development of the software architecture was driven by both hardware components and usage of open source languages for making the software architecture reusable and easy to understand. The architecture combin...

Object detection in camera images, using deep learning has been proven successfully in recent years. Rising detection rates and computationally efficient network structures are pushing this technique towards application in production vehicles. Nevertheless, the sensor quality of the camera is limited in severe weather conditions and through increas...

The rising popularity of self-driving cars has led to the emergence of a new research field in the recent years: Autonomous racing. Researchers are developing software and hardware for high performance race vehicles which aim to operate autonomously on the edge of the vehicles limits: High speeds, high accelerations, low reaction times, highly unce...

Real-time capability and robust software behavior have emerged as crucial issues since autonomous vehicles must react reliably to various traffic conditions when operating on our streets. The objective of our work is to understand and examine the processing latency of a software stack for autonomous vehicles. In this paper, we propose a framework b...

Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient and scalable perception algorithms, the maximum information should be extracted from the available sensor data. In this work, we present our concept for an end-to-end perception architecture, named DeepSTEP. The deep learning-based architecture pr...

The driving behavior of autonomous vehicles has a significant impact on safety for all traffic participants. Unlike current traffic participants, autonomous vehicles in the future will also need to adhere to safety standards and defined risk properties in order to achieve a high level of public acceptance. At the same time, successful autonomous ve...

Autonomous racing with scaled race cars has gained increasing attention as an effective approach for developing perception, planning and control algorithms for safe autonomous driving at the limits of the vehicle’s handling. To train agile control policies for autonomous racing, learning-based approaches largely utilize reinforcement learning, albe...

Violations of real-time properties and high latencies have emerged as crucial issues in autonomous vehicles since they can lead to unwanted vehicle behavior and critical maneuvers. Our study aims to provide a comprehensive understanding of latencies in a software stack for autonomous vehicles. In this paper, we present an evaluation workflow to ins...

Robot localization is an inverse problem of finding a robot’s pose using a map and sensor measurements. In recent years, Invertible Neural Networks (INNs) have successfully solved ambiguous inverse problems in various fields. This paper proposes a framework that approaches the localization problem with INN. We design a network that provides implici...

Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient and scalable perception algorithms, the maximum information should be extracted from the available sensor data. In this work, we present our concept for an end-to-end perception architecture, named DeepSTEP. The deep learning-based architecture pr...

In this paper, we present a novel method for integrating 3D LiDAR depth measurements into the existing ORB-SLAM3 by building upon the RGB-D mode. We propose and compare two methods of depth map generation: conventional computer vision methods, namely an inverse dilation operation, and a supervised deep learning-based approach. We integrate the form...

Given the increasing complexity of autonomous driving, it becomes more difficult to test driving functions and to optimize algorithm parameters. One major challenge is that many parameters and software components influence each other, so even small changes in parameters can lead to a high sensitivity in vehicle performance. Many approaches involve...

Autonomous vehicles (AVs) are being rapidly introduced into our lives. However, public misunderstanding and mistrust have become prominent issues hindering the acceptance of these driverless technologies. The primary objective of this study is to evaluate the effectiveness of a driving simulator to help the public gain an understanding of AVs and b...

Autonomous driving faces the difficulty of securing the lowest possible software execution times to allow a safe and reliable application. One critical variable for autonomous vehicles is the latency from the detection of obstacles to the final actuation response of the vehicle, especially in the case of high-speed driving. A prerequisite for auton...

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground...

The objective of this work is to provide a comprehensive understanding of the development of autonomous vehicle perception systems. So far, most autonomy perception research has been concentrated on improving perception systems’ algorithmic quality or combining different sensor setups. In our work, we draw conclusions from participating in the Indy...

In this paper, we present a novel method for integrating 3D LiDAR depth measurements into the existing ORB-SLAM3 by building upon the RGB-D mode. We propose and compare two methods of depth map generation: conventional computer vision methods, namely an inverse dilation operation, and a supervised deep learning-based approach. We integrate the form...

Automotive tire tread depth has a major influence on a car's safety and thus has to be closely monitored. However, there is currently no on-board solution that can measure tire wear with an accuracy of less than 1 mm. TireEye is an optical device mounted inside the wheel well and facing towards the road. It records the cross-section of the longitud...

Autonomous racing with scaled race cars has gained increasing attention as an effective approach for developing perception, planning and control algorithms for safe autonomous driving at the limits of the vehicle's handling. To train agile control policies for autonomous racing, learning-based approaches largely utilize reinforcement learning, albe...

Robot localization is an inverse problem of finding a robot's pose using a map and sensor measurements. In recent years, Invertible Neural Networks (INNs) have successfully solved ambiguous inverse problems in various fields. This paper proposes a framework that solves the localization problem with INN. We design an INN that provides implicit map r...

Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, the DRL training is an active exploration process typically derived in simulation environments. Although this simulation training is cheap and fast, applying DRL algo...

Although robotics courses are well established in higher education, the courses often focus on theory and sometimes lack the systematic coverage of the techniques involved in developing, deploying, and applying software to real hardware. Additionally, most hardware platforms for robotics teaching are low-level toys aimed at younger students at midd...

Autonomous Racing awards agents that react to opponents' behaviors with agile maneuvers towards progressing along the track while penalizing both over-aggressive and over-conservative agents. Understanding the intent of other agents is crucial to deploying autonomous systems in adversarial multi-agent environments. Current approaches either oversim...

The 3rd Japan Automotive AI Challenge was an international online autonomous racing challenge where 164 teams competed in December 2021. This paper outlines the winning strategy to this competition, and the advantages and challenges of using the Autoware.Auto open source autonomous driving platform for multi-agent racing. Our winning approach inclu...

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems like disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground f...

Autonomous vehicles (AVs) are being rapidly introduced into our lives. However, public misunderstanding and mistrust have become prominent issues hindering the acceptance of these driverless technologies. The primary objective of this study is to evaluate the effectiveness of a driving simulator to help the public gain an understanding of AVs and b...

Motorsport has always been an enabler for technological advancement, and the same applies to the autonomous driving industry. The team TUM Autonomous Motorsports will participate in the Indy Autonomous Challenge in October 2021 to benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapoli...

Scenario understanding and motion prediction are essential components for completely replacing human drivers and for enabling highly and fully automated driving (SAE-Level 4/5). In deeply stochastic and uncertain traffic scenarios, autonomous driving software must act beyond existing traffic rules and must predict critical situations in advance to...

Autonomous systems are composed of several subsystems such as mechanical, propulsion, perception, planning and control. These are traditionally designed separately which makes performance optimization of the integrated system a significant challenge. In this paper, we study the problem of using gradient-free optimization methods to jointly optimize...

Motorsport has always been an enabler for technological advancement, and the same applies to the autonomous driving industry. The team TUM Auton-omous Motorsports will participate in the Indy Autonomous Challenge in Octo-ber 2021 to benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapo...

In literature, scientists describe human mobility in a range of granularities by several different models. Using frameworks like MATSIM, VehiLux, or Sumo, they often derive individual human movement indicators in their most detail. However, such agent-based models tend to be difficult and require much information and computational power to correctl...

Solving a Minimum Lap Time Problem (MLTP), under the constraints stemming from a race car's driving dynamics, can be considered to be state of the art. Nevertheless, when dealing with electric race vehicles as in Formula E or the Roborace competition, solving an MLTP is not enough to form an appropriate competition strategy: Maximum performance ove...

Autonomous Racing is gaining increasing publicity as an attractive showcase of state-of-the-art technologies and the enhanced algorithms used for autonomous driving. The Indy Autonomous Challenge (IAC) tackled autonomous high-speed wheel-to-wheel racing at the famous Indianapolis Motor Speedway (IMS) in October 2021. To solve this problem, advanced...

The rising popularity of self-driving cars has led to the emergence of a new research field inrecent years: Autonomous racing. Researchers are developing software and hardware for high-performancerace vehicles which aim to operate autonomously on the edge of the vehicle’s limits: High speeds, highaccelerations, low reaction times, highly uncertain,...

In 2017, the German ethics commission for automated and connected driving released 20 ethical guidelines for autonomous vehicles. It is now up to the research and industrial sectors to enhance the development of autonomous vehicles based on such guidelines. In the current state of the art, we find studies on how ethical theories can be integrated....

High-performance autonomy often must operate at the boundaries of safety. When external agents are present in a system, the process of ensuring safety without sacrificing performance becomes extremely difficult. In this paper, we present an approach to stress test such systems based on the rapidly exploring random tree (RRT) algorithm. We propose t...

The rising popularity of driver-less cars has led to the research and development in the field of autonomous racing, and overtaking in autonomous racing is a challenging task. Vehicles have to detect and operate at the limits of dynamic handling and decisions in the car have to be made at high speeds and high acceleration. One of the most crucial p...

Automotive traffic scenes are complex due to the variety of possible scenarios, objects, and weather conditions that need to be handled. In contrast to more constrained environments, such as automated underground trains, automotive perception systems cannot be tailored to a narrow field of specific tasks but must handle an ever-changing environment...

Automotive traffic scenes are complex due to the variety of possible scenarios, objects, and weather conditions that need to be handled. In contrast to more constrained environments, such as automated underground trains, automotive perception systems cannot be tailored to a narrow field of specific tasks but must handle an ever-changing environment...

State-of-the-art 3D object detection for autonomous driving is achieved by processing lidar sensor data with deep-learning methods. However, the detection quality of the state of the art is still far from enabling safe driving in all conditions. Additional sensor modalities need to be used to increase the confidence and robustness of the overall de...

Connected and autonomous vehicles (CAVs) could reduce emissions, increase road safety, and enhance ride comfort. Multiple CAVs can form a CAV platoon with a close inter-vehicle distance, which can further improve energy efficiency, save space, and reduce travel time. To date, there have been few detailed studies of self-driving algorithms for CAV p...

With the evolution of self-driving cars, autonomous racing series like Roborace and the Indy Autonomous Challenge are rapidly attracting growing attention. Researchers participating in these competitions hope to subsequently transfer their developed functionality to passenger vehicles, in order to improve self-driving technology for reasons of safe...

Performance and robustness targets have been considered for controller design for decades. However, robust controllers usually suffer from performance limitations due to conservative uncertainty assumptions made a priori to system operation. The increased number of systems (e.g. autonomous vehicles) which require high-performance operation in safet...

The market for commercial vehicles offers great potential for the use of electric vehicles today and in the future due to its spatial mobility behaviour. With a disposition model, which is combined with an energy management system and a charging management system, it should be possible to carry out optimal deployment planning for conventional and e...

In circuit motorsport, race strategy helps to finish the race in the best possible position by optimally determining the pit stops. Depending on the racing series, pit stops are needed to replace worn-out tires, refuel the car, change drivers, or repair the car. Assuming a race without opponents and considering only tire degradation, the optimal ra...

n this paper we give an overview on methods for the optical detection of road surface damages and analyze the importance of contextual information. The objective is to improve the optical detection of road damages, especially potholes, based on images from windscreen mounted monocular cameras, as well as to reduce the complexity and thus save compu...

The way to full autonomy of public road vehicles requires the step-by-step replacement of the human driver, with the ultimate goal of replacing the driver completely. Eventually, the driving software has to be able to handle all situations that occur on its own, even emergency situations. These particular situations require extreme combined braking...

Electric vhicles and autonomous driving dominate current research efforts in the automotive sector. The two topics go hand in hand in terms of enabling safer and more environmentally friendly driving. One fundamental building block of an autonomous vehicle is the ability to build a map of the environment and localize itself on such a map. In this p...

Our dynamic, graph-based trajectory planner for race vehicles is now available open-source on GitHub! The planner demonstrated competitive behavior on a real race vehicle during the Roborace Season Alpha at speeds over 200kph, as well as in the simulative competition of the Indy Autonomous Challenge Hackaton 2. The planner is available on GitHub:...

The development of software components for autonomous driving functions should al-ways include an extensive and rigorous evaluation. Since real-world testing is expensive and safety-critical – especially when facing dynamic racing scenarios at the limit of handling – a favored approach is simulation-based testing. In this work, we pro-pose an open-...

The Scenario Architect provides a lightweight graphical user interface that allows a straightforward realization and manipulation of concrete driving testing scenarios. Exemplary use-cases are the validation of an online verification framework or training of a prediction algorithm. The Scenario Architect is available on GitHub: https://github.com/...

We wish to make the following corrections to the published paper [...]

Applying an optimal race strategy is a decisive factor in achieving the best possible result in a motorsport race. This mainly implies timing the pit stops perfectly and choosing the optimal tire compounds. Strategy engineers use race simulations to assess the effects of different strategic decisions (e.g., early vs. late pit stop) on the race resu...

The development of software components for autonomous driving functions should always include an extensive and rigorous evaluation. Since real-world testing is expensive and safety-critical -- especially when facing dynamic racing scenarios at the limit of handling -- a favored approach is simulation-based testing. In this work, we propose an open-...

The way to full autonomy of public road vehicles requires the step-by-step replacement of the human driver, with the ultimate goal of replacing the driver completely. Eventually, the driving software has to be able to handle all situations that occur on its own, even emergency situations. These particular situations require extreme combined braking...