Johannes Betz

Technical University of Munich | TUM · Department of Mobility Systems Engineering

Whether BMW, VW or Google: Almost all leading automobile and technology companies are researching and developing the multi-stage autonomy of vehicles, which enables a completely self-driven vehicle without a driver in autonomy level 5. Based on his assessment and experience, the driver had previously carried out environmental detection, localizatio...

This paper shows a software stack capable of planning a minimum curvature trajectory for an autonomous race car on the basis of an occupancy grid map and introduces a controller design that allows to follow the trajectory at the handling limits. The minimum curvature path is generated using a quadratic optimisation problem (QP) formulation. The key...

This paper presents a detailed description of the software architecture that is used in the autonomous Roborace vehicles by the TUM-Team. The development of the software architecture was driven by both hardware components and usage of open source languages for making the software architecture reusable and easy to understand. The architecture combin...

Object detection in camera images, using deep learning has been proven successfully in recent years. Rising detection rates and computationally efficient network structures are pushing this technique towards application in production vehicles. Nevertheless, the sensor quality of the camera is limited in severe weather conditions and through increas...

The rising popularity of self-driving cars has led to the emergence of a new research field in the recent years: Autonomous racing. Researchers are developing software and hardware for high performance race vehicles which aim to operate autonomously on the edge of the vehicles limits: High speeds, high accelerations, low reaction times, highly unce...

In recent years, different approaches for motion planning of autonomous vehicles have been proposed that can handle complex traffic situations. However, these approaches are rarely compared on the same set of benchmarks. To address this issue, we present the results of a large-scale motion planning competition for autonomous vehicles based on the C...

We present a novel autonomous driving framework, DualAD, designed to imitate human reasoning during driving. DualAD comprises two layers: a rule-based motion planner at the bottom layer that handles routine driving tasks requiring minimal reasoning, and an upper layer featuring a rule-based text encoder that converts driving scenarios from absolute...

This work aims to present a three-dimensional vehicle dynamics state estimation under varying signal quality. Few researchers have investigated the impact of three-dimensional road geometries on the state estimation and, thus, neglect road inclination and banking. Especially considering high velocities and accelerations, the literature does not add...

In this paper we investigate the use of Doppler Beam Sharpening (DBS) as a preprocessing step for radar data in an autonomous driving 3D detection pipeline to enhance the effective angular resolution. Due to the assumption of a static scene, the proposed method applies DBS together with a moving point filtering approach to reduce preprocessing arti...

Visual-inertial navigation system (VINS) is a state-of-the-art technology for estimating the motion and position of moving objects, such as drones and robots. Tightly coupled methods have high accuracy but low real-time performance, while loosely coupled systems are more efficient, especially for resource-constrained systems. This study proposes a...

This research aims to investigate professional racing drivers’ expertise to develop an understanding of their cognitive and adaptive skills to create new autonomy algorithms. An expert interview study was conducted with 11 professional race drivers, data analysts, and racing instructors from across prominent racing leagues. The interviews were cond...

Connected and Automated Vehicles (CAVs) and Robot Swarms (RS) have the potential to transform the transportation and manufacturing sectors into safer, more efficient, sustainable systems. However, extensive testing and validation of their algorithms are required. Small-scale testbeds offer a cost-effective and controlled environment for testing alg...

In the ever-evolving landscape of autonomous vehicles, competition and research of high-speed autonomous racing emerged as a captivating frontier, pushing the limits of perception, planning, and control. Autonomous racing presents a setup where the intersection of cutting-edge software and hardware development sparks unprecedented opportunities and...

Our research introduces a modular motion planning framework for autonomous vehicles using a sampling-based trajectory planning algorithm. This approach effectively tackles the challenges of solution space construction and optimization in path planning. The algorithm is applicable to both real vehicles and simulations, offering a robust solution for...

Traditional static visual-inertial navigation systems (VINS) confront substantial challenges in dynamic environments, while current dynamic VINS solutions struggle to maintain high-accuracy performance in static environments. Achieving higher localization accuracy in static environments often requires incorporating additional geometric structural i...

Teaching autonomous and intelligent transportation systems in higher education has traditionally focused on theory, often lacking comprehensive coverage of the practical techniques required for real-world applications. To overcome this, we developed a new university course centered around hands-on learning with a modular autonomous small-scale vehi...

Our research aims to generate robust, dense 3D depth maps for robotics, especially autonomous driving applications. Since cameras output 2D images and active sensors such as LiDAR or radar produce sparse depth measurements, dense depth maps need to be estimated. Recent methods based on visual transformer networks have outperformed conventional deep...

Autonomous vehicles require accurate and robust localization and mapping algorithms to navigate safely and reliably in urban environments. We present a novel sensor fusion-based pipeline for offline mapping and online localization based on LiDAR sensors. The proposed approach leverages four LiDAR sensors. Mapping and localization algorithms are bas...

Accurate and reliable localization and mapping are crucial prerequisites for autonomous driving to achieve path planning. However, in large-scale complex dynamic environ- ments, the traditional LiDAR loop closure detection methods that rely on radius search can easily result in false negatives, leading to the inability to correct accumulated errors...

ROS 2 is a framework consisting of software libraries for developing robot systems, such as autonomous driving systems, that consist of multiple interacting components. In ROS 2, each component is implemented as a node, which contains time-triggered and event-triggered tasks. These tasks communicate with each other via ROS 2 topics or shared memory...

Real-time capability and robust software behavior have emerged as crucial issues since autonomous vehicles must react reliably to various traffic conditions when operating on our streets. The objective of our work is to understand and examine the processing latency of a software stack for autonomous vehicles. In this paper, we propose a framework b...

Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient and scalable perception algorithms, the maximum information should be extracted from the available sensor data. In this work, we present our concept for an end-to-end perception architecture, named DeepSTEP. The deep learning-based architecture pr...

The driving behavior of autonomous vehicles has a significant impact on safety for all traffic participants. Unlike current traffic participants, autonomous vehicles in the future will also need to adhere to safety standards and defined risk properties in order to achieve a high level of public acceptance. At the same time, successful autonomous ve...

Autonomous racing with scaled race cars has gained increasing attention as an effective approach for developing perception, planning and control algorithms for safe autonomous driving at the limits of the vehicle’s handling. To train agile control policies for autonomous racing, learning-based approaches largely utilize reinforcement learning, albe...

Violations of real-time properties and high latencies have emerged as crucial issues in autonomous vehicles since they can lead to unwanted vehicle behavior and critical maneuvers. Our study aims to provide a comprehensive understanding of latencies in a software stack for autonomous vehicles. In this paper, we present an evaluation workflow to ins...

Robot localization is an inverse problem of finding a robot’s pose using a map and sensor measurements. In recent years, Invertible Neural Networks (INNs) have successfully solved ambiguous inverse problems in various fields. This paper proposes a framework that approaches the localization problem with INN. We design a network that provides implici...

Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient and scalable perception algorithms, the maximum information should be extracted from the available sensor data. In this work, we present our concept for an end-to-end perception architecture, named DeepSTEP. The deep learning-based architecture pr...

In this paper, we present a novel method for integrating 3D LiDAR depth measurements into the existing ORB-SLAM3 by building upon the RGB-D mode. We propose and compare two methods of depth map generation: conventional computer vision methods, namely an inverse dilation operation, and a supervised deep learning-based approach. We integrate the form...

Given the increasing complexity of autonomous driving, it becomes more difficult to test driving functions and to optimize algorithm parameters. One major challenge is that many parameters and software components influence each other, so even small changes in parameters can lead to a high sensitivity in vehicle performance. Many approaches involve...

Autonomous vehicles (AVs) are being rapidly introduced into our lives. However, public misunderstanding and mistrust have become prominent issues hindering the acceptance of these driverless technologies. The primary objective of this study is to evaluate the effectiveness of a driving simulator to help the public gain an understanding of AVs and b...

Autonomous driving faces the difficulty of securing the lowest possible software execution times to allow a safe and reliable application. One critical variable for autonomous vehicles is the latency from the detection of obstacles to the final actuation response of the vehicle, especially in the case of high-speed driving. A prerequisite for auton...

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground...

The objective of this work is to provide a comprehensive understanding of the development of autonomous vehicle perception systems. So far, most autonomy perception research has been concentrated on improving perception systems’ algorithmic quality or combining different sensor setups. In our work, we draw conclusions from participating in the Indy...

Autonomous vehicles face numerous challenges to ensure safe operation in unpredictable and hazardous conditions. The autonomous driving environment is characterized by high uncertainty, especially in occluded areas with limited information about the surrounding obstacles. This work aims to provide a trajectory planner to solve these unsafe environm...

The vehicle dynamics model is a fundamental prerequisite for advanced software development for intelligent vehicles. This incites the need for accurate mathematical modeling to match the driving dynamics of real vehicles as closely as possible. However, an accurate vehicle model has a variety of parameters that often rely on massive real-vehicle te...

In this paper, we present a novel method for integrating 3D LiDAR depth measurements into the existing ORB-SLAM3 by building upon the RGB-D mode. We propose and compare two methods of depth map generation: conventional computer vision methods, namely an inverse dilation operation, and a supervised deep learning-based approach. We integrate the form...

Automotive tire tread depth has a major influence on a car's safety and thus has to be closely monitored. However, there is currently no on-board solution that can measure tire wear with an accuracy of less than 1 mm. TireEye is an optical device mounted inside the wheel well and facing towards the road. It records the cross-section of the longitud...

Autonomous racing with scaled race cars has gained increasing attention as an effective approach for developing perception, planning and control algorithms for safe autonomous driving at the limits of the vehicle's handling. To train agile control policies for autonomous racing, learning-based approaches largely utilize reinforcement learning, albe...

Robot localization is an inverse problem of finding a robot's pose using a map and sensor measurements. In recent years, Invertible Neural Networks (INNs) have successfully solved ambiguous inverse problems in various fields. This paper proposes a framework that solves the localization problem with INN. We design an INN that provides implicit map r...

Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, the DRL training is an active exploration process typically derived in simulation environments. Although this simulation training is cheap and fast, applying DRL algo...

Although robotics courses are well established in higher education, the courses often focus on theory and sometimes lack the systematic coverage of the techniques involved in developing, deploying, and applying software to real hardware. Additionally, most hardware platforms for robotics teaching are low-level toys aimed at younger students at midd...

Autonomous Racing awards agents that react to opponents' behaviors with agile maneuvers towards progressing along the track while penalizing both over-aggressive and over-conservative agents. Understanding the intent of other agents is crucial to deploying autonomous systems in adversarial multi-agent environments. Current approaches either oversim...

The 3rd Japan Automotive AI Challenge was an international online autonomous racing challenge where 164 teams competed in December 2021. This paper outlines the winning strategy to this competition, and the advantages and challenges of using the Autoware.Auto open source autonomous driving platform for multi-agent racing. Our winning approach inclu...

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems like disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground f...

Autonomous vehicles (AVs) are being rapidly introduced into our lives. However, public misunderstanding and mistrust have become prominent issues hindering the acceptance of these driverless technologies. The primary objective of this study is to evaluate the effectiveness of a driving simulator to help the public gain an understanding of AVs and b...

Motorsport has always been an enabler for technological advancement, and the same applies to the autonomous driving industry. The team TUM Autonomous Motorsports will participate in the Indy Autonomous Challenge in October 2021 to benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapoli...

Scenario understanding and motion prediction are essential components for completely replacing human drivers and for enabling highly and fully automated driving (SAE-Level 4/5). In deeply stochastic and uncertain traffic scenarios, autonomous driving software must act beyond existing traffic rules and must predict critical situations in advance to...

Autonomous systems are composed of several subsystems such as mechanical, propulsion, perception, planning and control. These are traditionally designed separately which makes performance optimization of the integrated system a significant challenge. In this paper, we study the problem of using gradient-free optimization methods to jointly optimize...

Motorsport has always been an enabler for technological advancement, and the same applies to the autonomous driving industry. The team TUM Auton-omous Motorsports will participate in the Indy Autonomous Challenge in Octo-ber 2021 to benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapo...

In literature, scientists describe human mobility in a range of granularities by several different models. Using frameworks like MATSIM, VehiLux, or Sumo, they often derive individual human movement indicators in their most detail. However, such agent-based models tend to be difficult and require much information and computational power to correctl...

Solving a Minimum Lap Time Problem (MLTP), under the constraints stemming from a race car's driving dynamics, can be considered to be state of the art. Nevertheless, when dealing with electric race vehicles as in Formula E or the Roborace competition, solving an MLTP is not enough to form an appropriate competition strategy: Maximum performance ove...

Autonomous Racing is gaining increasing publicity as an attractive showcase of state-of-the-art technologies and the enhanced algorithms used for autonomous driving. The Indy Autonomous Challenge (IAC) tackled autonomous high-speed wheel-to-wheel racing at the famous Indianapolis Motor Speedway (IMS) in October 2021. To solve this problem, advanced...