Conference PaperPDF Available

Ontology based Scene Creation for the Development of Automated Vehicles

Authors:

Abstract

The introduction of automated vehicles without permanent human supervision demands a functional system description, including functional system boundaries and a comprehensive safety analysis. These inputs to the technical development can be identified and analyzed by a scenario-based approach. Furthermore, to establish an economical test and release process, a large number of scenarios must be identified to obtain meaningful test results. Experts are doing well to identify scenarios that are difficult to handle or unlikely to happen. However, experts are unlikely to identify all scenarios possible based on the knowledge they have on hand. Expert knowledge modeled for computer aided processing may help for the purpose of providing a wide range of scenarios. This contribution reviews ontologies as knowledge-based systems in the field of automated vehicles, and proposes a generation of traffic scenes in natural language as a basis for a scenario creation.
https://arxiv.org/pdf/1704.01006.pdf
... As an example for such a consistent and unified ontological basis, Jatzkowski et al. demonstrated how system skills required for the dynamic driving task can be speci-fied using a scene and skill ontology [30]. To methodically support a safety case, Bagschik and Menzel examined how a knowledge base can be leveraged for an automated scene creation [31] that in turn can be used to derive test scenarios from keywords [32]. In such a scenario-based approach, it is essential to assign observed concrete scenarios to scenario classes, an approach coined as tagging in prior work [33]. ...
... This enables us to rely on the Lane as a stable interface, and to interchange implementations according to the context. Furthermore, previous work has assumed that every concept required to represent happenings in traffic is represented in one of the six layers [31]. This work extends this idea by detaching conceptually separated parts which semantically span over multiple layers or might even be completely independent of the traffic domain. ...
Preprint
Full-text available
Knowledge representation and reasoning has a long history of examining how knowledge can be formalized, interpreted, and semantically analyzed by machines. In the area of automated vehicles, recent advances suggest the ability to formalize and leverage relevant knowledge as a key enabler in handling the inherently open and complex context of the traffic world. This paper demonstrates ontologies to be a powerful tool for a) modeling and formalization of and b) reasoning about factors associated with criticality in the environment of automated vehicles. For this, we leverage the well-known 6-Layer Model to create a formal representation of the environmental context. Within this representation, an ontology models domain knowledge as logical axioms, enabling deduction on the presence of critical factors within traffic scenes and scenarios. For executing automated analyses, a joint description logic and rule reasoner is used in combination with an a-priori predicate augmentation. We elaborate on the modular approach, present a publicly available implementation, and evaluate the method by means of a large-scale drone data set of urban traffic scenarios.
... Second, the adversary-based generation and testing approaches [18,61,22] model the surrounding agents (e.g., vehicles and pedestrians) as adversarial agents to generate safety-critical driving scenarios. Third, the knowledge-based generation and testing approaches [19,56,6] aim to integrate domain knowledge such as traffic rules as additional constraints to guide the testing scenario generation process. Recently, the latter two categories have shown efficient and effective evaluation results under specific driving environments and settings, and therefore we mainly focus on them in this work. ...
Preprint
Full-text available
As shown by recent studies, machine intelligence-enabled systems are vulnerable to test cases resulting from either adversarial manipulation or natural distribution shifts. This has raised great concerns about deploying machine learning algorithms for real-world applications, especially in the safety-critical domains such as autonomous driving (AD). On the other hand, traditional AD testing on naturalistic scenarios requires hundreds of millions of driving miles due to the high dimensionality and rareness of the safety-critical scenarios in the real world. As a result, several approaches for autonomous driving evaluation have been explored, which are usually, however, based on different simulation platforms, types of safety-critical scenarios, scenario generation algorithms, and driving route variations. Thus, despite a large amount of effort in autonomous driving testing, it is still challenging to compare and understand the effectiveness and efficiency of different testing scenario generation algorithms and testing mechanisms under similar conditions. In this paper, we aim to provide the first unified platform SafeBench to integrate different types of safety-critical testing scenarios, scenario generation algorithms, and other variations such as driving routes and environments. Meanwhile, we implement 4 deep reinforcement learning-based AD algorithms with 4 types of input (e.g., bird's-eye view, camera) to perform fair comparisons on SafeBench. We find our generated testing scenarios are indeed more challenging and observe the trade-off between the performance of AD agents under benign and safety-critical testing scenarios. We believe our unified platform SafeBench for large-scale and effective autonomous driving testing will motivate the development of new testing scenario generation and safe AD algorithms. SafeBench is available at https://safebench.github.io.
... How diverse and representative the testing scenarios are essentially determines the testing efficiency and effectiveness. Bagschik et al. [29] proposes a 5-layer model for testing scenarios, including road level (layer 1), traffic infrastructure (layer 2), manipulation of layer 1 and 2 (layer 3), object (layer 4) and environment (layer 5). A recent survey [2] shows that most works focus on sampling critical scenarios at layer 4 and layer 5, i.e., by controlling the trajectories of other traffic participants such as vehicles or pedestrians or manipulating the simulated weather. ...
Preprint
High-definition (HD) maps are essential in testing autonomous driving systems (ADSs). HD maps essentially determine the potential diversity of the testing scenarios. However, the current HD maps suffer from two main limitations: lack of junction diversity in the publicly available HD maps and cost-consuming to build a new HD map. Hence, in this paper, we propose, FEAT2MAP, to automatically generate concise HD maps with scenario diversity guarantees. FEAT2MAP focuses on junctions as they significantly influence scenario diversity, especially in urban road networks. FEAT2MAP first defines a set of features to characterize junctions. Then, FEAT2MAP extracts and samples concrete junction features from a list of input HD maps or user-defined requirements. Each junction feature generates a junction. Finally, FEAT2MAP builds a map by connecting the junctions in a grid layout. To demonstrate the effectiveness of FEAT2MAP, we conduct experiments with the public HD maps from SVL and the open-source ADS Apollo. The results show that FEAT2MAP can (1) generate new maps of reduced size while maintaining scenario diversity in terms of the code coverage and motion states of the ADS under test, and (2) generate new maps of increased scenario diversity by merging intersection features from multiple maps or taking user inputs.
... Another research direction could investigate whether structuring the input space in a way suitable for the AI functionality supports the detection of corner cases. Provided that the operational domain is conceptualized as an ontology, ontology-based testing [BMM18] may support automatic detection. A properly adapted generator may specifically select promising combinations of extreme parameter values and, thus, provide valuable input for synthetic test data generation. ...
Chapter
Deployment of modern data-driven machine learning methods, most often realized by deep neural networks (DNNs), in safety-critical applications such as health care, industrial plant control, or autonomous driving is highly challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability and implausible predictions to directed attacks by means of malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from so-called safety concerns, properties that preclude their deployment as no argument or experimental setup can help to assess the remaining risk. In recent years, an abundance of state-of-the-art techniques aiming to address these safety concerns has emerged. This chapter provides a structured and broad overview of them. We first identify categories of insufficiencies to then describe research activities aiming at their detection, quantification, or mitigation. Our work addresses machine learning experts and safety engineers alike: The former ones might profit from the broad range of machine learning topics covered and discussions on limitations of recent methods. The latter ones might gain insights into the specifics of modern machine learning methods. We hope that this contribution fuels discussions on desiderata for machine learning systems and strategies on how to help to advance existing approaches accordingly.
... Similarly, scenario models, used for scenario-based testing, verification and validation (e.g. [7], [8], [9], [10]), also heavily rely on assumptions and estimations. On the one hand, making wrong or inaccurate assumptions will likely impact safety, since a non adequate consideration of corner cases and rare events for testing and validation purposes would result in overly optimistic assessment results and, worst case scenario, in the deployment of an unsafe system. ...
Conference Paper
Ensuring safety is arguably one of the largest remaining challenges before widespread market adoption of Automated Driving Systems (ADSs). One central aspect is how to provide evidence for the fulfilment of the safety claims and, in particular, how to produce a predictive and reliable safety case considering both the absence and the presence of faults in the system. In order to provide such evidence, there is a need for describing and modelling the different elements of the ADS and its operational context: models of event exposure, sensing and perception models, as well as actuation and closed-loop behaviour representations. This paper explores how estimates from such statistical models can impact the performance and operation of an ADS and, in particular, how such models can be continuously improved by incorporating more field data retrieved during the operation of (previous versions of) the ADS. Focusing on the safe driving velocity, this results in the ability to update the driving policy so to maximise the allowed safe velocity, for which the safety claim still holds. For illustration purposes, an example considering statistical models of the exposure to an adverse event, as well as failures related to the system's perception system, is analysed. Estimations from these models, using statistical confidence limits, are used to derive a safe driving policy of the ADS. The results highlight the importance of leveraging field data in order to improve the system's abilities and performance, while remaining safe. The proposed methodology, leveraging a data-driven approach, also shows how the system's safety can be monitored and maintained, while allowing for incremental expansion and improvements of the ADS.
... Both the expert and data-driven approaches suffer from a limited variety of these accident and near-accident scenarios. State-of-the-art research on testing scenario generation focuses mainly on variation in the static part of the scenarios, namely the environment, infrastructure, and road network [4], [5]. The variation of dynamic elements such as surrounding vehicles is usually minimal in the generated scenarios. ...
Article
Full-text available
We develop a human driver behavior model (Cog-Mod) based on two complementary cognitive architectures; Queueing Network-Model Human Processor (QN-MHP) and Adaptive Control of Thought-Rational (ACT-R), to represent human cognition while driving. The proposed model can integrate different task-specific analytical driver models under a similar cognitive procedure. The model can simulate variable cognitive processing ability, resulting in different stopping distances in a scenario where the front vehicle brakes sharply when it enters a trigger distance. We evaluate the model based on the distribution of stopping distance with varying cognitive processing time. This approach is useful for modeling non-ego vehicles in scenario-based testing of automated vehicles (AVs).
... Given their importance for the modeling ADS, maps have been the subject of many studies focused on their compositional description and formalization. These studies often use ontologies and logics with associated reasoning mechanisms to check the consistency of descriptions and their accuracy with respect to desired properties [4,3]. Other works propose open source mapping frameworks for highly automated driving [1,2,16]. ...
Article
Full-text available
The paper proposes a method for the correct by design coordination of autonomous driving systems (ADS). It builds on previous results on collision avoidance policies and the modeling of ADS by combining descriptions of their static environment in the form of maps, and the dynamic behavior of their vehicles. An ADS is modeled as a dynamic system involving a set of vehicles coordinated by a Runtime that based on vehicle positions on a map and their kinetic attributes, computes free spaces for each vehicle. Vehicles are bounded to move within the corresponding allocated free spaces. We provide a correct by design safe control policy for an ADS if its vehicles and the Runtime respect corresponding assume-guarantee contracts. The result is established by showing that the composition of assume-guarantee contracts is an inductive invariant that entails ADS safety. We show that it is practically possible to define speed control policies for vehicles that comply with their contracts. Furthermore, we show that traffic rules can be specified in a linear-time temporal logic, as a class of formulas that constrain vehicle speeds. The main result is that, given a set of traffic rules, it is possible to derive free space policies of the Runtime such that the resulting system behavior is safe by design with respect to the rules.
... Given their importance for the modeling ADS, maps have been the subject of many studies focused on their compositional description and formalization. These studies often use ontologies and logics with associated reasoning mechanisms to check the consistency of descriptions and their accuracy with respect to desired properties [4,3]. Other works propose open source mapping frameworks for highly automated driving [1,2,16]. ...
Preprint
Full-text available
The paper proposes a method for the correct by design coordination of autonomous driving systems (ADS). It builds on previous results on collision avoidance policies and the modeling of ADS by combining descriptions of their static environment in the form of maps, and the dynamic behavior of their vehicles. An ADS is modeled as a dynamic system involving a set of vehicles coordinated by a Runtime that based on vehicle positions on a map and their kinetic attributes, computes free spaces for each vehicle. Vehicles are bounded to move within the corresponding allocated free spaces. We provide a correct by design safe control policy for an ADS if its vehicles and the Runtime respect corresponding assume-guarantee contracts. The result is established by showing that the composition of assume-guarantee contracts is an inductive invariant that entails ADS safety. We show that it is practically possible to define speed control policies for vehicles that comply with their contracts. Furthermore, we show that traffic rules can be specified in a linear-time temporal logic, as a class of formulas that constrain vehicle speeds. The main result is that, given a set of traffic rules, it is possible to derive free space policies of the Runtime such that the resulting system behavior is safe by design with respect to the rules.
... With the development of deep learning, target detection methods have started changing from classical machine learning methods to deep learning methods, representing a new paradigm of machine learning. Target detection has been widely used in face detection [5], automatic driving [6], text detection [7,8], and other fields. Traditional target detection methods are based on color or shape features for target extraction. ...
Article
Full-text available
Road traffic elements comprise an important part of roads and represent the main content involved in the construction of a basic traffic geographic information database, which is particularly important for the development of basic traffic geographic information. However, the following problems still exist for the extraction of traffic elements: insufficient data, complex scenarios, small targets, and incomplete element information. Therefore, a set of road traffic multielement remote sensing image datasets obtained by unmanned aerial vehicles (UAVs) is produced, and an improved YOLOv4 network algorithm combined with an attention mechanism is proposed to automatically recognize and detect multiple elements of road traffic in UAV imagery. First, the scale range of different objects in the datasets is counted, and then the size of the candidate box is obtained by the k-means clustering method. Second, mosaic data augmentation technology is used to increase the number of trained road traffic multielement datasets. Then, by integrating the efficient channel attention (ECA) mechanism into the two effective feature layers extracted from the YOLOv4 backbone network and the upsampling results, the network focuses on the feature information and then trains the datasets. At the same time, the complete intersection over union (CIoU) loss function is used to consider the geometric relationship between the object and the test object, to solve the overlapping problem of the juxtaposed dense test element anchor boxes, and to reduce the rate of missed detection. Finally, the mean average precision (mAP) is calculated to evaluate the experimental effect. The experimental results show that the mAP value of the proposed method is 90.45%, which is 15.80% better than the average accuracy of the original YOLOv4 network. The average detection accuracy of zebra crossings, bus stations, and roadside parking spaces is improved by 12.52%, 22.82%, and 12.09%, respectively. The comparison experiments and ablation experiments proved that the proposed method can realize the automatic recognition and detection of multiple elements of road traffic, and provide a new solution for constructing a basic traffic geographic information database.
Thesis
Cette thèse CIFRE, réalisé au sein de Stellantis, fournit une stratégie de génération de scénarios, modélisée par niveaux d’abstraction et orientée par la sensibilité du VA, pour une validation par simulation. Ce travail s’inscrit dans le périmètre du standard ISO PAS 21448 /SOTIF (Safety Of The Intended Functionality). Pour ce faire, la démarche suivie s’articule autour de cinq contributions : (1) Une analyse de l’architecture fonctionnelle du VA et la mise en évidence des challenges liés à la validation de sa sécurité : aspects normatifs, chaines de simulation, la présence d’incertitude dans l’environnement opérationnel du VA. (2) La proposition d’un cadre conceptuel (modèle de connaissance) sur lequel s’appuiera la méthodologie de génération des scénarios qui sera proposée par la suite. (3) Une synthèse sur les indicateurs manipulés dans la littérature, ainsi que ceux, que nous retiendrons dans notre stratégie de génération finale dont notamment l’indicateur de sensibilité. Elle donne également une structure du système de génération des scénarios et de validation par simulation de la sécurité du VA, ainsi que la manière dont les indicateurs seront exploités dans cette structure. (4) La proposition d’une heuristique de génération des scénarios et l’estimation de l’indicateur de risque associé au VA. Cette quatrième contribution, s’appuie sur les éléments développés dans les contributions précédentes : le modèle conceptuel proposé (contribution 2), la structure du système de génération et de validation ainsi que les indicateurs associés (contribution 3). (5) Enfin, la dernière contribution est une implémentation des propositions précédentes via un cas d’étude.Mots clés : Véhicule Autonome (VA), SOTIF (Safety Of The Intended Functionality), Limitation de performances fonctionnelles, Insuffisances fonctionnelles, Scénarios critiques, Métrique de sensibilité, Stratégie de génération de scénarios, Validation par simulation
Conference Paper
Full-text available
The project Automated Unmanned Protective Vehicle for Highway Hard Shoulder Road Works (aFAS) aims to develop an unmanned protective vehicle to reduce the risk of injuries due to crashes for road workers. To ensure functional safety during operation in public traffic the system shall be developed following the ISO 26262 standard. After defining the functional range in the item definition, a hazard analysis and risk assessment has to be done. The ISO 26262 standard gives hints how to process this step and demands a systematic way to identify system hazards. Best practice standards provide systematic ways for hazard identification, but lack applicability for automated vehicles due to the high variety and number of different driving situations even with a reduced functional range. This contribution proposes a new method to identify hazardous events for a system with a given functional description. The method utilizes a skill graph as a functional model of the system and an overall definition of a scene for automated vehicles to identify potential hazardous events. An adapted Hazard and Operability Analysis approach is used to identify system malfunctions. A combination of all methods results in operating scenes with potential hazardous events. These can be assessed afterwards towards their criticality. A use case example is taken from the current development phase of the project aFAS.
Chapter
Full-text available
In the future, the functions of autonomous driving could fundamentally change all road traffic; to do so, it would have to be implemented on a large scale, in series production.
Chapter
Full-text available
The development of autonomous vehicles currently focuses on the functionality of vehicle guidance systems.
Conference Paper
Full-text available
Mobile road works on the hard shoulder of German highways bear an increased accident risk for the crew of the protective vehicle which safeguards road works against moving traffic. The project “Automated Unmanned Protective Vehicle for Highway Hard Shoulder Road Works” aims at the unmanned operation of the protective vehicle in order to reduce this risk. Simultaneously, this means the very first unmanned operation of a vehicle on German roads in public traffic. This contribution introduces the project by pointing out the main objectives and demonstrates the current state of the system design regarding functionality, modes of operation, as well as the functional system architecture. Pivotal for the project, the scientific challenges raised by the unmanned operation – strongly related to the general challenges in the field of automated driving – are presented as well. The results of the project shall serve as a basis to stimulate an advanced discussion about ensuring safety for fully automated vehicles in public traffic operating at higher speeds and in less defined environments. Thus, this contribution aims at contacting the scientific community in an early state of the project.
Conference Paper
Full-text available
For the design and test of functional modules of an automated vehicle, it is essential to define interfaces. While interfaces on the perception side, like object lists, point clouds or occupancy grids, are to a certain degree settled already, they are quite vague in the consecutive steps of context modeling and in particular on the side of driving execution. The authors consider the scene as the central interface between perception and behavior planning & control. Within the behavior planning & control block, a situation is a central data container. A scenario is a common approach to substantiate test cases for functional modules and can be used to detail the functional description of a system. However, definitions of these terms are often-at best-vague or even contradictory. This paper will review these definitions and come up with a consistent definition for each term. Moreover, we present an example for the implementation of each of these interfaces.
Article
Full-text available
The introduction of highly automated driving and autonomous road vehicles will imply new functional safety challenges. The higher complexity and the partly implicit definition of the tasks for the E/E systems will make it harder to argue completeness and correctness of the safety requirements in each phase of the ISO 26262 lifecycle. This paper discusses the new situation in terms of an increasing semantic gap, and recommends to perform more safety refinement steps. As a consequence, ISO 26262 should be amended with activities prescribing new refinements levels.
Thesis
In dieser Arbeit wird ein Beitrag für den methodischen Test von automatisierten Fahrfunktionen mit Hilfe von virtuellen Umgebungen geleistet. Im ersten Teil wird die Notwendigkeit eines systematischen Testkonzepts begründet und die These aufgestellt, dass ein szenariobasiertes Testkonzept eine mögliche Lösung für das Testdilemma darstellen könnte. Dazu werden sechs Forschungsfragen aufgestellt, die für die Entwicklung eines szenariobasierten Ansatzes beantwortet werden müssen. Im zweiten Teil werden die Grundlagen und Voraussetzungen der Arbeit dargestellt. Hierfür werden Begriffe und Definitionen vorgestellt. Weiterhin wird der Begriff der Komplexität von Szenarien untersucht. Die Automatisierungsgrade und eine funktionale Systemarchitektur für automatisierte Fahrfunktionen werden vorgestellt. Der Teil schließt mit einer Klassifikation von verschiedenen X-in-the-Loop-Verfahren ab. Im dritten Teil wird das Testkonzept des modularen virtuellen Testbaukastens vorgestellt. Es werden Anforderungen definiert sowie der Aufbau und die Schnittstellen zwischen den Modulen des Testbaukastens präsentiert. Für die Auswahl und Analyse der Einflussparameter, die Testfallerstellung und die Testdurchführung mittels X-in-the-Loop-Verfahren werden Anforderungen definiert und der relevante Stand der Technik vorgestellt. Daraus wird der Forschungsbedarf abgeleitet. Für die Auswahl und Analyse der Einflussparameter wird ein Schema zur Beschreibung der Einflussparameter hergeleitet und Informationsquellen für die Auswahl und Analyse von Einflussparametern werden bewertet. Für die Testfallerstellung wird ein generisches Modell zur Beschreibung von Szenarien vorgestellt und eine kombinatorische Testfallableitung präsentiert. Für die Testdurchführung wird eine Zuordnungsmethode für Testfälle auf verschiedene X-in-the-Loop-Verfahren beschrieben. Zusätzlich werden Testtreiber für die Module einer funktionalen Systemarchitektur analysiert und die Testtreiber des modularen virtuellen Testbaukastens vorgestellt. Für die Testfallauswertung werden Anforderungen definiert und Methoden aus dem Stand der Technik zur Bewertung und zur Analyse der Testergebnissen präsentiert. Der Teil schließt mit einer Beschreibung der Limitationen des Testbaukastens ab. Der vierte Teil beschreibt die Anwendung des Testbaukastens im Fallbeispiel des Engstellenassistenten. Das Projekt wird vorgestellt und die verschiedenen Module des Testbaukastens werden angewendet.