Jasper Sprockhoff’s research while affiliated with German Aerospace Center (DLR) and other places

The complexity of safety-critical systems is continuously increasing. To create safe systems despite the complexity, the system development requires a strong integration of system design and safety activities. A promising choice for integrating system design and safety activities are model-based approaches. They can help to handle complexity through abstraction, automation, and reuse and are applied to design, analyze, and assure systems. In practice, however, there is often a disconnect between the model-based design and safety activities. At the same time, there is often a delay until recent approaches are available in model-based frameworks. As a result, the advantages of the models are often not fully utilized. Therefore, this article proposes a framework that integrates recent approaches for system design (model-based systems engineering), safety analysis (system-theoretic process analysis), and safety assurance (goal structuring notation). The framework is implemented in the systems modeling language (SysML), and the focus is placed on the connection between the safety analysis and safety assurance activities. It is shown how the model-based integration enables tool assistance for the systematic creation, analysis, and maintenance of safety artifacts. The framework is demonstrated with the system design, safety analysis, and safety assurance of a collision avoidance system for aircraft. The model-based nature of the design and safety activities is utilized to support the systematic generation, analysis, and maintenance of safety artifacts.

The development of Artificial Intelligence (AI) based systems is becoming increasingly prominent in various industries. The aviation industry is also gradually adopting AI-based systems. An example could be using Machine Learning algorithms for flight assistance. There are several reasons why adopting these technologies poses additional obstacles in aviation compared to other industries. One reason is strong safety requirements, which lead to obligatory assurance activities such as thorough testing to obtain certification. Amongst many other technical challenges, a systematic approach is needed for developing, deploying, and assessing test cases for AI-based systems in aviation. This paper proposes a method for iterative scenario-based testing for AI-based systems. The method contains three major parts: First, a high-level description of test scenarios; second, the generation and execution of these scenarios; and last, monitoring of scenario parameters during scenario execution. The scenario parameters, which can be for instance environmental or system parameters, are refined and the test steps are executed iteratively. The method forms a basis for developing iterative scenario-based testing solutions. As a domain-specific example, a practical implementation of this method is illustrated. For an object detection application used on an airplane, flight scenarios, including multiple airplanes, are generated from a descriptive scenario model and executed in a simulation environment. The parameters are monitored using a custom Operational Design Domain monitoring tool and refined in the process of iterative scenario generation and execution. The proposed iterative scenario-based testing method helps in generating precise test cases for AI-based systems while having a high potential for automation.

In recent years, there has been significant progress in Artificial Intelligence (AI), leading to an increasing interest for integration of AI-based functions into newly developed systems. AI promises several benefits, amongst others, beyond the state-of-the-art functions and performance. However, the use of AI-techniques also introduces new challenges regarding safety and security of systems and their certification. These challenges mostly originate from the "black box nature" of complex AI algorithms. To tackle the challenges, safety of the AI-based systems has to be addressed throughout the entire development and life cycle of the system. The adaption of existing methods to the development of AI-based systems is necessary. An established method for the development of complex systems is Model-Based Systems Engineering (MBSE), which offers several advantages for the systems engineering process. In this paper three application examples of how MBSE can support the engineering process of AI-based systems are presented using an application use case: An AI-based threat localization system. First, a systematic development framework is used to design and model the AI-based system. Second, it is demonstrated how safety analysis can be integrated into a model of the system to identify potentially hazardous scenarios, which could arise, for example, due to erroneous predictions by an AI. For the analysis, an approach called Model-Based STPA is utilized which is based on the System-Theoretic Process Analysis. Third, it is demonstrated how MBSE can help in performing scenario-based safety assessment. From the operational domain model, executable configurations are generated to run scenario-based test cases.