Michael Wagner's research while affiliated with Carnegie Mellon University and other places

... As a result, the argument is structured around what evidence is expected at that point in time, as if this is the final state of what will ever be known about the system by engineers responsible for assuring the system. However, modern assurance cases are rarely one-off exercises and stakeholders may anticipate that more information will become available, such as field data, that could strengthen (or weaken) the validity of the assurance case argument (Koopman & Wagner, 2020). This means that confidence in the assurance case will change (and hopefully increase) as further evidence becomes available. ...

... Katz et al. [97] present a robustness testing technique for unmanned autonomous systems, whose objective is to nd software faults in deep parts of complex systems. To do so, it tests internal units rst and works towards the outside to understand if faults can be activated from the outside of the system. ...

... .) the set of activities, means, and methods that shall be considered, throughout the lifecycle of a system, to produce results towards building arguments that confidently support the safety requirements/targets of such a system have been met" [1][2][3][4][5][6][7]. This is taken into account not only at the design time but also throughout operation, notably when online learning is at play [1,[8][9][10][11][12]. ...

... To fulfill safety requirements [73], it is crucial to not only produce accurate segments, but even more so make reliable predictions in the face of perturbations [55], distribution shifts [61, 112,125], uncommon situations [139] and out-ofdistribution (OOD) objects [14,53]. Modern Deep Neural Networks (DNNs) achieve impressive performance in segmentation tasks [3,27,49,92,106]; however, they struggle to generalize to data samples not seen during training, e.g., image corruptions [55,100], adversarial attacks [26,120], or change of style [40]. In the face of such events, they often produce overconfident probability estimates [47,52,89] even when they are wrong, which can impede the detection of failure modes and thus their adoption in the industry. ...

... Convolutional neural networks (CNNs) are a representative type of DNN and are used in various fields, such as image classification [3,4], face recognition [5], and video processing [6], as well as in safety-critical systems [7][8][9], such as those of autonomous vehicles, in which robustness is very important [10][11][12]. ...

... From a conceptual standpoint, contributions 1 and 2 stem from Safety ArtISt clearly identifying recommended design and V&V practices for several AI variant methods, thus going beyond technology-agnostic safety assurance approaches for AI-based systems, such as the ANSI UL4600:2020 [34], as well as research efforts focused on specific application domains [8,37,40] or AI variants [36][37][38][39][40][41]. Moreover, its cost-effectiveness is reflected in the subdivision of its V&V activities into two steps: "AI Preliminary V&V", which allows for identifying blocking safety issues with simplified yet important analyses of AI performance and fault tolerance sampling, and "AI Detailed V&V", whose costly, exhaustive analyses are triggered only if the "AI Preliminary V&V" is successful. ...

... To test for safety-critical or rare, low-probability adverse driving scenarios and corner cases, one must contend with certain risks and substantial costs and time. Even when there is an opportunity to test rare combinations of factors such as weather, lighting, traffic conditions, and sensor noise corruption that lead to failures in autonomous perception systems [3][4][5], replicating the same real-world testing conditions remains challenging and infeasible. ...

... The potential benefits of using FI into design phases of autonomous systems range from providing early opportunities for integration of inductive technologies-e.g., machine learning algorithms that use training sets to derive models of camera lens-to reducing costs and risks associated to autonomy functions. Such techniques have already been used successfully to find and characterize defects on autonomous vehicles [36]. ...

... In safety-critical systems such as autonomous driving, it is crucial to establish as much as possible about real-world performance prior to real-world deployment. High-severity, low-probability failures are especially important to capture and characterise as these are the ones most likely to be missed during standard development and testing [36]. ...

... Much of the complexity of modern vehicles comes because the functions enabled by all this software and computing power are highly interconnected; integrating streams of sensor data, mechanical actuators and ADAS including anti-lock brakes, forward collision mitigation, adaptive cruise control and blind spot detection with information delivered to, and inputs from, the driver (Buckl et al. 2012). The current shift toward electric (EV) and self-driving (SDV) vehicles is only increasing that complexity, which some argue makes the removal of the human driver from the loop not just a possibility or desirable, but a necessity (For a discussion, Wagner and Koopman 2015;Pelliccione et al. 2017). Crucially, not all of the integration that we observe is necessary just for driving the car. ...