# Matthew O'Kelly's research while affiliated with University of Pennsylvania and other places

## Publications (21)

Article
Autonomous vehicles (AVs) are already driving on public roads around the US; however, their rate of deployment far outpaces quality assurance and regulatory efforts. Consequently, even the most elementary tasks, such as automated lane keeping, have not been certified for safety, and operations are constrained to narrow domains. First, due to the li...
Preprint
Learning-based methodologies increasingly find applications in safety-critical domains like autonomous driving and medical robotics. Due to the rare nature of dangerous events, real-world testing is prohibitively expensive and unscalable. In this work, we employ a probabilistic approach to safety evaluation in simulation, where we are concerned wit...
Preprint
Balancing performance and safety is crucial to deploying autonomous vehicles in multi-agent environments. In particular, autonomous racing is a domain that penalizes safe but conservative policies, highlighting the need for robust, adaptive strategies. Current approaches either make simplifying assumptions about other agents or lack robust mechanis...
Article
Teaching autonomous systems is challenging because it is a rapidly advancing cross-disciplinary field that requires theory to be continually validated on physical platforms. For an autonomous vehicle (AV) to operate correctly, it needs to satisfy safety and performance properties that depend on the operational context and interaction with environme...
Preprint
While autonomous vehicle (AV) technology has shown substantial progress, we still lack tools for rigorous and scalable testing. Real-world testing, the $\textit{de-facto}$ evaluation method, is dangerous to the public. Moreover, due to the rare nature of failures, billions of miles of driving are needed to statistically validate performance claims....
Chapter
Full-text available
The testing of Autonomous Vehicles (AVs) requires driving the AV billions of miles under varied scenarios in order to find bugs, accidents and otherwise inappropriate behavior. Because driving a real AV that many miles is too slow and costly, this motivates the use of sophisticated ‘world simulators’, which present the AV’s perception pipeline with...
Preprint
Full-text available
In 2005 DARPA labeled the realization of viable autonomous vehicles (AVs) a grand challenge; a short time later the idea became a moonshot that could change the automotive industry. Today, the question of safety stands between reality and solved. Given the right platform the CPS community is poised to offer unique insights. However, testing the lim...
Preprint
Modern treatments for Type 1 diabetes (T1D) use devices known as artificial pancreata (APs), which combine an insulin pump with a continuous glucose monitor (CGM) operating in a closed-loop manner to control blood glucose levels. In practice, poor performance of APs (frequent hyper- or hypoglycemic events) is common enough at a population level tha...
Preprint
While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the $\textit{de facto}$ evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of miles in order to statistically validate...
Article
2018 Curran Associates Inc.All rights reserved. While recent developments in autonomous vehicle (AV) technology highlight substantial progress, we lack tools for rigorous and scalable testing. Real-world testing, the de facto evaluation environment, places the public in danger, and, due to the rare nature of accidents, will require billions of mile...
Article
Full-text available
This article elaborates the approaches that can be used to verify an autonomous vehicle (AV) before giving it a driver’s license. Formal methods applied to the problem of AV verification include theorem proving, reachability analysis, synthesis, and maneuver design. Theorem proving is an interactive technique in which the computer is largely respon...
Article
Full-text available
The testing of Autonomous Vehicles (AVs) requires driving the AV billions of miles under varied scenarios in order to find bugs, accidents and otherwise inappropriate behavior. Because driving a real AV that many miles is too slow and costly, this motivates the use of sophisticated `world simulators', which present the AV's perception pipeline with...
Conference Paper
Full-text available
Article
Full-text available
This paper details the design of an autonomous vehicle CAD toolchain, which captures formal descriptions of driving scenarios in order to develop a safety case for an autonomous vehicle (AV). Rather than focus on a particular component of the AV, like adaptive cruise control, the toolchain models the end-to-end dynamics of the AV in a formal way su...
Conference Paper
Full-text available
Relaxed notions of decidability widen the scope of automatic verification of hybrid systems. In quasi-decidability and δ-decidability, the fundamental compromise is that if we are willing to accept a slight error in the algorithm's answer, or a slight restriction on the class of problems we verify, then it is possible to obtain practically useful a...
Article
Full-text available
Autonomous vehicles (AVs) have already driven millions of miles on public roads, but even the simplest scenarios have not been certified for safety. Current methodologies for the verification of AV's decision and control systems attempt to divorce the lower level, short-term trajectory planning and trajectory tracking functions from the behavioral...
Article
Full-text available
Diabetes associated complications are affecting an increasingly large population of hospitalized patients. Since glucose physiology is significantly impacted by patient-specific parameters, it is critical to verify that a clinical glucose control protocol is safe across a wide patient population. A safe protocol should not drive the glucose level i...

## Citations

... AdvSim [18] benchmarks several black-box optimization algorithms to search adversarial trajectories to obtain safety-critical scenarios for the full autonomy stack, but its adversarial objective is targeted for planning, and it is designed for the single vehicle system. Another stream of work [31], [32] formulates the scenario generation problem as the rare event simulation to sample failure scenarios for the single-agent autonomy system. In contrast, we focus on producing challenging scenarios for the LiDAR-based multiagent V2X perception system where both the agents' poses and the selection of collaborators are searched to optimize an adversarial objective customized for perception. ...
... Motorsport racing has proven to enable knowledge transfer of cutting-edge research to the automotive industry [1,2,3]. In particular, autonomous racing presents a new frontier that promises to revolutionize autonomous driving by enabling and stress-testing new technologies and algorithms in the field of Self Driving Cars (SDC) [4,5,6,7]. For this reason, many autonomous racing competitions have recently emerged, featuring different platforms and form-factors, from full-scaled Indy Autonomous [6] and Formula Student Driverless [8] to scaled F1TENTH [5,9]. ...
... In an attempt to push the limits towards the development of new technologies, numerous competitions are organized and held in major international conferences. Above all, the F1/10 Autonomous Racing competition [2], [3] is one of the most popular; its name derives from the use of 1:10 scaled-down car models. Depending on the task objective, the problem is faced with different levels of detail and approximations [4], [5]. ...
... Recent courses on edX share similar motivations as ours but differ in the selection of topics [23], [24]. It is also worth mentioning broader open-source initiatives, such as the F1Tenth initiative [25] and BWSI (Beaver Works Summer Institute) [26], that provide introductory-level courses for seniors and K-12 students. ...
... Park et al. [41] developed scenarios for evaluating safety measures during take-over situations on virtually simulated highways. Similarly, Abbas et al. [42] demonstrated various dangerous situations during autonomous driving through a virtual simulator based on the Grand Theft Auto V game. Recently, simulation methods have been studied in conjunction with AI technologies. ...
... Research on the safety of autonomous vehicles (AVs) has so far focused on evaluating their impacts using simulation models (Morando et al. 2018;Papadoulis, Quddus, and Imprialou 2019), computational analysis (Kalra and Paddock 2016), scenario planning (Millard-Ball 2016), public perceptions (Moody, Bailey, and Zhao 2020), communication networks (Hussain and Zeadally 2019), liability and privacy issues (Lim and Taeihagh 2018), and predictions for potential economic savings (Clements and Kockelman 2017). Most of the existing literature that examined the safety implications of AVs has adopted modelling and quantitative approaches used in the engineering and mathematics fields (Morando et al. 2018;Papadoulis, Quddus, and Imprialou 2019;Abbas et al. 2019). Research by Kassens-Noor et al. (2020) shows that there are over 100,000 engineering articles on AVs, compared with only 200 articles covering the planning aspects. ...
... Second, formal specifications enable automated testing and monitoring for AV, e.g, see [8]- [15], for requirements based testing. Third, formal specifications on the perception system can also function as a requirements language between original equipment manufacturers (OEM) and suppliers. ...
... Testing scenario search and generation has also been widely applied to the testing and verification of autonomous systems [20], [21], [22], [23], [24]. Some approaches employ optimization or adaptive sampling techniques to accelerate finding test cases with highest risk to the system [25], [26], [27], [28], [29], [30], [31]. ...
... For example, manufacturers have used the technology to make dental implants (Dawood et al., 2015) or even bone tissues (Bose et al., 2013). CAD tools have also evolved with regards to visualisation features, with photo-realistic renderings becoming more commonplace, for instance to better picture violations of requirements in autonomous vehicle safety assessment (O'Kelly et al., 2017) or to simulate coating appearance depending on lightning (Jhamb et al., 2020). ...
... The idea behind [13] is that we can rst search the set of behaviors to nd those executions with low robustness. Assuming continuity of behavior, low-robustness executions are surrounded by other low-robustness executions, and possibly by executions with negative robustness (Figure 4). ...