Laura Humphrey's research while affiliated with Air Force Research Laboratory and other places

Publications (21)

Article
Full-text available
Subjective psychometric ratings are critical tools in human factors engineering. Despite widespread use, there is controversy about whether such ratings can be treated at cardinal levels of measurement. Answering this question is important given that a measure’s level determines what mathematical and statistical operations can be meaningfully appli...
Article
Full-text available
Situation awareness (SA), a measure of how well a person understands the situation, is frequently used to evaluate the safety and effectiveness of critical systems that depend on human behavior. While there are objective ways of measuring SA, subjective assessments, such as the SA rating technique (SART), are still widely used. However, it is not c...
Conference Paper
Sophisticated software is enabling new capabilities across a number of different domains, from driverless cars in the automotive industry, to embedded medical devices in the healthcare industry, to autonomous drones in the aerospace industry. Given the safety implications, there is a need for software in these domains to be highly reliable. However...
Chapter
Manually designing control logic for reactive systems is time-consuming and error-prone. An alternative is to automatically generate controllers using “correct-by-construction” synthesis approaches. Recently, there has been interest in synthesis from Generalized Reactivity(1) or GR(1) specifications, since the required computational complexity is r...
Article
Full-text available
Psychometrics are increasingly used to evaluate trust in the automation of systems, many of them safety-critical. There is no consensus on what the highest level of measurement is for trust. This is important as the level of measurement determines what mathematics and statistics can be meaningfully applied to ratings. In this work, we introduce a n...
Article
Full-text available
Psychometrics are increasingly being used to evaluate trust in the automation of safety-critical systems. There is no consensus on what the highest level of measurement is for psychometric trust. This is important as the level of measurement determines what mathematics and statistics can be meaningfully applied to ratings. In this work, we introduc...
Chapter
This paper presents initial, positive results from using SPARK to prove critical properties of OpenUxAS, a service-oriented software framework developed by AFRL for mission-level autonomy for teams of cooperating unmanned vehicles. Given the intended use of OpenUxAS, there are many safety and security implications; however, these considerations are...
Chapter
Full-text available
Designing protocols for multi-agent interaction that achieve the desired behavior is a challenging and error-prone process. The standard practice is to manually develop proofs of protocol correctness that rely on human intuition and require significant effort to develop. Even then, proofs can have mistakes that may go unnoticed after peer review, m...
Chapter
We introduce the concept of structured synthesis for Markov decision processes. A structure is induced from finitely many pre-specified options for a system configuration. We define the structured synthesis problem as a nonlinear programming problem (NLP) with integer variables. As solving NLPs is not feasible in general, we present an alternative...
Preprint
Full-text available
Shield synthesis is an approach to enforce a set of safety-critical properties of a reactive system at runtime. A shield monitors the system and corrects any erroneous output values instantaneously. The shield deviates from the given outputs as little as it can and recovers to hand back control to the system as soon as possible. This paper takes it...
Article
Full-text available
Symbolic motion planning for robots is the process of specifying and planning robot tasks in a discrete space, then carrying them out in a continuous space in a manner that preserves the discrete-level task specifications. Despite progress in symbolic motion planning, many challenges remain, including addressing scalability for multi-robot systems...
Preprint
Full-text available
Symbolic motion planning for robots is the process of specifying and planning robot tasks in a discrete space, then carrying them out in a continuous space in a manner that preserves the discrete-level task specifications. Despite progress in symbolic motion planning, many challenges remain, including addressing scalability for multi-robot systems...
Preprint
Full-text available
We introduce the concept of structured synthesis for Markov decision processes where the structure is induced from finitely many pre-specified options for a system configuration. The resulting synthesis problem is in general a nonlinear programming problem (NLP) with integer variables. As solving NLPs is in general not feasible, we present an alter...
Article
Full-text available
Shield synthesis is an approach to enforce safety properties at runtime. A shield monitors the system and corrects any erroneous output values instantaneously. The shield deviates from the given outputs as little as it can and recovers to hand back control to the system as soon as possible. In the first part of this paper, we consider shield synthe...
Conference Paper
Shield synthesis is an approach to enforce a set of safety-critical properties of a reactive system at runtime. A shield monitors the system and corrects any erroneous output values instantaneously. The shield deviates from the given outputs as little as it can and recovers to hand back control to the system as soon as possible. This paper takes it...
Article
We propose an approach to synthesize control protocols for autonomous systems that account for uncertainties and imperfections in interactions with human operators. As an illustrative example, we consider a scenario involving road network surveillance by an unmanned aerial vehicle (UAV) that is controlled remotely by a human operator but also has a...
Conference Paper
Full-text available
We propose an approach to synthesize control protocols for autonomous systems that account for uncertainties and imperfections in interactions with human operators. As an illustrative example, we consider a scenario involving road network surveillance by an unmanned aerial vehicle (UAV) that is controlled remotely by a human operator but also has a...

Citations

... However, these abstract statuses are usually difficult to quantify. Many studies have applied self-reported surveys for assessing different aspects of the cognitive status, such as the National Aeronautics and Space Administration Task Load Index (NASA-TLX) [13] and the Rating Scale Mental Effort (RSME) [26] for assessing mental workload, the Dundee Stress State Questionnaire (DSSQ) [27] for assessing task engagement and distraction, the Karolinska Sleepiness Scale (KSS) [28] for assessing fatigue and drowsiness, and the Situation Awareness Rating Technique (SART) [29] for assessing situational awareness. These instruments have been adopted and validated in many studies and provide a solution to quantify cognitive statuses. ...
... Although model trustworthiness is an on-going area of research in artificial intelligence and computer science literature (Chang et al., 2005;Wei et al., 2020;Markus et al., 2021), trust was not measured at any point in this thesis. Measuring the trustworthiness of a model has traditionally been done through surveys, resulting in psychometric measures of perceived trust (Miller, 2022). ...
... This inherently makes trust measurement and modeling controversial. There is an ongoing debate about the validity of psychometric measures: whether they are real or merely artificial/folk constructs (Annett, 2002;Dekker & Hollnagel, 2004;Dekker & Nyce, 2015); whether they converge with objective measures (Dekker & Nyce, 2015;Matthews et al., 2020); and whether they satisfy the requirements for cardinal levels of measurement (Bolton et al. n.d.-a, n.d.-b;Wei et al., 2019Wei et al., , 2020. ...
... These proposed ideas cover a wide range of possibilities such as allowing UAVs to land at vertistops or vertiports installed on roofs of existing buildings or within cloverleaf exchanges on freeways. Several simulation tools [9]- [11] have also been developed. ...
... Linear temporal logic (LTL) is a well known notation for specifying concurrent behavior [19]. It has found several applications to robotics, including in motion planning [25] and multi-robot coordination [2], [4]. (These use either the GR(1) fragment or finite LTL, which have good synthesis algorithms.) ...
... In [9] the research discusses a method to perform run-time assurance for learning systems with an assurance architecture designed in Architecture Analysis and Design Language (AADL) and formal contracts for each of the components modeled and verified in Assume Guarantee (AGREE) annex. In [12] Davis discusses an approach to use architectural analysis to prove that the protocol designed for multi-agents satisfies the specified properties. This effort also emphasized the use of AADL and AGREE for formal assurance. ...
... To simplify matters, assume that the airplane (P) is tracking the movement of one vehicle (V) that is about to cross the runway. Let us further assume that P tracks V using a perception module that can only determine the position of the vehicle with a certain accuracy [33], i.e., for every position of V, the perception module reports a noisy variant of the position of V. However, it is important to know that the plane obtains a sequence of these measurements. ...
... For example, human trust has been investigated in a multiagent setting in several studies, e.g. (Williams et al., 2021;Wang et al., 2018). However, there is still work to be done in understanding how trust is affected when robots differ in levels of autonomy, and exhibit varied performance and failure rates. ...
... However, The k-stabilizing shield synthesis problem is unrealizable for many safety-critical systems, because a finite number of deviations cannot be guaranteed. Humphrey et al. [34] addressed this problem by proposing the notion of admissible shields which was extended and generalized in Könighofer et al. [35] by assuming that systems have a cooperative behavior with respect to the shield, i.e., the shield ensures a finite number of deviations if the system chooses certain outputs. The authors presented a synthesis procedure that maximizes the cooperation between system and environment for satisfying the required enforced properties. ...
... Consequently, robot systems are still in their infancy in developing the ability to explain their own behavior when confronting noisy sensory inputs and executing complex multistep decision processes. Planner-based robot systems can generally provide an interpretable account for their actions to humans [e.g., by Markov decision processes (7,8), HTN (9), or STRIPS (10)], but these planners struggle to explain how their symbolic-level knowledge is derived from low-level sensory inputs. In contrast, robots equipped with deep neural networks (DNNs) (11) have demonstrated impressive performance in certain specific tasks due to their powerful ability to handle low-level noisy sensory inputs (12,13). ...