John Rushby's research while affiliated with SRI International and other places

Publications (146)

Preprint
Full-text available
Shared intentionality is a critical component in developing conscious AI agents capable of collaboration, self-reflection, deliberation, and reasoning. We formulate inference of shared intentionality as an inverse reinforcement learning problem with logical reward specifications. We show how the approach can infer task descriptions from demonstrati...
Preprint
Full-text available
I use mechanized verification to examine several first- and higher-order formalizations of Anselm's Ontological Argument against the charge of begging the question. I propose three different but related criteria for a premise to beg the question in fully formal proofs and find that one or another applies to all the formalizations examined. I also s...
Preprint
Full-text available
We use a mechanized verification system, PVS, to examine the argument from Anselm's Proslogion Chapter III, the so-called "Modal Ontological Argument." We consider several published formalizations for the argument and show they are all essentially similar. Furthermore, we show that the argument is trivial once the modal axioms are taken into accoun...
Preprint
Full-text available
Modal logics allow reasoning about various modes of truth: for example, what it means for something to be possibly true, or to know that something is true as opposed to merely believing it. This report describes embeddings of propositional and quantified modal logic in the PVS verification system. The resources of PVS allow this to be done in an at...
Preprint
Full-text available
An assurance case is intended to provide justifiable confidence in the truth of its top claim, which typically concerns safety or security. A natural question is then "how much" confidence does the case provide? We argue that confidence cannot be reduced to a single attribute or measurement. Instead, we suggest it should be based on attributes that...
Chapter
Ideally, assurance enables us to know that our system is safe or possesses other attributes we care about. But full knowledge requires omniscience, and the best we humans can achieve is well-justified belief. So what justification should be considered adequate for a belief in safety? We adopt a criterion from epistemology and argue that assurance s...
Chapter
I use mechanized verification to examine several first- and higher-order formalizations of Anselm’s Ontological Argument against the charge of begging the question. I propose three different but related criteria for a premise to beg the question in fully formal proofs and find that one or another applies to all the formalizations examined. I also s...
Article
We use a mechanized verification system, PVS, to examine the argument from Anselm’s Proslogion Chapter III, the so-called “Modal Ontological Argument.” We consider several published formalizations for the argument and show they are all essentially similar. Furthermore, we show that the argument is trivial once the modal axioms are taken into accoun...
Chapter
The functions of an autonomous system can generally be partitioned into those concerned with perception and those concerned with action. Perception builds and maintains an internal model of the world (i.e., the system’s environment) that is used to plan and execute actions to accomplish a goal established by human supervisors.
Preprint
System assurance is confronted by significant challenges. Some of these are new, for example, autonomous systems with major functions driven by machine learning and AI, and ultra-rapid system development, while others are the familiar, persistent issues of the need for efficient, effective and timely assurance. Traditional assurance is seen as a br...
Conference Paper
An assurance case provides a structured argument to establish a claim for a system based on evidence about the system and its environment. I propose a simple interpretation for the overall argument that uses epistemic methods for its evidential or leaf steps and logic for its reasoning or interior steps: evidential steps that cross some threshold o...
Conference Paper
Patients in intensive care often have a dozen or more medical devices and sensors attached to them. Each is a self-contained system that operates in ignorance of the others, and their integrated operation as a system of systems that delivers coherent therapy is performed by doctors and nurses. But we can easily imagine a scenario where the devices...
Conference Paper
An assurance case provides an argument that certain claims (usually concerning safety or other critical properties) are justified, based on given evidence concerning the context, design, and implementation of a system. An assurance case serves two purposes: reasoning and communication. For the first, the argument in the case should approach the sta...
Article
Full-text available
To become practical for assurance, automated for- mal methods must be made more scalable, automatic, and cost-effective. Such an increase in scope, scale, au- tomation, and utility can be derived from an emphasis on a systematic separation of concerns during verification. SAL (Symbolic Analysis Laboratory) attempts to address these issues. It is a...
Article
Aircraft automation designers are faced with the challenge to develop and improve automation such that it is transparent to the pilots using it. To identify problems that may arise between pilots and automation, methods are needed that can uncover potential problems with automation early in the design process. In this paper, simulation and model ch...
Article
Full-text available
We propose to validate experimentally a theory of software certification that proceeds from assessment of confidence in fault-freeness (due to standards) to conservative prediction of failure-free operation.
Conference Paper
We present an approach for detecting sensor spoofing attacks on a cyber-physical system. Our approach consists of two steps. In the first step, we construct a safety envelope of the system. Under nominal conditions (that is, when there are no attacks), the system always stays inside its safety envelope. In the second step, we build an attack detect...
Article
A synchronous observer is an adjunct to a system model that monitors its state variables and raises a signal flag when some condition is satisfied. Synchronous observers provide an alternative to temporal logic as a means to specify safety properties but have the advantage that they are expressed in the same notation as the system model-and thereby...
Conference Paper
To identify problems that may arise between pilots and automation, methods are needed that can uncover potential problems with automation early in the design process. Such potential problems include automation surprises, which describe events when pilots are surprised by the actions of the automation. In this work, agent-based, hybrid time simulati...
Conference Paper
A safety case must resolve concerns of two different kinds: how complete and accurate is our knowledge about aspects of the system (e.g., its requirements, environment, implementation, hazards) and how accurate is our reasoning about the design of the system, given our knowledge. The first of these is a form of epistemology and requires human exper...
Conference Paper
A synchronous observer is an adjunct to a system model that monitors its state variables and raises a signal when some condition is satisfied. Synchronous observers provide an alternative to temporal logic as a means to specify safety properties but have the benefit that they are expressed in the same notation as the system model. Model checkers th...
Article
Full-text available
This paper refines and extends an earlier one by the first author [1]. It considers the problem of reasoning about the reliability of fault-tolerant systems with two “channels” (i.e., components) of which one, A, because it is conventionally engineered and presumed to contain faults, supports only a claim of reliability, while the other, B, by virt...
Conference Paper
Failures in component-based systems are generally due to unintended or incorrect interactions among the components. For safety-critical systems, we may attempt to eliminate unintended interactions, and to verify correctness of those that are intended. We describe the value of partitioning in eliminating unintended interactions, and of assumption sy...
Article
Full-text available
Data sanitization has been studied in the context of architectures for high assurance systems, language-based information flow controls, and privacy-preserving data publication. A range of sanitization strategies has been developed to address the wide variety of data content and contexts that arise in practice. It is therefore tempting to separate...
Article
Avionics systems in modern and next-generation airborne vehicles combine and integrate various realtime applications to efficiently share the physical resources on board. Many of these real-time applications also need to fulfill fault-tolerance requirements — i.e., the applications have to provide a sufficient level of service even in presence of f...
Conference Paper
Full-text available
Breakdowns in complex systems often occur as a result of system elements interacting in ways unanticipated by analysts or designers. The use of task behavior as part of a larger, formal system model is potentially useful for analyzing such problems because it allows the ramifications of different human behaviors to be verified in relation to other...
Conference Paper
I outline the principal ideas of the Distributed Secure System (DSS) on which Brian Randell and I collaborated in the early 1980s, its modern manifestation as MILS, and continuing research challenges posed by these architectures.
Chapter
Suitable formalisms could allow the arguments of a safety case to be checked mechanically. We examine some of the issues in doing so.
Conference Paper
Littlewood introduced the idea that software may be possibly perfect and that we can contemplate its probability of (im)perfection. We review this idea and show how it provides a bridge between correctness, which is the goal of software verification (and especially formal verification), and the probabilistic properties such as reliability that are...
Article
We examine the problem of selecting a best value from a collection of sensor readings, and diagnosing faulty readings in such a collection. We focus on sensor interfaces that return a range of values and describe the "fusion functions" f;n (S) of Marzullo and F of Schmid and Schossmaier. We use PVS formally to prove the soundness of f;n (S) (i.e.,...
Article
Requirements engineering for information security poses two main challenges: eliciting what are the requirements for a particular system, and figuring out how to specify them in a way that is both perspicuous (to the problem owner) and useful (to the developer). In this short note, I look at some of the challenges in how to specify security require...
Article
Adaptive systems-those that can change their behavior at runtime-pose new challenges for certification, and particularly for traditional, standards-based methods of certification such as DO-178B. These traditional methods are effective in conservative fields because they can establish a solid basis in experience and can incorporate the lessons lear...
Conference Paper
Full-text available
To achieve the vision of information superiority, secure and timely sharing of information is needed between geographically separated platforms and users. However, often the producers and consumers of the information, as well as the information itself are separated in different security domains. A COTS marketplace of composable, high assurance comp...
Article
Formal methods offer the unique benefit that it is possible to examine all possible circumstances within a given scope, rather than merely sample them as with testing and simulation. This is possible even when huge numbers of discrete possibilities must be considered, such as in fault scenarios, and in the presence of real-time and continuous behav...
Conference Paper
By their very nature, loss of control accidents are unanticipated and rare, and their precursors are rare also. Onboard systems to detect and mitigate these precursors must work - and work correctly - when required but must not introduce new malfunctions or unintended functions. How can we provide assurance that software invoked in such rare and un...
Conference Paper
Full-text available
The early 1980s saw the development of some rather sophisticated distributed systems. These were not merely networked file systems: rather, using remote procedure calls, hierarchical naming, and what would now be called middleware, they allowed a collection of systems to operate as a coherent whole. One such system in particular was developed at Ne...
Article
Timed systems are notoriously hard to de- bug and to verify because the continuous nature of time allows vast numbers of different behaviors; em- bedded systems must often deal with faults, and these introduce another dimension of complexity. Simula- tion and testing provide little assurance in these do- mains because they can visit only a small fr...
Conference Paper
Traditional, standards-based approaches to certification are hugely expensive, of questionable credibility when development is outsourced, and a barrier to innovation. This paper is a call and a manifesto for new approaches to certification. We start by advocating a goal-based approach in which unconditional claims delivered by formal methods are c...
Conference Paper
The world at large cares little for verified software; what it cares about are trustworthy and cost-effective systems that do their jobs well. We examine the value of verified software and of verification technology in the systems context from two perspectives, one analytic, the other synthetic. We propose some research opportunities that could enh...
Chapter
Full-text available
A group membership protocol is presented and proven correct for a synchronous time-triggered model of computation with processors in a ring that broadcast in turn. The protocol, derived from one used for critical control functions in automobiles, accepts a very restrictive fault model to achieve low overhead and requires only one bit of membership...
Chapter
Full-text available
We have argued previously that the effectiveness of a verification system derives not only from the power of its individual features for expression and deduction, but from the extent to which these capabilities are integrated: the whole is more than the sum of its parts [19,21]. Here, we illustrate this thesis by describing a simple construct for t...
Conference Paper
Hybrid systems are at the core of most embedded and many other kinds of systems; formal methods for analysis of hybrid systems have made remarkable progress in the last decade and thus provide a strong foundation for assurance in the system core. But there are many systems issues that interact with the hybrid systems core and complicate the overall...
Conference Paper
Modern verification systems such as PVS are now reaching the stage of development where the formal verification of critical algorithms is feasible with reasonable effort. This paper describes one such verification in the field of fault tolerance. The distribution of single-source data to replicated computing channels (Interactive Consistency or Byz...
Article
Full-text available
The eorts of researchers over the past 20 years has yielded an impressive array of verification tools. However, no single tool or method is going to solve the verification problem. An entire spectrum of formal methods and tools are needed ranging from test case generators, static analyzers, and type checkers, to invariant generators, decision pro-...
Article
Abstract We describe sal-atg, a tool for automated test generation that will be distributed as part of the next release of SAL. Given a SAL specification augmented,with Boolean trap variables representing test goals, sal-atg generates an efficient set of tests to drive the trap variables to TRUE; SAL specifications are typically instrumented with t...
Article
s, linear arithmetic, and lists. The ground (i.e., quanti er-free) fragment of many combinations is decidable when the fully quanti ed combination is not, and practical experience indicates that automation of the ground case is adequate for most applications. Practical experience also suggests several other desiderata for an eective deductive servi...
Article
It is well-known that counterexamples produced by model checkers can provide a basis for automated generation of test cases. However, when this approach is used to meet a coverage criterion, it generally results in very inefficient test sets having many tests and much redundancy. We describe an improved approach that uses model checkers to generate...
Article
Secure systems are often built around a "security kernel"—a relatively small and simple component that guarantees the security of the overall system. In this paper we ask whether this approach can be used to ensure system properties other than security—in particular, we are interested in whether "safe ty" properties can be han- dled in this way. Ou...
Article
This paper reviews some of the diculties that arise in the veri cation of kernelized secure systems and suggests new techniques for their resolution.
Article
This report was prepared to supplement a forthcoming chapter on formal methodsin the FAA Digital Systems Validation Handbook1. Its purpose is to outlinethe technical basis for formal methods in computer science, to explain the use offormal methods in the specification and verification of software and hardware requirements,designs, and implementatio...
Article
SAL 2 augments the specification language and explicit-state model checker of SAL 1 with high-performance symbolic and bounded model checkers, and with novel infinite bounded and witness model checkers.
Conference Paper
We present a formal operational semantics for Stateflow, the graphical Statecharts-like language of the Matlab/Simulink tool suite that is widely used in model-based development of embedded systems. Stateflow has many tricky features but our operational treatment yields a surprisingly simple semantics for the subset that is generally recommended fo...
Article
The increasing performance of modern model-checking tools offers high potential for the computer-aided design of fault-tolerant algorithms. Instead of relying on human imagination to generate taxing failure scenarios to probe a fault-tolerant algorithm during development, we define the fault behavior of a faulty process at its interfaces to the rem...
Article
Full-text available
SAL 2 augments the specification language and explicit-state model checker of SAL 1 with high-performance symbolic and bounded model checkers, and with novel infinite bounded and witness model checkers. The bounded model checker can use several di#erent SAT solvers, while the infinite bounded model checker similarly can use several different ground...
Article
The project originally focused on compositional formal methods for aspect-oriented programs and was located in the PCES program. Soon after its inception, however, the project was moved to the SEC (Software Enabled Control) program where its focus shifted to formal analysis of mixed discrete/continuous (i.e., hybrid) systems. We developed a two-ste...
Article
The increasing performance of modern model-checking tools offers high potential for the computeraided design of fault-tolerant algorithms. Instead of relying on human imagination to generate taxing failure scenarios to probe a fault-tolerant algorithm during development, we define the fault behavior of a faulty process at its interfaces to the rema...
Article
The Needham-Schroeder authentication protocol is specified in SAL and its model checker is used to detect the flaw discovered by Gavin Lowe. The SAL simulator is used to further explore the model of the protocol. This provides a simple illustration in the use of SAL for this domain.
Article
This paper shows that deterministic consensus in synchronous distributed systems with link faults is possible, despite the impossibility result of (Gray, 1978). Instead of using randomization, we overcome this impossibility by moderately restricting the inconsistency that link faults may cause system-wide. Relying upon a novel hybrid fault model th...
Article
The emergence of complex safety-critical systems such as integrated modular avionics challenges the limits of traditional process-based assurance methods based on what DO-178B calls "reviews." The problem is that interaction between subsystems produces too many possible behaviors. Assurance for these cases needs support from what DO178B calls "anal...
Article
We formally specify the TTA startup algorithm of Paulitsch and Steiner in the SAL language. Using the SALenv model checker, we confirm that the algorithm succeeds in starting the cluster after at most one collision. We explore alternative algorithms and system parameters, and investigate behavior in the presence of faults. The limitations of finite...
Article
Full-text available
Formal analyses can provide valuable assurance for high confidence software and systems. The analyses can range from strong typechecking through test case generation and static analysis to model checking and full verification. In all cases, the tools that support the analyses use formal deduction in some way or other. ICS is a fully automatic, high...
Article
We describe and compare the architectures of four fault-tolerant, safety-critical buses with a view to deducing principles common to all of them, the main differences in their design choices, and the tradeoffs made. Two of the buses come from an avionics heritage, and two from automobiles, though all four strive for similar levels of reliability an...
Article
The objective of this research was to develop mechanisms and methods of analysis to support construction of survivable systems where survivable means systems able to withstand multiple kinds of faults among their components, including those induced deliberately by an active attacker. One class of architectures for survivability builds on classical...
Article
Airplanes are certified as a whole: there is no established basis for separately certifying some components, particularly software-intensive ones, independently of their specific application in a given airplane. The absence of separate certification inhibits the development of modular components that could be largely "precertified" and used in seve...
Article
Important aspects of both security and safety are related to process encapsulation and controlled flow of information through known interfaces. Par- titioning refers to architectural mechanisms that enforce these attributes. In this paper, we examine formal characterizations of partitioning.
Conference Paper
We describe formal verification of some of the key algorithms in the Time-Triggered Architecture (TTA) for real-time safety-critical control applications. Some of these algorithms pose formidable challenges to current techniques and have been formally verified only in simplified form or under restricted fault assumptions. We describe what has been...
Article
Airplanes are certified as a whole: there is no established basis for separately certifying some components, particularly software-intensive ones, independently of their specific application in a given airplane. The absence of separate certification inhibits the development of modular components that could be largely "precertified" and used in seve...
Article
We examine the problem of selecting a best value from a collection of sensor readings, and diagnosing faulty readings in such a collection. We focus on sensor interfaces that return a range of values and describe the "fusion functions" f;n (S) of Marzullo and F of Schmid and Schossmaier. We use PVS formally to prove the soundness of f;n (S) (i.e.,...
Article
To illustrate some of the power and convenience of its specification language and theorem prover, we use the PVS formal verification system to verify the soundness of a proof rule for assume-guarantee reasoning due to Ken McMillan.
Article
Simpson's four-slot fully asynchronous communication mechanism allows single reader and writer processes to access a shared memory in such a way that interference between concurrent reads and writes is avoided, the reader always accesses the most recent data stored by the writer, and neither process need wait for the other. In computer science parl...
Article
Safety- and security-critical systems both require encapsulation of code and data belonging to different applications and sensitivity levels. It must be impossible for a fault or Trojan Horse in one application to affect the operation or real-time performance of another, or for information of one sensitivity level to contaminate that of another. En...
Article
Avionics and control systems for aircraft use distributed, fault-tolerant computer systems to provide safety-critical functions such as flight and engine control. These systems are becoming modular, meaning that they are based on standardized architectures and components, and integrated, meaning that some of the components are shared by different f...
Article
Automation surprises occur when an automated system behaves differently than its operator expects. If the actual system behavior and the operator's ‘mental model’ are both described as finite state transition systems, then mechanized techniques known as ‘model checking’ can be used automatically to discover any scenarios that cause the behaviors of...
Conference Paper
This paper shows that deterministic consensus in synchronous distributed systems with link faults is possible, despite the impossibility result of Gray (1978). Instead of using randomization, we overcome this impossibility by moderately restricting the inconsistency that link faults may cause system-wide. Relying upon a novel hybrid fault model tha...
Article
Embedded systems for safety-critical applications often integrate multiple "functions" and must generally be fault-tolerant. These requirements lead to a need for mechanisms and services that provide protection against fault propagation and ease the construction of distributed fault-tolerant applications. A number of bus architectures have been dev...
Article
We describe an extended fault model for Byzantine Agreement due to Schmid and Weiss. The new fault model extends the previous "hybrid" fault model of Thambidurai and Park by the addition of omission-faulty nodes, and by the introduction of link faults. We formally verify the Hybrid Oral Messages Algorithm (OMH) under this new fault model.
Conference Paper
Human operators use mental models to guide their interaction with automated systems.We can “model the human” by constructing explicit descriptions of plausible mental models. Using mechanized formal methods, we can then calculate divergences between the actual system behavior and that suggested by the mental model. These divergences indicate possib...
Article
We describe some of the design choices that should be considered in the development and application of specification languages and verification systems. A principal issue is the need to reconcile the desire for expressiveness in the specification language with the ability to provide effective mechanical support. We argue that this reconciliation is...
Article
Full-text available
Modern passenger aircraft are highly automated, and problems at the interface between the automation and the pilot are implicated in several accidents. I use a simple example taken from the autopilot of a widely used aircraft type to demonstrate how formal methods can be used to analyze some aspects of these interfaces, and to expose potential prob...
Article
Formal methods have gained acceptance in the hardware field through a pragmatic approach that has succeeded in providing systematic, scalable, highly automated, and cost-effective treatments for certain stereotypical problems of practical importance. By identifying stereotypical problems, the effort required to develop effective formal methods has...
Article
A small modification to the interactive convergence clock synchronization algorithm allows it to tolerate a larger number of simple faults than the standard algorithm, without reducing its ability to tolerate arbitrary or "Byzantine" faults. Because the extended caseanalysis required by the new fault model complicates the already intricate argument...
Article
Human operators use mental models to guide their interaction with automated systems. We can "model the human" by constructing explicit descrip- tions of plausible mental models. Using mechanized formal methods, we can then calculate divergences between the actual system behavior and that suggested by the mental model. These divergences indicate pos...
Article
Abstract We formally verify the parameters on the timing of message windows in transmitters, receivers, and bus guardians for the Time-Triggered Architecture. i ii Contents
Article
Full-text available
Recent work has shown a convergence between the Human Factors and Formal Methods communities that opens promising new directions for collaborative work in calculating, predicting, and analyzing the behavior of complex aeronautical systems and their operators. Previously it has been shown that fully automatic, finitestate verification techniques can...
Article
Model checking has won some industrial acceptance in debugging designs. Theorem proving and formal verification are less popular. An approach built around automated abstractions could integrate theorem proving with model checking in an acceptable way and provide a bridge between refutation and verification.
Article
We show that the assumptions required of the authentication mechanism in Byzantine agreement protocols that use "signed messages" are stronger than generally realized, and require more than simple digital signatures. The protocols may fail if these assumptions are violated. We then present new protocols for Byzantine agreement that add authenticati...
Conference Paper
. I describe a systematic method for deductive verification of safety properties of concurrent programs. The method has much in common with the "verification diagrams" of Manna and Pnueli [17], but derives from different intuitions. It is based on the idea of strengthening a putative safety property into a disjunction of "configurations" that can e...
Article
The challenges in using theorem proving for verification of parallel systems are to achieve adequate automation, and to allow human guidance to be expressed in terms of the system under examination rather than the mechanisms of the prover. This paper provides an overview of techniques that address these challenges.
Article
The research performed in this project enhanced the PVS formal verification system to provide improved support for the development and assurance of fault-tolerant and safety-critical systems. The capabilities developed are freely available in the publicly distributed version of PVS. In addition, new and more systematic methods were developed for ve...
Conference Paper
First Page of the Article
Conference Paper
ion, Invariant Generation, and Theorem Proving ? John Rushby Computer Science Laboratory SRI International 333 Ravenswood Avenue Menlo Park, CA 94025, USA rushby@csl.sri.com Abstract. Mechanized formal methods that use both model checking and theorem proving seem to hold most promise for the future. Effective use of both technologies requires they...
Conference Paper
The most powerful tools for analysis of formal specifications are general-purpose theorem provers and model checkers, but these tools provide scant methodological support. Conversely, those approaches that do provide a well-developed method generally have less powerful automation. It is natural, therefore, to try to combine the better-developed met...
Article
This report examines the requirements for partitioning, mechanisms for their realization, and issues in providing
Conference Paper
Full-text available
We present a case study to demonstrate that the decomposition of a fault tolerant program into its components is useful in its mechanical verification. More specifically we discuss our experience in using the theorem prover PVS to verify Dijkstra's token ring program in a component based manner. We also demonstrate the advantages of component based...
Article
Since its founding, NASA has been dedicated to the advancement of aeronautics and space science. The NASA Scientific and Technical Information (STI) Program Office plays a key part in helping NASA maintain this important role. The NASA STI Program Office is operated by Langley Research Center, the lead center for NASA's scientific and technical inf...
Article
Full-text available
A specification language used in the context of an effective theorem prover can provide novel features that enhance precision and expressiveness. In particular, type checking for the language can exploit the services of the theorem prover. We describe a feature called “predicate subtyping” that uses this capability and illustrate its utility as mec...
Article
We describe a formal specification and mechanically checked verification of the Interactive Convergence Clock Synchronization Algorithm of Lamport and Melliar-Smith [16]. In the course of this work, we discovered several technical flaws in the analysis given by Lamport and Melliar-Smith, even though their presentation is unusually precise and detai...

Citations

... There are several other applications of automated reasoning to ontological arguments, including first and higher-order treatments of Anselm's traditional argument (Oppenheimer and Zalta 2011;Rushby 2013Rushby , 2020, modal treatments of that argument (Rushby 2019), and Gödel's modal argument (Benzmüller and Woltzenlogel-Paleo 2014). These other applications use moderately advanced logical constructions, such as definite descriptions (Oppenheimer and Zalta 2011;Rushby 2013), higher order logic (Rushby 2020), and first order modal logic (Benzmüller and Woltzenlogel-Paleo 2014;Rushby 2019), whose mechanized support is fairly intricate and prone to errors (Garbacz 2012). ...
... Model-centred assurance (Jha et al. 2020), a new alternative architecture proposed to enable autonomy, critiques this variant of the pattern. It asserts that 'perception functions of the primary system will surely be better resourced and more capable than those of the monitors' and concludes that the monitors should rather use the same input channels to construct a model of the environment. ...
... My focus here is the Proslogion II argument, represented completely formally in first-or higher-order logic, and explored with the aid of a mechanized verification system. Elsewhere, I use a verification system to examine renditions of the argument in modal logic (Rushby, 2019), and also the argument of Proslogion III (Rushby, 2021). Verification systems are tools from computer science that are generally used for exploration and verification of software or hardware designs and algorithms; they comprise a specification language, which is essentially a rich (usually higher-order) logic, and a collection of powerful deductive engines (e.g., satisfiability solvers for combinations of theories, model checkers, and automated and interactive theorem provers). ...
... The safety domain was one of the first to elaborate the safety cases concept. Safety cases were originally theorized by Tim Kelly [KBMB97] and then generalized by John Rushby [Rus15]. In particular, in [Rus15], Rushby claims that the introduction of this kind of methodology in the industries are a significant contribution to system and software assurance and certification. ...
... The strongest known software separation is provided by a separation kernel [8], [9]: the task of a separation kernel is to create an environment which is indistinguishable from that provided by a physically distributed system: it must appear as if each regime is a separate, isolated machine and that information can only flow from one machine to another along known external communication lines. (Rushby used the term regime to denote the hardware abstraction a separation kernel provides to a VM.) ...
... (M2) The data flow model makes explicit all interfaces, connections and data dependencies, and (clocked, named) flows -thus greatly reducing attack surfaces and unanticipated interactions. (M3) In synchronous reactive data flow languages, monitoring safety properties is easy and can be achieved using finite-state automata [55,29]. Not only can monitors be expressed within the model, but the same framework can be used to specify axioms and assumptions, to constrain behaviour, and specify test cases. ...
... Stein [Ste12] applied a distributed performance analysis for implementing admission control and enabling self-configuration of task-based real-time systems. Selfintegration is a concept that is mainly used in a systems-of-systems context [Rus16], [BGLT19]. It addresses the integration of individual systems into a systems-of-systems environment to provide new capabilities or services rather than the automation of the integration phase. ...
... Similarly, when we build proof procedures, another example of the principle of problem solving by decomposition, putting proofs together may be much more complex than simply "build the conjunction" of the proofs. Breaking claims down to ones that can be put together by conjunction enormously, and unnecessarily, complicates the decomposition problem. 3 There is also something to be said about attempts at reducing safety reasoning directly to First Order Logic (FOL) and using automated deduction support and proof calculi for expressing a safety argument (see [31,32]). These attempts try to provide a strong, well-defined, foundation for eliciting what is meant by a structured argument. ...
... Rushby [Rus14] illustrates how assurance arguments can be formalized with modern verification systems such as Isabelle or PVS to overcome some of the logical fallacies associated with informal ACs. Similarly, our framework allows reasoning using formal logic, but additionally supports the combination of formal and informal artifacts. ...
... One attractive idea is for the mode switch to be triggered by monitoring the runtime behavior of the system against its safety case. 16,17 It is plausible to make a case that such monitoring is "possibly perfect" and to estimate its probability of perfection. A perfect system is one that never fails; to say that a system has probability 0.999 of perfection means that of 1,000 systems engineered in a similar way, only 1 may be expected ever to suffer a failure. ...