Conference Paper

Verified Software: Theories, Tools, Experiments Vision of a Grand Challenge Project

Authors:
  • Microsoft Research Ltd., Cambridge, UK
Conference Paper

Verified Software: Theories, Tools, Experiments Vision of a Grand Challenge Project

If you want to read the PDF, try requesting it from the authors.

Abstract

The ideal of correct software has long been the goal of research in Computer Science. We now have a good theoretical understanding of how to describe what programs do, how they do it, and why they work.This understanding has already been applied to the design, development and manual verification of simple programs of moderate size that are used in critical applications.Automatic verification could greatly extend the benefits of this technology. This paper argues that the time is ripe to embark on an international Grand Challenge project to construct a program verifier that would use logical proof to give an automatic check of the correctness of programs submitted to it.Prototypes for the program verifier will be based on a sound and complete theory of programming; they will be supported by a range of program construction and analysis tools; and the entire toolset will be evaluated and evolve by experimental application to a large and widely representative sample of useful computer programs.The project will provide the scientific basis of a solution for many of the problems of programming error that afflict all builders and users of software today. This paper starts with an optimistic vision of a possible long-term future of reliable programming. It argues that scientific research will play an essential role in reaching these long-term goals. It suggests that their achievement should be accelerated by a major international research initiative, modelled on a Grand Challenge, with specific measurable goals. The suggested measure is one million lines of verified code.By definition, this consists of executable programs, together with their specifications, designs, assertions, etc., and together with a machine-checked proof that the programs are consistent with this documentation. We anticipate that the project would last more than ten years, consume over one thousand person-years of skilled scientific effort, drawn from all over the world.Each country will contribute only a proportion of the effort, but all the benefits will be shared by all. The paper concludes with suggestions for exploratory pilot projects to launch the initiative and with a call to volunteers to take the first steps in the project immediately after this conference.A possible first step will be to revise and improve this paper as a generally agreed report of goals and methods of the scientists who wish to engage in the project.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... There is a healthy trend in formal methods research driven by the idea of a Grand Challenge (GC) [28] expected to deliver "a comprehensive and unified theory of programming" and "a repository of verified software" [28,Section 2]. Mondex [29] was the first GC pilot project. ...
... There is a healthy trend in formal methods research driven by the idea of a Grand Challenge (GC) [28] expected to deliver "a comprehensive and unified theory of programming" and "a repository of verified software" [28,Section 2]. Mondex [29] was the first GC pilot project. ...
... In the sequel we will focus on such a Key * Dat volume level array which, at this level of refinement, is regarded as the abstract structure to be further refined. Diagram (28) above shows how this array, hereinafter referred to as the abstract store (AbsSt), unfolds into four data structureslabeled FIdx, Jrnl, RIdx, and F lash in the diagram once refined down to (journaled) FLASH level. In detail: ...
Article
Relational algebra offers to software engineering the same degree of conciseness and calculational power as linear algebra in other engineering disciplines. Binary relations play the role of matrices with similar emphasis on multiplication and transposition. This matches with Alloy’s lemma “everything is a relation” and with the relational basis of the Algebra of Programming. Altogether, it provides a simple and coherent approach to checking and calculating programs from abstract models. In this paper we put Alloy and the Algebra of Programming together in a case study originating from the Verifiable File System mini- challenge put forward by Joshi and Holzmann: verifying the refinement of an abstract file store model into a journaled (FLASH) data model catering for wear leveling and recovery from power loss. Our approach relies on diagrams to graphically express typed assertions. It interweaves model checking (in Alloy) with calculational proofs in a way which offers the best of both worlds. This provides ample evidence of the positive impact in software verification of Alloy’s focus on relations, complemented by induction-free proofs about data structures such as stores and lists.
... As a consequence , " the criterion of correctness is specified by types, assertions and other redundant annotations associated with the code of the program " , where " the compiler will work in combination with other program development and testing tools, to achieve any desired degree of confidence in the structural soundness of the system and the total correctness of its more critical components. " The extension in [67] to what is now called " verified software grand challenge " has broadened the focus from " a program verifier that would use logical proof to give an automatic check of the correctness of programs submitted to it " (op.cit. pg.1) to a vision that forsees the interactive use of " a range of program construction and analysis tools " (ibidem) and mentions the role of " reliable capture of requirements, and their faithful encoding as specifications " (pg.3) and of " other formal and informal documentation " (pg.4) besides " assertions internal to the program " . ...
... ASM Ground Models (System Blueprints): A Semantical Foundation for Program Verification A main point where the verified software challenge as formulated in [66, 67] must be extended to become of practical interest concerns the apparently implicit assumption that compilable programs constitute the true definition of the system they represent. The assumption expresses a widespread belief. ...
... Unfortunately it is still strongly debated what kind of language is suited to express ground models and which methods are appropriate for their analysis. To satisfy the above described ground model properties and to serve as basis for a practical program verification project, neither can the ground model language be confined to the syntax of some particular logic or specification language nor can the means of analysis be restricted to a priori fixed rule-based (a fortiori if mechanized) reasoning schemes, contrary to what some formulations in [66, 67] seem to suggest and also contrary to the view hold in [75] that " the " verification problem " is the theorem proving problem, restricted to a computational logic " . It would not solve the communication problem since the thorough training in mathematical logic it requires goes beyond the expertise that can reasonably be expected from every software practitioner or domain expert. ...
Article
We explain why for the verified software challenge proposed in (66,67) to gain practical impact, one needs to include rigorous defini- tions and analysis, prior to code development and comprising both ex- perimental validation and mathematical verification, of ground models, i.e. blueprints that describe the required application-content of programs. This implies the need to link via successive refinements the relevant prop- erties of such high-level models in a traceable and checkable way to code a compiler can verify. We outline the Abstract State Machines (ASM) method, a discipline for reliable system development which allows one to bridge the gap between informal requirements and executable code by combining application-centric experimentally validatable system mod- elling with mathematically verifiable stepwise detailing of abstract mod- els to compile-time-verifiable code.
... A "verifying compiler", as proposed by Tony Hoare and Jay Misra in their grand challenge [1,2], is one possible solution to achieve this goal. According to this approach, the compiler is able to determine the correctness of the code by using additional information, like, e.g. ...
... Graph Model SIBs (2) This bundle provides SIBs that deal with SLGs. There are SIBs for loading and traversing hierarchical SLGs as well as for retrieving information from contained SIBs (e.g. ...
Article
Full-text available
This paper presents Genesys, a framework for the high-level construction of property conform code generators. Genesys is an integral part of jABC, a flexible framework designed to enable a systematic model-driven development of systems and applications on the basis of an (extensible) library of well-defined, reusable building blocks. Within jABC, Genesys manages the construction of code generators for jABC’s models. So far, Genesys has been used to develop code generators for a variety of different target platforms, like a number of Java-based platforms, mobile devices, BPEL engines, etc. Since the code generators are themselves built within the jABC in a model-driven way, concepts like bootstrapping and reuse of existing components enable a fast evolution of Genesys’ code generation library, and a high degree of self-application. Due to its increasing complexity and its high degree of reuse, Genesys profits from model checking-based verification. This way, jABC’s models of code generators can be automatically checked wrt. well-formedness properties, to ensure that the models do indeed only consist of building blocks which are suitable for the considered target platforms, and whose versions are mutually compatible. It can be also be verified that the code generation process only starts after a successful initialization phase, and that the generated code is always packaged with all the required libraries. We will illustrate the ease of extension and flexibility of the Genesys framework by describing the intended coverage of diversity while highlighting the high potential of reuse.
... The key motivation for developing the link is to contribute to the 'Grand Challenge in Computing' (GC6) project. The GC project [13] is expected: ...
... After the start of 'Grand Challenge in Computing' (GC6) [13] project with a special focus on mission critical filestores, a number of efforts have been made to formalise the flash memory and filestores. Paper [4] gives the Z notation for the formal model of NAND flash memory. ...
... FMICS, the ERCIM Working Group on Formal Methods for Industrial Critical Systems (FMICS) [8], transfers and promotes the use formal methods technology in industry . The ongoing Verified Software Repository Grand Challenge [11] offers a great opportunity to reach this goal, resulting in a more robust and solid software industry in Europe. The FMICS-jETI platform [9] is a collaborative demonstrator based on the jETI technology [14, ?]. ...
... FMICS, the ERCIM Working Group on Formal Methods for Industrial Critical Systems (FMICS) [8], transfers and promotes the use formal methods technology in industry . The ongoing Verified Software Repository Grand Challenge [11] offers a great opportunity to reach this goal, resulting in a more robust and solid software industry in Europe. The FMICS-jETI platform [9] is a collaborative demonstrator based on the jETI technology [14, ?] . ...
Conference Paper
Full-text available
We revisit the automatic tool composition feature of the electronic tool integration platform under the perspective of planning. It turns out that in todays terminology, ETIs temporal logic-based synthesis of tool sequences is a form of planning-based automated orchestration. In contrast to Al-based planning approaches, our synthesis approach is not restricted to compute one solution, but it may compute all (shortest/minimal) solutions, with the intent to provide maximum insight into the potential design space.
... In [25], Hoare and Misra proclaim the verification grand challenge as an international project to construct a program verifier that would use logical proof to give an automatic check of the correctness of programs submitted to it. What sounds for a moment out of touch with reality is, based on their assumptions, within the reach of the next 20 years. ...
... In order to correct the receiver state machine implementation, source code lines 25 Based on the results in Table 6.6, the following consequences are drawn: ...
... In 2003, Tony Hoare proposed the Verifying Compiler as a Grand Challenge for Computer Science [Hoare 2003]. As the proposal started to gain the support of the community, it became the Grand Challenge in Verified Software [Hoare and Misra 2008] and then the Verified Software Initiative, which was officially launched at the 2008 Conference on Verified Software: Theories, Tools, and Experiments [Shankar and Woodcock 2008]. The UK effort is in the Grand Challenge in Dependable Systems Evolution (GC6), and current work includes building a Verified Software Repository [Bicarregui et al. 2006]. ...
... We emphasise the academic role of the repository in advancing science, but this does not preclude parts of the repository containing reusable verified components directed towards real-life application domains. The notion of verification will include the entire spectrum, from avoidance of specific exceptions like buffer overflow, general structural integrity (or crash-proofing), continuity of service, security against intrusion, safety, partial functional correctness , and (at the highest level) total functional correctness [Hoare and Misra 2008]. Similarly, the notion of verification will include the entire spectrum, from unit testing to partial verification through bounded model checking to fully formal proof. ...
Article
Full-text available
Formal methods use mathematical models for analysis and verification at any part of the program life-cycle. We describe the state of the art in the industrial use of formal methods, concentrating on their increasing use at the earlier stages of specification and design. We do this by reporting on a new survey of industrial use, comparing the situation in 2009 with the most significant surveys carried out over the last 20 years. We describe some of the highlights of our survey by presenting a series of industrial projects, and we draw some observations from these surveys and records of experience. Based on this, we discuss the issues surrounding the industrial adoption of formal methods. Finally, we look to the future and describe the development of a Verified Software Repository, part of the worldwide Verified Software Initiative. We introduce the initial projects being used to populate the repository, and describe the challenges they address.
... The key motivation for developing the link is to contribute to the 'Grand Challenge in Computing' (GC6) project. The GC project [13] is expected: ...
... After the start of 'Grand Challenge in Computing' (GC6) [13] project with a special focus on mission critical filestores, a number of efforts have been made to formalise the flash memory and filestores. Paper [4] gives the Z notation for the formal model of NAND flash memory. ...
Article
Full-text available
peer-reviewed Following the development of formalisms based on data and behavioural aspects of the system, there are number of attempts in which these two formalisms are mixed together to get benefit of both paradigms. 'Circus' being a living specification language with continuous collaboration from both academia and industry, is a combination of Z, CSP and the refinement calculus. To make use of the available and industry-proven tools for a particular programming paradigm, there is a need to develop a formally verified link between the one world and the other. The aim of this work is to develop a formally verified link between a state-rich process algebra i.e. 'Circus' to a state-free process algebra i.e. CSP. To achieve the research goal, the most suitable available tools are to identify. For developing link between targeted formal languages, we will identify the key translations required between the two languages. For ensuring correctness of the translation, we will formalise the key translation / refinement steps. These will form the theoretical core of the work and support the soundness of the link. In the end, we will select and verify a case study from the collection of software / hardware protocols. SFI
... In other words, we propose ways of deriving specifications from requirements in a systematic fashion. This is reminiscent of the work on deriving code from specification [71,72]. The similarity between the two can be summarised as follows: ...
... Hoare's Communicating Sequential Processes (CSP) [68] is a formal description language used in software engineering. Although its original purpose was to describe concurrency in programming [67], it has evolved and been applied to other areas of software engineering: for example, modelling and analysis of security protocols [141], specifying software architecture connections [4], describing system level interactions between software and hardware [135], and software verification [72,71]. It also influenced the development of the Occam programming language [78]. ...
Article
Full-text available
One of the problems with current practice in software development is that customer requirements are often not well captured, understood and analysed, and there is no clear traceable path from customer requirements to software specifications. This often leads to a mismatch between what the customer needs and what the software developer understands the customer needs. In addition to capturing, understanding and analysing requirements, requirements engineering (RE) aims to provide methods to allow software development practitioners to derive software specifications from requirements. Although work exists towards this aim, the systematic derivation of specifications from requirements remains an open problem. This paper provides a practical technique to implement the idea of problem progression as the basis for transforming requirements into specifications. The technique allows us to progress a software problem towards identifying its solution by carefully investigating the problem context and re-expressing the requirement statement until a specification is reached.
... Research in tool-supported verification has reached a point where several tools have been applied successfully in industrial projects, and use is becoming increasingly widespread [103, 158]. There may be reason to anticipate that verification tools will be as ubiquitous in the future as type checkers are today in software development: Hoare has introduced the concept verifying compiler [95]—a compiler that verifies the correctness of the programs it compiles—and envisions that such a compiler may be reality within 15-20 years [96, 97]. If this scenario unfolds, it is not unreasonable to think that the programmers of tomorrow may be producing verified software on a daily basis . ...
... @BULLET The embedded and mission-critical community sometimes prefers to use low-level languages, including assembly, for fear of the risks potentially introduced by compilers and other supporting tools. The " Verifying Compiler Grand Challenge " [38] [77] is an attempt to support the development of tools that — even with such programming languages — will guarantee, during the process of compiling and thanks to techniques described in the following sections, the reliability of the programs they process. §7 STATIC VERIFICATION TECHNIQUES ...
Conference Paper
ABSTRACT Achieving software reliability takes many complementary techniques, directed at the process or at the products. This survey summarizes some of the most fruitful ideas. 1 OVERVIEW
... Hoare's paper defines a set of criteria for an international effort to drive research in computer science forward towards automatic software verification. Hoare et al [22] proposed that the conditions set in [21] were met, and that the time to start such a long term international research project had arrived. The GC project is expected to " deliver a comprehensive and unified theory of programming " , " prototype for a comprehensive and integrated suite of programming tools " , and " deliver a repository of verified software " . ...
Conference Paper
Full-text available
Tool interoperability as a mean to achieve integration is among the main goals of the international Grand Challenge initiative. In the context of the Verifiable file system mini-challenge put forward by Rajeev Joshi and Gerard Holzmann, this paper focuses on the integration of different formal methods and tools in modelling and verifying an abstract file system inspired by the Intel ^{\textrm{\textregistered}} Flash File System Core. We combine high-level manual specification and proofs with current state of the art mechanical verification tools into a tool-chain which involves Alloy, VDM++ and HOL. The use of (pointfree) relation modelling provides the glue which binds these tools together.
... There are various approaches that aim at assuring the reliability of automatic code generation. A "verifying compiler", as proposed by Tony Hoare and Jay Misra in their grand challenge [HM05,Hoa03], is one possible solution to achieve this goal. According to this approach, the compiler is able to determine the correctness of the problem by using additional information, like e.g. ...
Article
Full-text available
Automatic code generation is an essential cornerstone of today’s model-driven approaches to software engineering. Thus a key requirement for the success of this technique is the reliability and correctness of code generators. This article describes how we employ standard model checking-based verification to check that code generator models developed within our code generation framework Genesys conform to (temporal) properties. Genesys is a graphical framework for the high-level construction of code generators on the basis of an extensible library of well-defined building blocks along the lines of the Extreme Model-Driven Development paradigm. We will illustrate our verification approach by examining complex constraints for code generators, which even span entire model hierarchies. We also show how this leads to a knowledge base of rules for code generators, which we constantly extend by e.g. combining constraints to bigger constraints, or by deriving common patterns from structurally similar constraints. In our experience, the development of code generators with Genesys boils down to re-instantiating patterns or slightly modifying the graphical process model, activities which are strongly supported by verification facilities presented in this article.
... Formal verification techniques complement testing; these techniques can, in some cases, even replace tests and provide an exhaustive coverage of the state space, which is considered by the standard DO-178C and its extension DO-333. Reliable software and the verifying compiler have 6 Grand Challenge been declared as one of the grand challenges in computer science [139,140]. ...
Thesis
Full-text available
Whenever software faults can endanger human life, property, or the environment, the absence of faults must be ensured with utmost care and the best technologies available. Evidence is needed showing that all requirements are satisfied and that the risk of faults is reduced. One technique to conduct such a verification task—composed of the software to verify, the specification to check, and a model of the environment—is software model checking. To conduct a verification task with a model checker, different models of the task are constructed. We distinguish between two types of task models: syntactic task models and semantic task models, which define the respective syntactic structure (control flow) and semantic structure (state transitions, invariants) of the verification task. When constructing such models, we can observe that similar structures and substructures reappear within and among different verification tasks. For example, the same assertions to check can appear in different functions, or the same predicate can be part of different invariants to describe sets of program states. Similarities that appear during the model construction process can be the result of solving similar reasoning problems, often solved using computationally expensive procedures (as typical for model checking), over and over again. Not reusing results of solving similar problems, not having a means for conducting repeated efforts automatically, or not trying to reduce the number of similar reasoning efforts, is a waste of precious resources. To address these problems, we present a common conceptual and technical foundation for sharing syntactic and semantic task artifacts for reuse, within and among verification runs. Both the syntactic construction of a verification task and the construction of its semantic model—which describes all possible behaviors and states—are covered. We study how commonalities and regularities in the task models can be taken into account to facilitate the process of sharing task artifacts for reuse, and to make the overall verification process more efficient and effective. We introduce abstract transducers as the theoretical foundation of this thesis: a type of finite-state transducers with an inherent notion of abstraction for states, the input alphabet, and its output alphabet. Abstracting these transducers allows us to widen both the set of input words for that they produce output and the sets of output words. Abstract transducers are instantiated as task artifact transducers to map from program structures to task artifacts to share. We show that the notion of abstraction provides a means for increasing the scope for that task artifacts are shared for reuse. We present two instances of task artifact transducers: Yarn transducers and precision transducers. We use Yarn transducers for providing code to weave into the control-flow structure of a computer program, and present the Loom analysis as a means for orchestrating the weaving process. Precision transducers provide a means for sharing abstraction precisions for reuse, thus aid in defining the level of abstraction of a semantic task model. For both types of transducers, we provide empirical evidence on their practical applicability, for example, to verify Linux kernel modules, and show that they can help in increasing the verification performance.
... In this paper, we show that we can develop verified component-based software in SPARK. Moreover we believe that this approach can scale up, and thus has the potential to address the scale problem posed by the Grand Challenge in Verified Software [11, 12]. Our belief is based on our view that the scale problem can only be tackled by a component-based approach, where software components are systematically composed into a complete system. ...
Conference Paper
SPARK is useful for developing reliable software for safety-critical systems, using the 'correctness-by-construction' approach. It also has verication tools that can be used to produce veried software. To tackle larger-scale development of veried software, compo- nents are useful. In this paper we show how to dene and im- plement software components in SPARK and use existing SPARK tools to produce veried component-based software. We demon- strate our approach on a missile guidance system.
... @BULLET Prove that e will always hold. (This assumes a " verifying compiler " equipped with proof capabilities — another concept promoted by Tony Hoare [7].) ...
Chapter
All object-oriented programs, but also those in C or Pascal as soon as they use pointers, are subject to the risk of run-time crash due to “null pointer dereferencing”. Until recently this was the case even in statically typed languages. Tony Hoare has called this problem his “billion-dollar mistake”. In the type system of ISO-standard Eiffel, the risk no longer exists: void safety (the absence of null pointer dereferencing) has become a property guaranteed by the type system and enforced by the compiler. The mechanism is fully implemented and major libraries and applications have been made void-safe. This presentation describes the principles of Eiffel’s void safety, their implementation and the lessons gained.
... Software correctness has become one major concern in software development [22]. Formal methods are expected to play a significant role in ensuring this. ...
Conference Paper
Formal specifications have been a focus of software engineering research for many years and have been applied in a wide variety of settings. Their use in software engineering not only promotes high-level verification via theorem proving or model checking, but also inspires the "correct-by- construction" approach to software development via formal refinement. Although this correct-by-construction method proves to work well for small software systems, it is still a Utopia in the development of large and complex software systems. This paper moves one step forward in this direction by designing and implementing a sound linkage between the high level specification language Object-Z and the object-oriented specification language Spec#. Such a linkage would allow system requirements to be specified in a high-level formal language but validated and used in program language level. This linking process can be readily integrated with an automated program refinement procedure to achieve correctness-by-construction. In case no such procedures are applicable, the obtained contract- based specification can guide programmers to manually generate program code, which can then be verified against the obtained specification using any available program verifiers.
... The feasibility of large-scale formal software verification rests squarely on the development of robust, sophisticated, and scalable verification tools. Recent advances in verification technology on a number of fronts have made it possible to contemplate a major push toward large-scale software verification [Hoare 2003;Hoare and Misra 2008]. These advances have already yielded practical tools for solving hard verification problems. ...
Article
Full-text available
Automated deduction uses computation to perform symbolic logical reasoning. It has been a core technology for program verification from the very beginning. Satisfiability solvers for propositional and first-order logic significantly automate the task of deductive program verification. We introduce some of the basic deduction techniques used in software and hardware verification and outline the theoretical and engineering issues in building deductive verification tools. Beyond verification, deduction techniques can also be used to support a variety of applications including planning, program optimization, and program synthesis.
... Although the Missile Guidance System is already quite large, it is nowhere near the 1 million lines that is the target of the Grand Challenge in Verified Software [5] . Therefore , we hope to attempt increasingly larger examples, in order to produce convincing evidence that our model is fit for purpose, as far as the scale problem is concerned. ...
Article
1 What? Our component-based approach to verified software is a result of cross-fertilisation be-tween verified software and component-based software development. In contrast to ap-proaches based on compositional verification techniques, our approach is designed to solve the scale problem in verified software. Compositional verification tends to be top-down, i.e. it partitions a system into sub-systems, and proves the whole system by proving the subsystems (Fig. 1). The subsys- of subsys Fig. 1. Compositional verification. tems, often called components, are system-specific, and are therefore not intended for reuse. It follows that their proofs cannot be reused in other systems. By contrast, our approach to verified software is bottom-up, starting with pre-existing pre-verified components, and composing them into verified composites. (Fig. 2). Com-Component . . .
... Program Verification: RQP can also be an important component for Hoare's Grand Challenge project of program verification [HM05]. In order to prove the correctness of a program, all possible states of a program must be computed. ...
Article
Full-text available
Testing is one of the most time-consuming and cost-intensive tasks in software development projects today. A recent report of the NIST [RTI02] estimated the costs for the economy of the Unites States of America caused by software errors in the year 2000 to range from $22.2 to $59.5 billion. Consequently, in the past few years, many techniques and tools have been developed to reduce the high testing costs. Many of these techniques and tools are devoted to automate various testing tasks (e.g., test case generation, test case execution, and test result checking). However, almost no research work has been carried out to automate the testing of database applications (e.g., an E-Shop application) and relational database management systems (DBMSs). The testing of a database application and of a DBMS requires different solutions because the application logic of a database application or of a DBMS strongly depends on the contents of the database (i.e., the database state). Consequently, when testing database applications or DBMSs new problems arise compared to traditional software testing. This thesis focuses on a specific problem: the test database generation. The test database generation is a crucial task in the functional testing of a database application and in the testing of a DBMS (also called test object further on). In order to test a certain behavior of the test object, we need to generate one or more test databases which are adequate for a given set of test cases. Currently, a number of academic and commercial test database generation tools are available. However, most of these generators are general-purpose solutions which create the test databases independently from the test cases that are to be executed on the test object. Hence, the generated test databases often do not comprise the necessary data characteristics to enable the execution of all test cases. In this thesis we present two innovative techniques (Reverse Query Processing and Symbolic Query Processing), which tackle this problem for different applications (i.e, the functional testing of database applications and DBMSs). The idea is to let the user specify the constraints on the test database individually for each test case in an explicit way. These constraints are then used directly to generate one or more test databases which exactly meet the needs of the test cases that are to be executed on the test object. In heutigen Softwareentwicklungsprojekten ist das Testen eine der kosten- und zeitintensivsten Tätigkeiten. Wie ein aktueller Bericht des NIST [RTI02] zeigt, verursachten Softwarefehler in den USA im Jahr 2000 zwischen 22, 2 und 59, 5 Milliarden Dollar an Kosten. Demzufolge wurden in den letzten Jahren verschiedene Methoden und Werkzeuge entwickelt, um diese hohen Kosten zu reduzieren. Viele dieser Werkzeuge dienen dazu die verschiedenen Testaufgaben (z.B. das Erzeugen von Testfällen, die Ausführung von Testfällen und das Überprüfen der Testergebnisse) zu automatisieren. Jedoch existieren fast keine Forschungsarbeiten zur Testautomatisierung von Datenbankanwendungen (wie z.B. eines E-Shops) oder von relationalen Datenbankmanagementsystemen (DBMS). Hierzu sind neue Lösungen erforderlich, da das Verhalten der zu testenden Anwendung stark vom Inhalt der Datenbank abhängig ist. Folglich ergeben sich für den Test von Datenbankanwendungen oder von Datenbankmanagementsystemen neue Probleme und Herausforderungen im Vergleich zum traditionellen Testen von Anwendungen ohne Datenbank. Die vorliegende Arbeit diskutiert ein bestimmtes Problem aus diesem Umfeld: Die Generierung von Testdatenbanken. Die Generierung von Testdatenbanken ist eine entscheidende Tätigkeit für den erfolgreichen Test einer Datenbankanwendung oder eines Datenbankmanagementsystems (im weiteren Verlauf auch Testobjekt genannt). Um eine bestimmte Funktionalität des Testobjekts zu testen, müssen die Daten in den Testdatenbanken bestimmte Charakteristika aufweisen. Zur Erzeugung einer Testdatenbank existieren verschiedene Forschungsprototypen wie auch kommerzielle Datenbankgeneratoren. Jedoch sind die existierenden Datenbankgeneratoren meist Universallösungen, welche die Testdatenbanken unabhängig von den auszuführenden Testfällen erzeugen. Demzufolge weisen die generierten Testdatenbanken meist nicht die notwendigen Datencharakteristika auf, die zur Ausführung einer bestimmten Menge von Testfällen notwendig sind. Die vorliegende Doktorarbeit stellt zwei innovative Ansätze vor (Reverse Query Processing und Symbolic Query Processing), die dieses Problem für unterschiedliche Anwendungen (d.h. für das funktionale Testen von Datenbankanwendungen und Datenbankmanagementsystemen) lösen. Die generelle Idee beider Ansätze ist, dass der Benutzer explizit für jeden Testfall die notwendigen Bedingungen an die Testdaten formulieren kann. Diese Bedingungen werden dann dazu genutzt, um eine oder mehrere Testdatenbanken zu generieren, die die gewünschten Datencharakteristika aufweisen, welche zur Ausführung der Testfälle notwendig sind.
... There are many examples of successful application for formal methods, e.g. [1], and there are challenges for the future [23]. There are two well established approaches to formal reasoning and system verification [13]. ...
Article
The work concerns automatic generation of logical specifications from requirements models. Logical specifications obtained in such a way can be subjected to formal verification using deductive reasoning. Formal verification concerns correctness of a model behaviour. Reliability of the requirements engineering is essential for all phases of software development processes. Deductive reasoning is an important alternative among other formal methods. However, logical specifications, considered as sets of temporal logic formulas, are difficult to specify manually by inexperienced users and this fact can be regarded as a significant obstacle to practical use of deduction-based verification tools. A method of building requirements models using some UML diagrams, including their logical specifications, is presented step by step. Organizing activity diagrams into predefined workflow patterns enables automated extraction of logical specifications. The crucial aspect of the presented approach is integrating the requirements engineering phase and the automatic generation of logical specifications. A system of the deduction-based verification is proposed. The reasoning process could be based on the semantic tableaux method. A simple yet illustrative example of the requirements elicitation and verification is provided.
... As mentioned by Hoare and Misra in [26], the time is ripe to embark on a Grand Challenge project to construct a verifying compiler. A verified software repository has been identified as an important step towards meeting such challenge. ...
Article
Full-text available
An important step in meeting the Verifying Compiler Grand Challenge is the Verified Software Repository. In the FMICS view, the repository should include proven correct software and tools to help establishing the correctness of the software in question. We propose to set up a collaborative demonstrator, based on the jETI technology, to provide tools to the repository and to orchestrate different tools.
... The core of this work is orchestration languages, software contracts, and Hoare logic [15,8,14,7]. We borrow the idea of footprints from separation logic [19] and the fundamentals of superposition from UNITY [9]. ...
Article
Firefighting evolving, open systems' failures is rather old school. Rather than hand-patching failures, as is done today, these sys-tems should self-repair. We provide a formal foundation for self-healing in evolving, open systems. The focus of our reasoning is dynamically com-posed open systems that experience partial failure. We talk about com-position from an orchestration perspective, and use the Orc language as a foundation for specifying compositional systems. Unfortunately, Orc's semantics makes no mention of the behavior of the programs under com-position, therefore, one cannot reason about the behavior of Orc pro-grams. In this paper, we explain how to provide Orc with a behavioral semantics i.e., software contracts, thereby synthesizing specifications of whole Orc programs from the specification of their parts. We also enrich Orc's semantics to account for programs that communicate via superpo-sition and message passing.
... -15 річний міжнародний проекттеорія і практика верифікації усіх видів продуктів та їхнє накопичення у Інтернет бібліотеках [19,20]; ...
... We are convinced that this perspective on modelling tools and even frameworks can be useful for the progress towards better software also for less critical application domains, like consumer IT products: they strike wherever time to market and first time correct are an issue. UK has initiated effort towards a verified software repository [5] within the framework of the UK Grand Challenges Programme [10]. The Verified Software Repository will assist in the development of software by facilitating access to a managed collection of analysis tools and a repository of case studies or challenge codes to exercise these tools. ...
Conference Paper
Full-text available
One of the goals of FMICS, the ERCIM Working Group on Formal Methods for Industrial Critical Systems (FMICS) [8], is to transfer and promote the use formal methods technology in industry. The ongoing Verified Software Repository Grand Challenge [11] offers a great opportunity to reach this goal, resulting in a more robust and solid software industry in Europe. We demonstrate here the current status of the FMICS-jETIplatform1, a collaborative demonstrator based on the jETI technology2, that provides as repository a collection of verification tools stemming from the activities of the FMICS working group and facilities to orchestrate them in a remote and simple way. At the same time FMICS-jETI itself is a contribution to the VSR repository and thus to the Grand Challenge.
... In 2005, a Verified Software Grand Challenge was proposed [27], and in its context a Verified Software Repository (VSR) [8]. Although a workshop produced several new publications related to the Mondex case study [44] and the site proposed the POSIX file store and a pacemaker as additional examples, there was not a lot of follow-up activity after that. ...
Chapter
Full-text available
Formal methods (in a broad sense) have been around almost since the beginning of computer science. Nonetheless, there is a perception in the formal methods community that take-up by industry is low considering the potential benefits. We take a look at possible reasons and give candidate explanations for this effect. To address the issue, we propose a repository of industry-relevant example problems with an accompanying open data storage for experiment results in order to document, disseminate and compare exemplary solutions from formal model based methods. This would allow potential users from industry to better understand the available solutions and to more easily select and adopt a formal method that fits their needs. At the same time, it would foster the adoption of open data and good scientific practice in this research field.
... Hoare and Misra [14] outline the importance of undertaking experiments involving the application of theories and tools in order to push forward scientific progress in formal methods. Experiments help us to understand the strengths and weaknesses of theories and tools. ...
Conference Paper
Full-text available
Event-B is a formal method used for specifying and reasoning about systems. Rodin is a toolset for developing system models in Event-B. Our experiment which is outlined in this paper is aimed at applying Event-B and Rodin to a flash-based filestore. Refinement is a useful mechanism that allows developers to sharpen models step by step. Two uses of refinement, feature augmentation and structural refinement, were employed in our development. Event decomposition and machine decomposition are structural refinement techniques on which we focus in this work. We present an outline of a verified refinement chain for the flash filestore. We also outline evidence of the applicability of the method and tool together with some guidelines.
Article
Full-text available
Data refinement is a special instance of refinement where a specification is refined by replacing the data type used in the specification. The theory of data refinement guarantees that this replacement does not adversely affect the functional behaviour of the programs that use these specifications. Object-oriented programming languages such as JML and Spec# support the specification and verification of object-oriented programs. We research their capabilities, identifying their strengths and weaknesses from both a specification and a tool-support point of view. This leads us to the conclusion that object-oriented specification languages should support a view of objects that abstracts away from the implementation details. We examine the specification and verification of programs that are written in this way, making use of existing language features, so that data refinements can be verified using existing verification tools. We propose a framework for the specification and verification of modular data refinement within an object-oriented environment. Objects are specified in terms of one data type and implemented in terms of another. Clients who interact with these objects are never concerned with the underlying implementation details as they interact directly with the abstract specification. A proof-of-concept tool is developed to demonstrate the viability and effectiveness of our proposed framework. This tool takes the form of an application that checks whether or not a program conforms to our framework for the modular data refinement of object-oriented programs.
Article
The Verifying Compiler (VC) project is a core component of the Dependable Systems Evolution Grand Challenge. The VC offers the promise of automatically proving that a program or component is correct, where correctness is defined by program assertions. While several VC prototypes exist, all adopt a semantics for assertions that is unsound. This paper presents a consolidation of VC requirements analysis (RA) activities that, in particular, brought us to ask targeted VC customers what kind of semantics they wanted. Taking into account both practitioners' needs and current technological factors, we offer recovery of soundness through an adjusted definition of assertion validity that matches user expectations and can be implemented practically using current prover technology. For decades, there have been debates concerning the most appropriate semantics for program assertions. Our contribution here is unique in that we have applied fundamental software engineering techniques by asking primary stakeholders what they want and, based on this, proposed a means of efficiently realizing the semantics stakeholders want using standard tools and techniques. We describe how support for the new semantics has been added to ESC/Java2, one of the most fully developed VC prototypes. Case studies demonstrate the effectiveness of the new semantics at uncovering previously indiscernible specification errors.
Conference Paper
The ultimate objective of a verifying compiler is to prove that proposed code implements a full behavioral specification. Experience reveals this to be especially difficult for programs that involve pointers or references and linked data structures. In some situations, pointers are unavoidable; in some others, verification can be simplified through suitable abstractions. Regardless, a verifying compiler should be able to handle both cases, preferably using the same set of rules. To illustrate how this can be done, we examine two approaches to full verification. One replaces language- supplied indirection with software components whose specifications abstract pointers and pointer- manipulation operations. Another approach uses abstract specifications to encapsulate data structures that pointers and references are often used to implement, limiting verification complications to inside the implementations of these components. Using a modular, specification-based tool we have developed for verification condition generation, we show that full verification of programs with and without the direct use of pointers can be handled similarly. There is neither a need to focus on selected pointer properties, such as the absence of null references or cycles, nor a need for special rules to handle pointers.
Article
Theories of concurrency can be distinguished by the set of processes that they model, and by their choice of pre-ordering relation used to compare processes and to prove their correctness. For example, theories based on CCS are often pre-ordered by simulation (or more commonly bisimulation), of which the main varieties are strong or weak or barbed. Theories based on CSP choose as their pre-order a refinement relation, defined as inclusion over sets of observations. The main varieties of observation are just traces, or failures and/or divergences. The processes of the CSP model are restricted to those that satisfy certain naturally arising ‘healthiness conditions’. This paper gives a unifying treatment of simulation and refinement, and illustrates it by the familiar varieties of CCS and CSP that are mentioned above. We consider the variations two at a time. A link between two theories is a function L, which maps the processes of its source theory onto those of its target theory. The image of L defines exactly the set of processes of the target theory. The ordering relation of the target theory is obtained by applying the link L to one or both operands before applying the source theory ordering. We will use the normal transition rules of a structured operational semantics to define a series of linking functions: W for weak simulation, T for trace refinement, R for refusals, D for divergences. We then show that each function is a retraction, in the sense that it is idempotent and decreasing and (in most cases) monotonic in its source ordering. Finally, we show that certain compositions of these functions are also retractions. The definition of a retraction ensures that (1) the processes of the target theory are a subset of those of the source theory; (2) all ordering theorems of the source theory are preserved in the target theory; (3) the healthiness conditions of the target theory are expressed as fixed-point equivalences of the form p ≡ Lp; (4) model-checking the target theory can be optimised, by applying L to only one of the two operands of the ordering. Finally, we show how the separately defined retractions can be composed in a way that preserves these important properties. In other words, the transition systems of several alternative versions of CCS, as well as the main standard versions of CSP, are retracts of the universal transition system that underlies CCS. The research reported here is a step towards completion of the unfinished business of the original ESPRIT Basic Research Action CONCUR [BRA 3009, 1989-92], which aimed to assimilate the theories and notations of CSP, ACP and CCS. A retraction is a good tool for this purpose, because it precisely codifies the similarities between the theories, and enables them to be used in combination, while preserving their essential and beneficial differences. Such unified families of theories may in due course serve as a rigorous foundation for a comprehensive programming toolset, one that provides reliable assistance at all stages of program design, development, testing and evolution. In this working draft, some of the later sections are incomplete.
Article
Proof scores are documents of comprehensible plans to prove theorems. The proof score approach to systems analysis is a method in which proof scores are used to verify that systems enjoy properties (or analyze systems). In this paper, we describe a way to analyze electronic commerce protocols with the proof score approach, which has been developed and refined through several case studies conducted.
Article
Full-text available
When arguments of non-trivial types are repeated in an operation invocation, alternative but otherwise functionally equivalent implementations often produce different results. To know what outcome to expect, a programmer is forced to examine the procedure body of the implementation currently in use. This clearly violates procedural abstraction and thwarts modular reasoning. It occurs because aliasing is introduced when repeated arguments are passed to procedures by copying references. A simple parameter-passing scheme in which initial values are passed for second and subsequent repeated arguments eliminates this problem. This scheme can be implemented efficiently, and it interacts well with by-value parameter passing. It localizes the effects of calls, and permits the introduction of a uniform proof rule for all procedure calls, simplifying formal modular analysis as well as informal yet rigorous reasoning that respects procedural abstraction.
Article
Full-text available
From a historical approach of the construction of the computer and computing, we argue in favor of a more vigorous interaction between Computer Science and Information Science. The arguments are established not only around the role of information as performer of both areas, but also from evidence of the impossibility of "exact" or "human" knowledge to solve by themselves the challenges of the construction of science and technology. In addition, we consider the hybridism between what is called "content" (what is studied) and the so-called "context" (where and when is studied), and the impossibility of a broad understanding of technological artifacts or scientific facts dissociated of the historical conditions in which they have been proposed. All these three questions make clear that, like the Science of Information, Computer Science is also constructed around the information, and therefore, the dialogue between these fields need to crave the level of co-construction of both fields.
Article
Message passing is widely used in industry to develop programs consisting of several distributed communicating components. Developing functionally correct message passing software is very challenging due to the concurrent nature of message exchanges. Nonetheless, many safety-critical applications rely on the message passing paradigm, including air traffic control systems and emergency services, which makes proving their correctness crucial. We focus on the modular verification of MPI programs by statically verifying concrete Java code. We use separation logic to reason about local correctness and define abstractions of the communication protocol in the process algebra used by mCRL2. We call these abstractions futures as they predict how components will interact during program execution. We establish a provable link between futures and program code and analyse the abstract futures via model checking to prove global correctness. Finally, we verify a leader election protocol to demonstrate our approach.
Conference Paper
The gap between on the one side the users' understanding of Business Process Models (BPMs) and on the other side the run behavior of model implementations is still with us. We introduce Abstract State Machine Nets (ASM Nets) whose component ASMs can be defined using a simple combination of textual (data-oriented) and intuitive graphical (control-flow oriented) yet semantically rigorously defined descriptive means. This allows the BP experts to design BPMs whose underlying ASM Nets constitute a reliable precise contract---a contract which guarantees the BP domain experts that the application-domain focussed understanding of the BPMs they design is also a correct understanding of the code behavior provided by the implementation of the models by software experts. This paves the way for the development of certifiably correct BPMs and their implementations. To illustrate one practical and one conceptual application of the concept we a) instantiate ASM Nets to model the behavioral meaning of the graphical and textual notations used in the commercial S-BPM tool suite with its focus on communication (service interaction) and b) show that applying the rigorous ASM refinement concept to ASM Nets supports IBM's Guard-Stage-Milestone approach to adaptive case management by an accurate conceptual foundation.
Conference Paper
Computer science researchers in the programming languages and formal verification communities, among others, have produced a variety of automated assistance and verification tools and techniques for formal reasoning. While there have been notable successes in utilizing these tools on the development of safe and secure software and hardware, these leading-edge advances remain largely underutilized by large populations of potential users that may benefit from them. In particular, we consider researchers, instructors, students, and other end users that may benefit from instant feedback from lightweight modeling and verification capabilities when exploring system designs or formal arguments. We describe Aartifact, a supporting infrastructure that makes it possible to quickly and easily assemble interacting collections of small domain-specific languages, as well as translations between those languages and existing tools (e.g., Alloy, SPIN, Z3) and techniques (e.g., evaluation, type checking, congruence closure); the infrastructure also makes it possible to compile and deploy these translators in the form of a cloud-based web application with an interface that runs inside a standard browser. This makes more manageable the process of exposing a limited, domain-specific, and logistically accessible subset of the capabilities of existing tools and techniques to end users. This infrastructure can be viewed as a collection of modules for defining interfaces that turn third-party formal modeling and verification tools and techniques into plug-ins that can be integrated within web-based interactive formal reasoning environments.
Book
Thousands of different programming languages exist, and many more are being created each year, yet all those involved in such work must acknowledge that it is the highest goal of programming language design to enable good ideas to be elegantly expressed. These are the words of Sir Charles Antony Richard Hoare, or Tony Hoare to his colleagues and friends, a man who has inspired generations of researchers in computer science. His work, grounded in practical programming, has had a profound impact on the evolution of the field. This includes the development of Hoare logic and Communicating Sequential Processes (CSP). Reflections on the Work of C.A.R. Hoare presents a comprehensive edited survey of all aspects of these subjects, with original contributions by more than 30 international leaders in the field. The book, while honoring Hoares important contributions, assembles a collection of chapters showing the state of the art in programming languages, sequential programs, concurrency, and unified theories of programming. Topics and features: demonstrates a principled combination of CSP and functional programming, and the expression of CSP semantics to -calculus; reviews methods for proving Hoare formulae based on the computation of postconditions and preconditions; investigates developments in game semantics and semantics based on the state monad; examines the satisfiability problem, void safety, and issues of auxiliary variables; introduces type families in Haskell and a description of Quicksort in Orc; describes an experiment using the Tokeneer archive, and a correctness proof of cryptographic protocols based on the Shadow Security model; presents a representation of CSP as a bigraphical reactive system, and shows how simple entities can be related to CSP processes; discusses the problem frames approach, and explores algebraic properties of the new programming combinators. This accessible monograph is an ideal overview of theoretical and empirical evolution in programming logic and semantics of programming languages. It will serve as an invaluable resource for professionals, researchers, libraries, and students who are interested in broadening their knowledge in all of the areas covered.
Article
Tool interoperability is among the main goals of the international Grand Challenge initiative. In the context of the Verifiable File System mini-challenge put forward by Joshi and Holzmann, our work has been focused on the integration of different formal methods and tools in a tool-chain for modelling and verification. The current paper shows how to adapt such a tool-chain to the task in hands, aiming at reducing tool integration costs. The refinement of an abstract file store model into a journaled (flash) data model catering for wear leveling and recovery from power loss is taken as case study. This shows that refinement steps can be carried out within a shorter, reduced life-cycle where model checking in Alloy goes hand in hand with manual proofs carried out in the (pointfree) algebra of binary relations. This provides ample evidence of the positive impact of Alloy’s lemma ’everything is a relation’ on software verification, in particular in carrying out induction-free proofs about data structures such as finite maps and lists.
Chapter
We describe an experiment conducted as part of a pilot project in the Verified Software Initiative (VSI). We begin by recounting the background to the VSI and its six initial pilot projects, and give an update on the current progress of each project. We describe one of these, the Tokeneer ID Station in greater detail. Tokeneer was developed by Praxis High Integrity Systems and SPRE for the US National Security Agency, and it has been acclaimed by the US National Academies as representing best practice in software development. To date, only five errors have been found in Tokeneer, and the entire project archive has been released for experimentation within the VSI. We describe the first experiment using the Tokeneer archive. Our objective is to investigate the dependability claims for Tokeneer as a security-critical system. Our experiment uses a model-based testing technique that exploits formal methods and tools to discover nine anomalous scenarios. We discuss four of these in detail.
Conference Paper
This paper presents a case study in modeling and verifying a POSIX-like file store for Flash memory. This work fits in the context of Hoare’s verification challenge and, in particular, Joshi and Holzmann’s mini-challenge to build a verifiable file store. We have designed a simple robust file store and implemented it in the form of a Promela model. A test harness is used to exercise the file store in a number of ways. Model checking technology has been extensively used to verify the correctness of our implementation. A distinguishing feature of our approach is the (bounded) exhaustive verification of power loss recovery.
Chapter
Full-text available
Tony Hoare’s many contributions to computing science are marked by insight that was grounded in practical programming. Many of his papers have had a profound impact on the evolution of our field; they have moreover provided a source of inspiration to several generations of researchers. We examine the development of his work through a review of the development of some of his most influential pieces of work such as Hoare logic, CSP and Unifying Theories.
Article
Full-text available
The state of knowledge in how to specify sequential programs in object-oriented languages such as Java and C# and the state of the art in automated verification tools for such programs have made measurable progress in the last several years. This paper describes several remaining challenges and approaches to their solution.
Article
The verifying compiler (VC) project proposals suggest that mainstream software developers are its targeted end-users. Like other software engineering efforts, the VC project success depends on appropriate end-user consultation. Industrial use of program assertions for the purpose of run-time assertion checking (RAC) is becoming commonplace. A likely next step on the path to VC adoption is the use of assertions in extended static checking (ESC), a fully automated form of static program verification (SPV). Unfortunately, all current VC prototypes supporting SPV, adopt a semantics which is unsound relative to the standard run-time interpretation of assertions. In this article, we report on the results of a survey in which we asked industrial developers what logical semantics they want program assertions to have, and whether consistency across RAC and SPV tools is important. Survey results indicate that developers are in favor of a semantics for assertions that is compatible with their current use in RAC.
Conference Paper
We propose an ASM-based method to integrate into current feature-based software design practice modular verification techniques.
Conference Paper
Difficulties in reasoning about functional correctness and relational properties of object-oriented programs,are reviewed. An approach using auxiliary state is briefly described, with emphasis on the author’s work. Some near term challenges are sketched. Formal verification depends on scientific theories of programming, which answer
Conference Paper
The pointfree transform offers to the predicate calculus what the Laplace transform offers to the differential/integral calculus: the possibility of changing the underlying mathematical space so as to enable agile algebraic calculation. This paper addresses the foundations of the transform and its application to a calculational approach to extended static checking (ESC) in the context of abstract modeling. In particular, a calculus is given whose rules help in breaking the complexity of the proof obligations involved in static checking arguments. The close connection between such calculus and that of weakest pre-conditions makes it possible to use the latter in ESC proof obligation discharge, where pointfree notation is again used, this time to calculate with invariant properties to be maintained. A connection with the “everything is a relation” lemma of Alloy is established, showing how close to each other the pointfree and Alloy notations are. The main advantage of this connection is that of complementing pen-and-paper pointfree calculations with model checking support wherever validating sizable abstract models.
Conference Paper
Full-text available
We propose a collaboration project to integrate the research effort and results obtained at UNU-IIST on formal techniques in component and object systems with research at TRDDC in modelling and development of tools that support object-oriented and component-based design. The main theme is an integration of verification techniques with engineering methods of modelling and design, and an integration of verification tools and transformation tools. This will result in a method in which a correct program can be developed through transformations that are either proven to be correct or by showing that the transformed model can be proven correct by a verification tool.
ResearchGate has not been able to resolve any references for this publication.