Technical ReportPDF Available

Composable Rationalized Microstandards

Authors:
  • Dependable Computing
  • Dependable Computing

Abstract and Figures

This report details research into the construction of rationalized microstandards to guide and assure adequate reverse and reengineering of software modules for improved correctness. The microstandards included cover informal reverse engineering of a software module used by multiple systems, assurance through formal specification and proof, and assured development of source code from formal specification under retrenchment rules. The researchers show how these standards area composable into a whole for reengineering, such that ecosystems of microstandards could be developed for common engineering activities.
Content may be subject to copyright.
Tech Report: DepComp-TR-2022-01
Composable
Rationalized Microstandards
Jonathan Rowanhill, PhD
Benjamin Hocking, PhD
07/15/2022
2
Abstract. The rationalized microstandard is a proposed structure for standards
meant to improve costs and strength of assurance for systems, processes, and
analyses. A rationalized microstandard includes an explicated rationale written
as an assurance argument template that explains why a conformant subject sup-
plying the required evidence is believed to have a claimed property. By writing
this rationale in a modeling language such as Goal Structuring Notation (GSN),
it becomes an artifact upon which iterative, community-based development of
standards can occur. However, the characteristics of complete arguments backing
such standards has not yet been examined. This work presents a study of the
characteristics of a prototype rationalized microstandard assuring correct reengi-
neering of a software module through formal specification. The potential stand-
ard’s characteristics are presented and analyzed, including the size, scope, form,
and required conformance evidence of its GSN argument. An initial characteri-
zation of the standard’s composition from other potential standards is reviewed.
The results of this study indicate that a working rationalized microstandard can
be built with reasonable argument, evidence, and opportunities for composabil-
ity.
Keywords: rationalized microstandard, assurance argument, standard, compos-
able standards
1 Introduction
Developing strong assurance arguments for systems is typically expensive and requires
significant expertise in domains of assurance. One way to potentially reduce cost and
risk in assurance is to encourage the development and application of rationalized stand-
ards [1]. The rationalized standard is a structure for standards in which they are built
from rigorous assurance arguments. Each such standard claims a property about con-
formant subjects. The standard’s assurance argument explains why the claim would
hold for an intended class of conformant subjects and backs up the argument with the
evidence conformant subjects must provide. Subjects that provide this evidence and are
of the intended class are assured because they have provided the evidence required by
the argument to support the standard’s claim.
A rigorous assurance argument can be modeled in a language such as Goal Struc-
turing Notation (GSN) [2]. Therefore, a rationalized standard’s argument is a concrete
artifact that can be maintained and contributed to as carefully governed “open source”.
This might enable continuous improvement of standards. It also introduces the potential
for an economy of scope in assurance emerging from an ecology of open-source stand-
ards shared in repositories. If these standards are small and well-focused, then they are
microstandards (or ministandards) that can improve the specificity of assurance for
properties such as safety [3]. Furthermore, just as open-source software is composed of
technology stacks, so too might rationalized microstandards be composed of assurance
stacks for systems, processes, and analyses.
However, it remains to be seen if practical rationalized microstandards can be built.
Furthermore, the size, argument form, required conformance evidence, and potential
for composability of such standards has only been speculatively described.
3
This paper presents a study of the feasible development and resulting properties of a
prototype rationalized microstandard. This microstandard is developed for specific but
re-usable engineering activitythe replacement of an existing software module with
one re-engineered using formal specification.
The remainder of this paper details the results of this study. Section 2 presents the
goals of the feasibility study and related work. Section 3 presents the results of the
feasibility study analyzing the development cost and content of the developed mi-
crostandard. The potential for composability within the microstandards is addressed.
Section 4 draws conclusions about the potential for future research and development of
modular standards driven by explicitly modeled rationale.
2 Feasibility Study
The goal for the presented work is to better understand the development cost and prop-
erties of potentially useful rationalized microstandards. The following questions and
characteristics are to be addressed and explored in his feasibility study:
Feasible Construction: The extent to which a reasonable, first iteration of a ration-
alized microstandard prototype can be constructed for a valuable use case.
Development Cost and Characterization: The costs in time for artifacts defining
a rationalized microstandard.
Argument: An understanding of the size and form of a concrete rationalized mi-
crostandard. This includes the structure of the argument as well as the semantics of
its reasoning, supporting claims, and evidence.
Conformance Cost and Characterization: The extent and form of evidence that
the must be obtained for conformance, and the nature of any process that lends to
reasonable completion of the example rationalized microstandard.
Composability: The extent to which the developed microstandard can rely on com-
position from other independently useful microstandards.
2.1 Use Case
The study is conducted for a use case of formal methods. Specifically, the standard is
designed for the use case of replacing a target software module with a version re-engi-
neered using formal specification, as illustrated in Fig. 1. A software source code mod-
ule M is to be replaced with a version M’ that is reengineered through formal specifi-
cation. The goal of this reengineering is to improve assurance of the module’s func-
tional and nonfunctional behavior in the systems of interest.
4
Fig. 1. An illustration of the formal methods use case. A software module used in one or more
software systems (illustrated in the intersection of the systems) is replaced by a version better
assured through formal specification and verification.
This use case typically arises when the module M is identified to be important in the
provision of critical functionality. By reengineering the module through a formal spec-
ification process, one can produce deductive proof of some requirements of the module
against the specification. With careful reasoning and specification and source code de-
sign, one can assure that these proofs hold for the resulting software implementation.
2.2 Related Work
There are many standards and conformance application processes that require inclusion
of assurance arguments or safety cases in included work, such as the UL-4600 standard
for autonomous vehicles [4] and ISO 26262 [5]. However, rationalized microstandards
are not relying on the use of argument as conformance evidence. Instead, a rationalized
microstandard’s argument represents the rationale behind the standard itself.
Past work has attempted to reverse engineer the rationale of the DO-178C standard
as an assurance argument [6]. Some standards do possess partial, informal rationale
either in prologue or as support for understanding asks of conformance and compliance.
There is significant work on use of safety case argument patterns [7] and argument
fragments [8] to achieve argument scale and strength through composability. Such work
achieves re-usability of an argument pattern by contextualizing it within a larger argu-
ment and extending it to be more specific. In contrast, the work presented here focuses
on complete arguments modeling detailed, contextually situated rationale for assuring
specific properties of specific classes of system and activities. Standards developers
concentrate on developing its rational and specifying required evidence. Applicants
only focus on following guidance and collecting the required evidence [1].
3 Analysis of a Prototype Microstandard
The following two prototype rationalized microstandards were developed:
5
Reengineering a Software Module Using Formal Specification: claims
a conformant reengineering of a software module with formal specification
produces a correct and complete software module.
Informally Reverse Engineering A Software Module: claims a conform-
ant reverse engineering effort produces correct and complete interface con-
tracts for a software module for systems and qualities of interest.
The resulting microstandards can be found on GitHub [9], where they are open to in-
spection, comment, issue reporting, and discussion. The development cost of both mi-
crostandards is analyzed. The remainder of analysis is performed on the reengineering
microstandard.
3.1 Development Effort
The rationale microstandards were developed by one engineer and reviewed and edited
by another engineer. An assurance argument, evidence template spreadsheet, and guid-
ance document were written for each standard. Table 1 shows the developed cost of the
prototype rationale microstandards rounded to the nearest ten hours.
Table 1. Measures of Development Effort for Prototype Rationalized Microstandards
Microstandard
Total
Time
(hrs.)
Argument De-
velopment
Time (hrs.)
Evidence Tem-
plate
Development
Time (hrs.)
Guidance
Document
Develop-
ment Time
(hrs.)
Reengineering using
Formal Specification
90
50
10
30
Reverse Engineering
of Module Contracts
90
50
20
20
Roughly 60% of development time was spent in argument work. Each standard’s argu-
ment went through more than 5 major iterative revisions before final prototype release.
Top-down changes occurred due to attempts to strengthen and clarify reasoning, while
bottom-up changes occurred as the practicality of obtaining evidence was considered.
3.2 Argument Form
The rationale for the reengineering microstandard was written as an assembling collec-
tion of Goal Structuring Notation (GSN) [2] modules. The total argument for the stand-
ard, assembled from its modules, is illustrated in Fig. 2. Overlaying the argument’s
structure are labelled regions of argument for which semantics is discussed in the next
sub-section.
6
Fig. 2. The Rationale of the Formal-specification-based Reengineering Microstandard
Table 2 presents the number of GSN nodes of each type utilized in the rationale.
Table 2. GSN Elements
Node Type
Count
Goals
126
Away Goals
8
Assumptions
15
Justifications
16
Context
65
Strategies
56
Solutions
86
3.3 Argument Semantics
As the rationale for the standard is inductive logic, it is important to directly present the
core reasoning to the reader. For reasons of presentation and discussion, this section
presents the rationale in an abstracted form, illustrated in simplified GSN. Each goal
node of the abstraction is labelled with a letter. These correspond with the letters label-
ling sections of the full argument illustrated in Fig. 2. Each shaded region of that figure
corresponds to an illustrated abstracted argument discussed in the remainder of this
subsection.
The root claim of the rationale is that the reengineered version, M’, of software mod-
ule M is fit for use in replacing the software module M in each set of systems of interest
for functionality and a specified set of nonfunctional qualities of interest (e.g., safety
and security). The support for this claim is presented in Fig. 3.
7
Fig. 3. The top level of the rationale for the formal-specification-based reengineering mi-
crostandard (abstract view).
The argument for satisfactory M’ is based on overarching properties [10] [11], with
innocuity generalized to assert that unintended behaviors of M’ do not result in unac-
ceptable impact on any of the specified nonfunctional qualities of interest.
The overarching property concept of defined intended behavior is replaced with the
more specific capture mechanism of interface contracts. Interface contracts define as-
sume/guarantee terms for M’ that if correct and complete, define all required behavior
for M’ so that each system satisfies its functional and nonfunctional requirements.
Intent. Fig. 4 presents an abstract view of the intent argument. Intent of the software
module is defined as a set of natural language assume/guarantee contracts on the inter-
face of the software modulefunctions and data accessible to the outside systems
source code where the module is either acted upon or acts upon the surrounding system
source code elements. The interface contracts capture the currently required behavior
of the module as interaction with systems of concern.
The argument is made that correct and complete interface contracts represent intent
for module M’ are developed from correct and complete interface contracts capturing
the actual behavior of the current software module, amended to define intent correctly
through sufficient detection and correction of implementation error, and updated for
current desired behavior from a complete set of behavior changes asked for by stake-
holders. The former claim is backed by the second developed microstandard.
D: Innoc uity
Unintended behavior of M’ does
not undermine qualities of interest.
C: Cor rectness
M’ Source Code Satisfies
Interface Contracts.
B: Int ent
Correct and Complete
Interface Contracts for M’.
Strategy
through satisfaction of generalized
Overarching Properties.
A: Sa tisfactory M
by Generalized Overarching
Properties
8
Fig. 4. The intent argument of the reengineering microstandard (abstract view).
Correctness. The correctness wing of the argument, the top of which is illustrated in
Fig. 5 below, occupies the bulk of the standard’s rationale. This is because verifying a
source module engineered from verified formal specification requires careful attention
to details of relationships between interface contracts, verifiable properties, formal
specification, proof construction, proof execution, and source code synthesis.
Fig. 5. The correctness argument of the formal-specification-based reengineering standard’s ra-
tionale (abstract view).
The correctness argument is based on the concept of implied contract guarantees. Each
contract guarantee (which may not be easily verified itself) is argued to be reasonably
implied by a set of sufficient, verifiable properties. Those properties are argued to be
each correctly verified indirectly against a formal specification and/or directly against
Micr ostandar d: Source Code V and V
A Microstandard for general V and V techniques.
P: Directly Verified Properties
On source code of M'.
O: Indirectly Verified Properties
On the formal specification of M'.
N: Prope rty Verificati on Coverage
All properties are verified on the
reeng ineered module .
I: Verified Properties
All properties are verified on M'.
H: Impli ed Contract Guarantees
Each contract is satisfied by a set of
properties of M'.
C: Cor rectness
M’ Source Code Satisfies
Interface Contracts
9
source code. In the latter case, a hypothesized microstandard for standard verification
techniques of source code is hypothesized but out of scope for this effort.
The center of the rationale is the argument of correct indirect verification through
formal specification. This is illustrated in Fig. 6, arguing that indirect verification is
correct because the properties are correctly verified on the module’s formal specifica-
tion, and the source code implementation of the specification is “sufficiently correct”
such that those properties will also hold for the source code.
Verification against formal specification is argued to be correct through a complete
mapping of properties to lemmas where each lemma is argued to correctly encode its
intended property in the context of the specification. The correctness of support lemmas
and axioms is also argued. It is further argued that property lemmas are all either proven
and/or verified with correct testing against an animated/executable specification.
The claim that the source code implementation of the specification is “sufficiently
correct” is supported by argument that the source code is “correct within accumulating
scalar value error” and that that verified properties continue to hold on the source code
implementation under the possible scalar value error. The latter is argued by numerical
analysis or having stated allowed error specifically in verified properties.
Fig. 6. The argument (abstract) assuring correct indirect verification of module properties.
Retrenchment models [12] formally map between corresponding values and operations
on data types off the specification and source code, e.g., between real numbers and
IEEE floating-point numbers. “Correct within scalar value error” is defined to mean
that for the same inputs (under retrenchment) to ‘execution’ of formal specification and
its implementation, the executions will follow the same conditional paths and only dif-
fer in accumulating error of scalar data value error. It is argued that this correctness
condition is achieved under a valid retrenchment model.
X: Properties Hold
Under possible scalar value error
under the retrenchment rules.
AA: Cor rect Part by Simulation Condition
A part of the specification is correct to within
only accumulated scalar value error under the
retrenchment model.
V: Va lid R etre nch ment M odel s
Valid retrenchent models for the
intended source language and
OS/Hardware environment(s).
W: C orrec t Sourc e Co de
to Within accumulated scalar value
error under the retrenchment models.
S: Sufficiently Correct Source Code
for the properties verified on the specification
to hold on the source code implementation.
CC: Lemm as Correctly Verified by Test ing
A subset of the lemmas.
BB: Subs et of Lemma s are Proven
A subset of lemmas are proven
Z: Lemmas Verifie d
including supporting lemmas.
Y: Co rre ct Lem mas and Ax io ms
To verify properties and provide
required support.
U: Lemm as Verified
All lemmas representing
properties are adequately verified.
R: Prope rties Verifie d on Formal Specificati on
of M'
O: Indirectly Verified Properties
On the formal specification of M'.
T: Le m m a fo r E a c h Pr o pe r ty
In the formal specification.
10
How one shows the correctness relation can vary by the nature of a specification and
its properties. Therefore, the standard argues that the specification consists of correct
parts that compose correctly. The argument for assuring the correctness criteria of a
part is illustrated in Fig. 7. The argument asserts two sufficient subclaims. First, the
implementation part correctly implements the formal semantics of its specification in
code and data definitions under the defined retrenchment models. Second, the specifi-
cation part stepwise simulates [12] its source code implementation under the retrench-
ment.
The first claim is assured by applying “mirroring” implementation rules between the
specification and source language [13]. In the standard, SPARK Ada is assumed for use
to prove the formal semantics of the specification and retrenchment rules are satisfied.
This involves showing that verification conditions generated by Spark Ada are ade-
quately verified and that they include consideration for overflow and underflow under
retrenchment.
Fig. 7. The argument for a sufficiently correct implementation part (abstract).
The second claim, that the functional specification part is a stepwise simulation of its
implementation source code (except for accumulating scalar value error) relies on as-
suring proper source code and analysis of specification based on specification type.
Either (a) it is proven that the source code satisfies the formally specified functionality,
or (b) it is shown that the source code is an operationally correct model of a function
under retrenchment and that all data stays in allowed ranges of source code without
imprecision of values from the specification, or (c) properties need to have been proven
against a nondeterministic specification [14] utilizing interval arithmetic and determi-
nation that any bound imprecision in lemmas is sound for the source code implementa-
tion.
Innocuity. The innocuity argument, illustrated abstractly in Fig. 8, claims that innocu-
ity holds because the interface contracts define all required behavior to satisfy all non-
II: Declarative Simulation
BY declarative specification as simulating
preconditions and postconditions.
HH: Determ inistic Simul ation
By pure refinement given
retrenchment models.
GG: Nondeterministic Simulation
by Nondeterministic Specification with
sound bound specified scalar error ranges.
Within Scal ar Valu e Er ror
Means that all conditional behavior
is correctly simulated and only
scalar error on values differs.
EE: Simulation to within Scalar Value Error
The specification part's definition (down to operations)
simulates the implementation part to within accumulating
scalar value error (under the retrenchment model)
LL: VCs Addressed
All verification conditions generated
by Spark ADA are either satisfied or
otherwise satisfactorily addressed.
KK: Accept able Ranges
The retrenchment mapping and
source code stay within allowed
ranges without overflow or underflow.
FF: Correct In Spa rk Ada
of the spark ada part
DD: Correc t Implemen tation of Op erations and Data
In Spark Ada for the Specification Part.
JJ: Correct Mirrored Implementation
The operations and data types of the specification
are mirrored correctly to source code.
AA: Correc t Part by Simulation C ondition
A part of the specification is correct to within
only accumulated scalar value error under the
retrenchment model.
11
functionals of interest for tracked system services. Therefore, correctness of the inter-
face contracts (intent) and their correct satisfaction (correctness) are sufficient to show
innocuity. In addition, the standard might recommend other approaches to system-level
verification against the new module’s source code, such as through behavior coverage
analysis using analysis from symbolic execution.
Fig. 8. The innocuity argument of the formal-specification-based reengineering microstand-
ard’s rationale (abstract).
Note that this innocuity argument relies on a GSN away goal (to the lower left) that
comes from within the argument implementing the reverse engineering microstandard
applied in the intent wing. This is an example of an implementation dependency be-
tween microstandards. The ramifications of this are discussed later.
3.4 Conformance Evidence and Conformance Work
The solution nodes of a microstandard’s rationale are important as they elucidate the
evidence required by conformant systems. As the standard developed, two types of so-
lutions were distinguished as follows:
Invariant Solutions: Solutions applying evidence that is independent of conform-
ance subjects, such as textbook knowledge or facts.
Conformance Solutions: Solutions applying evidence that is dependent on a partic-
ular conformance subject such as proofs of lemmas about a particular module.
Of the solution nodes of the standard, seven are invariant. The remainder are conform-
ance solutions.
Each conformance solution refers to one or more forms of required conformance
evidence. The developed microstandards supply conformance evidence templates in a
computer spreadsheet. Each evidence field has a unique name, a description, and the
requirements for presented evidence instances.
An analysis of the required conformance evidence fields shows that evidence fields
consist of recurring categories shown in Table 3.
Innocuity of Source Code via Symbolic Execution Analysis
The unintended behavior of the source code of the prime software module
is innocuous for the {qualities of interest} of the {systems of interest}
Intent
.
Q: Invoked Function Behavior Satisfies
Invoking Function Requirements
K: System Service-Lev el Verificati on
by the prime source code module's behavior.
Contracts are Suffici ent Satisfac tion of Syste m Requirem ents
for qualities of interest. This is a claim from within the
informalReverseEngineering microstsandard.
J: Interface Contracts Guarantee Innocuity
for attributes of interest
D: Innocui ty
Unintended behavior of M’ does
not undermine qualities of interest.
Correctn ess
.
12
Table 3. Evidence field types of the microstandard evidence template.
Field Type
Count
Artifact identifier
27
Boolean
13
Enumeration
3
Description
15
Part identifier
5
Issue
3
Sign-off
47
Technical artifact
17
Grand Total
117
Analysis of the instantiation requirements for evidence indicates that the expected scale
of required evidence for a conformant system is proportional to the number of lemmas
and axioms required to verify the properties that imply guarantees of interface con-
tracts.
3.5 Modular Composability
For this analysis, modular composability is defined as the ability to re-use the assur-
ance claims of one rationalized microstandard to support the assurance claims of an-
other rationalized microstandard such that both microstandards are useful apart from
one another. During the development of the reengineering microstandard’s intent wing
of its argument, it was determined by the authors that modular composability existed
between argument assuring reengineering and assuring reverse engineering of module
source code to interface contracts is correct and complete. Properties noted in this de-
coupling were:
Re-usable subject matter. The subjects about which properties are assured are val-
uable to more than a parent microstandard.
Common Context. The context in which a property about a subject is true is com-
mon to many use cases.
Composition was possible because of the following properties of the arguments:
Public Interface Claims: The root claim, or a set of claims of a microstandard’s
rationale are declared a public interface (e.g., as supported in SACM2 [16]) that is
less likely to be revised than the underlying argument.
Explicit, Necessary Context and Assumptions: Interface claims must clearly state
the assumptions and context that they require for their underlying argument.
Compatible Terminology and Ontological Constructs: Terminology and ontolog-
ical relations must be clear between the domains of standards.
Given these characteristics, an analysis was performed on the two produced mi-
crostandards to propose the assurance stack illustrated in Fig. 9. The stack consists of
modular rationalized microstandards. Edges of the illustrated stack are labeled with
13
some of the terminology and concepts that would be shared between them. Microstand-
ards illustrated in gray have developed rationale and evidence templates. All except the
two developed for this study would require separate development of guidance docu-
mentation.
Fig. 9. The prototype assurance stack of prototype and hypothesized rationalized microstand-
ards from which the reengineering microstandard is composed.
Note that many of the stack’s components are subjects of concern in existing safety
standards, such as reference in DO-178C to behavior coverage, test correctness, and
tool qualification [17], and in modules of academic papers such as retrenchment models
and simulation [12] and precision analysis [18]. Therefore, there is considerable ra-
tionale to apply to their further development and surrounding communities of interest
that might develop and maintain them.
(C) Correct
Verication of a
Formally Specied
Sou rce Code
Module against
Interface Contracts
(L) Correct and Complete
Reverse Engineering of a
Sou rce Code Modul e
(P) Correct
Verication
of Source
Code
(L) Correct Documentation
Mining
(L) Correct Algorithm
Performance Analysis
(K) Correct
Innocuity
Determination from
Behavior Coverage
Analysis of Source
Code
Quality
Requirem ents
Requirem ents
Inte rface Contrac t
Software Module
Systems of Interest
Qualities of Interest
(A) Reengineering Veried Software Modules through Formal Specication
(W) Correct-to-Within-
Sca lar-V alue -Erro r
Implementation of a
Formal Functional
Spe ci cat i on
(FF) Correct Spark
Ada Code fro m a
Formal Functional
Spe cica tion under
Retrenchment
(X) Correctly
Veried Properties
under Numerical
Analysis of
Sof twar e Sou rce
Code
(EE) Stepwise
Si mulat ion
Condi tion for
Sou rce Code in
Functional
Spe cica tion under
Retrenchment
(B) Correct and
Compl ete Interfa ce
Contracts from
Reverse Engineering
of a Software
Module
(V) Validity of
Retrenchment
Models
(L) Correct Verification of
Functional Formal
Spe cifi cat i ons
Inte rface Contrac t
Infe renceab le Comp letene ss
Co rrect to Withi n S cala r Va lue E rro r
Retren chment
Formal F unctional S peci cation
Co rrect to Withi n S cala r Va lue E rro r
14
4 Conclusions
A prototype rationalized microstandard was presented, and its form, reasoning, and re-
quired conformance evidence summarized. Based on this work, we believe that such
standards can be written and are of a manageable size and complexity comparable to
many open-source software projects.
We note that roughly 40% of required conformance evidence fields consisted of
sign-off activities from team leads and domain experts (and a few signatures from
stakeholders.) This is because many of the properties utilized to assure activities in
defining the intent for a software module, formal specification, and source code author-
ing rely on properties of artifacts that currently have no supported tooling. Some such
properties, such as completeness of property verification, can be automated through
spreadsheet mechanisms. But others, such as judging that lemma correctly encode in-
tended properties remain informally judged by development teams. There are many
opportunities for better tooling in end-to-end assurance that could emerge from study-
ing reference processes and the resulting assurance sign-off evidence.
The semantics of the prototype standard has been through many revisions and could
continue to improve for clarity, strength of reasoning, strength and practicality of con-
formance evidence, and usability of context and assumptions. In each iteration of the
standard, doubts and even defeaters were discovered for the rationale through applica-
tion of defeasible reasoning. It is likely that this would continue with further work on
the standard.
It is an open question as to whether teams of domain experts could work together to
achieve a central agreed upon rationale for a given microstandard. Norms and govern-
ance are required for participation in the modern open-source software community, and
groups building rationalized microstandards could adopt similar community models. In
addition, just as software ideas with re-use potential go through many revisions and
negotiation on interface and interface changes over time with other projects, one might
expect similar requirements on the relationships between standards in an assurance
stack.
Having developed rational microstandards for a practical use case, the authors intend
to apply the standards to ongoing research work in reverse engineering and formal
methods. Study of the utility and efficiency of the prototype standards will be attempted
for a practical but challenging reengineering application.
Acknowledgment: This work supported in part by the U.S. AFRL as part of the
CRDInAL project.
References
1. Knight, J. C., & Rowanhill, J. (2016, September). The indispensable role of rationale in
safety standards. In International Conference on Computer Safety, Reliability, and Secu-
rity (pp. 39-50). Springer, Cham.
15
2. Kelly, T., & Weaver, R. (2004, July). The goal structuring notationa safety argument nota-
tion. In Proceedings of the dependable systems and networks 2004 workshop on assurance
cases (p. 6).
3. Fenton, N. E., & Neil, M. (1998). A strategy for improving safety related software engineer-
ing standards. IEEE Transactions on Software Engineering, 24(11), 1002-1013.
4. Koopman, P., Ferrell, U., Fratrik, F., & Wagner, M. (2019, September). A safety standard
approach for fully autonomous vehicles. In International Conference on Computer Safety,
Reliability, and Security (pp. 326-332). Springer, Cham.
5. Birch, J., Rivett, R., Habli, I., Bradshaw, B., Botham, J., Higham, D., ... & Palin, R. (2013,
September). Safety cases and their role in ISO 26262 functional safety assessment. In Inter-
national Conference on Computer Safety, Reliability, and Security (pp. 154-165). Springer,
Berlin, Heidelberg.
6. Holloway, C. M. (2015, February). Explicate'78: Uncovering the implicit assurance case in
do-178c. In Safety-Critical Systems Symposium 2015 (SSS 2015) (No. NF1676L-20463).
7. Hawkins, R., Clegg, K., Alexander, R., & Kelly, T. (2011, September). Using a software
safety argument pattern catalogue: Two case studies. In International Conference on Com-
puter Safety, Reliability, and Security (pp. 185-198). Springer, Berlin, Heidelberg.
8. Šljivo, I., Gallina, B., Carlson, J., Hansson, H., & Puri, S. (2017). A method to generate
reusable safety case argument-fragments from compositional safety analysis. Journal of Sys-
tems and Software, 131, 570-590.
9. github.com/DependableComputing.
10. Holloway, C. M. (2019). Understanding the overarching properties (No. NF1676L-33745).
11. Aiello, M. A., Comar, C., & Ruiz, J. F. (2020). An Assurance Case based on Overarching
Properties for a TQL1 Code Generator. ERTS2020.
12. Banach, R., & Poppleton, M. (2000, August). Retrenchment, refinement, and simulation.
In International Conference of B and Z Users (pp. 304-323). Springer, Berlin, Heidelberg.
13. Hocking, A. B., Rowanhill, J. C., & Di Vito, B. L. (2020, October). An analysis of imple-
menting PVS in SPARK Ada. In 2020 AIAA/IEEE 39th Digital Avionics Systems Confer-
ence (DASC) (pp. 1-7). IEEE.
14. Rowanhill, J. Hocking, A.B, Hawkins, W. Nondeterministic Specification in the Reals for
Software Refinement. Technical Report. Dependable Computing. March 22, 2021.
15. Pratt, Vaughan R. "Semantical considerations on Floyd-Hoare logic." 17th Annual Sympo-
sium on Foundations of Computer Science (sfcs 1976). IEEE, 1976.
16. www.omg.org/spec/SACM/2.1/PDF
17. Jacklin, S. (2012). Certification of safety-critical software under DO-178C and DO-278A.
In Infotech@ Aerospace 2012 (p. 2473).
18. Cherubin, S., & Agosta, G. (2020). Tools for reduced precision computation: a survey. ACM
Computing Surveys (CSUR), 53(2), 1-35.
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
SPARK Ada's support for proofs of correctness make the programming language ideal for implementing a PVS specification. Algorithmically implementing a PVS specification in SPARK Ada allows users to maintain the rigor of PVS in executable code. The goal of such an implementation is to maintain the validity of the proofs showing the specification implements formal requirements specified in PVS as theorems. This then shows the implementation also satisfies those formal requirements. We synthesized portions of NASA's DAIDALUS (Detect and AvoID Alerting Logic for Unmanned Systems) PVS specification into SPARK Ada. To provide confidence in the correspondence between the PVS specification and the SPARK Ada implementation, we designed a formal synthesis process. This process, while currently manual, allows us to have increased confidence that the properties proven to hold for the specification will continue to hold for the implementation.
Conference Paper
Full-text available
In this paper, we argue that standards, especially those intended to support critical applications, should define explicitly both the properties expected to accrue from use of the standard and an explicit rationale that justifies the contents of the standard. Current standards do not include an explicit, comprehensive rationale. Without a rationale, the use, maintenance, and revision of standards is unnecessarily difficult. We introduce a new concept for standards, the rationalized standard. A rationalized standard combines: (a) an explicit goal defining a property desired for conformant systems, (b) guidance that, if followed correctly, should yield an entity with the property stated in the goal, and (c) the rationale showing the reasoning why there is assurance with reasonable confidence that a conformant entity will have the property defined by the goal. We illustrate the utility of an explicit rationale using an existing safety standard, ISO 26262.
Conference Paper
Full-text available
Compliance with the automotive standard ISO 26262 requires the development of a safety case for electrical and/or electronic (E/E) systems whose malfunction has the potential to lead to an unreasonable level of risk. In order to justify freedom from unreasonable risk, a safety argument should be developed in which the safety requirements are shown to be complete and satis-fied by the evidence generated from the ISO 26262 work products. However, the standard does not provide practical guidelines for how it should be devel-oped and reviewed. More importantly, the standard does not describe how the safety argument should be evaluated in the functional safety assessment proc-ess. In this paper, we categorise and analyse the main argument structures re-quired of a safety case and specify the relationships that exist between these structures. Particular emphasis is placed on the importance of the product-based safety rationale within the argument and the role this rationale should play in assessing functional safety. The approach is evaluated in an industrial case study. The paper concludes with a discussion of the potential benefits and chal-lenges of structured safety arguments for evaluating the rationale, assumptions and evidence put forward when claiming compliance with ISO 26262.
Conference Paper
Full-text available
Software safety cases encourage developers to carry out only those safety activities that actually reduce risk. In practice this is not always achieved. To help remedy this, the SSEI at the University of York has developed a set of software safety argument patterns. This paper reports on using the patterns in two real-world case studies, evaluating the patterns' use against criteria that includes flexibility, ability to reveal assurance decits and ability to focus the case on software contributions to hazards. The case studies demonstrated that the safety patterns can be applied to a range of system types regardless of the stage or type of development process, that they help limit safety case activities to those that are significant for achieving safety, and that they help developers nd assurance deficits in their safety case arguments. The case study reports discuss the difficulties of applying the patterns, particularly in the case of users who are unfamiliar with the approach, and the authors recognise in response the need for better instructional material. But the results show that as part of the development of best practice in safety, the patterns promise signicant benets to industrial safety case creators.
Article
The use of reduced precision to improve performance metrics such as computation latency and power consumption is a common practice in the embedded systems field. This practice is emerging as a new trend in High Performance Computing (HPC), especially when new error-tolerant applications are considered. However, standard compiler frameworks do not support automated precision customization, and manual tuning and code transformation is the approach usually adopted in most domains. In recent years, research have been studying ways to improve the automation of this process. This article surveys this body of work, identifying the critical steps of this process, the most advanced tools available, and the open challenges in this research area. We conclude that, while several mature tools exist, there is still a gap to close, especially for tools based on static analysis rather than profiling, as well as for integration within mainstream, industry-strength compiler frameworks.
Article
Safety-critical systems usually need to be accompanied by an explained and well-founded body of evidence to show that the system is acceptably safe. While reuse within such systems covers mainly code, reusing accompanying safety artefacts is limited due to a wide range of context dependencies that need to be satisfied for safety evidence to be valid in a different context. Currently, the most commonly used approaches that facilitate reuse lack support for systematic reuse of safety artefacts.
Conference Paper
The Radio Technical Commission for Aeronautics (RTCA) has recently released DO-178C and DO-278A as new certification guidance for the production of airborne and ground-based air traffic management software, respectively. Additionally, RTCA special committee SC-205 has also produced, at the same time, five other companion documents. These documents are RTCA DO-248C, DO-330, DO-331, DO- 332, and DO-333. These supplements address frequently asked questions about software certification, provide guidance on tool qualification requirements, and illustrate the modifications recommended to DO-178C when using model-based software design, object oriented programming, and formal methods. The objective of this paper is to first explain the relationship of DO-178C to the former DO-178B in order to give those familiar with DO-178B an indication of what has been changed and what has not been changed. With this background, the relationship of DO-178C and DO-278 to the new DO-278A document for ground-based software development is shown. Last, an overview of the new guidance contained in the tool qualification document and the three new supplements to DO-178C and DO-278A is presented. For those unfamiliar with DO-178B, this paper serves to provide an entry point to this new certification guidance for airborne and ground-based Communication, Navigation, Surveillance and Air Traffic Management (CNS/ATM) software certification.
Conference Paper
This paper deals with logics of programs. The objective is to formalize a notion of program description, and to give both plausible (semantic) and effective (syntactic) critera for the notion of truth of a description. A novel feature of this treatment is the development of the mathematics underlying Floyd-Hoare axiom systems independently of such systems. Other directions that such research might take are also considered. This paper grew out of, and is intended to be usable as, class notes...
Conference Paper
Retrenchment is introduced as a liberalisation of refinement intended to address some of the shortcomings of refinement as sole means of progressing from simple abstract models to more complex and realistic ones. In retrenchment the relationship between an abstract operation and its concrete counterpart is mediated by extra predicates, allowing the expression of non-refinement-like properties and the mixing of I/O and state aspects in the passage between levels of abstraction. Modulated refinement is introduced as a version of refinement allowing mixing of I/O and state aspects, in order to facilitate comparison between retrenchment and refinement, and various notions of simulation are considered in this context. Stepwise simulation, the ability of the simulator to mimic a sequence of execution steps of the simulatee in a sequence of equal length is proposed as the benchmark semantic notion for relating concepts in this area. One version of modulated refinement is shown to have particularly strong connections with automata theoretic strong simulation, in which states and step labels are mapped independently from simulator to simulatee. A special case of retrenchment, simple simulable retrenchment is introduced, and shown to have properties very close to those of modulated refinement. The more general situation is discussed briefly. The details of the theory are worked out for the B-Method, though the applicability of the underlying ideas is not limited to just that formalism.
Article
There are many standards which are relevant for building safety- or mission-critical software systems. An effective standard is one that should help developers, assessors and users of such systems. For developers, the standard should help them build the system cost-effectively, and it should be clear what is required in order to conform to the standard. For assessors, it should be possible to objectively determine compliance to the standard. Users, and society at large, should have some assurance that a system developed to the standard has quantified risks and benefits. Unfortunately, the existing standards do not adequately fulfil any of these varied requirements. We explain why standards are the way they are, and then provide a strategy for improving them. Our approach is to evaluate standards on a number of key criteria that enable us to interpret the standard, identify its scope and check the ease with which it can be applied and checked. We also need to demonstrate that the use of a standard is likely either to deliver reliable and safe systems at an acceptable cost or to help predict reliability and safety accurately. Throughout the paper, we examine, by way of example, a specific standard for safety-critical systems (namely IEC 1508) and show how it can be improved by applying our strategy