ArticlePDF Available

An Overview of JML Tools and Applications

Authors:
  • GrammaTech, Inc.

Abstract and Figures

The Java Modeling Language (JML) can be used to specify the detailed design of Java classes and interfaces by adding annotations to Java source files. The aim of JML is to provide a specification language that is easy to use for Java programmers and that is supported by a wide range of tools for specification typechecking, runtime debugging, static analysis, and verification.This paper gives an overview of the main ideas behind JML, details about JML’s wide range of tools, and a glimpse into existing applications of JML.
Content may be subject to copyright.
This article appears in Software Tools for Technology Transfer.
The original version is available at www.springerlink.com
An overview of JML tools and applications
Lilian Burdy1, Yoonsik Cheon2, David R. Cok3, Michael D. Ernst4, Joseph R. Kiniry5, Gary T.
Leavens6?, K. Rustan M. Leino7, Erik Poll5
1INRIA, Sophia-Antipolis, France
2Dept. of Computer Science, University of Texas at El Paso, El Paso, Texas, USA
3Eastman Kodak Company, R&D Laboratories, Rochester, New York, USA
4Computer Science & Artificial Intelligence Lab, MIT, Cambridge, Massachusetts, USA
5Dept. of Computer Science, University of Nijmegen, Nijmegen, the Netherlands
6Dept. of Computer Science, Iowa State University, Ames, Iowa, USA
7Microsoft Research, Redmond, Washington, USA
Received: date / Revised version: date
Abstract. The Java Modeling Language (JML) can be
used to specify the detailed design of Java classes and
interfaces by adding annotations to Java source files. The
aim of JML is to provide a specification language that is
easy to use for Java programmers and that is supported
by a wide range of tools for specification type-checking,
runtime debugging, static analysis, and verification.
This paper gives an overview of the main ideas be-
hind JML, details about JML’s wide range of tools, and
a glimpse into existing applications of JML.
1 Introduction
JML [57,58], the Java Modeling Language, is useful for
specifying detailed designs of Java classes and interfaces.
JML is a behavioral interface specification language for
Java; that is, it specifies both the behavior and the syn-
tactic interface of Java code. The syntactic interface of a
Java class or interface consists of its method signatures,
the names and types of its fields, etc. This is what is
commonly meant by an application programming inter-
face (API). The behavior of such an API can be pre-
cisely documented in JML annotations; these describe
the intended way that programmers should use the API.
In terms of behavior, JML can detail, for example, the
preconditions and postconditions for methods as well as
class invariants, in the Design by Contract style [73].
An important goal for the design of JML is that it
should be easily understandable by Java programmers.
This is achieved by staying as close as possible to Java
syntax and semantics. Another important design goal is
that JML not impose any particular design methodology
on users; instead, JML should be able to document Java
programs designed in any manner.
?Supported in part by US NSF grants CCR-0097907 and CCR-
0113181
The work on JML was started by Gary Leavens and
his colleagues and students at Iowa State University. It
has since grown into a cooperative, open effort. Several
groups worldwide are now building tools that support
the JML notation and are involved with the ongoing
design of JML. For an up-to-date list, see the JML web-
site, www.jmlspecs.org. The open, cooperative nature
of the JML effort is important both for tool developers
and users, and we welcome participation by others. For
potential users, the fact that there are several tools sup-
porting the same notation is clearly an advantage. For
tool developers, using a common syntax and semantics
can make it much easier to get users interested. After all,
one of the biggest hurdles to using a new specification-
centric tool is often the lack of familiarity with the as-
sociated specification language.
The next section introduces the
JML notation. Sections 3 through 7
then discuss the tools currently avail-
able for JML in more detail. Section 8
discusses the applications of JML in
the domain of Java Card, the Java
dialect for programming smartcards.
Section 9 discusses some related lan-
guages and tools, and Section 10 con-
cludes.
2 The JML Notation
JML blends Eiffel’s Design by Contract approach [73]
with the Larch tradition [41,20, 56] (both of which share
features and ideas with VDM [52]).1Because JML sup-
ports quantifiers such as \forall and \exists, and be-
cause JML allows model (i.e., specification-only) fields
1JML also has takes some features from the refinement calculus
[75], which we do not discuss in this paper.
2 Burdy et al.: An overview of JML tools and applications
and methods, specifications can easily be made more
precise and complete than is typical for Eiffel software.
However, following Eiffel’s use of its expression syntax in
assertions, JML uses Java’s expression syntax in asser-
tions; this makes JML’s notation easier for programmers
to learn than notations based on a language-independent
specification language, such as the Larch Shared Lan-
guage [58,59] or OCL [91].
Figure 1 gives an example of a JML specification that
illustrates its main features. JML assertions are written
as special annotation comments in Java code, either after
//@ or between /*@ ... @*/, so that they are ignored
by Java compilers but can be used by tools that sup-
port JML. Within annotation comments, JML extends
the Java syntax with several keywords—in the example
in Figure 1, the JML keywords invariant,requires,
assignable,ensures, and signals are used. It also ex-
tends Java’s expression syntax with several operators—
in the example \forall,\old, and \result are used;
these begin with a backslash so they do not clash with
existing Java identifiers.
The central ingredients of a JML specification are
preconditions (given in requires clauses), postcondi-
tions (given in ensures clauses), and invariants. These
are all expressed as boolean expressions in JML’s exten-
sion to Java’s expression syntax.
In addition to normal postconditions, the language
also supports exceptional postconditions, specified us-
ing signals clauses. These can be used to specify what
must be true when a method throws an exception. For
example, the signals clause in Figure 1’s debit method
specifies that debit may throw a PurseException and
that the balance will not change in that case (as specified
by the use of the \old keyword).
The assignable clause for the method debit spec-
ifies a frame condition, namely that debit will assign
only to the balance field. Although not a traditional
part of Design by Contract languages like Eiffel, such
frame conditions are essential for verification of code
when using some of the tools described later.
There are many additional features of JML that are
not used in the example in Figure 1. We briefly discuss
the most important of these below.
Model variables, which play the role of abstract val-
ues for abstract data types [23], allow specifications
that hide implementation details. For example, if in-
stead of a class Purse, we were specifying an inter-
face PurseInterface, we could introduce the bal-
ance as such a model variable. A class implementing
this interface could then specify how this model field
is related to the class’s particular representation of
balance.
JML comes with an extensive library that provides
Java types that can be used for describing behavior
mathematically. This library includes such concepts
as sets, sequences, and relations. It is similar to li-
braries of mathematical concepts found in VDM, Z,
LSL, or OCL, but allows such concepts to be used di-
rectly in assertions, since they are embodied as Java
objects.
The semantics of JML forbids side-effects in asser-
tions. This both allows assertion checks to be used
safely during debugging and supports mathemati-
cal reasoning about assertions. This semantics works
conservatively, by allowing a method to be used in
assertions only if it is declared as pure, meaning the
method does not have any side-effects and does not
perform any input or output [58]. For example, if
there is a method getBalance() that is declared as
pure,
/*@ pure @*/ int getBalance() { ... }
then this method can be used in the specification
instead of the field balance.
Finally, JML supports the Java modifiers (private,
protected, and public) that control visibility of
specifications. For example, an invariant can be de-
clared to be protected if it is not observable by
clients but is intended for use by programmers of
subclasses. (Technically the invariants and method
specifications in the Purse example of Figure 1 have
default or package visibility, and thus would only be
visible to code in the same package.)
3 Tools for JML
For a specification language, just as for a programming
language, a range of tools is necessary to address the
various needs of the specification language’s users such
as reading, writing, and checking JML annotations.
The most basic tool support for JML is parsing and
typechecking. This already provides an advantage over
informal comments, as parsing and typechecking will
catch any typos, type incompatibilities, references to
names that no longer exist, etc. The JML checker (jml )
developed at Iowa State University performs parsing and
typechecking of Java programs and their JML annota-
tions, and most of the other tools mentioned below in-
corporate this functionality.
The rest of this paper describes the various tools
that are currently available for JML. The following cat-
egorization serves also as an organization for the imme-
diately following sections of this paper. We distinguish
tools for checking of assertions at runtime, tools for stat-
ically checking of assertions (at or before compile-time),
tools for generating specifications, and tools for docu-
mentation.
3.1 Runtime assertion checking and testing
One way of checking the correctness of JML specifica-
tions is by runtime assertion checking, i.e., simply run-
Burdy et al.: An overview of JML tools and applications 3
public class Purse {
final int MAX_BALANCE;
int balance;
//@ invariant 0 <= balance && balance <= MAX_BALANCE;
byte[] pin;
/*@ invariant pin != null && pin.length == 4
@ && (\forall int i; 0 <= i && i < 4;
@ 0 <= pin[i] && pin[i] <= 9);
@*/
/*@ requires amount >= 0;
@ assignable balance;
@ ensures balance == \old(balance) - amount
@ && \result == balance;
@ signals (PurseException) balance == \old(balance);
@*/
int debit(int amount) throws PurseException {
if (amount <= balance) { balance -= amount; return balance; }
else { throw new PurseException("overdrawn by " + amount); }
}
/*@ requires p != null && p.length >= 4;
@ assignable \nothing;
@ ensures \result <==> (\forall int i; 0 <= i && i < 4;
@ pin[i] == p[i]);
@*/
boolean checkPin(byte[] p) {
boolean res = true;
for (int i=0; i < 4; i++) { res = res && pin[i] == p[i]; }
return res;
}
/*@ requires 0 < mb && 0 <= b && b <= mb
@ && p != null && p.length == 4
@ && (\forall int i; 0 <= i && i < 4;
@ 0 <= p[i] && p[i] <= 9);
@ assignable MAX_BALANCE, balance, pin;
@ ensures MAX_BALANCE == mb && balance == b
@ && (\forall int i; 0 <= i && i < 4; p[i] == pin[i]);
@*/
Purse(int mb, int b, byte[] p) {
MAX_BALANCE = mb; balance = b; pin = (byte[]) p.clone();
}
}
Fig. 1. Example JML specification
ning the Java code and testing for violations of JML as-
sertions. Such runtime assertion checks are accomplished
by using the JML compiler jmlc (Section 4.1).
Given that one often wants to do runtime assertion
checking in the testing phase, there is also a jmlunit tool
(Section 4.2), which combines runtime assertion checking
with unit testing.
3.2 Static checking and verification
More ambitious than testing if the code satisfies the
specifications at runtime is verifying that the code sat-
isfies its specification statically. This can give more as-
surance in the correctness of code as it establishes the
correctness for all possible execution paths, whereas run-
time assertion checking is limited by the execution paths
exercised by the test suite being used. Of course, correct-
ness of a program with respect to a given specification is
not decidable in general. Any verification tool must trade
4 Burdy et al.: An overview of JML tools and applications
off the level of automation it offers (i.e., the ability to
dispense with user interaction) and the complexity of the
properties and code that it can handle. There are several
tools for statically checking or verifying JML assertions,
providing different levels of automation and supporting
different levels of expressivity in specifications:
The program checker ESC/Java (Section 5.1) can
automatically detect certain common errors in Java
code and check relatively simple assertions.
ESC/Java2 (Section 5.2) extends ESC/Java to sup-
port more of the JML syntax and to add other func-
tionality.
The LOOP tool (Section 5.3) translates code anno-
tated with JML specifications to proof obligations
that one can then try to prove using the theorem
prover PVS. The LOOP tool can handle more com-
plex specifications and code than automatic checkers
like ESC/Java can, but at the price of more user
interaction.
The program checker JACK (Section 5.4) offers simi-
lar functionality to ESC/Java, but is more ambitious
in attempting real program verification.
3.3 Generating specifications
In addition to these tools for checking specifications,
there are also tools that help a developer write JML
specifications, with the aim of reducing the cost and ef-
fort of producing JML specifications:
The Daikon tool (Section 6.1) infers likely invariants
by observing the runtime behavior of a program.
The Houdini tool (Section 6.2) postulates annota-
tions for code, then uses ESC/Java to check them.
The jmlspec tool can produce a skeleton of a spec-
ification file from Java source and can compare the
interfaces of two different files for consistency.
3.4 Documentation
Finally, in spite of all the tools mentioned above, ulti-
mately human beings must read and understand JML
specifications. Since JML specifications are also meant
to be read and written by ordinary Java programmers, it
is important to support the conventional ways that these
programmers create and use documentation. The jml-
doc tool (Section 7.1) produces browsable HTML pages
containing both the API and the specifications for Java
code, in the style of pages generated by javadoc [38].
4 Runtime Assertion Checking and Testing
The most obvious way to use JML annotations is to test
them at runtime and report any detected violations. In
this section we discuss two tools, jmlc and jmlunit , that
work this way.
4.1 Runtime Assertion Checking
4.1.1 Overview and Goals
The goal of the JML compiler, jmlc, also known as the
runtime assertion checker, is to find inconsistencies be-
tween specifications and code by executing assertions at
runtime. The overall approach is to find such inconsis-
tencies dynamically, by executing JML’s assertions while
the program runs and notifying the user of any assertion
violations. As with other runtime assertion checkers, one
normally hopes to find that the code is incorrect with re-
spect to the specification. However, it may also be that
the specification itself is incorrect (with respect to what
the user has in mind), but the code is correct. Find-
ing problems in specifications is important for keeping
the specifications accurate and up-to-date; this solves a
common problem with informal documentation, which
cannot be mechanically checked against the program.
An important requirement for the runtime assertion
checker is that it be good at isolating problems, in the
sense that users of the tool should be able to quickly
pinpoint what in either the code or specifications must
be changed to correct an inconsistency. For this purpose,
jmlc must provide information that is helpful for users.
This includes both static information, such as what parts
of the specification were violated and where in the pro-
gram the violation was detected, as well as dynamic in-
formation about the values of variables and what method
calls led to the violation (a stack backtrace).
It is also helpful, for isolating problems, if the runtime
assertion checker can execute as large a subset of the
JML language as possible.
The runtime assertion checker must also be trustwor-
thy, in the sense that it must not generate false reports
of assertion violations. That is, every assertion violation
must be a report of an assertion that is false, according
to the JML semantics. In meeting this goal, the runtime
assertion checker can fail to report assertions that might
be false. For example, JML includes a way to write in-
formal descriptions in assertions; these informal descrip-
tions are merely pieces of English text, and so only a hu-
man reader can decide whether they are true or false. If
the runtime assertion checker were to assume some par-
ticular truth value for these it might report an assertion
violation when none actually existed. In such cases it is
better for the runtime assertion checker to not report a
violation. Similarly, it is also acceptable for the runtime
assertion checker to not execute some parts of assertions,
especially in postconditions. However, not being able to
execute some precondition could cause a method to fail
unexpectedly; thus jmlc should give a warning for non-
executable preconditions. In summary, it is better if the
runtime assertion checker can execute all assertions and
find all assertion violations, but this is a goal that can
be incrementally approached during the development of
the tool.
Burdy et al.: An overview of JML tools and applications 5
An important goal of the runtime assertion checker is
that its work should be transparent when no assertions
are violated. That is, except for time and space mea-
surements, a correct program compiled with jmlc should
behave just as if compiled with a normal Java compiler.
The transparency of runtime assertion checking is aided
by JML’s design, as assertions are not allowed to have
any side-effects [59].
Although jmlc does not have to be used with any
particular methodology, there are some general ideas for
using such tools that are helpful for beginners [73]. A
basic technique for using the runtime assertion checker
is to first specify preconditions for the normal behavior
of methods. This is easily done and helps ensure that
all methods are called in expected states. For debugging
purposes, it is also important to add toString methods
to all types involved, so that jmlc can display object
values in violation messages. Following this, one could
define invariants that describe the legal states of objects
of each class (see Section 6.1 for more on this topic). To
help debug implementations, one can then advance to
describing normal postconditions for methods. If one is
describing a library for untrusted clients, it may also be
useful to document when various exceptions are thrown
by writing exceptional postconditions.
4.1.2 Design of the Tool
The JML compiler was developed at Iowa State Univer-
sity as an extension to the MultiJava compiler [24]. It
compiles Java programs annotated with JML specifica-
tions into Java bytecode [19,21]. The compiled bytecode
includes instructions that check JML specifications such
as preconditions, normal and exceptional postconditions,
invariants, and history constraints.
Because the JML language provides such a rich set
of specification facilities, it presents new challenges in
runtime assertion checking. One of these challenges that
the current tool meets is supporting abstract specifica-
tions written in terms of specification-only declarations
such as model fields, ghost fields, and model methods.
This aspect of the JML compiler represents a significant
advance over the state of the art in runtime assertion
checking as represented by Design by Contract tools such
as Eiffel [73] or by Java tools such as iContract [55] or
Jass [9]. Other advances over such tools include (stateful)
interface specifications, multiple inheritance of specifica-
tions from interfaces, various forms of quantifiers and set
comprehension notation, support for strong and weak
behavioral subtyping [68,28], and a contextual interpre-
tation of undefinedness [21].
4.1.3 Example
The specifications and code in Figure 1 were debugged
using the runtime assertion checker in combination with
the unit testing tool described in Section 4.2.3. Using
jmlc on the example is straightforward; the user simply
tells the tool to compile the Purse.java file and then
runs a test driver using jmlrac as the virtual machine.
The jmlrac command is a version of the java command
that knows about the necessary runtime libraries for run-
time assertion checking. Assertion violations are printed
as messages on the console. We discuss details of this
kind of testing in Section 4.2.3.
4.1.4 Experience
The runtime assertion checker is one of the most widely
used JML tools. It has been used on several case stud-
ies. One of the most demanding of these case studies
is the checking of the built-in model types for JML it-
self, which have very rich and complete specifications. It
has been used in several undergraduate classes, but in
those cases it has also been used for simple, Design by
Contract style, specifications. It has also been used in
several of the other case studies mentioned in the rest
of this paper. It seems to be helpful to use the runtime
assertion checker before doing serious program verifica-
tion, to make sure that the easily found bugs are removed
before spending the effort to do verification.
In sum, the JML compiler brings programming ben-
efits to formal interface specifications by allowing Java
programmers to use JML specifications as practical and
effective tools for debugging, testing, and Design by Con-
tract.
4.1.5 Future Work
One of the main issues in the future work on jmlc is im-
proving both the speed of compilation and the speed of
executing runtime assertion checks. For the latter, there
seem to be several simple things that can be done to im-
prove execution speed. For example, caching the values
of model fields instead of recomputing them in several
places within an assertion would be helpful.
Another direction for future work is being pursued
at Virginia Tech by Stephen Edwards and his student
Roy Tan. They are building a version of the JML com-
piler that produces separate bytecode files for the nor-
mal code and for a runtime assertion checking wrapper.
Separating the runtime assertion checking code into this
wrapper has several advantages. In particular, decisions
about what classes should be checked can be made while
the program executes. It will also enable the addition of
runtime checks to code for which the source code is not
available.
4.1.6 Availability
The runtime assertion checker is part of the main JML
toolset available via www.jmlspecs.org, which is devel-
oped as an open source project hosted at SourceForge.
net.
6 Burdy et al.: An overview of JML tools and applications
4.2 Unit Testing
4.2.1 Overview and Goals
A formal specification can be viewed as a test oracle [84,
3], and JML’s runtime assertion checker can be used as
the decision procedure for the test oracle [22]. This idea
has been implemented as a unit testing tool for Java,
jmlunit, by combining JML with the popular unit testing
tool JUnit [10].
The main goal of the jmlunit tool is to significantly
automate unit testing of Java code. More specifically,
the goal is to free the programmer from writing the code
that decides whether unit tests pass or fail.
4.2.2 Design of the Tool
The jmlunit tool, developed at Iowa State University,
generates JUnit test classes that rely on the JML run-
time assertion checker. The test classes send messages
to objects of the Java classes under test. The testing
code catches assertion violation errors from such method
calls to decide if the test data violate the precondition
of the method under test; such assertion violation er-
rors do not constitute test failures. When the method
under test satisfies its precondition, but otherwise has
an assertion violation, then the implementation failed to
meet its specification, and hence the test data detects
a failure [22]. In other words, the generated test code
serves as a test oracle whose behavior is derived from
the specified behavior of the class being tested.
The user is still responsible for generating test data;
however, the generated test classes make it easy for the
user to supply this data. The tool comes with a frame-
work that includes sample test data for the built-in Java
value types. This framework allows one to combine, fil-
ter, and compose test data in several different ways to
create a variety of tests. In addition, the user can supply
handwritten JUnit test methods if desired. Such hand-
written tests are useful for exploring combinations of
method calls that the automatic testing ignores.
4.2.3 Example
In this subsection we discuss runtime assertion check-
ing and unit testing with jmlunit, based on Figure 1.
To do unit testing with jmlunit, one first runs the jm-
lunit tool on the Purse.java file (technically, one has
to use an option to tell the tool to test methods and
constructors with package visibility). This produces a
file, Purse JML TestData.java, into which test data is
placed, and another file Purse JML Test.java, which
contains a driver to run the tests. In the first file we
supplied data of the various types used as arguments to
the methods being tested; this consists of integers (0,
1, -1, -22, etc.), Purse objects (such as null and new
Purse(1,1,p), where pis a 4-element array of bytes),
and fresh byte arrays (such as null,new byte[] {},new
byte[] {0,0,0}, and new byte[] {0, 0, 0, 0}). To
run the tests, one first compiles the classes being tested
with jmlc (using a special option to flag unhandled and
unspecified exceptions as errors); then the classes pro-
duced by jmlunit are compiled with a normal Java com-
piler; finally one executes the automatically-generated
driver class, Purse JML Test, using jmlrac.
If all the annotations are removed from Figure 1, then
the unit testing process described in the previous para-
graph does not detect any errors. This is because the
unit testing tool is only testing for violations of asser-
tions and, if there are no assertions, then no violations
are detected. This illustrates the important observation
that the quality of the testing that jmlunit provides is
only as good as the specifications.
Consider a version of Figure 1 that only includes the
preconditions of the methods and the constructor, but
omits the invariants, the frame axioms, and all the nor-
mal and exceptional postconditions. Testing of Purse
produces 11 failures, all of which are similar to that
shown in Figure 2. (Printing of Purse objects is handled
by adding the obvious toString method to the code in
Figure 1.)
This error is the result of not specifying (i.e., delet-
ing) the exceptional postconditions of the debit method.
It shows that the condition in an exceptional postcondi-
tion can be alternatively considered as the negation of
a precondition for normal behavior; which makes sense,
because throwing an exception is not normal behavior.
If the precondition of the debit method is changed to
the following:
amount >= 0 && amount <= balance
then all of these failures go away. This also happens if
the debit method has the exceptional postcondition re-
stored from Figure 1, which tells the runtime assertion
checker that such exceptions are expected.
This kind of testing is also effective at finding various
omissions in preconditions. For example, if the precon-
dition in the checkPin method or the constructor that
specifies that the array must be of an appropriate length
is omitted, then the tests will encounter failures.
Checking preconditions will not show places where
the code is wrong, unless one method in the code calls
another incorrectly. For the most part, errors in code are
revealed by adding either invariants or postconditions.
If we add the invariants back into the version of Purse,
but still leave out the postconditions, then testing can
detect omitted initialization of the MAX BALANCE field in
the constructor (although Java itself detects missing ini-
tializations of final fields, so for JML to detect this error,
one also has to omit the final attribute from that field).
Similarly, with the invariants, the constructor’s precon-
dition must have the first line shown in Figure 1, or many
violations of the first invariant in the figure occur.
Burdy et al.: An overview of JML tools and applications 7
1) debit(Purse_JML_Test$TestDebit)junit.framework.AssertionFailedError:
Method ’debit’ applied to
Receiver: Purse(max=1, bal=0, pin={0123})
Argument amount: 1
Caused by: org.jmlspecs.jmlrac.runtime.JMLExceptionalPostconditionError:
by method Purse.debit regarding specifications at
File "Purse.java", line 9, character 17, when
’jml$e’ is PurseException: overdrawn by 1
at Purse.checkXPost$debit$Purse(Purse.java:256)
at Purse.debit(Purse.java:347)
Fig. 2. Example output from testing with jmlunit.
Adding postconditions from Figure 1 allows many
other errors in coding to be detected. For example, with
all the postconditions restored, omissions of initializa-
tions of the balance and pin fields are detected. The
postconditions can also detect incorrect coding in the
loop of the checkPin method, but doing so requires test
data for byte arrays that differ in only the positions not
checked by the code; we had to add such data to our
initial set of test data, since the original test data did
not detect these errors. Figuring out the right test data
to add in this case was subtle and could easily have been
missed.
4.2.4 Experience
Our experience shows that the tool allows one to per-
form unit testing with minimal coding effort and detects
many kinds of errors. Ironically, about half of our test
failures were caused by specification errors, which shows
that the approach is also useful for debugging specifica-
tions. In addition, the tool can report assertion coverage
information, identifying assertions that are always true
or always false, and thus indicating deficiencies in the
set of test cases. However, the approach requires specifi-
cations to be fairly complete descriptions of the desired
behavior, as the quality of the generated test oracles de-
pends on the quality of the specifications. Thus, the ap-
proach trades the effort one might spend in writing test
cases for effort spent in writing formal specifications.
4.2.5 Future Work
JML/JUnit testing is limited in that it only detects prob-
lems that are the result of single method or construc-
tor calls. Thus test data has to be carefully crafted so
that the method is applied to objects in states that will
fully exercise it. This process would be easier if the test
drivers would apply several methods in sequence to var-
ious pieces of data. One alternative for doing this would
be to generate such sequences of method calls automat-
ically. (An experimental version of Daikon can do this.)
Another alternative is to augment JML with facilities
to write specifications for blocks of example code to be
used in testing.
4.2.6 Availability
jmlunit is part of the main JML toolset. This toolset is
available via www.jmlspecs.org. It has been developed
as an open source project hosted at SourceForge.net.
5 Static Checking and Verification
In this section, we describe several tools for statically
checking—or verifying—JML annotations, providing dif-
ferent degrees of rigor and automation.
5.1 Extended Static Checking with ESC/Java
5.1.1 Overview and Goals
The ESC/Java tool [36], originally developed at Compaq
Research, performs what is called extended static check-
ing [27,60], compile-time checking that goes well beyond
type checking. It can check relatively simple assertions
and can check for certain kinds of common errors in Java
code, such as dereferencing null, indexing an array out-
side its bounds, or casting a reference to an impermissi-
ble type. ESC/Java supports a subset of JML. For this
subset it checks the consistency between the code and
the given JML annotations. The user’s interaction with
ESC/Java is quite similar to the interaction with a com-
piler’s type checker: the user includes JML annotations
in the code and runs the tool, and the tool responds with
a list of possible errors in the program.
5.1.2 Design of the Tool
JML annotations affect ESC/Java in two ways. First,
the given JML annotations help ESC/Java suppress spu-
rious warning messages. For example, in Figure 1, the
constructor’s precondition p != null lets ESC/Java de-
termine that the dereference of pin the constructor’s
body is valid, and thus no null-dereference warning is
produced. Second, annotations make ESC/Java do addi-
tional checks. For example, when checking a caller of the
Purse constructor, the precondition p != null causes
ESC/Java to emit a warning if the actual parameter
8 Burdy et al.: An overview of JML tools and applications
for pmay be passed in as null. In these two ways, the
use of JML annotations enables ESC/Java to produce
warnings not at the source locations where errors man-
ifest themselves at runtime, but at the source locations
where the errors are committed.
An interesting property of ESC/Java is that it is nei-
ther sound nor complete; that is, it neither warns about
all errors, nor does it warn only about actual errors.
This is a deliberate design choice: the aim is to increase
the cost-effectiveness of the tool. In some situations, con-
vincing a mechanical checker of the absence of some par-
ticular error may require a large number of JML anno-
tations (consider, for example, a hypothetical program
that dereferences null if four of the program’s large-
valued integer variables satisfy the equation in Fermat’s
Last Theorem). To make the tool more cost-effective,
it may therefore be prudent to ignore the possibility of
certain errors, which is what ESC/Java has been de-
signed to do. The ESC/Java User’s Manual [64] contains
a list of all cases of unsoundness and incompleteness in
ESC/Java.
Under the hood, ESC/Java is powered by detailed
program semantics and an automatic (non-interactive)
theorem prover, Simplify [26]. ESC/Java translates a
given JML-annotated program into verification condi-
tions [65,37,61]. Verification conditions are logical for-
mulas that are valid if and only if the program is free
of the kinds of errors being analyzed. Any verification-
condition counterexamples found by the theorem prover
are turned into programmer-sensible warning messages,
including the kind and source location of each potential
error [62]. The User’s Manual for ESC/Java [64] also
provides a detailed description of the semantics of JML
annotations, as they pertain to ESC/Java.
5.1.3 Example
We refrain from giving details of an ESC/Java example
here. Instead, we describe an example in the context of
ESC/Java’s successor, ESC/Java2 , in Section 5.2.3.
5.1.4 Experience
The first major experience with ESC/Java was to ap-
ply the tool to the sources of its own front end, over 40
KLOC of Java. This source was “fully annotated”, mean-
ing that enough specifications were given for ESC/Java
to check the front end for run-time errors (like null
dereferences and array-index bounds errors) and spec-
ification violations (like precondition violations) with-
out producing any warnings. This and some other early
experiences are described in the ESC/Java overview pa-
per [36].
Applications to Java Card are discussed in Section 8.
The experience applying ESC/Java to Java Card was
one of the motivations for the work on ESC/Java2 , as
maintaining different versions of the API specification,
one using ESC/Java’s dialect of JML and one using the
full JML language, was becoming a lot of work.
5.1.5 Availability
The final binary release (version 1.2.4) of ESC/Java is
available from Compaq/HP’s web site: www.research.
compaq.com/downloads.html. The source code (includ-
ing that of related tools, e.g. Houdini, Calvin, and Sim-
plify) is available as well. This source code release is ob-
scurely named the “Java Programming Toolkit Source
Release.” ESC/Java only runs on x86 machines with
Linux and Microsoft Windows, Sun’s SPARC with So-
laris, and Alpha processors with Hewlett-Packard’s Tru64
Unix.
5.2 ESC/Java2
5.2.1 Overview and Goals
Development of version 1 of ESC/Java had ceased by
the time the Compaq Systems Research Center became
part of HP Labs, where it was later dissolved. Conse-
quently, Cok and Kiniry have in progress a version 2 of
ESC/Java, built on the source code release provided by
Compaq and HP. This version has the following goals:
to migrate the code base of ESC/Java and the code
accepted by ESC/Java to Java 1.4;
to update ESC/Java to accept annotations consis-
tent with the current version of JML;
to increase the amount of JML that is checked, while
remaining true to the original engineering goals of
ESC/Java.
5.2.2 Design of the Tool
ESC/Java2 follows the design of ESC/Java. In addition,
ESC/Java2 , like ESC/Java, recognizes that the state-of-
the-art of static checking is such that not all mismatches
between code and specifications are reported by static
checking tools; that is, there are aspects which are un-
sound, typically because some of the Java semantics are
not yet fully modeled. Similarly, some generated warn-
ings are not actually errors in the program; that is, there
are aspects which are incomplete, typically because cur-
rent theorem provers are insufficiently powerful. It is a
goal of all such tools, including ESC/Java2 , to be as
sound and complete as is possible within reasonable en-
gineering limits, but since no existing tools fully model
or fully prove full multi-threaded Java (indeed, portions
of the semantics of the language are still being debated),
the authors of both ESC/Java and ESC/Java2 believe
that it is in the interests of users to be explicit about the
known sources of unsoundness and incompleteness.
ESC/Java2 does include improvements to ESC/Java
in the following areas, while retaining backwards com-
patibility in all but a few features:
Burdy et al.: An overview of JML tools and applications 9
It parses Java 1.4 (ESC/Java only parsed Java 1.3).
In particular ESC/Java2 handles the Java assert
statement. A tool option allows the user to choose
whether Java assert statements are treated as state-
ments that may throw exceptions (per the Java se-
mantics) or whether they are treated like assert
statements in JML, which are checked by the static
checker.
It handles the current binary format for Java classes.
It parses all of current JML. This is a somewhat mov-
ing target, since JML is the subject of ongoing dis-
cussion and research. Nevertheless the core part of
JML is stable and that is the portion that ESC/Java2
attempts to statically check. Some of the more eso-
teric features of JML (e.g. model programs) are only
parsed and are ignored for purposes of static check-
ing.
It allows specifications to be placed in (multiple) files
separate from the implementation, using JML’s re-
finement features. ESC/Java2 makes checks by com-
bining all available specifications and implementa-
tions. It also checks these specifications for consis-
tency.
It follows the JML semantics for specification inher-
itance. The constructs specific to ESC/Java version
1 (also_requires, etc.) were dropped.
It enlarges the set of JML features that are statically
checked, most importantly:
Pure methods, which may be included in annota-
tions;
Most aspects of assignable clauses;
Model fields, with the associated represents,in
and maps annotations.
5.2.3 Example
As an example, if the second invariant in Figure 1 is
omitted and the current ESC/Java2 tool is applied to
the source code, the warnings shown in Figure 3 are pro-
duced. The warning messages indicate the likely problem
and the source code location that violates the implicit
or explicit specification, namely, in this case, the implicit
specification that the left-hand operand of the derefer-
ence operation must not be a null reference and that the
index of an array reference must be less than the array
length.
If ESC/Java2 is applied to Purse.java as it stands
(using a current version of JML’s specifications for Java
system classes), a warning will be produced reflecting
the fact that the specifications of the behavior of clone
are not yet completed.
A source of unsoundness in ESC/Java(2) that is rel-
evant in the Purse example is its handling of loops:
by default, it will not attempt verification of the loop
in checkPin, but simply unroll it once. This makes it
easy for the programmer, who doesn’t have to supply a
loop invariant, but it may also miss errors. In contrast,
LOOP and JACK (and ESC/Java2 with the -loopSafe
switch) handle loops soundly, but then require users to
supply loop invariants. For this case, the loop invariant
as illustrated in Figure 4 would have to be given.
5.2.4 Experience
The first major partial verification using ESC/Java2 was
done in early 2004 when the Dutch Parliament decided
in 2003 to construct an Internet-based remote voting
system for use by Dutch expatriates. The SoS group at
the University of Nijmegen was part of an expert review
panel for the system and also performed a black-box
network and system security evaluation of this system
in late 2003. They also were responsible for designing,
implementing, and verifying the vote tally subsystem
of this system in early 2004. This implementation used
JML and ESC/Java2 extensively.
ESC/Java2 made a very positive impression on the
SoS developers. Its increased capabilities as compared
to Compaq ESC/Java, particularly with regards to han-
dling the full JML language, the ability to reason with
models and specifications with pure methods, are very
impressive. And, while the tool is still classified as an
“alpha” release, we found it to be quite robust (per-
haps unsurprising given its history, the use of JML and
ESC/Java2 in and on its own source code, and the fact
that it is passed through seven alpha releases thus far).
But there are still a number of issues with ESC/Java2
and JML that were highlighted by this verification effort
and are discussed in another paper [54].
5.2.5 Future Work
There are a number of major areas of development of
ESC/Java2 that will improve overall usability of the tool,
besides performance improvements.
The use of model variables and method calls in anno-
tation expressions. Model variables are an important
abstraction mechanism in writing specifications and
model methods allow much more readable and com-
pact specifications [23]. This is a current topic of re-
search and experimentation; most of what is needed
to support these features is a part of the current al-
pha release of ESC/Java2 [25].
Checking of the frame conditions specified by JML’s
assignable clause (also known as modifies). It is an
acknowledged unsoundness of ESC/Java that these
are not checked and faulty assignable clauses can
be a subtle source of errors. ESC/Java2 checks most
aspects of assignable clauses. However, the default
assignable clause in JML specifications is that ev-
erything is potentially modified; this interpretation
is not currently implemented.
Arithmetic. JML needs to have available for specifi-
cations both mathematical integers and reals as well
as the finite-precision approximations that are used
10 Burdy et al.: An overview of JML tools and applications
Purse.java:31: Warning: Possible null dereference (Null)
for (int i=0; i < 4; i++) { res = res && pin[i] == p[i]; }
^
---------------------------------------------------------------------
Purse.java:31: Warning: Array index possibly too large (IndexTooBig)
for (int i=0; i < 4; i++) { res = res && pin[i] == p[i]; }
^
Fig. 3. Example ESC/Java2 warnings
in computer programs. There is some initial work [18]
incorporating these into JML but as yet no axioma-
tization that enables reasoning with ESC/Java2 .
The most significant aspect of future work, however,
is experimentation with specification and static checking
of larger, more varied, and real-world bodies of source
code. Such experimentation is needed to verify that JML
has the facilities that are needed for realistic specifica-
tions and that static checking tools such as ESC/Java2
are capable of providing a benefit to working program-
mers.
5.2.6 Availability
An alpha version of ESC/Java2 is available from http:
//www.cs.kun.nl/sos/research/escjava. The tool is
a Java program that is fairly platform-independent, but
it uses the Simplify prover, which is only available on
Linux, Windows, Solaris, and MacOSX platforms.
5.3 Program Verification with LOOP
5.3.1 Overview and Goals
The LOOP project at the University of Nijmegen started
out as an exploration of the semantics of object-oriented
languages in general, and Java in particular. Only later
did it evolve to investigate verification of JML-annotated
Java. For a detailed overview of the LOOP project we
refer to [50].
5.3.2 Design of the Tool
The project began with the formalization of a denota-
tional semantics of sequential Java [51] in the language
of the theorem prover PVS [82]. An associated compiler,
called the LOOP tool [11], was developed, which trans-
lates any given sequential Java class into PVS theories
describing its semantics. In order to conveniently use this
as a basis for the specification and verification of Java
code, the LOOP tool was then extended to also provide
a formal semantics of JML, so that the tool now trans-
lates JML-annotated Java code into proof obligations for
PVS, which one can try to prove interactively, in PVS.
These proof obligations are expressed as a special kind of
Hoare statements about methods, and they are proved
using an associated Hoare logic [49] and weakest-pre-
condition calculus [47] for Java and JML, both of which
have been formalized in PVS.
A difference between LOOP and both ESC/Java(2)
and JACK (see Section 5.4 for the JACK tool) is that
it provides a so-called shallow embedding of Java and
JML in PVS, defining a formal denotational semantics
of both Java and JML in PVS. This has its advantages.
The Hoare logic and wp-calculi that are used have been
completely formalized and proven sound with respect
to these semantics in PVS, whereas both ESC/Java(2)
and JACK directly rely on an axiomatic semantics. Also,
our semantics of Java in PVS is still (symbolically) ex-
ecutable to a degree, as it lets PVS evaluate the de-
notation of a program. This has been very useful in the
extensive testing and debugging of our formal semantics,
where we compared the results of the normal execution
of a Java program, i.e. the result of executing its byte-
code on a Java VM, and the symbolic execution of its
semantics in PVS.
5.3.3 Example
Using the LOOP tool to verify the example in Figure 1
fails for the constructor, as it did for ESC/Java, because
the specifications of the behavior of clone are incom-
plete. The verification of the methods is fully automatic
using LOOP, using its weakest precondition calculus,
except that the verification of checkPin needs manual
interaction in PVS to supply the loop invariant, as the
tool doesn’t handle JML’s loop_invariant yet.
5.3.4 Experience
Case studies with the LOOP tool are discussed in [12, 46,
48]. Verification of JML-annotated code with the LOOP
tool (especially the required interactive theorem proving
with PVS) can be very labor-intensive, but allows verifi-
cation of more complicated properties than can be han-
dled by fully automated extended static checking using
ESC/Java. Because of this labor-intensive nature, one
will typically first want to use other, less labor-intensive,
approaches, such as runtime assertion checking or ex-
tended static checking, to remove some of the errors in
the code or specifications before turning to the LOOP
tool. Experiences with such a combined approach are
described in [13]. The possibility to do this is an impor-
Burdy et al.: An overview of JML tools and applications 11
tant —if not crucial— advantage of using a specification
language that is supported by a range of tools.
The LOOP tool generates a single proof obligation
for each method and constructor, expressed as a Hoare
statement. It does not, as commonly done in verification
condition generators, split this up into smaller verifica-
tion conditions. Instead, this splitting up is done inside
the theorem prover PVS, using dedicated proof strate-
gies. A disadvantage of this is that the size of proof obli-
gations that can be comfortably handled in PVS has
become a bottleneck.
5.3.5 Future Work
Ongoing work on the LOOP tool includes support for
the different forms of arithmetic as proposed in [18] and
investigations into proving information flow properties.
The longer term plans for the LOOP tool are currently
not clear.
5.3.6 Availability
The LOOP tool is not publicly available, simply because
it is not easy to use without intensive user support and
documentation that we cannot provide. Actually, LOOP
itself is easy enough to use — it is simply a compiler that
outputs PVS — but dealing with the large and numer-
ous PVS theories it outputs requires considerable (PVS)
expertise.
5.4 Static Verification with JACK
5.4.1 Overview and Goals
The JACK [15] tool was initially developed at the re-
search lab of Gemplus, a manufacturer of smartcards and
smartcard software. Further development is now happen-
ing at INRIA. JACK aims to provide an environment
for Java and Java Card program verification using JML
annotations. It implements a fully automated weakest
precondition calculus in order to generate proof obli-
gations from JML-annotated Java sources. Those proof
obligations can then be discharged using different theo-
rem provers.
The main design goals are an easily accessible user
interface, a high degree of automation, a high correctness
assurance, and prover independence.
5.4.2 Design of the Tool
The main goal of JACK is that it should be usable by
normal Java developers, allowing them to validate their
own code, following, in this way, the JML philosophy.
Thus, care has been taken to hide the mathematical for-
mulation of the underlying concepts. To allow develop-
ers to work in a familiar environment, JACK is inte-
grated as a plug-in to the Eclipse2IDE. This plug-in
allows users to generate proof obligations, to run the
automatic provers, and to inspect the generated lem-
mas. To facilitate this last task, JACK provides a ded-
icated proof obligation viewer. This viewer presents the
proof obligations as execution paths within the program,
highlighting the source code relevant to the proof obli-
gations. Moreover, goals and hypotheses are displayed
in a Java/JML-like notation. The user can then work
within its current development tool, add the JML anno-
tations and check partially the correctness of the code
in a familiar environment.
JACK ’s core is an implementation, in Java, of a
weakest precondition calculus. This ensures proof obliga-
tion generation without user interaction. Following this
step, automatic provers are used to prove the generated
lemmas. Users then have to check whether any remain-
ing lemmas are valid or not. To reduce the remaining
costly manual task – creating the JML annotation as-
sertions – we have developed and integrated in JACK a
prototype that annotates source code with assertions by
propagation of pre and post conditions. This is a way to
reduce the cost of using JML, since, at the moment, the
main issue when using JACK is the time spent annotat-
ing classes.
JACK is not based on a formalization of Java as
LOOP is; thus one cannot easily prove the formal cor-
rectness of the tool, and the implementation of the weak-
est precondition calculus can contain bugs. Nevertheless,
the aim of the tool is to be complete and sound (i.e. to
generate all proof obligations that are valid if and only if
the application respects its formalization). So, users can
choose to check partially the correctness of their applica-
tion by just reviewing the unproved proof obligations, or
they can also prove all the proof obligations using an in-
teractive theorem prover, thereby obtaining a complete
assurance on the development correctness.
JACK provides an interface to automatic theorem
provers. Currently, the prover of the Atelier B toolkit,
Simplify (the prover used in ESC/Java), and PVS are
integrated. These provers are integrated as plug-ins in
JACK . Since JACK is based on an intermediate lemma
formulation language, it is quite easy to integrate new
provers by implementing a translator from this interme-
diate language to the prover input. Interfacing several
provers increases the automatic proof ratio. This also al-
lows people to prove any remaining lemmas interactively
in their preferred prover.
The actually interfaced automatic provers can usu-
ally automatically prove up to 90% of the proof obliga-
tions. The remaining ones have to be proved outside of
JACK , using the classical B proof tool, PVS, or the Coq
proof assistant. However, JACK is meant to be used by
Java developers, who cannot be expected to use a proof
assistant. Therefore, in addition to the proved and un-
2http://www.eclipse.org
12 Burdy et al.: An overview of JML tools and applications
proved states, JACK adds a checked state, which allows
developers to indicate that they have manually checked
the proof obligation. In order to better handle those
cases, other different approaches could be investigated,
such as integration with test tools such as jmlunit, in-
tegration of other proof assistants, or perhaps support
from a proof-expert team.
5.4.3 Example
The code of the class given in the Figure 1 was proved
using JACK . To generate proof obligations automati-
cally, loop invariants have to be given explicitly in the
code. Here, the JML annotation of Figure 4 is added
in the body of the method checkPin before the for
statement. When this annotation is added, one can run
JACK , which then calculates proof obligations automat-
ically and proves them using the automated provers.
Here, only three proof obligations remain unproved due
to, yet again, the lack of complete specification of the
clone() method in the constructor.
5.4.4 Experience
Like ESC/Java,JACK tries to hide the complications of
the underlying theorem prover from the user, by provid-
ing a push-button tool that normal Java developers, and
not just formal methods experts, can and would like to
use. We believe that this may be a way to let non-experts
venture into the world of formal verification.
ESC/Java,LOOP, and JACK all use (or, in the
case of LOOP, have the option of using) a weakest pre-
condition calculus to generate verification conditions.
ESC/Java and LOOP generate one verification condi-
tion per method implementation, whereas JACK gener-
ates roughly one verification condition per syntactic code
path through the code. So each of JACK ’s verification
conditions is smaller than those generated by ESC/Java
and LOOP. On the other hand, JACK may generate
a very large number of verification conditions. Though
it generates just one verification condition per method,
ESC/Java factors its verification conditions differently
than the other two tools (see [37,61]) and therefore is
able to keep the one verification condition reasonably
small. More important than size, verification conditions
generated by ESC/Java often let the theorem prover
avoid redundant work. JACK ’s approach has the ad-
vantage that it is easy to pass the different verification
conditions to different theorem provers.
5.4.5 Future Work
To increase the automation of this validation phase, we
are currently thinking of interfacing JACK with a coun-
terexample detector or runtime test generator. We are
also still investigating the annotation generation and
propagation techniques since we consider that it can be
a way to reduce the cost of using the tool.
5.4.6 Availability
JACK is currently not publicly available.
6 Generating Specifications
Apart from checking that implementations meet specifi-
cations, a considerable barrier to entry in the use of any
formal specification language is writing specifications in
the first place. The JML tools discussed so far assume
the existence of a JML specification, and leave the task
of writing it to the programmer. This task can be time-
consuming, tedious, and error-prone, so tools that can
help in this task can be of great benefit.
6.1 Invariant Detection with Daikon
6.1.1 Overview and Goals
The Daikon invariant detector [31,32] is a tool that pro-
vides assistance in creating a specification. Daikon out-
puts observed program properties in JML syntax (as well
as other output formats) and automatically inserts them
into a target program.
6.1.2 Design of the Tool
The Daikon tool dynamically detects likely program in-
variants. In other words, given program executions, it
reports properties that were true over those executions.
The set of reported properties is also known as an op-
erational abstraction. Dynamic invariant detection op-
erates by observing values that a program computes
at runtime, generalizing over those values, and report-
ing the resulting properties. The properties reported by
Daikon encompass numbers (x <=y,y== ax +b), col-
lections (mytree.contains(x), mylist.isSorted()), point-
ers (n== n.next.prev), and implications (p != null
==>p.value >x); a complete list appears in the Daikon
user manual.
Like any dynamic analysis, the accuracy of the in-
ferred invariants depends in part on the quality and com-
pleteness of the test cases, and other executions may fal-
sify some of the reported properties. (Furthermore, the
actual behavior of the program is not necessarily the
same as its intended behavior.) However, Daikon uses
static analysis, statistical tests, and other mechanisms
to reduce the number of false positives [33]. Even if a
property is not true in general, Daikon’s output provides
valuable information about the test suite over which the
program was run. Combining invariant detection with
a static verifier such as ESC/Java helps to overcome
the problems of both techniques: the unsoundness of the
dynamic analysis and the static analysis’s need for an-
notations.
Burdy et al.: An overview of JML tools and applications 13
//@ loop_invariant 0 <= i <= 5;
//@ loop_invariant res == (\forall int j; 0 <= j && j < i; pin[j] == p[j]);
Fig. 4. Loop invariant
6.1.3 Example
In order to apply Daikon to a program, a user runs an
instrumented version of the program to create a data
trace file, then runs Daikon over the data trace file to
produce likely invariants. The instrumented version of
the program contains, at program points such as proce-
dure entries and exits, code that writes the values of all
variables in scope to a trace file. In some cases (as for
Daikon’s C front end), the instrumentation is performed
automatically on a compiled executable by a special run-
time system. In other cases (as for Daikon’s Java front
end), the user runs a source-to-source translator that
instruments the program, then runs the instrumented
program in place of the original.
Given a simple test suite that creates 1000 random
Purse objects and invokes debit on each one, the Daikon
tool automatically generates the annotations of Figure 1,
except that the current version of Daikon does not gen-
erate JML’s signals clauses. Daikon’s output is correct
JML that is parseable by the JML toolset.
6.1.4 Experience
Even with modest test suites, Daikon’s output is re-
markably accurate. In one set of experiments [80], over
90% of the properties that it reported were verifiable by
ESC/Java (the other properties were true, but were be-
yond the capabilities of ESC/Java), and it reported over
90% of the properties that ESC/Java needed in order to
complete its verification. For example, if Daikon gener-
ated 100 properties, users had only to delete less than
10 properties and to add another 10 properties in order
to have a verifiable set of properties. In another experi-
ment [81], users who were provided with Daikon output
(even from unrealistically bad test suites) performed sta-
tistically significantly better on a program verification
task than did users who did not have such assistance.
In addition to aiding the task of static checking as
described above, operational abstractions generated by
the Daikon invariant detector have been used to generate
and improve test suites [44,93, 40], automate theorem-
proving [78,79], identify refactoring opportunities [53],
aid program analysis [29,30], choose modalities [67], pre-
dict incompatibilities in component upgrades [71,72], de-
tect anomalies and bugs [89,43,87,14, 70], and isolate er-
rors [92,39,66], among other uses.
6.1.5 Future Work
As noted above, Daikon does not generate JML signals
clauses for exceptional method exits. Doing so requires
enhancements to the language-specific front ends, but
no significant changes to Daikon proper. Another in-
strumentation enhancement that we are pursuing is re-
placing the current Java instrumenter (which performs
a source-to-source translation) by one that is embed-
ded in the Java Virtual machine and works on compiled
Java programs. This change will simplify using Daikon
by reducing the work required of a user. Finally, mak-
ing Daikon work online taking data from a running
program rather than from a trace file will reduce the
number of steps to 1, which is the same as currently re-
quired to run any Java program (via the java command).
Our main research thrust is not to improve Daikon
itself, but to find more uses for the operational abstrac-
tions that it produces. Linking it to verification tools
from the JML toolset is just one application; some oth-
ers were noted above in Section 6.1.4.
6.1.6 Availability
Daikon is publicly available, in both source and com-
piled form, from http://pag.csail.mit.edu/daikon/.
Daikon includes front ends for Java, C, Perl, and other
languages and input formats.
Several other implementations of dynamic invariant
detection exist [43,87,45]. However, they do not produce
output in JML format, they are not publicly available,
and they check and report only a small fraction of the
properties that Daikon does [83].
6.2 Inferring annotations with Houdini
6.2.1 Overview and Goals
An obstacle to using program verification tools such as
ESC/Java on legacy code is the lack of annotations in
such a program. The warnings more likely point out
missing annotations than errors in the code. The Hou-
dini tool [35, 34] attempts to alleviate this problem by
supplying many of the missing annotations.
6.2.2 Design of the Tool
Houdini works by making up candidate annotations for
the given program. Such candidate annotations compare
fields and array lengths to -1, 0, 1, constants used in
array constructors, null,true, and false (depending
on the type of the field), and indicate that arrays and
sub-arrays contain no null elements. To find which of
the candidate annotations hold for the program, Hou-
dini repeatedly invokes ESC/Java, removing those can-
14 Burdy et al.: An overview of JML tools and applications
didate annotations that ESC/Java finds to be inconsis-
tent with the code. When all remaining candidate an-
notations are consistent with the code, Houdini invokes
ESC/Java a final time to produce warnings that are then
presented to the user. Houdini thus retains the precision
of ESC/Java, trading quick turnaround for a reduced
annotation effort.
Note that any user-supplied JML annotations in the
program still get used by Houdini, since they become
part of each invocation of ESC/Java . Thus, the benefits
of using JML annotations are the same for Houdini as for
ESC/Java, but Houdini can find program errors from a
smaller set of user-supplied JML annotations.
6.2.3 Example
If the class in Figure 1 is given to Houdini without
any annotations, then Houdini will produce a number
of candidate annotations, including the invariants 0 <=
balance and 1 <= balance and the Purse-constructor
preconditions 0 <= b and 1 <= b. If the given program
contains a call to the Purse constructor that passes in 0
for b, then the candidate precondition 1 <= b is refuted
and removed. Since the constructor assigns bto balance,
the candidate invariant 1 <= balance will then eventu-
ally also become refuted.
Houdini will also include balance <= MAX BALANCE
among many other candidate annotations, but will not
include, for example, the universal quantifications shown
in Figure 1.
6.2.4 Experience
Houdini has been applied to a number of real applica-
tion programs, the initial account of which is reported
in [35]. For each of the applications, Houdini (in concert
with ESC/Java) was able to find errors. The number of
warnings produced was generally larger than the num-
ber of warnings inspected by a user. For example, in the
36-KLOC program “Cobalt” [35], only 200 of the 540
warnings were inspected by a user, though this inspec-
tion revealed 8 errors. In the largest program to which
Houdini was applied, a systems administration tool com-
prising 500 KLOC of Java, the number of warnings pro-
duced was too large to be particularly useful, though an
inspection of 10 of the warnings still revealed 2 program
errors. The experience with Houdini , albeit limited, sug-
gests that it is possible for a user to inspect a program
for errors at a rate of upwards of 1000 LOC per hour.
6.2.5 Future Work
Though Houdini has found real errors, some problems
make the tool less effective than one would like. We men-
tion three such problems here.
First, Houdini’s simple strategy for producing candi-
date annotations limits the number of ESC/Java warn-
ings it can suppress. Future work might consider apply-
ing more static analysis or dynamic profiling to improve
the initial set of candidate annotations.
Second, to reduce the number of warnings produced,
it is important for Houdini to infer good class invari-
ants. Even in the cases where Houdini’s candidate set
includes the necessary invariants, Houdini may fail to
infer them because the first point at which they hold is
unknown. For example, ESC/Java generally checks that
an object’s invariant has been established before the ob-
ject’s constructor invokes any method on the object. But
the purpose of such a method invocation is sometimes
to help establish the object’s invariant in the first place.
In an attempt to improve this situation, Houdini uses a
special mode of ESC/Java , where ESC/Java inlines any
method call from a constructor. This mode allows Hou-
dini to infer better invariants, but sometimes produces
enormous verification conditions in ESC/Java. Future
work might find a better solution to this problem.
Third, to avoid forcing users to write loop invariants,
ESC/Java by default analyzes only a fixed number of
unrollings of each loop. If the loop is known always to
go through more iterations than are unrolled (for exam-
ple, if a for loop iterates exactly 10 times, where 10
is a constant mentioned in the loop head), then the ef-
fect is that ESC/Java’s analysis doesn’t ever reach the
other side of the loop. This may be acceptable in a man-
ual application of ESC/Java, since ESC/Java performs
modular checking method by method, and therefore the
checking of other methods is unaffected. However, for
Houdini, whose inference is more like that of a whole-
program analysis, this situation can have a paralyzing
effect on the entire program analysis. Houdini side steps
this situation by using a special mode of ESC/Java,
where ESC/Java in effect introduces a jump from its
last unrolling of the loop until after the loop. This is
much better for Houdini, but it also introduces execu-
tion paths that don’t exist in the given program, which
leads to other problems. Perhaps there are better solu-
tions.
6.2.6 Availability
Work on the Houdini tool petered out in 2001 with the
transformation of the Compaq Systems Research Center.
The sources of the final version of Houdini are available
in the ESC/Java source distribution, named the “Java
Programming Toolkit Source Release”, at http://www.
research.compaq.com/downloads.html.
7 Documentation
Generating human-readable web pages from JML spec-
ifications is accomplished by the jmldoc tool.
Burdy et al.: An overview of JML tools and applications 15
7.1 jmldoc
7.1.1 Overview and Goals
The goal of the jmldoc tool is to produce HTML pages
like those produced by the javadoc tool, but including
information from JML annotations as well. JML allows
specifications to be spread across a number of refinement
files. This is essential, for example, in the case that the
Java source code may not be modified to include specifi-
cations directly in the source code. Even within one file,
the specifications relevant to the class may be spread
throughout the file, making easy spotting of a relevant
invariant difficult. Also, JML enforces behavioral inher-
itance, in which an overriding method must satisfy the
specification of the methods it overrides. Accordingly,
jmldoc includes in the HTML representation of the spec-
ifications of a method the specifications of the methods
it overrides. By combining and grouping these specifica-
tions appropriately, jmldoc makes them more accessible
to the programmer. Particularly for those accustomed
to browsing the javadoc documentation of an API, the
inclusion of the additional specifications in a formal no-
tation as provided by jmldoc is expected to be a conve-
nience.
7.1.2 Design of the Tool
The jmldoc tool is designed to leverage as much of both
the JML tools and the javadoc tool as possible. It uses
the classes of the JML checker to parse, typecheck, and
provide an AST that includes specifications of each class
and method being documented. The javadoc tool pro-
vides a doclet API that allows some reuse of the javadoc
framework. Many of the contributed doclets use the pro-
vided classes to parse valid Java source code with javadoc
comments and then to do checks or alternative process-
ing on those files, such as producing PDF rather than
HTML or checking that all methods do indeed have
javadoc comments. The jmldoc tool instead alters (by
derivation) the mechanism that generates the HTML
pages in order that the output will contain in addition
information about the JML annotations in the source
files, as provided by the JML-generated AST. In this
way, the jmldoc tool remains consistent with the other
JML tools in their handling of the JML language, but
it also produces HTML pages consistent with other cur-
rent javadoc documentation and with that produced by
the javadoc tool itself. Aside from accepting additional
command-line options appropriate to JML, jmldoc is in-
tended to be a drop-in replacement for javadoc.
7.1.3 Example
An example of jmldoc’s output is shown in Figure 5;
it shows the current output produced for the method
HashMap.size as currently specified by JML specifica-
tions for Java system classes.
7.1.4 Experience
The main experience we have with jmldoc is in documen-
tation of packages that ship with JML, such as JML’s
built-in types for modeling and its samples, and with
documentation of parts of the Java standard libraries.
While these are used by JML users on a daily basis,
there have been no formal case studies of the usefulness
of jmldoc. Informal reports, however, have been positive.
7.1.5 Future Work
The tool is being maintained as part of the JML toolset,
but not being extended further other than to keep pace
with changes in the definition of JML itself. Extensive
maintenance is also needed to keep pace with changes
in the doclet API with each new version of Java. As
it happens, the portions of the doclet API that are ex-
tended by jmldoc have been changing significantly even
between minor releases of javadoc. If this rate of change
continues, the JML project may need to seek an alter-
native design that is not tied as closely to the current
appearance of javadoc documentation in order to lessen
the maintenance burden.
7.1.6 Availability
The jmldoc tool was authored by David Cok along the
lines of the goals espoused by Raghavan [88]. It is part of
the main JML toolset available via www.jmlspecs.org,
which is developed as an open source project hosted at
SourceForge.net.
8 Applications of JML to Java Card
Although JML is able to specify arbitrary sequential
Java programs, most of the serious applications of JML
and JML tools up to now have targeted Java Card.
Java CardT M is a dialect of Java specifically designed
for the programming of the latest generation of smart-
cards. Java Card is adapted to the hardware limitations
of smartcards; for instance, it does not support floating
point numbers, strings, object cloning3, or threads.
Java Card is a well-suited target for the application
of formal methods. It is a relatively simple language
with a restricted API. Moreover, Java Card programs,
called applets, are small, typically on the order of several
KBytes of bytecode. Additionally, correctness of Java
Card programs is of crucial importance, since they are
used in sensitive applications, e.g. as bank cards, iden-
tity cards, and in mobile phones. Furthermore, once such
3The fact that Java Card does not have cloning means that a
version of the Purse example in Figure 1 rewritten to Java Card
rather than Java does verify using ESC/Java,LOOP, or JACK .
Indeed, the absence of clone in Java Card is a reason why dealing
with clone has not been a priority in these tools.
16 Burdy et al.: An overview of JML tools and applications
Fig. 5. Example jmldoc output
smartcards are issued, it is difficult, if not impossible, to
fix any software errors.
JML, and several tools for JML, have been used for
Java Card, especially in the context of the EU-supported
project VerifiCard (www.verificard.org).
JML has been used to write a formal specification
of almost the entire Java Card API [86]. This experi-
ence has shown that JML is expressive enough to specify
non-trivial existing API classes. The runtime assertion
checker has been used to specify and verify a component
of a smartcard operating system [85].
ESC/Java has been used with great success to verify
a realistic example of an electronic purse implementation
in Java Card [16]. This case study was instrumental in
convincing industrial users of the usefulness of JML and
feasibility of automated program checking by ESC/Java
for Java Card applets. In fact, this case study provided
the motivation for the development of the JACK tool
discussed earlier, which is specifically designed for Java
Card programs. One of the classes of the electronic purse
has also been verified using the LOOP tool [12]. An
overview of the work on this electronic purse, and the
way in which ESC/Java and LOOP can be used to com-
plement each other, is given in [13].
As witnessed by the development of the JACK tool
by Gemplus, Java Card smartcard programs may be
one of the niche markets where formal methods have
a promising future. Here, the cost that companies are
willing to pay to ensure the absence of certain kinds of
bugs is quite high. It seems that, given the current state
of the art, using static checking techniques to ensure
relatively simple properties (e.g., that no runtime ex-
ception ever reaches the top-level without being caught)
seems to provide an acceptable return-on-investment. It
should be noted that the very simplicity of Java Card is
not without its drawbacks. In particular, the details of
its very primitive communication with smartcards (via
a byte array buffer) is not easily abstracted away from.
It will be interesting to investigate if J2ME (Java 2 Mi-
cro Edition), which targets a wider range of electronic
consumer products, such as mobile phones and PDAs, is
also an interesting application domain for JML.
9 Related Work
9.1 Java
Many runtime assertion checkers for Java exist, for ex-
ample Jass, iContract, and Parasoft’s jContract, to name
just a few. Each of these tools has its own specification
language, thus specifications written for one tool do not
work in any other tool. And while some of these tools
support higher-level constructs such as quantifiers, all
are quite primitive when compared to JML. For exam-
ple, none include support for purity specification and
checking, model methods, refinements, or unit test inte-
gration. The developers of Jass have expressed interest
in moving to JML as their specification language.
The ChAsE tool [17] is a static checker for JML’s
assignable clauses. It performs a syntactic check on
such clauses, which, in the spirit of ESC/Java, is nei-
ther sound nor complete, but which spots many mis-
takes made in the user’s assignable clauses. ChAsE was
developed to complement the functionality missing in
other tools: not checking assignable clauses was one of
the sources of unsoundness of ESC/Java. Also, assign-
able clauses are not checked by the runtime assertion
checker, making errors in assignable clauses hard to de-
tect. The functionality to check assignable clauses is now
incorporated in ESC/Java2 . Also, the JML runtime as-
sertion checker has started to incorporate some of this
functionality.
In addition to ESC/Java(2),LOOP , and JACK , sev-
eral other tools exist for the verification of Java code,
for instance Krakatoa [69], Jive [74], and KeY [1]. The
Krakatoa tool also uses JML as specification language; it
produces proof obligations for the theorem prover Coq.
It is planned that Jive will also start supporting JML.
Burdy et al.: An overview of JML tools and applications 17
The KeY tool uses OCL instead as its specification lan-
guage, and is integrated with a commercial CASE tool.
9.2 Other languages
SPARK (www.sparkada.com, [4]) is an initiative similar
to JML in many respects, but much more mature, and
targeting Ada rather than Java. SPARK (which stands
for Spade Ada Kernel) is a language designed for pro-
gramming high-integrity systems. It is a subset of Ada95
(with no object references and subclasses, for example)
enriched with annotations to enable tool support. This
includes tools for data- and information-flow analysis,
and for code verification, in particular to ensure the ab-
sence of runtime exceptions [2]. Spark has been success-
fully used to construct high-integrity systems that have
been certified using the Common Criteria, the ISO stan-
dard for the certification of information technology secu-
rity. SPARK and the associated tools are marketed by
Praxis Critical Systems Ltd., demonstrating that this
technology is commercially viable.
A more recent initiative that is very similar to JML
is Spec# [6]. The Spec# language extends C# with
contract specifications, analogously to the way JML ex-
tends Java. The Spec# compiler then introduces runtime
checks for the declared specifications (akin to jmlc), and
the Boogie program verifier tries to prove these speci-
fications statically using an automatic theorem prover
(akin the tools described in Section 5). One difference
between Spec# and JML is that Spec# builds in a new
methodology for object invariants [5,63,7], trading re-
strictions on the kinds of programs that can be written
for a sound modular reasoning technique.
9.3 OCL: UML’s constraint language
Despite the similarity in the acronyms, JML is very dif-
ferent in its aims from UML [90]. The most basic differ-
ence is that the UML aims to cover all phases of anal-
ysis and design with many notations, and it tries to be
independent of programming language, while JML only
deals with detailed designs (for APIs) and is tied to Java.
The model in JML refers to abstract, specification-only
fields that can be used to describe the behavior of vari-
ous types. By contrast, the model of UML refers to the
general modeling process (analysis and design) and is
not limited to abstractions of individual types.
JML does have some things in common with the Ob-
ject Constraint Language (OCL) [91], which is part of
the UML standard. Like JML, OCL can be used to spec-
ify invariants and pre- and postconditions. An important
difference is that JML explicitly targets Java, whereas
OCL is not specific to any one programming language.
One could say that JML is related to Java in the same
way that OCL is related to UML.
JML clearly has the disadvantage that it can not be
used for, say, C++ programs, whereas OCL can. But it
also has obvious advantages when it comes to syntax,
semantics, and expressivity. Because JML sticks to the
Java syntax and typing rules, a typical Java programmer
will prefer JML notation over OCL notation, and, for
instance, prefer to write (in JML):
invariant pin != null && pin.length == 5;
rather than the OCL:
inv: pin <> null and pin->size() = 5
JML supports all the Java modifiers such as static,
private,public, etc., and these can be used to record
detailed design decisions for different readers. Further-
more, there are legal Java expressions that can be used
in JML specifications but that cannot be expressed in
OCL.
More significant than these limitations, or differences
in syntax, are differences in semantics. JML builds on
the (well-defined) semantics of Java. So, for instance,
equals has the same meaning in JML and Java, as does
==, and the same rules for overriding, overloading, and
hiding apply. One cannot expect this for OCL, although
efforts to define a semantics for OCL are underway.
In all, we believe that a language like JML, which
is tailored to Java, is better suited for recording the de-
tailed design of Java programs than a generic language
like OCL. Even if one uses UML in the development of
a Java application, it may be better to use JML rather
than OCL for the specification of object constraints, es-
pecially in the later stages of the development. There
has been work on automatically translating OCL to JML
[42].
10 Conclusions
We believe that JML presents a promising opportunity
to gently introduce formal specification into industrial
practice. It has the following strong points:
1. JML is easy to learn for any Java programmer, since
its syntax and semantics are very close to Java. We
believe this a crucial advantage, as a big hurdle to
introducing formal methods in industry is often that
people are not willing, or do not have the time, to
learn yet another language.
2. There is no need to invest in the construction of a
formal model before one can use JML. Or rather: the
source code is the formal model. This brings further
advantages:
It is easy to introduce the use of JML gradually,
simply by adding the odd assertion to some Java
code.
JML can be used for existing (legacy) code and
APIs. Indeed, most applications of JML and its
tools to date have involved existing APIs and
code.
18 Burdy et al.: An overview of JML tools and applications
There is no discrepancy between the actual code
and the formal model. In traditional applications
of formal methods there is often a gap between
the formal model and the actual implementation,
which means that some bugs in the implementa-
tion cannot be found, because they are not part
of the formal model, and, conversely, some prob-
lems discovered in the formal model may not be
relevant for the implementation.
3. There is a growing availability of a wide range of tool
support for JML.
Unlike B, JML does not impose a particular design
methodology on its users. Unlike UML, VDM, and Z,
JML is tailored to specifying both the syntactic interface
of Java code and its behavior. Therefore, JML is better
suited than these alternative languages for documenting
the detailed design of existing Java programs.
As a common notation shared by many tools, JML
offers users multiple tools supporting the same notation.
This frees users from having to learn a whole new lan-
guage before they can start using a new tool. The shared
notation also helps the economics both for users and tool
builders. Any industrial use of formal methods will have
to be economically justified, by comparing the costs (the
extra time and effort spent) against the benefits (im-
provements in quality, number of bugs found). Having
a range of tools, offering different levels of assurance
at different costs, makes it much easier to start using
JML. One can begin with a technique that requires the
least time and effort (perhaps runtime assertion check-
ing) and then move to more labor-intensive techniques if
and when that seems worthwhile, until one has reached a
combination of tools and techniques that is cost-effective
for a particular situation.
Using any of the tools for static checking or verifi-
cation requires formal specifications of the APIs of any
system libraries used, and the cost of developing such
specifications is very high. Indeed, the largest case study
to date in using JML for specification is the ongoing work
in developing specifications for substantial parts of the
Java system libraries. Being able to reuse these same
specifications for different tools is an important advan-
tage.
Future Work
There are still many opportunities for further develop-
ment of both the JML language and its tools. For in-
stance, we would also like to see support for JML in
integrated development environments (such as Eclipse)
and integration with other kinds of static checkers.
A major recent extension to JML concerns the sup-
port for different forms of arithmetic, providing normal
mathematical integers in addition to Java’s n-bit 2’s-
complements integers [18].
One important aspect of future work is experiment-
ing with the use JML for specification of real-world code
and APIs, and using the associated tools. There has
been a lot of work on producing JML specifications of
the Java system libraries (these can be downloaded from
www.jmlspecs.org), but more work is needed.
Using JML to specify real-world code raises many
interesting issues. For instance, JML allows pure meth-
ods to be used in annotations, where pure methods are
defined as those which have no side-effects. But this is
a very strict definition, which can be impractical when
writing specifications, as many methods (including some
in core Java libraries) that programmers intuitively as-
sume to be pure are not pure, due to unobservable and
benevolent side-effects [59]. Work continues on a better
and more useful definition of purity, e.g. [8].
With more tools supporting JML, and the specifica-
tion language JML growing in complexity due to the dif-
ferent features that are useful for the different tools, one
important challenge is maintaining agreement on the se-
mantics of the language between the different tools. One
thing that has become very clear in the course of devel-
oping JML is that precisely defining the semantics of a
specification language such as JML is very tricky.
More generally, there are several fundamental issues
in the specification of object-oriented systems that are
still active topics of investigation. The notion of object
invariant is tricky in the presence of callbacks [5,7, 63,
77]. Another largely open issue is how concurrency prop-
erties should be specified.
As always in imperative programming, aliasing is a
major source of complications, and an important source
of bugs. For example, in the example in Figure 1 it is
probably important that in the constructor the field pin
is not simply aliased to the argument p, but that a new
array is created. However, the current specification does
not demand this. JML should offer practical ways to
constrain potential aliasing. A first proposal is given in
[76].
The subtleties involved in such open problems are ev-
idenced by the slightly different ways in which different
tools approach these problems. This reflects the research
(as opposed to industrial development) focus of most of
those involved in JML and its tools. Nevertheless, JML
seems to be successful in providing a common notation
and a semantics that is, at least for a growing core sub-
set, shared by many tools, and as a common notation,
JML is already proving to be useful to both tool devel-
opers and users.
Acknowledgements. Despite our long list of co-authors, still
more people have been involved in developing the tools dis-
cussed in this paper, including Joachim van den Berg, Ab-
hay Bhorkar, Kristina Boysen, Cees-Bart Breunesse, N´estor
Cata˜no, Patrice Chalin, Curtis Clifton, Kui Dai, Werner Di-
etl, Marko van Dooren, Cormac Flanagan, Mark Lillibridge,
Marieke Huisman, Bart Jacobs, Jean-Louis Lanet, Todd Mill-
Burdy et al.: An overview of JML tools and applications 19
stein, Peter Mueller, Greg Nelson, Jeremy Nimmer, Carlos
Pacheco, Arun Raghavan, Antoine Requet, Frederic Rioux,
Clyde Ruby, Jim Saxe, Raymie Stata, Roy Tan, and Martijn
Warnier. Thanks to Raymie Stata for his initiative in getting
the JML and ESC/Java projects to agree on a common syn-
tax, and to Michael M¨oller for the logo. Work on the JML
tools at Iowa State builds on the MultiJava compiler written
by Curtis Clifton as an adaptation of the Kopi Java compiler.
References
1. W. Ahrendt, Th. Baar, B. Beckert, R. Bubel, M. Giese,
R. H¨ahnle, W. Menzel, W. Mostowski, A. Roth,
S. Schlager, and P. H. Schmitt. The KeY tool. Soft-
ware and System Modeling, 2004. To appear.
2. Peter Amey and Roderick Chapman. Industrial strength
exception freedom. In ACM SigAda 2002, pages 1–9.
ACM, 2002.
3. Sergio Antoy and Dick Hamlet. Automatically checking
an implementation against its formal specification. IEEE
Transactions on Software Engineering, 26(1):55–69, Jan-
uary 2000.
4. John Barnes. High Integrity Software: The SPARK Ap-
proach to Safety and Security. Addison Wesley, 2003.
5. Mike Barnett, Robert DeLine, Manuel F¨ahndrich,
K. Rustan M. Leino, and Wolfram Schulte. Verification
of object-oriented programs with invariants. Journal of
Object Technology, 3(6):27–56, 2004.
6. Mike Barnett, K. Rustan M. Leino, and Wolfram Schulte.
The Spec# programming system: An overview. In Con-
struction and Analysis of Safe, Secure and Interoperable
Smart devices (CASSIS), LNCS, 2004. To appear.
7. Mike Barnett and David Naumann. Friends need a bit
more: Maintaining invariants over shared state. In Dex-
ter Kozen, editor, Mathematics of Program Construction,
volume 3125 of LNCS, pages 54–84. Springer, July 2004.
8. Mike Barnett, David A. Naumann, Wolfram Schulte, and
Qi Sun. 99.44% pure: Useful abstractions in specifi-
cations. In Formal Techniques for Java-like Programs.
Proceedings of the ECOOP’2004 Workshop, pages 11–
18, 2004. Technical Report NIII-R0426, University of
Nijmegen.
9. D. Bartetzko, C. Fischer, M. M¨oller, and H. Wehrheim.
Jass — Java with assertions. In Workshop on Runtime
Verification at CAV’01, 2001. Published in ENTCS, K.
Havelund and G. Rosu (eds.), 55(2), 2001.
10. Kent Beck and Erich Gamma. Test infected: Program-
mers love writing tests. Java Report, 3(7):37–50, 1998.
11. Joachim van den Berg and Bart Jacobs. The LOOP
compiler for Java and JML. In T. Margaria and W. Yi,
editors, TACAS’01, number 2031 in LNCS, pages 299–
312. Springer–Verlag, 2001.
12. Cees-Bart Breunesse, Joachim van den Berg, and Bart
Jacobs. Specifying and verifying a decimal representation
in Java for smart cards. In H. Kirchner and C. Ringeis-
sen, editors, AMAST’02, number 2422 in LNCS, pages
304–318. Springer–Verlag, 2002.
13. Cess-Bart Breunesse, N´estor Cata˜no, Marieke Huisman,
and Bart Jacobs. Formal methods for smart cards: an
experience report. Technical report, University of Ni-
jmegen, 2003. NIII Technical Report NIII-R0316. To
appear in Science of Computer Programming, Elsevier.
14. Yuriy Brun and Michael D. Ernst. Finding latent code
errors via machine learning over program executions.
In ICSE’04, Proceedings of the 26th International Con-
ference on Software Engineering, Edinburgh, Scotland,
May 26–28, 2004.
15. Lilian Burdy, Antoine Requet, and Jean-Louis Lanet.
Java applet correctness: A developer-oriented approach.
In D. Mandrioli K. Araki, S. Gnesi, editor, FME 2003,
volume 2805 of LNCS, pages 422–439. Springer–Verlag,
2003.
16. estor Cata˜no and Marieke Huisman. Formal specifica-
tion of Gemplus’s electronic purse case study. In L. H.
Eriksson and P. A. Lindsay, editors, FME 2002, volume
LNCS 2391, pages 272–289. Springer–Verlag, 2002.
17. estor Cata˜no and Marieke Huisman. CHASE: A static
checker for JML’s assignable clause. In Lenore D.
Zuck, Paul C. Attie, Agostino Cortesi, and Supratik
Mukhopadhyay, editors, VMCAI: Verification, Model
Checking, and Abstract Interpretation, volume 2575 of
LNCS, pages 26–40. Springer–Verlag, 2003.
18. Patrice Chalin. JML support for primitive arbitrary pre-
cision numeric types: Definition and semantics. Journal
of Object Technology, 3(6):57–79, 2004.
19. Yoonsik Cheon. A runtime assertion checker for the Java
Modeling Language. Technical Report 03-09, Depart-
ment of Computer Science, Iowa State University, Ames,
IA, April 2003. The author’s Ph.D. dissertation. Avail-
able from archives.cs.iastate.edu.
20. Yoonsik Cheon and Gary T. Leavens. The
Larch/Smalltalk interface specification language. ACM
Transactions on Software Engineering and Methodology,
3(3):221–253, July 1994.
21. Yoonsik Cheon and Gary T. Leavens. A runtime as-
sertion checker for the Java Modeling Language (JML).
In Hamid R. Arabnia and Youngsong Mun, editors, the
International Conference on Software Engineering Re-
search and Practice (SERP ’02), pages 322–328. CSREA
Press, June 2002.
22. Yoonsik Cheon and Gary T. Leavens. A simple and prac-
tical approach to unit testing: The JML and JUnit way.
In Boris Magnusson, editor, ECOOP 2002, volume 2374
of LNCS, pages 231–255. Springer–Verlag, June 2002.
23. Yoonsik Cheon, Gary T. Leavens, Murali Sitaraman, and
Stephen Edwards. Model variables: Cleanly supporting
abstraction in design by contract. Technical Report 03-
10, Department of Computer Science, Iowa State Uni-
versity, April 2003.
24. Curtis Clifton. MultiJava: Design, implementation, and
evaluation of a Java-compatible language supporting
modular open classes and symmetric multiple dispatch.
Technical Report 01-10, Department of Computer Sci-
ence, Iowa State University, Ames, Iowa, 50011, Novem-
ber 2001. Available from www.multijava.org.
25. David R. Cok. Reasoning with specifications containing
method calls in jml. In Formal Techniques for Java-like
Programs. Proceedings of the ECOOP’2004 Workshop,
pages 41–48, 2004. Technical Report NIII-R0426, Uni-
versity of Nijmegen.
26. David Detlefs, Greg Nelson, and James B. Saxe. Sim-
plify: A theorem prover for program checking. Technical
Report HPL-2003-148, HP Labs, July 2003.
27. David L. Detlefs, K. Rustan M. Leino, Greg Nelson, and
James B. Saxe. Extended static checking. Research Re-
20 Burdy et al.: An overview of JML tools and applications
port 159, Compaq Systems Research Center, December
1998.
28. Krishna Kishore Dhara and Gary T. Leavens. Forcing
behavioral subtyping through specification inheritance.
In 18th International Conference on Software Engineer-
ing, pages 258–267. IEEE Computer Society Press, 1996.
29. Nii Dodoo, Alan Donovan, Lee Lin, and Michael D.
Ernst. Selecting predicates for implications in program
analysis, March 16, 2002. Draft. http://pag.lcs.mit.
edu/~mernst/pubs/invariants-implications.ps.
30. Nii Dodoo, Lee Lin, and Michael D. Ernst. Selecting,
refining, and evaluating predicates for program analysis.
Technical Report MIT-LCS-TR-914, Massachusetts In-
stitute of Technology, Laboratory for Computer Science,
Cambridge, MA, July 21, 2003.
31. Michael D. Ernst. Dynamically Discovering Likely Pro-
gram Invariants. PhD thesis, University of Washington
Department of Computer Science and Engineering, Seat-
tle, Washington, August 2000.
32. Michael D. Ernst, Jake Cockrell, William G. Griswold,
and David Notkin. Dynamically discovering likely pro-
gram invariants to support program evolution. IEEE
Transactions on Software Engineering, 27(2):1–25, 2001.
33. Michael D. Ernst, Adam Czeisler, William G. Griswold,
and David Notkin. Quickly detecting relevant program
invariants. In ICSE 2000, Proceedings of the 22nd In-
ternational Conference on Software Engineering, pages
449–458, 2000.
34. Cormac Flanagan, Rajeev Joshi, and K. Rustan M.
Leino. Annotation inference for modular checkers. In-
formation Processing Letters, 77(2–4):97–108, February
2001.
35. Cormac Flanagan and K. Rustan M. Leino. Houdini, an
annotation assistant for ESC/Java. In J. N. Oliveira and
P. Zave, editors, FME 2001, volume LNCS 2021, pages
500–517. Springer–Verlag, 2001.
36. Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge,
Greg Nelson, James B. Saxe, and Raymie Stata. Ex-
tended static checking for Java. In ACM SIGPLAN 2002
Conference on Programming Language Design and Im-
plementation (PLDI’2002), pages 234–245, 2002.
37. Cormac Flanagan and James B. Saxe. Avoiding expo-
nential explosion: Generating compact verification con-
ditions. In Conference Record of the 28th Annual ACM
Symposium on Principles of Programming Languages,
pages 193–205. ACM, January 2001.
38. Lisa Friendly. The design of distributed hyperlinked
programming documentation. In S. Fra¨
iss`e, F. Gar-
zotto, T. Isakowitz, J. Nanard, and M. Nanard, editors,
IWHD’95, pages 151–173. Springer–Verlag, 1995.
39. Alex Groce and Willem Visser. What went wrong: Ex-
plaining counterexamples. In 10th International SPIN
Workshop on Model Checking of Software, pages 121–
135, Portland, Oregon, May 9–10, 2003.
40. Neelam Gupta and Zachary V. Heidepriem. A new struc-
tural coverage criterion for dynamic detection of program
invariants. In Proceedings of the 13th Annual Interna-
tional Conference on Automated Software Engineering
(ASE 2003), Montreal, Canada, October 8–10, 2003.
41. John V. Guttag, James J. Horning, et al. Larch: Lan-
guages and Tools for Formal Specification. Springer–
Verlag, New York, NY, 1993.
42. Ali Hamie. Translating the Object Constraint Language
into the Java Modeling Language. In Symposium on
Applied Computing. Proceedings of the 2004 ACM sym-
posium on applied computing (SAC’2004), pages 1531–
1535. ACM, 2004.
43. Sudheendra Hangal and Monica S. Lam. Tracking down
software bugs using automatic anomaly detection. In
ICSE’02, Proceedings of the 24th International Confer-
ence on Software Engineering, pages 291–301, Orlando,
Florida, May 22–24, 2002.
44. Michael Harder, Jeff Mellen, and Michael D. Ernst.
Improving test suites via operational abstraction. In
ICSE’03, Proceedings of the 25th International Confer-
ence on Software Engineering, pages 60–71, Portland,
Oregon, May 6–8, 2003.
45. Johannes Henkel and Amer Diwan. Discovering alge-
braic specifications from Java classes. In ECOOP 2003
— Object-Oriented Programming, 15th European Con-
ference, Darmstadt, Germany, July 23–25, 2003.
46. B. Jacobs, J. Kiniry, and M. Warnier. Java program
verification challenges. In FMCO 2002, volume 2852 of
LNCS, pages 202–219. Springer–Verlag, 2003.
47. Bart Jacobs. Weakest precondition reasoning for Java
programs with JML annotations. Journal of Logic and
Algebraic Programming, 58(1-2):61–88, 2004.
48. Bart Jacobs, Martijn Oostdijk, and Martijn Warnier.
Source Code Verification of a Secure Payment Applet.
Journ. of Logic and Algebraic Programming, 58(1-2):107–
120, 2004.
49. Bart Jacobs and Erik Poll. A logic for the Java Model-
ing Language JML. In H. Hussmann, editor, Fundamen-
tal Approaches to Software Engineering (FASE), volume
2029 of LNCS, pages 284–299. Springer–Verlag, 2001.
50. Bart Jacobs and Erik Poll. Java program verification
at Nijmegen: Developments and perspective. In Inter-
national Symposium on Software Security (ISSS’2003),
number 3233 in LNCS, pages 134–153. Springer–Verlag,
2004.
51. Bart Jacobs, Joachim van den Berg, Marieke Huisman,
Martijn van Berkum, Ulrich Hensel, and Hendrik Tews.
Reasoning about Java classes (preliminary report). In
OOPSLA’98, volume 33(10) of ACM SIGPLAN Notices,
pages 329–340. ACM, October 1998.
52. Cliff B. Jones. Systematic Software Development Using
VDM. International Series in Computer Science. Prentice
Hall, Englewood Cliffs, N.J., second edition, 1990.
53. Yoshio Kataoka, Michael D. Ernst, William G. Griswold,
and David Notkin. Automated support for program
refactoring using invariants. In ICSM 2001, Proceed-
ings of the International Conference on Software Main-
tenance, pages 736–743, Florence, Italy, November 6–10,
2001.
54. Joseph R. Kiniry and David R. Cok. ESC/Java2: Uniting
ESC/Java and JML: Progress and issues in building and
using ESC/Java2 and a report on a case study involving
the use of ESC/Java2 to verify portions of an internet
voting tally system. In Construction and Analysis of
Safe, Secure and Interoperable Smart devices (CASSIS),
LNCS. Springer–Verlag, 2004. To appear.
55. Reto Kramer. iContract – the Java design by con-
tract tool. TOOLS 26: Technology of Object-Oriented
Languages and Systems, Los Alamitos, California, pages
295–307, 1998.
Burdy et al.: An overview of JML tools and applications 21
56. Gary T. Leavens. An overview of Larch/C++: Behav-
ioral specifications for C++ modules. In Haim Kilov
and William Harvey, editors, Specification of Behav-
ioral Semantics in Object-Oriented Information Model-
ing, chapter 8, pages 121–142. Kluwer Academic Publish-
ers, Boston, 1996. An extended version is TR #96-01d,
Department of Computer Science, Iowa State University,
Ames, Iowa, 50011.
57. Gary T. Leavens, Albert L. Baker, and Clyde Ruby.
JML: A notation for detailed design. In Haim Kilov,
Bernhard Rumpe, and Ian Simmonds, editors, Behav-
ioral Specifications of Businesses and Systems, pages
175–188. Kluwer Academic Publishers, Boston, 1999.
58. Gary T. Leavens, Albert L. Baker, and Clyde Ruby. Pre-
liminary design of JML: A behavioral interface specifica-
tion language for Java. Technical Report 98-06u, Iowa
State University, Department of Computer Science, April
2003.
59. Gary T. Leavens, Yoonsik Cheon, Curtis Clifton, Clyde
Ruby, and David R. Cok. How the design of JML accom-
modates both runtime assertion checking and formal ver-
ification. In FMCO 2002, volume 2852 of LNCS, pages
262–284. Springer–Verlag, 2003. Also appears as tech-
nical report TR03-04, Dept. of Computer Science, Iowa
State University.
60. K. Rustan M. Leino. Extended static checking: A
ten-year perspective. In Reinhard Wilhelm, editor,
Informatics—10 Years Back, 10 Years Ahead, volume
2000 of LNCS. Springer–Verlag, 2000.
61. K. Rustan M. Leino. Efficient weakest preconditions.
Technical Report MSR-TR-2004-34, Microsoft Research,
April 2004.
62. K. Rustan M. Leino, Todd Millstein, and James B. Saxe.
Generating error traces from verification-condition coun-
terexamples. Science of Computer Programming, 2004.
To appear.
63. K. Rustan M. Leino and Peter Mller. Object invariants in
dynamic contexts. In ECOOP 2004 — Object-Oriented
Programming, 18th European Conference, pages 491–516,
Olso, Norway, June 16–18, 2004.
64. K. Rustan M. Leino, Greg Nelson, and James B. Saxe.
ESC/Java user’s manual. Technical Note 2000-002, Com-
paq SRC, October 2000.
65. K. Rustan M. Leino, James B. Saxe, and Raymie Stata.
Checking Java programs via guarded commands. Tech-
nical Note 1999-002, Compaq SRC, May 1999.
66. Ben Liblit, Alex Aiken, Alice X. Zheng, and Michael I.
Jordan. Bug isolation via remote program sampling.
In Proceedings of the ACM SIGPLAN 2003 Conference
on Programming Language Design and Implementation,
pages 141–154, San Diego, CA, June 9–11, 2003.
67. Lee Lin and Michael D. Ernst. Improving adaptability
via program steering. In ISSTA 2004, Proceedings of the
2004 International Symposium on Software Testing and
Analysis, Boston, MA, USA, July 12–14, 2004.
68. Barbara Liskov and Jeannette Wing. A behavioral notion
of subtyping. ACM Transactions on Programming Lan-
guages and Systems, 16(6):1811–1841, November 1994.
69. C. March´e, C. Paulin-Mohring, and X. Urbain. The
Krakatoa tool for certification of Java/JavaCard pro-
grams annotated in JML. Journal of Logic and Algebraic
Programming, 58(1-2):89–106, 2004.
70. Leonardo Mariani and Mauro Pezz`e. A technique for ver-
ifying component-based software. In International Work-
shop on Test and Analysis of Component Based Systems,
Barcelona, Spain, March 27–28, 2004.
71. Stephen McCamant and Michael D. Ernst. Predicting
problems caused by component upgrades. In Proceedings
of the 10th European Software Engineering Conference
and the 11th ACM SIGSOFT Symposium on the Founda-
tions of Software Engineering, pages 287–296, Helsinki,
Finland, September 3–5, 2003.
72. Stephen McCamant and Michael D. Ernst. Early identifi-
cation of incompatibilities in multi-component upgrades.
In ECOOP 2004 — Object-Oriented Programming, 18th
European Conference, Olso, Norway, June 16–18, 2004.
73. Bertrand Meyer. Object-oriented Software Construction.
Prentice Hall, New York, NY, second edition, 1997.
74. J. Meyer and A. Poetzsch-Heffter. An architecture
for interactive program provers. In S. Graf and
M. Schwartzbach, editors, TACAS’00, number 1785 in
LNCS, pages 63–77. Springer, Berlin, 2000.
75. Carroll Morgan. Programming from Specifications: Sec-
ond Edition. Prentice Hall International, Hempstead,
UK, 1994.
76. P. M¨uller, A. Poetzsch-Heffter, and G.T. Leavens. Modu-
lar specification of frame properties in JML. Concurrency
and Computation: Practice and Experience, 15(2):117–
154, 2003.
77. Peter M¨uller, Arnd Poetzsch-Heffter, and Gary T. Leav-
ens. Modular invariants for object structures. Technical
Report 424, ETH Zurich, October 2003.
78. Toh Ne Win and Michael D. Ernst. Verifying distributed
algorithms via dynamic analysis and theorem proving.
Technical Report 841, Massachusetts Institute of Tech-
nology, Laboratory for Computer Science, Cambridge,
MA, May 25, 2002.
79. Toh Ne Win, Michael D. Ernst, Stephen J. Garland, Dil-
sun Kırlı, and Nancy Lynch. Using simulated execution
in verifying distributed algorithms. Software Tools for
Technology Transfer, 6(1):67–76, 2004.
80. Jeremy W. Nimmer and Michael D. Ernst. Automatic
generation of program specifications. In ISSTA 2002,
International Symposium on Software Testing and Anal-
ysis, pages 232–242, Rome, Italy, 2002.
81. Jeremy W. Nimmer and Michael D. Ernst. Invariant
inference for static checking: An empirical evaluation. In
ACM SIGSOFT 10th International Symposium on the
Foundations of Software Engineering (FSE 2002), pages
11–20, 2002.
82. S. Owre, S. Ra jan, J.M. Rushby, N. Shankar, and M. Sri-
vas. PVS: Combining specification, proof checking, and
model checking. In R. Alur and T.A. Henzinger, editors,
Computer Aided Verification, number 1102 in LNCS,
pages 411–414. Springer–Verlag, 1996.
83. Jeff H. Perkins and Michael D. Ernst. Efficient incremen-
tal algorithms for dynamic detection of likely invariants.
In ACM SIGSOFT 12th International Symposium on the
Foundations of Software Engineering (FSE 2004), New-
port Beach, CA, USA, November 2004.
84. Dennis K. Peters and David Lorge Parnas. Using test
oracles generated from program documentation. IEEE
Transactions on Software Engineering, 24(3):161–173,
1998.
22 Burdy et al.: An overview of JML tools and applications
85. Erik Poll, Pieter Hartel, and Eduard de Jong. A Java
reference model of transacted memory for smart cards.
In Smart Card Research and Advanced Application Con-
ference (CARDIS’2002), pages 75–86. USENIX, 2002.
86. Erik Poll, Joachim van den Berg, and Bart Jacobs. For-
mal specification of the Java Card API in JML: the
APDU class. Computer Networks, 36(4):407–421, 2001.
87. Brock Pytlik, Manos Renieris, Shriram Krishnamurthi,
and Steven P. Reiss. Automated fault localization using
potential invariants. In AADEBUG’2003, Fifth Interna-
tional Workshop on Automated and Algorithmic Debug-
ging, Ghent, Belgium, September8–10, 2003.
88. Arun D. Raghavan. Design of a JML documentation
generator. Technical Report 00-12, Iowa State Univer-
sity, Department of Computer Science, July 2000.
89. Orna Raz, Philip Koopman, and Mary Shaw. Se-
mantic anomaly detection in online data sources. In
ICSE’02, Proceedings of the 24th International Confer-
ence on Software Engineering, pages 302–312, Orlando,
Florida, May 22–24, 2002.
90. Jim Rumbaugh, Ivar Jacobson, and Grady Booch. The
Unified Modeling Language Reference Manual. Addison-
Wesley Publishing Company, 1998.
91. Jos Warmer and Anneke Kleppe. The Object Constraint
Language: Precise Modeling with UML. Addison-Wesley
Publishing Company, 1999.
92. Tao Xie and David Notkin. Checking inside the black
box: Regression fault exposure and localization based on
value spectra differences. Technical Report UW-CSE-
02-12-04, University of Washington Department of Com-
puter Science and Engineering, Seattle, WA, USA, De-
cember 2002.
93. Tao Xie and David Notkin. Tool-assisted unit test selec-
tion based on operational violations. In Proceedings of
the 13th Annual International Conference on Automated
Software Engineering (ASE 2003), Montreal, Canada,
October 8–10, 2003.
... At run-time, violations of these conditions are monitored and reported. Pioneered by the Ei el language [51], programmers have embraced the design-bycontract methodology [52,53] to build high-assurance software in various languages with extensions of behavioral contracts, e.g., Java [14], C++ [16], Python [60], Haskell [91], Racket [35], Elixir [62], etc. Several studies [19,52] have also shown that behavioral contracts can e ectively support the design, development, testing, and debugging of software systems. ...
... Figure 7 depicts a simpli ed code snippet from the Umbrella project. Speci cally, the Umbrella project o ers a staking service where users can stake and unstake their tokens through the deposit (lines [11][12][13][14] and withdraw (lines 5-10) functions, respectively. The withdraw function updates the balances of a user (line 7), transfers the staked tokens (line 8), and nally checks if the transfer is successful (line 9). ...
... The Ei el programming language pioneered the "design-by-contract" methodology [51,53], advocating the idea of setting clear expectations between software components right at the outset of software development. Various languages have extensions with behavioral contracts, including but not limited to Java [14], C++ [16], Python [60], Haskell [91], Racket [35], Elixir [62], etc. ...
Article
Full-text available
Ensuring the reliability of smart contracts is of vital importance due to the wide adoption of smart contract programs in decentralized financial applications. However, statically checking many rich properties of smart contract programs can be challenging. On the other hand, dynamic validation approaches have shown promise for widespread adoption in practice. Nevertheless, as part of the programming environment for smart contracts, existing dynamic validation approaches have not provided programmers with a notion to clearly articulate the interface between components, especially for addresses representing opaque contract instances. We argue that the “design-by-contract” approach should complement the development of smart contract programs. Unfortunately, there is only limited linguistic support for that in existing smart contract languages. In this paper, we design a Solidity language extension ConSol that supports behavioral contracts. ConSol provides programmers with a modular specification and monitoring system for both functional and latent address behaviors. The key capability of ConSol is to attach specifications to first-class addresses and monitor violations when invoking these addresses. We evaluate ConSol using 20 real-world cases, demonstrating its effectiveness in expressing critical conditions and preventing attacks. Additionally, we assess ConSol’s efficiency and compare gas consumption with manually inserted assertions, showing that our approach introduces only marginal gas overhead. By separating specifications and implementations using behavioral contracts, ConSol assists programmers in writing smart contract code that is more robust and readable.
... Java thus considers these annotations to be comments. JML has several tools that can verify Java programs statically or validate them dynamically [36]. For example, static verification can be accomplished with OpenJML's extended static checker (ESC) [17]. ...
Conference Paper
Full-text available
Automated program repair (APR) tools generally use a test suite to localize bugs and validate patches. These patches may pass the test suite but still be incorrect, which is called overfitting. To better understand the relationship between code coverage and overfitting, we aim to quantify the reduction in overfitting achieved by having more tests to cover branches multiple times, once 100% branch coverage is reached. We also investigate whether having more such tests increases the chances that the generated patch is exact, meaning that the patched program is syntactically the same as the original, high quality correct code in our dataset. Our experiments used three different test suites, each covering all branches in the code: test suites that cover each branch at least 1, 3, or 5 times. We used seven well-known APR tools for Java on a dataset of buggy programs equipped with formal specifications. Using formal methods allows us to reliably and objectively check for overfitting. Our experimental results indicate that enlarging the test suite beyond 100% coverage of branches reduces overfitting. However, beyond a certain threshold, expanding the test suite to cover branches repeatedly does not reduce overfitting.
Chapter
Margaria and colleagues have emphasized a paradigm for system construction and assurance in which development is organized around building and refining one comprehensive model of the system (referred to as the “One Thing Approach” (OTA)). In this paper, we connect to several key OTA ideas by presenting an integrated collection of development artifacts for a simple safety-critical system (an Infant Incubator example called the Isolette). Our illustration uses AADL for defining system models, the HAMR model-driven development tool, the GUMBO model-based component contract language for specifying constraints, and the Slang/Logika framework for code-level automated property-based testing and verificationl. The artifacts illustrate a rigorous process, moving end-to-end from the concepts of operations and requirements to assurance cases and deployment on the formally verified seL4 microkernel. We describe pedagogical resources and tutorials that have been used to introduce both students and industry teams to formal-methods integrated with model based development.
Chapter
This chapter presents ACSL, the ANSI/ISO C Specification Language, focusing on its current implementation within the Frama-C framework. As its name suggests, ACSL is meant to express precisely and unambiguously the expected behavior of a piece of C code. It plays a central role in Frama-C, as nearly all plug-ins eventually manipulate ACSL specifications, either to generate properties that are to be verified, or to assess that the code is conforming to these specifications. It is thus very important to have a clear view of ACSL ’s semantics in order to be sure that what you check with Frama-C is really what you mean. This chapter describes the language in an agnostic way, independently of the various verification plug-ins that are implemented in the platform, which are described in more details in other chapters. It contains many examples and exercises that introduce the main features of the language and insists on the most common pitfalls that users, even experienced ones, may encounter.
Article
Class invariants — consistency constraints preserved by every operation on objects of a given type — are fundamental to building, understanding and verifying object-oriented programs. For verification, however, they raise difficulties, which have not yet received a generally accepted solution. The present work introduces a proof rule meant to address these issues and allow verification tools to benefit from invariants. It clarifies the notion of invariant and identifies the three associated problems: callbacks, furtive access and reference leak. As an example, the 2016 Ethereum DAO bug, in which $50 million were stolen, resulted from a callback invalidating an invariant. The discussion starts with a simplified model of computation and an associated proof rule, demonstrating its soundness. It then removes one by one the three simplifying assumptions, each removal raising one of the three issues, and leading to a corresponding adaptation to the proof rule. The final version of the rule can tackle tricky examples, including “challenge problems” listed in the literature.
Chapter
Use of simulation to support the design of software for robotic systems is pervasive. Typically, roboticists draw a state machine using an informal notation (not precise or machine checkable) to convey a design and guide the development of a simulation. This involves writing code for a specific simulator (using C, C++, or some proprietary language and API). Verification is carried out using simulation runs and testing the deployed system. The RoboStar technology supports a model-based, rather than this (simulation) code-centered, approach to development. Models are written using domain-specific notations in line with those accepted by roboticists. In this tutorial, we focus on modelling and verification using RoboChart, our design notation, and its tool, called RoboTool. In RoboChart, software controllers are described by timed state machines. The semantics is defined using a process algebra, namely, tock-CSP, which we can use for verification by model checking or theorem proving. Use of RoboChart complements simulation and testing.
Article
Full-text available
Design by Contract, proposed by Meyer for the programming language Eiffel, is a technique that allows run-time checks of specification violation and their treatment during program execution. Jass, Java with assertions, is a Design by Contract extension for Java allowing to annotate Java programs with specifications in the form of assertions. The Jass tool is a pre-compiler that translates annotated into pure Java programs in which compliance with the specification is dynamically tested. Besides the standard Design by Contract features known from classical program verification (e.g. pre- and postconditions, invariants), Jass additionally supports refinement, i.e. subtyping, checks and the novel concept of trace assertions. Trace assertions are used to monitor the dynamic behaviour of objects in time.
Conference Paper
A powerful approach to finding errors in computer software is to translate a given program into a verification condition, a logical formula that is valid if and only if the program is free of the classes of errors under consideration. Finding errors in the program is then done by mechanically searching for counterexamples to the verification condition. This paper gives an overview of the technology that goes into such program checkers, reports on some of the progress and lessons learned in the past ten years, and identifies some remaining challenges.
Article
Producing specifications by dynamic (runtime) analysis of program executions is potentially unsound, because the analyzed executions may not fully characterize all possible executions of the program. In practice, how accurate are the results of a dynamic analysis? This paper describes the results of an investigation into this question, determining how much specifications generalized from program runs must be changed in order to be verified by a static checker. Surprisingly, small test suites captured nearly all program behavior required by a specific type of static checking; the static checker guaranteed that the implementations satisfy the generated specifications, and ensured the absence of runtime exceptions. Measured against this verification task, the generated specifications scored over 90% on precision, a measure of soundness, and on recall, a measure of completeness. This is a positive result for testing, because it suggests that dynamic analyses can capture all semantic information of interest for certain applications. The experimental results demonstrate that a specific technique, dynamic invariant detection, is effective at generating consistent, sufficient specifications for use by a static checker. Finally, the research shows that combining static and dynamic analyses over program specifications has benefits for users of each technique, guaranteeing soundness of the dynamic analysis and lessening the annotation burden for users of the static analysis.
Article
This thesis describes the first steps of a project aimed at formal verification of Java programs. The work presented here is part of a larger project called LOOP, for Logic of Object Oriented Programming. A semantics for Java is described in type theory and it is shown how this semantics forms the basis for program verification. The verifications are done with the use of interactive theorem provers,namely PVS and Isabelle. Both theorem provers are described in some detail, resulting in a comparison of the strong and weak points of both systems. A Hoare logic is discussed, which is especially tailored to reasoning about Java. The LOOP project resulted in the construction of the so-called LOOP compiler, which takes Java classes as input and returns PVS or Isabelle theories, capturing the semantics of the Java program, as output. This thesis also briefly describes a specification language for Java, called JML (Java modeling language). This language can be used to specify Java classes.Finally, two non-trivial case studies are described in this thesis, verifying properties of standard Java library classes. The first case study verifies a class invariant over class Vector, the second case study verifies the functional behaviour of the class AbstractCollection
Article
We describe the basic structure of an environment for proving Java programs annotatedwith JML specications. Our method is generic with respect to the API,and thus well suited for JavaCard applets certication. It involves three distinctcomponents: the Why tool, which computes proof obligations for a core imperativelanguage annotated with pre- and post-conditions, the Coq proof assistant formodeling the program semantics and conducting the development of proofs, andnally the Krakatoa...
Article
More Java Gems presents the best articles and columns published in Java Report between 1997 and 1999. Dwight Deugo, Editor of Java Report, has carefully selected each article to be independent of any specific version of Java. The material relies mainly on those classes that are now part of the standard Java class library and APIs. Also, each article and column discusses Java topics and implementations that are not readily available in a single book. The book serves as an excellent reference to anyone involved with Java. The reader can learn more about the language, perform analysis, design and modeling, work on specific implementations, check performance, and perform testing. This book presents the good ideas of people who have used Java for 'real' applications.