Conference PaperPDF Available

Usability-Oriented Design of Liquid Types for Java

Authors:
  • Carnegie Mellon University and University of Lisbon

Abstract and Figures

Developers want to detect bugs as early in the development lifecycle as possible, as the effort and cost to fix them increases with the incremental development of features. Ultimately, bugs that are only found in production can have catastrophic consequences. Type systems are effective at detecting many classes of bugs during development, often providing immediate feedback both at compile-time and while typing due to editor integration. Unfortunately, more powerful static and dynamic analysis tools do not have the same success due to providing false positives, not being immediate, or not being integrated into the language. Liquid Types extend the language type system with predicates, augmenting the classes of bugs that the compiler or IDE can catch compared to the simpler type systems available in mainstream programming languages. However, previous implementations of Liquid Types have not used human-centered methods for designing or evaluating their extensions. Therefore, this paper investigates how Liquid Types can be integrated into a mainstream programming language, Java, by proposing a new design that aims to lower the barriers to entry and adapts to problems that Java developers commonly encounter at runtime. Following a participatory design methodology, we conducted a developer survey to design the syntax of LiquidJava, our prototype. To evaluate if the added effort to writing Liquid Types in Java would convince users to adopt them, we conducted a user study with 30 Java developers. The results show that LiquidJava helped users detect and fix more bugs and that Liquid Types are easy to interpret and learn with few resources. At the end of the study, all users reported interest in adopting LiquidJava for their projects.
Content may be subject to copyright.
User-driven Design and Evaluation of Liquid Types in Java
CATARINA GAMBOA, LASIGE, Faculdade de Ciências da Universidade de Lisboa, Portugal
PAULO ALEXANDRE SANTOS, LASIGE, Faculdade de Ciências da Universidade de Lisboa, Portugal
CHRISTOPHER S. TIMPERLEY, School of Computer Science, Carnegie Mellon University, USA
ALCIDES FONSECA, LASIGE, Faculdade de Ciências da Universidade de Lisboa, Portugal
Bugs that are detected earlier during the development lifecycle are easier and cheaper to x, whereas bugs
that are found during production are dicult and expensive to address, and may have dire consequences.
Type systems are particularly eective at identifying and preventing bugs early in the development lifecycle
by causing invalid programs to result in build failure.
Liquid Types are more powerful than those found in mainstream programming languages, allowing the
detection of more classes of bugs. However, while Liquid Types were proposed in 2008 with their integration in
ML and subsequently introduced in C (2012), Javascript(2012) and Haskell(2014) through language extensions,
they have yet to become widely adopted by mainstream developers. This paper investigates how Liquid Types
can be integrated in a mainstream programming language, Java, by proposing a new design that aims to lower
the barrier to entry and adapts to problems that Java developers commonly encounter at runtime.
To promote accessibility, we conducted a series of developer surveys to design the syntax of LiquidJava, our
prototype. To evaluate the prototype’s usability, we conducted a user study of 30 Java developers, concluding
that users intend to use LiquidJava and that it helped to nd more bugs and debug faster.
ACM Reference Format:
Catarina Gamboa, Paulo Alexandre Santos, Christopher S. Timperley, and Alcides Fonseca. 2021. User-driven
Design and Evaluation of Liquid Types in Java. 1, 1 (October 2021), 9 pages. https://doi.org/10.1145/nnnnnnn.
nnnnnnn
1 INTRODUCTION
Software quality is a major concern throughout software development [
16
]. Given the increased
costs of nding and addressing bugs later in the development lifecycle, many developers and
organizations aim to identify issues as early as possible when they are cheaper and easier to address
by “shifting left” [
22
]. Unsurprisingly, validation techniques that are integrated into editors have
proved widely popular and been adopted by many developers.
Strong type systems have been introduced in modern programming languages (e.g., Haskell,
Java), allowing developers to specify the expected type of operations and verify, at compile time, if
those types are respected. Editors typically integrate this verication to provide developers with
immediate feedback about the incorrect use of variables and values, allowing them to debug and
resolve the cause of identied issues more quickly.
Authors’ addresses: Catarina Gamboa, cvgamboa@fc.ul.pt, LASIGE, Faculdade de Ciências da Universidade de Lisboa,
Lisboa, Portugal; Paulo Alexandre Santos, pacsantos@fc.ul.pt, LASIGE, Faculdade de Ciências da Universidade de Lisboa,
Lisboa, Portugal; Christopher S. Timperley, ctimperley@cmu.edu, School of Computer Science, Carnegie Mellon University,
Pittsburgh, USA; Alcides Fonseca, amfonseca@fc.ul.pt, LASIGE, Faculdade de Ciências da Universidade de Lisboa, Lisboa,
Portugal.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and
the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires
prior specic permission and/or a fee. Request permissions from permissions@acm.org.
©2021 Association for Computing Machinery.
XXXX-XXXX/2021/10-ART $15.00
https://doi.org/10.1145/nnnnnnn.nnnnnnn
, Vol. 1, No. 1, Article . Publication date: October 2021.
arXiv:2110.05444v1 [cs.PL] 11 Oct 2021
2 Catarina Gamboa, Paulo Alexandre Santos, Christopher S. Timperley, and Alcides Fonseca
Renement types are more expressive than the relatively simple types of these programming
languages [
13
], as they extend the language with predicates that restrict the allowed values in
variables and methods. For example, Listing 1 shows a simple renement on the variable
r,
stating
that its value must be between 0 and 255. Renement types can be used to detect simple division
by zero errors and out-of-bounds array access bugs [
23
], but they have also been used to detect
security issues [6] and protocol violations [7].
@Renement("r >= 0 && r <= 255")
int r;
r = 90; // okay
r = 200 + 60; /okay in Java, Renement Type Error:
Type expected: (r >= 0 && r <= 255);
Renement found: (r == 200 + 60)/
Listing 1. Variable refinement in LiquidJava and verification of its assignments.
Although renement types have been introduced in programming languages such as ML [
11
],
Haskell [
21
], C [
19
] and Javascript[
8
], they have not yet been widely adopted by developers. There
are many plausible explanations for the lack of adoption, including, but not limited to, a lack of
awareness about renement types and the language extensions that provide them, developers
may lack experience in writing specications, or it could be that the rst languages to support
renement types were not widely used within the industry at the time.
The primary goal of this work is to explore how to promote the wide usage of renement types
by adding them to Java, one of the most popular programming languages in the world, and using
developer feedback to improve the usability of renement types in general.
2 LIQUIDJAVA DESIGN
LiquidJava represents the integration of Liquid Types in Java, having a design focused on usability
through the identication of system requirements (Section 2.1) and the guidance of the renements
language by Java developers (Section 2.2). The language features are presented in Section 2.3
and aim to adapt to problems that Java developers commonly encounter at runtime. Finally, we
implemented a prototype for LiquidJava and integrated it into an IDE to improve the usability of
the framework (Section 2.4).
2.1 Requirements
To promote the usability of renement types, we identied the following design requirements
based on popular characteristics of successful static verication techniques (e.g., the NonNull
annotation [9]) and feedback from developers [20]:
Renements are optional.
A valid Java program without renements should type-check,
allowing developers to incrementally adopt stronger types. This requirement can be fullled
by using parametric annotations, such as the @Refinement annotation;
Idiomatic renements.
The language of the renements should be as close to Java as
possible, so the developer can intuitively use and write the specication without learning a
dierent language. To gain exibility, we encoded the renements language as strings inside
the annotations. We created an online survey, detailed in Section 2.2, to access the best syntax
for the renements, and developed the grammar based on the survey results;
Renement type-checking should be decidable
without introducing unnecessary over-
head to the compilation process and providing interactive feedback to developers as they
, Vol. 1, No. 1, Article . Publication date: October 2021.
User-driven Design and Evaluation of Liquid Types in Java 3
type. To this end, we restricted the renements language to Liquid Types [
18
] which are
veriable by SMT solvers;
Renement type-checking works on top of an existing Java type-checker
to avoid
early deprecation as the Java release cycle increases. Therefore, we implemented the rene-
ment type checker on top of Spoon [
17
] which uses the Eclipse Compiler Kit, adopting Java
features as they are released.
These requirements guided the design of LiquidJava and the implementation of a prototype of
the renement type checker that was then integrated as an IDE plugin.
2.2 Syntax Survey
To assess the best syntax of the renements language, we created and shared an online survey
with multiple possible syntaxes for dierent LiquidJava features: type renements of variables
and methods, predicate aliases, and anonymous variables. The survey had 50 valid answers from
participants familiar with Java, from which the majority was "Not Familiar" with renement types.
Participants evaluated their preference for the syntax options, and the options that were most
preferable were used to guide the development of the renements grammar. For example, Figure 1
shows two possible syntaxes for renements in methods. The rst one adds the renements before
each basic type (before the parameters and the return type), while the second only adds one
annotation with all the renements and is inspired by renements of functional programming
languages. The answers show a preference for the rst syntax over the second since the rst has a
higher rate of "Preferable" answers and a lower rate of "Not Acceptable" answers. Similar analyses
were applied to the remaining features leading to the presented in Section 2.3.results
  








  





 

Fig. 1. Preferences on the Syntax for Methods Refinements.
2.3 LiquidJava features
LiquidJava supports the renement of variables (as shown in Listing 1), elds and methods (both
parameters and return types). Listing 2 shows an example of a method with the second parameter
and return types extended with renements.
Furthermore, renements can be used in classes to model the state of their objects. Classes are
considered the fundamental programming elements of the Java language [
15
]. Although classes
themselves do not have a specic value that can be rened, they can have methods that produce
changes to the internal state of the objects. In this view, we can rene the object state when a
method is called and when the method has ended with the annotation
@StateRefinement(
from="predicate", to = "predicate").
, Vol. 1, No. 1, Article . Publication date: October 2021.
4 Catarina Gamboa, Paulo Alexandre Santos, Christopher S. Timperley, and Alcides Fonseca
@Renement("_ >= a && _ <= b")
public static int inRange(int a, @Renement("b > a") int b){
return a + 1;
}
...
inRange(10, 20); //correct
inRange(10, 2); /Renement Type Error
Type expected: (b > a);
Renement found: (b == 2) && (a == 10)/
Listing 2. Annotation of method inRange with refinements and the verification of the method’s invocations.
@ExternalRenementsFor("java.net.Socket")
@StateSet({"unconnected", "bound", "connected", "closed"})
public interface SocketRenements {
@StateRenement(to="unconnected(this)")
public void Socket();
@StateRenement(from="unconnected(this)",to="bound(this)")
public void bind(SocketAddress add);
@StateRenement(from="bound(this)", to="connected(this)")
public void connect(SocketAddress add, int timeout);
@StateRenement(from="connected(this)")
public void sendUrgentData(int n);
@StateRenement(to="closed(this)")
public void close();
}
Listing 3. Socket class object state refinement
Fig. 2. DFA that represents the
Socket class states and transitions.
Listing 3 presents the modeling of the class Socket from the external library java.net and whose
methods follow an implicit order of invocations that the DFA of Figure 2 can describe. To model this
class, we can start by dening the possible disjoint states with the annotation @StateSet and then,
for each method, describe the allowed transitions. For example, the method
sendUrgentData
can only be invoked if the object is connected, which means that this method should only be called
after the
connect
method. The developers using the Socket class can now detect if the invocations
follow the correct protocol before executing the program.
2.4 Integration with IDE
Integrated development environments (IDEs) are indispensable tools for software development
nowadays. They integrate editor services that enhance developers productivity, such as content
completion, documentation popups, and real-time type checking [
14
]. Overall, IDEs provide instant
feedback to the developers while implementing the code, helping them correct errors before
executing the programs.
, Vol. 1, No. 1, Article . Publication date: October 2021.
User-driven Design and Evaluation of Liquid Types in Java 5
To enhance the usability of LiquidJava, we have created an IDE plugin to allow the developers
to use the lightweight verication integrated with the development environment. To create the
LiquidJava plugin, we used the Language Server Protocol [
2
] that decouples the creation of the
language server from the implementation of the interface for the target editor. By using LSP we
create a single implementation of the LiquidJava language server that can be paired with a client
for any editor that implements LSP (e.g., Visual Studio Code [3], Eclipse [10], Emacs [1]).
Fig. 3. IDE Plugin reporting an error in the incorrect usage of the Socket class.
For the implementation of the LiquidJava plugin, we chose to create a client for Visual Studio
Code, given that it is one most used IDEs for Java development [
4
,
5
]. The main features of the
plugin can be seen in Figure 3 and include:
Error Reporting
This informs the user, in real-time, of the code elements whose speci-
cation could not be proved and underlines their exact location;
Error Information
The problems tab shows the specication that LiquidJava failed to
verify. For more detailed information, the user can hover the error and get all the verication
conditions used to try to prove the specication.
The plugin is available for download on the project website [
12
], along with examples for the
usage of renements in variables, methods and classes.
3 EVALUATION
To evaluate the usability of our approach, we conducted a user study with volunteer Java developers,
to answer the following research questions:
RQ. 1 Are renements easy to understand?
RQ. 2 Is it easier and faster to nd implementation errors using LiquidJava than with plain Java?
RQ. 3 How hard is it to annotate a program with renements?
RQ. 4 Are developers open to using LiquidJava in their projects?
The study had 30 participants familiar with Java, of which 80% described themselves as being
only “Vaguely Familiar” or “Not Familiar” with renement types, and 90% had no previous contact
, Vol. 1, No. 1, Article . Publication date: October 2021.
6 Catarina Gamboa, Paulo Alexandre Santos, Christopher S. Timperley, and Alcides Fonseca
with LiquidJava. Each participant had an online synchronous study session with an interviewer
and followed the tasks described in 3.1. The results of the study are presented in 3.2.
3.1 Study Configuration
We conducted the synchronous sessions through the Zoom video platform, and gave the participants
a survey with the study guidelines and answer placeholders, and a GitHub repository with the
study les. The participants were also asked to share their screen with the VSCode editor and the
document with the answers.
The study was divided into six parts, as follows:
(1) Task 1: Find the error in plain Java
Participants had to nd and x semantic errors in two
Java programs, where the implementation did not correspond to the informal documentation
presented in the associated Javadoc.
(2) Task 2: Understand Renements without prior explanation
Participants had to in-
terpret the renements present in dierent sections of the code (variables, methods and
classes) and use them correctly and incorrectly. For example, the rst exercise had a variable
with a renement type annotation and the participants had to assign both a correct and then
an incorrect value. The results on this task aim to answer
RQ. 1
by counting the correct uses
of the renements.
(3) Overview of LiquidJava
We introduced participants to LiquidJava using a 4-minute video
and a webpage [
12
] explaining the examples of the previous task. Both resources were then
available to be used within the remainder of the study. In the rest of the study, participants
used LiquidJava through an IDE extension created for Visual Studio Code;
(4) Task 3: Find the error with LiquidJava
Similar to the Task 1, participants had to nd
and x the incorrect behaviour of the programs. However, for this task, they were aided by
the LiquidJava plugin. This task, paired with the Task 1, intend to answer
RQ. 2
. The exercises
were the same in both tasks, but they were split into two sets so that each participant could
have dierent exercises in each task. Hence, half the participants had one set of exercises
for Task 1 and a dierent one for Task 3, and the remaining half had the reverse set order.
Therefore, the plain Java results serve as a baseline for the LiquidJava results.
(5) Task 4: Annotate Java programs with LiquidJava
Participants were asked to add Liquid-
Java annotations to three Java programs that targeted the LiquidJava features of renements
on variables, elds, methods and classes. This part targets
RQ. 3
, and includes a nal question
about the diculty of adding the annotations.
(6) Final Comments
Participants had the opportunity to express their thoughts on the
overall experience of using LiquidJava by sharing what they most liked and disliked about the
framework and whether they would use LiquidJava in their projects. This overview aimed to
answer the last research question, RQ. 4, and provide feedback to improve the project.
3.2 Results
The answers were reviewed and evaluated, leading to the results presented in this section. We
evaluated the answers to the presented problems using the categories: Correct,Incorrect,Unanswered,
and Compiler Correct.Compiler correct represents the answers that lead to the expected behaviour
of the LiquidJava compiler without being completely correct according to the exercise.
Figure 4a shows that the renements on variables and methods were intuitive to use since a
large majority of participants answered correctly. The renements in the class were less intuitive,
and 26.7% of the participants even decided to leave this question unanswered. However, after the
short overview on LiquidJava, all participants could annotate the class with the protocol using
, Vol. 1, No. 1, Article . Publication date: October 2021.
User-driven Design and Evaluation of Liquid Types in Java 7
Variables Methods Class Protocol
Annotation Target
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
Number of Answers
86.7%
96.7%
46.7%
13.3%
3.3%
26.7%
26.7%
LiquidJava Usage before Overview
Correct Incorrect Unanswered
(a) Answers to Understanding Refinements without
prior explanation.
Variables Methods Class Protocol Class Fields
Annotation Target
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
Number of Answers
100.0%
80.0%
100.0%
43.3%
20.0%
56.7%
LiquidJava Annotation Results
Correct Compiler Correct Incorrect Unanswered
(b) Answers to Annotation with LiquidJava.
Fig. 4. Understandability of LiquidJava.
LiquidJava (Figure 5b), providing support for the notion that they are easy to learn and understand
(
RQ. 1
). When participants were asked to rate the ease of adding annotations on a scale of 0 (very
dicult) to 5 (very easy), 60% assigned it a value of 5, while the remaining 40% chose the value 4.
This result provides evidence that it is fairly easy to add the renement to the code (RQ. 3).
To Find To Fix
Answers
1
3
5
7
9
11
13
15
Number of Answers
6.7%
53.3%
53.3%
40.0% 46.7%
Using Java
To Find To Fix
Answers
100.0%
46.7%
53.3%
Using LiquidJava
Correct Compiler Correct Incorrect Unanswered
Answers to Socket
(a) Answers on finding and fixing bug in Socket exam-
ple.
100 200 300 400 500 600 700
Time (s)
Java
LiquidJava
Version
Comparison on Java and LiquidJava
time expent for bug finding
(b) Time for finding and fixing bug in ArrayDeque example.
Fig. 5. Using Java and LiquidJava to find and fix implementation bugs.
In Tasks 1 and 3, two programs were used. The rst one consisted of identifying the wrong order
in which methods are called (Listing 3). As shown in Figure 5a, almost no participant could nd or
x the error without the help of LiquidJava. In the second program, also with a wrong sequence
of invocations of ArrayDeque methods, every participant was able nd and x the bugs in both
Java and LiquidJava, but they were faster when using LiquidJava (Figure 5). This result shows that
LiquidJava might be more useful when added to lesser-known classes and protocols, and can reduce
the time spent on the localization of the bugs, answering to RQ. 2.
On the overview of the experience, participants reported that they mostly enjoyed LiquidJava’s
error reporting, state renement, and the syntax of its language. For the features that the participants
, Vol. 1, No. 1, Article . Publication date: October 2021.
8 Catarina Gamboa, Paulo Alexandre Santos, Christopher S. Timperley, and Alcides Fonseca
disliked, they referred to some aspects of the syntax and some plugin features, but the majority
answered that there was nothing they disliked in the usage of LiquidJava.
The last question of the study was Would you use LiquidJava in your projects? to which all the
participants answered yes, which gives condence that participants nd this tool accessible for its
gains (RQ. 4).
4 CONCLUSION
This work presents the design of LiquidJava, a project focused on promoting the usability of
renement types, specically in the Java language. To this end, we dened design requirements
and used the input of programmers familiar with Java to guide the syntax of the renements
language. The features of the LiquidJava join the existing renements concepts with the Java
features, introducing the renements of object state. An implementation of LiquidJava was created
and integrated within the Visual Studio Code editor. We developed a research study to evaluate
LiquidJava, and its results indicate that the renements are easy to understand and that developers
are interested in using LiquidJava in their projects.
REFERENCES
[1] [n.d.]. Gnu emacs. https://www.gnu.org/software/emacs/
[2] [n.d.]. Language Server Protocol. https://microsoft.github.io/language-server-protocol/
[3] 2016. Visual Studio Code. https://code.visualstudio.com/
[4] 2021. 2021 Java technology report. https://www.jrebel.com/blog/2021-java-technology-report
[5] 2021. JVM ecosystem REPORT 2021. https://snyk.io/jvm-ecosystem- report-2021/
[6]
Jesper Bengtson, Karthikeyan Bhargavan, Cédric Fournet, Andrew D. Gordon, and Sergio Maeis. 2008. Renement
Types for Secure Implementations. In Proceedings of the 21st IEEE Computer Security Foundations Symposium, CSF 2008.
IEEE Computer Society, 17–32. https://doi.org/10.1109/CSF.2008.27
[7]
Nuno Burnay, Antónia Lopes, and Vasco T. Vasconcelos. 2020. Statically Checking REST API Consumers. In Software
Engineering and Formal Methods - 18th International Conference, Frank S. de Boer and Antonio Cerone (Eds.), Vol. 12310.
265–283. https://doi.org/10.1007/978-3- 030-58768- 0_15
[8] Chugh R., Herman D., Jhala R. [n.d.]. Dependent Types for JavaScript. http://goto.ucsd.edu/~ravi/research/oopsla12-
djs.pdf.
[9]
Manuel Fähndrich and K. Rustan M. Leino. 2003. Declaring and checking non-null types in an object-oriented
language. In Proceedings of the 2003 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages
and Applications, OOPSLA 2003, Ron Crocker and Guy L. Steele Jr. (Eds.). ACM, 302–312. https://doi.org/10.1145/
949305.949332
[10] Eclipse Foundation. [n.d.]. Eclipse IDE 2021-06: The Eclipse Foundation. https://www.eclipse.org/eclipseide/
[11]
Timothy S. Freeman and Frank Pfenning. 1991. Renement Types for ML. In Proceedings of the ACM SIGPLAN’91
Conference on Programming Language Design and Implementation (PLDI), David S. Wise (Ed.). ACM, 268–277. https:
//doi.org/10.1145/113445.113468
[12] Catarina Gamboa. 2021. LiquidJava - Project Website. https://catarinagamboa.github.io/liquidjava.html
[13]
Ranjit Jhala and Niki Vazou. 2020. Renement Types: A Tutorial. CoRR abs/2010.07763 (2020). arXiv:2010.07763
https://arxiv.org/abs/2010.07763
[14]
Lennart C. L. Kats, Richard G. Vogelij, Karl Trygve Kalleberg, and Eelco Visser. 2012. Software development environ-
ments on the web: a research agenda. In ACM Symposium on New Ideas in Programming and Reections on Software,
Onward! 2012, part of SPLASH ’12, Tucson, AZ, USA, October 21-26, 2012, Gary T. Leavens and Jonathan Edwards (Eds.).
ACM, 99–116. https://doi.org/10.1145/2384592.2384603
[15]
David Holmes Ken Arnold, James Gosling. 2005. THE Java
Programming Language, Fourth Edition. Addison Wesley
Professional.
[16]
Upadhyay P. 2012. The Role of Verication and Validation in System DevelopmentLife Cycle. IOSR Journal of Computer
Engineering (IOSRJCE) 5, 1 (2012), 17–20.
[17]
Renaud Pawlak, Martin Monperrus, Nicolas Petitprez, Carlos Noguera, and Lionel Seinturier. 2016. SPOON: A
library for implementing analyses and transformations of Java source code. Softw. Pract. Exp. 46, 9 (2016), 1155–1179.
https://doi.org/10.1002/spe.2346
[18]
Patrick Maxim Rondon, Ming Kawaguchi, and Ranjit Jhala. 2008. Liquid types. In Proceedings of the ACM SIGPLAN
2008 Conference on Programming Language Design and Implementation, Rajiv Gupta and Saman P. Amarasinghe (Eds.).
, Vol. 1, No. 1, Article . Publication date: October 2021.
User-driven Design and Evaluation of Liquid Types in Java 9
ACM, 159–169. https://doi.org/10.1145/1375581.1375602
[19]
Rondon P., Bakst A., Kawaguchi M., Jhala R.,. [n.d.]. CSolve: Verifying C With Liquid Types. http://goto.ucsd.edu/
~rjhala/papers/csolve_verifying_c_with_liquid_types.pdf.
[20]
Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from building
static analysis tools at Google. Commun. ACM 61, 4 (2018), 58–66. https://doi.org/10.1145/3188720
[21]
Niki Vazou, Eric L Seidel, Ranjit Jhala, Dimitrios Vytiniotis, and Simon Peyton-Jones. 2014. Renement types for
Haskell. In ACM SIGPLAN Notices, Vol. 49. ACM, 269–282.
[22] Hyrum Wright, Titus Delafayette Winters, and Tom Manshreck. 2020. Software Engineering at Google.
[23]
Hongwei Xi and Frank Pfenning. 1998. Eliminating Array Bound Checking Through Dependent Types. In Proceedings
of the ACM SIGPLAN ’98 Conference on Programming Language Design and Implementation (PLDI). ACM, 249–257.
https://doi.org/10.1145/277650.277732
, Vol. 1, No. 1, Article . Publication date: October 2021.
... Liquid types allow programmers to enjoy many of the benefits of dependent types with minor annotation effort, by encoding refinements as first-order logic predicates, with standard operations on common data types such as integers, bit vectors, and arrays. This idea is then introduced in many other programming languages, like Haskell [21], TypeScript [22], and Java [5]. However, none of them have focused on how to refine the string type to inject syntactic and semantic requirements of strings, as FLAT does. ...
Preprint
Programmers regularly use strings to encode many types of data, such as Unix file paths, URLs, and email addresses, that are conceptually different. However, existing mainstream programming languages use a unified string type to represent them all. As a result, their type systems will keep quiet when a function requiring an email address is instead fed an HTML text, which may cause unexceptional failures or vulnerabilities. To let the type system distinguish such conceptually different string types, in this paper, we propose to regard \emph{formal languages as types} (FLAT), thereby restricting the set of valid strings by context-free grammars and semantic constraints if needed. To this end, email addresses and HTML text are treated as different types. We realize this idea in Python as a testing framework FLAT-PY. It contains user annotations, all directly attached to the user's code, to (1) define such \emph{language types}, (2) specify pre-/post-conditions serving as \emph{semantic oracles} or contracts for functions, and (3) fuzz functions via random string inputs generated from a \emph{language-based fuzzer}. From these annotations, FLAY-PY \emph{automatically} checks type correctness at runtime via \emph{code instrumentation}, and reports any detected type error as soon as possible, preventing bugs from flowing deeply into other parts of the code. Case studies on real Python code fragments show that FLAT-PY is enable to catch logical bugs from random inputs, requiring a reasonable amount of user annotations.
Article
Refinement types combine SMT decidable constraints with a compositional, syntax-directed type system to provide a convenient way to statically and automatically check properties of programs. However, when type checking fails, programmers must use cryptic error messages that, at best, point out the code location where a subtyping constraint failed to determine the root cause of the failure. In this paper, we introduce refinement type refutations, a new approach to explaining why refinement type checking fails, which mirrors the compositional way in which refinement type checking is carried out. First, we show how to systematically transform standard bidirectional type checking rules to obtain refutations. Second, we extend the approach to account for global constraint-based refinement inference via the notion of a must-instantiation: a set of concrete inhabitants of the types of subterms that suffice to demonstrate why typing fails. Third, we implement our method in HayStack—an extension to LiqidHaskell which automatically finds type-refutations when refinement type checking fails, and helps users understand refutations via an interactive user-interface. Finally, we present an empirical evaluation of HayStack using the regression benchmark-set of LiqidHaskell, and the benchmark set of G2, a previous method that searches for (non-compositional) counterexample traces by symbolically executing Haskell source. We show that HayStack can find refutations for 99.7% of benchmarks, including those with complex typing constructs (e.g., abstract and bounded refinements, and reflection), and does so, an order of magnitude faster than G2.
Article
How working statically-typed functional programmers write code is largely understudied. And yet, a better understanding of developer practices could pave the way for the design of more useful and usable tooling, more ergonomic languages, and more effective on-ramps into programming communities. The goal of this work is to address this knowledge gap: to better understand the high-level authoring patterns that statically-typed functional programmers employ. We conducted a grounded theory analysis of 30 programming sessions of practicing statically-typed functional programmers, 15 of which also included a semi-structured interview. The theory we developed gives insight into how the specific affordances of statically-typed functional programming affect domain modeling, type construction, focusing techniques, exploratory and reasoning strategies, and expressions of intent. We conducted a set of quantitative lab experiments to validate our findings, including that statically-typed functional programmers often iterate between editing types and expressions, that they often run their compiler on code even when they know it will not successfully compile, and that they make textual program edits that reliably signal future edits that they intend to make. Lastly, we outline the implications of our findings for language and tool design. The success of this approach in revealing program authorship patterns suggests that the same methodology could be used to study other understudied programmer populations.
Article
Programming language design requires making many usability-related design decisions. However, existing HCI methods can be impractical to apply to programming languages: languages have high iteration costs, programmers require significant learning time, and user performance has high variance. To address these problems, we adapted both formative and summative HCI methods to make them more suitable for programming language design. We integrated these methods into a new process, PLIERS, for designing programming languages in a user-centered way. We assessed PLIERS by using it to design two new programming languages. Glacier extends Java to enable programmers to express immutability properties effectively and easily. Obsidian is a language for blockchains that includes verification of critical safety properties. Empirical studies showed that the PLIERS process resulted in languages that could be used effectively by many programmers and revealed additional opportunities for language improvement.
Article
Creating modern software inevitably requires using application programming interfaces (APIs). While software developers can sometimes use APIs by simply copying and pasting code examples, a lack of robust knowledge of how an API works can lead to defects, complicate software maintenance, and limit what someone can express with an API. Prior work has uncovered the many ways that API documentation fails to be helpful, though rarely describes precisely why. We present a theory of robust API knowledge that attempts to explain why, arguing that effective understanding and use of APIs depends on three components of knowledge: (1) the domain concepts the API models along with terminology, (2) the usage patterns of APIs along with rationale, and (3) facts about an API’s execution to support reasoning about its runtime behavior. We derive five hypotheses from this theory and present a study to test them. Our study investigated the effect of having access to these components of knowledge, finding that while learners requested these three components of knowledge when they were not available, whether the knowledge helped the learner use or understand the API depended on the tasks and likely the relevance and quality of the specific information provided. The theory and our evidence in support of its claims have implications for what content API documentation, tutorials, and instruction should contain and the importance of giving the right information at the right time, as well as what information API tools should compute, and even how APIs should be designed. Future work is necessary to both further test and refine the theory, as well as exploit its ideas for better instructional design.
Article
Some blockchain programs (smart contracts) have included serious security vulnerabilities. Obsidian is a new typestate-oriented programming language that uses a strong type system to rule out some of these vulnerabilities. Although Obsidian was designed to promote usability to make it as easy as possible to write programs, strong type systems can cause a language to be difficult to use. In particular, ownership, typestate, and assets, which Obsidian uses to provide safety guarantees, have not seen broad adoption together in popular languages and result in significant usability challenges. We performed an empirical study with 20 participants comparing Obsidian to Solidity, which is the language most commonly used for writing smart contracts today. We observed that Obsidian participants were able to successfully complete more of the programming tasks than the Solidity participants. We also found that the Solidity participants commonly inserted asset-related bugs, which Obsidian detects at compile time.
Chapter
Consumption of REST services has become a popular means of invoking code provided by third parties, particularly in web applications. Nowadays programmers of web applications can choose TypeScript over JavaScript to benefit from static type checking that enables validating calls to local functions or to those provided by libraries. Errors in calls to REST services, however, can only be found at runtime. In this paper, we present SRS, a language that extends the support of static analysis to calls to REST services, with the ability to statically find common errors such as missing or invalid data in REST calls and misuse of the results from such calls. SRS features a syntax similar to JavaScript and is equipped with a rich collection of types and primitives to natively support REST calls that are statically validated against specifications of the corresponding APIs written in the HeadREST language.
Article
As increasingly complex software is developed every day, a growing number of companies use static analysis tools to reason about program properties ranging from simple coding style rules to more advanced software bugs, to multi-tier security vulnerabilities. While increasingly complex analyses are created, developer support must also be updated to ensure that the tools are used to their best potential. Past research in the usability of static analysis tools has primarily focused on usability issues encountered by software developers, and the causes of those issues in analysis tools. In this article, we adopt a more user-centered approach, and aim at understanding why software developers use analysis tools, which decisions they make when using those tools, what they look for when making those decisions, and the motivation behind their strategies. This approach allows us to derive new tool requirements that closely support software developers (e.g., systems for recommending warnings to fix that take developer knowledge into account), and also open novel avenues for further static-analysis research such as collaborative user interfaces for analysis warnings.
Preprint
In stream-based programming, data sources are abstracted as a stream of values that can be manipulated via callback functions. Stream-based programming is exploding in popularity, as it provides a powerful and expressive paradigm for handling asynchronous data sources in interactive software. However, high-level stream abstractions can also make it difficult for developers to reason about control- and data-flow relationships in their programs. This is particularly impactful when asynchronous stream-based code interacts with thread-limited features such as UI frameworks that restrict UI access to a single thread, since the threading behavior of streaming constructs is often non-intuitive and insufficiently documented. In this paper, we present a type-based approach that can statically prove the thread-safety of UI accesses in stream-based software. Our key insight is that the fluent APIs of stream-processing frameworks enable the tracking of threads via type-refinement, making it possible to reason automatically about what thread a piece of code runs on -- a difficult problem in general. We implement the system as an annotation-based Java typechecker for Android programs built upon the popular ReactiveX framework and evaluate its efficacy by annotating and analyzing 8 open-source apps, where we find 33 instances of unsafe UI access while incurring an annotation burden of only one annotation per 186 source lines of code. We also report on our experience applying the typechecker to two much larger apps from the Uber Technologies, Inc. codebase, where it currently runs on every code change and blocks changes that introduce potential threading bugs.
Article
For a static analysis project to succeed, developers must feel they benefit from and enjoy using it.