Jonathan de Halleux's research while affiliated with Microsoft and other places

Publications (68)

Conference Paper
Paper-based fabrication techniques offer powerful opportunities to prototype new technological interfaces. Typically, paper-based interfaces are either static mockups or require integration with sensors to provide real-time interactivity. The latter can be challenging and expensive, requiring knowledge of electronics, programming, and sensing. But...
Conference Paper
Paper-based fabrication techniques offer powerful opportunities to prototype new technological interfaces. Typically, paper-based interfaces are either static mockups or require integration with sensors to provide real-time interactivity. The latter can be challenging and expensive, requiring knowledge of electronics, programming, and sensing. But...
Conference Paper
The micro:bit (http://www.microbit.org) is a pocket-sized, programmable computing device, designed to engage people with computing technology. The micro:bit is visually appealing, fun, easy to code and inexpensive. It is widely available at schools in the United Kingdom and is now being rolled out world-wide. Key features of the micro:bit that make...
Conference Paper
The chance to influence the lives of a million children does not come often. Through a partnership between the BBC and several technology companies, a small instructional computing device called the BBC micro:bit will be given to a million children in the UK in 2016. Moreover, using the micro:bit will be part of the CS curriculum. We describe how M...
Conference Paper
Puzzles are the basic building block of Code Hunt contests. Creating puzzles and choosing suitable puzzles from the puzzle bank turns out to be a complex operation requiring skill and experience. Constructing a varied and interesting mix of puzzles is based on several factors. The major factor is the difficulty of the puzzle, so that the contest ca...
Conference Paper
Software engineering tools and environments are migrating to the cloud, enabling more people to participate in programming from many more devices. To study this phenomenon in detail, we designed, implemented and deployed Touch Develop (url www.touchdevelop.com), a cloud-based integrated development environment (CIDE), which has been online for the...
Conference Paper
Sophistication and flexibility of software development make it easy to leave security vulnerabilities in software applications for attackers. It is critical to educate and train software engineers to avoid introducing vulnerabilities in software applications in the first place such as adopting secure coding mechanisms and conducting security testin...
Patent
The description relates to media files and more particularly to media files that include a program and a human-understandable description of the program. One example can receive executable code. This example can determine a functionality provided by the executable code. The example can also create a media file that stores the executable code and pr...
Article
Producing industry impacts has been an important, yet challenging task for the research community. In this paper, we report experiences on successful technology transfer of Pex and its relatives (tools derived from or associated with Pex) from Microsoft Research and lessons learned from more than eight years of research efforts by the Pex team in c...
Article
Full-text available
Pex is an automatic white-box test-generation tool for .NET. We have established that games can be built on top of Pex to open the tool to students and to the general public. In particular, we have released Pex4Fun (www.pexforfun.com) and its successor Code Hunt (www.codehunt.com) as web-based educational gaming environments for teaching and learni...
Article
We are experiencing a technology shift: Powerful and easy-to-use mobile devices like smartphones and tablets are becoming more prevalent than traditional PCs and laptops. Mobile devices are going to be the first and, in less developed countries, possibly the only computing devices which virtually all people will own and carry with them at all times...
Patent
Full-text available
An isolation system is described for converting original product code into corresponding modified code. The isolation system operates by identifying a subset of original methods to be converted. For each such original method, the isolation system generates a modified part having at least one property with a type-safe delegate type which matches a s...
Conference Paper
Full-text available
Code Hunt (http://www.codehunt.com/) is an educational coding game (that runs in a browser) for teaching and learning computer science at scale. The game consists of a series of worlds and levels, which get increasingly challenging. In each level, the player has to discover a secret code fragment and write code for it. The game has sounds and a lea...
Conference Paper
Pex4Fun (http://www.pex4fun.com/) is a web-based educational gaming environment for teaching and learning programming and software engineering. Pex4Fun can be used to teach and learn programming and software engineering at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser - with In...
Patent
Full-text available
Described are symbolic finite automata for symbolically expressing and analyzing regular expression constraints, such as for use in program analysis and testing. A regular expression or pattern is transformed into a symbolic finite automaton having transitions that are labeled by formulas that denote sets of characters (rather than individual chara...
Conference Paper
This video demo features a new keyword programming environment for TouchDevelop, a popular touch-centric system for scripting mobile devices. The new environment allows users to simply enter a set of keywords, and its internal program synthesis engine automatically generates script snippets that most likely reflect the users' intent. Because the sy...
Conference Paper
Unit test generation tools typically aim at one of two objectives: to explore the program behavior in order to exercise automated oracles, or to produce a representative test set that can be used to manually add oracles or to use as a regression test set. Dynamic symbolic execution (DSE) can efficiently explore all simple paths through a program, e...
Patent
Described herein are interactive graphical user interfaces that visualize execution behavior of source code together with theoretical execution behavior of the source code together with the source code itself. Structure of the source code is analyzed to determine theoretical execution behavior of the source code. Thereafter, symbolic execution of t...
Conference Paper
Full-text available
We define and advocate the subfield of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for software engineering. In this subfield, gaming technologies o...
Conference Paper
Massive Open Online Courses (MOOCs) have recently gained high popularity among various universities and even in global societies. A critical factor for their success in teaching and learning effectiveness is assignment grading. Traditional ways of assignment grading are not scalable and do not give timely or interactive feedback to students. To add...
Patent
Full-text available
An extension of symbolic execution for programs involving contracts with quantifiers over large and potentially unbounded domains is described. Symbolic execution is used to generate, from a program, concrete test cases that exhibit mismatches between the program code and its contracts with quantifiers. Quantifiers are instantiated using symbolic v...
Patent
Full-text available
Disclosed is a process to generate and execute relevant, non-redundant test cases starting with an execution trace. An execution trace may be collected. A sequence of actions and the data involved in the actions may then be extracted from the execution trace and persisted separately from one another. Code for a non-deterministic program (“NDP”) may...
Conference Paper
Full-text available
Mobile devices are becoming the prevalent computing platform for most people. TouchDevelop is a new mobile development environment that enables anyone with a Windows Phone to create new apps directly on the smartphone, without a PC or a traditional keyboard. At the core is a new mobile programming language and editor that was designed with the touc...
Conference Paper
Applications in mobile-marketplaces may leak private user infor-mation without notification. Existing mobile platforms provide lit-tle information on how applications use private user data, making it difficult for experts to validate applications and for users to grant applications access to their private data. We propose a user-aware privacy contr...
Article
Dynamic symbolic execution (DSE) can efficiently explore all simple paths through a program, reliably determining whether there are any program crashes or violations of assertions or code contracts. However, if such automated oracles do not exist, the traditional approach is to present the developer a small and representative set of tests in order...
Article
Full-text available
From paper to computers, the way that we have been writing down thoughts and performing symbolic computations has been constantly evolving. Teaching methods closely follow this trend, leveraging existing technology to make teaching more effective and preparing students for their later careers with the available technology. Right now, in 2012, we ar...
Article
Code search has always been essential to software development; it is the cornerstone of activities such as program comprehension and maintenance. Traditionally, code search required learning of complex query languages with very steep learning curves.
Conference Paper
We are experiencing a technology shift: powerful and easy-to-use touchscreen-based mobile devices such as smartphones and tablets are becoming more prevalent than traditional PCs and laptops. Many mobile devices are going to be the first and, in less developed countries, possibly the only computing devices that virtually all people would own and ca...
Article
Pex4Fun (pexforfun.com) is a web-based serious gaming environment for teaching computing at many levels, from high school all the way through graduate courses. Unique to the Pex4Fun experience is a cloud-based program evaluation engine based on dynamic symbolic execution and SMT-solving, which provides customized feedback to the student and automat...
Article
We are experiencing a technology shift: Powerful and easy-to-use touchscreen-based mobile devices like smartphones and tablets are becoming more prevalent than traditional PCs and laptops. We propose that computer programming, and thus teaching of programming, can and should be done directly on the mobile devices themselves, without the need for a...
Conference Paper
Currently, testing is still the most important approach to reduce the amount of software defects. Software quality metrics help to prioritize where additional testing is necessary by measuring the quality of the code. Most approaches to estimate whether some unit of code is sufficiently tested are based on code coverage, which measures what code fr...
Conference Paper
Full-text available
High-coverage testing is challenging. Modern object-oriented programs present additional challenges for testing. One key difficulty is the generation of proper method sequences to construct desired objects as method parameters. In this paper, we cast the problem as an instance of program synthesis that automatically generates candidate programs to...
Conference Paper
Full-text available
Technology transfer is typically viewed as being from academia to industry but it can indeed go in either direction. Many of the same challenges then apply -- platform suitability, timeliness, support, and community building. In this paper, we describe recent efforts to transfer technology for research and teaching from an industrial research labor...
Article
Full-text available
The world is experiencing a technology shift. In 2011, more touchscreen-based mobile devices like smartphones and tablets will be sold than desktops, laptops, and netbooks combined. In fact, in many cases incredibly powerful and easy-to-use smart phones are going to be the first and, in less developed countries, possibly the only computing devices...
Conference Paper
Full-text available
Achieving high structural coverage is an important goal of software testing. Instead of manually producing test inputs that achieve high structural coverage, testers or developers can employ tools built based on automated test-generation approaches, such as Pex, to automatically generate such test inputs. Although these tools can easily generate te...
Conference Paper
Full-text available
Recent advances in software testing introduced parameterized unit tests (PUT), which accept parameters, unlike conventional unit tests (CUT), which do not accept parameters. PUTs are more beneficial than CUTs with regards to fault-detection capability, since PUTs help describe the behaviors of methods under test for all test arguments. In general,...
Conference Paper
Full-text available
An important goal of software testing is to achieve at least high structural coverage. To reduce the manual eorts of producing such high-covering test inputs, testers or developers can employ tools built based on automated structural test-generation approaches. Although these tools can easily achieve high structural coverage for simple programs, wh...
Conference Paper
Full-text available
Pex4Fun from Microsoft Research is a web-based serious gaming environment for teaching computer science. Pex4Fun can be used to teach and learn computer programming at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser - with Intellisense - and Pex4Fun executes it and analyzes it in...
Conference Paper
Full-text available
Software programs evolve throughout their lifetime undergoing various changes. While making these changes, software developers may introduce regression faults. It is desirable to detect these faults as quickly as possible to reduce the cost involved in fixing them. One existing solution is continuous testing, which runs an existing test suite to qu...
Conference Paper
Full-text available
Developer testing is a type of testing where developers test their code as they write it, as opposed to testing done by a separate quality assurance organization. Developer testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes faults early in the software development life cycle. Effec...
Conference Paper
Full-text available
Test coverage criteria including boundary-value and logical coverage such as Modified Condition/Decision Coverage (MC/DC) have been increasingly used in safety-critical or mission-critical domains, complementing those more popularly used structural coverage criteria such as block or branch coverage. However, existing automated test-generation appro...
Conference Paper
Full-text available
Mutation testing has been used to assess and improve the quality of test inputs. Generating test inputs to achieve high mutant-killing ratios is important in mutation testing. However, existing test-generation techniques do not provide effective support for killing mutants in mutation testing. In this paper, we propose a general test-generation app...
Conference Paper
Full-text available
Although much progress has been made in software verification, software testing remains by far the most widely used technique for improving software reliability. Among various types of testing, developer testing is a type of testing where developers test their code as they write it, as opposed to testing done by a separate quality assurance organiz...
Conference Paper
Full-text available
Recently there has been an upsurge of interest in both, Search–Based Software Testing (SBST), and Dynamic Symbolic Execution (DSE). Each of these two approaches has complementary strengths and weaknesses, making it a natural choice to explore the degree to which the strengths of one can be exploited to offset the weakness of the other. This paper...
Conference Paper
Unit tests of object-oriented code exercise particular sequences of method calls. A key problem when automatically generating unit tests that achieve high structural code coverage is the selection of relevant method-call sequences, since the number of potentially relevant sequences explodes with the number of methods. To address this issue, we prop...
Conference Paper
Isolating test cases from environment dependencies is often desirable, as it increases test reliability and reduces test execution time. However, code that calls non-virtual methods or consumes sealed classes is often impossible to test in isolation. Moles is a new lightweight framework which addresses this problem. For any .NET method, Moles allow...
Conference Paper
We describe a technique and a tool called Qex for generating input tables and parameter values for a given parameterized SQL query. The evaluation semantics of an SQL query is translated into a specific background theory for a satisfiability modulo theories (SMT) solver as a set of equational axioms. Symbolic evaluation of a goal formula together w...
Conference Paper
Unit testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes bugs early in the software development life cycle. However, manual unit testing is often tedious and insufficient. Testing tools can be used to enable economical use of resources by reducing manual effort. Recently parameteriz...
Conference Paper
Full-text available
Test coverage such as branch coverage is com- monly measured to assess the sufficiency of test inputs. To reduce tedious manual efforts in generating high-covering test inputs, various automated techniques have been proposed. Some recent effective techniques include Dynamic Symbolic Execution (DSE) based on path exploration. However, these existing...
Conference Paper
Concurrency is pervasive in large systems. Unexpected interference among threads often results in ldquoHeisenbugsldquo that are extremely difficult to reproduce and eliminate. We have implemented a tool called CHESS for finding and reproducing such bugs. When attached to a program, CHESS takes control of thread scheduling and uses efficient search...
Conference Paper
Full-text available
Dynamic symbolic execution is a structural testing technique that systematically explores feasible paths of the program under test by running the program with different test inputs to improve code coverage. To address the space-explosion issue in path exploration, we propose a novel approach called Fitnex, a search strategy that uses state-dependen...
Conference Paper
Full-text available
Unit testing is a technique of testing a single unit of a program in isolation. The testability of the unit under test can be reduced when the unit interacts with its environment. The construction of high-covering unit tests and their execution require appropriate interactions with the environment such as a file system or database. To help set up t...
Conference Paper
Programs that interact with the file system are a classical challenge for automated software testing. A common approach to handling this problem is to insert an abstraction layer between the application and the file system. However, even with a well-defined abstraction layer, the burden on the software developer or tester is still high: they have t...
Article
Full-text available
Recently parameterized unit testing has emerged as a promising and effective methodology to allow the separa-tion of (1) specifying external, black-box behavior (e.g., as-sumptions and assertions) by developers and (2) generat-ing and selecting internal, white-box test inputs (i.e., high-code-covering test inputs) by tools. A parameterized unit tes...
Conference Paper
Full-text available
An objective of unit testing is to achieve high structural cov- erage of the code under test. Achieving high structural cov- erage of object-oriented code requires desirable method-call sequences that create and mutate objects. These sequences help generate target object states such as argument or re- ceiver object states (in short as target states...
Conference Paper
Full-text available
Regression test generation aims at generating a test suite that can detect behavioral differences between the original and the modified versions of a program. Regression test generation can be automated by using dynamic symbolic execution (DSE), a state-of-the-art test generation technique, to generate a test suite achieving high structural coverag...
Conference Paper
Full-text available
One of the most challenging problems in deductive program verification is to find inductive program invariants typically expressed using quantifiers. With strong-enough invariants, existing provers can often prove that a program satisfies its specification. However, provers by themselves do not find such invariants. We propose to automatically gene...
Conference Paper
Pex automatically produces a small test suite with high code coverage for a .NET program. To this end, Pex performs a systematic program analysis (using dynamic symbolic execution, similar to path- bounded model-checking) to determine test inputs for Parameterized Unit Tests. Pex learns the program behavior by monitoring execution traces. Pex uses...
Conference Paper
This hands-on tutorial will teach the principles of Parameterized Unit Testing [5,4] with Pex [2], an automatic test input generator for .NET which performs a systematic program analysis, similar to path bounded model-checking. A parameterized unit test is simply a method that takes parameters, calls the code under test, and states assertions.
Article
Full-text available
A unit test for an object-oriented program involves a sequence of method calls, which create, mutate, or observe objects. Some method arguments may have primitive types. Recent advances in symbolic execution enable the effective generation of relevant primitive val-ues, given a fixed sequence of method calls. However, there exists a challenge in te...
Conference Paper
This hands-on tutorial teaches the principles of Parameterized Unit Testing in Visual Studio with Pex, an automatic test input generator. A parameterized unit test (PUT) is simply a method that takes parameters, calls the code under test, and states assertions. Given a PUT written in a .NET language, Pex automatically produces a small test suite wi...
Article
Full-text available
The number of applications available in mobile marketplaces is increasing rapidly. It's very easy to become overwhelmed by the sheer size of their codebase. We propose to use code clone analysis to help manage existing applications and develop new ones. First, we propose an automatic application ranking scheme based on (dis)similarity. Traditionall...
Article
Full-text available
Although there are various emerging serious games developed for education and training purposes, there exist few serious games for practitioners or students to improve their programming or problem-solving skills in the computer science domain. To provide an open platform for creating serious games in learning computer science, we have developed a w...

Citations

... The increase of immersion through haptic multimodal coupling can be used for virtual prototyping of human-machine systems. There is increasing research on multimodal prototypes that address haptics as an interaction resource in addition to the visual and acoustic resources (Bourguet, M. L., 2003; Kelly et al., 2018). The use of multimodal and therefore more immersive prototypes allows to track the voice of customer more accurately and thus to obtain more detailed feedback from user and expert testing (Carulli et al., 2013;Stark et al. 2010). ...
... Block-based environments provide a block-based editor, but, we identified that some tools also support a hybrid editor (text and blocks), which means that it is possible to interact with the underlying language either through a blocks editor or text-based editor. Based on this, 69 of the studied tools support a block editor only, while 15 support both blocks and text editor [6,7,12,15,18,20,22,34,40,62,76,77,90,143,164]. The remaining ten tools do not mention it at all. ...
... Firstly, a full-scale blocklevel debugger offering all tracing (stepping) and inspection (watches) features in analogy to source-level debuggers. The latter not only compares with Blockly, but also with all popular visual programming systems either with blocks like Scratch and App Inventor [5] or syntax-driven hierarchical composition like Touch Develop [6]. ...
... To address this gap and help students to develop programming skills, researchers have suggested using games, tangible tools, and motivating projects [9,17]. Teaching programming with interactive games not only provides the students with a fun way to learn concepts in the classroom, but also can be integrated into extracurricular activities to increase and sustain students' interest (e.g., PlayLogo 3D [34], Code Hunt [3], etc.). In recent years, researchers have combined games with physical interaction to support students' learning processes. ...
... Machine learning Fava et al., 2015 ;Inaba et al., 2015 ;Nunzio et al., 2016 ;Riegler et al., 2015 ; 6 Software development ( Biegel et al., 2014 ;LaToza et al., 2013 ;Snijders et al., 2014Snijders et al., , 2015Yakushin and Lee, 2014 ;Xie et al., 2015 ) 6 Innovation ( Armisen and Majchrzak, 2015 ;Brandtner et al., 2014 ;Cherinka et al., 2013 ;Lauto and Valentin, 2016 ;Roth et al., 2015 ) 5 ...
... Touch Develop by Microsoft (Ball et al., 2016) combined the imperative programming style with typical object-based elements (not including any class definition or inheritance features), while offering the syntactic illusion of message-passing regarding method invocations. ...
... Another game "Code Defenders" was designed to teach the concept of mutation testing [8]. "Code Hunt" [11] gets players to implement a working solution by getting test cases to pass. ...
... If no papers were found for a particular combination, the bar is omitted. Some popular papers [23][24] [25] present methods that make the code exploration more efficient. The tools extract automatically the code interface and generate path tests following a determined algorithm. ...
... Comme nous pouvons l'imaginer il existe de nombreuse stratégies d'exploration dont des résumés sont proposés dans [86,56,65]. ...
... The test that reached a pass is allowed to conclude (by possibly giving a later fail verdict) before stopping. The two specifications above can be realised by the use at runtime of some adequate solver for enumerating instantiations in a way similar to [56] and [3]. In this case, the instances are generated one at a time, until a pass event is observed; only the instances of the same symbolic test that generated the current concrete test are then considered, and no other symbolic tests are tried. ...