Nikolai Tillmann

Meta

In recent years, online programming and software engineering education via information technology has gained a lot of popularity. Typically, popular courses often have hundreds or thousands of students but only a few course staff members. Tool automation is needed to maintain the quality of education. In this paper, we envision that the capability...

Parameterized unit testing, recent advances in unit testing, is a new methodology extending the previous industry practice based on traditional unit tests without parameters. A parameterized unit test (PUT) is simply a test method that takes parameters, calls the code under test, and states assertions. Parameterized unit testing allows the separati...

Puzzles are the basic building block of Code Hunt contests. Creating puzzles and choosing suitable puzzles from the puzzle bank turns out to be a complex operation requiring skill and experience. Constructing a varied and interesting mix of puzzles is based on several factors. The major factor is the difficulty of the puzzle, so that the contest ca...

Software engineering tools and environments are migrating to the cloud, enabling more people to participate in programming from many more devices. To study this phenomenon in detail, we designed, implemented and deployed Touch Develop (url www.touchdevelop.com), a cloud-based integrated development environment (CIDE), which has been online for the...

Sophistication and flexibility of software development make it easy to leave security vulnerabilities in software applications for attackers. It is critical to educate and train software engineers to avoid introducing vulnerabilities in software applications in the first place such as adopting secure coding mechanisms and conducting security testin...

A tracing just-in-time (TJIT) compiler system is described for performing parallelization of code in a runtime phase in the execution of code. Upon detecting a hot loop during the execution of the code, the compiler system extracts trace information from sequentially recorded traces. In a first phase, the compiler system uses the trace information...

The description relates to media files and more particularly to media files that include a program and a human-understandable description of the program. One example can receive executable code. This example can determine a functionality provided by the executable code. The example can also create a media file that stores the executable code and pr...

Producing industry impacts has been an important, yet challenging task for the research community. In this paper, we report experiences on successful technology transfer of Pex and its relatives (tools derived from or associated with Pex) from Microsoft Research and lessons learned from more than eight years of research efforts by the Pex team in c...

Pex is an automatic white-box test-generation tool for .NET. We have established that games can be built on top of Pex to open the tool to students and to the general public. In particular, we have released Pex4Fun (www.pexforfun.com) and its successor Code Hunt (www.codehunt.com) as web-based educational gaming environments for teaching and learni...

Learning to code can be made more effective and sustainable if it is perceived as fun by the learner. Code Hunt uses puzzles that players have to explore by means of clues presented as test cases. Players iteratively modify their code to match the functional behavior of secret solutions. This way of learning to code is very different to learning fr...

We are experiencing a technology shift: Powerful and easy-to-use mobile devices like smartphones and tablets are becoming more prevalent than traditional PCs and laptops. Mobile devices are going to be the first and, in less developed countries, possibly the only computing devices which virtually all people will own and carry with them at all times...

A recent survey among developers revealed that half plan to use HTML5 for mobile apps in the future. An earlier survey showed that access to native device APIs is the biggest shortcoming of HTML5 compared to native apps. Several different approaches exist to overcome this limitation, among them cross-compilation and packaging the HTML5 as a native...

Mobile cloud computing can greatly enrich the capabilities of today’s pervasive mobile devices. Storing data on the cloud can enable features such as automatic backup, seamless transition between multiple devices, and multiuser support for existing apps. However, the process of converting local into cloud data types requires high expertise, is diff...

An isolation system is described for converting original product code into corresponding modified code. The isolation system operates by identifying a subset of original methods to be converted. For each such original method, the isolation system generates a modified part having at least one property with a type-safe delegate type which matches a s...

JavaScript has long outpaced its original target applications, being used not only for coding complex web clients, but also web servers, game development and even desktop applications. The most appealing advantage of moving applications to JavaScript is its capability to run the same code in a large number of different devices. It is not surprising...

Code Hunt (http://www.codehunt.com/) is an educational coding game (that runs in a browser) for teaching and learning computer science at scale. The game consists of a series of worlds and levels, which get increasingly challenging. In each level, the player has to discover a secret code fragment and write code for it. The game has sounds and a lea...

This book constitutes the refereed proceedings of the 8th International Conference on Tests and Proofs, TAP 2014, held in York, UK, in July 2014, as part of the STAF 2014 Federated Conferences. The 10 revised full papers and 4 short papers presented together with two tutorial descriptions were carefully reviewed and selected from 27 submissions. Th...

Dynamic Symbolic Execution (DSE) is a state-of-the-art test-generation approach that systematically explores program paths to generate high-covering tests. In DSE, the presence of loops (especially unbound loops) can cause an enormous or even infinite number of paths to be explored. There exist techniques (such as bounded iteration, heuristics, and...

Pex4Fun (http://www.pex4fun.com/) is a web-based educational gaming environment for teaching and learning programming and software engineering. Pex4Fun can be used to teach and learn programming and software engineering at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser - with In...

Programming for Mobile and Touch (PRoMoTo'13) was held at the 2013 ACM SIGPLAN conference on Systems, Programming, Languages and Applications (SPLASH 2013), October 2013 in Indianapolis, USA. Submissions for this event were invited in the general area of mobile and touch-oriented programming languages and programming environments, and teaching of p...

A finite domain approximation for symbolic terms of a symbolic state is derived, given some finite domains for basic terms of the symbolic state. A method is executed recursively for symbolic sub-terms of a symbolic term, providing a domain over-approximation that can then be provided to a solver for determining a more accurate domain. The method c...

TouchDevelop represents a new programming environment that enables users to develop mobile applications directly on mobile devices. TouchDevelop has successfully drawn a huge number of end users, who have published thousands of TouchDevelop scripts online. To enhance end-user programming on mobile devices, we conduct a comprehensive field study of...

In one embodiment, a trace optimizing engine may create an optimized trace of an operation segment of a software program. A memory may store a trace of an operation segment and an inherited stack frame from the operation segment. An optimizing trace compiler may resolve a local pointer of the operation segment to a memory object and insert a guard...

Described are symbolic finite automata for symbolically expressing and analyzing regular expression constraints, such as for use in program analysis and testing. A regular expression or pattern is transformed into a symbolic finite automaton having transitions that are labeled by formulas that denote sets of characters (rather than individual chara...

A TouchDevelop script appears to the user as statements in a language which is not unlike many other programming languages. This chapter covers the syntax and semantics of that language. The language is augmented by a powerful and rich API (Application Programming Interface), an API which significantly extends the programming capabilities of the To...

This appendix provides a worked example of using the TouchDevelop editor on a Windows phone. It does not cover all the editor’s features. Some experimentation is suggested for gaining familiarity with the editor.

TouchDevelop is a complete app creation ecosystem designed for touch, cloud connected, mobile devices. This chapter provides a brief introduction to the world of TouchDevelop scripting and the devices that support it.

Smart phones, tablets and computers provide calendars, e-mail, access to social media, some forms of instant messaging, SMS and phone calls. Even a computer might place a phone call through the use of a service like Skype. These are all forms of interaction which are covered in this chapter.KeywordsPhone CallSmart PhoneInstant MessagingAudio ClipPh...

There are many web services on the internet which allow client applications to query and store all kinds of structured information. Some web services require the user to authenticate in order to use protected resources.

Resources (services) provided by TouchDevelop are dependent on the capabilities of a particular platforms. Some methods of some datatypes may only be available on certain devices. They will not be listed by the TouchDevelop editor unless the target platform is set appropriately in the script properties.KeywordsOperating SystemGeneral Characteristic...

A smartphone or tablet can double as a portable music player. It can play music held as MP3 files on the device. It can play audio streamed over the internet, it can record sound using its microphone … and more.

A TouchDevelop script usually needs to interact with the user. While input via a microphone and output via speakers built into, or connected to, the device are certainly possibilities, a touchscreen or a screen plus a mouse is almost always used for input and output. In TouchDevelop, the screen is known as the wall. The API provides many ways in wh...

This appendix provides a worked example of creating a TouchDevelop script using an editor which runs inside your browser. It cannot cover all the editor’s features. Some experimentation is suggested for gaining familiarity with the editor. The script to be entered is shown in Figure A-1. It is published under the name rotor with code name /gtbd.Key...

This appendix reproduces material found on the TouchDevelop website at https://www.touchdevelop.com/docs/api . This appendix provides descriptions of the datatypes implemented in TouchDevelop. Appendix B covers services (also called resources).

TouchDevelop includes a “game board” API for writing simple sprite-based games. The API includes a primitive physics engine to simplify common game loops where objects are moving while they experience gravity and friction. Game board elements can be orchestrated using events based on touch and regular timing intervals.KeywordsBoard GameTouchDevelop...

A typical smartphone or tablet contains sensors which track the device’s location, movement and orientation. Scripts can use these sensors in many ways. Sensors can provide input for navigation aids, they can be an integral part of a game, and they can provide simple input to scripts. The possibilities are endless. These sensors are probably absent...

This appendix reproduces material found on the TouchDevelop website at https://www.touchdevelop.com/docs/api . It is provided here to make the book more self-contained. Appendix B covers the objects (known as resources or services) provided by the API. The datatypes are covered in Appendix C.KeywordsDatatypeAppendixScript ReviewCalendar ServiceWind...

A smartphones, tablets and laptops are commonly used for displaying photographs and videos. TouchDevelop scripts provide new ways to download, create, modify and display photographs. Scripts can be used to record videos using the device’s camera and to play back both those videos and videos obtained from elsewhere.KeywordsVideo FilePicture TypeStri...

A well-designed user interface that takes full advantage of the screen is essential for giving a professional look and feel to a program. The page and boxed constructs in TouchDevelop provide a powerful means of laying out information on the screen. A unique feature to assist script developers is that a running script can be suspended while the for...

Most mobile devices can be connected to the internet, either via a Wi-Fi connection or by using the data services of the cellphone company. Many useful scripts already exist to retrieve information from websites on the internet. By using the facilities of TouchDevelop, you can easily construct the additional scripts which extract the specific infor...

A state component saves a present state of a program or model. This state component can be invoked by the program or model itself, thereby making state a first-class citizen. As the state of the program evolves from the saved state, the saved state remains for reflection and recall, for example, for testing, verification, transaction processing, et...

Live programming allows programmers to edit the code of a running program and immediately see the effect of the code changes. This tightening of the traditional edit-compile-run cycle reduces the cognitive gap between program code and execution, improving the learning experience of beginning programmers while boosting the productivity of seasoned o...

Unit test generation tools typically aim at one of two objectives: to explore the program behavior in order to exercise automated oracles, or to produce a representative test set that can be used to manually add oracles or to use as a regression test set. Dynamic symbolic execution (DSE) can efficiently explore all simple paths through a program, e...

Described herein are interactive graphical user interfaces that visualize execution behavior of source code together with theoretical execution behavior of the source code together with the source code itself. Structure of the source code is analyzed to determine theoretical execution behavior of the source code. Thereafter, symbolic execution of t...

Massive Open Online Courses (MOOCs) have recently gained high popularity among various universities and even in global societies. A critical factor for their success in teaching and learning effectiveness is assignment grading. Traditional ways of assignment grading are not scalable and do not give timely or interactive feedback to students. To add...

Debugging mobile phone applications is hard, as current debugging techniques either require multiple computing devices or do not support graphical debugging. To address this problem we present GROPG, the first graphical on-phone debugger. We implement GROPG for Android and perform a preliminary evaluation on third-party applications. Our experiment...

We define and advocate the subfield of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for software engineering. In this subfield, gaming technologies o...

Disclosed is a process to generate and execute relevant, non-redundant test cases starting with an execution trace. An execution trace may be collected. A sequence of actions and the data involved in the actions may then be extracted from the execution trace and persisted separately from one another. Code for a non-deterministic program (“NDP”) may...

An extension of symbolic execution for programs involving contracts with quantifiers over large and potentially unbounded domains is described. Symbolic execution is used to generate, from a program, concrete test cases that exhibit mismatches between the program code and its contracts with quantifiers. Quantifiers are instantiated using symbolic v...

Mobile devices are becoming the prevalent computing platform for most people. TouchDevelop is a new mobile development environment that enables anyone with a Windows Phone to create new apps directly on the smartphone, without a PC or a traditional keyboard. At the core is a new mobile programming language and editor that was designed with the touc...

Dynamic symbolic execution (DSE) can efficiently explore all simple paths through a program, reliably determining whether there are any program crashes or violations of assertions or code contracts. However, if such automated oracles do not exist, the traditional approach is to present the developer a small and representative set of tests in order...

Applications in mobile-marketplaces may leak private user infor-mation without notification. Existing mobile platforms provide lit-tle information on how applications use private user data, making it difficult for experts to validate applications and for users to grant applications access to their private data. We propose a user-aware privacy contr...

Computers that once filled rooms now fit in our pockets, and unlike their predecessors, mobile computers abound. The mobile industry is surging, with more smartphones being sold to consumers than PCs [17]. But does the rise of mobility impact computer science education? We claim that computer science educators must seriously consider mobility as th...

From paper to computers, the way we have been writing down thoughts and performing symbolic computations has been constantly evolving. Teaching methods closely follow this trend, leveraging existing technology to make teaching more effective and preparing students for their later careers with available technologies. At present, we are in the middle...

From paper to computers, the way that we have been writing down thoughts and performing symbolic computations has been constantly evolving. Teaching methods closely follow this trend, leveraging existing technology to make teaching more effective and preparing students for their later careers with the available technology. Right now, in 2012, we ar...

TouchDevelop represents a radically new mobile application development model, as TouchDevelop enables mobile application development on a mobile device. I.e., with TouchDevelop, the task of programming say a Windows Phone is shifted from the desktop computer to the mobile phone itself. We describe a first experiment on independent, non-expert subje...

Code search has always been essential to software development; it is the cornerstone of activities such as program comprehension and maintenance. Traditionally, code search required learning of complex query languages with very steep learning curves.

We are experiencing a technology shift: powerful and easy-to-use touchscreen-based mobile devices such as smartphones and tablets are becoming more prevalent than traditional PCs and laptops. Many mobile devices are going to be the first and, in less developed countries, possibly the only computing devices that virtually all people would own and ca...

Platforms such as Windows Azure let applications conduct data-intensive cloud computing. Unit testing can help ensure high-quality development of such applications, but the results depend on test inputs and the cloud environment's state. Manually providing various test inputs and cloud states is laborious and time-consuming. However, automated test...

Pex4Fun (pexforfun.com) is a web-based serious gaming environment for teaching computing at many levels, from high school all the way through graduate courses. Unique to the Pex4Fun experience is a cloud-based program evaluation engine based on dynamic symbolic execution and SMT-solving, which provides customized feedback to the student and automat...

We are experiencing a technology shift: Powerful and easy-to-use touchscreen-based mobile devices like smartphones and tablets are becoming more prevalent than traditional PCs and laptops. We propose that computer programming, and thus teaching of programming, can and should be done directly on the mobile devices themselves, without the need for a...

Currently, testing is still the most important approach to reduce the amount of software defects. Software quality metrics help to prioritize where additional testing is necessary by measuring the quality of the code. Most approaches to estimate whether some unit of code is sufficiently tested are based on code coverage, which measures what code fr...

High-coverage testing is challenging. Modern object-oriented programs present additional challenges for testing. One key difficulty is the generation of proper method sequences to construct desired objects as method parameters. In this paper, we cast the problem as an instance of program synthesis that automatically generates candidate programs to...

Technology transfer is typically viewed as being from academia to industry but it can indeed go in either direction. Many of the same challenges then apply -- platform suitability, timeliness, support, and community building. In this paper, we describe recent efforts to transfer technology for research and teaching from an industrial research labor...

The world is experiencing a technology shift. In 2011, more touchscreen-based mobile devices like smartphones and tablets will be sold than desktops, laptops, and netbooks combined. In fact, in many cases incredibly powerful and easy-to-use smart phones are going to be the first and, in less developed countries, possibly the only computing devices...

The Inspector/Executor is well-known for parallelizing loops with irregular access patterns that cannot be analyzed statically. The downsides of existing inspectors are that it is hard to amortize their high run-time overheads by actually executing the loop in parallel, that they can only be applied to loops with dependencies that do not change dur...

Recent advances in software testing introduced parameterized unit tests (PUT), which accept parameters, unlike conventional unit tests (CUT), which do not accept parameters. PUTs are more beneficial than CUTs with regards to fault-detection capability, since PUTs help describe the behaviors of methods under test for all test arguments. In general,...

Achieving high structural coverage is an important goal of software testing. Instead of manually producing test inputs that achieve high structural coverage, testers or developers can employ tools built based on automated test-generation approaches, such as Pex, to automatically generate such test inputs. Although these tools can easily generate te...

Software-defect detection is an increasingly important research topic in software engineering. To detect defects in a program, static verification and dynamic test generation are two important proposed techniques. However, both of these techniques face their respective issues. Static verification produces false positives, and on the other hand, dyn...

An important goal of software testing is to achieve at least high structural coverage. To reduce the manual eorts of producing such high-covering test inputs, testers or developers can employ tools built based on automated structural test-generation approaches. Although these tools can easily achieve high structural coverage for simple programs, wh...

We present results for the “Impact Project Focus Area” on the topic of symbolic execution as used in software testing. Symbolic execution is a program analysis technique introduced in the 70s that has received renewed interest in recent years, due to algorithmic advances and increased availability of computational power and constraint solving techn...

Pex4Fun from Microsoft Research is a web-based serious gaming environment for teaching computer science. Pex4Fun can be used to teach and learn computer programming at many levels, from high school all the way through graduate courses. With Pex4Fun, a student edits code in any browser - with Intellisense - and Pex4Fun executes it and analyzes it in...

Software programs evolve throughout their lifetime undergoing various changes. While making these changes, software developers may introduce regression faults. It is desirable to detect these faults as quickly as possible to reduce the cost involved in fixing them. One existing solution is continuous testing, which runs an existing test suite to qu...

User-input validators play an essential role in guarding a web application against application-level attacks. Hence, the security of the web application can be compromised by defective validators. To detect defects in validators, testing is one of the most commonly used methodologies. Testing can be performed by manually writing test inputs and ora...

Developer testing is a type of testing where developers test their code as they write it, as opposed to testing done by a separate quality assurance organization. Developer testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes faults early in the software development life cycle. Effec...

It is well known that the later software errors are discovered during the development process, the more costly they are to repair, yet testing and automated analysis tools tend to be applied late in the development cycle. In this paper, we describe a future integrated testing environment (FITE) that continually analyzes code for a variety of functi...

Test coverage criteria including boundary-value and logical coverage such as Modified Condition/Decision Coverage (MC/DC) have been increasingly used in safety-critical or mission-critical domains, complementing those more popularly used structural coverage criteria such as block or branch coverage. However, existing automated test-generation appro...

Mutation testing has been used to assess and improve the quality of test inputs. Generating test inputs to achieve high mutant-killing ratios is important in mutation testing. However, existing test-generation techniques do not provide effective support for killing mutants in mutation testing. In this paper, we propose a general test-generation app...

Tracing just-in-time compilers (TJITs) determine frequently executed traces (hot paths and loops) in running programs and focus their optimization effort by emitting optimized machine code specialized to these traces. Prior work has established this strategy to be especially beneficial for dynamic languages such as JavaScript, where the TJIT interf...

Although much progress has been made in software verification, software testing remains by far the most widely used technique for improving software reliability. Among various types of testing, developer testing is a type of testing where developers test their code as they write it, as opposed to testing done by a separate quality assurance organiz...

Tracing just-in-time compilers (TJITs) determine frequently executed traces (hot paths and loops) in running programs and focus their optimization effort by emitting optimized machine code specialized to these traces. Prior work has established this strategy to be especially beneficial for dy- namic languages such as JavaScript, where the TJIT inte...

Recently there has been an upsurge of interest in both, Search–Based Software Testing (SBST), and Dynamic Symbolic Execution (DSE). Each of these two approaches has complementary strengths and weaknesses, making it a natural choice to explore the degree to which the strengths of one can be exploited to offset the weakness of the other. This paper...

Unit tests of object-oriented code exercise particular sequences of method calls. A key problem when automatically generating unit tests that achieve high structural code coverage is the selection of relevant method-call sequences, since the number of potentially relevant sequences explodes with the number of methods. To address this issue, we prop...