Conference Paper

IMUnit: improved multithreaded unit testing

DOI: 10.1145/2025113.2025145 Conference: SIGSOFT/FSE'11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC'11: 13rd European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5-9, 2011
Source: DBLP

ABSTRACT Multithreaded code is notoriously hard to develop and test. A multithreaded test exercises the code under test with two or more threads. Each test execution follows some schedule/interleaving of the multiple threads, and different schedules can give different results. Developers often want to enforce a particular schedule for test execution, and to do so, they use time delays (Thread.sleep in Java). Unfortunately, this approach can produce false positives or negatives, and can result in unnecessarily long testing time. This paper presents IMUnit, a novel approach to specifying and executing schedules for multithreaded tests. We introduce a new language that allows explicit specification of schedules as orderings on events encountered during test execution. We present a tool that automatically instruments the code to control test execution to follow the specified schedule, and a tool that helps developers migrate their legacy, sleep-based tests into event-based tests in IMUnit. The migration tool uses novel techniques for inferring events and schedules from the executions of sleep-based tests. We describe our experience in migrating over 200 tests. The inference techniques have high precision and recall of over 75%, and IMUnit reduces testing time compared to sleep-based tests on average 3.39x.

Download full-text


Available from: Vilas Jagannath, Mar 22, 2014
  • Source
    • "Beyond cooperative execution, a number of approaches can be considered in relation to thread scheduling and bug detection. Basic scheduling constraints for program events may be specified, e.g., as in [12] "
    [Show abstract] [Hide abstract]
    ABSTRACT: Bugs in multithreaded application can be elusive. They are often hard to trace and replicate, given the usual non-determinism and irreproducibility of scheduling decisions at runtime. We present Cooperari, a tool for deterministic testing of multithreaded Java code based on cooperative execution. In a cooperative execution, threads voluntarily suspend (yield) at interference points (e.g., lock acquisition), and code between two consecutive yield points of each thread always executes serially as a transaction. A cooperative scheduler takes over control at yield points and deterministically selects the next thread to run. An application test runs multiple times, until it either fails or the state-space of schedules is deemed as covered by a configurable policy that is responsible for the scheduling decisions. Beyond failed assertions in software tests, deadlocks and races are also detected as soon as they are exposed in the cooperative execution. Cooperari effectively finds, characterizes, and deterministically reproduces bugs that are not detected under unconstrained preemptive semantics, as illustrated by standard benchmark examples.
    PPPJ '14: 2014 International Conference on Principles and Practices of Programming on the Java platform: Virtual machines, Languages, and Tools, Krakow, Poland; 09/2014
  • Source
    • "There are two problems with a timeout-based approach. First, it is sensitive to the underlying environment, hence fragile [19]. For instance, the timeout should be set to a larger value when running the program on a slower machine . "
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing multithreaded programs is a hard problem, because it is challenging to expose those rare interleavings that can trigger a concurrency bug. We propose a new thread interleaving coverage-driven testing tool called Maple that seeks to expose untested thread interleavings as much as possible. It memoizes tested interleavings and actively seeks to expose untested interleavings for a given test input to increase interleaving coverage. We discuss several solutions to realize the above goal. First, we discuss a coverage metric based on a set of interleaving idioms. Second, we discuss an online technique to predict untested interleavings that can potentially be exposed for a given test input. Finally, the predicted untested interleavings are exposed by actively controlling the thread schedule while executing for the test input. We discuss our experiences in using the tool to expose several known and unknown bugs in real-world applications such as Apache and MySQL.
    Proceedings of the ACM international conference on Object oriented programming systems languages and applications; 10/2012
  • Source
    • "However, such mechanisms are not portable and reliable in general and may require nontrivial modifications in the program text. Techniques have been proposed to guide the execution to intended thread schedules in more portable and reliable ways, where the intended schedules are specified by the programmer relative to a global timer (ConAn [16], MultithreadedTC [21]) or a sequence of user-defined events expressed in linear temporal logic (IMUnit [10]). While these techniques give the programmer the ability to impose constraints on the interleaving of threads, they do not support systematic exploration of all executions satisfying these constraints. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing is the most widely-used methodology for software validation. However, due to the nondeterministic interleavings of threads, traditional testing for concurrent programs is not as effective as for sequential programs. To attack the nondeterminism problem, software model checking techniques have been used to systematically enumerate all possible thread schedules of a test program. But such systematic and exhaustive exploration is typically too time-consuming for many test programs. We believe that the programmer's help to guide the model checker towards interesting executions is critical to circumvent this problem. We propose a testing technique and a supporting tool called CONCURRIT, which provides a model checker that can be guided programmatically within test code. While writing a test, the programmer specifies a particular thread interleaving scenario in mind using an embedded domain-specific language (DSL), and CONCURRIT explores all and only the executions realizing the intended scenario. During the exploration, the programmer is also able to observe the execution (e.g., assert invariants) and constrain the future decisions of the model checker, all within the test code.We believe that providing the programmer the ability to observe and control the exploration of executions will lead to more effective and efficient testing for concurrent programs.
    Proceedings of the 4th USENIX conference on Hot Topics in Parallelism; 06/2012
Show more