Conference Paper

Improved multithreaded unit testing.

DOI: 10.1145/2025113.2025145 Conference: SIGSOFT/FSE'11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC'11: 13rd European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5-9, 2011
Source: DBLP

ABSTRACT Multithreaded code is notoriously hard to develop and test. A multithreaded test exercises the code under test with two or more threads. Each test execution follows some schedule/interleaving of the multiple threads, and different schedules can give different results. Developers often want to enforce a particular schedule for test execution, and to do so, they use time delays (Thread.sleep in Java). Unfortunately, this approach can produce false positives or negatives, and can result in unnecessarily long testing time. This paper presents IMUnit, a novel approach to specifying and executing schedules for multithreaded tests. We introduce a new language that allows explicit specification of schedules as orderings on events encountered during test execution. We present a tool that automatically instruments the code to control test execution to follow the specified schedule, and a tool that helps developers migrate their legacy, sleep-based tests into event-based tests in IMUnit. The migration tool uses novel techniques for inferring events and schedules from the executions of sleep-based tests. We describe our experience in migrating over 200 tests. The inference techniques have high precision and recall of over 75%, and IMUnit reduces testing time compared to sleep-based tests on average 3.39x.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Multithreaded programs are hard to develop and test. In order for programs to avoid unexpected concurrent behaviors at runtime, for example data-races, synchronization mechanisms are typically used to enforce a safe subset of thread interleavings. Also, to test multithreaded programs, devel- opers need to enforce the precise thread schedules that they want to test. These tasks are nontrivial and error prone. This paper presents EnforceMOP, a framework for specifying and enforcing complex properties in multithreaded Java programs. A property is enforced at runtime by blocking the threads whose next actions would violate it. This way the remaining threads, whose execution is safe, can make global progress until the system eventually reaches a global state in which the blocked threads can be safely unblocked and allowed to execute. Users of EnforceMOP can specify the properties to be enforced using the expressive MOP multi-formalism notation, and can provide code to be executed at deadlock (when no thread is safe to continue). EnforceMOP was used in two different kinds of applications. First, to enforce general properties in multithreaded programs, as a formal, semantic alternative to the existing rigid and sometimes expensive syntactic synchronization mechanisms. Second, to enforce testing desirable schedules in unit testing of multithreaded programs, as an alternative to the existing limited and often adhoc techniques. Results show that EnforceMOP is able to effectively express and enforce complex properties and schedules in both scenarios.
    Proceedings of the 2013 International Symposium on Software Testing and Analysis; 07/2013
  • [Show abstract] [Hide abstract]
    ABSTRACT: Testing multithreaded programs is a hard problem, because it is challenging to expose those rare interleavings that can trigger a concurrency bug. We propose a new thread interleaving coverage-driven testing tool called Maple that seeks to expose untested thread interleavings as much as possible. It memoizes tested interleavings and actively seeks to expose untested interleavings for a given test input to increase interleaving coverage. We discuss several solutions to realize the above goal. First, we discuss a coverage metric based on a set of interleaving idioms. Second, we discuss an online technique to predict untested interleavings that can potentially be exposed for a given test input. Finally, the predicted untested interleavings are exposed by actively controlling the thread schedule while executing for the test input. We discuss our experiences in using the tool to expose several known and unknown bugs in real-world applications such as Apache and MySQL.
    Proceedings of the ACM international conference on Object oriented programming systems languages and applications; 10/2012
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Bugs in multithreaded application can be elusive. They are often hard to trace and replicate, given the usual non-determinism and irreproducibility of scheduling decisions at runtime. We present Cooperari, a tool for deterministic testing of multithreaded Java code based on cooperative execution. In a cooperative execution, threads voluntarily suspend (yield) at interference points (e.g., lock acquisition), and code between two consecutive yield points of each thread always executes serially as a transaction. A cooperative scheduler takes over control at yield points and deterministically selects the next thread to run. An application test runs multiple times, until it either fails or the state-space of schedules is deemed as covered by a configurable policy that is responsible for the scheduling decisions. Beyond failed assertions in software tests, deadlocks and races are also detected as soon as they are exposed in the cooperative execution. Cooperari effectively finds, characterizes, and deterministically reproduces bugs that are not detected under unconstrained preemptive semantics, as illustrated by standard benchmark examples.
    PPPJ '14: 2014 International Conference on Principles and Practices of Programming on the Java platform: Virtual machines, Languages, and Tools, Krakow, Poland; 09/2014

Full-text (3 Sources)

Available from
May 28, 2014