Conference Paper

Empirical studies of test case prioritization in a JUnit testing environment

Dept. of Comput. Sci. & Eng., Nebraska Univ., Lincoln, NE, USA;
DOI: 10.1109/ISSRE.2004.18 Conference: Software Reliability Engineering, 2004. ISSRE 2004. 15th International Symposium on
Source: DBLP

ABSTRACT Test case prioritization provides a way to run test cases with the highest priority earliest. Numerous empirical studies have shown that prioritization can improve a test suite's rate of fault detection, but the extent to which these results generalize is an open question because the studies have all focused on a single procedural language, C, and a few specific types of test suites, in particular, Java and the JUnit testing framework are being used extensively in practice, and the effectiveness of prioritization techniques on Java systems tested under JUnit has not been investigated. We have therefore designed and performed a controlled experiment examining whether test case prioritization can be effective on Java programs tested under JUnit, and comparing the results to those achieved in earlier studies. Our analyses show that test case prioritization can significantly improve the rate of fault detection of JUnit test suites, but also reveal differences with respect to previous studies that can be related to the language and testing paradigm.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Automated program repair recently received considerable attentions, and many techniques on this research area have been proposed. Among them, two genetic-programming-based techniques, GenProg and Par, have shown the promis-ing results. In particular, GenProg has been used as the baseline technique to check the repair effectiveness of new techniques in much literature. Although GenProg and Par have shown their strong ability of fixing real-life bugs in nontrivial programs, to what extent GenProg and Par can benefit from genetic programming, used by them to guide the patch search process, is still unknown. To address the question, we present a new automated repair technique using random search, which is commonly considered much simpler than genetic programming, and implement a prototype tool called RSRepair. Experiment on 7 programs with 24 versions shipping with real-life bugs suggests that RSRepair, in most cases (23/24), outperforms GenProg in terms of both repair effectiveness (requiring fewer patch trials) and efficiency (requiring fewer test case executions), justifying the stronger strength of random search over genetic programming. According to experimental results, we suggest that every proposed technique using optimization algorithm should check its effectiveness by comparing it with random search.
    The 36th International Conference on Software Engineering (ICSE 2014), Hyderabad, India; 05/2014
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Regression testing is a testing process that is performed to provide confidence that changes in a part of software do not affect other parts of the software. Execution of the all existing test cases is the best way to re-establish the confidence.However, regression testing is an expensive process; there might be insufficient resources (e.g., time, testers) to allow for the re-execution of all test cases. Regression test prioritization techniques attempt to re-order a regression test suite based on some criteria so that highest priority test cases are executed earlier. This paper presents ComReg, a test case prioritization technique based on the dependency network of faults. We model a fault dependency network as a directed graph and identify leading faults to prioritize test cases for regression testing. We use a centrality aggregation technique which considers six network representative centrality metrics to identify leading faults in the fault dependency network. We also discuss the use of fault communities to select an arbitrary percentage of the test cases from a prioritized regression test suite. Finally, we present a case study which shows the use of the technique in a software development process.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Most techniques for automated program repair use test cases to validate the effectiveness of the produced patches. The validation process can be time-consuming espe-cially when the object programs ship with either lots of test cases or some long-running test cases. To alleviate the cost for testing, we first introduce regression test prioritization insight into the area of automated program repair, and present a novel prioritization technique called FRTP with the goal of reducing the number of test case executions in the repair process. Unlike most existing prioritization techniques frequently requiring ad-ditional cost for gathering previous test executions information, FRTP iteratively extracts that information just from the repair process, and thus incurs trivial performance lose. We also built a tool called TrpAutoRepair, which implements our FRTP technique and has the ability of automatically repairing C programs. To evaluate TrpAutoRepair, we compared it with GenProg, a state-of-the-art tool for automated C program repair. The experiment on the 5 subject programs with 16 real-life bugs provides evidence that TrpAutoRepair performs at least as good as GenProg in term of success rate, in most cases (15/16); TrpAutoRepair can significantly improve the repair efficiency by reducing efficiently the test case executions when searching a valid patch in the repair process.
    2013 29th IEEE International Conference on Software Maintenance (ICSM 2013), Eindhoven, The Netherland; 03/2014


1 Download
Available from