Johannes Mayer

Universität Ulm, Ulm, Baden-Württemberg, Germany

Are you Johannes Mayer?

Claim your profile

Publications (24)1.98 Total impact

  • Ralph Guderlei, Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing software with random output is a challenging task as the output corresponding to a given input dif- fers from execution to execution. Therefore, the usual ap- proaches to software testing are not applicable to random- ized software. Instead, statistical hypothesis tests have been proposed for testing those applications. To apply these sta- tistical hypothesis tests, either knowledge about the theo- retical values of statistical characteristics of the program output (e. g. the mean) or a reference implementation (e. g. a legacy system) are required to apply statistical hypothesis tests. But often, both are not available. In the present paper, it is discussed how a testing method called Metamorphic Testing can be used to construct sta- tistical hypothesis tests without knowing exact theoretical characteristics or having a reference implementation. For that purpose, two or more independent output sequences are generated by the implementation under test (IUT). Then, these sequences are compared according to the metamor- phic relation using statistical hypothesis tests.
    Quality Software, 2007. QSIC '07. Seventh International Conference on; 11/2007
  • Johannes Mayer, Sami Beydeda
    [Show abstract] [Hide abstract]
    ABSTRACT: The First International Workshop on Software Test Eval- uation (STEV'07) co-located with QSIC'07 is the first work- shop to address the oracle problem in software testing. Unfortunately, this important problem has often been ne- glected. The workshop is intended as a forum for those working in the field of software test oracles and its aim is to foster collaboration among those researcher in order to advance the state of the art. An program with presentations on a lot of topics in the field of test oracles guarantees an interesting and hopefully also inspiring workshop. 30350372
    Quality Software, 2007. QSIC '07. Seventh International Conference on; 11/2007
  • [Show abstract] [Hide abstract]
    ABSTRACT: Gleason grading is a common method used by pathologists to determine the aggressivity of prostate cancer on the basis of histological slide preparations. The advantage of this grading system is a good correlation with the biological behavior of the tumor, while its drawback is the subjectivity underlying the judgements of pathologists. Therefore, an automation of Gleason grading would be desirable. In this paper, we examined 780 digitized grayscale images of 78 different cases, which were split into a training and a test set. We developed two methods based on combinations of morphological characteristics like area fraction, line length, and Euler number to classify into the categories "Gleason score < 7" and "Gleason score > or = 7." In particular, the distinction between these two classes has great impact on the prognosis of patients. The agreement of each method with visual diagnosis was 87.18% and 92.31% within the training set and 66.67% and 64.10% within the test set, respectively.
    IEEE Transactions on Information Technology in Biomedicine 07/2007; 11(4):406-14. · 1.98 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: While Model Driven Architecture (MDA) has already been adopted by companies for the development of business and persistence layers, there has been no general adoption of a model-driven approach for graphical user interface (GUI) development. Therefore, the present paper describes a practical and pragmatic approach for task-dialog and ab-stract presentation modeling of GUIs. The presented model has already been used to generate user interfaces. It will be shown how UML activity diagrams can be extended in or-der to enable automated code genertation from the model— with some manual intervention (with a GUI builder to lay-out the GUI components). A sample application will be described in order to show how the presented model can be used.
    02/2007;
  • Christoph Schneckenburger, Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: One possibility to make testing strategies more effective is to incorporate knowledge about the typical geometric structure of failure-causing inputs within the input domain into the test data selection. For example Adaptive Random Testing is a testing strategy which is based on the idea of failure-causing inputs being clustered within the input domain. So far, there has been no empirical quantification about the location and the geometric shape of failure-causing inputs. Thus it is currently unknown whether the encouraging results gained by Adaptive Random Testing hold true in general. This work aims at introducing an approach which makes it possible to verify the assumption of clustered failure patterns. Possibly it furthermore enables the improvement of current Adaptive Random Testing methods and the development of further black box testing strategies incorporating knowledge about location and shape of failure patterns into test data selection. Therefore metrics for location and shape of failure patterns are specified. They are based on methods from image analysis.
    Fourth International Workshop on Software Quality Assurance, SOQUA 2007, in conjunction with the 6th ESEC/FSE joint meeting, Dubrovnik, Croatia, September 3-4, 2007; 01/2007
  • [Show abstract] [Hide abstract]
    ABSTRACT: Software testing research has mostly focused on deterministic software systems so far. In reality, however, randomized software systems (i.e. software systems with random output) also play an important role, e. g. for simulation purposes. Test evaluation is a real problem in that case. In previous work, statistical hypothesis tests have already been used, but test decisions have not been interpreted. Furthermore, those tests have only been applied if theoretic values on the distribution of program outputs had been available and not in case of golden implementations. In the present paper, we propose a general approach on how to apply statistical hypothesis tests in order to test randomized software systems. We exactly determine the confidence gained through these tests. We show that after passing a statistical hypothesis test, it can be guaranteed that at least the tested characteristics of the system under test are correct with a certain probability and accuracy of the result. Our approach is also applicable in case of golden implementations. Therefore, knowledge about the outputs' distribution is not necessary in that situation, which is a great advantage. Two case studies are described that have been conducted in order to assess the proposed approach. One of the case studies is based on a software system for the simulation of stochastic geometric models (among others) that evolved from the GeoStoch research project and is now used at France Télécom R&D, Paris, in order to calculate costs for communication networks and to plan new network structures.
    Fourth International Workshop on Software Quality Assurance, SOQUA 2007, in conjunction with the 6th ESEC/FSE joint meeting, Dubrovnik, Croatia, September 3-4, 2007; 01/2007
  • Ralph Guderlei, Johannes Mayer
    International Journal of Software Engineering and Knowledge Engineering. 01/2007; 17:757-781.
  • Johannes Mayer, Ralph Guderlei
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing image processing applications is a non-trivial task. Complex inputs have to be generated and complex test results have to be evaluated. In the present paper, models for random generation of images are proposed and compared. The study for their comparison uses mutants of one particular implementation of an image processing operator, namely an implementation of the Euclidean distance transform. Metamorphic relations, necessary properties, and special values are furthermore identified for this distance transform to enable automatic evaluation of test results. These criteria are also compared using mutation analysis. Based on the results, general hints are given on how to choose random models and automatically evaluate test results for testing in the field of image processing
    Sixth International Conference on Quality Software (QSIC 2006), 26-28 October 2006, Beijing, China; 01/2006
  • Source
    Johannes Mayer, Christoph Schneckenburger
    [Show abstract] [Hide abstract]
    ABSTRACT: Testing with randomly generated test inputs, namely Random Testing, is a strategy that has been applied succefully in a lot of cases. Recently, some new adaptive approaches to the random generation of test cases have been proposed. Whereas there are many comparisons of Random Testing with Partition Testing, a systematic comparison of random testing techniques is still missing. This paper presents an empirical analysis and comparison of all random testing techniques from the field of Adaptive Random Testing (ART). The ART algorithms are compared for effectiveness using the mean F-measure, obtained through simulation and mutation analysis, and the P-measure. An interesting connection between the testing effectiveness measures F-measure and P-measure is described. The spatial distribution of test cases is determined to explain the behavior of the methods and identify possible shortcomings. Besides this, both the theoretical asymptotic runtime and the empirical runtime for each method are given.
    2006 International Symposium on Empirical Software Engineering (ISESE 2006), September 21-22, 2006, Rio de Janeiro, Brazil; 01/2006
  • Johannes Mayer, Christoph Schneckenburger
    [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive random testing (ART) subsumes a family of random testing techniques that are designed to be more effective than pure random testing. These methods spread test cases more evenly within the input domain than a uniform distribution does. In the present paper, it is investigated why standard ART methods are less effective for higher failure rates. Therefore, the spatial distribution of the test cases generated by these methods is analyzed - also in higher dimensions - with a new approach. Based on the results of the analysis, improved algorithms are proposed that are equally effective for all failure rates as an empirical study reveals
    Sixth International Conference on Quality Software (QSIC 2006), 26-28 October 2006, Beijing, China; 01/2006
  • Johannes Mayer, Ralph Guderlei
    [Show abstract] [Hide abstract]
    ABSTRACT: Full software test automation requires automated test input generation, execution, and output evaluation. The latter task is non-trivial and usually referred to as the oracle problem in software testing. The present paper describes an empirical study on metamorphic testing, an approach to the oracle problem. This study was conducted with common Java implementations of determinant computation in order to evaluate the usefulness of the metamorphic testing approach and to establish general criteria that can be used to quickly assess metamorphic relations with respect to their suitability. The latter is very important, since metamorphic testing is based on so-called metamorphic relations on input-output tuples, which can easily be found. It is, however, crucial to evaluate these relations according to their usefulness. The empirical study enables us to derive general rules that can be used to quickly assess metamorphic relations and identify those that should be considered and studied in more detail with other methods (e.g. with mutation analysis)
    30th Annual International Computer Software and Applications Conference (COMPSAC 2006), 17-21 September 2006, Chicago, Illinois, USA; 01/2006
  • Source
    Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive Random Testing subsumes a class of algorithms that detect the first failure with less test cases than Ran- dom Testing. The present paper shows that a "reference method" in the field of Adaptive Random Testing is not ef- fective for higher dimensional input domains and clustered failure-causing inputs. The reason for this behavior is ex- plained, and a modified method is proposed and analyzed.
    Genetic and Evolutionary Computation Conference, GECCO 2006, Proceedings, Seattle, Washington, USA, July 8-12, 2006; 01/2006
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Nowadays, plugin-based applications are quite common. Eclipse is probably the most popular example of such an application. By means of plugins, end-users can add or remove functionality even at runtime. Besides the kernel, plugin-based applications can be kept very small and nearly everything can be designed as a plugin. However, if plugins are added at runtime, their ordering is difficult to organize. This can be observed for graphical user interface representations of plugins, such as menu or list items for example. In particular, the kernel may not refer to a single concrete plugin, since it has to be independent of concrete plugins - according to the plugin concept. Therefore, self-organization is proposed in the present paper as a solution to structure plugin-based applications. A pattern for linearly ordered plugins is presented. The end-user still retains the possibility to reorder the plugins manually according to his preferences. A sample application of the presented pattern in the context of graphical user interfaces is described
    2006 International Conference on Autonomic and Autonomous Systems (ICAS 2006), 16-21 July 2006, Silicon Valley, California, USA; 01/2006
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Plugin-based applications can be dynamically ex- tended at runtime. This leads to highly extensible systems on the one hand, but structuring the representation of plugins is difficult on the other hand. Often plugins are represented in graphical user interfaces (e.g. in menu structures) which are generated at runtime. But the kernel does not have—and may not have— any knowledge of the participating plugins. Therefore, the kernel can not care about the plugins' order. This may lead to menu structures with varying order each time the application is run. Obviously, this results in less intuitive and ergonomical graphical user interfaces. Self-organizing mechanisms producing a linear structure of plugins have been successfully applied to plugin- based applications in previous work. But with an increasing number of plugins involved, linearly ordered structures will soon become crowded. Therefore, in the present paper the self- organizing mechanisms are extended to enable hierarchically ordered plugins and presented in a pattern. The hierarchical order can be embedded in GUI tree structures, which lead to ergonomic and intuitive self-organizing menu structures. A sample application of the presented pattern in the context of Internet portals is described.
    Proceedings of the International Conference on Software Engineering Research and Practice & Conference on Programming Languages and Compilers, SERP 2006, Las Vegas, Nevada, USA, June 26-29, 2006, Volume 2; 01/2006
  • Source
    Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive Random Testing (ART) is designed to detect the first failure with fewer test cases than pure Random Testing. Since well-known ART methods, namely Distance-Based ART (D-ART) and Restriction-Based ART (RRT), have quadratic runtime, ART methods based on the idea of partitioning have been presented. ART by Random Partitioning is one of these partition-based ART algorithms. While having only a little bit more than linear asymptotic runtime, the number of test cases necessary to detect the first failure is substantially higher than that of D-ART and RRT. In the the present paper, an improved version of ART by Random Partitioning is presented employing the notion of restriction. The presented algorithm has the same very good runtime as ART by Random Partitioning while requiring significantly fewer test cases to exhibit the first failure.
    Proceedings of the International Conference on Software Engineering Research and Practice & Conference on Programming Languages and Compilers, SERP 2006, Las Vegas, Nevada, USA, June 26-29, 2006, Volume 1; 01/2006
  • Source
    Johannes Mayer, Christoph Schneckenburger
    [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive Random Testing (ART) denotes a family of random testing methods that are designed to be more effective than Random Testing (RT). Mostly, these methods have been in- vestigated using the mean F-measure, which denotes the random number of test cases necessary to detect the first failure. The two most important ART methods, namely Distance-Based ART (D- ART) and Restricted Random Testing (RRT), perform worse for higher failure rates than for lower failure rates. Furthermore, all previous publications on ART analyzed these methods for testing with unlimited resources. The present paper investigates, why D- ART and RRT behave better for lower failure rates. Therefore, the F-measure distribution and the spatial distribution of single test cases are analyzed. Thereby, shortcomings of D-ART and RRT are revealed. Improved ART methods are presented based on our findings. Furthermore, the usefulness of the F-measure distribution of testing with unlimited resources for resource- constrained testing is explained. Finally, the ART methods are compared to RT for both cases, i.e. with and without resource limitations.
    Proceedings of the International Conference on Software Engineering Research and Practice & Conference on Programming Languages and Compilers, SERP 2006, Las Vegas, Nevada, USA, June 26-29, 2006, Volume 1; 01/2006
  • Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: Random Testing is a strategy to select test cases based on pure randomness. Adaptive Random Testing (ART), a family of algorithms, improves pure Random Testing by taking common failure pattern into account. The best—in terms of the number of test cases necessary to detect the first failure—ART algorithms, however, are too runtime inefficient. Therefore, a modification of a fast, but not so good ART algorithm, namely ART by Bisection, is presented. This modification requires much less test cases than the original method while retaining its computational efficiency.
    Formal Methods and Software Engineering, 7th International Conference on Formal Engineering Methods, ICFEM 2005, Manchester, UK, November 1-4, 2005, Proceedings; 01/2005
  • Johannes Mayer
    [Show abstract] [Hide abstract]
    ABSTRACT: Adaptive Random Testing (ART) denotes a family of test case generation algorithms that are designed to detect common failure patterns better than pure Random Testing. The best known ART algorithms, however, use many distance computations. Therefore, these algorithms are quite inefficient regarding runtime. New algorithms combining Adaptive Random Testing by Bisection and the principle of localization are presented. These algorithms heavily reduce the amount of distance computation while exhibiting very good performance measured in terms of the number of test cases necessary to detect the first failure.
    Formal Approaches to Software Testing, 5th International Workshop, FATES 2005, Edinburgh, UK, July 11, 2005, Revised Selected Papers; 01/2005
  • Source
    Johannes Mayer, Ralph Guderlei
    [Show abstract] [Hide abstract]
    ABSTRACT: The oracle problem is addressed for random testing and testing of random- ized software. The presented Statistical Oracle is a Heuristic Oracle using statistical methods, especially statistical tests. The Statistical Oracle is applicable in case there are explicit formulae for the mean, the distribution, and so on, of characteristics com- putable from the test result. However, the present paper only deals with the mean. As with the Heuristic Oracle, the decision of the Statistical Oracle is not always correct. An example from image analysis is shown, where the Statistical Oracle has success- fully been applied.
    Testing of Component-Based Systems and Software Quality, Proceedings of SOQUA 2004 (First International Workshop on Software Quality) and TECOS 2004 (Workshop Testing Component-Based Systems); 01/2004
  • 01/2004;