To read the full-text of this research, you can request a copy directly from the authors.
Abstract
Software plays an increasingly important role in equipment and systems, both in terms of technical relevance and of development cost (often higher than 50% even for small systems). Unlike HW, SW does not go through a production phase. Also, SW cannot break or wear out. However, it can fail to satisfy its required function because of defects which manifest themselves while the system is operating (dynamic defects). A fault in the SW is thus caused by a defect, even if appears randomly in time, and SW problems are basically quality problems which have to be solved with quality assurance tools (configuration management, testing, and quality data reporting systems). In order to grant a software high quality level against a reasonable cost, the testing planning phase has to be study in detail. In order to do so, an adequate coverage of the product functionality has to be supplied to reduce the test time. The aim of the paper is suggesting an automated software testing as a solution to the problem of having to maximize the test plan coverage within the available time and to increase software reliability and quality in use.
To read the full-text of this research, you can request a copy directly from the authors.
... Testaussuunnittelun tarkasteleminen osissa on Bacioccola & al. (2008) mukaan edellytys ohjelmiston korkean laatutason saavuttamiselle kohtuullisilla kustannuksilla. Tämä johtaa sopivan kattavuuden hankkimiseen tuotteen toiminnallisuudelle testausajan vähentämiseksi. ...
... Kuva 3.12 Ohjelmiston kehitys-testausprosessi (Bacioccola & al., 2008). ...
... Satunnaisissa tekniikoissa testidata tuotetaan satunnaisesti, staattisissa tekniikoissa ohjelmaa ei suoriteta ja dynaamisissa tekniikoissa ohjelma suoritetaan. Bacioccola & al. (2008) esittämä automatisoitu ohjelmistotestaus voidaan luokitella dynaamiseksi tekniikaksi: automatisoinnin avulla voidaan suorittaa sekä suorituskykytestausta, joka on verrattavissa käyttäjien toimintaan että nopea ja helppo regressiotestaus. Ennen kuin kehitetään automaattinen testisarja on määriteltävä testitapaukset, järjestelmän rakenne, alku-ja loppuehdot, systeemiparametrit, toiminnan ehdot sekä syötteet ja tulosteet. ...
... The originality of the present work mainly consists in the introduction of a validation method able to fill the gap, by testing the SUT against its expected operational profile. A quantitative information (confidence level) about the reliability of the embedded software [20] may be achieved by a statistical exploration of the whole state space. ...
The number of measurement-based embedded systems increases continuously in automotive, avionic, defense, and communication industries. In order to ensure the system’s compliance with requirements, adequate solutions for functional testing are needed. The paper describes an original approach for the functional validation of measurement-based complex control systems. The main focus is on the automatic generation of realistic continuous data sets (test stimuli) within the operational ranges of the measurement system under test, by applying enhanced statistical techniques, which are able to reproduce both timing and data dependencies. A thorough case study using an air fuel ratio control system from the automotive domain is performed to assess the approach. Results indicate that the proposed method is suitable for automated functional testing of embedded control systems within a Software-in-the-Loop test environment.
... Figure 3). [1,8,9,10,15,20,33,47,54,58,67,79,100,112,115,118,132,143] [1, 5,7,8,9,10,11,13,14,16,19,21,22,30,37,38,39,40,41,42,43,44,45,46,47,48,49,51,53,55,59,62,64,74,76,77,79,81,85,90,91,98,99,102,105,108,111,112,113,114,115,119,120,122,125,129,134,142,143,144] Planning phase First of all, no articles were found that can be classified into the planning phase. ...
Context: Quality assurance effort, especially testing effort, is often a major cost factor during software development, which sometimes consumes more than 50 % of the overall development effort. Consequently, one major goal is often to reduce testing effort. Objective: The main goal of the systematic mapping study is the identification of existing approaches that are able to reduce testing effort. Therefore, an overview should be presented both for researchers and practitioners in order to identify, on the one hand, future research directions and, on the other hand, potential for improvements in practical environments. Method: Two researchers performed a systematic mapping study, focusing on four databases with an initial result set of 4020 articles. Results: In total, we selected and categorized 144 articles. Five different areas were identified that exploit different ways to reduce testing effort: approaches that predict defect-prone parts or defect content, automation, test input reduction approaches, quality assurance techniques applied before testing, and test strategy approaches. Conclusion: The results reflect an increased interest in this topic in recent years. A lot of different approaches have been developed, refined, and evaluated in different environments. The highest attention was found with respect to automation and prediction approaches. In addition, some input reduction approaches were found. However, in terms of combining early quality assurance activities with testing to reduce test effort, only a small number of approaches were found. Due to the continuous challenge of reducing test effort, future research in this area is expected.
... Regression test is the classic software automatic testing application and it is the main part of the acceptance test. This kind of software testing is suitable for automatic tests with continuous repetitions of the same operations or test sequences, carried out in order to verify that the new functionalities implemented doesn't penalize the old ones [1]. The continuous repetitions of an operations set permit to evaluate also the memory occupation by the software modules; so monitoring the occupation of the total memory of the computer and that one engaged by the single process, it is possible to underline the presence of memory leaks by one or more software modules. ...
In order to guarantee a software with high quality level vs a reasonable cost, the testing planning phase has to be study in detail, also to reduce the test time [1]. This paper introduces the result of the application of software automatic test to one of the most common software failure causes: the overflow due to memory leaks. The analysis carried out on a single software failure cause wants to show the total generality of the technique and to define some parameters of life of the software. The general value of the method is shown applying the technique to different software and for everyone of them defining the mean time to failure, where with failure we mean only the failure due to overflow; even if the overflow it isn't the only analyzable failure cause through the application of such software automatic test.
... The stress produced by a test simulates the real operating conditions found in the field. At the same time, during a memory occupation test, a regression test can be performed, be+ing it based on continuous repetition of the same operations or test sequences [24]. Since it identifies, in a relatively short time, failures in the field before the product is realised and then to correct them through software redesign, the implementation of this kind of automated tests allows to reduce costs and working time. ...
Software plays an increasingly important role in complex systems, especially for high-tech applications involved in important fields, such as transportation, financial management, communication, biomedical applications and so on. For these systems, performances such as efficient operation, fault tolerance, safety and security have to be guaranteed by the software structure, whose quality in use is assuming a growing importance from the industrial point of view. The basic problem is that the complexity of the task which software has to perform has often grown more quickly than hardware. In addition, unlike hardware, software cannot break or wear out, but can fail during its life cycle (dynamic defects) [1]. Software problems, essentially, have to be solved with quality assurance tools such as configuration management, testing procedures, quality data reporting systems and so on [2]. In this context, the paper proposes a new approach concerning the automated software testing as an aid to maximize the test plan coverage within the time available and also to increase software reliability and quality in use [3]. In this paper a method which combines accelerated automated tests for the study of software regression and memory overflow will be shown, in order to guarantee software with both a high quality level and a decrease of the testing time. The software will be tested by using test sequences reproducing the actual operating conditions and accelerated stress level. Moreover the research wishes to define some parameters of the software life and to show the generality of the proposed technique.
... It focuses on testing measurement software using new statistical tools for improving the code-and-debugging cycle. And an automated software testing has been realized in [16] for granting a software high quality level. These works focus on the design and testing of measurement software and do not offer a specific development approach to remarkably simplify the software design, particularly for simplifying the coding work. ...
Distributed measurement systems (DMSs) are widely adopted to assist the design and testing of household appliances. Unfortunately, a reference model rarely rules the development of those systems, leading to poor software quality and bringing many troubles in the process of maintenance and integration. Furthermore, the development of measurement systems often suffers from low efficiency due to the complex programming and debugging process. To address the aforementioned issues, we first present a universal reference model serving as a foundation in the design of DMSs, particularly for the household appliance testing industry. The model provides the design criteria and methodologies of DMSs in terms of data collection and management. Then, according to the model, an interactive platform is built to facilitate the software development of DMSs for both local and web-based applications. The platform consists of the reusable component libraries, the client-based assembly subsystem, the web-based assembly subsystem, the data collection subsystem, and the customization subsystem. The software development process for measurement systems is turned into configuring parameters through interactive dialog boxes. The development efficiency and performance of the system generated by the platform are evaluated. Moreover, the model and platform are validated by various development cases for manufacturing enterprises. The results demonstrate that the model is feasible and that the platform remarkably improves the development efficiency and software quality.
Test systems are widely adopted to assist the design and testing of domestic appliances. A test system mainly includes computers, data acquisition instruments and test software. Development of test software often suffers from low efficiency due to the complex design, programming and debugging process. In this paper, we design a platform to facilitate the development of test software based on Client/Server and Browser/Server hybrid architecture. The development process of the test software is turned into configuring parameters through the graphic user interface (GUI). The platform presented consists of the reusable component libraries, client-based assembly subsystem, Web-based assembly subsystem, data collection subsystem and customization subsystem. Our platform is validated by various cases of developing test software for manufacturing enterprises. The results demonstrate that our platform remarkably improves the development efficiency and quality of test software.
Memory leak has been exposed to be one of the most serious bugs which are hard to locate and fix in computer program. This paper describes three methods of detecting memory leaks in Solaris when programmers do applications. Using dbx can get specific lines of source codes where memory leak occurs and memory leak sizes. MDB and DTrace can inform the users allocation and reclamation of memory blocks, and fix the memory leak points. The experiments show these three tools can enable us to find memory leaks quickly and correctly. It provides new ways to detect memory leak and avoid this serious bug in program.
ContextA lot of different quality assurance techniques exist to ensure high quality products. However, most often they are applied in isolation. A systematic combination of different static and dynamic quality assurance techniques promises to exploit synergy effects, such as higher defect detection rates or reduced quality assurance costs. However, a systematic overview of such combinations and reported evidence about achieving synergy effects with such kinds of combinations is missing.Objective
The main goal of this article is the classification and thematic analysis of existing approaches that combine different static and dynamic quality assurance technique, including reported effects, characteristics, and constraints. The result is an overview of existing approaches and a suitable basis for identifying future research directions.MethodA systematic mapping study was performed by two researchers, focusing on four databases with an initial result set of 2498 articles, covering articles published between 1985 and 2010.ResultsIn total, 51 articles were selected and classified according to multiple criteria. The two main dimensions of a combination are integration (i.e., the output of one quality assurance technique is used for the second one) and compilation (i.e., different quality assurance techniques are applied to ensure a common goal, but in isolation). The combination of static and dynamic analyses is one of the most common approaches and usually conducted in an integrated manner. With respect to the combination of inspection and testing techniques, this is done more often in a compiled way than in an integrated way.Conclusion
The results show an increased interest in this topic in recent years, especially with respect to the integration of static and dynamic analyses. Inspection and testing techniques are currently mostly performed in an isolated manner. The integration of inspection and testing techniques is a promising research direction for the exploitation of additional synergy effects.
The use of techniques for automating the generation of software test cases is very important as it can reduce the time and cost of this process. The latest methods for automatic generation of tests use metaheuristic search techniques, i.e. genetic algorithms and simulated annealing. There is a great deal of research into the use of genetic algorithms to obtain a specific coverage in software testing but there is none using the metaheuristic Tabu search technique. In this paper, we explain how we have created an efficient testing technique that combines Tabu search with Korel chaining approach. Our technique automatically generates test data in order to obtain branch coverage in software testing.
Problems associated with understanding, verifying and re-engineering the way in which a system allocates and releases dynamic memory present significant challenges to the software maintainer. Because the questions underlying these problems are undecidable, no system can provide a completely fail safe certification. For example, in checking for memory leaks, a system can only warn of potential problems, but cannot guarantee that no leaks remain. We present an approach to modelling the dynamic memory access properties of a program using amorphous program slicing to create a Dynamic Memory Model (DMM). The slices are constructed from a transformed version of the original program in which heap access has been made explicit using a pseudo variable to denote the top of the heap. The DMM is a simplified version of the original program which is concerned solely with the dynamic memory access behaviour of the original. We illustrate the use of DMMs in problems of comprehension, verification and re-engineering. We introduce a proof-of-concept DMM construction algorithm, showing how slicing simplification power can be dramatically improved using domain-specific transformation rules
Memory leaks and memory corruption are two major forms of software bugs that severely threaten system availability and security. According to the US-CERT vulnerability notes database, 68% of all reported vulnerabilities in 2003 were caused by memory leaks or memory corruption. Dynamic monitoring tools, such as the state-of-the-art Purify, are commonly used to detect memory leaks and memory corruption. However, most of these tools suffer from high overhead, with up to a 20 times slowdown, making them infeasible to be used for production-runs. This paper proposes a tool called SafeMem to detect memory leaks and memory corruption on-the-fly during production-runs. This tool does not rely on any new hardware support. Instead, it makes a novel use of existing ECC memory technology and exploits intelligent dynamic memory usage behavior analysis to detect memory leaks and corruption. We have evaluated SafeMem with seven real-world applications that contain memory leak or memory corruption bugs. SafeMem detects all tested bugs with low overhead (only 1.6%-14.4%), 2-3 orders of magnitudes smaller than Purify. Our results also show that ECC-protection is effective in pruning false positives for memory leak detection, and in reducing the amount of memory waste (by a factor of 64-74) used for memory monitoring in memory corruption detection compared to page-protection.
Standard for software test documentation
Jan 1998
Ansi Ieee
St
ANSI IEEE St. 829,"Standard for software test documentation", 1998.
Software Testing Practice in industry
Jan 2006
N Juristo
A M Moreno
"Software Testing Practice in industry" N. Juristo, A. M. Moreno
(Universidad Poletecnica de Madrid), QA Labs CNF IEEE 2006
IEEE Standard for Software Verification and Validation Plans
Ansi Ieee
Std
ANSI IEEE Std. 1012, "IEEE Standard for Software Verification and Validation Plans", 1986.
Information Technology-Software Product Evaluation-Quality Characteristics and Guidelines for their Use
Jan 2001
Iso Iec
ISO/IEC 9126: Information Technology-Software Product Evaluation-Quality Characteristics and Guidelines for their Use, 2001.
Introduction to Software Reliability Engineering and Testing
Jan 1997
"Introduction to Software Reliability Engineering and Testing" J. D.
Musa CNF IEEE 1997