Conference Paper

Verifying Code Generation Tools for the B-Method Using Tests: A Case Study

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In this paper, we present a case study where two code generators for the B-Method were validated using software testing techniques. Our testing strategy is a combination of Grammar-Based Testing (GBT) and Model-Based Testing (MBT) techniques. The strategy consists of two steps. In the first step, grammar-based coverage criteria are used to generate a wide and meaningful set of test input models to validate the parsing capabilities of the code generators. In the second step, a MBT tool is used to validate the correctness of the output produced by these tools. The MBT tool generates a set of tests based on the same input model used by the code generation tools. The generated code is considered correct (consistent with the input model) if it passes this set of tests. Using this testing strategy, we were able to find problems in both code generation tools with moderate effort.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... They are used as a lower-cost complement of formal proofs, for an in-depth verification of the system. Software testing can quickly identify failures and expose defects introduced during the code generation and code maintenance [16]. Also, some certificates such as DO-178B, required by the Federal Aviation Administration (FAA), demand the use of software testing and measuring coverage over the code. ...
... BTestBox has undergone a great deal of improvement since the previous version, used in [7,16]. The main differences between the previous and new versions are: (i) inclusion of several code coverage criteria, the previous version supported only Statement Coverage; (ii) new HTML interface, including reports generated for each coverage criterion; and (iii) better support to new case studies both academic and industrial. ...
... A variety of lexical elements of the B language were created with nested instructions. They were designed to test the scalability of a B translator to LLVM [16]. Those files may be classified by the number of nesting in the body of the operations. ...
Chapter
Full-text available
The argument of correctness in refinement-based formal software design often disregards source code analysis and code generation. To mitigate the risk of errors in these phases, certifications issued by regulation entities demand or recommend testing the generated software using a code coverage criteria. We propose improvements for the BTestBox, a tool for automatic generation of tests for software components developed with the B method. BTestBox supports several code coverage criteria and code generators for different languages. The tool uses a constraint solver to produce tests, thus being able to identify dead code and tautological branching conditions. It also generates reports with different metrics and may be used as an extension to the Atelier B. Our tool performs a double task: first, it acts on the B model, by checking the code coverage. Second, the tool performs the translation of lower level B specifications into programming language code, runs tests and compares their results with the expected output of the test cases. The present version of BTestBox uses parallelisation techniques that significantly improve its performance. The results presented here are encouraging, showing performance numbers that are one order of magnitude better than the ones obtained in the tool’s previous version.
... Os testes gerados pelo BETA são eficazes e com baixo esforço de aplicação. Tais testes vêm oferecendo uma confiabilidade complementar, de acordo com os resultados empíricos obtidos na pesquisa e publicados em Moreira et al. (2015). Esses testes são baseados na especificação do modelo B e avaliam a coerência entre a especificação e o código LLVM-IR. ...
... Todos esses critérios são bem estabelecidas na comunidade de teste de software e definidos de acordo com os trabalhos de Ammann e Offutt (2008). Na tabela 6, o grupo de colunas (a) contém a numeração de identificação, o nome do modelo de teste B, o número 1 Counter 51 4 10 10 10 8 8 8 6 6 6 2 Swap 18 3 3 3 3 6 6 6 1 1 1 3 Calculator 48 6 10 10 10 26 26 26 6 6 6 4 Wd 27 3 5 5 5 5 5 5 4 4 4 5 Prime 10 1 4 4 4 5 5 5 3 3 3 6 Division 12 1 2 2 2 3 3 3 1 1 1 7 Team 36 2 3 3 3 7 7 7 Tabela 6 -Resultados dos testes aplicados com BETA em Moreira et al. (2015). ...
... Ou seja, todos os testes gerados foram aprovados e nenhum erro foi identificado no código gerado por B2LLVM e C4B para esses modelos. No período da publicação em Moreira et al. (2015), os módulos de 10 até 13 não eram traduzidos por B2LLVM. Atualmente, todos os módulos são traduzidos por B2LLVM. ...
Thesis
Full-text available
Assembly code synthesis is a process that requires rigorous care. Typically, this process in mature translators and compilers is relatively safe, though some errors have occasionally been identified. In a more restricted context, the translators used in the smaller communities and developed constantly have been more susceptible to these kind of errors. Considering this context, two translation and verification approaches using the B method have been presented. The first approach considers the B component translation to assembly code for a specific platform by using a formal model of the instruction set, which also allows to debug and to check program properties. This approach can formally ensure the semantic consistency of the translation, although the verification process is difficult and time-consuming. This verification effort was somewhat relieved by the development of BEval, a tool which has made better use of the verification techniques. After applying the automatic verification process using the Atelier B tool, BEval was still able to solve, in the certain models, many of the remaining proof obligations, reaching up to 88 \% of total obligations. However, the verification process of this translation approach is still complex, requiring several manual interactions. In order to make a translation more efficient and also safe, a second approach for the translating B to virtual machine code was developed. This second approach used the developed translator B2LLVM and applied the automatic generation of tests to check the consistency of the program specifications and their respective assembly code. This translator also supported the code coverage evaluation and the insertion of the traceability annotations. Thus, this study has significantly contributed to developing B translation to assembly code, providing rigorous support for the verification of the translation. Currently, the B2LLVM outweighs 20,9% the percentage of translatable test samples based on grammar compared to the C4B industrial translator. Furthermore, the generated code by B2LLVM has important advantages in the ample capacity of verification, integration and optimization.
... – Generation of test scripts: the work of [Souza Neto, 2015] contributed with a generator of test scripts for the tool. The scripts are only partially implemented but are a considered big improvement when compared with the test case specifications that were generated by the initial versions of the tool; ...
... It required a lot of effort, especially when the test specification defined too many, too complex test cases. With that in mind, we developed a generator of executable test scripts that automates part of the process of translation to concrete test cases [Souza Neto and Moreira, 2014] [Souza Neto, 2015]. These executable test scripts still have to be adapted and complemented with some information before they can be executed. ...
... The tests were generated using a model specified by [Moreira and Ierusalimschy, 2013]. The case study presented here was performed in [Souza Neto, 2015] and provided valuable feedback to improve BETA. Lua [Ierusalimschy et al., 1996] is a scripting language that was designed to be powerful, fast, lightweight, and embeddable. ...
Presentation
Full-text available
Thesis Proposal presented as a requirement to obtain the Ph.D. title at Federal University of Rio Grande do Norte. This work was supervised by Prof. Anamaria M. Moreira and Prof. Michael Leuschel.
... Some currently used correct-by-design model-based development approaches for reactive systems, notably the B-method and Event-B-method (Abrial, 1996;Moreira, 2015) too prescribe conducting formal verification and validation (V&V) throughout the design process: a progressive refinement process with a theorem proving system for proof obligations at each refinement stage. The premise is that detailed models cannot be built at the beginning of a development process. ...
... Formal validation of model-based development at instrumentation level is still poorly supported (Bringmann & Kramer, 2008). De Matos (2015) and Nielsen (2014) help comparing a strict formal modelbased development and a combination of formal and traditional unit-test based methods. (Malik & Roop, 2015) discuss code generation from method-B. ...
... In [38], BETA was used to contribute to the validation of two code generation tools for the B-method, b2llvm [39] and C4B 12 . Both code generators receive as input a B implementation, written in the B0 notation and produce code (LLVM code with a C interface, for b2llvm, and C, for C4B) as output. ...
... Both code generators receive as input a B implementation, written in the B0 notation and produce code (LLVM code with a C interface, for b2llvm, and C, for C4B) as output. The tests generated by BETA were used in [38] as oracles to verify the compliance of the code generated by b2llvm and C4B with the original B abstract machine. A set of 13 B modules (ATM, Sort, Calculator, Calendar, Counter, Division, Fifo, Prime, Swap, Team, TicTacToe, Timetracer and Wd) was used in this evaluation. ...
Article
Full-text available
BackgroundBETA (Bbased testing approach) is a toolsupported approach to generate test cases from Bmethod specifications through the application of input space partitioning and logical coverage criteria. The BETA tool automates the whole process, from the design of abstract test cases to the generation of executable test scripts. Methods In this paper, we present an empirical study that was performed to evaluate and contribute to the development of BETA. The study evaluated the applicability of BETA on problems with different characteristics and used techniques of code coverage and mutation analysis to measure the quality of the generated test cases. The study was carried out in different rounds, and the results of each round were used as a reference for the improvement of the approach and its supporting tool. ResultsThese case studies were relevant not only for the evaluation of BETA but also to evaluate how different features affect the usability of the approach and the quality of the test cases and to compare the quality of the test cases generated using different coverage criteria. Conclusions The results of this study showed that (1) BETAgenerated test scenarios for the different criteria follow theoretical expectations in terms of criteria subsumption; (2) the BETA implementation of the logical criteria generates more efficient test sets regarding code and mutation coverage than the input space partitioning ones; (3) it is important to go beyond the strict requirements of the criteria by adding some additional variation (randomization) of the input data; and (4) the algorithms designed to combine test requirements into test cases need to deal carefully with infeasible (logically unsatisfiable) combinations.
... Os trabalhos iniciados nesta pesquisa baseiam-se no trabalho desenvolvido anteriormente Moreira et al (2015), cujo orientador é autor da citada publicação, que apresenta diferentes abordagens de testes para geradores de código. Apesar de a abordagem do presente trabalho ser ...
... Embora essas estratégias tenham sido exploradas neste trabalho, ainda não foram aplicadas em larga escala, pois o projeto encontra-se incipiente. Convém ressaltar, contudo que o trabalho desenvolvido por Moreira et al (2015) que abrange o mesmo ambiente e possui estratégias similares mostrou resultados significativos e reconhecidos internacionalmente. ...
... E.g., for deadlock checking, we ask the constraint solver to find a state of a B model satisfying the invariant, such that no event or operation is enabled. -Model-based testing [34,16,31]. ...
Conference Paper
Constraint solving technology for formal models has made considerable progress in the last years, and has lead to many applications such as animation of high-level specifications, test case generation, or symbolic model checking. In this article we discuss the idea to use formal models themselves to express constraint satisfaction problems and to embed formal models as executable artefacts at runtime. As part of our work, we have developed a document generation feature, whose output is derived from such executable models. This present article has been generated using this feature, and we use the feature to showcase the suitability of formal modelling to express and solve various constraint solving benchmark examples. We conclude with current limitations and open challenges of formal model-based constraint solving.
... The tests generated by BETA were used as oracles to verify the compliance of the code generated by b2llvm and C4B with the original B abstract machine. This case study was presented in [10], mainly focusing on the validation of b2llvm and C4B. ...
Conference Paper
Full-text available
Software Testing and Formal Methods are two techniques that focus on software quality and that can complement each other. Many researchers in both fields have tried to combine them in different ways. BETA is one of the efforts in this line of work. It is a tool-supported approach to generate test cases based on B-Method specifications. In this paper, we present an empirical study that was made to evaluate BETA. We performed two case studies with different objectives and that used different techniques to measure, not only the quantity, but the quality of the test cases generated by the approach. With the results of these case studies, we were able to identify some flaws and propose improvements to enhance both the approach and the tool.
Article
Full-text available
A domain specific language (DSL) focuses on the essential concepts in a specific problem domain, and abstracts from low-level implementation details. The development of DSLs usually centers around the meta-model, grammar and code generator, possibly extended with transformations to analysis models. Typically, little attention is given to the formal semantics of the language, whereas this is essential for reasoning about DSL models, and for assessing the correctness of the generated code and analysis models. We argue that the semantics of a DSL should be defined explicitly and independently of any code generator, to avoid all kinds of complexities from low-level implementation details. As the generated analysis models must reflect some of these implementation details, we propose to formalize them separately. To assess the correctness and consistency of the generated code and analysis models in a practical way, we use conformance testing. We extensively illustrate this general approach using specific formalizations for an industrial DSL on collision prevention. We do not aim for a generic semantic model for any DSL, but this specific DSL indicates the potential of a modular semantics to facilitate reuse among DSLs.
Conference Paper
Full-text available
Automatic support to the generation and execution of tests is an important research topic, given the need to have quality testing of products within reasonable costs. The use of systematic testing techniques and adequate coverage criteria are means through which this support and quality may be obtained. The LGen (Lua Language Generator) tool aims to contribute to this scenario by generating sentences from the language of a given grammar in such a way that these sentences satisfy some given coverage criteria. In the current paper, we study the applicability of LGen to the domain of software product lines. More specifically, we propose to use LGen to generate testing configurations (specific products) for a family of products in a product line specified by a feature model. We applied the tool to some publicly available feature models of the SPLOT repository and generated test configurations from grammars which formally describe those feature models. We have attained full coverage of the used classical coverage criteria in all examples with a small number of test configurations , a crucial condition for such tests to be practical.
Article
Full-text available
Compilers are used for creating executable modules for programs written in high-level languages; therefore, the presence of errors in a compiler is a serious danger for the quality of the software developed with the use of this compiler. As in the case of any other software, testing is one of the most important methods of quality control and error detection in compilers. The survey is devoted to methods for generating, running, and checking the quality of compiler test suites, which are based on formal specifications of the programming language syntax and semantics.
Conference Paper
Full-text available
Grammar testing is discussed in the context of grammar engineering (i.e., software engineering for grammars). We propose a gen- eralisation of the known rule coverage for grammars, that is, context-dependent branch coverage. We investigate grammar testing, especially coverage analysis, test set generation, and integration of testing and grammar transformations. Grammar recovery is chosen as a subfield of grammar engineering to illustrate the developed concepts. Grammar recovery is concerned with the derivation of a language’s grammar from some available resource such as a semi-formal language reference.
Conference Paper
Full-text available
Given a grammar (or other sorts of meta-data), one can trivially derive combinatorially exhaustive test-data sets up to a specified depth. Without further efforts, such test-data sets would be huge at the least and explosive most of the time. Fortunately, scenarios of grammar-based testing tend to admit non-explosive approximations of naive combinatorial coverage. In this paper, we describe the notion of controllable combinatorial coverage and a corresponding algorithm for test-data generation. The approach is based on a suite of control mechanisms to be used for the characterization of test-data sets as well-defined and understandable approximations of full combinatorial coverage. The approach has been implemented in the C#-based test-data generator Geno, which has been successfully used in projects that required differential testing, stress testing and conformance testing of grammar-driven functionality.
Article
Full-text available
There is a growing awareness, both in industry and academia, of the crucial role of formally verifying the translation from high-level source-code representation into an low-level object code that is typically performed by an optimizing compiler. Formally verifying an optimizing compiler, as one would verify any other large program, is not feasible due to its size, ongoing evolution and modification, and, possibly, proprietary considerations. Translation validation is a novel approach that offers an alternative to the verification of translators in general and compilers in particular: Rather than verifying the compiler itself, one constructs a validation tool which, after every run of the compiler, formally confirms that the target code produced in the run is a correct translation of the source program. The paper presents voc--a methodology for translation validation of optimizing compilers. We distinguish between structure preserving optimizations, for which we establish a simulation relation between the source and target code based on computational induction, and structure modifying optimizations, for which we develop specialized "permutation-rules". The paper also describes voc-64---a prototype translation validator tool that automatically produces verification conditions for the global optimizations of the SGI Pro-64 compiler.
Article
Full-text available
This paper reports on the development and formal verification (proof of semantic preservation) of CompCert, a compiler from Clight (a large subset of the C programming language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its correctness. Such a verified compiler is useful in the context of critical software and its formal verification: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well.
Conference Paper
Full-text available
We describe LLVM (low level virtual machine), a compiler framework designed to support transparent, lifelong program analysis and transformation for arbitrary programs, by providing high-level information to compiler transformations at compile-time, link-time, run-time, and in idle time between runs. LLVM defines a common, low-level code representation in static single assignment (SSA) form, with several novel features: a simple, language-independent type-system that exposes the primitives commonly used to implement high-level language features; an instruction for typed address arithmetic; and a simple mechanism that can be used to implement the exception handling features of high-level languages (and setjmp/longjmp in C) uniformly and efficiently. The LLVM compiler framework and code representation together provide a combination of key capabilities that are important for practical, lifelong analysis and transformation of programs. To our knowledge, no existing compilation approach provides all these capabilities. We describe the design of the LLVM representation and compiler framework, and evaluate the design in three ways: (a) the size and effectiveness of the representation, including the type information it provides; (b) compiler performance for several interprocedural problems; and (c) illustrative examples of the benefits LLVM provides for several challenging compiler problems.
Conference Paper
We describe a method and a corresponding tool for grammar-based test-data generation (GBTG). The basic generation principle is to enumerate test data based on grammatical choices. However, generation is broken down into two phases to deal with context-sensitive properties in an efficient and convenient manner. The first phase enumerates test data (i.e., parse trees) with placeholders. The second phase instantiates the placeholders through post-processors. A DSL for grammar transformation is used to customize a given grammar, meant for parsing, to be more suitable for test-data generation. Post-processors are derived from a corresponding object-oriented framework. The actual tool, Xtextgen, extends theXtext technology for language development.
Conference Paper
Testing model-to-model (M2M) transformations is becoming a prominent topic in the current Model-driven Engineering landscape. Current approaches for transformation testing, however, assume having explicit model representations for the input domain and for the output domain of the transformation. This excludes other important transformation kinds, such as model-to-text (M2T) and text-to-model (T2M) transformations, from being properly tested since adequate model representations are missing either for the input domain or for the output domain. The contribution of this paper to overcome this gap is extending Tracts, a M2M transformation testing approach, for M2T/T2M transformation testing. The main mechanism we employ for reusing Tracts is to represent text within a generic metamodel. By this, we transform the M2T/T2M transformation specification problems into equivalent M2M transformation specification problems. We demonstrate the applicability of the approach by two examples and present how the approach is implemented for the Eclipse Modeling Framework (EMF). Finally, we apply the approach to evaluate code generation capabilities of several existing UML tools.
Conference Paper
Model-based testing is a technique to design abstract tests from models that partially describe the system's behaviour. Abstract tests are transformed into concrete tests, which include test input values, expected outputs, and test oracles. Although test oracles require significant investment and are crucial to the success of the testing, we have few empirical results about how to write them. With the same test inputs, test oracles that check more of the program state have the potential to reveal more failures, but may also cost more to design and create. This research defines six new test oracle strategies that check different parts of the program state different numbers of times. The experiment compared the six test oracle strategies with two baseline test oracle strategies. The null test oracle strategy just checks whether the program crashes and the state invariant test oracle strategy checks the state invariants in the model. The paper presents five main findings. (1) Testers should check more of the program state than just runtime exceptions. (2) Test oracle strategies that check more program states do not always reveal more failures than strategies that check fewer states. (3) Test oracle strategies that check program states multiple times are slightly more effective than strategies that check the same states just once. (4) Edge-pair coverage did not detect more failures than edge coverage with the same test oracle strategy. (5) If state machine diagrams are used to generate tests, checking state invariants is a reasonably effective low cost approach. In summary, the state invariant test oracle strategy is recommended for testers who do not have enough time. Otherwise, testers should check state invariants, outputs, and parameter objects.
Article
Tribute Foreword Introduction Part I. Mathematics: 1. Mathematical reasoning 2. Set notation 3. Mathematical objects Part II. Abstract Machines: 4. Introduction to abstract machines 5. Formal definition of abstract machines 6. Theory of abstract machines 7. Constructing large abstract machines 8. Examples of abstract machines Part III. Programming: 9. Sequencing and loop 10. Programming examples Part IV. Refinement: 11. Refinement 12. Constructing large software systems 13. Examples of refinement Appendixes Index.
Article
Grammars are traditionally used to recognize or parse sentences in a language, but they can also be used to generate sentences. In grammar-based test generation (GBTG), context-free grammars are used to generate sentences that are interpreted as test cases. A generator reads a grammar G and generates L(G), the language accepted by the grammar. Often L(G) is so large that it is not practical to execute all of the generated cases. Therefore, GBTG tools support ‘tags’: extra-grammatical annotations which restrict the generation. Since its introduction in the early 1970s, GBTG has become well established: proven on industrial projects and widely published in academic venues. Despite the demonstrated effectiveness, the tool support is uneven; some tools target specific domains, e.g. compiler testing, while others are proprietary. The tools can be difficult to use and the precise meaning of the tags are sometimes unclear. As a result, while many testing practitioners and researchers are aware of GBTG, few have detailed knowledge or experience. We present YouGen, a new GBTG tool supporting many of the tags provided by previous tools. In addition, YouGen incorporates covering-array tags, which support a generalized form of pairwise testing. These tags add considerable power to GBTG tools and have been available only in limited form in previous GBTG tools. We provide semantics for the YouGen tags using parse trees and a new construct, generation trees. We illustrate YouGen with both simple examples and a number of industrial case studies. Copyright © 2010 John Wiley & Sons, Ltd.
Conference Paper
Model transformation (MT) testing is gaining interest as the size and complexity of MTs grows. In general it is very difficult and expensive (time and computational complexity-wise) to validate in full the correctness of a MT. This paper presents a MT testing approach based on the concept of Tract, which is a generalization of the concept of Model Transformation Contract. A Tract defines a set of constraints on the source and target metamodels, a set of source-target constraints, and a tract test suite, i.e., a collection of source models satisfying the source constraints. We automatically generate input test suite models, which are then transformed into output models by the transformation under test, and the results checked with the USE tool (UML-based Specification Environment) against the constraints defined for the transformation. We show the different kinds of tests that can be conducted over a MT using this automated process, and the kinds of problems it can help uncovering.
Article
A fast algorithm is given to produce a small set of short sentences from a context free grammar such that each production of the grammar is used at least once. The sentences are useful for testing parsing programs and for debugging grammars (finding errors in a grammar which causes it to specify some language other than the one intended). Some experimental results from using the sentences to test some automatically generated simpleLR(1) parsers are also given.
Book
The B method is a means for specifying, designing and coding software systems. The long-awaited B Book is the standard reference for everything concerning this method. It contains the mathematical basis on which it is founded, the precise definitions of the notations used, and a large number of examples illustrating its use in practice. J.-R. Abrial, the inventor of B, has written the book in such a way that it can be used for self-study or for reference. It is in four parts, the first dealing with the mathematical foundations, including a systematic construction of predicate logic and set theory, and the definition of the various mathematical structures that are needed to formalize software systems; the author places special emphasis on the notion of proof. The second part contains a presentation of the Generalized Substitution Language and of the Abstract Machine Notation, which are both used to specify software systems; the author gives examples to show how large specifications can be constructed systematically. The next part introduces the two basic programming features of sequencing and loop, with examples showing how to construct small algorithms. The last part covers the very important notion of refinement. It shows how to construct large software systems by means of layered architectures of modules. It culminates with the presentation of several examples of complete development with a special emphasis on the methodological approach. Finally, appendices give summaries of all the logical and mathematical definitions, and of all the rules and proof obligations. With the appearance of The B Book, formal methods practitioners, computer scientists, and systems developers at last will have access to the definitive account of what will become one of the standard approaches to the construction of software systems.
Book
Extensively class-tested, this textbook takes an innovative approach to software testing: it defines testing as the process of applying a few well-defined, general-purpose test criteria to a structure or model of the software. It incorporates the latest innovations in testing, including techniques to test modern types of software such as OO, web applications, and embedded software. The book contains numerous examples throughout. An instructor's solution manual, PowerPoint slides, sample syllabi, additional examples and updates, testing tools for students, and example software programs in Java are available on an extensive website.
Article
Typescript. Thesis (Ph.D.)--New York University, Graduate School of Arts and Science, 2006. Includes bibliographical references (leaves 109-111).
Article
Unlike for conventional compilers for imperative programming languages such as C or ADA, no establishedmethods for safeguarding artifacts generated by model-based code generators exist despite progress in the field of formalverification. Several test approaches dominate the engineering practice. This paper describes a general and toolindependenttest architecture for code generators used in model-based development. We evaluate the effectiveness of ourtest approach by means of testing optimizations performed by the TargetLink code generator, a widely accepted andcomplex development tool used in automotive model-based development.
Article
We describe a translation validation infrastructure for the GNU C compiler. During the compilation the infrastructure compares the intermediate form of the program before and after each compiler pass and verifies the preservation of semantics. We discuss a general framework that the optimizer can use to communicate to the validator what transformations were performed. Our implementation however does not rely on help from the optimizer and it is quite successful by using instead a few heuristics to detect the transformations that take place. The main message of this paper is that a practical translation validation infrastructure, able to check the correctness of many of the transformations performed by a realistic compiler, can be implemented with about the effort typically required to implement one compiler pass. We demonstrate this in the context of the GNU C compiler for a number of its optimizations while compiling realistic programs such as the compiler itself or the Linux kernel. We believe that the price of such an infrastructure is small considering the qualitative increase in the ability to isolate compilation errors during compiler testing and maintenance.
Article
. Grammar testing is discussed in the context of grammar engineering (i.e., software engineering for grammars). We propose a generalisation of the known rule coverage for grammars, that is, contextdependent branch coverage. We investigate grammar testing, especially coverage analysis, test set generation, and integration of testing and grammar transformations. Grammar recovery is chosen as a subfield of grammar engineering to illustrate the developed concepts. Grammar recovery is concerned with the derivation of a language's grammar from some available resource such as a semi-formal language reference. 1 Introduction Grammar engineering Grammars are omnipresent in software development. They are used for the definition of syntax, exchange formats and others. Several important kinds of software rely on grammars, e.g., compilers, debuggers, profilers, slicing tools, pretty printers, (re-) documentation tools, language reference manuals, browsers and IDEs, software analysis tool...
Proposal: Translation of B Implementations to LLVM-IR
  • D Déharbe
  • V Medeiros
Déharbe, D., Medeiros, Jr., V.: Proposal: Translation of B Implementations to LLVM-IR. In: SBMF, Brasília -DF, SBMF (2013)
Automatic Generation of Tests from Language Descriptions (Text in Portuguese). Master’s thesis
  • C Hentz
Hentz, C.: Automatic Generation of Tests from Language Descriptions (Text in Portuguese). Master's thesis, UFRN, Natal, Brazil (2010)
Grammar-based test generation with yougen. Software: Practice and Experience
  • D M Hoffman
  • D Ly-Gagnon
  • P Strooper
  • H.-Y Wang
D. M. Hoffman, D. Ly-Gagnon, P. Strooper, and H.-Y. Wang. Grammar-based test generation with yougen. Software: Practice and Experience, 41(4):427-447, 2011.
Compiler correctness and implementation verification: The verifix approach
  • W Goerigk
  • A Dold
  • T Gaul
  • G Goos
  • A Heberle
  • F W Von Henke
  • U Hoffmann
  • H Langmaack
  • H Pfeifer
  • H Ruess
W. Goerigk, A. Dold, T. Gaul, G. Goos, A. Heberle, F. W. Von Henke, U. Hoffmann, H. Langmaack, H. Pfeifer, H. Ruess, et al. Compiler correctness and implementation verification: The verifix approach. 1996.
Tract able model transformation testing
  • M Gogolla
  • A Vallecillo
  • R B France
  • J M Kuester
  • B Bordbar
Gogolla, M., Vallecillo, A.: Tract able model transformation testing. In: France, R.B., Kuester, J.M., Bordbar, B., Paige, R.F. (eds.) ECMFA 2011. LNCS, vol. 6698, pp. 221-235. Springer, Heidelberg (2011)
Test-Data generation for Xtext
  • J Härtel
  • L Härtel
  • R Lämmel
Härtel, J., Härtel, L., Lämmel, R.: Test-Data generation for Xtext. In: Combemale, B., Pearce, D.J., Barais, O., Vinju, J.J. (eds.) SLE 2014. LNCS, vol. 8706, pp. 342-351. Springer, Heidelberg (2014)
Controllable combinatorial coverage in grammar-based testing
  • R Lämmel
  • W Schulte
Lämmel, R., Schulte, W.: Controllable combinatorial coverage in grammar-based testing. In: Uyar, M.U., Duale, A.Y., Fecko, M.A. (eds.) TestCom 2006. LNCS, vol. 3964, pp. 19-38. Springer, Heidelberg (2006)
Testing M2T/T2M transformations
  • M Wimmer
  • L Burgueño
  • A Moreira
  • B Schätz
  • J Gray
  • A Vallecillo
Wimmer, M., Burgueño, L.: Testing M2T/T2M transformations. In: Moreira, A., Schätz, B., Gray, J., Vallecillo, A., Clarke, P. (eds.) MODELS 2013. LNCS, vol. 8107, pp. 203-219. Springer, Heidelberg (2013)