J.F. Leathrum’s research while affiliated with Clemson University and other places

Experience with an innovative approach to requirements based testing in the IEEE POSIX arena is presented. The work described in this paper is based on a full scale conformance test suite development for the standard IEEE Std 10035-The Ada Language Binding to POSIX. The approach taken in this work is based on a formal description of test requirements which can be automatically translated into executable tests by the Clemson Automated Testing System (CATS). A brief design overview of CATS, as well as a design taxonomy for the testing requirements is presented. Lessons learned from this experienced recommendations for future work conclude the paper

Experience from a full scale effort to apply formal methods to automated testing in the open systems software arena is described. The formal method applied in this work is based upon the Clemson Automated Testing System (CATS) which includes a formal specification language, a set of guidelines describing how to use the method effectively, and tool support capable of translating formal specifications into executable tests. This method is currently being used to develop a full scale test suite for IEEE's Ada Language Binding to POSIX. Following an overview of CATS, an experience report consisting of results, lessons learned and future directions is presented.

The development of formal test specifications for an open system standard is described. The effort is being conducted within the environment provided by the Clemson Automated Testing System (CATS). CATS features the ability to automatically translate formal test specifications into executable tests. The formal test specifications are written in accordance with a specification language designed in support of this effort. An overview of the CATS architecture and formal test specification language provide a backdrop for an experience report on the development of formal test specifications for IEEE Std 1003.5 POSIX Ada Language Interfaces. A discussion of scale-up issues concludes the paper

The role of formal methods is examined in the context of the process of developing and adopting open standards. Against the broad backdrop of concerns for improving the quality of standards, issues of conformance assessment, test specification, and test methodology guidelines are considered. The experience gained from the attempts to formalize the test specifications for POSIX 2003.5 is presented as lessons learned. The tradeoffs associated with the various formal methods are considered in terms of the properties of common semantic model for assertions languages. The intent here is to collect the common features in a form that provides insights on issues such as encapsulation and inheritance of specifications, inter-operation semantics, state and control structures for assertions, and name space management conventions.

An abstract is not available.

This paper describes the lifecycle role of a conformance testing research facility in the open systems standards environment. This facility, the Clemson Automated Testing System (CATS), has demonstrated the value of integrating formalized test methods within all phases of standards development. IEEE's effort to develop a standard for operating systems interfaces (POSIX) has provided a working environment to investigate and evaluate the capabilities of CATS. In this arena, CATS has proven valuable in exposing critical issues in the emerging standard and in formulating feasible solutions on multiple occasions. The role of CATS in the areas of automated testing, profile development and real-time extensions is described. A discussion of future directions for CATS and testing in open system standards concludes the paper

This paper describes a data visualization tool developed to support prototype development and testing of software standard interfaces in the open systems environment. The visualization capability is an extension of the Clemson Automated Testing System (CATS). CATS is a research facility which has proven valuable in exposing and addressing critical issues in emerging areas such as the IEEE POSIX real-time extensions. Preliminary investigations with CATS involving real-time interfaces and statistical reasoning about large scenarios of tests has motivated the need for a data visualization capability. Current approaches to testing open systems interfaces include very limited visualization aids, such as 2D bar charts to represent statistical information about the test results. The visualization tool developed in this work extends these capabilities by introducing realism and abstraction via raytracing and hierarchical data representations. These capabilities support a more meaningful analysis of system behavior in that a new, more descriptive set of questions is possible. Experimental results achieved with CATS and the visualization of system behavior with respect to deadlines for real-time systems are presented. Future applications for the data visualization tool in the open systems standards arena are proposed.

An abstract is not available.

This paper illustrates the power and flexibility of a new approach to creating conformance test suites for open systems software standards such as POSIX. The approach incorporates an automated process for the generation and validation of conformance tests. The automated process has been implemented and comprises key components of the Clemson Automated Testing System (CATS). CATS is a testing facility designed to provide a life-cycle support environment for the development of conformance tests. This paper introduces the CATS facility and presents examples of its use