Scott R. Tilley’s research while affiliated with Florida Institute of Technology and other places

Download Free Sample This book summarizes the current hard problems in software testing as voiced by leading practitioners in the field. The problems were identified through a series of workshops, interviews, and surveys. Some of the problems are timeless, such as education and training, while others such as system security have recently emerged as increasingly important. The book also provides an overview of the current state of Testing as a Service (TaaS) based on an exploration of existing commercial offerings and a survey of academic research. TaaS is a relatively new development that offers software testers the elastic computing capabilities and generous storage capacity of the cloud on an as-needed basis. Some of the potential benefits of TaaS include automated provisioning of test execution environments and support for rapid feedback in agile development via continuous regression testing. The book includes a case study of a representative web application and three commercial TaaS tools to determine which hard problems in software testing are amenable to a TaaS solution. The findings suggest there remains a significant gap that must be addressed before TaaS can be fully embraced by the industry, particularly in the areas of tester education and training and a need for tools supporting more types of testing. The book includes a roadmap for enhancing TaaS to help bridge the gap between potential benefits and actual results. Table of Contents: Introduction / Hard Problems in Software Testing / Testing as a Service (TaaS) / Case Study and Gap Analysis / Summary / Appendix A: Hard Problems in Software Testing Survey / Appendix B: Google App Engine Code Examples / Appendix C: Sauce Labs Code Examples / References / Author Biographies

As software systems grow in size and complexity, testing is becoming an increasingly costly endeavor in terms of time, labor, and other resources. Despite the best efforts of software testers, defects continue to elude standard testing techniques, resulting in production environments where errors cost significantly more to repair.

This chapter presents a case study that provides the basis for an evaluation of TaaS against the issues identified in the HPST survey. The evaluation framework is described and applied to the tools examined in the case study. Of the twelve tools introduced in the previous chapter, three were chosen as candidates for additional analysis: Sauce Labs, SOASTA CloudTest Lite, and BlazeMeter. The remainder of this chapter details the case study and provides an analysis of the gap between current TaaS solutions and industry needs.

Software testing is a critical phase of the software lifecycle, responsible for assuring the quality of a system under test; however, testing suffers from a variety of challenges—both new and timeless. In order to evaluate the ability of TaaS to overcome these challenges, the needs of the industry must first be identified.

The architecture of a Web system affects many of its quality attributes, such as testability, security, and accessibility. The architecture itself is influenced by factors such as system requirements, infrastructure constraints, and interoperability needs. This session is the fifth in a series of special events held at WSE focused on selected aspects of research directions in Web systems evolution, and the first focused on issues related to the role of architecture.

Communication skills are essential to a successful career in computing. You may be an excellent programmer, but if you can't explain your work to your peers then your career will quickly plateau. This means the most important language you should master during your graduate studies is not C# or Java -- it's English. Whether it's writing reports for class, writing conference or journal papers to summarize research results, or writing a long document such as a thesis or a dissertation, it's all writing. Your special challenge is explaining complex concepts to your readers in a way that they can easily understand. Fortunately, this is a skill that you can learn.

One of the most powerful tools in the hacker's reverse engineering arsenal is the virtual machine. These systems provide a simple mechanism for executing code in an environment in which the program can be carefully monitored and controlled, allowing attackers to subvert copy protection and access trade secrets. One of the challenges for anti-reverse engineering tools is how to protect software within such an untrustworthy environment. From the perspective of a running program, detecting an emulated environment is not trivial: the attacker can emulate the result of different operations with arbitrarily high fidelity. This paper demonstrates a mechanism that is able to detect even carefully constructed virtual environments by focusing on the stochastic variation of system call timings. A statistical technique for detecting emulated environments is presented, which uses a model of normal system call behavior to successfully identify two commonly used virtual environments under realistic conditions.

Software testing is becoming an increasingly expensive and time-consuming endeavor. With advances in cloud computing, new methods of testing in cloud environments are allowing testers to take advantage of the vast resources of the cloud while demanding less upfront costs. Testing as a service (TaaS) is a model of software testing that offers accessible services that handle testing activities for consumers on a pay-for-use basis. The goal of this paper is to describe the state of academic research within TaaS, including a basic architecture, example services, recent studies, benefits, challenges, needs, and a road ahead.

Cloud Computing makes it possible for users to access a wide range of web services in the public domain and to embed these global services in their local applications. This promises to save a significant amount of individual development cost. The biggest obstacle to using this technology is the problem of trust. To gain trust in the services offered they have to be extensively tested, either by the user himself or by a trusted agent. This chapter deals with the testing of web services in the cloud. There are many similarities to testing web services in a local service-oriented architecture, but there are also significant differences. In a company specific SOA, testers can gain access to the source. This is not true of the cloud. There is no possibility of accessing the source. Therefore, testers must rely solely on the specification contained in the service level agreement – SLA – and the web service interface definition – WSDL or REST – to base their test upon. Testing in the cloud is strictly a black-box test. The goal of a cloud service test is also not to find errors but to assess the suitability of the service to the purpose of the user. It may be necessary to test several services in order to find that one best suited to the requirements of the user. To judge suitability it is necessary to define an ideal usage profile, including performance, security and other non-functional criteria, and to compare that with the actual profile of each potential service. For this both static and dynamic analysis methods must be applied. The chapter presents an automated approach to assessing cloud services and selecting that one most suitable to the user’s application.