Anneliese Andrews's research while affiliated with University of Denver and other places

Publications (92)

Article
This paper provides an approach for selective black-box model-based regression testing for web applications, emphasizing testing proper mitigation of external failures in web applications. This approach uses an existing model of web applications, FSMWeb, and extends its test generation capabilities to include selective regression testing of fail-sa...
Article
Full-text available
Model-based Testing (MBT) can play a key role in testing autonomous robotic systems. MBT leverages graph-based behavioral models, such as Communicating Extended Finite State Machines (CEFSMs), to describe the behavior of environment objects that autonomous robots interact with (e.g., moving obstacles). These environment objects interact with each o...
Conference Paper
Model-based Testing (MBT) can play a key role in testing autonomous robotic systems. MBT leverages graph-based behavioral models, such as Communicating Extended Finite State Machines (CEFSMs), to describe the behavior of environment objects that autonomous robots interact with (e.g., moving obstacles). These environment objects interact with each o...
Article
Full-text available
To enable effective and safe operations of autonomous robots in environments with unknowns and unpredictability, a key practical problem is how to test the functionality and assess the performance of real-time motion planning systems. This is a challenge because the underlying algorithms are real-time, sensing-based, and often non-deterministic. Th...
Conference Paper
Full-text available
It is a practically challenging problem to test the functionality of autonomous systems and assess their performance in environments with unknowns and unpredictability. Existing testing techniques are designed heavily based on testers experience and hardly take into account all scenarios. This paper applies a model-based testing technique to evalua...
Article
External failures like network changes can affect system operation negatively. Mitigation requirements try to prevent or reduce their effects. This paper presents a black box testing approach that tests fail-safe behavior in web applications. Failure mitigation tests are built based on a functional test suite. Mitigation requirements are used to bu...
Conference Paper
Full-text available
This paper describes a model-based test generation approach for testing autonomous systems interacting with their environment (i.e., world). Unlike other approaches that assume a static world with attributes and values, we present and test a dynamic world. We use Communicating Extended Finite State Machine (CEFSM) to illustrate an active world mode...
Conference Paper
Full-text available
This paper presents a selective regression testing approach for testing proper failure mitigation in safety-critical systems. Based on changes to the models used for test generation, it classifies tests as obsolete, retestable, and reusable. It also defines a process for generating new tests based on partial regeneration. It takes advantage of the...
Conference Paper
In this case study we investigate software reliability models and their applicability to process improvement at an IT help desk. We propose a model selection framework and demonstrate its success using real help desk incident data from a portfolio of 156 desktop software applications. Incidents are predicted at five intervals and measured against a...
Article
This paper proposes an approach for testing of safety-critical systems. It is based on a behavioral and a fault model. The two models are analyzed for compatibility, and necessary changes are identified to make them compatible. Then, transformation rules are used to transform the fault model into the same model type as the behavioral model. Integra...
Article
Many aerospace vehicles are vulnerable to security attacks that not only can cause loss of the vehicle, but also loss of life or compromise national security. This paper proposes an approach for security testing an aerospace launch system. Our approach is based on building a security test suite from a behavioral model, an attack type and a mitigati...
Article
Full-text available
In the aerospace domain, defects in mitigating failures can have very high costs. This makes it imperative to have systematic testing approaches for making sure that failure mitigations work properly. This paper describes a model-based testing approach that targets mitigations of system failures. It includes models of the system, of possible failur...
Conference Paper
Full-text available
This paper describes a model-based test generation approach for testing autonomous systems interacting with their environment (i.e., world). Unlike other approaches that assume a static world with attributes and values, we present and test the world dynamically. We build the world model in two steps: a structural model that constructs environmental...
Article
Full-text available
In formal experiments on software engineering, the number of factors that may impact an outcome is very high. Some factors are controlled and change by design, while others are are either unforeseen or due to chance. This paper aims to explore how context factors change in a series of formal experiments and to identify implications for experimentat...
Article
In recent years, the programming language design community has engaged in rigorous debate on the role of empirical evidence in the design of general purpose programming languages. Some scholars contend that the language community has failed to embrace a form of evidence that is non-controversial in other disciplines (e.g., medicine, biology, psycho...
Conference Paper
Costs associated with IT help desk operations present challenges to profitability goals in an organization. Minimizing software failures in an operational environment is important to customer experience, but more practically to the cost model for offering IT services. The user-facing behavior of software systems such as unscheduled downtime, slow p...
Article
Full-text available
The exploratory testing (ET) approach is commonly applied in industry, but lacks scientific research. The scientific community needs quantitative results on the performance of ET taken from realistic experimental settings. The objective of this paper is to quantify the effectiveness and efficiency of ET vs. testing with documented test cases (test...
Conference Paper
Most help desk operations keep records on incidents and how they are resolved. Rarely are these analyzed with respect to recommendations for improvements. At the same time, improvements have the potential to decrease costs of helpdesk operations and increase effectiveness. We analyzed incident reports of two products (one incident prone, the other...
Conference Paper
Full-text available
This paper proposes a genetic algorithm (GA)method to generate test scenarios for testing proper fail-safe behavior for web applications. Unlike other approaches which combine fault trees with state charts, we create mitigation tests from an existing functional black box test suite. A genetic algorithm is used that determines points of failures and...
Conference Paper
This paper proposes an approach for testing of safety critical systems. It is based on a behavioral and a fault model. The two models are analyzed for compatibility and necessary changes are identified to make them compatible. Then transformation rules are used to transform the fault model into the same model type as the behavioral model. Integrati...
Conference Paper
Full-text available
This paper provides a technique for testing proper failure mitigation in safety-critical systems. Unlike other approaches which integrate behavioral and failure models, and then generate tests from the integrated model, we build failure-mitigation tests from an existing behavioral test suite, using an explicit mitigation model for which we generate...
Conference Paper
This paper provides a tradeoff framework to determine whether selective regression testing or brute force regression testing is preferable. The framework is based on classifying tests as reusable, retest able, and obsolete. It uses the FSMWeb model as the behavioral model upon which test generation is based. To illustrate how our tradeoff framework...
Article
Code inspections and structural unit testing are two common defect detection methods in software development. This paper analyzes three replications of an experiment aimed at comparing which method finds more defects. The first two experiments use the same experimental design and materials, including the programs to be inspected or tested, while th...
Article
Web applications are fast becoming more widespread, larger, more interactive, and more essential to the international use of computers. It is well understood that web applications must be highly dependable, and as a field we are just now beginning to understand how to model and test Web applications. One straightforward technique is to model Web ap...
Article
Testing of web -based applications presents a number of challenges posed by the heterogeneity of hardware, op erating systems, software platforms, etc. The distributed nature of web-based applications increases behavioral di versity and contributes to complexity. Many web appli cations evolve rapidly and experience frequent changes. These character...
Article
Full-text available
A method for automatically identifying and classifying hyperactivated spermatozoa trajectories is described. This physiologically-based computerized algorithm captures the motion behavior of sperm during hyperactivation. A novel Minimum Bounding Square Ratio (MBSR) algorithm classifies spermatoza as hyperactivated, transitional or progressive. Clas...
Article
The unified modeling language (UML) designs come in a variety of different notations. UML designs can be quite large and interactions between various notations and the models they define can be difficult to assess. During the design phase, and between successive releases of systems, designs change. The impact of such changes and the resulting effec...
Article
Many organizations rely on web applications that use back-end databases to store important data. Testing such applications requires significant effort. Manual testing alone is often impractical, so testers also rely on automated testing techniques. However, current automated testing techniques may produce false positives (or false negatives) even i...
Article
AbstractThis chapter explores current state-of-the-art techniques that have been used in software design testing, either to test the designs or to test implementations against the designs. A common design notation that is in use today is UML. Originally, techniques have been designed with the intentions of testing implementations against their desi...
Article
Reproducing and learning from failures in deployed software is costly and difficult. Those activities can be facilitated, however, if the circumstances leading to a failure can be recognized and properly captured. To anticipate failures we propose to monitor system field behavior for simple trace instances that deviate from a baseline behavior expe...
Article
Early detection and correction of faults in the software design phase can reduce total cost and time to market of a software product. In this paper we describe an approach for testing UML design models to uncover inconsistencies. Our approach uses behavioral views such as Sequence Diagrams to simulate state change in an aggregate model. The aggrega...
Conference Paper
As model driven architectures (MDAs) gain in popularity, several techniques that test the UML models have been proposed. These techniques aim at early detection and correction of faults to reduce the overall cost of correcting them later in the software life-cycle. Recently, Pilskalns et al., 2003 proposed an approach to test the UML design models...
Article
Problems related to interactions between components is a sign of problems with the software architecture of the system and are often costly to fix. Thus it is very desirable to identify potential architectural problems and track them across releases to see whether some relationships between components are repeatedly change-prone.This paper shows a...
Article
Full-text available
RESUMEN RESUMEN Serious efforts are spent on defect detection during software development. So, it's important to use the most efficient and effective defect detection methods . This guide can help practitioners choose efficient and effective defect detection methods using the evidence - based software engineering (EBSE) approach . Using 12 empiri...
Conference Paper
Full-text available
Software defect data are typically used in reliability modeling to predict the remaining number of defects in order to assess software quality and release decisions. However, in practice such decisions are often constrained by availability of resources. As software gets more complex, testing and fixing defects become difficult to schedule. This pap...
Conference Paper
Security attacks continually threaten distributed systems, disrupting both individuals and organizations economically and physically. In the software lifecycle, early detection and correction of security flaws in the design phase can reduce overall costs associated with maintenance. Current software development methodologies such as the model drive...
Article
Full-text available
Designs almost always require tradeoffs between competing design choices to meet system require- ments. We present a framework for evaluating design choices with respect to meeting competing requirements. Specifically, we develop a model to estimate the performance of a UML design subject to changing levels of secu- rity and fault-tolerance. This a...
Conference Paper
During component based software engineering it is important for component developers to design components that show high cohesion within a component and low coupling between components. Empirical data shows that software artifacts possessing these properties are more maintainable. Current practice in design metric evaluation relies on extracting st...
Conference Paper
Full-text available
For model driven development approaches to succeed, there is a need for model validation techniques. This paper presents an approach to testing designs described by UML class diagrams, interaction diagrams, and activity diagrams. A UML design model under test is transformed into an executable form. Test infrastructure is added to the executable for...
Article
Researchers and practitioners are still trying to nd eective ways to model and test Web applications. This paper proposes a system-level testing technique that combines test generation based on nite state machines with constraints. We use a hierarchical approach to model potentially large Web applications. The approach builds hierarchies of Finite...
Conference Paper
A process is proposed to support comprehension of COTS components. In this process, software developers build a COTS comprehension model to support component selection. We integrate the process with a UML component based software development approach. We illustrate our approach using a hotel reservation system.
Conference Paper
To increase the likelihood for software project success, it is important to be able to identify the drivers of success. This paper compares three methods to identify similar projects with the objective to predict project success. The hypothesis is that projects with similar characteristics are likely to have the same outcome in terms of success. Tw...
Conference Paper
Fault links represent relationships between the types of mistakes made and the type of module being developed or modified. The existence of such fault links can be used to guide code reviews, walkthroughs, allocation of verification and validation resources, testing of new code development, as well as code maintenance. We present an approach for ca...
Conference Paper
This paper presents a systematic method for simultaneously defining a software architecture and selecting off-the-shelf components for reuse. The method builds upon existing techniques for component selection and architecture evaluation. We identify architectural decisions that have a large effect on the components used early in the process so that...
Article
The code that provides solutions to key software re-quirements, such as security and fault-tolerance, tends to be spread throughout (or cross-cut) the program modules that implement the "primary functionality" of a software system. Aspect-oriented programming is an emerging programming paradigm that supports imple-menting such cross-cutting require...
Conference Paper
Full-text available
The complex functionalities and high demands of software quality make manual testing of a web application ineffective. Automatic software testing methods can help to determine if a web application is working correctly, but existing methods are unable to test whether such an application interacts correctly with a back-end database. This paper elabor...
Article
The code that provides solutions to key software requirements, such as security and fault-tolerance, tends to be spread throughout (or cross-cut) the program modules that implement the "primary functionality" of a software system. Aspect-oriented programming is an emerging programming paradigm that supports implementing such cross-cutting requireme...
Conference Paper
Designs almost always require trade-offs between competing design choices to meet pervasive system dependability requirements (e.g., security, performance and fault tolerance system goals). In some cases, dependability requirements are realized by functionality that cross-cuts designs. Aspect-oriented modeling (AOM) methods allow developers to loca...
Conference Paper
Verification of high-assurance system designs before they are fabricated on chips reduces time and money costs related to chip testing. Test benches, which are used in simulating and verifying VHDL designs, need to perform efficiently and effectively. Test automation through the automated generation of test patterns increases the efficiency and eff...
Conference Paper
For model driven development approaches to succeed, there is a need for developing techniques for validating models. Studies show that many software faults occur in the design phase. Hence, it is essential to find and remove faults in design models. Currently, UML design models are typically evaluated using walkthroughs, inspections, and other info...
Conference Paper
Component based development in the design phase ne- cessitates a comprehensive look at both static and dynamic UML views. If a design is to incorporate third-party com- ponents, one must define component interfaces. We propose a method for defining components in the design phase that can be used for qualification purposes. Coupling and fre- quency...
Conference Paper
Full-text available
Code inspections and white-box testing have both been used for unit testing. One is a static analysis technique, the other, a dynamic one, since it is based on executing test cases. Naturally, the question arises whether one is superior to the other, or, whether either technique is better suited to detect or isolate certain types of defects. We inv...
Conference Paper
Full-text available
Systematic design testing, in which executable models of behaviors are tested using inputs that exercise scenarios, can help reveal flaws in designs before they are implemented in code. We present a testing method in which executable forms of the Unified Modeling Language (UML) models are tested. The method incorporates the use of test adequacy cri...
Conference Paper
Full-text available
Error detection and correction in the design phase can re- duce total costs and time to market. Yet, testing of design models usually consists of walk-throughs and inspections both of which lack the rigor of systematic testing. Test adequacy criteria for UML models help de- fine necessary objectives during the process of test creation. These test c...
Article
Full-text available
This paper describes an integrated method using various techniques to improve testing efficiency. It uses several empirical techniques (capture-recapture methods, an experience-based method, and a selection method for software reliability growth models) and describes ways to combine the defect estimates obtained to make release decisions. Results o...
Article
This paper presents a method for analyzing the impact software project factors have on project success as defined by project success factors that have been prioritized. It is relatively easy to collect measures of project attributes subjectively (i.e., based on expert judgment). Often Likert scales are used for that purpose. It is much harder to id...
Conference Paper
During software evolution, adaptive, and corrective maintenance are common reasons for changes. Often such changes cluster around key components. It is therefore important to analyze the frequency of changes to individual classes, but, more importantly, to also identify and show related changes in multiple classes. Frequent changes in clusters of c...
Article
Systematic design testing, in which executable models of behaviours are tested using inputs that exercise scenarios, can help reveal flaws in designs before they are implemented in code. In this paper a technique for testing executable forms of UML (Unified Modelling Language) models is described and test adequacy criteria based on UML model elemen...
Conference Paper
The verification of behavioral models is an important step before transferring a hardware design to the layout. A popular approach is to use a variety of code coverage measures to evaluate how much of the design has been simulated. Common coverage measures include branch coverage and bit toggle coverage. This paper presents a test pattern generatio...
Article
Full-text available
Defect analysis of software components can be used to guide testing, with the goal of focusing on parts of the software that were fault-prone in earlier releases or earlier life cycle phases, such as development. We replicate a study that adapted a reverse architecting technique using defect reports to derive fault architectures. A fault architectu...
Article
Full-text available
Estimating remaining defects (or failures) in software can help test managers make release decisions during testing. Several methods exist to estimate defect content, among them a variety of software reliability growth models (SRGMs). SRGMs have underlying assumptions that are often violated in practice, but empirical evidence has shown that many a...
Article
Anti-random testing has proved useful in a series of empirical evaluations. The basic premise of anti-random testing is to chose new test vectors that are as far away from existing test inputs as possible. The distance measure is either Hamming distance or Cartesian distance. Unfortunately, this method essentially requires enumeration of the input...
Article
This paper describes an AI planner assisted approach to generate test cases for system testing based on high level test objectives. We use four levels of test generation: the metaprocessor, the preprocessor, the AI planner, and the postprocessor levels. Test generation is based on an extended UML model of the system under test and a mapping of high...
Article
SUMMARY Components that have defects after release, but not during testing, are very undesirable as they point to 'holes' in the testing process. Either new components were not tested enough, or old ones were broken during enhancements and defects slipped through testing undetected. The latter is particularly pernicious, since customers are less fo...
Conference Paper
Understanding the behavior of components is an important task in the component-based software development process. Component users build mental models to understand a component when they use it for the first time. The models are also useful during the evolution of the component and the application that incorporates it. The process of component unde...
Conference Paper
Project success is influenced by many factors. Some are primary drivers of project success, others secondary. However, these secondary factors may be no less important to understand and learn from than the primary factors. In addition, project success drivers are often qualitative and subjective, eluding analysis through traditional statistical met...