Tsong Yueh Chen's research while affiliated with Swinburne University of Technology and other places

Publications (282)

Preprint
Full-text available
Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating la...
Preprint
Full-text available
Automated driving systems (ADS) are expected to be reliable and robust against a wide range of driving scenarios. Their decisions, first and foremost, must be well understood. Understanding a decision made by ADS is a great challenge, because it is not straightforward to tell whether the decision is correct or not, and how to verify it systematical...
Article
Context Metamorphic testing (MT) has been successfully applied to a wide scope of software systems. In these applications, the testing results of MT form the basis for drawing conclusions about the target system’s performance. Therefore, the effectiveness of MT is crucial to the trustfulness of the derived conclusions. Objective However, due to th...
Article
Over the past decade, metamorphic testing has gained rapidly increasing attention from both academia and industry, particularly thanks to its high efficacy on revealing real-life software faults in a wide variety of application domains. On the basis of a set of metamorphic relations among multiple software inputs and their expected outputs, metamor...
Article
Metamorphic Relations (MRs) play a key role in determining the fault detection capability of Metamorphic Testing (MT). As human judgement is required for MR identification, systematic MR generation has long been an important research area in MT. Additionally, due to the extra program executions required for follow-up test cases, some concerns have...
Article
Full-text available
Metamorphic testing is well known for its ability to alleviate the oracle problem in software testing. The main idea of metamorphic testing is to test a software system by checking whether each identified metamorphic relation (MR) holds among several executions. In this regard, identifying MRs is an essential task in metamorphic testing. In view of...
Article
This paper presents an automated, domain-independent, metamorphic testing platform called MTKeras. In this paper, we report on an investigation demonstrating the effectiveness and usability of MTKeras through five case studies in the four domains of image classification, sentiment analysis, search engines and database management systems. We also re...
Preprint
Full-text available
Regression is one of the most commonly used statistical techniques. However, testing regression systems is a great challenge because of the absence of test oracle in general. In this paper, we show that Metamorphic Testing is an effective approach to test multiple linear regression systems. In doing so, we identify intrinsic mathematical properties...
Preprint
Full-text available
Software testing is often hindered where it is impossible or impractical to determine the correctness of the behaviour or output of the software under test (SUT), a situation known as the oracle problem. An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a s...
Article
Full-text available
Software testing is often hindered where it is impossible or impractical to determine the correctness of the behaviour or output of the software under test (SUT), a situation known as the oracle problem. An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a s...
Article
Regression is one of the most commonly used statistical techniques. However, testing regression systems is a great challenge because of the absence of test oracle in general. In this paper, we show that Metamorphic Testing is an effective approach to test multiple linear regression systems. In doing so, we identify intrinsic mathematical properties...
Conference Paper
Full-text available
Since the popularization of social media, news has entered our lives digitally. While news is spreading broader and faster, fake news is becoming an increasingly popular topic. Fake news detection is therefore important in both social media and research areas. With artificial intelligence technology, software engineers have developed a lot of fake...
Conference Paper
Full-text available
Information is indispensable in modern society. People's daily communication and work depend on information transmission. Unsafe storage or transmission of data may result in privacy and security problems. One way to attempt to prevent such issues is to use encryption algorithms to transform information into encrypted forms. Because the encryption...
Conference Paper
Full-text available
Purpose-Computer Science (CS) has been rapidly growing in popularity, partly due to the great growth of big data (BD) and other new technologies, and the services that support them. BD involves data sets that are too large to be captured, curated, analyzed, or processed (for an acceptable time and cost) using traditional CS techniques. Although BD...
Article
Full-text available
Existing test case prioritization (TCP) techniques have limitations when applied to real-world projects, because these techniques require certain information to be made available before they can be applied. For example, the family of input-based TCP techniques are based on test case values or test script strings; other techniques use test coverage,...
Preprint
Full-text available
Training multiple-layered deep neural networks (DNNs) is difficult. The standard practice of using a large number of samples for training often does not improve the performance of a DNN to a satisfactory level. Thus, a systematic training approach is needed. To address this need, we introduce an innovative approach of using metamorphic relations (M...
Conference Paper
Full-text available
Diversity is the key factor for test cases to detect program failures. Adaptive random testing (ART) is one of the effective methods to improve the diversity of test cases. Being an ART algorithm, the evolutionary adaptive random testing (eAR) only increases the distance between test cases to enhance its failure detection ability. This paper presen...
Article
Context Previous studies proposed different kinds of approaches for class integration test order generation, and corresponding systems can be implemented based on these approaches. Such class integration test order generation systems can facilitate the process of software integration testing if they are implemented correctly. Objective However, a...
Article
Full-text available
Unsupervised machine learning is the training of an artificial intelligence system using information that is neither classified nor labeled, with a view to modeling the underlying structure or distribution in a dataset. Since unsupervised machine learning systems are widely used in many real-world applications, assessing the appropriateness of thes...
Article
Failure region, where failure-causing inputs reside, has provided many insights to enhance testing effectiveness of many testing methods. Failure region may also provide some important information to support other processes such as software debugging. When a testing method detects a software failure, indicating that a failure-causing input is ident...
Article
Mutation testing is basically a fault-based software testing technique, which has been proposed to measure the fault detection effectiveness of a test suite using programs with simulated faults (namely mutants). However, mutation testing is time consuming and computationally expensive because of the normal use of a large amount of mutants. Thus, re...
Conference Paper
Full-text available
This paper describes an ongoing project to develop a Virtual Reality platform to deliver phobia-inspired experiences. These experiences could induce a reaction in the user that may help the user overcome, or alleviate, the phobia. The platform includes monitoring sensors that could be used to measure how much impact the experience is having. The pr...
Preprint
Failure region, where failure-causing inputs reside, has provided many insights to enhance testing effectiveness of many testing methods. Failure region may also provide some important information to support other processes such as software debugging. When a testing method detects a software failure, indicating that a failure-causing input is ident...
Article
Test-suite-based automated program repair (APR) techniques acquire information from an input test suite to guide the repair process, aiming to produce a repair that can pass all test cases of the input test suite. Obviously, the input test suite has a critical impact on the repair effectiveness of APR techniques. This article reports on a study of...
Conference Paper
Full-text available
We present the design of MTKeras, a generic metamorphic testing framework for machine learning, and demonstrate its effectiveness through case studies in image classification and sentiment analysis.
Article
Combinatorial interaction testing (CIT) aims at constructing a covering array (CA) of all value combinations at a specific interaction strength, to detect faults that are caused by the interaction of parameters. CIT has been widely used in different applications, with many algorithms and tools having been proposed to support CA construction. To dat...
Article
Full-text available
Adaptive random testing (ART) has been proposed to enhance the effectiveness of random testing (RT) through more even spreading of the test cases. In particular, restricted random testing (RRT) is an ART algorithm based on the intuition of skipping all the candidate test cases that are within the neighborhoods (or zones) of previously executed test...
Chapter
Full-text available
Adaptive random testing (ART) has been proved to be effective in improving the failure detection ability of random testing. As a lightweight ART algorithm, the ART by bisection (ART-B) can realize test case generation in a linear order of time complexity, but its ability for finding failures is not so strong. In this study, the dynamic bisection is...
Article
Full-text available
The effectiveness analysis of risk evaluation formulas has become a significant research area in spectrum-based fault localization (SBFL). The risk evaluation formula is designed and widely used to evaluate the likelihood of a program spectrum to be faulty. There are numerous empirical and theoretical studies to investigate and compare the performa...
Article
Full-text available
In recent years, Service Oriented Architecture (SOA) has been increasingly adopted to develop distributed applications in the context of the Internet. To develop reliable SOA-based applications, an important issue is how to ensure the quality of web services. In this paper, we propose a dynamic random testing (DRT) technique for web services, which...
Chapter
Although both positive and negative testing are important for assuring quality of programs, generating a variety of test inputs for such testing purposes is difficult for machine learning software. This paper studies why it is difficult, and then proposes a new method of generating datasets that are test inputs to machine learning programs. The pro...
Article
A geographic information system (GIS) provides basic location-enabled services for many different applications related to navigation, education, and telecommunications. It is a foundation for analysis and visualization. Testing GIS is critical, but challenging due to the difficulty to assess the correctness of GIS outputs, which is called the test...
Article
Full-text available
Test-case prioritisation (TCP) attempts to schedule the order of test-case execution such that faults can be detected as quickly as possible. TCP has been widely applied in many testing scenarios such as regression testing and fault localisation. Abstract test cases (ATCs) are derived from models of the system under test and have been applied to ma...
Article
Full-text available
Test‐case prioritisation (TCP) attempts to schedule the order of test‐case execution such that faults can be detected as quickly as possible. TCP has been widely applied in many testing scenarios such as regression testing and fault localisation. Abstract test cases (ATCs) are derived from models of the system under test and have been applied to ma...
Conference Paper
Full-text available
Purpose-The discipline of computer science (CS) has seen a surge in popularity in education, with student demand the highest it has been for many years. A reason for this enthusiasm may be linked to the recent growth of the industry, boosted by the increasingly ubiquitous ABC of artificial intelligence, big data, and cloud computing. However, in ad...
Article
Full-text available
Abstract test cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (e.g., in regression testing), test case prioritization (TCP) has been proposed to improve the testing quality, aiming at ordering test c...
Article
Full-text available
Adaptive random testing (ART) was developed as an enhanced version of random testing to increase the effectiveness of detecting failures in programs by spreading the test cases evenly over the input space. However, heavy computation may be incurred. In this paper, three enhanced algorithms for fixed-size-candidate-set ART (FSCS-ART) are proposed ba...
Article
Full-text available
What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of t...
Article
Full-text available
Test case prioritization (TCP) plays an important role in identifying, characterizing, diagnosing and correcting faults quickly. TCP has been widely used to order test cases of different types, including model inputs (also called abstract test cases). Model inputs are constructed by modeling the program according to its input parameters, values, an...
Article
Testing scientific software is a difficult task due to their inherent complexity and the lack of test oracles. In addition, these software systems are usually developed by end user developers who are neither normally trained as professional software developers nor testers. These factors often lead to inadequate testing. Metamorphic testing is a sim...
Article
Full-text available
Modern information technology paradigms, such as online services and off-the-shelf products, often involve a wide variety of users with different or even conflicting objectives. Every software output may satisfy some users, but may also fail to satisfy others. Furthermore, users often do not know the internal working mechanisms of the systems. This...
Article
Combining spectrum-based fault localization (SBFL) with other techniques is generally regarded as a feasible approach as advantages from both techniques would be preserved. SENDYS which combines SBFL with slicing-hitting-set-computation is one of the promising techniques. However, all current evaluations on SENDYS were obtained via empirical studie...
Article
Full-text available
As part of a software testing process, output verification poses a challenge when the output is not numeric or textual, such as graphical. The industry practice of using human oracles (testers) to observe and verify the correctness of the actual results is both expensive and error-prone. In particular, this practice is usually unsustainable when de...
Article
Random testing and partition testing are two major families of software testing techniques. They have been compared both theoretically and empirically in numerous studies for decades, and it has been acknowledged that they have their own advantages and disadvantages and that their innate characteristics are fairly complementary to each other. In th...
Preprint
Unsupervised machine learning is a task of modeling the underlying structure of ``unlabeled data''. Since learning algorithms have been incorporated into many real-world applications, validating the implementations of those algorithms becomes much more important in the aim of software quality assurance. However, validating unsupervised machine lear...
Conference Paper
test cases are derived by modeling the system under test, and have been widely applied in practice, such as for software product line testing and combinatorial testing. Abstract test case prioritization (ATCP) is used to prioritize abstract test cases and aims at achieving higher rates of fault detection. Many ATCP algorithms have been proposed, us...
Article
Context Testing concurrent data structures remains a notoriously challenging task, due to the nondeterminism of multi-threaded tests and the exponential explosion on the number of thread schedules. Objective We propose an automated approach to generate a series of concurrent test cases in an adaptive manner, i.e., the next test cases are generated...
Conference Paper
Full-text available
We report on a novel use of metamorphic relations (MRs) in machine learning: instead of conducting metamorphic testing, we use MRs for the augmentation of the machine learning algorithms themselves. In particular, we report on how MRs can enable enhancements to an image classification problem of images containing hidden visual markers ("Artcodes")....
Preprint
Full-text available
We report on a novel use of metamorphic relations (MRs) in machine learning: instead of conducting metamorphic testing, we use MRs for the augmentation of the machine learning algorithms themselves. In particular, we report on how MRs can enable enhancements to an image classification problem of images containing hidden visual markers ("Artcodes")....
Preprint
Full-text available
In recent years, metamorphic testing has been successfully and systematically adopted within Adobe Systems to improve the cost effectiveness of its software process. In this industry experience report, we present a case where metamorphic testing has been applied to the Data Collection JavaScript Library of Adobe Analytics. This type of software is...
Conference Paper
Combinatorial Interaction Testing (CIT) aims at constructing an effective test suite, such as a Covering Array (CA), that can detect faults that are caused by the interaction of parameters. In this paper, we report on some empirical studies conducted to examine the fault detection capabilities of five popular CA constructors: ACTS, Jenny, PICT, CAS...
Preprint
Full-text available
As part of a software testing process, output verification poses a challenge when the output is not numeric or textual, such as graphical. The industry practice of using human oracles (testers) to observe and verify the correctness of the actual results is both expensive and error-prone. In particular, this practice is usually unsustainable when de...
Article
Full-text available
Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of w...
Conference Paper
Automated program repair (APR) is a promising approach to localize faults and generate patches for program under repair. One of the test suite based APR techniques, a method leveraging Adaptive search strategies and program Equivalence (AE), has been commonly used for program repair. AE assumes the availability of test oracles, which brings the ora...