About
273
Publications
58,748
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
5,790
Citations
Introduction
T.H. Tse received the PhD from LSE and was a Visiting Fellow at University of Oxford. He is an Honorary Professor at HKU after retiring from the full professorship in 2014. He is ranked internationally as no. 2 in metamorphic testing. He has been a Steering Committee Chair of IEEE International Conference on Software Quality, Reliability, and Security; and an Associate Editor of IEEE Transactions on Reliability. He is a Life Fellow of BCS. He was awarded an MBE by Queen Elizabeth II of UK.
Skills and Expertise
Current institution
Additional affiliations
January 2006 - June 2014
Publications
Publications (273)
Random testing is not only a useful testing technique in itself, but also plays a core role in many other testing methods. Hence, any significant improvement to random testing has an impact throughout the software testing community. Recently, Adaptive Random Testing (ART) was proposed as an effective alternative to random testing. This paper presen...
Algebraic specifications have been used in the testing of object-oriented programs and received much attention since the 1990s. It is generally believed that class-level testing based on algebraic specifications involves two independent aspects: the testing of equivalent and nonequivalent ground terms. Researchers have cited intuitive examples to i...
Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of w...
We propose a robustness testing approach for software systems that process large amounts of data. Our method uses metamorphic relations to check software output for erroneous input in the absence of a tangible test oracle. We use this technique to test two major citation database systems: Scopus and the Web of Science. We report a surprising findin...
- [...]
The digital world we live in, and the COVID-19 pandemic, has seen a proliferation of digital spaces adopted in the higher education sector. However, not all digital spaces are created equally. This chapter aims to:
• Explore the current state and extent to which universities are resourced to make their digital spaces accessible and inclusive.
• Ana...
The testing of large databases faces the test oracle problem, namely, that it is difficult to verify execution results against expected outcomes. Rigger and Su applied metamorphic testing through query partitioning and ternary logic partitioning techniques to alleviate the challenge. In Part (A) of our project, we conduct an in-depth investigation...
In this final chapter of the Handbook, we introduce emerging, innovative methods in software fault localization. First, we present scientific and systematic hypothesis-testing techniques, and show they may be applied in practice. Second, for fault localization in the absence of a test oracle, we present a semi-proving methodology based on metamorph...
QRS 2022 (The 22nd IEEE International Conference on Software Quality, Reliability, and Security) was originally planned to be held in Guangzhou, China from 5-9, December. However, due to the ongoing pandemic, certain districts of Guangzhou were listed as COVID-19 high-risk infectious areas during the conference dates. As a result, all the in-person...
Existing test case prioritization (TCP) techniques have limitations when applied to real-world projects, because these techniques require certain information to be made available before they can be applied. For example, the family of input-based TCP techniques are based on test case values or test script strings; other techniques use test coverage,...
The papers in this special section focus on software testing and program analysis. Software plays an integral part in our lives today because of its near-ubiquitous influence on our increasingly technological society. Taking appropriate steps to improve software quality is, therefore, of paramount importance. This has fueled the demand for techniqu...
When we consider real estate, we know that “There are three things that matter in property: location, location, location” — a statement attributed to Lord Harold Samuel in 1944.
The article titled “Adaptive random testing: The ART of test case diversity” by T.Y. Chen, F.-C. Kuo, R.G. Merkel, and T.H. Tse was published in Journal of Systems and Sof...
Deep neural networks (DNNs) have been widely used in classification tasks. Studies have shown that DNNs may be fooled by artificial examples known as adversaries. A common technique for testing the robustness of a classification is to apply perturbations (such as random noise) to existing examples and try many of them iteratively, but it is very te...
A series of online meetings were organized internationally to collect personal experiences from persons with disabilities (PwDs) about the impacts of COVID-19 on their daily lives. This was to ensure that the voices of PwDs are being heard, to ascertain the needs for support, and to campaign local leaders for an inclusive approach in the battle aga...
There has been an increasing concern from both industry and the research community about software robustness: the ability of dealing with erroneous input or unexpected situations. To assess robustness, the software under test (SUT) needs to be tested with invalid or erroneous input, and a major approach for this purpose is fuzzing, where random or...
Current research on the testing of machine translation software mainly focuses on functional correctness for valid, well-formed inputs. By contrast, robustness testing, which involves the ability of the software to handle erroneous or unanticipated inputs, is often overlooked. In this paper, we propose to address this important shortcoming. Using t...
Adaptive random testing (ART) has been proposed to enhance the effectiveness of random testing (RT) through more even spreading of the test cases. In particular, restricted random testing (RRT) is an ART algorithm based on the intuition of skipping all the candidate test cases that are within the neighborhoods (or zones) of previously executed test...
*INTRODUCTION*
The Accessible IT Development Association (AIDA) is a social enterprise in Hong Kong. We are a team of people with profound disabilities who provide professional web design and development services. In this paper, we share our experience of the challenges and successes since our establishment in 2013.
*INITIAL CHALLENGES*
Initially,...
Adaptive random testing (ART) was developed as an enhanced version of random testing to increase the effectiveness of detecting failures in programs by spreading the test cases evenly over the input space. However, heavy computation may be incurred. In this paper, three enhanced algorithms for fixed-size-candidate-set ART (FSCS-ART) are proposed ba...
An external web service may evolve without prior notification. In the course of the regression testing of a workflow-based web service, existing test case prioritization techniques may only verify the latest service composition using the not-yet-executed test cases, overlooking high-priority test cases that have already been applied to the service...
Many composite workflow services utilize non-imperative XML technologies such as WSDL, XPath, XML schema, and XML messages. Regression testing should assure the services against regression faults that appear in both the workflows and these artifacts. In this paper, we propose a refinement-oriented level-exploration strategy and a multilevel coverag...
Effective testing is essential for assuring software quality. While regression testing is time-consuming, the fault detection capability may be compromised if some test cases are discarded. Test case prioritization is a viable solution. To the best of our knowledge, the most effective test case prioritization approach is still the additional greedy...
Many existing studies measure the effectiveness of test case prioritization techniques using the average performance on a set of test suites. However, in each regression test session, a real-world developer may only afford to apply one prioritization technique to one test suite to test a service once, even if this application results in an adverse...
As users are increasingly concerned about energy efficiency, they are also increasingly intolerant of performance anomalies of programs that may cause significant energy waste. Bug localization is a bottleneck in the development of multithreaded programs. Although both static and dynamic performance bug localization techniques have been proposed, t...
A common type of study used by researchers to map out the landscape of a research topic is known as mapping study. Such a study typically begins with an exploratory search on the possible ideas of the research topic, which is often done in an unsystematic manner. Hence, the activity of formulating research questions in mapping studies is ill-define...
Applications written by end-user programmers are hardly energy-optimized by these programmers. The end users of such applications thus suffer significant energy issues. In this paper, we propose CrowdAdaptor, a novel approach toward locating energy-efficient configurations to execute the applications hosted in virtual machines on handheld devices....
Context-aware pervasive software is responsive to various contexts and their changes. A faulty implementation of the context-aware features may lead to unpredictable behavior with adverse effects. In software testing, one of the most important research issues is to determine the sufficiency of a test suite to verify the software under test. Existin...
In real life, a tester can only afford to apply one test case prioritization technique to one test suite against a service-oriented workflow application once in the regression testing of the application, even if it results in an adverse scenario such that the actual performance in the test session is far below average. It is unclear whether the fac...
Test case selection is a prime process in the engineering of test harnesses. In particular, test case diversity is an important concept. In order to achieve an even spread of test cases across the input domain, Adaptive Random Testing (ART) was proposed such that the history of previously executed test cases are taken into consideration when select...
In software testing, a test oracle refers to the mechanism for determining whether the results of the software under test agree with the expected outcomes. To achieve this, we need a means to determine the expected outcomes, a means to gauge the actual results, and a means to decide whether the actual results agree with the expected outcomes. In re...
Test case generation is a vital procedure in the engineering of test harnesses. In particular, the choice relation framework and the category-partition method play an important role, by requiring software testers to identify categories (intuitively equivalent to input parameters or environment conditions) and choices (intuitively equivalent to rang...
Context: Existing fault-localization techniques combine various program features and similarity coefficients with the aim of precisely assessing the similarities among the dynamic spectra of these program features to predict the locations of faults. Many such techniques estimate the probability of a particular program feature causing the observed f...
ContextTesting and debugging consume a significant portion of software development effort. Both processes are usually conducted independently despite their close relationship with each other. Test adequacy is vital for developers to assure that sufficient testing effort has been made, while finding all the faults in a program as soon as possible is...
Dynamic race detectors can explore multiple thread schedules of a multithreaded program over the same input to detect data races. Although existing sampling-based precise race detectors reduce overheads effectively so that lightweight precise race detection can be performed in testing or post-deployment environments, they are ineffective in detecti...
When a software system fails at a user site, information is often automatically sent to the developer for diagnostics. Usually, only a system log is included, with all the confidential data such as identities and financial information stripped off. Such a log message is found to be insufficient for debugging. In the present paper, the authors set o...
This paper extends the CHOiCe reLATion framEwork, abbreviated as CHOC'LATE, which assists software testers in the application of category/choice methods to testing. CHOC'LATE assumes that the tester is able to construct a single choice relation table from the entire specification; this table then forms the basis for test case generation using the a...
We present a flow analysis technique for detecting unreachable states and actions in concurrent systems. It is an enhancement of the approach by Cheung and Kramer. Each process of a concurrent system is modeled as a finite state machine, whose states represent process execution states and whose transitions are labeled by actions. We construct depen...
ContextEffective test case prioritization shortens the time to detect failures, and yet the use of fewer test cases may compromise the effectiveness of subsequent fault localization.Objective
The paper aims at finding whether several previously identified effectiveness factors of test case prioritization techniques, namely strategy, coverage granul...
A workflow-based web service may use ultra-late binding to invoke external web services to concretize its implementation at run time. Nonetheless, such external services or the availability of recently used external services may evolve without prior notification, dynamically triggering the workflow-based service to bind to new replacement external...
We present a generic framework, JS Cloud, for the remote execution of Java Script programs. Our work dynamically estimates whether a code partition should be executed remotely. The empirical result shows that JS Cloud can be useful if the Java Script engine on a handheld device is inefficient.
Two popular specification-based test case generation methods are the choice relation framework and the classification-tree methodology. Both of them come with associated tools and have been used in different applications with success. Since both methods are based on the idea of partition testing, they are similar in many aspects. Because of their s...
Search services are the main interface through which people discover information on the Internet. A fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the ranking quality because different a...
Fault localization commonly relies on both passed and failed runs, but passed runs are generally susceptible to coincidental correctness and modern software automatically produces a huge number of bug reports on failed runs. FOnly is an effective new technique that relies only on failed runs to locate faults statistically.
Energy efficiency at the software level has gained much attention in the past decade. This paper presents a performance-aware frequency assignment algorithm for reducing processor energy consumption using Dynamic Voltage and Frequency Scaling (DVFS). Existing energy-saving techniques often rely on simplified predictions or domain knowledge to extra...
Tabular expressions have been proposed as a notation to document mathematically precise but readable software specifications. One of the many roles of such documentation is to guide testers. This paper 1) explores the application of four testing strategies (the partition strategy, decision table-based testing, the basic meaningful impact strategy,...
Model-based testing helps test engineers automate their testing tasks so that they are more cost-effective. When the model is changed because of the evolution of the specification, it is important to maintain the test suites up to date for regression testing. A complete regeneration of the whole test suite from the new model, although inefficient,...
An ongoing, annual survey of publications in systems and software engineering identifies the top 15 scholars and institutions in the field over a 5-year period. Each ranking is based on the weighted scores of the number of papers published in TSE, TOSEM, JSS, SPE, EMSE, IST, and Software of the corresponding period. This report summarizes the resul...
Statistical fault localization techniques find suspicious faulty program entities in programs by comparing passed and failed executions. Existing studies show that such techniques can be promising in locating program faults. However, coincidental correctness and execution crashes may make pro- gram entities indistinguishable in the execution spectr...
An effective integration between testing and debugging should address how well testing and fault localization can work together productively. In this paper, we report an empirical study on the effectiveness of using adequate test suites for fault localization. We also investigate the integration of test case prioritization and statistical fault loc...
Existing fault localization techniques combine various program features and similarity coefficients with the aim of precisely assessing the similarities among the dynamic spectra of these program features to predict the locations of faults. Many such techniques estimate the probability of a particular program feature causing the observed failures....
Fault localization is a major activity in program debugging. To automate this time-consuming task, many existing fault-localization techniques compare passed executions and failed executions, and suggest suspicious program elements, such as predicates or statements, to facilitate the identification of faults. To do that, these techniques propose st...
A web service may evolve autonomously, making peer web services in the same service composition uncertain as to whether the evolved behaviors are compatible with its original collaborative agreement. Although peer services may wish to conduct regression testing to verify the agreed collaboration, the source code of the former service may be inacces...
An early step for most black-box testing methods is to identify a set of categories and choices (or their equivalents) from the specification. The identification is often performed in an ad hoc manner, thus the quality of categories and choices is in doubt. Poorly identified categories and choices will affect the comprehensiveness of test cases. In...
We present an integrated method for program proving, testing, and debugging. Using the concept of metamorphic relations, we select necessary properties for target programs. For programs where global symbolic evaluation can be conducted and the constraint expressions involved can be solved, we can either prove that these necessary conditions for pro...
We present an integrated method for program proving, testing, and debugging. Using the concept of metamorphic relations, we select necessary properties for target programs. For programs where global symbolic evaluation can be conducted and the constraint expressions involved can be solved, we can either prove that these necessary conditions for pro...
Pervasive computing systems often use middleware as a means to communicate with the changing environment. However, the interactions with the context-aware middleware as well as the interactions among applications sharing the same middleware may introduce faults that are difficult to reveal by existing testing techniques. Our previous work proposed...
Model-based testing helps test engineers automate their testing tasks so that they can be more cost-effective. When the model is changed due to the evolution of the specification, it is important to maintain the test suites up to date for regression testing. A complete regeneration of the whole test suite from the new model, although inefficient, i...
Tasks running on embedded systems are often asso- ciated with deadlines. While it is important to complete tasks before their associated deadlines, performance and energy con- sumption also play important roles in many usages of embedded systems. To address these issues, we explore the use of Dynamic Voltage and Frequency Scaling (DVFS), a standard...
Dynamic service compositions pose new verification and validation challenges such as uncertainty in service membership. Moreover, applying an entire test suite to loosely coupled services one after another in the same composition can be too rigid and restrictive. In this paper, we investigate the impact of service selection on service-centric testi...
