D. Richard Kuhn

National Institute of Standards and Technology | NIST · Computer Security Division

Artificial Intelligence and machine learning (AI/ML) failures can be challenging to analyze. Detailed understanding of how these systems fail is a valuable first step in designing better AI/ML safety and security practices.

Symbolic execution of smart contracts suffers from sequence explosion. Some existing tools limit the sequence length, thus being unable to adequately evaluate some functions. In this paper, we propose a symbolic execution approach without limiting the sequence length. In our approach, the symbolic execution process is a two-phase model that maximiz...

The data-intensive nature of machine learning (ML)-enabled systems introduces unique challenges in test and evaluation. We present an overview of combinatorial coverage, exploring its applications across the ML-enabled system lifecycle and its potential to address key limitations in performing test and evaluation for ML-enabled systems.

Verification of complex software systems is an important, yet challenging task. Testing is the most common method for assuring that software meets its specifications and is defect-free. To claim that software is defect-free and thus reliable , one has to show that it produces the “correct” output or “behaves” according to specification without fa...

Machine Learning (ML) models rely on capturing important feature interactions to generate predictions. This study is focused on validating the hypothesis that model predictions often depend on interactions involving only a few features. This hypothesis is inspired by t-way combinatorial testing for software systems. In our study, we utilize the not...

Many conventional software engineering methods for high-trust software are not well suited to assured autonomy, but concepts from combinatorial testing can add confidence by providing a quantitative measure of the usefulness of a dataset.

With the increased complexity of software systems, dependable, reliable, and trustworthy computing is of paramount importance. Of these qualities, dependability is of particular interest in mission critical systems, where failure can lead to loss of human life. The technology used to build such systems must meet the expectations of its stakeholders...

The ability to efficiently share information has both commercial and societal benefits—even more so when the data can be shared outside of a host organization. For example, improved data sharing between medical providers that are not in the same network will enhance patient care [1] as well as reduce costs by sharing data and minimizing redundant...

Stakeholders of technology-oriented systems expect some level of reliability from them. IEEE defines Reliability as “the ability of a system or component to perform its required functions under stated conditions for specified period of time” [10]. One of the crucial factors in determining a system’s quality is based on how reliable it is. However...

This article summarizes some recent novel approaches to the problem of verification, testing, and assurance of autonomous systems. These include proxy verification and combinatorial methods for input space coverage measurement, which also has applications to explainable artificial intelligence. The ideas are evolving rapidly and likely to lead to i...

This report summarizes a roundtable panel discussion held at the Second Annual IEEE Workshop on Assured Autonomy, AI, and Machine Learning. Eight expert panelists discussed ways to ensure that artificial intelligence and machine learning systems are safe.

Emerging technologies typically lead to disruptive innovation and catalyze transformation in how businesses compete, how society evolves, and how people live. This special issue focuses on new emerging technologies and exciting opportunities for technology-enabled transformation.

Simulation is a useful and effective way to analyze and study complex, real-world systems. It allows researchers, practitioners, and decision makers to make sense of the inner working of a system that involves many factors often resulting in some sort of emergent behavior. The number of parameter value combinations grows exponentially and it quickl...

Distributed ledger technology (DLT), including blockchain, has a number of properties that make it useful for distributed systems. However, the immutability of blockchain and most forms of DLT make it impossible to delete data, as is required for compliance with many privacy rules regarding personally identifiable information. Thus, there is a nee...

Industry continues to be challenged when attempting to share data among organizations, especially when the data comes from different database management systems (DBMS) and different DBMS schemas. Another concern is that privacy laws may require some types of data to be protected under local access policies. We describe a secure data sharing solutio...

We briefly review properties of explainable AI proposed by various researchers. We take a structural approach to the problem of explainable AI, examine the feasibility of these aspects and extend them where appropriate. Afterwards, we review combinatorial methods for explainable AI which are based on combinatorial testing-based approaches to fault...

With the broader adoption of AI-enabled software systems, it is necessary to provide assurance to the layman user that the AI system will behave as intended. This interactive tutorial will provide an overview of AI assurance, introduce a new set of assurance goals for AI systems, discuss the open challenges in AI assurance, and present recommendati...

Testing Internet of Things (IoT) systems is challenging. This is not only because of the various aspects of IoT systems, such as software, hardware, and network that need to be tested, but also because of the unexpected issues caused by a large number of heterogeneous devices brought together by IoT systems. When an IoT system has hundreds, or even...

Wide-scale adoption of intelligent algorithms requires artificial intelligence (AI) engineers to provide assurances that an algorithm will perform as intended. In this article, we discuss the formalization of important aspects of AI assurance, including its key components.

In this paper, we report on applying combinatorial testing to Internet of Things (IoT) home automation hub systems. We detail how to create a dedicated input parameter model of an IoT home automation hub system for use with combinatorial test case generation strategies. Further, we developed an automated test execution framework and two test oracle...

In the twenty-first century, our life will increasingly depend on software-based products and complex interconnected systems. Thus, the quality and security of software-based systems is a world-wide concern. Combinatorial testing is a versatile methodology for finding errors (bugs) and vulnerabilities in software-based systems. This paper offers a...

A correction to this paper has been published: https://doi.org/10.1007/s11786-021-00502-7

We propose a two-step TLS-based fingerprinting approach using combinatorial sequences and properties of TLS handshake messages. Our approach combines fingerprinting based on attributes of the initial ClientHello message with the observed behavior of TLS clients when presented with permuted handshake messages in order to enhance the granularity of t...

Explainable artificial intelligence (XAI) is a crucially important aspect of improving trust and reliability of artificial intelligent and machine learning (AI/ML) systems. In this special issue, we have included a diverse group of articles addressing different aspects of XAI in a variety of domains. From real-time systems to human-in-the-loop faul...

The adequacy of a test suite is typically assessed with respect to a criterion such as, for example, requirements coverage or code coverage. This paper presents a metric for the adequacy of a test suite with respect to the modeled test space. Most failures in real-life systems involve only a few parameters. So, a useful criterion for the coverage o...

Recent advancements in the field of deep learning have enabled its application in Autonomous Driving Systems (ADS). A Deep Neural Network (DNN) model is often used to perform tasks such as pedestrian detection, object detection, and steering control in ADS. Unfortunately, DNN models could exhibit incorrect or unexpected behavior in real-world scena...

Machine Learning (ML) models, a core component to artificial intelligence systems, often come as a black box to the user, leading to the problem of interpretability. Explainable Artificial Intelligence (XAI) is key to providing confidence and trustworthiness for machine learning-based software systems. We observe a fundamental connection between XA...

Combinatorial testing typically considers a single input model and creates a single test set that achieves $t$ -way coverage. This paper addresses the problem of combinatorial test generation for multiple input models with shared parameters. We formally define the problem and propose an efficient approach to generating multiple test sets, one for...

Sharing data between organizations is difficult due to different database management systems imposing different schemas as well as security and privacy concerns. We leverage two proven NIST technologies to address the problem: Next Generation Database Access Control and the data block matrix.

There is an opportunity to solve the database sharing problem of clinical trial data, while protecting proprietary, PII and other sensitive data through the integration of two proven NIST technologies: Next Generation Database Access Control (NDAC) and the data block matrix.

Understanding fault types can lead to novel approaches to debugging and runtime verification. Dealing with complex faults, particularly in the challenging area of embedded systems, craves for more powerful tools, which are now becoming available to engineers.

Embedded systems are becoming ubiquitous companions in all our lives. This article reviews the terminology and modern understanding of complex anomalies and state-of-the-art debugging. It details sophisticated omniscient debugging and runtime verification and describes a novel technique to combine the benefits of those processes.

Studies have shown that combinatorial testing (CT) can be effective for detecting faults in software systems. By focusing on the interactions between different factors of a system, CT shows its potential for detecting faults, especially those that can be revealed only by the specific combinations of values of multiple factors (multi-factor faults)....

Testing is the most commonly used approach for software assurance, yet it remains as much judgment and art as science. We suggest that structural coverage measures must be supplemented with measures of input space coverage, providing a means of verifying that an adequate input model has been defined.

There are multiple options for communication of data to and from mobile sensors. For tracking systems, Global Navigation Satellite System (GNSS) is often used for localization and mobile-phone technologies are used for transmission of data. Low-power wide area networks (LPWAN) is a newer option for sensor networks including mobile sensors. We devel...

Presents the views of five experts who examine distributed ledger technology and blockchain, discussing their status in terms of adoption and success.

We present a combinatorial coverage measurement analysis for test vectors provided by the NIST Cryptographic Algorithm Validation Program (CAVP), and in particular for test vectors targeting the AES block ciphers for different key sizes and cryptographic modes of operation. These test vectors are measured and analyzed using a combinatorial approach...

In this paper, we report on the applicability of combinatorial sequence testing methods to the problem of fingerprinting browsers based on their behavior during a TLS handshake. We created an appropriate abstract model of the TLS handshake protocol and used it to map browser behavior to a feature vector and use them to derive a distinguisher. Using...

Combinatorial interaction testing (CIT) is a well-known technique, but the industrial experience is needed to determine its effectiveness in different application domains. We present a case study introducing a unified framework for generating, executing and verifying CIT test suites, based on the open-source Avocado test framework. In addition, we...

Combinatorial interaction testing (CIT) is a well-known technique, but the industrial experience is needed to determine its effectiveness in different application domains. We present a case study introducing a unified framework for generating, executing and verifying CIT test suites, based on the open-source Avocado test framework. In addition, we...

Distributed ledger technology (DLT) offers new and unique advantages for information systems, but some of its features are not a good fit for many applications. We review the properties of DLT and show how two recently developed ideas can be used to retain its advantages while simplifying design.

Distributed ledger technology (DLT) offers new and unique advantages for information systems, but some of its features are not a good fit for many applications. We review the properties of DLT and show how two recently developed ideas can be used to retain its advantages while simplifying design.

This draft white paper identifies seventeen technical trust-related issues that may negatively impact the adoption of IoT products and services. The paper offers recommendations for mitigating or reducing the effects of these concerns while also suggesting additional areas of research regarding the subject of “IoT trust.” This document is intended...

As big data, cloud computing, grid computing, and the Internet of Things reshape current data systems and practices, IT experts are keen to harness the power of distributed systems to boost security and prevent fraud. How can these systems’ capabilities be used to improve processing without inflating risk?

Describes a data structure that provides integrity protection for distributed ledger systems while also allowing deletion of records, preserving hash-based integrity assurance that other blocks are unchanged. The datablock matrix data structure could be useful in meeting privacy requirements such as the European Union General Data Protection Regula...

Combinatorial testing has been shown to be a very effective strategy for software testing. After a failure is detected, the next task is to identify one or more faulty statements in the source code that have caused the failure. In this paper, we present a fault localization approach, called BEN, which produces a ranking of statements in terms of th...

Cryptographic hash functions are security-critical algorithms with many practical applications, notably in digital signatures. Developing an approach to test them can be particularly difficult, and bugs can remain unnoticed for many years. We revisit the National Institute of Standards and Technology hash function competition, which was used to dev...

Will our smart devices betray us? Can we trust our smart beds, pet feeders, and watches to maintain the level of privacy we want and expect? As the numbers of devices coming online reach staggering levels, serious questions must be raised about the level of cybertrust we can reasonably expect to have in our modern lifestyle.

Most security vulnerabilities result from ordinary coding errors. What does this mean for the prospects of more secure software?