Matthias Scheutz

Tufts University | Tufts · Department of Computer Science

This paper presents a novel semantics for the mA* epistemic action language that takes into consideration dynamic per-agent observability of events. Different from the original mA* semantics, the observability of events is defined locally at the level of possible worlds, giving a new method for compiling event models. Locally defined observability...

Recent work in human-robot teaming has demonstrated that when robots build and maintain “shared mental models,” the effectiveness of the whole human-robot team is overall better compared to a baseline with no shared mental models. In this work, we expand on this insight by introducing proactive behaviors in addition to shared mental models to inves...

As AI-enabled robots enter the realm of healthcare and caregiving, it is important to consider how they will address the dimensions of care and how they will interact not just with the direct receivers of assistance, but also with those who provide it (e.g., caregivers, healthcare providers, etc.). Caregiving in its best form addresses challenges i...

Robust estimation of systemic human cognitive states is critical for a variety of applications, from simply detecting inefficiencies in task assignments, to the adaptation of artificial agents behaviors to improve team performance in mixed-initiative human-machine teams. This study showed that human eye gaze, in particular, the percentage change in...

In dynamic open-world environments, agents continually face new challenges due to sudden and unpredictable novelties, hindering Task and Motion Planning (TAMP) in autonomous systems. We introduce a novel TAMP architecture that integrates symbolic planning with reinforcement learning to enable autonomous adaptation in such environments, operating wi...

For robots to become effective apprentices and collaborators, they must exhibit some level of autonomy, for example, recognizing failures and identifying ways to address them with the aid of their human teammates. In this systems paper, we present an integrated cognitive robotic architecture for a “robot apprentice” that is capable of assessing its...

Social and moral norms are a fabric for holding human societies together and helping them to function. As such they will also become a means of evaluating the performance of future human–machine systems. While machine ethics has offered various approaches to endowing machines with normative competence, from the more logic‐based to the more data‐bas...

Natural language instructions are effective at tasking autonomous robots and for teaching them new knowledge quickly. Yet, human instructors are not perfect and are likely to make mistakes at times, and will correct themselves when they notice errors in their own instructions. In this paper, we introduce a complete system for robot behaviors to han...

As human-machine teams are being considered for a variety of mixed-initiative tasks, detecting and being responsive to human cognitive states, in particular systematic cognitive states, is among the most critical capabilities for artificial systems to ensure smooth interactions with humans and high overall team performance. Various human physiologi...

Learning to detect, characterize and accommodate novelties is a challenge that agents operating in open-world domains need to address to be able to guarantee satisfactory task performance. Certain novelties (e.g., changes in environment dynamics) can interfere with the performance or prevent agents from accomplishing task goals altogether. In this...

This paper advances the theory and practice of the Domain Generalization (DG) problem in Machine Learning. Specifically, we consider the scenario where hypotheses consist of a representation mapping followed by a labeling function. Popular DG methods optimize a well-known upper bound for the risk in the unseen domain to learn both the optimal repre...

Goal-based agents need to be resilient to perturbations in the world. Existing resilience definitions emphasize maintenance-type goals and, consequently, describe how well systems can recover and return to a desirable operating state after a perturbation. An alternative formulation of resilience is required for achievement-type goals that empha...

Recent attention has been brought to robots that “disobey” or so-called “rebel” agents that might reject commands. However, any discussion of autonomous agents that “disobey” risks engaging in a potentially hazardous conflation of simply non-conforming behavior with true disobedience. The goal of this paper is to articulate a sense of what constitu...

Domain generalization (DG) is a branch of transfer learning that aims to train the learning models on several seen domains and subsequently apply these pre-trained models to other unseen (unknown but related) domains. To deal with challenging settings in DG where both data and label of the unseen domain are not available at training time, the most...

We propose a self-assessment framework which enables a robot to estimate how well it will be able to perform a known or possibly novel task. The robot simulates the task to generate a state distribution of possible outcomes and determines (1) the likelihood of overall success, (2) the most probable failure location, and (3) the expected time to tas...

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed an...

Machine ethics has sought to establish how autonomous systems could make ethically appropriate decisions in the world. While mere statistical machine learning approaches have focused on learning human preferences from observations and attempted actions, hybrid approaches to machine ethics attempt to provide more explicit guidance for robots based o...

In this paper, we propose a novel domain generalization (DG) framework based on a new upper bound to the risk on the unseen domain. Particularly, our framework proposes to jointly minimize both the covariate-shift as well as the concept-shift between the seen domains for a better performance on the unseen domain. While the proposed approach can be...

Transparent task-based communication between human instructors and robot instructees requires robots to be able to determine whether a human instruction can and should be carried out, i.e., whether the human is authorized, and whether the robot can and should do it. If the instruction is not appropriate, the robot needs to be able to reject it in a...

We propose RAPid-Learn: Learning to Recover and Plan Again, a hybrid planning and learning method, to tackle the problem of adapting to sudden and unexpected changes in an agent's environment (i.e., novelties). RAPid-Learn is designed to formulate and solve modifications to a task's Markov Decision Process (MDPs) on-the-fly and is capable of exploi...

In order for artificial agents to perform useful tasks in changing environments, they must be able to both detect and adapt to novelty. However, visual novelty detection research often only evaluates on repurposed datasets such as CIFAR-10 originally intended for object classification. This practice restricts novelties to well-framed images of dist...

As AI-enabled robots enter the realm of healthcare and caregiving, it is important to consider how they will address the dimensions of care and how they will interact not just with the direct receivers of assistance, but also with those who provide it (e.g., caregivers, healthcare providers, etc.). Caregiving in its best form addresses challenges i...

As AI-enabled robots enter the realm of healthcare and caregiving, it is important to consider how they will address the dimensions of care and how they will interact not just with the direct receivers of assistance, but also with those who provide it (e.g., caregivers, healthcare providers etc.). Caregiving in its best form addresses challenges in...

As development of robots with the ability to self-assess their proficiency for accomplishing tasks continues to grow, metrics are needed to evaluate the characteristics and performance of these robot systems and their interactions with humans. This proficiency-based human-robot interaction (HRI) use case can occur before, during, or after the perfo...

Potential applications of robots in private and public human spaces have prompted the design of so-called “social robots” that can interact with humans in social settings and potentially cause humans to attach to the robots. The focus of this paper is an analysis of possible benefits and challenges arising from such human-robot attachment as report...

Invariance principle-based methods, for example, Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG). Despite the promising theory, invariance principle-based approaches fail in common classification tasks due to the mixture of the true invariant features and the spurious invariant feature...

Understanding the spread of false or dangerous beliefs—often called misinformation or disinformation—through a population has never seemed so urgent. Network science researchers have often taken a page from epidemiologists, and modeled the spread of false beliefs as similar to how a disease spreads through a social network. However, absent from tho...

An agent’s autonomy can be viewed as the set of physically and computationally grounded algorithms that can be performed by the agent. This view leads to two useful notions related to autonomy: behavior potential and success potential, which can be used to measure of how well an agent fulfills its potential, call fulfillment. Fulfillment and succes...

As robots begin to occupy our social spaces, touch will increasingly become part of human–robot interactions. This paper examines the impact of observing a robot touch a human on trust in that robot. In three online studies, observers watched short videos of human–robot interactions and provided a series of judgments about the robot, which either d...

Environmental psychology aims to study human behavior with regard to the environment and how psychological techniques can be used to motivate behavior change. We argue that these concepts can be applied to interactive robots designed for other tasks, which then enables them to encourage sustainability behaviors in humans. We first present a literat...

Dialogue agents that interact with humans in situated environments need to manage referential ambiguity across multiple modalities and ask for help as needed. However, it is not clear what kinds of questions such agents should ask nor how the answers to such questions can be used to resolve ambiguity. To address this, we analyzed dialogue data from...

For the Domain Generalization (DG) problem where the hypotheses are composed of a common representation function followed by a labeling function, we point out a shortcoming in existing approaches that fail to explicitly optimize for a term, appearing in a well-known and widely adopted upper bound to the risk on the unseen domain, that is dependent...

The game of monopoly is an adversarial multi-agent domain where there is no fixed goal other than to be the last player solvent, There are useful subgoals like monopolizing sets of properties, and developing them. There is also a lot of randomness from dice rolls, card-draws, and adversaries' strategies. This unpredictability is made worse when unk...

Explainability has emerged as a critical AI research objective, but the breadth of proposed methods and application domains suggest that criteria for explanation vary greatly. In particular, what counts as a good explanation, and what kinds of explanation are computationally feasible, has become trickier in light of oqaque “black box” systems such...

Understanding the spread of false or dangerous beliefs - so-called mis/disinformation - through a population has never seemed so urgent to many. Network science researchers have often taken a page from epidemiologists, and modeled the spread of false beliefs as similar to how a disease spreads through a social network. However, absent from those di...

Attachment theory is a research area in psychology that has enjoyed decades of successful study, and has subsequently become explored in realms beyond that of the original infant-caregiver bonds. Now, attachment is studied in relation to pets, symbols (such as deities), objects, technologies, and notably for our purposes, robots. When we discuss at...

With robotics rapidly advancing, more effective human–robot interaction is increasingly needed to realize the full potential of robots for society. While spoken language must be part of the solution, our ability to provide spoken language interaction capabilities is still very limited. In this article, based on the report of an interdisciplinary wo...

Real-word intelligent agents must be able to detect sudden and unexpected changes to their task environment and effectively respond to those changes in order to function properly in the long term. We thus isolate a set of perturbations that agents ought to address and demonstrate how task-agnostic perturbation detection and mitigation mechanisms ca...

In the future, artificial agents are likely to make life-and-death decisions about humans. Ordinary people are the likely arbiters of whether these decisions are morally acceptable. We summarize research on how ordinary people evaluate artificial (compared to human) agents that make life-and-death decisions. The results suggest that many people are...

Intelligent agents that are confronted with novel concepts in situated environments will need to ask their human teammates questions to learn about the physical world. To better understand this problem, we need data about asking questions in situated task-based interactions. To this end, we present the Human-Robot Dialogue Learning (HuRDL) Corpus -...

Much research effort in HRI has focused on how to enable robots to learn new skills from observations, demonstrations, and instructions. Less work, however, has focused on how skills can be corrected if they were learned incorrectly, adapted to changing circumstances, or generalized/specialized to different contexts. In this paper, a skill modifica...

Language-enabled robots with moral reasoning capabilities will inevitably face situations in which they have to respond to human commands that might violate normative principles and could cause harm to humans. We believe that it is critical for robots to be able to reject such commands. We thus address the two key challenges of when and how to reje...

Trust in human-robot interactions (HRI) is measured in two main ways: through subjective questionnaires and through behavioral tasks. To optimize measurements of trust through questionnaires, the field of HRI faces two challenges: the development of standardized measures that apply to a variety of robots with different capabilities, and the explora...

Regular irradiation of indoor environments with ultraviolet C (UVC) light has become a regular task for many indoor settings as a result of COVID-19, but current robotic systems attempting to automate it suffer from high costs and inefficient irradiation. In this paper, we propose a purpose-made inexpensive robotic platform with off-the-shelf compo...

There is a close connection between health and the quality of one’s social life. Strong social bonds are essential for health and wellbeing, but often health conditions can detrimentally affect a person’s ability to interact with others. This can become a vicious cycle resulting in further decline in health. For this reason, the social management o...

As robots begin to enter roles in which they work closely with human teammates or peers, it is critical to understand how people trust them based on how they interpret the robot’s behavior. In this paper we investigated the interplay between trust in a robot and people’s perceptions of the robot’s emotional intelligence. We used a vignette-based me...

Trust in human-robot interactions (HRI) is measured in two main ways: through subjective questionnaires and through behavioral tasks. To optimize measurements of trust through questionnaires, the field of HRI faces two challenges: the development of stan- dardized measures that apply to a variety of robots with different capabilities, and the explo...

We present a survey of investigations of human trust in robots in the recent human-robot interaction literature. The included papers are all experimental HRI studies and were published in the years 2018 or 2019. We explore how trust is defined in these papers, as well as what types of questions about trust are investigated and how trust is being ob...

Symbolic planning models allow decision-making agents to sequence actions in arbitrary ways to achieve a variety of goals in dynamic domains. However, they are typically handcrafted and tend to require precise formulations that are not robust to human error. Reinforcement learning (RL) approaches do not require such models, and instead learn domain...

This paper addresses ethical challenges posed by a robot acting as both a general type of system and a discrete, particular machine. Using the philosophical distinction between “type” and “token,” we locate type-token ambiguity within a larger field of indefinite robotic identity, which can include networked systems or multiple bodies under a singl...

Robots are increasingly embedded in human societies where they encounter human collaborators, potential adversaries, and even uninvolved by-standers. Such robots must plan to accomplish joint goals with teammates while avoiding interference from competitors, possibly utilizing bystanders to advance the robot’s goals. We propose a planning framework...

It has been claimed that a main advantage of cognitive architectures (compared to other types of specialized robotic architectures) is that they are task-general and can thus learn to perform any task as long as they have the right perceptual and action primitives. In this paper, we provide empirical evidence for this claim by directly comparing a...

We propose a novel approach to the problem of false belief revision in epistemic planning. Our state representations are pointed Kripke models with two binary relations over possible worlds: one representing agents' necessarily true knowledge, and one representing agents' possibly false beliefs. State transition functions maintain S5n properties in...

Although temporal logic has been touted as a fruitful language for specifying interpretable agent objectives, there has been little emphasis on generating explanations for agents with temporal logic objectives. In this paper, we develop an approach to generating explanations for the behavior of agents planning with several temporal logic objectives...

Assistive robots are becoming an increasingly important application platform for research in robotics, AI, and HRI, as there is a pressing need to develop systems that support the elderly and people with disabilities, with a clear path to market. Yet, what remains unclear is whether current autonomous systems are already up to the task or whether a...