T. Fong’s research while affiliated with Mountain View College and other places

Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to fail. In this work, we present an approach where the registration system attempts to automatically determine the object model, pose, and articulation for user-selected points using a nonlinear iterative closest point algorithm. When the automated fitting is incorrect, the operator can iteratively intervene with corrections after which the system will refit the object. We present an implementation of our fitting procedure and evaluate it with a user study that shows that it can improve user performance, in measures of time on task and task load, ease of use, and usefulness compared to a manual registration approach. We also present a situated example that demonstrates the integration of our method in an end-to-end system for articulating a remote valve.

Affordance Templates (ATs) are a method for parameterizing objects for autonomous robot manipulations. In this approach, instances of an object are registered by positioning a model in a 3D environment, which requires a large amount of user input. We instead propose a registration method which combines autonomy and user corrections. For selected objects, the system determines both the model and corresponding pose autonomously. The user makes corrections only if the model or pose is incorrect. This method increases the level of autonomy compared to existing approaches which can reduce user input and time on task. In this paper, we present an overview of existing methods, a description of our method, preliminary results, and planned future work.

Purpose of Review This review provides an overview of the motivation, challenges, state-of-the-art, and recent research for human-robot interaction (HRI) in space. For context, we focus on NASA space missions, use cases, and systems (both flight and research). However, the discussion is broadly applicable to all activities in space that require or make use of human-robot teams. Recent Findings To date, HRI in space has largely been limited to remote interaction between humans on Earth and robots in space. This interaction is associated with telerobotic operations—from direct (manual) control to intermittent, supervisory control. Recent work, however, has begun to address a wide range of human-robot arrangements (co-located, remote, 1:1, groups, etc.). In addition, researchers have been studying human-robot teaming theory and system design, efficient interaction methods, and human-robot communication. Summary We begin by describing NASA’s use of robots in space for both deep space science and human exploration. We then describe several aspects of HRI that are important for space missions, with emphasis on factors that are critical or unique for the space environment. Next, we provide a brief overview of HRI associated with space systems, including technology demonstrations. Finally, we conclude with a short survey of recent research, which will affect human-robot interaction for both Artemis missions and future missions to Mars.

Our goal is to enable robots to communicate information about their internal state using expressive light signals. Since nonverbal cues are typically given context by a robot’s other actions, we combined light signals, varying their color, pattern, and frequency, with robot base motion and investigated the effects of these nonverbal signals on human observers’ attribution of certain task-related properties (e.g., urgency and safety) to the robot. The results show that variations in light signal parameters affect human attribution of common notification properties. Using these findings, we present a first step towards systematically generating new light signals that can convey different state-related properties relevant for human-robot interaction, validated in a video-based study.

Robotic science missions in remote environments, such as deep ocean and outer space, can involve studying phenomena that cannot directly be observed using on-board sensors but must be deduced by combining measurements of correlated variables with domain knowledge. Traditionally, in such missions, robots passively gather data along prescribed paths, while inference, path planning, and other high level decision making is largely performed by a supervisory science team. However, communication constraints hinder these processes, and hence the rate of scientific progress. This paper presents an active perception approach that aims to reduce robots' reliance on human supervision and improve science productivity by encoding scientists' domain knowledge and decision making process on-board. We use Bayesian networks to compactly model critical aspects of scientific knowledge while remaining robust to observation and modeling uncertainty. We then formulate path planning and sensor scheduling as an information gain maximization problem, and propose a sampling-based solution based on Monte Carlo tree search to plan informative sensing actions which exploit the knowledge encoded in the network. The computational complexity of our framework does not grow with the number of observations taken and allows long horizon planning in an anytime manner, making it highly applicable to field robotics. Simulation results show statistically significant performance improvements over baseline methods, and we validate the practicality of our approach through both hardware experiments and simulated experiments with field data gathered during the NASA Mojave Volatiles Prospector science expedition.

The Atacama rover astrobiology drilling studies (ARADS) project is iteratively developing a simulated Mars rover biomarker-detection mission over four successive field deployments at the Atacama Desert Mars-analog site in 2015–2019. The second ARADS deployment in February 2017 gathered ground-truth samples, studied biomarker distributions, tested an integrated 2 m sampling drill and sample transfer arm on the KREX2 rover, and tested four astrobiology instruments at sites near Yungay Station in the Atacama Desert.

Flawless operation of mobility systems, excavation, mining and in situ resource utilization (ISRU) operations, regolith transport, and many others critically depend on knowledge of the geotechnical properties of the soil. This paper presents Stinger, a geotechnical instrument design for robotic and astronaut deployment. Stinger combines an Apollo-based penetrometer approach for measuring bearing strength with a Lunokhod approach for measuring shear strength. The shear vane is initially housed inside the cone and is pushed out whenever shear tests are required. When the shear vane is out, the cone-vane is rotated to measure the shear strength of the soil. The test data from the Stinger breadboard shows a very good correlation with a stand-alone cone penetrometer and shear vane data.

Selecting suitable landing sites is fundamental to achieving success in robotic lander missions. However, due to sensing limitations, landing sites that are both safe and scientifically suitably may not be determined reliably from orbit prior to descent, especially where orbital sensing data is noisy or incomplete. In previous work we proposed an algorithm which allows landers to autonomously plan informative descent trajectories, exploiting information gathered during descent to land on high quality sites. In this paper, we improve the scalability of our approach through importance sampling. Our method substantially reduces planning times without sacrificing the quality of the selected landing sites.