Jessie Y. C. Chen

• PhD
• Senior Researcher at Army Research Laboratory

Human interactions with an Autonomous Intelligent Cyber-defense Agent (AICA) need to be systematically considered and evaluated across all stages of design and employment. Although some cyber and AI knowledgeable users will interact with AICAs at various touchpoints, the primary AICA users are frontline operators who are likely to have little to no...

This chapter reviews human factors research on agent transparency and its effects on human performance, workload, calibrated trust, and situation awareness. In particular, this review focuses on the Situation awareness-based Agent Transparency (SAT) framework and its implementations in various human-agent systems. The contexts of the studies review...

We examined the effects of communication style on human performance, workload, situation awareness, and trust in a robot in a human-robot team. In a 2 × 2 mixed factor study, participants were teamed with a simulated robot to conduct a cordon-and-search style task. Participants were assigned to a communication style (Directive vs. Non-directive; be...

The relationship of humans to robots has evolved from using tools, to controlling platforms, to supervising automated systems, to collaborating with autonomous agents. This chapter discusses the human element of the evolving man-machine symbiosis including both traditional and emerging human factors issues related to designing systems that emulate...

This chapter discusses frameworks and techniques that can be used to achieve transparent bidirectional human-agent communications. The first section describes models of transparency and related research showing the importance of insight into the agent's current actions, plans, and reasoning. We discuss visualization techniques (in the context of vi...

This book constitutes the refereed proceedings of the 13th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2021, held virtually as part of the 23rd HCI International Conference, HCII 2021, in July 2021. The total of 1276 papers and 241 posters included in the 39 HCII 2021 proceedings volumes was carefully reviewed and selecte...

Ongoing evolution of uninhabited systems and the interconnecting agent and cloud technologies reveal a rapid blurring of boundaries between the roles of humans and machines as well as the difference between the notions of uninhabited systems and other automated processes. In the future we need to focus less on supervising uninhabited systems and mo...

The eight papers in this special issue focus on agent and system transparency in human-machine systems. The concept of transparency is investigated in a variety of contexts of human agent interaction—from a single robot to multiple heterogeneous agents and swarms. Two studies examine the effects of individual and cultural differences and, based on...

Intelligent Agent-based transparency is an important tool for improving trust and reliance calibration in human-agent teaming. Further, flexibility in amount and type of transparency information provided to human collaborators may allow for the accommodation of known biases (e.g., misuse and disuse). To understand the utility of transparency manipu...

Agent transparency is an important contributor to effective human–agent teaming. However, agent transparency’s effects on human performance when the agent is unreliable have yet to be examined. This report examines how the transparency and reliability of a robotic autonomous squad member (ASM) affected a human observer’s task performance, workload,...

The primary purpose of this article is to determine the impact of a simulated agent's transparency on human performance and related variables, such as response time, workload, and trust calibration. The agent supports participants as they complete a base defense task by managing a team of heterogeneous unmanned vehicles and serves as a decision aid...

The 2 volume-set of LNCS 12190 and 12191 constitutes the refereed proceedings of the 12th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2020, which was due to be held in July 2020 as part of HCI International 2020 in Copenhagen, Denmark. The conference was held virtually due to the COVID-19 pandemic. A total of 1439 papers...

The 2 volume-set of LNCS 12190 and 12191 constitutes the refereed proceedings of the 12th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2020, which was due to be held in July 2020 as part of HCI International 2020 in Copenhagen, Denmark. The conference was held virtually due to the COVID-19 pandemic. A total of 1439 papers...

This book constitutes late breaking papers from the 22nd International Conference on Human-Computer Interaction, HCII 2020, which was held in July 2020. The conference was planned to take place in Copenhagen, Denmark, but had to change to a virtual conference mode due to the COVID-19 pandemic. From a total of 6326 submissions, a total of 1439 paper...

Agent transparency is an important contributor to human performance, situation awareness (SA), and trust in human–agent teaming. However, agent transparency's effects on human performance when the agent is unreliable have yet to be examined. This paper examined how the transparency and reliability of an autonomous robotic squad member (ASM) affecte...

ABSTRACT This article aims to investigate the Grand Challenges which arise in the current and emerging landscape of rapid technological evolution towards more intelligent interactive technologies, coupled with increased and widened societal needs, as well as individual and collective expectations that HCI, as a discipline, is called upon to address...

The report outlines various visualization techniques as the basis for enabling intelligent agent (software) transparency. We discuss visualizations as vehicles to capture the dynamics of the system being portrayed to give a human operator insight into the agent’s plans, logic, and expected outcomes (including uncertainties). Various techniques such...

This two-volume set LNCS 11574 and 11575 constitutes the refereed proceedings of the 11th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2019, held in July 2019 as part of HCI International 2019 in Orlando, FL, USA. HCII 2019 received a total of 5029 submissions, of which 1275 papers and 209 posters were accepted for publica...

This two-volume set LNCS 11574 and 11575 constitutes the refereed proceedings of the 11th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2019, held in July 2019 as part of HCI International 2019 in Orlando, FL, USA. HCII 2019 received a total of 5029 submissions, of which 1275 papers and 209 posters were accepted for publica...

Technological advances offer the promise of autonomous systems to form human-machine teams that are more capable than their individual members. Understanding the inner workings of the autonomous systems, especially as machine-learning (ML) methods are being widely applied to the design of such systems, has become increasingly challenging for the hu...

Effective collaboration between humans and agents depends on humans maintaining an appropriate understanding of and calibrated trust in the judgment of their agent counterparts. The Situation Awareness-based Agent Transparency (SAT) model was proposed to support human awareness in human–agent teams. As agents transition from tools to artificial tea...

In this Thematic Issue ‘Human-Autonomy Teaming’ of Theoretical Issues in Ergonomics Science, five U.S. military-funded research efforts are presented to discuss human factors issues in a variety of military human-autonomy teaming mission environments: dismounted infantry working with a small ground robot; intelligence analysis; human working with a...

A key element of system transparency is allowing humans to calibrate their trust in a system, given the implicit inherent uncertainty, emergent behaviors, etc. As robotic swarms progress towards real-world missions, such transparency becomes increasingly necessary in order to reduce the disuse, misuse and errors humans make when influencing and dir...

In a human–automation interaction study, automation assistance level (AL) was investigated for its effects on operator performance in a dynamic, multi–tasking environment. Participants supervised a convoy of manned and unmanned vehicles traversing a simulated environment in three AL conditions, while maintaining situation awareness and identifying...

This study focuses on the increase of situation awareness (SA) in human-agent mixed-initiative mission planning. Complex agent behavior and the failure to comprehend the agent’s proposed actions might result in a loss of SA and a decrease in trust. This study adopted the transparency strategies proposed by the SA-based Agent Transparency (SAT) mode...

This work describes and validates a concept of transparent behavior for adaptive automation in the field of military helicopter missions. The adaptive automation is implemented as a cognitive agent, to serve as an artificial co-pilot. It dynamically adjusts its level of assistance by choosing from different workload-adapted strategies of assistive...

This two-volume set LNCS 10909 and 10910 constitutes the refereed proceedings of the 10th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2018, held as part of HCI International 2018 in Las Vegas, NV, USA. HCII 2018 received a total of 4346 submissions, of which 1171 papers and 160 posters were accepted for publication after...

We examined how varying the transparency of agent reasoning affected complacent behavior, in the form of incorrect acceptances of an agent’s recommendations, in a route selection task. We were particularly interested in how participants’ eye movements might disambiguate whether the incorrect acceptances were due to complacency or incorrect informat...

To understand how the information available to an operator and the transparency of an intelligent agent’s reasoning interact to affect complacent behavior, 2 between-subjects experiments were conducted. Participants supervised a 3-vehicle convoy as it traversed a simulated environment and rerouted the convoy when needed with the assistance of an in...

We developed the user interfaces for two HRI tasking environments based on the Situation awareness-based Agent Transparency (SAT) model: dismounted infantry interacting with a ground robot (Autonomous Squad Member) and human interacting with an intelligent agent to manage a team of heterogeneous robotic vehicles (IMPACT). User testing showed that a...

Agent transparency has been proposed as a solution to the problem of facilitating operators’ situation awareness in human-robot teams. Sixty participants performed a dual monitoring task, monitoring both an intelligent, autonomous robot teammate and performing threat detection in a virtual environment. The robot displayed four different interfaces,...

This paper addresses issues related to integrating autonomy-enabled, intelligent agents into collaborative, human-machine teams. Interaction with intelligent machine agents capable of making independent, goal-directed decisions in human-machine teaming operations constitutes a major change from traditional human-machine interaction involving teleop...

Issues related to defining the Soldier’s role in future battlefields populated by autonomous systems are driving important US Army research programs. Mixed-initiative systems entailing shared decision-making between humans and intelligent software are a promising strategy that combines the advantages of human insight and autonomous control. This re...

This paper discusses two studies testing the effects of agent transparency in joint cognitive systems involving supervisory control and decision-making. Specifically, we examine the impact of agent transparency on operator performance (decision accuracy), response time, perceived workload, perceived usability of the agent, and operator trust in the...

Working with industry, universities and other government agencies, the U.S. Army Research Laboratory has been engaged in multi-year programs to understand the role of humans working with autonomous and robotic systems. The purpose of the paper is to present an overview of the research themes in order to abstract five research requirements for effec...

The purpose of this paper is to summarize display design techniques that are best suited for displaying information to support transparency of communication in autonomous systems interfaces. The principles include Ecological Interface Design, integrated displays, and pre-attentive cuing. Examples of displays from two recent experiments investigatin...

This book constitutes the refereed proceedings of the 9th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2017, held as part of HCI International 2017 in Vancouver, BC, Canada. HCII 2017 received a total of 4340 submissions, of which 1228 papers were accepted for publication after a careful reviewing process. The 45 papers pr...

This book focuses on the importance of human factors in the development of reliable and safe unmanned systems. It discusses current challenges such as how to improve perceptual and cognitive abilities of robots, develop suitable synthetic vision systems, cope with degraded reliability of unmanned systems, predict robotic behavior in case of a loss...

The present study investigated the effect of including information to support transparency, based on the Situation awareness-based Agent Transparency (SAT) model, in the interface for an autonomous agent known as the Autonomous Squad Member (ASM). In four different SAT model-based display conditions, participants used the ASM’s interface to gain in...

We evaluated the usability and utility of an unmanned vehicle management interface that was developed based on the Situation awareness–based Agent Transparency model. We sought to examine the effect of increasing levels of agent transparency on operator task performance and perceived usability of the agent. Usability and utility were assessed throu...

We examined how varying the transparency of agent reasoning affected operator workload in a route selection task, and how the differing measures of workload compared in assessing and understanding cognitive workload. Participants guided a three-vehicle convoy safely through a simulated environment of which they had a limited amount of information,...

We examined how the transparency of an agent’s reasoning affected the human operator’s complacent behavior in a military route selection task. Participants guided a three-vehicle convoy through a simulated environment in which they had a limited amount of information about their surroundings, all this while maintaining communication with command an...

Autonomous agents, both software and robotic, are becoming increasingly common. They are being used to supplement human operators in accomplishing complex tasks, often acting as collaborators or teammates. Agents can be designed to keep their human operators ‘in the loop’ by reporting information concerning their internal decision making process. T...

The U.S. Army Research Laboratory is engaged in a multi-year program focusing on the human role in supervising autonomous vehicles. We discuss this research with regard to patterns of human/intelligent agent (IA) interrelationships, and explore the dynamics of these patterns in terms of supervising multiple autonomous vehicles. The first design pat...

Objective: We used meta-analysis to assess research concerning human trust in automation to understand the foundation upon which future autonomous systems can be built. Background: Trust is increasingly important in the growing need for synergistic human-machine teaming. Thus, we expand on our previous meta-analytic foundation in the field of hu...

We developed the user interfaces for two Human-Robot Interaction (HRI) tasking environments: dismounted infantry interacting with a ground robot (Autonomous Squad Member) and human interaction with an intelligent agent to manage a team of heterogeneous robotic vehicles (IMPACT). These user interfaces were developed based on the Situation awareness-...

This study investigated how differing levels of information affected decisions human operators made in a route-selection task. Experiment 1 examined how information about resource usage/requirements affected route-selection decisions for a remotely based supervisor guiding a dismounted Soldier unit through an urban environment. Experiment 2 include...

Objective: We investigated the effects of level of agent transparency on operator performance, trust, and workload in a context of human-agent teaming for multirobot management. Background: Participants played the role of a heterogeneous unmanned vehicle (UxV) operator and were instructed to complete various missions by giving orders to UxVs thr...

The purpose of the paper is to discuss human centered design implications for shared decision making between humans and autonomous systems in complex environments. Design implications are generated based on empirical results from two research paradigms. In the first paradigm, an intelligent agent (RoboLeader) supervised multiple subordinate systems...

We discuss the Situation awareness-based Agent Transparency (SAT) model and its three levels of information to support agent transparency and operator situation awareness of the agent: agent’s current action or plan; agent’s reasoning information; and agent’s projections of future outcomes. Two studies—Autonomous Squad Member and IMPACT—were summar...

We use the Situation awareness-based Agent Transparency model as a framework to design a user interface to support agent transparency. Participants were instructed to supervise an autonomous robotic agent as it traversed simulated urban environments. During this task, participants were exposed to one of three levels of information used to support a...

Two experiments were conducted to examine the effects of level of information on human operators’ route selection decisions. Experiment 1 examined how information about resource usage/requirements affected route selection decisions for a remotely based supervisor guiding a dismounted soldier unit through an urban environment. Experiment 2 increased...

We conducted a human-in-the-loop robot simulation experiment. The effects of displaying transparency information, in the interface for an autonomous robot, on operator trust were examined. Participants were assigned to one of three transparency conditions and trust was measured prior to observing the autonomous robotic agent's progress and post obs...

The objective of the study was to investigate the effects of agent transparency on operator performance in the context of joint human-agent decision making in multi-robot management. The agent display configurations were based on the 3 levels of the situation awareness–based agent transparency model (basic-information, reasoning, and projections/un...

Studies investigating the effects of level of autonomy (LOA) on workload and performance in human-agent teams typically utilize subjective measures, but do not often incorporate physiological measures. This paper examines how well eye movement data collected in a recent experiment converges with the findings suggested by the subjective measures. Se...

Increasingly autonomous robotic systems are expected to play a vital role in aiding humans in complex and dangerous environments. It is unlikely, however, that such systems will be able to consistently operate with perfect reliability. Even less than 100% reliable systems can provide a significant benefit to humans, but this benefit will depend on...

Approved for public release; distribution is unlimited.

We examine the effects of transparency and communication modality on user trust in a human-robot interaction scenario. To better understand the emerging topic of transparency, we have varied the levels of information that a robotic teammate provided to participants and measured their trust responses. Additionally, we varied the modality (text, audi...

The human factors literature on intelligent systems was reviewed in relation to the following: efficient human supervision of multiple robots, appropriate human trust in the automated systems, maintenance of human operator's situation awareness, individual differences in human–agent (H-A) interaction, and retention of human decision authority. A nu...

The effectiveness of an active shutter-glasses stereoscopic display (SD) and a passive polarised SD was evaluated in a live robot-teleoperation task and a simulated indirect-vision driving task in various terrains. Overall, participants completed their tasks significantly faster with the SDs in three-dimensional (3D) mode than with the SDs in the b...

Approved for public release; distribution is unlimited.

The objective of this review is to extract design implications from multiyear US Army sponsored research investigating humans and autonomy. The programs covered diverse research paradigms: (a) effects of autonomy related to pedestrian safety during urban robotic missions, (b) supervision of multiple semi-autonomous robots assisted by an intelligent...

In the current experiment, we simulated a military multitasking environment and evaluated the effects of RoboLeader on the performance of human operators (i.e., vehicle commanders) who had the responsibility of supervising the plans/routes for a convoy of three vehicles while maintaining proper 360° local security around their own vehicle. We evalu...

In the current experiment, we simulated a multitasking environment and evaluated the effects of an intelligent agent, RoboLeader, on the performance of human operators who had the responsibility of managing the plans/routes for three vehicles (their own manned ground vehicle, an aerial robotic vehicle, and a ground robotic vehicle) while maintainin...

A military multitasking environment was simulated and three experiments were conducted to examine the effects of an intelligent agent, RoboLeader, on the performance of robotics operators. In the first experiment, RoboLeader was perfectly reliable; in the second experiment, RoboLeader's recommendations were manipulated to be either false-alarm pron...

A military targeting environment was simulated to examine the effects of an intelligent route-planning agent RoboLeader, which could support dynamic robot re-tasking based on battlefield developments, on the performance of robotics operators. We manipulated the level of assistance (LOAs) provided by RoboLeader as well as the presence of a visualisa...

A military multitasking environment was simulated to examine the effects of an intelligent agent, RoboLeader, on the performance of robotics operators. The participants' task was to manage a team of ground robots with the assistance of RoboLeader, an intelligent agent capable of coordinating the robots and changing their routes on the basis of batt...

In all human-robot interaction, trust is an important element to consider because the presence or absence of trust certainly impacts the ultimate outcome of that interaction. Limited research exists that delineates the development and maintenance of this trust in various operational contexts. Our own prior research has investigated theoretical and...

In all human-robot interaction, trust is an important element to consider because the presence or absence of trust certainly impacts the ultimate outcome of that interaction. Limited research exists that delineates the development and maintenance of this trust in various operational contexts. Our own prior research has investigated theoretical and...

We evaluate and quantify the effects of human, robot, and environmental factors on perceived trust in human-robot interaction (HRI). To date, reviews of trust in HRI have been qualitative or descriptive. Our quantitative review provides a fundamental empirical foundation to advance both theory and practice. Meta-analytic methods were applied to the...

We describe an approach examining multi-level collaboration challenges by integrating social, organizational, and cultural factors for human-robot teams operating in the real world. We discuss the research at three levels of social interaction: within a team, within a social environment, and within a culture. We first describe research exploring ps...

This work explores the theoretical foundations of trust which provide the framework for the development of our model of human-robot team trust. The pragmatic purpose for this model is to provide a greater understanding of the factors that facilitate the development of human operator trust in robotic teammates. We predicate the model’s structure wit...

RoboLeader is an intelligent agent that has the capabilities of coordinating a team of ground robots and revising route plans for the robots based on battlefield intelligence. Specifically, RoboLeader can support dynamic re-tasking based on battlefield developments as well as coordination between aerial and ground robots in pursuit of moving target...

Extensive US Army programs are being pursued to increase the effectiveness of unmanned vehicles for diverse missions during future combat. The following paper identified 23 human-robot interaction (HRI) guidelines related to interface design, procedural issues, individual differences and training implications based on three HRI research programs. T...

The purpose of this paper is to review research pertaining to the limitations and advantages of supervisory control for unmanned systems. We identify and discuss results showing technologies that mitigate the observed problems such as specialized interfaces, and adaptive systems. In the report, we first present an overview of definitions and import...

Robotic systems are being introduced into military echelons to extend warfighter capabilities in complex, dynamic environments. While these systems are designed to complement human capabilities (e.g., aiding in battlefield situation awareness and decision making, etc), they are often misused or disused because the user does not have an appropriate...

A military multitasking environment was simulated to examine the effects of unreliable automation on the performance of robotics operators. The main task was to manage a team of four ground robots with the assistance of RoboLeader, an intelligent agent capable of coordinating the robots and changing their routes based upon developments in the missi...

A military vehicle crew station environment was simulated and a series of three experiments was conducted to examine the workload and performance of the combined position of the gunner and robotics operator in a multitasking environment. The study also evaluated whether aided target recognition (AiTR) capabilities (delivered through tactile and/or...

In the FY09 DRI, we developed the RoboLeader intelligent agent that had the capabilities of coordinating a team of ground robots and revising route plans for the robots based on battlefield intelligence. In the current DRI, the capabilities of RoboLeader were expanded to deal more specifically with dynamic re-tasking requirements based on battlefie...

We simulated a generic military crew station and examined the workload and performance of robotics operators when interacting with a ground robot in the two modes of robotic autonomy, teleoperation or semi-autonomous. We examined the effect of autonomy and invocation strategies on performance. The operator had either full teleoperation (manual) or...

We investigated the effectiveness of RoboLeader, an intelligent agent that could help the human operator control a team of robots, for enhancing the overall human-robot teaming performance. We compared the operators' target detection performance in the 4-robot and 8-robot conditions. The results showed that the participants detected significantly f...

US Army researchers and support contractors are involved in a multi-year effort to understand the impact of human-robot interaction (HRI) and teaming for unmanned aerial vehicles (UAV) and unmanned ground vehicles (UGV) in future and current Army conflicts. The purpose of this paper is to summarize human-robotic principles derived from these progra...

A three-part experiment was conducted to investigate the usefulness of two types of 3D stereoscopic displays (SDs) for simulated indirect-vision driving (with various terrains) and live robot teleoperation. Results showed that, overall, participants completed their tasks significantly faster when they used an SD in 3D mode compared to the baseline...

A military reconnaissance environment was simulated to examine the performance of ground robotics operators who were instructed to utilise streaming video from an unmanned aerial vehicle (UAV) to navigate his/her ground robot to the locations of the targets. The effects of participants' spatial ability on their performance and workload were also in...

We developed an intelligent agent, RoboLeader, that could assist human operators in controlling a team of robots. More specifically, RoboLeader was able to help the operators with their route planning tasks. Although there were no significant differences between the RoboLeader and Baseline conditions for target detection performance, participants i...

Autonomous teams of robotic vehicles are gaining significant importance in military applications in the realm of reconnaissance and the execution of vital tasks. As teams of robotic vehicles grow in size and mission complexity, ever increasing burden is placed on the human operators charged with overseeing such operations. It is imperative that, in...

The purpose of this report is to review research pertaining to the limitations and advantages of supervisory control for unmanned vehicle (UV) systems. We identify and discuss results showing technologies that mitigate observed problems, such as specialized interfaces and adaptive systems. In the report, we first present an overview of definitions...