Aaron Steinfeld

Aaron Steinfeld
Carnegie Mellon University | CMU · Robotics Institute

Industrial and Operations Engineering, PhD
Many of my papers are posted online already. Please check Google Scholar, my website, or email me directly.

About

158
Publications
35,600
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
3,662
Citations
Additional affiliations
August 2001 - present
Carnegie Mellon University

Publications

Publications (158)
Conference Paper
This paper presents exploratory work on how the key components of transformer-based neural language models (self-attention mechanism and large-scale pre-trained models) can be leveraged to perform context retrieval, symbol manipulation, and propositional reasoning. We fine-tuned off-the-shelf GPT-2 and GPT-3 language models to simulate the proposit...
Preprint
The proposed symposium focuses understanding, modeling, and improving the efficacy of (a) communicating proficiency from human to robot and (b) communicating intent from a human to a robot. For example, how should a robot convey predicted ability on a new task? How should it report performance on a task that was just completed? How should a robot a...
Preprint
Full-text available
Recently, pedestrian behavior research has shifted towards machine learning based methods and converged on the topic of modeling pedestrian interactions. For this, a large-scale dataset that contains rich information is needed. We propose a data collection system that is portable, which facilitates accessible large-scale data collection in diverse...
Article
The human-robot interaction (HRI) community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To...
Article
Full-text available
Intelligent agents are part of the smartphones, smart speakers, and home robots that millions of us own and interact with regularly. However, little is known about if or how these agents should be embodied. Additionally, as conversations with agents move from heavily structured and transactional to more flexible and, at times, social, embodiment ma...
Conference Paper
Full-text available
Recently, transformer language models have been applied to build both task-and non-task-oriented dialogue systems. Although transformers perform well on most of the NLP tasks, they perform poorly on context retrieval and symbolic reasoning. Our work aims to address this limitation by embedding the model in an operational loop that blends both natur...
Conference Paper
Full-text available
Human-in-the-loop Machine Learning (HIL-ML) is a widely adopted paradigm for instilling human knowledge in autonomous agents. Many design choices influence the efficiency and effectiveness of such interactive learning processes, particularly the interaction type through which the human teacher may provide feedback. While different interaction types...
Conference Paper
Full-text available
Personalization of user experience has a long history of success in the HCI community. More recently the community has focused on adaptive user interfaces, supported by machine learning, that reduce interaction efforts and improves user experience by collapsing transactions and pre-filtering results. However, generally, these more recent results ha...
Conference Paper
Full-text available
Smart technologies have recently come under scrutiny for automating inequality. Given the current push towards developing and implementing smart cities policies that affect transportation systems in places like Kampala and Kigali, it is important to examine how the different modes of transportation meet the needs of diverse passengers and identify...
Preprint
This record contains the proceedings of the 2020 Workshop on Assessing, Explaining, and Conveying Robot Proficiency for Human-Robot Teaming, which was held in conjunction with the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI). This workshop was originally scheduled to occur in Cambridge, UK on March 23, but was moved to a...
Conference Paper
Full-text available
Artificial Intelligence (AI) plays an increasingly important role in improving HCI and user experience. Yet many challenges persist in designing and innovating valuable human-AI interactions. For example, AI systems can make unpredictable errors, and these errors damage UX and even lead to undesired societal impact. However, HCI routinely grapples...
Conference Paper
We describe an exploration of how kiosk-type stationary robots might provide navigation instructions for blind people. Inspired by a technique used by Orientation & Mobility experts in which a route is traced out on a person's palm, we developed five methods that supplement verbal instructions with physical movements. We explored the usability, str...
Article
While several studies note the challenges that people with disabilities face when using public transit, little work has investigated how mobile transit information apps affect accessibility. To address this gap, we recruited transit riders who are blind, who have low vision, who use mobility devices, and who have no disabilities. We asked them to u...
Conference Paper
Full-text available
The inevitable increase in real-world robot applications will, consequently, lead to more opportunities for robots to have observable failures. Although previous work has explored interaction during robot failure and discussed hypothetical danger, little is known about human reactions to actual robot behaviors involving property damage or bodily ha...
Conference Paper
Full-text available
Interactions with multiple conversational agents and social robots are becoming increasingly common. This raises new design challenges: Should agents and robots be modeled after humans, presenting their entity (i.e., social presence) as bound to a single body, or should they take advantage of non-human capabilities, such as moving their social pres...
Conference Paper
Full-text available
Clinical decision support tools (DST) promise improved health-care outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual ft as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically ge...
Preprint
There is great promise in creating effective technology experiences during situationally-induced impairments and disabilities through the combination of universal design and adaptive interfaces. We believe this combination is a powerful approach for meeting the UX needs of people with disabilities, including those which are temporary in nature. Res...
Research
Full-text available
Prior work suggests that an autonomous vehicle’s brand will have a significant impact on the trust people place in the vehicle’s autonomy (Carlson et al, 2014). This was associated with the perceived quality of the system and the competency of the engineering team. Therefore, it is important for autonomous vehicle companies to not just deliver safe...
Conference Paper
Full-text available
Autonomous vehicles have been in development for nearly thirty years and recently have begun to operate in real-world, uncontrolled settings. With such advances, more widespread research and evaluation of human interaction with autonomous vehicles (AV) is necessary. Here, we present an interview study of 32 pedestrians who have interacted with Uber...
Article
Bus fullness, namely bus crowding level, is one of the most critical metrics used to quantify public transportation service quality and consumer satisfaction. Often it is simply represented by the head count within a bus vehicle that is measured using the prevailing automatic passenger counter (APC)., Little is known, however, about how the precise...
Conference Paper
Full-text available
Machine learning (ML) plays an increasingly important role in improving a user’s experience. However, most UX practitioners face challenges in understanding ML’s capabilities or envisioning what it might be. We interviewed 13 designers who had many years of experience designing the UX of ML-enhanced products and services. We probed them to characte...
Conference Paper
The ability to communicate is crucial to leading an independent life. Unfortunately, individuals from developing communities who are deaf and hard of hearing tend to encounter difficulty communicating, due to a lack of educational resources. We present findings from a two-year deployment of Speak Up, a suite of voice-powered games to motivate speec...
Conference Paper
Social robots are becoming more prevalent in our daily environments. In these contexts, many researchers have studied how robots communicate with people. However, not enough attention has been paid to social interaction between robots and people with disabilities. In particular, little attention has been devoted to deaf and blind children interacti...
Conference Paper
This paper presents our design of an autonomous navigation system for a mobile robot that guides people who are blind and low vision in indoor settings. It begins by presenting user studies that shaped our design of the system, moves on to describing our model of human-robot coupled motion, and concludes by describing our autonomous navigation syst...
Conference Paper
Full-text available
We explored whether a robot can leverage social influences to motivate nearby bystanders to intervene and defend them from human abuse. We designed a between-subjects study where 48 participants took part in a memorization task and observed a confederate mistreating a robot both verbally and physically. The robot was either empathetic towards the p...
Article
Full-text available
Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challe...
Conference Paper
We conducted a 2x2 between-subjects experiment to examine the effects of two orientation and two gaze behaviors during group conversations for a mobile, low degree-of-freedom robot. For this experiment, we designed a novel protocol to induce changes in the robot's group and study different social contexts. In addition, we implemented a perception s...
Conference Paper
This work explores the capability of bi-directional manipulation during assistance to blind or low vision users. We describe a haptic approach that utilizes the manipulator arm to support interaction and provide navigational information supplemented with landmarks and spatial cues to users.
Conference Paper
As robots become more common in everyday life it will be increasingly important to understand how non-experts will view robot failure. In this study, we found that severity of failure seems to be tightly coupled with perceived risk to self rather than risk to the robot's task and object. We initially thought perceived severity would be tied to the...
Conference Paper
Participatory sensing systems use people and their smartphones as a sensing infrastructure, and getting people to make contributions remains a critical challenge. Little work details how system designers should combine different interactions to increase coverage of service location. Tiramisu, a participatory sensing system, invites transit riders t...
Conference Paper
Full-text available
Human-robot interaction (HRI) often relies on audio and visual channels for communication between robots and human users. While these methods have proven to be successful, the reliance on these two channels limits their effectiveness in extreme environments and for people with certain disabilities. In this paper, we present an alternative communica...
Conference Paper
Full-text available
Machine learning improves mobile user experience. Interestingly, envisioning apps with adaptive interfaces that reduce navigation and selection effort is not standard UX practice. When implementing an adaptive UI for our mobile transit app, we encountered a number of problems. Our original design did not log necessary information nor did it induce...
Conference Paper
Full-text available
Clinical decision support tools (DSTs) are computational systems that aid healthcare decision-making. While effective in labs, almost all these systems failed when they moved into clinical practice. Healthcare researchers speculated it is most likely due to a lack of user-centered HCI considerations in the design of these systems. This paper descri...
Chapter
Our research goals are to understand and model the factors that affect trust in intelligent systems across a variety of application domains. In this chapter, we present two methods that can be used to build models of trust for such systems. The first method is the use of surveys, in which large numbers of people are asked to identify and rank facto...
Conference Paper
Full-text available
Over the last two decades great advances have been made in medical decision support tools (DSTs). Interestingly, as these systems move out of labs and into clinical practice, many fail due to a lack of interaction design and the considerations for a user's context that this discipline brings. Today design researchers and practitioners are beginning...
Article
Dynamic changes can present significant challenges for visually impaired travelers to safely and independently navigate urban environments. To address these challenges, we are developing the NavPal suite of technology tools [1]. NavPal includes a dynamic guidance tool [2] in the form of a smartphone app that can provide real-time instructions based...
Conference Paper
Full-text available
Robots are predicted to serve in environments in which human honesty is important, such as the workplace, schools, and public institutions. Can the presence of a robot facilitate honest behavior? In this paper, we describe an experimental study evaluating the effects of robot social presence on people's honesty. Participants completed a perceptual...
Conference Paper
Assistive robots can enhance the safety, efficiency, and independence of people who are blind or low vision (B/LV) during urban travel. However, a clear understanding is still lacking in how best to introduce and describe an assistive robot to B/LV persons in a way that facilitates effective human-robot interaction. The goal of this study was to un...
Conference Paper
We present initial findings from an experiment in which participants played Mafia, an established role-playing game, with our robot. In one condition, the robot played like the rest of the participants and, in the other, the robot moderated the game. We discuss general aspects of the interaction, participants' perceptions, and the potential of this...
Conference Paper
Crowdsourced mobile sensing systems provide a counterpoint to the idea of fully automated sensing systems by transferring some or all of the sensing duties to the end users. Humans can easily sense in some ways that are impossible for machines to sense, leading to hybrid crowdsourced-automated systems. However, this transfer of sensing to humans co...
Article
We propose an assisted photography framework to help visually impaired users properly aim a camera and evaluate our implementation in the context of documenting public transportation accessibility. Our framework integrates user interaction during the image capturing process to help users take better pictures in real time. We use an image compositio...
Conference Paper
In this study, we explored the impact of a co-located sidekick on child-robot interaction. We examined child behaviors while interacting with an expressive furniture robot and his robot lamp sidekick. The results showed that the presence of a sidekick did not alter child proximity, but did increase attention to spoken elements of the interaction. T...
Conference Paper
Participatory sensing systems (PSS) require frequent injection of information that has a short shelf-life. The use of crowds to gather information for PSS is therefore particularly challenging. In this study, we explore the impact of two policies on user contributions. A quid-pro-quo policy exchanges contributions from users for access to critical...
Conference Paper
Many mobile applications rely on location information gained from location services on mobile devices. However, continuously tracking the device location with high accuracy drains the battery quickly. Furthermore, sensing the same location can be redundant when multiple devices are co-located. In this paper, we develop a crowdsourcing-based locatio...
Conference Paper
When remote robots operate in unstructured environments, they are typically controlled or monitored by an operator, depending upon the available autonomy levels and the current level of reliability for the available autonomy. When multiple autonomy modes are available, the operator must determine a control allocation strategy. We conducted two sets...
Conference Paper
Trust in automation plays a crucial role in human-robot interaction and usually varies during interactions. In scenarios of shared control, the ideal pattern is for the user's real-time trust in the robot to align with robot performance. This should lead to an increased overall efficiency of the system by limiting under-trust and over-trust. Howeve...
Conference Paper
Prior work in human trust of autonomous robots suggests the timing of reliability drops impact trust and control allocation strategies. However, trust is traditionally measured post-run, thereby masking the real-time changes in trust, reducing sensitivity to factors like inertia, and subjecting the measure to biases like the primacy-recency effect....
Article
The detection and tracking of moving objects is an essential task in robotics. The CMU-RI Navlab group has developed such a system that uses a laser scanner as its primary sensor. We will describe our algorithm and its use in several applications. Our system worked successfully on indoor and outdoor platforms and with several different kinds and co...
Conference Paper
We evaluate three interaction modes to assist visually impaired users during the camera aiming process: speech, tone, and silent feedback. Our main assumption is that users are able to spatially localize what they want to photograph, and roughly aim the camera in the appropriate direction. Thus, small camera motions are sufficient for obtaining a g...
Conference Paper
It is challenging to quantitatively measure a user's trust in a robot system using traditional survey methods due to their invasiveness and tendency to disrupt the flow of operation. Therefore, we analyzed data from an existing experiment to identify measures which (1) have face validity for measuring trust and (2) align with the collected post-run...
Conference Paper
Prior work in human-autonomy interaction has focused on plant systems that operate in highly structured environments. In contrast, many human-robot interaction (HRI) tasks are dynamic and unstructured, occurring in the open world. It is our belief that methods developed for the measurement and modeling of trust in traditional automation need altera...
Article
This study investigates travel tendencies among people with disabilities and senior citizens in New Delhi, India to reveal relationships between the desire for transportation independence, use of assistive technology for mobility, travel frequency, and satisfaction with available modes of transportation. Study volunteers received invitations to com...