Wendy Ju

Cornell University | CU · Information Science

The COVID-19 pandemic, travel restrictions, and social distancing measures have made it difficult to observe, monitor, or manage urban life. To capture the experience of being in New York City during the first year of the COVID-19 pandemic, we used a novel method of remote ethnography to interview people who were walking the city. We developed the...

This article explores and characterizes the pandemic urbanism of NYC during the first year of COVID-19. It analyzes changes to the sidewalk and the urban lifestyle using a novel method of remote ethnography: the integrated use of Zoom video conferencing and GPS smartphone tracking to interview participants as they walked and filmed the city. The da...

Empathic vehicles are expected to improve user experience in automated vehicles and to help increase user acceptance of technology. However, little is known about potential real-world implementations and designs using empathic interfaces in vehicles with higher levels of automation. Given advances in affect detection and emotion mitigation, we cond...

In order for autonomous vehicles to adapt to local norms in human driving, it is critical to profile how human driving differs across geographical locations. While ethnographers have qualitatively described regional differences in driving style, data-driven statistical models might help computer-driven cars drive like locals and recognize how local...

The proliferation of voice agents in consumer devices requires new tools for evaluating these systems beyond their technical functionality. This paper presents a toolkit for the evaluation of Voice User Interfaces (VUIs) with the intention of measuring the crucial factors of subjective enjoyment in the user experience. The PUEVA toolkit was constru...

The Roosevelt Island Digital Twin is a smart planning tool which combines a 3D model of the physical characteristic of the island (buildings, roads, parks, etc.), with socio-economic data and demographic information. It is a virtual representation of the island with layers of data and analysis that can be added to physical visualizations. The model...

This paper examines sensor fusion techniques for modeling opportunities for proactive speech-based in-car interfaces. We leverage the Is Now a Good Time (INAGT) dataset, which consists of automotive, physiological, and visual data collected from drivers who self-annotated responses to the question "Is now a good time?," indicating the opportunity t...

People interacting with voice assistants are often frustrated by voice assistants' frequent errors and inability to respond to backchannel cues. We introduce an open-source video dataset of 21 participants' interactions with a voice assistant, and explore the possibility of using this dataset to enable automatic error recognition to inform self-rep...

One key technique people use in conversation and collaboration is conversational repair. Self-repair is the recognition and attempted correction of one’s own mistakes. We investigate how the self-repair of errors by intelligent voice assistants affects user interaction. In a controlled human-participant study (N=101), participants asked Amazon Alex...

Travel restrictions and social distancing measures make it difficult to observe, monitor or manage physical fieldwork. We describe research in progress that applies technologies for real-time remote observation and conversation in on-road vehicles to observe field work on a farm. We collaborated on a pilot deployment of this project at Kreher Eggs...

Many producers of automated vehicle systems have begun testing autonomous vehicles on the road. In order to ensure safety and prevent crashes, human drivers are enlisted to monitor autonomous vehicles. However, operators of autonomous systems exhibit negative behavior adaptations in response to prolonged supervision of automation. To prevent the on...

The Interactions website (interactions.acm.org) hosts a stable of bloggers who share insights and observations on HCI, often challenging current practices. Each issue we'll publish selected posts from some of the leading and emerging voices in the field.

This paper provides a framework for examining human-vehicle interactions with respect to three dimensions that can involve models or simulations: the agents, the environments, and the scenarios. Agents are considered on a spectrum from human to artificial actors. Environments are considered on a spectrum from simulated to real. Scenarios are consid...

The effectiveness of social distancing as a disease-slowing measure is dependent on the degree of compliance that individuals demonstrate to such orders. In this ongoing research, we study outdoor pedestrian activity in New York City, specifically using (a) video streams gathered from public traffic cameras (b) dashcam footage from vehicles driving...

This paper presents a novel and exploratory investigation of how users might control future autonomous vehicles with user-defined in-air static hand gestures. In the era of autonomous vehicles, how to support “driving” without steering control may become a key issue affecting user experience. As the navigation interface of future autonomous cars wi...

The understanding of stress and its impact on human performance is crucial to mitigate human error in the face of a threat. This is especially the case for critical incidents on a ship bridge, where human error can easily lead to severe danger for crew, cargo, and other vessels. To overcome the current limitations of robust objective stress measure...

Drivers and pedestrians use various culturally-based non-verbal cues such as head movements, hand gestures, and eye contact when crossing roads. With the absence of a human driver, this communication becomes challenging in autonomous vehicle (AV)-pedestrian interaction. External human-machine interfaces (eHMIs) for AV-pedestrian interaction are bei...

Will automated driving help or hurt our efforts to remedy climate change? The overall impact of transportation and mobility on the global ecosystem is clear: changes to that system can greatly affect climate outcomes. The design of mobility and automotive systems will influence key factors such as driving style, fuel choice, ride sharing, traffic p...

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify...

Conventional interaction design methodologies cannot fully encompass the redefined relationships between humans and increasingly intelligent technology. New methods are necessary to address interaction at early stages in the design process. Both design metaphors and enactment techniques have been suggested, and this paper explores whether a combina...

2020 ACM. Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to...

To help both designers and people with tetraplegia fully realize the benefts of voice assistant technology, we conducted interviews with fve people with tetraplegia in the home to understand how this population currently uses voice-based interfaces as well as other technologies in their everyday tasks. We found that people with tetraplegia use voic...

Top-down simulations of autonomous intersections neglect considerations for the human experience of being in cars driving through these autonomous intersections. To understand the impact that perspective has on perception of autonomous intersections, we conducted a driving simulator experiment and studied the experience in terms of perception, feel...

With autonomous driving on the horizon, new research challenges appeared and subsequently, new methods and research instruments became necessary. To adapt to these emerging research questions, driving simulators, the cornerstone of automotive human factors research, have been tweaked, modified or developed from scratch. This one-day workshop invite...

For automated vehicles operating at SAE Level 4 capability, control could feasibly be passed from machine to human and vice versa -regardless of whether minimal risk condition exists as a fallback solution. We propose two Human-Machine Interfaces (HMIs) to assist in the management of these transitions: 1) A 'Responsibility Panel' providing the nece...

Advances in automotive sensing systems and speech interfaces provide new opportunities for smarter driving assistants or infotainment systems. For both safety and consumer satisfaction reasons, any new system which interacts with drivers must do so at appropriate times. We asked 63 drivers, ”Is now a good time?” to receive non-driving information d...

Recent research suggests that a robot's motors make sounds that can influence users' perception of the robot's characteristics. To more deeply understand users' associations with specific sonic characteristics, we adapted methods from sensory science including Check All That Apply (CATA) questions and Polarized Sensory Positioning (PSP) to tease ou...

In fields where in situ performance cannot be measured, ecological validity is difficult to estimate. Drawing on theory from social psychology and virtual reality, we argue that face validity can be a useful proxy for ecological validity. We provide illustrative examples of this relationship from work in search-and-rescue HRI, and conclude with som...

This course is a hands-on introduction to interactive electronics prototyping for people with a variety of backgrounds, including those with no prior experience in electronics. Familiarity with programming is helpful, but not required. Participants learn basic electronics, microcontroller programming and physical prototyping using the Arduino platf...

Automated driving is one of the most discussed disruptive technologies of this decade. It promises to increase drivers' safety and comfort, improve traffic flow, and lower fuel consumption. This has a significant impact on our everyday life and mobility behavior. Beyond the passengers of the vehicle, it also impacts others, for example by lowering...

Dominance is a key aspect of interpersonal relationships. To what extent do nonverbal indicators related to dominance status translate to a nonanthropomorphic robot? An experiment (N = 25) addressed whether a mobile robot's motion style can influence people's perceptions of its status. Using concepts from improv theater literature, we developed two...

Interactive systems present new opportunities for creating devices that attempt to learn the needs of people. However, inferring from data alone may not always allow for a true understanding of user needs. We suggest a vision of Social IoT where designers interact with users through machines as a new method for needfinding. We present a framework u...

We present the use of in-car virtual reality (VR) as a way to create calm, mindful experiences for passengers and, someday, autonomous vehicle occupants. Specifically, we describe a series of studies aimed at exploring appropriate VR content, understanding the influence of car movement, and determining the length and other parameters of the simulat...

The Interaction Engine is a framework for prototyping interactive, connected devices based on widely available single-board Linux computers. With microcontrollers, networking, and modular open-source software, these modules enable interaction modalities such as audio, video, tangible, and digital interfaces to be embedded into forms that go beyond...

Lack of trust can arise when people do not know what autonomous vehicles perceive in the environment. To convey this information without causing alarm or compelling people to act, we designed and evaluated a way to sonify an autonomous vehicle's perception of salient driving events using abstract auditory icons, or "earcons." These are localized in...

We propose a novel method for reliably inducing stress in drivers for the purpose of generating real-world participant data for machine learning, using both scripted in-vehicle stressor events and unscripted on-road stressors such as pedestrians and construction zones. On-road drives took place in a vehicle outfitted with an experimental display th...

In-car emotion detection and regulation have become an emerging and important branch of research within the automotive domain. Different emotional states can greatly influence human driving performance and user experience both in manual and automated driving conditions. The monitoring and regulation of relevant emotional states is therefore importa...

How will pedestrians from different regions interact with an approaching autonomous vehicle? Understanding differences in pedestrian culture and responses can help inform autonomous cars how to behave appropriately in different regional contexts. We conducted a field study comparing the behavioral response of pedestrians between metropolitan Mexico...

As technology becomes increasingly intelligent and progressively gains agency, the relationship between system and human is redefined. Conventional interaction design methodologies cannot fully encompass the emerging new types of relationships, and new methods are necessary to address interaction at early stages in the design process. Both design m...

Researchers and designers of in-vehicle interactions and interfaces currently have to choose between performing evaluation and human factors experiments in laboratory driving simulators or on-road experiments. To enjoy the benefit of customizable course design in controlled experiments with the immediacy and rich sensations of on-road driving, we h...

This course is a hands-on introduction to interactive electronics prototyping for people with a variety of backgrounds, including those with no prior experience in electronics. Familiarity with programming is helpful, but not required. Participants learn basic electronics, microcontroller programming and physical prototyping using the Arduino platf...

There is sustained and growing interest in human-robot teaming across academia and industry. Many critical questions still remain as to how to foster flexible, effective, teaming that allow humans and robots to work closely together. This panel will bring together experts on human-robot interaction (HRI) across academia and industry to discuss and...

The rise of evermore autonomy in vehicles and the expected introduction of self-driving cars have led to a focus on human interactions with such systems from an HCI perspective over the last years. Automotive User Interface researchers have been investigating issues such as transition control procedures, shared control, (over)trust, and overall use...

Motivated by the idea that slow breathing practices could transform the automobile commute from a depleting, mindless activity into a calming, mindful experience, we introduce the first guided slow breathing intervention for drivers. We describe a controlled in-lab experiment (N=24) that contrasts the effectiveness and impact of haptic and voice gu...

Autonomous robots in the home and on the road are fundamentally changing the way we live and interact. The visual expressions and interactions of these devices are well studied; however, more could be done to learn how sound could be a deliberate (or sometimes accidental) channel of communication from autonomous systems to humans. Combining enginee...

Stress affects the lives of millions of people every day. In-situ sensing could enable just-in-time stress management interventions. We present the first work to detect stress using the movements of a car's existing steering wheel. We extend prior work on PC peripherals and demonstrate that stress, expressed through muscle tension in the limbs, can...

Motivated by the idea that slow breathing practices could transform the automobile commute from a depleting, mindless activity into a calming, mindful experience, we introduce the first guided slow breathing intervention for drivers. We describe a controlled in-lab experiment (N =24) that contrasts the effectiveness and impact of haptic and voice g...

How might an actuated car seat become an expressive robot? To explore the possibilities of this novel interaction, we conducted a full design exploration from prototyping to validation, drawing on methods for embodied physical interaction design. First, we applied physical and digital puppeteering techniques to explore how a car seat can display em...

We introduce a Hidden Markov Model framework to formalize the beliefs that humans may have about the mode in which a semi-automated vehicle is operating. Previous research has identified various "levels of automation," which serve to clarify the di↵erent degrees of a vehicle's automation capabilities and expected operator involvement. However, a ve...

Background The daily commute could be a right moment to teach drivers to use movement or breath towards improving their mental health. Long commutes, the relevance of transitioning from home to work, and vice versa and the privacy of commuting by car make the commute an ideal scenario and time to perform mindful exercises safely. Whereas driving sa...

In this paper, we introduce two different transforming steering wheel systems that can be utilized to augment user experience for future partially autonomous and fully autonomous vehicles. The first one is a robotic steering wheel that can mechanically transform by using its actuators to move the various components into different positions. The sec...

This paper describes a Wizard of Oz study that was performed to gather insights on how automated vehicles (NHTSA’s Levels of Automation 2 and 3) should interact with human drivers. Twelve design improvisation sessions were conducted inside a driving simulator with interaction and interface design experts. The participants drove through a simulated...