Kori Inkpen

Microsoft

Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user’s domain expertise, mental models of an AI system, trust in recommendations, and more. This paper reports on a study that examines users’ inte...

Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. This work examines users' interaction with three simula...

In AI-assisted decision-making, effective hybrid (human-AI) teamwork is not solely dependent on AI performance alone, but also on its impact on human decision-making. While prior work studies the effects of model accuracy on humans, we endeavour here to investigate the complex dynamics of how both a model's predictive performance and bias may trans...

Details of the designs and mechanisms in support of human-AI collaboration must be considered in the real-world fielding of AI technologies. A critical aspect of interaction design for AI-assisted human decision making are policies about the display and sequencing of AI inferences within larger decision-making workflows. We have a poor understandin...

In AI-assisted decision-making, effective hybrid (human-AI) teamwork is not solely dependent on AI performance alone, but also on its impact on human decision-making. While prior work studies the effects of model accuracy on humans, we endeavour here to investigate the complex dynamics of how both a model's predictive performance and bias may trans...

Every dataset is flawed, often in surprising ways that data scientists might not anticipate. However, popular machine learning methods are mostly black-boxes. Due to their lack of interpretability, they might learn defective knowledge from these datasets, which can be difficult to detect. In this work, we show how interpretable machine learning met...

Understanding how racial information impacts human decision making in online systems is critical in today's world. Prior work revealed that race information of criminal defendants, when presented as a text field, had no significant impact on users' judgements of recidivism. We replicated and extended this work to explore how and when race informati...

Although systematic biases in decision-making are widely documented, the ways in which they emerge from different sources is less understood. We present a controlled experimental platform to study gender bias in hiring by decoupling the effect of world distribution (the gender breakdown of candidates in a specific profession) from bias in human dec...

Although systematic biases in decision-making are widely documented, the ways in which they emerge from different sources is less understood. We present a controlled experimental platform to study gender bias in hiring by decoupling the effect of world distribution (the gender breakdown of candidates in a specific profession) from bias in human dec...

This document presents the motivation, objectives and target audience, theme, submissions, structure as well as the expected outcome.

Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing a...

In recent years, AI systems have become both more powerful and increasingly promising for integration in a variety of application areas. Attention has also been called to the social challenges these systems bring, particularly in how they might fail or even actively disadvantage marginalised social groups, or how their opacity might make them diffi...

This document presents the motivation, objectives and target audience, theme, submissions, structure as well as the expected outcome.

Video conferencing is often used to connect geographically distributed users and can be used to help them engage in a shared activity in a mobile setting for remote collaboration. One key limitation of today's systems however is that they typically only provide one camera view, which often requires users to hold a mobile phone during the activity....

We developed a prototype which overlays local and remote participants in a video call and enables them to take group pictures together. These pictures serve as keepsakes of the event. The application uses real-time chroma key background removal to composite the remote person into the scene with the local group. We tested the prototype in a museum s...

When might human input help (or not) when assessing risk in fairness-related domains? Dressel and Farid asked Mechanical Turk workers to evaluate a subset of individuals in the ProPublica COMPAS data set for risk of recidivism, and concluded that COMPAS predictions were no more accurate or fair than predictions made by humans. We delve deeper into...

Although technology is evolving to connect people in new and exciting ways, it still falls short of truly embracing the power of human connection. And for kids, that gap is all too evident. It is time to take a step back and let children inspire us to create technology that supports truly magical experiences. Designing "for" kids will disrupt the t...

We designed and developed a "crowdcasting" prototype to enable remote people to participate in a live event through a collection of live streams coming from the event. Viewers could select from a choice of streams and interact with the streamer and other viewers through text comments and heart reactions. We deployed the prototype in three live even...

Live streaming services are a growing form of social media. Most live streaming platforms allow viewers to communicate with each other and the broadcaster via a text chat. However, interaction in a text chat does not work well with too many users. Existing techniques to make text chat work with a larger number of participants often limit who can pa...

The emergence of mobile live streaming as a popular app over the past couple years provides a new platform for socio-technical interaction in the world. This panel brings together several different perspectives on live streaming: companies who are producing commercial apps, researchers studying how people are using live streaming, and users. We hop...

Live streaming is becoming a popular way to share experiences with others. When concurrent video streams of the same event are available, it can be challenging to decide which streams to view and to manage the experience across multiple streams. We designed and developed the SocialStreamViewer prototype that aggregates multiple streams from an even...

Live streaming has recently emerged as a growing form of participatory social media. While current live streaming practice focuses on single stream experiences, there are increasing instances of events covered by multiple live streams. In order to explore how to support communication and participation in multi-stream experiences, we present the des...

We conducted a mixed methods study of the use of the Meerkat and Periscope apps for live streaming video and audio broadcasts from a mobile device. We crowdsourced a task to describe the content, setting, and other characteristics of 767 live streams. We also interviewed 20 frequent streamers to explore their motivations and experiences. Together,...

People can have experiences through video calls that are otherwise inaccessible. For example, someone who cannot leave the home may wish to experience visiting the zoo. We present TeleTourist, a system that uses video calls with strangers to share experiences for people with mobility restrictions. We designed TeleTourist to enhance immersion and pe...

Developing new prototypes for supporting personal communication raises many challenges when studying them in actual use. Personal communication tools are often used only with specific people (i.e., close friends and relatives) and often rely on the context of the home or other locations of personal interest. Thus, these prototypes are not readily s...

We developed a prototype called SeeSaw that explored using reaction video and auto reply to create an engaging video messaging experience. When viewing a video message, reaction video captures a video of the viewer's reaction to share back with the message sender. After finishing viewing the video message, auto reply immediately begins recording th...

Children diagnosed with a chronic illness, such as cancer, experience a vastly different childhood than their healthy counterparts. They may struggle with accepting that they are no longer seen as "normal". We surveyed 10 children who have a chronic illness and interviewed 15 healthcare professionals and 7 parents of chronically ill children to und...

Whether in private or professional life, individuals frequently adapt the technology around them and work with what they have at hand to accomplish a certain task. In this one-day workshop, we will discuss how this form of technology appropriation is used to satisfy communication needs. Thereby, we specifically focus on technology that was not inte...

Telephone calls and videoconferencing are ubiquitous parts of everyday life. As the content of the call may extend beyond just words, people share applications and media using techniques such as screen sharing and email attachments. Little is known about the prevalence of this behavior and the benefits it can provide. We conducted a survey and a la...

Mobile videoconferencing is increasingly being used to bring remote friends or family along to an activity happening outside the home, such as shopping or visiting a tourist attraction. We explored how including contextual information of the event, in addition to audio and video of the person at the event, impacts the shared experience. We studied...

Studies have shown positive impact of video blogs (vlogs) on patient education. However, we know little on how patient-initiated vlogs shape the relationships among vloggers and viewers. We qualitatively analyzed 72 vlogs on YouTube by users diagnosed with HIV, diabetes, or cancer and 1,274 comments posted to the vlogs to understand viewers' perspe...

We present the design, and evaluation of WaaZam, a video mediated communication system designed to support creative play in customized environments. Users can interact together in virtual environments composed of digital assets layered in 3D space. The goal of the project is to support creative play and increase social engagement during video sessi...

Consumers shopping in "brick-and-mortar" (non-virtual) stores often use their mobile phones to consult with others about potential purchases. Via a survey (n = 200), we detail current practices in seeking remote shopping advice. We then consider how emerging social platforms, such as social networking sites and crowd labor markets, could offer rich...

Described are systems and user interfaces for facilitating asynchronous communication with video threads. Implementations may enable members of a team to have threaded view of video messages, from which they can capture video, record their screen, and send, receive, and reply to a video message. The screen recording feature enables team members to...

Video-mediated communication (VMC) has become a feasible way to connect people in remote places for work and play. Nevertheless, little research has been done with regard to children and VMC. In this paper, we explore the behavior of a group of children, who exchanged video messages in an informal context. In particular, this paper presents the res...

While video communication is becoming quite popular among remote friends and family, recent usage practices have been extending beyond just talking heads to remotely sharing an experience by doing an activity together. However, current video chat tools are aimed at sharing talking heads and need to be reconsidered to support remotely sharing activi...

As children’s use of technology grows, we see video as an important communication medium for children to connect with their friends and family members. This chapter describes a series of research projects focused on connecting children with their friends using video. The VideoPlaydate project explored children’s use of synchronous video conferencin...

New approaches in the use of media and technology provide the opportunity to give children's voices a space to express their unique interests. These new forms of video and audio use create a new type of engagement for children to communicate in ways that makes it relevant to their identity. We are looking to tap into the already prevalent use of th...

Health video blogs (vlogs) allow individuals with chronic illnesses to share their stories, experiences, and knowledge with the general public. Furthermore, health vlogs help in creating a connection between the vlogger and the viewers. In this work, we present a qualitative study examining the various methods that health vloggers use to establish...

Video communication is moving beyond face-to-face discussions on desktop computers to sharing experiences out in the real world. We explored how mobile video could enable distributed family members to share experiences wherever they occurred - kids' sporting events, birthday parties, etc. We investigated how people used two technology probes to sha...

We conducted a study comparing avatar conferencing with video and audio conferencing for work scenarios. We studied nine four-person teams using a within-subjects design that measured users' perceptions and preferences across the conferencing conditions. Video was rated highest in all measures. Avatar and Audio were rated similarly, except for soci...

Visual analytics involves complex analytical processes that can often benefit from collaboration. Many researchers have explored co-located synchronous systems to help support collaborative visual analytics; however, the process can often be long and require a series of sessions. Providing support for asynchronous collaboration in visual analytics...

Work teams are often geographically distributed, and in some cases, experience large time-zone differences with no overlap in working hours. We explored the use of asynchronous video in temporally distributed teams. We developed VideoThreads, which provides a novel threadbased visualization of video messages. Based on a deployment to four teams, we...

Task and reference spaces are important communication channels for remote collaboration. However, all existing systems for sharing these spaces have an inherent weakness: they cannot share arbitrary physical and digital objects on arbitrary surfaces. We present IllumiShare, a new cost-effective, light-weight device that solves this issue. It both s...

Consumer-based synchronous video communication is on the rise and is viewed as a valuable medium to support long distance relationships. We were interested in the potential of asynchronous video to augment children's close friendships and what types of activities they would engage in using video. We explored both of these concepts through a 9-week...

Time Travel Proxy (TTP) enables participating in meetings that you cannot attend in real time, either because of time conflicts or global time zone differences. TTP uses lightweight video recordings to pre-record your contributions to a meeting, which are played on a tablet that serves as a proxy for you during the meeting. Reactions and responses...

This paper presents two studies conducted to eliminate possible confounds in previous work (Wallace, Mandryk and Inkpen, 2008). The first evaluates interaction with 2D applications on near and far displays based on performance, perceived workload and subjective measures. A constant field of view and resolution is maintained throughout the evaluatio...

Task and reference spaces are important channels of communication for remote collaboration. A number of systems exist for sharing these spaces, but all of them have an inherent flaw: none can share arbitrary physical and digital objects on arbitrary surfaces. We have created PixIO, a new cost-effective, light-weight peripheral device that solves th...

Co-located collaboration can be extremely valuable during complex visual analytics tasks. We present an exploratory study of a system designed to support collaborative visual analysis tasks on a digital tabletop display. Fifteen participant pairs employed Cambiera, a visual analytics system, to solve a problem involving 240 digital documents. Our a...

Video-mediated communication (VMC) has become a feasible way to connect people in remote places for work and play. Nevertheless, little research has been done with regard to children and VMC. In this paper, we explore the behavior of a group of children, who exchanged video messages in an informal context. In particular, we have analyzed 386 videos...

Compared to collocated interaction, videoconferencing disrupts the ability to use gaze and gestures to mediate interaction, direct reactions to specific people, and provide a sense of presence for the satellite (i.e., remote) participant. We developed a kinetic videoconferencing proxy with a swiveling display screen to indicate which direction that...

Pen pal programs for connecting students from around the world through letter writing have been popular for generations. However, traditional technologies have several limitations in supporting pen pal activities. In this study, we explored the potential of video-based asynchronous messaging in supporting the development of children’s cross-cultura...

People sometimes miss small parts of meetings and need to quickly catch up without disrupting the rest of the meeting. We developed an Accelerated Instant Replay (AIR) Conferencing system for videoconferencing that enables users to catch up on missed content while the meeting is ongoing. AIR can replay parts of the conference using four different m...

To understand how people might use a speech dialog system in the public areas of their homes, we conducted an exploratory field study in six households. For two weeks each household used a system that logged motion and usage data, recorded speech diary entries and used Experience Sampling Methodology (ESM) to prompt participants for additional exam...

The presence of computers in schools has grown tremendously over the last ten years. In the wake of this enormous growth, sound research onhowto e ectively design learning environments and successfully integrate computers into the classroom is needed. The research described in this dissertation evaluates computer-based collaborative learning enviro...

This paper describes results from a large-scale survey to explore users' comfort with different styles of avatars for workplace communication. Thirty-one avatars were evaluated based on users' ratings along several dimensions and grouped into five different clusters. The highest rated cluster was the set of formal, realistic avatars that users did...

We conducted interviews with sixteen members of teams that worked across global time zone differences. Despite time zone differences of about eight hours, collaborators still found time to synchronously meet. The interviews identified the diverse strategies teams used to find time windows to interact, which often included times outside of the norma...

Video-mediated communication (VMC) has become a feasible way to connect people in remote places for work and play. Nevertheless, little research has been done with regard to children and VMC. In this paper, we explore the behavior of a group of children, who exchanged video messages in an informal context. In particular, we have analyzed 386 videos...

Pen pal programs for connecting students from around the world through letter writing have been popular for generations. However, traditional technologies have several limitations in supporting pen pal activities. In this study, we explored the potential of video-based asynchronous messaging in supporting the development of children’s cross-cultura...

In high-end desktop videoconferencing systems, several windows compete for screen space, particularly when users also share an application. Ideally, the layout of these windows should satisfy both (a) layout guidelines for establishing a rich communication channel and (b) user preferences for window layouts. This paper presents an exploration of us...

When people attend meetings they may miss parts of the discussion if they, for example, step out to take a phone call, go to the bathroom, or have a momentary lapse in concentration. As a result, they may need to catch up on what they missed upon returning to the meeting. Asking other attendees for a recap is often disruptive. To avoid such disrupt...

Co-located collaboration can be extremely valuable during complex visual analytics tasks. This paper presents an exploratory study of a system designed to support collaborative visual analysis tasks on a digital tabletop display. Fifteen participant pairs employed Cam-biera, a visual analytics system, to solve a problem involving 240 digital docume...

We explore the design of a system for three-way collaboration over a shared visual workspace, specifically in how to support three channels of communication: person, reference, and task-space. In two studies, we explore the implications of extending designs intended for dyadic collaboration to three-person groups, and the role of each communication...

Current business conditions have given rise to distributed teams that are mostly collocated except for one remote member. These "hub-and-satellite" teams face the challenge of the satellite colleague being out-of-sight and out-of-mind. We developed a telepresence device, called an Embodied Social Proxy (ESP), which represents the satellite coworker...

We present an empirical investigation of video-mediated free play between 13 pairs of friends (ages 7 and 8). The pairs spent 10 minutes playing with each of four different prototypes we developed to support free play over videoconferencing. We coded each interaction for the types of play and the amount of social play observed. The children in our...

Location awareness can help facilitate a rendezvous of two or more persons. To further enhance the rendezvous experience, we conducted two complementary field studies to identify what information in a location-aware map application is important to rendezvous individuals (study 1) and to explore the use of autofocus, our automation technique to redu...

Previous work has demonstrated the benefits of spatial audio conferencing over monophonic when listening to a group conversation. In this paper we examined three-way distributed conversations while varying the presence of spatial video and audio. Our results demonstrate significant benefits to adding spatialized video to an audio conference. Specif...

We present an empirical investigation of video-mediated free play between 13 pairs of friends (ages 7 and 8). The pairs spent 10 minutes playing with each of four different prototypes we developed to support free play over videoconferencing. We coded each interaction for the types of play and the amount of social play observed. The children in our...

Current business conditions have given rise to a particular kind of distributed team that is mostly collocated except for one remote member. These "hub-and-satellite" teams face the challenge of leaving the satellite colleague out-of-sight and out-of-mind. We developed a telepresence device, called an Embodied Social Proxy (ESP), which represents t...

Multi-display groupware (MDG) systems, which typically comprise both public and personal displays, promise to enhance collaboration, yet little is understood about how they differ in use from single-display groupware (SDG) systems. While research has established the technical feasibility of MDG systems, evaluations have not addressed the question o...

The research presented in this paper compares user-generated and automatic graph layouts. Following the methods suggested by van Ham et al. (2008), a group of users generated graph layouts using both multi-touch interaction on a tabletop display and mouse interaction on a desktop computer. Users were asked to optimize their layout for aesthetics an...

The effectiveness of interaction with mobile devices can be impacted by handedness; however, support for handedness in the interface is rarely provided. The goal of this article is to demonstrate that handedness is a significant interface consideration that should not be overlooked. Four studies were conducted to explore left‐handed user interactio...

Designers' extensive software needs have not been adequately documented in the research literature, and are poorly supported by software. Without appropriate tools to support their needs, designers have difficulty knowing the best way to evolve the look and feel of interactive applications they are designing. In order to inform the design of new to...

We present results from a study examining how visual selection feedback and map layout impact interaction when expressing spatial queries on paper maps using handheld devices. A sequence analysis of gaze patterns indicates that efficient queries involved a progression of visual attention from the paper map to the handheld device for a city street m...

We present WIPDash, a visualization for software development teams designed to increase group awareness of work items and code base activity. WIPDash was iteratively designed by working with two development teams, using interviews, observations, and focus groups, as well as sketches of the prototype. Based on those observations and feedback, we...