A. J. Bernheim Brush's research while affiliated with Microsoft and other places

Publications (121)

Article
The pervasive availability of voice assistants may support children in finding answers to informational queries by removing the literacy requirements of text search (e.g., typing, spelling). However, most such systems are not designed for the specific needs and preferences of children and may struggle with understanding the intent of their question...
Conference Paper
The pervasive availability of voice assistants may support children in finding answers to informational queries by removing the literacy requirements of text search (e.g., typing, spelling). However, most such systems are not designed for the specific needs and preferences of children and may struggle with understanding the intent of their question...
Chapter
Lab of Things (LoT) is a flexible platform that lowers the barrier for experimental research that uses connected devices in homes and other physical spaces. LoT enables (1) easy interconnection of devices and implementation of experimental technology, (2) easy deployment and monitoring of field studies and analysis of data from experiments, and (3)...
Article
Researchers have studied pervasive computing in many domains, but few are as complex as the home. Decades of early-adopter use of smart home technologies have blazed the path and highlighted numerous research challenges. Today, the long-standing vision of pervasive computing, commonly referred to as the Internet of Things, is finally starting to co...
Chapter
In-home deployments can be used to study current behavior patterns or to evaluate prototype systems. Compared to other options such as lab studies, interviews, or surveys, in-home deployments offer unparalleled realism due to the in-situ nature of the deployment as part of the participants' normal lives. However, in-home deployments require careful...
Conference Paper
Time of use tiered pricing schedules encourage shifting electricity demand from peak to off-peak hours. Charging times for electric vehicles (EV) can be shifted into overnight hours, which are usually off-peak. EVs can also be used as energy storage devices, available during certain peak hours to power a house with electricity stored during off-pea...
Article
There are many different ways to evaluate a novel interactive system. However, placing that system into the hands of real users and allowing them to use it as they would like in their natural environments may arguably be the best approach to understand if it "really" works. This is because findings learned from user studies conducted in the lab or...
Article
Lately, much has been said about the Internet of Things and how it is going to change how we will live, work and play. Strategy Analytics forecasts that by 2020, every person on the planet will carry four connected devices [1]. The majority of this growth will be driven by the interconnection of devices, sensors, smart objects and the like, and is...
Article
Full-text available
Today, it is not uncommon to find ourselves remote from those we care about. Despite the impact of mobile and social technologies on connectedness, recent studies suggest that it could be these very technologies that exacerbate a sense of loneliness. In attempt to help people feel more connected, we designed and created BodyPods, a remotely paired...
Patent
Full-text available
The claimed subject matter relates to an architecture that can scale a content feed in terms of the amount of content delivered in order to facilitate satisfactory experiences in connection with a social network. In particular, the architecture can utilize a content feed to disseminate content between members of a network community, generally relat...
Article
People respond emotionally to other people, animals, or even objects like furniture. While current furniture is static in appearance, embedded electronics can enable furniture to change its appearance. A couch could show excitement during a party or anger when a pet scratches it. But would emotional furniture delight or annoy people? To explore the...
Article
Furniture is the building block of the spaces we inhabit. Its design and its functions shape how we use spaces, as individuals and as groups. While being an integral part of our lives, furniture is unaware of what happens around it. But what if furniture could change its appearance? What situations should it respond to? How might it communicate its...
Article
People interact with chairs frequently, making them a potential location to perform implicit health sensing that requires no additional effort by users. We surveyed 550 participants to understand how people sit in chairs and inform the design of a chair that detects heart and respiratory rate from the armrests and backrests of the chair respectivel...
Patent
Full-text available
Techniques are described for device locking with activity preservation at a specified level within a multi-level hierarchy of device states. Such locking enables a user to share a device with another user while specifying a particular level of access to the device, such as access to a particular class of applications, a specific application, or a s...
Patent
A home device management technique is described that allows a user to manage a network of distributed home devices. Special application modules, driver modules, and service exchange ports are employed. Each application modules implements some functionality in the home, preferably using one or more devices in the home distributed system. Each driver...
Conference Paper
Inspired by the availability of inexpensive connected devices-such as lights, water sensors, security cameras, power meters, and thermostats-we built the Lab of Things (LoT) platform to enable easy interaction with devices for a range of applications, including automation, security, energy management, and elder care. Our goal is to substantially lo...
Chapter
Lab of Things (LoT) is a flexible platform that lowers the barrier for experimental research that uses connected devices in homes and other physical spaces. LoT enables (1) easy interconnection of devices and implementation of experimental technology, (2) easy deployment and monitoring of field studies and analysis of data from experiments, and (3)...
Patent
Whenever an event occurs on a computing system which will accept a response from a user of the system, the system automatically determines whether or not to enable speech interaction with the system for the event response. Whenever speech interaction is enabled with the system for the event response, the system provides a notification to the user w...
Article
Emerging mobile applications that sense context are poised to delight and entertain us with timely news and events, health tracking, and social connections. Unfortunately, sensing algorithms quickly drain the phone's battery. Developers can overcome battery drain by carefully optimizing context sensing but that makes programming with context arduou...
Conference Paper
Emerging mobile applications that sense context are poised to delight and entertain us with timely news and events, health tracking, and social connections. Unfortunately, sensing algorithms quickly drain the phone's battery. Developers can overcome battery drain by carefully optimizing context sensing but that makes programming with context arduou...
Patent
Systems and methods that regulate range of access to personal information of a mobile unit's owner. The access control component can designate granularity for access levels and/or a spectrum of access modes—(as opposed to a binary choice of full access or no access at all). Such access can be based on a spectrum and/or discrete trust relationship b...
Conference Paper
HomeSys 2013 will be an inspiring, interactive, cross-disciplinary workshop for anyone conducting research into technology in homes. This includes anyone building novel systems, applications, or devices for the home, or studying existing or novel technology use in domestic settings, or anyone else with an interest in the intersection between techno...
Conference Paper
Researchers who develop new home technologies using connected devices often want to conduct large-scale field studies in homes to evaluate their technology, but conducting such studies today is extremely challenging. Inspired by the success of PlanetLab, which enabled development and evaluation of global network services, we are developing a shared...
Conference Paper
We investigate how technology usage in homes has changed with the increasing prevalence of mobile devices including Tablets and Smart Phones. We logged Internet usage from 86 Belgium households to determine their six most common Internet Activities. Next, we surveyed households about what devices they own, how they share those devices, and which de...
Conference Paper
We present the design, implementation, and evaluation of B4, a private WAN connecting Google's data centers across the planet. B4 has a number of unique characteristics: i) massive bandwidth requirements deployed to a modest number of sites, ii) elastic ...
Conference Paper
Enabling phones to infer whether they are currently in a pocket, purse or on a table facilitates a range of new interactions from placement-dependent notifications setting to preventing "pocket dialing". We collected data from 693 participants to understand where people keep their phone in different contexts and why. Using this data, we identified...
Conference Paper
In a neighborhood watch group, neighbors cooperate to prevent crime by sharing information and alerting police of suspicious activities. We propose a digital neighborhood watch (DNW) in which security cameras of individual homes work together to monitor the neighborhood. DNW could augment neighborhood watch by providing digital evidence of crime, i...
Article
Ever wondered if it was a good time to call your spouse, child, or friend? Do you sometimes just want to know if they are on their way to meet you? With Peek you can request the activity state (e.g. walking, in-vehicle, at-rest) and location of a phone when needed. Context is sent back automatically to people on the phone owner's pre-approved Peek...
Patent
Full-text available
The claimed subject matter relates to an architecture that can facilitate creation and management of an event-oriented transient network and can further manage decommission of the transient network. In particular, the architecture can construct temporary communities based upon a particular event, project, or activity; manage (e.g., filter, prioriti...
Conference Paper
We describe how two of our recent research technologies, .NET Gadgeteer and HomeOS, complement each other in enabling research using custom devices in home deployments. .NET Gadgeteer enables rapid prototyping of custom devices based on solderless assembly from a wide range of publicly available hardware modules, and managed C# software. HomeOS pro...
Article
Full-text available
Researchers who develop new home technologies using connected devices (e.g. sensors) often want to conduct large-scale field studies in homes to evaluate their technology, but conducting such studies today is quite challenging, if not impossible. Considerable custom engineering is required to ensure hardware and software prototypes work robustly, a...
Article
HCI community is particularly well situated to think about how technologies can be redesigned, leading to reductions not predicated upon a concept of individuals who make constant and active choices. To be relevant and fruitful for sustainability, it is crucial to look at the composition of energy and/or emissions for the particular context of inte...
Article
Full-text available
Most mobile phones and tablets support only two access control device states: locked and unlocked. We investigated how well all or-nothing device access control meets the need of users by interviewing 20 participants who had both a smartphone and tablet. We find all-or-nothing device access control to be a remarkably poor fit with users' preference...
Article
Full-text available
To explore the value of speech input focus for handling notifications, we built and deployed SpeechToast, an Outlook Add-in that replaces standard email notifications with a version that includes speech input commands (e.g. "open", "delete"). Notifications shown by SpeechToast have speech input focus when the audio context surrounding the computer...
Article
Most location-aware mobile applications only make use of the user's current location, but there is an opportunity for them to infer the user's future locations. We present Trajectory-Aware Search (TAS), a mobile local search application that predicts the user's destination in real-time based on location data from the current trip and shows search r...
Article
Full-text available
Bandwidth caps, a limit on the amount of data users can upload and download in a month, are common globally for both home and mobile Internet access. With caps, each bit of data consumed comes at a cost against a monthly quota or a running tab. Yet, relatively little work has considered the implications of this usage-based pricing model on the user...
Article
Full-text available
Determining who is interacting with a multi-user interactive touch display is challenging. We describe a technique for associating multi-touch interactions to individual users and their accelerometer-equipped mobile devices. Real-time device accelerometer data and depth camera-based body tracking are compared to associate each phone with a particul...
Conference Paper
Network devices for the home such as remotely controllable locks, lights, thermostats, cameras, and motion sensors are now readily available and inexpensive. In theory, this enables scenarios like remotely monitoring cameras from a smartphone or customizing climate control based on occupancy patterns. However, in practice today, such smarthome scen...
Article
LINC is an inkable tablet-based digital family cale ndar designed to overcome coordination challenges within existing family routines. As an awareness appliance, LINC can be pl aced in any home location to provide at-a-glance family calendar awareness which in turn can enable family coordination. In th is demo we will illustrate LINC's simple and f...
Article
Digital family calendars have the potential to help families coordinate, yet they must be designed to easily fit within existing routines or will simply not be used. To understand the critical factors affecting digital family calendar design, we extended LINC, an inkable family calendar, and conducted a four-week field study of its adoption and use...
Conference Paper
Full-text available
Home heating is a major factor in worldwide energy use. Our system, PreHeat, aims to more efficiently heat homes by using occupancy sensing and occupancy prediction to automatically control home heating. We deployed PreHeat in five homes, three in the US and two in the UK. In UK homes, we controlled heating on a per-room basis to enable further ene...
Conference Paper
People often search for local information (e.g., a restaurant, store, gas station, or attraction) from their mobile device. We show, via a survey of 929 mobile searchers at a large software company, that local searches tend to be highly contextual, influenced by geographic features, temporal aspects, and the searcher's social context. While locatio...
Conference Paper
Many potential pervasive computing applications could use predictions of when a person will be at a certain place. Using a survey and GPS data from 34 participants in 11 households, we develop and test algorithms for predicting when a person will be at home or away. We show that our participants’ self-reported home/away schedules are not very accur...
Conference Paper
Automatically identifying the person you are talking with using continuous audio sensing has the potential to enable many pervasive computing applications from memory assistance to annotating life logging data. However, a number of challenges, including energy efficiency and training data acquisition, must be addressed before unobtrusive audio sens...
Conference Paper
Full-text available
Visions of smart homes have long caught the attention of researchers and considerable effort has been put toward enabling home automation. However, these technologies have not been widely adopted despite being available for over three decades. To gain insight into this state of affairs, we conducted semi-structured home visits to 14 households with...
Article
Some initial insights from the experiences studying how households in South Africa use metered Internet bandwidth are reported. It is found that with the rapid spread of Internet-enabled mobile devices and wireless modems for laptop data access, more users in these regions will be connecting to the Internet via a metered connection than we might ha...
Conference Paper
Full-text available
To understand how people might use a speech dialog system in the public areas of their homes, we conducted an exploratory field study in six households. For two weeks each household used a system that logged motion and usage data, recorded speech diary entries and used Experience Sampling Methodology (ESM) to prompt participants for additional exam...
Article
Full-text available
Human Activity Recognition (HAR) has been a topic of investigation for multiple decades. Researchers have explored a variety of approaches including a service running in the background on a mobile phone, cameras in the environment, RFID readers on people, and many more. In this note, we advocate that for HAR to make the leap from interesting resear...
Conference Paper
Full-text available
We introduce activity-based navigation, which uses human activities derived from sensor data to help people navigate, in particular to retrace a "trail" previously taken by that person or another person. Such trails may include step counts, walking up/down stairs or taking elevators, compass directions, and photos taken along a user's path, in addi...
Conference Paper
Long-term personal GPS data is useful for many UbiComp services such as traffic monitoring and environmental impact assessment. However, inference attacks on such traces can reveal private information including home addresses and schedules. We asked 32 participants from 12 households to collect 2 months of GPS data, and showed it to them in visuali...
Article
Full-text available
Home heating is a major factor in worldwide energy use. We describe two experiments aimed at reducing the amount of time heating systems need to be on, without compromising occupants' comfort. The first resulted in a machine learning algorithm based on GPS data to predict when an occupant will arrive at home. The second examined how long it takes t...
Conference Paper
Despite the potential health benefits of being able to monitor and log one's food and drink intake, manually performing this task is notoriously hard. While researchers are still exploring methods of automating this process for food, less work has been done in automatically classifying beverage intake. In this paper, we present a novel method that...
Conference Paper
Full-text available
Nowadays it has become increasingly common for family members to be distributed in different time zones. These time differences pose specific challenges for communication with- in the family and result in different communication practices to cope with them. To gain an understanding of current chal- lenges and practices, we interviewed people who re...
Conference Paper
Full-text available
We argue that heterogeneity is hindering technological innovation in the home---homes differ in terms of their devices and how those devices are connected and used. To abstract these differences, we propose to develop a home-wide operating system. A HomeOS can simplify application development and let users easily add functionality by installing new...
Conference Paper
Newport is a collaborative application for sharing context (e.g. location) and content (e.g. photos and notes) during mobile phone calls. People can share during a phone call and sharing ends when the call ends. Newport also supports using a computer during a call to make it easier to share content from the phone or launch screen sharing if the cal...
Conference Paper
We present an empirical investigation of video-mediated free play between 13 pairs of friends (ages 7 and 8). The pairs spent 10 minutes playing with each of four different prototypes we developed to support free play over videoconferencing. We coded each interaction for the types of play and the amount of social play observed. The children in our...
Conference Paper
We present an empirical investigation of video-mediated free play between 13 pairs of friends (ages 7 and 8). The pairs spent 10 minutes playing with each of four different prototypes we developed to support free play over videoconferencing. We coded each interaction for the types of play and the amount of social play observed. The children in our...
Conference Paper
Despite the potential benefits of digital note taking tools, research has found that people continue to use paper for creating micronotes, informal personal notes such as reminders and to-dos. Design recommendations from formative studies suggest that "natural" input modalities such as voice and digital ink could help to overcome the drawbacks of t...
Conference Paper
Full-text available
All four of the most popular webmail providers - AOL, Google, Microsoft, and Yahoo! - rely on personal questions as the secondary authentication secrets used to reset account passwords. The security of these questions has received limited formal scrutiny, almost all of which predates webmail. We ran a user study to measure the reliability and secur...
Conference Paper
Full-text available
Although domestic computer use is increasing, most efforts to reduce energy use through improved power management have focused on computers in the workplace. We studied 20 households to better understand how people currently use power management strategies on their home computers. We saw computers in the home, particularly desktop computers, are le...
Conference Paper
Full-text available
Technology makes it possible to share many different types of information with coworkers. We conducted a large-scale survey (N=549) to better understand current sharing among coworkers, how people stay aware of collocated and remote coworkers, and whether their willingness to share different types of awareness information changes based on the locat...
Conference Paper
In this Special Interest Group (SIG) we plan to focus on discussions and activities surrounding the design of technologies to support families. Many researchers and designers study domestic routines to inform technology design, create novel interactive systems, and evaluate these systems through real world use. Bringing together researchers, design...
Conference Paper
Full-text available
Mobile phones are becoming increasingly personalized in terms of the data they store and the types of services they provide. At the same time, field studies have reported that there are a variety of situations in which it is natural for people to share their phones with others. However, most mobile phones support a binary security model that offers...
Article
Recently, interest in tabletop computing has surged, both within the research community and within the commercial sector. However, given the early stage of interactive tabletop adoption and current low availability, it is difficult to find end users with extended experience in using such devices. We conducted a survey of 58 tabletop researchers and...
Article
Full-text available
Everyday family life involves a myriad of mundane activities that need to be planned and coordinated. We describe findings from studies of 44 different families' coordination routines to understand how to best design technology to support them. We outline how a typology of calendars containing family activities is used by three different types of f...
Article
In recent years, computer and Internet technologies have broadened the ways that people can stay in touch. Through interviews with parents and grandparents, we examined how people use existing technologies to communicate and share with their extended family. While most of our participants expressed a desire for more communication and sharing with t...
Book
This book constitutes the refereed proceedings of the 7th International Conference on Pervasive Computing, Pervasive 2009, held in Nara, Japan, in May 2009. The 20 revised full papers and 7 revised short papers presented were carefully reviewed and selected from 147 initial submissions. The papers are organized in topical sections on digital displa...
Conference Paper
Knowledge workers often undertake tasks that involve a variety of information artifacts, including both paper and digital documents. In this paper, we first summarize findings from a study that illustrate some of the challenges of managing tasks that include both paper and digital content. We then introduce DocuDesk, a prototype interactive desk th...
Conference Paper
Full-text available
To better understand the potential for horizontal surfaces in day-to-day work, we conducted a field study. We collected and analyzed over a month of use data from eight participants who used horizontal displays in conjunction with their existing office computer setups. Our analysis of the system logs, observations, and interview data from the study...
Conference Paper
Full-text available
We present the results of our study of 15 working parents, and how they manage their life scheduling needs, that is, how they manage their personal and professional schedules across settings and calendaring tools. In particular, we discuss how their dual roles of parent and employee compel them to record personal information on their professional c...
Conference Paper
Full-text available
Repetitive strain injuries and ergonomics concerns have become increasingly significant health issues as a growing number of individuals frequently use computers for long periods of time. Currently, limited software mechanisms exist for managing ergonomics; the most well-known are "break-reminder" packages that schedule and recommend typing breaks....
Conference Paper
In this paper we present Family Accounts, a new user account model for shared home computers. We conducted a study with sixteen families, eight who used individual profiles at home, and eight who shared a single profile. Our results demonstrate that Family Accounts is a good compromise between a single shared profile and individual profiles for eac...
Conference Paper
Full-text available
Staying in touch with extended family members can be a challenge in part because of the time and effort required, even with the help of current technologies. To explore the value of sharing suggestions in sparking communication and facilitating sharing between extended families, we iteratively built SPARCS, a prototype that encourages frequent shar...