Abigail Sellen

• Principal Investigator at Microsoft

Generative AI (GenAI) has introduced myriad opportunities and challenges for higher education. Anticipating this potential transformation requires understanding students' contextualised practices and norms around GenAI. We conducted semi-structured interviews with 26 students and 11 educators from diverse departments across two universities. Ground...

Hybrid meetings limit inclusion for remote participants. The Hybridge experimental system provides different interfaces for remote and room endpoints, focusing on improving inclusion via shared spatiality and remote agency. In-room participants see remotes on displays around a table, and remotes see video integrated into a digital twin. Remotes can...

Building on cross-disciplinary insights to shape the future of human-AI interaction.

With the shift to hybrid meetings in work spaces, there is an increasing need to create a more inclusive hybrid meeting experience where people meeting together in a room interact with those joining remotely. This paper describes a design exploration, implementation, and evaluation of Perspectives, a novel hybrid meeting system that aimed to create...

When COVID-19 led to mandatory working from home, significant blind spots in supporting the sociality of working life—in the moment and over time—were revealed in enterprise video meetings, and these were a key factor in reports about videoconferencing fatigue. Drawing on a large study (N = 849) of one global technology company’s employees’ experie...

In March, 2007, a forum entitled HCI 2020: Human Values in a Digital Age, was held in Sanlúcar la Mayor, Spain, just outside Seville. Its purpose was to gather luminaries in computing, design, social sciences, and scientific philosophy to discuss, debate and help formulate an agenda for human-computer interaction (HCI) over the next decade and beyo...

Objectives: To examine the feasibility, reliability, granularity, and convergent validity of a video-based pairwise comparison technique that uses algorithmic support to enable automated rating of motor dysfunction in patients with multiple sclerosis (MS). Design: Feasibility and larger cross-sectional cohort study. Setting: The outpatient cli...

Purpose: Clinical ordinal rating scales of movements, e.g., the Expanded Disability Status Scale, have poor intra- and interrater reliability, are insensitive to subtle differences and result in coarse-grained ratings compared to relative comparative rating methods. We therefore established video-based setwise comparison as a fine-grained, reliable...

Supplemental material for Reference videos reduce variability of motor dysfunction assessments in multiple sclerosis

Sophisticated ubiquitous sensing systems are being used to measure motor ability in clinical settings. Intended to augment clinical decision-making, the interpretability of the machine-learning measurements underneath becomes critical to their use. We explore how visualization can support the interpretability of machine-learning measures through th...

Motor dysfunction, particularly ataxia, is one of the predominant clinical manifestations in patients with multiple sclerosis (MS). Assessment of motor dysfunction suffers from a high variability. We investigated whether the clinical rating of ataxia can be improved through the use of reference videos, covering the spectrum of severity degrees as d...

We present a body of work undertaken in response to the challenge outlined by Harper et al. in their paper, ?'What is a File?' [9]. Through a conceptual and design-led exploration of new file metaphors, we developed the 'file biography', a digital entity that encompasses the provenance of a file and allows the user to keep track of how it propagate...

In the coming years, emergency calling services in North America will begin to incorporate new modalities for reporting emergencies, including video-based calling. The challenge is that we know little of how video calling systems should be designed and what benefits or challenges video calling might bring. We conducted observations and contextual i...

Technological support for augmenting the relationship that people establish with remote places has been studied fairly little as the primary focus in telepresence studies is the connection between people. This paper addresses the design challenge for supporting "active place presence" at home. A prototype, Hole in Space, was created to explore the...

Volunteer-driven organisations curating historic documents, such as societies and charities, often work within a bring-your-own-device (BYOD) practice and their meetings are in varying situations. A recurring challenge is finding lightweight ways to enable them to share and collectively work with documents using their own devices while in situ. We...

In this paper, we present PathSync, a novel, distal and multi-user mid-air gestural technique based on the principle of rhythmic path mimicry; by replicating the movement of a screen-represented pattern with their hand, users can intuitively interact with digital objects quickly, and with a high level of accuracy. We present three studies that each...

The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak,...

Touchscreens continue to advance including progress towards sensing fingers proximal to the display. We explore this emerging pre-touch modality via a self-capacitance touchscreen that can sense multiple fingers above a mobile device, as well as grip around the screen's edges. This capability opens up many possibilities for mobile interaction. For...

A growing number of domains, including affect recognition and movement analysis, require a single, real number ground truth label capturing some property of a video clip. We term this the provision of continuum labels. Unfortunately, there is often an uncacceptable trade-off between label consistency and the efficiency of the labelling process with...

Email is far from dead; in fact the volume of messages exchanged daily, the number of accounts per user, and the number of devices on which email is accessed have been constantly growing. Most previous studies on email have focused on management and retrieval behaviour within a single account and on a single device. In this paper, we examine how pe...

Within the context of healthcare, there has been a long-standing interest in understanding the posture and movement of the human body. Gait analysis work over the years has looked to articulate the patterns and parameters of this movement both for a normal healthy body and in a range of movement-based disorders. In recent years, these efforts to un...

New multi-modal annotation tools hold the promise of bringing the benefits of face-to-face contact to remote, asynchronous interactions. One such system, RichReview++, incorporates new techniques to improve access to the embedded multimedia commentary and allows users to annotate with new modalities, like deictic gestures. We conducted a series of...

The use of depth-sensing computer vision to capture bodily movement is increasingly being exploited in healthcare. Yet, there are few descriptions of how real-world practices influence the design of such applications. To this end, we present the development and empirical evaluation of ASSESS MS, a system to support the clinical assessment of Multip...

The term “conceptual designing” refers to an activity that various practitioners already undertake, but for which we lack a clear definition. This article provides that definition and uses an example of a design concept called “Manhattan” to present how exactly this type of process happens. We define conceptual designing as a constructive framing a...

The transition from local and personally owned file-based media management to cloud-based streaming services such as Spotify and Netflix brings new opportunities for users, but also leaves gaps in their understanding and practice. In this paper we present findings from an interview study that explored early adopters' complex relationships with thei...

Gaze has been proposed as an ideal modality for supporting remote target selection. We explored the potential of integrating gaze with hand gestures for remote interaction on a large display in terms of user experience and preference. We conducted a lab study to compare interaction in a photo-sorting task using gesture only, or the combination of g...

Background Sensor-based recordings of human movements are becoming increasingly important for the assessment of motor symptoms in neurological disorders beyond rehabilitative purposes. ASSESS MS is a movement recording and analysis system being developed to automate the classification of motor dysfunction in patients with multiple sclerosis (MS) us...

Reference is increasingly made to ‘digital collections’, yet this term encompasses accumulated digital objects of varying form, purpose and value. We review social science literature on material collections and draw from in-depth interviews with 20 people in the UK in order to offer a clearer understanding of what constitutes a digital collection a...

This case study represents our efforts to investigate the uses of voice control versus gestural control in the OR. We present a system we expressly built to allow for both gestural or voice control at the choice of the surgeon. We explain our deployment of this system in the context of cardiothoracic surgery and present a vignette on how the system...

As design practice has become more integrated in HCI research, there are on-going discussions around the role of design in research. Design research may take different forms, among which 'Research for Design' and 'Research through Design'. While, by definition, these two differ in their focus and result- The first informs the creation of a design a...

This paper introduces a novel document annotation system that aims to enable the kinds of rich communication that usually only occur in face-to-face meetings. Our system, RichReview, lets users create annotations on top of digital documents using three main modalities: freeform inking, voice for narration, and deictic gestures in support of voice....

This paper presents new learning-based techniques for measuring disease progression in Multiple Sclerosis (MS) patients. Our system aims to augment conventional neurological examinations by adding quantitative evidence of disease progression. An off-the-shelf depth camera is used to image the patient at the examination, during which he/she is asked...

Methods of encryption and decryption are described which use a key associated with an event to encrypt/decrypt data associated with the event. The method of encryption comprises identifying a key associated with an event and encrypting data using the identified key. The encrypted data is then published along with details of the event.

Methods and apparatus for displaying and interacting with messages are described. Messages are displayed using one of a number of different visualization schemes provided. A user can interact with messages or change the selected visualization scheme using gestures and the mapping of gestures to particular actions is different for different visualiz...

Natural User Interfaces (NUI) offer rich ways for interacting with the digital world that make innovative use of existing human capabilities. They include and often combine different input modalities such as voice, gesture, eye gaze, body interactions, touch and touchless interactions. However much of the focus of NUI research and development has b...

Computer visions (CV) systems are increasingly finding new roles in domains such as healthcare. These collaborative settings are a new challenge for CV systems, requiring the design of appropriate interaction paradigms. The provision of feedback, particularly of what the CV system can 'see,' is a key aspect, and may not always be possible to presen...

While surgical practices are increasingly reliant on a range of digital imaging technologies, the ability for clinicians to interact and manipulate these digital representations in the operating theatre using traditional touch based interaction devices is constrained by the need to maintain sterility. To overcome these concerns with sterility, a nu...

We describe the design, implementation and deployment of Photobox, a domestic technology that prints four or five randomly selected photos from the owner's Flickr collection at random intervals each month. We deployed Photobox in three homes for fourteen months, to explore how the slow pace at which it operates could support experiences of anticipa...

Developments in computer vision technology have led to a plethora of new body tracking applications. These applications share a challenge in accounting for characteristics of the specific places in which they are intended to be used. We present the concept of Place-onas, representations of "typical" places, as a shared resource to support multidisc...

With current digital technologies, people have large archives of digital media, such as images and audio files, but there are only limited means to include these media in creative practices of crafting and making. Nevertheless, studies have shown that crafting with digital media often makes these media more cherished and that people enjoy being cre...

Planning a wedding is arguably one of the most complicated collaborative tasks people ever undertake. Despite the commonplace use of technologies in "wedding work," little research has looked at this from an HCI perspective. Based on an interview study, we illustrate how technology is used to deliver the sought-after fantasy and a practical, yet en...

Animation creation is described, for example, to enable children to create, record and play back stories. In an embodiment, one or more children are able to create animation components such as characters and backgrounds using a multi-touch panel display together with an image capture device. For example, a graphical user interface is provided at th...

Touchless interaction with medical images lets surgeons maintain sterility during surgical procedures.

A sketching and searching application for idea generation is described. In an embodiment, a software application is described which has a user interface which comprises a sketching area. When a user draws or annotates a sketch in the application, the application automatically searches for images based on the sketch and displays results in the form...

With the proliferation of digital reading technologies and their underlying ecosystem, practices of reading are currently undergoing significant changes. Despite the currency of the topic, we find there is little empirical research on how people incorporate digital reading technologies into their existing leisure-based reading practices. In this pa...

We contrast the Chameleon Lens, which uses 3D movement of a mobile device held in the nonpreferred hand to support panning and zooming, with the Pinch-Flick-Drag metaphor of directly manipulating the view using multi-touch gestures. Lens-like approaches have significant potential because they can support navigation-selection, navigation-annotation,...

Family life is complex and dynamic. It forms a core part of our existence. Underpinning family life, is family connection: how families not just communicate with each other, but how they share their lives and routines, how they engage in social touch, and how they negotiate being together, or being apart. This book explores the various ways in whic...

Retrieval and display of digital media items is described. For example, the digital media items may be photographs, videos, audio files, emails, text documents or parts of these. In an embodiment a dedicated apparatus having a touch display screen is provided in a form designed to look like a domestic fish tank. In an embodiment graphical animated...

In recent years the Web has evolved substantially, transforming from a place where we primarily find information to a place where we also leave, share and keep it. This presents a fresh set of challenges for the management of personal information, which include how to underpin greater awareness and more control over digital belongings and other per...

In this position paper we discuss 'Materialise' – a building set consisting of physical building blocks and digital media input that allows for the building of hybrid creations – as an example of a design that integrates technology in creative practice. We show it does so by facilitating interactive craft practice, aestheticizing technology, and al...

Web-based technologies are often built to capitalize on the flexibility and fluidity that is supported by the internet, with the value of 'access anywhere' underpinning a blurring of boundaries across home and work. Yet the home is well known in HCI to have a unique set of qualities that can use-fully be drawn upon when designing to support domesti...

In laboratory studies, multi-surface slate-based reading systems have shown great promise as platforms for active reading. However, the true utility of such a system can only be ascertained through the rigors of real world use. We conducted month-long deployments of a multi-slate reading system to support the active reading activities of graduate s...

Norman's critique is indicative of the issue that while using the word natural might have become natural, it is coming at a cost. In other words, precisely because the notion of naturalness has become so commonplace in the scientific lexicon of HCI, so it is becoming increasingly important, it seems that there is a critical examination of the conce...

Despite predictions of the paperless office, most knowledge workers and students still rely heavily on paper in most of their document practices. Research has shown that paper's dominance can be attributed to the fact that it supports a broad range of these users' diverse reading requirements. Our analysis of the literature suggests that a new clas...

As we go about our everyday routines we encounter and interact with numerous physical (e.g. furniture or clothes) and digital objects (e.g. photos or e-mails). Some of these objects may be particular cherished, for example because of memories attached to them. As several studies into cherished objects have shown, we have more difficulties identifyi...

We describe the design and implementation of Photobox, a device intended to be used over many years, which occasionally prints a randomly selected photo from the owner's Flickr collection inside of a wooden chest. We describe and reflect on how engaging in the design of this slow technology [5] led to some unexpected challenges and provoked us to r...

Systems that detect the unaugmented human body allow players to interact without using a physical controller. But how is interaction altered by the absence of a physical input device? What is the impact on game performance, on a player's expectation of their ability to control the game, and on their game experience? In this study, we investigate th...

Within medical settings there is a growing interest in exploring touchless interaction technologies. The primary motivation here is to avoid contact during interaction with data so as to maintain asepsis. However, there is another important property of touchless interaction that has significant implications for their use within such settings -- nam...

Material artifacts are passed down as a way of sustaining relationships and family history. However, new issues are emerging as families are increasingly left with the digital remains of their loved ones. We designed three devices to investigate how digital materials might be passed down, lived with and inherited in the future. We conducted in-home...

This paper describes a field study of an interactive surface deployed in three family homes. The tabletop technology provides a central place where digital content, such as photos, can be easily archived, managed and viewed. The tabletop affords multi-touch input, allowing digital content to be sorted, triaged and interacted with using one or two-h...

Lifelogging is a technically inspired approach that attempts to address the problem of human forgetting by developing systems that “record everything.” Uptake of lifelogging systems has generally been disappointing, however. One reason for this lack of uptake is the absence of design principles for developing digital systems to support memory. Synt...

People are amassing larger and more diverse collections of digital things. The emergence of Cloud computing has enabled people to move their personal files to online places, and create new digital things through online services. However, little is known about how this shift might shape people's orientations toward their digital things. To investiga...

This paper presents findings from a field study of 24 individuals who kept diaries of their web use, across device and location, for a period of four days. Our focus was on how the web was used for non-work purposes, with a view to understanding how this is intertwined with everyday life. While our initial aim was to update existing frameworks of '...

While electronic documents are increasingly prevalent in the workplace there are many texts — such as books, magazines and letters — which are not easily available in an electronic form. Since many electronic document systems depend upon documents existing exclusively, or at least predominantly, in electronic form, this suggests an opportunity for...

Photographic mementos are important signifiers of our personal memories. Rather than simply passive representations of memories to “preserve” the past, these photos are actively displayed and consumed in the context of everyday behavior and social practices. Within the context of these settings, these mementos are invoked in particular ways to mobi...

Artifacts play an important role as triggers for personal memory. They help in the recollection of past experience and in reminiscing about people, places, and times gone by. Of particular interest to us is one type of artifact, the heirloom, which may also have rich connections with memory, but often through the lens of the life of a deceased memb...

Family life is complex and dynamic. It forms a core part of our existence. Underpinning family life, is family connection : how families not just communicate with each other, but how they share their lives and routines, how they engage in social touch, and how they negotiate being together, or being apart. This book explores the various ways in whi...

What and who a family is, is continually changing. Family is a place, an ever changing set of social relationships, an evolving archive of precious artefacts and the actions collectively unfolding that bring all of these elements into meaningful cohesion. Over space and time familial structure shifts; it expands, contracts, solidifies and dissolves...

Family homes are replete with “stuff”—physical artifacts and collections of objects that say something about the people who live in that home, who they are, and what they are about. At the same time, increasingly, the stuff that we keep is digital, whether or not this is generated deliberately (as in photos and videos), or simply the result of our...

In this paper we explore the notion of creating personally evocative collections of content from publicly available material. Compared to the personal media that we look at, reminisce over, or personalise our offices with, public media offers the potential for a different type of nostalgia, signifiers of an era such as entertainment, products, or f...