Joel E Fischer

• Professor (Assistant) at University of Nottingham

Many of our everyday social interactions involve mobile devices. Yet, these tend to only provide good support for distributed social interactions. Although much HCI and CSCW research has explored how we might support collocated, face-to-face situations using mobile devices, much of this work exists as isolated exemplars of technical systems or inte...

In the aftermath of major disasters, first responders are typically overwhelmed with large numbers of, spatially distributed, search and rescue tasks, each with their own requirements. Moreover, responders have to operate in highly uncertain and dynamic environments where new tasks may appear and hazards may be spreading across the disaster space....

We present a field trial of how instructions from an intelligent planning agent are dealt with by distributed human teams, in a time-critical task setting created through a mixed-reality game. We conduct interaction analysis to examine video recorded field observations and game log data. The findings highlight the social process by which players in...

Large Language Models (LLMs) are seemingly infiltrating every domain, and the legal context is no exception. In this paper, we present the results of three experiments (total N = 288) that investigated lay people's willingness to act upon, and their ability to discriminate between, LLM- and lawyer-generated legal advice. In Experiment 1, participan...

While ethical challenges are widely discussed in HCI, far less is reported about the ethical processes that researchers routinely navigate. We reflect on a multispecies project that negotiated an especially complex ethical approval process. Cat Royale was an artist-led exploration of creating an artwork to engage audiences in exploring trust in aut...

Conversational user interfaces (CUIs) have become an everyday technology for people the world over, as well as a booming area of research. Advances in voice synthesis and the emergence of chatbots powered by large language models (LLMs), notably ChatGPT, have pushed CUIs to the forefront of human-computer interaction (HCI) research and practice. No...

Formal Modelling is often used as part of the design and testing process of software development to ensure that components operate within suitable bounds even in unexpected circumstances. In this paper, we use predictive formal modelling (PFM) at runtime in a human-swarm mission and show that this integration can be used to improve the performance...

ChatGPT, a chatbot using the GPT-n series large language model, has surged in popularity by providing conversation, assistance, and entertainment. This has raised questions about its agency and resulting implications on trust and blame, particularly when concerning its portrayal on social media platforms like Twitter. Understanding trust and blame...

This paper summarizes the structure and findings from the first Workshop on Troubles and Failures in Conversations between Humans and Robots. The workshop was organized to bring together a small, interdisciplinary group of researchers working on miscommunication from two complementary perspectives. One group of technology-oriented researchers was m...

Onboard virtual assistants with the ability to converse with users are gaining favour in supporting effective human-machine interaction to meet safe standards of operation in automated vehicles (AVs). Previous studies have highlighted the need to communicate situation information to effectively support the transfer of control and responsibility of...

In July 2021, the UK recorded an increase in self-isolation notifications to the users of the NHS Covid-19 contact-tracing app. The term pingdemic was coined and used widely in the media to refer to this period of increased notifications. Whilst existing research on language evolution during Covid-19 focuses on the negative connotations of the term...

In August 2020, the UK government and regulation body Ofqual replaced school examinations with automatically computed A Level grades in England and Wales. This algorithm factored in school attainment in each subject over the previous three years. Government officials initially stated that the algorithm was used to combat grade inflation. After publ...

Cat Royale is an artist-led exploration of trustworthy autonomous systems created by the TAS Hub's creative ambassadors Blast Theory. A small community of cats inhabits a purpose built `cat utopia' at the centre of which a robot arm tries to enrich their lives by playing with them. We initially present the design of Cat Royale as an autonomous syst...

BACKGROUND Since September 2020, the NHS Covid-19 contact tracing app has been used to mitigate the spread of Covid-19 in the UK. Since its launch, this app has been part of the discussion regarding the perceived social agency of decision-making algorithms. On the social media website Twitter, a plethora of views about the app have been found but o...

Background Since September 2020, the National Health Service (NHS) COVID-19 contact-tracing app has been used to mitigate the spread of COVID-19 in the United Kingdom. Since its launch, this app has been a part of the discussion regarding the perceived social agency of decision-making algorithms. On the social media website Twitter, a plethora of v...

The ‘digital plumber’ is a conceptualisation in ubicomp research that describes the work of installing and maintaining IoT devices. But an important and often understated element of commercial IoT solutions is their long-term socio-technical infrastructural nature, and therefore long-term installation and maintenance needs. This adds complexity to...

Although computational linguistic methods-such as topic modelling, sentiment analysis and emotion detection-can provide social media researchers with insights into online public discourses, it is not inherent as to how these methods should be used, with a lack of transparent instructions on how to apply them in a critical way. There is a growing bo...

During the COVID-19 pandemic, digital contact-tracing has been employed in many countries to monitor and manage the spread of the disease. However, to be effective such a system must be adopted by a substantial proportion of the population; therefore, public trust plays a key role. This paper examines the NHS COVID-19 smartphone app, the digital co...

BACKGROUND Digital contact tracing (DCT) apps have been recently implemented widely as a response to the Covid-19 pandemic. Research has focused on understanding the acceptance and adoption of these apps, but more work is needed to understand which factors may contribute to sustained use of the app. This is key in public health due to DCT apps need...

Background: Digital contact tracing (DCT) apps have been recently implemented as a response to the Covid-19 pandemic. Research has focused on understanding the acceptance and adoption of these apps, but more work is needed to understand which factors may contribute to their sustained use. This is key in public health due to DCT apps needing a high...

Technology ‘demos’ have become a staple in technology design practice, especially for showcasing prototypes or systems. However, demonstrations are also commonplace and multifaceted phenomena in everyday life, and thus have found their way into empirical research of technology use. In spite of their presence in HCI, their methodical character as a...

The Wizard of Oz method is an increasingly common practice in HCI and CSCW studies as part of iterative design processes for interactive systems. Instead of designing a fully-fledged system, the 'technical work' of key system components is completed by human operators yet presented to study participants as if computed by a machine. However, little...

Via audiovisual communications and a controllable physical embodiment, Mobile Robotic telePresence (MRP) systems aim to support enhanced collaboration between remote and local members of a given setting. But MRP systems also put the remote user in positions where they frequently rely on the help of local partners. Getting or 'recruiting' such help...

This paper presents findings from the deployment of a technology probe—the connected shower—and implications for the development of ‘living services’ or autonomous context-aware consumer-oriented IoT services that exploit sensing to gain consumer ‘insight’ and drive personalised service innovation. It contributes to the literature on water sustaina...

UbiComp has been envisioned to bring about a future dominated by calm computing technologies making our everyday lives ever more convenient. Yet the same vision has also attracted criticism for encouraging a solitary and passive lifestyle. The aim of this paper is to explore and elaborate these tensions further by examining the human values surroun...

UbiComp has been envisioned to bring about a future dominated by calm computing technologies making our everyday lives ever more convenient. Yet the same vision has also attracted criticism for encouraging a solitary and passive lifestyle. The aim of this paper is to explore and elaborate these tensions further by examining the human values surroun...

As the Internet of Things continues to take hold in the commercial world, the teams designing these new technologies are constantly evolving and turning their hand to uncharted territory. This is especially key within the field of secondary service design as businesses attempt to utilize and find value in the sensor data being produced by connected...

Drawing from Conversation Analysis (CA), we examine how the orientation towards progressivity in talk-keeping things moving-might help us better understand and design for voice interactions. We introduce progressivity by surveying its explication in CA, and then look at how a strong preference for progressivity in conversation works out practically...

This volume represents the proceedings of the CHI 2019 Workshop on New Directions for the IoT: Automate, Share, Build, and Care.

Predictions of people's behaviour increasingly drive interactions with a new generation of IoT services designed to support everyday life in the home, from shopping to heating. Based on the premise that such automation is difficult due to the contingent nature of people's practices, in this work we explore the nature of these contingencies in depth...

As the IoT is taking hold in the home, in healthcare, factories, and industry, new challenges and approaches arise for HCI research and design. For example, HCI is exploring agency delegation and automation to support the user in managing the deluge of IoT data, make decisions, or even take actions on behalf of the user, while economic models are b...

The domestic environment is a key area for the design and deployment of autonomous systems. Yet research indicates their adoption is already being hampered by a variety of critical issues including trust, privacy and security. This paper explores how potential users relate to the concept of autonomous systems in the home and elaborates further poin...

The domestic environment is a key area for the design and deployment of autonomous systems. Yet research indicates their adoption is already being hampered by a variety of critical issues including trust, privacy and security. This paper explores how potential users relate to the concept of autonomous systems in the home and elaborates further poin...

Research indicates that personal adoption of emerging ubicomp technologies is being notoriously hampered by a variety of critical issues including trust, privacy and security. Issues such as these cannot be studied and understood by evaluating computer systems in isolation, but rather by taking a 'big picture' approach and examining their synergy w...

Video showcase to be presented at the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow UK. Also available at https://vimeo.com/316738022

This paper presents the design and field study of the Connected Shower, a bespoke IoT device that captures water flow, temperature, shower-head movement, and shower product weight. We deployed the device in six UK homes for a week to understand the use of 'intimate data' as captured by IoT systems. Findings from our contextual interviews unpack a)...

This paper examines ‘the routine shop’ as part of a project that is exploring automation and autonomy in the Internet of Things. In particular we explicate the ‘work’ involved in anticipating need using an ethnomethodological analysis that makes visible the mundane, ‘seen but unnoticed’ methodologies that household members accountably employ to org...

This workshop invites the CSCW community to explore hybrid events - large collocated events where technology is used to support audience participation. We argue that the technology landscape has changed since the early studies in CSCW towards this context. Therefore, the research foci must similarly change and focus on studying the practices or pro...

The domestic environment is a key area for the design and deployment of autonomous systems. Yet research indicates their adoption is already being hampered by a variety of critical issues including trust, privacy and security. This paper explores how potential users relate to the concept of autonomous systems in the home and elaborates further poin...

Provenance network analytics is a novel data analytics approach that helps infer properties of data, such as quality or importance, from their provenance. Instead of analysing application data, which are typically domain-dependent, it analyses the data’s provenance as represented using the World Wide Web Consortium’s domain-agnostic PROV data model...

In this panel, we discuss the challenges that are faced by HCI practitioners and researchers as they study how voice assistants (VA) are used on a daily basis. Voice has become a widespread and commercially viable interaction mechanism with the introduction of VAs such as Amazon's Alexa, Apple's Siri, the Google Assistant, and Microsoft's Cortana....

Voice User Interfaces are becoming ubiquitously available, providing unprecedented opportunities to advance our understanding of voice interaction in a burgeoning array of practices and settings. We invite participants to contribute work-in-progress in voice interaction, and to come together to reflect on related methodological matters, social uses...

The Internet of Things (IoT) promises to enable applications that foster a more efficient, sustainable, and healthy way of life. If end-users are to take full advantage of these developments we foresee the need for future IoT systems and services to include an element of autonomy and support the delegation of agency to software processes and connec...

Provenance network analytics is a novel data analytics approach that helps infer properties of data, such as quality or importance, from their provenance. Instead of analysing application data, which are typically domain-dependent, it analyses the data's provenance as represented using the World Wide Web Consortium's domain-agnostic PROV data model...

Voice User Interfaces (VUIs) are becoming ubiquitously available, being embedded both into everyday mobility via smartphones, and into the life of the home via ‘assistant’ devices. Yet, exactly how users of such devices practically thread that use into their everyday social interactions remains underexplored. By collecting and studying audio data f...

We present fieldwork findings from the deployment of an interactive sensing system that supports the work of energy advisors who give face-to-face advice to low-income households in the UK. We focus on how the system and the data it produced are articulated in the interactions between professional energy advisors and their clients, and how they col...

We present fieldwork findings from the deployment of an interactive sensing system that supports the work of energy advisors who give face-to-face advice to low-income households in the UK. We focus on how the system and the data it produced are articulated in the interactions between professional energy advisors and their clients, and how they col...

This one-day workshop intends to bring together both academics and industry practitioners to explore collaborative challenges in speech interaction. Recent improvements in speech recognition and computing power has led to conversational interfaces being introduced to many of the devices we use every day, such as smartphones, watches, and even telev...

In this paper, we present the study of interactional arrangements that support the collaboration of headquarters (HQ), field responders, and a computational planning agent in a time‐critical task setting created by a mixed‐reality game. Interactional arrangements define the extent to which control is distributed between the collaborative parties. W...

This Special Issue focuses on the emerging use of wearable technologies for mobile collocated interactions. Rather than merely listing off the articles that follow, we wish to introduce this Special Issue by discussing the past, present, and future of this field in HCI through its theories, ongoing designs and research efforts, and emerging frame-...

We introduce Co-Curator, a prototype mobile app designed to support collection and sharing of referential sources of inspiration in face-to-face design ideation meetings. The design of the app stems from both ideas of repurposing mobile devices in everyday mundane tasks and existing practice amongst designers to share and collate sources of inspira...

Touch screen technology's first public appearance was in the early 2000s. Touch screens became a part of the daily life with the invention of smartphones and tablets. Now, this technology has the potential to be the next big change in flight deck design. To date, mobile devices are deployed by several air carriers to perform a host of non-safety cr...

Many interactional issues with Flight Management Systems (FMS) in modern flight decks have been reported. Avionics designers are seeking for ways to reduce cognitive load of pilots with the aim to reduce the potential for human error. Academic research showed that touch screen interfaces reduce cognitive effort and provide an intuitive way of inter...

In order to benefit from potential reduced operational costs and crew workload airlines are increasingly interested in touchscreen-based Electronic Flight Bags (EFB). This paper focuses on the specific domain of Search and Rescue (SAR) Helicopters. A first set of results aiming to explore and understand potential benefits and challenges of an EFB i...

Future flight deck designs from various avionics manufacturer incorporate touchscreen technology. There is little published research investigating the impact of inflight vibrations and increased G-Force (+Gz) on touchscreen usability. A Fitts' law experiment was conducted to understand the effect of +Gz on touchscreen usability. 2-Gz and 3-Gz condi...

IoT applications that act on behalf of people are becoming reality. In this workshop we explore the opportunities and challenges that arise when these proactive technologies become increasingly embedded in our world and carried on our bodies, particularly when converging with the emerging 'sharing economy' paradigm. IoT sensors enable new sharing e...

In this paper we unpack the use of conversational agents, or so-called intelligent personal assistants (IPAs), in multi- party conversation amongst a group of friends while they are socialising in a café. IPAs such as Siri or Google Now can be found on a large proportion of personal smartphones and tablets, and are promoted as ‘natural language’ in...

We demonstrate a prototype mobile application designed to support individually collecting personal sources of inspiration on mobile phones, and then the sharing and curating of these collected materials in a face-to-face situation.

Smart systems are becoming increasingly ubiquitous and consequently transforming our lives. The level of system autonomy plays a vital role in the development of smart systems as it profoundly affects how people and these systems interact with each other. However, to date, there are very few studies on human interaction with such systems. This pape...

This paper presents findings from a co-design project that aims to augment the practices of professional energy advisors with environmental data from sensors deployed in clients’ homes. Premised on prior ethnographic observations we prototyped a sensor platform to support the work of tailoring advice-giving to particular homes. We report on the dep...

We present the findings from a study of how people interleave mobile phone use with conversation in pubs. Our findings, informed by ethnomethodology and conversation analysis, unpack the interactional methods through which groups of people in pubs occasioned, sustained, and disengaged from mobile device use during conversation with friends. Fundame...

In the 25 years since Ellis, Gibbs, and Rein proposed the time-space taxonomy, research in the 'same time, same place' quadrant has diversified, perhaps even fragmented. This one-day workshop will bring together researchers with diverse, yet convergent interests in tabletop, surface, mobile, and wearable technologies, spaces and spatial interaction...

Autonomous agents are rapidly becoming collaborative partners in addressing diverse industry and social problem domains. With this shift brings a number of challenges and opportunities in understanding and designing for the dynamics involved in human and agents working collectively. This session will facilitate a wider discussion on the current soc...

The Orchid Project, in partnership with Rescue Global, is a collaborative project that drives the science of human-agent collectives to real-world applications in disaster response.During the 2014 Angel Thunder exercise in the USA, researchers from the Orchid Project embedded with Rescue Global to complete an ethnographic study (Fischer et al., 201...

The avionics industry is seeking to understand the challenges and benefits of touchscreens on flight decks. This paper presents an investigation of interactive displays on the flight deck focusing on the impact of target size, placement and vibration on performance. A study was undertaken with search and rescue (SAR) crew members in an operational...

Interactive autonomous systems are likely to be more involved in future energy systems to assist human users. Given this, we prototyped a future scenario in which householders are assisted in switching electricity tariffs by an agent-based interactive system. The system uses real-time electricity monitoring to instantiate a scenario where participa...

We consider a setting where an agent-based planner instructs teams of human emergency responders to perform tasks in the real world. Due to uncertainty in the environment and the inability of the planner to consider all human preferences and all attributes of the real-world, humans may reject plans computed by the agent. A na¨ıve solution that repl...

We consider a setting where a team of humans oversee the coordination of multiple Unmanned Aerial Vehicles (UAVs) to perform a number of search tasks in dynamic environments that may cause the UAVs to drop out. Hence, we develop a set of multi-UAV supervisory control interfaces and a multi-agent coordination algorithm to support human decision maki...

The increased popularity of mobile devices, combined with their smaller and more personalised nature, means that we have greater opportunities to use our devices all the time. This work looks at the behaviours surrounding device usage amongst groups of friends, with a goal of supporting future research in mobile HCI.

Command and control environments ranging from transport control rooms to disaster response have long been of interest to HCI and CSCW as rich sites of interactive technology use embedded in work practice. Drawing on our engagement with disaster response teams, including ethnography of their training work, we unpack the ways in which situa-tional un...