Tony Stockman

Queen Mary, University of London | QMUL · School of Electronic Engineering and Computer Science

The calibration of chemical sensors is important for ensuring signal integrity. However, for some sensors this can be time-consuming, and without prompt and adequate feedback users may be unaware of calibration errors until post-session analysis is conducted. To address this challenge, we present a real-time sonification framework designed to facil...

Stress recognition, particularly using machine learning (ML) with physiological data such as heart rate variability (HRV), holds promise for mental health interventions. However, limited datasets in affective computing and healthcare research can lead to inaccurate conclusions regarding the ML model performance. This study employed supervised learn...

Drawing upon the well-documented impact of long-term heart rate variability biofeedback (HRVB) on psychophysiological responses, this study seeks to explore the short-term effects arising from a single HRVB session during and after paced breathing exercise. The research aligns with the neurovisceral integration model, emphasizing the link between h...

The auditory virtual reality interface of Audiom, a web-based map viewer, was evaluated by thirteen blind participants. In Audiom, the user is an avatar that navigates, using the arrow keys, through geographic data, as if they are playing a first-person, egocentric game. The research questions were: What will make blind users want to use Audiom map...

Auditory overviews of routes can provide routing and map information to blind users enabling them to preview route maps before embarking on a journey. This paper investigates the usefulness of a system designed to do this through a Preliminary Survey, followed by a Design Study to gather the design requirements, development of a prototype and evalu...

The emergence of new technologies is providing opportunities to develop novel solutions that facilitate the integration of visually-impaired people in different activities of our daily life, including collective music making. This paper presents a study conducted with visually-impaired music performers, which involved a participatory approach to th...

Objects that move in depth on an approaching trajectory (looming) are often encountered in both the real and virtual worlds. Examples include navigating oncoming traffic, and sporting and gaming activities where judgements are made to avoid or attack approaching objects. How people react to looming objects may impact on their survival and progressi...

Crossmodal correspondence, a perceptual phenomenon which has been extensively studied in cognitive science, has been shown to play a critical role in people’s information processing performance. However, the evidence has been collected mostly based on strictly-controlled stimuli and displayed in a noise-free environment. In real-world interaction s...

Although it has been shown that augmented multimodal feedback has a facilitatory effect on motor performance for motor learning and music training, the functionality of haptic feedback combined with other modalities in rhythmic movement tasks has rarely been explored and analysed. In this paper, we evaluate the functionality of visual-haptic feedba...

Leveraging the perceptual phenomenon of crossmoal correspondence has been shown to facilitate peoples information processing and improves sensorimotor performance. However for goal-oriented interactive tasks, the question of how to enhance the perception of specific Crossmodal information, and how Crossmodal information integration takes place duri...

Humans sense of distance depends on the integration of multi sensory cues. The incoming visual luminance, auditory pitch and tactile vibration could all contribute to the ability of distance judgement. This ability can be enhanced if the multimodal cues are associated in a congruent manner, a phenomenon has been referred to as Crossmodal correspond...

While travelling to new places, maps are often used to determine the specifics of the route to follow. This helps prepare for the journey by forming a cognitive model of the route in our minds. However , the process is predominantly visual and thus inaccessible to people who are either blind or visually impaired (BVI) or doing an activity where the...

Proceedings of ICAD 2019

Augmented visual-audio feedback supports rhythmic motor performance in both sports training and sensorimotor synchronization practise. In home-based rehabilitation for minor stroke patients, training on a fine motor skill using rhythms not only helps to recover sophisticated motion ability but also increases their confidence and mental health recov...

The objective of the work reported here is to develop an understanding of cross-modal collaborative information-seeking (CCIS) between visually impaired (VI) and sighted users in order to learn how best to support it. In a previous article, we reported the CCIS process that occurred when 14 pairs of users, one sighted and one VI, performed web-base...

The Channel Strip mixer found on physical mixing desks is the primary Graphical User Interface design for most Digital Audio Workstations. While this metaphor provides transferable knowledge from hardware, there may be a risk that it does not always translate well into screen-based mixers. For example, the need to search through several windows of...

This paper presents SoundTrAD, a method and tool for design- ing auditory displays for the user interface. SoundTrAD brings together ideas from user interface design and soundtrack compo- sition and supports novice auditory display designers in building an auditory user interface. The paper argues for the need for such a method before going on to d...

Objects that move in depth (looming) are ubiquitous in the real and virtual worlds. How humans interact and respond to these approaching objects may affect their continued survival, and is dependent on the individuals capacity to accurately interpret depth and movement cues. However, many psychological studies investigating auditory looming depict...

Graphical displays are a typical means for conveying awareness information in groupware systems to help users track joint activities, but are not ideal when vision is constrained. Understanding how people maintain awareness through non-visual means is crucial for designing effective alternatives for supporting awareness in such situations. We prese...

ASYNCHRONY BETWEEN TACTILE AND AUDITORY feedback (action-sound latency) when playing a musical instrument is widely recognized as disruptive to musical performance. In this paper we present a study that assesses the effects of delayed auditory feedback on the timing accuracy and judgments of instrument quality for two groups of participants: profes...

There is a growing research interest in combining crossmodal research with tangible interaction design. However, most behavioural research on crossmodal perception and on multimodal tangible interaction evaluation are based on static and non-interactive signals. Grounded in the cognitive neuroscience studies and behavioural research on multisensory...

The Objects VR interface and study explores interactive music and virtual reality, focusing on user experience, understanding of musical functionality, and interaction issues. Our system offers spatio-temporal music interaction using 3D geometric shapes and their designed relationships. Control is provided by tracking of the hands, and the experien...

This paper presents an observational study of the interaction of professional percussionists with a simplified hand percussion instrument. We reflect on how the sound-producing gestural language of the percussionists developed over the course of an hour session, focusing on the elements of their gestural vocabulary that remained in place at the end...

Expression in musical practice is inextricably tied to the touch of the performer. In digital musical instruments (DMIs) the relationship of touch to sound is indirect: the nuances and fine detail of performer control can be flattened and limited during the translation of physical gesture to physical sound. The locus of this research is in the cont...

What can the practice of Soundtrack Composition bring to the design of auditory displays? Previous research has highlighted the lack of knowledge in the wider human-computer-interaction (HCI) community about the practice of auditory display (AD) design, providing evidence that there is a need to capture the rationale for AD design more effectively,...

When designing digital musical instruments the importance of low and consistent action-to-sound latency is widely accepted. This paper investigates the effects of latency (0- 20ms) on instrument quality evaluation and performer inter- action. We present findings from an experiment conducted with musicians who performed on an percussive digital musi...

The majority of Digital Audio Workstation designs represent mix data using a channel strip metaphor. While this is a familiar design based on physical mixing desk layout, it can lead to a visually complex interface incorporating a large number of User Interface objects which can increase the need for navigation and disrupt the mixing workflow. With...

The majority of Digital Audio Workstation designs represent mix data using a channel strip metaphor. While this is a familiar design based on physical mixing desk layout, it can lead to a visually complex interface incorporating a large number of User Interface objects which can increase the need for navigation and disrupt the mixing workflow. With...

The Objects VR interface and study explores interaction design at the crossover of interactive music and virtual environments, where it looks to understand users’ experience, conceptual models of musical functionality, and associated interaction strategies. The system presented in testing, offers spatio-temporal music interaction using 3D geometric...

This paper presents the Accessible Spectrum Analyser (ASA) developed as part of the DePic project (Design Patterns for Inclusive collaboration) at Queen Mary University of London. The ASA uses sonification to provide an accessible representation of frequency spectra to visually impaired audio engineers. The software is free and open source and is di...

As the amount of visual information within Digital Audio Workstations increases, the interface potentially becomes more cluttered and time consuming to navigate. The increased graphical information may tax available display space requirements and potentially overload visual perceptual and attentional bandwidth. This study investigates the extent to...

Previous studies of users with visual impairments access to the web have focused on human-web interaction. This study explores the under investigated area of cross-modal collaborative information seeking (CCIS), that is, the challenges and opportunities that exist in supporting visually impaired (VI) users to take an effective part in collaborative...

There is growing interest in the application of crossmodal perception to interface design. However, most research has focused on task performance measures and often ignored user experience and engagement. We present an examination of crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and a...

We examine how auditory displays, sonification and haptic interaction design can support visually impaired sound engineers, musicians and audio production specialists access to digital audio workstation. We describe a user-centred approach that incorporates various participatory design techniques to help make the design process accessible to this p...

Research has suggested that adding contextual information such as reference markers to data sonification can improve interaction with auditory graphs. This paper presents results of an experiment that contributes to quantifying and analysing the extent of such benefits for an integral part of interacting with graphed data: point estimation tasks. W...

Most previous studies of users with visual impairments (VI) access to the web have focused solely on individual human-web interaction. This study explores the under investigated area of cross-modal collaborative information seeking (CCIS), that is the challenges and opportunities that exist in supporting visually impaired users to take an effective...

We present a study investigating the impact of dynamic tactile feedback on performer navigation of a continuous pitch space on a digital musical instrument. Ten musicians performed a series of blind pitch selection and melodic tasks on a self- contained digital musical instrument with audio-frequency tactile feedback that was generated in response...

I. Introduction Studies reveal that often group members collaborate when searching for information even if they were not explicitly asked to collaborate [1]. The activity that involves a group of people engaging in a common information seeking task is called Collaborative Information Seeking (CIS). Over the past few years, CIS research has focused...

Research has suggested that adding contextual information such as reference markers to data sonification can improve interaction with auditory graphs. This paper presents results of an experiment that contributes to quantifying and analysing the extent of such benefits for an integral part of interpreting graphed data: point estimation tasks. We ex...

Research has suggested that adding contextual information such as reference markers to data sonification can improve interaction with auditory graphs. This paper presents results of an experiment that contributes to quantifying and analysing the extent of such benefits for an integral part of interpreting graphed data: point estimation tasks. We ex...

Previous studies of users with visual impairments access to the web have focused on human-web interaction. This study explores the under investigated area of cross-modal collaborative information seeking (CCIS), that is, the challenges and opportunities that exist in supporting visually impaired (VI) users to take an effective part in collabor...

The growth in mobile and multimodal Computing is leading to the consideration of alternative modes of information presentation, particularly in situations such as driving or walking in unfamiliar locations where the eyes are needed for primary navigation. We report the results of an experiment in which map information is presented to 10 normally si...

This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigati...

Due to the compartmentalisation of the mix into separate channel strips, Digital Audio Workstation displays often result in global attributes and relationships between channels becoming hard to discern. This is despite the fact that overviews of mix information may help support direct visual comparison and make the visual data more coherent, thereb...

Understanding the information-seeking behavior of visually impaired users is essential to designing search interfaces that support them during their search tasks. In a previous article, we reported the information-seeking behavior of visually impaired users when performing complex search tasks on the web, and we examined the difficulties encountere...

Methods used to engage users in the design process often rely on visual techniques, such as paper prototypes, to facilitate the expression and communication of design ideas. The visual nature of these tools makes them inaccessible to people living with visual impairments. In addition, while using visual means to express ideas for designing graphica...

When mixing audio on computers, users need to be able to attend to both visual and auditory information and share their attention between the two modalities. This study aims to investigate to what extent User Interface (UI) design has an effect on mixing workflow by analysing whether different presentation styles result in more efficient coordinati...

This paper describes our Experience-Centered Design (ECD) inquiry into the current and potential role of digital photography to support interpersonal expression in a class of children (aged 11–15 years) at a mixed special education needs school. Presented as a case study, we describe five classroom-based creative workshops that engaged pupils with...

Multi-session search tasks are complex and span more than one web session. Such tasks are challenging because searchers must keep track of their search progress and the information they encounter across sessions. Multi-session tasks can be cognitively taxing for visually impaired users because the lack of persistence of screen readers causes the lo...

This paper discusses progression in our research toward de- veloping SoundTrAD, a method for creating auditory displays that is based on soundtrack composition. We begin by motivating the need for such a method, before detailing the design of the first step of this method. This involved creating a cue sheet that permits designers to treat a human-c...

Most previous studies of users with visual impairments access the web have focused solely on human-web interaction. This study explores the under investigated area of cross-modal collaborative information seeking (CCIS), that is the challenges and opportunities that exist in supporting visually impaired users to take an effective part in collaborat...

Frequently family members are geographically separated for large parts of the day. This separation, allied to a busy schedule, can make it difficult to share daily experiences and maintain the feeling of connectedness. This paper describes an exploratory study to investigate family dynamics and the use of technology in families with primary school...

This paper examines the question of how to uncover patterns from the process of designing cross-modal collaborative systems. We describe how we use activity patterns as an approach to guide this process and discuss its potential as a practical method for developing design patterns.

Through out the design process, designers have to consider the needs of potential users. This is particularly important, but rather harder, when the designers interact with the artefact to-be-designed using different senses or devices than the users, for example, when sighted designers are designing an artefact for use by blind users. In such cases...

We present the evaluation of an interactive audio map system that enables blind and partially sighted users to explore and navigate city maps from the safety of their home using simulated 3D audio and synthetic speech alone. We begin with a review of existing literature in the areas of spatial knowledge and wayfinding, auditory displays and auditor...

Current Digital Audio Workstations include increasingly complex visual interfaces which have been criticised for focusing user's attention on visual rather than aural modalities. This study aims to investigate whether visual interface complexity has an influence on critical listening skills. Participants with experience mixing audio on computers we...

This paper describes our Experience-centered Design (ECD) inquiry into the current and potential role of digital photography to support interpersonal communication and expression in a class at a mixed special education needs school. Presented as a case study, we describe five classroom-based Creative Photography workshops that engaged pupils with a...

This paper details work toward the design of a method for cre- ating auditory displays for the human-computer interface, based on soundtrack composition. We begin with the benefits to this approach before discussing methods for auditory display design and the need for a unification of different design techniques. We then outline our on-going invest...

We survey the development of auditory displays as assistive technology for visually impaired users. Both single user and collaborative, cross-modal applications are discussed. The systems and design issues covered include desktop and mobile contexts. We conclude that audio in assistive technology displays has much untapped potential.

This research poster talks about the use of Thematic Analysis method to gain insights into family dynamics and the use of technology within families with primary school children.

This research poster talks about the use of Thematic Analysis in HCI for children.

Auditory displays use sound to convey information within the context of human-computer interaction. The use of sound in the interface is becoming more important as technologies are becoming increasingly ubiquitous and 'every-day'. However, its been argued that often ad-hoc approaches are used for the creation of auditory displays, which in turn oft...

This research poster talks about the use of Thematic Analysis for analysing multimedia data, which was collected through cultural probes and semi-structured interviews exploring how families with primary school children use technology for communication and experience sharing. Winner of 'Best Poster Prize' Award

Auditory-visual looming (the presentation of objects moving in depth towards the viewer) is a technique used in film (particularly those in 3D) to assist in drawing the viewer into the created world. The capacity of a viewer to perceptually immerse within the multidimensional world and interact with moving objects can be affected by the sounds (aud...

This project examines how visually impaired people search for web-based information. We first summarise the findings of a comparative study of 15 visually impaired and 15 sighted users performing complex search tasks individually. We then describe an observational study of 4 pairs of sighted and visually impaired people collaborating on inform...

Meals have traditionally been a site for togetherness. We explore the opportunities to design a technology platform that supports re- mote guests in experiencing togetherness and playfulness within the practices of a traditional dinner party. Through both visual, aural channels and remote agency, the guests shared a holistic telematic dining experi...

There is an increasing desire to remain connected when physically distant and computer-mediated communication (CMC) is one means of satisfying this desire. In particular, there is a growing trend for individuals to use commercially available technology to connect with friends and family in social and leisure settings. Drawing on this trend, perform...

An approach to designing hierarchy-based auditory displays that supports non-visual interaction with relational diagrams is presented. The approach is motivated by an analysis of the functional and structural properties of relational diagrams in terms of their role as external representations. This analysis informs the design of a multiple perspect...

Understanding search behavior is important and leads to more effective interfaces that support searchers throughout the search process. In this article, through an observational user study, we investigate the search behavior of 15 visually impaired and 15 sighted searchers while they complete complex search tasks online. We study complex search tas...

The presentation of objects moving in depth towards the viewer (looming) is a technique used in film (par-ticularly those in 3D) to assist in drawing the viewer into the created world. The sounds that accompany these looming objects can affect the extent to which a viewer can perceptually immerse within the multidi-mensional world and interact with...

Growing workload of parents and increased amount of children's after-school activities leads to decreased amount of daily contact between family members. Linking the concepts of calm technology, attention theory and connectedness-oriented communication, this PhD research aims to investigate the type of communication parents and primary school child...

We present the evaluation of a methodological design framework that supports expert and novice designers in creating auditory artefacts in human–technology interaction. We first motivate the development of our framework by analysing available guidance and the current practice in the field. Subsequently, we recapitulate on the design of the framewor...

This chapter focuses on the nonspeech audio used to display information. Auditory interfaces are bidirectional, communicative connections between two systems-typically a human user and a technical product. The side toward the machine involves machine listening, speech recognition, and dialog systems. The side toward the human uses auditory displays...