Nick Bryan-Kinns

Nick Bryan-Kinns
Queen Mary, University of London | QMUL

PhD

About

216
Publications
37,173
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,024
Citations
Introduction
Nick Bryan-Kinns is Professor of Interaction Design, Director of EPSRC+AHRC Media and Arts Technology Centre at Queen Mary University of London, Distinguished Prof at Wuhan Uni of Technology and Guest Prof at Huazhong Uni of Science & Technology, China. He is a Fellow of Royal Society of Arts, Fellow of British Computer Society, Senior Member of ACM, and leads the Sonic Interaction Design Lab in the Centre for Digital Music. Nick has published award winning international journal and conference papers on his extensively funded research on participatory design, collaboration, mutual engagement, interactive art, cross-modal interaction, and tangible interfaces. Awards include winner of the AT&T and NYU Connect Ability Challenge 2015, and winner of QM Public Engagement Awards 2015 and 2017.
Additional affiliations
October 2002 - March 2021
Queen Mary, University of London
Position
  • Professor

Publications

Publications (216)
Article
Visual materials are a widely used tool for stimulating creativity. This paper explores the potential for visual stimuli to support novices’ creative engagement with multimodal digital musical interfaces. An empirical study of 24 participants was conducted to compare the effect of abstract and literal forms of graphical scores on novices’ creative...
Conference Paper
Identifying points of engagement from a person’s interaction with computers could be used to assess their experience and to adapt user interfaces in real-time. However, it is difficult to identify points of engagement unobtrusively; HCI studies typically use retrospective protocols or rely on cumbersome sensors for real-time analysis. We present a...
Conference Paper
Full-text available
Recent advances in generative Artificial Intelligence (AI) and Human-Computer Interaction (HCI) have led to a design trend of user interfaces supporting novices in co-creating with AI. Typically, these tools aim to extend novices' abilities and democratise the more difficult aspects of creative practice. We suggest that designing for reflection mig...
Article
This article reports on an interview-based study with ten sound artists and composers, all engaged in situated sonic practices. We propose that these artists engage the ear and shape possible interactions with the artwork by altering the relationship between sound, the space in which it is heard and the people who hear it. Our interviews probe the...
Article
The rapid development of rural societies mixed with the infrastructural transformation of emerging economies bring both challenges and opportunities to Human-Computer Interaction (HCI) design as illustrated through the emergence of the field of HCI for Development (HCI4D). A key challenge for HCI4D is how local knowledge, expertise, and culture can...
Chapter
Full-text available
We present our study on the design and evaluation of sound samples for motion perception in a Virtual Reality (VR) application. In previous study we found our sound samples to be incoherent with the VR visual channel. In current research we designed four new samples and tested them adapting standard subjective evaluation protocols to our needs. Twe...
Conference Paper
Full-text available
Explainable AI has the potential to support more interactive and fluid co-creative AI systems which can creatively collaborate with people. To do this, creative AI models need to be amenable to debugging by offering eXplainable AI (XAI) features which are inspectable, understandable, and modifiable. However, currently there is very little XAI for t...
Article
Full-text available
Background Advances in mobile technology and public needs have resulted in the emergence of mobile health (mHealth) services. Despite the potential benefits of mHealth apps, older adults face challenges and barriers in adopting them. Objective The aims of this study are to understand older adults’ perception of mHealth services and to discover the...
Article
One of the important parts of the Internet economy is the car-sharing economy. Nevertheless, a few numbers of studies have been dedicated to the issues of safety of driving and usability resulting from the car-sharing software application. The current study aims at analyzing the effect of phone location and screen orientation on usability and safet...
Chapter
Music therapy is regarded as a non-pharmacological treatment to improve cognitive, psychological and behavioural alterations in patients with Alzheimer’s disease (AD). However, the general hearing loss in AD patients decreases their musical perception and the efficacy of music therapy. This paper presents a wearable vibrotactile prototype, a multis...
Conference Paper
Full-text available
Composing is a neglected area of music education. To increase participation, many technologies provide open-ended interfaces to motivate child autodidactic use, drawing influence from Papert’s LOGO philosophy to support children’s learning through play. This paper presents a case study examining which interactions with Codetta, a LOGO-inspired, blo...
Preprint
BACKGROUND Advances in mobile technology and public needs have resulted in the emergence of mobile health (mHealth) services. Despite the potential benefits of mHealth apps, older adults face challenges and barriers in adopting them. OBJECTIVE The aims of this study are to understand older adults’ perception of mHealth services and to discover the...
Chapter
This paper presents a study conducted in naturalistic setting with data collected from an interactive art installation. The audience is challenged in a Turing Test for artistic creativity involving recognising human-made versus AI-generated drawing strokes. In most cases, people were able to differentiate human-made strokes above chance. An analysi...
Article
This research explores how tangible interactive technology might offer opportunities for socialization and sensory regulation. We present a study carried out in an educational setting during leisure activities with a small group of children with autism who like music. We introduce Όλοι (pronounced Olly), a sonic textile Tangible User Interface (TUI...
Preprint
Full-text available
Extended Reality (XR) technology - such as virtual and augmented reality - is now widely used in Human Computer Interaction (HCI), social science and psychology experimentation. However, these experiments are predominantly deployed in-lab with a co-present researcher. Remote experiments, without co-present researchers, have not flourished, despite...
Chapter
Craft making is associated with tradition, cultural preservation, and skilled hand-making techniques. While there are examples of digital craft making analyses in the literature, Augmented Reality (AR) applied to craft making practice has not been explored, yet applying AR to craft making practices could bring insight into methods of combining virt...
Conference Paper
Full-text available
One of the most fundamental features of living organisms is their growth, a biological phenomenon that can be considered as a type of slow, tangible output responding to an environmental stimulus or an input. Given the relative slowness of growth, once it becomes part of game mechanics, the feature can lead to slow interactivity and slow gameplay i...
Chapter
Full-text available
We introduce a novel experimental system to explore the role of vibrotactile haptic feedback in Virtual Reality (VR) to induce the self-motion illusion. Self-motion (also called vection) has been mostly studied through visual and auditory stimuli and a little is known how the illusion can be modulated by the addition of vibrotactile feedback. Our s...
Conference Paper
Full-text available
We present Olly, a musical textile tangible user interface (TUI) designed around the observations of a group of five children with autism who like music. The intention is to support scaffolding social interactions and sensory regulation during a semi-structured and open-ended playful activity. Olly was tested in the dance studio of a special educat...
Conference Paper
Full-text available
Digital technologies play a fundamental role in New Interfaces for Musical Expression as well as music making and consumption more widely. This paper reports on two workshops with music professionals and researchers who undertook an initial exploration of the differences between digital platforms (software and online services) for music in the UK a...
Conference Paper
Full-text available
There are many studies of Digital Musical Instrument (DMI) design, but there is little research on the cross-cultural co-creation of DMIs drawing on traditional musical instruments. We present a study of cross-cultural co-creation inspired by the Duxianqin-a traditional Chinese Jing ethnic minority single stringed musical instrument. We report on h...
Article
Full-text available
This paper presents research about design a textile-based stretch sensor for making a reliable sensing system for dance movement sensing. We examined sixteen textile-based conductive stretch sensors that were made with commercially produced conductive materials. Through the analysis of their sensitivity, linearity, hysteresis, responsiveness, and f...
Article
Full-text available
Shared virtual environments (SVEs) have been researched extensively within the fields of education, entertainment, work, and training, yet there has been limited research on the creative and collaborative aspects of interactivity in SVEs. The important role that creativity and collaboration play in human society raises the question of the way that...
Article
Full-text available
Design thinking holds the key to innovation processes, but is often difficult to detect because of its implicit nature. We undertook a study of novice designers engaged in team-based design exercises in order to explore the correlation between design thinking and designers’ physical (observable) behavior and to identify new, objective, design think...
Poster
This paper reports on results from the project WEAR (Wearable technologists Engage with Artists for Responsible innovation), which between 2017-2018 has facilitated sustainable innovation processes in the field of smart textiles and wearable technology. 46 design-technology interdisciplinary projects and start-ups throughout Europe were selected th...
Conference Paper
Full-text available
This paper presents the results of the EU-funded innovation action project WEAR Sustain (Wearable technologists Engage with Artists for Responsible innovation), which involved a total of 46 teams in 2 rounds of seed funding with the goal of setting the stage for sustainable and ethical transformation of the field of textile wearable technology. Loo...
Conference Paper
Full-text available
Knit e-textile sensors can be used to detect stretch or strain, and when integrated directly into wearable garments, they can be used to detect movement of the human body. However, before they can reliably be used in real-world applications, the garment construction technique and the effects of wear due to washing need to be considered. This paper...
Article
Full-text available
Wearable technology is widely used for collecting information about the human body and its movement by placing sensors on the body. This paper presents research into electronic textile strain sensors designed specifically for wearable applications which need to be lightweight, robust, and comfortable. In this paper, sixteen stretch sensors, each wi...
Conference Paper
Full-text available
This paper presents an observational study of collaborative spatial music composition. We uncover the practical methods two experienced music producers use to coordinate their understanding of multi-modal and spatial representations of music as part of their workflow. We show embodied spatial referencing as a significant feature of the music produc...
Conference Paper
Playtime is an important activity for child development as it stimulates as well as predicts cognitive, motor, emotional and social skills. Children with autism may find it difficult to socialize, particularly initiating and maintaining human interactions. Consecutively, it is thought that playing with peers is often a challenge that many children...
Article
Full-text available
Growth in digital technologies relies on good quality User Experience informed by rigorous application of Human Computer Interaction principles. This paper presents examples of bridging between China and UK HCI as part of a portfolio of education and research practices developed between Queen Mary University of London, UK, and Hunan University, Chi...
Preprint
Full-text available
Shared Virtual Environments (SVEs) have been extensively researched for education, entertainment, work, and training, yet there has been limited research on the creative aspects of collaboration in SVEs. This raises questions about how to design virtual working spaces to support collaborative creativity in SVEs. In this paper, we outline an SVE nam...
Conference Paper
This paper investigates how haptic and auditory stimulation can be playfully implemented as an accessible and stimulating form of interaction for children. We present the design of Mazi, a sonic Tangible User Interface (TUI) designed to encourage spontaneous and collaborative play between children with high support needs autism. We report on a five...
Conference Paper
Environmental concerns have driven an interest in sustainable smart cities, through the monitoring and optimisation of networked infrastructures. At the same time, there are concerns about who these interventions and services are for, and who benefits. HCI researchers and designers interested in civic life have started to call for the democratisati...
Article
Full-text available
There is a growing interest in designing products and interactions across cultures. In this paper, we report on our attempts to use in-situ making and evaluation to facilitate a short co-design process with outside designers in an ethnic and rural community. We found that rapid prototyping in the local context provided a mechanism to quickly engage...
Conference Paper
Full-text available
Purpose: As wearable technologies and eTextile sectors mature they are being increasingly used in couture and high street fashion. However, much of the innovation in this space has been driven by technological and commercial imperatives. It is time to reconsider this technological landscape in the bigger picture of a sustainable human-centred world...
Conference Paper
Full-text available
We present a case study of a participatory design project in the space of sustainable smart cities and Internet of Things. We describe our design process that led to the development of an interactive seed library that tells the stories of culturally diverse urban food growers, and networked environmental sensors from their gardens, as a way to supp...
Article
Full-text available
Creative engagement with novel musical interfaces can be rewarding for non-musicians. However, designing novel musical interfaces for non-musicians can be challenging because they lack conceptual and technical musical skills. In this paper we explore the effects of task motivation (experiential goal vs utilitarian goal) and user interface mode (whe...
Conference Paper
Full-text available
This paper presents a case study of using conventional Data Mining techniques to identify clusters of creative contributions made by users in a collaborative music making system. Results of the clustering suggest that when people mutually engage with each other they tend to converge on similar creative contributions, whereas smaller clusters may in...
Conference Paper
Full-text available
In this position paper we present case studies of the use of Thematic Analysis in Sonic Interaction Design to inform iterative design and to reveal underlying conceptual understandings of new forms of interaction. One case study used Thematic Analysis to explore people's creative engagement with novel interactive music systems. The second case stud...
Conference Paper
Full-text available
This paper presents a study examining the effects of disflu-ent design on audience perception of digital musical instrument (DMI) performance. Disfluency, defined as a barrier to effortless cognitive processing, has been shown to generate better results in some contexts as it engages higher levels of cognition. We were motivated to determine if dis...
Article
Wearable technologies are a nascent market, growing exponentially and moving into our everyday lives more and more. They are exciting in their capacity to break down barriers between artists and designers and digital technology companies. Technology is becoming more efficient, accurate, and personalized. Hardware is becoming smaller, less visible,...
Article
Full-text available
Co-creation across cultures is a fertile area for the study of design and human computer interaction. Many studies have examined what can be learnt from cultures across the world and how cultures respond to interactive technology, and yet open questions remain on how to engage people in co-creation across cultures. In this article, we reflect on a...
Conference Paper
Graphical displays are a typical means for conveying awareness information in groupware systems to help users track joint activities, but are not ideal when vision is constrained. Understanding how people maintain awareness through non-visual means is crucial for designing effective alternatives for supporting awareness in such situations. We prese...
Preprint
Full-text available
When engaged in music composition and improvisation, composers who are working together may want to communicate in modalities other than sound since that is the primary medium of the creative activity. This raises questions about the Sonic Interaction Design of Virtual Environments, specifically in terms of how to design user experiences which supp...
Chapter
This chapter presents an overview of 3 graphical tools supporting soundscape assessment in different settings, indoors and outdoors. These research prototypes support the spatial organization of the perceptual information available to the participants and are designed based on surveying techniques used in architectural training to create a foundati...
Conference Paper
Full-text available
A challenge for future design education in our post-screen world is how to develop design methods, practices, and education for non-visual interaction with interactive systems. The design education approach reported in this paper combines aspects of Sonic Interaction Design, Interaction Design, and physical computing to allow designers to explore t...
Conference Paper
Sonic maps are an increasingly popular form of exploring soundscapes, and are a possible means of communicating the experience of a soundwalk. We describe how a printed key influenced exploration of an interactive textile sonic map. We explain the technology behind the map, employing capacitive sensing and real-time audio processing. The sonic map...
Conference Paper
Full-text available
This paper explores the relative effect of gesture size on audience perception of digital musical instrument (DMI) performance. In a study involving a total audience of 28 people (split into 2 groups of 13 and 15), we used a small and large version of a DMI to examine how the size of perform-ers' gestures might differ, and how this affects post-hoc...
Conference Paper
Full-text available
This paper presents a methodology for the study of audience perception of live performances, using a combined approach of post-hoc and real-time data. We conducted a study that queried audience enjoyment and their perception of error in digital musical instrument (DMI) performance. We collected quantitative and qualitative data from the participant...
Conference Paper
Full-text available
This paper reports on a study which sets out to examine the process of creative engagement of individual non-musicians when interacting with interactive musical systems (IMSs), and to identify features of IMSs that may support non-musician's creative engagement in music making. In order to creatively engage novices, we aimed to design IMSs which wo...
Conference Paper
Full-text available
This research investigates how applying interaction constraints to digital music instruments (DMIs) affects the way that experienced music performers collaborate and find creative ways to make live improvised music on stage. The constraints are applied in two forms: i) Physically implemented on the instruments themselves, and ii) hidden rules that...
Conference Paper
Full-text available
There is growing interest in designing culturally identifiable products for cross cultural consumers. However, literature rarely pays attention to the conflicting needs of these products in terms of cross-cultural usability and their unique cultural identity. This paper presents a design model addressing this problem through a process of meaning co...
Conference Paper
Full-text available
Although there are potential benefits of mobile technology, there are challenges and barriers for older people to adopting them. This exploratory study took place in East London in 2016, UK. It used questionnaire and interview, aiming at understanding how mobile technology is adopted among older people. Older people’s adoption of mobile technology...
Conference Paper
Full-text available
We present a case study of in-situ culturally inspired design of things driven by the challenge of ‘designing the new music box with local culture’ which resulted in the creation of four interactive prototypes. We compare our design practices and explore the multiple roles of things in in-situ making and evaluating across cultures.