Simon Holland

The Open University (UK) · Centre for Research in Computing (CRC)

Purpose Restoration of walking ability is a key goal to both stroke survivors and their therapists. However, the intensity and duration of rehabilitation available after stroke can be limited by service constraints, despite the potential for improvement which could reduce health service demands in the long run. The purpose of this paper is to prese...

Shedding is a term used to describe a musical conversation between drummers with the aim to improve their drumming vocabulary, gain confidence in real-time trading of musical ideas, develop an understanding for their original voice on the drum kit and enjoy the process of exploring creativity with a fellow drummer. However, in practice drummers hav...

Background: Rhythm, brain, and body are closely linked. Humans can synchronize their movement to auditory rhythms in ways that can improve the regularity of movement while reducing perceived effort. However, the ability to perform rhythmic movement may be disrupted by various neurological conditions. Many such conditions impair mechanisms that cont...

Nonlinear dynamic processes are fundamental to the behavior of acoustic musical instruments, as is well explored in the case of sound production. Such processes may have profound and under-explored implications for how musicians interact with instruments, however. Although nonlinear dynamic processes are ubiquitous in acoustic instruments, they are...

Background: Huntington disease (HD) is an inherited genetic disorder that results in the death of brain cells. HD symptoms generally start with subtle changes in mood and mental abilities; they then degenerate progressively, ensuing a general lack of coordination and an unsteady gait, ultimately resulting in death. There is currently no cure for H...

Background: Movement analysis in a clinical setting is frequently restricted to observational methods to inform clinical decision making, which has limited accuracy. Fixed-site, optical, expensive movement analysis laboratories provide gold standard kinematic measurements; however, they are rarely accessed for routine clinical use. Wearable inertia...

BACKGROUND Rhythm, brain and the body are closely linked. Humans can synchronise their movement to auditory rhythms with little apparent effort. However, the ability to perform rhythmic movement can be severely disrupted due to neurological conditions. Even in cases where the ability to perceive rhythms remain, a neurological condition may affect t...

BACKGROUND Huntington’s Disease (HD) is an inherited genetic disorder that results in the death of brain cells. HD symptoms generally start with subtle changes in mood and mental abilities and degenerate progressively, ensuing a general lack of coordination and an unsteady gait, ultimately resulting in death. There is currently no cure for HD. Walk...

BACKGROUND Movement analysis in the clinical setting is frequently restricted to observational methods to inform clinical decision making, which has limited accuracy. Fixed-site optical expensive movement analysis laboratories provide ‘gold-standard’ kinematic measurements, however they are rarely accessed for routine clinical use. Wearable inertia...

Algorithmic composition typically involves manipulating structural elements such as indeterminism, parallelism, choice, multi-choice, recursion, weighting, sequencing, timing, and looping. There exist powerful tools for these purposes, however, many musicians who are not expert programmers find such tools inaccessible and difficult to understand an...

Historically, audiences have had various ways to participate in live music performances, including clapping, dancing, swaying, whistling, and singing. More recently, mobile and wireless devices, such as smartphones have opened up powerful new opportunities for audience participation. However, design for technology-mediated audience participation (T...

This is the introductory chapter of a book dedicated to new research in, and emerging new understandings of, music and human-computer interaction—known for short as music interaction. Music interaction research plays a key role in innovative approaches to diverse musical activities, including performance, composition, education, analysis, productio...

Nonlinear dynamical processes play a central role in many acoustic instruments, yet they rarely feature in digital instruments, and are little understood from an interaction design perspective. Such processes exhibit behaviours that are complex, time-dependent, and chaotic, yet in the context of acoustic instruments can facilitate interactions that...

Rebecca Fiebrink is a Senior Lecturer at Goldsmiths, University of London, where she designs new ways for humans to interact with computers in creative practice. As a computer scientist and musician, much of her work focuses on applications of machine learning to music, addressing research questions such as: ‘How can machine learning algorithms hel...

Algorithmic music composition involves specifying music in such a way that it is non-deterministic on playback, leading to music which has the potential to be different each time it is played. Current systems for algorithmic music composition typically require the user to have considerable programming skill and may require formal knowledge of music...

Personalised physiotherapy using wearables.

In a straightforward meta-level shift of focus, we use design patterns as a medium and process for capturing insight about the process of design. We survey mainstream design genres, and draw conclusions about how they can help inform the design of intelligent systems.

Toward personalised physiotherapy

For people with neurological conditions such as stroke and brain injury, gait rehabilitation can lead to a significantly more independent lifestyle. Existing gait rehabilitation studies with stroke survivors using rhythmic haptic cueing via wearable devices have demonstrated improvements in temporal symmetry, increase in stride length and walking s...

Many people with long-term neurological and neurodegenerative conditions such as stroke, brain injury, multiple sclerosis or Parkinson's disease suffer from an impaired walking gait pattern. Gait improvement can lead to better fluidity in walking, improved health outcomes, greater independence, and enhanced quality of life. Existing lab-based studi...

This chapter considers the use of haptics for learning fundamental rhythm skills, including skills that depend on multi-limb coordination. Different sensory modalities have different strengths and weaknesses for the development of skills related to rhythm. For example, vision has low temporal resolution and performs poorly for tracking rhythms in r...

Stroke is one of the leading causes for long-term adult disabilities. More than half of all stroke survivors depend on others for everyday activities after they are discharged from hospital. One of the conditions a stroke survivor may experience is hemiparetic gait. Typical characteristics of hemiparetic gait are: reduced speed, increased step vari...

Many people with long-term neurological and neurodegenerative conditions such as stroke, brain injury, multiple sclerosis or Parkinson's disease suffer from an impaired walking gait pattern. Gait improvement can lead to better fluidity in walking, improved health outcomes, greater independence, and enhanced quality of life. Existing lab-based studi...

For people with neurological conditions such as stroke and brain injury, gait rehabilitation can lead to a significantly more independent lifestyle. Existing gait rehabilitation studies with stroke survivors using rhythmic haptic cueing via wearable devices have demonstrated improvements in temporal symmetry, increase in stride length and walking s...

Type 1 diabetes is a potentially life-threatening chronic condition that requires frequent interactions with diverse data to inform treatment decisions. While mobile technologies such as blood glucose meters have long been an essential part of this process, designing interfaces that explicitly support decision-making remains challenging. Dual-proce...

For those with chronic conditions, such as Type 1 diabetes, smartphone apps offer the promise of an affordable, convenient , and personalized disease management tool. However , despite significant academic research and commercial development in this area, diabetes apps still show low adoption rates and underwhelming clinical outcomes. Through user-...

Hundreds of diabetes self-management apps are available for smart phones, typically using a diary or logging methodology. This paper investigates how well such approaches help participants to make sense of collected data. We found that, while such systems typically support data and trend review, they are ill suited to helping users understand compl...

We present an algorithmic composition system designed to be accessible to those with minimal programming skills and little musical training, while at the same time allowing the manipulation of detailed musical structures more rapidly and more fluidly than would normally be possible for such a user group. These requirements led us to devise non-stan...

Mobile and sensor-based technologies have created new interaction design possibilities for technology-mediated audience participation in live music performance. However, there is little if any work in the literature that systematically identifies and characterises design issues emerging from this novel class of multi-dimensional interactive perform...

We have reviewed and assessed the reliability of a dead reckoning and drift correction algorithm for the estimation of spatial gait parameters using Inertial Measurement Units (IMUs). In particular, we are interested in obtaining accurate stride lengths measurements in order to assess the effects of a wearable haptic cueing device designed to assis...

Algorithmic composition systems allow for the partial or total automation of music composition by formal, computational means. Typical algorithmic composition systems generate nondeterministic music, meaning that multiple musical outcomes can result from the same algorithm - consequently the output is generally different each time the algorithm run...

Wearable technologies, in the form of small, light and inconspicuous devices, can be designed to help individuals suffering from neurological conditions carry out regular rehabilitation exercises. Current research has shown that walking to a rhythm can lead to significant improvements in various aspects of gait. Our primary aim is to provide a suit...

Classifying stroke survivors based on their walking abilities is an important part of the gait rehabilitation process. It can act as powerful indicator of function and prognosis in both the early days after a stroke and long after a survivor receives rehabilitation. This classification often relies solely on walking speed; a quick and easy measure,...

Restoring mobility and rehabilitation of gait are high priorities for rehabilitation of neurological conditions. Cueing using metronomic rhythmic sensory stimulation has been shown to improve gait, but most versions of this approach have used auditory and visual cues. In contrast, I propose the development of a prototype wearable system for rhythmi...

Restoring mobility and rehabilitation of gait are high priorities for rehabilitation of neurological conditions. Cueing using metronomic rhythmic sensory stimulation has been shown to improve gait, but most versions of this approach have used auditory and visual cues. In contrast, we propose the development of a prototype wearable system for rhythm...

Classifying stroke survivors based on their walking abilities is an important part of the gait rehabilitation process. It can act as powerful indicator of function and prognosis in both the early days after a stroke and long after a survivor receives rehabilitation. This classification often relies solely on walking speed; a quick and easy measure,...

Hundreds of diabetes self-management apps are available for smart phones, typically using a diary or logging methodology. This paper investigates how well such approaches help participants to make sense of collected data. We found that, while such systems typically support data and trend review, they are ill suited to helping users understand compl...

Music is an evolutionarily deep-rooted, abstract, real-time, complex, non-verbal, social activity. Consequently, interaction design in music can be a valuable source of challenges and new ideas for HCI. This workshop will reflect on the latest research in Music and HCI (Music Interaction for short), with the aim of strengthening the dialogue betwee...

In this position paper we present a new approach to music interaction design. We propose the TMAP Design Cards as a tool and method to generate and advance ideas for technology-mediated audience participation in live music (TMAP). The first draft set of cards was built on the TMAP Framework, which describes and maps out the design space of TMAP bas...

We compare three contrasting models of the perceived distance between root-position major and minor chords and test them against new empirical data. The models include a recent psychoacoustic model called spectral pitch-class distance, and two well-established music theoretical models – Tonnetz distance and voice-leading distance. To allow a princi...

Restoring mobility and rehabilitation of gait are high priorities for rehabilitation of neurological conditions. Cueing using metronomic rhythmic sensory stimulation via entrainment has been shown to improve gait, but almost all previous versions of this approach have used auditory or visual cues. In contrast, we have developed and pilot-tested a p...

Recovery of walking function is a vital goal of post-stroke rehabilitation. Cueing using audio metronomes has been shown to improve gait, but can be impractical when interacting with others, particularly outdoors where awareness of vehicles and bicycles is essential. Audio is also unsuitable in environments with high background noise, or for those...

This paper presents a first step in the development of a methodology to compare the ability of different sonifications to convey the fine temporal detail of the Electroencephalography (EEG) brainwave signal in real time. In EEG neurofeedback a person " s EEG activity is monitored and presented back to them, to help them to learn how to modify their...

Restoring mobility and rehabilitation of gait are high priorities for post-stroke rehabilitation. Cueing using metronomic rhythmic sensory stimulation has been shown to improve gait, but most versions of this approach have used auditory and visual cues. In contrast, we developed a prototype wearable system for rhythmic cueing based on haptics, whic...

In popular music genres, typical songs are pre-composed and leave little or no space for improvisation during a live performance. That applies for the performing musicians as well as for the spectators in terms of interactive audience participation. In this study we question these improvisational limits and try to identify strategies for involving...

This paper presents an analysis of the user interfaces of a range of algorithmic music composition software using the Cognitive Dimensions of Notations as the main analysis tool. Findings include the following: much of the reviewed software exhibits a low viscosity and requires significant user knowledge. The use of metaphor (staff notation, music...

Recovery of walking function is a major goal of post-stroke rehabilitation. Audio metronomic cueing has been shown to improve gait, but can be impractical and inconvenient to use in a community setting, for example outdoors where awareness of traffic is needed, as well as being unsuitable in environments with high background noise, or for those wit...

This paper presents an analysis, using Cognitive Dimensions (Blackwell & Green, 2003), of a representative selection of user interfaces for algorithmic composition software. Cognitive Dimensions are design principles for notations, user interfaces and programming language design, or from another viewpoint 'discussion tools' for designers (Green & B...

Over the last few decades there has been steady growth in research that addresses the real-time sonification of electroencephalographic (EEG) data. Diverse application areas include medical data screening, Brain Computer Interfaces (BCI), neurofeedback, affective computing and applications in the arts. The present paper presents an overview and cri...

Neural resonance theory (E.W. Large & J.F. Kolen. 1994. Resonance and the perception of musical meter. Connection Science, 6, 177--208) suggests that the perception of rhythm arises as a result of auditory neural populations responding to the structure of the incoming auditory stimulus. Here, we examine the extent to which the responses of a comput...

We introduce, review and analyse recent research in Music and Human-Computer Interaction (HCI), also known as Music Interaction. After a general overview of the discipline, we analyse the themes and issues raised by the other 15 chapters of this book, each of which presents recent research in this field. The bulk of this chapter is organised as an...

“Music Interaction” is the term for interaction design within the domain of music. In areas such as music, the ability to engage effectively in certain activities tends to be restricted to those who have acquired detailed knowledge of domain-specific theories, terminologies, concepts or processes. It can be challenging to design or enhance user int...

The Haptic Bracelets are a system designed to help people learn multi-limbed rhythms (which involve multiple simultaneous rhythmic patterns) while they carry out other tasks. The Haptic Bracelets consist of vibrotactiles attached to each wrist and ankle, together with a computer system to control them. In this chapter, we report on an early empiric...

Tonal Harmony is widely considered to be the most technical and complex part of music theory. Consequently harmonic skills can be hard to acquire. Furthermore, experience of the flexible manipulation of harmony in real time generally requires the ability to play an instrument. Even for those with instrumental skills, it can be difficult to gain cle...

This agenda-setting book presents state of the art research in Music and Human-Computer Interaction (also known as ‘Music Interaction’). Music Interaction research is at an exciting and formative stage. Topics discussed include interactive music systems, digital and virtual musical instruments, theories, methodologies and technologies for Music Int...

We describe the motivations behind the E-Sense project which will investigate augmented perception by building a range of novel tactile interfaces. As well as exploring the practical utility of these systems for real world tasks, we are particularly interested in the following question: how can we design tactile interfaces to mediate novel sensory...

Current design guidelines for conventional tangible systems suggest that the representational significance of tangible tokens is an important consideration in the design of tangible interaction, especially in collaborative contexts. Such advice might be assumed to imply that nomadic tangible systems that employ improvised tokens are liable to have...

We present a new Dynamic Tonality MIDI sequencer, Hex, that aims to make sequencing music in and across a large variety of novel tunings as straightforward as sequencing in twelve-tone equal temperament. In order to enable the intuitive visualization and dynamic manipulation of tuning, it replaces the piano roll used in conventional MIDI sequencers...

The complex and multidimensional nature of musical timbre is a problem for the design of intuitive interfaces for sound synthesis. A useful approach to the manipulation of timbre involves the creation and subsequent navigation or search of n-dimensional coordinate spaces or timbre spaces. A novel timbre space search strategy is proposed, based on w...

In this paper, we introduce a new approach to computer-aided microtonal improvisation by combining methods for (1) interactive scale navigation, (2) real-time manipulation of musical patterns and (3) dynamical timbre adaption in solidarity with the respective scales. On the basis of the theory of well-formed scales we offer a visualization of the u...

In this paper, we describe a playable musical interface for tablets and multi-touch tables. The interface is a generalized keyboard, inspired by the Thummer, and consists of an array of virtual buttons. On a generalized keyboard, any given interval always has the same shape (and therefore fingering); furthermore, the fingering is consistent over a...

Hex Player poster. See associated paper for more details.