ThesisPDF Available

_derivations and the Performer-Developer: Co-Evolving Digital Artefacts and Human-Machine Performance Practices

Authors:

Abstract and Figures

This thesis concerns the development and use of interactive performance systems designed for improvised musical performance. Written from the perspective of a performer-developer, the research traces the development of personal approaches to designing for musical interactivity in human-machine performance, culminating in the development of the _derivations interactive performance system and related creative outcomes. The contributions and outcomes of this research project are as follows: - The development of novel computer music techniques for use in interactive musical performance; - A novel self-reflective study of the development and use of interactive musical performance systems from the perspective of a performer-developer; - Theoretical perspectives on the design and use of interactive musical performance systems. In addition to the published thesis, this research has generated significant creative outcomes in the form of software, studio recordings, documentation of live performances, video documentation and a publicly available website dedicated to the _derivations system. These creative outcomes are also presented as significant contributions of this research. The creative practice underpinning this research is presented as a narrative of development, tracing advancements in the author’s practice towards the stabilisation of the _derivations system and its accompanying performance practice. Designed for use by instrumental improvisers, _derivations uses live-sampling and timbral matching techniques to generate autonomous responses to the live performance of an improvising musician, engaging the performer in a playful, improvised musical dialogue. This thesis outlines both formative programming experiments and stabilised software artefacts, tracing the author’s creative practice to reveal the iterative and cyclical patterns of development engaged in throughout this research. Employing a practice-based research approach, this project uses the creative practices of software programming and interactive musical performance to surface issues, concerns and interests concerning human-machine performance practice. A self-reflective methodology is employed to engage with emergent research themes arising throughout the development of my creative artefacts. The thesis concludes with three extended reflections-on-action that interrogate theoretical concerns relevant to the interactive computer music community. The first of these reflections addresses the relationship between human and material agencies in the practice of the performer-developer, whilst the second reflection interrogates the concept of musical interpretation in the context of human-machine performance. The final reflection proposes symbiosis as a novel interactive metaphor in the development of interactive musical systems.
Content may be subject to copyright.
A preview of the PDF is not available
... The first author's practice-based doctoral research was concerned with the development and use of a performance system designed for improvised, human-machine performance [4]. The central creative outcome of this research was the author's derivations system, an interactive performance system that uses live sampling, real-time audio analysis and timbral matching techniques to develop generative contributions to a performance with a human improviser. ...
... This system was developed to explore notions of machine agency and autonomy, as well as human-computer interaction in the context of improvised human-machine performance. For a full technical description of the software, see [4]. 2 With a musical background as a saxophonist and computer musician, the first author's research was undertaken from the perspective of a performer-developer. The author's derivations system is the culmination of an iterative development process that formed a large part of the author's personal creative practice. ...
... Performer movements were tracked using infrared motion-tracking system developed for this work. 4 In this paper, the focus will be on the first of the works, Encoded. The second work, Pixel Mountain was aesthetically similar and made use of the same basic technologies and creative strategies. ...
Conference Paper
Full-text available
This paper discusses practice-based research in the context of live performance with interactive systems. Practice-based research is outlined in depth, with key concepts and approaches contextualised with respect to research in the NIME field. We focus on two approaches, both of which are concerned with documenting, examining and reflecting on the real-world behaviours and experiences of people and artefacts involved in the creation of new works. The first approach is primarily based on reflections by an individual performer/developer (auto-ethnography) and the second on interviews and observations. The rationales for both approaches are presented along with findings from research which applied them in order to illustrate and explore the characteristics of both. Challenges, including the difficulty of balancing rigour and relevance and the risks of negatively impacting on creative practices are articulated, as are the potential benefits.
... We find that comparatively little focus has been given to the agency of computing, and exploring aspects of creation, creativity and aesthetics generated by computers within musical practice. Recent developments in machine learning and deep learning have demonstrated potential for seemingly artistic practice generated by computers, including in musical applications [5,15,18]. However, these have largely focused on autonomous sound synthesis or on non-real time composition (often based on established notation or musical data languages) [5,12,14,15,19,20,22,24]. ...
... Recent developments in machine learning and deep learning have demonstrated potential for seemingly artistic practice generated by computers, including in musical applications [5,15,18]. However, these have largely focused on autonomous sound synthesis or on non-real time composition (often based on established notation or musical data languages) [5,12,14,15,19,20,22,24]. ...
... The initial concept was to develop a computer program that could deliver individual commands to an ensemble of eight improvising musicians utilising visual cues. Considerable research and development has been made into systems that improvise audio alongside performing musicians with varying degrees of success [5,6,12,18,22,24], but there has been little investigation into computer led music performance involving human performers. Interestingly, a core concept in much previous research is the idea of player-computer interaction (PCI) occurring with both parties conceived of as equals, often discussed as collaboration [4,5,6,18]. ...
Conference Paper
Full-text available
This paper outlines the development, creation and initial presentation of a computer program designed to direct human performers in improvised musical performance. The work, titled "Electric Sheep", is grounded in models of play-based improvisation of the late 20th century, focused around American composer John Zorn's "game-pieces" of the 1980s. It seeks to overcome some of the technical limitations of previous game-pieces whilst also providing a functioning example of player-computer interaction (PCI) in improvised music practice. Utilising an iterative rehearsal and development process we were able to isolate and highlight the importance of non-verbal and non-musical communication between improvising musicians and offer suggestions for incorporating this kind of feedback into future systems. Through this work, we will highlight the value of exploring the intersection of PCI and musical play as a valuable method of forming insight into rich PCI interactions.
... My own system, _derivations, is a semi-autonomous software system developed for use in improvised performances with instrumental musicians. 3 The system listens to the ...
Article
Full-text available
The development of interactive performance systems is an active area of research in the field of live electronic music. Whilst various models and metaphors of interactivity have been proposed in the literature, the engagement of these systems in improvised performance remains under-researched. This paper explores the notion of musical interpretation in improvised human-machine performance practice from the perspective of a performer-developer. Through a consideration of the notion of the musical text, these creative artefacts and the performance practices they engender are situated within the context of interpretive musical practice. I argue that musical performances with these software systems may be seen as an instantiation of the combined musical ideas of the system developer, the musician navigating this space of ideas, and the live and interactive contributions of a machine to the performance. The paper concludes that the development of interactive software is akin to the creation a form of musical text.
... "Non-stabilised artefacts are those considered to be still within a development and innovation phase, they are artefacts for which meaning is still emerging [...] By contrast, stabilised artefacts are those artefacts that have exited this innovation network and entered the real world to be made use of." (Carey, 2016) This description is given in the context of actor-network theory. According to Akrich and Latour, stabilised technical objects can be considered 'instruments of knowledge' (Akrich and Latour 1992, p. 221). ...
Conference Paper
Full-text available
In this paper the authors present their initial findings from a study into the contemporary use of modular synthesisers. The study works towards the development of an evaluative process-model examining the strategies modular synthesiser composers use to manage algorithmic complexity. Including observations of users' compositional workflows, which provide insight into the process of building, exploring and rebuilding algorithmic music systems using modular synthesisers. We use these observations to consider what factors cause users to fix certain constraints and explore others. We argue that compositional practice with modular synthesisers provides the ideal testing ground for understanding how machine learning and other algorithmic composition techniques can be used in a co-creative context. Finally, we outline design principles for the design of creative music production tools that are inclusive of the unique requirements of algorithmic music composition systems.
Article
Electronic systems designed to improvise with a live instrumental performer are a constant mediation of musical language and artificial decision-making. Often these systems are designed to elicit a reaction in a very broad way, relying on segmenting and playing back audio material according to a fixed or mobile set of rules or analysis. As a result, such systems can produce an outcome that sounds generic across different improvisers, or restrict meaningful electroacoustic improvisation to those performers with a matching capacity for designing improvisatory electroacoustic processing. This article documents the development of an improvisatory electroacoustic instrument for pianist Maria Donohue as a collaborative process for music-making. The Donohue+ program is a bespoke electroacoustic improvisatory system designed to augment the performance capabilities of Maria, enabling her to achieve new possibilities in live performance. Through the process of development, Maria’s performative style, within the broader context of free improvisation, was analysed and used to design an interactive electronic system. The end result of this process is a meaningful augmentation of the piano in accordance with Maria’s creative practice, differing significantly from other improvising electroacoustic instruments she has previously experimented with. Through the process of development, Donohue+ identifies a practice for instrument design that engages not only with a performer’s musical materials but also with a broader free improvisation aesthetic.
Book
Full-text available
The Creative Reflective Practitioner explores research and practice through the eyes of people with a wholehearted commitment to creative work. It reveals what it means to be a reflective creative practitioner, whether working alone, in collaboration with others, with digital technology or doing research, and what we can learn from listening and observing closely. It gives the reader new insights into the fascinating challenge that having a reflective creative mindset can bring. Creative reflective practice is seen through practitioner ideas and works which have informed the writing at every level, supported by research studies and historical accounts. The practitioners featured in this book represent a broad spectrum of interdisciplinary creative activities producing works in film, music, drama, dance and interactive installations. Their work is innovative, full of new ideas and exciting to experience, offering engagement and challenge for audiences and participants alike. Practitioner interviews give a direct sense of how they see creative practice from the inside. The ways in which these different situations of practice stimulate and facilitate reflection in practice and how we can learn from this are described. Variations of reflective practice are discussed that extend the original concepts proposed by Donald Schön, and a contemporary dimension is added through the role of the digital in creative reflective practice as a tool, mediator, medium and partner. This book is relevant to people who wish to understand creativity and reflection in practice and how to learn from the practitioners themselves. This includes researchers in any discipline as well as students, arts professionals and practitioners such as artists, curators, designers, musicians, performers, producers and technologists. Purchasing Options:£ = GBP Paperback: 9781138632769 pub: 2019-12-16 Available for pre-order. Item will ship after 16th December 2019 £36.99 Hardback: 9781138632745 pub: 2019-12-16 Available for pre-order. Item will ship after 16th December 2019
Conference Paper
Full-text available
This paper presents the author's _derivations system, an interactive performance system for solo improvising instrumentalist. The system makes use of a combination of real-time audio analysis, live sampling and spectral re-synthesis to build a vocabulary of possible performative responses to live instrumental input throughout an improvisatory performance. A form of timbral matching is employed to form a link between the live performer and an expanding database of musical materials. In addition, the system takes into account the unique nature of the rehearsal/practice space in musical performance through the implementation of performer-configurable cumulative rehearsal databases into the final design. This paper discusses the system in detail with reference to related work in the field, making specific reference to the system's interactive potential both inside and outside of a real-time performance context.
Conference Paper
Full-text available
We propose a definition of symbiotic interaction that is informed by current developments in computing. We clearly distinguish this definition from previous ones and from selected paradigms that address the human-computer relationship. The definition is also informed by a variety of human-centered frameworks in human-computer interaction, including embodied interactions, situationist frameworks, and participatory and work-oriented design perspectives. Symbiotic interactions can be achieved by combining computation, sensing technology, and interaction design to realize deep perception, awareness, and understanding between humans and computers. Important aspects to implement are transparency, reciprocity, and collaborative use of resources for both computers and humans. The symbiotic relationship is also characterized by goals and agency independence of humans and computers. The definition sets the premise to discuss in a critical way future research agendas for symbiotic interactions that are sensitive to human-centered values.
Conference Paper
Full-text available
For researchers working at the nexus of the techno-scientific and the artistic, recent ontological theory that redefines the virtual condition is useful. This meta-theory is coupled with Pickering's notion of temporally emergent practice as a " dance of agency " , where experimental goals are tuned to accommodate experience. Philosophical groundwork to underpin a hybrid methodology appropriate for practice-based new media research is outlined.
Chapter
This chapter discusses cognitive processes in improvisation. Improvisation is thus central to the formation of new ideas in all areas of human endeavor. Its importance experientially rests with its magical and self-liberating qualities. Its importance scientifically is that it presents one with the clearest, least edited version of how one think, encoded in behavior. It is among the time-based arts, namely music, dance, theatre and mime that one find the greatest literature. From a survey of this material, certain facts emerge quite consistently, and allow the formulation of plausible cognitive models for improvisation. Much of the variety of improvisation comes from the many different types of referent which may be used, and the many kinds of relationships the improviser may choose to set up between the referent and the sounds, movement, words, etc., that constitute the improvised behavior.
Book
How is it possible to think new thoughts? What is creativity and can science explain it? When The Creative Mind: Myths and Mechanisms was first published, Margaret A. Boden's bold and provocative exploration of creativity broke new ground. Boden uses examples such as jazz improvisation, chess, story writing, physics, and the music of Mozart, together with computing models from the field of artificial intelligence to uncover the nature of human creativity in the arts, science and everyday life. The Second Edition of The Creative Mind has been updated to include recent developments in artificial intelligence, with a new preface, introduction and conclusion by the author. It is an essential work for anyone interested in the creativity of the human mind.
Article
Soundspotting is a new approach to creating musical streams by selecting and concatenating source segments from a large audio database using methods from music information retrieval. The soundspotting process computes a similarity score between a target audio segment and all the available segments in the source database and selects the closest-matching source to concatenate to the audio output stream forming a real-time response to the target. Examples of target signals are solo instruments, a synthetic signal generated by an algorithm such as frequency modulation, or a previous output of the soundspotting process, thus yielding an audio information feedback circuit. Soundspotting enhances the techniques of sampling, plunderphonics, remixing, and mashups by adding automatic audio organization and an external driving target signal. This article explores the techniques, technologies, and musical possibilities for soundspotting and shows how it extends the canon of existing computer music methods.
Article
Musicians are always quick to adopt and explore new technologies. The fast-paced changes wrought by electrification, from the microphone via the analogue synthesiser to the laptop computer, have led to a wide diversity of new musical styles and techniques. Electronic music has grown to a broad field of investigation, taking in historical movements such as musique concrète and elektronische musik, and contemporary trends such as electronic dance music and electronica. A fascinating array of composers and inventors have contributed to a diverse set of technologies, practices and music. This book brings together some novel threads through this scene, from the viewpoint of researchers at the forefront of the sonic explorations empowered by electronic technology. The chapters provide accessible and insightful overviews of core topic areas and uncover some hitherto less publicised corners of worldwide movements. Recent areas of intense activity such as audiovisuals, live electronic music, interactivity and network music are actively promoted.