Conference PaperPDF Available

Renegotiating Responsibilities in Human-Computer Ensembles

Authors:

Abstract and Figures

This paper proposes discussions for the (re)negotiation of responsibilities in performances that involve a collaboration between human and computer agents. Most of current research is human-hierarchical, leaving the machine the role of a mere tool at performer's service. In this paper a more balanced distribution of responsibilities between the two agent is proposed. Chimney, a software developed by one of the authors, is proposed as a design probe to reflect on this topic. Chimney allows the composer to control only the musical material of a piece, leaving its evolution to an algorithmic agent whose decisions are unpredictable. This redistribution of responsibilities results in a compositional shift that causes the roles of the composer / performer to overlap. The implications for the experience of the performer operating Chimney are also discussed.
Content may be subject to copyright.
RENEGOTIATING RESPONSIBILITIES IN
HUMAN-COMPUTER ENSEMBLES
Fabio Morreale
Raul Masu
Centre for Digital Music, EECS
Queen Mary University of London
f.morreale@qmul.ac.uk
interAction Lab
University of Trento
raul.masu@unitn.it
Figure 1. Chimney, the software used as a probe to reflect on new scenarios of human-machine collaborations.
ABSTRACT
This paper proposes discussions for the (re)negotiation
of responsibilities in performances that involve a collab-
oration between human and computer agents. Most of
current research is human-hierarchical, leaving the ma-
chine the role of a mere tool at performer’s service. In
this paper a more balanced distribution of responsibilities
between the two agent is proposed. Chimney, a software
developed by one of the authors, is proposed as a design
probe to reflect on this topic. Chimney allows the com-
poser to control only the musical material of a piece,
leaving its evolution to an algorithmic agent whose deci-
sions are unpredictable. This redistribution of responsi-
bilities results in a compositional shift that causes the
roles of the composer / performer to overlap. The impli-
cations for the experience of the performer operating
Chimney are also discussed.
1. INTRODUCTION
At the dawn of the last century, the digital revolution of-
fered musicians and researchers the possibility to explore
new forms of musical creativity using computers. Be-
sides offering unprecedented interaction possibilities
thanks to novel instruments and interfaces, musicians
started experimenting with new compositional strategies
by detaching human responsibilities to the compositional
process, which were partially or completely delegated to
computers. Such strategies had been extensively ex-
plored by George Lewis, one of the pioneer musical
pieces that “emerges from a nonhuman intelligence” [1].
Copyright: © 2016 Morreale an d Masu. This is an open-access article
distributed under the terms of the Creative Commons Attribution License
3.0 Unported, which permits unrestricted use, distribution, and repro-
duction in any medium, provided the original author and source are cre-
dited.
In his work, he explored performance interactions be-
tween improvising musicians and computer programs
that create new music material at performance time.
The work described in this paper keeps up with the
encounter between human and algorithmic agents to col-
laboratively operate to create new music. In particular,
we propose discussions about the role of the machine in
such hybrid ensemble. Traditionally, computer agents
have been relegated as tools to serve the performer. We
here advocate a more equal redistribution of the roles of
the two agents. Chimney, a software that shares the con-
trol over the performance between the musician and the
computer, is presented as a probe for discussion. In
Chimney the musician-part of the control over the com-
position is reduced to (i) selecting the musical material
to be played during the execution; and (ii) deciding their
likelihood of being played temporally closer to each
other.
To emphasise the non-hierarchical nature of the en-
semble, a visual identity was assigned to the computer
agent. An algorithmic random walker unsystematically
roams throughout the screen: as it encounters an object,
the sound associated to that specific object starts playing
with a sound level which is proportional to its distance to
the walker. The human agent can only decide the objects
to be placed on the canvas, their position, and their size.
Under these conditions, the musician cannot organise a
temporal structure, which is entirely controlled by the al-
gorithmic agent.
Such renegotiation of responsibilities results in two
principal compositional shifts: (i) the music is no longer
organised according to the phraseological-temporal
structure; (ii) the sound objects become the focus of the
composition. As a consequence, the system fosters the
musician to elaborate new compositional strategies.
It is worth remarking that the scope of interest of this
paper lies beyond music and art domains. Chimney be-
longs to a recent corpus of work of the authors which
aims at as well as other works from the authors [4] [5]
that encompass reflections on philosophical and cultural
concerns. Our pieces are probes that pose questions and
speculate about possible future scenarios of computa-
tional art. For instance, what would be the consequences
of (partially) delegating artistic creation to an autono-
mous agent? How would the role of art and the artist
change?
The remainder of the paper is structured as follows.
The next section presents the technical details of Chim-
ney; Section 3 describes the computational shift that a
composer needs to face when utilising this system; Sec-
tion 4 presents Alinearity, a piece written by one of the
authors for Chimney and a trumpet; Section 5 discusses
more insights about the experience of the trumpet player
that interacted with the system.
2. CHIMNEY
Chimney is a multi-platform open source application de-
veloped with Processing1. Quite an efforts was made to
1 https://processing.org
build a strong visual identity to the algorithm for two rea-
sons: (i) to provide the algorithmic agent a character,
thus emphasising the absence of hierarchy between the
two agents; (ii) to help the audience understand the per-
formance. The visual interface is composed of a canvas
that displays the status of the algorithmic walker and the
sonic material.
2.1 The random walker: the fly
The canvas is initially blank with the exception of a par-
ticle - the random walker - that moves throughout the
screen. The idea was to give the walker a behaviour that
recalled that of a fly. For this reason, we based the fly on
an adapted version of the Perlin Noise [6]. The Perlin
Noise is a random generator function originally devel-
oped to produce natural looking textures on computer
generated surfaces. This function produces a more natu-
ral, harmonic succession of numbers than that of pure
random functions.
To the original Perlin Noise function we added a few
features to match a our requirements. In particular, the fly
had:
1. to cover the entire space of the canvas;
2. to be independent of the presence or absence of
other objects in the canvas;
3. to have non-deterministic movements: the path
taken by the fly cannot be predicted nor controlled
in any possibly ways by the musician.
Figure 2 shows a possible path taken by the fly over a
period of two minutes.
Figure 2. A possible trajectory of the fly.
2.2 The compositional material: sound circles
The performer can interact with the canvas by adding
sound sources to it. These sound sources are displayed as
circles (Figure 1), whose placement and radius can be
controlled by the user interacting with the mouse. Once
positioned in the canvas, the performer can interact with
it by changing its size, by repositioning it and by deleting
it. Every time a new circle is added to the canvas, the
system sets it in idle state and mutes it. As the fly enters
a circle, the sound connected to that circle increases. The
maximum level is reached when the fly is at the centre of
the circle.
The sound sources are made of pre-recorded excerpts
stored as audio files. The user can access these files
through a sidebar that lists all the sound files contained
in a specific folder (Figure 1, on the left). Once selected
a new sound source, the next circle to be positioned in
the canvas will be associated to that sound source. A col-
our code helps identifying what sound each circle is as-
sociated to.
3. COMPOSITIONAL SHIFT
The theoretical ideas that lie behind the compositional
approach to Chimney was based on the early approach to
composition and improvisation of John Cage. In particu-
lar, during his first improvisation period (following the
Feisst’s analysis of Cages’ work [2]), Cage categorised
compositional process as a fourfold activity composed
of: structure, material, method, and form. The structure
is the temporal division of the sections, the material con-
sists of the sound objects (including noise and silence),
the method is the “note to note procedure”, and the form
is the “morphological line of the sound-continuity”. Fol-
lowing Cage’s opinion, material and methods can be both
improvised and composed. Form cannot be composed
but only improvised, as it results from the sum of the
other activities. Finally, structure is the only element that
cannot be improvised.
Having control on the structure allows a composer to
be in charge of the evolution of the piece. In fact, he can
move back and forth on the music timeline. Sarath for-
mulated this concept proposing the term expanding tem-
porality of a compositional process [7]. Given that the
composer does not work at performance time, the musi-
cal decisions are not directly influenced by the previous
events. The composer can freeze time, re-think passages,
and anticipate future musical phrases. As a result, the de-
velopment of the piece is based not only on previous
events, but also on the anticipation of the material to be
presented at successive moments. A different kind of
temporality is the inner directed temporality [7], a con-
cept proposed to describe the evolution typical of im-
provisation. When improvising, a musician cannot pre-
cisely anticipate how the music will develop. Thus, im-
provisers can mostly focus their attention on the present,
with little attention to the near past and future.
Chimney offers the composer the possibility to rede-
fine the edge of composing and improvising. Given that
the temporal evolution of the piece is not under his or her
control, the structure can no longer be composed. Chim-
ney restrains human control over the development of the
piece as the composer cannot step back and forth and re-
compose a particular event. As the music is shaped at run
time, the performance itself represents the final compo-
sitional process. As a consequence, performing with
Chimney requires an inner directed temporality ap-
proach. However, at the same, it is not pure improvisa-
tion: despite the freedom to take real time decisions the
performer cannot improvise the actual musical material.
The musical material itself is the element that bounds
performance to composition. The musical gesture, lim-
ited to organising pre-composed music, is composed out-
2 https://cycling74.com/products/max/
side before the performance, thus it is closer to the con-
cept of expanding temporality. The musical material be-
comes the very core of the piece and can be seen as a
meta composition of the music, which ultimately gener-
ates at performance time.
The method touches both the compositional and the
improvisational aspects of music making. It is involved
both in the creation of the sound sources and the strate-
gies adopted to place them on the canvas. The form and
the structure are both determined during the perfor-
mance, and consequently are improvised elements.
4. ALINEARITY
This section exemplifies the innovative aspects of Chim-
ney by presenting Alinearity, a piece of music composed
for Chimney by one of the authors. Alinearity was com-
posed to provide the performer with the necessary degree
of freedom to properly interact with Chimney while
maintaining a degree of harmonic coherence. The piece
can be performed as a solo piece for Chimney or as a duo
for Chimney and a melodic instrument. Alinearity was
performed in 2015 at the International Society for Impro-
vised Music Festival (Château-d'Oex, Switzerland) in a
version for Chimney and 10 string electric guitar and in
2016 at the International Conference on the Design of
Cooperative Systems 2016 (Trento, Italy) for Chimney
and Trumpet.
4.1 Description of the piece
The most important aspect of the compositional process
adopted for Alinearity is the creation of sound objects as
the main elements of the piece. The musical form is not
determined by the development of musical material mov-
ing toward a specific section. Thus, there are no cadences
and the material itself does not evolve. The development
of the piece is based on the overlapping of different
sound excerpts that create different situations. The musi-
cal form is shaped by the very transit of the fly through
different cluster of sounds (the circles in the canvas).
This transit can be gradual or sudden, depending on the
behaviour or the fly.
The musical material was composed to be congruous
and at the same time to open a wide range of possible
combinations that guarantee coherence and expressive-
ness to the piece. It consists of 20 short monodic musical
excerpts, each with a timbral, melodic, harmonic, and
rhythmic value pre-recorded using a synthesiser made
with Max-Msp2. The synthesiser is controlled with stand-
ard MIDI messages.
4.2 Harmony
The harmony of the excerpts is organised as a politonal-
ity over two tonality centres: Gmin and C#. The note of
the excerpts can either belong to one specific scale or to
both scales (C, which enharmonically corresponds to B#,
F#, and Bb, which enharmonically corresponds to A#).
More specifically, the musical excerpts are harmonically
organised as follows:
7 excerpts belonging to Gmin;
7 excerpts belonging to C#;
6 excerpts belonging to both tones.
Their combination can create situations with distinct key
notes, polytonal harmonies, or situations with a modal
appearance that lacks a clear tonic centre.
4.3 Melody and rhythm
From a melodic point of view, the excerpts are clustered
into three sets. They can be long pedal notes, very short
patterns, and short themes:
6 long pedal notes;
6 short melodies;
8 short patterns.
The harmony of the six long pedal note belongs to both
tonal centres. The short patterns have sharper rhythm and
can create particularly complex polyrhythmic structures.
4.4 Timbre and register
The excerpts spread across five octaves. Long notes have
low register and short notes have high register. As a con-
sequence, the six long pedal notes are the lowest register,
the six short melodies lays in the medium register, and
the eight short patterns have the highest register.
The core of the synthesiser is a bank of eight resonant
band pass filters, which processes white noise in input.
The central frequencies of the filters are composed of a
fundamental tone and its seven higher harmonic partials.
The resulting sound can be accurately manipulated by in-
teracting with its nosiness and brightness, as well as its
envelope. Lower notes have a slow transients and a high
level of nosiness. Higher notes have fast transients and
low nosiness.
5. EXAMINING EXPERIENCES
Alinearity can be performed as a solo interface or in a
duo with a melodic instrument. This sections analyses the
different experiences of the musicians who (a) performs
Chimney, and (b) plays the melodic instrument playing
along Chimney.
5.1 Performing Chimney
Given that Chimney has been mainly operated by one of
the authors, the following discussion is based on self-
analysis of the experience. When using Chimney, com-
position and performance are superimposed. The emer-
gence of this hybrid composer-performer was thoroughly
described by Vallis [8][9]. The implications of this su-
perimposition on the iteration strategies adopted during
the performance are substantial. Four main gestures, or
approaches to interaction, with Chimney emerged.
Placing excerpts and spectating the movements of the fly
In this first approach, the performer does not consider the
movements of the fly at all. The gesture is guided by his
knowledge of the musical material, thus by placing the
excerpts on the canvas. The activities of the human per-
former have no effects on the musical output, who is rel-
egated to being a spectator of the performer. Depending
3 An excerpt of the exhibition can be found at
https://youtu.be/8-zVa1tN3Ak
on the movements of the fly, the actions of the musician
can also fail to influence the music at all as the fly could
potentially never reach the circles. From a musical per-
spective, this approach is the closest approach to tradi-
tional composition. Indeed, it operates in an expanded
temporality, and all the performers’ activities result in
long term effects. This approach is typically adopted at
the beginning of a performance.
Fostering a gradual passage to new situations
In this approach, the performer positions in the canvas a
limited amount of excerpts and follows the movement of
the fly. As a consequence, the control is more balanced
between the human and the algorithm. The performer fol-
lows the changes introduced by the transit of the fly to a
new circle by adding or subtracting elements from the
canvas. This approach reduces the time spanning to a
more localised present. The performer can focus on the
previous events and on the changings that are occurring.
The time window could be fairly wide but always centred
on the present.
Following the fly to increase music complexity
In this approach, the performer closely follows the move-
ment of the fly by adding circles to its trajectory. The hu-
man has the highest control over the composition, thus
the stochastic aspect is consequently less influent. As the
performer gesture are closely related to the movement of
the fly the temporarily is more focused on the present,
similarly to the inner directed case earlier described
(Section 3). This approach is normally adopted to create
local musical climax and results in a complex structure.
Such complexity results from the number of overlapping
excerpts that is achieved by adding several circles that
follow the fly. The outcome is a much more complex tex-
ture articulated by a bigger number of musical objects.
Furthermore, the music becomes richer and the quantity
of rest decreases.
Removing all the musical excerpts
Chimney consents to delete all the excerpts that are visi-
ble on the canvas at once. In this case, the musical gesture
is similar to what typically occurs with traditional instru-
ments: to one action corresponds one musical outcome.
The temporality is focused on the very time of the key
pressing, but is influenced by recent developments in the
music. This kind of interaction normally occurs only
once during a performance at the end of the last climax.
The sudden rest that results from this gesture resolves the
tension and gently goes towards the end of the piece.
When Alinearity was performed as a duo piece for Chim-
ney and Trumpet at COOP 2016 in Trento (Figure 3)3,
peculiar human-human interactions occurred. The shift-
ing among the different approaches was determined both
by the personal taste on the musical output as well as by
the input received by the other musician. The trumpeter
sometimes proposes shifting that did not follow the
movement of the fly. The operator of Chimney - one of
the authors of the paper - who was controlling Chimney
was thereby required to pay careful attention on the trum-
pet. In these case, gradual transits to new situations were
not determined by the fly movements but rather by the
other musician.
Figure 3. A trumpeter improvising along to Chimney.
5.2 Improvising along to Chimney
This section describes the trumpeter experience of play-
ing along to Chimney. The trumpeter had no score to fol-
low and was instructed about how the musical excerpts
were composed. He was also given some time rehearsing
the piece to become familiar with the software and the
musical material.
After the concert, he was asked to elaborate on his
experience of playing along to Chimney. The objective
was to understand his musical strategies and to collect
reflections about the interaction among the three agents
(the trumpeter, Chimney, and the performer of Chim-
ney).
The trumpeter reported that understanding all the de-
tails of the music played by Chimney was particularly
demanding. In particular, as opposed to traditional im-
provisation, he did not feel free to propose new musical
material. He was always answering to Chimney pro-
posals. Each note, or phrase, was indeed primarily based
on listening. As a result, his gestures were mostly about
imitating, completing, or opposing what Chimney pro-
posed.
From a musical point of view, the trumpeter was
mostly influenced by harmony and rhythms proposed by
Chimney. The approaches he adopted can be clustered
into three main categories:
using exactly the same notes of the excerpts;
using the same scale of the excerpts;
moving gradually from one scale to the other.
The rhythm could be influenced by the single pattern or
by the overlapping of more than one. In the first case, the
excerpts themselves determined the rhythmical result. In
the second case, he was instead mostly influenced by the
movement of the fly. With respect to the form, the trum-
peter tended to follow the output of Chimney, enhancing
his rhythm and loudness coherently with the overlapping
of more excerpts.
When prompted to discuss his relation with the two
other agents, the trumpeter reported that he experienced
two distinct interactions. He perceived to be in closer
contact with Chimney as he felt he was duet-ting with it.
The performer controlling Chimney was only the access
to ask something to the computer agent. Only in a few
cases the main connection was between the two humans.
This was particularly the case when the performer was
following the fly or when he cleared the canvas. In these
cases, Chimney was considered closer to a musical in-
strument.
He also commented that he would have preferred a
higher complexity in the music generated by Chimney.
In those situations, the trumpeter compensated for this
perceived deficiency by increasing the rhythm complex-
ity or the loudness of his instrument. To conclude, the
inherent limitation of Chimney of having a limited mate-
rial and being non-responsive fostered the trumpeter to
find novel musical strategies.
6. CONCLUSIONS
Chimney was mainly developed as a design probe to re-
flect on the consequences of delegating part of the com-
positional process to an algorithmic agent. In this con-
text, a number of negotiations between the human and
the computer agent have to take place. The implication
of such negotiations are evident from a musical perspec-
tive and from the performer’s experience alike. With re-
spect to the musical perspective, the focus of composi-
tional effort shifts to the interaction strategies more than
the form, and a number of compositional decisions are
left to performance time. As a consequence, the distinc-
tion between composition and improvisation, which is
neat in traditional performances, becomes blurred.
The disruption of built-in hierarchies of human-
leads-computer-follows also caused a series of implica-
tions in the experience of the performer who is control-
ling Chimney and of other musicians that play along to
it. Whereas some of these implications can be attributed
to the non-responsive nature of the system, others are at-
tributable to the distinctive character of the interaction.
In particular, both Chimney performer and the musician
that improvises along to it, are withhold the possibility to
control all the aspects of the performance. Freed by a pre-
cise control on musical structure and declined the possi-
bilities to have veto buttons, they need to surrender a
more balanced collaboration with the algorithmic coun-
terpart.
7. ACKNOWLEDGEMENTS
We are grateful to Riccardo Terrin for having performed
Alinearity, and to the organisers of the COOP and ISIM
conference.
8. REFERENCES
[1] D. Charles. For the Birds. Boston: Marion Boyars,
1981.
[2] S. M. Feisst. John Cage and Improvisation: An Un-
resolved Relationship. Musical Improvisation: Art,
Education, and Society: 38-51. 2009.
[3] G. E. Lewis. “Interacting with Latter-Day Musical
Automata”. Contemporary Music Review 18(3):
99112, 1999
[4] R. Masu, A. Conci, Z. Menestrina, F. Morreale, A.
De Angeli. “Beatfield: An Open-Meaning Audio-
visual Exploration”, 2016
[5] F. Morreale. “Generative Everything 2083”. In Pro-
ceedings of the XVIII Generative Art Conference,
2015
[6] K. Perlin. "An image synthesizer." ACM Siggraph
Computer Graphics 19, no. 3: 287-296, 1985
[7] Sarath, Ed. "A new look at improvisation." Journal
of Music Theory (1996): 1-38.
[8] O. S. Vallis, Contemporary approaches to live
computer music: the evolution of the performer
composer. Victoria University of Wellington, 2013.
[9] O. S. Vallis, A. Kapur. "Community-based design:
The democratization of musical interface
construction." Leonardo music journal 21, 2011:
29-34, 2011.
... This paper proposes another example in which part of the control is delegated to the machine. Chimney, a software instrument developed by two of the authors [6], forces the musician to delegate timing decisions to a non-responsive autonomous agent. As a consequence, the human agent has limited control on the temporal evolution of the piece. ...
... An example of DMI in which the musician divests control on timing to the machine is offered by Chimney [6]. In this virtual DMI the control on the temporal evolution of the piece is delegated to a computer agent, an algorithmic random walker. ...
... An informal evaluation was conducted at a public concert in which Chimney was perfomed in a duo with a trumpet [6]. Both the musician that was controlling Chimney (the second author of this paper) and the trumpeter were interviewed at the end of the performace. ...
Conference Paper
Full-text available
Research on musical instrument design suggests that deliberate constraints can offer new creative experiences to the performer. At times, design constraints are physically embedded in the instrument to limit the interaction possibilities ; in other cases the constraints are given by delegating part of the control to the machine. In this paper we propose a case study related to the latter form of constraints: in Chimney the musician delegates control on timing to an autonomous agent. Elaborating on opinions collected from musicians interacting with Chimney, we reflect on the consequences of delegating part of the control to the machine. Without the possibility of influencing the temporal evolution of the piece, the human performer is pushed to find creative workarounds and to surrender to a more balanced collaboration with the machine.
... The performance took place in Trento, during the 12th International Conference on the Design of Cooperative Systems (figure 2). A complete description of the design of the instrument and the performance can be found in [28]. ...
... The DMI: Chimney. Chimney [28], is an open-source software application developed in Processing. In Chimney, the DMI composer/performer can select the musical patterns to be played during the performance, by uploading them in the software database before the performance. ...
Conference Paper
This paper presents ARCAA (Actors, Role, Context, Activity, Artefacts), a framework that supports designers in understanding the artefact ecology in the music performance scenario, in particular, allowing to frame the role of the different actors. The ARCAA framework is the result of the combination of two different areas of HCI: artefact ecology concept, and design framework for digital musical instruments. The model borrows three categories from MINUET an established design framework and rethinks them from an ecological perspective. In ARCAA, these three categories are used as three lenses to connect each human actor to her artefact ecology. Finally, the framework allows comparing how the various artefacts create connections among the different people involved. The second part of the paper describes a case study that shows a practical adoption of the framework.
... Before presenting some discussion on the usage of the score, we provide a brief description of the system and a previous work. A more comprehensive description of technical details can be found in a paper by Morreale and Masu [19]. ...
... Chimney forces the composer to devise a specific compositional approach, finding a new balance between composition and improvisation. The paper by Morreale and Masu presents the shifting from a time-based to a sonic object composing approach describing in detail a composition named Alinearity [19]. Alinearity could be performed as a solo piece or in duo with a trumpeter. ...
Conference Paper
Full-text available
This paper proposes the idea that a score for a Digital Musical In- strument (DMI) can be approached from a design perspective. In particular, we focus on the concepts of affordances and constraints. The paper describes the use of scores in a piece called XXV composed for Chimney, a DMI and Cello. Both the piece and the system are detailed with a focus on design concepts. In the conclusion, we compare our approach to other literature that discuss the use of score with DMIs. Based on this, and the results obtained, we present recommendations regarding the use of score in DMIs. Future work is also outlined.
... Recent theories [7,8] proposed considering musical creativity as a distributed network of musical agencies. This concept was originally proposed in the context of algorithmic music: distributing agencies to non-human performers can spark new forms of creativity [8,32]. Comments from our survey suggest that the artistic practice of 7 performers can be analysed under the lens of musical agencies redistribution. ...
Conference Paper
Full-text available
The term 'NIME'-New Interfaces for Musical Expression-has come to signify both technical and cultural characteristics. Not all new musical instruments are NIMEs, and not all NIMEs are defined as such for the sole ephemeral condition of being new. So, what are the typical characteristics of NIMEs and what are their roles in performers' practice? Is there a typical NIME repertoire? This paper aims to address these questions with a bottom up approach. We reflect on the answers of 78 NIME performers to an on-line questionnaire discussing their performance experience with NIMEs. The results of our investigation explore the role of NIMEs in the performers' practice and identify the values that are common among performers. We find that most NIMEs are viewed as exploratory tools created by and for performers, and that they are constantly in development and almost in no occasions in a finite state. The findings of our survey also reflect upon virtuosity with NIMEs, whose peculiar performance practice results in learning trajecto-ries that often do not lead to the development of virtuosity as it is commonly understood in traditional performance.
Article
Full-text available
Human-AI co-creativity involves both humans and AI collaborating on a shared creative product as partners. In a creative collaboration, interaction dynamics, such as turn-taking, contribution type, and communication, are the driving forces of the co-creative process. Therefore the interaction model is a critical and essential component for effective co-creative systems. There is relatively little research about interaction design in the co-creativity field, which is reflected in a lack of focus on interaction design in many existing co-creative systems. The primary focus of co-creativity research has been on the abilities of the AI. This paper focuses on the importance of interaction design in co-creative systems with the development of the Co-Creative Framework for Interaction design (COFI) that describes the broad scope of possibilities for interaction design in co-creative systems. Researchers can use COFI for modeling interaction in co-creative systems by exploring alternatives in this design space of interaction. COFI can also be beneficial while investigating and interpreting the interaction design of existing co-creative systems. We coded a dataset of existing 92 co-creative systems using COFI and analyzed the data to show how COFI provides a basis to categorize the interaction models of existing co-creative systems. We identify opportunities to shift the focus of interaction models in co-creativity to enable more communication between the user and AI leading to human-AI partnerships.
Preprint
Full-text available
Human-AI co-creativity involves both humans and AI collaborating on a shared creative product as partners. In a creative collaboration, interaction dynamics, such as turn-taking, contribution type, and communication, are the driving forces of the co-creative process. Therefore the interaction model is a critical and essential component for effective co-creative systems. There is relatively little research about interaction design in the co-creativity field, which is reflected in a lack of focus on interaction design in many existing co-creative systems. The primary focus of co-creativity research has been on the abilities of the AI. This paper focuses on the importance of interaction design in co-creative systems with the development of the Co-Creative Framework for Interaction design (COFI) that describes the broad scope of possibilities for interaction design in co-creative systems. Researchers can use COFI for modeling interaction in co-creative systems by exploring alternatives in this design space of interaction. COFI can also be beneficial while investigating and interpreting the interaction design of existing co-creative systems. We coded a dataset of existing 92 co-creative systems using COFI and analyzed the data to show how COFI provides a basis to categorize the interaction models of existing co-creative systems. We identify opportunities to shift the focus of interaction models in co-creativity to enable more communication between the user and AI leading to human-AI partnerships.
Chapter
This paper presents Beatfield, a musical installation that allows players to explore an audiovisual landscape by positioning tangible objects on an augmented game board. The underlying idea of the installation was the proposition of an artefact that could encourage heterogeneous interpretations. Beatfield had to offer a multitude of interpretations and ways of appropriating the system; there would be not a right or wrong way to play with it. To this end, the design of the installation integrated related work on open-ended interaction, ambiguity, and appropriation with enigmatic aesthetics, ambiguous interaction strategies, and unpredictable mapping between user input and audiovisual output. The results collected from a user study confirmed the potential of the installation to stimulate a variety of different experiences and interaction strategies.
Article
The advent of on-line communities has democratized the process of musical interface design and allowed users to directly participate in the future development of the devices they use. On-line communities, acting as centralized repositories for information pertaining to the development of an interface, allow users to discuss their experiences and ideas as well as providing a framework for managing information pertaining to an interface. This centralized access to information regarding the design, use and development of an interface both focuses and accelerates the developmental process.
Article
The idea of music that somehow plays itself, or emerges from a nonhuman intelligence, is a common, transculturally present theme in folklore, science, and art. Over the centuries, this notion has been expressed through the development of various technological means. This paper explores aspects of my ongoing encounter with computers in improvised music, as exemplified by my most recent interactive computer music compositions. These works involve extensive interaction between improvising musicians and computer music-creating programs at the performance (“real-time”) level. In both theory and practice, this means that both human musicians and computer programs play central organizing and structuring roles in any performance of these works. This paper seeks to explore aesthetic, philosophical, cultural and social implications of this work. In addition, the nature and practice of improvisation itself will be explored, since an understanding of this ubiquitous musical activity is essential to establishing the cultural and historical context of the work.
Article
The authors introduce the concept of a Pixel Stream Editor. This forms the basis for an interactive synthesizer for designing highly realistic Computer Generated Imagery. The designer works in an interactive Very High Level programming environment which provides a very fast concept/implement/view iteration cycle. Naturalistic visual complexity is built up by composition of non-linear functions, as opposed to the more conventional texture mapping or growth model algorithms. Powerful primitives are included for creating controlled stochastic effects. The concept of 'solid texture' to the field of CGI is introduced. The authors have used this system to create very convincing representations of clouds, fire, water, stars, marble, wood, rock, soap films and crystals. The algorithms created with this paradigm are generally extremely fast, highly realistic, and asynchronously parallelizable at the pixel level
John Cage and Improvisation: An Unresolved Relationship
  • S M Feisst
S. M. Feisst. John Cage and Improvisation: An Unresolved Relationship. Musical Improvisation: Art, Education, and Society: 38-51. 2009.
Generative Everything
  • F Morreale
F. Morreale. "Generative Everything 2083". In Proceedings of the XVIII Generative Art Conference, 2015
Contemporary approaches to live computer music: the evolution of the performer composer
  • O S Vallis
O. S. Vallis, Contemporary approaches to live computer music: the evolution of the performer composer. Victoria University of Wellington, 2013.