Conference PaperPDF Available

Composing for Hyperbow: A Collaboration Between MIT and the Royal Academy of Music.

Authors:

Abstract and Figures

In this paper we present progress of an ongoing collaboration between researchers at the MIT Media Laboratory and the Royal Academy of Music (RAM). The aim of this project is to further explore the expressive musical potential of the Hyperbow, a custom music controller first designed for use in violin performance. Through the creation of new repertoire, we hope to stimulate the evolution of this interface, advancing its usability and refining its capabilities. In preparation for this work, the Hyperbow system has been adapted for cello (acoustic and electric) performance. The structure of our collaboration is described, and two of the pieces currently in progress are presented. Feedback from the performers is also discussed, as well as future plans.
Content may be subject to copyright.
Composing for Hyperbow: A Collaboration Between
MIT and the Royal Academy of Music
Diana Young
MIT Media Laboratory
20 Ames Street
Cambridge, MA 02142, USA
diana@media.mit.edu
Patrick Nunn
Royal Academy of Muisc
Marylebone Road
London NW1 5HT, UK
patrick@patricknunn.com
Artem Vassiliev
Royal Academy of Muisc
Marylebone Road
London NW1 5HT, UK
avassiliev@hotmail.com
ABSTRACT
In this paper we present progress of an ongoing
collaboration between researchers at the MIT Media
Laboratory and the Royal Academy of Music (RAM). The aim
of this project is to further explore the expressive musical
potential of the Hyperbow, a custom music controller first
designed for use in violin performance. Through the creation
of new repertoire, we hope to stimulate the evolution of this
interface, advancing its usability and refining its
capabilities. In preparation for this work, the Hyperbow
system has been adapted for cello (acoustic and electric)
performance. The structure of our collaboration is described,
and two of the pieces currently in progress are presented.
Feedback from the performers is also discussed, as well as
future plans.
Keywords
Cello, bow, controller, electroacoustic music, composition.
1. INTRODUCTION
1.1 Background and Motivation
As the field of new music interfaces grows, great
achievements are being made in the creation of controllers,
as well as in composition and performance techniques
featuring them. Alternative controllers, such as The Hands of
Waisvisz [18], have been demonstrated to have truly
virtuosic capabilities. Controllers inspired by traditional
acoustic instruments, such as Tarabella’s Imaginary Piano
[16], Burtner’s MetaSax [1], Scavone’s Pipe [14], and the
Hyper-Flute of Palacio-Quintin [9], extend the sonic
possibilities, performance techniques, and metaphors of
their counterparts. New string interfaces are a subset of these
[2-4, 6-8, 11-12, 17].
All of the above interfaces represent many different
approaches to sensor design, mapping, composition, and
performance, yielding great musical and intellectual rewards.
However, it is quite difficult to find cases of new music
interfaces that have the benefits of a large associated
repertoire and large group of dedicated players. Of course,
these are not simple features to attain.
Often new interfaces are designed primarily for the use of the
designer alone, or for a small number of select musicians.
But even when the desire to disseminate is great, numerous
practical issues, such as the robustness of the hardware and
software involved, often interfere with the goal to make
controllers available to more users. And, of course, it is often
difficult to decide on the appropriate time in a new
controller’s development for feedback from a larger audience
and user group, as the urge to iterate is ever-present.
Our controller, the Hyperbow [19], is one of the many that
lack two of the basic characteristics, a large repertoire and
large group of players, possessed by well-established
instruments. The Hyperbow was built for use in real-time
violin performance applications, such as required in Tod
Machover’s Toy Symphony, in which the Hyperbow made its
debut. Given the rigorous demands bowed string players
make on their instruments, as well as the impositions of a
touring musical work, great care was taken to design the
system with a high level of playability, usability, and
robustness.
The Hyperbow performed well, achieving its task of an
expressive new music controller that enables traditional
violin technique. The Hyperbow system was successfully
used in four public performances of Tod Machover’s
ToySymphony (2001-2002) by two different violinists. Later,
it was featured in a performance of Michael Alcorn’s
Crossing the Threshold for the opening of SARC (2004).
However, although the Hyperbow had been used in
performance by several players (who produced invaluable
feedback concerning their experiences), it still cannot be
considered in the same class as well-established acoustic
and electric instruments. If the Hyperbow is ever to achieve
the unqualified status of a real music instrument, it must not
only provide wonderful new sonic possibilities, but must
also be associated with a significant repertoire and have
many players. Additionally, it must be easily maintained by
these players, and ultimately, have its own performance
practice.
In order to achieve these goals, it is essential that the
Hyperbow be placed in the hands of more composers and
performers. It is this belief that is the core motivation of this
collaborative work.
2. HYPERBOW REVISIONS
The first Hyperbow system [20], a descendent of the original
MIT Hyperstring Project [6], was designed to capture
elements of violin bowing gesture for use in real-time
performance applications. Installed on a commercial carbon
fiber violin bow and electric violin, it features
accelerometers on the frog, force sensors (composed of strain
gauges) mounted on the bow stick, and an electric field
position sensor that includes an antenna mounted behind
the bridge of the violin. This last component of the sensor
system is an adaptation of the position sensor first used in
the Hypercello project [10].
The Hyperbow system is battery-powered and transmits its
sensor data wirelessly via an RF communication module. An
external electronics board receives the data and sends it to
the computer via the serial/USB port.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and
that copies bear this notice and the full citation on the first page. To
copy otherwise, or republish, to post on servers or to redistribute to
lists, requires prior specific permission and/or a fee.
NIME 06, June 4-8, 2006, Paris, France.
Copyright remains with the author(s).
In January 2005, when this collaboration officially began,
the Hyperbow technology was already several years old.
Though we could have significantly redesigned the hardware
in preparation for this new work, we opted not to do this.
Instead, we decided to postpone a major revision to a later
date when the process could benefit from exposure to more
composers and performers.
However, because of the different range of bowing movement
required by the cello and the fact that we would be using
acoustic cellos, some small but critical revisions of the
existing hardware were required. Specifically, the electric
field position sensor was adapted to produce a signal with
greater amplitude emitting from the bow (so that it could be
detected by the original receiver from the greater bow-bridge
distance of the cello).
When used for violin performance, the position antenna was
mounted directly on the solid body of a Jensen electric
violin by means of a simple screw and right angle bracket.
Obviously, such a scheme was not possible for our work
with acoustic cello, so the antenna was fixed to a threaded
rod, which was then mounted on the underside of the
tailpiece by means of a plastic clamp. (This was a variation
of an arrangement used in the Digital Stradivarius controller
[15].)
Figure 1:Hyperbow and acoustic acoustic cello (© Roberto
Aimi).
Interestingly, although the antenna mounting just described
was functional, the musicians determined that it was easier
and more convenient to attach the antenna to the strings
with tape (behind the bridge). Also, they sometimes found it
useful to shift the location of the antenna with respect to the
center of the bridge, as pictured in Figure 1. These
improvisations arose as the participants experimented with
the Hyperbow and were taken as positive indications of
increasing comfort with and ownership of the technology.
3. A NEW COLLABORATION
3.1 Structure
This project includes researchers from the MIT Media
Laboratory, whose primary role is that of interface design,
and composers and cellists from the Royal Academy of
Music.
Two Hyperbows for cello were built at MIT and then
transferred to the RAM’s permanent instrument collection.
Though we were interested in an equal exchange of
information and ideas, one of our intentions was for the
artists at RAM to create, rehearse, and perform without any
need for outside technology support, and to be able to freely
develop their own individual work processes.
3.2 Schedule
This collaboration began in January 2005, at which time the
collaborators from MIT traveled to London for a week long
workshop. Two Hyperbows (slightly revised, as described
below) were presented to the colleagues at RAM, and the
week began by imparting technical knowledge of the
Hyperbow system and related software such as Max/MSP.
After these introductory exercises, each composer/cellist
team began the work of creating the first compositional
sketches of the project. These evolved throughout the week,
and on the last working day we made a presentation to the
RAM community.
The composers continued to independently develop their
pieces through the spring of 2005, and in June 2005, the
group met again, this time in Boston. After another week of
concentrated work together, we presented the progress at
MIT.
In November 2005 during the Association Européenne des
Conservatoires Congress 2005 at Birmingham
Conservatoire, the first performance for the outside
community was given of two of the works in development,
Patrick Nunn’s Gaia Sketches, and Artem Vassiliev’s
MODES. These two pieces were performed again a month later
at RAM for the third research seminar on the Hyperbow
collaboration entitled “New Tools, New Uses”, and are
described below by their composers.
4. COMPOSING FOR HYPERBOW
As described above, there have been four public
presentations of new works for Hyperbow and cello to date.
During the last two events (Novemeber and December 2005),
two new compositions were performed. Gaia Sketches, by
Patrick Nunn, incorporates the Hyperbow with acoustic
cello, while MODES, by Artem Vassiliev, features an electric
cello. Below, the composers discuss their works.
4.1 Gaia Sketches, by Patrick Nunn
4.1.1 Approach
As part of my own research, my intention was to explore
ways of extending the timbre of the acoustic cello in a
manner that would feel natural to the player without the need
to change their existing technique. By applying a method of
direct mapping of bow gesture data to control parameters,
the incoming audio signal from the acoustic cello could be
coloured and transformed.
This particular approach stems from a desire to create a
closer relationship between the performance gesture and
audio processing. This method of design permits the player
to actively feel and control the electronic processing, rather
like an extension of his/her own instrument.
The decision to use acoustic cello as opposed to the electric
cello was made on aesthetic and conceptual grounds. The
greater range of sonorities and resulting range of expression
obtained from the acoustic cello provided a wider contrast to
the sounds produced after the signal was processed. In
addition, the score requires the cellists sound to be localized
to their position in performance without the need for
separate amplification.
In order to obtain a clean audio signal from the acoustic
cello and to eliminate maximum external audio pollution, a
Fisherman C-100 cello pickup was attached to the bridge
4.1.2 Concept
Gaia Sketches were inspired by Rachel Rosenthal’s poem
titled Gaia mon amour a passionate portrayal of
humanities infliction upon the Earth and the retaliation of
the spirit of Earth through environmental events [13]. The
cellist (representing the human spirit) is positioned at the
centre point between four surrounding speakers
(representing Gaia Mother Earth).
In the early stages of testing, small composition sketches
were written to test the effectiveness of the seven parameters
when mapped onto audio processes. These sketches were
developed with the addition of feedback delays,
reverberations and twelve sequentially triggered samples
that act as an accompanying landscape to the seven-minute
composition.
Gaia sketches comprises essentially of a series of statements
constructed from variants of a four-note motif. Each
statement is subjected to transforming timbral states and
colourations that explore the interaction between the players
bowing gesture and the possibilities inherent in the chosen
mapping configurations.
The accompanying samples serve as an additional layer of
sonorities that encompass and surround the soloist. A series
of sequential modes (triggered by pitch recognition) change
the configuration of colouration of the incoming audio
signal and is further added to the accompanying samples.
4.1.3 Challenges
The interface for Gaia Sketches was programmed in
Max/MSP and allows for initial calibration of the bow.
Visual references are given to incoming bow data and pitch
values of the acoustic cello. Further control is given to mode
selection and signal values in and out of the attached audio
interface.
The mapping of raw data from the bow often produced
inconsistent effects in practice due to natural variations in
the cellist’s gesture or variations between different
performers. A calibration procedure was introduced before
each performance which scales the extremes of incoming
gesture data to a set of maximum and minimum values.
Figure 2: Gaia Sketches uses pitch recognition to trigger
combinations of “colour” states and stored sound files.
Although the Hyperbow controller offers seven parameters
of gesture data, the challenge was to find combinations of
bow parameters that were responsive enough for the player
to experience effectively for real-time manipulation of audio
processing.
Five coloration states transform the incoming audio signal
from the acoustic cello through a combination of bow
gesture to sound processing parameter mappings. These
include two comb filters, two resonant filters and a Prosoniq
Northpole VST plugin.
The two comb filters are independently set with different
feedforward and feedback coefficients to produce both
flanger and chorus type colourations of the signal. The bow
velocity, obtained from the rate of change in bow position
between frog and tip data, is directly mapped to the rate of
the filter. The downward force bow parameter is directly
mapped to the depth of the effect.
The first of the resonant filters directly maps the z-
acceleration (normal to the string) of the bow to the centre
frequency of the filter, exciting the sound during passages
involving techniques such as spiccato. The second maps the
bow position (obtained from frog and tip values) to the
centre frequency of the filter.
The final colouration is achieved through the use of a
Prosoniq Northpole VST plugin in which two simultaneous
mappings occur. The first directly maps the downward force
data to distortion level of the incoming cello signal. The
second directly maps bow velocity to the cutoff frequency
creating a ‘wah-wah’ effect.
Figure 3: In Gaia Sketches the Hyperbow is used to control
at least one parameter in each of the “colour” states.
4.1.4 Thoughts on the Outcome of Gaia Sketches
The realization of Gaia Sketches can be seen as an initial
exploratory step in the collaborative development between
designer, composer and musician. Working with new tools
such as the Hyperbow generates a unique set of possibilities
and challenges both musically and technically for a
composer. The most common issue experienced by all
composers was one of balance between the processes of
learning the required new skills, design, programming,
composition and testing.
Perhaps the most exciting of opportunities lies in the
potential for creating an intuitive link between the performer
and the sound processing through successful mapping
strategies. In Gaia Sketches, these mappings are
independent one-to-one types although the higher- level
controls used in the Prosoniq Northpole VST plugin are
indeed lower-level parameters that have been cross-
coupled. Although these simple mappings were successful
in attaining a reasonable amount of intuitive control for the
performer, the range of timbral diversity was rather limited
and would certainly benefit from further experimentation
into more complex mapping strategies.
By far, the most fundamental part of the composition
process involved a thorough understanding of the
relationship between the gestural data and the perceived
sound from the player’s instrument. The seven parameters of
the Hyperbow, although a little daunting at first, provide a
composer with a multitude of gestural information about
the performer. Through innovative design of sound
synthesis and processing and efficient mapping schemes, it
is possible to find suitable combinations that are not only
musically interesting but effective in their response.
With the possibility of future performances in mind, one aim
was to make the electronic processing as automated as
possible. This was achieved through the use of sequential
mode selection triggered by pitch recognition using Tristan
Jehan’s pitch~ object [5]. It is feasible that certain bow
parameters could have been used but at the expense of the
integrity of the composition. Similarly, the coordination of
specific pitches within the composition with the triggering
of electronic events can pose further challenges for the
composer. However, the pitch tracking method proved to be
reliable on most occasions although further improvements
are required to ensure complete autonomy.
Future plans include more sketches exploring the
interaction and relationships between gesture mapping and
sound processing culminating in a larger work for string
orchestra and cello with Hyperbow.
4.2 MODES for electric cello, Hyperbow and
computer interaction, by Artem Vassiliev
As this is my first piece involving live electronic
interaction, the proportion between computer-generated
musical events and episodes of unaccompanied cello was
chosen with the emphasis on the solo instrument. It is a
more familiar medium for me as a composer; therefore this
approach allowed me to explore the new technology without
a danger for this work to become a purely technological
study rather than a piece of music. It consists of three
sections called MODES, 2 interludes, prelude and postlude.
Live electronics appear only in MODES and their
compositional function here can be seen mostly as
ornamentation and distorted reflection of the main solo. The
musical implication was to create an atmosphere of a
meditative self-reflection, for which the timbre of an electric
cello was the most productive sound source to explore.
(Although initially this piece was intended for an acoustic
cello, it was transformed later during our collaboration with
Peter Gregson.)
While working on the electronic component of my
composition, I aimed to produce a result, which can at the
same time sound predictable enough to become a part of a
‘stable’ composition and flexibly follow all the nuances of
the soloist’s behavior, which inevitably varies with every
new performance, on stage. This was achieved by means of
Max/MSP and Logic software with inclusion of two VST
plug-ins. The Max patch is a result of my collaborative work
with Philippe Kocher, Mike Fabio and Patrick Nunn.
In the MODE 1 the amplitude of incoming cello sound
triggers one of the three delay lines (as show in Figure 4), so
an echoed version of the original sound is being
transformed and distorted in three different ways.
Figure 4: MODE 1 uses a variable delay. The amplitude of
the cello sound triggers one of the 3 delay lines.
The next two MODES in this composition require the use of
the Hyperbow. Among the multiple possibilities offered by
this powerful performer’s tool, the one that interested me as
a composer was to use it as a flexible trigger and the
controller for the electronic accompaniment in this piece.
Thus, in the MODE 2 the incoming pitch from the cello is
analyzed by the software (using FFT) and replicated with
random octave transpositions. The pitches are also mapped
according to the three predetermined harmonies. The fast
acceleration of the bow switches the programme to the next
chord. In order to prevent very rapid changes, the software
was set to react on this parameter every five seconds. The
tempo of resulting accompanying arpeggiated patterns is
regulated by the bow pressure (the stronger the pressure, the
slower the tempo). Transposed cello pitches are also doubled
by the sine wave synthesizer, which creates an additional
layer of texture in a higher register. Its amplitude envelope
follows the amplitude of the incoming audio signal from the
cello.
Figure 5: MODE 2 uses a sine wave synth to double
transposed pitches produced by the electric cello.
The MODE 3 is similar to the MODE 2, but instead of the
transposed cello sound and the sine wave synthesizer, two
other physical modeling software synthesizers were used in
this section. The performer can switch between them by
playing either closer to the tip of the bow or to the frog. The
harmony and the tempo are controlled in the same way as in
MODE 2 (fast bow movement will switch the programme to
the next chord and the bow pressure determines the tempo).
Although all sound events in the electronic part of my
composition are generated in realtime, they can be
controlled in order to match harmonically and rhythmically
the solo cello part. This flexibility is mainly achieved by
use of the Hyperbow. Our practical experience demonstrated,
however, that such an approach to the bow as a trigger
increases responsibility and demands an additional effort
from the performer. This is the one of the problems that I am
going to explore further and possibly to solve in my next
composition for cello with the Hyperbow and string
orchestra.
Figure 6: MODE 3 is similar to MODE 2, but uses two
external software synths (instead of a sine wave synth).
As my first work for live electronics, MODES has achieved
its goal. It gave me enough practical experience and
inspiration for the new project in which the involvement of
live electronics component is going to be more substantial.
My first excitement of working with the Hyperbow is now
leading me towards serious exploration in a broader area of
compositional interests and research activities. Thus, in the
new project I am going to try using only those parameters
that will not interfere with interpretive ideas of the
performer. In other words, I see the Hyperbow in my next
piece more as a controller rather than a trigger. The ideal
situation for such a performance would be when the
complexity and variety of functions of the Hyperbow will
become almost unnoticeable for the player, but at the same
time all the events in the electronic part will be accurately
synchronized with the solo instrument and the orchestra,
leaving enough space for an interpretation and for an
improvisation if it is required.
Apart from offering endless opportunities for an experiment,
the Hyperbow is continuing to develop over the period of
our ongoing collaboration. I see this fact as a great
advantage for me as a composer. It motivates me to continue
working on this project because certain improvements in
software and hardware components can be made even
according to the needs of one individual composition. Thus,
the composer can be as precise as possible in his/her
compositional demands. On the other hand, every new
composition written specifically for the Hyperbow extends
its repertoire and potentially increases chances for this
outstanding invention to become one day an industrial
standard in contemporary music making and performance
practice.
5. PERFORMING WITH THE HYPERBOW
In this project, we are extremely fortunate to have the
participation of highly skilled, classically trained,
adventurous cellists. To date, four such players, Philip
Sheppard, Shu-Wei-Tseng, Alexander Holladay, and Peter
Gregson, have performed the various new works with
Hyperbow composed throughout this collaboration.
Figure 7: Shu-Wei Tseng with Hyperbow and cello (©
Roberto Aimi).
Not surprisingly, the expectations, impressions, and
experiences of each individual cellist regarding the
Hyperbow vary considerably. Previous experiences, such as
whether or not the player has performed with an electric or
amplified acoustic cello before, and the degree of familiarity
with studio equipment, computers, or technology in general,
are important factors. Also, the amount of time spent with
the Hyperbow, in collaboration with the composers and in
rehearsal, is of course critical.
Both Shu-Wei Tseng and Peter Gregson, who recently
performed Gaia Sketches and MODES, respectively, took a
keen interest in the technical capabilities of the Hyperbow.
However, in reaction to various aspects of the experience,
their impressions varied. Tseng remarks, “I was immediately
put into a position where I have to be fully aware of what and
how I do things. This is wonderful…” Gregson observes that
producing the gesture data to produce the desired sound
“came down simply to feel.” Of course, their perceived
ability to control the sound output of the system was also
dependent on the mapping in place.
On the issue of the weight and ergonomics of the Hyperbow,
feedback also differed. One player stated that the increased
weight of the Hyperbow was of no concern, while another
disagreed. Interestingly, criticisms regarding the carbon
fiber bow itself were also expressed, as Tseng observed that
it produced sound louder than usual, and Gregson noted that
he found the frog to be too low.
Though the number of cellists who have played with the
Hyperbow is still quite small, the feedback gained from them
has been extremely helpful. Perhaps the most encouraging
piece of feedback received thus far was the suggestion by
two of the cellists to conduct a training session on the
maintenance and upkeep of the Hyperbow interface for the
performing cellists in the group. Such a session would
include instructions on when and how to change batteries,
remove the electronics board to enable rehairing of the bow,
debug potential problems in the operation of the electronics,
etc. Imparting these skills to Hyperbow performers is
essential to ensure the success of the interface, as we hope
that it will one day be entirely under the care of its users and
truly “gig-worthy”.
Though the number of performers who have played the
Hyperbow is still quite small, the benefits we have gained
from their contributions are great. As our collaboration
grows and the Hyperbow develops, we will continue to enlist
the essential help of these and other performers.
6. SUMMARY
We are greatly encouraged by the progress we have achieved
in the past year of our Hyperbow collaboration. The
successful deployment of the Hyperbow within a new
community of users, performances of new compositions, and
the enthusiasm of both composers and performers for this
work of art and research, are all positive results.
As we continue, we plan to further increase the number of
participants (composers and cellists) involved in the
project, in order to create a greater repertoire for the
Hyperbow and a larger body of knowledge regarding its
performance applications. It is our hope that through such
work we may soon be able to produce a significantly
improved and refined Hyperbow that will be useful to many
other musicians.
7. ACKNOWLEDGMENTS
Our thanks to Tod Machover, Simon Bainbridge, and Philip
Sheppard, Mike Fabio, Philippe Kocher, and Alexander
Holladay. Special thanks to Peter Gregson and Shu-Wei
Tseng for the comments included above and to Roberto Aimi
for providing the photographs.
8. REFERENCES
[1] Burtner, M. The Metasaxophone: Design of a New
Computer Music Controller. . In Proceedings of the
2002 International Computer Music Conference
(ICMC). Göteborg, 2002.
[2] Chafe, C. http://ccrma.stanford.edu/~cc/.
[3] Goto, S. The Aesthetics and Technological Aspects of
Virtual Musical Instruments: The Case of the SuperPolm
MIDI Violin. Leonardo Music Journal, 9:115–120,
1999.
[4] Goudeseune, C. A. A Violin Controller for Real-Time
Audio Synthesis. Tech. Rep., Integrated Systems
Laboratoy, University of Illinois at Urbana-Champaign,
2001.
[5] Jehan, T. http://web.media.mit.edu/~tristan/.
[6] Machover, T. and Chung, J. Hyperinstruments:
Musically Intelligent and Interactive Performance and
Creativity Systems. In Proceedings of the 1989
International Computer Music Conference (ICMC). San
Francisco, 1989.
[7] Nichols, C. The Vbow: An Expressive Musical
Controller Haptic Human-Computer Interface. PhD
Thesis, Stanford University, 2003.
[8] Overholt, D. The Overtone Violin. In Proceedings of the
New Interfaces for Musical Expression (NIME)
Conference. Vancouver, 2005.
[9] Palacio-Quintin, Cléo. The Hyper-Flute. In Proceedings
of the New Interfaces for Musical Expression (NIME)
Conference. Montreal, 2003.
[10] Paradiso, J. A. and Gershenfeld, N. A. Musical
Applications of Electric Field Sensing. Computer Music
Journal, vol. 21, no. 3, pp. 69-89, 1997. .
[11] Poepel, C. Synthesized Strings for String Players. In
Proceedings of the New Interfaces for Musical
Expression (NIME) Conference. Hamamatsu, 2004.
[12] Rose, J.
http://jonroseweb.com/
.
[13] Rosenthal, R. and Chaudhuri, U. Rachel’s Brain and
Other Storms: The Performance Scripts of Rachel
Rosenthal. Great Britain: Continuum International
Publishing Group, 2001.
[14] Scavone, G. The Pipe: Explorations With Breath
Control. In Proceedings of the New Interfaces for
Musical Expression (NIME) Conference. Montreal,
2003.
[15] Schoner, B. Probabilistic Characterization and
Synthesis of Complex Driven Systems. PhD Dissertation,
MIT Media Laboratory, Cambridge, MA, 2000.
[16] Tarabella, L. and Bertini, G. Giving Expression to
Multimedia Performance. In Proceedings of the 2000
ACM Workshops on Multimedia. Los Angeles, 2000.
[17] Trueman, D. and Cook, P. BoSSA: The Deconstructed
Violin Reconstructed. In Proceedings of the 1999
International Computer Music Conference (ICMC).
Beijing, 1999.
[18] Waisvisz, M. The Hands, A Set of Remote MIDI-
Controllers. In Proceedings of the 1985 International
Computer Music Conference (ICMC). San Francisco,
1985.
[19] Young, D. The Hyperbow Controller: Real-Time
Dynamics Measurement of Violin Performance.
Proceedings of the New Interfaces for Musical
Expression (NIME) Conference. Dublin, 2002.
[20] Young, D. New Frontiers of Expression Through Real-
Time Dynamics Measurement of Violin Bows. Master’s
Thesis, MIT Media Laboratory, Cambridge, MA, 2001
... This work has been published in several venues, such as the 2003 Symposium of Musical Acoustics , the 2003 edition of the New Interfaces for Musical Expression Conference , and the proceedings of the 2007 International Computer Music Conference [Young and Serafin, 2007]. In addition to this body of work, the Hyperbow, once adapted for use with acoustic and electric cello, served as an integral part of a collaboration with composers and cellists at the Royal Academy of Music, the focus of which was to further explore its expressive musical potential [Young et al., 2006]. ...
Book
Full-text available
What is a musical instrument? What are the musical instruments of the future? This anthology presents thirty papers selected from the fifteen year long history of the International Conference on New Interfaces for Musical Expression (NIME). NIME is a leading music technology conference, and an important venue for researchers and artists to present and discuss their explorations of musical instruments and technologies. Each of the papers is followed by commentaries written by the original authors and by leading experts. The volume covers important developments in the field, including the earliest reports of instruments like the reacTable, Overtone Violin, Pebblebox, and Plank. There are also numerous papers presenting new development platforms and technologies, as well as critical reflections, theoretical analyses and artistic experiences. The anthology is intended for newcomers who want to get an overview of recent advances in music technology. The historical traces, meta-discussions and reflections will also be of interest for longtime NIME participants. The book thus serves both as a survey of influential past work and as a starting point for new and exciting future developments.
... The accelerometers and the electronics for wireless data transmission were accommodated on a printed circuit board mounted on the frog.The Hyperbow controller was essentially designed for professional musicians to be used in new music performance scenarios (e.g. Tod Machover's Toy Symphony, Patrick Nunn's Gaia Sketches and MODES by Artem Vassiliev for Hyperbow adapted for cello,Young et al., 2006). To serve also as a research tool for measuring violin bowing technique, the original design was adjusted to incorporate units for measuring the orientation of the bow in relation to the violin(Young, 2007). ...
Thesis
Full-text available
Full text of the thesis available from: https://qmro.qmul.ac.uk/xmlui/handle/123456789/18378
... A metaphor is chosen to fit the observed movements within a music performance context. For example, when choosing a metaphor, designers may select a conducting system [Boulanger and Mathews, 1997], performance system [Weinberg et al., 2002], hyperinstrument [Young et al., 2006], or generative composition system [Xenakis, 2001]. Within a system, ideas as to whether the resulting experience should be similar to string instrument performance, or piano instrument performance, dictate which variables best describe the movement, and how to map the movement to particular kinds of musical events. ...
Article
As a culture, we have the capacity to lead creative lives. Part of that capacity lies in how something like music can touch on just about every aspect of human thinking and experience. If music is such a pervasive phenomenon, what does it mean for the way we consider our lives in health? There are three problems with connecting the richness of music to scientifically valid clinical interventions. First, it is unclear how to provide access to something as seemingly complex as music to a diverse group of subjects with various cognitive and physical deficits. Second, it is necessary to quantify what takes place in music interactions so that causality can be attributed to what is unique to the music experience compared to motivation or attention. Finally, one must provide the structure to facilitate clinical change without losing the communicative and expressive power of music. This thesis will demonstrate how new music technologies are the ideal interfaces to address the issues of scale, assessment, and structured intervention that plague the ability to introduce creative work into healthcare environments. Additionally, we describe the first neural interface for multisensory-based physical rehabilitation, with implications for new interventions in diverse settings. This thesis demonstrates the design and implementation of devices that structure music interaction from the neural basis of rehabilitation. At the conclusion of this research, it is possible to envision an area where users are empowered during scientifically based creative tasks to compose neurological change.
... • musical performance style and allows better support for the repreformance of the pieces particularly for Interactive Multimedia Performance (IMP) that involves one or more performers who interact with a computer based instruments [23,21], sensors systems and others [14] Manipulating multimedia content using computers is an essential part of a live IMP performance. Typical recording methods using simply performance outputs recorded in the form of audio and video media are not sufficient for a proper analysis (e.g. for studying the effect of a particular performing gesture on the overall quality of the performance) or reconstruction of a performance at a later time. ...
Conference Paper
Full-text available
This paper discusses interactive multimedia framework and tools that have been developed under the i-Maestro EC IST project (www.i-maestro.org) for technology-enhanced music education, and focuses on one of the applications which explores 3D motion capture data for real-time analysis and feedback to support the learning processes. The multi-modal data (3D motion, sensor, audio and video) captured with the system offers a more complete and more detailed description of a musical performance. Another project, CASPAR (Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval - www.casparpreserves.eu) which is developing a framework for digital preservation uses the multi-modal data for the contemporary arts ?est-bed in order to offer musicologist of the future a better records and description of the playing style and interpretation that may not be easily extracted/understood from a video recording.
... Relatedly, Schnell and Battier introduce the term composed instruments, for the very design of instruments and the constraints and affordances they offer might be seen as constituting the composition [13]. Young et al describe the process of composing for the Hyperbow controller , explaining how the development of a new repertoire can feed into the evolution of the instrument itself [14]. Hanne De Jaegher's work on social interaction presents a novel approach, framing interaction as an autonomous process and the idea of participatory sense making [1]. ...
Thesis
Full-text available
This thesis proposes an alternative approach to sound synthesis. It seeks to offer traditional string players a synthesiser which will allow them to make use of their existing skills in performance. A theoretical apparatus reflect- ing on the constraints of formalisation is developed and used to shed light on construction-related shortcomings in the instrumental developments of related research. Historical aspects and methods of sound synthesis, and the act of musical performance, are addressed with the aim of drawing con- clusions for the construction of algorithms and interfaces. The alternative approach creates an openness and responsiveness in the synthesis instrument by using implicit playing parameters without the necessity to define, spec- ify or measure all of them. In order to investigate this approach, several synthesis algorithms are developed, sounds are designed and a selection of them empirically compared to conventionally synthesised sounds. The algo- rithms are used in collaborative projects with other musicians in order to examine their practical musical value. The results provide evidence that im- plementations using the approach presented can offer musically significant differences as compared to similarly complex conventional implementations, and that—depending on the disposition of the musician—they can form a valuable contribution to the sound repertoire of performers and composers.
Chapter
We present case studies of unusual instruments that share the same excitation mechanism as that of the bowed string. The musical saw, Tibetan singing bow, glass harmonica, and bowed cymbal all produce sound by rubbing a hard object on the surface of the instrument. For each, we discuss the design of its physical model and present a means for expressively controlling it. Finally, we propose a new kind of generalized friction controller to be used in all these examples.
Conference Paper
Thanks to the development of new technology, musical instruments are no more tied to their existing acoustic or technical limitations as almost all parameters can be augmented or modified in real time. An increasing number of composers, performers, and computer programmers have thus become interested in different ways of "supersizing" acoustic instruments in order to open up previously-unheard instrumental sounds. This leads us to the question of what constitutes a super instrument and what challenges does it pose aesthetically and technically? Although the classical music performers have traditionally been dependent on their existing instrumental skills, various technological solutions can be used to reach beyond them. This paper focuses on the possibilities of enhancing composers' and performing pianists' technical and expressive vocabulary in the context of electroacoustic super instrument compositions. The discussion will be illustrated by two compositional case studies.
Chapter
So far we have used the OAIS terminology for digital preservation . Now we turn to a complementary way of looking at it. We can say that the challenge of digital preservation of scientific data lies in the need to preserve not only the dataset itself but also the ability it has to deliver knowledge to a future user community. This entails allowing future users to reanalyze the data within new contexts. Thus, in order to carry out meaningful preservation we need to ensure that future users are equipped with the necessary information to re-use the data.
Article
Smart handheld tools epitomize a mythical and technological quest for personal mastery of skill, delivering both might and mind in the hands of their holders. A recent spur of academic and industrial efforts has given rise to a new field of research in HCI, one devoted to smart handheld tools. Here, the authors offer a definition for smart handheld tools, discuss the tools' origins and motivation, and present a survey of prominent work by themselves and others in disciplines such as fabrication, painting, printing, and maintenance. They also discuss their experiences operating in this new territory and conclude with a vision of a hybrid creative practice: smart handheld instruments that enable synergetic cooperation with human skill, personal style, and computational assistance that results in accuracy, guidance, and protection for users. This article is part of a special issue on printing and fabrication.
Article
Full-text available
Traditional musical instruments provide compelling metaphors for human-computer interfacing, both in terms of input (physical, gestural performance activities) and output (sound diffusion). The violin, one of the most refined and expressive of traditional instruments, combines,a peculiar physical interface with a rich acoustic diffuser. We have built a new instrument that includes elements of both the violin's physical performance,interface and its spatial filtering audio diffuser, yet eliminates both the resonating body and the strings. The instrument, BoSSA (Bowed-Sensor-Speaker-Array), is an amalgamation and extension of our previous work with violin interfaces, physical models, and directional tonal radiation studies. In addition to describing the various physical and software elements that make up BoSSA, we discuss some of its musical features and potentials; we are particularly impressed by the sense of presenceand intimacyit provides, and by its potential for creating a new kind of electronic chamber,music. 1.0 Introduction Traditional musical,instruments provide compel-
Article
The author discusses how his critical stance against conformity in computer-based interactive art eventually led him to create his own instrument as a way towards individual artistic sensitivity and thought. He first outlines the development and creation of a virtual musical instrument, the SuperPolm, as well as its technical points. He then addresses the relationship between gesture and music and the variety of human perceptual experiences that may occur during a performance on a virtual musical instrument. Finally the author presents the background of the SuperPolm's development and discusses cultural and technological aspects of interactivity.
Conference Paper
In this paper we describe the experience of researchers and artists involved in the activities of the “computer ART lab” (cART lab) of the italian National Council of Research (C.N.R.) in Pisa, regarding the “wireless technology” developed for controlling in real-time and giving expression to interactive multimedia performances.Due to the daily increase in computers power and electronics systems able to sense the presence, the shape, the distance and the position of objects, a new field of investigation and implementation has been started in the last few years: computer recognition of human gesture [1][2]. As a result, the human body itself can now be considered as a natural and powerful expressive “interface” to give feeling to performances based on computer generated electro-acoustic music and computer generated visual-art. Modern human computer interfaces are extremely rich, incorporating traditional interface devices such as keyboard and mouse and a wealth of advanced media types: sound, video, animated graphics. The term multi-modal is often associated with such interfaces to emphasize that the combined use of multiple modes of perception is relevant to the user's interface [3][4].The most relevant devices and systems developed at cART lab for gesture recognition to be used in interactive multimedia performances are here reported.
Conference Paper
The Hyper-Flute is a standard Boehm flute (the model used is a Powell 2100, made in Boston) extended via electronic sensors that link it to a computer, enabling control of digital sound processing parameters while performing. The instrument's electronic extensions are described in some detail, and performance applications are briefly discussed.
Article
Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2000. Includes bibliographical references (leaves 194-204).