Conference PaperPDF Available

Three-dimensional sound design with Mosca

Authors:

Abstract and Figures

Mosca is a software extension class of the SuperCollider language for sound synthesis and algorithmic composition. It produces three-dimensional sound fields via an intuitive graphical user interface controlling a variety of hidden internal methods. Drawing from ambisonics' adaptable form of surround sound applicable to wide-ranging loudspeaker configurations and headphones, Mosca is suitable in a variety of applications. Recent improvements to the software include the incorporation of high-order ambisonics and the OSSIA/score project, enabling sophisticated control of audio spatialisation and synchronisation with a variety of media. Additionally, current development will bring location sensitivity to Mosca. Together with existing head-tracking capacities for binaural audio reproduction, the software will be highly useful in mixed and virtual reality projects. Mosca est une extension pour l'environnement de programmation audio SuperCollider. Cette extension permet la production de champs sonores tridimensionnels à travers une interface graphique intuitive et différentes méthodes internes. Tirant profit de la flexibilité du format ambisonique, compatible avec le rendu au casque et une large gamme de configurations d'enceintes, Mosca peut s'adapter à un grand nombre d'applications. Ce programme est actuellement développé par les auteurs pour intégrer des ordres ambisoniques supérieurs, et davantage de techniques de spatialisation, ainsi que le projet OSSIA/score, permettant un contrôle sophistiqué et la synchronisation, avec différents médias. En outre, les développements actuels apporteront à Mosca une localisation GPS de l'auditeur. Avec les possibilités actuelles de suivi du mouvement de la tête pour le rendu binaural, le programme pourra facilement être utilisé pour des projets de réalité virtuelle et mixte.
Content may be subject to copyright.
1
#18.ART • 2018 • ISSN: 2238-0272
17th to 19th of oct 2019
ANAIS
18º Encontro internacional de Arte, Ciência Tecnologia
18th International Meeting of Art, Science and Technology
18º Encontro internacional de Arte, Ciência Tecnologia
18th International Meeting of Art, Science and Technology
Edição I Edition
ISSN: 2238-0272
Fluxo de Dados: Visualização e Sonicação
85
#18.ART • 2018 • ISSN: 2238-0272
Iain Mott1 and Thibaud Keller2
Three-dimensional sound design with Mosca
Spatialisation tri-dimensionnel avec Mosca
Abstract
Mosca is a software extension class of the SuperCollider language for sound
synthesis and algorithmic composition. It produces three-dimensional sound
elds via an intuitive graphical user interface controlling a variety of hidden
internal methods. Drawing from ambisonics’ adaptable form of surround
sound applicable to wide-ranging loudspeaker congurations and head-
phones, Mosca is suitable in a variety of applications. Recent improvements to
the software include the incorporation of high-order ambisonics and the OS-
SIA/score project, enabling sophisticated control of audio spatialisation and
synchronisation with a variety of media. Additionally, current development
will bring location sensitivity to Mosca. Together with existing head-tracking
capacities for binaural audio reproduction, the software will be highly useful
in mixed and virtual reality projects.
Keywords: ambisonics, surround sound, electroacoustic music, SuperCollider,
OSSIA/score
Résumé
Mosca est une extension pour l’environnement de programmation audio SuperCol-
lider. Cette extension permet la production de champs sonores tridimensionnels à
travers une interface graphique intuitive et diérentes méthodes internes. Tirant
prot de la exibilité du format ambisonique, compatible avec le rendu au casque
et une large gamme de congurations d’enceintes, Mosca peut s’adapter à un grand
nombre d’applications. Ce programme est actuellement développé par les auteurs
pour intégrer des ordres ambisoniques supérieurs, et davantage de techniques de
1 Sound artist, composer and lecturer in sound design and voice at the Departamento de Artes
Cênicas, Instituto de Artes, Universidade de Brasília, http://cen.unb.br. Doctorate in arts from the
University of Wollongong entitled “Sound Installation and Self-listening”. Personal website: https://
escuta.org. Institutional email: iainmott@unb.br.
2 Software technician and system administrator at SCRIME, University of Bordeaux, https://scrime.u-
-bordeaux.fr. Member of the OSSIA team, https://ossia.io. Masters in computer music programming
from the university of Saint-Etienne, https://musinf.univ-st-etienne.fr/index.html. Personal page: ht-
tps://github.com/thibaudk. Organisation page: https://github.com/scrime-u-bordeaux. Institutional
email: thibaud.keller@u-bordeaux.fr.
Fluxo de Dados: Visualização e Sonicação
86
#18.ART • 2018 • ISSN: 2238-0272
spatialisation, ainsi que le projet OSSIA/score, permettant un contrôle sophistiqué
et la synchronisation, avec diérents médias. En outre, les développements actuels
apporteront à Mosca une localisation GPS de l’auditeur. Avec les possibilités actu-
elles de suivi du mouvement de la tête pour le rendu binaural, le programme pourra
facilement être utilisé pour des projets de réalité virtuelle et mixte.
Mots clefs: ambisoniques, spatialisation, musique électroacoustique, SuperCollider,
OSSIA/score
Introduction
Mosca is a software extension class (or quark) of the SuperCollider language for
sound synthesis and algorithmic composition, and was initially developed as part
of a research project entitled Cerrado Ambisônico by Iain Mott, assisted by the MCTI/
CNPq in the Edital Universal. The software produces surround sound and oers an
intuitive graphical user interface (GUI) for direct control. A variety of ambisonic and
spatial audio techniques are used to render three dimensional sound designs on
both headphones and wide-ranging loudspeaker congurations. Mosca makes ex-
tensive use of the the Ambisonic Toolkit (ATK) code library for ambisonic processing
and the Automation quark to sequence control data. Recent work by Thibaud Keller
has brought improvements to the GUI and seen the inclusion of additional spatial
audio libraries. Furthermore, communication with the multimedia sequencer OSSIA/
score is now enabled, facilitating live performances and synchronisation with other
real-time multimedia systems. Current work also includes location sensitivity via GPS
and accelerometers, derived from Mott’s sound mapping installation Botanica, pre-
sented in the proceedings of #16.Art.
As an extension of SuperCollider, Mosca is open source and runs on Linux, Mac and
Windows platforms. This article describes both current capabilities of Mosca and the
work in progress. Along with the documentation for the Mosca quark in SuperCol-
lider, this article serves as a practical guide to the software and provides the reader
with information to realise their own projects.
Ambisonic sound
Sound is not a static object. It is liberated from a source and propagated through
space by way of compression and rarefaction of air molecules. On a mild day of
20 degrees centigrade, it moves in waves through the air at the speed of approxi-
mately 343 metres per second. Various sounds arrive at our ears from all directions,
either directly or after having rst come into contact with surrounding objects and
materials. Ambisonics aims to both capture or synthesise and reproduce such
multidirectional sound elds for a certain region in space. Developed by Michael
Fluxo de Dados: Visualização e Sonicação
87
#18.ART • 2018 • ISSN: 2238-0272
Gerzon and others in the UK and the USA in the early 1970s 3, ambisonics involves
encoding and decoding steps. To encode ambisonics is to process a sound source
or sources, for example a solo musical instrument or a ock of birds, by capturing
the waveform’s inherent phase and level information over a number of channels
and along specic spatial axes. Encoding may be done synthetically, by way of
digital or analogue processing, or acoustically, using a so-called soundeld4 mi-
crophone (Fellgett, 1975) rst developed by Michael Gerzon and Peter Craven in
the 1970s (Batke, 2009) with a number of capsules to capture the incoming sound
from multiple angles at once. As a convention, sound elds are generally encod-
ed in the multichannel B-format, which may exist at dierent spatial resolutions,
or ambisonic orders (Hollerweger, 2008) and is independent of the loudspeaker
conguration used for reproduction. The rendering of an ambisonic sound eld
is performed in the decoding step, where the signal is processed for a specic
loudspeaker array, whether it be a 2-dimensional arrangement of loudspeakers
surrounding an audience, or a three-dimensional array of loudspeakers and even
headphones, delivering a full spherical sonic experience.
Sound Sources
Mosca takes a exible approach to encoding and decoding. Sound sources may
be any combination of mono, stereo or B-format material and the signals may
originate from le (loaded into memory or streamed from disc), from hardware
inputs (physical or from other applications like a DAW via the Jack Audio Connec-
tion Kit (Jack)5) or from sound synthesis processes inside SuperCollider itself (Mc-
Cartney, 1996; Wilson, Cottle, & Collins, 2011). A particular source is rst selected
by right-clicking in a blank section in the GUI (Figure 1), then selecting its index
from the drop-down menu, with the total number of sources dened when initial-
ising Mosca. Once a source is selected, it can be assigned an input and the dened
sources appear as numbered circles in the GUI. The centre of the larger blue circle
denes the location of the listener or audience and its periphery marks the max-
imum audible distance from the listening point. Sources may be positioned and
moved in space by clicking and dragging.
3 For information on the origins and nature of ambisonics and extensive lists of early publications
on the subject, see: www.michaelgerzonphotos.org.uk. Ambisonics is under ongoing development
and many of the leading researchers participate in the email discussion group “Sursound”: https://
mail.music.vt.edu/mailman/listinfo/sursound. The website http://www.ambisonic.net alsoprovides
resources and the Wikipedia page on ambisonics provides an excellent technical introduction: ht-
tps://en.wikipedia.org/wiki/Ambisonics.
4 For information about encoding conventions see: https://www.ambisonic.net/leformats.html
5 http://jackaudio.org
Fluxo de Dados: Visualização e Sonicação
88
#18.ART • 2018 • ISSN: 2238-0272
Figure 1. Typical Mosca work session
In order to spatialise the sound sources, mono and stereo inputs are processed
on a per-source basis by a selection of algorithm set from the “Library” pull-down
menu in the GUI. Users may choose between the ATK6, HOALib7 (Guillot, Paris, &
Deneu, n.d.), Ambitools (Grond & Lecomte, 2017)8, ADT(Heller & Benjamin, 2014)9,
BF-FMH, Josh and VBAP libraries10. The ATK and Josh are both restricted to 1st or-
der ambisonics, BF-FMH can either encode in 1st or 2nd order whereas Ambitools,
HOALib and ADTB oer up to 5th order ambisonics. The default ambisonic order of
Mosca is 1, however the user may enter a maximum order as an instantiation argu-
ment to perform higher order encoding and decoding. Josh is a simple ambisonic
granulator eect. VBAP on the other hand is a non-ambisonic method of sound
spatialisation unaected by the “maxorder” argument. The acronym stands for
vector based amplitude panning (Pulkki, 1997) and involves the panning of sound
sources between adjacent loudspeakers in an array, the details of which—angular
6 http://www.ambisonictoolkit.net
7 http://hoalibrary.mshparisnord.fr/en
8 https://github.com/sekisushai/ambitools
9 https://bitbucket.org/ambidecodertoolbox/adt
10 The last three libraries are provided by the ocial supercollider plugin repository
Fluxo de Dados: Visualização e Sonicação
89
#18.ART • 2018 • ISSN: 2238-0272
location and distance—must be passed to Mosca in the creation arguments for
the Mosca instance. Regardless of the chosen technique for a particular source,
stereo signals are treated as two distinct mono signals, separated spatially by an
adjustable angle parameter.
B-format signals up to 5th order, when used as input to Mosca, have no need to
undergo ambisonic encoding. The signal can be oriented and transformed directly
in various ways. The user can load or stream existing archival B-format material or
create their own audio les, either by using Mosca itself to record B-format les from
manipulated sources (mono, stereo or B-format) or record their own material acous-
tically with a sound eld microphone. Most commonly available soundeld micro-
phones are so-called A-format devices. Recorded signals using these microphones
will therefore need to be processed into B-format for use. The original Mosca project
has made use of such a microphone, the Core Sound Tetramic along with a TASCAM
DR-680 digital recorder and B-format material has been generated from raw record-
ings using the Linux software Tetraproc (Adriaensen, 2007). The methodology used
and a sample processing script are available online.11
Mosca uses two techniques for positioning B-format sources in space. Either the
push transformation from the ATK library or the Beam formation techniques avail-
able in Ambitools (Lecomte, Gauthier, Langrenne, Berry, & Garcia, 2016). Both eects
diminish as the source approaches the centre point, surrounding the listener and
allowing the inherent ambisonic coding of the original signal to dominate. With the
ATK, this signal can also be gradually striped of its directional attributes with the “Di-
rectivity” control. Fully rolled back, this parameter renders B-format inputs as omni-
directional signals with constituent sounds surrounding the listening point equally
from all directions. Additionally, rotation of incoming ambisonic sound elds around
their Z-axis can be performed with the “Rotation” parameter.
Mono and stereo sources are fully contracted by default, setting a focused position
in space. When fully de-contracted with the contraction control in the GUI, the signal
becomes omnidirectional. B-format sources, on the other hand, are de-contracted
by default and contraction causes them to become spatially focussed. The parame-
ter thus oers continuous control over the source’s width, from a narrow point to an
enveloping mass. When mono or stereo signals are de-contracted, the spread and
diuse GUI options of the ATK library oer two dierent types of spectral smearing
over the spherical ambisonic image. Both have distinct rendering qualities and users
may nd that one suits a specic class of sounds better than the other, as is the case
with all available libraries.
11 http://escuta.org/en/proj/research/ambiresources/item/222-shell-script.html
Fluxo de Dados: Visualização e Sonicação
90
#18.ART • 2018 • ISSN: 2238-0272
Distance and reverberation
Along with angular location, Mosca’s source signals are attenuated proportionally
to their distance from the origin, inside an audible radius of one-hundred metres. In
this way, as sources move from the centre of the GUI towards the periphery, they be-
come quieter before being silenced once outside the large blue circle. All sources are
subject to high-frequency attenuation with distance. Additionally, a proximity eect12
is available via the ATK as well as a near eld compensation13 through Ambitools.
Reverberation plays an important psychoacoustic role in the perceived distance of
a sounding object. Typically, as a sound approaches, the ratio of direct to reected
sound—for example from walls, oor and ceiling—increases. In other words, distant
sounds generally appear more reverberant than close sounds. Mosca provides two
reverberation controls, close and distant, each with a selectable reverberation types.
Along with the Freeverb14 and a plain all-pass option, convolution reverberation can be
used to accurately simulate the resonance of natural environments. The Mosca project
has created B-format tail room impulse responses (RIR) to that eect using Linux based
software written by Fons Adriaensen (2007). The methods used have been document-
ed online. 15 Banks of such RIRs may be passed to Mosca as initiation arguments.
The implementation of convolution reverberation, under the close control, can be
described as a 2nd order diuse A-format reverberation. This technique produces re-
verberation weighted in the direction of sound events encoded in the dry ambisonic
signal and involves conversion to and from A-format in order to apply the eect. The
encoded 2nd order ambisonic signal is converted to a 12-channel A-format signal
and convolved with a B-format RIR which has been upsampled to 2nd order and con-
verted to A-format impulse spectrum, a process that is performed automatically on
rst initialisation. A nal step converts this reverberated A-format signal back to 2nd
order B-format for decoding and audition (Anderson, 2011).
Presently in Mosca, the distant reverberation control attenuates a type of reverberation
described by John Chowning as local, whereby the return signal from the reverberation
process16 is mixed back with the source signal before the spatialisation phase, producing
a tightly spatially focused reverberant signal at the same angular location as the source.
Conversely in our implementation, the close reverberation control attenuates what
12 http://doc.sccode.org/Classes/FoaProximity.html
13 http://www.sekisushai.net/ambitools/hoa_encoder
14 https://ccrma.stanford.edu/~jos/pasp/Freeverb.html
15 http://escuta.org/en/proj/research/ambiresources/item/222-shell-script.html
16 In the case of Mosca, fed by the w component of the source’s B-format encoded signal.
Fluxo de Dados: Visualização e Sonicação
91
#18.ART • 2018 • ISSN: 2238-0272
Chowning described as global reverberation, a type of reverberation that envelopes
the listener, albeit, in our implementation, with a source-directed weighting. The overall
eect is that as the source recedes from the listener, and as it moves from close to dis-
tant control, the width of the reverberant eld narrows (Chowning, 1977). In the future,
Mosca will also oer the option of applying A-format convolution reverberation under
distant as well as close control.
Movement, control and automation
Mosca is well disposed to create sound elds with moving sound sources and its
name was inspired in part by ambisonic recordings of ies—moscas in Portuguese—
made inadvertently during eld recordings in the national park of Chapada dos Vea-
deiros 17. It is also a reference to Trevor Wishart’s recordings of bluebottle ies for his
electroacoustic composition Red Bird. After attempts of recording a y axed to a
substrate, he discovered that only with a y in motion—achieved by waving a blue-
bottle attached to a stick in a stereo eld—“could the aural image ‘y’ be recreated”
(Wishart, 1996, p. 151). To reproduce this characteristic of moving sound, a scalable
Doppler eect is implemented in Mosca for each source.
Creating movements and animating parameters in Mosca can be done in a number
of ways. A quick and intuitive method relies on the Automation quark 18 and direct
interaction with the GUI. If users select record in the automation transport without
selecting play, any changes made with the interface—for instance loading particular
audio les, positioning sources, adjusting various level controls—are recorded as a
single register in memory and may be saved and later recalled as such, in a named
directory. If play is selected while recording, users may record the spatial movement
of sources and changes to the controls, as a continuous stream of data. The transport
may be rewound any number of times to overlay additional changes or to override
prior alterations of the controls. Again, the recorded data may be saved and later
reloaded from a named directory for playback. The transport may be synchronised
to a digital audio workstation (DAW) using Midi Machine Control (MMC) messages
by selecting the slave to MCC checkbox in the GUI 19. This allows Mosca to spatialise
multitrack audio compositions, the individual tracks being sent to Mosca via Jack to
individual sources with HW-in selected20.
17 B-format audio recordings from this eld work are available online on an interactive map: https://
escuta.org/mapa
18 https://github.com/neeels/Automation
19 This method has been tested to work with the open source DAW Ardour on Linux.
20 When selected in the GUI, the user must enter the number of channels and the starting bus number.
Fluxo de Dados: Visualização e Sonicação
92
#18.ART • 2018 • ISSN: 2238-0272
Mosca can run without an active GUI by reading stored Automation data21 and may
also be controlled entirely through coded commands by way of proxies for the GUI
elements. Calling “myMoscaInstanceName.xboxProxy[i].value”, for example, will re-
turn the current x-coordinate of the source at index “i”. With the “valueAction” meth-
od, new values can be set in real time for any given parameter proxy. A full list of
available proxies is given in the help le for Mosca.
A major focus of Mosca’s rework was to enable communication with the intermedia se-
quencer OSSIA/score22 (Celerier, 2018). This open source project can be described as a
graphical language to formulate software interactions in time. The approach expands
on a traditional sequencer interface with the addition of exible durations, parallel
timelines, conditional branches and a host of communication protocols. Developed
as a virtual conductor, OSSIA/score allows the creation of scenarios where events and
processes obey “when”, “while” and “switch case” statements represented as Triggers,
Loops and Conditions. Open Sound Control (OSC) (Schmeder, Freed, & Wessel, 2010)
and OSCQuery23 compliant applications like Mosca can then be remotely controlled
and synchronised together inside a single scenario. Exposed on the network with the
OSSIA quark24 for supercollider, all of Mosca’s parameters appear within the device ex-
plorer of OSSIA/score. This new feature not only greatly improves the integration with
other systems, it also provides high level functionalities for editing large-scale spatial
audio compositions and real time interactive projects of all kinds.
Figure 2. OSSIA/score scenario example
21 Using initialisation arguments.
22 ossia.io
23 https://github.com/Vidvox/OSCQueryProposal
24 https://github.com/OSSIA/ossia-sclang
Fluxo de Dados: Visualização e Sonicação
93
#18.ART • 2018 • ISSN: 2238-0272
Decoding
A variety of options are available for decoding, each dependent on the rendering
system used (binaural, 2D ring or 3D array). A SuperCollider decoder object, chosen
from a number of types, may be passed to the Mosca instance in an initialisation
argument. If none is provided, raw ambisonic components of the chosen order are
outputted for processing with an external decoder, such as Ambdec (Adriaensen,
2011). The AmbiDecoderToolbox library for Octave and Matlab (Heller & Benjamin,
2014) may also be used and is recommended for higher-order decoding. It enables
users to create custom decoders calibrated specically for their setup. The scripting
and compiling steps for ADT are described in the SC-HOA quark tutorials (Grond &
Lecomte, 2017). As mentioned, VBAP involves neither ambisonic encoding nor de-
coding, however Mosca does require VBAP to be initialised with an array of loud-
speaker coordinates (2D or 3D) passed as an initialisation argument.
When a binaural ambisonic decoder is used for headphone reproduction, it may be
advantageous to use a head-tracking device. In this way the sound eld rotates cor-
respondingly with rotations of the head and the listener may better audition each
source. Mosca provides support for the Arduino 9-Axes Motion Shield25 sensor with
the Arduino Uno26 microprocessor. The Mosca help le provides information on how
the head-tracker should be installed and congured. Movements of the head are
displayed in the GUI as values of heading, pitch and roll as well as corresponding ro-
tations of sound sources. These orientation controls, like the origin parameters, only
aect the relative spherical coordinates for every source, leaving the absolute Carte-
sian coordinates intact. Both types of coordinates are available in OSSIA/score when
exposed to the network.
Current work
Work is currently underway to bring location sensitivity to Mosca using SuperCol-
lider code developed in the sound art project Botanica (Mott, 2017). The code will
enable Mosca to import a map and then calibrate it to position a mobile listener in
accordance with their physical location. Longitude and latitude serial data may be
used as input, be it from GPS or from any location sensing device. The system de-
veloped uses a Ublox NEO-6M GPS module27 which is wired directly to the Arduino
Uno with 9-Axes Motion Shield. Coupled with head tracking, listeners will be able to
walk in a direction of their choice, or travel by some other means, to explore sound-
scapes spanning physical space. Together with existing head-tracking capacities for
25 https://store.arduino.cc/usa/9-axis-motion-shield
26 https://store.arduino.cc/usa/arduino-uno-rev3
27 https://lastminuteengineers.com/neo6m-gps-arduino-tutorial/
Fluxo de Dados: Visualização e Sonicação
94
#18.ART • 2018 • ISSN: 2238-0272
binaural audio reproduction, location sensitivity will lend Mosca broad application
in mixed and virtual reality projects. The OSSIA quark is also under development to
take over from the now deprecated ossia-supercollider28 project, currently the only
way to enable OSCQuery communication. It will soon be recongured and based on
Pierre Cochard’s wsclang29 fork of SuperCollider and the changes then submitted to
the ocial SuperCollider repository.
Resources
The home page for Mosca is http://escuta.org/mosca. To use Mosca, SuperCollider
must be installed on a Linux, Mac or Windows computer along with SuperCollider’s
assortment of plugins, including HOA30 and the ATK31. Like SuperCollider and the
ATK, the Mosca source code and accompanying Arduino source for head-tracking,
are available on the Github site.32 As well as including the SuperCollider source code
for Mosca and help les, the repository, as mentioned above, contains the source
code for the Arduino-based head-tracker and should be loaded onto the device us-
ing the Arduino IDE33 software. The Mosca quark and additional prerequisite quarks
including the ATK may be loaded via the Quarks.gui interface in SuperCollider. Once
loaded, the user may access the Mosca help le and guide with full instructions and
code examples, by running the help command on the class name Mosca. Addition-
ally, the moscaproject.zip le contained within the git source may be used. Once
extracted, the archive contains the basic le structure for a Mosca project as well as
an example RIR le. A much larger project directory with B-format audio les is also
available on the site Escuta.org.34
Acknowledgements
Participation in this event was made possible with assistance from the Fundação de
Apoio a Pesquisa do Distrito Federal (FAP DF). The authors gratefully acknowledge
Joseph Anderson (ATK), Pierre Lecomte (Ambitools), Florian Grond (SC-HOA), Pierre
28 https://github.com/OSSIA/ossia-supercollider
29 https://github.com/pchdev/wsclang
30 https://github.com/scrime-u-bordeaux/sc3-pluginsHOA
31 http://www.ambisonictoolkit.net/download/supercollider
32 https://github.com/escuta/mosca
33 https://www.arduino.cc
34 http://escuta.org/tmp/moscaproject.zip. The archive includes a B-format audio recorded by Iain
Mott in Chapada dos Veadeiros and Brasilia as well as a Spitre recording by John Leonard, provided
with kind permission.
Fluxo de Dados: Visualização e Sonicação
95
#18.ART • 2018 • ISSN: 2238-0272
Guillot (HoaLib), Neels Hofmeyr (Automation), Jean-Michël Celerier (OSSIA/score),
Pierre Cochard (ossia-supercollider & wsclang) and members of the SuperCollider
users and dev lists for their assistance and valuable suggestions.
References
Adriaensen, F. (2007). A Tetrahedral Microphone Processor for Ambisonic Re-
cording. Presented at the Linux Audio Conference, Berlin.
Adriaensen, F. (2011). Ambdec 0.4.2 User Manual. Retrieved from http://kokki-
nizita.linuxaudio.org/linuxaudio/downloads/ambdec-manual.pdf
Anderson, J. (2011). Authoring complex Ambisonic soundelds: An artist’s tips
& tricks. Presented at the Digital Hybridity and Sounds in Space Joint Symposium,
University of Derby, UK. Retrieved from https://www.researchgate.net/publica-
tion/273944382_Authoring_complex_Ambisonic_soundfields_An_artist%27s_
tips_tricks
Batke, J.-M. (2009). The B-Format Microphone Revised. Ambisonics Symposium, 5.
Celerier, J.-M. (2018). Authoring interactive media: A logical & temporal approach
(PhD thesis). University of Bordeaux, France.
Chowning, J. M. (1977). The Simulation of Moving Sound Sources. Computer Music
Journal, 1(3), 48–52.
Fellgett, P. (1975). Ambisonics. Part one: General system description. Studio Sound,
17(8), 20–22, 40.
Grond, F., & Lecomte, P. (2017). Higher Order Ambisonics for SuperCollider. Pre-
sented at the Linux Audio Conference.
Guillot, P., Paris, E., & Deneu, M. (n.d.). La bibliothèque de spatialisation HOA pour
Max/MSP, Pure Data, VST, FAUST. Revue Francophone d’Informatique et Musique.
Retrieved from https://revues.mshparisnord.fr:443/rm/index.php?id=245
Heller, A. J., & Benjamin, E. M. (2014). The Ambisonic Decoder Toolbox: Extensions
for Partial-Coverage Loudspeaker Arrays. Presented at the Linux Audio Confer-
ence, Karlsruhe, Germany.
Hollerweger, F. (2008). An Introduction to Higher Order Ambisonics. Retrieved
from http://o.mur.at/writings/HOA-intro.pdf
Lecomte, P., Gauthier, P.-A., Langrenne, C., Berry, A., & Garcia, A. (2016). Filtrage direc-
tionnel dans un scène sonore 3D par une utilisation conjointe de Beamforming et
Fluxo de Dados: Visualização e Sonicação
96
#18.ART • 2018 • ISSN: 2238-0272
d’Ambisonie d’ordre élevé. 13ème Congrès Français d’Acoustique Joint Avec Le
20ème Colloque VIbrations, SHocks and NOise, CFA/VISHNO. Presented at the Le
Mans, Sarthe, France. Le Mans, Sarthe, France.
McCartney, J. (1996). SuperCollider: A New Real Time Synthesis Language. Interna-
tional Computer Music Conference, 257–258. Hong Kong: International Computer
Music Association.
Mott, I. (2017). Botanica: Navigable, immersive sound art. Presented at the 16o
Encontro Internacional de Arte e Tecnologia, Porto, Portugal.
Pulkki, V. (1997). Virtual Sound Source Positioning Using Vector Base Amplitude Pan-
ning. Journal of the Audio Engineering Society, 45(6).
Schmeder, A., Freed, A., & Wessel, D. (2010). Best Practices for Open Sound Control.
Presented at the Linux Audio Conference.
Wilson, S., Cottle, D., & Collins, N. (Eds.). (2011). The SuperCollider Book. Cambridge,
Massachusetts & London: MIT Press.
Wishart, T. (1996). On Sonic Art (2nd ed.; S. Emmerson, Ed.). Amsterdam: Harwood
Academic Publishers.
... The soundtracks for the videos, which include synchronised voice, offs, music and sound effects, are heard by the viewer on headphones. The audio is processed using the extension class to sound synthesis language SuperCollider called Mosca [8,10]. On headphones, Mosca is used to produce a three-dimensional rendering of audio via ambisonic mechanisms, positioning sounds in three-dimensional space (Figure 9). ...
... The spectator initiates the video and associated lighting events (see below) by pressing one of several buttons on the base of the peepshow; The buttons are coloured red, black, green and blue and respectively perform the functions of initiating the video and lighting sequence, stopping the sequence, adding Portuguese subtitles and adding English subtitles. The buttons are connected to 10 The ...
... Although I might try to investigate a way to integrate Immlib [7], it is not thought with a special focus on rhythm and sequencing, in the same way as the Audio Spray Gun [8]. And concerning 3Dj [9] or Mosca [10], they both offer a framework for real-time interactive 3D sound spatialization, but only based on trajectories. ...
... Since the tool's purpose is to play real concerts within physical spaces with many loudspeakers, the development of spatial aspects is currently not worth it. Despite the increasing number of spaces housing high-density loudspeaker arrays, the only times I had the opportunity to play with more than 8 loudspeakers were concerts at my own university thanks to an Ethernet-based system 10 . When I performed during the Journées d'informatique Musicale in 2019 or the ICMC conference in 2018, only 8 spatial loudspeakers could be used for performances 11 . ...
Conference Paper
Full-text available
This paper describes the beginning of the story and the insides of a sound performance tool implemented in Super-Collider, whose objective is to have an ergonomic spatio-temporal and spectral control over numerous sound objects in real time, in order to swing between spatialized textures and polyrhythms. After a brief review of some spatial / sequencing / performance tools with a graphical user interface built with Su-perCollider, the incentives behind the creation of another new tool are explained, as well as discussing development strategies and more recent considerations between coding and composing from a performer's perspective. This spatial performance tool is finally detailed by focus-ing on the composition process and structure of one of the core parameters: the space (motion and rendering algorithm). A way of setting up a library of predefined spatial-ization models with a dynamic and quick selection among both rendering algorithms and also concrete and more abstract spatial composition techniques is proposed and detailed practically through the combination of different parameter modules.
... Inspiré par le créateur de Mosca [1] (logiciel de spatialisation), l'artiste sonore australien Iain Mott, ce projet se propose de concevoir un dispositif permettant la création d'espaces sonores virtuels binauraux explorables physiquement par les spectateurs. Ce dispositif s'appuie sur un système de positionnement en intérieur par radiofréquence, précis et rapide. ...
... IL estégalement possible de déplacer et de faire pivoter l'ensemble du référentiel de l'espace sonore. Avec Mosca, Iain Mott a réalisé l'installation sonore immersive Botanica [2] , conçue par l'association d'un module GPS, d'un capteur inertiel et géomagnétique, d'un microcontrôleur, d'un ordinateur 1 SuperCollider est un environnement et un langage de programmation pour la synthèse audio en temps réel et la composition algorithmique. portable et d'un casque audio. ...
Conference Paper
Full-text available
Inspiré par le créateur de Mosca (logiciel de spatialisation), l’artiste sonore australien Iain Mott, ce projet se propose de concevoir un dispositif permettant la création d’espaces sonores virtuels binauraux explorables physiquement par les spectateurs. Ce dispositif s’appuie sur un système de positionnement en int érieur par radiofréquence, précis et rapide. Il est articulé autour d’une architecture logicielle sous licence libre et unesolution materielle abordable. Le studio de création et de recherche en informatique et médias expérimentaux (SCRIME), promoteur et expérimentateur des techniques despatialisation du son, est à l’origine de ce projet.
... The viewer listens to binaural audio on headphones. The soundtrack is processed using the extension class to sound synthesis language SuperCollider 22 called Mosca [7,8]. On headphones, Mosca is used to produce a three-dimensional rendering of audio via ambisonic mechanisms, positioning sounds in three-dimensional space. ...
Conference Paper
Full-text available
Miniature theatrical environments are described for a mixed-media installation featuring videotaped performance. The modules of the installation use the design of an 18th century French peepshow as a point of departure. The original viewing cabinets, made as amusements for parlours and salons, offered a primitive form of stereoscopy by way of a large bi-convex lens and gave individuals viewing two-dimensional coloured images the impression of depth and three-dimensionality, an effect often enhanced using prosceniums. In the current work, this basic form is hybridised with a contemporary expression of another archaic though persistent artistic technique, the 19th century theatrical illusion, the Pepper's ghost and in addition to two-dimensional scenography inside the peepshow, objects are included to better integrate the ghost image with its three-dimensional confines. LED lighting inside the cabinets is dynamically controlled and synchronised with the videotaped ghosts and further, ambisonic audio is delivered on headphones to promote a sense of immersion in the scene and an association of the viewer with focal characters.
Thesis
Full-text available
The research/creation project is focused on the development of a spatialised sound creation tool, which aims at facilitating the improvisation of electroacoustic music on multiple loudspeakers. More specifically, this tool, which is called Live 4 Life, allows to compose spatialised polyrhythms, combined with enveloping textures in real time. Currently, it is particularly difficult to dynamically and simultaneously control mul- tiple sounds in space, due in part to the layering of the composition process in multiple software programs and the complexity of individually spatialising each sound event. By providing the improviser with flexible spatialisation tools, it allows him to exploit spa- tial dimensions throughout the creative process, from composition to performance. The integration of other sound parameters, such as rhythm, spectrum or playback speeds, in a global creation tool also allows to create new links in the composition of space. The tool, implemented on the open source platform SuperCollider, offers an ap- proach comprising different time scales, where spatialisation is determined both locally on individual events and globally on sound streams. Spatially and temporally open forms can thus be realised from multiple pre-composed sequences, which are organised and modified in real time. This spatial performance tool emphasises the construction and de- formation of parameter loops, rhythmic or spatial. It also makes it possible to question the spatio-temporal development between sound objects themselves, their processing or their reflections, and opens up broader questions on the links that exist between space and real time.
Chapter
Full-text available
This article describes artistically and technically, “Botanica”, a sound art work for gardens by the author. In the work, participants, together with a guide, use headphones and a laptop-equipped shoulder bag to explore an immersive sound composition overlaid on the landscape. The sounds heard are, for the large part, derived from nature and involve audio recordings from the cerrado environments of Brazil’s central west. As participants walk through the garden, their movements are tracked by GPS and this allows them to hear a variety of sounds mapped to different locations. The overlapping soundscapes encountered are 3-dimensional and head-tracking on the headphones helps to deliver a interactive immersion responsive to the finest movements of the listener. Sounds may be fixed in space, acting like beacons, or may follow their own trajectory through the garden. The headphones are of an open type, allowing the natural sounds of the environment to blend with those of the overlay. Botanica is presented as a sonic fantasy. It is inspired by landscape, the biosphere and the elements and is returned to the environment in a form to be explored in a specific setting, the garden, another fantasy of and for nature. It is a sound composition organised in space rather than time and as such interacts directly with the environment and with human presence. Botanica is an outcome of his research in the field of ambisonics (a form a surround sound) and specifically the project “Ambisonic Cerrado” which involves the use of ambisonic field recordings of the cerrado in theatrical contexts. The author’s open source computer software “Mosca” (an ambisonic extension class to the “SuperCollider” music language) was created in the research and forms the technical basis of Botanica.
Article
Full-text available
Dans cet article, nous présentons la bibliothèque HOA, un ensemble de classes C++ dédié à la spatialisation du son et fondé sur des techniques ambisoniques d’ordre supérieur. A travers les mises en œuvre pour les logiciels Max/MSP et Pure Data, sous forme d’un plugin VST ainsi que le portage en langage FAUST, nous mettons en avant les difficultés rencontrées, les améliorations réalisées ainsi que les avantages de chaque plateforme logicielle pour notre approche particulière de l’espace sonore.
Conference Paper
Full-text available
When first working with Ambisonics, the artist is usually presented with encoding (panning) tools intended to spatialise single point sources. When complex soundfields are generated through this approach, the results may be regarded as created via a sound-scene based paradigm in some way modelling a real- world scene. In contrast, this paper presents a sound-kernel paradigm and techniques for the generation of complex soundfields. This approach instead focuses on the direct generation of complex soundfield kernels as a starting point. A number of spatial authoring approaches, ranging from time/spatial-domain to frequency/spatial-domain techniques, are presented, discussed and illustrated with sound examples—which include musical excerpts from the author’s catalogue. Strategies to ‘upgrade’ well known two-channel stereo effects to Ambisonics and soundfield kernel processing are also introduced. Additionally, spatial imagining transforms from the author’s Ambisonic ToolKit (ATK) are described and demonstrated, enabling the artist to control the generated complex soundfields. Keywords: ambisonics, surround sound, spatialisation, spatial filtering, digital audio effects
Conference Paper
Full-text available
We present extensions to the Ambisonic Decoder Toolbox to efficiently design periphonic decoders for non-uniform speaker arrays such as hemispherical domes and multilevel rings. These techniques include modified inversion, AllRAD, and spherical Slepian function-based decoders. We also describe a new backend for the toolbox that writes out full-featured decoders in the Faust DSP specification language, which can then be compiled into a variety of plug- in formats. Informal listening tests and performance measurements indicate that these decoders work well for speaker arrays that are difficult to handle with conventional design techniques. The computation is relatively quick and more reliable compared to non- linear optimization techniques used previously.
Article
Full-text available
A digital computer was used to generate four channels of information, which are recorded on a tape recorder. The computer program provides control over the apparent location and movement of a synthesized sound in an illusory acoustical space. The method controls the distribution and amplitude of direct and reverberant signals between the loudspeakers to provide the angular and distance information and introduces a Doppler shift to enhance velocity information.
Thesis
Interactive media design is a field which has been researched as soon as computers started showing audio-visual capabilities. A common research theme is the temporal specification of interactive media objects: how is it possible to create multimedia presentations whose schedule takes into account events external to the system.This problem is shared with another research field, which is interactive music and more precisely interactive scores. That is, musical works whose performance will evolve in time according to a given score.In both cases, it is necessary to specify the medias and musical data orchestrated by the system: this is the subject of the first part of this thesis, which presents a model tailored for the design of multimedia applications. This model allows to simplify distributed access and remote control questions, and solves documentation-related problems.Once this model has been defined, we construct by inspiration with well-known data-flow systems used in music programming, a computation structure able to control and orchestrate the applications defined previously, as well as handling audio-visual data input and output.Specifically, a notion of permanent environment is introduced in the data-flow model: it simplifies multiple use cases common when authoring interactive media and music, and improves performance when comparing to a purely node-based approach.Finally, a temporal graph structure is presented: it allows to score parts of the data graph in time. Especially, nodes of the data graph are studied in the context of both synchronous and delayed cases.A visual edition language is introduced to allow for authoring of interactive scores in a graphical model which unites the previously introduced elements.The temporal structure is then studied from the distribution point of view: we show in particular that it is possible to earn an additional expressive power by supposing a concurrent execution of specific objects of the temporal structure.Finally, we expose how the system is able to recreate multiple existing media systems: sequencers, live-loopers, patchers, as well as new multimedia behaviours.
Conference Paper
L’ambisonie d’ordre élevé (higher order ambisonics) est une technique de représentation d’un champ de pression sur une base tronquée d’harmoniques sphériques. La capture d’un champ sonore naturel est traditionnellement faite à l’aide d’un réseau sphérique de microphones pour obtenir les composantes ambisoniques. Cependant, un microphone sphérique offre également des possibilités de formation de voies (beamforming) dans le domaine des harmoniques sphériques : en combinant les différentes composantes ambisoniques, un signal monophonique est obtenu équivalent à celui issu d’un microphone virtuel respectant la directivité choisie pour la voie. Cet article présente une technique de filtrage directionnel d’une scène sonore tridimensionnelle encodée dans un formalisme ambisonique. En alliant les techniques de beamforming et d’ambisonie d’ordre élevé, il résulte une scène sonore tridimensionnelle filtrée spatialement avec la fonction de directivité et toujours encodée dans un formalisme ambisonique.