Content uploaded by Daniel Rudrich
Author content
All content in this area was uploaded by Daniel Rudrich on Jul 17, 2017
Content may be subject to copyright.
29th TONMEISTERTAGUNG - VDT INTERNATIONAL CONVENTION, November 2016
Efficient Spatial Ambisonic Effects for Live Audio
(Effiziente r¨
aumliche Ambisonics-Effekte zur Live-Beschallung)
D. Rudrich, F. Zotter, M. Frank
Institute of Electronic Music and Acoustics
University of Music and Performing Arts Graz, Austria
Email: {rudrich, zotter, frank}@iem.at
Abstract
Animated by the award-winning, free ambiX/mcfx VST plugins that improve efficiency and applicability of 3D audio
productions in Ambisonics, this contribution demonstrates an example of their successful usage and extension as live
effects in a latency-critical sound-reinforcement situation. We present 3D-audio live effects developed for and used
during the concert performance of Al Di Meola – Elysium Unplugged & More in July 2016, Graz:
•Spatial Chorus spins the sound source around the audience at controllable speed, similar to the effect inside a
Leslie speaker.
•Multi Delay creates an immersive reverb-like sound by a large number of parallel feedback delays. In contrast to
conventional reverbs, the clarity of the direct sound is not impaired.
•Rotational Delay consists of two feedback delays with a slightly clockwise/anti-clockwise shifting of the delayed
sound. This effect envelopes the audience with a flow of previously played phrases.
1. Introduction
Recent developments in Ambisonics enabled its practical
application in the production of 3D audio as discussed in [1].
Scientific works of D. Malham [2], A. Sontacchi [3], maybe
most of all the one of J. Daniel [4–6] were fundamental
in making higher-order Ambisonics applicable, and the var-
ious software tools such as iem ambi for Pd1, AmbDec2,
WigWare3, Ambisonic Studio4, ICST Ambisonics Tools for
Max/MSP5, Ambisonic Toolkit for Supercollider6, AmbIEM
for Supercollider7, HOA VST plugins8. An extensive list can
be found on Wikipedia9.
The most recent improvement of the applicability were ad-
vancements in reliable decoding [7, 8] and plugins to exten-
sively produce in Ambisonics, such as [9–13].
As outlined in [14], Ambisonics can be used as a capable
multi-venue infrastructure for several playback facilities [15].
Currently, its distribution and storage format is adopted by
new media companies such as youtube/google/oculus, sug-
gesting to use the ambiX file format [16].
Besides the complex broadcasting and multi-venue playback
capacities, efficient spatialization of sound sources is achieved
for large audiences [17, 18], and it provides the possibilities
to develop new immersive real-time audio effects, e.g. those
based on SDM [19], as discussed in [17].
1by Thomas Musil, https://git.iem.at/pd/iem ambi
2by Fons Adriaensen http://kokkinizita.linuxaudio.org
3by Bruce Wiggins, http://www.brucewiggins.co.uk/?page id=78
4by Daniel Courville, http://www.radio.uqam.ca/ambisonic/
5by Jan Schacher and Philippe Kocher,
https://www.zhdk.ch/index.php?id=icst ambisonicsexternals
6by Joseph Anderson, http://www.ambisonictoolkit.net/
7by Christopher Frauenberger et al
https://github.com/supercollider-quarks/AmbIEM
8http://www.mshparisnord.fr/hoalibrary/
9https://en.wikipedia.org/wiki/List of Ambisonic Software
Fig. 1: Ligeti hall with its 29 loudspeakers hanging from the ceiling,
and musicians in concert.
Much has been achieved recently in terms of Ambisonic
effects, recently, with the ambiX plugin suite, which is able
to implement some of the spatial mastering concepts [20],
Ambisonic Widening [21], and more natural things such as
mirroring, rotation, but also warping, as described in [10,22].
The mcfx plugin suite allows to implement complex spatial
convolutive SDM-based reverberation or the virtualization of
spatial MIMO systems [23].
So far, Ambisonics is applicable (semi-) offline in audio
sequencers. With the usage in Al Di Meola’s concert in July
2016 in Graz as both spatialization and live effect unit, we
prove that its efficiency is also real-time capable with low
latencies.
29th TONMEISTERTAGUNG - VDT INTERNATIONAL CONVENTION, November 2016
broadcast
ambisonic
hemisphere
loudspeaker
and balancing
object signal pre-processing
stage
workstation
ambisonics
address
public
scene-based spatial processing
pedal board
spatialization, effects (common & custom)
decoding
encoding
monitors
preamp, dynamics, EQ, level, monitoringmixing desk
DI signals
microphone &
Fig. 2: Audio Setup of Al Di Meola Concert in Graz, July 2016.
2. Concert Audio Setup
The concert audio setup is depicted in Fig. 2. The mixing desk
was used for standard audio signal processing of the micro-
phone and direct signals from stage including leveling, dy-
namics, EQ and monitoring. The processed signals were sent
to the ambisonics workstation (Reaper with ambiX and mcfx
plugin suite) for Ambisonic encoding and effect processing
(spatialization, widening, reverb, common and custom ef-
fects) with 5th-order resolution (36 channels). The blocksize
for the Ambisonic processing was set to 2×128 samples,
resulting in a latency of about 6 ms. A pedal board on stage
allowed a remote engaging and control of the custom effects
by the musician.
The full scene was decoded to the loudspeaker array of
the venue (hemisphere with 29 loudspeakers). Also the
Ambisonic signals and a video stream were broadcasted live
to another venue (12-loudspeaker hemisphere) to address a
larger audience. Both venues used 5th-order AllRAD [7].
3. Common Effects
The following spatialization/room effects were applied
to every input channel and considered direction, source
width/depth, first reflections, and reverberation.
3.1. Spatialization
To bring the mono signals into the Ambisonics domain, the
ambix encoder plugin was used. It allows signal spatialization
to any position of the Ambisonic sphere, either statically
or dynamically using its built-in OSC interface. E.g. an
automatization can be triggered moving the source on a
predefined trajectory. Also with sending random azimuth
and elevation angles, the signal can be spatialized to random
positions. Especially for percussive instruments this creates
an interesting and vivid effect.
3.2. Widening
With the ambix widening plugin a source can be widened
[21] which was used to create depth, which is useful to
separate two sound sources with similar direction and level,
e.g. rhythm and lead guitar. This achieved by choosing the
single-sided mode to prevent pre-echos and a modulation of
about 10 ms to create an impulse response similar to early
reflections. The modulation depth can be used to control the
perceived depth.
3.3. Slapback Delay
The Ambisonic slapback delay is a simple but effective way
to create a first reflection of the signal. This is easily done by
mirroring the input signal at the origin and applying a short
delay as shown in Fig. 3. It was used to efficiently add a room
impression to spatially pronounce transient signals.
For setting the delay time its important to consider the
physical delay introduced by the geometry of the loudspeaker
array, i.e. for rim positions the slapback signal should not
arrive before the direct sound but should also not be too long
to avoid a splitting into two separate auditory events. For the
concert a delay of 80 ms was used.
mcfx: delay
FX send
FX return ambix: mirror
80ms
Fig. 3: Signal flow of Slapback Delay.
3.4. Convolution Reverb
To further add more diffuse room information to the sig-
nals while taking advantage of the Ambisonic format a 5th-
order Ambisonic convolution reverb (mcfx convolver) was
employed. The impulse responses were achieved by upmixing
a first-order Ambisonic impulse response from the openAir10
library, as proposed in [17].
10freely available at http://www.openairlib.net
29th TONMEISTERTAGUNG - VDT INTERNATIONAL CONVENTION, November 2016
4. Custom Effects
The following set of custom effects was made available to the
musicians themselves for individual applying and controlling.
The effects can be seen as an additional tool for musical
expression and emphasis.
The Multi-Delay and Rotational-Delay effects are based on
feedback loops. Many DAWs do not support software feed-
backs due to the risk of user-induced audio instability. Al-
though the DAW Reaper supports the activation of recursive
signal routing, this required excessive CPU load, causing au-
dible clicks, so that this option was avoided. As a workaround,
we implemented feedback via the audio driver. Within the
RME TotalMix application, it was possible to create feedback
loops integrated into Reaper’s processing by live monitoring
of the corresponding inputs. Another possible solution is to
use virtual sound cards with signal routing capabilities.
4.1. Spatial Chorus
This effect was inspired by the legendary Leslie Cabinet
often used with the Hammond organ, which creates a signal
modulation by rotating the loudspeakers.
Here, not the loudspeakers are rotated but the signal itself:
a fast dynamic spatialization spins the signal around the
audience, resulting in a chorus-/tremolo-like sound. The
rotation speed can be controlled via a USB expression pedal,
similar to the break,slow, and fast settings of the Leslie speed
settings. This effect works very well with sustaining sounds
and rotation speeds between 720 and 5400 ◦
/s(2 to 15Hz).
4.2. Multi-Delay
Due to a large set of parallel feedback delays, the Multi-
Delay effect creates a reverb-like sound, which envelopes the
audience while preserving the clarity of the direct sound.
The signal flow is depicted in Fig. 4. It consists of 36 feedback
delays with different delay times, each of them affecting
one of the 36 spherical harmonics. The delay times range
from 100 ms to 300 ms. The feedback attenuation is set to
about 2.4 dB, which directly controls the length of the reverb.
Additional filters attenuate low-/high-end content with each
feedback iteration, leading to a thinning out of the signal. As
a last step, the delayed harmonics get rotated by 72◦.
FX send
feedback
ambix: rotator z
mcfx: filter
mcfx: gain delay
FX return
100...300ms
Fig. 4: Signal flow of Multi-Delay.
Due to the rotation and the application in the spherical
harmonics domain the spatialization of one delayed signal
changes with each iteration. This again leads to a quasi-
random spatio-temporal appearance, which is typical for a
diffuse reverb. In contrast to conventional reverbs, this
spatialized reverb preserves the clarity of the direct sound,
even at long reverberation times.
4.3. Rotational Delay
This effect embodies a dynamically spatialized enhancement
of a simple stereo delay. Instead of being spatialized to
a fixed position, each iteration of the feedback delay gets
rotated clockwise (or anti-clockwise, respectively) by a fixed
amount around the z-axis (see Fig. 5). The delay times and
the rotation values for the two feedback delays are chosen
in a way that the rotation speeds for the clockwise and anti-
clockwise rotational delay are about the same: 9◦/300 ms and
10◦/350 ms, respectively, resulting in a rotation speed of about
30 ◦
/s. Additionally, each iteration is filtered with a low-
and high-pass causing coloration of the delays. As a result,
the audience is enveloped by a stream of previously played
phrases moving sideways front to back. To prevent the delays
from moving to the front again, a directional filter attenuates
the delayed signal once reaching the area behind the audience.
FX return
ambix: directional loudness
mcfx: delay
FX send
feedback
ambix: rotator z
mcfx: filter
300ms/350ms
−9◦/+10◦
Fig. 5: Signal flow of Rotational Delay with values for the left and
right delay, respectively.
5. Conclusions
With Al Di Meola’s concert in Graz, 2016, Fig. 1, we
could succesfully show that ambiX plugins and the Reaper
DAW allow to build spatial audio effects with high-enough
throughput for low-latency performance. The prerequisite for
this endeavor to become successful was to enrich the virtuous
performance with acoustic instruments by an impressive 3D
audio experience while preserving the genre-based sound
expectations of both the musicians and the audience.
Due to the encouraging responses, we consider implementing
the above-described effects into separate VST plugins for
improved usability. For instance the Multi-Delay’s 36-channel
feedback delay network can also be used as a kernel for a
reverb within other standard playback formats as 5.1, 7.1 or
the like. What is more, separate VST plugins circumvent the
need of feedback routing within the DAW or audio driver.
6. Acknowledgments
Many thanks to Stefan Zaradic for initiative and idea, and for
connecting us with Al Di Meola and Sennheiser Austria (Hans
Freunberger, Wolfgang Guse), whose funding was gratefully
acknowledged. We thank Elisabeth Freismuth for hosting
the event at our university, the IEM-team (Alois Sontacchi,
Stefan Warum, Vincent Ederle) and the audio and stage crew
of MUMUTH for their support.
29th TONMEISTERTAGUNG - VDT INTERNATIONAL CONVENTION, November 2016
References
[1] M. Frank, F. Zotter, and A. Sontacchi, “Producing 3D
audio in Ambisonics,” in 57th AES Conf., Hollywood
CA, 2015.
[2] D. G. Malham and A. Myatt, “3D Sound Spatialization
using Ambisonic Techniques,” Computer Music Jour-
nal, vol. 19, no. 4, pp. 58–70, 1995.
[3] A. Sontacchi, “Dreidimensionale Schallfeldreproduk-
tion f¨
ur Lautsprecher- und Kopfh¨
oreranwendungen,”
PhD Thesis, TU Graz, 2003.
[4] J. Daniel, J.-B. Rault, and J.-D. Polack, “Acoustic
properties and perceptive implications of stereophonic
phenomena,” in AES 6th Int. Conf.: Spatial Sound
Reproduction, 1999.
[5] J. Daniel, Repr´
esentation des champs acoustiques,
application `
a la transmission et `
a la reproduction de
sc`
enes sonores complexes dans un contexte multim´
edia.
Universit´
e de Paris 6: Th`
ese de doctorat, 2000.
[6] J. Daniel, “Evolving views on hoa: From technological
to pragmatic concerns,” in Proc. 1st Ambisonics
Symposium, 2009.
[7] F. Zotter and M. Frank, “All-round Ambisonic panning
and decoding,” Journal of the Audio Eng. Soc., vol. 60,
no. 10, 2012.
[8] A. J. Heller and E. M. Benjamin, “The Ambisonic
decoder toolbox: Extensions for partial-coverage loud-
speaker arrays,” in Linux Audio Conference, ZKM
Karlsruhe, 2014.
[9] M. Kronlachner, “Ambisonics plug-in suite for pro-
duction and performance usage,” in Linux Audio
Conference, IEM Graz, 2013.
[10] M. Kronlachner, “Plug-in suite for mastering the produc-
tion and playback in surround sound and Ambisonics,”
in AES Student Design Competition 2014, Gold Award,
2014.
[11] P. Lecomte and P.-A. Gauthier, “Real-time 3D Ambison-
ics using faust, processing, pure data, and OSC,” in Proc.
DAFx-15, 2015.
[12] T. Carpentier, M. Noisternig, and O. Warusfel, “Twenty
years of ircam spat: Looking back, looking forward,” in
Proc. ICMC, 2015.
[13] A. Politis and D. Poirier-Quinot, “JSAmbisonics: A web
audio library for interactive spatial sound processing
on the web,” in Proc. Interactive Audio Systems
Symposium, York, 2016.
[14] M. Gurevic and D. D. S. Bertet, “Ambisonic spatializa-
tion for networked music performance,” in Proc. ICAD,
2011.
[15] M. Frank and A. Sontacchi, “Case Study on Ambisonics
for Multi-Venue and Multi-Targe Concerts and Broad-
casts,” J. Audio Eng. Soc, 2017, in press.
[16] C. Nachbar, F. Zotter, A. Sontacchi, and E. Deleflie,
“ambix - suggesting an Ambisonic format,” in Proc. 3rd
Ambisonics Symposium, Lexington, Kentucky, 2011.
[17] M. Frank and F. Zotter, “Spatial impression and direc-
tional resolution in the reproduction of reverberation,”
in Fortschritte der Akustik, DAGA, 2016.
[18] P. Stitt, S. Bertet, and M. van Walstijn, “Off-centre
localisation performance of ambisonics and HOA for
large and small loudspeaker array radii,” Acta Acustica
united with Acustica, vol. 100, no. 5, 2014.
[19] S. Tervo, J. P¨
atynen, A. Kuusinen, and T. Lokki, “Spatial
decomposition method for room impulse responses,”
J. Audio Eng. Soc, vol. 61, no. 1/2, pp. 17–28, 2013.
[Online]. Available: http://www.aes.org/e-lib/browse.
cfm?elib=16664
[20] F. Melchior, U. Michaelis, and R. Steffens, “Spatial
mastering - a new concept for spatial sound design in
object-based audio scenes,” in ICMC, 2011.
[21] F. Zotter, M. Frank, M. Kronlachner, and J.-W. Choi,
“Efficient phantom source widening and diffuseness in
Ambisonics,” in Joint EAA Symposium on Auralization
and Ambisonics, Berlin, 2014.
[22] M. Kronlachner, “Spatial transformations for the alter-
ation of Ambisonic recordings,” M. Thesis, University
of Music and Performing Arts, Graz, Institute of
Electronic Music and Acoustics, 2014.
[23] M. Zaunschirm, M. Frank, and F. Zotter, “An interactive
virtual icosahedral loudspeaker array,” in Fortschritte
der Akustik, DAGA, 2016.
