ArticlePDF Available

Asymmetrical Envelope Shapes in Sound Spatialization

Authors:
  • Conservatorio di Musica "G.B.Martini" di Bologna

Abstract and Figures

Amplitude-based sound spatialization without any further signal processing is still today a valid musical choice in certain contexts. This paper emphasizes the importance of the resulting envelope shapes on the single loudspeakers in common listening situations such as concert halls, where most listeners will find themselves in off-centre positions, as well as in other contexts such as sound installations. Various standard spatialization techniques are compared in this regard and a refinement is proposed, which results in asymmetrical envelope shapes. This method combines a strong sense of localization and a natural sense of continuity. Some examples of pratical application carried out by Tempo Reale are also discussed.
Content may be subject to copyright.
Asymmetrical Envelope Shapes
in Sound Spatialization
Francesco Canavese*, Francesco Giomi*, Damiano Meacci* and Kilian Schwoon
* Tempo Reale, Florence, Italy, {fc, fg, dm}@centrotemporeale.it
Hochschule für Künste, Bremen, Germany, k.schwoon@hfk-bremen.de
Abstract — Amplitude-based sound spatialization without
any further signal processing is still today a valid musical
choice in certain contexts. This paper emphasizes the
importance of the resulting envelope shapes on the single
loudspeakers in common listening situations such as concert
halls, where most listeners will find themselves in off-centre
positions, as well as in other contexts such as sound
installations. Various standard spatialization techniques are
compared in this regard and a refinement is proposed,
which results in asymmetrical envelope shapes. This method
combines a strong sense of localization and a natural sense
of continuity. Some examples of pratical application carried
out by Tempo Reale are also discussed.
I. SIMULATIONS AND PATTERNS
Most contemporary research on sound spatialization
focusses on the simulation of other spaces rather than the
actual physical listening space. The idea of placing “an
arbitrary (possibly time-varying) location within an
illusory acoustic space that we hear but do not see” [1]
was pioneered by John Chowning [2] and can be found
nowadays, for instance, in the sophisticated “holographic”
techniques of wave field synthesis [3, 4]. This concept
tries to “hide” loudspeakers as much as possible from the
listeners, in order to create convincing virtual sound
locations.
On the other hand, composers may wish to use
loudspeakers as “instruments” and create interesting
spatial patterns between them. This approach might be
defined as pattern-oriented as opposed to simulation-
oriented. The authors have developed a spatialization
system which originated in live electronic productions by
the Italian composer Luciano Berio. His use of electronic
spatialization seems to be a natural extension of the
principles of his instrumental writing, where “identical
notes or similar figures pass between groups that are
similar in timbre, but separated in space” [5]. In this kind
of musical context, a homogeneous sound quality and
sonic presence is important. Spatial movements should be
achieved by purely amplitude-based methods, without
altering the signals using techniques such as delay,
reverberation or filtering, which are generally involved in
the simulation of spatial depth.
The problem of a more or less small privileged listening
area (sweet spot), which characterizes simulation-oriented
spatialization systems, is less relevant in a pattern-oriented
approach, although patterns are usually also more evident
from a central listening position. In any case, it is useful to
consider not only the privileged central perspective, but to
analyze what actually happens in off-centre listening
positions, where the effective envelope shape applied by a
spatialization algorithm to a single loudspeaker located
close to the listener becomes perceptually significant.
II. COMPARING SPATIALIZATION METHODS
There are some advantages and disadvantages of
common amplitude-based spatialization techniques that
will be examined by comparing one of the most simple
trajectories, a regular rotation on a circular octophonic
loudspeaker setup.
In classical amplitude (or intensity) panning, transitions
between adjacent loudspeakers are controlled by curves
that provide a constant intensity [6]. This obviously
creates a symmetrical envelope beginning at the peak of
the previous loudspeaker and ending at the peak of the
following one (Fig. 1). While this method works fine in a
central listening position, and is also acceptable for slow
movements in off-centre positions, it creates an
undesirable effect of artificial interruption on the single
loudspeakers when the movement becomes too fast.
Whereas the rising envelope shape is tolerable for the
listener, the fast decay and the following zero amplitude
have a rather disturbing quality. Belladonna and Vidolin
noted this very early [7] and implemented a generic
“offset” in their spatialization system (spAAce). Instead of
returning to zero amplitude, a low offset amplitude is kept
continuously on all speakers (Fig. 2). An interesting
analogy can be observed in an implementation of the same
trajectory using Ambisonics. In this spatialization
technique, a sound field is constructed from directional
and omnidirectional components of a previously encoded
signal [8, 9]. Ambisonics implies modulations of ampli-
tude and phase on each loudspeaker. Fig. 3 shows only the
amplitude variations: depending on the weight of the
omnidirectional component of the encoded signal, rather
“blurred” envelope shapes are generated that never
actually return to zero amplitude.
Fig. 1. Envelope shapes in classical amplitude panning.
Fig. 2. Amplitude panning with offset.
Fig. 3. Amplitude curves derived from Ambisonics.
Both spAAce and Ambisonics avoid the problem of
disturbing envelope shapes at high speed on the single
loudspeakers, which is typical for classical amplitude
panning. But they do so by basically smoothing the
movement, and therefore they lose a strong sense of
localization. This is due to the fact, that in all these
techniques sounds “arrive” at a certain loudspeaker in the
same way they “leave” it, generating thus symmetrical
envelope shapes.
III. AN ASYMMETRICAL APPROACH
Considering loudspeakers as instruments in a pattern-
oriented approach, envelope shapes created by
spatialization algorithms can be understood musically as
“articulations”. In order to achieve a strong sense of
localization, the sound on each loudspeaker must be rather
accentuated at the beginning, whereas the decay should be
relatively long, giving way smoothly to the sound on the
next loudspeaker. Therefore, asymmetrical envelope
shapes with a well-defined attack and a longer decay are
necessary.
At Tempo Reale a set of spatialization objects for use in
Max/MSP was developed [10]. These are based on linear
interpolations, which in a second instance are rescaled in
order to obtain constant intensity. The gain factors G for n
loudspeakers are multiplied by a rescaling factor R, which
is calculated as:
For efficiency reasons, R is not calculated at sampling
rate, but only once for each MSP signal vector (which can
be reduced to a single sample in the current MSP version).
Within each signal vector, the interpolation is linear. For
basic transitions between two loudspeakers, this generates
a light S-like curve which very gradually rises/decays near
the extreme values, whereas it is relatively steep at the
centre (Fig. 4). From a listening position close to a
loudspeaker, this curve is often preferable to the standard
square-root or sinusoidal functions used in stereo panning,
which are both very steep near zero.
In the Tempo Reale spatialization system, movements
are generated by scheduled sequences of lists representing
gain values. The interpolation times can be defined
individually for each list. Loudspeaker patterns are usually
described by pseudo-binary gain values, using “1.” for the
active and “0.” for the non-active speakers. If a pattern has
more than one active loudspeaker at the same time, the
gain factors are automatically rescaled as described above:
a pseudo-binary pattern such as (1. 0. 1.) would generate
the effective gain factors (0.71 0. 0.71). Actually, it is
possible to choose arbitrary lists of gain factors, as they
only represent proportions.
A rotation is simply generated by a sequence of lists
scheduled at regular intervals (Table I). It is then possible
to create asymmetrical envelope shapes by defining a
decay factor for successive loudspeaker configurations,
producing a sort of “shadow” of the previous
configurations. For each new loudspeaker configuration in
Table II, the previous gain values are multiplied by a
constant factor d=0.5. This list is then superimposed on
the current list by selecting the higher value at each
position. Fig. 5 shows the corresponding envelope shapes.
Fig. 6 illustrates the envelope shapes obtained by applying
different decay factors to the list atoms and rescaling the
linear interpolations as described above. In all these cases
the curves start rising at the peak of the previous
Fig. 4. S-like amplitude curve.
TABLE I.
SEQUENCE OF LISTS FOR A SIMPLE ROTATION.
spk1
spk2
spk3
spk4
spk5
spk6
spk7
spk8
step1
1.
0.
0.
0.
0.
0.
0.
0.
step2
0.
1.
0.
0.
0.
0.
0.
0.
step3
0.
0.
1.
0.
0.
0.
0.
0.
step4
0.
0.
0.
1.
0.
0.
0.
0.
step5
0.
0.
0.
0.
1.
0.
0.
0.
step6
0.
0.
0.
0.
0.
1.
0.
0.
step7
0.
0.
0.
0.
0.
0.
1.
0.
step8
0.
0.
0.
0.
0.
0.
0.
1.
TABLE II.
LISTS FOR A ROTATION WITH A CONSTANT DECAY FACTOR d=0.5.
spk1
spk3
spk4
spk5
spk6
spk7
spk8
step1
1.
0.
0.
0.
0.
0.
0.
step2
0.5
0.
0.
0.
0.
0.
0.
step3
0.25
1.
0.
0.
0.
0.
0.
step4
0.13
0.5
1.
0.
0.
0.
0.
step5
0.06
0.25
0.5
1.
0.
0.
0.
step6
0.03
0.13
0.25
0.5
1.
0.
0.
step7
0.02
0.06
0.13
0.25
0.5
1.
0.
step8
0.01
0.03
0.06
0.13
0.25
0.5
1.
step9
1.
0.02
0.03
0.06
0.13
0.25
0.5
step10
0.5
0.01
0.02
0.03
0.06
0.13
0.25
Fig. 5. Envelope shapes generated by the lists in Table II.
loudspeaker configuration, but the decay phase is more or
less extended, depending on the decay factor. It can also
be seen how the rising curves vary according to the gain
amount distributed over the other loudspeakers.
The Tempo Reale spatialization also provides routines
for generating symmetrically “blurred” gain distributions
in each list, basically by applying a blur factor to adjacent
loudspeakers. With a blur factor b=0.5, a list such as
(0. 0. 1. 0. 0.) is, for instance, transformed into
(0.25 0.5 1. 0.5 0.25). For regular rotations, this method
also generates symmetrically blurred envelope shapes in
time, rather similar to those of the Ambisonics example
discussed above. Both methods (blurred positions and
extended decays over time) can be freely combined and
may create a great variety of asymmetrical shapes
(Table III, Fig. 7).
IV. SOME EXAMPLES
As mentioned above, the Tempo Reale spatialization
system had been initially developed for Luciano Berio's
live electronic projects. His approach to spatialization in
his late work was extremely pattern-oriented. He
developed a notation system in which he basically defined
sequences of loudspeaker configurations with holding
times (tp) and movement times (tm) for the transitions to
the next loudspeaker configurations. Fig. 8 shows the
notation of a sequence with continuously changing
durations of tp and tm. Moreover, the number of active
loudspeakers in each configuration varies between one
and two. Applying the usual automatic rescaling
mechanisms and a decay factor, the resulting envelope
shapes of such a simple sequence becomes rather complex
(Fig. 9). In the current implementation, the decay factor is
only applied to the scheduled lists that generate the
pattern. Therefore, during the holding times there is no
variation of the gain factors left over from the previous
configurations (the amounts of shadow”). This
emphasizes the contrast between holding and movement,
but of course other implementations, which might
generate more continuous decaying envelopes, may also
have musical significance. As a strategy in live electronic
performance practice, the decay factors are usually
decided in the preproduction phase in the studio. As it is
often necessary to adapt this parameter to the specific
reverberation characteristics of the actual performance
space, the performance system used in these productions
[11] provides efficient rescaling mechanisms for the decay
factors on the level of the single sequence as well as on a
global level. This allows a precise adjustment of the
envelope shapes during the rehearsals.
In his live electronic projects [5], Berio only once used
a classical loudspeaker setup with an octophonic circle
around the audience, namely in Ofanìm (1988–1997) for
female voice, two children’s choirs, two instrumental
groups, and live electronics. In his works of musical
theater Outis (1996) and Cronaca del Luogo (1999) he
experimented with vertical loudspeaker positions. This
idea can also be found in Altra voce for alto flute, mezzo-
soprano, and live electronics (1999), where two diverging
diagonal lines of loudspeakers reach from the musicians at
the center of the stage to the upper left and right corners of
the concert hall. This kind of geometry only makes sense
in a pattern-oriented approach to spatialization. The
Fig. 6. Comparison of rotations with different
decay factors (d=0.3/0.5/0.7).
TABLE III.
ROTATION WITH BLUR FACTOR B=0.4 AND DECAY FACTOR d=0.7.
spk3
spk4
spk5
spk6
spk7
spk8
step1
0.16
0.06
0.03
0.06
0.16
0.4
step2
0.4
0.16
0.06
0.03
0.06
0.16
step3
1.
0.4
0.16
0.06
0.03
0.06
step4
0.7
1.
0.4
0.16
0.06
0.03
step5
0.49
0.7
1.
0.4
0.16
0.06
step6
0.34
0.49
0.7
1.
0.4
0.16
step7
0.24
0.34
0.49
0.7
1.
0.4
step8
0.17
0.24
0.34
0.49
0.7
1.
step9
0.16
0.17
0.24
0.34
0.49
0.7
step10
0.4
0.16
0.17
0.24
0.34
0.49
Fig. 7. Envelope shapes generated by the lists in Table III.
well-defined attacks of the envelope shapes are especially
important in this case to make the different diagonal
loudspeaker positions distinguishable to our ears.
Apart from concert productions, Tempo Reale is often
also involved in projects of sound installation art, like the
one realized in 2002 at the new Auditorium in Rome [12].
Only very rarely, there are central listening positions in
these works: the visitors are free to move in space, in
every position a valid listening experience must be
possible. In most of these productions, loudspeaker
distributions are chosen to create interesting relationships
with particular spatial situations, emphasizing or
transforming geometrical properties of an architecture. For
instance, in the exhibition “Visible cities. Renzo Piano
Building Workshop” (Milan, 2007), Tempo Reale realized
a sound installation, entitled Memory, with 19 loud-
speakers placed in a huge space where the central area
was not accessible. The loudspeakers were hanging from
the ceiling, forming different paths and listening areas
(Fig. 10). Sound movements where mainly structured as
linear trajectories of varying length. Even in a position
directly underneath a loudspeaker, the transition to the
adjacent speaker was clearly perceptible due to the clear
attack of its envelope shape. At the same time the decay
mechanism provided a smooth fadeout on the loudspeaker
above one's head.
V. CONCLUSIONS
The Tempo Reale spatialization system is the result of
practical research carried out by a team of and musicians
and developers dealing with live electronic projects and
sound installations. The basic effort was to overcome
psychoacoustical problems of classical amplitude panning
in a pattern-oriented compositional approach.
Concentrating on the envelope shapes applied to the single
loudspeakers signals, and transforming them with the
asymmetrical methods described in this article has led to
successful psychoacoustical and musical results in a great
variety of situations. These strategies can be applied in a
standard 5.1-channel surround sound setup, as well as
with complex and non-standard loudspeakers
distributions. Movements can be appreciated even in
listening positions relatively close to single loudspeakers,
while retaining a strong overall sense of localization.
The concept of timed sequences of loudspeaker
configurations is very easy to grasp and to deal with for
composers and sound artists. On the other hand, the
resulting envelope shapes can be rather complex and
sophisticated, having a precise control over the
“articulation” of each loudspeaker by adjusting only two
parameters (decay and blur factor). The flexibility of these
adjustments is important, as can be understood in analogy
to human interpreters: musicians more or less
unconsciously change their way of playing, especially
tempo, dynamics and articulation, depending on the
reverberation characteristics of concert halls and their
positions on stage. The possibility of easily modifying the
envelope shapes of the spatialization system is a relevant
new option for sound diffusion and underlines the concept
of loudspeakers as “instruments”. Furthermore, the very
nature of a pattern-oriented approach is to avoid conflicts
between a virtual space and an actual physical listening
space, which may have very particular characteristics on
its own.
Fig. 8. Notation of a spatialization pattern by Luciano Berio.
Fig. 9. Envelope shapes corresponding to Fig. 8.
Fig. 10. Loudspeaker configuration for the installation Memory,
realized for the exhibition “Visible cities” by Renzo Piano.
REFERENCES
[1] F. R. Moore, Elements of Computer Music, Englewood Cliffs,
New Jersey: 1990, p. 387.
[2] J. Chowning, “The simulation of moving sound sources”, Journal
of the Audio Engineering Society, vol. 19 n. 1, pp. 2-6, 1971.
[3] A. J. Berkhout, “A holographic approach to acoustic control”,
Journal of the Audio Engineering Society, 36(12):977-995, 1988.
[4] D. de Vries and M. M. Boone. “Wave field synthesis and analysis
using array technology”, IEEE workshop on applications of signal
processing to audio and acoustics, October 1999.
[5] F. Giomi, D. Meacci and K. Schwoon, “Live Electronics in
Luciano Berio's Music”, Computer Music Journal, vol. 27 n. 2,
2003, pp. 31-32.
[6] see [1], pp. 353-359.
[7] A. Belladonna and A. Vidolin. “spAAce: un programma di
spazializzazione per il Live Electronics”, Proceedings of the
Second International Conference on Acoustics and Musical
Research, Ferrara: 1995, pp. 113-118.
[8] M. Gerzon. “Periphony: With-Height Sound Reproduction”,
Journal of the Audio Engineering Society, vol. 21 n. 1, pp. 2-10
[9] J. C. Schacher and P. Kocher, “Ambisonics Spatialization Tools
for Max/MSP”, Proceedings of the International Computer Music
Conference, ICMA, New Orleans: 2006.
[10] D. Zicarelli “An Extensible Real-Time Signal Processing
Environment for Max”, Proceedings of the International
Computer Music Conference, ICMA, Ann Arbor: 1998, pp. 463-
466
[11] F. Canavese, F. Giomi, D. Meacci, and K. Schwoon, “An SQL-
Based Control System for Live Electronics”, Proceedings of the
International Computer Music Conference, Barcelona: 2005, pp.
753-756.
[12] F. Giomi, D. Meacci and K. Schwoon, “Electroacoustic Music in a
Multi-Perspective Architectural Context: A sound installation for
Renzo Piano's Auditorium in Rome”, Organised Sound, vol. 8 n.
2, 2003, pp. 157-162.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
A digital computer was used to generate four channels of information, which are recorded on a tape recorder. The computer program provides control over the apparent location and movement of a synthesized sound in an illusory acoustical space. The method controls the distribution and amplitude of direct and reverberant signals between the loudspeakers to provide the angular and distance information and introduces a Doppler shift to enhance velocity information.
Conference Paper
Full-text available
The concept of wave field synthesis (WFS) was introduced by Betkhout in 1988. It enables the generation of sound fields with natural temporal and spatial properties within a volume or area bounded by arrays of loudspeakers. Applications are found in real time performances as well as in reproduction of multitrack recordings. A logic next step was the formulation of a new wave field analysis (WFA) concept by Berkhout in 1997, where sound fields in enclosures are recorded with arrays of microphones and analyzed with postprocessing techniques commonly used in acoustical imaging. This way, both the temporal and spatial properties of the sound field can be investigated and understood. WFS and WFA meet in auralization applications: sound fields measured (or modeled) along arrays of microphone positions can be generated by arrays of loudspeakers for perceptual evaluation
Article
The development of electronic music by Luciano Berio is discussed. Three compositions of Berio such as Outis, Ofanìm and Altra voce are presented. The technological solutions designed and implemented for the live performance of these three works are also discussed.
Article
This paper describes the musical and technical design and the realisation of a large-scale sound installation produced and realised by Centro Tempo Reale for the inauguration of Renzo Piano's new Auditorium in Rome. The installation, conceived by Luciano Berio, was mainly dedicated to electroacoustic music: the complex technological set-up offered a new instrument for which electroacoustic pieces were transcribed. The techniques and the aesthetic implications of this transcription process within the context of an installation are discussed.
Article
In the past the temporal aspects of sound fields have obtained considerably more attention than the spatial aspects. Therefore most electroacoustic arguments are based on temporal frequencies and temporal reflection sequences. It is shown that sound control should be based on 'acoustic holography' featuring the spatial reconstruction of direct and reflected wave fields with desired wavefront properties at each moment of time. As holographically reconstructed sound fields cannot be distinguished from true sound fields, it is argued that holographic sound systems are the ultimate in sound control. A description is given of the holographic sound system ACS (patent pending) and measurements are shown.
spAAce: un programma di spazializzazione per il Live Electronics
  • A Belladonna
  • A Vidolin
A. Belladonna and A. Vidolin. "spAAce: un programma di spazializzazione per il Live Electronics", Proceedings of the Second International Conference on Acoustics and Musical Research, Ferrara: 1995, pp. 113-118.
Ambisonics Spatialization Tools for Max/MSP
  • J C Schacher
  • P Kocher
J. C. Schacher and P. Kocher, "Ambisonics Spatialization Tools for Max/MSP", Proceedings of the International Computer Music Conference, ICMA, New Orleans: 2006.
An SQL-Based Control System for Live Electronics
  • F Canavese
  • F Giomi
  • D Meacci
  • K Schwoon
F. Canavese, F. Giomi, D. Meacci, and K. Schwoon, "An SQL-Based Control System for Live Electronics", Proceedings of the International Computer Music Conference, Barcelona: 2005, pp. 753-756.