Content uploaded by Adrian Moore
Author content
All content in this area was uploaded by Adrian Moore on Mar 30, 2016
Content may be subject to copyright.
Cross-modality in multi-channel acousmatic music: the physical and virtual in
music where there is nothing to see.
Adrian Moore
The University of Sheffield
a.j.moore@shef.ac.uk
ABSTRACT
Acousmatic music asks for very active listening and is of-
ten quite challenging. Allowing for cross-modality enables
strong, often physical associations to emerge and potentially
affords greater understanding during the work and greater
recollection of the work. Composers of acousmatic music are
often aware of the need to engage the listener at numerous
levels of experience and understanding, creating sounds that
tend both towards the superficial and the structural. Acous-
matic music in multi-channel formats affords certain degrees
of freedom, allowing the listener to more easily prioritise
their listening.
This freedom potentially affords a more immersive experience
making cross-modal listening easier through the acceptance
of a plausible sonic reality no matter how foreign the sound-
world and despite being deprived of any direct visual cues,
senses of sight and touch are engaged.
1. MAKING SENSE OF THE UNKNOWN
In 2001: A Space Odyssey, Moon-Watcher first hears a sound
then goes to explore:
It was a rectangular slab, three times his height
but narrow enough to span with his arms, and it
was made of some completely transparent mate-
rial; indeed, it was not easy to see except when
the sun glinted on its edges. As Moon-Watcher
had never encountered ice, or even crystal-clear
water, there were no natural objects to which he
could compare this apparition. It was certainly
rather attractive, and though he was wisely cau-
tious of most new things, he did not hesitate for
long before sidling up to it. As nothing hap-
pened, he put out his hand, and felt a cold, hard
surface. [1]
Of course prior to finding the slab, Moon-Watcher had at-
tempted to reach out and touch the moon. And if we look
Copyright: c
2015 Adrian Moore et al. This is an open-access article dis-
tributed under the terms of the Creative Commons Attribution License 3.0
Unported, which permits unrestricted use, distribution, and reproduction in
any medium, provided the original author and source are credited.
at the picture, I’m not sure what would scare Moon-Watcher
more; the fact that the slab is not casting a shadow or the
question “who the hell dug this up?”
But the fact of the matter is, that within a natural environ-
ment, we suddenly have the alien. And why does Moon-
Watcher almost immediately go up to it and touch it, and then
proceed to taste it? Because for him, the situation is real.
For acousmatic music to work, despite the unreal or sur-
real nature of new sonic worlds, whether sounds are natu-
ral recordings, synthetic sounds, or manipulated versions of
soundfiles, if the environment is perceived to be as real or
as plausible as possible, we should be able to do more than
just hear it. We should be able to attempt to understand it. We
might even become part of it as we feel moved beyond a mere
academic attempt to decipher sounds. It is through reaching
out and engaging the senses of sight and touch (through vitual
metaphorical connections) that a work can afford a listener
something more personal than an auditory experience.
As the composed sonic environment becomes more com-
plex, the multi-channel distribution of sounds becomes in-
creasingly important. This paper sets out a personal research
path - investigated through practice for some time - that con-
siders how the multi-channel distribution of a polyphony of
sounds can create an environment that forces a listener to en-
gage other senses; to feel ‘near / far’ and to see ‘dark / light’.
As composers we are sculpting sound. The analogy of a
potter working with clay is often used to explain the ‘hands-
on’ nature of acousmatic music where rules are formulated
upon what we hear. We often rely upon the inherent tactility
of this analogy to mitigate against a lack of visual presence
on stage.
It is very interesting to note that whilst composers and psy-
chologists have approached the problem of cross-modality
in music from vastly different starting points and methods,
their conclusions are surprisingly similar. Interesting too that
the differences between working with acousmatic music and
western classical music are not as divisive as one might at
first think.
We can compare Wishart [2] and Smalley [3] when talking
about aspects of landscape and environment to Gaver’s no-
tions of ‘hard’ and ‘soft’ and the natural sonification of ma-
terials [4, 5]; we can place Godøy’s action gestures of ‘hit’
and ‘blow’ [6, 7, 8] alongside Smalley’s notions of surrogacy
[9] and underpinning our empirical listening strategies is Gib-
son’s ecological notions of affordances [10] which highlight
the unpredictability of our experiential listening.
2. COMPOSITION IN THE STUDIO
Let us not forget that despite the acousmatic veil of the loud-
speaker and the listening challenges this provokes, many sonic
gestures, even if developed purely through computer code,
have their shapes rooted in instrumental gesture and an ana-
logue musique concr`
ete practice that began with the mixing
desk and has moved via the mouse to the Kinect, LEAP mo-
tion, video capture and a growing number of instrumental in-
terfaces.
For example, in the analogue studio, the motion trajectories
of sound were always exaggerated through fader gestures and
this practice continues today during expressive ‘sound diffu-
sion’.
2.1 The survival instinct
Our instinctual listening modes allow us to analyse the audi-
tory scene (see Bregman [11]), dissect it for gestalt properties
relating to figure and ground and then prioritise our listen-
ing towards what we think is important, based upon proxim-
ity, position, amplitude (denoting size) and pitch (denoting
speed), often leading to an assessment of threat. Our dis-
section of an acousmatic phrase works in a similar fashion
though the threats are different. As we dissect, we begin to
traverse a continuum of gesture (metaphor to the body) and
texture (metaphor to our sense of touch). This division and
comparison (especially during composition and reflection) af-
fords frequent use of onomatopoeia as we internalise sounds
with words. Again, the composer’s categorisation of sounds
as ‘short/long/high/low’ resonates with a long line of empir-
ical research based mainly in instrumental music; from Pratt
[12] through to Rusconi et al [13].
As a composer of acousmatic music I am often led from
a ‘quick and dirty’ approximation of a sound’s key charac-
teristics to techniques of development that bring the sound
towards its polar opposite (short - make longer, low - make
higher). This approximation is often compounded by an ex-
aggeration of a sound’s properties, something Peter Lennox
calls cartoonification [14]. This process also affords embod-
ied cognition; retaining key features often through imitation,
exaggeration and approximation, setting up cross-modal links
in the process. As part of my teaching I tend to use a key set of
polar opposites to define both sounds and technical processes
and consider that we choose transformations based upon a re-
action against a sound’s most potent descriptor. This use of
metaphor normally encourages the composer to start thinking
about environment and the listener’s place within it.
3. ENVIRONMENT AND THE MULTICHANNEL
CONDITION.
The multi-channel disposition of sounds (and to a lesser ex-
tent the multi-channel diffusion of stereo material) can afford
an immediate immersion, a throwing in at the deep end if you
like. From that point, there is the potential (as yet not rigor-
ously explored, only speculated upon here) for the listener to
orient their listening with a certain degree of freedom, as there
is the potential for the sonic landscape or environment to have
a degree of redundancy within it - something that, whilst not
counter to the musical flow, affords a degree of polyphony
beyond that achieved by a stereo mix, by employing multiple
channels of audio output through multiple speakers. Through
envelopment and immersion the separation of gesture and tex-
ture can be made explicit: so too the horizontal and vertical
in sound.
However, despite this immersion, the listener must still ne-
gotiate their presence within the space. Smalley’s notions of
personal space, distal space and distorted space [3], and ideas
of being in (flowing with) and out of (reflecting upon) time,
help us place ourselves as: inside and touching; inside and
searching; or outside and searching (three modes that invoke
touch and sight). The links between space and time help the
composer with transformations and ultimately enable struc-
tures to take on form.
3.1 Sounds all around me. Or, is multi-channel music
relieving me of my imagination?
Whilst I am convinced of the relative increases in expressive
potential afforded by working in multi-channel, it does not to
any extent diminish the carefully crafted stereo masterpieces
of the last sixty years which are no less complex and demand
an equal degree of active listening. Multi-channel acousmatic
music may afford sonic redundancies and may introduce lis-
tener freedom.
Denis Smalley’s article on Space-form and the Acousmatic
Image [3] runs to definitions of some 50 different types of
space. Of particular importance are the spaces most closely
associated with distance in relation to the listener. When it
comes to identifying sound shapes, spectral space is clearly
important. Spectral space is a metaphorical impression of
space similar to that of pitch-space; a very low bass rum-
ble accompanied by a very shrill whine will normally define
a large spectral space with low density (nothing in the mid-
dle). Similarly, broadband noise will also (in theory) define
the same space but the density is such that we do not perceive
height or depth, rather simply ‘quantity’.
Spectral space is closely related to perspectival space which
engages with the ‘tracking’ of a sound and a closer under-
standing of it (potentially outside of the course of time’s flow).
Smalley cites Bayle’s notion of environment (with horizon,
temperature, climate and the predominance of high to low
defining gravity). Plausible sonic environments where agents
interact and where nature’s rules such as gravity are followed
can be found in numerous pieces of acousmatic music. Exam-
ples include: the bouncing ball archetype, the doppler effect,
the tendency of higher frequency sounds to require less am-
plitude to be heard and for these sounds often to have more
agile spatial movement. Smalley places these environments
and agents within a navigable gestural space - the perception
of source-cause - interacting within an ensemble space - a
‘scene’ - and focused within an arena space - a frame.
Imagine being close to the loudspeakers listening to a mul-
titude of dry, crisp, aggressive sonic gestures. Our position
is defined by our avoidance of these sounds (as though they
were throwing punches at us). This situation is like landing
in the middle of a dense forest. It is us that have to move. Our
situation is defined by what we bump into. Time feels more
of a challenge.
However, consider a similar environment when our acous-
tic space is viewable and experienced from within but felt to
be ‘distant’. This is the environmental approach and is espe-
cially prominent in multi-channel pieces where loudspeakers
are playing vastly different sounds or granulating across mul-
tiple channels. It is also the case where we perceive a land-
scape in the stereo field. We feel like we could rotate in our
seat and appreciate another perspective as though we were in
a boat in the middle of a lake (figure 1). Normally the com-
ponents of the soundworld are distant from us. The trees and
wildlife are at the periphery of the lake: we can’t reach out
and touch them but we can appreciate their movement from
afar. Time is ephemeral.
Figure 1. peripheral surroundings
Smalley describes the transmodality of spatial perception in
acousmatic music. Key to this is the following:
Transmodal linking occurs automatically when
the sonic materials seem to evoke what we imag-
ine to be the experience of the world outside the
music, and in acousmatic listening (not just acousm-
satic music) transmodal responses occur even though
these senses are not directly activated in order
only to listen. [3, 39]
Smalley cites Chion’s perception of rhythm as ‘trans-sensorial’.
He also cites his own previous work on gesture and surro-
gacy and the body’s understanding of energy transferral. Af-
ter cursory contextualisation of a number of psychological
approaches to perception (in particular the works of Handel
and No¨
e) Smalley reveals his need to be ‘able to act in the
soundscape, being physically able to be in it; know birds as
live, three-dimensional beings that fly; feel the wind; look
at, touch and enter water (how could I know that water flows
from its sound alone?); walk around and climb a tree; travel in
a car; move out of the house into the landscape’, concluding
that; ‘an acousmatic musical work has the potential to harness
my enactment, my spatial enactment.’ [3, 40]
It is through the idea of environment that we can construct a
more embodied experience. More importantly we can accept
that reduced listening is but a fraction of what we should be
doing when attending to an acousmatic work.
3.2 The unreal landscape
Acousmatic compositions are often unreal to many but in fact
the sonic landscapes, no matter how foreign, are to be ex-
pected. Only sound is going to come out of the loudspeaker
after all; not cotton wool, nails or some exotic scent. Perhaps
our only fear is of sound power and frequency content. 1
However, once the practicalities are dealt with, we can con-
centrate upon the auditory scene. Trevor Wishart’s imaginary
landscape remains unrealistic but he claims that, ‘for most lis-
teners it would remain a real landscape’ [2, 47]. In consider-
ing the multi-channel concert environment, Wishart continues
to describe how the listener’s frame of reference may be ma-
nipulated through ‘spinning’. Moreover, proximity and am-
plitude are again related to ‘psychological or social distance’
[2, 48]. We do need to be careful about words like ‘spinning’
however. Any sound made to spin that is not spectrally in-
clined to do so encourages ‘spinning’ as an (extra) musical
parameter and relegates sound to a mere test-tone.
3.3 It’s ‘fake’ but ‘real’
Perhaps the best our multi-channel environments can become
is ‘plausible’. And plausibility relies very much upon a cer-
tain degree of naturalness. Therefore, as a painter would set
his canvas in a frame and proceed to wash it down, the multi-
channel sonic environment is ‘created’ by its boundaries and
horizons.
On the one hand whilst Wishart suggests that ‘imposed’
(gestural) and ‘intrinsic’ morphologies [2, 59] can be blurred
by the acousmatic veil, from a composer’s point of view we
should ask if this is a good thing. Our multi-channel spaces
allow for redundancies that can potentially give morphologies
time to make themselves understood and separate the ‘im-
posed’ from the ‘intrinsic’.
3.4 Balance and redundancy
Perhaps the most powerful aspect of the multi-channel en-
vironment is the ability to separate layer from layer, figure
from ground, texture from gesture. John Young has created
numerous works for 24 independent channels and the result-
ing spaces are indeed plausible with very clearly defined fore-
grounds, surround spaces, and highly identifiable gestural ma-
terial. It would be na¨
ıve to suggest that all wisp-like sounds
must be of short duration and be heavily dispersed around the
space, but their intrinsic morphology suggests an ephemeral
energy profile. Imagine reaching out for a feather floating in
the wind. The feather’s profile may be in full 3-d (high-low,
front-back, left-right) or its descent from high to low may be
approximated to a plane.
1The unbalanced work performed at excessive volume is one that has
failed and unless forewarned and/or provided with earplugs, the listener may
suffer actual bodily harm.
Bernard Parmegiani’s huge cycle De Natura Sonorum high-
lights a number of methodologies that demonstrate how pro-
cess and compositional style are embedded with cross-modal
thought. In movement V, Etude ´
elastique we hear ‘swirling’
sounds and exponential spectral shapes: the metaphor of elas-
ticity and effervescence is tangible. It is clear Parmegiani
is thinking about sound diffusion as channel separation is
very fixed (headphone listening highlights this). Large ener-
gies are centrally panned; whip-like gestures are left or right
focused. Large energies also ‘duck’ the surrounding back-
ground noise (a kind of mask). The compositional question
- and the one that consistently reverts back to a cross-modal
naturalness - is this: to what degree can we engage with the
motion of the initial objects such that a performance diffusion
shapes the sound and is more than mere panning? The very
opening gestures of this movement could mirror the feather
example earlier. They may start in a distant plane and (as an
imposed morphology though diffusion) move to occupy the
surround space. (We may ‘see’ the sound (feather) from a
distance but later ‘feel’ it as its flight surrounds us.)
So it was the case with my work Surface and a number of
more recent works in the 5.1 and 7.1 studio that the centre
loudspeaker became a point of focus, draining the space to a
highly visible loudspeaker or becoming the demonic figure,
spewing forth sound into the space.
More recently in works such as The Battle and Counterat-
tack I have engaged with a surreal painterly version of the
battlefield; Shakespeare’s Birnam Wood meets Twin Peaks’
forest, the idea of visible and covert forces, the structured
attack descending into hand-to-hand combat, the rules of en-
gagement, the art of war. These two works are entirely picto-
rial and my compositional aim was to bring the listener into
the action through plausible environments and a strong sense
of cross-modality.
This is particularly evident in Counterattack where the phys-
icality of sound becomes slightly more ominous through care-
ful but strong use of the Low-frequency effects channel (LFE
or sub). Low frequencies are dark and immobile. Set against
a ‘wash’ generated through varying (and multiple) reverber-
ation plug-ins on pairs of channels, a ‘fake’ expansive space
can be created. Low frequencies alone go part way to creat-
ing a horizontal plane or a sense of depth. Some sort of high-
frequency biased or coloured noise is also required - a kind of
‘air’ to provide enclosure, creating an upper canopy. This un-
occupied space could be felt as ‘cold’. Within this nebulous
space, it is aesthetically viable and compositionally practical
to introduce more obvious ‘forces’. Acousmatic music again
becomes a ‘cinema for the ear’ as a real scene emerges and a
narrative develops.
3.5 Redundancy...again
And so it is that narrative can begin to dictate structure over
materials; form over content. Until that is, the narrative (an
energy of sorts) wanes and materials must, once again sus-
tain the momentum and direct the composition. Materials dis-
persed in the form of multi-channel environments encourage
the use of metaphor and cross-modality (visualising sound,
feeling an environment or texture). This enables the listener
to become immersed in sound whilst actively selecting streams
of sound. It also enables the composer to ‘project’ as well as
‘diffuse’ sound.
4. REFERENCES
[1] A. C. Clarke, 2001: a space odyssey. Hachette UK,
2010.
[2] T. Wishart, “The Relation of Language to Materials,” in
The Language of Electroacoustic Music, S. Emmerson,
Ed. London: Macmillan, 1986, pp. 41–60.
[3] D. Smalley, “Space-form and the acousmatic image,” Or-
ganised Sound, vol. 12, no. 01, pp. 35–58, 2007.
[4] W. W. Gaver, “What in the world do we hear?: An eco-
logical approach to auditory event perception,” Ecologi-
cal psychology, vol. 5, no. 1, pp. 1–29, 1993.
[5] ——, “How do we hear in the world? Explorations in eco-
logical acoustics,” Ecological psychology, vol. 5, no. 4,
pp. 285–313, 1993.
[6] R. I. Godøy, “Motor-mimetic music cognition,”
Leonardo, vol. 36, no. 4, pp. 317–319, 2003.
[7] ——, “Gestural-Sonorous Objects: embodied extensions
of Schaeffer’s conceptual apparatus,” Organised Sound,
vol. 11, no. 02, pp. 149–157, 2006.
[8] R. I. Godøy and M. Leman, Musical gestures: Sound,
movement, and meaning. Routledge, 2010.
[9] D. Smalley, “The listening imagination: listening in
the electroacoustic era,” Contemporary Music Review,
vol. 13, no. 2, pp. 77–107, 1996.
[10] J. J. Gibson, The ecological approach to visual percep-
tion. Routledge, 1986.
[11] A. S. Bregman, Auditory scene analysis: The perceptual
organization of sound. MIT press, 1994.
[12] C. C. Pratt, “The spatial character of high and low tones.”
Journal of Experimental Psychology, vol. 13, no. 3, p.
278, 1930.
[13] E. Rusconi, B. Kwan, B. L. Giordano, C. Umilta, and
B. Butterworth, “Spatial representation of pitch height:
the SMARC effect,” Cognition, vol. 99, no. 2, pp. 113–
129, 2006.
[14] P. Lennox and T. Myatt, “Perceptual Cartoonification in
Multi-Spatial Sound Systems,” International Community
for Auditory Display, 2011.
