Content uploaded by Matthias Kulcke
Author content
All content in this area was uploaded by Matthias Kulcke on Feb 24, 2021
Content may be subject to copyright.
The Sound of Spatial Relation
Music as sequential representation of multidimensional adjacencies
Matthias Kulcke1
1HafenCity University/Glueckstadt School of Music
1matthias@kulcke.de
This article scrutinizes the possible use of musical notations as graphic and
sound as acoustic representations of multidimensional adjacencies. The
commonly used representation of adjacencies as a graph or a bubble diagramm
may only supply limited clarity in certain cases, especially if several layers of
parameters are involved. This research shows the possible use of adjacency
matrices, to produce variations of musical notation, and by this musical sound
events relating to spatial configurations. This musical representation of spatial
design is introduced into the design process as an additional feedback mechanism
between human and material via machine.
Keywords: Adjacency, Music, Space
INTRODUCTION
Music has been used by some architects to add yet
another poetical element to architecture. There have
been several attempts to introduce sound architec-
ture as a term or as a built environment by musicians
and architects.
Adjacencies play an important role in architec-
tural design and they do so in several different layers,
e.g. in relation to human movement or sensory ap-
paratus. Thus reachability relating to access as well
as sound, vision, smell and other sensory layers may
be identified, described and analyzed in the course of
the development of an architectural design.
Information on nearness, however complex, may
be described, stored and used conveniently in an
information technology system as multidimensional
matrices.
Humans tend to need other forms of representa-
tion e.g. as a graph or a bubble diagram (for a cur-
rent example of digital design using adjacency rep-
resentation see Lorenz and Wurzer 2019). Their use-
fulness may be limited by complexity, if several layers
are depicted and/or a certain number of parameters
is exceeded. Therefore the development of further
materializations to represent degrees of nearness as
feedback tools between computational analysis and
the human user evaluating computational output is
bound to be useful in an iterative design process.
The author has been active in the field of elec-
tronic and algorithmic music since 1994 as one of the
composers and as guitarist within the industrial mu-
sic group KiEw parallel to his architectural endeav-
ours and is currently lecturer in the field of architec-
ture at the HafenCity University as well as the head-
master of the Glueckstadt School of Music. The no-
tion to connect electronic music with spatial design
stems from this professional background. It is among
other things a reflection on an underlying language
D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1 - eCAADe 38 |717
to handle the diverse materials (in reference to Ma-
chover 1985) of music and architecture.
MUSICAL COMPOSITION AND ADJACEN-
CIES
A sound event may be viewed as a wittgensteinian
object, the configuration of sound events to form
among others melody lines and chords as a wittgen-
steinian Sachverhalt. But although sound events are
in themselves configurations of loudness, duration,
pitch, distortion etc. - micro- and macroconfigura-
tions of sound events are nonetheless definable and
adjustable by more or less complex matrices describ-
ing rules of nearness/adjacency and thus chronology
i.e. the timeline of appearance and disappearance of
sound events.
Figure 1
flow chart for the
production of an
adjacency matrix
out of musical
notation
One strategy of discovering a composer’s craft and
artistic unique approach to sound is to deduce his
specific rules and constraints in the assembly of
sound events within his selectionspace of sound ma-
terial from given examples (see e.g. Ruschkowski).
This can be done, among other methods, by the
use of adjacency matrices in combination with tech-
niques of statistical analysis.
Musical composition of melody lines consists
of variations of one tone following another, of the
length of a note succeeding the same or another
length of a note resulting in tonal movement and
rhythm. The order of tonal succession, note length,
loudness and other parameters defines not only a
composition but is also revealing in regards to com-
positional rules and constraints.
These rules and constraints can be coded as ad-
jacency matrices (fig. 1). By using these matrices a
possibly vast number of other compositions adher-
ing to the compositional style stored in this multidi-
mensional matrix may be produced (fig. 2).
Thus adjacency description through matrix rep-
resentations may allow for analysis, compositional
fingerprinting and testing of theses regarding com-
positional rules and constraints.
TRANSFORMING ADJACENCIES INTO MU-
SIC
Once the adjacency matrices are formulated and ad-
ditional optional rules and constraints for its use
defined considering the number of appearances of
certain adjacencies, a number of configurations of
sound events can be derived from this material.
Melody Lines and Chords
Nearness and nearness evaluation can be repre-
sented in tonal and harmonic representation, in mu-
sical notation and sound. The choice of the specific
sounds arranged as chords to represent e.g. relations
of indifference, harmony, tension, conflict and clash
(fig. 3) is up to the user and may as described above
be stored in a matrix, producing these sounds upon
triggering, controlled by rules and regulations estab-
lished beforehand.
718 |eCAADe 38 - D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1
Figure 2
relations of sound
event sequence,
bubble diagram
and 2D adjacency
matrix
Figure 3
relations of
soundevents from a
tonal perspective
Thus certain spatial relations in the dynamic pro-
cess of arrangement in the design phase may trigger
sounds according to the compositional matrix and
the rules and regulations in regards to e.g. desirable,
indifferent and conflicting nearness.
Tonal relations in melody lines and chord pro-
gressions according to a compositional strategy can
be stored in multidimensional matrices. Several lay-
ers of refinement, like matrices that are valid only af-
ter a number of iterations of sound production or ma-
trices that regulate the importance of a specific adja-
cency in comparison to others, may serve a differen-
tiated outcome (fig. 4). The rather simplistic tonal ap-
proach to the expression of e.g. indifference or con-
flict is owed to its descriptive purposes serving the
D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1 - eCAADe 38 |719
method proposed here. Higher degrees of relational
complexity and broader fields of sound expression
are easily introduced e.g. through the idea of music
as a cognitive experiment (as discussed among oth-
ers by Ungeheuer 1991) dating back to the 1950s.
Figure 4
multidimensional
matrix representing
nearness and tonal
relations in
nearness
SPATIAL DESIGN AND SOUND EVENTS
It is important to note, that the objective of this article
is not to simulate a perception of (see Manning 1988)
or mimic space (Riethmüller 1976) through the use
of sound events. If musical compositional strategies
can be laid down in the form of adjacency matrices
and be connected with descriptive musical expres-
sion (which is never free of a personal element -> see
e.g. Hamlin and Roads 1985) referring to nearness,
then it is possible to supply spacerelevant adjacen-
cies in the process of dynamic change with meaning-
ful sound events to aid the human cognition in iden-
tifying complex substructures while manipulating a
given configuration of spatial elements.
Didactic Scenarios
The roles in this musically enhanced design process
can be distributed differently according to time bud-
get as well as experience in variations of didactic sce-
narios:
The instructor defines sound events triggered by
experimental spatial arrangement and also the ac-
companying rules and constraints. The students then
experiment with design variations in the combina-
tion and order of rooms and architectural elements
of a given spatial setting (fig. 5).
The spatial elements and their interrelations are
given. In addition the teacher only defines the com-
positional matrix and leaves the definition of addi-
tional rules and constraints up to the individual stu-
dents or the student group. On this basis the stu-
dents experiment with design variations in the com-
bination and order of rooms of a given spatial setting
(fig. 6). In a more sophisticated setting, they may use
an individually chosen spatial task.
The compositional matrix is defined individually
by the students, the rules and constraints to trigger
sounds according to spatial constellations are pro-
duced as a group effort based on interviews or ques-
tionnaires with future users (fig. 7).
Figure 5
didactic scenario 1
Figure 6
didactic scenario 2
Figure 7
didactic scenario 3
The proposed scenarios are designated for stu-
dents of architecture in experimental courses with
interdisciplinary outlook. The three variations are
sorted according to a rising degree of responsibility
on part of the students. The first scenario is meant
to be used with bachelor, the other two with master
students. A number of further learning scenarios are
conceivable, especially if the cooperation with a de-
partment of music is sought.
720 |eCAADe 38 - D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1
Figure 8
.csv data imported
into SonicPi and
converted into
sound events
Space Moved by Sound
A more experimental and musically more sophisti-
cated approach is the musical interaction with the
spatial arrangement. If an undesirable sound is pro-
duced by a chosen spatial arrangement, students can
insert sound e.g. via microphone into the system - by
singing a harmonically more desirable pitch in rela-
tion to the tonic to trigger an automated spatial al-
teration of the system (see Stroud on the simplicity
of sound manipulation by voiced pitch in comparison
to oscilloscope dials).
TECHNICAL PROTOTYPING
To realize these didactic concepts prototypical soft-
ware has been developed as a first step to produce
sound responses to spatial changes in simplified ar-
chitectural floorplans, both laid down in adjacency
matrices, connected by rules and constraints for the
detection of significant spatial change and triggering
of an auditive response. This software is embedded
in a workflow including the use of SonicPi, a software
that serves as a platform for musical programming
which was chosen out of a generally very diverse va-
riety of existing musical software (Mazzola 2006) due
to its easy access free of charge and simple data con-
nectivity. In practice the compositional matrices and
the matrices to describe spatial relations are stored
in several .csv-files which are accessed and converted
into sound events by SonicPi (fig. 8).
NEARNESS IN BUILT ARCHITECTURE
Sound and Orientation
In built architecture orientation is a crucial problem
in big complexes, expecially those who are meant to
serve people with impairments of senses and bodily
features vital to pinpoint ones location. A system of
individual nearness detection could be enhanced by
utilizing the auditive layer e.g. to aid the visually im-
paired.
GPS trackers may serve to individualize these ori-
entational clues (am I northbound, to high up or too
far down in relation to my destination floor?).
Harmonies and pitch may be connected to cer-
tain rooms, gradual alteration of harmony and pitch
could signal the floor level and other spatial param-
eters relating to current and desired location. Used
this way, the music truly becomes an expression of
reflexes as Adorno wrote on the subject of Strawin-
sky (Adorno 1978).
CONCLUSION AND OUTLOOK
The general question of course arises, how the artis-
tic content can best be read in which phase of the
D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1 - eCAADe 38 |721
creative or receptive process. Should the special-
ist, i.e. the fellow composer rather be drawn to the
compressed representation of the adjacency matrix
or is the quality of the musical work of art solely dis-
cernible from a full exemplary score?
Or is it only the striking instance of how these
symbolically laid down rules and constraints play out
in the real (see Miyazaki 2013), the “wild-life” of its
performative presentation by the virtuos interpreters
who doesn’t know about the secrets of composition
(Harnoncourt 1987)?
Architecture deals in this sense with similar artis-
tic questions as music does. When does building re-
veal itself? Is it the sketches, the drafted plans, the
perfect rendering, the crafted building or the user
and his interaction with the built environment who
unveils the truth and beauty of the art?
Using the outlined method of adjacency repre-
sentation via audio transcription and signals is all the
more interesting to be applied as a means to sig-
nal relative nearness of space and user, e.g. user
destination and user location, bearing these ques-
tions in mind. Especially, when dealing with the el-
derly, crucial practicality and aesthetics may form a
striking overlap of high professional interest. Several
clues as to individual location are desirable, all the
more if certain senses (e.g. the visual apparatus) are
less functional than others. People with a reduced
or impaired eyesight may experience facilitated ori-
entation in vast building complexes like hospitals if
sounds signal information relevant to location - also
they may experience architecture on one more re-
fined aesthetic level.
REFERENCES
Adorno, TW 1978, Philosophie der neuen Musik,
Suhrkamp Verlag, Frankfurt am Main
Hamlin, P and Roads, C 1985, ’Interview with Herbert
Brün’, in Roads, C (eds) 1985, Composers and the
Computer, William Kaufmann Inc., Los Altos, Califor-
nia, pp. 1-15
Harnoncourt, N 1987, Der musikalische Dialog - Gedanken
zu Monteverdi, Bach und Mozart, dtv/Bärenreiter,
München
Lorenz, W and Wurzer, G 2019 ’Visual Representation of
Adjacencies’, Architecture in the Age of the 4th Indus-
trial Revolution Proceedings of the 37th eCAADe and
23rd SIGraDi Conference Volume 2, Porto, pp. 11-20
Machover, T 1985, ’Thoughts on Computer Music Com-
position’, in Roads, C (eds) 1985, Composers and the
Computer, William Kaufmann Inc., Los Altos, Califor-
nia, pp. 89-111
Manning, P 1988, Electronic & Computer Music, Clarendon
Press, Oxford
Mazzola, G 2006, Elemente der Musikinformatik,
Birkhäuser Verlag, Basel
Miyazaki, S 2013, Algorhythmisiert - Eine Medienarchäolo-
gie digitaler Signale und (un)erhörter Zeiteffekte, Kul-
turverlag Kadmos, Berlin
Riethmüller, A 1976, Die Musik als Abbild der Realität - Zur
dialektischen Widerspiegelungstheorie in der Ästhetik,
Beihefte zum Archiv für Musikwissenschaft Band XV,
Franz Steiner Verlag, Wiesbaden
Ruschkowski, A 2010, Elektronische Klänge und musikalis-
che Entdeckungen, Philipp Reclam jun. GmbH & Co.
KG, Stuttgart
Stroud, J 2003, ’The Psychological Moment in Percep-
tion’, in Pias, C (eds) 2003, Cybernetics - Kybernetik -
The Macy-Conferences 1946-1953, Vol. I, Transactions,
diaphanes, Zürich-Berlin, pp. 41-65
Ungeheuer, E 1991, ’Musik als Wahrnehmungsexper-
iment - Analytische Betrachtungen zur Elektron-
ischen Musik der 50er Jahre’, in Ruschkowski, A
(eds) 1991, Die Analyse elektroakustischer Musik - Eine
Herausforderung an die Musikwissenschaft?, Rucksal-
druck GmbH, Berlin, pp. 27-35
722 |eCAADe 38 - D1.T5.S2. EDUCATION AND DIGITAL THEORY – ETHICS, CYBERNETICS, FEEDBACK, THEORY - Volume 1