Conference PaperPDF Available

Sound Shifting: From Soundscape to Soundshape

Conference Paper

Sound Shifting: From Soundscape to Soundshape

Abstract and Figures

Sound Shifting is an artistic research project that focuses on the physical representation of sound - this means the visualization and materialization of invisible phenomena that significantly shape our perception. We present a system that allows the transformation from sound into form in real-time by using a newly developed machine, the Audio Foam Cutter. This machine converts sound into polystyrene stripes that are arranged into sculptural objects. The resulting sound sculptures provide information about the represented sounds by their shape and aesthetic features and expand the range of our auditory perception to the tangible domain. The sound sculptures are snapshots of our soundscape and form a physical archive of sound representations. The Sound Shifting project aims to create an awareness of the materiality of sonic movements and affects.
Content may be subject to copyright.
Sound Shifting: From Soundscape
to Soundshape
Abstract
Sound Shifting is an artistic research project that
focuses on the physical representation of sound this
means the visualization and materialization of invisible
phenomena that significantly shape our perception. We
present a system that allows the transformation from
sound into form in real-time by using a newly
developed machine, the Audio Foam Cutter. This
machine converts sound into polystyrene stripes that
are arranged into sculptural objects. The resulting
sound sculptures provide information about the
represented sounds by their shape and aesthetic
features and expand the range of our auditory
perception to the tangible domain. The sound
sculptures are snapshots of our soundscape and form a
physical archive of sound representations. The Sound
Shifting project aims to create an awareness of the
materiality of sonic movements and affects.
Author Keywords
Sound Shifting; Hybrid Art; Sound Visualization; Sonic
Materialism; Data Visualization.
CCS Concepts
Human-centered computing~Human computer
interaction (HCI); Human-centered
computing~Interface design prototyping
Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights
for third-party components of this work must be honored. For all other
uses, contact the Owner/Author.
TEI '19, March 1720, 2019, Tempe, AZ, USA
© 2019 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-6196-5/19/03.
https://doi.org/10.1145/3294109.3301257
Reinhard Gupfinger
Martin Kaltenbrunner
University of Art and Design
Linz, Austria
reinhard.gupfinger@ufg.at
martin.kaltenbrunner@ufg.at
Luise Wolf
Author and journalist
Berlin, Germany
luisewolf@posteo.de
Figure 1: Detail, Soundscape of Linz, 2017
Photo © Katharina Anna Loidl
Hybrid Materials: TEI Arts and Performance Track
TEI '19, March 17–20, 2019, Tempe, AZ, USA
467
Introduction
How does sound look like as sculpture? At present, we
are constantly surrounded and pervaded by a wide
variety of sounds. We communicate with our
environment by deciphering the meanings of these
sounds and acting accordingly. We often associate
sounds that we experience in different situations with
certain memories and feelings. [[1]] Therefore, we pay
particular attention to the sounds of everyday life and
their repetitions. These sounds of everyday life are very
important, and our goal is to find new ways of making
these sounds tangible as sculptures.
At the center of the Sound Shifting project is the study
and representation of everyday and characteristic
sounds from an artistic perspective. In order to achieve
sculptural results, various methods of making sound
three-dimensional are tested - from rapid 3D printing
(to test formal qualities) to traditional casting
techniques (to preserve the sound sculptures for many
years to come). The use of new rapid prototyping
processes, such as 3D printing, offers new methods for
implementing the sculptural qualities of sound.
Another focus of the Sound Shifting project is the
current discussion about sound in public space as part
of the neo-liberalization of our cities. The project aims
to call attention to and discuss this topic through the
physical representation of problematic sounds. (Figure
2) The poetic aspect of these works is that sound is
made graspable and tangible in its most natural
movement with techno-scientific precision.
Sound Shifting creates an awareness of the materiality
of sound by showing it as sculptural collages from the
traces of a medium that is fleeting and marginal. Sound
is placed into a framework, and so its hidden essence
unfolds a mystical transcendence by making its
energetic impressions, its gravitations and its currents
visible in space. These "sound blocks" (Gilles Deleuze &
Félix Guattari) seem as organic as they are discreet, as
mystical as they are futuristic, cryptic, yet equally
simple and obvious. They show the inner tensions of
the sound itself. You can see and grasp the (sound)
shadows of the waves and experience the rhythm not
as an impact, but as a point of the highest compression
of a single flowing sound wave. The existence of sound
is brought to light in these multi-sized sculptures and
reveal a sonic materialism.
Related Work
From the 70s on, we have seen many approaches to
representing sound or music visually. [3] Currently,
there are a number of digital and analog processes for
visualizing sound on screen. Less common methods try
to represent sound as a physical object. Highly relevant
research has already been done in the context of the
physical representation of data. [6,7] The introduction
of 3D printing technology has expanded the possibilities
for experimenting with objects based on sound.
Through the use of modeling software, sound models
are generated and transformed into sculptures by 3D
printing. An interesting project in the context of the
physical representation of sound is Microsonic
Landscapes [[4]] by the Mexican artist group Realität.
They have visualized music albums and songs by
famous bands such as Portishead or Einstürzende
Neubauten as physical objects. Also, the artist Luke
Jerram is working with this transformation process. On
the basis of an audio recording of the Hiroshima atomic
bomb explosion, he produced a sculpture by
stereolithography. [5]
Figure 2: Hells Bells 2017
This piece refers to an incident
from 2015. A man from Linz,
Austria sued the Catholic Church
to stop nighttime chiming. [2]
The sound reliefs show 12 hours
of chiming of the New Cathedral
in Linz.
Photo ©Katharina Anna Loidl
Hybrid Materials: TEI Arts and Performance Track
TEI '19, March 17–20, 2019, Tempe, AZ, USA
468
Figure 3: Audio Foam Cutter (2016); Photo ©Katharina Anna Loidl Figure 4: Screenshot of audio analysis software
Implementation
The Audio Foam Cutter (AFC) transforms sound into
polystyrene and other similar materials. The device
consists of a hot-wire cutter mounted on a servomotor
that is controlled by audio signals. This combination
allows users to cut the sound input vertically and in
real-time into the material as it is passed through the
machine. The speed with which the material passes
through the machines can be adjusted and is
approximately one second per centimeter. The resulting
cutouts are used to create casting molds for different-
sized objects. Furthermore, the polystyrene stripes can
stand alone as tiny sound sculptures. The software
currently controlling the AFC and analyzing the audio
signal has been programmed with Max/MSP.
AFC projects depict sound sculpturally through its wave
troughs and heights, its traces and depths, its
vibrations and pulsations. They divide the time of the
music into timelines and stop the time of music by
capturing and presenting it in sculpture. They break
down the frequencies, the volumes and the harmonies
and reassemble them in a cluster, in a sculptural track,
in a physical spectrogram. Currently, we are working on
prototypes to read out and play back the sound of
these physical objects. For that, we are trying different
approaches such as the use of lasers or RFID devices.
Thus, in the end, the objects would become sculptural
records that can be replayed again.
Video Link to AFC
https://vimeo.com/193745588
Figure 5: Live Demo @ Ars
Electronica Center, Linz, 2016
The components of the system
and the audio signal flow.
Hybrid Materials: TEI Arts and Performance Track
TEI '19, March 17–20, 2019, Tempe, AZ, USA
469
1. Sound: Record characteristic sounds. This is based
on a research phase that investigates current topics of
our auditory environment.
2. Analysis: The sound recordings are analyzed using
a specially developed software, and the appropriate
frequencies and volumes are adapted and calibrated for
the cutting process.
3. Cutting: The AFC hot-wire cutting device enables
real-time transformation of the sound tracks into
polystyrene. The louder the sounds, the higher the cut
amplitude.
4. Casting: The cast is carried out with ceramic casting
powder or bronze. Finally, the polystyrene strips are
removed from the mold and the relief is cleaned.
Presentation
In the exhibition setup, people are invited to interact
with the AFC and transform their own voice into a small
piece of polystyrene. Additionally, we present the work
process and the resulting archive of transformations of
sounds into small three-dimensional objects. (Figure 7)
We also hope that we can demonstrate the new
machines for playing back the sounds cast in the
objects. In summary, the exhibition demonstrates how
invisible sonic phenomena can be represented
sculpturally and help to overcome the widespread idea
that sound is ephemeral and immaterial.
The project has been presented at numerous festivals
and exhibitions, such as Ars Electronica, Linz; ADAF,
Athens; LAB 30, Augsburg and Traklhaus in Salzburg.
Acknowledgements
We would like to thank the Kunstuniversität Linz for its
support of this project, as well as the Linz Culture
department.
References
[1] Holger Schulze. 2008. Sound Studies: Traditionen-
Methoden-Desiderate. Transcript Verlag.
[2] Hells bells! Austrian sues Catholic Church to stop
nighttime chiming. 2015. Retrieved November 6,
2018 from https://www.rt.com/news/230399-
austrian-sues-linz-cathedral
[3] Atari Video Music. Retrieved November 3, 2018
http://www.atarimuseum.com/videogames/dedicat
ed/videomusic/videomusic.html
[4] Microsonic Landscapes. Retrieved November 6,
2018 from http://www.realitat.com/microsonic
[5] 28 seconds of Hiroshima. Retrieved November 6,
2018 from https://theartstack.com/artist/luke-
jerram/artwork-28-seconds-of-hiroshima
[6] Jack Zhao and Andrew Vande Moere. 2008.
Embodiment in data sculpture: a model of the
physical visualization of information. In Proceedings
of the 3rd International Conference on Digital
Interactive Media in Entertainment and Arts
(DIMEA '08). ACM, New York, NY, USA, 343350.
DOI: https://doi.org/10.1145/1413634.1413696
[7] Yvonne Jansen, Pierre Dragicevic, Petra Isenberg,
Jason Alexander, Abhijit Karnik, Johan Kildal,
Sriram Subramanian, and Kasper Hornbæk. 2015.
Opportunities and Challenges for Data
Physicalization. In Proceedings of the 33rd Annual
ACM Conference on Human Factors in Computing
Systems (CHI '15). ACM, New York, NY, USA,
32273236. DOI:
https://doi.org/10.1145/2702123.2702180 Psy.
2012.
Figure 6: Eisenbahnbrücke, 2016
The Linz railway bridge was one
of the three bridges across the
Danube in Linz. In the summer of
2016, the bridge was demolished.
The sound relief contains the
characteristic sounds of this
bridge.
Photo ©Katharina Anna Loidl
Figure 7: Sound Barrier, 2015
Hybrid Materials: TEI Arts and Performance Track
TEI '19, March 17–20, 2019, Tempe, AZ, USA
470
Conference Paper
Full-text available
'Soundscapes' are an increasingly active topic in Human-Computer Interaction (HCI) and interaction design. From mapping acoustic environments through sound recordings to designing compositions as interventions, soundscapes appear as a recurring theme across a wide body of HCI research. Based on this growing interest, now is the time to explore the types of studies in which soundscapes provide a valuable lens to HCI research. In this paper, we review papers from conferences sponsored or co-sponsored by the ACM Special Interest Group on Computer-Human Interaction in which the term 'soundscape' occurs. We analyse a total of 235 papers to understand the role of soundscapes as a research focus and identify untapped opportunities for soundscape research within HCI. We identify two common soundscape conceptualisations: (1) Acoustic environments and (2) Compositions, and describe what characterises studies into each concept and the hybrid forms that also occur. On the basis of this, we carve out a foundation for future soundscape research in HCI as a methodological anchor to form a common ground and support this growing research interest. Finally, we offer five recommendations for further research into soundscapes within HCI.
Conference Paper
This work-in-progress paper presents three tangible user interface prototypes that use sensor technology in combination with mechanical actuators to provide an interactive physical display of sound and music data for people with visual impairments. The prototypes can be used either separately or combined as input and output devices utilizing pin-based, string-based and wheel-based interaction elements. They were developed as part of the research project Tangible Signals. In the paper, each prototype will be presented separately, including discussion of concept, interaction modalities, hardware design and possible use cases.
Conference Paper
Full-text available
Physical representations of data have existed for thousands of years. Yet it is now that advances in digital fabrication, actuated tangible interfaces, and shape-changing displays are spurring an emerging area of research that we call Data Physicalization. It aims to help people explore, understand, and communicate data using computer-supported physical data representations. We call these representations physicalizations, analogously to visualizations – their purely visual counterpart. In this article, we go beyond the focused research questions addressed so far by delineating the research area, synthesizing its open challenges, and laying out a research agenda.
Chapter
Was höre ich - jetzt? Wie höre ich in diesem Raum oder auf diesem Platz? Die Buchreihe Sound Studies möchte ein Sprechen aus, mit und über Klang eröffnen - über Fach- und Methodengrenzen hinweg, über die Grenzen wissenschaftlichen Sprechens hinaus. Es werden Fragen gestellt nach dem gegenwärtigen, historischen und künftigen Leben von Menschen und Tieren mit Dingen und Lauten; nach den Sounds, die sie gestalten, durch die sie handeln und fremde wie auch vermeintlich vertraute Kulturen erkunden. Der erste Band bietet eine Einführung in Traditionen, aktuelle künstlerische Ansätze und gegenwärtige Methoden - eine allgemeine Einführung in das Forschungsfeld Sound Studies, welches quer zu etablierten Disziplinen und Ausdrucksformen liegt. Er versammelt Texte zu historischen Beispielen der elektronischen Musik, des Radios und der Netzmusik, zu funktionalen Klängen, zur Akustischen Architektur, zur Pop- und Medienmusikwissenschaft, zu medienhistorischen Betrachtungen von Zeit und Klang, zur akustischen Markenkommunikation und zu experimentell-künstlerischen Ansätzen. Mit Beiträgen von Sam Auinger, Roger Behrens, Diedrich Diederichsen, Florian Dombois, Wolfgang Ernst, Golo Föllmer, Thomas Hermann, Daniel Ott, Holger Schulze, Martin Supper, Elena Ungeheuer, Carl-Frank Westermann u.v.a.m.
Conference Paper
Information is becoming pervasive in the contemporary society, and is increasingly saturating the visual senses and the cognitive efforts of the lay masses. As our attention for visual impulses and cognitive effort has become more competitive, new approaches are being pursued to convey information to people in memorable and intuitive ways. With human's inherent proficiency in comprehending the physical affordances present in the real world, some researchers and designers are investigating how meaningful insights can be conveyed by way of "sculpting" data. This paper proposes a domain model to establish the concept of data sculpture as a data-based physical artifact, possessing both artistic and functional qualities, that aims to augment a nearby audience's understanding of data insights and any socially relevant issues that underlie it. This paper also proposes a model of embodiment to capture and analyze the wide and multi-layered spectrum of existing data sculptures. In this model, the introduced concepts of metaphorical distances are used as a means to measure embodiment in data sculpture. The models provide groundwork for useful design principles for effective information communication of socially relevant, data-driven insights to a large, lay audience using data sculpture.
Sound Studies: Traditionen-Methoden-Desiderate
  • Holger Schulze
Holger Schulze. 2008. Sound Studies: Traditionen-Methoden-Desiderate. Transcript Verlag.
  • Atari Video Music
Atari Video Music. Retrieved November 3, 2018 http://www.atarimuseum.com/videogames/dedicat ed/videomusic/videomusic.html