Stephen Roddy

University College Cork | UCC · Radical Humanities Laboratory

This commentary considers the application of artificial intelligence (AI) and machine learning (ML) technologies to music and the sonic arts. It critiques the classical computational theory of mind (CCTM), a doctrine deriving from functionalism, which codifies the “mind” as a mathematical function in which the symbolic representations of the intern...

This chapter chronicles developments in the application of artificial intelligence (AI) and machine learning/deep learning (ML/DL) technologies in the creative arts from both technical and creative points of view. It then highlights two problematic themes that emerge in the discourse around AI/ML technologies in the arts. These are the master-serva...

Overview of ethical questions and concerns arising from the application of AI technologies in the arts and suggested strategies for dealing with these challenges.

This article introduces the Signal to Noise Loops project, which consisted of a series of performances and installations that took place worldwide between 2017 and 2022. The project utilized open data from a network of Internet of Things sensors placed around Dublin, Ireland, in the context of experimental music performance and composition. This ne...

This paper explores the thought of key Kyoto School philosophers Nishida Kitar_ō, Tanabe Hajime, and Nishitani Keiji as a novel framework through which to consider the historical development and philosophical underpinnings of sound art. Connections and divergences between these thinkers and key developments in the history of sound art are analyzed...

This paper explores the design, evaluation, and formalisation of a framework for the creation of soundscape sonifications: data-to-sound mappings which make use of sounds recorded from real-world sources to communicate information to a listener. This approach is informed by design principles from the field of embodied cognitive science. It opens wi...

Nao Tokui is an artist, researcher, and DJ who has been finding interesting new ways of integrating artificial intelligence and machine learning (AI/ML) techniques into his eclectic body of work since 2015. Stephen Roddy talked with Nao about his approach to AI in his own work and where he sees the relationship between AI and musical creativity goi...

This paper presents a technique for controlling the proportion of information present in parameter mapping sonifications which use time-series data. It suggests treating parameter mapping sonifica- tion as the addition of a modulated, data bearing signal to a carrier signal and the use of a low pass filter on the time-series data to con- trol the a...

Signal to Noise Loops v4 is a data-driven audiovisual piece. It is informed by principles from the fields of IoT, Sonification, Gener-ative Music, and Cybernetics. The piece maps data from noise sensors placed around Dublin City to control a generative algorithm that creates the music. Data is mapped to control the sound synthesis algorithms that d...

This chapter examines the historically prevalent models of cognition that have shaped research methods and techniques in some of the fields associated with sound design. It discusses the efficacy of disembodied models of cognition, which, in favour of reductive explanations, overlook how embodied and perceptual experience can shape and constrain co...

This piece is an online audiovisual installation that uses machine-learning techniques to reflect on the interplay of the artist and the artifact in the context of technologically mediated arts collaborations. This project grew in dialogue with another ongoing project titled 58+1/63 Indices on the Body. That project is a collaboration between the a...

This paper describes an auditory display system for smart city data for Dublin City, Ireland. It introduces and describes the different layers of the system and outlines how they operate individually and interact with one another. The system uses a deep learning model called a variational autoencoder to generate musical content to represent data po...

This article explores how conceptual metaphor theory can be applied to the problem of representing temporal context in the sonification of time series data. It opens with an introduction to some of the conceptual metaphors involved in our understanding of time and music. Two of these metaphors are extended to the domain of Auditory Display and act...

This is a theoretical paper that considers the mapping problem, a foundational issue which arises when designing a sonification, as it applies to sonic information design. We argue that this problem can be addressed by using models from the field of embodied cognitive science, including embodied image schema theory, conceptual metaphor theory and c...

Signal to Noise Loops i++ is a live performance for the PerformIOT system. This system involves the application of techniques and concepts from the field of data-driven music to achieve a balanced co-ordination between algorithmic composition, live looping and improvisation in the context of live electronic music performance. The performance utilis...

IoT and AI-Driven Audio in the Smart City: A Rhythmanalysis inspired approach.

This article explores the mapping problem in parameter mapping sonification: the problem of how to map data to sound in a way that conveys meaning to the listener. We contend that this problem can be addressed by considering the implied conceptual framing of data-to-sound mapping strategies with a particular focus on how such frameworks may be info...

This is a position paper which describes work in progress to develop an AI/ML driven auditory ambient information system which incorporates generative music techniques and considers some of the factors involved the design and development of a system of this nature. The system is intended to represent and communicate information about cryptocurrency...

This paper presents two evaluations intended to examine if listeners are more likely to associate certain vowel formant profiles with specific data types in an auditory display context. The data types and sounds chosen to reflect those data types are informed by findings from the field of cognitive science. The results of the evaluations suggest th...

The third wave of HCI has seen the widespread adoption of design principles borrowed from and informed by breakthroughs in the field of embodied cognitive science. These developments have taken place primarily in the contexts of visual media and interaction, but they are also of importance to the design of auditory displays and interactive systems...

This paper explores topics in embodied cognition, soundscape composition and sonification. It explains the compositional decisions and technical considerations that went into the composition of the piece The Good Ship Hibernia, which is an example of embodied soundscape sonification. This explanation is undertaken within the context of an approach...

Historical Sonification of IOT Network Traffic leveraging Embodied Music Cognition Listening Skills.

This paper explores how the concept of Absolute Nothingness as developed in the thought of three key Kyoto School thinkers Nishida Kitarō, Tanabe Hajime and Nishitani Keiji has influenced the practice of sound art. The paper examines the influence of these three philosophers on D.T. Suzuki and John Cage, the Mono-ha movement, and the Fluxus movemen...

While technology, particularly mobile and smart technology, can be viewed as furthering the eradication of real and "authentic" place; the argument can also be made that these same technologies, can also be instrumental in their recovery. Both alienator and facilitator, the impermeable membrane of the mobile screen and the headphone cuts us off fro...

Auditory display is the use of sound to present information to a listener. Sonification is a particular type of auditory display technique in which data is mapped to non-speech sound to communicate information about its source to a listener. Sonification generally aims to leverage the temporal and frequency resolution of the human ear and is a usef...

Sonification is the communication of data using sound. This thesis is concerned with meaning-making in sonification. It examines how meaning emerges during sonification listening through the lens of embodied cognitive science. It suggests that approaches to sonification which exploit the embodied nature of meaning-making can leverage aesthetic dime...

This paper presents a sonification listening model built from models of embodied cognitive meaning-making faculties. The aim of such a model is to aid in understanding how meaning is applied to auditory stimuli at the cognitive level.this in turn can aid auditory display designers in creating more effective auditory displays. The concept of 'scale'...

Historically, ‘Western Culture’ has been predominantly visuocentric tending to treat sound as an immaterial and secondary phenomenon. The roots of this bias can be traced back through the works of Kant and Hume to classical Pythagorean thought that reduces sound to the sum of its mathematical and mechanical components. Sound was often thought of as...

Aesthetics are gaining increasing recognition as an important topic in auditory display. This article looks to embodied cognition to provide an aesthetic framework for auditory display design. It calls for a serious rethinking of the relationship between aesthetics and meaning-making in order to tackle the mapping problem which has resulted from hi...

Transmissions require the encoding, communication and decoding of meaningful information across some medium or other. Historically, both the artist and the scientist have tended to treat physical space (and the sound wave in particular) as the medium by which music and sound are transmitted. This paper argues against this conception of the sonic me...

This paper makes a case for the use of an embodied cognition framework, based on embodied schemata and cross-domain This paper makes a case for the use of an embodied cognition mappings, in the design of auditory display. An overview of framework, based on embodied schemata and cross-domain research that relates auditory display with embodied cogni...