Carmine-Emanuele Cella

• PhD applied mathematics
• Professor (Assistant) at University of California, Berkeley

In this paper we present the first steps towards the creation of a tool which enables artists to create music visualizations using pre-trained, generative, machine learning models. First, we investigate the application of network bending, the process of applying transforms within the layers of a generative network, to image generation diffusion mod...

The problem of target-based computer-aided orchestration is a recurring topic in the contemporary music community. Because of its complexity, computer-aided orchestration remains a partially unsolved problem and several systems have been developed in the last twenty years. This article presents a practical overview of the recently introduced Orchid...

This article introduces the Orchidea Orchestral Qualities framework (OOQ), an extension of the Orchidea environment for computer-aided orchestration. Traditional target-based orchestration generally reconstructs a target sound “as faithfully as possible” with a collection of samples. But more often than not, composers do not have specific targets i...

This paper has two different aims: presenting the problem of target-based computer-assisted orchestration and introducing a new computational framework to solve it, called Orchidea. After the definition of the static and the dynamic versions of the problem, a historic perspective will be discussed. The cultural context of assisted orchestration wil...

Target-based computer-assisted orchestration can be thought of as the process of searching for combinations of orchestral sounds in a database of sound samples to match a given sound (called target) while respecting specific symbolic constraints (such as the musical instruments that we can use). It is modeled as a combinatorial optimization problem...

Creating a formal model for timbre is one of the most compelling open questions in music research. In contrast to more traditional perceptually-oriented approaches, often aimed at sound analysis, we introduce a three-dimensional geometric model of timbre, specifically designed for sound synthesis. The proposed model relies on the properties of spac...

In this paper we will perform a preliminary exploration on how neural networks can be used for the task of target-based computer-assisted musical orchestration. We will show how it is possible to model this musical problem as a classification task and we will propose two deep learning models. We will show, first, how they perform as classifiers for...

White light can be decomposed into different colors, and a complex sound wave can be decomposed into its partials. While the physics behind transverse and longitudinal waves is quite different and several theories have been developed to investigate the complexity of colors and timbres, we can try to model their structural similarities through the l...

We present a new data representation for music modeling and generation called a Flexible Grid. This representation aims to balance flexibility with structure in order to encode all the musical events (notes or rhythmic onsets) in a dataset without quantizing or discarding any temporal information. In experiments with a dataset of MIDI drum performa...

This paper is about the story of my relationship, as a contemporary music composer, with computational tools that are situated in the areas of signal processing, machine learning and music information retrieval (MIR). I believe that sharing this story can be useful to the MIR community since it illustrates the problems that can arise when you try t...

This paper introduces OrchideaSOL, a free dataset of samples of extended instrumental playing techniques, designed to be used as default dataset for the Orchidea framework for target-based computer-aided orchestration. OrchideaSOL is a reduced and modified subset of Studio On Line, or SOL for short, a dataset developed at Ircam between 1996 and 199...

The wavelet scattering transform is an invariant and stable signal representation suitable for many signal processing and machine learning applications. We present the Kymatio software package, an easy-to-use, high-performance Python implementation of the scattering transform in 1D, 2D, and 3D that is compatible with modern deep learning frameworks...

Within the last 15 years, the field of Music Information Retrieval (MIR) has made tremendous progress in the development of algorithms for organizing and analyzing the ever-increasing large and varied amount of music and music-related data available digitally. However, the development of content-based methods to enable or ameliorate multimedia retr...

The wavelet scattering transform is an invariant signal representation suitable for many signal processing and machine learning applications. We present the Kymatio software package, an easy-to-use, high-performance Python implementation of the scattering transform in 1D, 2D, and 3D that is compatible with modern deep learning frameworks. All trans...

In the past years, several hybridization techniques have been proposed to synthesize novel audio content owing its properties from two audio sources. These algorithms, however, usually provide no feature learning, leaving the user, often intentionally, exploring parameters by trial-and-error. The introduction of machine learning algorithms in the m...

In the past years, several hybridization techniques have been proposed to synthesize novel audio content owing its properties from two audio sources. These algorithms, however, usually provide no feature learning, leaving the user, often intentionally, exploring parameters by trial-and-error. The introduction of machine learning algorithms in the m...

This manifesto paper will introduce machine listening intelligence, an integrated research framework for acoustic and musical signals modelling, based on signal processing, deep learning and computational musicology.

Musical performance combines a wide range of pitches, nuances, and expressive techniques. Audio-based classification of musical instruments thus requires to build signal representations that are invariant to such transformations. This article investigates the construction of learned convolutional architectures for instrument recognition, given a li...

This short paper will present Vuza, a new functional language for computer music and creative coding. The keypoint of the language is to bring the expressivity and the flexiblity of functional programming to digital art and computer music and make possible to embed such power in host applications. Vuza is a general purpose language with specific ex...

In this article some advanced methods for sound hybridizations by means of the theory of sound-types will be shown. After a short presentation of the theory, a formal definition will be given. A framework implementing the theory will be presented in detail, with an emphasis on the modules aimed at the sound transformations. Hybridization is achieve...

Sound-types are a new method to represent and manipulate sounds in a quasi-symbolic way by means of low-level features and subsequent analysis stages. After the presentation of the basic ideas, a full analysis-synthesis framework and some applications will be shown.

It is sometimes desirable, in the purpose of analyzing recorded piano tones, to remove from the original signal the noisy components generated by the hammer strike and by other elements involved in the piano action. In this article we propose an efficient method to achieve such result, based on adaptive filtering and automatic estimation of fundame...

In this article we will propose a new approach for music description, based on the connection between the symbolic (logic) level and the signal level. This approach relies on the possibility of representing sounds in terms of types inferred by some low-level descriptions of signals and subsequent learning stages. We will present simple type theory...

We present a set of extensions to the Sound Description In- terchange Format (SDIF) for the purpose of storage and/or transmission of general audio descriptors. The aim is to al- low portability and interoperability between the feature ex- traction module of an audio information retrieval application and the remaining modules, such as training, cla...