Fabio Morreale

University of Auckland · School of Music

Music making is among the activities that best fulfil a person’s full potential, but it is also one of the most complex and exclusive: successful music making requires study and dedication, combined with a natural aptitude that only gifted individuals possess. This thesis proposes new design solutions to reproduce the human ability to make music. I...

In this paper, we reflect on our experience of designing, developing and evaluating interactive spaces for collaborative creativity. In particular, we are interested in designing spaces which allow everybody to compose and play original music. The Music Room is an interactive installation where couples can compose original music by moving in the sp...

Multidisciplinary research recently has been investigating solutions to offer new experiences of music making to musically untrained users. Our approach proposes to distribute the process of music making between the user and an autonomous agent by encoding this collaboration in the emotional domain. In this framework, users communicate the emotions...

This paper proposes discussions for the (re)negotiation of responsibilities in performances that involve a collaboration between human and computer agents. Most of current research is human-hierarchical, leaving the machine the role of a mere tool at performer's service. In this paper a more balanced distribution of responsibilities between the two...

The Music Room is an interactive installation that allows visitor to compose classical music by moving throughout a space. The distance between them and their average speed maps the emotionality of music: in particular, distance influences the pleasantness of the music, while speed influences its intensity. This paper focuses on the evaluation of v...

This study explores the integration of Occupational Therapy techniques within the evaluation of movement and functionality concerning users of Accessible Musical Instruments (AMI). It examines the current application of these techniques in design methodologies and contemplates their potential adaptation and incorporation into AMI design. The paper...

In this paper, we reflect on the focus of "newness" in NIME research and practice and argue that there is a missing O (for "Old") in framing our academic discourse. A systematic review of the last year's conference proceedings reveals that most papers do, indeed, present new instruments, interfaces , or pieces of technology. Comparably few papers f...

Many modern digital products use Machine Learning (ML) to emulate human abilities, knowledge, and intellect. In order to achieve this goal, ML systems need the greatest possible quantity of training data to allow the Artificial Intelligence (AI) model to develop an understanding of “what it means to be human”. We propose that the processes by which...

This article explores the concept of intimacy in the relationship between a musician and their instrument, specifically in the context of designing digital and accessible musical interfaces (DMI/AMI) with disabled musicians. We argue that current DMI/AMI design frameworks are lacking in their consideration of this relationship and that this deficie...

Ethically-defensible research requires wide-ranging, holistic, and deep consideration. It is often overseen by Research Ethics Committees, Institutional Research Boards or equivalents but not all organisations have these and where they do, their degree of independence from organisational priorities varies (perhaps leading to research that would cre...

The algorithmic recommendations used by digital platforms have significant impacts on users behaviours and preferences. For instance, Spotify and Tinder ground their platforms on recommendation algorithms that nudge users to either listen to specific songs or romantically match with specific users. Despite their powerful influence, there is little...

AI applications for music creation have been available since the last century but, until recently, their adoption has been limited to a small niche of researchers and engineers and their ontology limited to experimentation in computational creativity. The ongoing transformation of the music industry, the increasing injection of capital into AI-musi...

Due to the COVID-19 crisis, music communities who found themselves in social and physical isolation have been trying to find alternative solutions to keep some form of connection. Network performance is one of these solutions, one that is specifically aimed at enhancing communities’ connectivity beyond one’s intimate surroundings. In order for netw...

So far, NIME research has been mostly inward-looking, dedicated to divulging and studying our own work and having limited engagement with trends outside our community. Though musical instruments as cultural artefacts are inherently political, we have so far not sufficiently engaged with confronting these themes in our own research. In this paper we...

In NIME design, thorough attention has been devoted to feedback modalities, including auditory, visual and haptic feedback. How the performer executes the gestures to achieve a sound on an instrument, by contrast, appears to be less examined. Previous research showed that auditory imagery, or the ability to hear or recreate sounds in the mind even...

This paper presents ARCAA (Actors, Role, Context, Activity, Artefacts), a framework that supports designers in understanding the artefact ecology in the music performance scenario, in particular, allowing to frame the role of the different actors. The ARCAA framework is the result of the combination of two different areas of HCI: artefact ecology c...

We present the Magpick, a sensor-augmented plectrum for electric guitar, as a case study in designing research products which deliberately seek a creative tension between conceptual understanding and embodied practice. Building on the notion of "counterfactual artefacts", we seek to explore how the counterfactual might extend to embodied practice....

This manuscript aims to present the Magpick, a new technology for electric guitar, and to describe an autobiographical account of the early explorations of this new technology in a performance context.

This paper explores the process by which designers come to terms with an unfamiliar and ambiguous sensor material. Drawing on craft practice and material-driven interaction design, we developed a simple yet flexible sensor technology based on the movement of conductive elements within a magnetic field. Variations in materials and structure give ris...

Participatory artworks aim at creating an experience that is open to interpretation. We argue that such interpretations should not be just entirely predetermined by the cre-ators' expectations; rather, they should vary among audience members. We also argue that audience members that experienced the artwork could act as co-designers for the successi...

This paper introduces the Magpick, an augmented pick for electric guitar that uses electromagnetic induction to sense the motion of the pick with respect to the permanent magnets in the guitar pickup. The Magpick provides the guitarist with nuanced control of the sound that coexists with traditional plucking-hand technique. The paper presents three...

This paper presents a model to codesign interpretively flexible artefacts. We present the case study of Beatfield, a multimodal system that allows users to control audiovisual material by means of tangible interaction. The design of the system was coauthored by individuals with different background and interests to encourage a range of difference i...

Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evalua...

Extensive training with a musical instrument results in the automatization of the bodily operations needed to manipulate the instrument: the performer no longer has to consciously think about the instrument while playing. The ability of the performer to automate operations on the instrument is due to sensorimotor mechanisms that can predict changes...

The design of traditional musical instruments is a process of incremental refinement over many centuries of innovation. As a result, the shape and form of instruments are well established and recognised across cultures. Conversely, digital musical instruments (DMIs), being unconstrained by requirements of efficient acoustic sound production and er-...

An oft-cited aspiration of digital musical instrument (DMI) design is to create instruments, in the words of Wessel and Wright, with a 'low entry fee and no ceiling on virtuosity'. This is a difficult task to achieve: many new instruments are aimed at either the expert or amateur musician, with few instruments catering for both. There is often a ba...

The term 'NIME'-New Interfaces for Musical Expression-has come to signify both technical and cultural characteristics. Not all new musical instruments are NIMEs, and not all NIMEs are defined as such for the sole ephemeral condition of being new. So, what are the typical characteristics of NIMEs and what are their roles in performers' practice? Is...

This paper proposes the idea that a score for a Digital Musical In- strument (DMI) can be approached from a design perspective. In particular, we focus on the concepts of affordances and constraints. The paper describes the use of scores in a piece called XXV composed for Chimney, a DMI and Cello. Both the piece and the system are detailed with a f...

The purpose of this paper is to present Robinflock, an al-gorithmic system for automatic polyphonic music generation to be used with interactive systems targeted at children. The system allows real time interaction with the generated music. In particular, a number of parameters can be independently manipulated for each melody line. The design of th...

This paper presents a methodology for the study of audience perception of live performances, using a combined approach of post-hoc and real-time data. We conducted a study that queried audience enjoyment and their perception of error in digital musical instrument (DMI) performance. We collected quantitative and qualitative data from the participant...

Many digital musical instrument design frameworks have been proposed that are well suited for analysis and comparison. However, not all provide applicable design suggestions, especially where subtle, important details are concerned. Using traditional lutherie as a model, we conducted a series of interviews to explore how violin makers “go beyond th...

The purpose of this paper is to present Robinflock, an al-gorithmic system for automatic polyphonic music generation to be used with interactive systems targeted at children. The system allows real time interaction with the generated music. In particular, a number of parameters can be independently manipulated for each melody line. The design of th...

This paper investigates the involvement of preschoolers in the late stages of the design of Child Orchestra, a musical interactive playground for the kindergarten. The study presents a retrospective analysis of the design of Child Orchestra. The system was iteratively prototyped and developed following UCD methods and a design in use approach in a...

This half-day workshop will explore the craft of digital musical instrument design. Craft practice is central to the working process of both acoustic and digital instrument builders. Unlike the higher-level NIME design frameworks and tax-onomies that appear in the literature, craft knowledge is often personal, subjective, and occasionally difficult...

Every new edition of NIME brings dozens of new DMIs and the feeling that only a few of them will eventually break through. Previous work tried to address this issue with a deductive approach by formulating design frameworks; we addressed this issue with a inductive approach by elaborating on successes and failures of previous DMIs. We contacted 97...

Research on musical instrument design suggests that deliberate constraints can offer new creative experiences to the performer. At times, design constraints are physically embedded in the instrument to limit the interaction possibilities ; in other cases the constraints are given by delegating part of the control to the machine. In this paper we pr...

This position paper discusses toolkits for creating digital musical instruments. Musical interaction imposes stringent technical requirements on interactive systems, including high spatial and temporal precision and low latency. Social and community factors also play an important role in musical interface toolk-its, including design sharing and the...

This paper reflects on the dynamics and practices of building a maker community around a new hardware platform. We ex- amine the factors promoting the successful uptake of a maker platform from two perspectives: first, we investigate the tech- nical and user experience considerations that users identify as the most important. Second, we explore the...

When performing a piece of music, body, senses, and cognition are strictly connected to each other. This connection, however, is not always particularly evident. As a consequence, it is extremely important for musicians to be able to control their performance by relying on other sensorial modalities that complement the auditory cue. Sight, in parti...

The recent increase in interest for online multimedia streaming platforms has availed massive amounts of multimedia information that need to be indexed to be searchable and retrievable. User-centric implicit affective indexing employing emotion detection based on psycho-physiological signals, such as electrocardiogra-phy (ECG), galvanic skin respon...

Most of musicians can control their performance by relying on different sensorial modalities that complement the auditory cue. Vision, in particular, offers most instrumentalists an essential support: it helps them developing techniques, identifying errors, correcting expressiveness, and memorise complex passages. By contrast, when performing a pie...

This paper presents Beatfield, a musical installation that allows players to explore an audiovisual landscape by positioning tangible objects on an augmented game board. The underlying idea of the installation was the proposition of an artefact that could encourage heterogeneous interpretations. Beatfield had to offer a multitude of interpretations...

The last decades have witnessed a significant growth of interest in in- teractive systems that allow the audience to alter the auditory content of an art- work. Generally, the artwork itself is the primary goal of the artist, who arbitrarily maps audience behaviours into musical output following their personal aesthetic. However, given this subject...

This paper presents MINUET, a framework for musical interface design grounded in the experience of the player. MINUET aims to provide new perspectives on the design of musical interfaces, referred to as a general term that comprises digital musical instruments and interactive installations. The ultimate purpose is to reduce the complexity of the de...

In this paper, we reflect on the evaluation of The Music Room, an interactive installation that was designed with the aim of allowing everybody to experience musical creativity. Couples entering The Music Room can compose original piano music by moving throughout the space. The metaphor of emotion mediates the interaction between people's behaviour...

TwitterRadio is an interactive installation designed to explore the social world of Twitter through music. The idea behind this project is to access the musical domain to display information about the latest trends and news. The system automatically generates tonal compositions that are supposed to match the emotional contents of the tweets, as wel...

The purpose of this paper is to present Robin, an algorithmic composer specifically designed for interactive situations. Users can interact in real time with the algorithmic composition by means of control strategies based on emotions. This study aims at providing a system for automatic music generation to be applied to interactive systems for musi...

This paper presents The Music Room, an interactive installation where couples compose original music. The music is generated by Robin, an automatic composition system, according to relative distance between the users and the speed of their own movements. Proximity maps the pleasantness of music, while speed maps its intensity. The Music Room was ex...

This paper gives an overview of my PhD research that aims at contributing toward the definition of a class of interfaces for music creation that target non-musicians. In particular, I am focusing on the differences on design and evaluation procedures with respect to traditional interfaces for music creation that are usually intended to be used by m...

Current systems aiming at facilitating and extending musical expression typically rely on concepts that non-musicians can hardly comprehend. To attract such users, different interaction metaphors laying outside of the musical domain need to be considered. This study is a step towards the definition of a new class of interfaces for musical expressio...

In a multiple choice auditory experimental task, listeners had to discriminate walks over floors made of concrete, wood, gravel, or dried twigs. Sound stimuli were obtained by mixing temporal and spectral signal components, resulting in hybrid formulations of such materials. In this way we analyzed the saliency of the corresponding cues of time and...