Jon McCormack

Monash University (Australia) · Faculty of Information Technology

This paper presents a study on the use of a real-time music-to-image system as a mechanism to support and inspire musicians during their creative process. The system takes MIDI messages from a keyboard as input which are then interpreted and analysed using state-of-the-art generative AI models. Based on the perceived emotion and music structure, th...

This paper explores the design of an expressive visual instrument that embraces the unique movement style of a dancer living with physical disability. Through a collaboration between the dancer and an interaction designer/visual artist, the creative qualities of wearable devices for motion tracking are investigated, with emphasis on integrating the...

This paper presents the design and initial assessment of a novel device that uses generative AI to facilitate creative ideation, inspiration, and reflective thought. Inspired by magnetic poetry, which was originally designed to help overcome writer's block, the device allows participants to compose short poetic texts from a limited vocabulary by ph...

In this paper we present the design and development of the Transhuman Ansambl, a novel interactive singing-voice interface which senses its environment and responds to vocal input with vocalisations using human voice. Designed for live performance with a human performer and as a standalone sound installation, the ansambl consists of sixteen bespoke...

The development of remote sensing technologies, such as LiDAR and photogrammetry, accompanied by the exponential growth of easy-to-use 3D scanning applications and associated post-processing software, have made point cloud datasets accessible to a widening range of artists, designers, and creative technologists. Does this growing interest and incre...

The development of remote sensing technologies, such as Light Detection and Ranging (LiDAR) and photogrammetry, accompanied by the exponential growth of easy-to-use 3D scanning applications and associated post-processing software, have made point cloud datasets accessible to a widening range of artists, designers and creative technologists. Does th...

In creative design, where aesthetics play a crucial role in determining the quality of outcomes, there are often multiple worthwhile possibilities, rather than a single ``best'' design. This challenge is compounded in the use of computational generative systems, where the sheer number of potential outcomes can be overwhelming. This paper introduces...

In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for...

As we become increasingly entangled with digital technologies, the boundary between human and machine is progressively blurring. Adopting a performative, posthumanist perspective resolves this ambiguity by proposing that such boundaries are not predetermined, rather they are enacted within a certain material configuration. Using this approach, dubb...

In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for...

In this paper, we present a series of design methodologies which have emerged from the interaction of living (material) and non-living (machine) behaviours. ‘Bio Scaffolds’ explores a series of design tectonics that emerge from the co-creation between human, machine and natural intelligences. This research examines mycelium’s ability to biodegrade...

Human collaboration with systems within the Computational Creativity (CC) field is often restricted to shallow interactions, where the creative processes, of systems and humans alike, are carried out in isolation, without any (or little) intervention from the user, and without any discussion about how the unfolding decisions are taking place. Fruit...

Open AI's language model, GPT-3, has shown great potential for many NLP tasks, with applications in many different domains. In this work we carry out a first study on GPT-3's capability to communicate musical decisions through textual explanations when prompted with a textual representation of a piece of music. Enabling a dialogue in human-AI music...

In this paper we examine the concept of complexity as it applies to generative and evolutionary art and design. Complexity has many different, discipline specific definitions, such as complexity in physical systems (entropy), algorithmic measures of information complexity and the field of “complex systems”. We apply a series of different complexity...

Many creative generative design spaces contain multiple regions with individuals of high aesthetic value. Yet traditional evolutionary computing methods typically focus on optimisation, searching for the fittest individual in a population. In this paper we apply quality-diversity search methods to explore a creative generative system (an agent-base...

Many creative generative design spaces contain multiple regions with individuals of high aesthetic value. Yet traditional evolutionary computing methods typically focus on optimisation, searching for the fittest individual in a population. In this paper we apply quality-diversity search methods to explore a creative generative system (an agent-base...

In this paper we examine the concept of complexity as it applies to generative and evolutionary art and design. Complexity has many different, discipline specific definitions, such as complexity in physical systems (entropy), algorithmic measures of information complexity and the field of "complex systems". We apply a series of different complexity...

‘Bio Scaffolds’ explores a series of design tectonics that emerge from a co-creation between human, machine and natural intelligences. This research establishes an integral connection between form and materiality by enabling biological materials to become a co-creator within the design and fabrication process. In this research paper, we explore a h...

Point cloud datasets, the pertinent product of remote sensing technologies, such as LiDAR and photogrammetry, are increasingly being used as a new digital medium in art practice. This paper presents an overview of purposely selected and prominent artworks, emerging from traditional representational onto epistemologies, to identify currently prevail...

Design-especially of physical objects-can be understood as creative acts solving practical problems. In this paper we describe a biologically-inspired developmental model as the basis of a generative form-finding system. Using local interactions between cells in a two-dimensional environment, then capturing the state of the system at every time ste...

Design - especially of physical objects - can be understood as creative acts solving practical problems. In this paper we describe a biologically-inspired developmental model as the basis of a generative form-finding system. Using local interactions between cells in a two-dimensional environment, then capturing the state of the system at every time...

In this paper we examine the concept of complexity as it applies to generative art and design. Complexity has many different, discipline specific definitions, such as complexity in physical systems (entropy), algorithmic measures of information complexity and the field of “complex systems”. We apply a series of different complexity measures to thre...

The use of evolutionary methods in design and art is increasing in diversity and popularity. Approaches to using these methods for creative production typically focus either on optimisation or exploration. In this paper we introduce an evolutionary system for design that combines these two approaches, enabling users to explore landscapes of design...

In this paper we examine the concept of complexity as it applies to generative art and design. Complexity has many different, discipline specific definitions, such as complexity in physical systems (entropy), algorithmic measures of information complexity and the field of "complex systems". We apply a series of different complexity measures to thre...

Accurate evaluation of human aesthetic preferences represents a major challenge for creative evolutionary and generative systems research. Prior work has tended to focus on feature measures of the artefact, such as symmetry, complexity and coherence. However, research models from psychology suggest that human aesthetic experiences encapsulate facto...

This research examines the reflexive dimensions of cinematic virtual reality (CVR) storytelling. We created Anonymous, an interactive CVR piece that employs a reflexive storytelling method. This method is based on distancing effects and is used to elicit audience awareness and self-reflection about loneliness and death. To understand the audience’s...

This paper introduces a new form of real-time affective interface that engages the user in a process of conceptualisation of their emotional state. Inspired by Barrett's Theory of Constructed Emotion, `Mirror Ritual' aims to expand upon the user's accessible emotion concepts, and to ultimately provoke emotional reflection and regulation. The interf...

Mirror Ritual is an interactive installation that challenges the existing paradigms in our understanding of human emotion and machine perception. In contrast to prescriptive interfaces, the work's real-time affective interface engages the audience in the iterative conceptualisation of their emotional state through the use of affectively-charged mac...

Machines incorporating techniques from artificial intelligence and machine learning can work with human users on a moment-to-moment, real-time basis to generate creative outcomes, performances and artefacts. We define such systems collaborative, creative AI systems, and in this article, consider the theoretical and practical considerations needed f...

This paper describes practice-based research in the reflexive dimensions of cinematic VR, using alienation and disembod-iment as rhetorical devices. Cinematic VR currently focuses on immersive illusion rather than inducing self-awareness, but VR can also create reflexive contexts for eliciting self and social reflection. Anonymous is an interactive...

The generation of music that adapts dynamically to content and actions has an important role in building more immersive, memorable and emotive game experiences. To date, the development of adaptive music systems for video games is limited by both the nature of algorithms used for real-time music generation and the limited modelling of player action...

The generation of music that adapts dynamically to content and actions has an important role in building more immersive, memorable and emotive game experiences. To date, the development of adaptive music systems for video games is limited by both the nature of algorithms used for real-time music generation and the limited modelling of player action...

Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn't exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which af...

This paper examines five key questions surrounding computer generated art. Driven by the recent public auction of a work of “AI Art” we selectively summarise many decades of research and commentary around topics of autonomy, authenticity, authorship and intention in computer generated art, and use this research to answer contemporary questions ofte...

This paper examines five key questions surrounding computer generated art. Driven by the recent public auction of a work of `AI Art' we selectively summarise many decades of research and commentary around topics of autonomy, authenticity, authorship and intention in computer generated art, and use this research to answer contemporary questions ofte...

Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn't exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which af...

While visual cues are traditionally used for visual analytics, multimodal interaction technologies offer many new possibilities. This chapter explores the opportunities and challenges for developers and users to utilize and represent data through non-visual sensory channels to help them understand and interact with data. Users are able to experienc...

Computational music systems that afford improvised creative interaction in real time are often designed for a specific improviser and performance style. As such the field is diverse, fragmented and lacks a coherent framework. Through analysis of examples in the field, we identify key areas of concern in the design of new systems, which we use as ca...

Libraries are increasingly offering on-line digital access to their collections. However, traditional search-based interfaces are restrictive and do not encourage the user to explore the collection in the same way that a physical collection does. We present the Mitchell WordCloud, a novel on-line interface to the David Scott Mitchell collection of...

Improvisational interfaces are human-machine systems for improvising creatively with. To facilitate research in this field, we seek some general approaches to evaluating the quality of a particular interface, and the success of a given improvisation. Drawing on a broad array of improvisational practices across music, visual art and drama, we draw o...

Improvisational interfaces are human-machine systems for improvising creatively with. To facilitate research in this field, we seek some general approaches to evaluating the quality of a particular interface, and the success of a given improvisation. Drawing on a broad array of improvisational practices across music, visual art and drama, we draw o...

This paper describes the artwork Codeform, an interactive, evolutionary ecosystem of virtual “creatures” created by scanning museum visitor admission tickets at the Ars Electronica Center in Linz, Austria, in 2014. The paper first describes technical mechanisms used to realize the work and then discusses the issues of finding a genotypic representa...

Evolutionary Art typically involves a tradeoff between the size and flexibility of genotype space and its mapping to an expressive phenotype space. Ideally we would like a genotypic representation that is terse but expressive, that is, we want to maximise the useful variations the genotype is capable of expressing in phenotype space. Terseness is n...

In this paper the authors pose 10 questions they consider the most important for understanding generative computer art. For each question, the authors briefly discuss its implications and suggest how it might form the basis for further discussion.

In this paper we pose ten questions we consider the most important for understanding generative computer art. For each question, we briefly discuss the implications and suggest how it might form the basis for further discussion.

This paper addresses problems in computational creative discovery, either autonomous or in synergetic tandem with humans. A computer program generates output as a combination of base primitives whose interpretation must lie outside the program itself. Concepts of combinatoric and creative emergence are analysed in relation to creative outputs being...

This paper discusses issues in evolutionary art related to Art Theory and Aesthetics with a view to better understanding how they might contribute to both research and practice. Aesthetics is a term often used in evolutionary art, but is regularly used with conflicting or naïve understandings. A selective history of evolutionary art as art is provi...

In this article we argue that a framework for the description, analysis and comparison of generative artworks is needed. Existing ideas from kinetic art and other domains in which process description is prominent are shown to be inadequate. Therefore, we propose a new framework that meets this need and facilitates the long-term aim of constructing...

We use a new measure of creativity as a guide in an interactive evolutionary art task and tie the results to natural language usage of the term “creative.” Following previous work, we explore a tractable definition of creativity, one emphasizing the novelty of systems, and its addition to an interactive application. We next introduce a generative e...

This chapter is an edited conversation on the topic of computational evaluation of artistic artefacts. The participants were Harold Cohen, Frieder Nake, David Brown, Jon McCormack, Paul Brown and Philip Galanter. It began at the Dagstuhl seminar on computers and creativity, held in Germany in 2009 and continued over a period of several months via e...

In this paper we introduce and reflect on current research questions regarding computers and creativity. Creativity is an enigmatic yet widely discussed phenomena. With the now widespread adoption of computers and information technologies, the nature of creativity and how we think about it has changed significantly. We argue for a shift in thinking...

This final chapter proposes a number of questions that we think are important for future research in relation to computers and creativity. Many of these questions have emerged in one form or another in the preceding chapters and are divided into four categories as follows: how computers can enhance human creativity; whether computer art can ever be...

This paper investigates the performance and suitability of evolutionary algorithms for music composition by enhancing representation schemes. First, we argue that genetic programming (GP) is well suited to capture higher order musical structures due to its hierarchical representation. Representational enhancements are proposed on the standard GP tr...

This paper examines how models from ecology and evolutionary multi-agent systems can be used for creative applications. We outline a minimal, general specification for an ecosystem to be applied to the production of creative artefacts, based on spaces, materials, features and actions. Using this specification, we consider how basic evolutionary pri...

In this paper, we explore a generative art system de- signed to promote the creation of a diverse range of aesthetically pleasing images. We introduce our system, EvoEco, an agent- based pixel-level means of generating images based on artificial ecosystems. This art system is driven by interactive evolutionary computation, and further augmented usi...

One of the goals of artificial life in the arts is to develop systems that exhibit creativity. We argue that creativity {it per se} is a confusing goal for artificial life systems because of the complexity of the relationship between the system, its designers and users, and the creative domain. We analyse this confusion in terms of factors affectin...

This paper describes new research into the use of evolutionary ecosystem models as tools for creative artistic discovery. We give a brief history of the use of artificial evolution in the arts, and discuss difficulties with interactive and optimisation-based approaches to creativity. We present an approach based on evolutionary ecosystemic models....

In this paper, we introduce a new image database, consisting of examples of artists' work. Successful classification of this database suggests the capacity to automatically recognize an artist's aesthetic style. We utilize the notion of Transform-based Evolvable Features as a means of evolving features on the space, these features are then evaluate...

This paper describes modelling methods based on biological development for use in computer graphics applications, specifically the automated growth and development of complex organic shapes that are difficult to model directly. We examine previous approaches, including grammar-based methods, embedded systems and cellular models. Each system can be...

We address the question of how processes from evolution-ary biological ecosystems can be abstracted and beneficially applied in creative domains. Evolution is a process capa-ble of generating appropriate (fit) novelty in biological sys-tems, so it is interesting to ask if it can do so in other, non-biological systems. Past approaches have focused o...

We explore a new definition of creativity — one which emphasizes the statistical capacity of a system to generate previously unseen patterns — and discuss motivations for this perspective in the context of machine learning. We show the definition to be computationally tractable, and apply it to the domain of generative art, utilizing a collection o...

Throughout the short history of interactive digital music, there have been frequent calls for a new language of interaction that incorporates and acknowledges the unique capabilities of the computational medium. In this paper we suggest that a conceptualisation of possible modes of performance–time interaction can only be sensibly approached in lig...

This paper advances new methods for ecosystemic approaches to evolutionary music and art. We explore the biological concept of the niche and its role in evolutionary dynamics, applying it to creative com- putational systems. Using the process of niche construction organisms are able to change and adapt their environment, and potentially that of oth...

This chapter describes a novel type of artistic artificial life software environment. Agents that have the ability to make and listen to sound populate a synthetic world. An evolvable, rule-based classifier system drives agent behavior. Agents compete for limited resources in a virtual environment that is influenced by the presence and movement of...

This report summarises the discussion and experimental work produced by the authors at the 2009 symposium Computational Creativity: An Interdisciplinary Approach, Dagstuhl Leibniz-Zentrum für Informatik. It outlines the motivation for using computational techniques to stimulate human creativity, briefly summarising its historical context and predec...

This paper describes a new generative software system for music composition. A number of state-based, musical agents traverse a user-created graph. The graph consists of nodes (representing events), connected by edges, with the time between events determined by the physical length of the connecting edge. As the agents encounter nodes they generate...

Artificial ecosystems extend traditional evolutionary approaches in generative art in several unique and attractive ways. However some of these traits also make them difficult to work with in a creative context. This paper addresses the issue by adapting predictive modelling tools from theoretical ecology. Inspired by the ecological concept of spec...

Evolutionary computation (EC) techniques are e?cient, nature-inspired pl- ning and optimization methods based on the principles of natural evolution and genetics. Due to their e?ciency and simple underlying principles, these me- ods can be used in the context of problem solving, optimization, and machine learning. A large and continuously increasin...

Evolution is one of the most interesting and creative processes we currently understand, so it should come as no surprise that artists and designers are embracing the use of evolution in problems of artistic creativity. The material in this section illustrates the diversity of approaches being used by artists and designers in relation to evolution...

This chapter examines the possibilities and challenges that lie ahead for evolutionary music and art. Evolutionary computing methods have enabled new modes of creative expression in the art made by humans. One day, it may be possible for computers to make art autonomously. The idea of machines making art leads to the question: what do we mean by ‘m...

The problem confronting any contemporary artist wishing to use technology is in the relationship between algorithmic and creative processes. This relationship is traditionally a conflicting one, with the artist trying to bend and adapt to the rigour and exactness of the computational process, while aspiring for an unbounded freedom of expression. S...

ABSTRACT This paper discusses the concept of an articial,ecosystem for use in machine-assisted creative discovery. Properties and processes from natural ecosystems are abstracted and applied to the design of creative systems, in a similar way that evolutionary computing,methods use the metaphor of Darwinian evolution to solve problems in search and...

Art, it was once said, is anything you can get away with. So it is not surprising that evolutionary approaches to music and art research are challenging our notions of what is classified as "Art" and who is the "creator" of this work. The relatively new field of Evolutionary Music and Art (EMA) falls within the spectrum of Evolutionary Computing. I...