New Atlantis: Audio Experimentation in a Shared Online World

Full text

HAL Id: hal-01791975
https://hal.archives-ouvertes.fr/hal-01791975
Submitted on 22 Jun 2018
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
New Atlantis: Audio Experimentation in a Shared
Online World
Peter Sinclair, Roland Cahen, Jonathan Tanant, Peter Gena
To cite this version:
Peter Sinclair, Roland Cahen, Jonathan Tanant, Peter Gena. New Atlantis: Audio Experimentation
in a Shared Online World. Mitsuko Aramaki, Richard Kronland-Martinet, Sølvi Ystad. Bridging
People and Sound. 12th International Symposium, CMMR 2016, São Paulo, Brazil, July 5–8, 2016,
Revised Selected Papers, 10525, Springer, pp.229-246, 2017, Lecture Notes in Computer Science, 978-
3-319-67738-5. <10.1007/978-3-319-67738-5_14>. <hal-01791975>

New Atlantis: Audio Experimentation in A shared
Online World
Pete Sinclair 1, Roland Cahen2, Jonathan Tanant3 and Peter Gena4
1 Ecole Superieur d’Art d’Aix. Locus Sonus Research Unity. Aix-En-Provence, France
2 Ecole Nationale Superiere de Creation Industrielle. (ENSCI les ateliers), Paris France.
3 Independant Software Engineer, Jon Lab. Tilly, France.
4 School of the Arts Institute Chicago. (SAIC) Chicago, USA.
peter.sinclair@ecole-art-aix.fr
roland.cahen@ensci.com
jonathan@free.fr
pgena@artic.edu
Abstract. Computer games and virtual worlds are "traditionally" visually
orientated, and their audio dimension often secondary. In this paper we will
describe New Atlantis a virtual world that aims to put sound first. We will
describe the motivation, the history and the development of this Franco-
American project and the serendipitous use made of the distance between
partner structures. We explain the overall architecture of the world and discuss
the reasons for certain key structural choices. New Atlantis' first aim is to
provide a platform for audio-graphic design and practice, for students as well as
artists and researchers, engaged in higher education art or media curricula. We
describe the integration of student’s productions through workshops and
exchanges and discuss and the first public presentations of NA that took place
from January 2016. Finally we will unfold perspectives for future research and
the further uses of New Atlantis.
Keywords: audiographic creation, audio for virtual environments, sound
spatialisation, networked music.
1 Introduction
New Atlantis is a shared (multi-user) online virtual world dedicated to audio
experimentation and practice. Unlike most online worlds where image is the primary
concern, in NA sound comes first. NA provides a context for new-media students to
showcase research projects that explore the relationship between sound, virtual 3D
image and interactivity. It offers a pedagogical platform for audiographic animation,
real-time sound synthesis, object sonification and acoustic simulation. It is a place to
organize virtual sound installations, online concerts, Soundwalks and other
audiovisual art experiences.

2 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
The name New Atlantis comes from the title of an unfinished 1628 utopian novel by
philosopher Francis Bacon [1], which describes a legendary island somewhere in the
ocean, doted with extraordinary audio phenomena that might be considered as
premonitory of today’s electronic and digital audio techniques. We have adopted
some of Bacon’s ideas and nomenclature to create classes for the virtual world such
as “Sound Houses”, “Sound Pipes”, “Trunks” and “Helps”. In NA all elements are
intended to have audio qualities: spaces resonate, surfaces reflect and collisions
activate the multiple sounds of the objects involved. A collection of purpose built
scripts implement low level sound synthesis and multiple parameter interactivity,
enabling the creation of complex sound sources and environments linked to animation
or navigation in the visual scene.
NA can be accessed via a web viewer or as a standalone application. It is organized as
“spaces” that can be accessed independently but that share the same basic principles
of navigation and specific scripts. Multi-user, it can be shared by several players at
the same time making it suitable for group playing in both the gaming and the musical
sense of the word. Every registered user can create and host individual or shared
spaces which he or she can decide to make persistent or not. At the time of writing,
we are working on a limited number of public “spaces” that contain multiple “Sound
Houses” (architectural elements with specific acoustics) and other sound objects.
These can be visited by navigating through the scene or created within the scene. The
audio “mix” of these different sources varies with distance, so placing and navigating
between sound objects can become a musical experience. In public spaces, players
can interact with one another and with shared objects potentially playing together.
New Atlantis project is not only about creating a multi user virtual universe, but also
about making it together while learning. It is experimental, creative and educational.
New opportunities for further development in NA are created through the
organization of workshops, courses or events that group art students in different
locations, working together at a distance. It is a way to encourage ubiquitous working
groups of students to share immaterial and non-local (international) art projects. Most
sound and music education schemes tend to be oriented towards established
disciplines. NA on the other hand encourages experimental design projects and the
exploration of new or emerging creative fields. Francis Bacon’s New Atlantis
proposed a model for the role of science and art in society that placed education at the
heart of culture. Our project emphasizes discovery, cultural exchange,
experimentation, learning and furthering of knowledge in an educational and creative
environment.

New Atlantis 3
Fig 1. Students showcasing their projects in New Atlantis. ENSCI les ateliers, September 2015.
2 Context and History
“We have also sound-houses, where we practise and demonstrate all sounds
and their generation. We have harmony which you have not, of quarter-
sounds and lesser slides of sounds. Divers instruments of music likewise to
you unknown, some sweeter than any you have; with bells and rings that are
dainty and sweet. We represent small sounds as great and deep, likewise
great sounds extenuate and sharp; we make divers tremblings and warblings
of sounds, which in their original are entire. We represent and imitate all
articulate sounds and letters, and the voices and notes of beasts and birds. We
have certain helps which, set to the ear, do further the hearing greatly; we
have also divers strange and artificial echoes, reflecting the voice many
times, and, as it were, tossing it; and some that give back the voice louder
than it came, some shriller and some deeper; yea, some rendering the voice,
differing in the letters or articulate sound from that they receive. We have all
means to convey sounds in trunks and pipes, in strange lines and distances.”
[1]
The origins of the NA project go back to 2005 when the Locus Sonus, ESA-Aix
(Ecole Superieur d’Art d’Aix-En-Provence) and SAIC (School of the Art Institute of
Chicago) were awarded FACE[2] funding for an academic and research exchange
program. The original impulse occurred through collaboration between teams of the
3d and sound departments of the partner establishments, which led to the observation
that there was scope for research into the area of audio in games and other virtual
environments.
That NA refers to a Utopian model can be interpreted in several different ways.
Firstly as the above citation demonstrates, the original text by Francis Bacon

4 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
describes an island territory that was home to numerous extraordinary audio
phenomena. Beyond this the novel predicts the principles of the contemporary
research university as seen in the description of “Salomon's House” which promotes
international exchange: “For the several employments and offices of our fellows, we
have twelve that sail into foreign countries ... who bring us the books and abstracts,
and patterns of experiments of all other parts” [1] and trans-disciplinary research:
“We have three that collect the experiments of all mechanical arts, and also of liberal
sciences, and also of practices which are not brought into arts”[1]. These ideas,
combined with the fact that we are indeed engaged in creating a utopian world that
might be considered as situated (albeit in our imagination) somewhere in the ocean –
between Europe and America – combine to make NA a suitable reference.
Since the ESA-Aix/SAIC partnership was separated by the Atlantic Ocean, we rapidly
adopted networked solutions for our collaborations: video conferencing and remote
desktop were used to exchange lessons and conferences and an interconnected 3d
cave were set up with an interface in both Aix and Chicago. The first experiments in
3d audio-graphy took place in Second Life[3], using Pure Data[4] as an audio engine.
The process involved sending html commands from second life to an external server
that did the audio synthesis and streamed the result back to second life; the project
was presented in 2009 at the “Second Nature” festival in Aix-En-Provence[5]. This
system worked well enough to convince us that it was worthwhile pursuing the
development of sophisticated audio features for virtual environments, however, it was
difficult to implement and the delay due to streaming was problematic.
The decision was made to build our own multi user world using Panda 3d[6] with
Pure Data[4] bundled as an audio engine. Ecole Nationale Superieure de Creation
Industrielle (ENSCI, les ateliers) became associated with the project at this point. This
first version of NA was developed during multiple workshops that took place in
Chicago and Aix en Provence between 2007 and 2011. The project involved a
relatively complex path finding system used to calculate acoustics and custom-built
client server software [7]. Although a working version was tested successfully during
a workshop at ENSAB in 2011[8], it was decided to abandon the development based
on Panda/Pd in favor of Unity3d[9], a more recent and efficient platform offering
greater scope for audio programming and possessing built in networking capabilities.
3 New Atlantis Project Aims
There are multiple aims associated with the NA project: as mentioned above the
project emerges from an international exchange program and one of its first ambitions
is to provide a platform for international academic, artistic, cultural and scientific
exchange. The more specific aim is to further research into audio for virtual
environments with the precise goals of providing an educational tool for students and
a synchronized platform for remote music and sound art practices.

New Atlantis 5
3.1 Research Into Audio for Virtual and networked Environments,
background.
The history of audio development in game environments is relatively short and, at
least when this project was initiated, somewhat lacking in substance. Arguably, this
might be put down to competition between audio and visual requirements in terms of
processing power on personal computers or game boxes. If in recent years companies
such as AudioGaming[10] that are specialized in audio for game environments have
started to appear, for the essential they provide sound design services for the
commercial game market, rather than considering the virtual world as a possible audio
interface. Therefore the historical origins of this project might be considered from
several other angles. One approach is that of visual interfaces for musical
compositions such as UPIC[11] originally developed by Iannis Xenakis or navigable
scores such as Fontana Mix 1958 by John Cage[12]. Distant listening, remote
performance and other streamed and networked art forms are another thread that has
been largely investigated by Locus Sonus[13]. Experiments with remote musical
presence started as early as 1992 when during an event organized by Michel Redolfi;
Jean Claude Risset and Terry Riley in Nice played with David Rosenboom and
Morton Subotnick in Los Angeles using Disklaviers and a satellite connection[14]. In
1967, the late Maryanne Amacher conceived of and produced City Links, in Buffalo,
NY. The 28-hour performance took live microphone feeds from five different
locations in the city. Victor Grauer and Max Neuhaus were involved in the Buffalo
production and Neuhaus subsequently did similar work that came to be known as
“telematic performance”. The rapidly developing discipline of sonification[15] is
equally useful when reflecting on the sound of virtual objects. In effect the different
members of the research team have pursued these different lines of investigation over
the past decades.
We should not however ignore early forays into audio-graphic creation an example
being Ivan Chabanaud1 and co-author Roland Cahen’s Icarus [16]. When this virtual
reality project was started in 1995, sound synchronization techniques where
cumbersome: a Silicon graphics machine rendering visual objects was connected to
Opcode Max 3 (MIDI only) using an external MIDI expander and the IRCAM’s
spatialisation system. More recent audio-graphic projects that R. Cahen has been
involved in include BANDONEON[17], PHASE[18], ENIGMES[19], and
TOPOPHONIE[20].
Other recent research initiatives focusing on shared virtual environments such as
UDKOSC[21] use the OSC protocol to associate external audio engines such as Pd or
Supercollider with a virtual environment. Although the original version of NA
1 Ivan Chabanaud, was a fascinating french digital artist who gave wings to his
audience as they became icarus for the space of a visit. Our friend Ivan died the 4th
september 2015.

6 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
followed this line of investigation we have switched to the using the audio engine
incorporated in Unity for the sake of simplicity and versatility on the user side (see
section 6.). The Avatar Orchestra Metaverse, formed in 2007, is another Interesting
experimental project led by American Sound Artist, Pauline Oliveros: “The Avatar
Orchestra Metaverse is a global collaboration of composers, artists and musicians that
approaches the virtual reality platform Second Life as an instrument itself”[22].
However this approach is very different to that adopted for New Atlantis in the sense
that it is dependent on the resources of the existing virtual world Second Life. Proteus
by Ed Key and David Kanaga [23] is a rare example of a virtual world where the
combination of low-resolution graphics and generative composition create an ever-
evolving landscape/soundscape, offering a delicate interaction between the audio and
visual components.
3.2 Fields of Investigation
Interactive sound and music with virtual objects
Audiographic design is a multimodal approach that consists of coordinating graphical
form and behavior with auditory events in the design of virtual objects. These objects,
although they are visual images, can incorporate physical simulation so they can
interact with us and between themselves. While artists and designers often have a
limited culture of multimodality, creating audiographic objects or compositions
requires interconnections between various domains of expertise such as sound design,
graphic design, 3D animation, real time interaction and coding. New Atlantis has been
conceived to provide an adequate platform for this type of training. Designing
multimodal interactions and implementing them in a virtual world is more complex
than creating simple visual representations or video. It obliges the author to resolve
more design issues and to give objects deeper consistency, bringing them close to a
physical reality, even if they remain immaterial. At a certain point, the simulated
object becomes the object in its own right and the notion of “virtual” is modified.
Arguably, virtual objects are somehow non-material objects that incarnate their own
reality. This has long been the case in music as well as with other abstract artistic
activities, when an initial manipulation of representation ceases to be the finality.
Since the audiographic relationship is constructed, it can also be fictitious. An
object’s sound can be modified to become quite different from the “real life” original,
while paradoxically appearing very real or possibly more than real (hyperreal). An
early example of this phenomenon can be found in Jacques Tati’s film Mon Oncle
where “retouched” audio recordings focus attention unnaturally on specific banal
visual objects [24]. We consider that this bending, schematizing or detournement of
the relation between the visual object and its associated sound is a fruitful terrain for
investigation. Virtual composition also allows us to interact with sound objects using
a variety of expressive devices such as physical modeling, avatar form and behavior,
fictional scenarios and gameplay, opening a multitude of new forms of narration.

New Atlantis 7
We are currently developing new tools for New Atlantis in collaboration with FRE
PRISM that will incorporate LMA’s intuitive synthesizer control system. The system
is based on a principal of control by metaphorical description of material qualities and
physical actions. It is thus highly suitable for the creation of virtual objects in
particular since the simulated object does not necessarily require a real-world model.
Spatialization, Sound Navigation and Virtual Acoustics
Sound navigation consists of browsing through different sound sources placed in a
spatialized sound scene, thereby composing a musical form from the mix produced by
movements relative to these sources. It is an artistic transposition of our sound
experience in the physical world [25]. In virtual worlds, sounds are mainly spatialized
through orientation and distance. The simple act of navigation allows us to create
spatial mixes or canons as well as subject and object motion sequences. Spatial
acoustics are another important part of our audio perception and indeed they
participate in our natural audio interactions as we activate reverberant spaces through
our own actions such as footsteps or vocalizations [26]. This praxeology[27] of sound
space can also be developed in virtual environments and the juxtaposition of different
acoustic responses as we navigate while generating sounds or listening to sounds
generated by other players, provides an original approach to real-time audio signal
processing. Short simple sound sources such as clicks or collision sounds can be used
to activate and compare virtual acoustics and continuous sound sources can be
multiplied to create complex harmonics. Such indeterminate forms of navigation,
inspired by SoundWalking or other Soundscape related activities (see The Tuning of
the World by R.Murray Schafer [28]), are alternatives too more permanently authored
audiographic forms.
3.3 A Synchronized Platform for Remote Music and Sound Art Practices
Networked Playing and Performance.
In the case of NA “playing” may mean “gameplay” i.e. in general video games
parlance “the specific way in which players interact with a game”[29], or it can
equally be used to designate musical playing. In the preceding section, we approached
NA as a compositional tool but it can also be considered as an audio “sandpit”, as a
shared instrument, or as a stage for public performances. The fact that these activities
are synchronized online means that NA also opens a line of investigation into
remotely shared musical and sound art practices. This in turn invites speculation as to
the possibility of a creating a new paradigm in musical practice and distribution, a
question that we wish to investigate in future research.
Networked music performances have existed since the beginning of the Internet and
even before through the use of telephone and radio (see examples in section 3.1.).
Until now however, such activities have, for the most part, focused on mixing
networked (streamed, midi or OSC controlled sounds) with locally sounding
instruments an example is Jean Claude Risset’s use of the Disc Clavier, another is

8 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
French musician Eric M’s practice of using Locus Sonus’ “open microphone” live
audio streams [30] to introduce indeterminate sounds into his DJ sets. With NA
however, the virtual sound space is distributed, and co-users share events, functions,
triggers, controllers and audio recordings (made on the fly) through a dedicated
server. The fact that all connected users share the “current state” of a space (see
technical explications in section 3) means that as long as an instance of a space is
active, they experience the same evolving situation from wherever they are (within
the limits of the network’s capabilities). There is no distinction per se between
musician and audience, this allows for inventive approaches to shared performance
(for example, if one player introduces a sound source object or makes a recording,
other players can hear it, move it or modify its parameters).
3.4 An Educational Tool
Locus Sonus[31] is a research unit based at ESA - Aix and funded by the French
ministry for culture. Its main concern is practice-based research into sound art. Locus
Sonus’ research investigates virtual and acoustic sound space in a permanent
exploration of “new auditoriums”[13]. ENSCI les Ateliers, have worked on numerous
sound design projects including sounding objects, industrial sound design,
audiography, sound spatialization, auditory interfaces but also sound art, electronic
music, radio broadcasts, soundtracks for films, and various research projects [32]. The
School of the Art Institute of Chicago’s Art and Technology department teach virtual
and augmented reality, gaming, 3d modeling, computer imaging, live performance
audio and immersive audio[33]. Between them, these three structures provide the
multidisciplinary expertise, creative and educational context that the NA project is
founded on. The recent addition of the FRE PRISM as a new partner adds elements of
fundamental research in the domain of digital audio to this rich consortium.
Teaching Sound in Art and Design Schools
Over the last 20 years sound education in France has progressively shifted from the
conservatoires (music academies) to schools of art and design. The main reason for
this is that, apart some rare exceptions, conservatoires were designed to teach solely
classical music theory and instrumental practice and have therefore experienced
difficulty in adapting to the challenges of XXth and XXIst century music and sound
art production. As a consequence sound arts, sound design, some performing arts and
sound for multimedia are now taught in many art and design schools including ENSCI
and ESA-Aix. Most of the students concerned are consequently visual artists and
rarely trained musicians. On the one hand this is a drawback, because they have a
limited sonic culture, and on the other it is an advantage because they don’t have
preconceived models or cultural bias. Similarly, unlike traditional composers who
arguably consider music as a result of relationships between pitch and rhythm, artists
tend to use sound as material. Since their culture is primarily visual, these students

New Atlantis 9
have a tendency to create sounds in relation to images and to be concerned with
multimedia and interaction.
Working and Creating Online in Virtual Workshops, Complementary Skills &
Delocalized Working Groups
Creating and playing music together and by extension audiographic composition and
performance are a great way to link people; this holds for artists, designers and
students as well as the game audience. NA “bridges people with and through sound”
since it is a group project within which participants with different skills and different
degrees of expertise can come together and cooperate. Thus although the permanent
team of NA is constituted of experienced developers, researchers, artists and teachers,
this team has been completed from the outset of the project by successive generations
of post graduate students. The architecture of NA has been designed in such a way as
to encourage a variety of artistic approaches and methodologies, a creative participant
can easily build a new and unique space (or level) however different propositions can
also be easily combined into a single space. This method of creating content is
gradually enlarging and enriching the scope of the project through a process of
experimentation, feedback and exchange. On another level, as described above,
creating content for the world is also be a challenging pedagogical exercise. Students
are encouraged to work in teams pooling skill sets and sharing experience. The NA
team is spread over several geographic locations, rather than this becoming a
handicap it has turned into a fundamental characteristic of the project. NA is
progressively becoming the place where we meet to work and remote collaboration is
in the process of becoming second nature.
4 Architecture and Development
The particular aims of New Atlantis, described above, imply that certain decisions
have been made regarding the architecture and other parameters of the world. To give
some examples: the world needs to be shared online, it should be easy to use by a
wide range of players (including those creating content) and audio performance –
including synchronization between players – is paramount.
4.1 System Architecture
New Atlantis consists of three main software components:
The first is a MySQL database / PHP backend. The second is a standalone app (Mac
OS X/Windows), named the Viewer, which allows the player to navigate the world,
host multiplayer sessions, manage his account (spaces, assets…) and upload content.
The third (optional) component is the SDK, which allows the advanced participant to
build a viewer and to create components and contents using Unity3D authoring
software.

10 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
Fig 2. System architecture overview.
Entities and Related Concepts
A Space is an independent New Atlantis world. It is not necessarily related to the
Bacon Sound Houses concept: several Sound Houses could exist in one Space, or one
Sound House could be split in several Spaces. The absolute rule is that one Space is
totally independent: nothing communicates with, exits to or enters from the outside.
An Object is a composite-audio-graphic-3d-interactive object created by a participant
(designer/artist/developer) in Unity and uploaded to a Space in the form of an “Asset
Bundle”. Objects have qualities (in particular audio capabilities) and consist of a data
packages with associated states. These include the updating position, orientation, and
other related parameters in the currently activated space. When not in a space, an
object is referred to as an asset.
A player (user or visitor) is a human user of the New Atlantis platform. The system
holds parameters related to the players, such as email, login, password and a list of
assets.
The viewer is the multi-purpose application that is the main tool a player has to use to
visit and navigate a space.
4.2 Synchronization and Data Persistence
As a shared audio world, New Atlantis has stringent needs when it comes to
synchronization since, as described above, we wish players to be able to play together
musically. This requires that what each player hears on different connected computers
is as close as is possible to the same thing.
New Atlantis
standalone client application
(standalone, client or server)
New Atlantis
web server
Unity3d engine
New Atlantis SDK (C# mono scripting)
PHP / MySQL scripting
MySQL database
New Atlantis web service / backend
networking rendering audio inputs
Unity master server and facilitator
objectsusers spaces assets
Network :
LAN
internet
Display :
Screens
Projectors
VR
Audio output :
Stereo
5.1
7.1
Controllers :
Keyboard
mouse
gamepad
Audio
acoustics
Audio
components
New Atlantis assets (Asset Bundles)
and
New Atlantis viewer
Master
server
Navigation Other
components
Tools

New Atlantis 11
Fig 3. Synchronization and data persistence architecture overview.
A MySQL database / PHP backend holds the main state of objects and a web service
allow the access to this database, including creation, modification and deleting of
objects. Any player is able to host a session on his/her personal computer and accept
other players running the same viewer application, from virtually anywhere in the
world (within the limits of firewall and network structures). This is implemented
using Unity’s built-in advanced network features (NAT punchthrough, Master server,
Facilitator server) – so instead of running a dedicated server, we decided to take this
simpler and more flexible approach where any user is able to host a New Atlantis
session with a chosen Space (this method is used by most multiplayer video games).
We have had to define and implement the way Unity’s network system manages
synchronization, when several players are connected together in a session. We
decided to use an authoritative server scheme, meaning that the server runs the entire
simulation and sends objects states to the clients that are running the scene without
the logic (solely the rendering). The only exception to this is the avatar position,
which is not sent by the server but generated by each client to guarantee good local
fluidity (to avoid lag in the avatar’s own position). The number of objects that need to
be synchronized has a direct effect on performance and we tried several variations on
“depths” of synchronization, from every single object to only a few main ones. There
is inevitably a compromise that has to be made between performance and accuracy in
simultaneity between clients (largely dependent on bandwidth). We should insist here
on the special needs of working with sound: principally good temporal
synchronization. To sum up, in the viewer application, two types of synchronization
occur:
1. A synchronization of the objects contained in a given space: the viewer
app connects to the web service and downloads an XML manifest (list) of
all objects in the space with their initial state (position, orientation, name).
Each object is an Asset Bundle that is then downloaded by the viewer and
held in cache for future retrieval.

12 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
2. A client/server synchronization with Unity Network engine:
synchronization of avatars positions, created objects, audio recorded
during playing... The Unity Network engine implementation works with
TCP/IP and UDP/IP messages sending with a proprietary format
including synchronized parameters serialization. We have had to
implement RPCs (Remote Procedure Calls) functions to handle special
cases and audio-graphic synchronization.
4.3 Content Creation: Tradeoff Between Simplicity and Features
As described in section 2, NA has several levels of usage. Technically, the system
needs to be used by a wide range of players with skills that range between basic
gaming and professional design and programming. Advanced players, such as
students in workshops, can use the NA SDK package in Unity, while a player without
such skills is still be able to run the viewer app and access the spaces without doing
any programming. It should be mentioned that some of the students during the
workshops and throughout the duration of project development have even
programmed substantial improvements and add-ons to the platform.
We have identified the following users:
1. Simple player/visitor – visits and navigates a space.
2. Advanced player/visitor – creates spaces with existing assets.
3. Builder – builds new assets to be used and shared.
4. Core developer – creates new standard NA components.
Unity is used both as a content creation platform and as the game engine. This means
that while it is tempting to allow the builder to use Unity’s authoring tool with as few
restrictions as possible, it is necessary to forbid practices that could lead to issues,
such as bad synchronization or malfunctioning components.
The need to standardize accepted scripts
For security and design reasons, builder created scripts cannot be included in Asset
Bundles. User-created Objects can reference existing scripts, but a new script (at the
time of writing), needs to be included in the viewer to be available in the system. A
workaround would be to build a .NET assembly using Mono Develop or Microsoft
Visual Studio and use .NET reflection to make it available at runtime. However this
is still in development and is not a perfect solution since it would also enable
potentially unlimited access to scripting with the implied security problems.
Performance considerations
Resources are a major concern when it comes to user-generated content: a player
could potentially build a highly complex object that is too “greedy” with the resources

New Atlantis 13
of visiting computers (including video rendering performance, download time, CPU
processing and memory). We have decided to manage this in two ways. Firstly, by
defining “good practice” guidelines, these include: limiting the number of
simultaneously playing sound sources; reducing the number of physical collisions and
rigid bodies; reducing the size of audio clips; avoiding triggering too many audio
sources too frequently; being careful with the use of custom DSP processing. In
addition to these guidelines, we have limited the Asset Bundle upload size to a
maximum of 7MB. The second approach is via the implementation of active
optimization schemes, such as dynamic audio sources management as well as
dynamic components (distant audio sources and objects in general are deactivated).
4.4 The Viewer App and the Standard Tools and Navigation
As we aim for high coherency between spaces and the player’s experience, we
decided to introduce some standard concepts.
1. Navigation: after a few early discussions and tests concerning a user-
definable avatar and navigation system, we finally decided to provide a
standard navigation system, that includes a few cameras and standard
controls (with keyboard and mouse or with a dedicated gamepad).
2. Interactions and tools: we designed all interactions as “tools”, that are
selectable in the viewer by the player. A large number of these tools have
been suggested many of which concern variations on modes of audio
interaction (rubbing or dragging as well as simply colliding). At the time of
writing roughly half of these ideas are implemented including sound-playing
tools, an object thrower, physical interactions, flashlight, sunlight, trunk
creation (audio recorder)...
Audio Components
We have built a library of audio components to be used on audiographic objects in
New Atlantis, these include:
1. Audio synthesis: noise generation, oscillators, FM synthesis, loopers,
wave terrain synthesis...
2. Audio triggering: play/stop an audio source based on specific events, such
as a collision, a volume intersection (named a trigger), a threshold
distance with the listener…
3. Audio parameter modulation: pitch, volume, panning...
4. Experimental audio filtering: implementation of Bacon’s audio helps.
5. Audio recording: with content synchronization over the network (the
“trunk”).
This list of audio components continues to augment with the ultimate intention of
providing a comprehensive toolbox for DSP interaction and sound generation.

14 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
Other standard components
Because in New Atlantis designers can only use built-in scripts, it has also been
necessary to provide a wide range of other standard components, including
animations, visuals, move and rotate, teleport, GUI, particles triggering, physics…
4.5 Audio Spatialisation and Virtual Acoustics
A standard approach to audio spatialisation divides perception into three classes: the
direct path, the early reflections and the late reverberation. As most 3D game engines
today, Unity has a simple 3D audio sources management, which we used to spatialize
an audio source’s direct path. Unity also has a concept of Reverb zones with
dedicated parameters (such as presets, level, reverb time....), but the Reverb effect is
applied to the final mix and is triggered by the listener position – this means that by
default, it is not possible to have sources in different spaces and have each source
/space resonate independently since all sources are reverberated simultaneously. To
compensate for this we have introduced the possibility for an Audio source to make
its immediately surrounding space resonate (NAReverbEffector / Resonator) by using
volume triggering, this allows us to have more complex shapes than standard Unity
reverb zones (simple spherical volumes). This is a workaround since instead of the
listener it is the audio source that is reverberated (we found this to be a better
compromise).
Fig 4. New Atlantis space “Ljubljana Island” showing reverb zones.
First Reflections
We have conducted some experiments concerning first reflections, that consisted of
sending rays in all directions around the audio source, determining which rays hit a
colliders (corresponding to an audio reflecting surface in our approximation), and
then calculating the distance source-to-collider and collider-to-listener to define the

New Atlantis 15
audio delay for this path. The audio delay for each ray is then introduced in a multitap
delay System (a FIR (Finite Impulse Response) filter) and applied to the audio source
(using C# OnAudioFilterRead() custom DSP code). Although still an approximation,
this gave interesting results – a sense of scale, small spaces (with a short slap back
echo) and big spaces (cliffs, canyons etc.). However at the time of writing we have
not yet included this as a standard feature in NA.
Audio Sources Directivity and Roll-off
By default, the Unity audio engine does not provide management of audio source
directivity. We have introduced the possibility to set a curve defining the source
volume depending on the angle (similar to a polar diagram). Sound attenuation is
implemented either by manual adjustment of roll off curves, where the builder defines
how the sound attenuates over distance, or by using a physically accurate model for
sound attenuation, (the well known inverse square distance law) where the builder
simply defines the Sound Pressure Level at 1m in dB (dB SPL).
5 First Results - Student Workshops Public Presentations
5.1 Period September 2014 - January 2016 Description & Method
New Atlantis design and development took place between September 2014 and
September 2015, during workshops in Chicago and Aix-En-Provence and through
regular online (skype) meetings. Subsequently a workshop was organized in ENSCI
les Ateliers in Paris that included a group of 15 students and the whole team of
researchers, artists from Aix, Chicago, Troy and Paris. Within a week, students who
had never worked with either sound or 3D environments were able to create a set of
audiographic objects. At this point we programmed the 2016 performance (see
below). From October 2015 to January 2016 the team worked separately but
connectedly to prepare this performance: in Paris, another group of 10 students
created their own objects, helped to design graphic interface elements such as the
avatar and prepared for the upcoming performance. At the same time a Masters
student from ESA-Aix (Alex Amiel) was in residence in SAIC Chicago, helping to
build the Chicago space, a group of Masters students were working in ESA-Aix to
create the Aix space and Ben Chang was working in RPI-Troy to create his own
space. From January 10 to 15 another workshop with a fresh group of students from
design art and gaming schools took place at Le Cube. This time, the framework was
more advanced and the group was able to work and produce results much faster.
5.2 Presentations January 16th 2016
For the performance on January 16th 2016 participants and audience were gathered in
four different venues: Le Cube (Issy les Moulineaux, Paris France), the Vasarely
Foundation (Aix en Provence), SAIC (Chicago) and Rensselaer Polytechnic Institute

16 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
(Troy - New York state). The sound was played over 7.1 or 5.1 surround systems. As
well as being connected via NewAtlantis the participants in different venues were
connected via Skype during the performance and the audience was able listen to their
commentaries and exchanges live. Five NA spaces were visited during the 80-minute
performance that started at 9pm in Paris and 2pm in Chicago. The audience reaction
was enthusiastic and there was a positive reaction towards the prospect of continuing
development.
6 Conclusions and Further Research
One of the aims of NA is to provide a platform for international academic, artistic,
cultural and scientific exchange. In this sense the project has indeed succeeded in
creating a robust bridge between the American and French research teams and
students. The remaining challenge is for New Atlantis to become a real-time tool or
platform for international artistic and musical collaboration. Hopes are high however
since music and sound are after all internationally comprehensible. Possibly the most
important step, that of making the app publicly available has yet to be made.
6.1 Game architecture and Development
With the ultimate goal of making all things audio in NA (i.e. that all visual elements
have an audio counterpart) the concept of acoustic materials, could allow us to define
the way a 3D surface interact with incoming sound, with parameters such as
absorption at several audio frequencies, audio diffusion and reflection. We intend to
implement audio path-finding using Unity’s ray casting capabilities – precisely
calculating the contribution of each source to each space and applying a chain of
audio effects based on the traversed space’s characteristics. This was implemented in
the previous Panda3D/Pd version of NA it now has to be ported to Unity. Other
advanced topics will be addressed in the future, such as accurate sound occlusion
(possibly using ray casting between the source and the listener), and more advanced
audio spatialization algorithms with the Unity Spatialization SDK. It is our intention
to incorporate audio streaming capabilities into NA permitting the real time inclusion
“real world” captured sounds. This would enable a voice object for example whereby
a visitor could detach his or her voice (from navigation) and use it to make a distant
space resound.
6.2 Integration of and Intuitive synthesizer.
Locus Sonus has recently joined with the audio group of the CNRS Laboratory LMA,
now FRE PRISM with a program (accord cadre) funded by MCC (Ministry for
Culture and Communication) and CNRS (Centre Nationale pour la Recherche
Scientifique). In this context it is planned to integrate the LMA’s research into
intuitive synthesizer interaction. In effect this synthesizer that is controlled by

New Atlantis 17
“semantic descriptions of sound events”[34][35], including non-existent ones, would
appear to be an ideal solution for audio graphic experimentation in NA.
6.3 Interface and Graphic Design
Before releasing a public version of the New Atlantis app in is necessary to improve
the ergonomics of the user interface. The design process is programmed and we hope
to achieve this goal within the coming months.
Fig 5. New Atlantis performance synchronized between: Le Cube, Vasarely Foundation, SAIC
and RPI. January 2015. Photo: Le Cube.

18 Pete Sinclair, Roland Cahen, Jonathan Tanant and Peter Gena
7 New Atlantis Team 2016
Coordination:
Peter Sinclair (Locus Sonus - ESAAix), Peter Gena (SAIC), Roland Cahen (ENSCI)
Development
Jonathan Tanant (JonLab) : lead developer and software architect , Components developement :
Alexandre Amiel (ESAAix)
Faculty/Research:
Mark Anderson (3d Graphics SAIC), Robb Drinkwater (Audio programming SAIC), Michael
Fox (3d GraphicsSAIC), Jerome Joy (Audio Locus Sonus)
Ben Chang (Programming , 3d Graphics)
Students:
Daan De Lange, Théo Paolo, Alexandre Amiel, Antoine Langlois (ESAAix) Adrien Giordana,
Luca Notafrancesco, Marion Talou, Dorian Roussel, Valentin Moebs, Anaïs Maurette de
Castro, Thomas Signollet, Oscar Gillet, Juliette Gueganton, Mathilde Miossec, Gamzar Lee,
David Guinot, Paul Barret, Gaëtan Marchand, Aristide Hersant, Louis Fabre, Blanche Garnier,
Lucas Dubosque, Alexandra Radulescu.
Organization:
Anne Roquigny (Locus Sonus), Julie Karsenty (ESAAix)
Partners: Locus Sonus ESAAix Ecole supérieure d’art d’Aix-en-Provence
ENSCI Les Ateliers, Rensaler Polytechnic Institute, Troy, School of the art Institute of
Chicago, Chicago USA
Previous Participants:
Ricardo Garcia, Gonzague Defos de Rau, Margarita Benitez, Anne Laforet, Jerome Abel, Eddie
Breitweiser, Sébastien Vacherand.
References
1. Bacon, F.: The New Atlantis. no publisher given, United Kingdom (1628)
2. French-Americain Cultural Exchange, http://face-foundation.org/index.html
3. Second Life official site, http://secondlife.com/
4. Pure Data – Pd community site, https://puredata.info/
5. LS-SL Seconde Nature, http://www.secondenature.org/LS-SL-LOCUS-SONUS-IN-
SECOND-LIFE.html
6. Panda3D – Free 3D game Engine, https://www.panda3d.org/
7. NA version1, http://locusonus.org/w/?page=New+Atlantis
8. LaForet, A.: New Atlantis, un monde virtuel sonore. In Locus Sonus 10 ans
d’expérimentation en art sonore. pp. – Le Mot et Le Reste, Marseille (2015)
9. Unity Game Engine, https://unity3d.com/
10. AudioGaming, http://www.audiogaming.net/game-sound-design
11. UPIC https://en.wikipedia.org/wiki/UPICÒ
12. Cage, J. : Fontana Mix. Peters, New York, nº EP 6712, (1960)

New Atlantis 19
13. Sinclair, P., Joy, J.: Locus Sonus 10 ans d’expérimentation en art sonore. pp. – Le Mot et
Le Reste, Marseille (2015)
14. Polymeneas-Liontiris. T, Loveday-Edwards A. (2013) The Disklavier in Networked Music
Performances In: ATINER, 4th Annual International Conference on Visual and Performing
Arts 3-6 June 2013, Athens: Greece.
15. Sinclair, P.: ed. Sonification - What Where How Why. AI & Society (Springer) 27, no. 2
(05 2012).
16. Installation Icare de Ivan Chabanaud, http://www.musicvideoart.heure-
exquise.org/video.php?id=2151
17. BANDONEON, http://roland.cahen.pagesperso-orange.fr/bandoneon/Bandoneon.htm
http://www.edit-revue.com/?Article=200
18. Cahen, R.: Sound Navigation in the PHASE installation: producing music as performing a
game using haptic feedback. Subsol, Gérard, ed.
19. ENIGMES, http://projetenigmes.free.fr/wiki/index.php?title=Accueil
20. TOPOPHONIE, http://www.topophonie.com
21. UDKOSC, https://ccrma.stanford.edu/wiki/UDKOSC
22. Avatar Orchestra Metaverse, http://avatarorchestra.blogspot.fr/
23. (btween 22 and 23) Key Ed., Kanaga D.: Proteus – A game by Twisted Trees,
http://twistedtreegames.com/proteus/
24. Jacques Tati: Composing in Sound and Image,
https://www.criterion.com/current/posts/3337-jacques-tati-composing-in-sound-and-image
25. Cahen, R. Rodet, X. Lambert, JP.:Virtual Storytelling: Using Virtual Reality Technologies
for Storytelling: Third International Conference, ICVS 2005, Strasbourg, France, November
30-December 2, 2005: Proceedings.
26. Sinclair, P.: Inside Zeno’s Arrow: Mobile Captation and Sonification Audio Mobility Vol.
9 No. 2. 2015, http://wi.mobilities.ca/
27. Thibaud, JP.: Towards a praxiology of sound environment. Sensory Studies -
Sensorial Investigations, 2010, pp.1-7.
28. Schafer, RM.: The Tuning Of The World. Alfred Knopf, New York, (1977)
29. Key Ed., Kanaga D.: Proteus – A game by Twisted Trees,
http://twistedtreegames.com/proteus/
30. Locus Sonus Sound Map, http://locusonus.org/soundmap/051/
31. Locus Sonus, http://locusonus.org
32. Cahen, R.: Teaching Sound--Design @ENSCI les Ateliers, Sound-Design and innovation: a
virtuous circle. Cumulus 2015 Proceedings
33. SAIC, http://www.saic.edu/academics/departments/ats/
34. Conan S., Thoret E., Aramaki M., Derrien O., Gondre C., Kronland-Martinet R., Ystad S.
(2014). An intuitive synthesizer of continuous interaction sounds: Rubbing, Scratching and
Rolling. Computer Music Journal, 38(4) 24-37, doi:10.1162/COMJ_a_00266
35. Pruvost L., Scherrer B., Aramaki M., Ystad S., Kronland-Martinet R. Perception-Based
Interactive Sound Synthesis of Morphing Solids’ Interactions. Proceedings of the Siggraph
Asia 2015, Kobe, Japon, 2-5 Novembre 2015.