Conference PaperPDF Available

Abstract and Figures

Sonification has gained importance in the last few years due to the technological development in the areas of sound synthesis and manipulation. This area allows to transform and understand large data sets that can be too ample and complex to analyse without these tools. This paper describes the development of a musical sonification project applied to the area of Literature that maps the data of the Portuguese epic poem, The Lusiads by Luís de Camões, into sounds. This work intends to show that Sonification can be applied to different and not so common areas and create new ways of reading and understanding texts, in this case, a well-known and important poem from Portuguese Literature.
Content may be subject to copyright.
2 017.x C o A x.o rg
A PORTUGUESE EPOPEE
SEEN THROUGH SOUND
Lisbon
Computation
Com munication
Aesthetics
& X
Abstract
Sonification has gained importance in the last few
years due to the technological development in the
areas of sound synthesis and manipulation. This ar-
ea allows to transform and understand large data
sets that can be too ample and complex to ana-
lyse without these tools. This paper describes the
development of a musical sonification project ap-
plied to the area of Literature that maps the data
of the Portuguese epic poem, The Lusiads by Luís
de Camões, into sounds. This work intends to show
that Sonification can be applied to different and not
so common areas and create new ways of reading
and understanding texts, in this case, a well-known
and important poem from Portuguese Literature.
Keywords
Auditory Display
Data Representation
Musical Sonification
Information Design
Lusiads
ÂNGELA COELHO
aacoelho@student.dei.uc.pt
PEDRO MARTINS
pjmm@dei.uc.pt
AMÍLCAR CARDOSO
amilcar@dei.uc.pt
CISUC, Informatics
Engineering Department
University of Coimbra
Coimbra, Portugal
216 1. INTRO DUCTION
Sonification is a very interdisciplinary and relatively new field, which is gaining
more and more importance due to the huge progress of computers in recent
years (Kramer et al. 2010, Hermann and Hunt 2005). The evolution of computers
not only improved the technologies available but also increased the generation
of large and complex amounts of data, which changed the way we learn, commu-
nicate and explore the information received (Kramer et al. 2010, Hermann and
Hunt 2005). All these factors led to the emergence of the sonification field, which
joins concepts like perception, acoustic, design, arts and engineering (Kramer
et al. 2010).
This field can be briefly described as a subtype of the auditory display that uses
non-verbal sounds to represent information (Barrass and Kramer 1999, Kramer et
al. 2010, Hermann, Hunt and Neuhoff 2011). The sonification method transforms
data, its characteristics and relationships, into acoustic signals, where sound has
the function to communicate data and provide support for information analysis
(Frazier 2013, Hermann and Hunt 2005, Kramer et al. 2010, Minciacchi and
Rosenboom 2015, Park et al. 2010, Vicinanza 2014). There is also a Sonification
approach called Musical Sonification, which was explored in the project described
in this paper to take advantage of the action of listening to music and its character-
istics and particularities to represent information (Ben-Tal and Berger 2004). It is
possible to use the changes over the course of a music such as pitch, amplitude, tim-
bre, tempo, rhythm to create a mental image that can be used to represent data.
Sonification can easily communicate larger and dynamic data because it pro-
vides two new dimensions to represent it: the sound itself (and its characteris-
tics) and the idea of time (Kramer et al. 2010, Minciacchi and Rosenboom 2015).
A sonification system allows showing vast amounts of data in a small period, gi-
ving a general overview of the information, as well as the existing trends and
patterns. Also the interdisciplinarity of this field provides conditions to use soni-
fication to improve visualisation systems.
The work presented in this paper explores the field of Musical Sonification
applied to poetry. This project transforms the characteristics of a popular and
relevant Portuguese poem, The Lusiads, into music. A preliminary sketch of this
work was presented at the 4th International Workshop on Musical Metacreation
(MUME), which allowed to test different approaches for the sonification: how
many instruments should be used and how to use them and how to represent
different levels of information (Coelho, Martins, and Cardoso 2016).
Comparisons between music and language are traditionally established at the
syntax and rhetoric level, which derives from the fact that both music and lan-
guage are sounds organised in time (Lerdahl 2001). Although these two areas
share the same roots, the evolution led to the specialisation of each one: music
in pitch organisation and language in word and sentence meaning (Lerdahl 2001).
However, poetry exists in this evolutionary divergence because it combines spe-
ech with elements of a musical heritage: rhythmic and metrical patterns. The
sound can represent something that maps or graphics are not capable: the rhythm
of the data (House and Brooks 2013, Vanhemert 2013). According to these com-
parisons, it makes sense to explore the poetry in Sonification, which can enrich
the poetry analysis and turn this process easier and more appealing to people
unfamiliar with poetry.
2 17
The will to create a different sonification process, which can take advantage of
the sound characteristics to provide poetry information, led to the development
of this project. The sonification in Literature can create a new way of reading and
understanding text, which was the motivation for this work. Our main goal was
to build a software application that could convey information about the poem
and create a new vision of a so well-known and essential book of Portuguese Lit-
erature, where the user can experience the poem through a customised naviga-
tion and explore different sections of the story told. This work explores the use
of sonification techniques in poetry, where it is not common to apply these type
of processes, but whose understanding can be improved with their application.
With this purpose in mind, we created a musical and visual interface that reflects
not only the sequential structure of the poem but also its external structure.
To the best of our knowledge, there are not many sonification works in this
domain, one of the few exceptions being the sonification of Chinese poetry, Text-
to-Music (Huang, Lu, and Ren 2011). This project transforms the characteristics
and dynamics of the poem to durations of musical elements, which allows people,
who are not familiar with Chinese poetry, to appreciate it in an easier way (Huang,
Lu, and Ren 2011; Ren 2007).
To better explain the model of Musical Sonification presented herein, the re-
mainder of this paper starts by providing an overview of the key aspects of The
Lusiads. Then, there is a description of the application built and the sonification
process applied in this work. The paper ends with a conclusion, where the Soni-
fication area and the results obtained with this project are objects of reflection.
2. THE LUSIADS
The Lusiads is a Portuguese epic poem written by Luís de Camões (1524-1580).
This book was published in 1572, and it has been the subject of numerous analy-
ses over time (Sena 1980). The poem belongs to the epic genre, a literary genre
that comes from the Greek-Latin Antiquity (Camões 2011, Gaio n.d., Pais 1994).
This book is inspired by Virgil’s Aeneid or Homer’s Iliad and Odyssey, i.e., epopees.
It is written in verse, in a high style, and intends to magnify the achievements
of the heroes, in this case, the Portuguese people. Camões narrates the voyage
of the explorer Vasco da Gama to India, and through this story, he tells the Por-
tuguese deeds and extols the strength of his people (Sena 1980).
2.1. Poem characteristics
The poem has a total of 1102 stanzas of eight verses each, which are divided into
ten cantos and two epic cycles (Fig. 1). The verses are decasyllables because they
contain ten poetic syllables and the stanza has a rhyme scheme constant throug-
hout the whole poem, consisting of crusade rhymes in the first six verses and
paired rhymes in the last two a b a b a b c c (Pais 1994, Gaio n.d.). The internal
structure of the poem follows the epic genre rules, so it is divided in Proposition,
Invocation, Dedication and Narration (Pais 1994, Gaio n.d.).
The Lusiads can also be divided, as shown in Fig. 1, into four narrative plans,
which intersect and coexist over the narrative (Michelli 2003, Sena 1980):
1. Travel plan, which consists in the narration of the events during the
trip from Lisbon to India;
218 2. Gods’ plan, which includes the gods’ interventions, intersected with
the travel plan, where the gods make decisions that affect the fate of
the Portuguese people;
3. Portugal History plan, which is the narration of Portugal’s history
by Vasco da Gama to the King of Malindi;
4. Poet plan, mostly located at the end of each canto, where Camões
reflects the state of the world.
The poem is also divided, according to some studies, in various subnarratives
(Fig. 1) that represent the main events of the book (Sena 1980). Different char-
acters make the narration of the story, such as Luís de Camões itself or Vasco
da Gama (Pais 1994).
2.2. Dataset
The characteristics to be sonified can be divided into three main areas, as shown
in Fig. 2:
1. Narrators, where the intervention of the different narrators
was explored;
2. Narrative Plans, where the intersection and intensity of the plans
of the story was sonified;
3. Subdivisions, which includes the sonification of the subnarratives,
episodes and emotions identified in the poem.
Furthermore, we also explored the verses characteristics and the rhyme scheme
to create the rhythm and the prophecies presented during the poem. We created
a database to keep all this information and to communicate with the sonifica-
tion system.
3. PROJECT
The project of Musical Sonification that was developed transforms the main fea-
tures of the poem, The Lusiads, into sound. The principal idea was to build a
software application that allows the user to experience the poem in an innova-
Fig. 1
General structure of the
poem The Lusiads.
219 tive way and to obtain different types of information depending on the custom
navigation performed by the user.
3.1. Features
To satisfy this goal, the application offers different features to improve the user
experience. The user can control the reproduction of the sound result and nav-
igate across the different sections of the book by the timeline in the interface
(Fig. 3 [A]). The project also has a visual component that works as a label and
guides the user through the sonified information, which improves the process
of sound visualisation (Fig. 3 [B]). The application allows the user to select the
information that he / she wants to hear by the filters (Fig. 3 [C]), which improves
the understanding process between what is listened and what is shown.
Besides the previous features, the application offers, as a form to navigate
through the poem, three types of zoom, whose information has different levels
of specificity (Fig. 4). Therefore, the user can select the section to be sonified:
1. Zoom 0: Where the user can listen to the sonification of the whole
poem and only the higher level divisions of the poem are sonified.
The sound result in this zoom has a total of 80 bars, which means
that each bar represents 13 stanzas of the poem.
2. Zoom 1: This type of zoom is available after selecting a canto,
subnarrative or bigger episode, which is sonified. The difference
between this and the previous zoom is that at this level, the presence
of each plan is sonified instead of its intensity. The scale used is eight
stanzas of the selected excerpt per bar.
3. Zoom 2: Level of higher detail, which is activated after the choice
of a specific episode and offers a sonification of the selected passage.
The scale applied at this level is one stanza to one bar.
Fig. 2
Information sonified
in the application.
Fig. 3
Screenshot of
the application.
220
Zooms
The link in Fig. 5 has a demonstration of the application’s functionalities being
in use.
3.2. Technologies and resources
The development of all these functionalities demanded the use of different tech-
nologies that were able to communicate with each other. In this way, we used
the Processing language to create the interface presented, to control the action
of the remaining software and to establish the communication between the user
and the other tools. At this level, the connection to the database is identified via
the BezierSQLib library, which allows the access to the information in the data-
base by SQL. This data is then processed in Processing and sent to Max / MSP
using the oscP5 library. At this point, a Max / MSP patcher is responsible for the
sonification, according to the information received. This process involves the
creation of text files that store the sonification inputs. After this, the patcher
plays the sound result of the subdivisions of the poem and sends the MIDI notes
of the remaining information to be played through Ableton Live. Max is also res-
ponsible for sending the information required by Processing to construct the
labels and the visual component. Ableton Live is the MIDI sequencer respon-
sible for the reproduction of the narrators and narrative plans. It distributes the
Fig. 4
Scheme of
different zooms.
Fig. 5
Demonstration of the final
application (https://vimeo.
com/201050778)
221 information received by Max/MSP for the respective MIDI channels. We also
used a database of sound files, Soundtracks, datasets for music and emotion,
as a resource (Eerola and Vuoskoski 2010). This dataset is an archive of movie
sound samples classified in a set of emotions. These samples are applied in the
sonification of the subdivisions of the poem.
4. SONIFICATION PROPOSAL
The sonification model applied follows essentially two sonification techniques:
auditory icons and parameter mapping sonification. The first one takes advantage
of the association of information with familiar sounds and the last one consists
in the association of information to different sound characteristics (Barrass and
Kramer 1999). The final sonification has 120 bpm and a variable number of bars,
according to the dimension of the selected excerpt. The link in Fig. 6 has a de-
monstration of the sonification results for zoom 0.
4.1. Rhyme scheme
The creation of the harmonic structure of the output music is based on a simple
model of 8 bars. This pattern is mapped from the rhyme scheme of the poem (a
b a b a b c c): a is mapped to C major, b with F major and c with G major (Fig. 7).
The result is a simple cyclic chord progression (C F C F C F G G), one chord/bar,
which results in a music in C Major, with no modulations.
Fig. 6
Sonification of the
zoom 0 (https://vimeo.
com/180120006)
Fig. 7
Harmonic structure
of the Sonification.
222 4.2. Narrators
The narrators’ interventions in The Lusiads are transformed into sound with the
use of parameter mapping. Through the whole book, there are 43 different nar-
rators that were divided, for the purpose of the sonification, into six types: main,
secondary, mythological, crew, Portuguese and foreign. Each of these types is
associated with a different instrument, respectively, tuba, flute, trombone, oboe
and horn. Each instrument plays the song during the intervention of the cor-
respondent narrator type. The melody played at this stage follows the pattern
described earlier and is computed at the bar level: for each block, a variable
sequence of 10 notes (six eighth notes and four sixteenth notes) of the chord’s
scale is generated. Therefore, the bar has the same size of the poem verse, 10
notes to 10 syllables. At the visual level, each bar of the narrator is represented
by a blue circle, whose position and colour change according to the rhyme and
the narrator type, respectively (Fig. 3 [D]). Although this information is sonified
at every zoom, the sound results change because with the increase of the detail
is possible to hear narrators with minor interventions.
4.3. Narrative Plans
Narrative plans are also transformed into sound with the application of parameter
mapping. The four plans — Travel, Gods, Portugal History, Poet are associated,
each one, with a different percussion instrument conga bongo, xylophone, clog
box and clavestine. Each instrument follows a different rhythmic pattern, repea-
ted at every bar. The plans are also represented by a diamond, whose colour
changes with the type of plan (Fig. 3 [E]).
At zoom 0, the intensity of each plan, in other words, the variation of its num-
ber of stanzas in each canto, is mapped to the volume of the correspondent
sound and shown by the variation of the diamond size. At the other levels, the
presence of each plan in the story is mapped, so each instrument plays when
the corresponding plan exists.
4.4. Subdivisions
As shown in Fig. 2, the subdivisions of the poem were divided into three groups:
subnarratives, episodes and emotions. The method followed here was auditory
icons, so each subdivision is assigned with a familiar soundtrack. Every group in
this section is represented by a stroke, whose colour scheme changes with the
type represented (Fig. 3 [F]).
Subnarratives
The subnarratives of the poem are played only in zoom 0, before the selection
of a canto or subnarrative. The poem has eight subnarratives (Fig. 1) that were
grouped into three types to favour the sonification process: Dedication, Travel
and History. Each subnarrative group is associated with a music sample that
resembles it, which plays during the corresponding event.
223 Episodes
The episodes are sonified in zoom 1, after the selection of a canto, subnarrative
or a longer episode. The poem can be divided in 56 episodes that, in order to
be mapped, were grouped into seven types: fantastic, geographic, opinion, reign,
warlike, lyric and naturalist. Each type has a similar thematic sound sample that
is played during the specific episode.
Emotions
The emotions presented in every episode were collected and stored in an emo-
tion scale: anger, concern, tranquillity and love. This information is mapped into
sound in zoom 2, after the selection of a specific episode. Each emotion is as-
signed to a correspondent sample of the database resource, which is played
when the emotion is identified.
4.5. Prophecies
For last, the poem also contains prophecies that tell historical Portuguese events
that occur after the time of the narrative. These prophecies are sonified in the
last zoom with the application of reverb, while they are present, which gives an
echo sensation. The goal in this case was to give the sensation of a prophetic
and magical environment because these sections of the poem are mainly con-
trolled by mythological figures.
5. CONCLUSIONS AND REFLEXIONS
Sonification is not a traditional mapping method, which leads to the existence
of issues and challenges to overcome for the improvement of this area. How to
choose the best sonification technique for a dataset? Can all kinds of data be
sonified? Can a sonification system be understood without a visual component?
How to make the sound representation easy-to-understand? Will the technology
be prepared for the integration and expansion of Sonification? These are some of
the issues that emerge from the analysis of the Sonification field. It is necessary
to investigate and explore these questions in order to establish suitable method-
ologies and identify the most promising techniques to address them.
The principal purpose of the project presented herein was to explore Sonification
in an area where its application is not typical: poetry. This work allowed us to ex-
periment and test different approaches of sonification until we found one that we
believed to be suitable for the project. During this process, we were confronted
with some challenges: the creation of the dataset of the poem, the representation
of so many levels of information, the choice of the instruments and samples to
be used, the technological limitations and the development of a visual guide to
the sonification. The solution we found to overcome these issues was the crea-
tion of an interactive sonification system that allows the user to explore the poem
in a customised way and at his / her own time. This type of interactive systems
expands the sonification process because it allows the user to compare different
sections of the story and get conclusions. The user can choose the section that
he / she would like listen to and the type of information he / she wants to receive,
which turns the understanding process of the poem easier. In order to test the
224 usefulness of this product, some usability tests were applied that, although are
not able to fully test the sonification process, were able to understand that the
users can create an analysis process from the sonification results. These tests
cannot evaluate in detail the user experience, which will be an area to fully ana-
lyse in future work, with more suitable techniques. However, it was possible to
understand that users were capable of getting conclusions about the story of
the poem and create correlations between different types of information and
sections of the poem.
The goal of this project was to create an application that can communicate
information and also be able to produce a new vision of a book so well-known in
the Portuguese Literature. Besides, the results of this work allow us to understand
that sound can communicate information and how it can relate with language.
This project reinforces the idea that music and language are connected and share
the same roots. Poetry emerges between these two areas and joins character-
istics of both fields, so it makes sense to explore the poetry in Sonification, to
take advantage of the sound as a communication tool and to improve the under-
standing process of complex and long poetic narratives.
REFERENCES
Barrass, Stephen, and Gregory Kramer.
‘Using Sonification.’ Multimedia Systems 7,
no. 1 (1999): 23-31.
Ben-Tal, Oded, and Jonathan Berger.
‘Creative Aspects of Sonification.’ Leonardo
37, no. 3 (2004): 229-33.
Camões, Luís de. ‘Os Lusíadas’. Porto, Por-
tugal: Porto Editora, 2011.
Coelho, Ângela, Pedro Martins, and Amílcar
Cardoso. ‘A Musical Sonification of the
Portuguese Epopee.’ Paper presented at the
MUME, Paris, France, 2016.
Eerola, Tuomas, and Jonna K Vuoskoski.
A Comparison of the Discrete and Dimen-
sional Models of Emotion in Music.’
Psychology of Music 39, no. 1 (2010): 18-49.
Frazier, Sarah. ‘Sonification: Data Like
You’ve Never Heard Before.’ (2013).
http://earthzine.org/2013/07/23/sonifica-
tion-data-like-youve-never-heard-before/
Gaio, Tiago. ‘Os Lusíadas.’
http://oslusiadas.no.sapo.pt/indice.html
Hermann, Thomas, and Andy Hunt.
‘Guest Editors’ Introduction: An Introduction
to Interactive Sonification.’ IEEE Multimedia,
2005, 20-24.
Hermann, Thomas, Andy Hunt, and John
G. Neuhoff. ‘Introduction.’ Chap. 1 In The
Sonification Handbook, edited by Thomas
Hermann, Andy Hunt and John G. Neuhoff,
1-6. Berlin, Germany: Logos Verlag, 2011.
House, Brian, and Katherine Brooks.
‘Quotidian Record: Artist Brian House
Turns Tracking Data into a Vinyl Music
Experience.’ (2013). http://www.huffing-
tonpost.com/2013/07/26/quotidien-re-
cord_n_3653658.html
Huang, Chih-Fang, Hsiang-Pin Lu, and
Jenny Ren. ‘Algorithmic Approach to Sonifi-
cation of Classical Chinese Poetry.’
Multimedia Tools and Applications 61,
no. 2 (2011): 489-518.
Kramer, Gregory, Bruce Walker, Terri Bone-
bright, Perry Cook, John H Flowers, Nadine
Miner, and John Neuhoff.
‘Sonification Report: Status of the Field and
Research Agenda.’ Faculty Publications, De-
partment of Psychology (2010): Paper 444.
Lerdahl, Fred. ‘The Sounds of Poetry Viewed
as Music.’ Annals of the New York Academy
of Sciences 930 (2001): 337-54.
Michelli, Regina. ‘Os Lusíadas, De Luís De
Camões: Breve Viagem Estrutural.’
Augustus, 2003, 39-48.
Minciacchi, Diego, and David Rosenboom.
‘Sonification, Perceptualizing Biological In-
formation.’ (2015). http://journal.frontiersin.
org/researchtopic/3579/sonification-percep-
tualizing-biological-information-overview
Pais, Amélia Pinto. ‘Para Compreender
Os Lusíadas.’ 2ª ed. Porto, Potugal: Areal
Editores, 1994.
Park, Sihwa, Seunghun Kim, Samuel Lee,
and Woon Seung Yeo. ‘Online Map Interface
for Creative and Interactive Music-Making.’
Paper presented at the New Interfaces
for Musical Expression (NIME), Sydney,
Australia, 2010.
Ren, Jenny; Winsor, Phil; Huang, Chih-Fang.
‘Developing Chinese Style Algorithmic Com-
position Using Markov Chains — from the
Classical Chinese-Poetry Perspective.’ Paper
presented at the International Computer
Music Conference (ICMC), Copenhagen,
Denmark, 2007.
Sena, Jorge de. ‘A Estrutura De “Os Lusía-
das” E Outros Estudos Camonianos E De
Poesia Peninsular Do Século XVI’. Obras
De Jorge De Sena. 2 ed. Lisboa, Portugal:
Edições 70, 1980.
Vanhemert, Kyle. ‘Artist Turns a Year’s
Worth of Tracking Data into a Haunting
Record.’ (2013). https://www.wired.com/
2013/07/a-years-worth-of-location-data-
transformed-into-a-beautiful-record/
Vicinanza, Domenico. ‘Sonification: Repre-
senting Data through Music.’ (2014 ).
http://www.bbc.co.uk/news/maga-
zine-25975712
... In the poetry field, Coelho, Martins and Cardoso [3] created a A Musical Sonification of the Portuguese Epopee, specifically of The Lusiads, by Luís de Camões. It is an interactive sonification where the user can explore the poem by choosing different levels of "zooming", listening to it as a whole, as a canto/ subnarrative or a specific episode and therefore customising the experience. ...
... The three values were calculated so we could understand the distribution of the data and compare some discrepancy caused by extremely large or small values. In general, the mean of the level of participants that chose the right answer is at the intermediate level (3). The only value that stands from this scenario is in the first question, were the mean of the five participants that scored the highest value reached the second highest level of musical knowledge (4). ...
Chapter
Sonification is a scientific field that seeks to explore the potential of sound as an instrument to convey and interpret data. Its techniques have been developing significantly with the growth of technology and supporting hardware and software, which have spread in our daily environment. This allowed the establishment of new communication tools to share information, opinion and feelings as part of our daily routine.
... In the poetry field, Coelho, Martins and Cardoso [3] created a A Musical Sonification of the Portuguese Epopee, specifically of The Lusiads, by Luís de Camões. It is an interactive sonification where the user can explore the poem by choosing different levels of "zooming", listening to it as a whole, as a canto/ subnarrative or a specific episode and therefore customising the experience. ...
... The three values were calculated so we could understand the distribution of the data and compare some discrepancy caused by extremely large or small values. In general, the mean of the level of participants that chose the right answer is at the intermediate level (3). The only value that stands from this scenario is in the first question, were the mean of the five participants that scored the highest value reached the second highest level of musical knowledge (4). ...
Conference Paper
Full-text available
Sonification is a scientific field that seeks to explore the potential of sound as an instrument to convey and interpret data. Its techniques have been developing significantly with the growth of technology and supporting hardware and software, which have spread in our daily environment. This allowed the establishment of new communication tools to share information, opinion and feelings as part of our daily routine. The aim of this project was to unite the social media phenomena with sonification, using Twitter data to extract user’s emotions and translate them into musical compositions. The focus was to explore the potential of music in translating data as personal and subjective as human emotions, developing a musically complex and captivating mapping based on the rules of Western Music. The music is accompanied by a simple visualization, which results in emotions being heard and seen with the corresponding tweets, in a multimodal experience that represents Twitter’s emotional reality. The mapping was tested through an online survey, and despite a few misunderstandings, the results were generally positive, expressing the efficiency and impact of the developed system.
Conference Paper
Full-text available
Sonification is a field that has gained importance in the last few years due to the technological development in the areas of sound synthesis and manipulation. This area can be seen as the auditory counterpart of Information Visualization and it has real meaning when the data sets are too ample and complex to be seen graphically. This paper describes work in progress on the development of a musical sonification of The Lusiads, a Portuguese epic poem by Luís de Camões. This work intends primarily to create and explore new ways of reading the Portuguese epopee.
Article
Full-text available
A goal of sonification research is the intuitive audio representation of complex, multidimensional data. The authors present two facets of this research that may provide insight into the creative process. First, they discuss aspects of categorical perception in nonverbal auditory scene analysis and propose that these characteristics are simplified models of creative engagement with sound. Second, they describe the use of sonified data in musical compositions by each of the authors and observe aspects of the creative process in the purely aesthetic use of sonified statistical data.
Conference Paper
Full-text available
In this paper, we discuss the musical potential of COM-Path -an online map based music-making tool -as a novel and unique interface for interactive music composition and performance. COMPath provides an intuitive environment for creative music making by sonification of georeferenced data. Users can generate musical events with simple and familiar actions on an online map interface; a set of lo-cal information is collected along the user-drawn route and then interpreted as sounds of various musical instruments. We discuss the musical interpretation of routes on a map, review the design and implementation of COMPath, and present selected sonification results with focus on mapping strategies for map-based composition.
Article
Full-text available
The primary aim of the present study was to systematically compare perceived emotions in music using two different theoretical frameworks: the discrete emotion model, and the dimensional model of affect. A secondary aim was to introduce a new, improved set of stimuli for the study of music-mediated emotions. A large pilot study established a set of 110 film music excerpts, half were moderately and highly representative examples of five discrete emotions (anger, fear, sadness, happiness and tenderness), and the other half moderate and high examples of the six extremes of three bipolar dimensions (valence, energy arousal and tension arousal). These excerpts were rated in a listening experiment by 116 non-musicians. All target emotions of highly representative examples in both conceptual sets were discriminated by self-ratings. Linear mapping techniques between the discrete and dimensional models revealed a high correspondence along two central dimensions that can be labelled as valence and arousal, and the three dimensions could be reduced to two without significantly reducing the goodness of fit. The major difference between the discrete and categorical models concerned the poorer resolution of the discrete model in characterizing emotionally ambiguous examples. The study offers systematically structured and rich stimulus material for exploring emotional processing.
Article
Full-text available
The idea behind sonification is that synthetic nonverbal sounds can represent numerical data and provide support for information processing activities of many differentkinds. This article describes some of the ways that sonification has been used in assistive technologies, remote collaboration, engineering analyses, scientific visualisations, emergency services and aircraft cockpits. Approaches for designing sonifications are surveyed, and issues raised by the existing approaches and applications are outlined. Relationsare drawn to other areas of knowledge where similar issueshave also arisen, such as human-computer interaction, scientific visualisation, and computer music. At the end is a listof resources that will help you delve further into the topic.
Article
Full-text available
An extended parallel is developed between musical and prosodic structures, using the author's cognitively oriented music theory and recent work in generative phonology. For illustration, the sounds of a short poem by Robert Frost are treated entirely in musical terms. The poem is assigned a phonological stress grid and then musical grouping and meter. These structures enable a durational realization. Phonological stress also helps assign the poem's normative melodic contour. Finally, the similarities and differences in sound repetition are given hierarchical structure by means of musical prolongational theory. These formal parallels suggest a corresponding realization in brain localization and function. Evidence from the neuropsychological literature is cited in support of this view. The picture emerges that grouping, meter, duration, contour, and timbral similarity are mind/brain systems shared by music and language, whereas linguistic syntax and semantics and musical pitch relations are systems not shared by the two domains.
Article
Full-text available
Today's computing technology is radically different from that of 10 years ago. Devices such as smartphones, tablets, and even wearable devices are found wherever we are. Researchers and developers can take advantage of this new era by knowing that the public has personal access to highly interactive multimedia devices. Interfaces involving sound are already in the hands of millions of people. For information display, sound promises an alternative to squeezing information through small screens that then force us to attend to them, thus making us lose awareness of our immediate environment. This special issue looks at some of the upcoming research on how such interaction with sound can be used in a variety of areas and applications.
Article
The classical Chinese poetry is a remarkable form of art in traditional Chinese character. However, it is difficult for people who are unfamiliar with ancient Chinese to experience the artistic content of the poetry. In this study, a sonification scheme, Tx2Ms (Text-to-Music), is proposed to extract the poetry features between lines in verses; moreover, dynamics and interval relations are modeled to map those features to the movement of multi-dimensional musical elements such as durations. This conversion is based on poetry intonation and acoustic analysis of the pronunciations of poems; and a stochastic compositional algorithm is created by applying a Markov chain. In addition, the best pentatonic mode for a specific poem is recommended according to the formants analysis. Therefore, the sonification of classical Chinese poetry not only provides a novel way for people to appreciate Chinese poetry but also enriches the state of mind and imagery in the delivery process, and the experiment results show that the proposed system is successfully accepted by most people.
Os Lusíadas'. Porto, Portugal: Porto Editora
  • Luís Camões
  • De
Camões, Luís de. 'Os Lusíadas'. Porto, Portugal: Porto Editora, 2011.