2 017.x C o A x.o rg
A PORTUGUESE EPOPEE
SEEN THROUGH SOUND
Sonification has gained importance in the last few
years due to the technological development in the
areas of sound synthesis and manipulation. This ar-
ea allows to transform and understand large data
sets that can be too ample and complex to ana-
lyse without these tools. This paper describes the
development of a musical sonification project ap-
plied to the area of Literature that maps the data
of the Portuguese epic poem, The Lusiads by Luís
de Camões, into sounds. This work intends to show
that Sonification can be applied to different and not
so common areas and create new ways of reading
and understanding texts, in this case, a well-known
and important poem from Portuguese Literature.
University of Coimbra
216 1. INTRO DUCTION
Sonification is a very interdisciplinary and relatively new field, which is gaining
more and more importance due to the huge progress of computers in recent
years (Kramer et al. 2010, Hermann and Hunt 2005). The evolution of computers
not only improved the technologies available but also increased the generation
of large and complex amounts of data, which changed the way we learn, commu-
nicate and explore the information received (Kramer et al. 2010, Hermann and
Hunt 2005). All these factors led to the emergence of the sonification field, which
joins concepts like perception, acoustic, design, arts and engineering (Kramer
et al. 2010).
This field can be briefly described as a subtype of the auditory display that uses
non-verbal sounds to represent information (Barrass and Kramer 1999, Kramer et
al. 2010, Hermann, Hunt and Neuhoff 2011). The sonification method transforms
data, its characteristics and relationships, into acoustic signals, where sound has
the function to communicate data and provide support for information analysis
(Frazier 2013, Hermann and Hunt 2005, Kramer et al. 2010, Minciacchi and
Rosenboom 2015, Park et al. 2010, Vicinanza 2014). There is also a Sonification
approach called Musical Sonification, which was explored in the project described
in this paper to take advantage of the action of listening to music and its character-
istics and particularities to represent information (Ben-Tal and Berger 2004). It is
possible to use the changes over the course of a music such as pitch, amplitude, tim-
bre, tempo, rhythm to create a mental image that can be used to represent data.
Sonification can easily communicate larger and dynamic data because it pro-
vides two new dimensions to represent it: the sound itself (and its characteris-
tics) and the idea of time (Kramer et al. 2010, Minciacchi and Rosenboom 2015).
A sonification system allows showing vast amounts of data in a small period, gi-
ving a general overview of the information, as well as the existing trends and
patterns. Also the interdisciplinarity of this field provides conditions to use soni-
fication to improve visualisation systems.
The work presented in this paper explores the field of Musical Sonification
applied to poetry. This project transforms the characteristics of a popular and
relevant Portuguese poem, The Lusiads, into music. A preliminary sketch of this
work was presented at the 4th International Workshop on Musical Metacreation
(MUME), which allowed to test different approaches for the sonification: how
many instruments should be used and how to use them and how to represent
different levels of information (Coelho, Martins, and Cardoso 2016).
Comparisons between music and language are traditionally established at the
syntax and rhetoric level, which derives from the fact that both music and lan-
guage are sounds organised in time (Lerdahl 2001). Although these two areas
share the same roots, the evolution led to the specialisation of each one: music
in pitch organisation and language in word and sentence meaning (Lerdahl 2001).
However, poetry exists in this evolutionary divergence because it combines spe-
ech with elements of a musical heritage: rhythmic and metrical patterns. The
sound can represent something that maps or graphics are not capable: the rhythm
of the data (House and Brooks 2013, Vanhemert 2013). According to these com-
parisons, it makes sense to explore the poetry in Sonification, which can enrich
the poetry analysis and turn this process easier and more appealing to people
unfamiliar with poetry.
The will to create a different sonification process, which can take advantage of
the sound characteristics to provide poetry information, led to the development
of this project. The sonification in Literature can create a new way of reading and
understanding text, which was the motivation for this work. Our main goal was
to build a software application that could convey information about the poem
and create a new vision of a so well-known and essential book of Portuguese Lit-
erature, where the user can experience the poem through a customised naviga-
tion and explore different sections of the story told. This work explores the use
of sonification techniques in poetry, where it is not common to apply these type
of processes, but whose understanding can be improved with their application.
With this purpose in mind, we created a musical and visual interface that reflects
not only the sequential structure of the poem but also its external structure.
To the best of our knowledge, there are not many sonification works in this
domain, one of the few exceptions being the sonification of Chinese poetry, Text-
to-Music (Huang, Lu, and Ren 2011). This project transforms the characteristics
and dynamics of the poem to durations of musical elements, which allows people,
who are not familiar with Chinese poetry, to appreciate it in an easier way (Huang,
Lu, and Ren 2011; Ren 2007).
To better explain the model of Musical Sonification presented herein, the re-
mainder of this paper starts by providing an overview of the key aspects of The
Lusiads. Then, there is a description of the application built and the sonification
process applied in this work. The paper ends with a conclusion, where the Soni-
fication area and the results obtained with this project are objects of reflection.
2. THE LUSIADS
The Lusiads is a Portuguese epic poem written by Luís de Camões (1524-1580).
This book was published in 1572, and it has been the subject of numerous analy-
ses over time (Sena 1980). The poem belongs to the epic genre, a literary genre
that comes from the Greek-Latin Antiquity (Camões 2011, Gaio n.d., Pais 1994).
This book is inspired by Virgil’s Aeneid or Homer’s Iliad and Odyssey, i.e., epopees.
It is written in verse, in a high style, and intends to magnify the achievements
of the heroes, in this case, the Portuguese people. Camões narrates the voyage
of the explorer Vasco da Gama to India, and through this story, he tells the Por-
tuguese deeds and extols the strength of his people (Sena 1980).
2.1. Poem characteristics
The poem has a total of 1102 stanzas of eight verses each, which are divided into
ten cantos and two epic cycles (Fig. 1). The verses are decasyllables because they
contain ten poetic syllables and the stanza has a rhyme scheme constant throug-
hout the whole poem, consisting of crusade rhymes in the first six verses and
paired rhymes in the last two — a b a b a b c c (Pais 1994, Gaio n.d.). The internal
structure of the poem follows the epic genre rules, so it is divided in Proposition,
Invocation, Dedication and Narration (Pais 1994, Gaio n.d.).
The Lusiads can also be divided, as shown in Fig. 1, into four narrative plans,
which intersect and coexist over the narrative (Michelli 2003, Sena 1980):
1. Travel plan, which consists in the narration of the events during the
trip from Lisbon to India;
218 2. Gods’ plan, which includes the gods’ interventions, intersected with
the travel plan, where the gods make decisions that affect the fate of
the Portuguese people;
3. Portugal History plan, which is the narration of Portugal’s history
by Vasco da Gama to the King of Malindi;
4. Poet plan, mostly located at the end of each canto, where Camões
reflects the state of the world.
The poem is also divided, according to some studies, in various subnarratives
(Fig. 1) that represent the main events of the book (Sena 1980). Different char-
acters make the narration of the story, such as Luís de Camões itself or Vasco
da Gama (Pais 1994).
The characteristics to be sonified can be divided into three main areas, as shown
in Fig. 2:
1. Narrators, where the intervention of the different narrators
2. Narrative Plans, where the intersection and intensity of the plans
of the story was sonified;
3. Subdivisions, which includes the sonification of the subnarratives,
episodes and emotions identified in the poem.
Furthermore, we also explored the verses characteristics and the rhyme scheme
to create the rhythm and the prophecies presented during the poem. We created
a database to keep all this information and to communicate with the sonifica-
The project of Musical Sonification that was developed transforms the main fea-
tures of the poem, The Lusiads, into sound. The principal idea was to build a
software application that allows the user to experience the poem in an innova-
General structure of the
poem The Lusiads.
219 tive way and to obtain different types of information depending on the custom
navigation performed by the user.
To satisfy this goal, the application offers different features to improve the user
experience. The user can control the reproduction of the sound result and nav-
igate across the different sections of the book by the timeline in the interface
(Fig. 3 [A]). The project also has a visual component that works as a label and
guides the user through the sonified information, which improves the process
of sound visualisation (Fig. 3 [B]). The application allows the user to select the
information that he / she wants to hear by the filters (Fig. 3 [C]), which improves
the understanding process between what is listened and what is shown.
Besides the previous features, the application offers, as a form to navigate
through the poem, three types of zoom, whose information has different levels
of specificity (Fig. 4). Therefore, the user can select the section to be sonified:
1. Zoom 0: Where the user can listen to the sonification of the whole
poem and only the higher level divisions of the poem are sonified.
The sound result in this zoom has a total of 80 bars, which means
that each bar represents 13 stanzas of the poem.
2. Zoom 1: This type of zoom is available after selecting a canto,
subnarrative or bigger episode, which is sonified. The difference
between this and the previous zoom is that at this level, the presence
of each plan is sonified instead of its intensity. The scale used is eight
stanzas of the selected excerpt per bar.
3. Zoom 2: Level of higher detail, which is activated after the choice
of a specific episode and offers a sonification of the selected passage.
The scale applied at this level is one stanza to one bar.
in the application.
The link in Fig. 5 has a demonstration of the application’s functionalities being
3.2. Technologies and resources
The development of all these functionalities demanded the use of different tech-
nologies that were able to communicate with each other. In this way, we used
the Processing language to create the interface presented, to control the action
of the remaining software and to establish the communication between the user
and the other tools. At this level, the connection to the database is identified via
the BezierSQLib library, which allows the access to the information in the data-
base by SQL. This data is then processed in Processing and sent to Max / MSP
using the oscP5 library. At this point, a Max / MSP patcher is responsible for the
sonification, according to the information received. This process involves the
creation of text files that store the sonification inputs. After this, the patcher
plays the sound result of the subdivisions of the poem and sends the MIDI notes
of the remaining information to be played through Ableton Live. Max is also res-
ponsible for sending the information required by Processing to construct the
labels and the visual component. Ableton Live is the MIDI sequencer respon-
sible for the reproduction of the narrators and narrative plans. It distributes the
Demonstration of the final
221 information received by Max/MSP for the respective MIDI channels. We also
used a database of sound files, Soundtracks, datasets for music and emotion,
as a resource (Eerola and Vuoskoski 2010). This dataset is an archive of movie
sound samples classified in a set of emotions. These samples are applied in the
sonification of the subdivisions of the poem.
4. SONIFICATION PROPOSAL
The sonification model applied follows essentially two sonification techniques:
auditory icons and parameter mapping sonification. The first one takes advantage
of the association of information with familiar sounds and the last one consists
in the association of information to different sound characteristics (Barrass and
Kramer 1999). The final sonification has 120 bpm and a variable number of bars,
according to the dimension of the selected excerpt. The link in Fig. 6 has a de-
monstration of the sonification results for zoom 0.
4.1. Rhyme scheme
The creation of the harmonic structure of the output music is based on a simple
model of 8 bars. This pattern is mapped from the rhyme scheme of the poem (a
b a b a b c c): a is mapped to C major, b with F major and c with G major (Fig. 7).
The result is a simple cyclic chord progression (C F C F C F G G), one chord/bar,
which results in a music in C Major, with no modulations.
Sonification of the
zoom 0 (https://vimeo.
of the Sonification.
222 4.2. Narrators
The narrators’ interventions in The Lusiads are transformed into sound with the
use of parameter mapping. Through the whole book, there are 43 different nar-
rators that were divided, for the purpose of the sonification, into six types: main,
secondary, mythological, crew, Portuguese and foreign. Each of these types is
associated with a different instrument, respectively, tuba, flute, trombone, oboe
and horn. Each instrument plays the song during the intervention of the cor-
respondent narrator type. The melody played at this stage follows the pattern
described earlier and is computed at the bar level: for each block, a variable
sequence of 10 notes (six eighth notes and four sixteenth notes) of the chord’s
scale is generated. Therefore, the bar has the same size of the poem verse, 10
notes to 10 syllables. At the visual level, each bar of the narrator is represented
by a blue circle, whose position and colour change according to the rhyme and
the narrator type, respectively (Fig. 3 [D]). Although this information is sonified
at every zoom, the sound results change because with the increase of the detail
is possible to hear narrators with minor interventions.
4.3. Narrative Plans
Narrative plans are also transformed into sound with the application of parameter
mapping. The four plans — Travel, Gods, Portugal History, Poet — are associated,
each one, with a different percussion instrument — conga bongo, xylophone, clog
box and clavestine. Each instrument follows a different rhythmic pattern, repea-
ted at every bar. The plans are also represented by a diamond, whose colour
changes with the type of plan (Fig. 3 [E]).
At zoom 0, the intensity of each plan, in other words, the variation of its num-
ber of stanzas in each canto, is mapped to the volume of the correspondent
sound and shown by the variation of the diamond size. At the other levels, the
presence of each plan in the story is mapped, so each instrument plays when
the corresponding plan exists.
As shown in Fig. 2, the subdivisions of the poem were divided into three groups:
subnarratives, episodes and emotions. The method followed here was auditory
icons, so each subdivision is assigned with a familiar soundtrack. Every group in
this section is represented by a stroke, whose colour scheme changes with the
type represented (Fig. 3 [F]).
The subnarratives of the poem are played only in zoom 0, before the selection
of a canto or subnarrative. The poem has eight subnarratives (Fig. 1) that were
grouped into three types to favour the sonification process: Dedication, Travel
and History. Each subnarrative group is associated with a music sample that
resembles it, which plays during the corresponding event.
The episodes are sonified in zoom 1, after the selection of a canto, subnarrative
or a longer episode. The poem can be divided in 56 episodes that, in order to
be mapped, were grouped into seven types: fantastic, geographic, opinion, reign,
warlike, lyric and naturalist. Each type has a similar thematic sound sample that
is played during the specific episode.
The emotions presented in every episode were collected and stored in an emo-
tion scale: anger, concern, tranquillity and love. This information is mapped into
sound in zoom 2, after the selection of a specific episode. Each emotion is as-
signed to a correspondent sample of the database resource, which is played
when the emotion is identified.
For last, the poem also contains prophecies that tell historical Portuguese events
that occur after the time of the narrative. These prophecies are sonified in the
last zoom with the application of reverb, while they are present, which gives an
echo sensation. The goal in this case was to give the sensation of a prophetic
and magical environment because these sections of the poem are mainly con-
trolled by mythological figures.
5. CONCLUSIONS AND REFLEXIONS
Sonification is not a traditional mapping method, which leads to the existence
of issues and challenges to overcome for the improvement of this area. How to
choose the best sonification technique for a dataset? Can all kinds of data be
sonified? Can a sonification system be understood without a visual component?
How to make the sound representation easy-to-understand? Will the technology
be prepared for the integration and expansion of Sonification? These are some of
the issues that emerge from the analysis of the Sonification field. It is necessary
to investigate and explore these questions in order to establish suitable method-
ologies and identify the most promising techniques to address them.
The principal purpose of the project presented herein was to explore Sonification
in an area where its application is not typical: poetry. This work allowed us to ex-
periment and test different approaches of sonification until we found one that we
believed to be suitable for the project. During this process, we were confronted
with some challenges: the creation of the dataset of the poem, the representation
of so many levels of information, the choice of the instruments and samples to
be used, the technological limitations and the development of a visual guide to
the sonification. The solution we found to overcome these issues was the crea-
tion of an interactive sonification system that allows the user to explore the poem
in a customised way and at his / her own time. This type of interactive systems
expands the sonification process because it allows the user to compare different
sections of the story and get conclusions. The user can choose the section that
he / she would like listen to and the type of information he / she wants to receive,
which turns the understanding process of the poem easier. In order to test the
224 usefulness of this product, some usability tests were applied that, although are
not able to fully test the sonification process, were able to understand that the
users can create an analysis process from the sonification results. These tests
cannot evaluate in detail the user experience, which will be an area to fully ana-
lyse in future work, with more suitable techniques. However, it was possible to
understand that users were capable of getting conclusions about the story of
the poem and create correlations between different types of information and
sections of the poem.
The goal of this project was to create an application that can communicate
information and also be able to produce a new vision of a book so well-known in
the Portuguese Literature. Besides, the results of this work allow us to understand
that sound can communicate information and how it can relate with language.
This project reinforces the idea that music and language are connected and share
the same roots. Poetry emerges between these two areas and joins character-
istics of both fields, so it makes sense to explore the poetry in Sonification, to
take advantage of the sound as a communication tool and to improve the under-
standing process of complex and long poetic narratives.
Barrass, Stephen, and Gregory Kramer.
‘Using Sonification.’ Multimedia Systems 7,
no. 1 (1999): 23-31.
Ben-Tal, Oded, and Jonathan Berger.
‘Creative Aspects of Sonification.’ Leonardo
37, no. 3 (2004): 229-33.
Camões, Luís de. ‘Os Lusíadas’. Porto, Por-
tugal: Porto Editora, 2011.
Coelho, Ângela, Pedro Martins, and Amílcar
Cardoso. ‘A Musical Sonification of the
Portuguese Epopee.’ Paper presented at the
MUME, Paris, France, 2016.
Eerola, Tuomas, and Jonna K Vuoskoski.
‘A Comparison of the Discrete and Dimen-
sional Models of Emotion in Music.’
Psychology of Music 39, no. 1 (2010): 18-49.
Frazier, Sarah. ‘Sonification: Data Like
You’ve Never Heard Before.’ (2013).
Gaio, Tiago. ‘Os Lusíadas.’
Hermann, Thomas, and Andy Hunt.
‘Guest Editors’ Introduction: An Introduction
to Interactive Sonification.’ IEEE Multimedia,
Hermann, Thomas, Andy Hunt, and John
G. Neuhoff. ‘Introduction.’ Chap. 1 In The
Sonification Handbook, edited by Thomas
Hermann, Andy Hunt and John G. Neuhoff,
1-6. Berlin, Germany: Logos Verlag, 2011.
House, Brian, and Katherine Brooks.
‘Quotidian Record: Artist Brian House
Turns Tracking Data into a Vinyl Music
Experience.’ (2013). http://www.huffing-
Huang, Chih-Fang, Hsiang-Pin Lu, and
Jenny Ren. ‘Algorithmic Approach to Sonifi-
cation of Classical Chinese Poetry.’
Multimedia Tools and Applications 61,
no. 2 (2011): 489-518.
Kramer, Gregory, Bruce Walker, Terri Bone-
bright, Perry Cook, John H Flowers, Nadine
Miner, and John Neuhoff.
‘Sonification Report: Status of the Field and
Research Agenda.’ Faculty Publications, De-
partment of Psychology (2010): Paper 444.
Lerdahl, Fred. ‘The Sounds of Poetry Viewed
as Music.’ Annals of the New York Academy
of Sciences 930 (2001): 337-54.
Michelli, Regina. ‘Os Lusíadas, De Luís De
Camões: Breve Viagem Estrutural.’
Augustus, 2003, 39-48.
Minciacchi, Diego, and David Rosenboom.
‘Sonification, Perceptualizing Biological In-
formation.’ (2015). http://journal.frontiersin.
Pais, Amélia Pinto. ‘Para Compreender
Os Lusíadas.’ 2ª ed. Porto, Potugal: Areal
Park, Sihwa, Seunghun Kim, Samuel Lee,
and Woon Seung Yeo. ‘Online Map Interface
for Creative and Interactive Music-Making.’
Paper presented at the New Interfaces
for Musical Expression (NIME), Sydney,
Ren, Jenny; Winsor, Phil; Huang, Chih-Fang.
‘Developing Chinese Style Algorithmic Com-
position Using Markov Chains — from the
Classical Chinese-Poetry Perspective.’ Paper
presented at the International Computer
Music Conference (ICMC), Copenhagen,
Sena, Jorge de. ‘A Estrutura De “Os Lusía-
das” E Outros Estudos Camonianos E De
Poesia Peninsular Do Século XVI’. Obras
De Jorge De Sena. 2 ed. Lisboa, Portugal:
Edições 70, 1980.
Vanhemert, Kyle. ‘Artist Turns a Year’s
Worth of Tracking Data into a Haunting
Record.’ (2013). https://www.wired.com/
Vicinanza, Domenico. ‘Sonification: Repre-
senting Data through Music.’ (2014 ).