Conference PaperPDF Available

Abstract and Figures

Sonification is a field that has gained importance in the last few years due to the technological development in the areas of sound synthesis and manipulation. This area can be seen as the auditory counterpart of Information Visualization and it has real meaning when the data sets are too ample and complex to be seen graphically. This paper describes work in progress on the development of a musical sonification of The Lusiads, a Portuguese epic poem by Luís de Camões. This work intends primarily to create and explore new ways of reading the Portuguese epopee.
Content may be subject to copyright.
A Musical Sonification of the Portuguese Epopee
ˆ
Angela Coelho, Pedro Martins, Am´
ılcar Cardoso
CISUC, Informatics Engineering Department, University of Coimbra
aacoelho@student.dei.uc.pt, {pjmm, amilcar}@dei.uc.pt
Abstract
Sonification is a field that has gained importance in the
last few years due to the technological development in
the areas of sound synthesis and manipulation. This area
can be seen as the auditory counterpart of Information
Visualization and it has real meaning when the data sets
are too ample and complex to be seen graphically. This
paper describes work in progress on the development
of a musical sonification of The Lusiads, a Portuguese
epic poem by Lu´
ıs de Cam˜
oes. This work intends pri-
marily to create and explore new ways of reading the
Portuguese epopee.
Introduction
In the late 50s, sound waves were digitally represented for
the first time (Duarte 2014). This event made sound synthe-
sis and sound composition possible by a computer, which
changed both the music and computing world and led to the
emergence of the Sonification area.
Sonification can be briefly described as a subtype of au-
ditory display that uses non-verbal sounds to represent in-
formation (Barrass and Kramer 1999; Kramer et al. 2010;
Minciacchi and Rosenboom 2015; Neuhoff 2011). It con-
sists in the transformation of data and its relationships into
acoustic signals (Frazier 2013). This area assumes that the
sound is capable of representing data and providing support
for information processing and analysis to a form that can
be understood by the user (Frazier 2013; Hermann and Hunt
2005; Kramer et al. 2010; Minciacchi and Rosenboom 2015;
Park et al. 2010; Vicinanza 2014).
This field is very interdisciplinary because it joins differ-
ent concepts, such as perception, acoustic, design, arts and
engineering, which leads to the necessity of connection be-
tween all these areas to develop auditory systems (Kramer
et al. 2010). The huge progress of computers in the last few
years led to the generation of large amounts of data and
changed the way we learn, communicate and explore the
world around us (Hermann and Hunt 2005; Kramer et al.
2010).
Although Information Visualization showed up to assist
researchers in the analysis of large volumes of data, its
This work is licenced under Creative Commons ”Attribution 4.0
International” licence, the International Workshop on Musical
Metacreation, 2016, (www.musicalmetacreation.org).
techniques become sometimes insufficient due to the expo-
nential increase of information that the user wants to ac-
cess (Hermann and Hunt 2005; Kramer et al. 2010). In this
way, sound can be a solution to this phenomenon: New
forms of representing larger and dynamic data can be cre-
ated through Sonification, without becoming too complex
for the user understanding (Kramer et al. 2010; Minciacchi
and Rosenboom 2015). Sonification provides two new di-
mensions to transmit data: the sound itself (and its character-
istics) and the idea of time. A sonification project can show
huge amounts of data in just a few minutes, giving a general
overview of the information, as well as the existing trends
and patterns. Also, the interdisciplinarity of this field pro-
vides conditions to use sonification to improve visualization
systems. Powerful audio technologies have also been devel-
oped, which, more than ever, show the importance of sound
as a data representation technique (Kramer et al. 2010).
The work presented herein explores the field of Sonifi-
cation applied to poetry. The project intends to transform
the characteristics of a well-known Portuguese poetry book,
The Lusiads, into music. This project was motivated by the
will to create a different sonification process, which can take
advantage of the sound characteristics to provide poetry in-
formation. In this way, it will be created a musical output
that reflects not only the sequential structure of the poem,
but also its external structure. To better explain the work in
progress, the remainder of this paper starts by providing an
overview of works in Sonification, namely in Musical Soni-
fication. This is followed by a description of the main as-
pects of The Lusiads. Then, there is a description of the gen-
eral features of the project and what has already been devel-
oped. The paper ends with a conclusion, where the Sonifica-
tion area and the results obtained so far with this work are
objects of reflection.
Applications of Sonification
The main area where Sonification is used is in scientific re-
search: engineering, construction of artificial models, seis-
mic or medical studies (Barrass and Kramer 1999; Frazier
2013; Kramer et al. 2010). Sonification has also a great
importance in areas where vision cannot acquire informa-
tion, such as manufacturing and production control, traffic
and visual impairment situations (Barrass and Kramer 1999;
Begault et al. 1996; Kramer et al. 2010; Krygier 1994).
MUME 2016 - The Fourth International Workshop on Musical Metacreation, ISBN #978-0-86491-397-5
Sonification is mainly applied to larger, complex and dy-
namic datasets (Frazier 2013; Kramer et al. 2010; Minci-
acchi and Rosenboom 2015), as it can provide alternative
ways of representing information, where the size is reduced
without a significant loss of information and various dimen-
sions can be simultaneously represented (Hermann and Hunt
2005; Kramer et al. 2010).
This work explores a Musical Sonification, an approach
in Data Sonification that intends to take advantage of the
act of listening to music and the aspects involved in this
action (Ben-Tal and Berger 2004). It is possible to use the
changes over the course of a music such as frequency, am-
plitude, timbre to create a mental image that can be used to
represent data.
There are a few projects that explored this type of soni-
fication. Between 1999 and 2000, Marty Quinn created a
musical performance, The Climate Symphony, where the
data of an ice core extracted in Greenland was transformed
into sound (Quinn 2001). This ice block provided infor-
mation of the climate history for the past 110 000 years,
which after being sonified allowed to understand the cli-
mate evolution in the world. Another interesting project is
the sonification system, Text-to-Music, which intends to of-
fer a new way for helping people to appreciate Chinese
poetry (Huang, Lu, and Ren 2011). This system maps the
characteristics, dynamics and relations of the verses into
durations of musical elements (Huang, Lu, and Ren 2011;
Ren, Phil, and Huang 2007). This allows to sonify the poem
according to its pronunciation, which is an aspect that has a
great importance in the analysis of Chinese poetry (Huang,
Lu, and Ren 2011). Hard Data is also an important project in
this area because it musically explores the statistical data of
the war between Iraq and USA (DuBois 2009a; 2009b). This
sonification system was developed by Luke Dubois in 2009
and was initially a public timeline to allow anyone to create
his/her own musical version of these data (DuBois 2009a).
However, during the development of the project, Dubois cre-
ated a website with his own version of the data sonification
and then created a string quartet version for the Mivos quar-
tet, which focuses on the casualty statistics. Another explo-
ration in this area is the Quotidian Record project, where
Brian House creates a musical composition that features a
continuous year of his location-tracking data (House 2012b;
2012a). Brian develops an algorithm that identifies the loca-
tions by latitude and longitude and associates each one with
a musical duration or harmonic relation. The musical result
is recorded on a vinyl that contains the markings of the time
and the names of cities to which he traveled.
A Sonification of The Lusiads
The project that is being developed intends to transform the
main features of the poem, The Lusiads, into sound. We in-
tend to build a software application that allows the user to
experience the poem in an innovative way and to obtain dif-
ferent types of information depending on the custom navi-
gation performed by the user.
Before introducing the sonification model, a description
of The Lusiads structure and main characteristics will be
given.
The Lusiads
The Portuguese epic poem The Lusiads by Lu´
ıs de Cam˜
oes
(1524-1580) has been the subject of numerous analyses over
time (Sena 1980). The poem was first presented by Lu´
ıs de
Cam˜
oes to Dom Sebastian I to whom he dedicates the poem,
and was published in 1572.
The poem is inserted in the epic genre, a literary genre that
comes from the Greek-Latin Antiquity (Cam˜
oes 2011; Gaio
nd; Pais 1994). This book is like Virgil’s Aeneid or Homer’s
Iliad and Odyssey, i.e., an epopee, as it is written in verse, in
a high style, and intends to magnify the achievements of the
heroes, in this case, the Portuguese people.
Cam˜
oes narrates the travel of Vasco da Gama to India and
through this story he tells the Portuguese deeds and extols
the strength of his people (Sena 1980).
Poem Characteristics
The poem, in terms of external structure, has a total of
1102 stanzas of eight verses each and is divided into 10
cantos (Gaio nd; Pais 1994). The verses are decasyllables
because they contain ten syllables and the stanza has a
rhyme scheme constant throughout the all poem (Fig. 1),
consisting of crusade rhymes in the first six verses and
paired rhymes in the last two – abababcc(Gaio nd;
Pais 1994).
Figure 1: Excerpt of a stanza and the rhyme scheme.
The internal structure of the poem follows the epic genre
rules (Gaio nd; Pais 1994):
1. Proposition, where the poet presents what is going to sing;
2. Invocation, a section where the poet asks for support and
protection to deities (nymphs of the Tagus river);
3. Dedication, where the author dedicates the poem to some-
one, in this case Dom Sebastian I;
4. Narration, which represents the story itself.
The Lusiads can be also divided into four narrative plans,
which intersect over the narrative (Fig. 2) (Michelli 2003;
Sena 1980):
1. Travel plan, that consists in the narration of the events of
the trip from Lisbon to India;
Figure 2: Structure of the poem.
2. Gods’ plan, that includes the gods’ interventions, inter-
sected with the travel plan, where the gods make decisions
that affect the fate of the Portuguese people;
3. Portugal History plan, which is the narration of Portugal’s
history by Vasco da Gama to the King of Malindi;
4. Poet plan, essentially located at the end of each canto,
where Cam˜
oes reflects about the state of the world;
Different characters are responsible for the narration. The
most frequent ones are Lu´
ıs de Cam˜
oes, the main narrator,
and Vasco da Gama.
Dataset
The characteristics to be sonified can be divided into three
different areas (Fig. 3):
1. External structure, where is intended to explore the verses
characteristics and the rhyme scheme for creating the
rhythm;
2. Internal structure: exploration of the subnarratives of the
poem, the intersection between the different narrative
plans and the subunits or episodes that divide the poem;
3. Sequential structure, where is intended to identify the dif-
ferent narrators of the book and also analyze the different
story times, since there are prophecies to tell facts that
occurred afterwards.
Figure 3: Dataset divided in areas.
General features
The sonification system under development has interactiv-
ity as a core functionality. It is intended that the user can
navigate in the system to get different types of information
depending on what he/she exploits, which enables a custom
experience of the Portuguese epopee. In this way, the ap-
plication will offer three types of zoom, whose information
will have different levels of specificity:
1. Zoom 0: Where the user can listen to the whole book and
only the higher level divisions of the poem will be soni-
fied.
2. Zoom 1: This type of zoom will be available after select-
ing a canto or subnarrative, which will be sonified.
3. Zoom 2: Level of higher detail, which is activated after the
choice of a subunit or a specific episode and will offer a
sonification of the selected excerpt.
The project will also have a visual component to work as a
label and to guide the user through the sonified information,
which will improve the process of sound visualization.
First Prototype
At this stage of development, there is an implementation of
zoom 0, which was done using Processing and the MidiBus
library to create the MIDI notes and MIDI messages and to
establish the connection with a sequencer (currently Able-
ton Live Lite 9). In zoom 0 only three types of information
are used (Fig. 4): the main narrators (Lu´
ıs de Cam˜
oes and
Vasco da Gama), the frequency of the narrative plans during
the poem and the presence and intersection of the various
subnarratives along the text.
The output at this zoom level is a sound file of 120 bpm,
with 80 bars. The harmonic structure is cyclic, based on a
simple pattern of 8 bars mapped from the rhyme scheme of
the poem (abababcc): ais mapped to C major,bwith
F major and cwith G major. The result is a cyclic chord
progression (CFCFCFGG), one chord/bar, which results
in a music in C Major, with no modulations.
Figure 4: Sonified characteristics.
The melody is computed at the bar level: for each bar, a
variable sequence of 10 notes (6 eighth notes and 4 sixteenth
notes) of the chord’s scale is generated. The first and last
notes are always the tonic of the scale. This melody is played
by two different melodic instruments, which correspond to
the two main narrators:
1. Lu´
ıs de Cam˜
oes - Tuba;
2. Vasco da Gama - Flute.
The four narrative plans are associated, each one, with a dif-
ferent percussion instrument, playing during the entire piece.
Each instrument follows a different rhythmic pattern. The
volume of the sound changes according to the variation of
the number of stanzas that the respective plan has in each
canto:
1. Travel Plan – Bongo;
2. Gods Plan – Xylophone;
3. Portugal History Plan – Clog boxes;
4. Poet Plan – Claves.
The subnarratives were grouped into three groups:
1. Dedication, which includes the Introduction and the Con-
clusion of the poem.
2. Travel, which includes Travel of Mozambique to Malindi,
Travel of Bethlehem to Malindi, Travel of Malindi to Cali-
cut, India and Return.
3. History, which includes Portugal’s History.
Each group is associated with a harmonic instrument, which
plays, in the presence of the specific subnarrative, the chords
of the corresponding rhyme (awith the Cchord, bwith the
Fchord and cwith the Gchord):
1. Dedication – Harp;
2. Travel – Violin;
3. History – Cello.
In this way, the goal in zoom 0 is the user to be able to
distinguish the different narrators, the intensity of each nar-
rative plan and to understand the evolution of the story by
the intersection of the subnarratives.
The sound file associated with zoom 0 is available
at https://soundcloud.com/ngela-coelho-
438647153/sonification-lusiadas and it is
recommended to follow the sonification with Fig. 4.
For the other zooms the goal is to follow a similar system
of division by types of instruments and explore the other
characteristics of the poem such as the intersection of the
narratives plans, the types of subunits, the other narrators
and the prophecies.
Reflection and Conclusions
How to choose the best sonification technique for a dataset?
Can all kinds of data be sonified? Can a sonification sys-
tem be understood without a visual component? How to
make the sound representation easy-to-understand? Will the
technology be prepared for the integration and expansion of
Sonification?
These are some of the issues that emerge from the anal-
ysis of the Sonification field. It is necessary to investigate
and explore these questions in order to establish suitable
methodologies and identify the most promising techniques
to address them. During the development of this project was
realized that in most cases, the questions that appear during
a sonification project are answered by the type of data that is
being sonified. Each kind of information requires a different
approach to achieve the best possible result.
The work presented herein intends to explore the Sonifi-
cation area in a field where its application is not common:
poetry, more specifically the epic poetry. The main goal is to
transform The Lusiads into different types of sound accord-
ing to the level of detail chosen. Although this is a work in
progress, the project is allowing us to experiment and test al-
ternative configurations for the sonification. For instance, it
is allowing us to understand the number of different instru-
ments that the sonification should use in the poem represen-
tation, how to use them and how to represent so many levels
of information. While the proposed zoom 0 meets the goals
that were originally established, it is mandatory to analyze
how the project will be received by the user, the best way to
communicate and validate it. By the end of this project, it is
expected that with the final application the sound result will
be better understood, due to the interaction system that will
be created for the user navigation and the visual component
that will help to follow the auditory product.
To be able to prove the usefulness of the application, it is
also imperative to validate it with final users. We intend to
assess whether the different levels of detail of the story are
perceived from the sonification and to what point relevant
information is retained.
This application is intended to not only be an innovative
sonification process, but also to be a design product, which
can communicate information and also be able to produce a
new vision of a book so well-known in the Portuguese liter-
ature.
References
Barrass, S., and Kramer, G. 1999. Using sonification. Mul-
timedia Syst. 7(1):23–31.
Begault, D. R.; Wenzel, E. M.; Shrum, R.; and Miller, J.
1996. A virtual audio guidance and alert system for com-
mercial aircraft operations. In Proceedings of the 3rd Inter-
national Conference on Auditory Display (ICAD’96).
Ben-Tal, O., and Berger, J. 2004. Creative aspects of sonifi-
cation. Leonardo 37(3):229–233.
Cam˜
oes, L. d. 2011. Os Lus´
ıadas. Porto Editora.
Duarte, M. L. 2014. A contribuic¸˜
ao de Max Mathews para
aM
´
usica Computadorizada. In Proceedings of the Simpom.
DuBois, R. L. 2009a. Hard data. Retrieved from
http://lukedubois.com/.
DuBois, R. L. 2009b. Hard data. Retrieved from
http://archive.turbulence.org/Works/harddata/.
Frazier, S. 2013. Sonification: Data like you’ve never heard
before. Retrieved from Earthzine website:
http://earthzine.org/2013/07/23/sonification-data- like-
youve-never- heard-before/.
Gaio, T. n.d. Os Lus´
ıadas. Retrieved from
http://oslusiadas.no.sapo.pt/indice.html.
Hermann, T., and Hunt, A. 2005. Guest editors’ introduc-
tion: An introduction to interactive sonification. IEEE Mul-
timedia 12(2):20–24.
House, B. 2012a. Quotidian record. Retrieved from
http://brianhouse.net/works/quotidian_record/.
House, B. 2012b. Quotidian record. Retrieved from
http://blog.brianhouse.net/post/31228661266.
Huang, C.-F.; Lu, H.-P.; and Ren, J. 2011. Algorithmic
approach to sonification of classical chinese poetry. Multi-
media Tools and Applications 61(2):489–518.
Kramer, G.; Walker, B.; Bonebright, T.; Cook, P.; Flowers,
J. H.; Miner, N.; and Neuhoff, J. 2010. Sonification report:
Status of the field and research agenda. Psychiatry and Psy-
chology Commons.
Krygier, J. B. 1994. Sound and geographic visualization. In
Visualization in modern cartography. Oxford, U.K.: Perga-
mon. 149–166.
Michelli, R. 2003. Os lus´
ıadas, de lu´
ıs de cam˜
oes: Breve
viagem estrutural. Augustus 8(16):39–48.
Minciacchi, D., and Rosenboom, D. 2015. Sonification,
perceptualizing biological information. Retrieved from
Frontiers website:
http://journal.frontiersin.org/researchtopic/3579/
sonification-perceptualizing- biological-information-
overview.
Neuhoff, J. G. 2011. Perception, cognition and action in
auditory displays. In Hermann, T.; Hunt, A.; and Neuhoff,
J. G., eds., The Sonification Handbook. Berlin, Germany:
Logos Verlag. 63–85.
Pais, A. P. 1994. Para compreender Os Lus´
ıadas (2a ed.).
Porto, Portugal: Areal Editores.
Park, S.; Kim, S.; Lee, S.; and Yeo, W. S. 2010. Online
map interface for creative and interactive music-making. In
Proceedings of the New Interfaces for Musical Expression
(NIME).
Quinn, M. 2001. Research set to music: The climate sym-
phony and other sonifications of ice core, radar, DNA, seis-
mic and solar wind data. In Proceedings of the 7th Interna-
tional Conference on Auditory Display (ICAD’01).
Ren, J. W.; Phil; and Huang, C.-F. 2007. Developing chinese
style algorithmic composition using markov chains from the
classical chinese-poetry perspective. In Proceedings of the
International Computer Music Conference, 214–219.
Sena, J. d. 1980. A estrutura de ”Os Lus´
ıadas” e outros
estudos camonianos e de poesia peninsular do s´
eculo XVI
(2a ed.). Lisbon, Portugal: Edic¸ ˜
oes 70.
Vicinanza, D. 2014. Sonification: Representing data through
music. Retrieved from BBC News website:
http://www.bbc.co.uk/news/magazine-25975712.
... In the poetry field, Coelho, Martins & Cardoso [3] created a A Musical Sonification of the Portuguese Epopee, specifically of The Lusiads, by Luís de Camões. It is an interactive sonification where the user can explore the poem by choosing di↵erent levels of "zooming", listening to it as a whole, as a canto/subnarrative or a specific episode and therefore customising the experience. ...
Conference Paper
Full-text available
Sonification is a scientific field that seeks to explore the potential of sound as an instrument to convey and interpret data. Its techniques have been developing significantly with the growth of technology and supporting hardware and software, which have spread in our daily environment. This allowed the establishment of new communication tools to share information, opinion and feelings as part of our daily routine. The aim of this project was to unite the social media phenomena with sonification, using Twitter data to extract user’s emotions and translate them into musical compositions. The focus was to explore the potential of music in translating data as personal and subjective as human emotions, developing a musically complex and captivating mapping based on the rules of Western Music. The music is accompanied by a simple visualization, which results in emotions being heard and seen with the corresponding tweets, in a multimodal experience that represents Twitter’s emotional reality. The mapping was tested through an online survey, and despite a few misunderstandings, the results were generally positive, expressing the efficiency and impact of the developed system.
... This project transforms the characteristics of a popular and relevant Portuguese poem, The Lusiads, into music. A preliminary sketch of this work was presented at the 4th International Workshop on Musical Metacreation (MUME), which allowed to test different approaches for the sonification: how many instruments should be used and how to use them and how to represent different levels of information (Coelho, Martins, and Cardoso 2016). ...
Conference Paper
Full-text available
Sonification has gained importance in the last few years due to the technological development in the areas of sound synthesis and manipulation. This area allows to transform and understand large data sets that can be too ample and complex to analyse without these tools. This paper describes the development of a musical sonification project applied to the area of Literature that maps the data of the Portuguese epic poem, The Lusiads by Luís de Camões, into sounds. This work intends to show that Sonification can be applied to different and not so common areas and create new ways of reading and understanding texts, in this case, a well-known and important poem from Portuguese Literature.
Article
Full-text available
A goal of sonification research is the intuitive audio representation of complex, multidimensional data. The authors present two facets of this research that may provide insight into the creative process. First, they discuss aspects of categorical perception in nonverbal auditory scene analysis and propose that these characteristics are simplified models of creative engagement with sound. Second, they describe the use of sonified data in musical compositions by each of the authors and observe aspects of the creative process in the purely aesthetic use of sonified statistical data.
Conference Paper
Full-text available
In this paper, we discuss the musical potential of COM-Path -an online map based music-making tool -as a novel and unique interface for interactive music composition and performance. COMPath provides an intuitive environment for creative music making by sonification of georeferenced data. Users can generate musical events with simple and familiar actions on an online map interface; a set of lo-cal information is collected along the user-drawn route and then interpreted as sounds of various musical instruments. We discuss the musical interpretation of routes on a map, review the design and implementation of COMPath, and present selected sonification results with focus on mapping strategies for map-based composition.
Article
RESUMO: Esta obra narra uma história magnífica. O livro refere-se a uma viagem através de "mares nunca de antes navegados": a descoberta do caminho marítimo para as Índias pelos portugueses. Estruturalmente, Os Lusíadas pode ser dividido em quatro pla-nos: ideológico (abarcando os pensamentos e comentários do Poeta), referencial (sobre a viagem portuguesa), mítico (focalizando os deuses da mitologia romana) e histórico (abor-dando a História de Portugal). Este artigo pretende apresentar um breve comentário sobre a obra e uma visão estrutural dos dez cantos que a compõem. ABSTRACT: This work tells a magnificent history. The book concerns about the trip through "seas never navigated before": the maritime way to Indian's discovery by Portu-guese men. Structurally. Os Lusíadas can be divided into four plans: ideological (showing the Poet's thoughts and comments), referential (about the Portuguese trip), mythological (focusing the Gods of roman mythology) and historical (about Portuguese History). This paper intends to present a brief comment about the book and a structural view of the ten cantos which compose it. Palavras-chave: Literatura Portuguesa, Luís de Camões, Os Lusíadas, análise estrutural.
Article
The classical Chinese poetry is a remarkable form of art in traditional Chinese character. However, it is difficult for people who are unfamiliar with ancient Chinese to experience the artistic content of the poetry. In this study, a sonification scheme, Tx2Ms (Text-to-Music), is proposed to extract the poetry features between lines in verses; moreover, dynamics and interval relations are modeled to map those features to the movement of multi-dimensional musical elements such as durations. This conversion is based on poetry intonation and acoustic analysis of the pronunciations of poems; and a stochastic compositional algorithm is created by applying a Markov chain. In addition, the best pentatonic mode for a specific poem is recommended according to the formants analysis. Therefore, the sonification of classical Chinese poetry not only provides a novel way for people to appreciate Chinese poetry but also enriches the state of mind and imagery in the delivery process, and the experiment results show that the proposed system is successfully accepted by most people.
Article
The Climate Symphony and Other Sonifications of Ice Core, Radar, DNA, Seismic and Solar Wind Data is a one-hour performance/presentation of sonification research by Marty Quinn of Design Rhythmics Sonification Research Lab and BAE Systems. It was presented in November 2000 at the National Science Foundation at the invitation of the Director's Office of Public Affairs and the Office of Polar Programs and was warmly received. This paper describes the Climate Symphony portion of the presentation in detail.