Content uploaded by Dimitrios Vassilakis
Author content
All content in this area was uploaded by Dimitrios Vassilakis on Jun 07, 2019
Content may be subject to copyright.
“JAZZ MAPPING” AN ANALYTICAL
AND COMPUTATIONAL APPROACH TO
JAZZ IMPROVISATION
Dimitrios Vassilakis
Anastasia Georgaki
Christina Anagnostopoulou
Department of Music Studies
National and Kapodistrian
University Of Athens
info@dimitriosvassilakis.com
Department of Music Studies
National and Kapodistrian
University of Athens
georgaki@music.uoa.gr
Department of Music Studies
National and Kapodistrian
University Of Athens
chrisa@music.uoa.gr
ABSTRACT
“Jazz mapping" is a multi-layered analytical approach to
jazz improvisation. It is based on hierarchical segmenta-
tion and categorization of segments, or constituents, ac-
cording to their function in the overall improvisation. The
approach aims at identifying higher-level semantics of
transcribed and recorded jazz solos. At these initial stag-
es, analytical decisions are rather exploratory and rely on
the input of one of the authors and experienced jazz per-
former. We apply the method to two well-known solos,
by Sonny Rollins and Charlie Parker, and discuss how
improvisations resemble story-telling, employing a broad
range of structural, expressive and technical tools, usually
associated with linguistic production, experience, and
meaning. We elucidate the implicit choices of experi-
enced jazz improvisers, who have developed a strong
command over the language and can communicate ex-
pressive intent, elicit emotional responses, and unfold
musical “stories” that are memorable and enjoyable to
fellow musicians and listeners. We also comment on po-
tential artificial intelligence applications of this work to
music research and performance.
1. INTRODUCTION
1.1 Goals: The project aims at advancing our current un-
derstanding of jazz improvisation and, by extension, of
musical creativity. It introduces and applies a musical
language-mapping scheme that can support the creation
of a large annotated corpus of transcribed solos, assist in
the pedagogy of improvisation and serve as a reference
point in human and artificial musicianship research. The
utility of the approach may also extend to research in
other domains that explore hierarchical sequential data
and real-time decision making, such as generative model-
ing of natural language and speech.
1.2 Related work: Formal music analysis is usually
concerned with breaking the musical surface into
segments based on similarity, and with studying how
these are put together syntactically as a piece of music
unfolds in time, thus attributing internal cohesion [1]
Semiotic analysis (paradigmatic and syntagmatic) is a
typical example of a method which categorizes segments
according to similarity [2] Paradigmatic analysis has been
computationally modeled in the past [3], [4].
At the same time, a significant body of research
literature addressing jazz improvisation has been
developing over the last couple of decades. This work
includes topics on jazz storytelling [5], including
references on the concept by well known jazz musicians
and scholars. Some [6] introduce the concept of re-telling
to refer to the re-working of a standard, based on a
famous recording of a master, stressing the important
tension between individual voice and tradition. Others [7]
explore machine learning of jazz grammars, using basic
building-blocks or “slopes,” touching upon the antitheses
of abstraction versus vocabulary, and attempting to
codify harmonic tension.
A relevant work that focused on Sonny Rollins’s
thematic improvisation [8] will be explored further,
below.
Weimar’s Jazzomat Research Project [9] has produced
several databases of annotated solos and licks, including
the “Dig That Lick” database. Studies on the use of pat-
terns in jazz [10], [11], have stressed the importance of
auditory and motor patterns organizing into a stored
menu of pattern libraries.
Researchers at the Georgia Institute of Technology
have been developing robotic applications of computer
improvisation [12] that are informing and are being in-
formed by our work.
Francois Pachet in 2001 produced The Continuator,
later developed in the European project MIROR
(mirorproject.eu) [13], focuses on learning sequences by
linear analyses of input patterns and phrases to generate a
response. Improvisations have been generated in real time
based on input of musical sequences [14]. Explorations
on the improvisers’ thought processes during a duo [15]
have attempted to reveal the intent and the scheme or
scenario behind an improvisation. Musical passage
coding as “phrase” and “variation” has been used to
assist a music program to acquire “common sense,” [16],
while a very interesting interview of Ornette Coleman by
Jacques Derrida touches on the relationship between
language and jazz improvisation.1
All the above approaches deal with a structural analysis
of jazz improvisations, thus studying the jazz vocabulary
1http://www.ubu.com/papers/Derrida-Interviews-Coleman_1997.pdf
Interview originally appeared in French in the magazine Les In-
rockuptibles no. 115 (20 aout-2 septembre 1997): 37-40,43.
and syntax, but they are not progressing deeper into the
semantics of the language.
Based on the above approaches, and while we
acknowledge that the topic of semantics in jazz might be
too complex to describe with a formal syntactic analysis,
we make a first attempt in interpreting the various
constituents that result from the analysis, together with
their function and style in the improvisation, expanding
into issues of semantics, syntactical analyses, story telling
and thematic development.
1.3 Proposed outcome: The “jazz mapping” project has
potential implications to machine learning and Artificial
Intelligence (AI) system development. It can provide
means for AI to manage in a human-like way the essen-
tial human tension among past, present, and future char-
acterizing all decision-making. This potential can be real-
ized through “teaching” an AI system the rules that gov-
ern annotation and how these rules dynamically interact
and change when actualized as experienced present or
“now”.
We will begin by identifying and adapting to jazz im-
provisation musical contexts basic human communication
tools/codes, concepts and structures such as: question and
answer/call and response, fragment, lick, phrase, thematic
development, short/long, memorable or abstract, and ref-
erences among phrases. A similar approach can potential-
ly be used to explore concepts such as harmonic tension,
phrasing, articulation, expressiveness, sonic character or
“sound,” etc., to generate jazz solos much like a jazz im-
proviser/storyteller would, using layers of multi-
reference.
A pattern database will be also created as those anno-
tated phrases licks, fragments and patterns will have mul-
tiple uses on “describing” or “outlining” chords and
chord sequences helping to address issues like originality
and personal voice and different approaches of players
like vertical versus linear, voice leading versus modal or
free.
2. THE JAZZ MAPPING APPROACH
2.1 Constituents in syntactic analysis.
In order to analyze an improvisation through mapping we
propose a novel method which consists of the following
levels: Jazz improvisational structural elements and map-
pings, thematic analyses by defining segments, licks and
phrases and annotation of syntax and structure.
In our analysis, we define 3 types of constituents, listed
here by increasing duration and/or complexity:
1. Segment
2. Lick
3. Phrase
Each of the constituents found would carry a tag describ-
ing the function in the improvisation, such as: re-
sponse/answer, reference, or new idea.
2.2 Definitions
Below we attempt a definition for each constituent, bear-
ing in mind that this is not a fully formal approach yet,
therefore the criteria for a constituent to belong to a cate-
gory are not fully explicit, and rely to some extend on the
context of the piece under analysis.
Segment: very short but salient theme, fragment, angu-
lar/linear/long single note, usually one bar (e.g. the the-
matic seed in John Coltrane’s “Love Supreme” Ex.1).
Ex.1
(John Coltrane goes on to build part of his solo using this
fragment in different keys).
Segment duration does also depend on tempo; visual ana-
logue: a Lego piece or a brick.
Lick: a memorable theme usually between two and four
bars (e.g. Ex.2, the opening in Charlie Parker’s “Now’s
The Time”).
Ex.2
Lick is longer than a segment and shorter than a phrase
(again, dependent on tempo, typically not longer than
four bars); often musicians transpose favorite licks in a
variety of keys to enhance their “vocabulary” in a certain
style; can also be used as “mannerisms” to reference an-
other performer or style; visual analogue: a larger, more
salient and recognizable structure such as a door or a
window.
Phrase: longer sequence of notes2 that may or may not
contain discernible segments or licks; visual analogue: an
entire room or part of a space that can contain
legos/bricks, doors, windows, etc.
Here is Ex.3, Dexter Gordon’ s 7 bar long phrase from
“Cheesecake”.
Ex.3
Additionally a constituent, according to its function,
would acquire one of the following characterisations:
a response/answer to a previous element in the same
piece (reaction to an internal/local musical event),
a reference to a previous element in the same piece (allu-
2 Our initial focus on horn solos imposes a maximum phrase duration
based on breath capacity, which can, of course, be exceeded when using
circular breathing techniques.
sion to an internal/local event), or an independent new
idea. In terms of score annotation for the mapping we use
S for segment, L for lick, P for phrase, different colours
for each one, plus r for reference/relationship, a for an-
swer/response, while location is described with brackets.
3. ANNOTATION SYNTAX
In annotating the above constituents as a piece unfolds in
time, i.e. syntactically, we developed textual annotations
to describe constituents and their locations.
3.1 Location
Location is annotated as “measure number and beat num-
ber within the measure”. For example, location “1.3”
means” third beat of the first measure”. Longer durations
are annotated analogously. For example, an element’s
duration of "1.1 - 2.4" means that the element starts on
beat 1 of measure 1 and ends on beat 4 of measure 2.
3.2 Constituents and Qualifiers
Segment = S(Index, Reference, Response)
Lick = L(Index, Reference, Response)
Phrase = P(Index, Reference, Response)
Index: numerical order of appearance of a structural ele-
ment
Reference: 1,2,3… = a first/second/third reference; 0 =
not a reference
Response: 1,2,3… = a first/second/third response; 0 = not
a response
If both Reference and Response are 0 the element quali-
fies as a New Idea.
3.3 Annotation Example
For Measure 1 in Sonny Rollins’s “St. Thomas” we
would write 1.1 - 1.4; S(1, 0, 0) to indicate:
beats 1-4 of measure 1 outline the first distinct segment
of the piece which is not a reference or response to any
other element but a new idea.
For Measure 2 we would write 2.1 - 2.4; S(1, 0, 1) to
indicate: beats 1-4 of measure 2 constitute the 1st re-
sponse (and not a reference) to the 1st segment, which
was introduced in measure 1.
For Measures 15-17 we would write 15.1 - 17.1; S(1, 2,
3) to indicate: the portion beginning at measure 15, beat 1
and ending at measure 17, beat 1 constitutes the 2nd ref-
erence and 3rd response to the 1st segment.
3.4 Additional definitions
*: Mannerism
A segment, lick, or phrase that exemplifies a performer’s
style; a structural element that sounds like a quintessen-
tial Sonny Rollins, Charlie Parker, or any artist passage.
For example: S(1,0,0)* describes the piece’s 1st segment,
which is neither a reference nor a response, nor a wholly
new idea but, rather, a stylistic mannerism, pointing to a
specific style characteristic to an artist or genre.
This designation helps differentiate between references to
elements within a given solo and references to the per-
forming artist's "memory bank."
The following Ex.4 is a Parker mannerism on “Now’ s
The Time”, that we see in a more elaborate version below
at our analyses of “Au Privave”.
Ex.4
**: Quote
A segment that directly incorporates a well-known and
recognizable structural element from another piece (e.g. a
theme from Beethoven’s 5th symphony, a lick from a
Jazz standard or a well-known pop song, or another play-
er’s favorite phrase.
The use of quotes in jazz improvisation is happening
often so if the quotes are properly labelled inside a well-
formed database of phrases, fragments and licks, then we
can annotate adding specifically the source of the quote
and a double asterisk: S(1,0,0)**.
4. ANNOTATION EXAMPLES
4.1 Sonny Rollins solo on “St. Thomas”.
Score analyses with brackets and annotation definitions.
(We also use colors to help identify the constituents
Segment=green, Lick=red, Phrase=blue):
Annotation:
1.1 - 1.4; S(1, 0, 0)
2.1 - 2.4; S(1, 0, 1)
3.1 - 3.4; S(1, 0, 2)
4.1 - 5.1; S(1, 0, 3)
5.2 - 5.4; S(1, 1, 0)
6.1 - 6.4; S(1, 1, 1)
7.1 - 7.2; S(1, 1, 0)
7.3 - 8.1; S(1, 1, 2)
8.2 - 9.1; S(1, 1, 3)
9.3 - 13.2; L(1, 0, 0)
13.1 - 13.3; S(1, 2, 0)
13.3 - 13.4; S(1, 2, 1)
14.2 - 14.4; S(1, 2, 2)
15.1 - 17.1; S(1, 2, 3)
17.1 - 17.4; S(1, 0, 0)
17.4 - 18.2; S(1, 0, 1)
18.2 - 18.4; S(1, 0, 2)
19.1 - 19.4; S(1, 0, 3)
20.1 - 21.2; S(1, 0, 4)
21.3 - 24.4; L(2, 0, 0)
25.1 - 32.1; P(1, 0, 0)
31.4 - 32.1; S(1, 3, 0)
32.3 - 33.4; L(3, 0, 0)*
34.2 - 35.4; L(3, 0, 1)
36.1 - 37.1; S(2, 0, 0)
37.3 - 37.4; S(2, 0, 1)
38.3 - 39.4; L(4, 0, 0)
41.1 - 42.4; L(5, 0, 0)
44.1 - 52.1; P(2, 0 ,0)
53.2 - 53.3; S(3, 0 ,0)
54.2 - 57.2; L(6, 0, 0)
57.4 - 59.4; L(7, 0, 0)
60.1 - 61.4; L(3, 1, 1)
62.1 - 63.4; L(7, 0, 2)
65.1 - 69.3; P(3, 0, 0)
69.4 - 71.4; L(8, 0 , 0)
72.1 - 73.1; L(8, 0, 1)
73.3 - 74.4; L(9, 0, 0)
75.1 - 76.4; L(10, 0, 0)
77.1 - 79.4; L(11, 0, 0)
80.1 - 81.1; S(2, 1, 0)
4.2 Charlie Parker solo on “Au Privave”.
Score Analyses:
Annotation:
1.1 - 3.4; L(1, 0, 0)
4.4 - 5.3; S(1, 0, 0)
5.4 - 6.3; S(1, 0, 1)
7.1 - 11.4; P(1, 0, 0)
12.4 - 19.4; P(2, 0, 0)
20.1 - 23.1; L(2, 0, 0)*
23.2 - 24.1; S(2, 0, 0)
24.2 - 25.1; S(2, 0, 1)
25.2 - 27.4; L(3, 0, 0)
28.1 - 29.4; L(4, 0, 0)
30.1 - 31.1; L(4, 0, 1)
31.4 - 33.1; L(5, 0, 0)
33.3 - 35.4; L(6, 0, 0)
36.1 - 37.4; L(7, 0, 0)
4.3 Comments on the 2 solos
For this paper we analyzed 2 solos from different periods
of jazz and from different players. We see a much longer
solo on Sonny Rollins, as it is later hard bop period, and
he is thus able to expand into thematic development,
while Charlie Parker takes a much shorter solo on the
blues but he is the one who presented the new bebop lan-
guage that forms the basis of modern jazz improvisation
to this day. He doesn’t refer back to himself like Sonny
was able to do later on, he introduces new ideas and also
plays one of his favorite phrases on the double time that
since then has become a sort of parkerism for the jazz
community. We have a sense that Parker was able to play
so much “music” in a very short solo, while Sonny on a
longer solo creates movement, interest and innovation by
his thematic development approach.
We see how Sonny Rollins uses the opening segment to
built thematic development in many instances of the solo,
not only as related segments, but also as part of licks and
longer phrases. These elements mark a great development
in the syntax and the story telling of a jazz solo.
Both players share the love of the blues, a very basic el-
ement in jazz improvisation and their both have a great
swing “feel”.
Many of the above segments, licks and phrases are part
of the jazz vocabulary of today and we witness here the
development of jazz from two masters of their art, who
among others defined the language and also created a
very strong personal voice.
5. METHODOLOGY DISCUSSION
5.1 Sequential information (Thematic development)
Identifying locations in time for each element provides
the structural skeleton that can support future automation
of such analyses and AI-system-generated thematic de-
velopment.
For example a sequence may proceed as:
Segment, answer, answer, lick, related segment, answer,
repeat, original segment, new lick, new segment, answer,
phrase(that may or may not contain previously intro-
duced segments or licks), first lick reference, answer etc.
Codification of sequential development may also find
applications in speech analysis and several temporal art
forms.
5.2 How to call and answer
There are plenty of instances of this paradigm in improvi-
sation. What we would learn is the transformation func-
tion that takes us from the initial structural element (such
as the segment, lick, or phrase) into the response.
Similarity or contrast can both form the basis of a ques-
tion/answer procedure.
We also have information that describes the sequence
of the responses so we could learn how the first response
differs from the second, or the third, and so on.
For example, in the first 4 measures of Sonny Rollins on
St. Thomas we see there is an initial segment, a response
segment, a 2nd response segment, and a third response
segment. In this example each response has more notes
than the previous. Such trends are learnable.
5.3 Transformations or referencing and embellishing
This has similarities to the call / response paradigm.
However, a reference to a previous element is not neces-
sarily a “response” but can serve a different thematic
structure function.
Repeated phrases: here we either annotate as the same
segment/lick/phrase, but in the case of small alterations to
the original then this again is mapped as reference and
answer.
5.4 Hierarchical
Three examples to look out for:
a) Combine segments to create licks
b) Combine licks to create phrases
c) Freely combine all three elements
While there are instances where a lick or phrase is made
of smaller elements, not every lick or phrase can be de-
scribed this way. Often, licks and phrases are original and
do not reference other elements.
5.5 Structural interchange
Cross-reference among the three identified structural el-
ements provides another means of thematic development
during improvisation. Our analytical approach can cap-
ture this feature through double annotation on the specific
bar or bars. See, for example, the end of Lick1 and
Phrase1 on the Sonny Rollins solo where he ends restat-
ing the 1st segment idea.
5.6 Voice leading concept
In be-bop, hard-bop and modern jazz styles voice leading
is frequently used to end or connect licks, phrases and
themes. In a more open, modal or free playing this is not
so evident. Rather, harmonic tension, sound, articulation
and note density within phrases provide the most im-
portant cues. We anticipate that an upcoming multi-
layered mapping will address this issue.
5.7 Emotion and creativity
Emotion: A common mechanism in music, also employed
here, is creating patterns of tension and release that play
with the listeners’ expectations.
To what extent something can be characterized as inter-
esting or emotional is contingent on what preceded it and
what, eventually, follows. A player known for a specific
style or mannerism – say, a linear approach – can inhibit
expectations by switching to a vertical approach, or by
inserting unexpected pauses, long notes, or sound effects.
Variations such as these that increase interest and elicit
affective responses are manifestations of the performer’s
creativity and capacity to unfold a musical improvisation
as a compelling story.
5.8 Inspiration
One way to approach “inspiration” could be in terms of
compelling, unexpected structures that arise out of ran-
domness. As jazz musicians deal with randomness, if
suddenly - in playing or practice - we get a struc-
ture/phrase that stands out in terms of being memorable
or highly organized/structured then we recognize this as
inspiration that usually becomes a new composition or a
favorite mannerism.
5.9 Thematic development and multi reference
References to previous elements, whether as straight re-
peats or augmented, diminished, displaced, or otherwise
modified, can be considered a form of self-reference.
Feeding a database of such manipulations and thematic
developments to machine learning algorithms can support
the development of AI systems that exhibit self-
referential behavior and, by extension, apparent self-
awareness.
6. CONCLUSIONS
We have proposed an analytical method that supports
systematic annotation of a wide variety of jazz solos and
can reveal the musical language characteristics of indi-
vidual players and styles. The annotated constituents per
solo will eventually feed a database of musical segments,
licks, and phrases that can imply and outline a specific
chord or a longer harmonic progression. We anticipate
that this database will enhance the “bag of tricks” of the
jazz player and help the jazz educator explain jazz styles,
performers’ personal voices, and characteristic manner-
isms.
In jazz, performers always strive to develop a personal
voice that can stand next to that of the masters. The
knowledgeable player or listener can usually identify,
after only a few notes, a master performer who has de-
veloped language and mannerisms that are immediately
evident.
A personal voice consists of sounds and sound struc-
tures with certain recognizable and personal qualities that
function as a performer’s signature. The mappings sup-
ported in this study can help reveal and codify these sig-
natures and organize them into systematic categories.
Further work is required to better define stylistic con-
stituents, flexible enough to codify a broad range of styles
and personal voices. As we proceed, we will seek the
insights of top jazz improvisers, worldwide, and assess
the resulting database through AI machine learning and
performance.
Acknowledgments
We would like to thank Gil Weinberg, Alexander Lerch,
Mason Bretan, Frank Clark, and Athanassios Economou
(Georgia Institute of Technology), Martin Norgaard and
Mariana Montiel (Georgia State University) and Pantelis
Vassilakis (AcousticsLab) for their advice, friendship,
wisdom and research collaborations.
Deep thanks to David Liebman and fellow players Jeff
“Tain” Watts, Essiet Okon Essiet, Sylvia Cuenca, Benito
Gonzales, Ralph Peterson, Dave Kikoski, Theo Hill,
Craig Bailey - among many others - that helped by their
playing, discussions and insights.
We would also like to thank the graduate students at the
Department of Music Studies, National and Kapodistrian
University of Athens.
REFERENCES
[1] N. Cook, A Guide to Musical Analysis, OUP, 1987.
[2] J.J. Nattiez, Fondements d’une Semiologie de la
Musique, Union Generale d’ Editions, 1975.
[3] C. Anagnostopoulou and G. Westermann,
“Categorisation in music: A computational model
for Paradigmatic analysis,” Proceedings ICMC,
Thessaloniki, Greece, 1997.
[4] E. Cambouropoulos, Towards a general
computational theory of musical structure, PhD
Thesis, University of Edinburgh, 1998.
[5] F. Okiji, “Storytelling in Jazz Work as
Retrospective,” Collaboration Journal of the Society
for American Music, 11(1), February 2017, pp. 70-
92.
[6] S. Bjerstedt, Story Telling in jazz Improvisation,
PhD Thesis, Lund University, 2014.
[7] J. Gillick, K. Tang, and R.M. Keller, “Machine
Learning of Jazz Grammars,” Computer Music
Journal, 2008, pp. 111–222.
[8] G. Schuller, “Sonny Rollins and the challenge of
thematic improvisation,” The Jazz Review, 1(1).
1958.
[9] J. Abeßer, E. Cano, K. Frieler, and M. Pfleiderer,
“Dynamics in jazz improvisation – score-informed
estimation and contextual analysis of tone intensities
in trumpet and saxophone solos,” Proceedings of the
9th Conference on Interdisciplinary Musicology –
CIM14, Berlin, Germany, 2014.
[10] M. Norgaard, M. Montiel, and J. Spencer, “Chords
not required: Incorporating horizontal and vertical
aspects independently in a computer improvisation
algorithm,” Proceedings of the International
Symposium on Performance Science, 2013, pp. 725–
730.
[11] M. Norgaard, “How jazz musicians improvise: The
central role of auditory and motor patterns,”
Music Perception. 31(3), 2014, pp. 271-287.
[12] M. Bretan and G. Weinberg, “A survey of robotic
musicianship,” Communications of the ACM, 59(5),
2016, 100-109.
[13] A.R. Addessi, L. Ferrari, S. Carlotti, and F. Pachet,
“Young children musical experience with a flow
machine,” Proceedings of ICMPC9 and 6th ESCOM
Conference, 2006.
[14] G. Assayag and S. Dubnov, “Using Factor Oracles
for machine Improvisation,” Soft Computing, 8(9),
2004 c; Imperial College Press, Lecture Notes
Series, 2016, pp.61-74.
[15] D. Mendonca and W. Wallace, “Cognition in Jazz
Improvisation: An Exploratory Study,” Proceedings
of the Annual Meeting of the Cognitive Science
Society, 26(26) 2004.
[16] D. Horowitz, “Representing Musical Knowledge in
a Jazz Improvisation System,” MIT Media Lab,
Cambridge, MA, 1995.
