Content uploaded by Michele Zappavigna
Author content
All content in this area was uploaded by Michele Zappavigna on Jul 17, 2019
Content may be subject to copyright.
Chapter 9
The Coupling of Gesture and Phonology
Michele Zappavigna, Chris Cléirigh, Paul Dwyer and J. R. Martin
1 Introduction
Analysis of gesture is a new region in Systemic Functional Linguistic (SFL)
inquiry, although it is relatively established in other disciplines such as
anthropology and cognitive science (Efron 1941, Morris et al. 1979, McNeill
1992, Goldin-Meadow 2003, Kendon 2004). Early SFL-oriented work includes
exploration of gesture realizing process type and interpersonal meaning –
i.e. meaning related to the negotiation of social relations (Martinec 2000a, 2001),
discourse semantics and gesture (Hood 2007), and a case study on non-verbal
communication by a child with an intellectual disability (Dreyfus 2006). These
perspectives arise out of Halliday’s (1985) initial framing of gesture as paralinguistic,
functioning to support the meaning made in talk. Gestures are, in Halliday’s
model, ‘not part of the grammar, but rather additional variations by which the
speaker signals the import of what he [sic] is saying’ (Halliday 1985: 30). As a
mode of expression, gesture has a prosodic structure which we might think of as
akin to an intonation contour because it cannot be systematically divided into
constituent units, unlike, for example, grammatical structures (Halliday 1985).
Kendon (2004: 125) asserts that gesture is ‘fully-integrated’ into the expres-
sion of meaning as an ‘ensemble’. The kind of resource-based view of gesture
that he proposes accords with the theoretical stance of SFL:
It appears that there is exibility in the gesture-speech relationship and this,
it will be suggested, is best understood in terms of a point of view that sees
gesture and speech as two different kinds of expressive resource available to
speakers, and that the gestures employed within an utterance, like the words
that are employed, are components of a speaker’s nal product . . . Gesture is
a partner with speech in the utterance as nally constructed. (Kendon 2004:
111; italics in original)
One way in which SFL takes up the idea of partnership is the concept of coupling,
that is, co-selection of functional features in a text (Martin 2008; Zappavigna et al.
2008, Martin this volume). Patterns of couplings form syndromes, where a syn-
drome is de ned as the recurrent co-selection of features in a text or corpus con-
tributing to a particular rhetorical strategy (Zappavigna et al. 2008). For example,
220 New Discourse on Language
within a particular text, [tone 4] may occur with a [declarative] realizing
[reservation].1 This particular coupling may, in turn, be involved in more complex
logogenetic patterning. The weight of a syndrome, that is, how tightly features
are coupled, depends on how frequently particular features co-occur. This
co-occurrence is both predicted by and in uences the system of language.
However, as we began our analyses we were faced with the problem that we
currently do not have a pre-existing analytical unit for investigating gesture.
In addition, since we cannot systematically relate gesture to lexicogrammar,
determining how gestures are motivated by meaning is more dif cult than
determining, for example, how discourse semantic meanings are realized in
phonology. Because of these dif culties we proposed using the tone group as
coterminous with a gestural unit. This is because both intonation and gesture
involve bodily movements, and postulating a relationship between the two, as
expression systems, offers a means to relate gesture to the information unit as a
unit of content. This method is broadly consistent with Kendon’s (2004)
approach of exploring how gesture and intonation work in partnership. In
addition, we are able to apply the concept of coupling, thus far only theorized
monomodally, to look at the relationship between gesture and phonology.
It should be noted that the correspondences of gesture and talk that are
made in this chapter presuppose fuzzy boundaries. It would make no sense to
proceed by identifying a lexical item in the verbiage and look only at the cor-
responding gesture in the exact frame in the time series. In other words, we are
looking for the meanings in the spoken text (within each information unit real-
ized by a tone group) that motivate the gestures. The coupling is approximate
since gestures require preparatory movements and the articulators for gesture
(arms, hands etc.), being larger, are slower than those for speech (tongue, lips
etc.). Kendon notes:
. . . the preparation of the gesture phrase begins in advance of the parts of the
spoken expressions to which it is to be linked semantically. This means that,
when using gesture, the speaker must already have organized it at the same
time as the plan for the spoken phrase with which is it is to co-occur is
organized. Gesture and speech thus, are planned for together and gestural
expression is a fully integrated component of the utterance’s construction.
(Kendon 2004: 125)
Since we adopt a systemic functional analytical approach, we operate by
considering how the meanings construed by the coupling of gesture and speech
function in the particular context of situation in which they are deployed.
2 A Brief Introduction to Phonology
Our phonological analyses are based upon Halliday’s (1970) systemic functional
phonology which was in uenced by Firth’s (1948) work on non-segmental
The Coupling of Gesture and Phonology 221
prosody and work in Chinese phonology (Wang 1936). This approach to
phonology takes the foot as the fundamental rhythmic unit of spoken English
(for a related discussion of rhythm see Caldwell this volume). A foot consists of a
beat, or Ictus, which can be realized by a salient syllable or be silent, optionally
followed by one or more weak syllables. Foot boundaries are annotated with /
and silent beats with ^, as in the following:
//That’s / all / ^ / that’s / what I’d / like to / know//
The unit of intonation is the tone group, which consists of one or more feet.
The example above from our data corresponds to a single tone group; tone
boundaries are annotated with //. A tone group realizes ‘one unit of informa-
tion, one ‘block’ in the message that the speaker is communicating; and so it
can be of any length’, corresponding to the clause in the unmarked case
(Halliday 1970: 3).
Within the tone group, the part given most emphasis by the speaker and
falling on a salient syllable is referred to as the tonic syllable (shown in bold
type above), and the foot in which it occurs is the tonic foot. The tonic carries
the main (de ning) pitch movement in the tone group either by covering the
widest pitch range or by immediately following a jump in pitch. Within a tone
group, the part extending from the tonic to the end of the tone group is called
the tonic segment and the rest is referred to as the pretonic segment. Table 9.1
shows how we have chosen to lay out our analyses so that the structures we have
introduced in this section can be systematically related to gesture.
The system of pitch movement choices is known as tone. There are ve
simple tones:
[tone 1]: fall
[tone 2]: rise
[tone 3]: level (or low rise)
[tone 4]: fall-rise
[tone 5]: rise-fall
There are also two compound tones, which are sequences of two tones:
[tone 13] and [tone 53]. What makes these compounds is that the second tone
cannot be preceded by its own pretonic segment, and so always directly follows
the preceding tonic.
Table 9.1 An example of rhythm and intonation analysis
1
Pretonic Segment Tonic Segment
That’s all ∆ that’s what I’d like to know
222 New Discourse on Language
Key is the name given to systems that are realized by tone (Halliday and
Matthiessen 2004). It construes meanings about the speaker’s attitude to the
negotiation (for example, ‘politeness, assertiveness, indifference’ (Halliday
1970: 22). Couplings of particular tones and particular grammatical features
construe speci c interpersonal meanings. For example, [tone 4] (fall rise)
together with a [declarative] realizes the key feature [reservation].
3 Data: The Affray Youth Justice Conference
We analyse gesture within a particular context of situation: Youth Justice Con-
ferencing (YJC). Conferencing, a form of restorative justice, affords a young
person who has committed an offence the opportunity to talk about the offence
in the presence of the victim and support people, who are also encouraged to
discuss their experiences. In New South Wales, Australia, YJC acts as an alterna-
tive to the Children’s Court for sentencing a young person who has committed
a criminal offence that meets certain requirements in terms of severity. The con-
ferences are usually held with participants seated in a circle con guration and in
meeting rooms in public buildings such as Police Citizen Youth Clubs or Council
chambers. Participants are encouraged by the Convenor, who runs the confer-
ence, to share their feelings and experiences relating to the offence. These par-
ticipants can include the victim of the offence, support people for both the
young person and the victim, police of cers such as Youth Liaison Of cers and
Ethnic Community Liaison Of cers, and members of the local community.
While there may be a perception that YJCs are events in which emotions are
dramatically played out, carrying with them large and animated gestures, the
conferences that we have observed and recorded do not tend to support this
assumption. Gesturing in these conferences was typically relatively constrained,
particularly when produced by the young person who tended to occupy mini-
mal gestural space, perhaps in order to remain a small target for criticism.
The conference that is the subject of our analysis was convened as a result of
an affray offence by a young male. Hereon, for brevity, it will be referred to as
the Affray YJC. The Affray YJC was video recorded and the researchers also sat
in the conference circle as silent observers. To minimize the visual presence of
recording equipment in the hope of reducing their impact upon conference
participants, video-recordings were made using a camera mounted on a micro-
phone stand recording to DVD. The cameras were positioned in the corners of
the room to further reduce their visual presence (Figure 9.1).
The Affray YJC was a slight variation on the typical circle formation of a confer-
ence as participants were seated around a rectangular table formed by three desks
pushed together (Figure 9.2). Thus, some gesturing occurred underneath the
table and was not visible to other participants or our cameras. This table was also
involved in some of the participants’ gestures: for example, as a surface upon
which to beat hands to emphasize the salience of a point being made in the talk.
The Coupling of Gesture and Phonology 223
F 9.1 Con guration for recording a YJC
F 9.2 Seating con guration of the ‘Affray YJC’ (participants have been
blurred to maintain anonymity)
224 New Discourse on Language
This conference was one hour and a half in duration and was transcribed
using Transcriber (Boudahmane, Manta, Antoine, Galliano, and Barras 2001,
http://trans.sourceforge.net/). Text 1, the data analysed in this chapter, was a
95-second extract sampled from this conference by looking for an example of
gestural prosody, that is, repetition of a particular gesture (pleading hands)
across a phase of discourse also marked by a shift in eld and tenor variables.2
For example, there was a shift in terms of eld (content) to talk about family
from talk about the legal system, and also a shift in terms of tenor (negotiating
relationships) to increased talk about affect (anger).
In exploring intermodal co-patterning we found that we required tools to
assist with tracking and representing logogenesis, the unfolding of text. The
video was annotated using ELAN (Max-Planck-Institute for Psycholinguistics
2008, www.mpi.nl/tools/). ELAN was chosen as an annotation tool as it allowed
us to track multiple data series while concurrently tracking the time series of
the video text. These data series took the form of different ‘annotation tiers’
(see for example the tier labelled ‘tone group’ near the bottom of Figure 9.3).
This practical endeavour of annotating the text generated a number of theo-
retical concerns relating to how to model co-articulation so that we account for
both the sequencing of the text and relationships between different modes.
A challenge is to determine what we can understand to be co-expressed when
we are dealing with modes that have a constituent, or alternatively, a prosodic
structure, where boundaries between units are fuzzy. This challenge has led us
and others (Zhao this volume) to imagine what a metalanguage that can account
for semiotic dynamism might be. The work herein is a step alongside Zhao
toward theorizing the kinds of relationships involved in logogenetic coupling,
that is, patterns of coupling unfolding in a text.
F 9.3 An example of tone group analysis performed in ELAN
The Coupling of Gesture and Phonology 225
We now turn to the text which will be analysed in detail in this chapter,
Text 1, which is an interaction between the young person and a Ethnic Com-
munity Liaison Of cer (ELO). This text was taken from the commissioned
recount stage of the conference and is an example where an ELO intervenes in
a recount that does not appear to be functioning optimally.
The commissioned recount is the stage of the conference where a convenor
will ask a young person to give an account of the crime and then to re ect on
the implications of their behaviour. The manual used in New South Wales
as part of the convenors’ training includes a scripted outline detailing the
structure of a typical conference (Youth Justice Conferencing Directorate
2005). It indicates that the convenor should invite the young person to ‘tell
their story’. The young person, however, in the conferences that we have
observed, will rarely produce a packaged recount of events. Both the details of
the offence and related emotion talk are usually jointly constructed through
prompting by the convenor.
Therefore, conceiving of the conference in terms of genre, that is, as staged
activity that serves a particular social goal within a culture (Martin and Rose
2008, Martin this volume), we refer to the story that the young person tells as
a commissioned recount rather than simply a recount. This functional label
re ects the way the convenor, ELO (in the case of the Affray YJC) and young
person (YP) jointly construe the talk that unfolds, and the observation that we
have made over the course of observing a range of conferences involving differ-
ent kinds of offences, that the young person tends not to offer forth an extended,
spontaneous delivery. Conferencing is a macro-genre,3 and the commissioned
recount functions as a recontextualized element genre within this macro-genre;
in other words, it is a smaller genre performing a function within the larger
genre of conferencing while retaining its own particular staging and patterns of
meaning. Commissioned recount typically occurs after the gathering and legal
framing of the conference and before the responses by third parties to the
young person’s ‘story’. The commissioned recount tends to have the following
structure (^ conventionally signals ‘followed by’):
Orientation ^ Record of events ^ (Re-orientation) ^ (Extension) ^ Interpre-
tation ^ Rami cations
Elsewhere we have provided a more detailed analysis of the function of this
genre (Martin et al. forthcoming). The orientation stage in the commissioned
recount sets the recount in time and space and introduces the main partici-
pants involved in the conference. The record of events which follows presents
the sequence of events leading up to and constituting the offence from the
young person’s perspective. The next stage, re-orientation is optional and
wraps up the recount and returns participants from reconstructed past events
to the spatio-temporal setting of the conference itself. The extension stage
is also optional and functions to elicit a fuller account from the young person.
226 New Discourse on Language
Within the interpretation stage the recount is evaluated, as emotions and
social values in relation to the offence are explored. Finally, within the
rami cations stage, some of the pertinent consequences of the offence are
canvassed.
Text 1 is part of the interpretation stage of the commissioned recount.
It comes after the Convenor has attempted to elicit, extend and nally
begin to jointly evaluate the YP’s recount. The transcript of Text 1 is repro-
duced below:
ELO: OK. Mate y- [frustrated noise]. Mate, you’ve gotta understand brother,
what- what everyone’s trying to do here is, mate, you might look at
everybody and- and you know and everyone’s saying, ‘Say the truth’
but if you were ang- let’s just say from the police. OK. And, mate,
police can defend themselves. Do you understand? I’m not going to
speak for police. What I’m saying to ya is if you’re doing the wrong
thing and they’re catching ya, why would you be angry with them?
Why would- why would you not look at what you’re doing to get your-
self into this trouble? Why would you not look at how you’re getting
your mum into this crap, how you’re getting your family into this
crap? Why don’t you look at why we’re here today? Are we here
because of me?
YP: No.
ELO: Are we- who are we here for?
YP: Because of me.
ELO: Not because of you as a person, because of something you’re doing
that’s not right. We’re not targeting you personally, brother. It’s what
you’re doing that’s no good. You’re probably a good person. What
you’re doing is not good. You understand the difference?
YP: Yes.
ELO: We’re not saying, ‘YP, you’re a this and you’re a that’. What we’re
saying is your behaviour is getting you into trouble, man. You’re
hurting your family, brother. You’re hurting your [Arabic], brother.
You understand?
YP: Yes.
ELO: If I didn’t care about ya, man, I didn’t care about your mum, I didn’t
care about, you know, the Den and everything, I wouldn’t even be
here. I mean, I’ve nished my work. But if we- if everyone here today
could help you just to sorta think to yourself, ‘What am I doing to my
family? What am I doing to myself?’, man, Shalam, that’s what it’s all
about. That’s what today is all about. It’s about you sitting down and
having a look- Why don’t you come- I think you should come and sit
here and look at ya Mum. Come and sit here. I want- Sorry. I want you
to sit there. I want you to sit over there [inaudible chatter about
moving chairs].
The Coupling of Gesture and Phonology 227
4 Analysis: The Pleading Clasp Prosody
The gestural prosody that characterizes Text 1 involved a gesture of hands
pressed together in what we term a ‘pleading clasp’. It is similar to the hand
formation used for praying in many cultures, although not typically in Islamic
culture.4 The contemporary association of prayer with supplication has its
origins in the Latin, precari, meaning ‘to ask earnestly, beg’ (Harper 2001: 11).
The pleading clasp gesture seemed to carry some of this meaning as it was often
coupled with instances of [tone 3] that can realize ‘pleading’ when used with a
vocative (Halliday 1967, 1970). The use of the vocative ‘man’ together with
[tone 3] shows the ELO trying to subvert the tenor that the relatively formal
situation imposes. This was a way of engaging more closely with the young
person in the hope of being maximally persuasive.
The rst instance that the ELO produces is roughly commensurate with pray-
ing hands, although his ngers are slightly splayed (LHS of Figure 9.4). The ELO
also clasps his hands together interlocking his ngers (RHS of Figure 9.4).
The ELO most often pointed his nger tips away from his body when forming
the pleading clasp, meaning that his hands were in a horizontal position
(Figure 9.5). We take this pleading clasp as the ‘rest’ (Kendon 2004) or ‘home
position’ (Sacks and Schegloff 2002) in Text 1.
Figure 9.6 shows the intervals in which the ELO uses pleading clasp gestures.
The time series is shown at the bottom of the diagram in seconds. Black intervals
represent pleading hand gestures and grey intervals represent all other types of
gesture. As the pleading clasp prosody illustrated in this gure is the most consis-
tent prosodic structure (since the grey regions represent a collection of different
F 9.4 ELO’s pleading clasp and ‘Praying Hands’ (study for an Apostle gure
of the ‘Heller’ altar by Albrecht Dürer, c.1508)
F 9.5 An example of the pleading hand clasp
228 New Discourse on Language
kinds of gesture), we take this gesture as the unmarked option and proceed, in
the sections that follow, by considering the different choices that the ELO makes
to deviate from this gesture.
5 Beating, Pitching and Articulating
In our extract, gestures operated to support the meanings made in the spoken
discourse produced by the ELO. If we take phonology, another expression
system, for which we have an existing rank scale and have developed methods
of analysis, as a means to explore gesture, we may note three ways in which the
two expression plane systems seem to couple:
Pitching
z
Beating z
Articulating z
Gestural pitching, that is, movement up or movement down of the gesture, is
like intonation and appears to realize key and information. We propose pitching
as an analogue of the tone system in phonology (Halliday 1967, 1970). We are
using ‘pitching’ to refer only to those gestures that move ‘in tune’ with the pitch
contour of the concurrent tone group, for example when a gesture rises and
falls (in space) as the intonation rises and falls (in pitch). Gestural beating is like
rhythm and realizes salience. We propose ‘beating’ as the gestural analogue of
rhythm in phonology (Halliday 1967, 1970), with its function of adding salience
to the co-timed meanings. Gestural articulation is akin to phonological articula-
tion and realizes a range of possibilities such as deixis and representation. The
following sections explain each of these possibilities in turn.
6 Pitching the Intonation of Speech
The ELO’s gesture would often move in the same direction as the pitch move-
ment, realizing interpersonal key. The ELO’s gesture at times, for example,
mimicked the rise and fall tone contour of [tone 5] (Figure 9.7). [Tone 5] has
the potential to realize meanings to do with exasperation and disappointment
(Halliday 1985). The example in Figure 9.7 is a tone group from the clause
complex, ‘If I didn’t care about, you know, the Den and everything, man,
I wouldn’t even be here’. By echoing the rise and fall of the pitch, the gesture
0
5 101520 2530 35 4045 50 5560 6570 7580 8590 95
F 9.6 Stretches where ELO uses pleading clasp gestures
The Coupling of Gesture and Phonology 229
realizes the same meaning as the intonation (tone 5) in that grammatical con-
text (exasperation, frustration etc.).
7 Beating the Rhythm of Speech
Within Text 1 there are many examples where gesture beats the rhythm and
pitches the tone of the spoken language. Gestures can beat on tonic syllables,
salient syllables, or all syllables. Increasing the frequency of a gesture intensi es
the salience of the information.
The ELO devoted a relatively large amount of semiotic energy to emphasizing
his message, attempting to hold the YP’s attention in order to in uence his
behaviour. In so doing, he made use of different beating gestures where we he
would repetitively beat parts of his body, either in the air, against his body, or on
the tabletop, increasing the frequency of the beat at salient syllables in his talk.
For example as the ELO calls attention to the function of the conference in his
verbiage, saying ‘That’s what today is all about’, he raises the frequency of the
gesture shown in Figure 9.8. This gesture involved clasping the hands together
Tone 5
wouldn’t even be here
right hand retracts, making a whip-like opening and
upturning ‘rise-fall’ trajectory back to clasping with the left hand
Tone type
Tone group
Feet
Gesture
Pretonic Segment Tonic Segment
F 9.7 Gesture echoing pitch movement
Tone type Tone 1
Tone
group
Feet That’s is all aday iswhat to bout
clasp beats towards chest on salients, with extra beat on ‘a(bout)’
Gesture
Beats ◊ ◊ ◊ ◊ ◊ ◊
Pretonic Segment Tonic Segment
F 9.8 Increased salience realized as increased frequency of gestural beats
230 New Discourse on Language
and beating toward the chest on salient syllables, with an additional beat
afforded to the tonic segment which realizes the culmination of new
information.
Increase in salience also had the function of increasing the intensity, or,
graduation (Martin and White 2005) of evaluations that ELO made about the
young person’s behaviour. Again increased frequency of a beating gesture
supported the increase in salience. By beating on (interpersonal) evaluative
meanings, and thus giving them (textual) salience, the gestures also served to
upscale those attitudes in terms of graduation. For example, the ELO says ‘Your
behaviour is getting you into trouble, man’, while adopting an arched hand
position with ngers fanned and beating his ngers on the table in front of him
(Figure 9.9). The beats fall on ‘get-’ and every subsequent syllable in the tone
group, highlighting the negative judgement.
Hood (2007) has also noted ‘vibrating’ gestures that function to upscale
graduation in face-to-face teaching of advanced level classes in writing academic
English. She distinguishes these from beating gestures, of the kind we have
identi ed here.
8 Articulating with Speech
Three functions of articulating gestures used by the ELO could be identi ed:
representation, deixis and contrast. We will explain each of these in turn in the
subsections that follow.
Representation
Gesture can contribute to the realization of meaning iconically, representing,
or ‘miming’, for example, a process (Martinec 2000a). These types of gesture
Tone type
Tone
group
Feet
Gesture
Beats
Pretonic Segment Tonic Segment
Tone 1
∆ is getting you into trouble man
finger-fanned right hand arches right beating the desk on salient ‘get-’
and beats on every following syllable
◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊
F 9.9 Increased salience scaling-up evaluation
The Coupling of Gesture and Phonology 231
were not common in Text 1. One instance observed occurred when the ELO
made reference to police, saying:
And, mate, police can defend themselves. Do you understand? I’m not going
to speak for police.
In this instance the ELO slaps one hand on another when he talks of police
being able to ‘defend’ themselves, a gesture iconic to the action of fending off
an attacker. Surprisingly, iconic gestures of this kind were relatively rare in our
sample.
Deixis
Gesture was also used to realize deixis, identifying persons in the exchange (see
also Kendon 2004 on deictic gestures). At points during the phase, one of the
ELO’s hands broke from the pleading clasp and was used to point to interlocu-
tors. He says to the YP:
But if everybody here could help you today to just sorta think to yourself,
‘What am I doing to my family, what am I doing to myself?’, man, Shalam,
that’s what it’s all about.
The ELO identi es two types of participants: the people seated around the
table and the YP. He makes a circular gesture to refer to everyone at the table
and points at the YP to identify him (Figure 9.11). Thus his gestural expression
varies with the meaning being distinguished: circular for plural rst person ‘we
all’, pointed for singular second person ‘you’. The Deictic ‘here’ is also coupled
with a downward pointing gesture.
Feet ∆ po fend them lice can de selves
left hand moves left on ‘police’, then claps right hand on ‘defend’ into
clasp
Gesture
Tone type
Tone
group Pretonic Segment Tonic Segment
Tone 1
F 9.10 Gesture representing a process
232 New Discourse on Language
Contrast
Contrast was often realized through gesture in the ELO’s discourse. In so doing
he used his physical gestural space as an abstract semiotic space. He placed semi-
otic objects in speci c positions in his gestural space to express certain meanings.
For example, he used space interpersonally, creating a shared space between
himself and the young person and another unshared space (Figure 9.12).
The ELO made use of this space to support contrasts made in his spoken
discourse that contributed to an overall rhetoric of oppositions. The ELO
Tone 4
But
if
we
can
∆
if
everybody here
could
day
right hand moves
out and left and
around table for
‘everybody’
(simultaneously
mimicked by
head swivel),
Tone
type
Tone
group
Feet
Gesture
Pretonic Segment Tonic Segment
help you
to
points
down at
table for
‘here’,
and to YP
for ‘you’
(personal
reference),
reclasping
at tonic
‘today’
F 9.11 Gesture realizing deixis
shared
semiotic
space
YP
unshared
semiotic space
unshared
semiotic space
ELO
F 9.12 Gesture space as semiotic space
The Coupling of Gesture and Phonology 233
tended to place positively evaluated targets in the shared space between himself
and the YP (Figure 9.12). This was part of a general correlation between ges-
tures and the targets of evaluative polarity. The ELO tended to direct gestures
coupled with negative evaluation towards the left, away from the YP. In turn,
gestures towards the YP and to the right were coupled with positive evaluation.
For example, the ELO says:
Why would you not look at how you’re getting your mum into this crap, how
you’re getting your family into this crap. Why don’t you look at why we’re
here today? Are we here because of me?
In this example, positively judged items such as ‘mother’ (invoked as a token of
positive evaluation in Islamic culture which construes the role of the mother as
highly-valued) are associated with gestures on the right of the shared space
between the ELO and YP, and items construing negative judgement such as ‘crap’
are associated with gestures on the left of this shared space (Figure 9.13).
As Figure 9.13 illustrates, beating gestures coupled with the positively evalu-
ated (+ value) behaviour of ‘looking at’ (i.e. re ecting upon) one’s behaviour
occur on the left whereas ‘crap’ (the negative situation that the young person
has drawn his mother into), is coupled with gestures on the right of the ELO’s
gesture space, construing a contrastive negative evaluation (- value).
Elsewhere we have discussed the ‘alternating’ evaluative prosody adopted
by the ELO which has a ‘to-and-fro’ structure, moving back and forth between
evaluative polarities (Zappavigna et al. forthcoming). Contributing to this
Why
would you
not
look at how
you’re
getting
your
mum into
this
crap
beating prayer clasp
moves leftward for ‘look’
(+ value, contrast with
failing to look)
then a little right for
‘mum’(contrast: +value)
then back to
the right for
‘crap’
(contrast:-
value)
Tone 1
◊◊ ◊◊◊ ◊
Tone type
Tone
group
Feet
Gesture
Beats
Pretonic Segment Tonic Segment
F 9.13 Gesture supporting evaluative contrast
234 New Discourse on Language
tendency was alternation between positively and negatively judged targets.
Judgement is part of the attitude system de ned in appraisal theory as ‘the
region of meaning construing our attitudes to people and the way they behave –
their character (how they measure up)’ (Martin and White 2005: 52; on
appraisal see also Tian this volume, Knox et al. this volume). For example, we
identi ed logogenetic patterns such as the following (positive judgement
marked in bold italic; negative judgement in small caps):
It’s what you’re doing that’s . You’re probably a good person. What
you’re doing is . You understand the difference?
Our analysis suggests that the gestural prosody acts to support the construal of
such evaluative contrast and to a rhetoric of oppositions more generally.
A challenge for future work will be nding ways of visually representing the logo-
genetic unfolding of gesture in tandem with speech so that the kind of coupling
suggested in this chapter can be elaborated with evidence from longer stretches
of discourse and ultimately with evidence from multimodal corpora.
9 Conclusion
The analysed sample of video-taped discourse acts as a pilot for testing out the
hypothesis that phonological structure can act as a useful starting point for
analysing gesture. By considering the coupling of gesture and phonology we
have proposed a methodology for handling the complexity of gesture, while
focusing on how gesture contributes to meaning-making.
In our sample, gesture has proven to hold a somewhat capricious relationship
to the meaning expressed in spoken discourse, roaming all over the semantic sys-
tems in the logogenesis of a text, one moment realizing deixis, another realizing
key, as the semantic systems ‘compete’ for additional realization by gesture. It was
found that gestures could indeed be chunked syntagmatically with tone groups,
thus providing a domain in which paradigmatic systems of gestural features can be
said to operate. Because the tone group realizes – and is coterminous with – the
information unit, this also provided a content plane domain within which to
consider the meanings being realized gesturally. We found three ways in which
gestural and phonological features optionally coupled in this syntagmatic domain:
pitching, beating and articulating. This is a modest, rst step into systematizing
gesture that we hope to follow with analyses of larger stretches of discourse.
Notes
1 Square brackets are used to represent features in system networks.
2 Matthiessen (1995: 36) suggests that context is ‘functionally diversi ed’, that is,
combinations of eld, tenor and mode de ne the way language is used. Brie y,
The Coupling of Gesture and Phonology 235
‘ eld’ refers to content, ‘tenor’ to the social relationships enacted, and ‘mode’ to
the communication channel.
3 Macro-genres are complexes of smaller genres (see Martin and Rose 2008).
4 The ELO locates himself, at the beginning of his interaction with the young per-
son, as a member of the Lebanese Muslim community and identi es the young
person as also part of this community.
References
Efron, D. (1941), Gesture, Race and Culture. The Hague: Mouton.
Firth, J. R. (1948), ‘Sounds and prosodies’. Transactions of the Philosophical Society:
127–52.
Goldin-Meadow, S. (2003), Hearing Gesture: How our Hands Help us Think.
Cambridge, MA: Harvard University Press.
Halliday, M. A. K. (1967), Intonation and Grammar in British English. The Hague:
Mouton.
—— (1970), A Course In Spoken English: Intonation. London: Oxford University
Press.
—— (1985), Spoken and Written Language. Victoria, NSW, Australia: Deakin
University.
—— (2004), An Introduction to Functional Grammar. London: Arnold.
Harper, D. (2001), Online Etymology Dictionary. Retrieved 27 November, 2008, from
Online etymology dictionary: http://www.etymonline.com.
Hood, S. (2007), ‘Embodying language: bringing gesture in from the paralinguistic
cold’, in Proceedings of Semiotic Margins: Reclaiming Meaning. Sydney, NSW,
Australia.
Kendon, A. (2004), Gesture: Visible Action as Utterance. Cambridge: Cambridge
University Press.
Martin, J. R. (2008), ‘Tenderness: realization and instantiation in a Botswanan
town’, in N. Norgaard (ed.), Systemic Functional Linguistics in Use, Odense: Odense
Working Papers in Language and Communication, vol. 29, 31–62.
Martin, J. R. and Rose, D. (2008), Genre Relations: Mapping Culture. London:
Equinox.
Martin, J. R. and White, P. (2005), Language of Evaluation: Appraisal in English.
Basingstoke: Palgrave Macmillan.
Martin, J. R., Zappavigna, M. and Dwyer, P. (forthcoming), ‘Negotiating shame:
Exchange and genre structure in youth justice conferencing’, in C. Lipovsky and
A. Mahboob (eds), Critical Dimensions in Applied Linguistics. Newcastle: Cambridge
Scholars Press.
Martinec, R. (2000a), ‘Types of process in action’. Semiotica, 130, (3/4), 243–68.
Martinec, R. (2000b), ‘Rhythm in multimodal texts’. Leonardo, 33, (4), 289–97.
Martinec, R. (2001), ‘Interpersonal resources in action’. Semiotica, 135, (1/4), 117–45.
Matthiessen, C. M. I. M. (1995), Lexicogrammatical Cartography: English Systems. Tokyo:
International Language Sciences.
McNeill, D. (1992), Hand and Mind: What Gestures Reveal About Thought. Chicago and
London: University of Chicago Press.
236 New Discourse on Language
Morris, D., Collett, P. M. and O’Shaughnessy, M. (1979), Gestures: Their Origins and
Distribution. London: Cape.
Sacks, H. and Schegloff, E. (2002), ‘Home position’. Gesture, 2, (2), 133–46.
Youth Justice Conferencing Directorate (2005), Conference Convenor Training
Package. Unpublished materials distributed at workshops for new convenors
conducted by staff from the NSW Department of Juvenile Justice. NSW,
Australia.
Wang, L. (1936), Zhongguo Yinyunxue (Chinese Phonology). Shanghai: Commercial
Press.
Zappavigna, M., Dwyer, P. and Martin, J. (2008), ‘Syndromes of meaning: exploring
patterned coupling in a NSW Youth Justice Conference’, in A. Mahboob, and
N. Knight (eds), Questioning Linguistics. Newcastle: Cambridge Scholars
Publishing. 103–17.
Zappavigna, M., Cléirigh, C., Dwyer, P. and Martin, J. (forthcoming), ‘Visualising
appraisal prosody’, in A. Mahboob and N. Knight (eds), Appliable Linguistics:
Texts, Contexts and Meanings. London/New York: Continuum.
