The Effect of Gesture on Persuasive Speech

Full text

The effect of gesture on persuasive speech
Judith Peters, Marieke Hoetjes
Centre for Language Studies, Radboud University Nijmegen, the Netherlands
j.l.peters@student.ru.nl, m.hoetjes@let.ru.nl
Abstract
Speech perception is multimodal, with not only speech, but also
gesture presumably playing a role in how a message is
perceived. However, there have not been many studies on
the
effect that hand gestures may have on speech perception in
general, and on persuasive speech in particular. Moreover, we
do not yet know whether an effect of gestures may be larger
when addressees are not involved in the topic of the discourse,
and are therefore more
focus
ed on peripheral cues, rather than
the content of the message. In the current study participants
were shown a speech with or without gestures. Some
participants were involved in the topic of the speech, others
were not. We studied five measures of persuasiveness. Results
showed that for all but one measure, viewing the video with
accompanying gestures made the speech more persuasive.
In
addition, there were several interactions, showing that the
performance of the speaker and the factual accuracy of the
speech scored high especially for those participants who not
only saw gestures but were also not involved in the topic of the
speech.
Index Terms
: multimodal speech perception, gesture,
persuasion, dual processing models
1. Introduction
Human communication is multimodal. When people speak,
they generally use not only speech, but also gestures. In the past
few decades it has been established that speech and gesture are
closely
related
[1, 2]. Despite the ubiquity of gestures in human
communication, they are relatively understudied, and although
the field of gesture studies has grown, with several hypotheses
about the relationship between speech and gesture being
proposed [e.g. 3, 4-7], many aspects of the relationship between
speech and gesture are still unknown. One of the questions often
asked in gesture research is why people gesture when they
speak. Studies have suggested that people gesture not only for
themselves – to help speech production [8]-, but also for the
addressee –to help speech perception [9]. However, most
studies have focused on gesture production, and gesture
perception has received relatively little attention. The current
study focuses on the effect of gesture in speech perception, by
studying the effect of gesture production on the perceived
persuasiveness of a political speech.
Gestures can be defined as symbolic movements of the hands
and arms “related to ongoing talk and to the expressive effort or
intention”[10]. This definition excludes self-adaptors, such as
touching one’s hair or removing an imaginary speck of dust
from one’s clothes. This definition also indicates that speech
and gesture are related. Studies have shown that speech and
gesture are closely related at a semantic, pragmatic, and
temporal level. For example, it has been shown that gesture is
temporally synchronized with co-expressive speech
[1]
. Also,
speech and gesture are acquired together [see 11, for an
overview], and break down together in cases of disfluency, for
example during stuttering [12]. Additional evidence of the close
relationship between speech and gesture has been given by the
fact that also congenitally blind people, who have never seen
anyone gesture, gesture when they speak [8]. In short, these
studies suggest that we cannot
get a complete picture of human
communication if gesture is not taken into account.
In the present study, we focus on the role of gesture in one
specific type of persuasive communication, political speeches.
Political speeches differ from more usual face-to-face
communication in several ways. Speeches tend to be
monologues, and the addressee does not have an active role in
the communicative process
. Also, one of the main goals of a
political speech is to convince addressees about a particular
point of view. To this end, speakers can use various pragmatic
devices. One possible pragmatic device is using hand gestures
while speaking. A question is whether hand gestures play a role
in the extent to which such a speech is considered persuasive.
Only a few previous
studies
have been conducted on the use
of gesture in political speeches. Streeck [13] conducted a case
study in which he discussed the type of hand gestures that 2004
Democratic Party presidential candidates produced during the
primary campaign. He showed that the politicians used many
pragmatic gestures, which presumably help the addressee
visually structure and process speech. However, since this was
a case study, we do not know whether the gestures that the
politicians produced actually had an effect on the audience. A
study by Maricchiolo et al. [14] did take the audience into
account. In their experimental study, they showed participants
video clips of a political speech which was either accompanied
by one of four types of gestures, or by no gestures. The results
were that the participants who saw a video clip with a gesture
found the speech more persuasive than if they did not see a
gesture. However, the definition of gestures as used in the study
by Marrichiolo et al. can be considered problematic (self-
adaptors were considered to be gestures). Also, the participants
in the study were all involved in the topic of the speech. This
could be of importance, because whether an addressee is
involved or not might play a role in how a message is processed.
In communication research, several influential message
processing models have been proposed. The Elaboration
Likelihood Model (ELM) [15] and the Heuristic Systematic
Model (HSM)
[16] both propose that a recipient can be
persuaded by a message in one of two ways, depending on the
level of involvement (also called elaboration). If the addressee
is highly involved, he or she will consider the quality of the
proposed arguments carefully to determine the level of
persuasiveness –this is called the central route. If, however, the
addressee is not that involved, or even uninterested, the
information in the message will be processed more
superficially, and not the quality of the proposed arguments, but
heuristics affect whether the addressee is persuaded by the
Copyright © 2017 ISCA
INTERSPEECH 2017
August 20–24, 2017, Stockholm, Sweden
http://dx.doi.org/10.21437/Interspeech.2017-194659

message – this is called the peripheral route. It might be the case
that nonverbal signals such as gestures are one of the heuristics
used by addressees who follow the peripheral route.
As far as we are aware, only one previous study has
addressed the role of gestures in persuasive communication
while also considering the dual processing models. Jackob,
Roessing, and Petersen [17] presented students with one of
three videos: without vocal emphasis and without gestures, with
vocal emphasis but without gestures, or with vocal emphasis
and gestures. The speech in the video was evaluated with regard
to three dimensions of persuasiveness: performance of the
speaker, characteristics of the speaker, and characteristics of the
argumentation. It was found that although vocal emphasis and
gestures helped
increase the perception of some features of the
speech, the content of the speech was most important. However,
the authors mention that these results can be explained by the
fact that it was likely that the participants, who were highly
educated and involved students, took the central route. This
means that we still do not know what the effect is of gestures in
persuasive speeches for addressees who take the peripheral
speech processing route.
In the present study the goal is to investigate the effect that
hand gestures have in persuasive speeches. We conducted a
study in which participants (students) were presented with a
speech about a local educational issue. Participants either saw a
video of the speech, including the many spontaneous hand
gestures that were produced by the speaker, or listened to the
speech while viewing several photo stills of the speaker. In
addition, some participants were encouraged to take the central
processing route, while others were encouraged to take the
peripheral processing route. The hypothesis is that viewing
hand gestures will make the speech more persuasive for the
addressees, in particular when the addressees are not highly
involved (i.e. when they take the peripheral route).
2. Method
2.1. Participants
In total, 116 participants took part in the experiment (112
females, age M= 25, SD= 8.35, range 18-56 years old). All
participants were (part-time) students of pedagogy at a local
university of applied sciences, and were unfamiliar with the
goal of this study. Participants took part in groups and were
randomly allocated to one of the 4 experimental conditions
.
2.2. Material
Participants were presented with a video of a pre-recorded
speech of about 2 minutes. In this speech, the speaker discussed
the merits of introducing a new final exam at the local
university of applied sciences. The speaker in the video was
introduced to the participants as being a local politician (which
was not actually the case).
There were 2 versions of the speech (manipulating gesture
production). One version (Gesture) used the original sound and
video. In the original video the speaker spontaneously produced
many hand gestures. The other version of the speech (No
Gesture) had the original sound, but instead of the original
video, twelve screenshots taken from the video were presented
(see figure 1 for an example), one after another, in the course of
1 Note that in this study, a higher score indicated less agreement
with the items.
the 2-minute speech. These screenshots showed the speaker at
moments during the speech when he did not produce any
gestures. This means that in the second version, although there
was comparable visual input insofar that the same speaker was
shown, participants did not see any movement.
The speech was introduced to the participants in one of two
ways (manipulating level of elaboration): by either stating that
the speech was about an exam which might be introduced at the
local university of applied sciences next year, i.e. the
participants themselves would have to take this exam (Central
condition), or by stating that the exam might be introduced in
2025, i.e. the participants themselves would not have to take
this exam (Peripheral condition).
To check whether participants who thought they might have
to take the exam next year actually followed the central
processing route, participants, after listening to the speech, were
presented with three 7-point Likert scale items on the level of
elaboration about the exam (α=.78. Example item: The final
general exam will influence my life). Participants in the Central
condition showed more elaboration (M=4.5, SD
=1.19) than
participants in the Peripheral condition (M=2.73, SD=0.97),
t(112)=8.85, p<.001.
Figure 1: Example screenshot from video material.
2.3. Instruments
To analyse persuasion, participants were presented with a
questionnaire after they had listened to and viewed the speech.
Persuasion was analysed using three dimensions
[based on 17].
Each dimension was measured using three 7-point Likert scale
items, ranging from 1 (agree completely) to 7 (disagree
completely) 1. We measured
1) the perceived performance of the speaker (α=.83. Example
item: I thought the speech was lively),
2) the perceived characteristics of the speaker (α=.85. Example
item: I thought the speaker was competent), and
3) the perceived characteristics of the argumentation (α=.61.)
Due to this low reliability, the three items of this final
dimension were analysed separately (I thought the speech was
sincere/factually accurate/interesting).
2.4. Procedure
The experiment was conducted in groups. Participants were
asked to first read a text introducing the experiment, which
contained information about the topic of the speech (the new
final exam), including information on when this new exam was
to be introduced (depending on the condition, either next year,
or 2025). Each group of participants was then presented with
660

the audio and, depending on the condition, the video or the
screenshots of the speech. After the speech had been played,
participants had to fill out the questionnaire individually.
Participants were able to ask questions beforehand and were
debriefed once the questionnaire had been filled out. The total
duration of the experiment for each participant was about 10
minutes.
2.5. Design and analysis
The study consisted of a between subjects design, with 2
independent variables: Gesture (levels: Gesture/No gesture)
and Elaboration (levels: Central/Peripheral). Persuasion was
analysed on three dimensions (see Instruments). Statistical
analyses consisted of 2x2 ANOVAs.
3. Results
3.1. Perceived performance of the speaker
There was no main effect of Elaboration on the perceived
performance of the speaker. However, there was a main effect
of Gesture: participants who saw the speech without gestures
(M=4.71, SD=1.22) were more negative about the performance
of the speaker than the participants who saw the speech with
gestures (M=4.13, SD= 1.18)
,
F(1,111)=7.681, p=.007, η2=.065.
This main effect was qualified by an interaction between
Elaboration and Gesture, F(1,111)=8.02, p=.006, η2=.067.
Participants in the Gesture x Central condition (M=4.59,
SD=1.12) were more negative about the performance of the
speaker
than participants in the Gesture x Peripheral condition
(M=3.68, SD=1.07), see also figure 2.
Figure 2: Mean scores for perceived performance of the
speaker, for all 4 conditions. A higher score indicates less
agreement with the items.
3.2. Perceived characteristics of the speaker
There was a main effect of Elaboration on the perceived
characteristics of the speaker. Participants in the Central
condition (
M=3.60, SD=.93) were more negative about the
characteristics of the speaker than the participants in the
Peripheral condition (M=2.96, SD=1.02), F(1,111)=11.22,
p=.001, η2=.092. There was also a main effect of Gesture.
Participants who saw the speech without gestures (M=3.51,
SD
=.94)
were more negative about the characteristics of the
speaker than the participants who saw the speech with gestures
(M=3.07, SD=1.07) , F(1,111)=4.609, p=.034, η2=.04. There was
no interaction effect between Elaboration and Gesture.
3.3.
Perceived characteristics of the argumentation
Due to the low reliability (α=.61.), the three questions
measuring the perceived characteristics of the argumentation
were analysed separately.
3.3.1. Sincerity of the speech
There was a main effect of Elaboration on the perceived
sincerity of the speech. Participants in the Central condition
(M=3.7, SD=1.10) found the speech less sincere than
participants in the Peripheral condition (M
=3.2,
SD=1.20),
F(1,111)= 5.019, p=.027, η2=.043. There was also a main effect
of Gesture. Participants who saw the speech without gestures
(M=3.73, SD=1.10) found the speech more insincere than
participants who saw the speech with gestures (M=3.17,
SD=1.17) , F(1,111)=6.81, p=.010, η2=.058. There was no
interaction effect between Elaboration and Gesture.
3.3.2. Factual accuracy of the speech
There were no main effects of Elaboration or Gesture on the
perceived factual accuracy of the speech. There was, however,
an interaction effect between Elaboration and Gesture,
F(1,111)=4.72, p
=.032, η
2=041. Participants in the Gesture x
Central condition (M=4.12, SD=1.11) perceived the speech to
be less factually accurate than participants in the Gesture x
Peripheral condition (M=3.37, SD=1.16), see also figure 3.
Figure 3: Mean scores for factual accuracy of the speech, for
all 4 conditions. A higher score indicates less agreement with
the items.
3.3.3. Interestingness of the speech
There was no main effect of Elaboration on the perceived
interestingness of the speech. There was, however, a main
effect of Gesture. Participants who saw the speech without
gestures (M=4.40, SD
=1.49) foun
d the speech less interesting
than participants who saw the speech with gestures (M=3.70,
SD=1.45) , F(1,111)=6.52, p=.012, ƞ2=.055. There was no
interaction effect between Elaboration and Gesture.
4. Discussion
This study focused on one aspect of multimodal speech
perception, namely hand gestures. The goal of the present study
was to study the effect of hand gestures in
political speeches on
the persuasiveness of the speech. The hypothesis was that
viewing hand gestures would make the speech more persuasive
for addressees, in particular when the addressees are not highly
involved (i.e. when they take the peripheral processing route).
The results showed that for four of the five measures,
participants who viewed the video without
hand gestures indeed
found the speech less persuasive than the participants who saw
hand gestures while listening to the speech. When there was an
0
2
4
6
Central Peripheral
Gesture
No gesture
0
2
4
6
Central Peripheral
Gesture
No gesture
661

interaction between level of elaboration and gesture (this was
the case for two measures, performance of the speaker and
factual accuracy) this interaction was as expected, with a
positive effect of gesture only for participants who took the
peripheral route.
The results from this study are in line with previous
experimental work on the effect of gestures on the perception
of persuasive speech [e.g. 14, 17], but extent previous
knowledge in an important way. In the study by Maricchiolo et
al. [14], and the study by Jackob et al. [17], it was found that
viewing gestures made a speech more persuasive. However, in
both studies all participants were presumably involved in the
topic of the speech. Because gestures can be used as a heuristic
cue in message processing, we need to be able to distinguish
between people who are carefully considering the content and
quality of an argument (and thus take the central processing
route), and people who are less involved in a topic and can be
persuaded of the quality of a message by heuristic cues (and
thus take the peripheral processing route). The present study has
made this distinction, and found that for several aspects of
p
ersuasiveness,
there is only an effect of viewing gestures for
people who take the peripheral route. This means that using
hand gestures in a political speech can be a useful strategy when
trying to persuade people who are not that interested in the topic
or who are not paying close attention.
An important alternative explanation of the results is that it
may not be the gestures per se, but other aspects of the moving
image of the video which made the speech in the gesture
condition
more persuasive. In the
gesture condition,
participants were presented with a video, which included
gestures, but also all other movements that the speaker made.
For example facial expressions and overall body movements
could (also) be a reason of someone’s increased level of
persuasiveness. The setup used in this study cannot exclude this
possibility and cannot distinguish between effects of the various
nonverbal elements of the speech
. Also, we cannot rule out that
participants simply considered the gesture condition more fun
to watch merely because of watching a moving image, which
was in turn reflected in the persuasion measures.
In the current study, it was decided to create the no gesture
condition by using the sound of the original video, accompanied
by twelve video screenshots. This way, the speech was
completely
identical in both conditions. As many other aspects
as possible were also kept identical: the screenshots in the no
gesture condition showed the same speaker in the same context
as in the gesture condition. This means that visual aspects such
as the appearance of the speaker were kept constant.
Participants in the no gesture condition were presented with
twelve screenshots in the course of the two minutes that the
speech took, so there was also some ‘movement’ in the no
gesture condition. We decided to manipulate the material this
way because we wanted the speech to be as natural as possible,
and identical between conditions. Creating a no gesture
condition by having the speaker produce the same speech
without gestures would probably have led to changes in aspects
of speech, and could have made the speech less fluent.
However, although the current setup had the advantage of
having identical speech in both conditions, it had the
disadvantage, as mentioned above, of more visual differences
between the conditions than only the use of gestures.
To address these points, an alternative setup in future
research would be one where the no gesture condition only
shows the speaker’s face, and not the hands, as compared to the
gesture condition where the entire upper body can be shown.
Another option would be to use a virtual agent which can be
programmed to use gestures, or not. Virtual agents are also one
of the domains in which
the findings fr
om this study can be
applied to speech technology systems.
The current study focused on political speeches. Naturally,
there are many other situations in which persuasive speech is
used, for example in face to face dialogue. In order to claim that
the current findings apply to persuasive speech in general more
studies in other persuasive speech domains
would have to be
done, with similar findings.
Finally, it can be noted that the politician in the video clip
was male, and practically all participants in the study were
female. It might be the case that male speakers are considered
more (or less) persuasive, and it might also be that the gender
of the participant plays a role. To determine whether this is the
case, future studies could include gender of the speaker and of
the participant in the experimental design.
5. Conclusion
The goal of this study was to determine whether viewing
hand gestures makes a speech more persuasive, in particular
when the addressee is not very involved in the topic. Findings
firstly showed that a political speech without hand gestures was
considered less persuasive than a political speech with hand
gestures.
Secondly, the results showed that when people were
not very involved with the topic, viewing a speech with hand
gestures led to more agreement with items measuring perceived
performance of the speaker and perceived accuracy of the
speech than when perceiving a speech without hand gestures.
We can therefore conclude that viewing gestures can positively
influence speech perception, and that gestures can be used as a
pragmatic device, especially when trying to persuade
addressees who are not very involved or not that interested in
the topic under discussion.
6. Acknowledgements
We would like to thank Raymond Reesink for his help in
producing the material, and Peter Berkers and the teachers at
the HAN for their help in facilitating this study during teaching
hours.
7. References
[1] D. McNeill, Hand and mind. What gestures reveal about thought.
Chicago: University of Chicago Press, 1992.
[2] A. Kendon, Gesture. Visible action as utterance. Cambridge:
Cambridge University Press, 2004.
[3] S. Kita, M. Alibali, and M. Chu, "How do gestures influence
thinking and speaking? The Gesture-for-Conceptualization
Hypothesis", Psychological Review, vol. 124, pp 245-266, 2017.
[4] S. Kita and A. Özyürek, "What does cross-
linguistic variation in
semantic coordination of speech and gesture reveal?: Evidence for
an interface representation of spatial thinking and speaking",
Journal of Memory and Language, vol. 48, pp. 16-32, 2003.
[5] R. M. Krauss, Y. Chen, and R.F. Gottesman, "Lexical gestures
and lexical access: A process model, in Language and gesture
", in
D. McNeill (ed.), Language and Gesture, Cambridge University
Press: New York, 2000, pp. 261-283.
[6] J. P. de Ruiter, "The production of gesture and speech, in
Language and gesture",
in D. McNeill (ed.), Language and
Gesture, Cambridge University Press: Cambridge, 2000, pp. 284-
311.
662

[7] A. B. Hostetter and M. Alibali, "Language, gesture, action! A test
of the gesture as simulated action framework", Journal of Memory
and Language, vol. 63, pp. 245-257, 2010.
[8] J. M. Iverson and S. Goldin-
Meadow,
"Why people gesture when
they speak", Nature, vol. 396, p. 228, 1998.
[9] R. M. Krauss, "Why do we gesture when we speak?", Current
Directions in Psychological Science, vol. 7, pp. 54-
60
, 1998.
[10] M. Gullberg, "Some reasons for studying gesture and second
language acquisition (hommage a Adam Kendon)", International
Review of Applied Linguistics, vol. 44, pp. 103-124, 2006.
[11] M.
Gullberg, K. De Bot, and V. Volterra,
"Gestures and some key
issues in the study of language development", Gesture, vol. 8, no.
2, pp. 149-179, 2008.
[12] R. I. Mayberry and J. Jaques, "Gesture production durin g stuttered
speech: Insights into the nature of gesture-speech integration", in
D. McNeill (ed.), Language and Gesture, Cambridge University
Press: New York, 2000, pp. 199-213.
[13] J. Streeck, "Gesture in political communication: A case study of
the democratic presidential candidates dur
ing the 2004 primary
campaign", Research on Language and Social Interaction, vol.
41, no. 2, pp. 154-186, 2008.
[14] F. Maricchiolo, F., A. Gnisci, M. Bonaiuto, and G. Ficca, "Effects
of different types of hand gestures in persuasive speech on
receivers' evalu
ations
", Language and cognitive processes, vol.
24, no. 2, pp. 239-266, 2009.
[15] R. E. Petty and J.T. Cacioppo, Communication and Persuasion.
Central and Peripheral Routes to Attitude Change. New York:
Springer, 1986.
[16] S. Chaiken,
"
Heuristic versus systematic processing and the use
of source versus message cues in persuasion", Journal of
Personality and Social Psychology, vol. 39, no. 5, pp. 752-766,
1980.
[17] N. Jackob, T. Roessing, and T. Petersen, "The effects of verbal
and nonverbal elements in persuasive communication: Findings
from two multi-method experiments", Communications, vol. 36,
pp. 245-271, 2011.
663