Conference PaperPDF Available

Why do you think this joke told by robot is funny? The humor style matters

Why do you think this joke told by robot is funny? The humor style
Heng Zhang1and Chuang Yu2and Adriana Tapus1
Abstract Humor usually plays a positive role in social activ-
ities. We posit that endowing a social robot with humor ability
can enhance expressive human-robot interaction. People’s per-
ception on humor is different, and therefore, making the robot
expressing humor in an appropriate way is a challenge. The
main aim of this paper is to explore the correlation between
people’s perception on different types of jokes and their humor
styles (Affiliative, Self-enhancing, Self-defeating, Aggressive).
In the experiment, we used the humanoid robot Pepper to
perform different types of jokes. Both subjective (jokes rating)
and objective measures (RGB and thermal images) were used.
The latter method was employed to extract facial features
(facial action unit and facial temperature). After extracting and
analyzing the data of both measurement methods, we found
that the Self-defeating humor style positively affects people’s
rating on all types of jokes. In addition, there is also a positive
correlation between people’s humor style scores and the degree
of happiness.
Humor is pervasive in our social life. We can enjoy humor
in various situations, which is attributed to the positive role of
humor in social events [1]. Appropriate humor can make the
social atmosphere relaxed and conducive to communication
[2]. Those who are good at using humor in social situations
are also appreciated by more people. They will meet less
communication barriers and build better interpersonal rela-
tionship with others [3] [4].
In addition to promote the interpersonal communication,
previous studies also show that the appropriate use of humor
can have many other positive effects. In education, humor
has been shown to reduce students’ tension and stress [5],
and also contribute to improve student’s score in exams [6].
Humor is also used by nurses as a strategy to alleviate
patients’ fears of illnesses and injury [7]. In team work,
humor can help leader construct effective leadership and
facilitate cohesion between team members [8], [3].
Humor has so many benefits, yet humans are considered
to be the only creatures capable of using and understanding
humor. As a result, humor is possible to be an essential
trait for human beings from the evolutionary perspective
[9]. As man-made intelligent agent, the robots are expected
*This work was supported by ENSTA Paris, Institut Polytechnique de
Paris, France and the CSC PhD Scholarship.
1Heng Zhang is with Phd candidate of U2IS, ENSTA Paris, Institut Poly-
technique de Paris, Paris, France
2Chuang Yu is with the postdoctor of Cognitive Robotics
Laboratory, University of Manchester, Manchester, UK
1Adriana Tapus is with the faculty of U2IS, EN-
STA Paris, Institut Polytechnique de Paris, Paris, France
to be endowed with human social capabilities, and humor
should be an indispensable one. According to the CASA
paradigm [10], the positive effect of humor in human-human
interaction will also be seen in human-robot interaction.
Humor is complex and affected by many different factors.
Consequently, it is necessary to research what factors influ-
ence people’s perception of humor and what role they play.
In this paper, we aim to explore how the humor style affect
people’s perception on some specific humorous contents.
The rest of the paper is organized as follows: firstly,
we discuss some relevant related work about humor style
and humor-robot interaction in Section II; Our experimental
design and results are presented in Section III and Section
IV, respectively. Finally, the conclusions and the perspective
on future works are part of Section V.
A. Humor Style
People’s sense of humor is affected by multi-factors such
as society, family, culture, and hence, the humor style is
different among people [11]. Although sense of humor is
multi-faceted, it is considered a stable personality trait for in-
dividuals [12]. Consequently, psychologists tried to quantify
and classify human’s humor styles into several categories.
They proposed some measurement methods, such as Coping
Humor Scale(CHS) [13], Sense of Humor Scale(MSHS)
[14], Sense of Humor Questionnaire(SHQ) [15], Humor
Style Questionnaire(HSQ) [16] and many others. Among
them, currently the most widely used one is Martin’s Humor
Style Questionnaire(HSQ) [16]. This is also the humor model
used in this paper.
Affiliative humor involves amusing all people by saying
funny things. Those who are high in Affiliative humor tend
to put others at ease and use humor to facilitate interper-
sonal relationships [17]. Compared with Affiliative humor,
Self-enhancing humor focuses more on intrapsychic part. It
involves high self-esteem as well as facing negative aspects
with a positive mindset [16]. Self-defeating humor is a kind
of negative humor style, which relates to low self-esteem and
avoidance [18]. People who are high in Self-defeating humor
tend to amuse others by degrading themselves. The “class
clown” is the embodiment of this humor style. Aggressive
humor is another humor style associated with negativity. It
involves expressing humor without considering the potential
negative impact on other people [16].
Research on humor type is providing researchers with
a new perspective on some traditional research questions.
Through the analysis of participants’ humor styles, Hampes
et al. [19] found that loneliness personality trait is negatively
correlated with Affiliative humor, and positively correlated
with Self-defeating humor. Furthermore, Neves et al. [20]
researched the role of humor style in the relationship be-
tween leaders and employees. They drew the conclusion that
Affiliative and Self-enhancing humor styles of the leader
could help employees build trust in the leader, especially for
employees with low core-self-evaluations, but the Aggressive
humor could have the opposite effect. Schneider et al. [21]
conducted meta-analyses on previous studies. They found an
association between humor styles and mental health, and also
that different humor styles can impact differently on mental
B. Social Robot with Humor
Social robots are playing more and more important roles
in human society. Especially during the Covid-19 pandemic,
there is a growing requirement and interest in using social
robots in our daily life [22] [23]. Compared to industrial
robots, people expect that a social robot not only provides
functional capabilities but also emotional ones [24]. Since
happiness is a kind of positive emotion that benefits physical
and mental health [25], humor that directly and efficiently
induces happiness is naturally a very promising ability for
social robots. In addition, a survey has demonstrated that
humor could reduce people’s tension, and also compensate
for imperfect performance of robots in interaction, resulting
in better interaction experience [26].
There has been some studies on the factors that affect
people’s perception on robot humor and the roles humor
plays in Human-Robot Interaction. After analyzing 22 per-
formances in the wild, Vilk et al. [27] drew conclusion that
the good timing makes the performance significantly funnier
when the social robot delivers a talk show. Wendt et al. [28]
researched the nonverbal factors in robot humor. The results
showed that the humorous nonverbal behavior would make
the robot more human-like and entertaining. Weber et al. [29]
proposed an approach to build users’ models dynamically
so that the robot could adapt its humor according to users’
reaction to the previous joke. Menne et al. [30] found that
robot told clever jokes was perceived as more intelligent.
Therefore, the user’s trust in the robot increases when it
performs the various tasks. In the medical context, Johanson
et al. [31] demonstrated through repeated measures that the
humorous robot was rated as having more empathy and
sociable personality, resulting in increasing patients’ positive
In this work, we aim exploring the factors and mechanisms
that influence human’s perception on robot’s humor. Our
current work mainly focuses on the relationship between
individuals’ humor styles and their perception on different
types of jokes told by a social robot.
A. Experimental Platform
In this work, we used the social robot Pepper to tell
jokes. It has 17 degrees of freedom, which allows it to
make a variety of gestures. During the experiment, we
used both RGB and thermal cameras to record participants’
facial features. The RGB camera is a Logitech HD webcam
C930e (1080p, 30FPS), and the thermal camera is an Optris
PI 640i USB-powered Infrared Camera (640×480, 32FPS,
temperature range -20C˜100C). These two cameras were
fixed on an octopus tripod. The experimental setup is shown
in Figure 1.
Fig. 1. Experimental setup
ROS was used to process the thermal and RGB images.
The framework is as shown in Figure 2. The extracted
temperatures and AU data were recorded into a .csv file.
The process was done offline.
Fig. 2. Thermal and RGB image processing framework
B. Performance Content
During the experiment, Pepper robot tells four kinds of
jokes accompanied by gestures.
1) Jokes: We selected 20 jokes from both the Jester joke
dataset and quotes from famous talk-show comedians. we
chose 5 jokes for each of the four humor styles. The selection
criteria were as follows.
Affiliative humor style: This kind of jokes should be
accepted by most people. There are no strong offensive
connotations in these jokes. The positive or negative
comments in jokes are not directed at anyone present.
Self-enhancing humor style: This kind of jokes are often
told from a first-person perspective. The content should
be positive and demonstrate the speaker’s optimism
about the event in the joke.
Self-defeating humor style: The same with the Self-
enhancing humor style, this kind of jokes are also
Humor Style Joke
Affiliative Husband asked his wife: Why is it good that there are female astronauts?
Wife replied: Because if the crew gets lost, at least the woman will ask for directions.
Self-enhancing If you think you are too small to be effective, you have never been in the dark with a mosquito.
Self-defeating I remember the time I was kidnapped and they sent back a piece of my finger to my father. My old man said he wanted more proof.
Aggressive Q: If your dog is barking at the back door, and your wife is yelling at the front door, who do you let in first?
A: The dog, of course. It will shut up once you let it in.
usually told from the first-person perspective. However,
these jokes are negative and usually speakers make fun
of their own shortcomings to amuse others.
Aggressive humor style: This kind of jokes usually
mock someone with a certain aggressiveness.
Some examples of the jokes used can be found in Table
2) Verbal and Paraverbal Robot Behavior: Microsoft
Azure TTS (Text to Speech) was used to generate Pepper’s
speech. According to the previous research, the prosody is
a key factor that affect people’s perception on humor [32].
Therefore, we used a series of SSML (Speech Synthesis
Markup Language) format scripts as the input of Azure
TTS, so that we can control robot’s speaking prosody. One
example of the SSML script is as shown in Figure 3.
Fig. 3. SSML script
Some of the participants in the experiment are Chinese.
Therefore, we translated all the jokes into Chinese and also
generated the corresponding Chinese version of the speech.
3) Nonverabal Robot Behavior - Gestures: Pepper’s
ALAnimatedSpeech function is able to generate gestures
automatically based on the speech content. Since the gestures
generated by this method have a certain randomness, we
choreographed Pepper’s gestures manually by Choregraphe.
The previous studies have shown that iconic gestures
contribute to illustrate the speech contents [33], and improve
people’s ratings on robot naturalness [34]. Consequently,
we designed some iconic gestures when choreographing
the robot gestures. For example, there was a scene where
one person slapped another person in one joke. When the
performance came to this episode, Pepper also made a cor-
responding punching motion, which made the performance
more interesting.
C. Questionnaires
Participants had to fill out two questionnaires during the
experiment. The first one was used to collect ratings on the
funniness of the 20 jokes, and the second one was used to
evaluate participants’ humor styles.
1) Jokes Rating Questionnaire: In order to check if there
are significant differences between the four types of jokes in
terms of funniness, we made a questionnaire including the
20 jokes previously mentioned. Participants were asked to
rate each of the jokes from 1 to 10, where 1 is associated to
”Not funny at all” and 10 to ”Very funny”.
2) Humor Style Questionnaire: As previously mentioned,
the Humor Style Questionnaire (HSQ) developed by Martin
et al. [16] is widely used to measure individual’s humor
style. This questionnaire is capable of measuring two positive
and two negative dimensions of humor: Affiliative, Self-
enhancing, Aggressive, and Self-defeating, respectively. The
Affiliative humor is a positive style of humor, used to
amuse others, to facilitate relationships and interactions, and
to reduce interpersonal tension [17]. The Self-enhancing
humor is also a positive style of humor. Individuals with
this humor style tend to have a humorous perspective even
in situations of stress or adversity [35]. The Aggressive
humor is a negative style of humor related to sarcasm and
derision and used to manipulate others. And finally, the Self-
defeating humor is also a negative style of humor. Individuals
characterized by this humor style tend to amuse others by
doing or saying funny things at one’s own expense [16].
The questionnaire is a 32-item self-report inventory with
each question on a 7 point scale (ranging from totally agree
to totally disagree).
D. Scenario
The experiment was conducted in a quiet room in the lab-
oratory in order to avoid distractions from the surroundings.
Participants were asked to relax and get acquainted with
the environment. Then, we started the robot performance
and began to record participants’ facial features by RGB
and thermal cameras. During the whole process, Pepper
performed all the jokes of one type before passing to the next
type. The performance order was: Affiliative, Self-enhancing,
Self-defeating, Aggressive, respectively.
There was a ten-seconds interval between each joke. Every
time Pepper finished telling a joke, the participant would
rate robot’s performance from 1 to 10, where 1 is ”Not
funny at all” and 10 is ”Very funny”. At the end of the
experiment, participants were asked to fill out the Humor
Style Questionnaire (HSQ).
A. Participants
14 participants took part in this experiment in total. After
the experiment, we found that data from two participants was
not completely recorded. Therefore, we removed these two
participants from our analysis. The 12 remaining participants
(2 female and 10 male), are students of Institut Polytechnique
de Paris (IP Paris). Their age ranges from 20 to 25 (2
participants, 16.7%) and from 26 to 30 (10 participants,
83.3%), respectively.
For each participant, HSQ (Humor Style Questionnaire)
provides a value for each of the four dimensions of humor
(that are considered independent, even if there are some
positive correlations among them [16]). According to the
results of HSQ (Humor Style Questionnaire), 8 participants
have a dominant Affiliative humor style (66.7%), 3 partic-
ipants are mainly characterized by Self-enhancing humor
style (25.0%), and 1 participant is mainly characterized by
Aggressive humor style (8.3%). No participant is mainly
characterized by the Self-defeating humor style. However,
HSQ provides for each participant a score in each of the
four humor styles.
B. Online Joke Ratings
The Joke Rating Questionnaire was posted online. 77 stu-
dents mainly from Institut Polytechnique de Paris (France),
University of Manchester (UK), and Harbin Institute of
Technology (China) participated in the survey. The average
rating was calculated for each joke. Out of 5 jokes from
each humor type, we selected 4 jokes that have near ratings.
This allowed us to avoid for significant bias on experimental
results due to large differences of funniness between each
type of jokes. The average ratings of each type of jokes after
selection are shown in TableII.
Humor Style Joke 1 Joke 2 Joke 3 Joke 4 Average ratings
Affiliative 4.857 4.922 4.494 5.494 4.942
Self-enhancing 5.195 4.792 5.403 5.454 5.211
Self-defeating 4.571 5.039 4.961 4.753 4.831
Aggressive 4.195 4.532 5.338 5.441 4.877
TableII resumes the means of the 77 participants’ ratings
for the 16 selected jokes. The last column indicates the
average ratings of each type of joke. From the data, it can
be observed that each type of joke has very close ratings.
We analysed each humor style one by one, and we have
found some trends.
Each participant rated the funniness of the 16 jokes (4 per
humor style). For each participant and for each joke humor
style, an average score (on the 4 jokes per joke humor style)
was calculated. We checked the normality of data by using
Shapiro Wilk test firstly. The results are as following: for
Self-defeating humor style scores, p = 0.414 >0.05; for the
ratings of Affiliative type of jokes, p = 0.085 >0.05; for the
ratings of Self-enhancing type of jokes, p = 0.116 >0.05; for
the ratings of Aggressive type of jokes, p = 0.282 >0.05.
C. Face to Face Robot Interaction: Participants’ Ratings
After the robot’s performance, the participants rated each
joke from 1 to 10. The average ratings for each type joke
are shown in TableIII
Humor Style Affiliative Self-enhancing Self-defeating Aggressive
Average Ratings 4.854 4.958 5.0 5.375
All data satisfy the normal distribution. Therefore, we
calculated the Pearson correlation coefficient between par-
ticipant humor style and ratings of each humor style joke,
respectively. Statistical analysis yielded the following signif-
icant results.
The results showed that there are significant positive cor-
relations between the scores of participant’s Self-defeating
humor style and the ratings of each type of jokes. For
Affiliative type of jokes, r = 0.655, p = 0.021 <0.05; for Self-
enhancing type of jokes, r = 0.594, p = 0.042 <0.005; for
Self-defeating type of jokes, r = 0.679, p = 0.015 <0.05; for
Aggressive type of jokes, r = 0.642, p = 0.024 <0.05. Figure
4 shows the trends and distributions of participants’ ratings
on each type of jokes in different Self-defeating humor style
score. Individuals with higher Self-defeating scores will rate
higher the funniness of all types of jokes. This is similar
with other works, e.g., in [36], the authors have found that
self-defeating humor style can be seen as a facilitator for
social relationships at one’s own expense, especially in the
Chinese culture (our participants are 80% Chinese).
In addition, the Kruskal-Wallis one-way analysis of vari-
ance method was also applied to test participants’ ratings on
the four types of jokes, and the result shows that there are
significant differences between participants’ ratings on the
different types of jokes (K = 23.384, p <0.05).
D. Facial Action Unit Processing Results
The Facial Action Coding System (FACS) [37] is widely
used for facial action identification. According to the mech-
anism of FACS, we can recognize human’s emotion through
the combination of several AUs. In the previous research
papers [38] [39], some considered the combination of AU6
Fig. 4. Participants’ Ratings and Self-defeating Humor Style Scores. In all
four box plots, the horizontal coordinates represent the Self-defeating humor
style scores of participants; the vertical coordinates represent participants’
average ratings on jokes
and AU12 as the sign of happiness. There are also some
studies suggested that AU12 is the most critical sign of
happiness, and other AUs are not necessary [40] [41].
In our experiments, we found that AU6 (cheek raiser)
and AU12 (lip corner puller) appear simultaneously only
when a smile reaches a certain intensity. When the smile
is weak, only AU12 can be detected. The research in [42]
also supports our findings. Considering that the intensity of
the smile is individual dependent, we only used data from
AU12 to assess participants’ smiles.
We used OpenFace to process action units. An example of
a processed image is as shown in Figure 5. OpenFace served
as a Rosnode, it published messages containing AU12 data
as well as the timestamps at 30 FPS.
Fig. 5. RGB (left) and Thermal (right) Image Processing
According to the time spent on each joke and the times-
tamps, we divided the data into four segments corresponding
to the four humor types of jokes. Each segment contains
data measured during the robot’s performance for a type
of joke. In every frame where the AU12 was detected, the
presence data of AU12 becomes 1, otherwise it is 0. We
counted the number of AU12 detected during each type of
joke. Consequently, for each participant, we obtained four
AU12 numbers (corresponding to the four segments). The
degree of laughter varied greatly from person to person and
the number of detected AU12 between participants varied
a lot. Therefore, we normalized the AU12 data for each
participant: the AU12 number for each type of joke is divided
by the AU12 total number for the whole performance (e.g.,
the normalized AU12 number in Affiliative type jokes =
AU12 number in Affiliative type jokes / the total number
of AU12 in the whole performance). The normalized values
are between 0 and 1.
Since the participants’ humor style scores and AU12
data satisfied normal distribution, we calculated the Pearson
correlation coefficient. The obtained result (r = 0.3434, p
<0.05) shows that there is a positive correlation between
individual’s humor style scores and the number of AU12.
In other words, the happiness (as per number of smiles)
increases with the humor style scores.
E. Thermal Features Processing Results
During the experiment, both thermal and RGB images
were recorded simultaneously. We selected six regions on
the human face to detect the thermal features. These are
forehead, right cheek, left cheek, nose, lower lip, and chin,
respectively. These regions are as shown in the right part of
Figure 5. When processing the thermal images, the average
temperature of each of these six regions in each frame is
extracted and recorded in a csv format file. We analysed
each participant’s temperature data with his/her humor style
scores. We found that people who have high Self-enhancing
scores will have higher lip temperature variation slope when
watching Aggressive type jokes than those who have lower
scores of Self-enhancing humor style (Shapiro-Wilk test, p
= 0.313 >0.05; one-way ANOVA, p = 0.014 <0.05). Due
to the intra and inter variability of participants with respect
to humor style, we will continue investigating the variation
of the temperature on the face of participants with a larger
subject pool.
In this paper, we mainly explored the relationship between
individuals’ perception on robot humor and their humor
styles. In our experiment, we used both subjective mea-
surement methods (participants’ self-rating) and objective
measurement methods (thermal and RGB image processing).
After analysing 12 participants data, we found that the
Self-defeating humor style significantly affects participants
ratings on all types of jokes. With the increasing of the
Self-defeating humor style scores, participants ratings on
jokes also increase. In addition, there is also a positive
correlation between participants’ humor style scores and
degree of happiness/smile.
In our current work, there are still some limitations. Due
to Covid-19 pandemic, the closed experimental environment
and the necessity of removing the masks during the exper-
iments, the number of participants (only 14 participants)
is limited. Furthermore, during the experiments we had
to consider health and safety procedures. In addition, the
jokes (the dataset) selected for this work are mainly from
American joke websites, which have some American cultural
Future work will focus on the development of some
interactive scenarios, in which participants will be able to
experience robot’s humor in a more immersive experimental
setup, not just as a spectator.
This work was supported by ENSTA Paris, Institut Poly-
technique de Paris (IP Paris), France and China Scholarship
Council (CSC) (PhD scholarship).
[1] N. A. Kuiper, R. A. Martin, and K. A. Dance, “Sense of humour
and enhanced quality of life,” Personality and individual differences,
vol. 13, no. 12, pp. 1273–1283, 1992.
[2] S. A. Lei, J. L. Cohen, and K. M. Russler, “Humor on learning
in the college classroom: Evaluating benefits and drawbacks from
instructors’ perspectives, Journal of instructional Psychology, vol. 37,
no. 4, pp. 326–332, 2010.
[3] E. Romero and A. Pescosolido, “Humor and group effectiveness,
Human Relations - HUM RELAT, vol. 61, pp. 395–418, 03 2008.
[4] A. Ziv, “The social function of humor in interpersonal relationships,
Society, vol. 47, pp. 11–18, 01 2009.
[5] C. D. Rainsberger, “Reducing stress and tension in the classroom
through the use of humor. 1994.
[6] A. Ziv, “Teaching and learning with humor: Experiment and replica-
tion,” The Journal of Experimental Education, vol. 57, no. 1, pp. 4–15,
[7] J. S. Dowling, “Humor: A coping strategy for, Pediatric nursing,
vol. 28, no. 2, 2002.
[8] J. Holmes and M. Marra, “Humor and leadership style,” 2006.
[9] M. Gervais and D. S. Wilson, “The evolution and functions of laughter
and humor: A synthetic approach,” The Quarterly review of biology,
vol. 80, no. 4, pp. 395–430, 2005.
[10] B. Reeves and C. Nass, The media equation: How people treat
computers, television, and new media like real people. Cambridge
university press Cambridge, United Kingdom, 1996.
[11] A. Cann and C. Matson, “Sense of humor and social desirability:
Understanding how humor styles are perceived, Personality and
Individual Differences, vol. 66, pp. 176–180, 2014.
[12] W. Ruch, The sense of humor: Explorations of a personality charac-
teristic. Walter de Gruyter, 2010, vol. 3.
[13] R. A. Martin and H. M. Lefcourt, “Sense of humor as a moderator of
the relation between stressors and moods.” Journal of personality and
social psychology, vol. 45, no. 6, p. 1313, 1983.
[14] J. A. Thorson and F. C. Powell, “Development and validation of a mul-
tidimensional sense of humor scale,” Journal of clinical psychology,
vol. 49, no. 1, pp. 13–23, 1993.
[15] S. Svebak, “The development of the sense of humor questionnaire:
From shq to shq-6,” 1996.
[16] R. A. Martin, P. Puhlik-Doris, G. Larsen, J. Gray, and K. Weir, “Indi-
vidual differences in uses of humor and their relation to psychological
well-being: Development of the humor styles questionnaire, Journal
of research in personality, vol. 37, no. 1, pp. 48–75, 2003.
[17] H. M. Lefcourt, Humor: The psychology of living buoyantly. Springer
Science & Business Media, 2001.
[18] M. S. Fabrizi and H. R. Pollio, A naturalistic study of humorous
activity in a third, seventh, and eleventh grade classroom,” Merrill-
Palmer Quarterly (1982-), pp. 107–128, 1987.
[19] W. P. Hampes, “Correlations between humor styles and loneliness,”
Psychological Reports, vol. 96, no. 3, pp. 747–750, 2005.
[20] P. Neves and G. Karagonlar, “Does leader humor style matter and to
whom?” Journal of managerial psychology, 2020.
[21] M. Schneider, M. Voracek, and U. S. Tran, ““a joke a day keeps the
doctor away?” meta-analytical evidence of differential associations of
habitual humor styles with mental health,” Scandinavian Journal of
Psychology, vol. 59, no. 3, pp. 289–300, 2018.
[22] S. S. Kim, J. Kim, F. Badu-Baiden, M. Giroux, and Y. Choi, “Prefer-
ence for robot service or human service in hotels? impacts of the
covid-19 pandemic,” International Journal of Hospitality Manage-
ment, vol. 93, p. 102795, 2021.
[23] L. Aymerich-Franch and I. Ferrer, “The implementation of
social robots during the covid-19 pandemic,” arXiv preprint
arXiv:2007.03941, 2020.
[24] B. T. Tay, S. C. Low, K. H. Ko, and T. Park, “Types of humor that
robots can play, Computers in Human Behavior, vol. 60, pp. 19–28,
[25] R. T. Howell, M. L. Kern, and S. Lyubomirsky, “Health benefits: Meta-
analytically determining the impact of well-being on objective health
outcomes,” Health Psychology Review, vol. 1, no. 1, pp. 83–136, 2007.
[26] A. Nijholt, A. I. Niculescu, A. Valitutti, and R. E. Banchs, “Humor
in human-computer interaction: a short survey,” Adjunct Proceedings
of INTERACT, pp. 527–530, 2017.
[27] J. Vilk and N. T. Fitter, “Comedians in cafes getting data: evaluating
timing and adaptivity in real-world robot comedy performance, in
Proceedings of the 2020 ACM/IEEE international conference on
human-Robot Interaction, 2020, pp. 223–231.
[28] C. S. Wendt and G. Berg, “Nonverbal humor as a new dimension
of hri,” in RO-MAN 2009-The 18th IEEE International Symposium
on Robot and Human Interactive Communication. IEEE, 2009, pp.
[29] K. Weber, H. Ritschel, I. Aslan, F. Lingenfelser, and E. Andr´
e, “How
to shape the humor of a robot-social behavior adaptation based on
reinforcement learning,” in Proceedings of the 20th ACM international
conference on multimodal interaction, 2018, pp. 154–162.
[30] I. M. Menne, B. P. Lange, and D. C. Unz, “My humorous robot:
effects of a robot telling jokes on perceived intelligence and liking,
in Companion of the 2018 ACM/IEEE International Conference on
Human-Robot Interaction, 2018, pp. 193–194.
[31] D. L. Johanson, H. S. Ahn, J. Lim, C. Lee, G. Sebaratnam, B. A.
MacDonald, and E. Broadbent, “Use of humor by a healthcare robot
positively affects user perceptions and behavior, 2020.
[32] R. Oliveira, P. Arriaga, M. Axelsson, and A. Paiva, “Humor–robot
interaction: a scoping review of the literature and future directions,
International Journal of Social Robotics, vol. 13, no. 6, pp. 1369–
1383, 2021.
[33] C. Bartneck, T. Belpaeme, F. Eyssel, T. Kanda, M. Keijsers, and
S. ˇ
c, Human-robot interaction: An introduction. Cambridge
University Press, 2020.
[34] C.-M. Huang and B. Mutlu, “Multivariate evaluation of interactive
robot systems,” Autonomous Robots, vol. 37, no. 4, pp. 335–349, 2014.
[35] N. A. Kuiper, R. A. Martin, and L. J. Olinger, “Coping humour,
stress, and cognitive appraisals. Canadian Journal of Behavioural
Science/Revue canadienne des sciences du comportement, vol. 25,
no. 1, p. 81, 1993.
[36] Y.-C. Chan, W.-C. Hsu, Y.-J. Liao, H.-C. Chen, C.-H. Tu, and C.-
L. Wu, Appreciation of different styles of humor: An fmri study,”
Scientific reports, vol. 8, no. 1, pp. 1–12, 2018.
[37] P. Ekman and W. V. Friesen, “Facial action coding system, Environ-
mental Psychology & Nonverbal Behavior, 1978.
[38] M. H. Mahoor, S. Cadavid, D. S. Messinger, and J. F. Cohn, A
framework for automated measurement of the intensity of non-posed
facial action units,” in 2009 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition Workshops. IEEE, 2009,
pp. 74–80.
[39] B. Fasel and J. Luettin, “Recognition of asymmetric facial action
unit activities and intensities, in Proceedings of the International
Conference on Pattern Recognition (ICPR 2000), vol. 1, no. CONF,
2000, pp. 1100–1103.
[40] M.-A. Ruiz-Belda, J.-M. Fern´
andez-Dols, P. Carrera, and K. Barchard,
“Spontaneous facial expressions of happy bowlers and soccer fans,
Cognition and Emotion, vol. 17, no. 2, pp. 315–326, 2003.
[41] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and
I. Matthews, “The extended cohn-kanade dataset (ck+): A complete
dataset for action unit and emotion-specified expression,” in 2010
ieee computer society conference on computer vision and pattern
recognition-workshops. IEEE, 2010, pp. 94–101.
[42] Y. Tong, W. Liao, and Q. Ji, “Facial action unit recognition by exploit-
ing their dynamic and semantic relationships,” IEEE transactions on
pattern analysis and machine intelligence, vol. 29, no. 10, pp. 1683–
1699, 2007.
The ability to express semantic co-speech gestures in an appropriate manner of the robot is needed for enhancing the interaction between humans and social robots. However, most of the learning-based methods in robot gesture generation are unsatisfactory in expressing the semantic gesture. Many generated gestures are ambiguous, making them difficult to deliver the semantic meanings accurately. In this paper, we proposed a robot gesture generation framework that can effectively improve the semantic gesture expression ability of social robots. In this framework, the semantic words in a sentence are selected and expressed by clear and understandable co-speech gestures with appropriate timing. In order to test the proposed method, we designed an experiment and conducted the user study. The result shows that the performances of the gesture generated by the proposed method are significantly improved compared to the baseline gesture in three evaluation factors: human-likeness, naturalness and easiness to understand.
Facial expressions are one of the most practical and straightforward ways to communicate emotions. Facial Expression Recognition has been used in lots of fields such as human behaviour understanding and health monitoring. Deep learning models can achieve excellent performance in facial expression recognition tasks. As these deep neural networks have very complex nonlinear structures, when the model makes a prediction, it is not easy for human users to understand what is the basis for the model’s prediction. Specifically, we do not know which facial units contribute to the classification more or less. Developing affective computing models with more explainable and transparent feedback for human interactors is essential for a trustworthy human-robot interaction. Compared to “white-box" approaches, “black-box” approaches using deep neural networks, which have advantages in terms of overall accuracy but lack reliability and explainability. In this work, we introduce a multimodal affective human-robot interaction framework, with visual-based and verbal-based explanation, by Layer-Wise Relevance Propagation (LRP) and Local Interpretable Mode-Agnostic Explanation (LIME). The proposed framework has been tested on the KDEF dataset, and in human-robot interaction experiments with the Pepper robot. This experimental evaluation shows the benefits of linking deep learning emotion recognition systems with explainable strategies.KeywordsExplainable roboticsFacial Expression Recognition (FER)eXplainable Artificial Intelligence (XAI)Human-Robot Interaction (HRI)
The natural co-speech facial action as a kind of non-verbal behavior plays an essential role in human communication, which also leads to a natural and friendly human-robot interaction. However, a lot of previous works for robot speech-based behaviour generation are rule-based or handcrafted methods, which are time-consuming and with limited synchronization levels between the speech and the facial action. Based on the Generative Adversarial Networks (GAN) model, this paper developed an effective speech-driven facial action synthesizer, i.e., given an acoustic speech, a synchronous and realistic 3D facial action sequence is generated. In addition, a mapping between the 3D human facial action to the real robot facial action that regulates Zeno robot facial expressions is also completed. The evaluation results show the model has potential for natural human-robot interaction.KeywordsSocial robotFace actionHuman-robot interaction
Full-text available
Abstract Humor is a pervasive feature of everyday social interactions that might be leveraged to improve Human-Robot Interactions (HRI). Goals: Our goal is to evaluate how the use of humor can improve HRI and enhance the user’s perception of the robot, as well as to derive implications for future research and development of humorous robots. Method: We conducted a systematic search of 7 digital libraries relevant in the areas of HRI and Psychology for papers that were relevant to our goal. We identified 431 records, published between 2000 and August of 2020, of which 12 matched our eligibility criteria. The included studies reported the results of original empirical research that involved direct or video-mediated interaction of humans and robots. Results and Conclusion: Humor seems to have a positive effect in improving the user’s perception of the robot, as well as the user’s evaluation of the interaction. However, the included studies present a number of limitations in their approaches to robotic humor that need to be surpassed before reaching a final verdict on the value of humor in HRI.
Full-text available
Purpose The interest on leader humor styles is recent. By applying a trustworthiness framework, the authors examine (1) how leader humor styles contribute to performance and deviance via trust in the supervisor and (2) who benefits/suffers the most from different leader humor styles. Design/methodology/approach The authors tested their hypotheses in a sample of 428 employee–supervisor dyads from 19 organizations operating in the services sector. Findings Affiliative and self-enhancing leader humor styles are particularly beneficial for employees with low core-self-evaluations, helping them develop trust in the supervisor and consequently improving their performance. An aggressive leader humor style, via decreased trust in the supervisor, reduces performance, regardless of employees' core self-evaluations. Self-enhancing and self-defeating leader humor styles also present significant relationships with organizational deviance. Research limitations/implications Limitations include the cross-sectional design and the limited number of mechanisms examined. Practical implications Organizations need to train leaders in the use of humor and develop a culture where beneficial humor styles are endorsed, while detrimental humor styles are not tolerated. Originality/value These findings contribute to the literatures on trust and humor, by showing that the use of humor is not as trivial as one could initially think, particularly for those with low core self-evaluations, and by expanding our knowledge of the mechanisms by which different leader humor styles may influence performance and deviance.
Full-text available
The role of robots in society keeps expanding and diversifying, bringing with it a host of issues surrounding the relationship between robots and humans. This introduction to human-robot interaction (HRI), written by leading researchers in this developing field, is the first to provide a broad overview of the multidisciplinary topics central to modern HRI research. Students and researchers from robotics, artificial intelligence, psychology, sociology, and design will find it a concise and accessible guide to the current state of the field. Written for students from diverse backgrounds, it presents relevant background concepts, describing how robots work, how to design them, and how to evaluate their performance. Self-contained chapters discuss a wide range of topics, including the different communication modalities such as speech and language, non-verbal communication and the processing of emotions, as well as ethical issues around the application of robots today and in the context of our future society.
Full-text available
Humor styles are important in facilitating social relationships. Following humor styles theory, this functional magnetic resonance imaging (fMRI) study is the first to use “one-liner” humor to investigate the neural correlates involved in appreciating humor styles that differ in terms of target (self or other) and motivation (benign or detrimental). Interestingly, we observed greater activation in response to humor that facilitates relationships with others (self-defeating and affiliative humor) than to humor that enhances the self (self-enhancing and aggressive humor). Self-defeating humor may play an important role in Chinese culture in facilitating social relationships at one’s own expense. Psychophysiological interaction (PPI) analysis revealed temporal pole (TP)-frontal functional connectivity underlying the appreciation of self-directed humor, and temporo-parietal junction (TPJ)-frontal connectivity underlying the appreciation of other-directed humor. Amygdala-frontal coupling was observed during the appreciation of detrimental humor, while nucleus accumbens (NAc)-temporal coupling and midbrain-frontal coupling appear to play a role in the affective experience of amusement in response to benign humor. This study contributes to our understanding of the neural correlates of appreciating different humor styles, including humor that facilitates social relationships.
Robots and artificial intelligence (AI) technologies are becoming more prominent in the tourism industry. Nowadays, consumers are faced with multiple options involving both human and robot interactions. A series of experimental studies were implemented. Four experiments demonstrated that consumers had a more positive attitude toward robot-staffed (vs. human-staffed) hotels when COVID-19 was salient. The results were different from previous studies, which were conducted before the COVID-19 pandemic. Since the moderating role of perceived threat in consumers’ preference for robot-staffed hotels was significant, the respondents’ preference was attributed to the global health crisis. This research provides a number of theoretical and managerial implications by improving the understanding of technology acceptance during a health crisis.
This volume brings together the current approaches to the definition and measurement of the sense of humor and its components. It provides both an overview of historic approaches and a compendium of current humor inventories and humor traits that have been studied. Presenting the only available overview and analysis of this significant facet of human behavior, this volume will interest researchers from the fields of humor and personality studies as well as those interested in the clinical or abstract implications of the subject. © 1998 by Walter de Gruyter GmbH & Co., D-10785 Berlin. All rights reserved.
Conference Paper
A shared sense of humor can result in positive feelings associated with amusement, laughter, and moments of bonding. If robotic companions could acquire their human counterparts' sense of humor in an unobtrusive manner, they could improve their skills of engagement. In order to explore this assumption, we have developed a dynamic user modeling approach based on Reinforcement Learning, which allows a robot to analyze a person's reaction while it tells jokes and continuously adapts its sense of humor. We evaluated our approach in a test scenario with a Reeti robot acting as an entertainer and telling different types of jokes. The exemplary adaptation process is accomplished only by using the audience's vocal laughs and visual smiles, but no other form of explicit feedback. We report on results of a user study with 24 participants, comparing our approach to a baseline condition (with a non-learning version of the robot) and conclude by providing limitations and implications of our approach in detail.
Conference Paper
Research suggests, interpersonal competences such as having a sense of humor can help establish sociality in human-robot interaction. This study tested the effect of different types of jokes told by either a human or a robot (NAO) on the perceived intelligence and liking of the narrator. Results of a mixed-design ANOVA showed that only clever jokes could increase the attribution of intelligence to a robot. No significant differences were found for different types of jokes on liking the robot.