ArticlePDF Available

Nonverbal “Accents” Cultural Differences in Facial Expressions of Emotion

Authors:

Abstract

We report evidence for nonverbal "accents," subtle differences in the appearance of facial expressions of emotion across cultures. Participants viewed photographs of Japanese nationals and Japanese Americans in which posers' muscle movements were standardized to eliminate differences in expressions, cultural or otherwise. Participants guessed the nationality of posers displaying emotional expressions at above-chance levels, and with greater accuracy than they judged the nationality of the same posers displaying neutral expressions. These findings indicate that facial expressions of emotion can contain nonverbal accents that identify the expresser's nationality or culture. Cultural differences are intensified during the act of expressing emotion, rather than residing only in facial features or other static elements of appearance. This evidence suggests that extreme positions regarding the universality of emotional expressions are incomplete.
PSYCHOLOGICAL SCIENCE
Research Report
VOL. 14, NO. 4, JULY 2003 Copyright © 2003 American Psychological Society
373
NONVERBAL “ACCENTS”:
Cultural Differences in Facial Expressions of Emotion
Abigail A. Marsh, Hillary Anger Elfenbein, and Nalini Ambady
Harvard University
Abstract—
We report evidence for nonverbal “accents,” subtle differ-
ences in the appearance of facial expressions of emotion across cul-
tures. Participants viewed photographs of Japanese nationals and
Japanese Americans in which posers’ muscle movements were stan-
dardized to eliminate differences in expressions, cultural or otherwise.
Participants guessed the nationality of posers displaying emotional
expressions at above-chance levels, and with greater accuracy than
they judged the nationality of the same posers displaying neutral ex-
pressions. These findings indicate that facial expressions of emotion
can contain nonverbal accents that identify the expresser’s nationality
or culture. Cultural differences are intensified during the act of ex-
pressing emotion, rather than residing only in facial features or other
static elements of appearance. This evidence suggests that extreme po-
sitions regarding the universality of emotional expressions are incom-
plete.
Do facial expressions of emotion constitute a universal “language,
or are there unique ways of expressing emotions within different cul-
tures? This question has become a running debate among psycholo-
gists (e.g., Ekman, 1972, 1997; Elfenbein & Ambady, 2002a, 2002b;
Matsumoto, 2002; Mesquita, Frijda, & Scherer, 1997; Russell, 1994).
Frequently, when two psychological theories compete, the answer lies
somewhere in the middle. Seminal studies on facial expression of
emotion have supported the universality hypothesis, demonstrating
that people of different cultural backgrounds can display similar ex-
pressions in response to similar stimuli (e.g., Camras, Oster, Campos,
Miyake, & Bradshaw, 1997; Ekman, 1972; Ekman & Rosenberg,
1997). However, although there may exist basic commonalities, it
would be reasonable to expect local variations in emotional expression
across cultures. Emotional expression may function as a universal lan-
guage, but one with regional accents. In the present research, we in-
vestigated the existence of nonverbal “accents,” or variations in facial
expressions across cultures.
In linguistics, the word
accent
denotes the characteristic differ-
ences in pronunciation used by subsets of speakers of a single lan-
guage. People come to speak in the accent of speakers around them
(Baron-Cohen & Staunton, 1994; Munro, Derwing, & Flege, 1999).
This may occur not only because similarities in accents help to iden-
tify members of a group, but also because similar spoken accents im-
prove the understanding of verbal communication. Listeners have
more difficulty in understanding speakers using unfamiliar accents
(Munro & Derwing, 1995).
Research on the judgment of emotions across cultures suggests
that people also have more difficulty understanding nonverbal com-
munication of individuals from foreign cultures. Classic judgment
studies demonstrated that facial expressions of the basic emotions of
anger, disgust, fear, happiness, sadness, and surprise can be accurately
recognized across cultural groups, which implies the existence of a
universal schema for the appearance of basic emotions (Ekman, 1972,
1997; Ekman et al., 1987; Ekman, Sorensen, & Friesen, 1969; Izard,
1971). However, when viewing American stimuli, non-Americans
generally do not recognize emotions as accurately as Americans do.
For example, in Izard’s (1971) large-scale study, Americans correctly
identified the emotions in 83% of the facial photographs of Ameri-
cans, and Europeans scored between 75% and 83%, whereas Japanese
scored 65% and Africans only 50%.
Along these lines, a recent meta-analysis of cross-cultural emotion
recognition has provided support for an in-group advantage (Elfenbein
& Ambady, 2002a, 2002b), whereby people more accurately recog-
nize emotional expressions of members of their own cultural in-group
than those of members of a cultural out-group. This effect was consis-
tent across a range of emotions, experimental methods, and nonverbal
channels of communication, and was smaller for cultural groups who
were physically closer to one another or had greater cross-cultural ex-
posure. Interestingly, this meta-analysis found a significant in-group
advantage in many studies with designs in which the expressers’ back-
ground was not obvious from the stimulus materials. In Izard’s (1971)
study, Caucasian groups from European nations identified Caucasian
Americans’ expressions less accurately than did Americans.
In such studies, for participants’ accuracy to differ across cultural
groups implies the existence of cultural variants in the emotional ex-
pressions themselves. Such variants could serve to mark expressers’
nationality. If emotional communication is a universal language, the
coding for the basic muscle movements of certain expressions may be
inherited, but finer-grained elements of expressions may be further
shaped by the expressive styles of the people in one’s environment.
Thus, specific appearances of facial expressions may differ among
cultures, as is the case for other forms of nonverbal behavior (Halber-
stadt, 1985). The existence of nonverbal accents could render the fa-
cial expressions of persons in cultures outside one’s own less easily
interpretable. Just as there is an in-group advantage for understanding
people’s speech, cultural differences could lead to an in-group advan-
tage in emotion recognition.
In the current study, we aimed to test for nonverbal accents in emo-
tional expressions. The strongest test for cultural variations in expression
would be to use stimuli already pretested for “stimulus equivalence”
(Matsumoto, 2002) in terms of identical facial muscle movement
across members of different cultural groups. For this reason, we used
photographs of Japanese and Japanese Americans in Matsumoto and
Ekman’s (1988) Japanese and Caucasian Facial Expressions of Emo-
tion (JACFEE) and Japanese and Caucasian Neutral Faces (JACNeuF)
sets. These photographs were coded according to the Facial Affect
Coding System (FACS; Ekman & Friesen, 1978), a version of the
same systems used by Ekman (1972) and Camras et al. (1997) in their
studies providing evidence for the universality of facial expressions.
“The muscles innervated in the expressions corresponded to the uni-
versal signals of emotion,” Matsumoto (2002) wrote of these photo-
Address correspondence to Abigail Marsh, William James Hall, Depart-
ment of Psychology, Harvard University, 33 Kirkland St., Cambridge, MA
02138; e-mail: amarsh@wjh.harvard.edu.
PSYCHOLOGICAL SCIENCE
Nonverbal “Accents”
374
VOL. 14, NO. 4, JULY 2003
graphs, and they “are equivalent in emotion signaling properties
across encoder cultures” (p. 238). Given that exacting facial coding
systems have found these photographs to be identical in all measur-
able respects, use of these stimuli constitutes a conservative test of the
hypothesis that there are subtle differences in expressive style across
cultural groups.
Because the purpose of our study was to test whether the style of
facial expressions differs across cultures, we wanted to exclude alter-
native explanations for participants’ ability to identify the nationality
of the individuals in the photographs. A crucial consideration was that
a person might be able to identify shared nationality from the physiog-
nomy of facial features, apparel, or other features. Thus, it was impor-
tant to demonstrate that nonverbal accents arise in the expression of
emotion itself, rather than only in features of the static face. We used
JACFEE and JACNeuF photographs for this reason also, because they
control for lighting, background color, and clothing. Furthermore, the
sets of nonexpressive and expressive photographs include the same in-
dividuals, allowing us to control for differences in static facial features
across individual posers. Thus, it would provide strong evidence for
nonverbal accents if participants were able to discern the nationality of
posers in these emotional photographs. It would also provide evidence
against alternative hypotheses concerning permanent facial differ-
ences if participants were substantially less able to discern the nation-
ality of the same posers with neutral expressions. Such a finding
would allow us to conclude that cross-cultural differences in the ap-
pearance of facial expressions emerge in the act of expressing emo-
tion.
METHOD
Participants
Seventy-nine adults (40 females and 39 males) identifying them-
selves as native to the United States or Canada were recruited to par-
ticipate either for course credit or for $5 in compensation. Sixty-one
(77.2%) identified themselves as Caucasian, 6 (7.6%) as Black or Af-
rican American, and 5 (6.3%) as Asian American; 7 (8.9%) did not
identify their ethnic background.
Stimulus Materials
The JACFEE and JACNeuF sets provided all the stimulus photo-
graphs (Matsumoto & Ekman, 1988). We used photographs of 18 pos-
ers, 9 Japanese nationals (i.e., individuals of Japanese citizenship and
ancestry) and 9 Japanese Americans (i.e., American citizens of Japa-
nese ancestry). For each individual, we selected both an emotional ex-
pression (anger, disgust, fear, sadness, or surprise) and a neutral
expression, for a total of 36 photos.
1
Happy expressions were omitted,
as no Japanese nationals pose happiness in the JACFEE set. Each par-
ticipant judged the nationality of all 18 posers, viewing one of two sets
of the 18 posers’ photographs. The assignment of stimulus set to each
participant was random. Each set contained some of the posers dis-
playing emotional expressions and some displaying neutral expres-
sions; expressions were balanced across nationality, and photographs
were presented in a different random order for each participant. All
participants thus judged a mix of neutral and emotional expressions
displayed by the 18 Japanese-national and Japanese American posers.
Procedure
Participants completed the study in a private, sound-attenuated
room, using the software program SuperLab™ on an IBM-compatible
desktop computer. Photographs appeared on the monitor in gray scale
at 433
289 pixels. As is commonly done in such judgment studies
(e.g., Matsumoto, 1993), we first familiarized participants with the set
of photographs while asking them to make a different set of judg-
ments, unrelated to the nationality of the posers. During the familiar-
ization trials, participants judged six personality traits (efficient,
independent, ignorant, open, likable, and rude) for the 18 faces. After
the familiarization trials, the instructions presented on the computer
informed participants that half of the faces they had seen were Japa-
nese and the other half were Japanese American. This information en-
sured that participants understood that each individual shown had an
equal chance of being Japanese or Japanese American, which encour-
aged similar base rates in responses and limited test bias. Participants
then viewed the 18 photographs again and judged the nationality of
each individual poser.
RESULTS
The analyses we report use Wagner’s (1993) unbiased hit rate, cal-
culated as arcsin[hit rate * (1
false alarm rate)]. Although we mini-
mized response biases by informing participants that half of the
photos depicted Japanese and half depicted Japanese Americans, the
unbiased hit rate corrects for any lingering response biases across par-
ticipants, such as bias in the level of false alarms. Wagner (1993) also
provided a procedure to calculate the accuracy level expected due to
chance guessing, which is analogous to calculating expected values
for a chi-square analysis. All scores reported refer to the degree of ac-
curacy above that expected due to chance. Thus, a score of 0 would
indicate the participant was unable to distinguish Japanese from Japa-
nese American faces.
Participants could identify the nationality of emotional expressions
at above-chance levels, single-sample
t
(78)
5.18,
p
.000001, two-
tailed,
r
.51. Participants could also identify the nationality of neu-
tral expressions at above-chance levels, single-sample
t
(78)
2.41,
p
.05, two-tailed,
r
.26. A comparison of the effect sizes for accu-
racy judging nationality from emotional and neutral expressions, how-
ever, indicated significantly greater accuracy for photographs showing
emotional expressions,
Z
1.77,
p
.05. A 2 (expression type: emo-
tional, neutral)
2 (participant’s sex: male, female) repeated measures
analysis of variance (ANOVA) with stimulus set as a covariate simi-
larly showed a main effect for expression type. Nationality judgments
were significantly more accurate for emotional expressions than neu-
tral expressions,
F
(1, 76)
10.34,
p
.005,
.34. The main effect
of participants’ gender and the interaction of expression type and par-
ticipants’ gender were not significant,
F
(1, 76)
0.76 and
F
(1, 76)
0.20, respectively.
The results of a one-way repeated measures ANOVA by expression
(fear, disgust, sadness, surprise) indicated that accuracy in judgments
of nationality differed among the four expressions,
F
(3, 234)
6.28,
1. From the JACFEE set, we used Photographs 05, 21, 23, 30, 31, 45, 47,
53, and 56 (Japanese nationals) and Photographs 06, 22, 24, 29, 32, 46, 48, 54,
and 55 (Japanese Americans). From the JACNeuF set, we used Photographs
30, 31, 32, 33, 38, 49, 50, 54, and 56 (Japanese nationals) and Photographs 29,
34, 35, 36, 39, 43, 46, 48, and 51 (Japanese Americans).
PSYCHOLOGICAL SCIENCE
Abigail A. Marsh, Hillary Anger Elfenbein, and Nalini Ambady
VOL. 14, NO. 4, JULY 2003
375
p
.001,
.27. Note that only two anger exemplars from the
JACFEE could be used, so that each subject saw only the neutral ex-
pressions or the anger expressions of the anger expressers, and thus
anger was excluded from these analyses. Accuracy in judging nation-
ality was highest for sad expressions (
M
0.38,
SE
0.05), followed
by surprised expressions (
M
0.24,
SE
0.05), then fear expressions
(
M
0.15,
SE
0.05) and then disgust expressions (
M
0.12,
SE
0.04). Post hoc paired
t
tests showed that the nationality of individuals
expressing sadness was recognized more accurately than that of indi-
viduals expressing the three other emotions:
t
(78)
4.20,
p
.001,
for disgust;
t
(78)
3.16,
p
.01, for fear; and
t
(78)
2.05,
p
.05,
for surprise (all two-tailed). No other differences among emotions
were significant. The evidence suggests that expressive information
carries information about individuals’ nationality that is detectable to
naive observers. By contrast, observers are less able to determine the
nationality of the same individuals’ neutral expressions.
DISCUSSION
This study provides evidence for the existence of nonverbal ac-
cents—cultural variations in the appearance of basic facial expres-
sions of emotion. These findings suggest that extreme positions
regarding the universality of emotion are incomplete. That is, across
cultures, emotions may be expressed in a manner largely consistent
with universal prototypes, but there can still exist subtle cultural dif-
ferences in the appearance of these universal emotions. Participants
less accurately distinguished the cultural background of posers when
they judged neutral rather than emotional expressions of the same pos-
ers. This result indicates that emotional expressions carry information
about nationality or culture beyond the information conveyed by per-
manent differences between cultural groups in their members’ physi-
ognomy of facial features, apparel, or other static features. Differences
may exist in the extent to which various emotional expressions convey
evidence of cultural origins. However, because expresser and emotion
were confounded in the photo set we used, the results of this analysis
are difficult to interpret.
It is possible that nonverbal accents differentially shape even the
appearance of static facial features of members of different cultures.
Zajonc, Adelmann, Murphy, and Niedenthal (1987) showed that the
physical appearances of people in prolonged social contact converge
over time. They explained this finding in terms of permanent changes
in the structure of facial musculature, as a result of people who are in
contact with one another tending to develop common habits in using
certain muscles. This phenomenon might similarly entail permanent
changes in the facial appearance of members of an ethnic or national
group, and would make it possible to identify members of a particular
national or ethnic group on the basis of their neutral facial expres-
sions.
Our test for nonverbal accents in this study was conservative and
stringent. Participants accurately identified posers’ cultural back-
grounds in emotional expressions expressly designed for stimulus
equivalence. That is, the photograph set we used was designed with
the explicit purpose of eliminating all possible cultural differences in
the appearance of emotional expressions. Using the FACS (Camras et
al., 1997; Ekman, 1972; Matsumoto, 2002), Matsumoto and Ekman
(1988) validated these emotional expressions by identifying muscle
movement patterns that corresponded to their prototypes for universal
emotions. Thus, the current findings suggest that cultural differences
in the appearance of facial expressions are so robust that they are
nearly impossible to eliminate. These cultural differences also seem
not to be explicitly detectable using the FACS. They likely consist of
very slight variants in the muscle movements, or in the relative inten-
sity of these movements. This assumption is supported by the varia-
tion in the accuracy of nationality judgments across emotional
expressions. The expressions are, by definition, composed of different
sets of muscle movements. It therefore would be expected that differ-
ent emotional expressions would convey cultural differences to differ-
ent extents. More detailed analysis of the specific components of
expression carrying the implicitly detectable nonverbal accents is
needed.
The current study demonstrated people’s sensitivity to subtle dif-
ferences in the appearance of facial expressions across cultures. It
would be worthwhile for further work to examine how people use this
information in naturalistic situations. Nonverbal accents may provide
a mechanism underlying well-known social phenomena such as attrac-
tion to that which is familiar (Byrne, 1997) and xenophobia, the fear
or dislike of foreigners or strangers (Hall, 1959; Warnecke, Masters, &
Kempter, 1992). Attractiveness, familiarity, and similarity to self may
be mutually reinforcing qualities (Moreland & Zajonc, 1982). If peo-
ple from one’s own culture look familiar when expressing emotion,
this may help one to form favorable attitudes about individuals with
similar cultural backgrounds. Our findings suggest that people can
judge cultural background through nonverbal accents, just as they can
judge the geographic backgrounds of people speaking a common lan-
guage—for example, a Texan versus a Scot—through verbal accents.
REFERENCES
Baron-Cohen, S., & Staunton, R. (1994). Do children with autism acquire the phonology
of their peers? An examination of group identification through the window of bilin-
gualism.
First Language
,
14
, 241–248.
Byrne, D. (1997). An overview (and underview) of research and theory within the attrac-
tion paradigm.
Journal of Social & Personal Relationships
,
14
, 417–431.
Camras, L.A., Oster, H., Campos, J.J., Miyake, K., & Bradshaw, D. (1997). Japanese and
American infants’ responses to arm restraint. In P. Ekman & E.L. Rosenberg (Eds.),
What the face reveals: Basic and applied studies of spontaneous expression using
the Facial Action Coding System (FACS)
(pp. 289–301). New York: Oxford Univer-
sity Press.
Ekman, P. (1972). Universals and cultural differences in facial expressions of emotion. In
J. Cole (Ed.),
Nebraska symposium on motivation, 1971
(pp. 207–282). Lincoln:
University of Nebraska Press.
Ekman, P. (1997). What we have learned by measuring facial behavior. In P. Ekman &
E.L. Rosenberg (Eds.),
What the face reveals: Basic and applied studies of sponta-
neous expression using the Facial Action Coding System (FACS)
(pp. 469–485).
New York: Oxford University Press.
Ekman, P., & Friesen, W.V. (1978).
Facial Affect Coding System: A technique for the mea-
surement of facial movement
. Palo Alto, CA: Consulting Psychologists Press.
Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K.,
Krause, R., LeCompte, W.A., Pitcairn, T., Ricci-Bitti, P.E., Scherer, K., Tomita, M.,
& Tzavaras, A. (1987). Universals and cultural differences in the judgments of facial
expressions of emotion.
Journal of Personality and Social Psychology
,
53
, 712–717.
Ekman, P., & Rosenberg, E.L. (Eds.). (1997).
What the face reveals: Basic and applied
studies of spontaneous expression using the Facial Action Coding System (FACS)
.
New York: Oxford University Press.
Ekman, P., Sorensen, E.R., & Friesen, W.V. (1969). Pancultural elements in facial displays
of emotions.
Science
,
164
, 86–88.
Elfenbein, H.A., & Ambady, N. (2002a). Is there an ingroup advantage in emotion recog-
nition?
Psychological Bulletin
,
128
, 243–249.
Acknowledgments—This study was supported in part by two National
Science Foundation Graduate Fellowships to the first and second authors, a
Rand grant, and a Presidential Early Career Award for Scientists and Engi-
neers (Grant BCS-9733706) to the third author. We thank Dana Gavrieli,
Ryan Jimenez, and Lauren Sheerr for their assistance in this research.
PSYCHOLOGICAL SCIENCE
Nonverbal “Accents”
376
VOL. 14, NO. 4, JULY 2003
Elfenbein, H.A., & Ambady, N. (2002b). On the universality and cultural specificity of
emotion recognition: A meta-analysis.
Psychological Bulletin
,
128
, 203–235.
Halberstadt, A.G. (1985). Race, socioeconomic status, and nonverbal behavior. In A.W.
Siegman & S. Feldstein (Eds.),
Multichannel integrations of nonverbal behavior
(pp. 227–266). Hillsdale, NJ: Erlbaum.
Hall, E.T. (1959).
The silent language
. Garden City, NY: Doubleday.
Izard, C.E. (1971).
The face of emotion
. New York: Appleton-Century-Crofts.
Matsumoto, D. (1993). Ethnic differences in affect intensity, emotion judgments, display
rule attitudes, and self-reported emotional expression in an American sample.
Moti-
vation and Emotion
,
17
, 107–123.
Matsumoto, D. (2002). Methodological requirements to test a possible ingroup advantage
in judging emotions across cultures: Comments on Elfenbein and Ambady and evi-
dence.
Psychological Bulletin
,
128
, 236–242.
Matsumoto, D., & Ekman, P. (1988).
Japanese and Caucasian facial expressions of emo-
tion (JACFEE)
[Slides]. San Francisco, CA: San Francisco State University, Depart-
ment of Psychology, Intercultural and Emotion Research Laboratory.
Mesquita, B., Frijda, N.H., & Scherer, K.R. (1997). Culture and emotion. In J.W. Berry,
P.R. Dasen, T.S. Saraswathi, Y.H. Poortinga, & J. Pandey (Eds.),
Handbook of cross-
cultural psychology: Vol. 2. Basic processes and human development
(2nd ed., pp.
255–297). Boston: Allyn & Bacon.
Moreland, R.L., & Zajonc, R.B. (1982). Exposure effects in person perception: Familiarity,
similarity, and attraction.
Journal of Experimental Social Psychology
,
18
, 395–415.
Munro, M.J., & Derwing, T.M. (1995). Foreign accent, comprehensibility, and intelligibil-
ity in the speech of second language learners.
Language Learning
,
45
, 73–97.
Munro, M.J., Derwing, T.M., & Flege, J.E. (1999). Canadians in Alabama: A perceptual
study of dialect acquisition in adults.
Journal of Phonetics
,
27
, 385–403.
Russell, J.A. (1994). Is there universal recognition of emotion from facial expression? A
review of the cross-cultural studies.
Psychological Bulletin
,
115
, 102–141.
Wagner, H.L. (1993). On measuring performance in category judgment studies of nonver-
bal behavior.
Journal of Nonverbal Behavior
,
17
, 3–28.
Warnecke, A.M., Masters, R.D., & Kempter, G. (1992). The roots of nationalism: Nonver-
bal behavior and xenophobia.
Ethology & Sociobiology
,
13
, 267–282.
Zajonc, R.B., Adelmann, P.K., Murphy, S.T., & Niedenthal, P.M. (1987). Convergence in
the physical appearance of spouses.
Motivation & Emotion
,
11
, 335–346.
(R
ECEIVED
10/24/01; R
EVISION
ACCEPTED
9/23/02)
... Finally, the current study only included interviews that were conducted in Israel and the sample was relatively homogenous ethnically. Although nonverbal emotions are considered universal, emotional displays may be different in other cultural contexts (Hamilton et al., 2016;Marsh et al., 2003). ...
Article
Full-text available
Statements by alleged victims are important when child abuse is prosecuted; triers-of-fact often attend to nonverbal emotional expressions when evaluating those statements. This study examined the associations among interviewer supportiveness, children’s nonverbal emotions, and informativeness during 100 forensic interviews with alleged victims of child abuse. Raters coded the silent videotapes for children’s nonverbal emotional expressions while other raters coded the transcripts for interviewer support, children’s verbal emotions, and informativeness. Results showed that children’s nonverbal signals were more common than and preceded the verbal signs. Interviewer support was associated with children’s expressivity. When children expressed more nonverbal emotions, they were more responsive during the pre-substantive phases and more informative about the abuse. Nonverbal emotions partially mediated the association between support and informativeness. The findings underline the value of nonverbal emotional expression during forensic interviews and demonstrate how the interviewers’ supportive demeanor can facilitate children’s emotional displays and increase informativeness.
... The literature provides us with some solutions (e.g., see [4,14,30]), but further studies are needed to understand which of these is best suited to our case. Another essential aspect to consider is the existence of differences in nonverbal communication between genders and cultures, as described in [13,16]. To address this problem, we must review the existing literature for commonalities among the reactions of different groups of individuals to give more importance to particular features. ...
Conference Paper
In this paper, we present the rationale and the ideas behind META4RS, a museum itinerary recommender system. The system leverages deep learning techniques to acquire data about the visitor's position while ensuring her anonymity. Moreover, the visitor's appraisal of the artwork she observes is inferred implicitly based on the emotional reactions she expresses while watching a given artwork. We are not aware of any such recommender system proposed in the research literature. However, this system should ensure several advantages: (i) it is non-intrusive since it makes use of simple badges and off-the-shelf cameras while ensuring the anonymity of the visitor; (ii) it is independent of the type of museum; (iii) it offers personalized itineraries to visitors based on their implicitly inferred interests and preferences. Specifically, we illustrate the background and describe the architecture of the proposed system, discussing the steps required for its implementation. We also provide details of what has already been done and what remains to be done, outlining the open problems.
... Culture clearly influences nonverbal communication (Marsh et al., 2003), one of the three sub-criteria related to SC/I in autism (APA, 2013). For example, avoiding eye contact is a common symptom of autism, although reduced eye contact may also be a sign of respect in Native American and East Asian cultures (Akechi et al., 2013;Uono & Hietanen, 2015). ...
Article
Social communication and interaction deficits are a diagnostic criteria of autism and integral to practitioner and researcher conceptualization. Culture is an influential factor in expectations for, and demonstration of, social communication and interaction skills, but there is limited research published in autism journals on this topic. This paucity of autism research examining cultural factors related to social communication and interaction may contribute to known identification disparities for racial and ethnic populations minoritized by systemic factors and research bias. We call for increased commitment from researchers to recruit racially and ethnically minoritized participants, prioritize investigating cultural expectations and perceptions of social communication and interaction, and evaluate measures related to social communication for cultural and linguistic responsivity. Lay Summary A diagnosis of autism requires the presence of deficits in social communication and interaction. Examples of these behaviors and skills include holding a back-and-forth conversation, the use of nonverbal communicative behaviors (e.g., gestures), and developing and maintaining social relationships. Culture influences the expectations for, and presentation of, these behaviors. However, research on this topic is lacking. Conducting more research related to culture and social communication could help reduce the disparities in autism identification across racially and ethnically minoritized populations.
... Our findings are in line with previous research showing that perceivers can identify group membership from speech segments [33] and language dialects [34]. Facial expressions also allow perceivers to infer the nationality of producers, even from visually similar groups like White Americans and Australians [35], and Japanese and Japanese-Americans [36]. Our study adds laughter to this suite of communicative signals by providing evidence that laughter can convey information about cultural group identity. ...
Article
Full-text available
Laughter is a ubiquitous social signal. Recent work has highlighted distinctions between spontaneous and volitional laughter, which differ in terms of both production mechanisms and perceptual features. Here, we test listeners' ability to infer group identity from volitional and spontaneous laughter, as well as the perceived positivity of these laughs across cultures. Dutch ( n = 273) and Japanese ( n = 131) participants listened to decontextualized laughter clips and judged (i) whether the laughing person was from their cultural in-group or an out-group; and (ii) whether they thought the laughter was produced spontaneously or volitionally. They also rated the positivity of each laughter clip. Using frequentist and Bayesian analyses, we show that listeners were able to infer group membership from both spontaneous and volitional laughter, and that performance was equivalent for both types of laughter. Spontaneous laughter was rated as more positive than volitional laughter across the two cultures, and in-group laughs were perceived as more positive than out-group laughs by Dutch but not Japanese listeners. Our results demonstrate that both spontaneous and volitional laughter can be used by listeners to infer laughers’ cultural group identity. This article is part of the theme issue ‘Voice modulation: from origin and mechanism to social impact (Part II)’.
... Relatedly, our results are based on white Western participants interpreting facial expressions displayed by same-ethnicity faces, using commonly used English-language emotion terms. Future work will prioritize examining whether these results generalize to or vary across other cultures, face identities, and languages (e.g., see Jack et al., 41 Marsh et al., 84 Elfenbein, 85 and Dailey et al. 86 ). ...
Article
Full-text available
Human facial expressions are complex, multi-component signals that can communicate rich information about emotions,1, 2, 3, 4, 5 including specific categories, such as “anger,” and broader dimensions, such as “negative valence, high arousal.”6, 7, 8 An enduring question is how this complex signaling is achieved. Communication theory predicts that multi-component signals could transmit each type of emotion information—i.e., specific categories and broader dimensions—via the same or different facial signal components, with implications for elucidating the system and ontology of facial expression communication.⁹ We addressed this question using a communication-systems-based method that agnostically generates facial expressions and uses the receiver’s perceptions to model the specific facial signal components that represent emotion category and dimensional information to them.10, 11, 12 First, we derived the facial expressions that elicit the perception of emotion categories (i.e., the six classic emotions¹³ plus 19 complex emotions³) and dimensions (i.e., valence and arousal) separately, in 60 individual participants. Comparison of these facial signals showed that they share subsets of components, suggesting that specific latent signals jointly represent—i.e., multiplex—categorical and dimensional information. Further examination revealed these specific latent signals and the joint information they represent. Our results—based on white Western participants, same-ethnicity face stimuli, and commonly used English emotion terms—show that facial expressions can jointly represent specific emotion categories and broad dimensions to perceivers via multiplexed facial signal components. Our results provide insights into the ontology and system of facial expression communication and a new information-theoretic framework that can characterize its complexities.
Chapter
We are constantly forming impressions about those around us. Social interaction depends on our understanding of interpersonal behavior - assessing one another's personality, emotions, thoughts and feelings, attitudes, deceptiveness, group memberships, and other personal characteristics through facial expressions, body language, voice and spoken language. But how accurate are our impressions and when does such accuracy matter? How is accuracy achieved and are some of us more successful at achieving it than others? This comprehensive overview presents cutting-edge research on this fast-expanding field and will be essential reading for anyone interested in the psychology of interpersonal perception. A wide range of experts in the field explore topics including age and gender effects, psychopathology, culture and ethnicity, workplaces and leadership, clinicians' skills, empathy, meta-perception, and training people to be more accurate in their perceptions of others.
Chapter
The study of emotional expressions has a long tradition in psychology. Although research in this domain has extensively studied the social context factors that influence the expresser's facial display, the perceiver was considered passive. This 2007 book focuses on more recent developments that show that the perceiver is also subject to the same social rules and norms that guide the expresser's behavior and that knowledge of relevant emotion norms can influence how emotional expressions shown by members of different groups are perceived and interpreted. Factors such as ethnic-group membership, gender and relative status all influence not only emotional expressions but also the interpretation of emotional expressions shown by members of different groups. Specifically, the research presented asks the question of whether and why the same expressions shown by men or women, members of different ethnic groups, or individuals high and low in status are interpreted differently.
Chapter
The study of emotional expressions has a long tradition in psychology. Although research in this domain has extensively studied the social context factors that influence the expresser's facial display, the perceiver was considered passive. This 2007 book focuses on more recent developments that show that the perceiver is also subject to the same social rules and norms that guide the expresser's behavior and that knowledge of relevant emotion norms can influence how emotional expressions shown by members of different groups are perceived and interpreted. Factors such as ethnic-group membership, gender and relative status all influence not only emotional expressions but also the interpretation of emotional expressions shown by members of different groups. Specifically, the research presented asks the question of whether and why the same expressions shown by men or women, members of different ethnic groups, or individuals high and low in status are interpreted differently.
Chapter
In a world of connected smart devices where data is easily available, there is an opportunity to enhance existing technology to better adapt towards office workers’ needs. In this work, we explore the idea of how emotion can be one of many data points to be taken into account in a room planning calendar environment. We present Emoti-Office, a system that (1) collects emotional and contextual data in an office setting, (2) considers how to use this data and (3) how to link emotions with the desired output of a room suggestion or work environment to be reserved and used. Emoti-Office contributes to the field of office working research and development by proposing a context-aware and adaptive design which uses emotions to improve office workers’ environment by adapting to their needs.KeywordsOfficeEmotionsWorkersNeeds
Article
Full-text available
We draw on the process model of emotion regulation (Gross, Emotion,13(3), 359–365 2013) to propose that the two main components of paternalistic leadership – authoritarianism and benevolence, would induce negative emotions in employees that cannot be freely expressed at work, and how the act of emotion suppression would result in employee work–family conflict. Data from 218 employee–spouse dyads from three Chinese companies provide strong support for our hypotheses, revealing that both authoritarian leadership and benevolent leadership are positively related to employee work–family conflict through the mediation of employee emotion suppression in the workplace. Furthermore, employee power distance orientation positively influences the effect of authoritarian leadership, but negatively impacts the effect of benevolent leadership, on employee emotion suppression. These findings reveal the dark side of paternalistic leadership and shed light on the mechanisms through which leadership affect employee experience in the life domain and its boundary condition. We discuss the theoretical and practical implications in the context of Chinese organizations and beyond.
Article
Full-text available
Normal children whose parents have different native languages tend to develop an accent which is closer to their peers than to either parent. It was predicted that children with autism, because of their social deficits, might not acquire the accent of their peers, perhaps because of the lack of the normal drive to identify with peers. Bilingualism was used as a window into such social factors in language acquisition. Using audiotaped speech samples, the study found that in a sample of children with autism who were brought up in England and whose mothers were not English, 83.3% acquired their mother's (non-English) accent. In contrast, among normally- developing siblings of children with autism who were brought up in England and whose mothers were not English, only 12.5% acquired their mother's (non-English) accent. We suggest that such studies of unusual populations are of value in furthering our understanding of the larger population of children with autism, and the influences on normal social development.
Article
While we have known for centuries that facial expressions can reveal what people are thinking and feeling, it is only recently that the face has been studied scientifically for what it can tell us about internal states, social behavior, and psychopathology. Today's widely available, sophisticated measuring systems have allowed us to conduct a wealth of new research on facial behavior that has contributed enormously to our understanding of the relationship between facial expression and human psychology. The chapters in this volume present the state-of-the-art in this research. They address key topics and questions, such as the dynamic and morphological differences between voluntary and involuntary expressions, the relationship between what people show on their faces and what they say they feel, whether it is possible to use facial behavior to draw distinctions among psychiatric populations, and how far research on automating facial measurement has progressed. © 1997, 2005 by Oxford University Press, Inc. All rights reserved.
Article
One of the chief goals of most second language learners is to be understood in their second language by a wide range of interlocutors in a variety of contexts. Although a nonnative accent can sometimes interfere with this goal, prior to the publication of this study, second language researchers and teachers alike were aware that an accent itself does not necessarily act as a communicative barrier. Nonetheless, there had been very little empirical investigation of how the presence of a nonnative accent affects intelligibility, and the notions of “heavy accent” and “low intelligibility” had often been confounded. Some of the key findings of the study—that even heavily accented speech is sometimes perfectly intelligible and that prosodic errors appear to be a more potent force in the loss of intelligibility than phonetic errors—added support to some common, but weakly substantiated beliefs. The study also provided a framework for a program of research to evaluate the ways in which such factors as intelligibility and comprehensibility are related to a number of other dimensions. The authors have extended and replicated the work begun in this study to include learners representing other L1 backgrounds (Cantonese, Japanese, Polish, Spanish) and different levels of learner proficiency, as well as other discourse types (Derwing & Munro, 1997; Munro & Derwing, 1995). Further support for the notion that accent itself should be regarded as a secondary concern was obtained in a study of processing difficulty (Munro & Derwing, 1995), which revealed that nonnative utterances tend to require more time to process than native-produced speech, but failed to indicate a relationship between strength of accent and processing time.The approach to L2 speech evaluation used in this study has also proved useful in investigations of the benefits of different methods of teaching of pronunciation to ESL learners. In particular, it is now clear that learner assessments are best carried out with attention to the multidimensional nature of L2 speech, rather than with a simple focus on global accentedness. It has been shown, for instance, that some pedagogical methods may be effective in improving intelligibility while others may have an effect only on accentedness (Derwing, Munro, & Wiebe, 1998).
Article
Do people discriminate between foreigners and fellow-countrymen on the basis of nonverbal behavior? If so, are responses to outsiders primarily based on cognitive judgments or emotional reactions? Are there specific head and body movements that elicit these responses? This article presents experimental evidence showing that when American adults are presented with televised images of leaders from France, Germany, and the United States, they feel more negatively when seeing the foreigners and judge them more negatively than their fellow citizens. These effects do not occur when the images are presented with sound so that viewers are aware of nationality. The negative emotions and judgments therefore seem to be due to preconscious monitoring of nonverbal cues rather than to consciously held cognitive prejudices against foreigners or to other factors.
Article
Although the first research on race and socioeconomic differences in nonverbal behavior was conducted in the 1930's, interest in these issues was not sustained. In order to further understanding of these previous studies, all the nonverbal categories that included more than three studies investigating race and class differences were examined. The resultant categories included proxemics (interpersonal distance and body orientation), touch, eye gaze, and verbal/nonverbal interaction. Meta-analysis of the studies on proxemics and eye gaze showed that racial differences in proxemic behavior appear to be mediated by social class differences, though blacks and whites share a developmental pattern of increasing interpersonal distance, and blacks engage in less direct body orientation. Although rates of eye gaze differed by race, patterns of looking more while listening than speaking are shared interracially. The findings suggest that there are differences in the nonverbal domain across race, class, and culture. For research and application, an awareness of differences rather than the assumption of sameness is important. (LLL)