Recognizing Spontaneous Facial Expressions of Emotion in a Small-Scale Society of Papua New Guinea

Abstract

We report 2 studies on how residents of Papua New Guinea interpret facial expressions produced spontaneously by other residents of Papua New Guinea. Members of a small-scale indigenous society, Trobrianders (Milne Bay Province; N = 32, 14 to 17 years) were shown 5 facial expressions spontaneously produced by members of another small-scale indigenous society, Fore (Eastern Highlands Province) that Ekman had photographed, labeled, and published in The Face of Man (1980), each as an expression of a basic emotion: happiness, sadness, anger, surprise, and disgust. Trobrianders were asked to use any word they wanted to describe how each person shown felt and to provide valence and arousal ratings. Other Trobrianders (N = 24, 12 to 14 years) were shown the same photographs but asked to choose their response from a short list. In both studies, agreement with Ekman’s predicted labels was low: 0% to 16% and 13% to 38% of observers, respectively.

Full text

doi: 10.1037/emo0000236
©American Psychological Association, [2017]. This paper is not the copy of record and may not exactly replicate the authoritative document published in
the APA journal. Please do not copy or cite without the author's permission. The final article is available, upon publication, at: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 1
!
Running Head: SPONTANEOUS FACIAL EXPRESSIONS
Recognizing Spontaneous Facial Expressions of Emotion in a Small-Scale Society of
Papua New Guinea
Carlos Crivelli1, James A. Russell2, Sergio Jarillo3,4, & José-Miguel Fernández-Dols1
1 Facultad de Psicología, Universidad Autónoma de Madrid, Spain
2 Department of Psychology, Boston College, MA, USA
3 Anthropology Division, American Museum of Natural History, NY, USA
4 Arts of Africa, Oceania & the Americas, The Metropolitan Museum of Art, NY, USA
ACCEPTED FOR PUBLICATION IN
EMOTION
Author Note:
This research was funded by the Spanish Government’s grant PSI2014-57154-P, by a
grant from Boston College awarded to James A. Russell, and by a Predoctoral
Fellowship (FPI-UAM2012) awarded to Carlos Crivelli. We thank Antonio Pardo for
comments on a draft of this article. We thank the National Research Institute of Papua
New Guinea, Moses Moyobova, Dorothy Kwenama, Tabini Moses, Mr. Leseta, Frida
Kenny, Gilbert, and the members of Kaduwaga and Vakuta primary schools for their
help in the field.
Correspondence concerning this article should be addressed to James A. Russell,
Department of Psychology, McGuinn Hall, 140 Commonwealth Avenue, Boston
College, Chestnut Hill, MA (02467). E-mail: james.russell@bc.edu

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 2
!
Abstract
We report two studies on how residents of Papua New Guinea interpret facial
expressions produced spontaneously by other residents of Papua New Guinea. Members
of a small-scale indigenous society, Trobrianders (Milne Bay Province; N = 32, 14 to 17
years) were shown 5 facial expressions spontaneously produced by members of another
small-scale indigenous society, Fore (Eastern Highlands Province) that Ekman had
photographed, labeled, and published in The Face of Man (1980), each as an expression
of a basic emotion: happiness, sadness, anger, surprise, and disgust. Trobrianders were
asked to use any word they wanted to describe how each person shown felt and to
provide valence and arousal ratings. Other Trobrianders (N = 24, 12 to 14 years) were
shown the same photographs but asked to choose their response from a short list. In
both studies, agreement with Ekman’s predicted labels was low: 0 to 16% and 13 to
38% of observers, respectively.
Keywords: spontaneous facial expressions; indigenous societies; emotion perception;
cross-cultural diversity; universality thesis

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 3
!
Recognizing Spontaneous Facial Expressions of Emotion in a Small-Scale Society
of Papua New Guinea
Of extreme importance in the science of emotion has been a series of studies of
facial expressions conducted in Papua New Guinea (Ekman, 1972; Ekman & Friesen,
1971; Ekman, Sorenson, & Friesen, 1969; Sorenson, 1975, 1976). By purporting to
demonstrate that certain facial expressions convey to Papua New Guineans the same
emotions they convey to Westerners, these studies inspired a research program on the
universality of facial expressions and of emotions more generally, resurrected an
interest in Darwin’s analysis of facial expression, provided tools for neuroscience, and
inspired applications from tests of emotional intelligence to emotion regulation to
techniques for border security (Leys, 2010; Plamper, 2015). Debate has ensued over the
empirical results from studies of facial expressions and over the conceptual
framework—Basic Emotion Theory—undergirding that research and application
(Crivelli, Jarillo, Russell, & Fernández-Dols, 2016; Ekman, 1994, 2003, 2016; Ekman
& Cordaro, 2011; Jack, Blais, Scheepers, Schyns, & Caldara, 2009; Levenson, 2011).
All the same, according to a survey conducted in 2014, over 80% of emotion
researchers surveyed accepted the conclusion drawn from the original studies in Papua
New Guinea: certain facial expressions convey basic emotions universally (Ekman,
2016)—a conclusion often presented in psychology textbooks (Gilovich, Keltner, &
Nisbett, 2011; Myers & DeWall, 2015; Schacter, Gilbert, & Wegner, 2011). In this
article, we report two studies that challenge that conclusion.
The studies conducted in Papua New Guinea (Ekman, 1972; Ekman & Friesen,
1971; Ekman, Sorenson, & Friesen, 1969; Sorenson, 1975, 1976) have received various
criticisms (Russell, 1994; but see Ekman, 1994; Izard, 1994; Russell, 1995). Key

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 4
!
criticisms concern the methodological choices made when designing those studies.
First, one of the original researchers pointed to ways in which the Papua New
Guinean observers might have been influenced by the translators who helped conduct
the studies (Sorenson, 1975, 1976).
Second, responses by the Papua New Guinean observers were gathered with a
multiple-choice format. When observers select the predicted option from a limited set,
they might not be indicating an automatic “decoding” of the meaning “encoded” in a
signal, but only which is the best of the options available. Rather than “recognizing” a
specific emotion from the facial expression, they might narrow their choices through a
process of elimination (DiGirolamo & Russell, 2016; Nelson & Russell, 2016) or on the
basis of broad affective dimensions such as valence and arousal (Russell, 1997, 2003;
Yik, Widen, & Russell, 2013). Such speculations are reinforced by studies that used a
more open-ended response format in Western (Izard, 1971), Eastern (Haidt & Keltner,
1999), and pastoralist African samples (Gendron, Roberson, van der Vyver, & Barrett,
2014); these studies found lower amounts of recognition of emotion from the face than
is typically found with multiple-choice formats. Sorenson (1975) used a free labeling
format with observers from Papua New Guinea. For the hunter-gatherers Fore with least
contact with Westerners, the amount of recognition was low for disgust, contempt,
sadness (< 26%), modest for fear and surprise (33%), moderate for anger (51%), and
high for happiness (87%). Open-ended formats can be criticized for their reliance on the
observer having the needed term readily available. In short, multiple-choice formats
might overestimate the amount of agreement with the experimenter’s prediction, but
more open-ended formats might underestimate the amount. In the present study, we
explored both types of format.
And, third, the facial expressions presented to the Papua New Guinean observers

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 5
!
were all posed. Posed expressions were deliberately designed to convey one and only
one emotion, and the photographs shown to observers had been selected as the ones
most successful at doing so. Studies of recognition of emotion from spontaneously
produced facial expressions have found substantially less agreement on the emotion
conveyed (Hess & Blairy, 2001; Kayyal & Russell, 2013; Naab & Russell, 2007;
Wagner, MacDonald, & Manstead, 1986).
During his visits to Papua New Guinea in the 1960s, Paul Ekman photographed
facial expressions produced spontaneously. They were produced by a group of hunter-
gatherers—the Fore of the Eastern Highlands—who had no knowledge of the uses of a
camera and therefore cannot be said to be camera-shy. As in most nonindustrialized,
small-scales societies, Fore lives were lived in hamlets in a communal fashion
(Sorenson, 1976; Sorenson et al., 1972). So, Fore everyday life was easily observed and
photographed. In 1980, Ekman published a set of these photographs. Ekman knew the
individuals he photographed, their situation, and their behavior. Based on this
information and on the facial muscle movements visible on the face, Ekman specified
the emotions conveyed by the faces he photographed.
Two studies exploited this valuable archive of photographs by showing them to
Western observers who were asked what emotions were being expressed (Kayyal &
Russell, 2013; Naab & Russell, 2007). Responses failed to match Ekman’s predictions
at the level anticipated. For example, fewer than 30% of English-speaking United States
(U.S.) Americans, English-speaking Palestinians, and Arabic-speaking Palestinians, on
average, selected the predicted emotion (Kayyal & Russell, 2013). This result is far
from what was thought needed to support universality. Haidt and Keltner (1999)
specified that to support the claim of universal recognition, 70-90% of responses should
match the prediction. A possible explanation for the low amount of recognition,

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 6
!
however, is the phenomenon of in-group bias (Elfenbein, 2013). Expressions are better
recognized by members of groups more similar to those who produced the expressions.
Here we report two studies on how residents of Papua New Guinea interpret the facial
expressions that Ekman (1980) photographed in Papua New Guinea.
The Present Studies
Here we report two studies testing the claim of universal recognition by showing
Ekman’s (1980) photographs of the Fore taken nearly a half of a century ago to
observers from another small-scale society in Papua New Guinea. Our observers came
from a different ethnic group—the Trobrianders, a group of subsistence fishermen and
horticulturalists living in a small archipelago located in the Solomon Sea and
approximately 200 km east of mainland Papua New Guinea (Leach & Leach, 1983;
Malinowski, 1929/1969, 1935/1965; Young, 1998). Trobrianders live in small villages
built with trees, palms, and coconut fronds. In the Trobriand Islands, there is no
electricity, no running water, no sewers, and limited medical assistance (for a more
detailed description of the Trobrianders, see Crivelli, Jarillo, Russell, & Fernández-
Dols, 2016). Because of ethnic differences, our study is best considered a within-nation
rather than within-culture comparison (Ojalehto & Medin, 2015).
In the first study, Trobrianders were shown five of Ekman’s photographs each
purported to show an expression of a single basic emotion. Trobrianders were asked to
use any word they wanted to describe how each person shown felt and to provide
valence and arousal ratings. The second study was carried out to clarify the results of
the first with a different and literate sample of Trobrianders. The experimenter showed
the same five spontaneous facial expressions, but this time forced participants to choose
among specified labels from a short list.
Both studies were conducted entirely in Trobrianders’s vernacular, Kilivila (see

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 7
!
Fellows, 1901; Senft, 1986, 2010), a language that the experimenter speaks fluently.
Kilivila—which belongs to the Austronesian family, Papuan tip cluster—is spoken
exclusively in the archipelago, whereas Tok Pisin (the Papua New Guinea lingua franca)
is completely unheard of in the Trobriand Islands. Elementary schoolchildren (5-9 years
old) use exclusively their vernacular during classes and are instructed in Trobriand
culture such as environmental knowledge (e.g., names of botanical and animal species)
or traditional folklore (e.g., dancing and singing). Despite significant governmental
efforts to introduce English as the common language, alphabetization attempts have
resulted in a slow switch from oral to written Kilivila. It is not until students reach
primary school (10-16 years old) that they learn how to spell, read, and write in their
vernacular. Schools in the Trobriand Islands lack sufficient teachers, infrastructure, and
basic facilities, resulting in students’ loss of interest and high dropout rate. For the
convenience of the reader, we describe the studies as if they were conducted in English,
with only occasional translations into Kilivila.
Study 1: Free Labeling
The language of the Trobrianders includes commonly used and understood
terms that are approximate translations of Ekman’s (1980) predictions: happiness
(mwasawa) for the smiling face, sadness (ninamwau) for the pouting face, anger (leya)
for the scowling face, surprise (eyowa lopola) for the brow raiser face, and disgust
(minena) for the nose scrunching face. Trobriand emotion concepts are not exact
translations of, but greatly overlap with, their English counterparts. Indeed, emotion
terms generally lack exact translation from one language to another (e.g., Kollareth &
Russell, 2016; Wierzbicka, 2014). For example, eyowa lopola (literally, his or her
insides have jumped) implies fast timing, little cognitive elaboration, and neutral
valence, whereas ekau nanogu (literally, it has taken my mind) suggests more cognitive

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 8
!
elaboration and positive valence; both phrases were scored as a correct response for the
surprise face.
Method
Participants. Thirty-two adolescents (M age = 14.84, SE = 0.14, age range: 14-
17 years old; 17 male) were recruited in Kaduwaga (n = 12) and Vakuta (n = 20)
villages (Trobriand Islands, Milne Bay Province, Papua New Guinea). Instruction in
English usually begins around age 10-12 years. Twenty-six of the participants could
understand and speak some English, with proficiency ranging from understanding a few
words to the production and understanding of some simple sentences.
Facial expressions. Five spontaneous facial expressions of emotion—
happiness, sadness, anger, surprise, and disgust—produced by the Fore people (Eastern
Highlands Province, Papua New Guinea) were selected from a set of 68 photographs
published by Ekman (1980). We selected the five facial expressions based on two
criteria: (a) they showed a clear and frontal view of the face, and (b) Ekman (1980)
assigned to the expression a single label for a basic emotion. Each facial expression was
also coded with the facial action coding system (Ekman, Friesen, & Hager, 2002); see
Appendix, Table A1. Images were formatted with a similar size (average size 7.4 cm X
5.2 cm) and laminated.
Procedure. The study was carried out during class time to avoid leaking of
information. Participants arrived sequentially to an isolated testing area to be tested
individually. In a preliminary step, the experimenter ascertained that each participant
was able to make quantitative judgments on a 4-point ordinal scale. The participant saw
four cards with different quantities of fish (none, 3, 9, and 18 fish) and was asked to
touch the card that had no (gala), some (pikekita), a lot (bidubadu), or all (sena
bidubadu) the fish. All participants successfully passed this test.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 9
!
Next, the experimenter showed the participant one of the faces selected
randomly from the set of five and asked, “What does the person in the picture feel?”
The experimenter asked the participant to provide only one descriptor for the face or to
answer, “I don’t know” if they really didn’t know. In the event that the participant
provided more than one descriptor, the experimenter asked for the single best
descriptor. In the event that the participant described the face with only a general
positive or negative label (e.g., feeling bad) or described the face on a general arousal
dimension (e.g., sleepy), the experimenter asked for a more specific descriptor. These
procedures were crafted to nudge the participant toward providing a term for a discrete
emotion (or at least a discrete feeling), but without forcing them to do so and without
hinting just which emotion was expected. All the same, despite these nudges, the
participant had the final word on what label was taken as his or her response.
Next, for the same facial expression, the experimenter asked for a judgment of
the core affect—the broad dimensions of valence and arousal (Russell, 1980, 2003)—
the face expressed. The experimenter first asked participants whether the person was
feeling good (bwena) or bad (gaga). The dichotomous choice was followed by a request
for an intensity rating. For example, for a face that had been judged to be feeling good,
the experimenter asked the participant, “Is he feeling a little good (pikekita bwena)?
very good (sena bwena)? or extremely good (kena sena bidubadu bwena)?” In the same
vein, the experimenter asked whether the person was feeling sleepy (enunupila matala)
or aroused (emamata) followed by an intensity judgment. For example, for a face
judged to be aroused, the experimenter asked the participant, “Is he a little bit aroused
(pikekita emamata)? very aroused (sena emamata)? or extremely aroused (sena
bidubadu emamata)?”
This procedure was then repeated for each of the remaining faces. The order of

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 10
!
presentation of the faces was randomized separately for each participant.
Translation of responses. The experimenter’s knowledge of the vernacular
helped to translate Kilivila responses into English (Crivelli, Jarillo, & Fridlund, 2016;
Crivelli, Jarillo, Russell, & Fernández-Dols, 2016; Fernández-Dols & Crivelli, 2014).
Additionally, we relied on an ethnographic database of Trobrianders’ emotion concepts
generated in several islands of the archipelago with the help of mono- and bilingual
informants (for an analysis of emotion concepts in Trobrianders’ vernacular, see Senft,
1986, 1998). Following a liberal approach, we grouped together several labels into a
broader category (see the Appendix, Table A2). In some cases, dialectal differences
justified the grouping of labels. For example, bwena—translated as good—is used in all
islands of the Trobriand archipelago except in Vakuta Island where boina is used
instead. In other cases, we grouped together labels derived from the same root. For
example, togigila (translated as a laughing man) is the result of adding the nominal
classificatory prefix to- (i.e., a human male) with gigila (laughter).
Scoring and data analysis. Every participant provided one label or said, “I
don’t know,” per trial. Data analyses were performed with R (Kabacoff, 2015; R Core
Team, 2014), using functions in different packages. In order to assess whether or not the
distribution of labels generated was uniform, we performed chi-square goodness-of-fit
tests for the distribution of label’s frequencies within every facial expression. In order to
test whether any given label would be dubbed “characteristic” for certain facial display,
we obtained p vales and standardized residuals by bootstrapping 10,000 replicates for
simulation. Labels with standardized residuals higher than 2 SD will show a
significantly displacement of cases towards that category and, eventually, it will show
that the label is the most characteristic for that facial expression (Agresti, 2013).
To convert the core affect responses to a quantitative score, participant’s initial

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 11
!
dichotomous response was used to determine the score’s sign: positive valence/aroused
received a positive score, negative valence/sleepy a negative score, and “I don’t know”
a score of zero. The intensity judgment then was assigned a number from 1 to 3. Final
scores thus ranged from -3 to +3. As it happens, no participant said, “I don’t know.”
Results
Labels. The labels generated for each facial expression, along with the
proportion of participants who gave that label, is shown in Table 1. A fuller list with all
labels for all faces and the frequency of occurrence is provided in the Appendix (Table
A2).
The emotion labels predicted by Ekman (1980) occurred, but rarely. For no face
did a majority of Trobrianders produce the predicted label. For no face was the
predicted label modal. The mean proportion responding with the predicted label across
the five facial expressions was .08 (95% CI [.04, .13]). The highest proportion occurred
for face predicted to convey sadness (.16, 95% CI [.06, .32]). Lower values occurred for
the happy (.13, 95% CI [.04, .29]), disgust (.06, 95% CI [.01, .21]), and anger (.03, 95%
CI [.00, .17]) faces. For the surprised face, no Trobriander provided the expected label
or labels closely related to the expected label (ekau nanogu, eyowa lopola).
The data provided other indications that the Trobianders did not see a one-to-one
correspondence between the face and the predicted emotion. Thus, the predicted label
(ninamwau, sadness) provided for the sad face (.16) was also provided for surprised
(.16), disgusted (.09), and angry (.06) faces, but never for the happy face. A possible
interpretation of the .16 “correct” labeling of the sad face is therefore that the label
ninamwau was simply used more frequently than other labels for negative faces.
Altogether, the label ninamwau was used more often for some “incorrect” face than for
the “correct” one.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 12
!
The modal labels were unexpected: gigila (laughing, smiling) and gibulwa (feels
like avoiding social interaction). Gigila was modal for the happy face (.44, 95% CI [.28,
.61]), χ2 = 21.69, p < .0001. Gibulwa was modal for the other four faces: the sad face
(.22, 95% CI [.11, .39]), angry face (.56, 95% CI [.39 .72]), surprised face (.19, 95% CI
[.09, .36]), and disgusted face (.22, 95% CI [.11, .39]). Gibulwa was thus a general
term, but most characteristic for the angry face and not equally distributed across the
four faces for which it was modal, χ2 = 36.89, p < .0001. It might be argued that gigila
should be counted as correct for the happy face, gibulwa for the angry face. We
postpone discussion of that argument until the General Discussion section.
We also found that, despite our urgings, some Trobrianders did not produce a
label for the emotion expressed by three of the five facial expressions. “I don’t know”
was the second modal category for the sad (.19) and disgusted (.19) faces and the third
modal category for the surprised face (.16).
Dimensions of core affect. Figure 1 shows Trobrianders’ mean ratings for each
face on the valence and arousal dimensions. Friedman tests showed significant
differences across faces in valence, χ2(4, N = 32) = 68.68, p < .001, and arousal, χ2(4, N
= 32) = 46.23, p < .001. For valence, Trobrianders attributed pleasure to the happy face
(M = 1.38, SE = 0.33), and displeasure to the sad (M = -0.53, SE = 0.31), angry (M = -
1.25, SE = 0.27), surprised (M = -0.91, SE = 0.29), and disgusted (M = -2.94, SE = 0.04)
faces. For arousal, Trobrianders attributed high arousal to the happy (M = 1.06, SE =
0.31), sad (M = 1.44, SE = 0.22), angry (M = 2.13, SE = 0.21), and surprised (M = 1.38,
SE = 0.25) faces and low arousal to the disgusted face (M = -1.31, SE = 0.32).
For comparison purposes, we relied on a sample of participants from the United
States who had rated the five facial expressions on the core affect dimensions with
bipolar 7-point Likert scales (Kayyal & Russell, 2013). We turned all ratings into

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 13
!
dichotomous scores (Table 2). A majority of Trobrianders and U.S. Americans had the
same dominant dichotomous valence score for all 5 faces and the same dominant
dichotomous arousal score for 4 of 5 faces. The one case of disagreement was the
disgusted face’s arousal ratings. Only a small proportion of Trobrianders considered the
disgusted face as highly activated (.19, 95% CI [.09, .36]), whereas a majority of U.S.
Americans did (.80, 95% CI [.66, .89]), χ2(1, N = 77) = 25.88, p < .001, 95% CI [-.76, -
.40].
The proportion of Trobrianders’ attributions of positive or negative valence and
high or low arousal was similar but not identical to U.S. Americans’ attributions.
Significantly more Trobrianders than U.S. Americans rated sad, angry, and surprised
faces as aroused (see Table 2). A two-sample test for equality of proportions showed the
former difference in the proportion of Trobrianders’ and U.S. Americans’ attributions of
high arousal; for sad faces, χ2(1, N = 78) = 5.36, p = .021, 95% CI [.07, .44]; angry
faces, χ2(1, N = 78) = 11.14, p < .001, 95% CI [.19, .53]; and surprise faces, χ2(1, N =
72) = 7.38, p = .007, 95% CI [.12, .51].
Discussion
Shown spontaneous facial expressions produced by the Fore (Eastern Highlands
Province, Papua New Guinea), Trobrianders (Milne Bay Province, Papua New Guinea)
responded easily and with confidence. They occasionally produced the emotion labels
predicted by Ekman (1980). For three out of the five faces, however, despite our
urgings, almost 20% of Trobrianders stated that they did not know which emotion was
displayed, whereas, when asked about the dimensions of core affect, no Trobriander
said, “I don’t know.”
Ekman’s predicted labels were rare, provided for four of the five faces by a
small proportion of respondents (the highest proportion was .16). Further, the predicted

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 14
!
labels were also offered for the “wrong” face for four of the five faces. Trobrianders’
more frequent responses were gigila (laughing, smiling) for the happy face and gibulwa
(feeling like avoiding social interaction) for the sad, angry, surprised, and disgusted
faces.
Trobrianders also showed a significant overlap with U.S. Americans judging the
dimensions of core affect for the same faces. These data suggest that Trobrianders may
share with Westerners a process of interpreting facial expressions based on valence and
arousal—a process seen even in children of both Western (Kayyal & Russell, 2013;
Widen, 2013; Widen & Russell, 2008, 2010) and small-scale, indigenous societies
(Crivelli, Jarillo, Russell, & Fernández-Dols, 2016).
Study 2: Multiple Choice
Study 2 was designed to complement Study 1 in its focus on the same five Fore
spontaneous facial expressions. It might be argued that Study 1’s open-ended response
format was too open. Haidt and Keltner (1999) provided evidence that open-ended
response formats tend to show less agreement, whereas more constrained response
formats are more likely to support predictions from Basic Emotion Theory. Therefore,
multiple-choice response formats have often been used and recommended (e.g., ;
Boucher & Carlson, 1980; Ekman & Friesen, 1971).
Specifically, Study 2 addressed two issues. First, perhaps the predicted emotion
label did not come to mind, but would be endorsed if it were made available. We
therefore used here the common forced-choice response format that included all the
predicted labels. Frank and Stennett (2001) showed that adding certain alternatives
enhanced the performance of a forced-choice format. Here, we added two, one for any
emotion not listed and one for not knowing the meaning of the face. Second, in Study 1,
participants had frequently used two unexpected labels: a word for a behavior, gigila

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 15
!
(laughing, smiling), and a general emotion term, gibulwa (feeling like avoiding social
interaction). Perhaps these labels were used only because the predicted labels did not
come to mind. Do the Trobrianders genuinely prefer these labels when Ekman’s (1980)
predicted labels are also available?
The method of Study 2 was similar to that of Study 1, with two major
exceptions. First, the response format required respondents to choose one among
Ekman’s (1980) predicted emotion labels, two modal terms found in Study 1, itwali
(other [emotion]) and gala anukwali (I do not know). Second, to ensure that all response
options were clear and available, we recruited a sample of literate Trobrianders who
read the response options from a list.
Method
Participants. Twenty-nine adolescents, who had not participated in Study 1,
were recruited in Vakuta Island. We excluded five of these adolescents (2 male) who
failed our test of literacy (see below). The final sample thus consisted of 24 literate
adolescents (M age = 12.96, SE = 0.11, age range: 12-14 years old; 13 male). Twenty of
them could understand and speak some English words.
Facial expressions. The same set of faces used in Study 1 were used in Study 2
(see the Appendix, Table A1).
Response format. The response format consisted of nine written terms. Five of
the labels were predicted by Ekman (1980): mwasawa (happiness), ninamwau
(sadness), leya (anger), eyowa lopola (surprise, startle), and minena (disgust). Two of
the labels were Study 1’s modal categories: gigila (laughing, smiling) and gibulwa
(feels like avoiding social interaction). And, two of the remaining labels were itwali
(other emotion) and gala anukwali (I do not know). On the actual questionnaire, only
the Kilivila terms were listed. The items were always presented in the same order:

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 16
!
gilbuwa, ninamwau/mwau, minena, eyowa lopola, gigila, leya, mwasawa, itwali, and
gala anukwali.
Literacy test. To be eligible for Study 2, Trobrianders had to pass a literacy test.
The experimenter handed each potential participant a written list with all the response
options, which they were asked to read aloud twice. Then, they were asked whether they
understood the meaning of all the labels. All but five potential participants passed the
test. The five (2 males) who failed the test were dismissed from the study but rewarded
with candy. Thus, all actual participants could read the list and reported that they
understood the meaning of all the labels.
Procedure. The procedure was the same as in Study 1 except as follows. For
every trial, the participant was asked to read, once again, the entire list of labels before
selecting one for the face shown.
Scoring and data analysis. Data analyses were performed with R (Kabacoff,
2015; R Core Team, 2014). For computing a Cochran tests on the proportions of Table
3’s main diagonal we used the “coin” package (Hothorn, Hornik, van de Wiel, &
Zeileis, 2008); for binomial tests we used the “binom” package (Dorai-Raj, 2014) and
we relied on Agresti-Coull’s method for estimating 95% CIs (Agresti & Coull, 1998).
Results
Ekman’s predicted labels. The overall mean proportion of Trobrianders who
selected Ekman’s (1980) predicted emotion label for the predicted face was .23 (95% CI
[.17, .32]). Proportions “correct” (i.e., matching prediction) were low and similar across
faces. The highest “correct” proportion occurred for the disgust face (.38, 95% CI [.21,
.57]), followed by the sad face (.29, 95% CI [.15, .49]), surprise face (.21, 95% CI [.09,
.41]), happy face (.17, 95% CI [.06, .37]), and angry face (.13, 95% CI [.04, .32]). The
similarity of proportions for different faces seen in the overlapping of the 95% CIs was

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 17
!
confirmed by a Cochran’s Q test for matched samples applied to Table 3’s main
diagonal proportions, χ2(4, N = 24) = 5.8, p = .215.
Some researchers have compared the obtained “correct” proportion against a
theoretical proportion set to rule out guessing. In Study 2, if the participants had
selected from the nine options completely randomly, then the cutoff value to rule out
guessing would be .11. Only two of Ekman’s predicted labels, ninamwau (sadness) and
minena (disgust), significantly exceeded this cutoff value by an exact right unilateral
binomial test (all ps < .013).
On the contrary, Study 1’s modal label gigila (laughing, smiling) was
consistently selected as the most characteristic label for the happy face (.67, 95% CI
[.47, .82]), χ2 = 81.75, p < .001. Gibulwa (feels like avoiding social interaction) was the
modal label for the angry face (.50, 95% CI [.31, .69]) and the disgusted face (.33, 95%
CI [.18, .53]). Exact right unilateral binomial tests ruled out guessing (all ps < .004).
Misattributions. Trobrianders also occasionally selected an “incorrect” emotion
label for a face even when the “correct” label was on the list. For example, .29 of
respondents selected ninamwau (sad) for the sad face, but the same proportion of
respondents selected ninamwau (sad) for the angry face. Indeed, every one of Ekman’s
predicted labels was “misattributed” to an “incorrect” face, and every face received one
of the “incorrect” labels from the set of five Ekman labels. McNemar tests for giving
probabilities were computed with simulated p values based on 10,000 replicates in order
to test the distribution of emotion labels across all faces (Table 4). We found
misattributions of Ekman’s predicted labels across all five facial expressions, except for
the label minena (disgust), which showed no misattribution. The label mwasawa
(happiness) was attributed to both happy (.17) and surprised (.13) faces; the label
ninamwau (sadness) was attributed to happy (.13), sad (.29), angry (.29), and surprised

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 18
!
(.21) faces; the label leya (anger) was attributed to sad (.17), angry (.13), and disgusted
(.13) faces; and the label eyowa lopola (surprise) was attributed to sad (.17) and
surprised (.21) faces.
Study 1’s modal labels. The modal labels from Study 1, gigila and gibulwa,
were also chosen, even though Ekman’s predicted labels were available on the response
list. The label gigila (laughing, smiling) was matched to the happy face by a majority of
respondents. The label gibulwa (feeling like avoiding social interaction) was modal for
the angry face and selected for all the faces (sad, angry, surprised, and disgusted) except
the happy one. The results with gibulwa mirrored the role of the valence dimension seen
in Study 1 (see Figure 1).
Trobrianders did not prefer Ekman’s predicted labels over the two modal labels
seen in Study 1. A two-sample permutation test on the number of times Trobrianders
matched faces to Ekman’s (1980) predictions or Study 1’s modal labels was used to
validate Study 1’s results. In the present study, Trobrianders selected gigila (laughing,
smiling) significantly more times than mwasawa (happiness) for the happy face, p =
.001, 95% CI [-.77, -.25]. They selected gibulwa (feeling like avoiding social
interaction) significantly more times than leya (anger) for the angry face, p = .011, 95%
CI [-.64, -.11]. For the remaining three faces—sad, surprised, and disgusted—matching
scores’ proportions did not differ significantly between Ekman’s predictions and the
two modal labels of Study 1 (Table 5).
Discussion
Change of response format from free labeling to multiple choice had only subtle
effects on the resulting portrait of Trobrianders’ interpretation of facial expressions.
Contrary to Study 1, here only .06 of overall responses were “I don’t know.” Ekman’s
(1980) predicted labels continued to fare poorly. Although Ekman’s predicted emotion

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 19
!
label was modal for three of the five faces, in no case did a majority of Trobrianders
select that label. In the remaining two cases (happy and angry faces), the modal label
was the same as in Study 1. For disgusted faces, Ekman’s (1980) and Study 1’s
predictions were not significantly different. Ekman’s (1980) predictions are also
challenged by the large number of misattributions.
In contrast, Study 1’s modal label for the happy face, gigila (laughing, smiling),
was similarly attributed to the happy face in Study 2. Likewise, Study 1’s modal label
for sad, angry, surprised, and disgusted faces, gibulwa (feeling like avoiding social
interaction), was matched to the same faces in Study 2.
General Discussion and Conclusion
The facial stimuli studied here were a unique set of expressions spontaneously
produced by hunter-gatherers living in the Eastern Highlands of Papua New Guinea,
relatively isolated from the West and unaware of the uses of a camera at the time the
photographs were taken. Paul Ekman (1980) had taken the photographs, knew the
individuals, their situations, and their behavior. Based on this information and on the
facial muscle movements visible on the face, Ekman specified the emotion conveyed by
the faces we studied. The emotion attributed to the face within Basic Emotion Theory
can also be seen in coding by the facial action coding system (Ekman, Friesen, &
Hager, 2002).
Despite the importance of the classic facial expression studies in Papua New
Guinea, ours is the first study of Papua New Guineans interpreting facial expressions
spontaneously produced by other Papua New Guineans. The participants studied here,
Trobrianders, were also relatively culturally and visually isolated not only from Western
and Eastern industrialized countries but from other provinces of Papua New Guinea as
well.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 20
!
Trobrianders did occasionally select Ekman’s predicted labels in an open-ended
free labeling procedure and in a multiple-choice response format. Still, the overall
pattern of responding was not highly supportive of the predictions of Basic Emotion
Theory. Haidt and Keltner (1999) had set 70% to 90% agreement on the predicted label
as their expected level of support of the Basic Emotion Theory. Here, this amount of
agreement was never achieved. With forced choice, agreement with Ekman’s prediction
was consistently low, ranging from 13% to 38% of respondents. With free labeling,
agreement ranged from 0% to 16%. Further, Ekman’s predicted emotion labels were
often chosen for the “wrong” facial expression.
Free labeling and multiple-choice response format each have advantages and
disadvantages and are best viewed as complementary. Free labeling captures the
spontaneously produced categorization of the face but may underestimate agreement
with prediction because respondents did not have the needed word readily available.
Forced choice, in contrast, may overestimate agreement with prediction because
respondents indicate only which option is best among those listed, but possibly not their
spontaneous interpretation.
The limitations of the present study are largely obvious. One limitation of any
cross-cultural comparison stems from problems in translation. Emotion terms in
different languages do not translate one-to-one (Wierzbicka, 1999, 2014). Translation is
more of a problem for free labeling than for multiple choice. It might be argued that
more terms freely generated should have been counted as correct for Ekman’s
predictions. All terms, other than idiosyncratic ones, generated for each facial
expression are listed in the Appendix (Table A2), and the reader can rescore. For
example, one might argue that gigila (i.e., laughter, giggle, smile) should be counted as
a synonym of happy. Ethnographic and ethnolinguistic data, however, indicate that

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 21
!
gigila is not a synonym of happy (Fellows, 1901; Senft, 1986). Gigila refers to a
behavior and does not imply happiness. Trobrianders have a different word for
happiness, mwasawa. Further, in Study 2, when respondents were forced to choose,
17% chose mwasawa, and 67% chose gigila, for the “happy face.” Similarly, one might
argue that gibulwa should be counted as correct for the anger face. But Kilivila has a
word for anger (leya), and gibulwa is not its synonym. Gibulwa expresses a more
general feeling (feels like avoiding social interaction) and was selected for all faces
(sad, angry, surprised, and disgusted) except the happy one.
Results from both of our studies suggested that the broad dimensions of core
affect played a role in the interpretation of faces by the Trobrianders. When a core
affect interpretation of faces was assessed directly, Trobrianders largely agreed with
Westerners’ interpretation on valence for all 5 faces, and on arousal for four of five
faces. Further, use of gigila and gibulwa was consistent with a valence interpretation:
gigila was used only for the happy face, gibulwa for all negative facial expressions.
Still, core affect is clearly only a part of the interpretation placed on the faces.
Our two studies were designed to test a prediction from Basic Emotion Theory
and, secondarily, from Core Affect Theory. They were not designed to test other
theories, nor did they. All the same, the results do suggest that other theories merit
attention. The frequent use of gibulwa suggests that social messages were a part of the
interpretation Trobriander respondents placed on facial movements, even when directly
compared with emotion messages. This finding resonates with Fridlund’s (1994)
Behavioral Ecology Theory based on evolutionary considerations. This finding also
resonates with Frijda and Tcherkassof’s (1997) theory that facial expressions convey
action intentions. The contrast between the older findings of Ekman and Friesen (1971)
with the Fore and our findings with Trobrianders resonates with Elfenbein’s (2013)

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 22
!
theory of dialects of facial expressions. Perhaps residents of Papua New Guinea have
different dialects in their facial communication. More generally, the present findings
suggest that we need to reopen the discussion on the production and interpretation of
facial movements and encourage theories of all sorts.
It is customary at this point to say how nice it would be to have more data from
more groups on more faces. Perhaps. But much evidence confirms that both core affect
and Basic Emotion Theory provide at best only a first rough approximation of an
account of how people make sense of the facial expressions of others. Attempts to test a
Basic Emotion Theory account of facial expressions with the same methods appear to
be producing diminishing returns. Our results invite the development of new methods,
but also new theories that seek a better understanding of facial expressions and their
interpretation. Such theories could suggest new categories of facial movements, new
messages conveyed by those faces, and new processes by which facial movements are
produced and interpreted—and hence might breathe new life into this field.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 23
!
Author contributions
CC, JMFD, and JAR developed the study concept. All authors contributed to the study
design. CC performed data collection. All authors performed the data analysis and
contributed to the interpretation of the results. CC and JAR drafted the paper, whereas
JMFD and SJ provided critical revisions. All authors approved the final version of the
paper for submission.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 24
!
References
Agresti, A. (2013). Categorical data analysis (3rd ed.). Hoboken, NJ: Wiley.
Agresti, A., & Coull, B. A. (1998). Approximate is better tan “exact” for interval
estimation of binomial proportions. The American Statistician, 52, 119-226.
Boucher, J. D., & Carlson, G. E. (1980). Recognition of facial expression in three
cultures. Journal of Cross-Cultural Psychology, 11, 263-280.
Crivelli, C., Jarillo, S., & Fridlund, A. J. (2016). A multidisciplinary approach to
research in small-scale societies: Emotions and facial expressions in the field.
Frontiers in Psychology, 7:1073.
Crivelli, C., Jarillo, S., Russell, J. A., & Fernández-Dols, J. M. (2016). Reading
emotions from faces in two indigenous societies. Journal of Experimental
Psychology: General, 145, 830-843.
DiGirolamo, M. A., & Russell, J. A. (2016). The emotion seen in a face can be a
methodological artifact: The process of elimination hypothesis. [Manuscript
submitted for publication].
Dorai-Raj, S. (2014). binom: Binomial confidence intervals for several
parameterizations. Retrieved from http://cran.r-project.org/package=binom
Ekman, P. (1972). Universal and cultural differences in facial expressions of emotion.
In J. R. Cole (Ed.), Nebraska Symposium on Motivation (Vol. 19, pp. 207-283).
Lincoln, NE: Nebraska University Press.
Ekman, P. (1980). The face of man. New York, NY: Garland.
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48, 384-392.
Ekman, P. (1994). Strong evidence for universals in facial expressions: A reply to
Russell’s mistaken critique. Psychological Bulletin, 115, 268-287.
Ekman, P. (2003). Emotions revealed: Recognizing faces and feelings to improve

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 25
!
communication and emotional life. New York, NY: Times Books.
Ekman, P. (2016). What scientists who study emotion agree about. Perspectives in
Psychological Science, 11, 31-34.
Ekman, P., & Cordaro D. (2011). What is meant by calling basic emotions. Emotion
Review, 3, 364-370.
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion.
Journal of Personality and Social Psychology, 17, 124-129.
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial
displays of emotion. Science, 164, 86-88.
Elfenbein, H. A. (2013). Nonverbal dialects and accents in facial expressions of
emotion. Emotion Review, 5, 90-96.
Fellows, S. B. (1901). Grammar of the Kilivila dialect (together with a vocabulary), in
Annual Report of British New Guinea 1900-1901. Brisbane, Australia:
Government Printer.
Fernández-Dols, J. M., & Crivelli, C. (2014). Recognition of facial expressions: Past,
present, and future challenges. In M. Mandal, & A. Awasthi (Eds.),
Understanding facial expressions in communication: Cross-cultural and
multidisciplinary perspectives (pp. 19-40). New Delhi, India: Springer.
Frank, M. G., & Stennett, J. (2001). The forced-choice paradigm and the perception of
facial expressions of emotion. Journal of personality and social psychology, 80,
75-85.
Fridlund, J. A. (1994). Human facial expression: An evolutionary view. San Diego, CA:
Academic Press.
Frijda, N. H., & Tcherkassof, A. (1997). Facial expressions as modes of action
readiness. In J. A. Russell & J. M. Fernández-Dols (Eds.), The psychology of

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 26
!
facial expression (pp. 78-102). Cambridge, UK: Cambridge University Press.
Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Perceptions
of emotion from facial expressions are not culturally universal: Evidence from a
remote culture. Emotion, 14, 251-262.
Gilovich, T., Keltner, D., & Nisbett, R. E. (2011). Social psychology (2nd ed.). New
York, NY: Norton.
Haidt, J., & Keltner, D. (1999). Culture and facial expression: Open-ended methods find
more expressions and a gradient of recognition. Cognition and Emotion, 13(3),
225-266.
Hess, U., & Blairy, S. (2001). Facial mimicry and emotional contagion to dynamic
emotional facial expressions and their influence on decoding accuracy.
International Journal of Psychophysiology, 40, 129-141.
Hothorn, T., Hornik, K., van de Wiel, M. A., & Zeileis, A. (2008). Implementing a class
of permutation tests: The coin package. Journal of Statistical Software, 28, 1-23.
Izard, C. E. (1971). The face of emotion. New York, NY: Appleton-Century-Crofts.
Izard, C. E. (1994). Innate and universal facial expressions: Evidence from
developmental and cross-cultural research. Psychological Bulletin, 115, 288-
299.
Jack, R. E., Blais, C., Scheepers, C., Schyns, P. G., & Caldara, R. (2009). Cultural
confusions show that facial expressions are not universal. Current Biology, 19,
1543-1548.
Kabakoff, R. I. (2015). R in action: Data analysis and graphics with R (2nd ed.).
Shelter Island, NY: Manning.
Kayyal, M. H., & Russell, J. A. (2013). Palestinians and Americans judge spontaneous
facial expressions of emotion. Emotion, 13, 891-904.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 27
!
Kollareth, D., & Russell, J. A. (2016). The English word disgust has no exact
translation in Hindi or Malayalam. Cognition and Emotion, 30, 1-12.
Leach, J. W., & Leach, E. R. (1983). The Kula: New perspectives on Massim exchange.
Cambridge, UK: Cambridge University Press.
Levenson, R. W. (2011). Basic emotion questions. Emotion Review, 3, 379-386.
Leys, R. (2010). How did fear become a scientific object and what kind of object it is?
Representations, 110, 66-104.
Malinowski, B. (1965). Coral gardens and their magic: A study of the methods of tilling
the soil and agricultural rites in the Trobriand Islands, vol. 1 and vol. 2. New
York, NY: American Books. (Original work published 1935)
Malinowski, B. (1969). The sexual life of savages in North-Western Melanesia. New
York, NY: Harcourt, Brace and World. (Original work published 1929)
Myers, D. G., & DeWall, C. N. (2015). Psychology (11th ed.). New York, NY: Worth.
Naab, P. J., & Russell, J. A. (2007). Judgments of emotion from spontaneous facial
expressions of New Guineans. Emotion, 7, 736-744.
Nelson, N. L., & Russell, J. A. (2016). A facial expression of pax: Assessing children’s
“recognition” of emotion from faces. Journal of Experimental Child
Psychology, 141, 49-64.
Ojalehto, B. L., & Medin D. L. (2015). Perspectives on culture and concepts. Annual
Review of Psychology, 66, 249-275.
Plamper, J. (2015). The history of emotions: An introduction. Oxford, UK: Oxford
University Press.
R Core Team (2014). R: A language and environment for statistical computing. Vienna,
Austria: R Foundation for Statistical Computing. Retrieved from http://www.r-
project.org

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 28
!
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social
Psychology, 39, 1161-1178.
Russell, J. A. (1994). Is there universal recognition of emotion from facial expression?
A review of the cross-cultural studies. Psychological Bulletin, 115, 102-141.
Russell, J. A. (1995). Facial expressions of emotion: What lies beyond minimal
universality? Psychological Bulletin, 118, 379-391.
Russell, J. A. (1997). Reading emotion from and into faces: Resurrecting a dimensional-
contextual perspective. In J. A. Russell & J. M. Fernández-Dols (Eds.), The
psychology of facial expression (pp. 295-320). New York, NY: Cambridge
University Press.
Russell, J. A. (2003). Core affect and the psychological construction of emotion.
Psychological Review, 110, 145-172.
Schacter, D. L., Gilbert, D. T., & Wegner, D. M. (2011). Psychology (2nd ed.). New
York, NY: Worth.
Senft, G. (1986). Kilivila: The language of the Trobriand Islanders. New York, NY:
Mouton de Gruyter.
Senft, G. (1998). Body and mind in the Trobriand Islands. Ethos, 26, 73-104.
Senft, G. (2010). The Trobriand islanders’ ways of speaking. New York, NY: Mouton
de Gruyter.
Sorenson, E. R. (1975). Culture and the expression of emotion. In T. R. Williams (Ed.),
Psychological anthropology (pp. 361-372). Chicago, IL: Aldine.
Sorenson, E. R. (1976). The edge of the forest: Land, childhood, and change in a New
Guinea protoagricultural society. Washington, DC: Smithsonian Institution
Press.
Sorenson, E. R., Claessen, H. J. M., du Torr, B. M., Griffith, J., Hockings, P., Jablonko,

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 29
!
A., … Watson, J. B. (1972). Socio-ecological change among the Fore of New
Guinea. Current Anthropology, 13, 349-383.
Wagner, H. L., MacDonald, C., & Manstead, A. (1986). Communication of individual
emotions by spontaneous facial expressions. Journal of Personality and Social
Psychology, 50, 737-743.
Widen, S. C. (2013). Children’s interpretation of facial expressions: The long path from
valence-based to specific discrete categories. Emotion Review, 5, 72-77.
Widen, S. C., & Russell, J. A. (2008). Young children's understanding of other's
emotions. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook
of emotions (3rd ed., pp. 348-363). New York, NY: Guilford Press.
Widen, S. C., & Russell, J. A. (2010). Differentiation in preschooler’s categories for
emotion. Emotion, 10, 651-661.
Wierzbicka, A. (1999). Emotions across languages and cultures: Diversity and
universals. Cambridge, UK: Cambridge University Press.
Wierzbicka, A. (2014). Imprisoned in English: The hazards of English as a default
language. New York, NY: Oxford University Press.
Yik, M., Widen, S. C., & Russell, J. A. (2013). The within-subjects design in the study
of facial expressions. Cognition and Emotion, 27, 1062-1072.
Young, M. W. (1998). Malinowski’s Kiriwina: Fieldwork photography 1915-1918.
Chicago, IL: University of Chicago Press.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 30
!
Table 1
Proportion of Trobrianders Providing Each Label for Five Fore Spontaneously Produced Faces, Study 1
Labels
Faces
Kilivila
English
P
Happiness
Gigila, togigila
Laughing, smiling
.44*
Mwasawa
Happiness, play
.13
Mwamwasila
Magic of attraction, radiance
.13
Bwena, boina, bo bwena
Good
.09
Gibulwa
Feels like avoiding social interaction
.06
Gala anukwali
I don’t know
.06
Idiosyncratic
.09
Sadness
Gibulwa
Feels like avoiding social interaction
.22
Gala anukwali
I don’t know
.19
Ninamwau, mwau
Sadness
.16
Bwena, boina, bo bwena
Good
.09
Ekatowla
Sick
.09
Kokola
Fear
.06
Idiosyncratic
.19
Anger
Gibulwa
Feels like avoiding social interaction
.56*
Ekabelu
Wry face
.09
Ninamwau, mwau
Sadness
.06
Bwena, boina, bo bwena
Good
.06
Gala anukwali
I don’t know
.06
Idiosyncratic
.16
Surprise
Gibulwa
Feels like avoiding social interaction
.19
Ninamwau, mwau
Sadness
.16

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 31
!
Gala anukwali
I don’t know
.16
Bwena, boina, bo bwena
Good
.09
Ekatowla
Sick
.06
Kaikai wowola, lawari
Worried
.06
Idiosyncratic
.28
Disgust
Gibulwa
Feels like avoiding social interaction
.22*
Gala anukwali
I don’t know
.19
Ninamwau, mwau
Sadness
.09
Minena
Disgust
.06
Gaga
Bad
.06
Mayuyu
Pain
.06
Ekabelu
Wry face
.06
Ekasigegina
Snarling
.06
Ekalimisimisi, Ekamakwesi
Rejecting, refusing
.06
Idiosyncratic
.16
Note. N = 32. Proportions are rounded up. Results for predicted terms in bold. Kilivila (Senft, 1986) is the
Austronesian language spoken in the Trobriand archipelago. Asterisks represent chi-square goodness-of-fit tests’
standardized residuals higher than 2 SD. Idiosyncratic = Labels provided once (proportion < .04). Only one
Trobriander provided the predicted term (leya, anger) for the angry face.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 32
!
Table 2
Proportion of Trobrianders and U.S. Americans Rating a Set of Fore Faces on Core Affect Measures, Study 1
Society
Faces
Core affect
Trobrianders
U.S. Americans
Χ2
p
95% CI
Valence
Happiness
+
.75**
.65*
0.55
.457
[-.11, .29]
−
.25
.35
Sadness
+
.31
.00
−
.69*
1.00***
16.76
<.001
[-.49, -.18]
Anger
+
.19
.00
−
.81***
1.00***
9.16
.003
[-.35, -.09]
Surprise
+
.28
.17
−
.72*
.83***
0.50
.481
[-.32, .11]
Disgust
+
.00
.00
−
1.00***
1.00***
n/a
n/a
n/a
Arousal
Happiness
+
.75**
.73**
0.01
.999
[-.19, .22]
−
.25
.27
Sadness
+
.88***
.61
5.36
.021
[.07, .44]
−
.13
.39
Anger
+
.94***
.57
11.14
<.001
[.19, .53]
−
.06
.44
Surprise
+
.88***
.55
7.38
.007
[.12, .51]

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 33
!
−
.13
.45
Disgust
+
.19
.80***
25.88
<.001
[-.76, -.40]
−
.81***
.20
Note. Proportions are rounded up. Stimuli could either be rated as positive (+) or negative (−) on the dimensions of
valence and arousal. Asterisks represent right unilateral binomial tests’ p values (π = .50). CI = confidence interval.
*p < .05 (one-tailed)
**p < .01 (one-tailed)
***p < .001 (one-tailed)

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 34
!
Table 3
Proportion of Trobrianders Matching a Face to an Emotion Label, Study 2
Labels
Happiness
Sadness
Anger
Surprise
Disgust
Faces
Mwasawa
Ninamwau
Leya
Eyowa
lopola
Minena
Gigila
Gibulwa
Other
I don’t know
χ²
p
Happy
.17
.13
.00
.00
.00
.67*
.00
.00
.04
81.75
<.001
Sad
.04
.29
.17
.17
.04
.00
.13
.04
.13
14.25
.078
Angry
.00
.29*
.13
.04
.04
.00
.50*
.00
.00
52.05
<.001
Surprised
.13
.21
.04
.21
.04
.00
.17
.13
.08
9.75
.306
Disgusted
.00
.04
.13
.04
.38*
.00
.33*
.04
.04
35.25
<.001
Note. N = 24. Proportions are round up. Results for predicted terms in bold. To obtain p values, chi-square goodness-of-fit tests were
computed on rows by bootstrapping 10,000 replicates for simulation. Gigila = Laughing, smiling. Gibulwa = Feels like avoiding social
interaction.
* Values with standardized residuals higher than 2 SD.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 35
!
Table 4
McNemar Tests for Labels Across Faces
Labels
Faces
Happiness
Sadness
Anger
Surprise
Disgust
Gigila
Gibulwa
Happy
.17b
.13ab
.00a
.00a
.00a
.67b
.00a
Sad
.04ab
.29b
.17b
.17b
.04a
.00a
.13ab
Angry
.00a
.29b
.13ab
.04ab
.04a
.00a
.50c
Surprised
.13ab
.21ab
.04ab
.21b
.04a
.00a
.17b
Disgusted
.00a
.04a
.13ab
.04ab
.38b
.00a
.33bc
Note. Proportions are rounded up. Proportions with different subscripts in the same column differed significantly at
p < .05 according to McNemar tests. Gigila = laughing, smiling. Gibulwa = feels like avoiding social interaction.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 36
!
Table 5
Estimated Difference of Proportions Between Basic Emotion Theory’s Predicted Label and Study 1’s Modal
Labels
Labels
Faces
BET
Study 1’s modal
p
95% CI
Happiness
.17
.67
.001
[-.77, -.25]
Sadness
.29
.13
.287
[-.08, .40]
Anger
.13
.50
.011
[-.64, -.11]
Surprised
.21
.17
.999
[-.19, .27]
Disgust
.38
.33
.999
[-.25, .33]
Note. Proportions are rounded up. P values and 95% CIs for the difference of matching scores’ proportions
between Basic Emotion Theory and Study 1’s modal labels were computed through two-sample permutation
tests. The distribution under the null hypothesis was computed from all possible permutations. Study 1’s
modal labels were gigila (laughing, smiling) for the happy faces and gibulwa (feeling like avoiding social
interaction) for sad, angry, surprised, and disgusted faces. BET = Basic Emotion Theory. CI = confidence
interval.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 37
!
Figure 1. Balloon plot showing Trobrianders’ 20% trimmed means for valence
and arousal ratings of facial expressions of “emotion” spontaneously produced by Fore
members. The size of the plotted points represent a nonlinear transformation of the
median absolute deviation (MAD) computed for valence and arousal ratings.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 38
!
Appendix
Stimulus Set and Raw Data from Study 1
Table A1
Stimulus Set for Study 1 and Study 2
Ekman’s (1980) prediction
Plate
Action units
Happy
37B
2L + 7 + 12 + 25
Sad
14
1 + 4 + 5 + 7
Angry
17
4 + 5 + 7
Surprised
7
1 + 2 + 4 + 25
Disgusted
16
4 + 6 + 9 + 10
Note. Plate = the identification number provided by Ekman (1980) for the selected
Fore facial expressions. Action units = an anatomical coding system to identify and
describe facial muscles’ contractions as coded by Ekman, Friesen, & Hager’s (2002)
Facial Action Coding System (FACS). We randomly selected one picture from the
two available happy (plates 37B and 8f) and surprised (plates 7 and 8m) faces.

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 39
!
Table A2
Number of Subjects Providing an Emotion Label for a Set of Fore Faces
Labels
Faces
English
Kilivila
Happiness
Sadness
Anger
Surprise
Disgust
Happiness, play
Mwasawa
4
0
1
0
0
Sadness
Ninamwau, mwau
1
5
2
5
3
Anger
Leya
1
0
1
0
1
Surprise
Eyowa lopola
0
0
0
0
0
Disgust
Minena
0
0
0
0
2
Laughing, smiling
Gigila, togigila
14
0
0
0
0
Avoiding social interaction
Gibulwa
2
7
18
6
7
I don’t know
Gala anukwali
2
6
2
5
6
Good
Bwena, boina, bo bwena
3
3
2
3
0
Bad
Gaga
0
0
1
1
2
Magic of attraction, radiance
Mwamwasila
4
1
1
1
0
Fear
Kokola
0
2
0
1
1
Pain
Mayuyu
0
1
0
0
2
Sick
Ekatowla
0
3
1
2
0
Wry face
Ekabelu
0
0
3
0
2

doi: 10.1037/emo0000236
SPONTANEOUS FACIAL EXPRESSIONS 40
!
Snarling
Ekasigegina
0
0
0
1
2
Crying
Evalam
1
0
0
1
0
Confused
Enakaka
0
0
0
1
0
Embarrassment
Mwasila
0
0
0
1
0
Headache
Gedageda dabala
0
0
0
0
1
Worried
Kaikai wowola, lawari
0
0
0
2
0
Looking
Egigisa
0
1
0
0
0
Rejecting, not wanting
Ekalimisimisi
0
0
0
0
1
Doubting
Ewowoya
0
1
0
0
0
Opening wide the eyes
Etolatola matala
0
0
0
1
0
Rejecting, refusing
Ekamakwesi
0
0
0
0
1
Shut up
Ekapatu
0
0
0
1
0
Smart, healthy
Salau
0
1
0
0
0
Squint
Emitupayuyu
0
0
0
0
1
Bored
Kalanunumata
0
1
0
0
0
Note. Kilivila (Senft, 1986) is the Austronesian language spoken in the Trobriand archipelago.