Content uploaded by Dolichan Kollareth
Author content
All content in this area was uploaded by Dolichan Kollareth on Aug 21, 2020
Content may be subject to copyright.
1
New Basic Emotions?
Running head: NEW BASIC EMOTIONS?
On Evidence for a Dozen New Basic Emotions:
A Methodological Critique
Dolichan Kollareth, John Esposito, Yiran Ma, Hiram Brownell,
& James A. Russell
Boston College
© 2020, American Psychological Association. This paper is not the copy of record
and may not exactly replicate the final, authoritative version of the article. Please
do not copy or cite without authors' permission. The final article will be available,
upon publication, via its DOI: 10.1037/emo0000904
Author Note
Department of Psychology and Neuroscience, Boston College.
We thank Annmary Devassykutty for help gathering data.
For data from each of the studies, see https://osf.io/hs2fv/?
view_only=991e317cc44b4d388dc390a5b4535fcc
Correspondence concerning this article should be addressed to James A. Russell,
Department of Psychology and Neuroscience, 300 McGuinn Hall, Boston College, 140
Commonwealth Ave, Chestnut Hill, MA 02467. Email: james.russell@bc.edu.
2
New Basic Emotions?
Abstract
Much theory, research, and application regarding emotion is based on a set of basic emotions.
But the question remains: which emotions are in that set? One proposal is to expand the classic
set of six with 12 new ones, each indicated by a facial expression purported to convey that one
specific emotion universally. A series of studies offered as support for this proposal relied on
presenting participants with the emotion label embedded in a story and then asking them to
choose among four facial expressions or none. Here we critique that response procedure (used in
various studies) as confounding emotion with story. Our Study 1 (N = 1230 residents of the
United States) found that the same response procedure could “show” that the facial expressions
used in that previous research convey emotions other than the ones that had been proposed. Our
Study 2 (N = 64 in India and N = 56 in China) found similar results with participants who speak
non-Indo-European languages (Malayalam and Mandarin). Altogether, our results question
whether the proposed set of new basic emotions is warranted given problems in the response
procedure in which an emotion is embedded in a story.
Keywords: basic emotions; facial expression; limited-choice-from-array; culture
3
New Basic Emotions?
On Evidence for a Dozen New Basic Emotions:
A Methodological Critique
In their report on “recent advances in basic emotion theory [p. 57],” Keltner and Cordaro
(2017) summarized their own extensive research program purported to find at least 18 emotions
each with a specific universal facial signal (Cordaro, Keltner, Tshering, Wangchuk, & Flynn,
2016; Cordaro et al., 2019; Keltner & Cordaro, 2016; 2017; Keltner, Tracy, Sauter, Cordaro, &
2016; Keltner, Sauter, Tracy, & Cowen, 2019). The research program offers one way to answer a
question fundamental to the basic-emotion approach to the science of emotion: what are the basic
emotions? The results summarized by Keltner and Cordaro purported to identify an expanded set
of discrete emotions each of which has a universal facial signal. This expanded version of basic
emotion theory has already inspired important research, theory, and application (Adolphs &
Anderson, 2018; Ekman, 2017; Levenson, 2011; Scarantino, 2015).
Here we critique the response procedure used in this research program. Respondents are
given an emotion term embedded in a story and asked to choose among a limited set of facial
expressions. A similar response procedure has been used before (Camras & Allison, 1985;
Dashiell, 1927; Ekman & Cordaro, 2011; Ekman, Sorenson, & Friesen, 1969; Gendron,
Roberson, van der Vyver, & Barrett, 2014; Simon-Thomas, Keltner, Sauter, Sinicropi-Yao, &
Abramson, 2009). In addition, a similar response procedure was used in other studies on basic
emotions, but instead of choosing among facial expressions, respondents were asked to choose
among recorded vocalizations (Cordaro et al., 2016; Sauter, Eisner, Ekman, & Scott, 2010). In
such studies, the criterion for an emotion to be considered “basic” was the unique and universal
4
New Basic Emotions?
association between that emotion and a specific emotional signal. Implicitly, they sought a one-
to-one correspondence. Thus, each of the facial expressions, for example, is named by the one
emotion it is supposed to convey, as the “facial expression of disgust” is hypothesized to convey
disgust and nothing but disgust – or, in the study by Cordaro et al. (2019), the “facial expression
of sympathy” is hypothesized to convey sympathy and nothing but sympathy.
To examine this response procedure, we replicated and extended the recent study by
Cordaro et al. (2019). They reported evidence from naïve observers in nine Western and Eastern
industrialized societies: China, Germany, India, Japan, Pakistan, Poland, South Korea, Turkey,
and the United States. Their key finding was that their proposed 12 “new” basic emotions
(boredom, confusion, embarrassment, pain, shame, amusement, contentment, coyness, desire,
interest, pride, and sympathy) each had a facial expression as recognizable as had the six
traditional “classic” basic emotions (happiness, surprise, fear, anger, disgust, and sadness)
proposed by Ekman (1972). The mean percentage of participants who agreed with their
prediction was 82% for their new emotions and 75% for the classic six; these two figures are not
statistically different. Cordaro and colleagues interpreted this result as an indication that the
predicted facial expression was the signal for its corresponding emotion.
The specific response procedure used in the Cordaro et al. (2019) study can be termed a
limited-choice-from-array format. Respondents read an easily understood short story, which
included an emotion term, and then were asked to select the facial expression of the story
protagonist from an array of four expressions, or to choose ‘none of these’. The four faces
provided included the predicted face and three foils, all of similar valence.
Cordaro et al.’s (2019) specific procedure was designed to eliminate some problems
previously noted with this type of response procedure. One such arises when options are of
5
New Basic Emotions?
different valences: if foils were of both positive and negative valence, participants can use a
foil’s valence to eliminate it from consideration (thereby inflating the likelihood of selecting the
predicted option by chance among remaining options). To eliminate this problem, Cordaro et al
made all options of the same valence. Another problem is that even if none of the options were
adequate, participants were forced to choose one anyway. To eliminate this problem, Cordaro et
al. followed Frank and Stennett’s (2001) advice to add “none of the above” to the list of options.
Even with the refinements implemented by Cordaro et al. (2019), other problems remain
with this response procedure. One problem is that respondents cannot indicate their spontaneous
interpretation of the facial expression. When Kollareth et al. (2020) gathered data on
respondents’ spontaneous interpretation (as indicated by a free-labeling response format) of
facial expressions used in Cordaro et al.’s study, results failed to converge with theirs. This result
prompted us to examine the limited-choice-from-array response procedure in further research
with new samples of participants and a different design.
Here we focus on just one potentially powerful problematic design feature of the limited-
choice-from-array procedure: the confounding of emotion with a specific situation described in a
story. Because of this problem, results are more precisely described as showing an association
between a facial expression and its predicted emotion when embedded in a specific situation
(described in a story). In other words, participants are associating the facial expression with an
emotion+story. And it might be the story that determines which facial expression is chosen. This
possibility is reinforced by evidence that context influences what emotion is “recognized” from a
facial expression (Carroll & Russell, 1996; Widen, Pochedly, & Russell, 2015). Consider
Cordaro et al.’s (2019) story for amusement: “His friend just told him a very funny story, and he
feels very amused by it.” We do not know if the respondent thought the chosen facial expression
6
New Basic Emotions?
occurs with a genuine feeling (amusement) or as a social gesture acknowledging the attempt at
humor (whether successful or not). Or perhaps the respondent thought that the facial expression
occurs only when both conditions are fulfilled. Similarly, consider their story for coyness: “He is
flirting shyly with someone across the room, because he is feeling coy.” We do not know if the
respondent thought that the chosen facial expression occurs with a genuine feeling (coyness), or
with an instrumental social act of flirting (whatever the internal feeling), or if both are required.
Because of the confounding of emotion with situation, we hypothesize that, with the
limited-choice-from-array format, the same facial expression would be associated with different
emotions when embedded in different situations and that the same emotion in different situations
would be associated with different facial expressions. If, with this format, different emotions in
different situations appear “recognizable” from one facial expression, and vice versa, then those
emotions would fail to meet Cordaro et al.’s (2019) one-to-one criterion for signaling a specific
discrete emotion.
In our Study 1, we tested the hypothesis that, with the response procedure used in
Cordaro et al. (2019), the specific facial expression they proposed as a unique signal for a
specific emotion can be “shown” to signal a different emotion than the one they proposed. We
therefore developed a series of predictions (to be described shortly) about what that different
emotion would be.
In our Study 2, we tested the same hypothesis with respondents in India and China.
Keltner, Cordaro, and colleague’s criterion for concluding that an emotion is to be considered
basic was that that emotion is recognized from one specific facial expression panculturally. Our
Study 2 tested our hypothesis with participants who speak non-Indo-European languages,
7
New Basic Emotions?
Malayalam and Mandarin, respectively. The method was changed slightly for practical reasons
and to ensure robustness of conclusion across small changes.
The two studies were approved by the Boston College Institutional Review Board,
Protocol Number: 17.001.01. For data from each of the studies, see https://osf.io/hs2fv/?
view_only=991e317cc44b4d388dc390a5b4535fcc (Kollareth, 2020).
Study 1A and 1B: Participants in the United States
Study 1 tested the hypothesis that, with the response procedure used by Cordaro et al.
(2019), their facial expressions each allegedly of a specific emotion would indicate an emotion
other than the one they predicted. The alternative emotion was a classic emotion in Study 1A,
and a new emotion not predicted by Cordaro et al. in Study 1B. Both in Study 1A and 1B, there
were two conditions: A Replication condition and an Alternative condition. In the Replication
condition, we replicated Cordaro et al.’s results with their facial expressions, emotions, and
stories. In the Alternative condition, we paired the same facial expression used in the Replication
condition with our alternative emotion embedded in an alternative story. Because we were
hypothesizing emotion-face pairings other than the standard well-studied ones, we had little prior
research to draw on. Instead, the authors attempted to anticipate what alternative emotion-face
pairings might receive empirical support. Clearly, our specific alternative pairings were
preliminary but, if successful, would effectively demonstrate the ambiguous nature of the facial
expressions in and of themselves.
For each pair of Replication and Alternative conditions (i.e., two different emotion+story
stimuli), the response array -- four facial expressions (target plus three foils) and “none of these”
-- stayed the same. Thus, the facial expression defined as the target in the Cordaro et al.’s (2019)
study remained and was presented along with the same three foils. What varied across the two
8
New Basic Emotions?
conditions was the emotion and the story in which it was embedded. If our prediction is borne
out – if participants choose the same face as identified by Cordaro et al., given our different
emotion+story combination, then the same facial expression would, by the logic of the limited-
choice-from-array response procedure, have been shown to be a signal for two different
emotions.
Studies 1A and 1B were conducted concurrently and independently. The design of Study
1A was 2 (Replication vs Alternative emotion) X 8 (Faces) = 16 cells, each of which contained a
different set of participants. The design of Study 1B was 2 (Replication vs Alternative emotion)
X 12 (Faces) = 24 cells. Each participant was randomly assigned to one of the 16 + 24 = 40 cells
in a fully between-subjects design.
Method
In Study 1A and 1B, the same facial expression (along with 3 foils and “none of these”)
appeared in two conditions: In the Replication condition, we used the emotion+story used by
Cordaro et al. (2019); in the Alternative condition, we used the same predicted facial expression
as in the Replication condition (and the same foils) but now preceded by our newly created
alternative emotion+story. In both conditions the same facial expression was the predicted
choice. And, in both conditions, we followed the Cordaro et al.’s method, in which they
embedded their emotion term within a brief story and asked participants to pick the face of the
protagonist from a set of four facial expressions, plus “none of these.” We did not know the set of
facial foils Cordaro et al. used in their trials, and therefore we made our own sets with
photographs used by Cordaro et al., ensuring that the valence of all foils was the same. The
design for Study 1A included 16 between-subject cells (2 Replication vs Alternative X 8 target
9
New Basic Emotions?
facial expressions); the design for Study 1B included 24 between-subject cells (2 Replication vs
Alternative X 12 target facial expressions).
Participants. Participants were 1230 adults over the age of 18 who identified as native
English speakers (M = 35.75 years, SD = 11.5 years; 606 male; 617 female; 7 unspecified sex).
Number of participants within each of the 40 between-subject cells combined across Study 1A
and 1B ranged from 25-40. All participants were recruited from Amazon Mechanical Turk and
awarded $0.20 for their participation. Inclusion criteria also required current residency in the
United States and speaking English as a native language. Participants identified themselves as
Caucasian (70.9%), Asian (10.9%), Hispanic (7.1%), African American (6.7%), Other (2.3%),
American Indian (1.7%), and unspecified race (0.4%). Participants were screened by Amazon
Mechanical Turk such that only residents of the United States could take the survey. Twelve
participants’ responses had been excluded as they failed to provide a Mechanical Turk generated
code at the end of the survey.
To determine appropriate sample sizes we relied on the detailed discussion of power
presented by Cordaro et al. (2019) and their conclusion based on findings of emotions “reliably
identified at well above chance levels.” On each trial, a participant chose among 4 faces and
“none of these,” yielding a chance performance of 20%. To reject the null hypothesis of 20%
chance responding with a power of approximately .8, a sample size of 25 participants responding
to each trial is adequate. We therefore set 25 as the minimum sample size for each trial. We set
the criterion of success as reliably above 20%; Cordaro et al. also mentioned 58% as a criterion
for success, but found that 23.3% of their trials failed to meet this criterion.
Facial expressions. All facial expressions used here either as target or as a foil were
those used by Cordaro et al. (2019, Table 1), who provided the photographs, which are also
10
New Basic Emotions?
provided by Keltner et al. (2019) and published online at
http://www.neuroslam.com/emotionwise.
Stories. Stories for the Replication condition were taken directly from Cordaro et al.
(2019). Stories for the Alternative condition for Study 1A are given in Table 1, for Study 1B in
Table 2.
Procedure. After giving informed consent for participation, participants were provided
with the following instructions: “On the following pages you are going to see four faces
accompanied by a short story. Please read the story, then select the facial expression that best
portrays how the protagonist is feeling (ex: Person A, B, C, etc.). Note that 'NONE OF THESE'
is an acceptable answer.” Participants then read a specific story, followed by “Please select the
facial expression that best portrays how Person [ ] is feeling.” Participants were presented with
four facial expressions in addition to ‘NONE OF THESE’. After the facial expression task,
participants answered demographic questions about their sex, age, most fluent language,
education, and ethnicity. Participants were also encouraged to give feedback and identify any
problems in taking the survey.
Results and Discussion
Matching scores, the proportion of participants who chose the predicted face, are shown
separately for each face in Table 1 for Study 1A and in Table 2 for Study 1B. Also listed are 95%
confidence intervals for each face based on N = 1000 bootstrap samples. With 4 faces and the
“none of these” response option, we chose .20 as our benchmark for random responding. The
matching proportion is significantly different from chance (p < .05, two-tailed) if the lower
boundary of the 95% confidence interval is higher than .20. All significant results were
11
New Basic Emotions?
confirmed by sign test using exact probabilities. These results are collapsed across participant
gender. See the supplement for analyses of gender effects.
Results from the Replication condition largely replicated the results reported by Cordaro
et al. (2019) with participants in the United States, for which Cordaro et al. reported matching
scores that ranged from .56 to .95. Results in our Replication condition, across Studies 1A and
1B, ranged from .15 to .97. The range was broader, but most results were similar to theirs. The
one unusually low matching score of .15 occurred with the “pride” face and did not differ from
chance of .20. With this one result set aside, the range was .50 to .97 for these 19 of 20 trials, in
which a majority of respondents chose the predicted face. Because our foils likely at least
occasionally differed from theirs, our results clearly replicated theirs.
The results of the eight trials in the Alternative condition of Study 1A (Table 1) supported
our hypothesis that the faces used in the Cordaro et al. (2019) study signal classic emotions by
the same criterion Cordaro et al. used to conclude that they each signal a new basic emotion. For
all 8 trials, matching scores ranged from .48 to .94, all of which were significantly different from
chance. In each case, the modal response was the facial expression we had predicted. Not only
did the 8 faces have a matching score that met the criterion of being a signal of a classic emotion,
6 of the 8 achieved a matching score higher than that achieved as a signal for the new emotion in
the Replication condition. In sum, facial expressions each purported to convey a new (i.e., non-
classic) emotion were found to convey a classic emotion by the same criterion.
(Insert Table 1)
The results of the 12 trials in the Alternative condition of Study 1B (Table 2) supported
our hypothesis that the faces used in the Cordaro et al. (2019) study signal non-classic emotions
not predicted by their theory. Matching scores ranged from .52 to .89, all significantly different
12
New Basic Emotions?
from chance. In each case, the modal response was the facial expression we had predicted, and in
three cases, matching scores in the Alternative condition were greater than those in the
Replication condition. In sum, Cordaro et al.’s facial expressions were each found to convey a
new emotion not predicted by their theory.
(Insert Table 2)
In summary, the limited-choice-from-array procedure used in Cordaro et al.’s (2019)
study produced results here that challenge the validity of that procedure as one providing support
for a unique connection between one emotion and one facial expression. The same facial
expression chosen for one emotion in the Replication condition was chosen for a different
emotion in the Alternative condition. With their method, we found that a given facial expression
appears to signal two or more different emotions depending on the story used in the procedure:
For example, Cordaro et al.’s “facial expression of embarrassment” was “shown” to signal
embarrassment, happiness, amusement, and coyness. The same face can appear to signal the new
basic emotion proposed by Cordaro et al., a classic basic emotion, a different Cordaro et al.
“new” basic emotion, or even another emotion not in their theory. Even one instance of the
method producing such results would suffice to raise questions about the method. Here, 20 out of
20 of instances examined in the Alternative condition produced such results.
Study 2: Cross-cultural Replication in India and China
A strength of the research program carried out by Keltner, Cordaro, and colleagues (e.g.,
Cordaro et al., 2016; Keltner & Cordaro, 2016; 2017; Keltner et al., 2016; Keltner et al., 2019)
and of the Cordaro et al. (2019) study specifically is its cross-cultural perspective. In addition,
extensive evidence has shown that facial expressions occur in dialects (Elfenbein, Beaupre,
Lévesque, & Hess, 2007). Other research has raised even more sweeping doubts about the claim
13
New Basic Emotions?
of cross-cultural universality of facial signals of specific emotions (Crivelli, Russell, Jarillo, &
Fernández-Dols, 2016; Jack, Garrod, Yu, Caldara, & Schyns, 2012). In our Study 2, we tested
our hypothesis in two societies, India and China. Cordaro et al. had replicated their results in,
among others, India and China. We omitted the Replication condition for we assume that their
specific results would replicate in these societies, just as they did in the United States in our
Study 1.
Here our aim was to replicate the Alternative condition results of Study 1, although not
with identical stories. Instead, we revised stories to be more suitable to the society studied. In
finding alternative face-emotion associations, we had little to go on. Our study is therefore a
preliminary attempt to find situations in which each of the facial expressions might occur and
what alternative emotion might accompany it in India and in China. We also did not know the set
of foils Cordaro et al. (2019) used, and therefore we formed our own sets: for both targets and
foils, we again used the photographs provided by Cordaro et al and ensured that the valence of
predicted face and all foils was the same. We studied all 12 new emotions proposed by Cordaro
et al. (amusement, boredom, confusion, contentment, coyness, desire, embarrassment, interest,
pain, pride, shame, and sympathy). We included each face claimed to correspond to one of the
proposed new basic emotions; thus 12 trials in total. For each face, we predicted that it would be
selected as matching an alternative emotion+situation, that is, a different emotion from that used
by Cordaro et al. For each of our alternative emotions, we created an accompanying situation
story.
In describing our study in India and China and its results, we use terms in English for the
convenience of the reader, but the actual studies were carried out entirely in Malayalam or
Mandarin, respectively. For each sample, the study consisted of 12 trials. In some, we used an
14
New Basic Emotions?
emotion term (albeit predicted for a different face) drawn from Cordaro et al. (2019): in India,
embarrassed, sympathy, pride, amusement, bored, anger; and in China, pride, embarrassed,
surprised, proud, bored. In the other of the 12 trials, we used other emotions: in India,
depression, excited, determined, effort, disappointment, uncomfortable; and in China, distressed,
impatient, confident, shy, effort, disappointment, anxious.
Method
The method is the same as in Study 1 except as noted. In contrast to the between-subject
design in Study 1A and 1B, participants in both the Chinese and the Malayalam samples
responded to all 12 trials. That is, as in Cordaro et al. (2019), the design was within-subject.
Although the within-subject design can exaggerate amount of agreement, we wanted to mirror
Cordaro et al.’s method as much as practical.
Participants. Participants in India were 64 undergraduate students from St. Xavier’s
College in Thumba, India (M = 20.08 years, SD = 1.35 years; 26 male; 38 female). Participants
in China were 56 undergraduate students living in Shanghai (M = 20.14 years, SD = 1.02 years;
28 male; 28 female). The same power analysis used in Study 1 applies here and indicated a
minimum of 25 per trial. Beyond this, to ensure power equal to that in the Keltner-Cordaro
project, we aimed at recruiting more participants than the number Cordaro et al. (2019) had
recruited: their numbers were n = 44 in India and n = 54 in China.
Stories. For stories used in India, see Table 3 and in China, Table 4.
Translation. The emotion words in English were given to six English-Malayalam
bilingual speakers and six English-Mandarin bilingual speakers, who provided the best
translation for each of these words. Authors of this study included a native Malayalam speaker
and a native Mandarin speaker, who consulted bilingual dictionaries in respective languages and
15
New Basic Emotions?
found the most appropriate translation for these words (Ho, 2006; Pillai, 2002). Those words
showing the greatest agreement between the dictionaries and among the bilingual speakers were
chosen to be the translations. Translations for the emotion words in Malayalam in India:
depression - vishadam, excited - avesham, determined - nischayam, effort - parishramam,
disappointment - nirasa, uncomfortable – asugakaram, embarrassed – chammal, sympathy –
sahathapam, pride – ahamkaram, amusement – vinodam, bored – mushippu, anger – deshyam;
and in Mandarin in China: distressed – ku-nao, impatient – bu-nai-fan, confident – bu-xie-yi-gu,
shy – hai-xiu, effort – nu-li, disappointed – shi-wang, anxious – bu-an, surprised - bu-ke-si-yi,
embarrassed - gan-ga, pride/proud - de-yi, bored – wu-liao. Researchers who are also native
language speakers did the translations for the stories and instructions.
Procedure. The study was conducted face-to-face with an experimenter, a native speaker
of Malayalam or Mandarin. The experimenter provided a laptop and assisted participants. The
data were then collected on-line, using a different random order of presentation for each
participant.
Results and Discussion
Results for the Indian sample are shown in Table 3, which shows matching scores, along
with the 95% confidence intervals based on N = 1000 bootstrap samples. Data for each face were
analyzed separately to maintain comparability to results for Studies 1A and 1B. Thus, for all
analyses, each participant contributed only a single data point. As in Study 1A and 1B, we used .
20 as the benchmark for random responding, and we present analyses of gender effects in the
supplement.
Matching scores for the Indian sample are shown in Table 3. For their new emotions,
Cordaro et al. (2019) reported matching scores that ranged from .15 to .87 in India. The matching
16
New Basic Emotions?
scores in our sample showed a similar but somewhat smaller range: .16 to .75. In 8 of the 12
trials, the predicted face was selected by half or more of participants. In 11 of the 12 trials, our
predicted face was modal; in those 11 trials, our predicted face was selected above chance level.
(Insert Table 3)
Matching scores for the Chinese sample are shown in Table 4. For their new emotions,
Cordaro et al. (2019) reported matching scores that ranged from .32 to .99 in China. The
matching scores in our sample showed a similar but a lower range: .09 to .82. In 5 of the 12
trials, our predicted face was selected by half or more of participants. In 8 of the 12 trails, our
predicted face was modal. In 10 of the 12 trails, our predicted face was selected above chance
level.
(Insert Table 4)
An emotion specified by Cordaro et al. (2019) for one face was used here for another
face, challenging the view that the facial expression they offered uniquely conveys one emotion.
In some of our 12 emotion+story stimuli, we used an emotion term drawn from Cordaro et al.
However, participants selected another face for these emotions: Indian participants selected the
“amusement” face for embarrassed protagonist, “confusion” face for sympathetic protagonist,
“contentment” face for prideful protagonist, “embarrassment” face for amused protagonist,
“interest” face for bored protagonist, and “pride” face for angry protagonist. Chinese participants
selected “contentment” face for embarrassed protagonist, “interest” face for surprised
protagonist, and “amusement” face and “coy” face for prideful or proud protagonist.
A comparison across the two cultural groups further indicates that a face may be
associated with different emotions in different cultures. In 8 of our 12 trials, our Indian and
Chinese participants selected the face for our newly developed different emotions. For example,
17
New Basic Emotions?
consider the “amusement face” shown by Cordaro et al. (2019) to be selected as the expression
of amusement by participants in the United States. The “amusement” face was selected for an
embarrassed protagonist in the Indian sample, but for a prideful protagonist in the Chinese
sample. Of course, in each of the trials we used a different story and different embedded
emotion between the cultural groups (except in two cases where the “pain” face was given for
effortful protagonist and “shame” face for disappointed protagonist).
These results replicated the general message of Study 1 with somewhat different stories:
that is, Cordaro et al.’s (2019) findings are dependent on the response procedure they used. With
their method, their faces were associated in our study with emotions not predicted by their
theory. Even one instance of their method producing such results would suffice to raise
questions about the method when used in cross-cultural research. Here, the large majority of
instances produced such results.
General Discussion and Conclusion
To provide evidence for 12 new basic emotions in addition to the classic six, Cordaro et
al. (2019) used the limited-choice-from-array response format in which emotion and situation are
confounded. Our two studies challenge their response procedure and therefore conclusions based
on that procedure. Our results showed that there was no one-to-one correspondence between a
new facial expression and an emotion embedded in a story. When that procedure was used, the
new facial expressions proposed by Cordaro et al. were chosen for different emotions (embedded
in stories). This finding recurred in all three samples (from the United States, India, and China),
and in a majority of alternative trials (in 20 of 20 trials in the United States, in 11 of 12 trials in
India, and in 8 of 12 trials in China; so, in 39 of 44 trials altogether). The facial expressions they
18
New Basic Emotions?
proposed do not each signal only one emotion – if the results from their response procedure are
to be believed.
The limited-choice-from-array method can be used to demonstrate that the same facial
expression signals different emotions, either those proposed by Cordaro et al. (2019) or still
others. Study 1A’s Alternative condition showed that facial expressions claimed by Cordaro et al.
to signal one of their new emotions instead met their own criterion for signaling a classic Ekman
(1972) emotion. Study 1B’s Alternative condition showed that facial expressions claimed by
Cordaro et al. to signal one new emotion instead met their own criterion for signaling a different
new emotion or one not on their list. Study 2 extended these findings cross-culturally. By the
limited-choice-from-array method, the same face could be established as conveying two or three
different emotions, and one emotion as conveyed by two or three different faces.
Any conclusion must be evaluated in light of how convincing was the method used to
gather the data. Earlier, we articulated our doubts about the limited-choice-from-array response
format. The two studies reported here reinforced those doubts. Other reinforcement can be seen
in the failure of an alternative response format – free labeling – to replicate Cordaro et al.’s
(2019) results (Kollareth et al., 2020). Either of these findings, even one instance of such a
finding, would suffice to raise questions, but the large number of such findings makes a
compelling case. Doubts are also raised by research more generally, results of which question
claims of universal signals of emotion (Crivelli et al., 2016; Jack et al. 2012).
Future studies are required to examine the limitations of the present ones. The studies
reported here relied on facial expressions available through and previously studied by Cordaro et
al. (2019). Other facial stimuli, especially more dynamic portrayals, could yield other results.
Our participant samples were limited to three societies, all with extensive contact with each other
19
New Basic Emotions?
and with other societies. Tests of universality require more cross-cultural research, especially
with societies more isolated from the rest of the world. Such future research needs a careful look
at the response procedure to be used.
Finally, it would be a mistake to infer from the present results that the various facial
expressions examined here do, in fact, signal the two or three emotions we predicted. The
problems with the limited-choice-from-array format do not warrant that conclusion. Perhaps the
situation depicted in the accompanying story alone accounted for the choice of a facial
expressions. Although research could be carried out to pit emotion against story in accounting for
choice of facial expression, we question of the value of pursuing a response procedure that
inherently confounds emotion with story.
20
New Basic Emotions?
References
Adolphs, R. & Anderson, D. J. (2018). The neuroscience of emotion: A new synthesis. New
Jersey: Princeton University Press.
Camras, L. A., & Allison, K. (1985). Children's understanding of emotional facial expressions
and verbal labels. Journal of Nonverbal Behavior, 9(2), 84-94.
Carroll, J. M., & Russell, J. A. (1996). Do facial expressions signal specific emotions? Judging
emotion from the face in context. Journal of Personality and Social Psychology, 70(2),
205-218.
Cordaro, D. T., Keltner, D., Tshering, S., Wangchuk, D., & Flynn, L. M. (2016). The voice
conveys emotion in ten globalized cultures and one remote village in
Bhutan. Emotion, 16(1), 117-128.
Cordaro, D. T., Sun, R. Kamble, S., Hodder, N., Monroy, M., Cowen, A., Bai, Y., & Keltner, D.
(2019). The recognition of 18 facial-bodily expressions across nine cultures. Emotion, 19,
1528-1537.
Advance online publication. http://dx.doi.org/10.1037/emo0000576.
Crivelli, C., Russell, J. A., Jarillo, S., & Fernández-Dols, J. M. (2016). The fear gasping face as a
threat display in a Melanesian society. Proceedings of the National Academy of
Sciences, 113(44), 12403-12407.
Dashiell, J. F. (1927). A new method of measuring reactions to facial expression of
emotion. Psychological Bulletin, 24, 174-175.
Ekman, P. (1972). Universal and cultural differences in facial expressions of emotions. In J. K.
Cole (Ed.), Nebraska symposium on motivation, 1971 (pp. 207-283). Lincoln: University
of Nebraska Press.
21
New Basic Emotions?
Ekman, P. (2017). Facial expressions. In J. M. Fernandez Dols & J. A. Russell (eds.). The
science of facial expression (pp. 39-56). New York, NY: Oxford University Press.
Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion
Review, 3(4), 364-370.
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in the facial display
of emotions. Science, 164, 86-88.
Elfenbein, H. A., Beaupré, M., Lévesque, M., & Hess, U. (2007). Toward a dialect theory:
cultural differences in the expression and recognition of posed facial
expressions. Emotion, 7(1), 131-146.
Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Perceptions of emotion
from facial expressions are not culturally universal: evidence from a remote
culture. Emotion, 14(2), 251-262.
Frank, M. G., & Stennett, J. (2001). The forced-choice paradigm and the perception of facial
expressions of emotion. Journal of Personality and Social Psychology, 80(1), 75-85.
HZ, Y. (2006). Chinese-English English-Chinese (mandarin) dictionary & phrasebook. New
York: Hippocrene Books.
Jack, R. E., Garrod, O. G., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of
emotion are not culturally universal. Proceedings of the National Academy of
Sciences, 109(19), 7241-7244.
Keltner, D. & Cordaro, D. T. (2016). Understanding multimodal emotional expressions: Recent
advances in basic emotion theory. Emotion Researcher, ISRE’s Sourcebook for Research
on Emotion and Affect, Andrea Scarantino (ed.),
http://emotionresearcher.com/understanding-multimodal-emotional-expressions-recent-
22
New Basic Emotions?
advances-in-basic-emotion-theory/, accessed 12 October 2016.
Keltner, D. & Cordaro, D. T. (2017). Understanding multimodal emotional expressions: Recent
advances in basic emotion theory. In J. M. Fernandez Dols & J. A. Russell (Eds.), The
science of facial expression (pp. 57-76). Oxford, UK: Oxford University Press.
Keltner, D., Sauter, D., Tracy, J. L., & Cowen, A. (2019). Emotional expression: Advances in
basic emotion theory. Journal of Nonverbal Behavior. https://doi.org/10.1007/s10919-
019-00293-3.
Keltner, D., Tracy, J., Sauter, D. A., Cordaro, D. C., & McNeil, G. (2016). Expression of
emotion. In L. F. Barrett, M. Lewis, & J. M. Haviland-Jones. (Eds.), Handbook of
emotions, 4th Ed. (Pps. 467-482). New York: Guilford.
Kollareth, D. (2020, May 24). Method critique. Retrieved from osf.io/hs2fv
Kollareth, D., Esposito, J., Ma, Y., Sangster, K. M., Brownell, H., & Russell, J.A. (2020). On
Twelve Proposed New Basic Emotions: A Failure of Convergent Validity. Manuscript
Submitted for Publication.
Levenson, R. W. (2011). Basic emotion questions. Emotion Review, 3(4), 379-386.
Pillai, T. Ramalingam (2002). In English-English Malayalam dictionary (6thed.). Kottayam: DC
Books.
Russell, J. A. (1993). Forced-choice response format in the study of facial expression.
Motivation and Emotion, 17, 41-51.
Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010). Cross-cultural recognition of basic
emotions through nonverbal emotional vocalizations. Proceedings of the National
Academy of Sciences, 107(6), 2408–2412.
Scarantino, A. (2015). Basic emotions, psychological construction, and the problem of
23
New Basic Emotions?
variability. In L. F. Barrett & J. A. Russell (Eds.), The psychological construction of
emotion (pp. 334-376). New York: Guilford.
Simon-Thomas, E. R., Keltner, D. J., Sauter, D., Sinicropi-Yao, L., & Abramson, A. (2009). The
voice conveys specific emotions: Evidence from vocal burst displays. Emotion, 9(6), 838-
846.
Widen, S. C., Pochedly, J. T., & Russell, J. A. (2015). The development of emotion concepts: A
story superiority effect in older children and adolescents. Journal of Experimental Child
Psychology, 131, 186-192.
24
New Basic Emotions?
Table 1
The United States: Proportion of Participants Who Chose the Predicted Face in a Replication Condition and an Alternative
Condition, Study 1A
Replication Condition:
Original Cordaro et al.
(2019) story+emotion
Alternative Condition: Alternative story+emotion
Cordaro et al.
(2019) name for
the predicted
face
nProportion
choosing
predicted face
(95% CI)
Alternative
emotion
Alternative story+emotion nProportion
choosing
predicted face
(95% CI)
Pride 34 .15 (.04, .26) Anger Person K is confronting a problematic
roommate who has been stealing
many of Person K’s items. Person K
feels angry.
30 .80 (.64, .93)
Amusement 36 .92 (.81, 1.00) Happiness Person S is with a group of friends,
talking and enjoying themselves.
Person S is very happy.
33 .94 (.85, 1.00)
Coyness 30 .70 (.52, .86) Happiness Person T is with a group of friends,
talking and enjoying themselves.
30 .93 (.82, 1.00)
25
New Basic Emotions?
Person T is very happy.
Embarrassment 31 .55 (.36, .73) Happiness Person U is thinking about a recent
date with a romantic partner. Person U
feels happy.
31 .48 (.30, .66)
Interest 35 .60 (.44, .76) Surprise Person V has just been told a secret by
a friend. Person V is surprised.
31 .77 (.62, .91)
Pain 31 .97 (.89, 1.00) Anger Person W has just heard that a cousin
borrowed the family car and caused
extensive damage. Person W is very
angry.
33 .82 (.68, .94)
Shame 32 .75 (.60, .89) Sadness Person X’s parents both passed away
over the past month. Person X feels
very sad.
36 .89 (.78, .98)
Sympathy 39 .89 (.80, .98) Sadness Person Y is attending college, but has
failed all the courses this semester.
Person Y is very sad.
30 .90 (.78, 1.00)
Note. Participants chose among four faces (the predicted face plus 3 foils all of the same valence) plus “none of these”, which yielded
a random response probability of .20. 95% Confidence Intervals (CI) are based on N=1000 bootstrap samples. Bolded proportions
and Confidence Intervals indicate that the lower bound of the 95% confidence interval excludes –is higher than-- chance level (.20),
26
New Basic Emotions?
the result is statistically significant at the .05 level, two tailed. In the Replication condition, the story+emotion was that used by
Cordaro et al. (2019); in the Alternative condition, the story+emotion was created anew for the present study.
27
New Basic Emotions?
Table 2
The United States: Proportion of Participants Who Chose the Predicted Face in a Replication Condition and an Alternative
Condition, Study 1B
Replication Condition:
Original Cordaro et al.
(2019) story+emotion
Alternative Condition: Alternative story+emotion
Cordaro et al.
(2019) name for
the predicted
face
nProportion
choosing
predicted face
(95% CI)
Alternative
emotion
Alternative story+emotion nProportion
choosing
predicted face
(95% CI)
Amusement 35 .83 (.70, .94) Pride Person A is telling the news to a
highly competitive schoolmate whom
Person A has known for a very long
time. Person A was accepted into a
prestigious law school, while the rival
was rejected. Person A feels very
proud.
28 .54 (.34, .73)
Confusion 33 .85 (.71, .97) Sympathy Person C is listening to a close friend 27 .63 (.44, .81)
28
New Basic Emotions?
mourn over a loved one’s death.
Person C feels sympathy for the
friend.
Contentment 26 .96 (.86, 1.00) Pride Person D has received an award for
“outstanding scholar of the year” from
faculty. Person D is feeling deeply
proud.
26 .65 (.47, .84)
Embarrassment 26 .81 (.65, .95) Amusement Person G has just overheard an
inappropriate but funny comment.
Person G feels amused.
28 .82 (.67, .96)
Embarrassment 27 .52 (.32, .71) Coy Person O has a crush on someone and
has just received a flattering
compliment from that person. Person
O couldn’t help but act coy.
25 .72 (.54, .89)
Boredom 33 .88 (76, .97) Depression Person B has received only letters of
rejection from all of Person B’s top-
choice law schools after years of hard
work. Person B feels extremely
depressed.
27 .74 (.57, .90)
29
New Basic Emotions?
Coyness 29 .69 (.53, .85) Excitement Person E is meeting up with old
friends this weekend, after not seeing
them for a while. Person E is very
excited.
28 .57 (.39, .75)
Interest 28 .50 (.32,.68) Skepticism Person H has just been told something
by a child, but it seems to be a fib.
Person H feels skeptical.
40 .62 (.47, .76)
Pain 29 .93 (.82, 1.00) Concentration Person J urgently needs his car keys
and is trying to remember where they
are. Person J is concentrating and is
feeling focused.
27 .52 (.33, .71)
Pain 28 .89 (.78, 1.00) Effort Person Q is trying to open a jar-lid
that won’t open. Person Q is straining
to get it open to impress a friend.
Person Q is putting in a lot of effort.
27 .89 (.75, 1.00)
Sympathy 36 .94 (.85, 1.00) Discomfort Person M has just gotten on to an
extremely crowded train, with barely
any room to stand. Person M feels
quite uncomfortable.
35 .89 (.78, .97)
30
New Basic Emotions?
Sympathy 30 .83 (.68, .96) Frustration Person R has been working through a
math problem for an hour and yet
cannot come up with the right answer.
Person R is very frustrated.
30 .83 (.69, .96)
Note. Participants chose among four faces (the predicted face plus 3 foils all of the same valence) plus “none of these”, which yielded
a random response probability of .20. 95% Confidence Intervals (CI) are based on N=1000 bootstrap samples. Bolded proportions
and Confidence Intervals indicate that the lower bound of the 95% confidence interval excludes –is higher than-- chance level (.20),
the result is statistically significant at the .05 level, two tailed. In the Replication condition, the story+emotion was that used by
Cordaro et al. (2019); in the Alternative condition, the story+emotion was created anew for the present study.
31
New Basic Emotions?
Table 3
India: Proportion of Participants Who Chose the Predicted Face for a Different story + New Emotion, Study 2
Cordaro et al.
(2019) name for
the predicted face
New emotion New emotion with new story Proportion choosing
predicted face
(95% CI)
Amusement Embarrassed Someone tells person A that person A is physically very attractive.
Person A is embarrassed.
.41 (.29, .52)
Boredom Depression Person B spent a whole year preparing for medical entrance, but
could not get into any of the reputed medical colleges. Person B
feels very depressed.
.69 (.56, .80)
Confusion Sympathy Person C is listening to a close friend mourn over a loved
one’s death. Person C feels deep sympathy for the friend.
.53 (.41, .66)
Contentment Pride Person D has just got an admission to Study engineering in IIT
Mumbai. Person D feels very proud.
.70 (.59, .81)
Coy Excited Person E is meeting up with old friends this weekend, after not
seeing them for a while. Person E is very excited.
.73 (.62, .84)
Desire Determined Person F is trying to figure out a difficult math problem, but is not
succeeding. Person F is concentrating and feels determined.
.16 (.07, .24)
Embarrassment Amusement Person G has just overheard a funny but not-nice joke. Person G
feels amused.
.75 (.64, .86)
Interest Bored Person H is compelled to sit through a dull class. Person H must .42 (.31, .55)
32
New Basic Emotions?
stay awake but feels bored.
Pain Effort Person I is trying to open a jar-lid that won’t open. Person I is
straining to get it open to impress a friend. Person I is putting in a
lot of effort.
.64 (.52, .75)
Pride Anger Person J is about to confront another person who has been
disturbing Person J’s partner. Person J feels angry.
.58 (.45, .70)
Shame Disappointment Person K has just learned that Person K’s friend’s train got
canceled and the friend can no longer come for a visit. Person K
feels very disappointed.
.72 (.60, .83)
Sympathy Uncomfortable Person L has just gotten on to an extremely crowded train, with
barely any room to stand. Person L feels quite uncomfortable.
.47 (.34, .59)
Note. Participants chose among four faces (the predicted face plus 3 foils all of the same valence) plus “none of these”, which yielded
a random response probability of .20. 95% Confidence Intervals (CI) are based on N = 1000 bootstrap samples. Bolded proportions
and Confidence Intervals indicate that the lower bound of the 95% confidence interval excludes –is higher than-- chance level (.20),
the result is statistically significant at the .05 level, two tailed. Each story+emotion for the new emotions was created anew for the
present study.
33
New Basic Emotions?
Table 4
China: Proportion of Participants Who Chose the Predicted Face for a Different Story + New Emotion, Study 2
Cordaro et al.
(2019) name for
the predicted face
New emotion New emotion with new story Proportion choosing
predicted face
(95% CI)
Amusement Pride Person A gets the first place in one exam. Person A feels proud
about it.
.41 (.29, .53)
Boredom Distressed Person B takes a very difficult Math problem during an exam.
Person B tries hard but still cannot think of an answer. Person B
feels distressed about this problem.
.77 (.66, .87)
Confusion Impatient Person C went back to home after working for a whole day. Person
C found that the five-years-old son haven’t finished homework yet.
Person C feels impatient.
.61 (.48, .73)
Contentment Embarrassed Person D meets an old classmate during the high school reunion.
The old classmate greeted Person D. However, person D cannot
think of the name of this classmate and don’t know how to greet
back. Person D feels embarrassed about this situation.
.34 (.22, .47)
Coy Proud Person E led some friends in successfully tricking the person who
had bullied Person E. Person E feels proud about it.
.34 (.22, .47)
Desire Confident Person F is the king among friends. Person F doesn’t care about .29 (.17, .40)
34
New Basic Emotions?
being challenged by others. Person F feels confident.
Embarrassment Shy During the family reunion in the Chinese New Year, Person G was
praised by parents in front of all the relatives. Person G feels a
little bit shy.
.64 (.52, .77)
Interest Surprised Person H’s friends’ prepared a big birthday surprise. On seeing that
Person H felt very surprised.
.34 (.22, .47)
Pain Effort Person I tried best to lift a heavy barbell in the gym. Person I is
paying much effort.
.82 (.71, .91)
Pride Bored Person J looked on computer screen to do repetitive tasks for the
whole day. Person J felt very bored.
.09 (.02, .17)
Shame Disappointment Person K was told that Person K could not attend the interview for
a top university. Person K felt disappointed.
.70 (.56, .80)
Sympathy Anxious Person L’s college entrance examination will be released in three
minutes. Closer to the time, Person L felt more anxious.
.34 (.21, .46)
Note. Participants chose among four faces (the predicted face plus 3 foils all of the same valence) plus “none of these”, which yielded
a random response probability of .20. 95% Confidence Intervals (CI) are based on N = 1000 bootstrap samples. Bolded proportions
and Confidence Intervals indicate that the lower bound of the 95% confidence interval excludes –is higher than-- chance level (.20),
the result is statistically significant at the .05 level, two tailed. Each story+emotion for the new emotions was created anew for the
present study.
A preview of this full-text is provided by American Psychological Association.
Content available from Emotion
This content is subject to copyright. Terms and conditions apply.