ArticlePDF Available

Affective polarization in a word: Open-ended and self-coded evaluations of partisan affect

PLOS
PLOS One
Authors:

Abstract and Figures

The literature finds that partisanship drives negative emotional evaluations of out-partisans. Yet, scholars base these insights on measures–like thermometers, candidate evaluations, and social-distance measures–that discount the sentiment attached to individuals’ negative attitudes. We introduce a unique measure of affect capturing the motivation underpinning partisans’ attitudes. Our measure asks respondents for one-word to describe voters in their party and the opposing party. Then respondents code the sentiment behind their word choice themselves. Together, our measure produces qualitative and quantitative measures of respondents’ affect. We find that our self-coded open-ended measure has strong face validity and correlates strongly with existing affect measures. This measure advances our understating of partisan affect by allowing scholars a window into respondents’ state of mind. Scholars can easily apply our measure’s procedure beyond partisanship to other groups of interest.
This content is subject to copyright.
RESEARCH ARTICLE
Affective polarization in a word: Open-ended
and self-coded evaluations of partisan affect
Spencer KieselID*
, Sharif Amlani
Department of Political Science, University of California, Davis, Davis, California, United States of America
These authors contributed equally to this work.
*skiesel@ucdavis.edu
Abstract
The literature finds that partisanship drives negative emotional evaluations of out-partisans.
Yet, scholars base these insights on measures–like thermometers, candidate evaluations,
and social-distance measures–that discount the sentiment attached to individuals’ negative
attitudes. We introduce a unique measure of affect capturing the motivation underpinning
partisans’ attitudes. Our measure asks respondents for one-word to describe voters in their
party and the opposing party. Then respondents code the sentiment behind their word
choice themselves. Together, our measure produces qualitative and quantitative measures
of respondents’ affect. We find that our self-coded open-ended measure has strong face
validity and correlates strongly with existing affect measures. This measure advances our
understating of partisan affect by allowing scholars a window into respondents’ state of
mind. Scholars can easily apply our measure’s procedure beyond partisanship to other
groups of interest.
Introduction
Partisanship, as a social identity, has led to increasingly negative evaluations of out-party
members, also known as affective polarization [13]. But, to what extent are these evaluations
negative and what is the sentiment behind them?
To measure the dislike between partisans, scholars rely on thermometer measures, candi-
date evaluations, and social-distance measures of partisans’ willingness to engage with oppo-
nents [2,4]. However, these measures suffer from a common weakness: closed answered scales
[5]. These measures use a numerical scale to quantify respondents’ affect. While this approach
makes quantitative analysis easier, it does not capture the motivations underlying their feel-
ings, making it difficult to understand the mechanisms driving respondents’ affective
evaluations.
Therefore, scholars are cross-pressured: how can we measure what people feel and why they
feel it? We argue that existing measures provide a decent evaluation of respondents’ affect but
do not illuminate the motivations underlying partisans’ evaluations. In this article, we present
a new protocol to measure affect. Our two-question measure starts by asking respondents to
report one word characterizing partisans who share their party identification and one word
PLOS ONE
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 1 / 18
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Kiesel S, Amlani S (2025) Affective
polarization in a word: Open-ended and self-coded
evaluations of partisan affect. PLoS ONE 20(1):
e0310772. https://doi.org/10.1371/journal.
pone.0310772
Editor: Mike Farjam, University of Hamburg:
Universitat Hamburg, GERMANY
Received: January 15, 2024
Accepted: September 5, 2024
Published: January 16, 2025
Copyright: ©2025 Kiesel, Amlani. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: Full Replication code
and data is uploaded with this submission and can
be run in R
Funding: Funding for this research was provided
through the UC Davis Political Science Department.
The funders had no role in study design, data
collection and analysis, decision to publish, or
preparation of the manuscript.
Competing interests: The authors have declared
that no competing interests exist.
characterizing partisans with the opposing party identification. Then, we ask respondents to
code the sentiment behind their own word on a seven-point scale from extremely negative to
extremely positive, with neutral as the midpoint option. This procedure produces a word and
a numeric score both generated by the respondent. The word provides insight into what moti-
vates a respondent’s affect, and the score tells us the intensity of that affect. In combination,
scholars obtain a qualitative and a quantitative evaluation of the respondents’ affect towards in
and out partisans.
Our measure is grounded in Zaller’s Receive-Accept-Sample model of public opinion for-
mation [6]. When prompted, respondents draw from the distribution of their top-of-mind
characterizations of partisans and provide a word that matches the underlying sentiments they
hold. Thus, the words reflect each respondent’s most salient top of mind considerations.
When we evaluate the robustness of our measurement procedure, we find that our one-word
measure has high internal and external validity. First, when we look at respondents’ word
selection and the code they assign to their word, the measure performs as we expect: respon-
dents have positive evaluations of in-partisans and negative evaluations of out-partisans. Sec-
ond, when we compare our measure of affect with established measures of affect, we find the
measures are strongly related but not perfectly related, suggesting that our measure contributes
a unique perspective to our understanding of the latent dimensions of affect.
Our two-question measure provides reliable and valid estimates of respondents’ affect and
has at least four major benefits. First, open-ended responses give researchers a window into
the respondents’ state of mind that closed ended responses do not. Second, our measure more
directly captures respondents’ affect by eliciting spontaneous evaluations open to whatever
natural language occurs to respondents. Third, we overcome barriers associated with open-
ended responses by asking respondents to score their own words. This procedure quantified
open ended responses underlying affective evaluations without the difficulties associated with
matching open-ended responses to sentiment dictionaries. Finally, we show that our measure
is comparable to existing measures but is not perfectly correlated, suggesting that our measure
captures a unique affect of the latent concept of affect. Our measure balances the competing
concern researchers face between easy to use closed ended items and the rich but unwieldy
data provided by open ended questions.
Furthermore, our measure produces findings beyond what purely quantitative measures
can show that underscore the degree to which American politics occurs in a climate where par-
tisans not only disagree with opponents on policy issues but view them as illegitimate or
malevolent. The words respondents use to describe out-partisans—such as “sheep,” “sheeple,”
“misguide,” “uninformed,” “misinform,” “uneducated,” “brainwash”—reflect this feature of
contemporary politics. These demeaning and patronizing characterizations highlight the abyss
of affective polarization between both parties in a much more visceral way than thermometer
averages. They reveal a deeply entrenched in-group versus out-group mentality, signal distrust
and dehumanization of opponents, and reduce complex policy disagreements to the impulsive
vilification of the opposing side. These words highlight that some Americans not only see pol-
icy differences between partisans but think in harsh pejorative terms.
In the following five sections, we discuss how our measure relates to existing methods; pres-
ent the data and methods we used to construct and test our measure; provide evidence of inter-
nal validity; provide evidence of external validity by comparing our estimates to those
produced by well established measures such as thermometers; and argue that the qualitative
data produced by one-word measures can provide novel insights about the dimensions of
affect. Finaly, we conclude by summarizing by summarizing the limitations of our measure
and outlining future work.
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 2 / 18
Measuring affect
Affect refers to a feeling or emotion [7]. When applied to a group in survey research, affect
refers to the feelings or emotions a respondent has towards that group [810]. Scholars study-
ing partisan affect draw on three measures of affect: thermometers, candidate evaluations, and
social-distance measures. The most common tool scholars use to measure partisan affect are
feeling thermometers [1,2,11]. Respondents rate in and out-party on a scale ranging from 0 to
100, as a representation of their feelings toward each group [3,12]. A second measure for parti-
san affect is respondents’ evaluations of major party leaders [1315]. In the United States,
respondents evaluate the Republican and Democratic candidates for president on 10-point
scales of overall favorableness, trustworthiness, recklessness, and to what extent the respondent
shares their values [13]. Abroad, Reiljan (2020) [14] developed the Affective Polarization Index
(API). This index takes survey responses using 0 to 10 like–dislike scale for each party [14] and
their leader [15] and applies a weighted average of the difference between in-party and out-
party like-dislike scores. The index weights this difference by the electoral size of each party. A
third measure of affect is social-distance measures that use social distance questions, such as
those introduced by Iyengar, Sood, and Lelkes (2012) [16] that ask respondents how troubled
they would be by a family member marrying an out-party member.
While scholars use these measures of affect widely and we fully acknowledge their useful-
ness, they are not without their limitations. First, survey researchers did not develop thermom-
eter scales to measure affect [12]. As Wilcox, Sigelman, and Cook (1989, 251) [17] warn, “if
one uses a feeling thermometer to measure affect toward any particular group, one will have to
bear in mind that some respondents respond to feeling thermometers in an unusual manner.
This may pose a particular problem when feeling thermometers are used to identify supporters
of particular social groups.” Feeling thermometers also suffer from “inter-personal incompara-
bility” [18,19]. That is, people tend to interpret feeling thermometer scales differently making
comparing evaluations across individuals tenuous [18,19]. Additionally, respondents tend to
bias their responses towards the ends of both scales and around the 50 mark, suggesting that
respondents do not use the full range of the scale and some limit themselves to certain areas of
it [17]. Weisberg and Miller (1980) [12] find that mislabeling thermometer scores may over or
under estimate respondents’ actual feeling.
Second, party leader ratings exclusively focus on elites as potential drivers of affect, which
may or may not be the actual reason behind a respondent’s dislike for the out-party. Further-
more, the use of like-dislike scales or trait rating scales means that researchers select the
dimensions that respondents rate leaders on. This method imposes the researchers’ chosen
dimensions onto the respondents, who are then limited to providing information only within
the predefined parameters set by the researchers, rather than based on their own perspectives.
Third, social distance measures like those used by Iyengar, Sood, and Lelkes (2012) [16] cap-
ture respondents’ willingness to engage with out-partisans not their feelings about them, which
ultimately defines affect. Social-distance measures are a consequence of negative affect toward
opposing partisans and do not capture their state of mind about them.
In addition to their individual weaknesses, existing measures compress affective evaluations
into closed-answer scales, preventing researchers from discerning their basis and potentially
overlooking crucial information. Researchers then gain little insight into the motivations driv-
ing respondent’s closed answer responses and are left to hypothesize as to the reasons on their
own or include additional survey items and use more survey time. Yet, researchers who make
and deploy their own survey on affective polarization have an opportunity to gain insight into
respondent’s motivations directly from the respondents themselves. Accepting the expedient
tradeoff is often prudent to conducing practical research and existing measures have many
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 3 / 18
useful applications. However, an ideal measure is one that provides researchers with both the
expediency of an easy-to-use scale and a method for extracting the motivations underlying
respondents’ affect. This measure is our contribution: an easy-to-use survey instrument that
captures respondents’ affective evaluations quantitatively and qualitatively, addressing prob-
lems that plague existing affective measures.
As opposed to closed-answer responses, which may be easier to analyze but can suffer from
measurement error or internal validity issues [5,20], we rely on open ended questions. By
doing so, we allow respondents to explain their state of mind unrestricted by an artificially
generated scale. To avoid the pitfalls of open-ended text data we rely on respondent self-coding
[20]. While our approach is novel in measuring affective polarization, we build on previous
scholars who use open-ended responses to generate quantitative data. For instance, Roberts
et al. (2014) [21] use a structural topic model to classify the topics of open-ended survey ques-
tions using an out-of-the-box topic classifier. In addition, we draw on Glazier, Boydstun, and
Feezell (2021) [20], who highlight the advantages of respondents coding their own open-ended
responses. They demonstrate that these “self-coded evaluations” can mitigate concerns about
intercoder reliability and reduce the bias that may occur when researchers code respondents’
open-ended answers. Our work also builds on Zollinger (2024) [22], who explores in-group
and out-group identity formation based on voters’ qualitative responses. We ask respondents
to provide us with one word to describe in and out partisans, while Zollinger (2024) [22] asks
voters to describe a portrait of ingroup and out group partisans. While Zollinger (2024) [22]
methodology is distinct from ours, we both aim to identify the cleavages that separate voters in
the political parties we investigate through qualitative responses. Our work builds on research
of scholars who devise creative methodologies for extracting quantitative information from
qualitative survey response [2325]; in addition to research that outlines the perils of
researcher coded open-ended survey questions [26,27].
We base our measure in the Receive-Accept-Sample (RAS) model of public opinion forma-
tion [6]. Specifically, Zaller’s theory of survey responses which posits that, rather than walking
around with fixed opinions, respondents construct answers to survey questions when
prompted. Survey responses are generated by sampling top-of-mind considerations. The RAS
model’s Response axiom posits that, in answering survey questions, respondents average
salient considerations to construct an answer. Furthermore, because the salience of consider-
ations changes over time, each respondent’s answer represents a draw from a probability field
of possible answers. The RAS model’s main contribution is in explaining response instability
over time. However, we leverage the cognitive process proposed in The Nature and Origins of
Mass Opinion [6] to measure a part of the opinion construction process.
When prompted with any survey question asking respondents to evaluate out partisans, an
answer will be constructed when the question is posed. Respondents will generate an image of
a partisan, their characterization of the group, based in salient considerations at the time. Then
respondents will necessarily have to describe this image and characterize it in words. Our mea-
sure prompts respondents to provide researchers with the most salient word in their mind
used to describe the group being asked about. We then ask them to provide us a numerical rat-
ing. This process is shown in a flow chart in Fig 1 below while an image of our survey instru-
ment is shown in the following section in Fig 2.
Note that, according to the RAS model, this process would unfold similarly for a thermom-
eter score. Respondents would be prompted by the question to create an image of partisans
based in salient considerations. They would then have to think about that image in terms of
language. However, thermometer scores only have respondents provide a numerical rating for
their feelings towards the summation of salient considerations regarding partisans. They pro-
vide no insight into the image or language a respondent uses to characterize the group in
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 4 / 18
question. This does not mean thermometers are not perfectly valid measure. Indeed, they have
widespread applications and uses as a solid workhorse measurement tool. However, if a
researcher wants more detailed insights into respondents’ considerations, they are faced with
the difficulty of working with unwieldy open-ended data. Should a researcher desire increased
detail while avoiding the complexities and potential burdens associated with extensive open-
ended responses, there is a need for a novel methodological approach that provides a balance
between depth and practicality.
Building on current literature, our new measure of affective polarization has four major
advantages over existing measures. First, it provides a more direct measurement of respon-
dents’ affect by eliciting top of mind natural language considerations rather than constraining
respondents to a scale. Second, by utilizing respondent self-scoring, we eliminate the difficul-
ties associated with quantifying open ended responses [20]. Third, the new measure captures a
distinct aspect of affect, as it produces similar predictive estimates to existing measures but is
not perfectly correlated. Finally, one-word responses provide qualitative value that is useful for
defining the motivations behind affective polarization that scholars propose in theory and that
we show in this paper.
Methodology
In the summer of 2021, we conducted a survey asking respondents for their one-word evalua-
tions of in and out-partisans. Our survey yielded more than 1,300 high-quality and nationally
representative respondents recruited using Lucid’s survey platform. To construct our new
measure we first asked respondents to provide qualitative data by describing Democrats and
Fig 1. Model of response generation.
https://doi.org/10.1371/journal.pone.0310772.g001
Fig 2. One word survey question example.
https://doi.org/10.1371/journal.pone.0310772.g002
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 5 / 18
Republicans using only one word. Then, we ask respondents to self-code the sentiment behind
their word choice on a 7-point scale ranging from “extremely negative” (-3) to “extremely posi-
tive” (3), with “neutral” (0) as the midpoint. Fig 2 shows two examples of how a respondent
might engage with our question.
We prompt the respondent to provide one word that describes Democratic and Republican
voters. The first question represents the qualitative portion of our measure where we ask
respondents to consider their feelings and express them through natural language. This
engagement offers researchers important information about the reasons behind a respondent’s
affect. The follow-up question allows the respondent to self-code the sentiment behind their
word choice. This process takes the sentiment coding out of the researchers’ hands, empower-
ing the respondent to provide the affect behind their word choice.
Importantly, we do not ask respondents to code their affective feelings, but to code the
word that comes to their mind when we prompt their affective feelings, making our measure
of affect distinct from existing measures. Our goal is to quantify the sentiment behind their
attitudes, not simply their attitudes themselves. While attitudes are more general evaluations,
they are not necessarily rooted in emotional evaluations in the same way that sentiments are.
An attitude may be a general belief, but a sentiment is a general reaction. Our two-part ques-
tion prompts this reaction. If affect is rooted in emotional evaluations, then quantifying senti-
ment should better capture these evaluations than measuring beliefs. Fig 3 illustrates the
distribution of respondents’ self-coded word on our seven-point scale. The figure reports par-
tisans’ sentiment about both in and out-groups.
First, both Democrats and Republicans have positive affect toward individuals sharing their
party identification. 85 percent of Democrats and 81 percent of Republicans chose words that
they coded as positive. When describing their in-group, Democrats and Republicans have the
same median sentiment score, 2, and similar in-group sentiment averages 1.9 and 1.8, respec-
tively. Second, Democrats and Republicans have negative affect toward individuals with an
opposing party identification: 68 percent of Democrats and 69 percent of Republicans coded
their out-party word negatively. The median sentiment score was -2 and the average sentiment
scores were -1.1 for Democrats and -1.2 for Republicans describing members of the out-party.
Prima facie, our open ended survey responses follow a distribution that current literature might
expect: strong emotional affect in favor of one’s own party and against their out-party [2,16].
Fig 3. Distribution of respondents’ self-coded word on a seven-point scale.
https://doi.org/10.1371/journal.pone.0310772.g003
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 6 / 18
To validate our one-word measure, we focus on providing evidence that our one-word eval-
uations and their self-code have strong internal and external validity. Therefore, in the sections
to follow we illustrate internal validity in two ways. First, we look at the words respondents
report and the accompanying codes. Second, we compare the self-coded one-word responses
to the most common measures of affect researchers use in the literature. Our goal is to show
that our measure of affect has high face validity, high internal validity, and uncovers motiva-
tions underlying affective polarization that existing measures cannot. The following sections
conducts two tests to evaluate the validity of our one-word affect measure. To start, we provide
evidence showing self-coded one word evaluations measure respondents’ affect toward in-par-
tisans and out-partisans. Then, we examine how well our one-word measure compares with
existing affect measures. Finally, we provide evidence of the post collection value of text data
produced by one-word measures by exploring the dimensions of affect and showing systematic
differences in polarization between respondents who used policy vs valance words.
Internal validity
We begin our internal validity exercise by looking directly at the open-ended words that indi-
viduals report about their feelings towards in and out partisans. These words validate our mea-
sure and capture the emotional responses driving respondents’ affective feelings towards each
group. We find that individuals’ chosen words and codes display meaningful affect.
Figs 4and 5shows the distribution of the most popular words that Democrats and Republi-
cans reported about their in-partisans and the average self-coded responses for each word. In
each case, the plurality words partisans report about their own group are ideological in nature.
Around 6 percent of Democrats report the word “liberal” and 15 percent of Republicans report
the word “conservative.” Yet, Fig 4 reports more clearly that an overwhelming majority of
words tend to be affective in nature: Democrats characterize themselves as “smart,” “good,”
“caring,” and “compassionate;” while Republicans characterize themselves as “smart,” patri-
otic,” “American,” and “good.” Additionally, when we ask respondents to code these words,
the responses we receive are congruent with our expectations about respondents’ feelings
Fig 4. Distribution of the most common words about in-partisans.
https://doi.org/10.1371/journal.pone.0310772.g004
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 7 / 18
towards members of their own party. On average, respondents’ self-coded word reports a posi-
tive evaluation of in-partisans.
Next, we find that out-group responses are even more affective. Figs 4and 5show the distri-
bution of the most popular words that partisans report about out-partisans and the average
self-coded response for each word. Like the previous results, the most common words are
ideological in nature, “liberal” and “conservative.” However, Fig 6 shows that negative emo-
tional evaluations of out-partisans account for a majority of the words. Democrats characterize
Republicans as being “selfish,” “stupid,” “racist,” and “hateful;” while, Republican characterize
Democrats as being “stupid,” “socialists,” “sheep,” and “confused”.
Importantly, when respondents code their word, the responses are congruent with our
expectations about partisans’ feelings toward out-partisans. On average, Fig 6 reports that
respondents select a word and code it as negative when asked to evaluate out-partisans.
Together, these results suggest that the one-word evaluations and their subsequent codes
have strong face validity. Respondents select words based on their emotional evaluation of in
and out-partisans and the codes they assign are congruent with the literature’s expectations.
Additionally, we can see in Fig 6 that our measure provides a novel insight into an asymmetry
in levels of affective polarization. Among Democrats the main ideological word used to charac-
terize the our party “Conservative”, has an average rating of -.47 suggesting a slightly negative
evaluation. However, Republican respondents on average rate the main ideological word used
to characterize the out party “Liberal” as -1.5 suggesting a highly negative evaluation. Our mea-
sure reveals that ratings of the parties’ main ideological words are asymmetrically polarized.
Republicans view liberalism far more negatively than Democrats view conservatism. This dis-
tinction would be obscured with any other measure. For example, sentiment dictionaries
applied to open ended answer would classify liberal and conservative as neutral, while closed
ended answers would produce only the end point numerical values. Our measure shows that
Fig 5. Word cloud of the most common words about in-partisans.
https://doi.org/10.1371/journal.pone.0310772.g005
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 8 / 18
Republicans feel far more negatively towards the ideology of the Democratic party than Demo-
crats do towards the ideology of the Republican party. See Fig 7 for a qualitative presentation
of out-partisan sentiments.
Comparison with existing measures
Next, we compare our one-word self-coded measure of affect with three well-established mea-
sures: thermometer scores, candidate evaluations, and a social-distance measure. We selected
these measures as a benchmark because of their widespread use in the literature [24].
Researchers most frequently use thermometers to measure respondents’ affect [2,3]. In our
survey we asked respondents to rate their feelings about Democratic and Republican voters on a
scale from 0 to 100. Then, to leverage trait ratings we use Lelkes, Sood, and Iyengar’s (2017)
questions and average the scales together to create one measure for each party. Lastly, we create
an affect measure based on Iyengar, Sood, and Lelkes’ (2012) [16] finding that respondents are
unhappy seeing their son or daughter marry a member of the out-party. Building on this
research, we included three questions in our survey asking how happy the respondent would be
if their son or daughter married someone of the out-party, to live in a neighborhood composed
of out partisans, and shop at a grocery store that contributed campaign contributions to out-
partisan candidates [4,28]. We average the scale together to create one measure for each party.
When we compare respondents’ self-coded one-word evaluation of affect to the established
measures, Fig 8 reports a remarkably strong relationship. First, when we compare one-word
evaluations to Democratic and Republican feeling thermometers the correlations are 0.76 and
0.77, respectively. Second, the correlations between Democratic and Republican one-word
evaluations and candidate evaluations of Joe Biden and Donald Trump are 0.73 and 0.73,
respectively. Lastly, the correlations between Democratic and Republican one-word evalua-
tions and lifestyle evaluations are 0.63 and 0.65, respectively. Together, the correlations report
a strong relationship between one-word evaluation and established measures of affect.
While the correlations between the one-word evaluations and established measures are
strongly related, they are not perfectly correlated; this is where their value lies. The residual
Fig 6. Distribution of the most common words about out-partisans.
https://doi.org/10.1371/journal.pone.0310772.g006
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 9 / 18
correlation suggests our measure captures something unique about the latent dimension that
current measures do not. In fact, we should be concerned if these measures were very strongly
related because it would suggest that the one-word measure does not make a meaningful
contribution.
To truly illustrate how our approach stands out from existing affect measures we need to
leverage the words provided by respondents. Next, we examine the sematic undertones provided
Fig 7. Word cloud of the most common words about out-partisans.
https://doi.org/10.1371/journal.pone.0310772.g007
Fig 8. Scatterplot of self-coded one word evaluations vs existing measures.
https://doi.org/10.1371/journal.pone.0310772.g008
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 10 / 18
by responses and argue that our one-word measure captures distinctions between policy and
valence as drivers of affective polarization. We explore these dimensions in the next section.
Dimensions of affect
Current measures of affect, like thermometers, social-distance measures, and candidate evalua-
tions [4], do an excellent job quantifying the divide between partisans. However, they fall short
in their ability to qualitatively uncover heterogeneous motivations for respondents’ affective
evaluations and dislike of their out-party.
Literature on emotional evaluations suggests that affect is based on subjective interpreta-
tions of the world [29]. These interpretations originate from a multidimensional structure
[30], and inform how we evaluate the world around us [31]. Current measures of affective
polarization reduce affective expressions to a single number and omit useful information
about the motivations behind the respondent’s affective evaluations. This section leverages the
content behind individuals’ one-word selection to define dimensions of respondents’ affect
that we anticipate contribute to affective polarization in the political system and that have yet
to be codified by previous literature. We contribute to the literature on affective polarization
by defining and testing two dimensions that may underpin individuals’ affective evaluations of
out-partisans. This novel property of our measure adds a different perspective to our under-
standing of affect and its role in American politics.
We hypothesize that two key motivations drive respondents’ affective evaluations of out-
partisans and contribute to affective polarization in the United States: policy and valence. On
the one hand, partisans’ animus between Democrats and Republicans may be the result of pol-
icy and ideological differences. Evidence suggests that the mass public is ideologically divided
[32], that the public sees their world through a partisan lens and interpret the world, even
basic facts, differently depending on their party identification [33,34]; while policy divisions at
the elite level trickle down to the voters who follow their lead [35].
On the other hand, partisans’ animus between Democrats and Republicans may be the
result of negative character evaluations of the out-party [36,37]. A respondent may contrast
the policy divisions between themselves and an out-partisan and then ascribe a character (or
valence) attribute onto an out-partisan because of their beliefs. For example, a Democrat
might view a Republican’s anti-abortion stance as fundamentally opposed to their own belief
in reproductive rights, leading them to attribute negative character traits, such as being oppres-
sive or misogynistic, to the Republican. This theory is based in Tajfel and Turner (1979) [38]
who define conditions that produce intergroup conflict based on an in-group ascribing nega-
tive character evaluations to an out-group they perceive as inferior.
To examine the policy and valence dimension of affective polarization, we hand-code code
respondents’ one-word evaluations into three groups: neither policy nor valence, policy, or
valence. The purpose of this exercise is to quantitatively assess the dimensions of a respon-
dents’ affect towards in and out partisans. As our theory proposes, we anticipate that respon-
dents’ affect is driven by either policy or valence. Therefore, we can better understand the
respondents’ visceral reactions, gut response, and top-of-bucket state-of-mind affect by coding
each one-word response.
We apply the following coding rules to each category of words. First, a policy word is any
word that talks about policy or has an ideological direction to it. We code words like “liberal,”
“conservative,” “socialist,” and “fascists” as policy. Second, a valence word is any word that
talks about demeanor, behavior, or character [36,37]. We code words like “stupid,” “unin-
formed,” “sheep,” “hateful” and vulgar characterizations as a valence evaluation. Additionally,
we also code words ascribing positive evaluations like “smart,” “correct,” “intelligent” and
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 11 / 18
“good” as valence as well. Lastly, we code words that neither directly describe character or
have a policy or ideological angle to them as “neither policy nor valence.” These include words
like “voter,” “money,” and “workers.” We represent the valence and policy dimension as two
dummy variables that serve as our key independent variables, with words representing neither
policy nor valence serving as the reference category. Together, our independent variable repre-
sents the two dimensions of affect, valence and policy, that respondents rely on to evaluate vot-
ers. In the model, we only use out-party evaluations to create these two dimensions. In the
appendix, we report the full results of the models using in-party evaluations to create these two
dimensions in Tables 1A-4A in S1 Appendix. We use these independent variables to predict
the affective polarization scores using our respondent-coded one-word evaluation.
Affective polarization scores serve as our dependent variable. We create our measure of
affective polarization by applying the same formula scholars use to create thermometers, can-
didate evaluation, and social-distance affective polarization to the respondent’s self-coding of
their one-word evaluation:
Affective Polarization ¼In Party Evaluation Out Party Evaluation
The formula for affective polarization uses respondents’ self-coded word and takes their in-
party affect evaluation and subtracts it from a respondents’ out-party affect evaluation. The
formula generates a score ranging from -6 representing extreme in-party dislike to 6 represent-
ing extreme out-party dislike, with 0 indicating indifference between both parties. As a theo-
retical extension, we also use the policy and valence dimension to predict affective
polarizations using thermometers, candidate evaluation, and social-distance measures.
We employ an ordinary least squares model that regresses affective polarization (using each
measure of interest) onto our policy and valence dimension using the following formula:
Affective Polarization
¼aþb1ðPolicy DimensionÞ þ b2ðValence DimensionÞ þ biXi
ð Þ þ dsþε
Affective Polarization represents the values of affective polarization derived using each mea-
sure (self-coded words, thermometers, candidate evaluation, and social-distance). We stan-
dardize each measure of affective polarization so that mean is equal to 0 and the standard
deviation is 1 to interpret the coefficients on the same scale. Policy Dimension represents
words that have a policy or ideological meaning; while the Valence Dimension covers any
words that have a positive or negative character evaluation. Together, these variables represent
our key independent variables, and we compare their coefficient to the base term: non-policy
or valence words.
X
i
represents our control variables. Our controls include the respondents self-coded one-
word evaluation of out-party voters. This control is the most important because it tests whether
the valence or policy dimensions contributes predictive power to affective polarization, beyond
merely the positive or negative evaluation of the word. Our model also controls include demo-
graphic characteristics such as the respondents’ age,income,gender,education and ethnicity.
We also include terms measuring extremism in respondents party identification and ideology.
Finally, we include political engagement measures: whether the respondent donated to a politi-
cal candidate and whether they voted in the 2020 election.
We report the results of our key independent variable across three model specifications: base
model (includes only our dependent and independent variable), control model (includes our
controls along with our independent variable) and a state fixed effects model (δ
s
). We report the
tables for each full specification across each affective polarization measure in Tables 1A-4A in
S1 Appendix. We also report alternative model specifications in Fig 3A in S1 Appendix.
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 12 / 18
We find that the valence dimension contributes to predicting affective polarization. Fig 9
reports the results of the linear model regressing the affective measures onto the policy and
valence dimensions. When predicting affective polarization using the self-coded one-word
evaluations, the valence and policy dimension preform equally as well (positive and statistically
significant), with the policy dimension outperforming in the state fixed effects model. When
predicting affective polarization using thermometer scores or candidate evaluations, the
valence dimension outperforms the policy dimension. Finally, nether the policy nor valence
dimensions are useful beyond the positive or negative one-word evaluations in predicting
affective polarization using social distance measures. In each model, the coefficient represent-
ing respondents’ one-word evaluations of out-partisan voters is significant, positive, and out-
performs both the valence and policy dimensions in each model. Yet, across key indicators of
affective polarization, both the valence and policy dimensions contribute additional predictive
power that suggests that valence and policy evaluations of out-partisans may motivate affective
polarization.
Taking the control model, we find that when compared to non-policy or valence words,
valence words increase affective polarization as measured using words, thermometers, and
candidate evaluations measures by 0.12, 0.19, and 0.2 units, respectively. Meaning that valence
evaluations of the out-party increase affective polarization by a fifth to at least a tenth of a stan-
dard deviation.
Consequently, even when we include respondents’ one-word evaluations of out-partisans
into the model, the valence dimension is a significant predictor of affective polarization. This
effect is meaningful because it shows that the valence dimension exerts an effect that is inde-
pendent from the positive/negative sentiment of the word alone. Therefore, the content of
respondents’ affect (i.e., character evaluation) is meaningful in understanding affective polari-
zation. This additional content is a key contribution of our one word measures.
These results report that character evaluations of out-partisans are associated with greater
affective polarization than policy or ideological assessment of the out-party. The results imply
that the partisan animus behind affective polarization is not solely rooted in policy but also in
the character evaluations that partisan project onto their counterpart. While not unique in this
regard, these finding do further our understanding of how negative character evaluations drive
Fig 9. Estimates for the effect of out-group policy and valence dimension on affective polarization.
https://doi.org/10.1371/journal.pone.0310772.g009
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 13 / 18
the distrust and dislike that partisans harbor for one another. Indeed, our findings fit with and
connect to a broader literature on discursive polarization in communication between partisans
[39]. Other scholars, such as Bruggemann and Meyer (2023) [39], have found that out group
communication is characterized by hostile and dismissive interactions at least partially driven
by negative constructions of the outgroup [39]. These findings should raise the alarm that pol-
icy gridlock, media characterizations, and firebrand rhetoric fundamentally reveal divisions
among partisans that go well beyond rational policy dimensions but embed themselves in the
emotional consciousness of partisans.
These results are striking because they reveal that established quantitative measures of affect
contain key information about the nature of respondents’ affect, which we show that one-
word evaluations help to clarify. Without the qualitative assessment that one-word answers
offer, scholars may be missing key variation in respondents’ affective attitude about out-parti-
sans obscured by affective polarization scores. Here our qualitative exercise highlights the
character dimensions that comprise the emotional affective dimensions that explains affective
polarization. Further, these results lend support to Tajfel and Turner’s (1979) [38] social iden-
tity theory that the literature has taken as true and we formalize in our analysis.
In sum, these results show that dimensions of respondent’s affect towards out-partisans
contribute to our understanding of affective polarization. Particularly, respondents harboring
negative character evaluations tend to have higher levels affective polarization, as measured by
words, thermometers, and candidate evaluations. These dimensions contribute independently
to respondent’s one-word evaluations and makes them unique in explaining affective
polarization.
Conclusion
As political divisions expand, our need to measure affect and polarization in the public becomes
even more valuable. However, while extremely useful, current measures of affect, such as ther-
mometers scores, candidate evaluations, and social-distance measures are closed-ended in their
nature. To better measure these divisions, we introduce a unique measure of affect that draws
on innovations in open-ended responses. Our open-ended approach measures partisan affect
by allowing respondents to provide their top-of-mind evaluation, without the restrictions of a
bounded scale. Then, having respondents self-code their answer provides researchers with an
easy-to-use scale the retains the rich information open ended responses provide.
If scholars aim to capture affective evaluations, then we argue that the measures we intro-
duce captures respondents’ emotional evaluations about partisans; thereby enhancing the
validity of affective analyses. We support our argument through a battery of internal and exter-
nal validity checks. First, we report the raw words that respondents selected and show face
validity in their emotional evaluations of in and out-partisans ("F*cktards” and “Poopybutts”).
Second, we show that our new measures are highly, but not perfectly correlated, to established
measures of affect [4,16]. These results imply that our measures capture a unique feature of
affect that existing measures may not. This unique feature better captures the sentiment behind
their emotional evaluations.
Together, our analysis suggests one-word evaluations contribute to our understanding of
partisan affect and affective polarization by revealing the motivations behind respondents’
evaluations. The type of analysis presented here can only be done using self-coded one-word
responses since traditional measures reduce emotional responses onto a bounded scale and
fully open-ended responses are too unwieldy for quantitative analysis.
Despite our efforts, our protocol it not without limitations. First, unless self-coding open-
ended responses are widely implemented in major national surveys (CCES/ANES), only
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 14 / 18
researchers who have control over the questions in their own survey can implement self-coding.
This is a solvable problem. If CCES and ANES, introduce self-coded one-word responses, then
scholars can use open-ended responses or validate existing affect measures. Second, our survey
item involves respondents typing a response and then coding it. This two-step process takes
additional time compared to traditional scales. This cost is unavoidable. Yet, we believe the ben-
efits outlined above are well worth the survey time. Third, this paper uses one word and one
self-code of that word to measure affect. In effect, this approach asks respondents to take com-
plex human emotions and motivations and summarize them into a single word, which may be
seen as being reductionist. Methodologically, it may miss out on the opportunity to obtain
deeper insights into the respondents’ motivations by inviting them to share more detailed quali-
tative insights. Longer open-ended text fields, qualitative interviews, and focus groups may
serve as useful alternatives to a single word and result in collecting more comprehensive qualita-
tive data. While our method introduces a new survey protocol to affective polarization research,
we recognize that it might not provide the depth of insight achievable through more expansive
qualitative techniques. Therefore, leveraging a self-coding protocol alongside deeper quantita-
tive analysis could powerfully enable researchers to construct a qualitative dataset, converting
them into quantitative insights, and test hypotheses about respondents’ emotions, attitudes, or
motivations. Irrespective of the quantitative approach, allowing respondents to self-code their
qualitative responses enrich the dataset with nuanced insights directly from the source,
enhances the accuracy, usefulness and relevance of the data collected. We encourage future
researchers to pair our self-coding procedure with a longer qualitative response from partici-
pants to gain the benefits of self-coding along with richer qualitative insights.
While our measure relies on a self-coded evaluation, alternative methods to quantify open-
ended responses exist. For example, large language models and natural language processing
are growing in popularity due to their ability to process large amounts of text and evaluate it
across various dimensions [23,24,40]. These models can classify underlying sentiment (posi-
tive, neutral, or negative) [41] and emotion (anger, sadness, joy, etc.) [42], identify people,
places, or things, and group common themes together across responses [43]. These processing
tools are powerful, particularly for text that have already been collected by researchers, text
that the author cannot evaluation themselves, or long-form open-ended survey responses that
researchers want to quantify. Text processing tools with these capabilities are becoming
increasingly democratized. Free and open-source text processing tools are available on sites
like GitHub and Hugging Face. While proprietary LLM’s like OpenAI’s ChatGPT, Google’s
Gemini, Anthropic’ Claude 3 and more are becoming cheaper, more affordable, and more
accessible every year. For researcher who are interested in getting started immediately, we
believe that they should leverage ChatGPT as a revolutionary text classification tool [4446].
Through prompt engineering, researchers can use ChatGPT as a research assistant to analyze
survey responses, classify respondents’ emotions and sentiments, and extract common themes
within a text corpus. Leveraging these advancements in natural language processing, particu-
larly ChatGPT, researcher can take in long-form qualitative data and derive quantitative mea-
sures efficiently and accurately.
Despite limitations and alternative classification methods, for scholars looking to move for-
ward with our survey protocol, we have several practical avenues for future research. First,
scholars can apply our protocol comparatively to study affective polarization partisans in other
counties. Whether there are cross-national differences in the dimensions of partisan affect is a
question our protocol is well suited to answer. Additionally, differences in how partisans char-
acterize each other across parties in multi-party democracies is likely a promising area of
research given existing work on the topic [11]. Second, scholars can use our protocol to mea-
sure divisions between any group of individuals. For example, Amlani and Kiesel (2022) [47]
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 15 / 18
examine one word evaluations of vaccinated or unvaccinated Americans. Third, we asked
respondents to code the sentiment of their word; however, scholars can also ask respondents
to choose from a preexisting list of emotions (i.e., angry, frustration, sad, or happy). This addi-
tional step would provide scholars with a quantitative understanding of the emotions that
underlie respondents’ choice of words.
Supporting information
S1 Appendix.
(DOCX)
Acknowledgments
We thank and appreciate the support and suggestions given by Chris Hare and Amber Boy-
dstun throughout our work on this project.
Author Contributions
Conceptualization: Spencer Kiesel.
Data curation: Spencer Kiesel.
Formal analysis: Spencer Kiesel.
Funding acquisition: Spencer Kiesel.
Investigation: Spencer Kiesel.
Methodology: Spencer Kiesel, Sharif Amlani.
Visualization: Sharif Amlani.
Writing original draft: Sharif Amlani.
Writing review & editing: Sharif Amlani.
References
1. Gidron N, Adams J, Horne W. American Affective Polarization in Comparative Perspective. Elements in
American Politics [Internet]. 2020 Nov [cited 2022 Jun 27]; Available from: https://www.cambridge.org/
core/elements/american-affective-polarization-in-comparative-perspective/1E3584B482D51DB25
FFFB37A8044F204
2. Iyengar S, Lelkes Y, Levendusky M, Malhotra N, Westwood SJ. The Origins and Consequences of
Affective Polarization in the United States. Annu Rev Polit Sci. 2019 May 11; 22(1):129–46.
3. Lelkes Y, Westwood SJ. The Limits of Partisan Prejudice. The Journal of Politics. 2017 Apr; 79
(2):485–501.
4. Druckman JN, Levendusky MS. What Do We Measure When We Measure Affective Polarization? Pub-
lic Opinion Quarterly. 2019 May 21; 83(1):114–22.
5. Reja U, Manfreda KL, Hlebec V, Vehovar V. Open-ended vs. Close-ended Questions in Web Question-
naires. 2003; 19.
6. Zaller J. The nature and origins of mass opinion. Cambridge university press; 1992.
7. Barrett LF, Bliss-Moreau E. Affect as a Psychological Primitive. Adv Exp Soc Psychol. 2009; 41:167–
218. https://doi.org/10.1016/S0065-2601(08)00404-8 PMID: 20552040
8. Barsade SG, Gibson DE. Group emotion: A view from top and bottom. In: Composition. US: Elsevier
Science/JAI Press; 1998. p. 81–102. (Research on managing groups and teams, Vol. 1.).
9. Barsade SG, Knight AP. Group Affect. Annual Review of Organizational Psychology and Organizational
Behavior. 2015; 2(1):21–46.
10. Kelly JR, Barsade SG. Mood and Emotions in Small Groups and Work Teams. Organizational Behavior
and Human Decision Processes. 2001 Sep 1; 86(1):99–130.
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 16 / 18
11. Gidron N, Adams J, Horne W. Who Dislikes Whom? Affective Polarization between Pairs of Parties in
Western Democracies. Comparative Political Studies. 2022;
12. Weisberg HF, Miller AH. Evaluation of the Feeling Thermometer: A Report to the National Election
Study Board Based on Data from the 1979 Pilot Survey. 1980; 35.
13. Lelkes Y, Sood G, Iyengar S. The Hostile Audience: The Effect of Access to Broadband Internet on Par-
tisan Affect. American Journal of Political Science. 2017; 61(1):5–20.
14. Reiljan A. ‘Fear and loathing across party lines’ (also) in Europe: Affective polarisation in European
party systems. European J Political Res. 2020 May; 59(2):376–96.
15. Reiljan A, Garzia D, Ferreira Da Silva F, Trechsel AH. Patterns of Affective Polarization toward Parties
and Leaders across the Democratic World. Am Polit Sci Rev. 2023 Jun 29;1–17.
16. Iyengar S, Sood G, Lelkes Y. “Affect, not ideology: A social identity perspective on polarization”: Erra-
tum. Public Opinion Quarterly. 2012; 76(4):819–819.
17. Wilcox C, Sigelman L, Cook E. Some Like It Hot: Individual Differences in Responses to Group Feeling
Thermometers. The Public Opinion Quarterly. 1989; 53(2):246–57.
18. Brady HE. The Perils of Survey Research: Inter-Personally Incomparable Responses. Political Method-
ology. 1985; 11(3/4):269–91.
19. Winter N, Berinsky AJ. What’s Your Temperature? Thermometer Ratings and Political Analysis. In Atlanta,
GA; 1999. Available from: https://www.nicholasjgwinter.com/assets/papers/Winter-Berinsky-1999.pdf
20. Glazier RA, Boydstun AE, Feezell JT. Self-coding: A method to assess semantic validity and bias when
coding open-ended responses. Research & Politics. 2021 Jul 1; 8(3):20531680211031752.
21. Roberts ME, Stewart BM, Tingley D, Lucas C, Leder-Luis J, Gadarian SK, et al. Structural Topic Models
for Open-Ended Survey Responses. American J Political Sci. 2014 Oct; 58(4):1064–82.
22. Zollinger D. Cleavage Identities in Voters’ Own Words: Harnessing Open-Ended Survey Responses.
American Journal of Political Science. 2024; 68(1):139–59.
23. Barbera
´P, Boydstun AE, Linn S, McMahon R, Nagler J. Automated Text Classification of News Articles:
A Practical Guide. Polit Anal. 2021 Jan; 29(1):19–42.
24. Grimmer J, Stewart BM. Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods
for Political Texts. Political Analysis. 2013; 21(3):267–97.
25. Mossholder KW, Settoon RP, Harris SG, Armenakis AA. Measuring Emotion in Open-ended Survey
Responses: An Application of Textual Data Analysis. Journal of Management. 1995; 21(2).
26. Mikhaylov S, Laver M, Benoit K. Coder Reliability and Misclassification in Comparative Manifesto Proj-
ect Codings. 66th MPSA annual national conference. 2008 May 1;
27. Montgomery AC, Crittenden KS. Improving Coding Reliability for Open-Ended Questions. The Public
Opinion Quarterly. 1977; 41(2):235–43.
28. Levendusky M, Malhotra N. Does Media Coverage of Partisan Polarization Affect Political Attitudes?
2015; 20.
29. Marcus GE. Emotions in politics. Annual review of political science. 2000; 3(1):221–50.
30. Davis KL, Panksepp J. The brain’s emotional foundations of human personality and the Affective Neuro-
science Personality Scales. Neuroscience & Biobehavioral Reviews. 2011; 35(9):1946–58. https://doi.
org/10.1016/j.neubiorev.2011.04.004 PMID: 21527289
31. Tesser A, Martin L. The psychology of evaluation. 1996;
32. Abramowitz AI, Saunders KL. Is Polarization a Myth? The Journal of Politics. 2008 Apr; 70(2):542–55.
33. Bartels LM. Beyond the Running Tally: Partisan Bias in Political Perceptions. Political Behavior. 2002
Jun 1; 24(2):117–50.
34. Campbell A, Center U of MSR, Converse PE, Miller WE, Stokes DE. The American Voter. University of
Chicago Press; 1960. 576 p.
35. Lenz GS. Learning and Opinion Change, Not Priming: Reconsidering the Priming Hypothesis. American
Journal of Political Science. 2009; 53(4):821–37.
36. Stokes DE. Spatial Models of Party Competition. Am Polit Sci Rev. 1963 Jun; 57(2):368–77.
37. Stone WJ. Candidates and Voters: Ideology, Valence, and Representation in U.S Elections [Internet].
Cambridge: Cambridge University Press; 2017 [cited 2020 Feb 19]. Available from: http://ebooks.
cambridge.org/ref/id/CBO9781108225021
38. Tajfel H, Turner J. An integrative theory of intergroup conflict. In: The Social Psychology of Intergroup
Relations. Monterey, CA: Brooks Cole; 1979. p. 33–47.
39. Bru¨ggemann M, Meyer H. When debates break apart: discursive polarization as a multi-dimensional
divergence emerging in and through communication. Communication Theory. 2023; 33(2–3):132–42.
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 17 / 18
40. Jelodar AB, Sun Y. Joint object and state recognition using language knowledge. In: 2019 IEEE Interna-
tional Conference on Image Processing (ICIP). IEEE; 2019. p. 3352–6.
41. Trabelsi Z, Saidi F, Thangaraj E, Veni T. A survey of extremism online content analysis and prediction
techniques in twitter based on sentiment analysis. Secur J. 2023 Jun 1; 36(2):221–48.
42. Cochrane C, Rheault L, Godbout JF, Whyte T, Wong MWC, Borwein S. The Automatic Analysis of
Emotion in Political Speech Based on Transcripts. Political Communication. 2022 Jan 2; 39(1):98–121.
43. Greene D, Cross JP. Exploring the Political Agenda of the European Parliament Using a Dynamic Topic
Modeling Approach. Political Analysis. 2017; 25(1):77–94.
44. Belal M, She J, Wong S. Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis [Inter-
net]. arXiv; 2023 [cited 2024 Mar 11]. Available from: http://arxiv.org/abs/2306.17177
45. Fatouros G, Soldatos J, Kouroumali K, Makridis G, Kyriazis D. Transforming sentiment analysis in the
financial domain with ChatGPT. Machine Learning with Applications. 2023 Dec 15; 14:100508.
46. Susnjak T. Applying BERT and ChatGPT for Sentiment Analysis of Lyme Disease in Scientific Litera-
ture. In 2024 [cited 2024 Mar 11]. p. 173–83. Available from: http://arxiv.org/abs/2302.06474
47. Amlani S, Kiesel S. Analysis | How much do vaccinated Americans dislike the unvaccinated? We mea-
sured. Washington Post [Internet]. 2022 Jan 27 [cited 2022 Jun 30]; Available from: https://www.
washingtonpost.com/politics/2022/01/27/vaxxed-anger-pandemic-public-opinion/
PLOS ONE
Affective Polarization in a Word: Open-Ended and Self-Coded Evaluations of Partisan Affect
PLOS ONE | https://doi.org/10.1371/journal.pone.0310772 January 16, 2025 18 / 18
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Research indicates that affective polarization pervades contemporary democracies worldwide. Although some studies identify party leaders as polarizing agents, affective polarization has been predominantly conceptualized as a product of in-/out-party feelings. This study compares levels of party affective polarization (PAP) and leader affective polarization (LAP) cross-nationally, using data from the Comparative Study of Electoral Systems. Applying like–dislike scales and an identical index to both concepts, we reveal that while the two strongly correlate, LAP is systematically lower than PAP. The United States emerges as an exceptional case, being the only country where LAP significantly exceeds PAP. Drawing on regime input/output and institutions as theoretical building blocks, we explore cross-national variations and show that the relative strength of LAP vis-à-vis PAP is increased by presidential regime type, poor government performance, and low party system fragmentation. The findings of this study contribute to the thriving research on affective polarization and personalization of politics.
Article
Full-text available
While dislike of opposing parties, that is, affective polarization, is a defining feature of contemporary politics, research on this topic largely centers on the United States. We introduce an approach that analyzes affective polarization between pairs of parties, bridging the US two-party system and multiparty systems in other democracies. Analyzing survey data from twenty Western democracies since the mid-1990s, first, we show that partisans' dislike of out-parties is linked to elite policy disagreements on economic issues and, increasingly over time, also to cultural issues. Secondly, we argue and empirically demonstrate that governing coalition partners in parliamentary democracies display much warmer feelings toward each other than we would expect based on elite policy (dis)agreements. Third, we show that radical right parties are disliked much more intensely than we would expect based on policy disputes and coalition arrangements. These findings highlight the policy-based and institutional underpinnings of affective polarization.
Article
Full-text available
Fundamental transformations of underlying cleavage structures in advanced democracies should become evident in new collective identities. This article uses quantitative text analysis to investigate how voters describe their ingroups and outgroups in open‐ended survey responses. I look at Switzerland, a paradigmatic case of electoral realignment along a “second,” universalism–particularism dimension of politics opposing the far right and the new left. Keyness statistics and a semi‐supervised document scaling method (latent semantic scaling) serve to identify terms associated with the poles of this divide in voters’ responses, and hence to measure universalist/particularist identities. Based on voters’ own words, the results support the idea of collective identities consolidating an emerging cleavage: Voters’ identity descriptions relate to far right versus new left support, along with known sociostructural and attitudinal correlates of the universalism–particularism divide, and they reveal how groups opposed on this dimension antagonistically demarcate themselves from each other.
Article
Full-text available
Nowadays, extremist organizations use social networks, such as Twitter, to flourish their dark activities. Usually, to polarize new members, these organizations attempt to share their radical propaganda by posting tweets. Practically, Sentiment Analysis (SA) techniques are widely used to classify the polarity of these extremist tweets, to derive appropriate conclusions for decision-making purposes, and to make valuable predictions about future violent and terrorist events. To study the influence of social networks-based malicious activities on human security and safety, this paper surveyed different Machine and Deep Learning based techniques used for Tweets SA, and discussed their strengths and weaknesses and recent trends in the field. Furthermore, the conducted survey work highlighted promising key areas and potential challenges that require further consideration to implement more effective methods to combat extremism in online social networks.
Article
Full-text available
Automatic sentiment analysis is used extensively in political science. The digitization of legislative transcripts has increased the potential application of established tools for the automated analyses of emotion in text. Unlike in writing, however, expressing emotion in speech involves intonation, facial expressions, and body lan- guage. Drawing on a new dataset of annotated texts and videos from the Canadian House of Commons, this paper does three things. First, we examine whether transcripts capture the emotional content of speeches. We find that transcripts capture sentiment, but not emotional arousal. Second, we compare strategies for the automated analysis of sentiment in text. We find that leading approaches performed reasonably well, but sentiment dictionaries generated using word embeddings surpassed these other approaches. Finally, we test the robustness of the approach based on word embeddings. Although the methodology is reasonably robust to alternative specifications, we find that dictionaries created using word embeddings are sensitive to the choice of seed words and to training corpus size. We conclude by discussing the implications for analyses of political speech.
Article
Full-text available
Open-ended survey questions can provide researchers with nuanced and rich data, but content analysis is subject to misinterpretation and can introduce bias into subsequent analysis. We present a simple method to improve the semantic validity of a codebook and test for bias: a “self-coding” method where respondents first provide open-ended responses and then self-code those responses into categories. We demonstrated this method by comparing respondents’ self-coding to researcher-based coding using an established codebook. Our analysis showed significant disagreement between the codebook’s assigned categorizations of responses and respondents’ self-codes. Moreover, this technique uncovered instances where researcher-based coding disproportionately misrepresented the views of certain demographic groups. We propose using the self-coding method to iteratively improve codebooks, identify bad-faith respondents, and, perhaps, to replace researcher-based content analysis.
Chapter
This chapter presents a practical guide for conducting sentiment analysis using Natural Language Processing (NLP) techniques in the domain of tick-borne disease text. The aim is to demonstrate the process of how the presence of bias in the discourse surrounding chronic manifestations of the disease can be evaluated. The goal is to use a dataset of 5643 abstracts collected from scientific journals on the topic of chronic Lyme disease to demonstrate using Python, the steps for conducting sentiment analysis using pretrained language models and the process of validating the preliminary results using both interpretable machine learning tools, as well as a novel methodology of leveraging emerging state-of-the-art large language models like ChatGPT. This serves as a useful resource for researchers and practitioners interested in using NLP techniques for sentiment analysis in the medical domain.
Article
“Polarization” is a common diagnosis of the state of contemporary societies. Yet, few studies theorize or systematically analyze how polarization evolves in media content. To guide future empirical studies, we introduce a public sphere perspective on polarization. Discursive Polarization, defined as divergence emerging in public communication, may disrupt the public sphere if left untamed. Its analysis should combine the study of ideological polarization (increasing disagreement about issues) and affective polarization (growing disaffection between groups) as evolving in communication. Both processes may be measured in media content. We propose a framework combining the study of journalism and digital communication networks, investigating (1) content and (2) networked interactions regarding both political issues and social identity formation. The exploration of how the public sphere is disrupted in the process of Discursive Polarization may help us to understand the wider social phenomenon of polarization: before societies break apart, debates break apart.
Book
American political observers express increasing concern about affective polarization, i.e., partisans' resentment toward political opponents. We advance debates about America's partisan divisions by comparing affective polarization in the US over the past 25 years with affective polarization in 19 other western publics. We conclude that American affective polarization is not extreme in comparative perspective, although Americans' dislike of partisan opponents has increased more rapidly since the mid-1990s than in most other Western publics. We then show that affective polarization is more intense when unemployment and inequality are high; when political elites clash over cultural issues such as immigration and national identity; and in countries with majoritarian electoral institutions. Our findings situate American partisan resentment and hostility in comparative perspective, and illuminate correlates of affective polarization that are difficult to detect when examining the American case in isolation.