Technical ReportPDF Available

The Development and Psychometric Properties of LIWC-22

February 2022

DOI:10.13140/RG.2.2.23890.43205

Authors:

Ryan L. Boyd

Stony Brook University

Ashwini Ashokkumar

Stanford University

Sarah Seraj

University of Texas at Austin

James W. Pennebaker

University of Texas at Austin

The words that people use in everyday life tell us about their psychological states: their beliefs, emotions, thinking habits, lived experiences, social relationships, and personalities. From the time of Freud’s writings about “slips of the tongue” to the early days of computer-based text analysis, researchers across the social sciences have amassed an extensive body of evidence showing that people’s words have tremendous psychological value. To appreciate some of the truly great pioneers, check out (Allport, 1942), Gottschalk and Gleser (1969), Stone et al., (1966), and Weintraub (1989). Although promising, the early computer methods floundered because of the sheer complexity of the task. In order to provide a better method for studying verbal and written speech samples, we originally developed a text analysis application called Linguistic Inquiry and Word Count, or LIWC (pronounced “Luke”). The first LIWC application was developed as part of an exploratory study of language and disclosure (Francis & Pennebaker, 1992). The second (LIWC2001), third (LIWC2007), fourth (2015), and now fifth (LIWC-22) versions updated the original application with increasingly expanded dictionaries and sophisticated software design (Pennebaker et al., 2001, 2007, 2015). The most recent evolution, LIWC-22 (Pennebaker et al., 2022), has significantly altered both the dictionary and the software options to reflect new directions in text analysis. As with previous versions, the program is designed to analyze individual or multiple language files quickly and efficiently. At the same time, the program attempts to be transparent and flexible in its operation, allowing the user to explore word use in multiple ways.

Content uploaded by Ryan L. Boyd

Content may be subject to copyright.

The Development and Psychometric

Properties of LIWC-22

Ryan L. Boyd, Ashwini Ashokkumar,

Sarah Seraj, and James W. Pennebaker

The University of Texas at Austin

General email correspondence should be sent to Ryan L. Boyd or James W. Pennebaker. Specific

inquiries about LIWC-22 should be directed to the LIWC team via https://www.liwc.app/contact.

The LIWC-22 program is owned and distributed by Pennebaker Conglomerates and is intended

for academic, university-based research purposes only. For all other uses of LIWC (e.g.,

commercial use, non-profit organization use), Receptiviti, Inc. has exclusive distribution rights.

The official reference for this paper is:

Boyd, R. L., Ashokkumar, A., Seraj, S., & Pennebaker, J. W. (2022). The development

and psychometric properties of LIWC-22. Austin, TX: University of Texas at Austin.

https://www.liwc.app

LIWC-22 Development Manual Page 1

Last Updated: 2022-02-16

Table of Contents

The Development and Psychometric Properties of LIWC-22 ........................................................ 2

The LIWC-22 Text Processing Modules ........................................................................................ 2

The Main Text Processing Module: LIWC Analysis .................................................................. 2

Companion Processing Modules ................................................................................................. 3

Dictionary workbench .............................................................................................................. 3

Word frequencies and word clouds .......................................................................................... 3

Topic modeling with the Meaning Extraction Method ............................................................ 3

Narrative arc ............................................................................................................................. 4

Language style matching .......................................................................................................... 4

Contextualizer .......................................................................................................................... 4

Case studies .............................................................................................................................. 4

Prepare transcripts .................................................................................................................... 4

The LIWC-22 Dictionary and its Development .............................................................................. 5

Development of the LIWC-22 Dictionary ................................................................................... 6

LIWC-22: Establishing the Psychometrics .................................................................................. 8

The Test Kitchen Corpus .......................................................................................................... 8

Table 1. The Test Kitchen Corpus of 31 Million Words ...................................................... 9

Quantifying the Reliability and Validity of LIWC-22 ........................................................... 10

Table 2. LIWC-22 Language Dimensions and Reliability ................................................. 11

Variability and Context in the Measurement of Verbal Behavior ......................................... 14

Table 3. Selected Descriptive Statistics from the Test Kitchen Corpus ............................. 15

Significant Changes in LIWC-22 from Previous Versions of LIWC ........................................ 17

New and updated LIWC-22 language categories ................................................................... 17

LIWC categories no longer included ..................................................................................... 21

Table 4. Comparisons Between LIWC-22 and LIWC2015 ................................................ 22

LIWC Dictionary Translations ..................................................................................................... 24

Acknowledgements ....................................................................................................................... 25

References ..................................................................................................................................... 26

Appendix A: The Test Kitchen Corpus ........................................................................................ 32

Table A1. The Test Kitchen Corpus ................................................................................... 33

Corpus and Text File Selection ................................................................................................. 34

Individual Corpus Characteristics ............................................................................................. 35

Appendix B: Recommended Further Reading .............................................................................. 38

LIWC-22 Development Manual Page 2

The Development and Psychometric Properties of LIWC-22

The words that people use in everyday life tell us about their psychological states: their beliefs,

emotions, thinking habits, lived experiences, social relationships, and personalities. From the

time of Freud’s writings about “slips of the tongue” to the early days of computer-based text

analysis, researchers across the social sciences have amassed an extensive body of evidence

showing that people’s words have tremendous psychological value. To appreciate some of the

truly great pioneers, check out (Allport, 1942), Gottschalk and Gleser (1969), Stone et al.,

(1966), and Weintraub (1989).

Although promising, the early computer methods floundered because of the sheer complexity of

the task. In order to provide a better method for studying verbal and written speech samples, we

originally developed a text analysis application called Linguistic Inquiry and Word Count, or

LIWC (pronounced “Luke”). The first LIWC application was developed as part of an exploratory

study of language and disclosure (Francis & Pennebaker, 1992). The second (LIWC2001), third

(LIWC2007), fourth (2015), and now fifth (LIWC-22) versions updated the original application

with increasingly expanded dictionaries and sophisticated software design (Pennebaker et al.,

2001, 2007, 2015).

The most recent evolution, LIWC-22 (Pennebaker et al., 2022), has significantly altered both the

dictionary and the software options to reflect new directions in text analysis. As with previous

versions, the program is designed to analyze individual or multiple language files quickly and

efficiently. At the same time, the program attempts to be transparent and flexible in its operation,

allowing the user to explore word use in multiple ways.

The LIWC-22 Text Processing Modules

At its core, LIWC-22 consists of software and a “dictionary” — that is, a map that connects

important psychosocial constructs and theories with words, phrases, and other linguistic

constructions. To avoid confusion, words contained in texts that are read and analyzed by LIWC-

22 are referred to as target words. Words in the LIWC-22 dictionary file will be referred to as

dictionary words. Groups of dictionary words that tap a particular domain (e.g., negative

emotion words) are variously referred to as “subdictionaries” or “word categories” or just

“categories.”

The Main Text Processing Module: LIWC Analysis

LIWC-22 is designed to accept written or transcribed verbal text which has been stored as a

digital, machine-readable file in one of multiple formats, including plain text (.txt), PDF, RTF, or

standard Microsoft Word files (.docx). The software can also process texts inside of multiple

spreadsheet formats, including those saved as Comma Separated Values (CSV) or Microsoft

Excel (.xlsx) format. The default LIWC-22 dictionary can be run in two ways: 1) via the standard

Graphical User Interface (GUI), as has been the case with all previous versions of LIWC, and 2)

via a Command Line Interface (CLI) from your command line, or from other platforms that can

interface with your system’s command line (for example, R or Python). LIWC-22 is compatible

with both PC and Mac platforms.

LIWC-22 Development Manual Page 3

During operation, the LIWC-22 processing module accesses each text in your dataset, compares

the language within each text against the LIWC-22 dictionary. Like all previous versions of

LIWC, the LIWC-22 text processing module works by counting all of the words in a target text,

then calculating the percentage of total words that are represented in each of the LIWC

subdictionaries. After processing each text sample, the LIWC processing module writes the

output to a data table that can be exported in several formats of your choosing, including

spreadsheets (e.g., CSV for MS Excel), a newline-delimited JSON, and so on. As described in

more detail below, the LIWC output includes the file name, total word count, and the percentage

of words that were captured in the text for each language dimension.

Companion Processing Modules

All previous versions of LIWC have been defined largely by their central feature: the LIWC

basic processing module. An important distinguishing characteristic of LIWC-22 is that the

program includes a group of companion processing modules that provide additional analytic

methods for language researchers. These additional features provide new ways to analyze and

understand your text samples. These features include the following:

Dictionary workbench

Historically, creating custom dictionaries was a tedious, esoteric process that only a

conscientious programmer could love. The dictionary workbench makes it possible for users to

build a dictionary using a simple point-and-click interface, accompanied by a built-in error-

checking tool. When completed, users can evaluate the psychometric properties of their

dictionary — namely, Cronbach’s α — on a dataset of their choosing. Note that the underlying

format of the LIWC-22 dictionary is considerably more flexible and dynamic than past versions

of LIWC. Nevertheless, older LIWC-formatted dictionary files can be imported and seamlessly

converted into the new dictionary format.1

Word frequencies and word clouds

When analyzing a new data set, it is sometimes helpful to see a word frequency table to

determine what words are most commonly used. In addition to word frequencies, LIWC-22 can

build a word cloud that can be saved in an image format.

Topic modeling with the Meaning Extraction Method

The traditional LIWC program was built on the assumption that many of the most important

social and psychological dimensions of language could be found through the analysis of function

words (e.g., pronouns, prepositions) and emotion words. Sometimes, however, it is important to

know the content of what people are saying in addition to their linguistic styles. Although there

are various types of topic modeling, LIWC-22’s topic modeling feature was built in

consideration of a psychologically informed method: the Meaning Extraction Method (MEM;

Chung & Pennebaker, 2008). The MEM allows users to subject their text analyses to factor

analyses (SVD, or other methods) to discover dominant themes and meaning in their dataset.

1 Note that the older LIWC dictionary format (.dic) can be converted into the newer format (.dicx) but the new

cannot be converted into the old format.

LIWC-22 Development Manual Page 4

Narrative arc

Language researchers have struggled with ways to devise computerized methods to analyze

narrative structure. Recent work on the Arc of Narrative points to three underlying processes that

are shared in most stories: staging, plot progression, and cognitive tension (Boyd, Blackburn, et

al., 2020). The narrative arc module automatically assesses texts for how each narrative structure

“unfolds” throughout the story, providing corresponding graphs and metrics that reflect the

degree to which each text resembles a normative narrative shape.

Language style matching

When describing the “style” of someone’s language, we typically are basing our evaluation on

the ways in which authors use function words (e.g., pronouns, articles, auxiliary verbs,

prepositions, and other short, common words). The Language Style Matching (LSM) module

compares the language style among different texts. LSM analyses can provide a metric of how

similarly two people talk with each other in dyads or larger groups, whether they write in similar

styles, or more broadly how similarly two or more groups may be writing or thinking alike

(Ireland & Pennebaker, 2010).

Contextualizer

When trying to understand what words can tell us about people, context is crucial (Boyd &

Schwartz, 2021). Often, we might have a sense that certain types of words are being used a

certain way, or that they reflect a particular type of psychosocial process. However, it is

important — crucial, even — that we critically examine our assumptions about word use. Often,

the best way to do this is to look at how words are being used in their broader context; this

approach is often referred to as “keywords in context.” The Contextualizer module offers a

simple way to extract words and their immediate context. For example, if we want to better

understand what the word “love” is conveys in a dataset, we can extract a set number of words

before and after each appearance of the word “love” in our texts. From there, it is possible to

simply inspect the usage of the word or, better yet, come up with a text analysis method (within

LIWC-22, perhaps?) to analyze the words surrounding its usage.

Case studies

All previous versions of LIWC were written for researchers who typically analyzed large

numbers of text files. Many users, however, have wanted to be able to simply dive into a single

text to understand it more deeply through a close analysis. The Case Studies module essentially

brings the other modules into a single location to allow users to focus on and explore a single

text. While other people are out climbing mountains, dancing at the club with their soul mates, or

enjoying expensive designer drugs with movie stars, you can bask in the cold, sterile glow of

your computer screen, obsessively analyzing the narrative structure of email from a coworker or

LIWCing the dialogue from your favorite episode of Rick and Morty.

Prepare transcripts

A new module has been added that helps users to clean conversation transcripts to run different

types of LIWC analyses.

LIWC-22 Development Manual Page 5

The LIWC-22 Dictionary and its Development

The LIWC-22 Dictionary is the heart of the text analysis strategy. The internal dictionary is

composed of over 12,000 words, word stems, phrases, and select emoticons. Each dictionary

entry is part of one or more categories, or subdictionaries, designed to assess various

psychosocial constructs. For example, the word cried is part of 10-word categories: affect,

tone_pos, emotion, emo_neg, emo_sad, verbs, focuspast, communication, linguistic, and

cognition. Hence, if the word cried is found in the target text, each of these 10 subdictionary

scale scores will be incremented. Most, but not all, of the LIWC-22 categories are arranged

hierarchically. All sadness words, by definition, belong to the broader “emo_neg”, “emotion”,

“tone_neg”, as well as the overall “affect” category. Note too that word stems can be captured by

the LIWC-22 system. For example, the dictionary includes the stem hungr* which allows for any

target word that matches the first five letters to be counted as “food” word (including hungry,

hungrier, hungriest). The asterisk, then, denotes the acceptance of all letters, hyphens, or

numbers following its appearance.

Each of the default LIWC-22 categories is composed of a list of dictionary words designed to

capture that dimension. The selection of words that make up the categories has involved multiple

steps. When LIWC was first conceived, the idea was to identify a group of words that tapped

into basic emotional and cognitive dimensions often studied in social, health, and personality

psychology. As our understanding of the psychology of verbal behavior has matured, the breadth

and depth of word categories in the LIWC dictionary has expanded considerably.

For LIWC-22, we have completely rebuilt the text processing engine, including the flexibility of

LIWC-formatted dictionaries. Dictionaries can now accommodate numbers, punctuation, short

phrases, and even regular expressions. These additions allow the user to read "netspeak"

language that is common in Twitter and Facebook posts, as well as SMS (short messaging

service, a.k.a. “text messaging”) and SMS-like modes of communication (e.g., Snapchat, instant

messaging). For example, "b4" is coded as a preposition and ":)" is coded as a positive tone

word.

In this latest version of LIWC, several new categories have been added, others overhauled

considerably, and a small number have been removed. With the advent of more powerful

analytic methods and more diverse language samples, we have been able to build more

internally-consistent language dictionaries with enhanced psychometric properties, in general.

This means that many of the dictionaries in previous LIWC versions may have the same name,

but the words making up the dictionaries have been altered (categories subjected to major

changes are described in a later section).

LIWC-22 Development Manual Page 6

Development of the LIWC-22 Dictionary

The construction of the LIWC dictionaries has significantly evolved over the years. Earlier

iterations of the LIWC dictionary relied extensively on large groups of human raters. With

increasing computational power, however, more recent versions have depended on establishing a

harmony between the domain expertise and knowledge of human raters and sets of increasingly

complex algorithms and statistical models. Below, we present an overview of the process used to

create the LIWC-22 dictionary.

Step 1. Word Collection. In the design and development of the LIWC category scales, sets of

words were first generated for each conceptual dimension, using the LIWC2015 dictionary as a

starting point. Within the Psychological Processes category, for example, the “affect”

subdictionaries were based on words from several sources, including previous versions of the

LIWC dictionary. We drew on common emotion rating scales, such as the PANAS (Watson et

al., 1988), Roget’s Thesaurus, and standard English dictionaries. Following the creation of

preliminary category word lists, 3-4 judges individually generated word lists for each category,

then held group brainstorming sessions in which additional words relevant to the various

dictionaries were generated and added to the initial lists. Similar schemes were used for the other

subjective dictionary categories.

Step 2. Judge Rating Phase. Once the grand list of words was amassed, each word in the

dictionary was independently examined by 3-4 judges and qualitatively rated in terms of

“goodness of fit” for each category. In order for a word to be retained in a given category, a

majority of judges had to agree on its inclusion. In cases of disputes, judges individually and

jointly inspected several corpora and online sources to help determine a word’s most common

use, inflection, and psychological meaning. Words for which judges could not decide on

appropriate category placement were removed from the dictionary.

Step 3. Base Rate Analyses. Once a working version of the dictionary was constructed from

judges’ ratings, the Meaning Extraction Helper (MEH; Boyd, 2018) was used to determine how

frequently dictionary words were used in various contexts across a large, diverse corpus of texts:

we refer to this as the “Test Kitchen” corpus. The Test Kitchen corpus contains 15,000 texts

from a diverse set of 15 corpora, including blog posts, spoken language studies, social media,

novels, student writing, and several others; we discuss this corpus in much greater detail in a

later section. Most relevant to this section, however, is that these analyses were used to root out

dictionary words that did not occur at least once across multiple corpora.

Step 4. Candidate Word List Generation. The 5,000 most frequently-used words in the Test

Kitchen corpus were identified. Of these 5,000 words, those that were not already in the LIWC

dictionary were considered candidates for inclusion. For several linguistic categories (e.g., verbs,

adjectives), Stanford’s CoreNLP and custom-made analytic software was used to identify high

base rate exemplars that were treated as candidates for inclusion (see: Boyd, 2020; Manning et

al., 2014). All candidate words were then correlated with all dictionary categories in order to

identify common words that were 1) not yet included in the dictionary, and 2) showed acceptable

conceptual and statistical fit with existing categories. Words that correlated positively with

dictionary categories were added to a list of candidate words for possible inclusion. All candidate

words were reviewed by teams of 3-4 judges who voted on 1) whether words should be included

in the dictionary and 2) whether words were a sound conceptual fit for specific dictionary

categories. Judges’ rating procedures were parallel to those outlined in Step 2. Finally, the four

LIWC-22 Development Manual Page 7

authors (RLB, AA, SS, and JWP) jointly worked on evaluating each word in randomly-assigned

teams of two to determine whether they should be cross-categorized into other LIWC-22

categories.

Step 5. Psychometric Evaluation. Following all previously-described steps, each language

category was separated into its constituent words. Each word was then quantified as a percentage

of total words. All words for each category were used to compute internal consistency statistics

for each language category as a whole. Words that were detrimental to the internal consistency of

their overarching language category were added to a candidate list of words for omission from

the final dictionary. A group of 4 judges (the authors of this document) then reviewed the list of

candidate words and voted on whether words should be retained. Words for which no majority

could be established were omitted.2 Several linguistic categories, such as pronouns and

prepositions, constitute established linguistic constructs and were therefore not a part of the

omission process.

Step 6. Refinement Phase. After Steps 1-5 were complete, they were repeated in their entirety.

This was done to catch any possible mistakes/oversights that might have occurred throughout the

dictionary creation process. The psychometrics of each language category changed negligibly

during each refinement phase. During the last stage of the final refinement phase, all four judges

reviewed the dictionary in its entirety for mistakes.

Step 7. Addition of Summary Variables. In addition to standard LIWC dimensions based on

percentage of total words, four summary variables were calculated: analytical thinking

(Pennebaker et al., 2014), clout (Kacewicz et al., 2014), authenticity (M. L. Newman et al.,

2003), and emotional tone (Cohn et al., 2004). Each summary variable builds upon previously-

published research from our lab; measures are calculated, then converted to percentiles based on

standardized scores from large comparison corpora. The summary variables are the only non-

transparent dimensions in the LIWC-22 output. The summary measures have been adjusted

against new norms but are conceptually consistent with the scores calculated in LIWC2015.

2 Worry not: the authors remain very good friends to this day.

LIWC-22 Development Manual Page 8

LIWC-22: Establishing the Psychometrics

From the beginning, the top priority of creating LIWC has been to build a scientifically sound

system that is both reliable and valid. For each iteration of LIWC, the dictionaries have been

modernized to try to keep up with subtle (and not-so-subtle) shifts in language. At the same time,

the world of text-based data science has grown exponentially, providing new methods and data

that facilitate increasingly well-validated versions of the dictionaries. For LIWC-22, we have

been able to build a large text corpus that includes traditional and contemporary English

language samples across multiple contexts. This “Test Kitchen” corpus was used for multiple

purposes in the creation and testing of the LIWC-22 dictionary, ranging from word selection to

the assessment of the dictionaries’ reliability and validity.

The Test Kitchen Corpus

The assessment of any text analysis system requires a large set of text samples drawn across

multiple authors and contexts. Several impressive corpora exist, including archives from Twitter,

Facebook, Reddit, movie transcripts, Wikipedia, the British National Corpus, Project Gutenberg,

and, for researchers associated with Google, just about anything ever posted on the web. The

challenge for psychological researchers, however, is to assemble a large array of texts that

broadly represent the ways words are used by everyday people in everyday life.

In previous versions of LIWC development, we relied on whatever datasets we had collected or

could find. For LIWC-22, we sought to build a curated corpus that would broadly represent the

many ways in which language is used. Once built, we could rely on the corpus as a “test kitchen”

to both quantify and qualify our LIWC-22 dictionary and, at the same time, obtain estimates

about the context-dependence of verbal behavior (insofar as word use reflects a certain class of

verbal behavior as well as social behavior and psychologically meaningful behavior more

broadly defined).

The Test Kitchen corpus was constructed from randomly selected subsets of text from across 15

different types of English language sets. The original datasets included thousands, sometimes

millions of writings or transcribed speech samples, including blogs, emails, movie dialog, social

media posts, natural conversations, etc. Some of the data repositories were collected by our or

other labs, others came from public archives. From each of the 15 data sets, we randomly

selected 1,000 text samples with a minimum of 100 words. For any texts with more than 10,000

words, an algorithm was written to select 10,000 continuous words from a random starting point

in the document. As can be seen in Table 1, the Test Kitchen corpus includes 1,000 texts from

each of the 15 different sources, for a total of 15,000 texts, each averaging over 2,000 words. The

overall word count of the entire corpus is over 31 million words. To the degree possible, all

personally identifying information was stripped.

Note that for most corpora, single texts reflected the writings from a single person. For example,

each text from the Blog or Email corpus included multiple blog entries or emails from the same

person. For additional information on the Test Kitchen Corpus, see Appendix A. Due to the

nature of several of the data sources, the Test Kitchen corpus is not readily available for research

use and cannot be made publicly available.

LIWC-22 Development Manual Page 9

Table 1. The Test Kitchen Corpus of 31 Million Words

Corpus Description

Word Count M

(SD)

Applications

Technical college admissions essays

1506 (501)

Blogs

Personal blogs from blogger.com

2144 (1920)

Conversations

Natural conversations

614 (495)

Enron Emails

Internal emails from Enron

316 (376)

Facebook

Facebook posts from mypersonality.com

2195 (2034)

Movies

Transcribed movie dialogue

6633 (2459)

Novels

Novels from Project Gutenberg

5703 (189)

NYT

New York Times articles

744 (494)

Individuals’ Reddit comments

1751 (1945)

Short Stories

Short stories

2977 (2211)

SOC

Stream of consciousness essays

656 (256)

Speeches

U.S. Congressional speeches

950 (1241)

TAT

Thematic Apperception Test, online website

326 (63)

Tweets

Collected tweets from individual accounts

4442 (2858)

Yelp

Restaurant reviews posted to Yelp

99 (1)

Overall mean

2070 (2466)

Note: Each corpus is composed of 1,000 texts, each originally between approximately 100 to 10,000

words. After data collection, some texts became slightly smaller or larger because of data cleaning

procedures. For a more detailed description of the Test Kitchen Corpus, see Appendix A.

LIWC-22 Development Manual Page 10

Quantifying the Reliability and Validity of LIWC-22

Assessing the reliability and validity of text analysis programs is a tricky business. One might

reasonably assume that it is psychometrically acceptable to calculate the internal consistency of a

LIWC category in the same way as one might do with a self-report questionnaire; this

assumption would be wrong. Consider the fact that a questionnaire that taps into the construct of

anger or aggression, for example, typically asks participants to respond to a number of questions

about their feelings or behaviors related to anger, in general. Reliability coefficients, then, are

computed by correlating people’s responses to the various questions. The more highly they

correlate, the reasoning goes, the more the questionnaire items all measure the same thing. Voila!

The scale is deemed to be internally consistent, and therefore reliable.

A similar strategy can be used with words. But be warned: the psychometrics of natural language

are not as straight-forward as with questionnaires. The reason becomes deceptively obvious to

those who dedicate their lives to the study of verbal behavior. Once you say something, you

generally do not need to say it again in the same social media post, essay, or conversation. The

nature of discourse, then, is we usually say something and then move on to the next thought or

topic. Repeating the same idea over and over again is not the rule in verbal behavior, but the

exception. Yet, this repeated, concurrent sampling of a construct is the foundation of self-report

questionnaire design. It is important, then, to understand that acceptable boundaries for natural

language reliability coefficients are lower than those commonly seen elsewhere in psychological

tests. Put simply: behavior — particularly verbal behavior — has very different psychometric

properties than more “broad strokes” assessments of human psychology commonly used

elsewhere.

The LIWC-22 Anger scale, for example, is made up of 181 anger-related words, word stems, and

phrases. In theory, the more that people use one type of anger word in a given text, the more they

should use other anger words in the same text. To test this idea, we can determine the degree to

which people use each of the 181 anger words across a select group of text files and then

calculate the intercorrelations of the word use. In order to calculate these statistics, each

dictionary word was measured as a percentage of total words per text (Cronbach’s α) or,

alternatively, in a binary “present versus absent” manner (Kuder–Richardson Formula 20; Kuder

& Richardson, 1937). The scores were then entered as an “item” in a standard internal

consistency calculation, providing internal consistency metrics across all corpora.

Both Kuder–Richardson Formula 20 and the raw, unstandardized Cronbach’s α metric are

presented in Table 2; both metrics were derived from the entire Test Kitchen corpus.

Importantly, the traditional Cronbach’s α method, calculated from relative word frequencies,

tends to sorely underestimate reliability in language categories due the highly variable base rates

of word usage within any given category. The Kuder–Richardson Formula 20 should generally

be considered to be a more “true” approximation of each category’s true internal consistency.

LIWC-22 Development Manual Page 11

Table 2. LIWC-22 Language Dimensions and Reliability

The Development and Psychometric Properties of LIWC-22

Abstract