Conference PaperPDF Available

Within and Between-Person Differences in Language Used Across Anxiety Support and Neutral Reddit Communities

Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, pages 182–193
New Orleans, Louisiana, June 5, 2018. c
2018 Association for Computational Linguistics
Within and Between-Person Differences in Language Used Across Anxiety
Support and Neutral Reddit Communities
Molly E. Ireland and Micah Iserman
Department of Psychological Sciences, Texas Tech University, Lubbock, Texas
Although many studies have distinguished be-
tween the social media language use of people
who do and do not have a mental health con-
dition, within-person context-sensitive com-
parisons (for example, analyzing individuals’
language use when seeking support or dis-
cussing neutral topics) are less common. Two
dictionary-based analyses of Reddit communi-
ties compared (1) anxious individuals’ com-
ments in anxiety support communities (e.g.,
/r/PanicParty) with the same users’ comments
in neutral communities (e.g., /r/todayilearned),
and, (2) within popular neutral communities,
comments by members of anxiety subreddits
with comments by other users. Each compari-
son yielded theory-consistent effects as well as
unexpected results that suggest novel hypothe-
ses to be tested in the future. Results have rel-
evance for improving researchers’ and practi-
tioners’ ability to unobtrusively assess anxiety
symptoms in conversations that are not explic-
itly about mental health.
1 Introduction
Approaches to automatically identifying general
psychological distress or specific mental health
conditions tend to focus on between-person com-
parisons, often including yoked controls that are
matched on demographic characteristics (Copper-
smith et al.,2016;Smith et al.,2017). Particularly
in the area of computational linguistics, which his-
torically has focused more on prediction or classi-
fication than psychological insight (cf. Schwartz
et al.,2013), within-sample variance due to differ-
ences in communicative contexts is typically ig-
nored. Such differences (for example, in how in-
dividuals who are distressed talk when they are
seeking support versus having conversations that
are irrelevant to mental health) may wash out in
sufficiently large text samples; likewise, a com-
mon research aim is to classify a person’s men-
tal health condition or distress level accurately in
the absence of contextual information, given that
such information is frequently unavailable (Cop-
persmith et al.,2015;Schwartz et al.,2016). When
within-person analyses—comparing a person with
themselves, versus matched controls—have been
carried out in computational linguistics, the aim
has typically been to identify change points over
time or temporal patterns that precede important
events, such as suicide attempts or panic attacks
(Benton et al.,2017;Coppersmith et al.,2016;
De Choudhury et al.,2016;Loveys et al.,2017).
It is clearly useful to be able to recognize dis-
tress or clinically relevant changes in situations
where contextual data is absent or sparse. How-
ever, when details about the communicative con-
text are available, understanding how individuals’
goals and the social context influence language use
may be valuable in interpreting linguistic signals
more accurately. For example, using language to
identify mental health conditions or classify symp-
tom severity (i.e., triage) in support settings, such
as crisis support forums, may be very different
from attempting the same classification in every-
day conversations about topics other than mental
health (Friedenberg et al.,2016).
Research in psychology supports the premise
that certain emotions, personality traits, and men-
tal health symptoms manifest differently across
various settings, with negative affective traits be-
ing virtually invisible in many situations (Ireland
and Mehl,2014;Mehl et al.,2012). For exam-
ple, in transcripts of naturalistic recordings of stu-
dents’ everyday lives, neuroticism correlated with
increased physical activity for men and decreased
verbosity and laughter for women, with no other
linguistic correlates for either sex (Mehl et al.,
2006). Neuroticism—described by Jack Block as
“an overinclusive, easy-to-invoke, societally eval-
uative wastebasket label” (Block,2010, p. 9)—is
the Big Five trait that is typically the least legible,
or most difficult to reliably and accurately detect
in verbal or nonverbal behavior (Tskhay and Rule,
2014). Neuroticism is characterized by vulnerabil-
ity to stress and negative affect, including depres-
sion, anxiety, and irritability (John and Srivastava,
There are two main reasons for the difficulty of
detecting neuroticism in everyday social interac-
tions. First, expressing negative affect publicly is
often non-normative or socially undesirable. That
is, people tend to dislike and avoid negativity—
particularly sadness (Tiedens,2001). Separately,
neuroticism involves internalizing emotions such
as anxiety and sadness (Zahn-Waxler et al.,2000),
which are directed inward and do not require the
involvement of other people (in contrast with other
Big Five facets, such as gregariousness or con-
formity). As a result of these characteristics,
even people ranking high in neuroticism will of-
ten avoid verbalizing their negative thoughts and
feelings in public (e.g., conversations at work) and
reveal those traits through negative emotional lan-
guage only in private (e.g., diaries; Holleran and
Mehl,2008;Jarrold et al.,2011;Mehl et al.,2006,
Avoiding self-disclosures of sadness or anxiety
may be particularly common among men (Nadeau
et al.,2016), given that men are discouraged
from expressing emotion in most cultures (Garside
and Klimes-Dougan,2002), and negative affect
or neuroticism is more normative among women
(Schmitt et al.,2008). For both sexes, strategically
suppressing or masking negative affect in order to
avoid social censure may present a barrier to cop-
ing with psychological distress, given that disclos-
ing negative emotions is a critical step in seeking
social support (Davison et al.,2000;Taylor et al.,
Building on personality research on how neu-
roticism manifests across public and private con-
texts (Mehl et al.,2012), we are specifically in-
terested in how individuals may suppress indica-
tors of negative affect (anxiety, sadness, or irrita-
tion) as they move from talking in support-seeking
settings—where presumably expressing negative
affect is more normative—to neutral settings. As
a test case, we analyzed users in subreddit com-
munities for general anxiety, social anxiety, health
anxiety, and panic disorder.
We focused on anxiety because it is enormously
common, has severe consequences for individuals’
well-being and health, and has been overlooked,
relative to depression, in studies of language and
clinical psychology. Several studies have investi-
gated anxiety in concert with other disorders (Cop-
persmith et al.,2014,2015;Gkotsis et al.,2017),
but studies that focus on a single condition more
commonly focus on depression (De Choudhury
et al.,2013; for a review, see Conway and OCon-
nor,2016). Worldwide, anxiety is the second most
prevalent mental health condition and, among all
mental disorders, accounts for the second greatest
variance in disability-adjusted life years (White-
ford et al.,2013). Anxiety is frequently comorbid
with depression (Sartorius et al.,1996), the pri-
mary cause of suicidality, but contributes unique
variance to the prediction of suicide attempts and
deaths by suicide (Khan et al.,2002).
Past research on the linguistic indicators of anx-
iety on social media has shown that anxious indi-
viduals’ language use resembles the more general
distress pattern observed in other mental health
conditions (particularly depression) and neuroti-
cism (Resnik et al.,2013,2015). This pattern
includes more references to negative affect (par-
ticularly anxiety words for anxious individuals),
greater self-focus, more tentativeness, more refer-
ences to health, and, in some cases, more socially
distant language, relative to average (Coppersmith
et al.,2014,2015;Resnik et al.,2013,2015).
We selected Reddit for analysis because of its
large base of daily active users and broad range of
well-defined, active communities (or subreddits)
on both mental health and other topics (Barthel
et al.,2016). Subreddits are defined by clear de-
scriptions and rules. For example, the sidebar of
one anxiety support forum states, “Welcome to
/r/PanicParty. This subreddit is intended to be a
place of help and support for those suffering from
anxiety and panic disorders. As a result, at least
for the more narrowly defined mental health com-
munities, subreddits comprise relatively coherent
groups of people who all assert that they have
the symptoms described in the group’s rules. Al-
though not all commenters will be suffering from
the anxiety symptoms they are discussing at the
time of posting, there is an expectation that com-
munity members have experienced anxiety them-
selves and are not participating solely in an ex-
pert (or voyeuristic) capacity. Because the same
Reddit users often post in both mental health sup-
Subreddit Anxiety Comparison
AskReddit 5746 5901
relationships 580 571
politics 506 1096
Advice 382 173
funny 378 749
pics 361 763
aww 351 443
news 326 743
AskWomen 322 426
worldnews 312 623
todayilearned 273 473
gaming 253 548
Showerthoughts 239 464
CasualConversation 238 530
videos 194 509
gifs 177 232
Fitness 159 232
SkincareAddiction 123 331
teenagers 70 180
actuallesbians 69 115
Table 1: Number of posts in non-anxiety forums by
each group.
port forums and general forums about neutral top-
ics (such as /r/AskReddit or /r/IamA), Reddit al-
lows for within-person same-site comparisons that
would not be possible in most other online anxi-
ety support communities (such as 7 Cups1or Dai-
Reddit is a popular news sharing and social me-
dia site used by 4-6% of adult internet users (Dug-
gan and Smith,2013). Its users are approximately
67% male, and 64% of all Reddit users are be-
tween 18 and 29, based on recent Pew research
(Barthel et al.,2016). Given that concealing nega-
tive emotions may be a particular concern among
men (Nadeau et al.,2016), and given the relatively
low participation of men in most psychology con-
venience samples, the possibility of oversampling
male users may be a benefit rather than a limitation
of Reddit analyses. Furthermore, the site’s use of
upvotes and downvotes (or “karma”) tends to dis-
courage most everyday users—that is, people not
using dedicated “trolling” accounts—from behav-
ing more antisocially than they would in real life
(Barthel et al.,2016;Chen and Wojcik,2016).
The following study analyzes naturalistic lan-
guage use on Reddit to ask two simple, ex-
ploratory research questions: (1) In a within-
person analysis, how do individuals use language
differently in mental health support forums ver-
sus neutral contexts? (2) In a between-person
analysis, do anxiety forum members and compar-
ison users who do not belong to anxiety forums
talk differently when posting in subreddits that
are not explicitly about mental health? We ex-
plored both question across all available Linguis-
tic Inquiry and Word Count (LIWC; Pennebaker
et al.,2015) categories, with special attention to
the categories that have previously served as in-
dicators of anxiety, and commonly used individ-
ual words (Coppersmith et al.,2015). Our aim
is to produce insights that will be useful in clin-
ical practice, particularly for clinicians interested
in monitoring clients between sessions or on an
outpatient basis after a health crisis, such as a sub-
stance use relapse or suicide attempt.
2 Method
We collected three sets of text from two groups
of users. For the anxiety group, we col-
lected the recent activity of members of six
anxiety-related subreddits (or forums; /r/Anxiety,
/r/HealthAnxiety, /r/PanicAttack, /r/panicdisorder,
/r/PanicParty, and /r/socialanxiety). The member-
ships of these forums vary, with /r/Anxiety and
/r/socialanxiety having over 80,000 members, and
the rest under 3,000 members. From this sample of
anxiety-poster activity, we identified the 20 most
common non-anxiety-related forums, then identi-
fied a sample of users who posted in those com-
mon forums but not in any anxiety-related forums
(referred to as comparison users; Table 1).
We collected and processed texts in R (R Core
Team,2018), using the jsonlite (Ooms,2014) and
RedditExtractoR (Rivera,2015) package. The ini-
tial scraping of the anxiety related subreddits re-
sulted in 2,636 replies from 1,423 unique users.
From each user’s profile, we collected 100 of their
most recent replies and dropped anyone with fewer
than 50 words in anxiety-related forums. This
left 28,154 replies from 1,409 unique users. We
then combined the text from each user by con-
text (anxiety versus non-anxiety-related forums)
and processed the texts with LIWC (Pennebaker
et al.,2015). We also translated texts into a
document-term matrix for word-level analyses,
which involved some cleaning to better identify
overwhelmingly domain
immaturity astronaut
wraps leggings
attacks heightened
Figure 1: 100 words or marks with the largest log-odds
when predicting the given source. Size and color are
determined by scaled log-odds.
word boundaries, and case standardization. Fi-
nally we excluded users with fewer than 50 words
in either their non-anxiety set, which resulted in
15,516 replies from 523 unique users.
For the comparison sample, we first collected
up to 102 of the most popular threads from each of
the 20 non-anxiety-related forums. After exclud-
ing content from the anxiety-posting users, this
resulted in 139,680 replies from 73,976 unique
users. From this potential set of users, we aimed
to identify a sample similar to the anxiety-posting
users, in terms of their non-anxiety-related forum
activity. To do this, we drew random users from
the potential set of users one by one; if the user
had more than 50 words across their replies, we
looked at which non-anxiety-related forums they
posted in. If including the user would not increase
the Canberra distance3between the anxiety post-
ing sample and the comparison by .044or more,
they would be added to the comparison set. This
was done until there were 523 users included in
the comparison group. The resulting sample in-
cluded 15,102 replies aggregated into 1,046 texts
(two per user), which, in terms of percentages of
subreddits, is .189 Canberra distance (.989 Pear-
son’s r) from the anxiety-poster sample. The final
dataset included 1,569 texts, with 523 from each
source (anxiety, non-anxiety, and comparison).
3 Results and Discussion
3.1 Full Sample Analyses
To capture an initial picture of the data, we con-
structed word clouds based on logistic regressions
(calculated for each word, with the word and the
percent of each user’s political posts predicting
each source separately; Figure 1), and fit a deci-
sion tree (using the rpart package; Therneau and
Atkinson,2018) to the entire dataset (Figure 2).
The decision tree’s predictions matched the real
sample 68% of the time; that is to say, knowing the
values of the anx, shehe, and netspeak LIWC cat-
egories, what percentage of words in the text were
captured by LIWC (Dic), and the frequency of that
and know, you could use these rules to appropri-
ately categorize the texts 68% of the time, within
this sample. Both of these visualizations give
3Mean of |ab|/a +b, where aand bare the subreddit
percentage vectors.
4This is the value we found to be inclusive enough to al-
low a sufficient number of comparisons in with minimal harm
to the comparison makeup.
LIWC_anx >= 0.042
LIWC_Dic >= -0.38
LIWC_shehe < -0.69
LIWC_netspeak >= -0.12
that >= 0.32
know >= 0.58
.33 .33 .33
.81 .05 .14
.11 .46 .42
.18 .34 .48
.50 .21 .29
.08 .38 .54
.05 .51 .44
.07 .70 .23
.04 .43 .53
.04 .80 .16
.04 .35 .61
.10 .28 .62
.01 .65 .34
yes no
Figure 2: Decision tree fit to the full dataset, predicting source sample. Each colorful node is named for the
dominant class, with the probability of each class and the percent of total sample underneath. The split values
under each node are z-scores.
anxiety comparison non-anxiety
anxiety .922 .113 .224
comparison .017 .662 .315
non-anxiety .061 .226 .461
Table 2: Confusion matrix comparing actual text
source to that predicted by the decision tree fit to the en-
tire sample. Cells show probability of predicted source
(rows) given actual source (columns), with columns
summing to 1.
the similar impression that, at minimum, anxiety-
related words characterize the texts from anxiety-
related forums. Table 2breaks down the decision
tree’s accuracy to make this point again; 92% of
the anxiety posts were accurately classified, com-
pared with 66% of the comparison texts, and only
46% of the non-anxiety texts (non-anxiety posts
from anxiety posters).
The word clouds in Figure 1are bound to be
somewhat specific to the users that we sampled
and may not generalize well to new data; never-
theless, they provide a vivid snapshot of the con-
tent of each sample, and some patterns in these
word-level correlates fit with past research on anx-
iety disorders. Figure 1shows that anxiety users’
neutral posts are characterized by references to
unpleasant aspects of relationships (separating,
doormat) or other people (immaturity, pestering),
counterbalanced to a degree by a few positive af-
fective words (wellbeing, masterpiece, hugged).
The same group of posts seemed to use more
moral words than the comparison or anxiety forum
posts, with terms that may reflect concerns about
harm (humane, wronged), subversion or question-
ing authority (denies, dissent), and perhaps unfair-
ness or injustice (gays, inmates, interracial, greed;
Graham et al.,2009). In contrast, the comparison
posts seemed to discuss social injustice in a less
personal or more analytic way (indictment, coun-
sel, Vladimir).
There were a few commonalities between words
used in neutral and anxiety support forums by
anxiety forum members. Echoing past find-
ings concerning anxious individuals’ greater use
of LIWC’s health category on Twitter (Copper-
smith et al.,2015), health references were more
common in anxiety users’ posts in both neutral
(nurses, overdosing) and anxiety forums (meds,
strokes). References to specific symptoms (pal-
pitations, hyperventilating), medications (propra-
nolol, mirtazapine), and behavioral coping strate-
gies (mindfulness, meditation) were more com-
mon in anxiety support forums. Although posts
in anxiety forums do refer to anxiety more of-
ten and more specifically than the two comparison
samples (panic, nervousness, spiraling), anxiety
users’ posts in neutral forums were also character-
ized by broader negative affective terms, such as
curse and bawling. Finally, anxiety forums, rela-
tive to the two comparison samples, used higher
rates of psychological terms that are not nec-
essarily unique to the etiology or treatment of
anxiety—including stressor, subconsciously, and
amygdala— perhaps reflecting users’ research on
or knowledge about psychology more broadly.
Category/Word log-odds z p
Intercept .071 .881 .378
anxiety .945 3.736 .000
notice .501 3.094 .002
LIWC: Dic .366 2.663 .008
parents .234 2.386 .017
though .204 2.429 .015
me .191 2.299 .022
sometimes .182 2.023 .043
LIWC: health .156 1.834 .067
LIWC: negemo .152 1.730 .084
LIWC: home .113 1.415 .157
was .107 1.019 .308
and .103 1.025 .305
LIWC: anx .064 .608 .543
LIWC: focuspast .064 .592 .554
depression .053 .421 .674
LIWC: conj .019 .175 .861
set -.064 -.838 .402
america -.129 -1.505 .132
LIWC: netspeak -.154 -1.819 .069
LIWC: funct -.165 -1.211 .226
panic -.183 -1.999 .046
LIWC: number -.198 -2.412 .016
asian -.228 -2.730 .006
russia -.253 -2.399 .016
guns -.258 -1.931 .053
punct: qmark -.277 -3.410 .001
lucky -.413 -2.423 .015
thank -.440 -4.208 .000
Table 3: Logistic regression in the between-person
analysis predicting user group (anxious or non-
anxious) within non-anxiety forums, with positive log-
odds associated with anxiety forum users.
Anxiety versus non-anxiety
anxiety non-anxiety
anxiety .960 .069
non-anxiety .040 .931
Non-anxiety versus comparison
non-anxiety comparison
non-anxiety .672 .351
comparison .328 .649
Table 4: Accuracies of within and between person re-
gression models broken down
3.2 Out-of-Sample Predictions
Next, we explored how predictable the texts’
source would be outside of the sample. To do
this, we randomly selected 174 users (1/3 of the
sample; keeping the number of texts from each
source about even) each from the anxiety-posting
sample and comparison sample to be held out for
testing, then used the remaining sample of 699
users for training. We considered most LIWC
categories (excluding percentile and punctuation
variables, which were processed manually) and all
unique words, making for 12,297 variables. From
here, we separated the data into a set only includ-
ing the anxiety and non-anxiety posts, and a set
only including the non-anxiety and comparison
posts. For each of these sets, we fit regularized
(elastic net, using the glmnet package; Friedman
et al.,2010) logistic regressions and decision trees,
both predicting each text’s source. We considered
both of these methods for their potential to reduce
the number of variables and thus make the results
more interpretable.
Within-Person Comparison. The first sample
we tested contained two sets of posts from each
user, with the goal of predicting which set of fo-
rums the given post was coming from (anxiety-
related or non-anxiety-related). To find the opti-
mal penalty parameter (α; affecting the smooth-
ness of weighting) for the model, we tested 5 val-
ues from 0 to 1 (considering L1and L2regular-
ization, and in between). The optimal weighting
parameter (λ; affecting the strength of weighting)
was selected by cross-validation within the train-
ing set. For the reported model, α=.25 and
Regularization left 216 variables with coeffi-
cients greater than 0. Among these, positive pre-
dictors of anxiety-related forums with the largest
coefficients were the dictionary (Dic; % of dic-
tionary words captured) and anxiety (anx) LIWC
categories. The positive predictors of non-anxiety
forums with the largest coefficients were the male,
sexual, and female LIWC categories, and the word
the. This model accurately classified 94.54% of
the test sample texts (Table 4).
In other words, when posting in anxiety forums,
people tended to use higher-frequency words and,
unsurprisingly, used words related to anxiety (e.g.,
scare, worried) more often. When the same peo-
ple moved to other non-anxiety-related forums,
they discussed men, women, and sex. Whether
this pattern represents masking (intentionally im-
itating Reddit norms in order to appear typical),
a type of disengagement coping (avoiding distress
through distractions), the anxiety forum members’
personalities when they are not feeling anxious, or
even the source of users’ anxiety itself is unclear
based on these data alone (Carver and Connor-
Smith,2010). Fitting a decision tree to the within-
person sample with the same outcome yielded
similar results (Figure 3), accurately classifying
83.82% of the texts in the test sample.
Between-Person Comparison. The between-
person analysis attempted to answer the poten-
tially more challenging question of how to distin-
guish anxiety forum members’ and others’ com-
ments in neutral forums. The same sort of reg-
ularized model was fit here in the same manner;
α=.25,λ=.199. This model accurately classi-
fied 66.09% of test sample texts (Table 4). Regu-
larization left 28 variables with coefficients over 0.
These are presented in Table 3, which also shows
the results of an unregularized logistic regression,
including only those variables, and fit to the en-
tire dataset. The decision tree for this set had an
out-of-sample accuracy of 60.01%.
Results showed that, relative to the compari-
son sample (people who were not members of
popular anxiety forums), anxiety subreddit mem-
bers posting in neutral forums used more common
words and more conjunctions (Coppersmith et al.,
2015), perhaps reflecting a simpler and more con-
versational (as opposed to analytical) writing style
(Pennebaker et al.,2014). Notably, anxiety fo-
rum members used more anxious language than
others even in neutral forums that were ostensi-
bly irrelevant to mental health. Finally, anxiety
forum members showed signs of being less social
than others, asking fewer questions (fewer whats,
fewer question marks) and thanking other posters
less often—perhaps reflecting social withdrawal,
which has been implicated in both the etiology and
maintenance of anxiety disorders (especially so-
cial anxiety; Rubin et al.,2009).
Finally, consistent with past findings regarding
neuroticism and anxiety, anxiety forum members
were more self-focused (more me) than compari-
son users (Tackman et al.,2018). That me and not I
predicted anxiety in this sample could indicate that
anxiety users’ self-focus specifically takes a more
passive or less agentic form, discussing events
or actions that that happened to them rather than
LIWC_anx >= 0.038
LIWC_Dic >= 0.69
LIWC_female < 0.19
yes no
Figure 3: Decision tree fit to the within-person test set
contrasting posts in anxiety and non-anxiety forums by
the same users. The number at the center of each node
is the probability of anxiety posters within that split.
65 70 75 80 85 90 95 100
Figure 4: Interaction between the social LIWC cate-
gory and the percent of words captured by LIWC.
their own actions or thoughts. Recent research has
examined psychological differences in subjective
and objective first-person singular pronouns (Iver-
sus me, respectively) in depression (Zimmermann
et al.,2017), finding that the objective me is more
indicative of depression than the subjective I. Our
results suggest that it may be worthwhile to revisit
the subjective vs. objective distinction in research
on anxiety as well.
Next, we explored the data more visually, with a
focus on LIWC variables of interest. For example,
Figure 4shows an interaction between the anxi-
ety and non-anxiety posts. Posts that are particu-
larly well captured by LIWC (Dic) but use very
few social words seem to be the main cause of
this interaction. Texts fitting this description seem
to describe experiences with anxiety and treat-
0 2 4 6 8 10 12
Figure 5: Interaction between the i and you LIWC cat-
ment, as in “I don’t know why, and I just wake
up like that sometimes. So far, breathing exercises
just makes it worse, but maybe I’m not doing it
right” (r/PanicAttack). Comments in non-anxiety-
related forums did not tend to involve this sort of
recounting, but it occurred occasionally; for ex-
ample, “This happened to me when I was in col-
lege. I was trying to sleep because I had to be up
early the next morning” (r/AskReddit). These are
well-captured and low in social language because
they are describing individuals’ thoughts and ex-
perience in a particular moment or sequence.
Finally, past LIWC research has demonstrated
the centrality of personal pronouns in understand-
ing how focus on oneself versus others relates
to personality and mental health (Tackman et al.,
2018). Figure 5shows the association between I
and you within each sample. A negative correla-
tion between first-person singular pronouns (i) and
second-person singular pronouns (you) is promi-
nent in the anxiety sample, which appears to be
most driven by texts with high Iuse and low
you use. Considering that approaching personal
challenges from a first-person rather than second-
person perspective tends to be associated with in-
creased psychological distress, the pronoun usage
of people posting in anxiety forums could repre-
sent a ruminative or otherwise suboptimal method
of seeking and providing support (Dolcos and Al-
barracin,2014;Kross and Ayduk,2011).
4 Future Work and Limitations
The aims of this study were to observe how
anxious individuals’ language use changes from
support-seeking to neutral settings, and investi-
gate whether those same anxiety-subreddit users’
language could be differentiated from others’ lan-
guage use in neutral forums. As a preliminary
proof-of-concept study, the present findings pro-
vide a foundation for future work on these top-
ics; however, our approach had several limitations.
First, after selecting only the 20 most common
neutral subreddits that anxiety community mem-
bers also posted in, and after excluding users who
did not use at least 50 words in each context, the
sample for the within-person comparison was rel-
atively small (N= 523). Future analyses may
hand code all 5,562 subreddits that the users in the
original anxiety sample also posted in, providing
a more nuanced portrait of how individuals with
anxiety post across popular and niche communi-
Partly due to the relatively small sample, we
primarily used a dictionary approach to analyzing
these texts. Because of their transparency, theory-
driven nature, and ease of use, they are more read-
ily disseminated to researchers outside of com-
putational linguistics (such as practicing clini-
cians) than more mathematically sophisticated or
data-driven natural language processing methods
(Tausczik and Pennebaker,2010). LIWC is also
arguably more appropriate than open-vocabulary
approaches in smaller samples (N<5,000), where
individual words or topics may not occur often
enough to be useful predictors (Schwartz et al.,
2016,2017). However, dictionary approaches also
have many acknowledged limitations. The LIWC
affect categories in particular can be difficult to
interpret without significant text cleaning that we
did not carry out in this study (e.g., disambiguat-
ing uses of like) and may not be reliably related
to self-reported positive emotions (Sun et al.,un-
der review). In dictionaries, it may also be unclear
whether the effect of an entire category is being
driven by one or a few relatively common words
(see Ireland et al.,2015).
In terms of psychological insights offered by
this study, a primary concern is whether the in-
dividuals in our sample are representative of other
individuals with anxiety. People commonly have
separate handles for different purposes in order to
provide some privacy, or use “throwaway” reddit
usernames when they wish to discuss personally
identifiable or intimate information on Reddit. It
may be relatively rare to share personal details re-
lating to mental health conditions (e.g., describing
recent panic attacks at work or childhood physi-
cal abuse) and then chat about less intimate topics
(e.g., video games or world news) in other sub-
reddits under the same username. In our sample,
only about one-third (37.12%) of the people who
used at least 50 words in anxiety subreddits also
used at least 50 words in popular neutral or non-
anxiety subreddits under the same name. By defi-
nition then, the people with sufficient text to ana-
lyze in both contexts are atypical, even for mem-
bers of Reddit anxiety forums. Speculatively, peo-
ple who are willing to use consistent usernames in
support-seeking and neutral contexts may be more
extraverted (John and Srivastava,1999), more ver-
bally disinhibited (Swann Jr and Rentfrow,2001),
or lower in self-monitoring (the tendency to al-
ter one’s behavior to fit social expectations; Ickes
et al.,1986), relative to an average person—all
characteristics that may limit the generalizabil-
ity of our results. More simply, they could have
milder anxiety symptoms (particularly for social
anxiety) or better overall mental health than those
who post only in mental health forums.
Along the same lines, the six anxiety commu-
nities that we sampled from do not provide full
coverage of all anxiety disorders; there are also
notable differences among the conditions those
subreddits represent. Panic disorder, social anxi-
ety, and generalized anxiety are in the same broad
category of the Diagnostic and Statistical Manual
of Mental Disorders 5 (Anxiety Disorders), but
those conditions have key differences in both eti-
ology and treatment (APA,2013). Future analyses
should determine whether changes in language use
from support-seeking to neutral contexts are simi-
lar across all mental health conditions that relate
to the experience of chronic negative affect, in-
cluding depression, PTSD, and bipolar disorder,
among many others. Within anxiety disorders as
well, it is unclear whether our results will gener-
alize to communities focusing on more narrowly
defined or less common conditions, such as spe-
cific phobias or agoraphobia.
Finally, by collapsing across posts, we sacri-
ficed granularity for parsimony. That is, for the
moment, we intentionally ignored a wealth of po-
tentially useful information about specific subred-
dits, time, upvotes, and thread structure. There is
clearly much more to be explored, particularly in
terms of social and temporal dynamics (see Cop-
persmith et al.,2016). For example, due to social
anxiety or simply the cognitive burden of inhibit-
ing negative emotions, anxiety users may be less
socially engaged—and therefore receive fewer up-
votes and replies—relative to controls when they
post in neutral communities. They also may post
more slowly, less often, or in atypical tempo-
ral patterns, relative to less anxious Reddit users
(Loveys et al.,2017).
5 Conclusion
Two sets of analyses explored how individuals’
language use changes from support-seeking to
neutral settings, and further demonstrated that
anxious individuals’ language use can be differen-
tiated from comparison posts even in neutral set-
tings, when the topics of conversation rarely fo-
cus on mental health. Results revealed not only
face-valid content differences (e.g., in references
to anxiety, negative affect, and social language),
but also subtler stylistic differences (e.g., in self-
focus, conjunctions, word frequency, and ques-
tions). Findings were largely consistent with past
research and existing theory (Coppersmith et al.,
2015;Mehl et al.,2012;Tackman et al.,2018),
while also suggesting novel data-driven hypothe-
ses to be tested in future research.
We are particularly encouraged by some of the
unexpected results (for example, regarding ques-
tion marks and thanks) that, despite not being di-
rectly predicted by past work, are nevertheless
consistent with research and theory on the nature
of anxiety. In terms of informing future behavior
change interventions, it may be especially valu-
able to identify behavior patterns in neutral set-
tings that maintain or exacerbate anxiety—for ex-
ample, being less interactive or positive even when
ostensibly engaging in prosocial behavior, such as
posting in discussion forums.
Information about the communication context
is typically unavailable in large-scale social media
classification tasks; however, clinicians or medi-
cal practitioners often operate at the level of indi-
vidual clients. In cases with abundant information
about the person and the context—for example,
when reviewing chat messages from online outpa-
tient therapy sessions (Wolf et al.,2010) or analyz-
ing clients’ social media messages between health
center visits (Padrez et al.,2015)—appreciating
how aspects of the situation influence the linguis-
tic signal of psychological distress may prove to
have near-future applied value.
APA. 2013. Diagnostic and statistical manual of men-
tal disorders-5. American Psychiatric Association.
Michael Barthel, Galen Stocking, Jesse Holcomb, and
Amy Mitchell. 2016. Nearly eight-in-ten reddit
users get news on the site. Pew Research Center,
Adrian Benton, Margaret Mitchell, and Dirk Hovy.
2017. Multitask learning for mental health condi-
tions with limited social media data. In Proceedings
of the 15th Conference of the European Chapter of
the Association for Computational Linguistics: Vol-
ume 1, Long Papers, volume 1, pages 152–162.
Jack Block. 2010. The five-factor framing of person-
ality and beyond: Some ruminations. Psychological
Inquiry, 21(1):2–25.
Charles S Carver and Jennifer Connor-Smith. 2010.
Personality and coping. Annual review of psychol-
ogy, 61:679–704.
Eric Evan Chen and Sean P Wojcik. 2016. A practical
guide to big data research in psychology. Psycho-
logical methods, 21(4):458.
Mike Conway and Daniel OConnor. 2016. Social me-
dia, big data, and mental health: current advances
and ethical implications. Current opinion in psy-
chology, 9:77–82.
Glen Coppersmith, Mark Dredze, and Craig Harman.
2014. Quantifying mental health signals in twitter.
In Proceedings of the Workshop on Computational
Linguistics and Clinical Psychology: From Linguis-
tic Signal to Clinical Reality, pages 51–60.
Glen Coppersmith, Mark Dredze, Craig Harman, and
Kristy Hollingshead. 2015. From adhd to sad: An-
alyzing the language of mental health on twitter
through self-reported diagnoses. In Proceedings
of the 2nd Workshop on Computational Linguistics
and Clinical Psychology: From Linguistic Signal to
Clinical Reality, pages 1–10.
Glen Coppersmith, Kim Ngo, Ryan Leary, and An-
thony Wood. 2016. Exploratory analysis of social
media prior to a suicide attempt. In Proceedings
of the Third Workshop on Computational Lingusitics
and Clinical Psychology, pages 106–117.
Kathryn P Davison, James W Pennebaker, and Sally S
Dickerson. 2000. Who talks? the social psychology
of illness support groups. American Psychologist,
Munmun De Choudhury, Michael Gamon, Scott
Counts, and Eric Horvitz. 2013. Predicting depres-
sion via social media. ICWSM, 13:1–10.
Munmun De Choudhury, Emre Kiciman, Mark Dredze,
Glen Coppersmith, and Mrinal Kumar. 2016. Dis-
covering shifts to suicidal ideation from mental
health content in social media. In Proceedings of
the 2016 CHI conference on human factors in com-
puting systems, pages 2098–2110. ACM.
Sanda Dolcos and Dolores Albarracin. 2014. The inner
speech of behavioral regulation: Intentions and task
performance strengthen when you talk to yourself
as a you. European Journal of Social Psychology,
Maeve Duggan and Aaron Smith. 2013. 6% of online
adults are reddit users. Pew Internet & American
Life Project, 3:1–10.
Meir Friedenberg, Hadi Amiri, Hal Daum´
e III, and
Philip Resnik. 2016. The umd clpsych 2016 shared
task system: Text representation for predicting triage
of forum posts about mental health. In Proceedings
of the Third Workshop on Computational Lingusitics
and Clinical Psychology, pages 158–161.
Jerome Friedman, Trevor Hastie, and Robert Tibshi-
rani. 2010. Regularization paths for generalized lin-
ear models via coordinate descent.Journal of Sta-
tistical Software, 33(1):1–22.
Rula Bayrakdar Garside and Bonnie Klimes-Dougan.
2002. Socialization of discrete negative emotions:
Gender differences and links with psychological dis-
tress. Sex roles, 47(3-4):115–128.
George Gkotsis, Anika Oellrich, Sumithra Velupillai,
Maria Liakata, Tim JP Hubbard, Richard JB Dob-
son, and Rina Dutta. 2017. Characterisation of men-
tal health conditions in social media using informed
deep learning. Scientific reports, 7:45141.
Jesse Graham, Jonathan Haidt, and Brian A Nosek.
2009. Liberals and conservatives rely on different
sets of moral foundations. Journal of personality
and social psychology, 96(5):1029.
Shannon E Holleran and Matthias R Mehl. 2008. Let
me read your mind: Personality judgments based on
a persons natural stream of thought. Journal of Re-
search in Personality, 42(3):747–754.
William Ickes, Susan Reidhead, and Miles Patterson.
1986. Machiavellianism and self-monitoring: As
different as” me” and” you”. Social Cognition,
Molly E Ireland and Matthias R Mehl. 2014. Natural
language use as a marker. The Oxford handbook of
language and social psychology, pages 201–237.
Molly E Ireland, H Andrew Schwartz, Qijia Chen,
Lyle H Ungar, and Dolores Albarrac´
ın. 2015.
Future-oriented tweets predict lower county-level
hiv prevalence in the united states. Health Psychol-
ogy, 34(S):1252.
William Jarrold, Harold S Javitz, Ruth Krasnow, Bart
Peintner, Eric Yeh, Gary E Swan, and Matthias
Mehl. 2011. Depression and self-focused language
in structured interviews with older men. Psycholog-
ical reports, 109(2):686–700.
Oliver P John and Sanjay Srivastava. 1999. The big five
trait taxonomy: History, measurement, and theoret-
ical perspectives. Handbook of personality: Theory
and research, 2(1999):102–138.
Arif Khan, Robyn M Leventhal, Shirin Khan, and Wal-
ter A Brown. 2002. Suicide risk in patients with anx-
iety disorders: a meta-analysis of the fda database.
Journal of affective disorders, 68(2):183–190.
Ethan Kross and Ozlem Ayduk. 2011. Making
meaning out of negative experiences by self-
distancing. Current directions in psychological sci-
ence, 20(3):187–191.
Kate Loveys, Patrick Crutchley, Emily Wyatt, and Glen
Coppersmith. 2017. Small but mighty: Affective
micropatterns for quantifying mental health from so-
cial media language. In Proceedings of the Fourth
Workshop on Computational Linguistics and Clini-
cal Psychology—From Linguistic Signal to Clinical
Reality, pages 85–95.
Matthias R Mehl, Samuel D Gosling, and James W
Pennebaker. 2006. Personality in its natural habitat:
Manifestations and implicit folk theories of person-
ality in daily life. Journal of personality and social
psychology, 90(5):862.
Matthias R Mehl, Megan L Robbins, and Shannon E
Holleran. 2012. How taking a word for a word
can be problematic: Context-dependent linguistic
markers of extraversion and neuroticism. Journal of
Methods and Measurement in the Social Sciences,
Miranda M Nadeau, Michael J Balsan, and Aaron B
Rochlen. 2016. Mens depression: Endorsed experi-
ences and expressions. Psychology of Men & Mas-
culinity, 17(4):328.
Jeroen Ooms. 2014. The jsonlite package: A practi-
cal and consistent mapping between json data and r
objects.arXiv:1403.2805 [stat.CO].
Kevin A Padrez, Lyle Ungar, Hansen Andrew
Schwartz, Robert J Smith, Shawndra Hill, Tadas
Antanavicius, Dana M Brown, Patrick Crutchley,
David A Asch, and Raina M Merchant. 2015. Link-
ing social media and medical record data: a study of
adults presenting to an academic, urban emergency
department. BMJ Qual Saf, pages bmjqs–2015.
James W. Pennebaker, Ryan L. Boyd, Kayla Jordan,
and Kate Blackburn. 2015. The development and
psychometric properties of liwc2015.UT Fac-
ulty/Researcher Works.
James W Pennebaker, Cindy K Chung, Joey Frazee,
Gary M Lavergne, and David I Beaver. 2014.
When small words foretell academic success: The
case of college admissions essays. PloS one,
R Core Team. 2018. R: A Language and Environment
for Statistical Computing. R Foundation for Statis-
tical Computing, Vienna, Austria. Version 3.4.4.
Philip Resnik, William Armstrong, Leonardo
Claudino, Thang Nguyen, Viet-An Nguyen,
and Jordan Boyd-Graber. 2015. Beyond lda:
exploring supervised topic modeling for depression-
related language in twitter. In Proceedings of the
2nd Workshop on Computational Linguistics and
Clinical Psychology: From Linguistic Signal to
Clinical Reality, pages 99–107.
Philip Resnik, Anderson Garron, and Rebecca Resnik.
2013. Using topic modeling to improve prediction
of neuroticism and depression in college students.
In Proceedings of the 2013 conference on empiri-
cal methods in natural language processing, pages
Ivan Rivera. 2015. RedditExtractoR: Reddit Data Ex-
traction Toolkit. R package version 2.0.2.
Kenneth H Rubin, Robert J Coplan, and Julie C
Bowker. 2009. Social withdrawal in childhood. An-
nual review of psychology, 60:141–171.
Norman Sartorius, T Bedirhan ¨
un, Yves Lecrubier,
and Hans-Ulrich Wittchen. 1996. Depression co-
morbid with anxiety: Results from the who study
on” psychological disorders in primary health care.”.
The British journal of psychiatry.
David P Schmitt, Anu Realo, Martin Voracek, and
uri Allik. 2008. Why can’t a man be more like a
woman? sex differences in big five personality traits
across 55 cultures. Journal of personality and social
psychology, 94(1):168.
H Andrew Schwartz, Johannes C Eichstaedt, Mar-
garet L Kern, Lukasz Dziurzynski, Stephanie M Ra-
mones, Megha Agrawal, Achal Shah, Michal Kosin-
ski, David Stillwell, Martin EP Seligman, et al.
2013. Personality, gender, and age in the language
of social media: The open-vocabulary approach.
PloS one, 8(9):e73791.
H Andrew Schwartz, Salvatore Giorgi, Maarten Sap,
Patrick Crutchley, Lyle Ungar, and Johannes Eich-
staedt. 2017. Dlatk: Differential language analysis
toolkit. In Proceedings of the 2017 Conference on
Empirical Methods in Natural Language Process-
ing: System Demonstrations, pages 55–60.
H Andrew Schwartz, Maarten Sap, Margaret L Kern,
Johannes C Eichstaedt, Adam Kapelner, Megha
Agrawal, Eduardo Blanco, Lukasz Dziurzynski,
Gregory Park, David Stillwell, et al. 2016. Predict-
ing individual well-being through the language of
social media. In Biocomputing 2016: Proceedings
of the Pacific Symposium, pages 516–527. World
Robert J Smith, Patrick Crutchley, H Andrew
Schwartz, Lyle Ungar, Frances Shofer, Kevin A
Padrez, and Raina M Merchant. 2017. Variations
in facebook posting patterns across validated patient
health conditions: A prospective cohort study. Jour-
nal of medical Internet research, 19(1).
Jessie Sun, H. Andrew Schwartz, Youngseo Son, Mar-
garet L. Kern, and Simine Vazire. under review.
The language of well-being: Tracking within-person
emotion fluctuations through everyday speech.
William B Swann Jr and Peter J Rentfrow. 2001. Blir-
tatiousness: cognitive, behavioral, and physiological
consequences of rapid responding. Journal of Per-
sonality and Social Psychology, 81(6):1160.
Allison M Tackman, David A Sbarra, Angela L Carey,
M Brent Donnellan, Andrea B Horn, Nicholas S
Holtzman, To’Meisha S Edwards, James W Pen-
nebaker, and Matthias R Mehl. 2018. Depression,
negative emotionality, and self-referential language:
A multi-lab, multi-measure, and multi-language-
task research synthesis. Journal of personality and
social psychology.
Yla R Tausczik and James W Pennebaker. 2010. The
psychological meaning of words: Liwc and comput-
erized text analysis methods. Journal of language
and social psychology, 29(1):24–54.
Shelley E Taylor, David K Sherman, Heejung S Kim,
Johanna Jarcho, Kaori Takagi, and Melissa S Duna-
gan. 2004. Culture and social support: who seeks
it and why? Journal of personality and social psy-
chology, 87(3):354.
Terry Therneau and Beth Atkinson. 2018. rpart: Re-
cursive Partitioning and Regression Trees. R pack-
age version 4.1-13.
Larissa Z Tiedens. 2001. Anger and advancement
versus sadness and subjugation: the effect of neg-
ative emotion expressions on social status confer-
ral. Journal of personality and social psychology,
Konstantin O Tskhay and Nicholas O Rule. 2014. Per-
ceptions of personality in text-based media and osn:
A meta-analysis. Journal of Research in Personal-
ity, 49:25–30.
Harvey A Whiteford, Louisa Degenhardt, J ¨
Rehm, Amanda J Baxter, Alize J Ferrari, Holly E
Erskine, Fiona J Charlson, Rosana E Norman,
Abraham D Flaxman, Nicole Johns, et al. 2013.
Global burden of disease attributable to mental
and substance use disorders: findings from the
global burden of disease study 2010. The Lancet,
Markus Wolf, Cindy K Chung, and Hans Kordy. 2010.
Inpatient treatment to online aftercare: E-mailing
themes as a function of therapeutic outcomes. Psy-
chotherapy Research, 20(1):71–85.
Carolyn Zahn-Waxler, Bonnie Klimes-Dougan, and
Marcia J Slattery. 2000. Internalizing problems of
childhood and adolescence: Prospects, pitfalls, and
progress in understanding the development of anxi-
ety and depression. Development and psychopathol-
ogy, 12(3):443–466.
Johannes Zimmermann, Timo Brockmeyer, Matthias
Hunn, Henning Schauenburg, and Markus Wolf.
2017. First-person pronoun use in spoken language
as a predictor of future depressive symptoms: Pre-
liminary evidence from a clinical sample of de-
pressed patients. Clinical psychology & psychother-
apy, 24(2):384–391.
... Instagram is a popular social media platform that has gained traction especially amongst adolescents and young adults (25)the age groups most at risk for the emergence of SSD. Despite its popularity, the majority of prior research to date has focused on text based platforms such as Facebook (26,27), Twitter (21,28), or Reddit (29,30). Contrary to these other popular networks, Instagram's primary focus is on images and videos as opposed to text (31). ...
... The majority of prior work, aiming to explore associations between mental health status and social media activity, has been limited by relying primarily on data from individuals with presumed psychiatric conditions, without the ability to validate the disclosure or diagnosis (38,39). For example, self-disclosures based on specific disease related terms or statements that have been mentioned by platform users (21,23,40) or affiliation with mental health related communities (29,30). Further, social media use of individuals with SSD has been studied little compared to other psychiatric conditions, such as depression, and research has mainly focused on linguistic analysis (39). ...
... We also compared interaction effects between day of the week and group. We identified a main effect of the day of the week on Instagram behavior [Q (6,30.12) = 12.24, p < 0.001]. ...
Full-text available
Background and Objectives: Prior research has successfully identified linguistic and behavioral patterns associated with schizophrenia spectrum disorders (SSD) from user generated social media activity. Few studies, however, have explored the potential for image analysis to inform psychiatric care for individuals with SSD. Given the popularity of image-based platforms, such as Instagram, investigating user generated image data could further strengthen associations between social media activity and behavioral health. Methods: We collected 11,947 Instagram posts across 68 participants (mean age = 23.6; 59% male) with schizophrenia spectrum disorders (SSD; n = 34) and healthy volunteers (HV; n = 34). We extracted image features including color composition, aspect ratio, and number of faces depicted. Additionally, we considered social connections and behavioral features. We explored differences in usage patterns between SSD and HV participants. Results: Individuals with SSD posted images with lower saturation ( p = 0.033) and lower colorfulness ( p = 0.005) compared to HVs, as well as images showing fewer faces on average ( SSD = 1.5, HV = 2.4, p < 0.001). Further, individuals with SSD demonstrated a lower ratio of followers to following compared to HV participants ( p = 0.025). Conclusion: Differences in uploaded images and user activity on Instagram were identified in individuals with SSD. These differences highlight potential digital biomarkers of SSD from Instagram data.
... The papers returned for the mental health query are the largest category of papers and are fairly homogeneous in terms of data sources in languages. The vast majority use English-language social media data (e.g., Jamil et al. 2017;Mikal et al. 2017;Ireland and Iserman 2018;Loveys et al. 2018) to detect generic mental health concerns, depression or anxiety (often without following a formal definition of these conditions). Some studies such as Gkotsis et al. (2016a) focus on detecting suicide ideation in social media. ...
... For example, separating the content used to define disorder status (e.g. membership of a mental health forum) from content used to characterise language use (posts by those users on other forums) 37 . When this approach was taken, accuracy was substantially worse. ...
Full-text available
Depressed individuals use language differently than healthy controls and it has been proposed that social media posts can be used to identify depression. Much of the evidence behind this claim relies on indirect measures of mental health and few studies have tested if these language features are specific to depression versus other aspects of mental health. We analysed the Tweets of 1006 participants who completed questionnaires assessing symptoms of depression and 8 other mental health conditions. Daily Tweets were subjected to textual analysis and the resulting linguistic features were used to train an Elastic Net model on depression severity, using nested cross-validation. We then tested performance in a held-out test set (30%), comparing predictions of depression versus 8 other aspects of mental health. The depression trained model had modest out-of-sample predictive performance, explaining 2.5% of variance in depression symptoms (R2 = 0.025, r = 0.16). The performance of this model was as-good or superior when used to identify other aspects of mental health: schizotypy, social anxiety, eating disorders, generalised anxiety, above chance for obsessive-compulsive disorder, apathy, but not significant for alcohol abuse or impulsivity. Machine learning analysis of social media data, when trained on well-validated clinical instruments, could not make meaningful individualised predictions regarding users’ mental health. Furthermore, language use associated with depression was non-specific, having similar performance in predicting other mental health problems.
... We also experimented with Linguistic Inquiry Word Count (LIWC) features, a closed-vocabulary English lexicon containing 64 categories (excluding punctuation categories), ranging from linguistic dimensions to psychological processes covering emotions and personal concerns, traditionally used in psychological studies (Pennebaker et al., 2007). In social media analysis, LIWC has been shown to contain signals for mental health disorders (Ireland and Iserman, 2018;Wolohan et al., 2018;Mitchell et al., 2015), including CLPSYCH and MULTITASK. ...
... In general, these works deploy supervised learning algorithms and extensive feature engineering to derive textual, social, and sentiment features (e.g., [Sharma and De Choudhury 2018;De Choudhury et al. 2017;Park et al. 2015;Tsugawa et al. 2015;De Choudhury et al. 2014]). Few works address anxiety Ireland and Iserman 2018;Gruda and Hasan 2019], or its comorbidity with other disorders, including depression [Cohan et al. 2018;Bagroy et al. 2017]. More recently, deep learning techniques have been explored for the classification of depression [Yates et al. 2017;Mann et al. 2020], chronic stress [Lin et al. 2014], and anxiety [Shen and Rudzicz 2017]. ...
Full-text available
The use of social networks to expose personal difficulties has enabled works on the automatic identification of specific mental conditions, particularly depression. Depression is the most incapacitating disease worldwide, and it has an alarming comorbidity rate with anxiety. In this paper, we explore deep learning techniques to develop a stacking ensemble to automatically identify depression, anxiety, and comorbidity, using data extracted from Reddit. The stacking is composed of specialized single-label binary classifiers that distinguish between specific disorders and control users. A meta-learner explores these base classifiers as a context for reaching a multi-label, multi-class decision. We developed extensive experiments using alternative architectures (LSTM, CNN, and their combination), word embeddings, and ensemble topologies. All base classifiers and ensembles outperformed the baselines. The CNN-based binary classifiers achieved the best performance, with f-measures of 0.79 for depression, 0.78 for anxiety, and 0.78 for comorbidity. The ensemble topology with best performance (Hamming Loss of 0.29 and Exact Match Ratio of 0.47) combines base classifiers according to three architectures, and do not include comorbidity classifiers. Using SHAP, we confirmed the influential features are related to symptoms of these disorders.
... Among mental health studies using Reddit data, many papers address the problem of predicting mental health status through machine learning methods, such as Support Vector Machine (SVM) [31,32,33,34], Logistic Regression [35,36,37] and Deep Learning [15,38,23,39,40]. Those papers framed their research questions as classification tasks, in contrast to our approach of predicting the effect of the social interactions as a regression task. ...
Full-text available
In recent years, Online Social Networks have become an important medium for people who suffer from mental disorders to share moments of hardship, and receive emotional and informational support. In this work, we analyze how discussions in Reddit communities related to mental disorders can help improve the health conditions of their users. Using the emotional tone of users’ writing as a proxy for emotional state, we uncover relationships between user interactions and state changes. First, we observe that authors of negative posts often write rosier comments after engaging in discussions, indicating that users’ emotional state can improve due to social support. Second, we build models based on SOTA text embedding techniques and RNNs to predict shifts in emotional tone. This differs from most of related work, which focuses primarily on detecting mental disorders from user activity. We demonstrate the feasibility of accurately predicting the users’ reactions to the interactions experienced in these platforms, and present some examples which illustrate that the models are correctly capturing the effects of comments on the author’s emotional tone. Our models hold promising implications for interventions to provide support for people struglling with mental illnesses.
... We also experimented with Linguistic Inquiry Word Count (LIWC) features, a closed-vocabulary English lexicon containing 64 categories (excluding punctuation categories), ranging from linguistic dimensions to psychological processes covering emotions and personal concerns, traditionally used in psychological studies (Pennebaker et al., 2007). In social media analysis, LIWC has been shown to contain signals for mental health disorders (Ireland and Iserman, 2018;Wolohan et al., 2018;Mitchell et al., 2015), including CLPSYCH and MULTITASK. ...
Multiple studies have demonstrated that behavior on internet-based social media platforms can be indicative of an individual's mental health status. The widespread availability of such data has spurred interest in mental health research from a computational lens. While previous research has raised concerns about possible biases in models produced from this data, no study has quantified how these biases actually manifest themselves with respect to different demographic groups, such as gender and racial/ethnic groups. Here, we analyze the fairness of depression classifiers trained on Twitter data with respect to gender and racial demographic groups. We find that model performance systematically differs for underrepresented groups and that these discrepancies cannot be fully explained by trivial data representation issues. Our study concludes with recommendations on how to avoid these biases in future research.
Background The study of depression and anxiety using publicly available social media data is a research activity that has grown considerably over the past decade. The discussion platform Reddit has become a popular social media data source in this nascent area of study, in part because of the unique ways in which the platform is facilitative of research. To date, no work has been done to synthesize existing studies on depression and anxiety using Reddit. Objective The objective of this review is to understand the scope and nature of research using Reddit as a primary data source for studying depression and anxiety. Methods A scoping review was conducted using the Arksey and O’Malley framework. MEDLINE, Embase, CINAHL, PsycINFO, PsycARTICLES, Scopus, ScienceDirect, IEEE Xplore, and ACM academic databases were searched. Inclusion criteria were developed using the participants, concept, and context framework outlined by the Joanna Briggs Institute Scoping Review Methodology Group. Eligible studies featured an analytic focus on depression or anxiety and used naturalistic written expressions from Reddit users as a primary data source. Results A total of 54 studies were included in the review. Tables and corresponding analyses delineate the key methodological features, including a comparatively larger focus on depression versus anxiety, an even split of original and premade data sets, a widespread analytic focus on classifying the mental health states of Reddit users, and practical implications that often recommend new methods of professionally delivered monitoring and outreach for Reddit users. Conclusions Studies of depression and anxiety using Reddit data are currently driven by a prevailing methodology that favors a technical, solution-based orientation. Researchers interested in advancing this research area will benefit from further consideration of conceptual issues surrounding the interpretation of Reddit data with the medical model of mental health. Further efforts are also needed to locate accountability and autonomy within practice implications, suggesting new forms of engagement with Reddit users.
Full-text available
Data-driven methods for mental health treatment and surveillance have become a major focus in computational science research in the last decade. However, progress in the domain, in terms of both medical understanding and system performance, remains bounded by the availability of adequate data. Prior systematic reviews have not necessarily made it possible to measure the degree to which data-related challenges have affected research progress. In this paper, we offer an analysis specifically on the state of social media data that exists for conducting mental health research. We do so by introducing an open-source directory of mental health datasets, annotated using a standardized schema to facilitate meta-analysis.
Full-text available
Depressive symptomatology is manifested in greater first-person singular pronoun use (i.e., I-talk), but when and for whom this effect is most apparent, and the extent to which it is specific to depression or part of a broader association between negative emotionality and I-talk, remains unclear. Using pooled data from N = 4,754 participants from 6 labs across 2 countries, we examined, in a preregistered analysis, how the depression–I-talk effect varied by (a) first-person singular pronoun type (i.e., subjective, objective, and possessive), (b) the communication context in which language was generated (i.e., personal, momentary thought, identity-related, and impersonal), and (c) gender. Overall, there was a small but reliable positive correlation between depression and I-talk (r = .10, 95% CI [.07, .13]). The effect was present for all first-person singular pronouns except the possessive type, in all communication contexts except the impersonal one, and for both females and males with little evidence of gender differences. Importantly, a similar pattern of results emerged for negative emotionality. Further, the depression–I-talk effect was substantially reduced when controlled for negative emotionality but this was not the case when the negative emotionality–I-talk effect was controlled for depression. These results suggest that the robust empirical link between depression and I-talk largely reflects a broader association between negative emotionality and I-talk. Self-referential language using first-person singular pronouns may therefore be better construed as a linguistic marker of general distress proneness or negative emotionality rather than as a specific marker of depression.
Full-text available
The number of people affected by mental illness is on the increase and with it the burden on health and social care use, as well as the loss of both productivity and quality-adjusted life-years. Natural language processing of electronic health records is increasingly used to study mental health conditions and risk behaviours on a large scale. However, narrative notes written by clinicians do not capture first-hand the patients’ own experiences, and only record cross-sectional, professional impressions at the point of care. Social media platforms have become a source of ‘in the moment’ daily exchange, with topics including well-being and mental health. In this study, we analysed posts from the social media platform Reddit and developed classifiers to recognise and classify posts related to mental illness according to 11 disorder themes. Using a neural network and deep learning approach, we could automatically recognise mental illness-related posts in our balenced dataset with an accuracy of 91.08% and select the correct theme with a weighted average accuracy of 71.37%. We believe that these results are a first step in developing methods to characterise large amounts of user-generated content that could support content curation and targeted interventions.
Full-text available
The massive volume of data that now covers a wide variety of human behaviors offers researchers in psychology an unprecedented opportunity to conduct innovative theory- and data-driven field research. This article is a practical guide to conducting big data research, covering data management, acquisition, processing, and analytics (including key supervised and unsupervised learning data mining methods). It is accompanied by walkthrough tutorials on data acquisition, text analysis with latent Dirichlet allocation topic modeling, and classification with support vector machines. Big data practitioners in academia, industry, and the community have built a comprehensive base of tools and knowledge that makes big data research accessible to researchers in a broad range of fields. However, big data research does require knowledge of software programming and a different analytical mindset. For those willing to acquire the requisite skills, innovative analyses of unexpected or previously untapped data sources can offer fresh ways to develop, test, and extend theories. When conducted with care and respect, big data research can become an essential complement to traditional research.
Background Social media is emerging as an insightful platform for studying health. To develop targeted health interventions involving social media, we sought to identify the patient demographic and disease predictors of frequency of posting on Facebook. Objective The aims were to explore the language topics correlated with frequency of social media use across a cohort of social media users within a health care setting, evaluate the differences in the quantity of social media postings across individuals with different disease diagnoses, and determine if patients could accurately predict their own levels of social media engagement. Methods Patients seeking care at a single, academic, urban, tertiary care emergency department from March to October 2014 were queried on their willingness to share data from their Facebook accounts and electronic medical records (EMRs). For each participant, the total content of Facebook posts was extracted. Using the latent Dirichlet allocation natural language processing technique, Facebook language topics were correlated with frequency of Facebook use. The mean number of Facebook posts over 6 months prior to enrollment was then compared across validated health outcomes in the sample. Results A total of 695 patients consented to provide access to their EMR and social media data. Significantly correlated language topics among participants with the highest quartile of posts contained health terms, such as “cough,” “headaches,” and “insomnia.” When adjusted for demographics, individuals with a history of depression had significantly higher posts (mean 38, 95% CI 28-50) than individuals without a history of depression (mean 22, 95% CI 19-26, P=.001). Except for depression, across prevalent health outcomes in the sample (hypertension, diabetes, asthma), there were no significant posting differences between individuals with or without each condition. Conclusions High-frequency posters in our sample were more likely to post about health and to have a diagnosis of depression. The direction of causality between depression and social media use requires further evaluation. Our findings suggest that patients with depression may be appropriate targets for health-related interventions on social media.
Conference Paper
History of mental illness is a major factor behind suicide risk and ideation. However research efforts toward characterizing and forecasting this risk is limited due to the paucity of information regarding suicide ideation, exacerbated by the stigma of mental illness. This paper fills gaps in the literature by developing a statistical methodology to infer which individuals could undergo transitions from mental health discourse to suicidal ideation. We utilize semi-anonymous support communities on Reddit as unobtrusive data sources to infer the likelihood of these shifts. We develop language and interactional measures for this purpose, as well as a propensity score matching based statistical approach. Our approach allows us to derive distinct markers of shifts to suicidal ideation. These markers can be modeled in a prediction framework to identify individuals likely to engage in suicidal ideation in the future. We discuss societal and ethical implications of this research.