ArticlePDF Available

Abstract and Figures

Social media may serve as an important platform for the monitoring of population-level opioid abuse in near real-time. Our objectives for this study were to (i) manually characterize a sample of opioid-mentioning Twitter posts, (ii) compare the rates of abuse/misuse related posts between prescription and illicit opiods, and (iii) to implement and evaluate the performances ofsupervised machine learning algorithms for the characterization of opioid-related chatter, which can potentially automate social media based monitoring in the future.. We annotated a total of 9006 tweets into four categories, trained several machine learning algorithms and compared their performances. Deep convolutional neural networks marginally outperformed support vector machines and random forests, with an accuracy of 70.4%. Lack of context in tweets and data imbalance resulted in misclassification of many tweets to the majority class. The automatic classification experiments produced promising results, although there is room for improvement.
Content may be subject to copyright.
Towards Automating Location-Specific Opioid Toxicosurveillance from Twitter via Data
Science Methods
Abeed Sarkera, Graciela Gonzalez-Hernandeza, Jeanmarie Perroneb
a Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia,
Pennsylvania, U.S.A.,
b Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, U.S.A.
Abstract
Social media may serve as an important platform for the
monitoring of population-level opioid abuse in near real-time.
Our objectives for this study were to (i) manually characterize
a sample of opioid-mentioning Twitter posts, (ii) compare the
rates of abuse/misuse related posts between prescription and
illicit opiods, and (iii) to implement and evaluate the
performances ofsupervised machine learning algorithms for
the characterization of opioid-related chatter, which can
potentially automate social media based monitoring in the
future.. We annotated a total of 9006 tweets into four
categories, trained several machine learning algorithms and
compared their performances. Deep convolutional neural
networks marginally outperformed support vector machines
and random forests, with an accuracy of 70.4%. Lack of context
in tweets and data imbalance resulted in misclassification of
many tweets to the majority class. The automatic classification
experiments produced promising results, although there is
room for improvement.
Keywords:
Social media, Opioids, Surveillance
Introduction
The problem of opioid (prescription and illicit) addiction and
overdose is having lethal consequences all over the United
States [1]. The 2015 National Survey on Drug Use and Health
(NSDUH) estimated that 11.5 million adults misused/abused
prescription opioids, and among adults with prescription opioid
use, 12.5% reported misuse [2]. The number of opioid overdose
deaths continue to rise alarmingly, with 174 people dying from
drug overdoses daily [3], and the current rate of opioid
prescriptions is three times higher than in the 90s. Between
2014 and 2015, opioid related death rates increased by 15.6%,
continuing a trend from 1999, and this increase was driven by
illicit opioids other than methadone [4]. Despite the significant
acceleration of the crisis in recent years, surveillance measures
are slow, and deriving estimates from surveys, such as the
NSDUH, is belated. There is almost a two-year lag between the
occurrence of overdose related deaths and the time by which
the statistics are publicized. Such a lag in the process of data
collection and synthesis makes it impossible to determine the
trajectory of the epidemic or identify geographic areas that are
Available at: https://www.drugabuse.gov/related-
topics/trends-statistics/overdose-death-rates. Accessed:
October 22, 2018.
more greatly impacted by the crisis at a specific point of time.
Whether its illicit or prescription opioids, the vast numbers of
people affected means that a comprehensive public health
approach is needed to curb the crisis, going beyond simply
changing patterns of prescribing [5]. Kolodny and Frieden [1]
recommended 10 steps that the federal government should take
to reverse the opioid epidemic, and, as their first point, the
authors outlined the need for real-time assessment of the
numbers, patterns, or trends of opioid misuse/addiction.
In this paper, we explore the possibility of using social media,
namely Twitter, as a resource for performing real time
surveillance of opioid abuse, including both prescription and
illicit opioids. Past studies have shown that users post
information related to drug abuse on social media [6]–[8].
However, there is a lack of analysis of the differences in abuse-
related chatter versus other types of chatter, such as
consumption, although it is well known that not all drug-related
chatter represents abuse [9]. There is also a lack of
understanding regarding the differences between the chatter
associated with prescription and illicit opioids (e.g., what
proportions of illicit vs. prescription opioid mentioning chatter
represent abuse?). Unsupervised methods that primarily rely on
the volume of data do not take into account the large amounts
of noise that is present in social media data (e.g., [10]). There
are currently no prototype end-to-end, automated pipelines that
can enable the real time surveillance of opioid abuse/misuse via
social media. In this paper, we take the first steps in addressing
these gaps. We present (i) data collection strategies from
Twitter, including the use of automatically generated
misspellings and geolocation metadata, (ii) an analysis of the
contents of tweets mentioning prescription and illicit opioids,
and (iii) a comparison of several supervised classification
approaches. Our experiments show that opioid chatter on
Twitter can vary significantly between prescription and illicit
opioids, with some illicit opioid keywords being too ambiguous
to be useful for data collection. We also show that using
annotated data, we can train supervised learning algorithms to
automatically characterize tweets. We suggest that such a
supervised classification system, paired with geolocation
metadata from Twitter, can be used to perform localized
surveillance of opioid abuse/misuse. We present our pilot
methods using the state of Pennsylvania as example.
MEDINFO 2019: Health and Wellbeing e-Networks for All
L. Ohno-Machado and B. Séroussi (Eds.)
© 2019 International Medical Informatics Association (IMIA) and IOS Press.
This article is published online with Open Access by IOS Press and distributed under the terms
of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0).
doi:10.3233/SHTI190238
333
Methods
Data Collection
We collected data from Twitter using names of prescription and
illicit opioid keywords (e.g., Percocet® and heroin), including
street names (e.g., china white, tar, skag, percs) and common
misspellings (e.g., percoset, heorin). We used the list of drug
slang terms recently released by the Drug Enforcement Agency
(DEA) of the United States to create an initial list of possible
slang terms for different prescription and illicit opioids [11].
We manually reviewed the terms and removed those we were
sure to be too ambiguous. For example, some of the slang terms
associated with heroin, as per the document, are ‘basketball’,
coffee’, ‘lemonade’ and ‘whiskey’. Through manual searches
of the Twitter web interface, we could not find any instances
where these terms were used to refer to opiods. Therefore, we
removed these to reduce the retrieval of noise. This strategy led
us to use a total of 56 unique names of opioids. Since drug
names are often misspelled on social media, we automatically
generated misspellings for these keywords using a misspelling
generation system [12]. Table 1 presents some sample opioid-
related keywords and their automatically generated
misspellings. After collecting an initial set, we analyzed
samples of retrieved tweets for each keyword. We discovered
that despite our initial filtering of keywords, approximately
85% of the tweets were retrieved by 4 keywords—tar (~6.5%),
dope (~54%), smack (~20.5%) and skunk (~4%)—and in these
tweets, these keywords were almost invariably unrelated to
opioids and represented something else. We, therefore,
removed these keywords for our final data collection. In this
manner, we collected tweets between the years 2012 to 2015,
only including those geolocated from within Pennsylvania and
excluding retweets.
Table 1– Sample of opioid-related keywords and their
automatically-generated frequently occurring misspellings
Keyword
Generated Misspellings
Tramadol
trammadol tramadal tramdol
tramadols tramado tramedol
tramadoll tramadole tramidol
tamadol tranadol tramodol
tremadol
Heroin
herione herroine heroins
heroine heroin heorin herion
Methadone
methadones methadose methodone
mehtadone metadone methadon
methdone
Oxycontin
oxicontin oxcotin oycotin
oxycotins oycontin oxycontins
oxycoton oxicotin ocycotin
oxycodin oxycottin oxycotine
ocycontin
Codeine
codiene coedine codine codene
codein
Dilaudid
delaudid dialudid dilaudad
diluadid diaudid dilaudin
dilauded dilauid dillaudid
Annotation
Our intent was to compare the distributions of tweets for
prescription and illicit opioids and to attempt to train supervised
learning algorithms—both of which require manual
annotation/labeling of a sample of tweets. Based on manual
inspection of the collected data,. we decided to manually code
the tweets into 4 broad categories—self-reported abuse,
information sharing, non-English, and unrelated. Details about
these categories are as follows.
Abuse-related (A)
Abuse-indicating or possible abuse by the poster or by someone
the user knows or is communicating with. This category also
includes admissions of abuse in the past. For illicit opioids, any
consumption is considered to be abuse. For prescription
opioids, consumption is considered to be abuse only when there
is evidence that the user is taking the drug without a
prescription, through a non-standard route (e.g., injecting,
snorting) or in combination with other substances in order to
experience certain sensations.
Information Sharing/Seeking/Providing (I)
Tweets in which the poster is asking for information or
providing information about an opioid. This category also
includes expressions of medical use (e.g., mentions of having a
prescription or taking painkillers after surgery), and sharing of
news articles or other media that contain information about
opioids. General statements about the drug may are also put into
this class.
Non-English (N)
Tweets that are not written in English belong to this category.
Unrelated (U)
This category includes tweets that are not about the drug or
opioid, but about something else. This category also includes
tweets that make metaphorical comparisons (e.g., I am addicted
to X like heroin). Some examples of tweets belonging to this
category: handle related (@codeine_CXXX), heroine (hero),
cooking (brown sugar). This category also includes tweets
about movies or lyrics of songs that mention opioids, but don’t
have any information value. Table 2 presents examples of
tweets belonging to these four categories.
Table 2– Sample tweets and their categories; opioid keywords
shown in bold
Tweet
Category
@username naa, i just popped a few percs at
2, i drink, sip lean. Wbu?
A
Sooooo heroine addicts robbed the house 3
houses away from me...makes me feel safe
I
Ok I thought that it was just a really funny
oxy
clean commercial but turns out it was just
the Spanish channel
U
Te quieroo muchito mi hermana negra
N
We iteratively annotated a set of 100 tweets and discussed the
disagreements between pairs of annotators. The disagreements
on the initial set were resolved via discussion, and the same
process was executed twice until an acceptable level of
agreement was reached. In the final set, disagreements for
overlapping tweets were resolved by a third annotator—the first
author of this article.
Analysis and Supervised Learning
Prescription versus Illicit Opioids
Using the annotated dataset, we compared the volumes of
prescription and illicit opioids in the sample to better
understand which of these two broad classes of opioids were
A. Sarker et al. / Towards Automating Location-Specific Opioid Toxicosurveillance from Twitter via Data Science Methods334
more frequently discussed on Twitter. Since the sample for
annotation was drawn randomly, we assumed that the
distributions of prescription and illicit opiod mentions
represented their natural distribution in publicly available
Twiter chatter. We also assessed the differences in the
distributions of the four tweet categories for these two types of
opioids by comparing their proportions. The results of these
comparisons are presented in the Results section.
Supervised Machine Learning
To train and evaluate several machine learning algorithms, we
first split the annotated data into training (~80%) and test
(~20%) sets. We used the training set for analysis, algorithm
training and feature analyses, and held out the test set for
evaluation. Our intent was primarily to assess the utility of the
annotated corpus for supervised machine learning, with the
assumption that if supervised classification produced adequate
performance, they can be employed in the future for real time
monitoring. We trained and optimized three different
classification algorithms over the dataset—support vector
machines (SVMs), random forests (RFs) and deep
convolutional neural network (d-CNN), and compared their
performances with a naïve bayes (NB) baseline classifier.
SVMs and RFs have been shown in the past to perform well for
text classification tasks, particularly because of their suitability
for handling large feature spaces. Meanwhile, CNN based
classifiers have become popular in the recent past, and they
work particularly well in the presence of large annotated data
sets. For the SVMs, RF and NB classifiers, we performed basic
feature engineering based on our findings from past work on
the topic of automatic prescription medication abuse detection
from social media [13]. As features, we used preprocessed n-
grams (n=1—3), word clusters or generalized representations
of words, and the presence or absence of abuse-indicating
terms. We used 10-fold cross validation over the training set for
the RF and SVM classifiers to find optimal parameter values.
For the SVMs, we optimized the kernel and the cost parameter.
For the RF classifier, we optimized the number of trees. For the
d-CNN classifier, we used dense word vectors, or word
embeddings as input. We obtained pre-trained word
embeddings from our past work [14]. We used a three-layer
convolutional neural network, and for optimizing the various
hyperparameters, we split the training set further into two sets
and used the larger set for training and the smaller set for
validation. For NB, SVM and RF classifiers, we used
implementations provided by the python scikit-learn library
[15], and for the d-CNN classifier, we used the TensorFlow
library [16]. Figure 1 summarizes our entire processing
workflow for this study—from spelling variant generation
through to supervised classification of tweets.
Results
A total of 9006 tweets mentioning both prescription and illicit
opioids were annotated by 4 annotators. Among 550
overlapping tweets, average inter-annotator agreement was
0.75 (Cohen’s kappa [17]). The final data set consisted of 1748
abuse tweets, 2001 information tweets, 4830 unrelated tweets,
and 427 non-English tweets. The majority of the tweets
mentioned illicit opioids—7038 illicit and 2257 prescription.
Figure 2 shows the distributions of illicit and prescription
opioid mentioning tweets in our annotated set, illustrating that
although the relative volume of illicit opioid tweets is much
Note that the sum is of these two numbers is greater than the
total number of tweets annotated (9006) since some tweets
mention both prescription and illicit opioids.
higher, a significantly larger proportion of these tweets are
unrelated to opioids. The significantly higher number of
unrelated tweets for illicit opioid mentioning posts suggests that
such tweets have higher amounts of noise associated with them,
and may be more difficult to mine knowledge from despite the
large volume.
Table 3 presents the performance of the three classifiers and the
NB baseline over the test set. In total, we used 7204 tweets for
trainining and 1802 tweets for evaluation. For the d-CNN
classifier, the training set was further split into 6304 for training
and 900 for validation. It can be seen that, in terms of overall
accuracy, macro-averaged recall and precision, the d-CNN
classifier marginally outperforms the two traditional
benchmark classification approaches (SVMs and RF) despite
the relatively small amount of annotated data that was used. All
the three classifiers perform significantly better than the NB
baseline. The high performance of the d-CNN classifier is
encouraging because such deep neural network based
classifiers have more room for improvement, compared to their
traditional counterparts, as more data is annotated.
Figure 1 The Twitter data processing workflow for this study
Discussion
Our experiments produced very promising results and showed
that automatic machine learning based approaches may in fact
provide a possible mechanism for monitoring opioid abuse in
near real time for targeted geographic locations (e.g., at the state
leve). By combining geolocation information and manually
annotated data, we were able to automatically characterize
opioid-mentioning chatter from Pennsylvania with moderate
accuracy. Table 4 shows three sample tweets, their automatic
classifications, location by county and timestamp.
Our manual categorization efforts revealed the difficulty of
annotating tweets with high inter annotator agreement. Creating
a specific annotation guideline and several iterations of
discussions over small sets of overlapping annotations helped
improve agreement, although in many cases, due to the lack of
context in the tweets, the assigned category depended on the
subjective assessement of the annotator. This suggests that
A. Sarker et al. / Towards Automating Location-Specific Opioid Toxicosurveillance from Twitter via Data Science Methods 335
thorough annotation guidelines and such an iterative approach
to annotation are very important for achieving acceptable
agreement levels for complex annotation tasks such as this.
We found that illicit opioid mentioning tweets were particularly
highly noisy, with references to song lyrics or movie quotes,
which led to a large proportion of them to be labeled as
unrelated. The high proportions of unrelated tweets for both
types of opioids, and particularly for illicit opioids, illustrate the
importance of a supervised classification system for automatic
surveillance. Keyword-based surveillance methods, which rely
on the volume of data collected using specific keywords, are
evidently not suitable for opioid toxicosurveillance since most
of the data retrieved by the keywords will be unrelated noise.
The amount of noise may increase or decrease based on events
publicized over media outlets. In addition, as our initial analysis
of the retrieved data showed, if ambiguous keywords are to be
used, the vast majority of tweets collected via the ambiguous
keywords (e.g., dabs) can be noise, and this noise may mask the
real abuse related signals. Thus, when designing surveillance
strategies for similar tasks via social media, care must be taken
to identify noisy keywords that may invalidate the surveillance
process by bringing in too much noise.
The automatic classification experiments produced acceptable
performances, suggesting that automated, real-time opioid
toxicosurveillance may be a possibility. In the future, we will
explore additioal classification strategies for further improving
performance. A brief error analysis revealed that lack of context
in tweets caused our learning algorithms to often misclassify
tweets to the majority class (U). To better understand the
characteristics of the missclassified tweets, more analyses are
required.
In the future, we will also apply supervised classifiers trained
using our annotated data to automatically characterize
unlabeled posts collected over a longer time period to better
understand how opioid abuse related tweets are distributed over
time and more fine-grained geolocations. Such an analysis may
reveal specific time periods that are associated with higher rates
of abuse. We will also explore how the opioid abuse rates
reported on Twitter correlate, if at all, with real-world data
regarding the opioid crisis, such as geolocation-centric opioid
overdose death rates.
Conclusions
Our study suggests that Twitter is a promising platform to
perform real-time surveillance of opioid abuse/misuse.
Although we have only used geolocation data to identify the
origins of tweets at the state level, it may be possible to further
narrow down to the county or city level, particularly as the
volume of data grows over time. Our manual categorization of
the data and analyses shows that keyword based data collection
from Twitter results in the retrieval of significant amounts of
noise. Therefore, studies attempting to use streaming Twitter
data for surveillance must be wary of the amount of noise
retrieved per keyword and only use keywords that are
unambiguous. The same protocol should be followed for
research involving data from other social networks. Our
annotation also showed that even when using keywords with
high signal-to-noise ratios, the number of unrelated tweets is
significantly higher for illicit opioids compared to prescription
opioids. Thus, the total volume of opioid related chatter may
not be indicative of the real abuse or misuse of opioids, but may
be driven by other factors such as news articles or the release of
movies/songs. To overcome this problem, we employed a
supervised classification approach to automatically categorize
the tweets, and we found a deep convolutional neural network
to produce the best performance with an overall accuracy of
70.4%. In the future, we will try to improve on this
classification performance by employing more advanced
strategies, and also use the output of the classifiers to perform
downstream geolocation-centric analyses.
Table 3– Classifier accuracies over the test set
Classifier
Recall
Precis-
ion
Accu-
racy
(%)
95% CI
Naïve Bayes
0.61
0.58
53.9
51.6-56.3
Random
Forest
0.66
0.70
70.1
67.9-72.2
Support
Vector
Machines
0.68
0.70
69.9
67.8-72.1
Deep
Convolutional
Neural
Network
0.70
0.71
70.4
68.2-72.5
Figure 2– Distributions of tweets belonging to each category
for illicit and prescription opioid mentioning tweets. The charts
show that aabout 75% of the tweets in the sample mention illicit
opioids, and that illicit opioid mentioning tweets have much
higher proportions of unrelated information (including non-
English tweets), while prescription opioid mentioning tweets
have higher proportions of misuse/abuse and information
oriented tweets.
A. Sarker et al. / Towards Automating Location-Specific Opioid Toxicosurveillance from Twitter via Data Science Methods336
Acknowledgements
Research reported in this publication was supported in part by
the National Institute on Drug Abuse of the National Institutes
of Health under Award Number R01DA046619. The content is
solely the responsibility of the authors and does not necessarily
represent the official views of the National Institutes of Health.
The data collection and annotation efforts were partly funded
by a grant from the Pennsylvania Department of Health. The
Titan Xp used for this research was donated by the NVIDIA
Corporation. The authors would like to thank Karen O’Connor,
Alexis Upshur and Annika DeRoos for performing the
annotations. This study was approved by the institutional
review board at the University of Pennsylvania.
References
[1] A. Kolodny and T.R. Frieden, Ten steps the federal
government should take now to reverse the opioid
addiction epidemic, JAMA 318 (2017), 1537-1538.
[2] B. Han, W.M. Compton, C. Blanco, E. Crane, J. Lee, and
C.M. Jones, Prescription opioid use, misuse, and use
disorders in U.S. adults: 2015 National Survey on Drug
Use and Health, Ann Intern Med 167 (2017), 293-301.
[3] H. Jalal, J.M. Buchanich, M.S. Roberts, L.C. Balmert, K.
Zhang, and D.S. Burke, Changing dynamics of the drug
overdose epidemic in the United States from 1979 through
2016, Science 361 (2018), 1184.
[4] R.A.R. Rudd, P. Seth, F. David, and L. Scholl, Increases in
drug and opioid-involved overdose deaths — United
States, 2010–2015, Morb Mortal Wkly Rep 65 (2016),
1445-1452.
[5] A. Schuchat, D. Houry, and G.P. Guy, New data on opioid
use and prescribing in the United States, JAMA 318 (2017,
425-426.
[6] L. Shutler, L.S. Nelson, I. Portelli, C. Blachford, and J.
Perrone, Drug use in the Twittersphere: a qualitative
contextual analysis of tweets about prescription drugs, J
Addict Dis 34 (2015), 303-310.
[7] M. Chary, N. Genes, C. Giraud-Carrier, C. Hanson, L.S.
Nelson, and A.F. Manini, “Epidemiology from tweets:
estimating misuse of prescription opioids in the USA from
social media, J Med Toxicol 13 (2017) 278-286.
[8] D. Cameron et al., PREDOSE: a semantic web platform
for drug abuse epidemiology using social media, J Biomed
Inform 46 (2013), 985-997.
§ The tweets and their metadata have been modified to protect
the anonymity of the actual users.
[9] A. Sarker et al., Social media mining for toxicovigilance:
automatic monitoring of prescription medication abuse
from Twitter, Drug Saf 39 (2016), 231-240.
[10] R.L. Graves, C. Tufts, Z.F. Meisel, D. Polsky, L. Ungar,
and R.M. Merchant, Opioid discussion in the
Twittersphere, Subst Use Misuse 53 (2018), 2132-2139.
[11] DEA Houston Division, Slang Terms and Code Words: A
Reference for Law Enforcement Personnel, US Drug
Enforcement Administration, Washington, DC, 2018.
[12] A. Sarker and G. Gonzalez-Hernandez, An unsupervised
and customizable misspelling generator for mining noisy
health-related text sources, J Biomed Inform 88 (2018), 98-
107.
[13] A. Sarker et al., Social media mining for toxicovigilance:
automatic monitoring of prescription medication abuse
from Twitter, Drug Saf 39 (2016), 231-240.
[14] A. Sarker and G. Gonzalez, A corpus for mining drug-
related knowledge from Twitter chatter: language models
and their utilities, Data in Brief 10 (2017), 122-131.
[15] F. Pedregosa et al., Scikit-learn: machine learning in
Python, J Mach Learn Res 12 (2011), 2825-2830.
[16] M. Abadi et al., TensorFlow: Large-Scale Machine
Learning on Heterogeneous Distributed Systems, Google
Research, Mountain View, CA, 2016.
[17] J. Cohen, A coefficient of agreement for nominal scales,
Educ Psychol Meas 20 (1960), 37-46.
Address for correspondence
Abeed Sarker, Ph.D.
Mailing Address: Level 4, 423 Guardian Drive, Division of
Informatics, Department of Biostatistics, Epidemiology and
Informatics, Perelman School of Medicine, University of
Pennsylvania, Philadelphia, PA 19104, U.S.A.
Email: abeed@pennmedicine.upenn.edu
Phone: +1-215-746-1700
Table 4
– Sample tweets and classification in real-time with geolocation information (county level) and timestamps§
Tweet
Class
County
Timestamp
Enjoying this healthy breakfast
recommendation
frm @username. Oatmeal
w/raisins/walnuts/brown sugar
frm @username
Unrelated
Philadelphia
12:37:11 XX
-XX- 2015
@username i
shouldnt have done all that heroin this morning
Abuse
Allegheny
13:32:34 XX
-XX-2015
I know everyone is socialized different and wired uniquely. I
still want to smack a ******* for not staying in their lane.
Unrelated
Philadelphia
13:54:21 XX
-XX-2015
its on the news.. kensington oxys on the loose
Information
Philadelphia
15:01:55 XX
-XX-2015
A. Sarker et al. / Towards Automating Location-Specific Opioid Toxicosurveillance from Twitter via Data Science Methods 337
... However, this may not be efficient due to the huge use of informal and slang terms on social media by users. Moreover, some researchers have considered this problem as a text classification problem and explored different machine learning models in this domain [6]- [9]. However, the selection of the best features (e.g., count vectors, TF-IDF vectors, bag-of-words, n-grams, etc.) is often a very challenging task due to the presence of huge variations of the social text. ...
... To automate and to get better accuracy supervised machine learning models are used widely in text classification tasks, particularly in online medical-related text data [4], [7], [9], [17]. Sarker et al. [9] present an approach to perform localized surveillance of opioid abuse using machine learning models. ...
... To automate and to get better accuracy supervised machine learning models are used widely in text classification tasks, particularly in online medical-related text data [4], [7], [9], [17]. Sarker et al. [9] present an approach to perform localized surveillance of opioid abuse using machine learning models. They extract word n-grams and abuse indicating terms as features from tweets and explore different machine learning-based models to detect opioid abuse. ...
... Mining medical text available on social media and EHR for various purpose such as identify death due to drug overdose [173], mining adverse event [151], serendipitous drug usage [1,2], detecting medication [159], drug abuse detection [167]suffers many problems such as imbalanced data [173]; missing context tweet, hard interpretation of consumer vocabulary, misspellings or ambiguity with common words [159] and high lexical variability [165].These barriers remain a major issue in building effective classifiers for medical text. ...
... Mining medical text available on social media and EHR for various purpose such as identify death due to drug overdose [173], mining adverse event [151], serendipitous drug usage [1,2], detecting medication [159], drug abuse detection [167]suffers many problems such as imbalanced data [173]; missing context tweet, hard interpretation of consumer vocabulary, misspellings or ambiguity with common words [159] and high lexical variability [165].These barriers remain a major issue in building effective classifiers for medical text. ...
Article
Full-text available
The extensive growth of data in the health domain has increased the utility of Deep Learning in health. Deep learning is a highly advanced successor of artificial neural networks, having powerful computing ability. Due to the availability of fast data storage and hardware parallelism its popularity grows in the last five years. This in article presents a comprehensive literature review of research deploying deep learning medical imaging and medical NLP including tasks, pipelines, and challenges. In this work, we have presented an extensive survey of deep learning architecture deployed in the fields of medical imaging and medical natural language processing. This paper helps in identifying suitable combination of Deep learning, Natural language processing and medical imaging to enhance diagnosis. We have highlighted the major challenges in deploying deep learning in medical imaging and medical natural language processing. All the results are presented in pictorial form. This survey is very helpful for novices working in the area of health informatics.
... As an example, machine learning algorithms were derived to characterize opioid-related chatter and to automate opioid monitoring and toxicosurveillance on the Twitter platform. 47 Another illustrative example is described in a study using deep neural networks to detect medication mentions on the Twitter platform, which may be integrated for pharmacovigilance, toxicosurveillance, and other public health measures. 48 Similar to clinical studies containing an introductory table with demographic and baseline clinical outcome data on patients' characteristics, studies of social media may decide to provide baseline details of content before analysis. ...
Article
The prominence of social media in contemporary society has extended significantly into the health care arena, where both patients and health care providers have used social media platforms to gather, communicate, learn, and share medical content and personal experience in real time. The medical literature has also seen an exponential increase in the number of studies that use data derived from social media coverage of various medical issues and topics. In this guide, we present a step-by-step framework for health care professionals and researchers to conduct studies that use data from social media platforms. We present 6 overarching steps: focus on framing a question that is appropriate for social media evaluation, identification of social media outlet and selection criteria of content, systematic data extraction, assessment of quality of content and sources of bias, analysis of data, and interpretation of study findings. Each step is illustrated with published examples.
Article
Recruiting people from diverse backgrounds to participate in health research requires intentional and culture-driven strategic efforts. In this study, we utilize publicly available Twitter posts to identify targeted populations to recruit for our HIV prevention study. Natural language processing and machine learning classification methods were used to find self-declarations of ethnicity, gender, age group, and sexually-explicit language. Using the official Twitter API we collected 47.4 million tweets posted over 8 months from two areas geo-centered around Los Angeles. Using available tools (Demographer and M3), we identified the age and race of 5,392 users as likely young Black or Hispanic men living in Los Angeles. We then collected and analyzed their timelines to automatically find sex-related tweets, yielding 2,166 users. Despite a limited precision, our results suggest that it is possible to automatically identify users based on their demographic attributes and Twitter language characteristics for enrollment into epidemiological studies.
Article
Full-text available
Background Modifiable risky health behaviors, such as tobacco use, excessive alcohol use, being overweight, lack of physical activity, and unhealthy eating habits, are some of the major factors for developing chronic health conditions. Social media platforms have become indispensable means of communication in the digital era. They provide an opportunity for individuals to express themselves, as well as share their health-related concerns with peers and health care providers, with respect to risky behaviors. Such peer interactions can be utilized as valuable data sources to better understand inter-and intrapersonal psychosocial mediators and the mechanisms of social influence that drive behavior change. Objective The objective of this review is to summarize computational and quantitative techniques facilitating the analysis of data generated through peer interactions pertaining to risky health behaviors on social media platforms. Methods We performed a systematic review of the literature in September 2020 by searching three databases—PubMed, Web of Science, and Scopus—using relevant keywords, such as “social media,” “online health communities,” “machine learning,” “data mining,” etc. The reporting of the studies was directed by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Two reviewers independently assessed the eligibility of studies based on the inclusion and exclusion criteria. We extracted the required information from the selected studies. Results The initial search returned a total of 1554 studies, and after careful analysis of titles, abstracts, and full texts, a total of 64 studies were included in this review. We extracted the following key characteristics from all of the studies: social media platform used for conducting the study, risky health behavior studied, the number of posts analyzed, study focus, key methodological functions and tools used for data analysis, evaluation metrics used, and summary of the key findings. The most commonly used social media platform was Twitter, followed by Facebook, QuitNet, and Reddit. The most commonly studied risky health behavior was nicotine use, followed by drug or substance abuse and alcohol use. Various supervised and unsupervised machine learning approaches were used for analyzing textual data generated from online peer interactions. Few studies utilized deep learning methods for analyzing textual data as well as image or video data. Social network analysis was also performed, as reported in some studies. Conclusions Our review consolidates the methodological underpinnings for analyzing risky health behaviors and has enhanced our understanding of how social media can be leveraged for nuanced behavioral modeling and representation. The knowledge gained from our review can serve as a foundational component for the development of persuasive health communication and effective behavior modification technologies aimed at the individual and population levels.
Research
Full-text available
We present a synopsis of publications focused on machine learning (ML) or artificial intelligence (AI) applications in healthcare for the year 2019. We appreciate the work of researchers and authors who have contributed significantly to the advancement of science in this area.
Article
Full-text available
Importance Automatic curation of consumer-generated, opioid-related social media big data may enable real-time monitoring of the opioid epidemic in the United States. Objective To develop and validate an automatic text-processing pipeline for geospatial and temporal analysis of opioid-mentioning social media chatter. Design, Setting, and Participants This cross-sectional, population-based study was conducted from December 1, 2017, to August 31, 2019, and used more than 3 years of publicly available social media posts on Twitter, dated from January 1, 2012, to October 31, 2015, that were geolocated in Pennsylvania. Opioid-mentioning tweets were extracted using prescription and illicit opioid names, including street names and misspellings. Social media posts (tweets) (n = 9006) were manually categorized into 4 classes, and training and evaluation of several machine learning algorithms were performed. Temporal and geospatial patterns were analyzed with the best-performing classifier on unlabeled data. Main Outcomes and Measures Pearson and Spearman correlations of county- and substate-level abuse-indicating tweet rates with opioid overdose death rates from the Centers for Disease Control and Prevention WONDER database and with 4 metrics from the National Survey on Drug Use and Health for 3 years were calculated. Classifier performances were measured through microaveraged F1 scores (harmonic mean of precision and recall) or accuracies and 95% CIs. Results A total of 9006 social media posts were annotated, of which 1748 (19.4%) were related to abuse, 2001 (22.2%) were related to information, 4830 (53.6%) were unrelated, and 427 (4.7%) were not in the English language. Yearly rates of abuse-indicating social media post showed statistically significant correlation with county-level opioid-related overdose death rates (n = 75) for 3 years (Pearson r = 0.451, P < .001; Spearman r = 0.331, P = .004). Abuse-indicating tweet rates showed consistent correlations with 4 NSDUH metrics (n = 13) associated with nonmedical prescription opioid use (Pearson r = 0.683, P = .01; Spearman r = 0.346, P = .25), illicit drug use (Pearson r = 0.850, P < .001; Spearman r = 0.341, P = .25), illicit drug dependence (Pearson r = 0.937, P < .001; Spearman r = 0.495, P = .09), and illicit drug dependence or abuse (Pearson r = 0.935, P < .001; Spearman r = 0.401, P = .17) over the same 3-year period, although the tests lacked power to demonstrate statistical significance. A classification approach involving an ensemble of classifiers produced the best performance in accuracy or microaveraged F1 score (0.726; 95% CI, 0.708-0.743). Conclusions and Relevance The correlations obtained in this study suggest that a social media–based approach reliant on supervised machine learning may be suitable for geolocation-centric monitoring of the US opioid epidemic in near real time.
Article
Full-text available
Background: Data collection and extraction from noisy text sources such as social media typically rely on keyword-based searching/listening. However, health-related terms are often misspelled in such noisy text sources due to their complex morphology, resulting in the exclusion of relevant data for studies. In this paper, we present a customizable data-centric system that automatically generates common misspellings for complex health-related terms, which can improve the data collection process from noisy text sources. Materials and methods: The spelling variant generator relies on a dense vector model learned from large, unlabeled text, which is used to find semantically close terms to the original/seed keyword, followed by the filtering of terms that are lexically dissimilar beyond a given threshold. The process is executed recursively, converging when no new terms similar (lexically and semantically) to the seed keyword are found. The weighting of intra-word character sequence similarities allows further problem-specific customization of the system. Results: On a dataset prepared for this study, our system outperforms the current state-of-the-art medication name variant generator with best F1-score of 0.69 and F14-score of 0.78. Extrinsic evaluation of the system on a set of cancer-related terms showed an increase of over 67% in retrieval rate from Twitter posts when the generated variants are included. Discussion: Our proposed spelling variant generator has several advantages over the existing spelling variant generators-(i) it is capable of filtering out lexically similar but semantically dissimilar terms, (ii) the number of variants generated is low, as many low-frequency and ambiguous misspellings are filtered out, and (iii) the system is fully automatic, customizable and easily executable. While the base system is fully unsupervised, we show how supervision may be employed to adjust weights for task-specific customizations. Conclusion: The performance and relative simplicity of our proposed approach make it a much-needed spelling variant generation resource for health-related text mining from noisy sources. The source code for the system has been made publicly available for research.
Article
Full-text available
Background: Despite the continuing epidemic of opioid misuse, data on the prevalence of prescription opioid use, misuse, and use disorders are limited. Objective: To estimate the prevalence of prescription opioid use, misuse, and use disorders and motivations for misuse among U.S. adults. Design: Survey. Setting: The 2015 National Survey on Drug Use and Health (NSDUH). Participants: 72 600 eligible civilian, noninstitutionalized adults were selected for NSDUH, and 51 200 completed the survey interview. Measurements: Prescription opioid use, misuse, and use disorders. Results: Weighted NSDUH estimates suggested that, in 2015, 91.8 million (37.8%) U.S. civilian, noninstitutionalized adults used prescription opioids; 11.5 million (4.7%) misused them; and 1.9 million (0.8%) had a use disorder. Among adults with prescription opioid use, 12.5% reported misuse; of these, 16.7% reported a prescription opioid use disorder. The most commonly reported motivation for misuse was to relieve physical pain (63.4%). Misuse and use disorders were most commonly reported in adults who were uninsured, were unemployed, had low income, or had behavioral health problems. Among adults with misuse, 59.9% reported using opioids without a prescription, and 40.8% obtained prescription opioids for free from friends or relatives for their most recent episode of misuse. Limitation: Cross-sectional, self-reported data. Conclusion: More than one third of U.S. civilian, noninstitutionalized adults reported prescription opioid use in 2015, with substantial numbers reporting misuse and use disorders. Relief from physical pain was the most commonly reported motivation for misuse. Economic disadvantage and behavioral health problems may be associated with prescription opioid misuse. The results suggest a need to improve access to evidence-based pain management and to decrease excessive prescribing that may leave unused opioids available for potential misuse. Primary funding source: U.S. Department of Health and Human Services.
Article
Full-text available
The U.S. opioid epidemic is continuing, and drug overdose deaths nearly tripled during 1999-2014. Among 47,055 drug overdose deaths that occurred in 2014 in the United States, 28,647 (60.9%) involved an opioid (1). Illicit opioids are contributing to the increase in opioid overdose deaths (2,3). In an effort to target prevention strategies to address the rapidly changing epidemic, CDC examined overall drug overdose death rates during 2010-2015 and opioid overdose death rates during 2014-2015 by subcategories (natural/semisynthetic opioids, methadone, heroin, and synthetic opioids other than methadone).* Rates were stratified by demographics, region, and by 28 states with high quality reporting on death certificates of specific drugs involved in overdose deaths. During 2015, drug overdoses accounted for 52,404 U.S. deaths, including 33,091 (63.1%) that involved an opioid. There has been progress in preventing methadone deaths, and death rates declined by 9.1%. However, rates of deaths involving other opioids, specifically heroin and synthetic opioids other than methadone (likely driven primarily by illicitly manufactured fentanyl) (2,3), increased sharply overall and across many states. A multifaceted, collaborative public health and law enforcement approach is urgently needed. Response efforts include implementing the CDC Guideline for Prescribing Opioids for Chronic Pain (4), improving access to and use of prescription drug monitoring programs, enhancing naloxone distribution and other harm reduction approaches, increasing opioid use disorder treatment capacity, improving linkage into treatment, and supporting law enforcement strategies to reduce the illicit opioid supply.
Article
Full-text available
In this data article, we present to the data science, natural language processing and public heath communities an unlabeled corpus and a set of language models. We collected the data from Twitter using drug names as keywords, including their common misspelled forms. Using this data, which is rich in drug-related chatter, we developed language models to aid the development of data mining tools and methods in this domain. We generated several models that capture (i) distributed word representations and (ii) probabilities of n-gram sequences. The data set we are releasing consists of 267,215 Twitter posts made during the four-month period— November, 2014 to February, 2015. The posts mention over 250 drug-related key words. The language models encapsulate semantic and sequential properties of the texts.
Article
Full-text available
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.
Article
Better understanding of the dynamics of the current U.S. overdose epidemic may aid in the development of more effective prevention and control strategies. We analyzed records of 599,255 deaths from 1979 through 2016 from the National Vital Statistics System in which accidental drug poisoning was identified as the main cause of death. By examining all available data on accidental poisoning deaths back to 1979 and showing that the overall 38-year curve is exponential, we provide evidence that the current wave of opioid overdose deaths (due to prescription opioids, heroin, and fentanyl) may just be the latest manifestation of a more fundamental longer-term process. The 38+ year smooth exponential curve of total U.S. annual accidental drug poisoning deaths is a composite of multiple distinctive subepidemics of different drugs (primarily prescription opioids, heroin, methadone, synthetic opioids, cocaine, and methamphetamine), each with its own specific demographic and geographic characteristics.
Article
Background: The rise in opioid use and overdose has increased the importance of improving data collection methods for the purpose of targeting resources to high-need populations and responding rapidly to emerging trends. Objective: To determine whether Twitter data could be used to identify geographic differences in opioid-related discussion and whether opioid topics were significantly correlated with opioid overdose death rate. Methods: We filtered approximately 10 billion tweets for keywords related to opioids between July 2009 and October 2015. The content of the messages was summarized into 50 topics generated using Latent Dirchlet Allocation, a machine learning analytic tool. The correlation between topic distribution and census region, census division, and opioid overdose death rate were quantified. Results: We evaluated a tweet cohort of 84,023 tweets from 72,211 unique users across the US. Unique opioid-related topics were significantly correlated with different Census Bureau divisions and with opioid overdose death rates at the state and county level. Drug-related crime, language of use, and online drug purchasing emerged as themes in various Census Bureau divisions. Drug-related crime, opioid-related news, and pop culture themes were significantly correlated with county-level opioid overdose death rates, and online drug purchasing was significantly correlated with state-level opioid overdoses. Conclusions: Regional differences in opioid-related topics reflect geographic variation in the content of Twitter discussion about opioids. Analysis of Twitter data also produced topics significantly correlated with opioid overdose death rates. Ongoing analysis of Twitter data could provide a means of identifying emerging trends related to opioids.
Article
The United States is in the midst of the worst drug addiction epidemic in its history. Prescriptions for and deaths from opioids both quadrupled between 1995 and 2010. By 2015, an estimated 92 million individuals in the United States were prescribed an opioid and there were more than 33 000 deaths from an opioid-involved overdose.
Article
Background The misuse of prescription opioids (MUPO) is a leading public health concern. Social media are playing an expanded role in public health research, but there are few methods for estimating established epidemiological metrics from social media. The purpose of this study was to demonstrate that the geographic variation of social media posts mentioning prescription opioid misuse strongly correlates with government estimates of MUPO in the last month. Methods We wrote software to acquire publicly available tweets from Twitter from 2012 to 2014 that contained at least one keyword related to prescription opioid use (n = 3,611,528). A medical toxicologist and emergency physician curated the list of keywords. We used the semantic distance (SemD) to automatically quantify the similarity of meaning between tweets and identify tweets that mentioned MUPO. We defined the SemD between two words as the shortest distance between the two corresponding word-centroids. Each word-centroid represented all recognized meanings of a word. We validated this automatic identification with manual curation. We used Twitter metadata to estimate the location of each tweet. We compared our estimated geographic distribution with the 2013–2015 National Surveys on Drug Usage and Health (NSDUH). ResultsTweets that mentioned MUPO formed a distinct cluster far away from semantically unrelated tweets. The state-by-state correlation between Twitter and NSDUH was highly significant across all NSDUH survey years. The correlation was strongest between Twitter and NSDUH data from those aged 18–25 (r = 0.94, p < 0.01 for 2012; r = 0.94, p < 0.01 for 2013; r = 0.71, p = 0.02 for 2014). The correlation was driven by discussions of opioid use, even after controlling for geographic variation in Twitter usage. Conclusions Mentions of MUPO on Twitter correlate strongly with state-by-state NSDUH estimates of MUPO. We have also demonstrated that a natural language processing can be used to analyze social media to provide insights for syndromic toxicosurveillance.
Article
The United States is in the midst of an opioid overdose epidemic. Between 1999 and 2010, prescription opioid–related overdose deaths increased substantially in parallel with increased prescribing of opioids.¹ In 2015, opioid-involved drug overdoses accounted for 33 091 deaths, approximately half involving prescription opioids.² Additionally, an estimated 2 million individuals in the United States have opioid use disorder (addiction) associated with prescription opioids, accounting for an estimated $78.5 billion in economic costs annually.³ Proven strategies are available to manage chronic pain effectively without opioids, and changing prescribing practices is an important step in addressing the opioid overdose epidemic and its adverse effects on US communities.