ArticlePDF Available

Abstract and Figures

Background Recruitment for research is usually expensive and time consuming. Facebook (FB) recruitment has become widely utilized in recent years. The main aim of this study was to assess FB as a recruitment tool in a study for Spanish- and English-speaking smokers. Additionally, the study set out to compare performance of ads by language (Spanish vs. English), location (U.S. vs. San Francisco) and content (self-efficacy ad vs. fear appeal ad). Methods Participants of a one-condition smoking cessation webapp study were recruited utilizing FB ads and posts through two phases: a recruitment-focused phase and an experimental phase comparing language, location and content. Results During the recruitment phase 581 participants in total (U.S. = 540, San Francisco = 41) provided consent. Of the U.S. participants 275 were Spanish-speakers and 265 English-speakers. The cost-per-consent was $25.81 for Spanish-speakers, and $15.49 for English-speakers. During the experimental phase U.S. users performed better (i.e. more clicks, engagement and social reach) than San Francisco users, Spanish-speakers engaged more than English-speakers, and the self-efficacy ad performed better than the fear appeal ad. Conclusions This study showed that although there were differences in cost-per-consent for Spanish- and English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing performance of ads by location, language, and ad content may contribute to developing more efficient campaigns.
Content may be subject to copyright.
Contents lists available at ScienceDirect
Internet Interventions
journal homepage: www.elsevier.com/locate/invent
Facebook for recruiting Spanish- and English-speaking smokers
Eduardo L. Bunge
1
, Lesley A. Taylor
1
, Melissa Bond
1
, Taylor N. Stephens
1
, Kara Nishimuta
1
,
Alinne Z. Barrera
1
, Robert Wickham, Ricardo F. Muñoz
,1
Palo Alto University, United States of America
ARTICLE INFO
Keywords:
Recruitment
Facebook
Smoking
Cessation
Spanish-speakers
ABSTRACT
Background: Recruitment for research is usually expensive and time consuming. Facebook (FB) recruitment has
become widely utilized in recent years. The main aim of this study was to assess FB as a recruitment tool in a
study for Spanish- and English-speaking smokers. Additionally, the study set out to compare performance of ads
by language (Spanish vs. English), location (U.S. vs. San Francisco) and content (self-ecacy ad vs. fear appeal
ad).
Methods: Participants of a one-condition smoking cessation webapp study were recruited utilizing FB ads and
posts through two phases: a recruitment-focused phase and an experimental phase comparing language, location
and content.
Results: During the recruitment phase 581 participants in total (U.S. = 540, San Francisco = 41) provided
consent. Of the U.S. participants 275 were Spanish-speakers and 265 English-speakers. The cost-per-consent was
$25.81 for Spanish-speakers, and $15.49 for English-speakers. During the experimental phase U.S. users per-
formed better (i.e. more clicks, engagement and social reach) than San Francisco users, Spanish-speakers en-
gaged more than English-speakers, and the self-ecacy ad performed better than the fear appeal ad.
Conclusions: This study showed that although there were dierences in cost-per-consent for Spanish- and
English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing per-
formance of ads by location, language, and ad content may contribute to developing more ecient campaigns.
1. Introduction
Recruitment for human subjects research is usually expensive and
time consuming (Thornton et al., 2016). Online recruitment has been
shown to be more ecient than oine methods in terms of total par-
ticipants enrolled, enables accessibility to larger and more diverse
participants, requires shorter recruitment periods, and reduces overall
study recruitment costs (Christensen et al., 2017;Lane et al., 2015;
Whitaker et al., 2017). However, there are concerns regarding sample
representativeness (Choi et al., 2017) and minimal research exists on
reach and enrollment of participants from diverse backgrounds (e.g.,
non-English speaking; Lane et al., 2015;Kayrouz et al., 2016;Bunge
et al., 2018).
The most commonly utilized online recruitment sources include:
Google Ads, Amazon Mechanical Turk (AMT), Facebook (FB), and
Craigslist (Temple and Brown, 2011). Evidence suggests that these
methods of recruitment result in similar samples and are as cost-
eective as more traditional face-to-face recruitment methods
(Thornton et al., 2016). A systematic review of 110 health and mental
health studies utilizing FB recruitment concluded that, on average, re-
cruitment costs were $6.79 per participant and few dierences were
noted when compared to samples recruited using face-to-face methods
(Thornton et al., 2016). Furthermore, FB may also be more time-ef-
fective compared to traditional recruitment methods (e.g., emails, print
advertisements and media releases), with one study nding participants
were recruited up to 2.5 times faster (Kayrouz et al., 2016).
FB allows for various recruitment options (see Akers and Gordon,
2018), such as posting general ads, promoting public fan pages, and
boosting posts. One study found that of these techniques, promoting FB
pages and boosting posts demonstrated the largest impact on recruit-
ment rates (Kayrouz et al., 2016). Furthermore, FB oers the promising
advantage of potential respondent driven sampling, also known as
snowball sampling, which capitalizes on FB's inherent peer networking
structure (i.e., FB users recruit other FB users or non-FB peers; Pedersen
https://doi.org/10.1016/j.invent.2019.02.002
Received 16 November 2018; Received in revised form 9 February 2019; Accepted 13 February 2019
This research was supported by funds from the California Tobacco-Related Disease Research Grants Program Oce of the University of California (PI: Muñoz),
grant number 24RT-0027. This research was supported in part by a grant from Google Adwords.
Corresponding author at: Palo Alto University, 1791 Arastradero Rd., Palo Alto, CA 94304, United States of America.
E-mail address: rmunoz@paloaltou.edu (R.F. Muñoz).
1
i4Health at Palo Alto University.
Internet Interventions 17 (2019) 100238
Available online 26 February 2019
2214-7829/ © 2019 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/BY-NC-ND/4.0/).
T
and Kurz, 2016). FB maximizes the potential to access the aforemen-
tioned hard-to-reach individuals,given the omnipresent use of FB
among various populations (Lane et al., 2015).
Several smoking cessation studies recruited participants through FB
(Akers and Gordon, 2018;Carlini et al., 2014;Ramo and Prochaska,
2012;Ramo et al., 2014, 2015a, 2015b;Sadasivam et al., 2013, 2016;
Thornton et al., 2013). Ramo and Prochaska (2012) examined FB as a
recruitment channel for surveying tobacco and other substance use for
individuals 1825 and reported a nal cost per valid, completed survey
of $4.28. FB estimated that 2.8% of accounts for people aged 1825
were reached through tobacco and marijuana keywords (Ramo and
Prochaska, 2012). Ramo et al. (2014) employed multiple ad types for a
smoking cessation trial over a seven-week period, such as ads in news
feeds, promoted posts, sponsored stories, and standard ads, averaging
$8.80 per eligible, consented participant. They found that images of
smoking and news feed ads yielded the greatest reach and clicks at the
lowest cost, and posited that this may be partially due to news feed ads
being viewable by mobile device. Ramo et al. (2015a, 2015b) con-
ducted a mixed-methods study surveying young adults about tobacco
and social media use to see if participants were interested in a smoking
cessation intervention through FB and, if so, how FB should be used to
help this population quit smoking. About one third of their sample
(31%) reported that they would use FB to quit smoking, and interest in
using FB was greater among those who were more motivated to quit,
had made a quit attempt within the last year, and had previously used
an online source to quit (Ramo et al., 2015a, 2015b). Sadasivam et al.
(2013) created a technology-assisted tobacco intervention to recruit
people through peer marketing. The initial wave of their participants,
or their seeds,(n = 190) were recruited through FB ads, the peer
recruits (n = 569) were recruited through the seeds, and the overall
cost per smoker recruited was $29.80 (Sadasivam et al., 2013). Inter-
estingly, FB can target people who are not searching for smoking ces-
sation information as well, a benecial recruitment outcome that would
not be accessible with other online recruitment methods such as Google
Ads. For example, Akers and Gordon (2018) conducted a clinical trial to
teach support skills to partners of smokeless users and recruited 1145
female partners of male smokeless tobacco users over a 15-month
period.
While FB oers the benet of broad reach, potential drawbacks
include privacy concerns and a general lack of clear guidance around
human subjects issues(Pedersen and Kurz, 2016, p. 5). Additionally,
FB recruitment outcomes vary depending on how campaigns are de-
signed and conducted, with some studies reporting that FB is more
expensive than other similar online methods (Frandsen et al., 2014;
Hener et al., 2013). Hener et al. (2013) compared the cost of various
recruitment strategies (i.e., standard media, broadcast emails, word of
mouth, Google AdWords, and social media) for a pilot study on web-
based Acceptance and Commitment Therapy (ACT) for smoking cessa-
tion. Results demonstrated that FB ads and posts were the least cost
eective method, costing $172.76 per participant vs. $46.98 per par-
ticipant for standard media, for example, which included press releases
on local television, radio, newspaper, and online news outlets (Hener
et al., 2013). Similarly, for an in-person smoking cessation clinical trial,
Frandsen et al. (2014) compared FB ads to traditional methods (yers,
word of mouth, and newspaper advertisement) and found that the cost
per participant using traditional methods was less than half of that for
FB ads.
Other common limitations in the existing research that utilized FB
recruitment strategies include: restriction of sample to individuals that
have internet access and use FB regularly (Ramo et al., 2010), and ads
that do not include diverse languages (Amon et al., 2014). To date, no
FB recruitment studies have reported dierences by language for
smoking cessation studies. Although Carlini et al. (2014) conducted a
study for low English prociency Brazilians, with FB ads in Portuguese
(the cost was $31.10 per smoker), it did not compare recruitment
outcomes by language (English vs. Portuguese).
Few studies have reported on recruitment methods for Spanish-
speaking populations for smoking cessation studies (Graham et al.,
2012;Muñoz et al., 2006). Graham et al. (2012) focused on developing
culturally-specic advertisements to recruit Spanish-speaking popula-
tions for a smoking cessation website. They rotated ads systematically
across four popular Latino websites and found that loss-framed ads
yielded a higher click-through rate than gain-framed ads, and surface-
targeted ads outperformed deep-targeted ads for clicks, click-through
rate, and number of registrants (Graham et al., 2012). A series of
smoking cessation studies were also done by Muñoz and colleagues in
both English and Spanish (Muñoz et al., 2006;Muñoz et al., 2009;
Muñoz et al., 2012;Leykin et al., 2012). Most of the recruitment eorts
were done through Google Ads. These studies examined the feasibility
but not the costs of recruiting Spanish-speaking smokers worldwide. For
example, in one study, 94,158 individuals from 152 countries and
territories visited the site (Muñoz et al., 2012). A total of 9173 Spanish-
and English-speaking smokers provided consent, however the recruit-
ment costs were not reported (Muñoz et al., 2012).
The aim of the current study was to assess FB as a recruitment tool
in a study for Spanish- and English-speaking smokers. Phase 1 examined
ad lib recruitment of English- and Spanish-speaking adult smokers
utilizing FB ads and posts for a smoking cessation webapp (recruitment-
focused phase). Phase 2 systematically compared performance of re-
cruitment methods by language (Spanish-speakers vs. English-
speakers), location (San Francisco vs. U.S.) and content (self-ecacy ad
vs. fear appeal ad) (experimental phase). Two main metrics were ana-
lyzed: number of clicks and engagement.
2. Method
2.1. Participants
Participants were recruited if they resided in the U.S. and reported
being 18 years of age and older. The recruitment period was from
January 9th, 2018 to May 16th, 2018. Participants were recruited for a
one-condition smoking cessation study utilizing a webapp, where they
completed questionnaires about their smoking habits and quit con-
dence, tracked their quit attempts, and received periodic follow up
texts to track their quit progress. The responsive webapp was designed
to work in a web browser on a computer or mobile device and did not
require downloading a specic app or other software.
2.2. Assessments and measures
An extensive inventory of metrics exists to assess the success and
connectivity of advertisement/recruitment strategies employed. The
following metrics have been dened based on the Facebook Business
(2018) stipulations, as FB is the recruitment channel of interest for this
article. Metrics were divided in two: those based on FB algorithms and
those based on participant behaviors. The metrics based on FB algo-
rithms include: Click-Through Rate, Impression, Reach, Social Reach,
Relevance, Unique Outbound Click. The metrics based on participant
behaviors include: Clicks, Consents, Conversion rate, Cost-per-consent,
Cost-per- click, Link clicks, Post comments, Post reactions, Post shares,
Shares. See Table 1 for a description of each metric type.
2.3. Procedures
The initial recruitment channels utilized in this study were Google
AdWords, announcements in relevant list serves, local iers, and dis-
tribution of 3 × 5 cards in Spanish and English with links to the web-
site. The low recruitment rates led the researchers to add FB recruit-
ment. This study only reports on participants recruited through FB.
Participants were recruited in two phases: a recruitment-focused
phase and an experimental phase comparing language, location and
content (self-ecacy ad vs. fear appeal ad). The recruitment-focused
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
2
phase acted as a pilot test for recruiting participants with several dif-
ferent strategies that varied at a fast pace. Once the recruitment target
was reached, the experimental phase focused on systematic compar-
isons between ads that performed well during the previous phase.
2.3.1. Recruitment-focused phase
The recruitment-focused phase primarily aimed to recruit as many
participants as possible in a short period of time. Starting on January
9th 2018 up to April 30th 2018, two community managers (CM) were
hired to recruit participants. One CM focused on recruitment of English-
speaking smokers and the other CM focused on Spanish-speaking
smokers. Each CM implemented her own recruitment strategy for her
target audience. The English strategy mostly focused on Facebook Ads
through the Ads Manager tool and the Spanish strategy mostly focused
on boosted posts directly from Facebook. Across all campaigns between
January 9th and May 16th, 2018 a total of 70 ads were run over
19 weeks; 50 ads in English and 20 ads in Spanish. Regarding posts, a
total of 62 posts were run over 19 weeks; 15 posts in English and 47
posts in Spanish. During this phase, both ads and posts presented dif-
ferent text content, images, starting time and locations.
2.3.2. Experimental phase
The second phase systematically compared performance of recruit-
ment methods by language (Spanish-speakers vs. English-speakers),
location (San Francisco only vs. US) and content (self-ecacy ad vs.
fear appeal ad). The experimental phase ran from May 3rd May 16th
2018. For this phase two of the best performing ads from the recruit-
ment-focused phase were selected based on CPC, CTR, and relevance.
Campaigns were run by language (Spanish-speakers vs. English-
speakers), location (San Francisco vs. US) and content (self-ecacy ad
vs. fear appeal ad). For direct comparison, the ad images and content
are shown in Fig. 1. Content for the self-ecacy ad included How to
Quittext, while the fear appeal ad included Just one cigarette a day
can double your risk of heart diseasetext.
The complete study was approved by both the University of
California San Francisco (15-17597) and Palo Alto University (15-042-
H) Institutional Review Boards.
2.4. Analysis
Due to large variability and lack of controls during Phase 1, in-
ferential statistics could not be completed for the recruitment-focused
phase, so only descriptive analyses are provided. Phase 2, the experi-
mental phase, was analyzed using a 2 × 2 × 2 binomial logistic re-
gression with the three factors of language, location and content. The
binomial logistic regression allows for direct comparison across lan-
guage, location, and content despite the dierence in Impressions or
Reach because it treats each dependent variable as a yes/no outcome
with a total N of the denominator variable (e.g., Impressions or Reach).
Since Facebook does not reveal how their unique metrics (i.e., re-
levance) are calculated, only continuous outcomes with enough power
to detect dierences were included in the analysis. Because both Reach
and impressions are determined by FB proprietary algorithms, there are
likely dierences in how dierent populations are targeted. Although
impressions are often used as a base to calculate common industry
metrics (e.g., CTR, CPC), Reach may be more appropriate for re-
searchers looking to evaluate dierences on an individual level. Both
Impressions and Reach are determined by FB's proprietary algorithms,
but once an individual is reached (sees the ad at least once), some
metrics are based on the individual's behavior (clicks, shares, and en-
gagement), not the FB algorithm. Therefore, clicks, total engagement,
and social reach were run with both Impressions and Reach as de-
nominators. UOC was only run with Reach as denominator because it
measures unique clicks, and using impressions (which are not unique)
Table 1
Campaign type and metrics.
Campaign type
Ads Paid advertisements that may appear in any given user's news feed or side-panel (run across FB, Instagram, Audience Network, and Messenger), can be targeted to specic
populations.
Posts FB Posts (text, images, or videos) are displayed within users' news feeds and can be self-posted or generated by their friends or followed pages.
Boosted Posts. Users can pay to boosttheir posts' visibility.
Metrics based on Facebook
Click-Through Rate (CTR) Percentage of times individuals see an ad and click on a link. The ratio of times that individuals click a link in an ad compared to the amount of
times these individuals see the ad.
Engagement The total number of actions that people take involving the ads.
Impressions Number of times an ad is shown on screen for the targeted audience.
Reach Number of individuals that see an ad at least once. (This diers from impressions which includes multiple views of an ad by the same
individual.)
Relevance Calculated based on an ad's positive feedback (e.g., app installs, clicks, video views) and negative feedback (e.g., User clicks I don't want to see
thison the ad) and ranges from 1 (lowest) to 10 (highest). Higher relevancy can lower cost and improve delivery.
Social reach Number of individuals that see an ad when displayed with social information, acting as word-of-mouth for advertisers (e.g., other FB friends
who engaged with the ad or FB page).
Unique Outbound Clicks (UOC) Number of clicks on links that redirect users oFB and to the targeted site.
Metrics based on participant behaviors
Clicks Number of clicks on an ad. Counts multiple types of clicks, such as links to other destinations and links to expanded ad experiences.
Consents Number of individuals consented to participate in the research study.
Conversion rate Metric calculated based on the Unique Outbound Clicks (UOC; see denition above) and FB-attributed consents obtained.
Cost per consent Cost per consenting individual obtained.
Cost per link click (CPC) Average cost per each link click, benchmark for ad eciency and performance. Calculated by dividing the total amount spent (U.S. $) per link click.
Link clicks Number of clicks on ad links to specic destinations or experiences, both on and oFB-owned properties (e.g., websites, App stores, click-to-call, etc.).
Post comments Number of comments per ad, which counts all comments made by individuals while the ad was running.
Post reactions Number of times individual users reacted to a post with a click, which includes reactions such as likeand love.
Post shares Number of shares per ad while running (may also include Instagram shares sent to inboxes), but does not count engagement with the post after the
share.
Notes. All denitions are based on Facebook Business' (2018) Glossary of Terms.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
3
as a denominator could cause confounded results.
3. Results
During the recruitment period, between January 9th and May 16th,
2018, the 70 ads and 62 posts received 2,357,181 impressions and
yielded 30,169 unique outbound clicks for a total cost of $19,289.15.
Total time spent managing the ads by the Spanish speaking and English
speaking CMs was 164 h and 139 h, respectively.
3.1. Recruitment-focused phase
During the recruitment phase there were 581 participants in total:
540 were from elsewhere in the U.S. and 41 from San Francisco. Of the
540 U.S. participants that provided consent, 275 were Spanish-speakers
and 265 English-speakers. A total budget of $11,203.40 was allocated
for both languages; the nal budget used was $7098.96 for Spanish
language ads and $4104.44 for English language ads. Metrics generated
for the Spanish and English language ads, respectively, were 1.83% vs.
1.77% for CTR, $0.43 vs. $0.44 for CPC, and a cost-per-consent of
$25.81 vs. $15.49 (See Table 2).
3.2. Experimental phase
During the experimental phase a total of 43 participants were re-
cruited (37 for U.S. and six for San Francisco). A total budget of $2600
was allocated for both languages and the nal budget used was for
$1299.87 Spanish and $1298.72 for English. Table 3 shows descriptive
outcomes for ads only by location. Of the 37 U.S. participants that
provided study consent, 18 were Spanish-speakers and 19 English-
speakers. For Spanish-speakers, CTR was 1.41%, CPC was $0.41, and
cost-per-consent was $27.78. For English-speakers, CTR was 1.44%,
Fig. 1. Example of Spanish and English Facebook ads.
Notes. Ads (a) and (b) illustrate fear appeal, and ads (c) and (d) illustrate self-ecacy messages.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
4
CPC was $0.49, and cost-per-consent was $26.31. Of the six SF parti-
cipants that provided consent, four were Spanish-speakers and two
English-speakers. For Spanish-speakers, CTR was 0.64%, CPC was
$1.23, and cost-per-consent was $199.98. For English-speakers, CTR
was 0.60%, CPC was $1.09, and cost-per-consent was $399.39 (See
Table 3).
3.3. Impressions as denominator
Chi-square values and odd-ratios for the logistic binomial regres-
sions calculated with impressions as the denominator can be found in
Table 4. For clicks, there was a signicant main eect observed for
location (X
2
= 581.35, p < .005) and content (X
2
= 35.02,
p < .005):U.S. participants were 2.28 times more likely to click an ad
than San Francisco participants, individuals were 1.23 times more
likely to click on the self-ecacy ad than the fear appeal ad. For En-
gagement, there was a signicant main eect for location (X
2
= 832.88,
p < .005), language (X
2
= 9.65, p < .005), and content (X
2
= 29.10,
p < .005): U.S. participants were 2.56 times more likely to engage;
Spanish-speaking participants were 1.11 times more likely to engage.
The content variable showed that the self-ecacy ad was 1.20 times
more likely to be engaged with compared to the fear appeal ad. Re-
garding Social Reach, there was a signicant main eect for location
(X
2
= 31.25, p < .005), language (X
2
= 1077.74, p < .005), and
content (X
2
= 19.44, p < .005), as well as signicant two-way inter-
actions for location with language (X
2
= 82.63, p < .005) and content
(X
2
= 6.35, p < .05). The odds-ratios indicate that, overall, U.S. par-
ticipants were 1.52 times more likely to reach a new person via social
means (i.e. shares, comments, etc.), Spanish-speaking participants were
7.32 times more likely to reach new users via social means, and the self-
ecacy ad was 1.40 times more likely to reach people via social means.
These main eects are qualied by the signicant interactions of lo-
cation by language and content. Specically, Spanish-speakers within
San Francisco (OR = 14.39) were far more likely to reach new users via
social means than were Spanish-speakers in the U.S. (OR = 3.72). Si-
milarly, the self-ecacy ad shown within San Francisco (OR = 1.70)
was more likely to reach new users via social means than the same ad in
the U.S. sample (OR = 1.15).
3.4. Reach as denominator
Chi-square values and odd-ratios for the logistic binomial regres-
sions run with Reach as the denominator can be found in Table 5. When
run with Reach as the denominator, signicant main eects of location
(X
2
= 192.64, p < .005), language (X
2
= 14.43, p < .005), and con-
tent (X
2
= 20.13, p < .005) were found for clicks, as well as a sig-
nicant interaction between location and content (X
2
= 6.89,
p < .01). In general, users in the U.S. were 1.61 times more likely to
click, Spanish-speakers were 1.14 times more likely to click, and the
self-ecacy ad received 1.17 times the amount of clicks than did the
fear appeal ad. Furthermore, the self-ecacy ad was far more eective
at receiving clicks in the U.S. (OR = 1.28) than it was in San Francisco
(OR = 1.07). For engagement, main eects were revealed for location
(X
2
= 327.64, p < .005), language (X
2
= 41.96, p < .005), and con-
tent (X
2
= 15.25, p < .005), such that users in the U.S. were 1.81
times more likely to engage, Spanish-speakers were 1.24 times more
likely to engage, while the self-ecacy ad was 1.14 times more likely to
be engaged with. Signicant interactions also emerged for engagement
for location with language (X
2
= 13.45, p < .005) and content
(X
2
= 8.01, p < .005), indicating that Spanish-speakers (OR = 1.40)
or those who saw the self-ecacy ad (OR = 1.25) were most likely to
engage if they were in the U.S. rather than San Francisco. Regarding
social reach, main eects were found for language (X
2
= 1224.57,
p < .005), and content (X
2
= 14.11, p < .005), which suggests that
Spanish-speakers were 8.21 times more likely to be reached via social
means, while the self-ecacy ad was 1.33 times more likely to reach
users via social means. Signicant interactions were also found for lo-
cation with language (X
2
= 56.46, p < .005) and content (X
2
= 4.03,
p < .05). Social reach was far more likely to be exhibited for Spanish-
speakers within San Francisco (OR =14.38) as well as the self-ecacy
ad shown within San Francisco (OR = 1.55). Within UOC, signicant
main eects emerged for location (X
2
= 163.36, p < .005), language
(X
2
= 12.15, p < .005), and content (X
2
= 20.87, p < .005), sug-
gesting that users in the U.S. were 1.57 times more likely to click on an
outbound link, Spanish-speakers were 1.13 times more likely to click on
an outbound link, and the self-ecacy ad was 1.18 times more likely to
have outbound links be clicked.
4. Discussion
Few articles have reported on recruitment methods for smoking
cessation studies conducted among Spanish-speaking populations
(Graham et al., 2012;Muñoz et al., 2006). The main aim of this study
was to assess FB as a recruitment for a one-condition smoking cessation
webapp study for Spanish- and English-speaking smokers. Recruitment
was conducted in two phases, the rst phase aimed to maximize re-
cruitment outcomes during a specied period of time without
Table 2
Recruitment-focused phase (U.S.) n = 540: recruitment costs and performance
metrics by language including posts and ads.
Metric type Total Spanish English
Impressions 1,430,119 901,367 528,752
Reach 1,034,038 632,339 401,699
Frequency 1.38 1.45 1.32
Link clicks 25,835 16,486 9349
UOC 23,613 15,072 8541
Shares 1887 1513 374
Comments 491 359 132
Reactions 3828 3120 708
Social reach 8359 7106 1253
CTR, % 1.81 1.83 1.77
CPC, US$ $0.43 $0.43 $0.44
Consent to the study (FB) 540 275 265
Facebook mobile 475 266 209
Facebook website 65 9 56
Conversion rate (FB) 2.29% 1.82% 3.10%
Total spent $11,203.40 $7098.96 $4104.44
Cost-per-consent (FB, US$) $20.75 $25.81 $15.49
Note. CPC = cost-per-click; CTR = click-through-rate; FB = Facebook;
UOC = Unique Outbound Clicks; all costs measured in U.S. dollars (US$).
Table 3
Descriptives for experimental phase (U.S. and San Francisco), recruitment costs
and performance metrics by language for ads only.
Metric type U.S. SF
Spanish English Spanish English
Impressions 86,200 70,210 100,827 122,040
Reach 54,913 56,232 50,999 61,334
Frequency 1.57 1.25 1.98 1.99
Link clicks 1212 1010 649 730
Unique Outbound Clicks (UOC) 1117 951 621 696
Shares 102 31 6 3
Comments 44 13 1 1
Reactions 153 47 30 17
Social reach 616 135 937 80
Click-through-rate (CTR), % 1.41 1.44 0.64 0.60
Cost-per-click (CPC), US$ $0.41 $0.49 $1.23 $1.09
Consents (FB) 18 19 4 2
Facebook mobile 17 14 4 2
Facebook website 1 5 0 0
Conversion rate (FB) 1.61% 2.00% 0.62% 0.29%
Total spent $499.96 $499.94 $799.91 $798.78
Cost-per-consent (FB,US$) $27.78 $26.31 $199.98 $399.39
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
5
implementing a predetermined or systematic approach to the ads. This
phase served as a pilot study to identify the necessary recruitment
procedures for an experimental design. Once the recruitment needs
were met (i.e., as many participants as possible using a fast-paced ad
lib, free-form approach), an experimental phase was implemented to
systematically assess dierences in recruitment campaigns by location,
language, and content of the ads.
Overall results from the recruitment-focused phase indicate that
Facebook ads and posts were eective in recruiting Spanish- and
English-speaking smokers to an online smoking cessation webapp. A
total of 581 participants provided consent (540 were from US and 41
from San Francisco), the number of U.S. participants that consented by
language were 275 for Spanish-speakers and 265 for English-speakers;
and the overall cost-per-consent was $20.75. Since the English-lan-
guage and Spanish-language campaigns were dierent (English-
language strategy focused mostly on ads and the Spanish-language
strategy focused mostly on boosted posts), the outcomes of such cam-
paigns are presented for informative purposes, but no generalizable
assumptions can be made based on such outcomes. The cost-per-con-
sent for the English campaign was $15.49, which is similar to the cost
reported by Thornton et al. (2016) of $18.18 for English language
studies that did not oer incentives. However, the cost-per-consent for
the current study was higher than for studies that oered incentives
(see Ramo et al., 2014 - $8.80 per-consent). A possible explanation for
this dierence may be that non-incentivized intervention studies that
aim to change behavior may require extra motivation and commitment
from individuals when compared to briefer or incentivized studies.
Furthermore, comparisons between the current study and previous
studies may be impacted by changes in FB's proprietary algorithm.
Additionally, as FB becomes a more popular advertising platform,
Table 4
Binomial logistic regression results for clicks, engagement, and social reach, with impressions as denominator.
Clicks Engagement Social reach
X
2
OR X
2
OR X
2
OR
Location
US vs. SF Only
581.35
⁎⁎⁎
2.28 832.88
⁎⁎⁎
2.56 31.25
⁎⁎⁎
1.52
Language
Spanish vs.
English
0.33 1.02 9.65
⁎⁎⁎
1.11 1077.74
⁎⁎⁎
7.32
Adset
Self E. vs.
Fear
35.02
⁎⁎⁎
1.23 29.10
⁎⁎⁎
1.20 19.44
⁎⁎⁎
1.40
Clicks Engagement Social reach
X
2
OR X
2
OR X
2
OR
US SF US SF US SF
Location Language
Spanish vs. English
2.56 0.965 1.08 0.0 1.11 1.11 82.63
⁎⁎⁎
3.72 14.39
Location Adset
Self E. vs. Fear
2.20 1.29 1.17 2.68
⁎⁎
1.26 1.13 6.35
1.15 1.70
Signicant at p < .05.
⁎⁎
Signicant at p < .01.
⁎⁎⁎
Signicant at p < .005.
Table 5
Binomial logistic regression results for clicks, engagement, social reach, and UOC, with reach as denominator.
Clicks Engagement Social reach UOC
X
2
OR X
2
OR X2 OR X
2
OR
Location
US vs. SF Only
192.64
⁎⁎⁎
1.61 327.64
⁎⁎⁎
1.81 0.82 1.07 163.36
⁎⁎⁎
1.57
Language
Spanish vs. English
14.43
⁎⁎⁎
1.14 41.96
⁎⁎⁎
1.24 1224.57
⁎⁎⁎
8.21 12.15
⁎⁎⁎
1.13
Adset
Self E. vs. Fear
20.13
⁎⁎⁎
1.17 15.25
⁎⁎⁎
1.14 14.11
⁎⁎⁎
1.33 20.87
⁎⁎⁎
1.18
Clicks Engagement Social reach UOC
X
2
OR X
2
OR X2 OR X
2
OR
US SF US SF US SF US SF
Location Language
Spanish vs. English
3.56 1.22 1.07 13.45
⁎⁎⁎
1.40 1.10 56.46
⁎⁎⁎
4.69 14.38 2.36 1.20 1.07
Location Adset
Self E. vs. Fear
6.89
⁎⁎
1.28 1.07 8.01
⁎⁎⁎
1.25 1.04 4.03
1.14 1.55 2.68 1.28 1.09
Signicant at p < .05.
⁎⁎
Signicant at p < .01.
⁎⁎⁎
Signicant at p < .005.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
6
competition for ad space increases, which consequently aects both
costs and click-through-rates. Finally, recruitment may vary depending
on the types of ads that each study used, therefore, comparing outcomes
of studies using dierent ads may be hard to evaluate.
Interestingly, the Spanish campaign had a higher cost-per-consent
($25.81) than the English campaigns. It is dicult to clearly ascertain
the reason for this outcome as there is a dearth of studies that have
examined these metrics among Spanish-speakers specically and, more
broadly, within smoking cessation web-based studies. Despite the fact
that the Spanish language is the second most common native language
spoken worldwide, it may be more expensive to recruit Spanish
speakers in the U.S. given greater costs to target recruitment eorts.
Additional external factors may further impede our ability to draw
strong conclusions from this study. First, FB's proprietary algorithm
determines how the dierent audiences are reached. For example, in
our study Spanish-speakers were delivered ads more frequently than
English-speakers. Second, the methods used by the community man-
agers in the recruitment phase for each language were dierent.
Together with previous studies using other online recruitment methods,
such as Google ads (Graham et al., 2012;Muñoz et al., 2006;Muñoz
et al., 2009;Leykin et al., 2012;Muñoz et al., 2012), this study de-
monstrates the feasibility of recruiting Spanish speaking smokers in the
U.S. and contributes to the still limited literature on online recruitment
methods for Spanish-speakers. For example, previous studies that have
shown the feasibility of recruiting Spanish speaking smokers worldwide
(Muñoz et al., 2006;Muñoz et al., 2009;Leykin et al., 2012;Muñoz
et al., 2012), were done using Google Ads, and were carried out more
than six years ago.
During the experimental phase a series of systematic comparisons
between languages, locations, and ad content were concurrently de-
ployed using the same ads. Direct comparisons were calculated using
both Impressions and Reach as denominators. Although Impressions are
often used as a standard to calculate common industry metrics (e.g.,
CTR, CPC), Reach may be more appropriate for researchers looking to
evaluate dierences on an individual level. Both Impressions and Reach
are determined by FB's proprietary algorithm, but once an individual is
reached (sees the ad at least once), some metrics are based on the in-
dividual's behavior (clicks, shares, and engagement), not the FB algo-
rithm.
The results of the analysis using Impressions as the denominator
indicated that U.S. participants had higher rates of clicks, engagements,
and social reach than San Francisco participants. Spanish-speaking
participants had higher rates of engagement and social reach than
English-speakers. The self-ecacy ad had higher rates of clicks, en-
gagements, and social reach than the fear ad. Additionally, Spanish-
speakers within San Francisco had higher rates of social reach
(OR = 14.39) than in the U.S.; and the self-ecacy ad when shown
within San Francisco had higher rates of social reach than more broadly
in the U.S.
When Reach was the denominator users in the US had higher rates
of clicks, UOCs, and engagements than users in San Francisco. Spanish-
speakers had higher rates of clicks, UOCs, engagement, and social reach
than English-speakers. Additionally, Spanish-speakers who saw the self-
ecacy adset had higher rates of engagement and social reach in the
U.S. compared to San Francisco. The self-ecacy ad had higher rates of
clicks, UOCs, engagements, and social reach than the fear ad.
Additionally, the self-ecacy ad had higher rates of clicks and en-
gagement in the U.S. compared to San Francisco.
Overall, results followed a similar pattern when evaluated based on
Impressions or Reach. U.S. users performed better (i.e. more clicks,
engagement, social reach, UOC) than San Francisco users, Spanish-
speakers engaged more than English-speakers, and the self-ecacy ad
performed better than the fear appeal ad. When using Reach as a de-
nominator, there were signicant interactions for clicks (location by
language) and engagement (location by language, and location by type
of ad) that were not evident when using impressions as the
denominator. These results highlight the relevance of considering me-
trics other than industry standards (i.e., Impressions as a denominator)
in order to capture a more accurate representation of users' behavior
when recruiting for smoking cessation studies.
5. Limitations and future directions
There are several limitations to the current study. First, because
during the recruitment phase each community manager had dierent
strategies, generalizable conclusions are dicult to draw. The recruit-
ment phase was successful in achieving higher conversion rates and
lower cost-per-consent than the experimental phase. However, it is
unclear if these outcomes were due to heterogeneity between campaign
strategies, or continued recruitment from the same audience pool, or
ads becoming less eective over time. Future studies could improve
their recruitment eorts if direct comparisons between ad type and
languages are implemented from the beginning.
Secondly, FB's proprietary algorithm made it challenging to de-
termine what eects were due to the algorithm versus dierences in
individual behavior, keeping in mind that both can aect cost.
Additionally, due to the types of data provided by FB, statistical ana-
lysis could only be performed on certain variables.
Third, it is dicult to evaluate dierences between users residing in
the broader U.S. and those within San Francisco only. The San
Francisco campaign was less eective than the U.S. campaign, but that
may be explained by dierences in smoking prevalence rates and po-
pulation size. Therefore, these results should not be generalized to other
U.S. cities or states without further considerations of demographic and
smoking dierences within regions. Future studies could benet from
comparing ad performance between dierent states or cities. For ex-
ample, it is possible that ads may have performed better in states with
higher rates of smokers. Akers and Gordon (2018) suggested to analyze
the tobacco use prevalence by state to conduct eective comparisons
between states.
Fourth, ads in the experimental phase were previously run in the
recruitment phase and originally designed for the English-speaking
audience. Ads designed for an English-speaking audience then trans-
lated into Spanish may perform dierently than ads designed for a
Spanish-speaking audience and translated into English. Even though the
text approximated a literal translation, that does not guarantee that
they were equally perceived by both audiences. Thus, the design of the
study did not allow for evaluation of language translation eects.
Fifth, while a variety of ad content were used during the recruit-
ment-focused phase, only two types of content (fear appeal vs. self-ef-
cacy) were used in the experimental phase. Therefore, more studies
are needed in order to grasp a robust understanding of the eects of ads
as results may not be generalizable to other types of content. Future
studies should attempt to replicate the comparisons employed in the
current study and expand them to other types of content.
Finally, although the experimental phase did have signicant nd-
ings, it only yielded 43 participants (37 for US and 6 for San Francisco)
over a two week period. As such, the impact of these ndings in relation
to Spanish speakers should be interpreted with caution and replicated
with a larger sample.
6. Conclusion
To the best of our knowledge this is the rst study to report on the
use of Facebook ads to recruit Spanish-speaking smokers and to com-
pare ad performance by location, language and ad content for the re-
cruitment of smoking cessation webapp studies. This study demon-
strates that FB is an eective platform for recruiting both Spanish- and
English-smokers into clinical trials of smoking cessation webapps.
Although there were dierences in cost-per-consent for Spanish- and
English-speakers, recruitment of Spanish-speakers through Facebook is
feasible. Furthermore, comparing an ad's performance by location,
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
7
language, and ad content may contribute to developing more ecient
and eective campaigns. With increased awareness regarding the im-
portance of including minorities and women in research, being able to
eectively reach and recruit diverse audiences is critical. Therefore,
Facebook's ability to target diverse audiences cost-eectively makes it
an important recruitment tool.
Conict of interest
Facebook for Recruiting Spanish- and English-Speaking Smokers.
The authors report no conict of interest.
References
Akers, L., Gordon, J.S., 2018. Using Facebook for large-scale online randomized clinical
trial recruitment: eective advertising strategies. J. Med. Internet Res. 20 (11), e290.
https://doi.org/10.2196/jmir.9372.
Amon, K.L., Campbell, A.J., Hawke, C., Steinbeck, K., 2014. Facebook as a recruitment
tool for adolescent health research: a systematic review. Acad. Pediatr. 14 (5),
439447. https://doi.org/10.1016/j.acap.2014.05.049.
Bunge, E., Cook, H.M., Bond, M., Williamson, R.E., Cano, M., Barrera, A.Z., Leykin, Y.,
Muñoz, R.F., 2018. Comparing Amazon mechanical Turk with unpaid internet re-
sources in online clinical trials. Internet Interv. 12, 6873. https://doi.org/10.1016/j.
invent.2018.04.001.
Carlini, B.H., Saoti, L., Rue, T.C., Miles, L., 2014. Using internet to recruit immigrants
with language and culture barriers for tobacco and alcohol use screening: a study
among Brazilians. J. Immigr. Minor. Health 17 (2), 553560. https://doi.org/10.
1007/s10903-013-9934-1.
Choi, I., Milne, D.N., Glozier, N., Peters, D., Harvey, S.B., Calvo, R.A., 2017. Using dif-
ferent Facebook advertisements to recruit men for an online mental health study:
engagement and selection bias. Internet Interv. 8, 2734. https://doi.org/10.1016/j.
invent.2017.02.002.
Christensen, T., Riis, A.H., Hatch, E.E., Wise, L.A., Nielsen, M.G., Rothman, K.J., ...
Mikkelsen, E.M., 2017. Costs and eciency of online and oine recruitment meth-
ods:a web-based cohort study. J. Med. Internet Res. 19 (3), e58. https://doi.org/10.
2196/jmir.6716.
Facebook Business. (2018). Glossary of ad terms. Retrieved from https://www.facebook.
com/business/help/447834205249495.
Frandsen, M., Walters, J., Ferguson, S.G., 2014. Exploring the viability of using online
social media advertising as a recruitment method for smoking cessation clinical trials.
Nicotine Tob. Res. 16 (2), 247251. https://doi.org/10.1093/ntr/ntt157.
Graham, A.L., Fang, Y., Moreno, J.L., Strei, S.L., Villegas, J., Muñoz, R.F., ... Vallone,
D.M., 2012. Online advertising to reach and recruit Latino smokers to an internet
cessation program: impact and costs. J. Med. Internet Res. 14 (4), e116. https://doi.
org/10.2196/jmir.2162.
Hener, J.L., Wyszynski, C.M., Comstock, B., Mercer, L.D., Bricker, J., 2013. Overcoming
recruitment challenges of web-based interventions for tobacco use: the case of web-
based acceptance and commitment therapy for smoking cessation. Addict. Behav. 38
(10), 24732476. https://doi.org/10.1016/j.addbeh.2013.05.004.
Kayrouz, R., Dear, B.F., Karin, E., Titov, N., 2016. Facebook as an eective recruitment
strategy for mental health research of hard to reach populations. Internet Interv. 4,
110. https://doi.org/10.1016/j.invent.2016.01.001.
Lane, T.S., Armin, J., Gordon, J.S., 2015. Online recruitment methods for web-based and
mobile health studies: a review of the literature. J. Med. Internet Res. 17 (7), e183.
https://doi.org/10.2196/jmir.4359.
Leykin, Y., Aguilera, A., Torres, L.D., Pérez-Stable, E.J., Muñoz, R.F., 2012. Interpreting
the outcomes of automated internet-based randomized trials: example of an inter-
national smoking cessation study. J. Med. Internet Res. 14 (1), e5. https://doi.org/10.
2196/jmir.1829.
Muñoz, R.F., Lenert, L.L., Delucchi, K., Stoddard, J., Perez, J.E., Penilla, C., Pérez-Stable,
E.J., 2006. Toward evidence-based internet interventions: a Spanish/English web site
for international smoking cessation trials. Nicotine Tob. Res. 8 (1), 7787. https://
doi.org/10.1080/14622200500431940.
Muñoz, R.F., Barrera, A.Z., Delucchi, K., Penilla, C., Torres, L.D., Pérez-Stable, E.J., 2009.
International Spanish/English internet smoking cessation trial yields 20% abstinence
rates at 1 year. Nicotine Tob. Res. 11 (9), 10251034. https://doi.org/10.1093/ntr/
ntp090.
Muñoz, R.F., Aguilera, A., Schueller, S.M., Leykin, Y., Pérez-Stable, E.J., 2012. From
online randomized controlled trials to participant preference studies: morphing the
San Francisco stop smoking site into a worldwide smoking cessation resource. J. Med.
Internet Res. 14 (3), e64. https://doi.org/10.2196/jmir.1852.
Pedersen, E.R., Kurz, J., 2016. Using Facebook for health-related research study re-
cruitment and program delivery. Curr. Opin. Psychol. 9, 3843. https://doi.org/10.
1016/j.copsyc.2015.09.011.
Ramo, D.E., Prochaska, J.J., 2012. Broad reach and targeted recruitment using Facebook
for an online survey of young adult substance use. J. Med. Internet Res. 14 (1), e28.
https://doi.org/10.2196/jmir.1878.
Ramo, D.E., Hall, S.M., Prochaska, J.J., 2010. Reaching young adult smokers through the
internet: comparison of three recruitment mechanisms. Nicotine Tob. Res. 12 (7),
768775. https://doi.org/10.1093/ntr/ntq086.
Ramo, D.E., Rodriguez, T.M.S., Chavez, K., Sommer, M.J., Prochaska, J.J., 2014.
Facebook recruitment of young adult smokers for a cessation trial: methods, metrics,
and lessons learned. Internet Interv. 1 (2), 5864. https://doi.org/10.1016/j.invent.
2014.05.001.
Ramo, D.E., Liu, H., Prochaska, J.J., 2015a. A mixed-methods study of young adults'
receptivity to using Facebook for smoking cessation: if you build it, will they come?
Am. J. Health Promot. 29 (4), e126e135. https://doi.org/10.4278/ajhp.130326-
QUAL-128.
Ramo, D.E., Young-Wol, K.C., Prochaska, J.J., 2015b. Prevalence and correlates of
electronic-cigarette use in young adults: ndings from three studies over ve years.
Addict. Behav. 41, 142147. https://doi.org/10.1016/j.addbeh.2014.10.019.
Sadasivam, R.S., Vloz, E.M., Kinney, R.L., Rao, S.R., Houston, T.K., 2013. Share2Quit:
web-based peer-driven referrals for smoking cessation. JMIR Res. Prot. 2 (2), e37.
https://doi.org/10.2196/resprot.2786.
Sadasivam, R.S., Cutrona, S.L., Luger, T.M., Volz, E., Kinney, R., Rao, S.R., ... Houston,
T.K., 2016. Share2Quit: online social network peer marketing of tobacco cessation
systems. Nicotine Tob. Res. 19 (3), 314323. https://doi.org/10.1093/ntr/ntw187.
Temple, E.C., Brown, R.F., 2011. A comparison of internet-based participant recruitment
methods: engaging the hidden population of cannabis users in research. J. Res. Pract.
7 (2), 120.
Thornton, L.K., Baker, A.L., Johnson, M.P., Lewin, T., 2013. Perceived risk associated
with tobacco, alcohol and cannabis use among people with and without psychotic
disorders. Addict. Behav. 38 (6), 22462251. https://doi.org/10.1016/j.addbeh.
2013.02.003.
Thornton, L., Batterham, P.J., Fassnacht, D.B., Kay-Lambkin, F., Calear, A.L., Hunt, S.,
2016. Recruiting for health, medical or psychosocial research using Facebook: sys-
tematic review. Internet Interv. 4, 7281. https://doi.org/10.1016/j.invent.2016.02.
001.
Whitaker, C., Stevelink, S., Fear, N., 2017. The use of Facebook in recruiting participants
for health research purposes: a systematic review. J. Med. Internet Res. 19 (8), e290.
https://doi.org/10.2196/jmir.7071.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
8
... A lot of preparation and planning was involved to ensure a range of recruitment strategies was incorporated. As this was the first tinnitus ICBT trial with Spanish participants, much research was done to investigate how to improve reaching this population [e.g., (48)(49)(50)(51)(52)]. The Spanish speakers were furthermore involved in the research team and during the intervention adaptation (20). ...
Article
Full-text available
Introduction: An internet-based cognitive behavioral therapy (ICBT) offers a way to increase access to evidence-based tinnitus care. To increase the accessibility of this intervention, the materials were translated into Spanish to reach Spanish as well as English speakers. A clinical trial indicated favorable outcomes of ICBT for tinnitus for the population of the United States. In view of later dissemination, a way to increase the applicability of this intervention is required. Such understanding is best obtained by considering the perspectives and experiences of participants of an intervention. This study aimed to identify the processes that could facilitate or hinder the clinical implementation of ICBT in the United States. Methods: This study evaluated the processes regarding enrolment, allocation, intervention delivery, the outcomes obtained, and the trial implementation. The study sample consisted of 158 participants who were randomly assigned to the experimental and control group. Results: Although the recruitment was sufficient for English speakers, recruiting the Spanish participants and participants belonging to ethnic minority groups was difficult despite using a wide range of recruitment strategies. The allocation processes were effective in successfully randomizing the groups. The intervention was delivered as planned, but not all the participants chose to engage with the materials provided. Compliance for completing the outcome measures was low. The personal and intervention factors were identified as barriers for the implementation whereas the facilitators included the support received, being empowering, the accessibility of the intervention, and its structure. Conclusion: An understanding regarding the factors contributing to the outcomes obtained, the barriers and facilitators of the results, engagement, and compliance were obtained. These insights will be helpful in preparing for the future dissemination of such interventions. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT04004260. Registered on 2 July 2019.
... The recruitment of a substantial portion of Spanish-speaking smokers for Version 3 was successful: 46% of our consented participants were Spanish speaking. The improvement in recruitment for Version 3 was primarily attributable to our use of social media methods, including Facebook ads (8), suggesting that learning the communication channels used by the population of interest is essential to reaching them. ...
Article
Full-text available
Background: Smoking cessation Internet interventions have been shown to be comparable in effectiveness to the nicotine patch. The aim of this study was to develop a Spanish/English smoking cessation web app using input from low-income smokers, and to evaluate modifications to the online intervention in terms of its ability to engage smokers. Methods: Three versions of a smoking cessation web app were developed and tested. Measures of engagement, such as completion of study registration, utilization of cigarette, mood, and craving trackers, and completion of follow-up assessments, were collected to determine whether changes in the website resulted in increased engagement. Results: The third version of the website, which featured improved look-and-feel and fewer barriers to engagement, markedly increased tracker engagement from the first two versions. However, follow-up rates remained low across all three versions. Conclusions: The increase in engagement was attributed to the following modifications: A more inviting landing page with key intervention elements available immediately; an easily accessible dashboard with users' data; and tracking tools that were more user friendly. We conclude that in addition to adequate and functional elements, design principles are key factors in increasing engagement in online interventions.
... With 9 in 10 US adults seeking information on the web 1 and 7 in 10 using social media platforms, 2 the use of online mediums to recruit and to collect research data from diverse populations has become a common and cost-effective practice in health sciences research over the last 5 years. [3][4][5][6] This form of recruitment and data collection is currently in use in large-scale biomedical research projects, such as the National Institute of Health's Precision Medicine Initiative, 7 which plans to recruit a diverse sample of 1 000 000 Americans through social media campaigns. Such projects also intend to collect digital information (electronic health records information, data from fitness devices, and even social media and web searches) to enhance our understanding of early risk factors for different disease states. ...
Article
Full-text available
Importance Using social media to recruit participants is a common and cost-effective practice. Willingness to participate (WTP) in biomedical research is a function of trust in the scientific team, which is closely tied to the source of funding and institutional connections. Objective To determine whether WTP and willingness to share social media data are associated with the type of research team and online recruitment platform. Design, Setting, and Participants This mixed-methods longitudinal survey and qualitative study was conducted over 2 points (T1 and T2) using Amazon’s Mechanical Turk (MTurk) platform. Participants were US adults aged 18 years or older who use at least 1 social media platform. Recruitment was stratified to match race/ethnicity proportions of the 2010 US Census. The volunteer sample consisted of 914 participants at T1, and 655 participants completed the follow-up survey 5 months later (T2). Main Outcomes and Measures Outcomes were (1) past experience with online research and sharing social media data for research; (2) WTP in research advertised online; (3) WTP in a study sponsored by a pharmaceutical company, a university, or a federal agency; and (4) willingness to share social media data. Opinions were solicited regarding the European Union’s General Data Protection Regulation statute, which came into effect between T1 and T2. Results Of 914 participants completing the first survey (T1), 604 (66.1%) were aged 18 to 39 years and 494 (54.0%) were female. Of these, 655 participants (71.7%) responded at T2. While 680 participants (74.4%) indicated WTP in biomedical research, only 454 (49.3%) were willing to share their social media data. Participants were significantly less likely to participate in federally sponsored (odds ratio [OR], 0.58; 95% CI, 0.51-0.64; P < .001) or pharmaceutical company (OR, 0.59; 95% CI, 0.53-0.66; P < .001) research than university-led studies. They were also less likely to share their social medial data for federal (OR, 0.65; 95% CI, 0.58-0.72; P < .001) or pharmaceutical company (OR, 0.50; 95% CI, 0.44-0.56; P < .001) research compared with academic studies. Willingness to participate in pharmaceutical company–led research decreased 11.89% from T1 to T2 (OR for T2, 0.62; 95% CI, 0.54-0.77; P < .001). Reasons for WTP were interest in furthering science, financial incentives, trust in the organization, and data security. While 63.0% of respondents reported seeing new privacy policy emails related to the General Data Protection Regulation law, only 27.1% indicated this positively influenced their WTP. Thematic analysis of responses indicated that WTP may improve with stronger data security measures. Conclusions and Relevance This study suggests that researchers may see reduced online research participation and data sharing, particularly for research conducted outside academia.
... We have worked on studies about perinatal depression on Spanish-speaking women (Barrera and Nichols, 2015;Carter et al., 2019) and online prevention of postpartum depression . We have also worked on studies to reach Spanish-speaking smokers (Muñoz et al., 2006(Muñoz et al., , 2009, and how to increase recruitment of Spanish-and English-speaking participants (Barrera et al., 2014;Bunge et al., 2019). ...
Article
Full-text available
This article is a partially revised version of a keynote address presented at the 10th Scientific Meeting of the International Society for Research on Internet Interventions (ISRII) in Auckland, New Zealand. It addresses six points: 1) the meanings of indigeneity, diversity, and equity, 2) the strong emotional reactions elicited by the inequities experienced by indigenous groups throughout the world, 3) the aspirations of members of ISRII in terms of what we would like our field to accomplish to address these inequities, 4) the United Nations goal of making health care a universal human right, 5) the difficulties encountered by other health sciences in attempting to include diverse populations into major studies, and 6) ways in which the Internet interventions and digital health field could include indigeneity, diversity, and equity in our work, and by doing so, contribute to the United Nations goal of making health care a universal human right. The authors suggest that providing access to health care to all people, no matter where they are on the socioeconomic continuum, is a key strategy to pursue. The field of Internet interventions could contribute by creating digital apothecaries that would develop, evaluate, and disseminate evidence-based Massive Open Online Interventions to anyone in the world who needs them. Keywords: Indigeneity, Diversity, Equity, Internet interventions, Digital apothecaries, Universal human rights
Article
Introduction: Social media, including Facebook outreach, is increasingly being used as a participant recruitment tool, and may be particularly useful in tobacco and smoking cessation studies. The Recruitment Innovation Center at Vanderbilt University Medical Center partnered with Project LUNA, a smoking cessation study, to conduct a pilot social media campaign aimed at increasing study recruitment. Methods: Two posts encouraging study participation were developed and promoted on Facebook to users with an interest in smoking-related topics, with a link to a study-specific webpage. Facebook and website analytics were collected, including impressions, clicks, click through rates, website traffic, and clicks to the study screening form. Study screening and enrollment data were also collected. Results: The Facebook campaign ran in June 2019 in the greater Houston area. In total, the Facebook posts logged 1,179,844 impressions, 6490 clicks, and an overall click-through rate of 0.55%. There were no differences in response to the two different promotional posts. Approximately 3812 unique individuals visited an intermediary study page, with 473 expressing interest in the study. Forty-three potential participants contacted the study team, resulting in study enrollment and randomization of 23 participants, with an estimated cost per enrolled participant of $441. Conclusions: The social media campaign was successful at increasing outreach and interest in the LUNA study. However, the price-per-participant enrolled was higher than in comparable tobacco cessation studies. These results and lessons learned may be beneficial to others considering social media as a recruitment method for their clinical research trial.
Article
Full-text available
Background Social media holds exciting promise for advancing mental health research recruitment, however, the extent and efficacy to which these platforms are currently in use are underexplored. Objective: A systematic review was conducted to characterize the current use and efficacy of social media in recruiting participants for mental health research. Method A literature review was performed using MEDLINE, EMBASE, and PsychINFO. Only non-duplicative manuscripts written in the English language and published between 1/1/2004–3/31/2019 were selected for further screening. Data extracted included study type and design, participant inclusion criteria, social media platform, advertising strategy, final recruited sample size, recruitment location, year, monetary incentives, comparison to other recruitment methods if performed, and final cost per participant. Results A total of 176 unique studies that used social media for mental health research recruitment were reviewed. The majority of studies were cross-sectional (62.5%) in design and recruited adults. Facebook was overwhelmingly the recruitment platform of choice (92.6%), with the use of paid advertisements being the predominant strategy (60.8%). Of the reviewed studies, substance abuse (43.8%) and mood disorders (15.3%) were the primary subjects of investigation. In 68.3% of studies, social media recruitment performed as well as or better than traditional recruitment methods in the number and cost of final enrolled participants. The majority of studies used Facebook for recruitment at a median cost per final recruited study participant of $19.47. In 55.6% of the studies, social media recruitment was the more cost-effective recruitment method when compared to traditional methods (e.g., referrals, mailing). Conclusion Social media appears to be an effective and economical recruitment tool for mental health research. The platform raises methodological and privacy concerns not covered in current research regulations that warrant additional consideration.
Article
Background and objectives: A key aspect of the Internet that facilitates research is social media, especially the meteoric rise in the use of Facebook as one of the primary applications for social connectivity. Facebook can be considered a rich source of data due to the high amounts of demographic information shared by users and stored in the system, as well as the way in which users share their thoughts and behaviour in their natural environment. This report focuses on one functionality that as yet remained under-discussed and perhaps under-utilised as a tool in health research - the ability for participants or interested parties to leave comments directly on Facebook posts relating to research projects. Further, this report provides some considerations for researchers intending to use Facebook in recruiting participants for research. Methods: The study was a Bowel Cancer Awareness Study conducted by the Centre for Rural Health, University of Tasmania to assess bowel cancer risk awareness in Tasmania. Participants were recruited to complete a survey on LimeSurveyTM via a Facebook page used to advertise the study. Several comments were made on the Facebook page over a three-month period, which were then further categorised and thematically analysed. Specifically, these comments were reviewed to determine how valuable Facebook comments can be for research. Results: The Facebook advertisement for the survey reached 136,640 people at a cost of $0.04 (4 cents) per person. From the range of comments received, four separate functions of these comments became apparent upon analysis - the capacity to ask questions, the ability to interact with others, promotion of the survey, and suggestions for future research. Discussion: From the bowel cancer awareness study, we found that Facebook comments on our study page allowed for an opportunity for ongoing contact with respondents, opportunity to receive feedback, address concerns and harness future research ideas. The ability to further utilize the potential for Facebook comments to enrich data collection and health research warrants a continuous and sustained interest, as research methodology progressively utilises the Internet and social media platforms as an effective and affordable option.
Article
Full-text available
Targeted Facebook advertising can be an effective strategy to recruit participants for a large-scale online study. Facebook advertising is useful for reaching people in a wide geographic area, matching a specific demographic profile. It can also target people who would be unlikely to search for the information and would thus not be accessible via Google AdWords. It is especially useful when it is desirable not to raise awareness of the study in a demographic group that would be ineligible for the study. This paper describes the use of Facebook advertising to recruit and enroll 1145 women over a 15-month period for a randomized clinical trial to teach support skills to female partners of male smokeless tobacco users. This tutorial shares our study team's experiences, lessons learned, and recommendations to help researchers design Facebook advertising campaigns. Topics covered include designing the study infrastructure to optimize recruitment and enrollment tracking, creating a Facebook presence via a fan page, designing ads that attract potential participants while meeting Facebook's strict requirements, and planning and managing an advertising campaign that accommodates the rapid rate of diminishing returns for each ad.
Article
Full-text available
Internet interventions face significant challenges in recruitment and attrition rates are typically high and problematic. Finding innovative yet scientifically valid avenues for attaining and retaining participants is therefore of considerable importance. The main goal of this study was to compare recruitment process and participants characteristics between two similar randomized control trials of mood management interventions. One of the trials (Bunge et al., 2016) was conducted with participants recruited from Amazon's Mechanical Turk (AMT), and the other trial recruited via Unpaid Internet Resources (UIR). Methods The AMT sample (Bunge et al., 2016) consisted of 765 adults, and the UIR sample (recruited specifically for this study) consisted of 329 adult US residents. Participants' levels of depression, anxiety, confidence, motivation, and perceived usefulness of the intervention were assessed. The AMT sample was financially compensated whereas the UIR was not. Results AMT yielded higher recruitment rates per month (p < .05). At baseline, the AMT sample reported significantly lower depression and anxiety scores (p < .001 and p < .005, respectively) and significantly higher mood, motivation, and confidence (all p < .001) compared to the UIR sample. AMT participants spent significantly less time on the site (p < .05) and were more likely to complete follow-ups than the UIR sample (p < .05). Both samples reported a significant increase in their level of confidence and motivation from pre- to post-intervention. AMT participants showed a significant increase in perceived usefulness of the intervention (p < .0001), whereas the UIR sample did not (p = .1642). Conclusions By using AMT, researchers can recruit very rapidly and obtain higher retention rates; however, these participants may not be representative of the general online population interested in clinical interventions. Considering that AMT and UIR participants differed in most baseline variables, data from clinical studies resulting from AMT samples should be interpreted with caution.
Article
Full-text available
Background Social media is a popular online tool that allows users to communicate and exchange information. It allows digital content such as pictures, videos and websites to be shared, discussed, republished and endorsed by its users, their friends and businesses. Adverts can be posted and promoted to specific target audiences by demographics such as region, age or gender. Recruiting for health research is complex with strict requirement criteria imposed on the participants. Traditional research recruitment relies on flyers, newspaper adverts, radio and television broadcasts, letters, emails, website listings, and word of mouth. These methods are potentially poor at recruiting hard to reach demographics, can be slow and expensive. Recruitment via social media, in particular Facebook, may be faster and cheaper. Objective The aim of this study was to systematically review the literature regarding the current use and success of Facebook to recruit participants for health research purposes. Methods A literature review was completed in March 2017 in the English language using MEDLINE, EMBASE, Web of Science, PubMed, PsycInfo, Google Scholar, and a hand search of article references. Papers from the past 12 years were included and number of participants, recruitment period, number of impressions, cost per click or participant, and conversion rate extracted. Results A total of 35 studies were identified from the United States (n=22), Australia (n=9), Canada (n=2), Japan (n=1), and Germany (n=1) and appraised using the Critical Appraisal Skills Programme (CASP) checklist. All focused on the feasibility of recruitment via Facebook, with some (n=10) also testing interventions, such as smoking cessation and depression reduction. Most recruited young age groups (16-24 years), with the remaining targeting specific demographics, for example, military veterans. Information from the 35 studies was analyzed with median values being 264 recruited participants, a 3-month recruitment period, 3.3 million impressions, cost per click of US $0.51, conversion rate of 4% (range 0.06-29.50), eligibility of 61% (range 17-100), and cost per participant of US $14.41. The studies showed success in penetrating hard to reach populations, finding the results representative of their control or comparison demographic, except for an over representation of young white women. Conclusions There is growing evidence to suggest that Facebook is a useful recruitment tool and its use, therefore, should be considered when implementing future health research. When compared with traditional recruitment methods (print, radio, television, and email), benefits include reduced costs, shorter recruitment periods, better representation, and improved participant selection in young and hard to reach demographics. It however, remains limited by Internet access and the over representation of young white women. Future studies should recruit across all ages and explore recruitment via other forms of social media.
Article
Full-text available
A growing number of researchers are using Facebook to recruit for a range of online health, medical, and psychosocial studies. There is limited research on the representativeness of participants recruited from Facebook, and the content is rarely mentioned in the methods, despite some suggestion that the advertisement content affects recruitment success. This study explores the impact of different Facebook advertisement content for the same study on recruitment rate, engagement, and participant characteristics. Five Facebook advertisement sets (“resilience”, “happiness”, “strength”, “mental fitness”, and “mental health”) were used to recruit male participants to an online mental health study which allowed them to find out about their mental health and wellbeing through completing six measures. The Facebook advertisements recruited 372 men to the study over a one month period. The cost per participant from the advertisement sets ranged from $0.55 to $3.85 Australian dollars. The “strength” advertisements resulted in the highest recruitment rate, but participants from this group were least engaged in the study website. The “strength” and “happiness” advertisements recruited more younger men. Participants recruited from the “mental health” advertisements had worse outcomes on the clinical measures of distress, wellbeing, strength, and stress. This study confirmed that different Facebook advertisement content leads to different recruitment rates and engagement with a study. Different advertisement also leads to selection bias in terms of demographic and mental health characteristics. Researchers should carefully consider the content of social media advertisements to be in accordance with their target population and consider reporting this to enable better assessment of generalisability.
Article
Full-text available
Background The Internet is widely used to conduct research studies on health issues. Many different methods are used to recruit participants for such studies, but little is known about how various recruitment methods compare in terms of efficiency and costs. Objective The aim of our study was to compare online and offline recruitment methods for Internet-based studies in terms of efficiency (number of recruited participants) and costs per participant. Methods We employed several online and offline recruitment methods to enroll 18- to 45-year-old women in an Internet-based Danish prospective cohort study on fertility. Offline methods included press releases, posters, and flyers. Online methods comprised advertisements placed on five different websites, including Facebook and Netdoktor.dk. We defined seven categories of mutually exclusive recruitment methods and used electronic tracking via unique Uniform Resource Locator (URL) and self-reported data to identify the recruitment method for each participant. For each method, we calculated the average cost per participant and efficiency, that is, the total number of recruited participants. Results We recruited 8252 study participants. Of these, 534 were excluded as they could not be assigned to a specific recruitment method. The final study population included 7724 participants, of whom 803 (10.4%) were recruited by offline methods, 3985 (51.6%) by online methods, 2382 (30.8%) by online methods not initiated by us, and 554 (7.2%) by other methods. Overall, the average cost per participant was ?6.22 for online methods initiated by us versus ?9.06 for offline methods. Costs per participant ranged from ?2.74 to ?105.53 for online methods and from ?0 to ?67.50 for offline methods. Lowest average costs per participant were for those recruited from Netdoktor.dk (?2.99) and from Facebook (?3.44). Conclusions In our Internet-based cohort study, online recruitment methods were superior to offline methods in terms of efficiency (total number of participants enrolled). The average cost per recruited participant was also lower for online than for offline methods, although costs varied greatly among both online and offline recruitment methods. We observed a decrease in the efficiency of some online recruitment methods over time, suggesting that it may be optimal to adopt multiple online methods.
Article
Full-text available
Recruiting participants is a challenge for many health, medical and psychosocial research projects. One tool more frequently being used to improve recruitment is the social networking website Facebook. A systematic review was conducted to identify studies that have used Facebook to recruit participants of all ages, to any psychosocial, health or medical research. 110 unique studies that used Facebook as a recruitment source were included in the review. The majority of studies used a cross-sectional design (80%) and addressed a physical health or disease issue (57%). Half (49%) of the included studies reported specific details of the Facebook recruitment process. Researchers paid between $1.36 and $110 per completing participants (Mean=$17.48, SD=$23.06). Among studies that examined the representativeness of their sample, the majority concluded (86%) their Facebook-recruited samples were similarly representative of samples recruited via traditional methods. These results indicate that Facebook is an effective and cost-efficient recruitment method. Researchers should consider their target group, advertisement wording, offering incentives and no-cost methods of recruitment when considering Facebook as a recruitment source. It is hoped this review will assist researchers to make decisions regarding the use of Facebook as a recruitment tool in future research.
Article
Full-text available
Recent reports indicate that Facebook (FB) may facilitate recruitment of hard to reach participants into mental health research. The present study aimed to contribute to this emerging literature by exploring recruitment data from a recently completed trial of online treatment for symptoms of anxiety and depression that targeted Arab people. The present study compared traditional recruitment strategies such as media releases, emails, and print advertisements with Facebook strategies including boosting posts, promoting websites, events and FB fan pages. The main outcomes of interest were the number of started applications and the time and cost associated per application associated with the FB and traditional recruitment strategies. A target sample of 350 was sought and a total of 81 participants applied to participate over the 42-week recruitment period. Overall, 86% of the resultant applications occurred via FB recruitment and a Poisson regression analysis indicated the FB strategies were more time-effective, recruiting participants 2.5 times faster than the traditional strategies. However, there were no differences in cost-effectiveness for FB ($US37 per participant) and traditional strategies ($US40 per participant). The findings of the current study add to existing literature detailing the value of FB recruitment strategies, alongside more traditional strategies, as a way of recruiting hard-to-reach populations for research. However, more research is needed to explore alternative and optimal strategies for the successful recruitment of hard to reach populations via FB and other online social media platforms.
Article
Full-text available
ABSTRACT Background: Internet and mobile health (mHealth) apps hold promise for expanding the reach of evidence-based health interventions. Research in this area is rapidly expanding. However, these studies may experience problems with recruitment and retention. Web-based and mHealth studies are in need of a wide-reaching and low-cost method of recruitment that will also effectively retain participants for the duration of the study. Online recruitment may be a low-cost and wide-reaching tool in comparison to traditional recruitment methods, although empirical evidence is limited. Objective: This study aims to review the literature on online recruitment for, and retention in, mHealth studies. Methods: We conducted a review of the literature of studies examining online recruitment methods as a viable means of obtaining mHealth research participants. The data sources used were PubMed, CINAHL, EbscoHost, PyscINFO, and MEDLINE. Studies reporting at least one method of online recruitment were included. A narrative approach enabled the authors to discuss the variability in recruitment results, as well as in recruitment duration and study design. Results: From 550 initial publications, 12 studies were included in this review. The studies reported multiple uses and outcomes for online recruitment methods. Web-based recruitment was the only type of recruitment used in 67% (8/12) of the studies. Online recruitment was used for studies with a variety of health domains: smoking cessation (58%; 7/12) and mental health (17%; 2/12) being the most common. Recruitment duration lasted under a year in 67% (8/12) of the studies, with an average of 5 months spent on recruiting. In those studies that spent over a year (33%; 4/12), an average of 17 months was spent on recruiting. A little less than half (42%; 5/12) of the studies found Facebook ads or newsfeed posts to be an effective method of recruitment, a quarter (25%; 3/12) of the studies found Google ads to be the most effective way to reach participants, and one study showed better outcomes with traditional (eg in-person) methods of recruitment. Only one study recorded retention rates in their results, and half (50%; 6/12) of the studies recorded survey completion rates. Conclusions: Although online methods of recruitment may be promising in experimental research, more empirical evidence is needed to make specific recommendations. Several barriers to using online recruitment were identified, including participant retention. These unique challenges of virtual interventions can affect the generalizability and validity of findings from Web-based and mHealth studies. There is a need for additional research to evaluate the effectiveness of online recruitment methods and participant retention in experimental mHealth studies.
Article
Introduction: Although technology-assisted tobacco interventions (TATIs) are effective, they are underused due to recruitment challenges. We tested whether we could successfully recruit smokers to a TATI using peer marketing through a social network (Facebook). Methods: We recruited smokers on Facebook using online advertisements. These recruited smokers (seeds) and subsequent waves of smokers (peer recruits) were provided the Share2Quit peer recruitment Facebook app and other tools. Smokers were incentivized for up to seven successful peer recruitments and had 30 days to recruit from date of registration. Successful peer recruitment was defined as a peer recruited smoker completing the registration on the TATI following a referral. Our primary questions were (1) whether smokers would recruit other smokers and (2) whether peer recruitment would extend the reach of the intervention to harder-to-reach groups, including those not ready to quit and minority smokers. Results: Overall, 759 smokers were recruited (seeds: 190; peer recruits: 569). Fifteen percent (n = 117) of smokers successfully recruited their peers (seeds: 24.7%; peer recruits: 7.7%) leading to four recruitment waves. Compared to seeds, peer recruits were less likely to be ready to quit (peer recruits 74.2% vs. seeds 95.1%), more likely to be male (67.1% vs. 32.9%), and more likely to be African American (23.8% vs. 10.8%) (p < .01 for all comparisons). Conclusions: Peer marketing quadrupled our engaged smokers and enriched the sample with not-ready-to-quit and African American smokers. Peer recruitment is promising, and our study uncovered several important challenges for future research. Implications: This study demonstrates the successful recruitment of smokers to a TATI using a Facebook-based peer marketing strategy. Smokers on Facebook were willing and able to recruit other smokers to a TATI, yielding a large and diverse population of smokers.
Article
Facebook has become an important tool for recruiting research participants and for program delivery. Given the wide use of Facebook, there is much potential for the site to help with recruitment efforts in both physical and behavioral health care arenas; reaching groups typically difficult to recruit and providing outreach to individuals that may not have received services elsewhere. Health studies using Facebook have generally reported success, including cost-effectiveness, recruitment of samples in brief periods of time, and ability to locate participants for follow-up research. Still, the use of Facebook for research and program delivery is a relatively new area that warrants more research attention and guidance around issues like validity of data, representativeness of samples, and protections of human subjects.