Facebook for recruiting Spanish- and English-speaking smokers

Abstract

Background
Recruitment for research is usually expensive and time consuming. Facebook (FB) recruitment has become widely utilized in recent years. The main aim of this study was to assess FB as a recruitment tool in a study for Spanish- and English-speaking smokers. Additionally, the study set out to compare performance of ads by language (Spanish vs. English), location (U.S. vs. San Francisco) and content (self-efficacy ad vs. fear appeal ad).

Methods
Participants of a one-condition smoking cessation webapp study were recruited utilizing FB ads and posts through two phases: a recruitment-focused phase and an experimental phase comparing language, location and content.

Results
During the recruitment phase 581 participants in total (U.S. = 540, San Francisco = 41) provided consent. Of the U.S. participants 275 were Spanish-speakers and 265 English-speakers. The cost-per-consent was 25.81forSpanish−speakers,and25.81 for Spanish-speakers, and 15.49 for English-speakers. During the experimental phase U.S. users performed better (i.e. more clicks, engagement and social reach) than San Francisco users, Spanish-speakers engaged more than English-speakers, and the self-efficacy ad performed better than the fear appeal ad.

Conclusions
This study showed that although there were differences in cost-per-consent for Spanish- and English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing performance of ads by location, language, and ad content may contribute to developing more efficient campaigns.

Full text

Contents lists available at ScienceDirect
Internet Interventions
journal homepage: www.elsevier.com/locate/invent
Facebook for recruiting Spanish- and English-speaking smokers
☆
Eduardo L. Bunge
1
, Lesley A. Taylor
1
, Melissa Bond
1
, Taylor N. Stephens
1
, Kara Nishimuta
1
,
Alinne Z. Barrera
1
, Robert Wickham, Ricardo F. Muñoz
⁎,1
Palo Alto University, United States of America
ARTICLE INFO
Keywords:
Recruitment
Facebook
Smoking
Cessation
Spanish-speakers
ABSTRACT
Background: Recruitment for research is usually expensive and time consuming. Facebook (FB) recruitment has
become widely utilized in recent years. The main aim of this study was to assess FB as a recruitment tool in a
study for Spanish- and English-speaking smokers. Additionally, the study set out to compare performance of ads
by language (Spanish vs. English), location (U.S. vs. San Francisco) and content (self-efficacy ad vs. fear appeal
ad).
Methods: Participants of a one-condition smoking cessation webapp study were recruited utilizing FB ads and
posts through two phases: a recruitment-focused phase and an experimental phase comparing language, location
and content.
Results: During the recruitment phase 581 participants in total (U.S. = 540, San Francisco = 41) provided
consent. Of the U.S. participants 275 were Spanish-speakers and 265 English-speakers. The cost-per-consent was
$25.81 for Spanish-speakers, and $15.49 for English-speakers. During the experimental phase U.S. users per-
formed better (i.e. more clicks, engagement and social reach) than San Francisco users, Spanish-speakers en-
gaged more than English-speakers, and the self-efficacy ad performed better than the fear appeal ad.
Conclusions: This study showed that although there were differences in cost-per-consent for Spanish- and
English-speakers, recruitment of Spanish-speakers through Facebook is feasible. Furthermore, comparing per-
formance of ads by location, language, and ad content may contribute to developing more efficient campaigns.
1. Introduction
Recruitment for human subjects research is usually expensive and
time consuming (Thornton et al., 2016). Online recruitment has been
shown to be more efficient than offline methods in terms of total par-
ticipants enrolled, enables accessibility to larger and more diverse
participants, requires shorter recruitment periods, and reduces overall
study recruitment costs (Christensen et al., 2017;Lane et al., 2015;
Whitaker et al., 2017). However, there are concerns regarding sample
representativeness (Choi et al., 2017) and minimal research exists on
reach and enrollment of participants from diverse backgrounds (e.g.,
non-English speaking; Lane et al., 2015;Kayrouz et al., 2016;Bunge
et al., 2018).
The most commonly utilized online recruitment sources include:
Google Ads, Amazon Mechanical Turk (AMT), Facebook (FB), and
Craigslist (Temple and Brown, 2011). Evidence suggests that these
methods of recruitment result in similar samples and are as cost-
effective as more traditional face-to-face recruitment methods
(Thornton et al., 2016). A systematic review of 110 health and mental
health studies utilizing FB recruitment concluded that, on average, re-
cruitment costs were $6.79 per participant and few differences were
noted when compared to samples recruited using face-to-face methods
(Thornton et al., 2016). Furthermore, FB may also be more time-ef-
fective compared to traditional recruitment methods (e.g., emails, print
advertisements and media releases), with one study finding participants
were recruited up to 2.5 times faster (Kayrouz et al., 2016).
FB allows for various recruitment options (see Akers and Gordon,
2018), such as posting general ads, promoting public fan pages, and
boosting posts. One study found that of these techniques, promoting FB
pages and boosting posts demonstrated the largest impact on recruit-
ment rates (Kayrouz et al., 2016). Furthermore, FB offers the promising
advantage of potential respondent driven sampling, also known as
snowball sampling, which capitalizes on FB's inherent peer networking
structure (i.e., FB users recruit other FB users or non-FB peers; Pedersen
https://doi.org/10.1016/j.invent.2019.02.002
Received 16 November 2018; Received in revised form 9 February 2019; Accepted 13 February 2019
☆
This research was supported by funds from the California Tobacco-Related Disease Research Grants Program Office of the University of California (PI: Muñoz),
grant number 24RT-0027. This research was supported in part by a grant from Google Adwords.
⁎
Corresponding author at: Palo Alto University, 1791 Arastradero Rd., Palo Alto, CA 94304, United States of America.
E-mail address: rmunoz@paloaltou.edu (R.F. Muñoz).
1
i4Health at Palo Alto University.
Internet Interventions 17 (2019) 100238
Available online 26 February 2019
2214-7829/ © 2019 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/BY-NC-ND/4.0/).
T

and Kurz, 2016). FB maximizes the potential to access the aforemen-
tioned “hard-to-reach individuals,”given the omnipresent use of FB
among various populations (Lane et al., 2015).
Several smoking cessation studies recruited participants through FB
(Akers and Gordon, 2018;Carlini et al., 2014;Ramo and Prochaska,
2012;Ramo et al., 2014, 2015a, 2015b;Sadasivam et al., 2013, 2016;
Thornton et al., 2013). Ramo and Prochaska (2012) examined FB as a
recruitment channel for surveying tobacco and other substance use for
individuals 18–25 and reported a final cost per valid, completed survey
of $4.28. FB estimated that 2.8% of accounts for people aged 18–25
were reached through tobacco and marijuana keywords (Ramo and
Prochaska, 2012). Ramo et al. (2014) employed multiple ad types for a
smoking cessation trial over a seven-week period, such as ads in news
feeds, promoted posts, sponsored stories, and standard ads, averaging
$8.80 per eligible, consented participant. They found that images of
smoking and news feed ads yielded the greatest reach and clicks at the
lowest cost, and posited that this may be partially due to news feed ads
being viewable by mobile device. Ramo et al. (2015a, 2015b) con-
ducted a mixed-methods study surveying young adults about tobacco
and social media use to see if participants were interested in a smoking
cessation intervention through FB and, if so, how FB should be used to
help this population quit smoking. About one third of their sample
(31%) reported that they would use FB to quit smoking, and interest in
using FB was greater among those who were more motivated to quit,
had made a quit attempt within the last year, and had previously used
an online source to quit (Ramo et al., 2015a, 2015b). Sadasivam et al.
(2013) created a technology-assisted tobacco intervention to recruit
people through peer marketing. The initial wave of their participants,
or their “seeds,”(n = 190) were recruited through FB ads, the peer
recruits (n = 569) were recruited through the seeds, and the overall
cost per smoker recruited was $29.80 (Sadasivam et al., 2013). Inter-
estingly, FB can target people who are not searching for smoking ces-
sation information as well, a beneficial recruitment outcome that would
not be accessible with other online recruitment methods such as Google
Ads. For example, Akers and Gordon (2018) conducted a clinical trial to
teach support skills to partners of smokeless users and recruited 1145
female partners of male smokeless tobacco users over a 15-month
period.
While FB offers the benefit of broad reach, potential drawbacks
include privacy concerns and a general “lack of clear guidance around
human subjects issues”(Pedersen and Kurz, 2016, p. 5). Additionally,
FB recruitment outcomes vary depending on how campaigns are de-
signed and conducted, with some studies reporting that FB is more
expensive than other similar online methods (Frandsen et al., 2014;
Heffner et al., 2013). Heffner et al. (2013) compared the cost of various
recruitment strategies (i.e., standard media, broadcast emails, word of
mouth, Google AdWords, and social media) for a pilot study on web-
based Acceptance and Commitment Therapy (ACT) for smoking cessa-
tion. Results demonstrated that FB ads and posts were the least cost
effective method, costing $172.76 per participant vs. $46.98 per par-
ticipant for standard media, for example, which included press releases
on local television, radio, newspaper, and online news outlets (Heffner
et al., 2013). Similarly, for an in-person smoking cessation clinical trial,
Frandsen et al. (2014) compared FB ads to traditional methods (flyers,
word of mouth, and newspaper advertisement) and found that the cost
per participant using traditional methods was less than half of that for
FB ads.
Other common limitations in the existing research that utilized FB
recruitment strategies include: restriction of sample to individuals that
have internet access and use FB regularly (Ramo et al., 2010), and ads
that do not include diverse languages (Amon et al., 2014). To date, no
FB recruitment studies have reported differences by language for
smoking cessation studies. Although Carlini et al. (2014) conducted a
study for low English proficiency Brazilians, with FB ads in Portuguese
(the cost was $31.10 per smoker), it did not compare recruitment
outcomes by language (English vs. Portuguese).
Few studies have reported on recruitment methods for Spanish-
speaking populations for smoking cessation studies (Graham et al.,
2012;Muñoz et al., 2006). Graham et al. (2012) focused on developing
culturally-specific advertisements to recruit Spanish-speaking popula-
tions for a smoking cessation website. They rotated ads systematically
across four popular Latino websites and found that loss-framed ads
yielded a higher click-through rate than gain-framed ads, and surface-
targeted ads outperformed deep-targeted ads for clicks, click-through
rate, and number of registrants (Graham et al., 2012). A series of
smoking cessation studies were also done by Muñoz and colleagues in
both English and Spanish (Muñoz et al., 2006;Muñoz et al., 2009;
Muñoz et al., 2012;Leykin et al., 2012). Most of the recruitment efforts
were done through Google Ads. These studies examined the feasibility
but not the costs of recruiting Spanish-speaking smokers worldwide. For
example, in one study, 94,158 individuals from 152 countries and
territories visited the site (Muñoz et al., 2012). A total of 9173 Spanish-
and English-speaking smokers provided consent, however the recruit-
ment costs were not reported (Muñoz et al., 2012).
The aim of the current study was to assess FB as a recruitment tool
in a study for Spanish- and English-speaking smokers. Phase 1 examined
ad lib recruitment of English- and Spanish-speaking adult smokers
utilizing FB ads and posts for a smoking cessation webapp (recruitment-
focused phase). Phase 2 systematically compared performance of re-
cruitment methods by language (Spanish-speakers vs. English-
speakers), location (San Francisco vs. U.S.) and content (self-efficacy ad
vs. fear appeal ad) (experimental phase). Two main metrics were ana-
lyzed: number of clicks and engagement.
2. Method
2.1. Participants
Participants were recruited if they resided in the U.S. and reported
being 18 years of age and older. The recruitment period was from
January 9th, 2018 to May 16th, 2018. Participants were recruited for a
one-condition smoking cessation study utilizing a webapp, where they
completed questionnaires about their smoking habits and quit con-
fidence, tracked their quit attempts, and received periodic follow up
texts to track their quit progress. The responsive webapp was designed
to work in a web browser on a computer or mobile device and did not
require downloading a specific app or other software.
2.2. Assessments and measures
An extensive inventory of metrics exists to assess the success and
connectivity of advertisement/recruitment strategies employed. The
following metrics have been defined based on the Facebook Business
(2018) stipulations, as FB is the recruitment channel of interest for this
article. Metrics were divided in two: those based on FB algorithms and
those based on participant behaviors. The metrics based on FB algo-
rithms include: Click-Through Rate, Impression, Reach, Social Reach,
Relevance, Unique Outbound Click. The metrics based on participant
behaviors include: Clicks, Consents, Conversion rate, Cost-per-consent,
Cost-per- click, Link clicks, Post comments, Post reactions, Post shares,
Shares. See Table 1 for a description of each metric type.
2.3. Procedures
The initial recruitment channels utilized in this study were Google
AdWords, announcements in relevant list serves, local fliers, and dis-
tribution of 3 × 5 cards in Spanish and English with links to the web-
site. The low recruitment rates led the researchers to add FB recruit-
ment. This study only reports on participants recruited through FB.
Participants were recruited in two phases: a recruitment-focused
phase and an experimental phase comparing language, location and
content (self-efficacy ad vs. fear appeal ad). The recruitment-focused
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
2

phase acted as a pilot test for recruiting participants with several dif-
ferent strategies that varied at a fast pace. Once the recruitment target
was reached, the experimental phase focused on systematic compar-
isons between ads that performed well during the previous phase.
2.3.1. Recruitment-focused phase
The recruitment-focused phase primarily aimed to recruit as many
participants as possible in a short period of time. Starting on January
9th 2018 up to April 30th 2018, two community managers (CM) were
hired to recruit participants. One CM focused on recruitment of English-
speaking smokers and the other CM focused on Spanish-speaking
smokers. Each CM implemented her own recruitment strategy for her
target audience. The English strategy mostly focused on Facebook Ads
through the Ads Manager tool and the Spanish strategy mostly focused
on boosted posts directly from Facebook. Across all campaigns between
January 9th and May 16th, 2018 a total of 70 ads were run over
19 weeks; 50 ads in English and 20 ads in Spanish. Regarding posts, a
total of 62 posts were run over 19 weeks; 15 posts in English and 47
posts in Spanish. During this phase, both ads and posts presented dif-
ferent text content, images, starting time and locations.
2.3.2. Experimental phase
The second phase systematically compared performance of recruit-
ment methods by language (Spanish-speakers vs. English-speakers),
location (San Francisco only vs. US) and content (self-efficacy ad vs.
fear appeal ad). The experimental phase ran from May 3rd –May 16th
2018. For this phase two of the best performing ads from the recruit-
ment-focused phase were selected based on CPC, CTR, and relevance.
Campaigns were run by language (Spanish-speakers vs. English-
speakers), location (San Francisco vs. US) and content (self-efficacy ad
vs. fear appeal ad). For direct comparison, the ad images and content
are shown in Fig. 1. Content for the self-efficacy ad included “How to
Quit”text, while the fear appeal ad included “Just one cigarette a day
can double your risk of heart disease”text.
The complete study was approved by both the University of
California San Francisco (15-17597) and Palo Alto University (15-042-
H) Institutional Review Boards.
2.4. Analysis
Due to large variability and lack of controls during Phase 1, in-
ferential statistics could not be completed for the recruitment-focused
phase, so only descriptive analyses are provided. Phase 2, the experi-
mental phase, was analyzed using a 2 × 2 × 2 binomial logistic re-
gression with the three factors of language, location and content. The
binomial logistic regression allows for direct comparison across lan-
guage, location, and content despite the difference in Impressions or
Reach because it treats each dependent variable as a yes/no outcome
with a total N of the denominator variable (e.g., Impressions or Reach).
Since Facebook does not reveal how their unique metrics (i.e., re-
levance) are calculated, only continuous outcomes with enough power
to detect differences were included in the analysis. Because both Reach
and impressions are determined by FB proprietary algorithms, there are
likely differences in how different populations are targeted. Although
impressions are often used as a base to calculate common industry
metrics (e.g., CTR, CPC), Reach may be more appropriate for re-
searchers looking to evaluate differences on an individual level. Both
Impressions and Reach are determined by FB's proprietary algorithms,
but once an individual is reached (sees the ad at least once), some
metrics are based on the individual's behavior (clicks, shares, and en-
gagement), not the FB algorithm. Therefore, clicks, total engagement,
and social reach were run with both Impressions and Reach as de-
nominators. UOC was only run with Reach as denominator because it
measures unique clicks, and using impressions (which are not unique)
Table 1
Campaign type and metrics.
Campaign type
Ads Paid advertisements that may appear in any given user's news feed or side-panel (run across FB, Instagram, Audience Network, and Messenger), can be targeted to specific
populations.
Posts FB Posts (text, images, or videos) are displayed within users' news feeds and can be self-posted or generated by their friends or followed pages.
Boosted Posts. Users can pay to “boost”their posts' visibility.
Metrics based on Facebook
Click-Through Rate (CTR) Percentage of times individuals see an ad and click on a link. The ratio of times that individuals click a link in an ad compared to the amount of
times these individuals see the ad.
Engagement The total number of actions that people take involving the ads.
Impressions Number of times an ad is shown on screen for the targeted audience.
Reach Number of individuals that see an ad at least once. (This differs from impressions which includes multiple views of an ad by the same
individual.)
Relevance Calculated based on an ad's positive feedback (e.g., app installs, clicks, video views) and negative feedback (e.g., User clicks “I don't want to see
this”on the ad) and ranges from 1 (lowest) to 10 (highest). Higher relevancy can lower cost and improve delivery.
Social reach Number of individuals that see an ad when displayed with social information, acting as word-of-mouth for advertisers (e.g., other FB friends
who engaged with the ad or FB page).
Unique Outbound Clicks (UOC) Number of clicks on links that redirect users offFB and to the targeted site.
Metrics based on participant behaviors
Clicks Number of clicks on an ad. Counts multiple types of clicks, such as links to other destinations and links to expanded ad experiences.
Consents Number of individuals consented to participate in the research study.
Conversion rate Metric calculated based on the Unique Outbound Clicks (UOC; see definition above) and FB-attributed consents obtained.
Cost per consent Cost per consenting individual obtained.
Cost per link click (CPC) Average cost per each link click, benchmark for ad efficiency and performance. Calculated by dividing the total amount spent (U.S. $) per link click.
Link clicks Number of clicks on ad links to specific destinations or experiences, both on and offFB-owned properties (e.g., websites, App stores, click-to-call, etc.).
Post comments Number of comments per ad, which counts all comments made by individuals while the ad was running.
Post reactions Number of times individual users reacted to a post with a click, which includes reactions such as “like”and “love.”
Post shares Number of shares per ad while running (may also include Instagram shares sent to inboxes), but does not count engagement with the post after the
share.
Notes. All definitions are based on Facebook Business' (2018) Glossary of Terms.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
3

as a denominator could cause confounded results.
3. Results
During the recruitment period, between January 9th and May 16th,
2018, the 70 ads and 62 posts received 2,357,181 impressions and
yielded 30,169 unique outbound clicks for a total cost of $19,289.15.
Total time spent managing the ads by the Spanish speaking and English
speaking CMs was 164 h and 139 h, respectively.
3.1. Recruitment-focused phase
During the recruitment phase there were 581 participants in total:
540 were from elsewhere in the U.S. and 41 from San Francisco. Of the
540 U.S. participants that provided consent, 275 were Spanish-speakers
and 265 English-speakers. A total budget of $11,203.40 was allocated
for both languages; the final budget used was $7098.96 for Spanish
language ads and $4104.44 for English language ads. Metrics generated
for the Spanish and English language ads, respectively, were 1.83% vs.
1.77% for CTR, $0.43 vs. $0.44 for CPC, and a cost-per-consent of
$25.81 vs. $15.49 (See Table 2).
3.2. Experimental phase
During the experimental phase a total of 43 participants were re-
cruited (37 for U.S. and six for San Francisco). A total budget of $2600
was allocated for both languages and the final budget used was for
$1299.87 Spanish and $1298.72 for English. Table 3 shows descriptive
outcomes for ads only by location. Of the 37 U.S. participants that
provided study consent, 18 were Spanish-speakers and 19 English-
speakers. For Spanish-speakers, CTR was 1.41%, CPC was $0.41, and
cost-per-consent was $27.78. For English-speakers, CTR was 1.44%,
Fig. 1. Example of Spanish and English Facebook ads.
Notes. Ads (a) and (b) illustrate fear appeal, and ads (c) and (d) illustrate self-efficacy messages.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
4

CPC was $0.49, and cost-per-consent was $26.31. Of the six SF parti-
cipants that provided consent, four were Spanish-speakers and two
English-speakers. For Spanish-speakers, CTR was 0.64%, CPC was
$1.23, and cost-per-consent was $199.98. For English-speakers, CTR
was 0.60%, CPC was $1.09, and cost-per-consent was $399.39 (See
Table 3).
3.3. Impressions as denominator
Chi-square values and odd-ratios for the logistic binomial regres-
sions calculated with impressions as the denominator can be found in
Table 4. For clicks, there was a significant main effect observed for
location (X
2
= 581.35, p < .005) and content (X
2
= 35.02,
p < .005):U.S. participants were 2.28 times more likely to click an ad
than San Francisco participants, individuals were 1.23 times more
likely to click on the self-efficacy ad than the fear appeal ad. For En-
gagement, there was a significant main effect for location (X
2
= 832.88,
p < .005), language (X
2
= 9.65, p < .005), and content (X
2
= 29.10,
p < .005): U.S. participants were 2.56 times more likely to engage;
Spanish-speaking participants were 1.11 times more likely to engage.
The content variable showed that the self-efficacy ad was 1.20 times
more likely to be engaged with compared to the fear appeal ad. Re-
garding Social Reach, there was a significant main effect for location
(X
2
= 31.25, p < .005), language (X
2
= 1077.74, p < .005), and
content (X
2
= 19.44, p < .005), as well as significant two-way inter-
actions for location with language (X
2
= 82.63, p < .005) and content
(X
2
= 6.35, p < .05). The odds-ratios indicate that, overall, U.S. par-
ticipants were 1.52 times more likely to reach a new person via social
means (i.e. shares, comments, etc.), Spanish-speaking participants were
7.32 times more likely to reach new users via social means, and the self-
efficacy ad was 1.40 times more likely to reach people via social means.
These main effects are qualified by the significant interactions of lo-
cation by language and content. Specifically, Spanish-speakers within
San Francisco (OR = 14.39) were far more likely to reach new users via
social means than were Spanish-speakers in the U.S. (OR = 3.72). Si-
milarly, the self-efficacy ad shown within San Francisco (OR = 1.70)
was more likely to reach new users via social means than the same ad in
the U.S. sample (OR = 1.15).
3.4. Reach as denominator
Chi-square values and odd-ratios for the logistic binomial regres-
sions run with Reach as the denominator can be found in Table 5. When
run with Reach as the denominator, significant main effects of location
(X
2
= 192.64, p < .005), language (X
2
= 14.43, p < .005), and con-
tent (X
2
= 20.13, p < .005) were found for clicks, as well as a sig-
nificant interaction between location and content (X
2
= 6.89,
p < .01). In general, users in the U.S. were 1.61 times more likely to
click, Spanish-speakers were 1.14 times more likely to click, and the
self-efficacy ad received 1.17 times the amount of clicks than did the
fear appeal ad. Furthermore, the self-efficacy ad was far more effective
at receiving clicks in the U.S. (OR = 1.28) than it was in San Francisco
(OR = 1.07). For engagement, main effects were revealed for location
(X
2
= 327.64, p < .005), language (X
2
= 41.96, p < .005), and con-
tent (X
2
= 15.25, p < .005), such that users in the U.S. were 1.81
times more likely to engage, Spanish-speakers were 1.24 times more
likely to engage, while the self-efficacy ad was 1.14 times more likely to
be engaged with. Significant interactions also emerged for engagement
for location with language (X
2
= 13.45, p < .005) and content
(X
2
= 8.01, p < .005), indicating that Spanish-speakers (OR = 1.40)
or those who saw the self-efficacy ad (OR = 1.25) were most likely to
engage if they were in the U.S. rather than San Francisco. Regarding
social reach, main effects were found for language (X
2
= 1224.57,
p < .005), and content (X
2
= 14.11, p < .005), which suggests that
Spanish-speakers were 8.21 times more likely to be reached via social
means, while the self-efficacy ad was 1.33 times more likely to reach
users via social means. Significant interactions were also found for lo-
cation with language (X
2
= 56.46, p < .005) and content (X
2
= 4.03,
p < .05). Social reach was far more likely to be exhibited for Spanish-
speakers within San Francisco (OR =14.38) as well as the self-efficacy
ad shown within San Francisco (OR = 1.55). Within UOC, significant
main effects emerged for location (X
2
= 163.36, p < .005), language
(X
2
= 12.15, p < .005), and content (X
2
= 20.87, p < .005), sug-
gesting that users in the U.S. were 1.57 times more likely to click on an
outbound link, Spanish-speakers were 1.13 times more likely to click on
an outbound link, and the self-efficacy ad was 1.18 times more likely to
have outbound links be clicked.
4. Discussion
Few articles have reported on recruitment methods for smoking
cessation studies conducted among Spanish-speaking populations
(Graham et al., 2012;Muñoz et al., 2006). The main aim of this study
was to assess FB as a recruitment for a one-condition smoking cessation
webapp study for Spanish- and English-speaking smokers. Recruitment
was conducted in two phases, the first phase aimed to maximize re-
cruitment outcomes during a specified period of time without
Table 2
Recruitment-focused phase (U.S.) n = 540: recruitment costs and performance
metrics by language including posts and ads.
Metric type Total Spanish English
Impressions 1,430,119 901,367 528,752
Reach 1,034,038 632,339 401,699
Frequency 1.38 1.45 1.32
Link clicks 25,835 16,486 9349
UOC 23,613 15,072 8541
Shares 1887 1513 374
Comments 491 359 132
Reactions 3828 3120 708
Social reach 8359 7106 1253
CTR, % 1.81 1.83 1.77
CPC, US$ $0.43 $0.43 $0.44
Consent to the study (FB) 540 275 265
Facebook mobile 475 266 209
Facebook website 65 9 56
Conversion rate (FB) 2.29% 1.82% 3.10%
Total spent $11,203.40 $7098.96 $4104.44
Cost-per-consent (FB, US$) $20.75 $25.81 $15.49
Note. CPC = cost-per-click; CTR = click-through-rate; FB = Facebook;
UOC = Unique Outbound Clicks; all costs measured in U.S. dollars (US$).
Table 3
Descriptives for experimental phase (U.S. and San Francisco), recruitment costs
and performance metrics by language for ads only.
Metric type U.S. SF
Spanish English Spanish English
Impressions 86,200 70,210 100,827 122,040
Reach 54,913 56,232 50,999 61,334
Frequency 1.57 1.25 1.98 1.99
Link clicks 1212 1010 649 730
Unique Outbound Clicks (UOC) 1117 951 621 696
Shares 102 31 6 3
Comments 44 13 1 1
Reactions 153 47 30 17
Social reach 616 135 937 80
Click-through-rate (CTR), % 1.41 1.44 0.64 0.60
Cost-per-click (CPC), US$ $0.41 $0.49 $1.23 $1.09
Consents (FB) 18 19 4 2
Facebook mobile 17 14 4 2
Facebook website 1 5 0 0
Conversion rate (FB) 1.61% 2.00% 0.62% 0.29%
Total spent $499.96 $499.94 $799.91 $798.78
Cost-per-consent (FB,US$) $27.78 $26.31 $199.98 $399.39
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
5

implementing a predetermined or systematic approach to the ads. This
phase served as a pilot study to identify the necessary recruitment
procedures for an experimental design. Once the recruitment needs
were met (i.e., as many participants as possible using a fast-paced ad
lib, free-form approach), an experimental phase was implemented to
systematically assess differences in recruitment campaigns by location,
language, and content of the ads.
Overall results from the recruitment-focused phase indicate that
Facebook ads and posts were effective in recruiting Spanish- and
English-speaking smokers to an online smoking cessation webapp. A
total of 581 participants provided consent (540 were from US and 41
from San Francisco), the number of U.S. participants that consented by
language were 275 for Spanish-speakers and 265 for English-speakers;
and the overall cost-per-consent was $20.75. Since the English-lan-
guage and Spanish-language campaigns were different (English-
language strategy focused mostly on ads and the Spanish-language
strategy focused mostly on boosted posts), the outcomes of such cam-
paigns are presented for informative purposes, but no generalizable
assumptions can be made based on such outcomes. The cost-per-con-
sent for the English campaign was $15.49, which is similar to the cost
reported by Thornton et al. (2016) of $18.18 for English language
studies that did not offer incentives. However, the cost-per-consent for
the current study was higher than for studies that offered incentives
(see Ramo et al., 2014 - $8.80 per-consent). A possible explanation for
this difference may be that non-incentivized intervention studies that
aim to change behavior may require extra motivation and commitment
from individuals when compared to briefer or incentivized studies.
Furthermore, comparisons between the current study and previous
studies may be impacted by changes in FB's proprietary algorithm.
Additionally, as FB becomes a more popular advertising platform,
Table 4
Binomial logistic regression results for clicks, engagement, and social reach, with impressions as denominator.
Clicks Engagement Social reach
X
2
OR X
2
OR X
2
OR
Location
US vs. SF Only
581.35
⁎⁎⁎
2.28 832.88
⁎⁎⁎
2.56 31.25
⁎⁎⁎
1.52
Language
Spanish vs.
English
0.33 1.02 9.65
⁎⁎⁎
1.11 1077.74
⁎⁎⁎
7.32
Adset
Self Eff. vs.
Fear
35.02
⁎⁎⁎
1.23 29.10
⁎⁎⁎
1.20 19.44
⁎⁎⁎
1.40
Clicks Engagement Social reach
X
2
OR X
2
OR X
2
OR
US SF US SF US SF
Location ∗Language
Spanish vs. English
2.56 0.965 1.08 0.0 1.11 1.11 82.63
⁎⁎⁎
3.72 14.39
Location ∗Adset
Self Eff. vs. Fear
2.20 1.29 1.17 2.68
⁎⁎
1.26 1.13 6.35
⁎
1.15 1.70
⁎
Significant at p < .05.
⁎⁎
Significant at p < .01.
⁎⁎⁎
Significant at p < .005.
Table 5
Binomial logistic regression results for clicks, engagement, social reach, and UOC, with reach as denominator.
Clicks Engagement Social reach UOC
X
2
OR X
2
OR X2 OR X
2
OR
Location
US vs. SF Only
192.64
⁎⁎⁎
1.61 327.64
⁎⁎⁎
1.81 0.82 1.07 163.36
⁎⁎⁎
1.57
Language
Spanish vs. English
14.43
⁎⁎⁎
1.14 41.96
⁎⁎⁎
1.24 1224.57
⁎⁎⁎
8.21 12.15
⁎⁎⁎
1.13
Adset
Self Eff. vs. Fear
20.13
⁎⁎⁎
1.17 15.25
⁎⁎⁎
1.14 14.11
⁎⁎⁎
1.33 20.87
⁎⁎⁎
1.18
Clicks Engagement Social reach UOC
X
2
OR X
2
OR X2 OR X
2
OR
US SF US SF US SF US SF
Location ∗Language
Spanish vs. English
3.56 1.22 1.07 13.45
⁎⁎⁎
1.40 1.10 56.46
⁎⁎⁎
4.69 14.38 2.36 1.20 1.07
Location ∗Adset
Self Eff. vs. Fear
6.89
⁎⁎
1.28 1.07 8.01
⁎⁎⁎
1.25 1.04 4.03
⁎
1.14 1.55 2.68 1.28 1.09
⁎
Significant at p < .05.
⁎⁎
Significant at p < .01.
⁎⁎⁎
Significant at p < .005.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
6

competition for ad space increases, which consequently affects both
costs and click-through-rates. Finally, recruitment may vary depending
on the types of ads that each study used, therefore, comparing outcomes
of studies using different ads may be hard to evaluate.
Interestingly, the Spanish campaign had a higher cost-per-consent
($25.81) than the English campaigns. It is difficult to clearly ascertain
the reason for this outcome as there is a dearth of studies that have
examined these metrics among Spanish-speakers specifically and, more
broadly, within smoking cessation web-based studies. Despite the fact
that the Spanish language is the second most common native language
spoken worldwide, it may be more expensive to recruit Spanish
speakers in the U.S. given greater costs to target recruitment efforts.
Additional external factors may further impede our ability to draw
strong conclusions from this study. First, FB's proprietary algorithm
determines how the different audiences are reached. For example, in
our study Spanish-speakers were delivered ads more frequently than
English-speakers. Second, the methods used by the community man-
agers in the recruitment phase for each language were different.
Together with previous studies using other online recruitment methods,
such as Google ads (Graham et al., 2012;Muñoz et al., 2006;Muñoz
et al., 2009;Leykin et al., 2012;Muñoz et al., 2012), this study de-
monstrates the feasibility of recruiting Spanish speaking smokers in the
U.S. and contributes to the still limited literature on online recruitment
methods for Spanish-speakers. For example, previous studies that have
shown the feasibility of recruiting Spanish speaking smokers worldwide
(Muñoz et al., 2006;Muñoz et al., 2009;Leykin et al., 2012;Muñoz
et al., 2012), were done using Google Ads, and were carried out more
than six years ago.
During the experimental phase a series of systematic comparisons
between languages, locations, and ad content were concurrently de-
ployed using the same ads. Direct comparisons were calculated using
both Impressions and Reach as denominators. Although Impressions are
often used as a standard to calculate common industry metrics (e.g.,
CTR, CPC), Reach may be more appropriate for researchers looking to
evaluate differences on an individual level. Both Impressions and Reach
are determined by FB's proprietary algorithm, but once an individual is
reached (sees the ad at least once), some metrics are based on the in-
dividual's behavior (clicks, shares, and engagement), not the FB algo-
rithm.
The results of the analysis using Impressions as the denominator
indicated that U.S. participants had higher rates of clicks, engagements,
and social reach than San Francisco participants. Spanish-speaking
participants had higher rates of engagement and social reach than
English-speakers. The self-efficacy ad had higher rates of clicks, en-
gagements, and social reach than the fear ad. Additionally, Spanish-
speakers within San Francisco had higher rates of social reach
(OR = 14.39) than in the U.S.; and the self-efficacy ad when shown
within San Francisco had higher rates of social reach than more broadly
in the U.S.
When Reach was the denominator users in the US had higher rates
of clicks, UOCs, and engagements than users in San Francisco. Spanish-
speakers had higher rates of clicks, UOCs, engagement, and social reach
than English-speakers. Additionally, Spanish-speakers who saw the self-
efficacy adset had higher rates of engagement and social reach in the
U.S. compared to San Francisco. The self-efficacy ad had higher rates of
clicks, UOCs, engagements, and social reach than the fear ad.
Additionally, the self-efficacy ad had higher rates of clicks and en-
gagement in the U.S. compared to San Francisco.
Overall, results followed a similar pattern when evaluated based on
Impressions or Reach. U.S. users performed better (i.e. more clicks,
engagement, social reach, UOC) than San Francisco users, Spanish-
speakers engaged more than English-speakers, and the self-efficacy ad
performed better than the fear appeal ad. When using Reach as a de-
nominator, there were significant interactions for clicks (location by
language) and engagement (location by language, and location by type
of ad) that were not evident when using impressions as the
denominator. These results highlight the relevance of considering me-
trics other than industry standards (i.e., Impressions as a denominator)
in order to capture a more accurate representation of users' behavior
when recruiting for smoking cessation studies.
5. Limitations and future directions
There are several limitations to the current study. First, because
during the recruitment phase each community manager had different
strategies, generalizable conclusions are difficult to draw. The recruit-
ment phase was successful in achieving higher conversion rates and
lower cost-per-consent than the experimental phase. However, it is
unclear if these outcomes were due to heterogeneity between campaign
strategies, or continued recruitment from the same audience pool, or
ads becoming less effective over time. Future studies could improve
their recruitment efforts if direct comparisons between ad type and
languages are implemented from the beginning.
Secondly, FB's proprietary algorithm made it challenging to de-
termine what effects were due to the algorithm versus differences in
individual behavior, keeping in mind that both can affect cost.
Additionally, due to the types of data provided by FB, statistical ana-
lysis could only be performed on certain variables.
Third, it is difficult to evaluate differences between users residing in
the broader U.S. and those within San Francisco only. The San
Francisco campaign was less effective than the U.S. campaign, but that
may be explained by differences in smoking prevalence rates and po-
pulation size. Therefore, these results should not be generalized to other
U.S. cities or states without further considerations of demographic and
smoking differences within regions. Future studies could benefit from
comparing ad performance between different states or cities. For ex-
ample, it is possible that ads may have performed better in states with
higher rates of smokers. Akers and Gordon (2018) suggested to analyze
the tobacco use prevalence by state to conduct effective comparisons
between states.
Fourth, ads in the experimental phase were previously run in the
recruitment phase and originally designed for the English-speaking
audience. Ads designed for an English-speaking audience then trans-
lated into Spanish may perform differently than ads designed for a
Spanish-speaking audience and translated into English. Even though the
text approximated a literal translation, that does not guarantee that
they were equally perceived by both audiences. Thus, the design of the
study did not allow for evaluation of language translation effects.
Fifth, while a variety of ad content were used during the recruit-
ment-focused phase, only two types of content (fear appeal vs. self-ef-
ficacy) were used in the experimental phase. Therefore, more studies
are needed in order to grasp a robust understanding of the effects of ads
as results may not be generalizable to other types of content. Future
studies should attempt to replicate the comparisons employed in the
current study and expand them to other types of content.
Finally, although the experimental phase did have significant find-
ings, it only yielded 43 participants (37 for US and 6 for San Francisco)
over a two week period. As such, the impact of these findings in relation
to Spanish speakers should be interpreted with caution and replicated
with a larger sample.
6. Conclusion
To the best of our knowledge this is the first study to report on the
use of Facebook ads to recruit Spanish-speaking smokers and to com-
pare ad performance by location, language and ad content for the re-
cruitment of smoking cessation webapp studies. This study demon-
strates that FB is an effective platform for recruiting both Spanish- and
English-smokers into clinical trials of smoking cessation webapps.
Although there were differences in cost-per-consent for Spanish- and
English-speakers, recruitment of Spanish-speakers through Facebook is
feasible. Furthermore, comparing an ad's performance by location,
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
7

language, and ad content may contribute to developing more efficient
and effective campaigns. With increased awareness regarding the im-
portance of including minorities and women in research, being able to
effectively reach and recruit diverse audiences is critical. Therefore,
Facebook's ability to target diverse audiences cost-effectively makes it
an important recruitment tool.
Conflict of interest
Facebook for Recruiting Spanish- and English-Speaking Smokers.
The authors report no conflict of interest.
References
Akers, L., Gordon, J.S., 2018. Using Facebook for large-scale online randomized clinical
trial recruitment: effective advertising strategies. J. Med. Internet Res. 20 (11), e290.
https://doi.org/10.2196/jmir.9372.
Amon, K.L., Campbell, A.J., Hawke, C., Steinbeck, K., 2014. Facebook as a recruitment
tool for adolescent health research: a systematic review. Acad. Pediatr. 14 (5),
439–447. https://doi.org/10.1016/j.acap.2014.05.049.
Bunge, E., Cook, H.M., Bond, M., Williamson, R.E., Cano, M., Barrera, A.Z., Leykin, Y.,
Muñoz, R.F., 2018. Comparing Amazon mechanical Turk with unpaid internet re-
sources in online clinical trials. Internet Interv. 12, 68–73. https://doi.org/10.1016/j.
invent.2018.04.001.
Carlini, B.H., Safioti, L., Rue, T.C., Miles, L., 2014. Using internet to recruit immigrants
with language and culture barriers for tobacco and alcohol use screening: a study
among Brazilians. J. Immigr. Minor. Health 17 (2), 553–560. https://doi.org/10.
1007/s10903-013-9934-1.
Choi, I., Milne, D.N., Glozier, N., Peters, D., Harvey, S.B., Calvo, R.A., 2017. Using dif-
ferent Facebook advertisements to recruit men for an online mental health study:
engagement and selection bias. Internet Interv. 8, 27–34. https://doi.org/10.1016/j.
invent.2017.02.002.
Christensen, T., Riis, A.H., Hatch, E.E., Wise, L.A., Nielsen, M.G., Rothman, K.J., ...
Mikkelsen, E.M., 2017. Costs and efficiency of online and offline recruitment meth-
ods:a web-based cohort study. J. Med. Internet Res. 19 (3), e58. https://doi.org/10.
2196/jmir.6716.
Facebook Business. (2018). Glossary of ad terms. Retrieved from https://www.facebook.
com/business/help/447834205249495.
Frandsen, M., Walters, J., Ferguson, S.G., 2014. Exploring the viability of using online
social media advertising as a recruitment method for smoking cessation clinical trials.
Nicotine Tob. Res. 16 (2), 247–251. https://doi.org/10.1093/ntr/ntt157.
Graham, A.L., Fang, Y., Moreno, J.L., Streiff, S.L., Villegas, J., Muñoz, R.F., ... Vallone,
D.M., 2012. Online advertising to reach and recruit Latino smokers to an internet
cessation program: impact and costs. J. Med. Internet Res. 14 (4), e116. https://doi.
org/10.2196/jmir.2162.
Heffner, J.L., Wyszynski, C.M., Comstock, B., Mercer, L.D., Bricker, J., 2013. Overcoming
recruitment challenges of web-based interventions for tobacco use: the case of web-
based acceptance and commitment therapy for smoking cessation. Addict. Behav. 38
(10), 2473–2476. https://doi.org/10.1016/j.addbeh.2013.05.004.
Kayrouz, R., Dear, B.F., Karin, E., Titov, N., 2016. Facebook as an effective recruitment
strategy for mental health research of hard to reach populations. Internet Interv. 4,
1–10. https://doi.org/10.1016/j.invent.2016.01.001.
Lane, T.S., Armin, J., Gordon, J.S., 2015. Online recruitment methods for web-based and
mobile health studies: a review of the literature. J. Med. Internet Res. 17 (7), e183.
https://doi.org/10.2196/jmir.4359.
Leykin, Y., Aguilera, A., Torres, L.D., Pérez-Stable, E.J., Muñoz, R.F., 2012. Interpreting
the outcomes of automated internet-based randomized trials: example of an inter-
national smoking cessation study. J. Med. Internet Res. 14 (1), e5. https://doi.org/10.
2196/jmir.1829.
Muñoz, R.F., Lenert, L.L., Delucchi, K., Stoddard, J., Perez, J.E., Penilla, C., Pérez-Stable,
E.J., 2006. Toward evidence-based internet interventions: a Spanish/English web site
for international smoking cessation trials. Nicotine Tob. Res. 8 (1), 77–87. https://
doi.org/10.1080/14622200500431940.
Muñoz, R.F., Barrera, A.Z., Delucchi, K., Penilla, C., Torres, L.D., Pérez-Stable, E.J., 2009.
International Spanish/English internet smoking cessation trial yields 20% abstinence
rates at 1 year. Nicotine Tob. Res. 11 (9), 1025–1034. https://doi.org/10.1093/ntr/
ntp090.
Muñoz, R.F., Aguilera, A., Schueller, S.M., Leykin, Y., Pérez-Stable, E.J., 2012. From
online randomized controlled trials to participant preference studies: morphing the
San Francisco stop smoking site into a worldwide smoking cessation resource. J. Med.
Internet Res. 14 (3), e64. https://doi.org/10.2196/jmir.1852.
Pedersen, E.R., Kurz, J., 2016. Using Facebook for health-related research study re-
cruitment and program delivery. Curr. Opin. Psychol. 9, 38–43. https://doi.org/10.
1016/j.copsyc.2015.09.011.
Ramo, D.E., Prochaska, J.J., 2012. Broad reach and targeted recruitment using Facebook
for an online survey of young adult substance use. J. Med. Internet Res. 14 (1), e28.
https://doi.org/10.2196/jmir.1878.
Ramo, D.E., Hall, S.M., Prochaska, J.J., 2010. Reaching young adult smokers through the
internet: comparison of three recruitment mechanisms. Nicotine Tob. Res. 12 (7),
768–775. https://doi.org/10.1093/ntr/ntq086.
Ramo, D.E., Rodriguez, T.M.S., Chavez, K., Sommer, M.J., Prochaska, J.J., 2014.
Facebook recruitment of young adult smokers for a cessation trial: methods, metrics,
and lessons learned. Internet Interv. 1 (2), 58–64. https://doi.org/10.1016/j.invent.
2014.05.001.
Ramo, D.E., Liu, H., Prochaska, J.J., 2015a. A mixed-methods study of young adults'
receptivity to using Facebook for smoking cessation: if you build it, will they come?
Am. J. Health Promot. 29 (4), e126–e135. https://doi.org/10.4278/ajhp.130326-
QUAL-128.
Ramo, D.E., Young-Wolff, K.C., Prochaska, J.J., 2015b. Prevalence and correlates of
electronic-cigarette use in young adults: findings from three studies over five years.
Addict. Behav. 41, 142–147. https://doi.org/10.1016/j.addbeh.2014.10.019.
Sadasivam, R.S., Vloz, E.M., Kinney, R.L., Rao, S.R., Houston, T.K., 2013. Share2Quit:
web-based peer-driven referrals for smoking cessation. JMIR Res. Prot. 2 (2), e37.
https://doi.org/10.2196/resprot.2786.
Sadasivam, R.S., Cutrona, S.L., Luger, T.M., Volz, E., Kinney, R., Rao, S.R., ... Houston,
T.K., 2016. Share2Quit: online social network peer marketing of tobacco cessation
systems. Nicotine Tob. Res. 19 (3), 314–323. https://doi.org/10.1093/ntr/ntw187.
Temple, E.C., Brown, R.F., 2011. A comparison of internet-based participant recruitment
methods: engaging the hidden population of cannabis users in research. J. Res. Pract.
7 (2), 1–20.
Thornton, L.K., Baker, A.L., Johnson, M.P., Lewin, T., 2013. Perceived risk associated
with tobacco, alcohol and cannabis use among people with and without psychotic
disorders. Addict. Behav. 38 (6), 2246–2251. https://doi.org/10.1016/j.addbeh.
2013.02.003.
Thornton, L., Batterham, P.J., Fassnacht, D.B., Kay-Lambkin, F., Calear, A.L., Hunt, S.,
2016. Recruiting for health, medical or psychosocial research using Facebook: sys-
tematic review. Internet Interv. 4, 72–81. https://doi.org/10.1016/j.invent.2016.02.
001.
Whitaker, C., Stevelink, S., Fear, N., 2017. The use of Facebook in recruiting participants
for health research purposes: a systematic review. J. Med. Internet Res. 19 (8), e290.
https://doi.org/10.2196/jmir.7071.
E.L. Bunge, et al. Internet Interventions 17 (2019) 100238
8