Available via license: CC BY 4.0
Content may be subject to copyright.
1
nemployment is among the largest problems
facing most western societies. In the
aftermath of the ‘great recession’ unemployment
across the developing world has been slow to
recover, with much of the developed world yet to
reach pre-2008 levels.
This problem is costly to unemployed
individuals and society. Governments are paying
unemployment benefits to more people, for longer,
and hysteresis is dampening productivity. As well
as these shorter-term consequences, long term
impacts may be even greater. Gregg (2001) and
Macmillan (2011) find considerable evidence of
employment "scarring,"; a phenomenon whereby
people (particularly young people) who experience
a spell of unemployment due to a crisis are more
likely to experience similar spells later in life.
There exist many different theoretical models
of employment. As a basis for this experiment, we
assume some validity in models, such as Saks (2006)
and Meyer (1990), that posit job search activity is a
component of the production function of
employment, and that more productive work
search (for example, attending specific recruitment
events where some participants will almost
certainly gain employment) is key to finding work
faster.
In this study, we present the results of a
randomized controlled trial designed to test the
effectiveness of mobile phone text messaging in
compelling jobseekers registered at Bedford
Jobcentre in the UK to attend specific recruitment
events. Such events are run by Jobcentre Plus (in
Bedford and elsewhere) in partnership with large
employers seeking to hire several staff at once.
These arrangements are generally exclusive,
meaning only those invited by the Jobcentre are in
the applicant pool. Although we do not have data
on who was hired in our study, it seems reasonable
to suppose that an increase in attendance might
lead to an increase in employment. This is not,
however, a necessary condition: even if job search
U
Abstract: Finding a job, especially in a recovering or uncertain economy, is challenging. Jobseekers Allowance
(JSA) welfare benefit claimants in the United Kingdom have many competing options available to them in terms
of how they direct their efforts in looking for work. Jobcentres, the organizations that support job seekers, have
very strong links to the labor market and often run recruitment events in direct partnership with large
employers seeking to hire in bulk. Attendance at these events, or any other specific job search activity, is
typically low. This article reports the results of a randomized control trial designed to test the effectiveness of
mobile phone text messaging in compelling jobseekers to attend such events. Tailored text messages are found
to significantly increase the likelihood of attendance. We find text messages to be particularly effective when
they seek to induce reciprocity and address low morale in the recipient.
Keywords: Labor market, Employment, Field experiment, Reciprocity, Personalization
Supplements: Open data, Open materials
Journal of Behavioral
Public Administration
Vol 2(1), pp. 1-9
DOI: 10.30636/jbpa.21.24
Michael Sanders*, Elspeth Kirkman*
I’ve booked you a place, good luck:
Applying behavioral science to improve attendance at
high-impact job recruitment events
* The Behavioral Insights Team, London, the U.K.
Address correspondence to Michael Sanders at
(michael.t.sanders@kcl.ac.uk)
Copyright: © 2019. The authors license this article
under the terms of the Creative Commons Attribution
4.0 International License.
Research Article
Sanders & Kirkman, 2019
2
activity of the type we study is not useful, the
implications of our study suggest that the
interventions studied could be directed to other,
perhaps more effective, behaviors.
Study Context
This study’s sample consisted of Jobseeker’s
Allowance (JSA) claimants accessing employment
services at Bedford Jobcentre. Jobcentre Plus
operates local Jobcentres across the United
Kingdom. Jobcentres have two functions relevant
to this trial: (1) to link their clients to employment
opportunities locally; (2) to determine whether
claimants are complying with the work search
requirements for receiving JSA.
To understand the implementation and theory
of change associated with this trial some context on
the process is required. First, when jobseekers
enroll at Bedford Jobcentre, they are assigned an
Advisor who they will see each time they attend.
This Advisor will document information on the
claimant and their work skills, including tagging
their record with Standard Occupational
Classification (SOC) codes to describe these skills.
Attendance at the Jobcentre typically involves
giving this Advisor evidence to verify job search
activity in the weeks (typically two) since the last
appointment. If the claimant cannot show
sufficient evidence, the Advisor can enact sanctions.
Receipt of JSA is conditioned on demonstrating
compliance at these appointments. As such,
claimants usually know their Advisor by name, they
may have a personal rapport, and there is a clear
role each party plays in their relationship.
Second, some information on the events
themselves: Jobcentres have direct relationships
with the local labor market which often make them
the partner of choice for finding employees to fill
many roles quickly - for example, if a new factory
or super market opens. Jobcentres select claimants
to invite to these events based on whether their
‘standard occupational classification’ (SOC) code
matches the skills required by the employer. All
eligible claimants are then sent an SMS message
inviting them to attend the event. Attendance at
these mass recruitment events, in contrast to many
other types of job search activity, is not mandatory.
Unless the claimant had an appointment with their
Advisor very recently it is likely that text message is
the only way they might hear about such an event.
The messages typically sent (the control in our
experiment) do not indicate anything beyond basic
information about the role(s) or the event.
Literature Review
Although we were not able to conduct in depth
research with claimants, conversations with
Jobcentre staff helped us to identify potential
barriers to attendance and formulate a few working
hypotheses as to how jobseekers determine
whether to attend an event, what barriers suppress
turnout and what might be done to overcome them.
First, claimants may ignore a text message, assume
it was sent in error, or even think it is not genuine
if they do not recognize the sender number. As
such, we speculate that signaling the message is
legitimate, directed, and purposeful may help.
Second, we heard that claimants often fixate on
what they know to be mandatory job search
activities; since this activity is not mandatory it may
not be sufficiently compelling. We speculate that
including an official signal of endorsement may
make this opportunity seem more compelling.
Third, claimants may not appreciate that the
Jobcentre staff is actively working to help them find
employment through these events. We speculate
that drawing attention to the administrative effort
of planning may drive increased attendance. Finally,
claimants may be demoralized as a result of
unemployment and may think their efforts will not
be successful. Including morale boosting language
that seeks to overcome a shrinking locus of control
may, we speculate, boost attendance.
With these four hypotheses in mind, we
reviewed the behavioral science literature to inform
the design of our messaging interventions. The
resulting treatments build on one another with each
including the same language as the last. As such, the
study should be read as a test of the combination
of these treatments and not as a clean test of each
potential mechanism. Below we discuss both the
use of the texting channel and the evidence base on
how to address the barriers to attendance we
observed.
Text Messages as a Call to Action
A strong research precedent suggests that text
messages can provide a high impact, low cost,
channel through which to galvanize action. Text
messages have been trialed in a variety of domains
such as repayment of court fines (Haynes et al.,
2013), loan repayment (Morten, Karlan, & Zinman,
2013) and saving energy (Gleerup et al., 2010). Text
Journal of Behavioral Public Administration, 2(1)
3
messages were already used as one of the
communication channels deployed by Bedford
Jobcentre Plus to promote recruitment events of
this nature.
Using Personalization to Grab Attention
and Boost Legitimacy
Drawing on a well-established body of research,
Haynes et al. (2013) demonstrate that text messages
including the recipient’s first name drive a 41
percent increase in the repayment amounts of
court-ordered fines compared to texts without
personalization.
Adding the Advisor Name to
Signal Official Endorsement
Morten, Karlan, & Zinman (2012) explore the
effect of drawing attention to the relationship
between loan recipient and loan officer on
soliciting repayments via text message. Of the
conditions they trial, “messages that mention the
loan officer’s name significantly, substantially, and
robustly improve repayment rates relative to
messages that mention the client’s name and/or to
the no-message control group” (p. 9). This is only
true for borrowers who have already met their loan
officer, implying that social connection explains the
difference in results. In this experiment, we build
on these findings by attempting to test the
difference between elements of the Jobseeker/
Advisor altercast.
Highlighting Effort to Induce Reciprocity and Signal
that the Jobcentre Invests in These Opportunities
Highlighting effort on the part of Jobcentre staff
might change the decision calculus of the claimant
since it signals that the staff consider these events
sufficiently valuable to invest time in planning them.
Beyond this, literature on the use of reciprocity –
unconditional gift giving or service by one party to
another – suggests this may turn out to be an
effective way to encourage a desired behavior (see
Falk and Fischbacher, 2006). Both theoretical
models and experimental results show that calling
on social conventions of reciprocity is
disproportionately effective when trying to get
individuals to comply with a request. For example,
an experiment into interventions that impact
charitable donations by employees at a large bank
showed that 11 percent of people given the small
gift of a bag of sweets donated a day’s salary
compared to just 5 percent whose donation was
solicited by the traditional awareness raising
campaign tactics (Behavioural Insights Team, 2013).
Field studies of reciprocity conducted to date (e.g.,
Behavioural Insights Team, 2013; Falk, 2007;
Alpizar, Carlsson, & Johansson-Stenman, 2008),
aim to elicit reciprocity by the giving of physical
gifts, such as sweets, fridge magnets and postcards.
In this experiment, we consider whether gifting a
service (the exertion of a small quantity of effort to
book an appointment), can be similarly effective at
motivating behavior.
Evoking Luck to Overcome
a Shrinking Locus of Control
The literature suggests that job seekers with an
internal locus of control conduct more work search
activity than those who believe that their future
outcomes are determined by external factors (e.g.
Caliendo, Cobb-Clark, & Uhlendorff, 2010). As
such, targeting jobseekers with an external locus of
control may help those less likely to attend to show
up. Building on a body of research linking beliefs in
luck to changes in action and behavior, Sagone et
al. (2014) show that those with an external locus of
control – in our domain, those who may be less
likely to perform enough job search activity – have
a stronger belief in the notion of “luck”. In this
experiment, we evoke the concept of luck as a
means to increase the likelihood those with an
external locus of control will take action.
The Intervention
Based on the literature review, we designed three
new messages, each of which builds on the one
before, to test against the control. The control is
the standard language used in such recruitments
prior to the experiment.
Experimental Design
We use a randomized control trial design to
determine the effect of variations on a text message
on attendance rates at specific recruitment events
held by Bedford Jobcentre Plus. Randomization is
conducted at the individual participant level, within
each recruitment event separately, yielding
randomization stratified by recruitment event. All
participants (n = 1,224) were drawn from the pool
of active JSA claimants at Bedford Jobcentre Plus.
Sanders & Kirkman, 2019
4
Participants were randomly assigned to
receive one of the four intervention messages
(coded zero to three). Each participant was part of
the trial for one recruitment event only, so each
participant appears in our data only once. If a
participant is eligible for a second event, they were
sent the same message as previously but excluded
from the experiment and hence from the data that
we received.
The study investigates the relationship
between message variant and attendance rate at
specific recruitment events. These events took
place between May and December 2013 and
attendance was measured based on a binary
measure of attendance indicating whether the
jobseeker showed up at any time during the
recruitment session to which they were invited.
Data were also returned on the success rate for
delivery of messages, although we do not know
whether a message that is delivered is ever opened.
As the sessions were for specific jobs, it was
possible to identify suitable participants using
information in the Jobcentre Plus database on their
skills and suitability for the type of work (as
denoted by their SOC code). Jobseekers identified
in this way were then texted in the order that their
records were returned by the system. The
maximum number of text recipients per
recruitment event was determined based on
Jobcentre capacity, staff availability and an
expected attendance rate. The upshot is that staff
are available beyond likely demand, meaning they
would be able to handle the recruitment even if 50
percent of those texted were to attend.
Owing to resource constraints, the same
Jobcentre Plus staff member was responsible for
Table 1
Message Variants Used in the Experiment
Treatment
name
Message text (formulation, and example with additive changes in bold)
Control
Formulation: [number] new [type of job] are now available at [company]. Come to
Bedford Jobcentre on [date and time] and ask for [staff member name] to find out
more.
Example: 8 new Picker Packer jobs are now available at Pro FS. Come to Bedford
Jobcentre on Monday 10 June between 10am and 4pm and ask for Sarah to find out more.
+ Claimant
name
Formulation: Hi [jobseeker name]. [number] new [type of job] are now available at
[company]. Come to Bedford Jobcentre on [date and time] and ask for [staff member
name] to find out more.
Example: Hi Elspeth, 8 new Picker Packer jobs are now available at Pro FS. Come to
Bedford Jobcentre on Monday 10 June between 10am and 4pm and ask for Sarah to find
out more.
+Advisor
name
Formulation: Hi [jobseeker name]. [number] new [type of job] are now available at
[company]. Come to Bedford Jobcentre on [date and time] and ask for [staff member
name] to find out more. [Advisor name]
Example: Hi Elspeth, 8 new Picker Packer jobs are now available at Pro FS. Come to
Bedford Jobcentre on Monday 10 June between 10am and 4pm and ask for Sarah to find
out more. Michael
+ Reciprocity
& Luck
Formulation: Hi [jobseeker name]. [number] new [type of job] are now available at
[company]. Come to Bedford Jobcentre on [date and time] and ask for [staff member
name] to find out more. I’ve booked you a place. Good luck, [Advisor name]
Example
:
Hi Elspeth, 8 new Picker Packer jobs are now available at Pro FS. Come to
Bedford Jobcentre on Monday 10 June between 10am and 4pm and ask for Sarah to find
out more. I’ve booked you a place. Good luck, Michael
Journal of Behavioral Public Administration, 2(1)
5
sending the text messages and running the
recruitment sessions. The procedure for ensuring
the staff member remained blind to the treatment
is documented in the Appendix.
Results
Data were analyzed in Stata version 12. Our data
contain 1,224 observations of participants who
were eligible to receive text messages as part of this
trial. For each participant we observe their
treatment assignment, the name of their Jobcentre
advisor, and the details of the job they are invited
to apply for, including the job title, the employer,
and the number of posts available. For variables
relating to the job role and their advisor, we
conduct Chi^2 tests of balance, and find no
significant imbalance between these variables and
treatment assignment for job title (p=0.142), the
employer (p=0.22), or advisor name (p=0.77).
There is significant imbalance on the number of
jobs available, driven by one role advertised to 3
participants only with 200 potential jobs available.
As well as the text message they were sent, we
observe whether they received the text message (i.e.,
whether the phone number held by DWP for them
is correct). In total, roughly 25 percent of the text
messages (307 participants) failed to be received.
The rate of message failure between conditions is
reported in column 5 of table 2. One condition
(Advisor name) has significantly lower failure than
the control, but the other arms are not significantly
different and do not differ significantly from each-
other in their failure rate. Finally, our data contain
information about the session to which they were
assigned, and whether they attended that session.
Table 2 below reports the results of a set of
regression analyses. Column 1 shows an OLS
(linear probability) model regressing attendance on
treatment variables, with control the omitted
category. Column 2 presents marginal effects from
a logistic regression with the same specification.
Column 3 shows a two staged OLS regression, with
the first stage regressing receipt of messages on
treatment variables and the second stage regressing
attendance on predictions from first stage. Column
4 replicates column 1, restricting the sample only to
participants who did not receive messages, as a
manipulation check.
Column 1 in the table shows Intention to
Treat (ITT) estimates for the full sample, that is, for
all participants whom the Jobcentre attempted to
reach in each treatment group. We identify a strong
positive impact on attendance in the final,
Table 2
Regression Estimates of Treatment Effects
Analysis
(1)
(2)
(3)
(4)
(5)
Attendance
(OLS ITT)
Attendance
(Logistic
ITT)
Attendance
(2SLS)
Attendance
(DACE)
Failure
(Exogeneity
Check)
+ Claimant Name
0.043
0.059
0.055
0.053
-0.069
(0.032)
(0.041)
(0.040)
(0.059)
(0.036)
+ Advisor Name
0.069*
0.090*
0.089*
0.020
-0.070*
(0.031)
(0.040)
(0.039)
(0.057)
(0.035)
+ Reciprocity & Luck
0.163***
0.188***
0.220***
0.042
-0.035
(0.030)
(0.41)
(0.041)
(0.054)
(0.035)
Constant
0.105***
0.117***
0.105***
0.119**
0.295***
(0.022)
(0.022)
(0.022)
(0.039)
(0.026)
Notes: * p < .05, ** p < .01, *** p < .001 (two-tailed test). Standard Deviations appear in the
parentheses below the means.
N = 1224 (307 for column (4))
Sanders & Kirkman, 2019
6
Reciprocity & Luck, treatment. Table 3 reports
average levels of attendance by treatment, and 95
percent confidence intervals around each.
Reciprocity & Luck performs significantly better
than any of the other treatments. This finding is
replicated in Table 2 using a logistic regression,
shown in column 2 (marginal effects are presented).
Results are not sensitive to excluding the role with
200 potential jobs described above.
Column 3 in Table 2 reports the results of a
two-stage-least squares (2SLS) instrumental
variables approach in which assignment to
conditions is used as an instrument for whether
participants received that message. As might have
been predicted, this serves to enhance the size of
the effect, to a 22 percentage point increase in
attendance in the most successful treatment group.
One potential concern is that there may have
been imbalance in randomization by chance, and so
participants assigned to our treatments may have
been more likely to attend than those in the control
group even in the absence of treatment. To test this,
in column (4) we conduct a “Defier Average Causal
Effect” (DACE) model, which estimates whether
treatment assignment had any effect on those
individuals who did not receive a text message,
owing to delivery failure. Since these individuals
were not, in fact, treated, we should expect no
significant impact on this group if randomization
has been successful. We find that this is the case,
with no significant impact of treatment assignment
on outcomes (attendance) for this untreated group.
Although treated groups are more likely to respond
than untreated groups, this effect is highly
insignificant (p>0.371), and the ordering of these
differences is not the same as among the ITT or
2SLS analyses. We are therefore content that
randomization has been successful and that the
differences observed between groups is a result of
our treatments.
Finally, column (5) contains a further balance
check to determine the success of randomization.
In this specification, the dependent variable is
whether a text message failed to be received—that
is, whether treatment assignment (which should be
exogenously applied) predicts failure to receive the
text message. Here, we find that all treatment
groups are slightly less likely to fail to receive a text
than the control group. For all but treatment 2 this
difference is insignificant. In treatment 2, the
difference is significant (p=0.046), however, the
effect is not sufficient to lead us to question our
primary results.
Discussion
It is worth being clear that the additive design of
our treatments does not allow us to conclusively
determine the most effective mechanism for
boosting attendance. In other words, we cannot be
sure whether the Reciprocity + Luck message
would be most effective in the absence of bundling
this language with that of the other treatments.
Additionally, our conclusions about impact are
limited since we were not able to obtain data on
whether those who attended events were hired and,
most importantly, whether it is likely those who
attended because of our new messages were hired.
Nonetheless, the findings do show that
varying the message sent to jobseekers influences
their decision on whether to engage with a public
service, and hence could be of broader relevance.
One way to assess whether this finding is
meaningful is to compare it to the effect of text
messages in comparable domains. Whilst many
other experiments use the medium of text message
Table 3
Average Attendance and Confidence Intervals by Treatment
Regression Estimates
Average
Attendance Rate
Lower Bound CI
Upper Bound CI
Control
10.5%
6.9%
14.1%
+ Claimant Name
14.8%
10.8%
19.0%
+ Advisor Name
17.4%
13.2%
21.6%
+ Reciprocity & Luck
26.8%
22.2%
31.5%
Journal of Behavioral Public Administration, 2(1)
7
to affect action, such as attendance, we have not
encountered studies that directly foreshadow what
we investigate in this paper. Our study was
designed to be powered to detect small effect sizes.
Following Cohen (1988), the minimum detectable
effect size in our study is a Cohen’s h of 0.22. The
results suggest that the effects of the two
personalization treatments are similar in absolute
and standardized size to those found in other text
message studies, in particular Karlan et al. (2012),
and Haynes et al. (2013). This implies a degree of
generalizability, which could mean this approach
would work equally well for other employment-
related activities.
The effect of the Reciprocity & Luck
condition is somewhat larger than other effects
observed in the literature. We see two likely
explanations for this: first, this may relate to the
difference in outcomes - in our case, attendance,
while in Haynes et al. and Karlan et al. the
probability of payment. Second, the treatment
effectively bundles several mechanisms beyond
those tested in other conditions. We also believe
that further study to separate out elements of the
mechanism, for example, testing the effect of the
line “I’ve booked you a place, good luck!” without
personalization, or disentangling the elements of
reciprocity and luck, could be fruitful to ensure
maximum effect is being achieved.
Beyond the mechanics of the study, we must
pause to consider generalizability of the findings.
First, this study was conducted during a recession,
meaning that labour market conditions were not
usual. Although this is likely to have an impact on
the types of job search that are most fruitful, as well
as the expected probability of getting a job, we
believe that low cost methods to connect claimants
to opportunities are likely to yield benefits (with the
goal of shortening spells of unemployment) in any
market. Second, we see applications for this
approach in a range of other domains. From market
failures (Madrian (2014)), to poverty alleviation
(Bertrand & Mullainathan (2004)), the use of
behavioural tweaks to improve the impact of
government intervention is well documented.
While SMS reminders are just one way to intervene,
they have exciting potential to address the real
factors that weigh into decision making at the
moment a decision is being made. Indeed, beyond
the research we discuss previously, there is a body
of evidence demonstrating the positive impact of
SMS messages across the policy spectrum, from
increasing voter turnout (see Dale & Strauss (2009)),
to smoking cessation (see Free et al. (2011)), to
educational attainment (see Bergman (2015).
Conclusion
The objective of this study was to investigate the
effects of variations on a text message on
attendance rates. Our results show that the
"reciprocity & luck condition" (T3: "... I've booked
you a place. Good luck..."), which includes the
language of the other treatments, is significantly
more effective than any other message in attracting
participants, leading to an attendance rate of 26.8
percent compared to 10.5 percent in the control.
Although the effect is not of the same magnitude,
the authors also find that using the Jobseeker’s first
name and the name of their Advisor is powerful in
prompting attendance. These findings are in line
with our initial hypotheses.
References
Alpizar, F., Carlsson, F., & Johansson-Stenman, O.
(2008). Anonymity, reciprocity, and conformity:
Evidence from voluntary contributions to a national
park in Costa Rica. Journal of Public Economics, 92,
1047–1060.
Bergman, P. (2015). Parent-child information frictions
and human capital investment: Evidence from a field
experiment. Working Paper Available at
https://ssrn.com/abstract=2622034
Bertrand, M., Mullainathan, S., & Shafir, E. (2004). A
behavioral-economics view of poverty. American
Economic Review, 94(2), 419-423.
Bryan, C. J., Walton, G. M., Rogers, T., & Dweck, C. S.
(2011). Motivating voter turnout by invoking the
self. Proceedings of the National Academy of Sciences, 108,
12653–12656.
Caliendo, M., Cobb-Clark, D. A., & Uhlendorff, A.
(2015). Locus of control and job search
strategies. Review of Economics and Statistics, 97(1), 88-
103.
Dale, A., & Strauss, A. (2009). Don't forget to vote: text
message reminders as a mobilization tool. American
Journal of Political Science, 53(4), 787-804.
Sanders & Kirkman, 2019
8
Falk, A. (2007). Gift Exchange in the Field. Econometrics
75, 1501–1511
Falk, A., & Fischbacher, U. (2006). A theory of
reciprocity. Games and Economic Behavior, 54, 293–315
Free, C., et al. (2011). Smoking cessation support
delivered via mobile phone text messaging (txt2stop):
a single-blind, randomised trial. The Lancet, 378(9785),
49-55.
Gleerup, M. (2010), et al. The Effect of Feedback by
Text Message (SMS) and Email on Household
Electricity Consumption: Experimental Evidence.
Energy Journal 31.3
Gregg, P. (2001). The impact of youth unemployment
on adult unemployment in the NCDS. The economic
journal, 111, 626–653.
Haynes, L. C., Green, D. P., Gallagher, R., John, P., &
Torgerson, D. J. (2013). Collection of Delinquent
Fines: An Adaptive Randomized Trial to Assess the
Effectiveness of Alternative Text Messages. Journal of
Policy Analysis and Management, 32, 718–730
Madrian, B. C. (2014). Applying insights from behavioral
economics to policy design. Annu. Rev. Econ., 6(1),
663-688.
Macmillan, L. (2011). Measuring the intergenerational
correlation of worklessness. Centre for Market and
Public Organisation.
Meyer, Bruce D, (1990). Unemployment Insurance and
Unemployment Spells. Econometrica, Econometric Society,
vol. 58(4), pages 757-782, July.
Karlan, D., Morten, M., & Zinman, J. (2012). A personal
touch: Text messaging for loan repayment (No. w17952).
National Bureau of Economic Research.
Pratkanis, A. R., & Gliner, M. D. (2004). And when shall
a little child lead them? Evidence for an altercasting
theory of source credibility. Current Psychology, 23, 279–
304
Pratkanis, A. R. (2007). Social influence analysis: An
index of tactics. The science of social influence:
Advances and future progress, 17–82
Sagone, E., & De Caroli, M. E. (2014). Locus of control
and beliefs about superstition and luck in adolescents:
What's their relationship?. Procedia-Social and Behavioral
Sciences, 140, 318-323.
Team, B. I. (2013). Applying behavioural insights to
charitable giving. London: Cabinet Office
Hogg, M. A., & Terry, D. I. (2000). Social identity and
self-categorization processes in organizational
contexts. Academy of management review, 25(1), 121-140.
Saks, A. M. (2006). Multiple predictors and criteria of job
search success. Journal of Vocational Behavior, 68(3),
400-415.
Tiggemann, M., & Winefield, A. H. (1984). The effects
of unemployment on the mood, self‐esteem, locus of
control, and depressive affect of school‐leavers.
Journal of occupational Psychology, 57, 33–42
Weinstein, E. A., & Deutschberger, P. (1963). Some
dimensions of altercasting. Sociometry, 454–466
Appendix
Appendix A. Detail on Blinded Messaging Procedure
When a recruitment is initiated the following procedure is put in place:
1. Once participants are identified using SOC codes, their information (first name, last name, advisor name,
phone number, national insurance number) is entered into a data template designed by the researchers.
2. The Jobcentre Plus staff member responsible for data entry then runs a code written by the researchers (by
pressing a button), which handles the randomization and data preparation as follows:
i. Assigns each participant to one of the four trials arms at random.
ii. Uses a conditional message formula to generate the text to be sent based on personal details and trial
arm assignment.
iii. Replaces the National Insurance number with a unique identifier (so that the information can be shared
with the research team without the need for personal data sharing).
iv. Creates two visible sheets: one with the text message and phone numbers only (plus instructions on
the day and time the messages should be set to send at, to control for variations between sessions),
which can be pasted into the text messaging machine; one with the names of all participants, their
unique ID number and blank columns to record attendance plus any comments.
v. Creates a hidden (password protected) sheet that only the research team can access, which contains
unique identification numbers plus the text condition for each participant.
Journal of Behavioral Public Administration, 2(1)
9
The text messages are then sent using the first of the sheets and the attendance of participants is recorded on
the day(s) of the session(s) using the second. At the end of each recruitment episode, the Jobcentre staff send
the attendance sheet and the hidden sheet (which is contained in the same document and remains uneditable
without the password) to the researchers along with a delivery report for all SMS text messages. The hidden
data is decrypted and the unique identifiers are then used to match attendance to text condition. As an assurance
exercise, the delivery records are checked to ensure that failure is consistent across conditions (indicating that
the randomization has been performed effectively) and that a sufficient number of texts were delivered.
