Effects of Job Development and Job Support on Competitive Employment of Persons With Severe Mental Illness

Article (PDF Available)inPsychiatric Services 56(10):1237-44 · November 2005with72 Reads
DOI: 10.1176/appi.ps.56.10.1237 · Source: PubMed
Abstract
Few studies have sought to determine which specific supported employment services improve employment outcomes for people with pyschiatric disabilities. This study examined the effects of job development and job support among other services on acquisition and retention of competitive employment. Data used in the analysis came from seven sites of the Employment Intervention Demonstration Program. Employment data were collected weekly for a period up to 24 months for 1,340 participants. A random-effects meta-analysis was conducted. Job development increased the probability of obtaining competitive employment. The effects of job development on job acquisition remained after the effects of other factors were controlled for. Job support was associated with more months in the first competitive job but not total hours worked. However, no evidence for the causal role of job support was found in analyses that tested the effects of job support after the job support was provided. The causal role of job support alone was also cast in doubt by the fact that a substantial overlap existed between individuals who received job support and vocational counseling. Job development is a very effective service when the goal is job acquisition. Job support is associated with retention of a first competitive job, but its causal role is questionable.

Full-text (PDF)

Available from: Paul B Gold, May 08, 2014
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11223377
A
lthough the principles, operat-
ing practices, and outcomes of
employment interventions for
persons with severe mental illness
have been investigated (1,2), few
studies have explored direct effects of
specific program services on employ-
ment outcomes. Services are groups
of activities that have one or more
common goals. Job development is
one example of a service. Principles
for employment programs have been
identified, and the evidence of their
effectiveness has been reviewed
(2–4). Principles focus on the manner
in which services are provided.
Stronger empirical evidence exists for
some principles—a focus on compet-
itive employment, eligibility based on
client interest, and rapid job search—
than others, such as integration of vo-
cational and clinical services, atten-
tion to consumer preferences, time-
unlimited and individualized support,
and benefits counseling (2–4).
Several recent studies and reviews
have highlighted the effectiveness of
supported employment services for
people with severe mental illness
(1,2,5–9). Studies have shown that
supported employment helps con-
sumers with severe mental illness ob-
tain competitive employment (2) and
Effects of Job Development and Job
Support on Competitive Employment
of Persons With Severe Mental Illness
HH.. SStteepphheenn LLeeffff,, PPhh..DD..
JJuuddiitthh AA.. CCooookk,, PPhh..DD..
PPaauull BB.. GGoolldd,, PPhh..DD..
MMaarrcciiaa TToopprraacc,, PPhh..DD..
CCrryyssttaall BBl
lyylleerr,, PPhh..DD..
RRiicchhaarrdd WW.. GGoollddbbeerrgg,, PPhh..DD..
WWiilllliiaamm MMccFFaarrllaannee,, MM..DD..
MMiicchhaaeell SShhaaffeerr,, PPhh..DD..
II.. EEllaaiinnee A
Alllleenn,, PPhh..DD..
TTeerreessiittaa CCaammaacchhoo--GGoonnssaallvveess,, PPhh..DD..
BBaarrbbaarraa RRaaaabb,, BB..AA..
Dr. Leff, Dr. Camacho-Gonsalves, and Ms. Raab are affiliated with the Human Services
Research Institute, 2269 Massachusetts Avenue, Cambridge, Massachusetts 02140 (e-mail,
sleff@hsri.org). Dr. Cook is with the mental health science research programs department
at the University of Illinois at Chicago. Dr. Gold is with the department of psychiatry at
the Medical University of South Carolina in Charleston. When this work was done, Dr.
Toprac was affiliated with the Texas Department of Mental Health and Mental Retarda-
tion in Austin. She is currently a private consultant. Dr. Blyler is with the Center for Men-
tal Health Services in Rockville, Maryland. Dr. Goldberg is with the department of psy-
chiatry at the University of Maryland in Baltimore. Dr. McFarlane is with the Center for
Psychiatric Research of the Maine Medical Center in Portland. Dr. Shafer is with the com-
munity rehabilitation program at the University of Arizona Health Sciences Center in Tuc-
son. Dr. Allen is with Babson College in Babson Park, Massachusetts.
Objectives: Few studies have sought to determine which specific support-
ed employment services improve employment outcomes for people with
pyschiatric disabilities. This study examined the effects of job develop-
ment and job support among other services on acquisition and retention
of competitive employment. Methods: Data used in the analysis came
from seven sites of the Employment Intervention Demonstration Pro-
gram. Employment data were collected weekly for a period up to 24
months for 1,340 participants. A random-effects meta-analysis was con-
ducted. Results:
Job development increased the probability of obtaining
competitive employment. The effects of job development on job acquisi-
tion remained after the effects of other factors were controlled for. Job
support was associated with more months in the first competitive job but
not total hours worked. However, no evidence for the causal role of job
support was found in analyses that tested the effects of job support after
the job support was provided. The causal role of job support alone was also
cast in doubt by the fact that a substantial overlap existed between indi-
viduals who received job support and vocational counseling. Conclusions:
Job development is a very effective service when the goal is job acquisi-
tion. Job support is associated with retention of a first competitive job, but
its causal role is questionable. (Psychiatric Services 56:1237–1244, 2005)
lef.qxd 9/21/2005 1:18 PM Page 1237
Advertisement:
that higher fidelity to supported em-
ployment is positively correlated with
competitive employment rates (10)
and other employment outcomes
(3,4). However, studies have shown
that job tenure is often short (11).
One formulation of supported em-
ployment is individual placement and
support (12), which has received the
most systematic research (2). Howev-
er, the principles and operating prac-
tices of supported employment, like
those of other employment interven-
tions, are embodied in a number of
different types of program and deliv-
ery mechanisms. Less is known about
the effectiveness of specific services.
Thus employment programs have
been referred to as “black boxes”
(13). The possibility remains that cer-
tain employment services work bet-
ter than others, independent of pro-
gram context.
Programs such as employment in-
terventions, which provide many dif-
ferent vocational and nonvocational
services, have “complex arrange-
ments and soft boundaries” (14). One
question about replicating socially
complex interventions generally and
evidence-based ones specifically that
has yet to be answered for most inter-
ventions is whether all the services
and complex arrangements currently
included in the interventions are nec-
essary to achieve desired outcomes.
To better understand this issue, we
examined data from a two-year, sev-
en-state demonstration program, the
Employment Intervention Demon-
stration Program (EIDP) (15), which
was funded by the Substance Abuse
and Mental Health Services Adminis-
tration. Specifically, we examined
types and amounts of services re-
ceived in relation to the achievement
and retention of competitive employ-
ment (termed “dismantling” re-
search) (16). We implemented dis-
mantling analyses to increase our un-
derstanding of factors that improve
employment outcomes and to identi-
fy more cost-effective and less de-
manding approaches to achieving
these outcomes.
The study reported here is unique
in that it enabled us to look at specif-
ic services across multiple sites and
program models. This report focuses
on two employment services hypoth-
esized by the EIDP investigators to
be particularly effective in obtaining
and retaining a competitive job—job
development and on-site job support,
respectively. We focused on competi-
tive jobs because they best reflect re-
covery and community integration,
two objectives of community care for
persons with severe mental illness
(17). Our study differs from other
studies of the EIDP that have been
reported in that sites focused on site-
specific primary outcomes that have
no relation to this meta-analytic study
of job development and job support
and their relationship to employment
outcome (5,18,19). Reports from the
EIDP Coordinating Center pool data
for the sites and look at the data from
a program level and not at the level of
specific services (20–22).
The receipt of a service can be
measured in terms of penetration
(whether or not the service is re-
ceived) and amount (how much of
the service is received). We focused
on service penetration, because we
had no theoretical or empirical basis
for distinguishing amounts of service
that were too little, appropriate, or
too much.
We hypothesized that participants
who received job development would
be more likely to acquire competitive
jobs than those who did not receive it.
Another hypothesis was that partici-
pants who received job development
sooner would likely be more prepared
for work and more likely to acquire
competitive jobs than those who re-
ceived it later. With respect to job sup-
port, we hypothesized that partici-
pants who received job support would
work more months and hours than
those who did not receive job support.
Methods
Data used in the analysis came from
seven sites of the EIDP study. The
sites were in Arizona, Connecticut,
Maine, Maryland, Massachusetts,
South Carolina, and Texas. The
sites, conditions, and methods are
described elsewhere (15). Individu-
als were included in the study if they
were 18 years or older at enroll-
ment, were willing and able to pro-
vide informed consent, had a DSM-
IV diagnosis of mental illness, and
were unemployed.
Procedures
Interview assessments with EIDP
participants elicited information
about demographic characteristics,
previous employment, current in-
come, clinical indicators, and other
relevant information at the time of
study enrollment (baseline) and at
six-month follow-up intervals for 24
months. Some data were collected on
a weekly basis, including vocational
data on the nature of the participant’s
job (among those employed), job ti-
tle, wages earned, hours worked, ben-
efits received, and level of workplace
integration. The sites also collected
data on the types of vocational and
clinical services received by EIDP
participants. The EIDP common pro-
tocol and data collection methods are
described elsewhere (23). Sites re-
ceived approval of human subjects’
protection and confidentiality safe-
guards required for recruitment, in-
formed consent, and data manage-
ment procedures from their organiza-
tions’ institutional review boards. Re-
cruitment of study participants took
place between February 1996 and
May 2000, and all participants re-
ceived monetary compensation for
each interview.
Analysis sample
A total of 1,340 persons from the sev-
en sites were included in this analysis.
Persons doing paid work at baseline
(N=28) and those with baseline but no
follow-up employment data (N=98)
were excluded.
Measures
Employment outcome variables. Job
development was defined as direct or
indirect contact with potential em-
ployers or networking with individu-
als or organizations that had job infor-
mation. The outcome variable for the
job acquisition analyses was whether
or not the individual obtained com-
petitive employment. In this study,
the criteria for competitive employ-
ment included four components: pay
at minimum wage or higher, a job lo-
cated in a mainstream integrated set-
ting, a job that was not set aside for
mental health consumers, and a job
that is held independently (that is, not
controlled by a service agency). The
first two criteria match the Depart-
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11223388
lef.qxd 9/21/2005 1:18 PM Page 1238
ment of Labor’s definition of compet-
itive employment (24), and the sec-
ond two were developed to differenti-
ate competitive employment from
transitional employment placements
or enclave jobs. Under this definition,
transitional employment placements,
such as those provided through Inter-
national Center for Clubhouse De-
velopment (ICCD) clubhouses, were
not categorized as competitive em-
ployment. The final report for the
Massachusetts site of the EIDP pro-
vides for further details of the site-
specific analysis that included such
transitional placements (25). For
each month in the study, EIDP par-
ticipants were categorized as having
obtained a competitive job (coded as
1) or not (coded as 0). Jobs were ex-
cluded if they were temporary and
lasted less than five days. Participants
were dropped from the analysis for
the study months after they obtained
a competitive job.
Job support was defined as on-site
counseling, support, and problem
solving. Three outcome variables
were selected for the job retention
analyses. First, total duration of the
first competitive job in months was
calculated by counting the number of
days from the start date to the end
date and dividing by 30.4 (the average
number of days in a month). Second,
total hours worked in the first com-
petitive job was calculated. Third,
whether the participant retained the
job for an entire month was docu-
mented. For each month, partici-
pants were categorized as being still
employed at the end of the month
(coded as 1) or not (coded as 0). A to-
tal of 503 jobs were included in the
analyses. This sample excluded jobs
that began less than nine months be-
fore data tracking ended (N=72) to
maximize the number of jobs ana-
lyzed but to exclude arbitrarily trun-
cated jobs.
Vocational services variables. In
addition to job development and job
support, eight other types of voca-
tional services were tracked: voca-
tional assessment and evaluation;
client-specific collaboration with an
employer; vocational support groups;
collaboration with family or friends;
vocational treatment planning and ca-
reer development; off-site skills train-
ing or education; off-site vocational
counseling; and transportation. Defi-
nitions for these other services are
available elsewhere (26).
Demographic and clinical vari-
ables. Because participants were not
randomly assigned to services, we
needed to explore the possibility of
selection bias and potential con-
founders. Demographic and clinical
variables explored to control for se-
lection bias were gender, race, previ-
ous work experience, a diagnosis of
schizophrenia, a diagnosis of a sub-
stance use disorder, education, mari-
tal status, children living in the same
household, Supplemental Security
Income (SSI) and Social Security
Disability Income (SSDI) status,
work motivation (measured by a scale
created by the EIDP steering com-
mittee), Positive and Negative Syn-
drome Scales (PANSS) scores (27) (a
measure of psychiatric symptoms),
and hours in nonvocational services.
These demographic and clinical vari-
ables were included on the basis of a
review of the literature by EIDP in-
vestigators to identify characteristics
of participants that might be associat-
ed with vocational outcomes. Other
potential covariates were excluded
from the EIDP study because of their
high correlations with other covari-
ates or lack of variance across the
sample. To identify potentially biasing
demographic and clinical variables,
we conducted regression analyses to
determine which variables were asso-
ciated with both service group mem-
bership and outcomes.
We postulated two potential con-
founders of service effects. One was
the receipt of vocational services in
addition to the services studied (mul-
tiple vocational services). A second
was the degree to which programs in-
tegrated vocational and clinical serv-
ices. The EIDP Level of Integration
Scale, which ranged from 0 to 100,
was used, and programs that scored
above 50 were categorized as having
integrated services; those that scored
below 50 were categorized as having
nonintegrated services. Further in-
formation about the EIDP Level of
Integration Scale is available in an-
other report of the EIDP project
(15). Programs that were more inte-
grated were similar to supported em-
ployment programs. We implement-
ed analyses to test whether these vari-
ables altered findings for job develop-
ment and job support.
Data analyses
Random-effects meta-analyses were
fit to the data over multiple sites.
Random-effects models consider
within- and between-site variability
and adjust the effect sizes to account
for sample size and site variability.
This method makes detecting statisti-
cal significance harder if there are be-
tween-site variations. The meta-
analysis of multisite trials is well es-
tablished in clinical research (28,29)
and adds clarity to other analyses of
the EIDP data (5,18–22). The analy-
ses provide a synthesis of the results
over all sites controlling for and clear-
ly showing the effects of site and se-
lectively testing the importance of
differences in participant mix to give
overall effect sizes for the entire
study. Cochran’s Q statistic was used
to examine variability between sites; a
significant Q rejects the null hypothe-
sis of homogeneity and indicates that
variability among the effect sizes is
greater than what would be expected
from subject-level sampling error
alone (30). All our analyses showed
consistency between sites.
Effect sizes for the job acquisition
and job retention variables were cal-
culated by using Comprehensive
Meta-Analysis (31) statistical soft-
ware. All hypotheses were two-tailed,
and the standard p value of p<.05 was
used for rejection of the null hypothe-
sis. Because participants were not ran-
domly assigned to services, we con-
structed multiple groups to test the
possibility that selection and contextu-
al effects were responsible for our
findings. Thus we refer to our disman-
tling data analytic approach as “con-
structed comparisons” meta-analysis.
In this method successive groups are
constructed to investigate potentially
confounding variables. However,
there is no way to completely rule out
the possibility of selection bias or po-
tential confounders that are attributa-
ble to unobserved characteristics.
Relative probability of acquisition
and retention of a job. To examine ef-
fects of job development and job sup-
port on acquisition and retention of
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11223399
lef.qxd 9/21/2005 1:18 PM Page 1239
competitive employment, the meta-
analyses we conducted compared the
relative probability of job acquisition
or retention given the participant’s
previous receipt of job development
or job support. The relative probabil-
ity of obtaining or retaining a job was
defined as the ratio of the percentage
of participants who received a service
and who had a desired outcome to
those who did not receive the service
but who had a desired outcome.
Methods for analyzing data for job
development and job support are de-
scribed separately below.
Job development and overall ef-
fects. To rule out “reverse causali-
ty”—for example, participants who al-
ready had jobs received job develop-
ment services for career advance-
ment—we carried out analyses in
which job acquisition was analyzed as
a function of previous receipt of job
development. We postulated that the
effects of job development on job ac-
quisition would be seen in the month
that the service was received or the
month immediately afterward, which
allowed us to analyze data for individ-
ual months and for multimonth peri-
ods. Because individuals who did not
acquire a job would appear more than
once in multimonth analyses, we
computed weighted monthly aver-
ages for persons who obtained com-
petitive employment and who did and
did not receive job development for
multiple months. The inputs into the
meta-analyses were monthly average
employment rates for participants
who received and did not receive job
development in that month or the
previous month (weighted by total N
for the month) and average monthly
number of participants who did and
did not receive job development.
Job development and effects of
time. Because one of our hypotheses
about the effects of job development
involved time, time periods were con-
structed on the basis of the number of
months a participant was in the pro-
gram. The months were divided into
three groups: months 1 to 6, months 7
to 12, and months 13 to 24. Time pe-
riods were chosen in order to have
enough participants in each period.
Separate meta-analyses examined
outcomes for each period and across
all months.
Job development and effects of mul-
tiple services. To examine the poten-
tially confounding effects of multiple
vocational services, a meta-analysis
was conducted to examine whether
effects found for job development
were maintained when participants
received multiple services.
Other unplanned analyses were
conducted to compare the relative
probability of obtaining a job while
examining the effect of other specific
vocational services and job develop-
ment on job acquisition. Separate
analyses compared participants who
received only job development serv-
ices in a given month with those who
received only the comparison voca-
tional service.
Job development and effects of pro-
gram context. To examine the poten-
tially confounding effects of integrat-
ed services, meta-analyses were con-
ducted to determine whether the ef-
fects found for job development were
maintained when participants who
received job development in the con-
text of integrated services were com-
pared with those who received inte-
grated services but not job develop-
ment. Analyses were conducted to in-
vestigate the role of job development
in programs that were not integrated.
Job support and overall effects. The
overall effects of job support were ex-
amined by meta-analyses over sites.
The inputs into the meta-analyses for
total duration of employment and to-
tal hours were the overall mean and
the number of participants who did or
did not receive job support during
their first competitive job. The inputs
for the job retention variable were the
rates of participants who were still
employed at the end of the month for
those who did or did not receive job
support in that month or in the previ-
ous month (weighted by total number
of participants for the month) and the
average monthly number of partici-
pants who received and did not re-
ceive job support.
To again rule out reverse causali-
ty—the possibility that people with
longer-lasting jobs would have a high-
er probability of receiving unneeded
job support, which would result in an
association between the two vari-
ables—we carried out analyses in
which job retention was analyzed as a
function of previous receipt of job
support. We postulated that the ef-
fects of job support on job retention
would be seen in the month the sup-
port was received or the month after-
ward, which again allowed us to ana-
lyze data for individual months and
for multimonth periods. Because re-
tention of a job would appear more
than once in multimonth analyses, we
computed weighted monthly aver-
ages for participants who retained
competitive employment and did or
did not receive job support for multi-
ple months.
Additional analyses to explore po-
tential selection or confounder vari-
ables were not conducted because no
significant difference was found in
the relative probability of retaining
competitive employment after the re-
ceipt of job support.
Job support and effects of program
context. The effects of program con-
text could not be tested in the job
support analyses, because the num-
ber of people who received job sup-
port in programs with nonintegrated
services was low. Of the 103 jobs held
by participants in programs with non-
integrated services, job support was
provided for only 15.
Results
Baseline characteristics
The mean±SD age of the 1,340 par-
ticipants in the study was 38.4±9.3
years, and 716 participants (53 per-
cent) were men. Less than half the
participants were white (653 partici-
pants, or 49 percent). A total of 408
participants (30 percent) were black,
205 (15 percent) were Hispanic, 35
(3 percent) were American Indian or
Alaskan Native, ten (.7 percent) were
Asian, and 27 (2 percent) were classi-
fied as other. A total of 482 partici-
pants (36 percent) had less than a
high school degree, and 468 (35 per-
cent) had no work experience in the
five years before they entered the
EIDP program. A total of 1,196 par-
ticipants (89 percent) were not mar-
ried, and 1,013 (76 percent) had no
children under the age of 18 years
living with them. A total of 465 (35
percent) received only SSI, 330 (25
percent) received only SSDI , and
167 (13 percent) received both SSI
and SSDI.
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11224400
lef.qxd 9/21/2005 1:18 PM Page 1240
All participants received a DSM-IV
axis I primary diagnosis. The most
prevalent diagnosis was schizophrenia
(429 participants, or 32 percent), fol-
lowed by major depression (265 par-
ticipants, or 20 percent), schizoaffec-
tive disorder (247 participants, or 18
percent), bipolar disorder (213 partic-
ipants, or 17 percent), psychotic disor-
der (44 participants, or 3 percent), and
other (95 participants, or 7 percent).
More than a third of the participants
(498 participants, or 37 percent) had a
primary or secondary diagnosis of a
substance use disorder. The mean
PANSS scores were 14.4±5.1 on the
positive subscale, 16.3±5.5 on the neg-
ative subscale, and 34.3±8.8 on the
general subscale (31).
Employment services received
Job development was the service most
used by EIDP participants (840 par-
ticipants, or 58 percent). On-site job
support was one of the least used em-
ployment services (423 participants,
or 29 percent). More than half of the
participants (812 participants, or 56
percent) received vocational assess-
ment and evaluation and off-site job
skills training and education (747 par-
ticipants, or 52 percent). Half of the
participants (726 participants, or 50
percent) received vocational treat-
ment planning or career develop-
ment. Fewer than half of the partici-
pants (707 participants, or 49 percent)
received off-site vocational counsel-
ing, collaboration with family and
friends (558 participants, or 39 per-
cent), collaboration with employers
(463 participants, or 32 percent), and
vocational support groups (439 partic-
ipants, or 30 percent). Transportation
was the employment service least
used (410 participants, or 28 percent).
Job development
Competitive jobs obtained. Of the
1,340 participants, 575 (43 percent)
obtained one or more competitive
jobs. A total of 362 of the 575 partici-
pants (63 percent) obtained only one
competitive job.
Effects of job development on job
acquisition. Table 1 shows the relative
probability of obtaining competitive
employment in an average month on
the basis of receipt of job develop-
ment. The table presents the average
number of participants who received
job development per month at each
site, the average weighted monthly
percentage of those who received job
development and obtained a job, the
average number of participants who
did not receive job development per
month at each site, the average
weighted monthly percentage of those
who did not receive job development
and obtained a job, the random effect
for each individual site, and the over-
all effect for all sites when the ran-
dom-effects model was used.
As shown in Table 1, the relative
probability of obtaining competitive
employment was significantly greater
for participants who received job de-
velopment. This finding was consis-
tent within and between sites, and the
overall effect across sites was also sig-
nificant. The effect was consistent at
all sites and statistically significant at
two sites. This pattern of findings
shows the benefit of multisite studies
when samples at individual sites are
small. Individuals who received job
development were almost five times
more likely to obtain competitive em-
ployment than individuals who did
not receive job development.
Effect of time period. Table 2 shows
that effect sizes were similar for dif-
ferent time periods.
Effects of demographic and clinical
variables. Having previous employ-
ment was the only variable signifi-
cantly associated with ever obtaining
a competitive job and ever receiving
job development (β=1.79, p<.001
and β=1.34, p=.02, respectively).
Thus we implemented subgroup
analyses only for this variable. Re-
sults in Table 2 show that regardless
of work experience, receipt of job de-
velopment increased the likelihood
of obtaining competitive employ-
ment and that individuals with no
previous work experience had virtu-
ally no chance of acquiring a compet-
itive job without job development.
Effects of multiple services. Table
2 also shows that the effects found
for job development were main-
tained for participants who received
multiple services but were not statis-
tically significant.
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11224411
TTaabbllee 11
Acquisition of competitive employment in an average month among 1,340 participants at seven sites of the Employment In-
tervention Demonstration Program, by whether or not the participant received job development services
Received job development Did not receive job development
Average Average
weighted weighted
Average monthly % Average monthly %
number who obtained number who obtained
Site per month employment per month employment Effect CI p
1 20 11.6 184 .8 9.20 1.37–61.81 .01
2 26 9.3 30 1.8 2.31 .22–24.01 .47
3 22 5.4 124 .6 5.64 .37–86.81 .16
4 22 10.95 111 .8 10.09 .96–106.50 .02
5 26 6.7 76 2.1 2.92 .43–19.71 .25
6 11 10.3 89 1.4 8.09 .54–120.40 .07
7 37 10.6 67 3.7 3.62 .70–8.84 .10
Combined 4.90 2.20–10.94 <.001
lef.qxd 9/21/2005 1:18 PM Page 1241
Results not shown here indicated
that participants who received only
job development had a greater likeli-
hood of obtaining competitive em-
ployment than those who received
only a comparison vocational service,
although none of these differences
were statistically significant. It is pos-
sible that the lack of significant find-
ings is attributable to the small num-
bers of participants who received only
one service.
Effects of program context. Table 2
also shows that job development
helped participants obtain competi-
tive employment in both integrated
and nonintegrated services. The ef-
fect was stronger in nonintegrated
services than in integrated services,
because there was such a small likeli-
hood of getting a competitive job
without job development when serv-
ices were not integrated. In addition,
provision of job development is rare
in nonintegrated services.
Job support
Effects of job support on job reten-
tion. Data presented in Table 3 show
a significant association between
number of months in the first com-
petitive job and receipt of job sup-
port. The association with total hours
worked approached significance.
However, these analyses did not take
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11224422
TTaabbllee 22
Acquisition of competitive employment in an average month among 1,340 participants in the Employment Intervention
Demonstration Program, by receipt of job development and time period, work experience, receipt of other vocational serv-
ices, and program context
Average weighted monthly
% who obtained employment
Variable Average N % Weighted SD OR CI p
Period
Months 1–6 5.71 2.51–13.01 <.001
No job development 895 2.3 12.6
Job development 298 11.6 30.5
Months 7–12 4.48 1.98–10.13 <.001
No job development 699 2.1 11.0
Job development 194 7.80 24.9
Months 13–24 5.94 1.97–17.92 .002
No job development 563 .8 6.2
Job development 84 6.99 17.85
Work experience
(past five years)
Worked 2.52 1.59–4.03 <.001
No job development 372 2.0 10.1
Job development 97 10.1 24.7
Did not work 4.27 1.88–9.71 .001
No job development 267 .96 4.2
Job development 58 7.6 18.2
Received multiple vocational
services 2.06 .71–5.98 .184
No job development 74 2.3 6.9
Job development 125 10.2 26.6
Program context
Integrated services 3.80 1.59–9.23 .003
No job development 307 2.3 9.7
Job development 142 9.5 25.2
Nonintegrated services 14.42 3.52–59.01 <.001
No job development 404 .9 6.1
Job development 24 3.98 7.5
TTaabbllee 33
Weighted mean effect sizes for receipt of job support and job retention among
participants in the Employment Intervention Demonstration Program
Hours or
months worked
Retention and support N Mean SD ES p for ES
a
CI
Total duration of job (months) .31 .02 .04 to .50
No job support 309 3.90 4.51
Job support 194 5.54 5.56
Total hours worked .21 .06 –.01 to .43
No job support 298 345.01 449.69
Job support 184 415.53 555.86
a
Standardized difference between the two means. The pooled standard deviation was used to stan-
dardize the difference.
lef.qxd 9/21/2005 1:18 PM Page 1242
into account whether the job support
was delivered before job retention,
leaving open the possibility that job
retention allowed persons to receive
job support rather than vice-versa.
As shown in Table 4, no significant
difference was found in the relative
probability of retaining versus losing
competitive employment for the en-
tire month after the receipt of job sup-
port. Additional analyses not shown
indicated that a significant overlap ex-
isted between the participants who re-
ceived job support and those who re-
ceived vocational counseling.
Discussion and conclusions
Job acquisition and retention are im-
portant outcomes to consumers (1)
and to policy makers (32). If re-
sources are limited, programs might
decide to focus more on job develop-
ment than on job support. These
nonexperimental data do not estab-
lish such modifications as an evi-
dence-based course of action, and
the resulting services would not be
supported employment. However,
these data do underscore and sup-
port the importance of job develop-
ment for job acquisition.
On average, receipt of job support
was positively correlated with the
number of months and hours worked
in the first competitive job. However,
the analyses that took the time of in-
tervention into account did not rule
out the possibility of reverse causality.
The weaker results for job support go
along with the weaker effects of em-
ployment interventions on job reten-
tion compared with job acquisition
that have been found in other studies
(11). We need to experiment with
services specifically focused on this
admittedly difficult outcome and to
design studies appropriate for this
task. We knew when individuals need-
ed job development, because they had
no job. We had no equivalent measure
of a need for job support. Thus we
were unable to determine the effects
of job support only on persons judged
to need this service.
It is important to point out that al-
though the analyses show that job de-
velopment was effective for the ac-
quisition of competitive employment
for 57 percent of persons who re-
ceived the service, 43 percent of
EIDP participants (N=318) who re-
ceived job development services were
unable to obtain competitive employ-
ment. How these persons should be
served is another important question
to be addressed.
It is important to note other limita-
tions to this study. Participants were
randomly assigned to programs but
not to services. Although we exam-
ined the possible effects of selection
bias and potential confounders, the
possibility of bias and confounders re-
mains. Ideally, randomized studies or
statistical modeling should be con-
ducted to confirm the findings pre-
sented here. Also, more studies are
needed to establish the generalizabil-
ity of our findings to different sub-
groups of people with severe mental
illness and currently available voca-
tional rehabilitation programs. In ad-
dition, the analyses were limited to
services received during the first
competitive job. It is possible that
services received during other com-
petitive jobs may have different ef-
fects on job acquisition and job reten-
tion. Indeed, in the EIDP study, sec-
ond and third competitive jobs lasted
longer on average than first competi-
tive jobs.
It is also important to point out that
the service data were collected differ-
ently at the sites. Some sites collected
the data for the study by using the def-
initions provided by the Coordinating
Center; most others had to cross-ref-
erence service definitions used in
their systems to those provided. These
differences in data collection may well
have affected the results. Future stud-
ies of obtaining competitive employ-
ment could focus on the day from
study entry to the day of beginning the
first competitive job. Future studies
could also conduct higher-order
Markov analysis that would allow de-
termination of whether the accumu-
lated amount of job services is related
to the probability of a person’s con-
verting from never being employed to
obtaining a job for the period that be-
gins at the start of the study and ends
at job acquisition.
This study only begins to unpack
the black box of vocational rehabilita-
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11224433
TTaabbllee 44
Retention of employment in an average month among 1,340 participants at seven sites of the Employment Intervention
Demonstration Program, by whether or not the participant received job support
Received job support Did not receive job support
Average Average
weighted weighted
Average monthly % Average monthly %
number who retained number who retained
Site per month employment per month employment Effect CI p
1 18 78.2 13 81.4 .92 .66–1.29 .63
2 11 82.5 8 88.4 .91 .66–1.25 .52
3 6 77.6 20 77.1 1.11 .65–1.90 .71
4 15 87.6 16 80.5 1.07 .78–1.45 .68
5 4 81.4 22 34.4 .97 .53–1.79 .92
6 2 59.3 18 82.9 .60 .15–2.44 .26
7 5 78.6 51 79.7 .99 .63–1.57 .98
Combined .98 .83–1.14 .76
lef.qxd 9/21/2005 1:18 PM Page 1243
tion programs. The effects of patterns
of services received (as suggested by
cluster analysis) on job acquisition
and retention should be explored.
Our analysis of the confounding ef-
fects of multiple services left this door
ajar. Future studies could group job
support and vocational counseling
into a composite category. In addi-
tion, when the effects of job support
were examined, one site had negative
results compared with the other. This
and other site variation might be ex-
plored to gain a deeper understand-
ing of effective services.
Finally, given the importance of re-
turn-to-work initiatives for persons
with severe mental illness, this study
provides support for prioritizing sup-
ported employment and other pro-
grams that emphasize individual job
development.
Acknowledgments
The research was funded by cooperative
agreement SM51820 from the Center for
Mental Health Services of the Substance
Abuse and Mental Health Services Ad-
ministration.
References
1. Cook JA, Razzano L: Vocational rehabilita-
tion for persons with schizophrenia: recent
research and implications for practice.
Schizophrenia Bulletin 26:87–103, 2000
2. Bond GR: Supported employment: Evi-
dence for an evidence-based practice. Psy-
chiatric Rehabilitation Journal 27:345–359,
2004
3. Drake RE, Becker DR, Bond GR, et al: A
process analysis of integrated and non-inte-
grated approaches to supported employ-
ment. Journal of Vocational Rehabilitation
18:51–58, 2003
4. Gowdry E, Carlson L, Rapp CA: Practices
differentiating high-performing from low-
performing supported employment pro-
grams. Psychiatric Rehabilitation Journal
26:232–239, 2003
5. Mueser KT, Clark RE, Haines M, et al: The
Hartford study of supported employment
for persons with severe mental illness. Jour-
nal of Consulting and Clinical Psychology
72:479–490, 2004
6. Schneider J: Is supported employment
cost-effective? A review. International
Journal of Psychosocial Rehabilitation
7:145–156, 2003
7. Twamley EW, Jeste DV, Lehman AF: Voca-
tional rehabilitation in schizophrenia and
other psychotic disorders: a literature re-
view and meta-analysis of randomized con-
trolled trials. Journal of Nervous Mental
Disease 191:515–523, 2003
8. Bond GR, Becker DR, Drake RE, et al: Im-
plementing supported employment as an
evidence-based practice. Psychiatric Ser-
vices 52:313–322, 2001
9. Crowther RE, Marshall M, Bond GR, et al:
Helping people with severe mental illness
to obtain work: systematic review. British
Medical Journal 322:204–208, 2001
10. Becker DR, Smith J, Tanzman B, et al: Fi-
delity of supported employment programs
and employment outcomes. Psychiatric
Services 52:834–836, 2001
11. Becker DR, Drake RE, Bond GR, et al: Job
terminations among persons with severe
mental illness participating in supported
employment. Community Mental Health
Journal 34:71–82, 1998
12. Becker DR, Drake RE: A working life for
people with severe mental illness. New
York, Oxford University Press, 2003
13. Kaufman CL: The self -help employment
center: some outcomes from the first year.
Psychosocial Rehabilitation Journal 18:
145–162, 1995
14. Wolfe N: Using randomized controlled tri-
als to evaluate socially complex services:
problems, challenges, and recommenda-
tions. Journal of Mental Health Policy and
Economics 3:97–109, 2000
15. Cook JA, Carey MA, Razzano LA, et al: The
pioneer: the Employment Intervention
Demonstration Program. New Directions
for Evaluation, no 94:31–46, 2002
16. Strayhorn JM: Control groups for psy-
chosocial intervention outcome studies.
American Journal of Psychiatry 144:
275–282, 1987
17. Bond GR, Salyers MP, Collins AL, et al:
How evidence-based practices contribute
to community integration. Community
Mental Health Journal 40:569–588 , 2004
18. Lehman AF, Goldberg R, Dixon LB, et al:
Improving employment outcomes for per-
sons with severe mental illnesses. Archives
of General Psychiatry 59:165–172, 2002
19. Gold PB, Meisler N, Santos AB, et al: Ran-
domized trial of supported employment in-
tegrated with assertive community treat-
ment for rural adults with severe mental ill-
ness. Schizophrenia Bulletin, in press
20. Cook JA, Leff HS, Blyler C, et al: Results of
a multi-site randomized implementation
effectiveness trial of supported employ-
ment interventions for individuals with se-
vere mental illness. Archives of General
Psychiatry 62: 505–512, 2005
21. Cook JA, Lehman AF, Drake RE, et al: In-
tegration of psychiatric and vocational serv-
ices: a multi-site randomized implementa-
tion effectiveness trial of supported em-
ployment, submitted for publication.
American Journal of Psychiatry, in press
22. Burke JK, Cook JA, Grey DD, et al: Demo-
graphic characteristics as predictors of em-
ployment outcomes in a multi-site study of
people with severe mental illness. Commu-
nity Mental Health Journal, in press
23. Cook JA, Razzano LA, Burke JK, et al: Em-
ployment Intervention Demonstration
Program: Common protocol and docu-
mentation. EIDP Coordinating Center,
University of Illinois at Chicago, Chicago,
IL, 2001
24. Department of Labor: Job Training Pro-
gram Act, Disability Grant Program Fund-
ed Under Title III, Section 323, and Title
IV, Part D, Section 452. Federal Register
63:15216–15227, 1998
25. An Experimental Comparison of PACT
and Clubhouse. Final Report from the
Massachusetts Site of the Employment In-
tervention Demonstration Project. Avail-
able at http://files.fountainhouse.org/
samhsa_final.pdf
26. Employment Intervention Demonstration
Program: Cross-site Common Service Cat-
egories and Definitions. Available at www.
psych.uic.edu/eidp/service_categories.pdf
27. Kay SR, Fiszbein A, Opler LA: The Positive
and Negative Syndrome Scale (PANSS) for
schizophrenia. Schizophrenia Bulletin
13:261–276, 1987
28. Anello C, O’Neill R: Does research synthe-
sis have a place in drug development poli-
cy? Synopsis of issues: assessment of safety
and postmarketing surveillance. Clinical
Research and Regulatory Affairs. 13:13–21,
1996
29. Kalaian SA: Meta-analysis methods for syn-
thesizing treatment effects in multisite
studies: hierarchical linear modeling (hlm)
perspective. Practical Assessment, Re-
search, and Evaluation, 8:15, 2003. Avail-
able at http://pareonline.net/getvn.asp?
v=8&n=15
30. Lipsey MW, Wilson DB: Practical Meta-
Analysis. Thousand Oaks, Calif, Sage, 2001
31. Borenstein M, Rothstein H: Comprehen-
sive Meta-Analysis. Englewood, NJ, Bio-
stat, 1999
32. New Freedom Commission on Mental
Health: Achieving the Promise: Transform-
ing Mental Health Care in America. Final
Report. DHHS pub no SMA-03-3832.
Rockville, Md, US Department of Health
and Human Services, 2003
PSYCHIATRIC SERVICES http://ps.psychiatryonline.org October 2005 Vol. 56 No. 10
11224444
lef.qxd 9/21/2005 1:18 PM Page 1244
    • "However, evidence of a quantifiable relationship between employment specialist support and job tenure was less clear. Leff et al. (2005) reported a significant association between tenure in the first job and on-site job support although less than half of their study participants received this support, and receiving job support did not alter the risk of losing the job in the following month. In Bond and Kukla's (2011a) study, face-to-face meetings between the worker and employment specialist, rather than telephone contact, and meeting in the agency office were positively associated with months of employment, but not months in the worker's first job. "
    [Show abstract] [Hide abstract] ABSTRACT: Background: Enabling people with severe mental illness to sustain employment remains a challenge. This is despite most wishing to be employed, and the development of effective vocational interventions and employment supports for this population. To better understand how to enable their sustained involvement in the workforce, this review sought to identify, analyse and summarise studies investigating the factors that impact the job tenure of workers with severe mental illness, irrespective of the type of employment support they received. Methods: An integrative literature review approach was employed to locate, appraise and synthesise quantitative and qualitative research focused on job tenure published in the 20 years up to 2013. Findings from nineteen studies were extracted and integrated using thematic analytic strategies. Results: Job tenure was mostly conceptualised across the reviewed studies as time spent in individual jobs rather than as ongoing participation in the workforce. Three themes describe the factors contributing to job tenure: (1) the worker's experience of doing the current job; (2) natural supports in the workplace; and (3) strategies for integrating work, recovery and wellness, each of which could either support or impede ongoing employment. Conclusion: Occupational therapists, other vocational specialists and mental health staff can use these factors as a guide to supporting people with severe mental illness in employment. More detailed examination of job tenure is required in future research not only on job duration but also on the quality of jobs held, their value for career development and the role of services in supporting tenure.
    Article · Mar 2016
    • "The results that some fidelity domains are more difficult to implement do not imply that they should be given lower priority. Quite the contrary: studies have shown that some frequently lower-rated items are strongly associated with better outcomes (Cook et al., 2005; Leff et al., 2005). This suggests that IPS teams need to devote special efforts to address these critical areas. "
    [Show abstract] [Hide abstract] ABSTRACT: Background: Fidelity scales are used to monitor adherence to evidence-based practices. The underlying assumption is that high fidelity predicts better outcomes. The IPS-25 is a fidelity scale measuring adherence to the Individual Placement and Support (IPS) model. A previous study found a significant association between the IPS-25 and competitive employment outcome. The current study sought to replicate this finding. Objective: We hypothesized that fidelity, as measured by the IPS-25, would predict program-level competitive employment rate. ............... (omitted) ............... Conclusions: The study demonstrated that IPS programs adhering to good fidelity are more likely to achieve enhanced competitive employment outcomes than the sites that have low fidelity. Keywords: Individual Placement and Support (IPS) model, IPS fidelity scale, predictive validity, competitive employment, supported employment
    Full-text · Article · Oct 2015
    • "Employment for people with psychiatric disabilities is a complex phenomenon influenced by a multitude of factors , including labor market conditions, individual work environments, worker characteristics, and public disability policies [1] . Although effective models of vocational rehabilitation have been developed to help people with psychiatric disabilities obtain employment, most jobs that result are low-wage, part-time, and temporary23. Prior research indicates that job separations of workers with serious mental illness are typically voluntary, yet high proportions of workers quit without having obtained another position45. "
    [Show abstract] [Hide abstract] ABSTRACT: We explored the relative effects of adverse working conditions, job satisfaction, wages, worker characteristics, and local labor markets in explaining voluntary job separations (quits) among employed workers with psychiatric disabilities. Data come from the Employment Intervention Demonstration Program in which 2,086 jobs were ended by 892 workers during a 24 mo observation period. Stepped multivariable logistic regression analysis examined the effect of variables on the likelihood of quitting. Over half (59%) of all job separations were voluntary while 41% were involuntary, including firings (17%), temporary job endings (14%), and layoffs (10%). In multivariable analysis, workers were more likely to quit positions at which they were employed for 20 h/wk or less, those with which they were dissatisfied, low-wage jobs, non-temporary positions, and jobs in the structural (construction) occupations. Voluntary separation was less likely for older workers, members of racial and ethnic minority groups, and those residing in regions with lower unemployment rates. Patterns of job separations for workers with psychiatric disabilities mirrored some findings regarding job leaving in the general labor force but contradicted others. Job separation antecedents reflect the concentration of jobs for workers with psychiatric disabilities in the secondary labor market, characterized by low-salaried, temporary, and part-time employment.
    Full-text · Article · Sep 2015
Show more

Recommended publications

Discover more