Neurocognitive enhancement therapy with vocational services:
Work outcomes at two-year follow-up
Morris D. Bella,b,⁎, Wayne Zitoa,b, Tamasine Greigb, Bruce E. Wexlerb
aVA Connecticut Healthcare System, Yale University School of Medicine, United States
bDepartment of Psychiatry, Yale University School of Medicine, United States
Received 28 April 2008; received in revised form 25 June 2008; accepted 29 June 2008
Available online 19 August 2008
Neurocognitive enhancement therapy (NET) is a remediation program for the persistent and function-limiting cognitive
impairments of schizophrenia. In a previous study in veterans, NET improved work therapy outcomes as well as executive function
and working memory. The present study aimed to determine whether NETcould enhance functional outcomes among schizophrenia
and schizoaffective patients in a community mental health center receiving community-based vocational services. Method: Patients
(N=72) participated in a hybrid transitional and supported employment program (VOC) and were randomized to either NET+VOC
or VOC only. NET+VOC included computer-based cognitive training, work feedback and a social information information-
processing group. VOC only also included two weekly support groups. Active intervention was 12 months with 12 month follow-up.
Follow-up rate was 100%. Results: NET+VOC patients worked significantly more hours during the 12 month follow-up period,
reached a significantly higher cumulative rate of competitive employment by the sixth quarter, and maintained significantly higher
rates of employment. Conclusion: NET training improved vocational outcomes, suggesting the value of combining cognitive
remediation with other rehabilitation methods to enhance functional outcomes.
© 2008 Published by Elsevier B.V.
Keywords: Cognitive remediation; Vocational rehabilitation; Neuropsychology; Schizophrenia; Supported employment
Cognitive remediation treatments utilizing combina-
tions of computer-based cognitive exercises, individual
in cognitive performance for patients with schizophrenia
(Kurtz et al., 2001; Twamley et al., 2003; McGurk et al.,
to obtain performance comparable to healthy controls on
tasks used in training (Wexler et al., 1997). In another,
many people were able to reach normal levels of
performance on neuropsychological tests separate from
the training exercises (i.e., within 1 SD below the mean
value of normal subjects, Bell et al., 2001). While some
studies suggest that the gains from cognitive remediation
do not generalize to functional outcomes in the commu-
nity (Pilling et al., 2002; Twamley et al., 2003), others
present encouraging results (Wykes et al., 2007a,b). One
randomized study of 44 clients with severe mental illness
Available online at www.sciencedirect.com
Schizophrenia Research 105 (2008) 18–29
⁎Corresponding author. Psychology Service 116B, VA Connecticut
Healthcare System, West Haven, United States. Tel.: +1 203 932
5711x2281; fax: +1 203 937 4883.
E-mail address: Bell.Morris_D+@West-Haven.VA.Gov(M.D.Bell).
0920-9964/$ - see front matter © 2008 Published by Elsevier B.V.
and a history of job failure found that competitive
employment rates, work hours and wages were signifi-
cantly better for subjects randomized to the “Thinking
Skills for Work Program,” compared to those who
received standard supported employment services alone.
These findings remained robust at two to three year
follow-up (McGurk et al.,2007a,b; McGurk et al.,2005).
Bell et al. (2005) examined work performance and work
productivityina randomizedclinicaltrial comprising145
patients who participated in work therapy (WT) alone, or
work therapy+neurocognitive enhancement therapy
(NET+WT). NET intervention was very similar to the
6 months rather than for 12 months. Outpatients were
placed in transitional employment positions in a Depart-
ment of Veterans Affairs (VA) medical center for six
months, and followed for an additional 6 months.
Participants receiving NET+WT worked significantly
more hours in the 6 months following training than did
those receiving only WT. Moreover, responders to the
cognitive intervention demonstrated the best work out-
comes. In the present study, we hypothesized that the
benefit found for NET with work therapy in the VA for
center (CMHC) with job placements in the community.
The vocational program at this CMHC was a hybrid,
combining elements of transitional and supported
employment programs. Participants received services
(e.g., job development, job coaching) for 12 months and
were followed for an additional 12 months. Specifically,
we hypothesized that clients randomized to NET+
vocational program (NET+VOC) would work more
total hours (transitional and competitive), work more
competitive hours, and have better employment rates at
follow-up than participants randomized to VOC only.
Seventy-seven stable outpatients with diagnoses of
schizophrenia or schizoaffective disorder were recruited
at a large urban community mental health center.
Participants met diagnostic criteria based on the Struc-
tured Clinical Interview for DSM-IV (First et al., 1996)
from the Diagnostic and Statistical Manual of Mental
Disorders (American Psychiatric Association & Task
Force on DSM-IV, 1994), conducted by research
psychologists, and were eligible if they were clinically
stable (i.e., GAFN30, no housing changes, psychiatric
medication alterations, or psychiatric hospitalizations in
the 30 days prior to intake). Exclusion criteria included
history of traumatic brain injury, known neurological
diseases, developmental disability, and active substance
abuse within the past 30 days. Three randomized subjects
were subsequently excluded for failing to meet exclusion
criteria, and one withdrew voluntarily before receiving
services. An additional participant classified as a missing
person by the police was excluded from analyses. Thus,
(See Fig. 1). They completed intake, were randomized
and reached the two-year follow-up between 2000 and
2005. Thanks to the high degree of engagement
engendered by the vocational program, our follow-up
rate for employment data was 100%.
This was an unblinded randomized clinical trial with
an experimental arm (neurocognitive enhancement
therapy (NET) plus vocational rehabilitation (VOC))
and active control arm (VOC only). Randomization was
stratified based on degree of cognitive impairment and
vocational experience. Participants were classified as
cognitivelyimpaired iftheyscored 1SDbelowthemean
(for a previously established schizophrenia sample) on 2
or more of 6 neuropsychological indicators (Bell et al.,
2001). Participants were classified as work experienced
one year. Vocational outcomes were tracked over
24 months. This trial is registered as NCT00339170.
A neuropsychological test battery administered at
intake consisted of the Wechsler Adult Intelligence
Scale — III (WAIS-III; Wechsler, 1997a), Wechsler
Memory Scale — III (WMS-III; Wechsler, 1997b),
Hopkins Verbal Learning Test (HVLT; Brandt and
Benedict, 2001), Verbal Fluency (Benton et al., 1994),
Wisconsin Card Sorting Test (WCST; Bell et al., 1997b;
Heaton, 1981) Gorham's Proverbs (Gorham, 1950;
Marengo et al., 1980), Bell Lysaker Emotion Recogni-
tion Test (BLERT; Bell et al., 1997a) Trail Making Test
(Reitan and Wolfson, 1985), Tower of London (Davis,
1999) and Continuous Performance Test (CPT; Loong,
1991). The 6 variables for determining severity of
cognitive impairment classification were: CPT total
score wrong, HVLT Trial 1, WMS-R Visual Reproduc-
tion, WCST categories correct, Gorham's Proverbs
Bizarreness, and BLERT total score. These were
selected as representative of the major domains of
cognitive processes commonly impaired in schizophrenia
19M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
(attention, verbal and visual memory, executive function,
cognitive disorganization, and affect recognition).
Symptoms were assessed using the Positive and
Negative Syndrome Scale (PANSS; Kay et al., 1987). A
five component model based on factor analysis of the
PANSS (Bell et al., 1994) is used as an alternative to the
rationally derived categories of positive, negative, and
general symptoms. The five components are Positive,
Negative, Cognitive, Hostility, and Emotional Discom-
fort (Table 1). Demographic variables were obtained at
intake from participant interviews, clinician interviews,
and medical chart reviews.
Work performance and cognitive functioning on the
job was assessed bi-weekly using the Work Behavior
Inventory (WBI; Bryson et al., 1997, 1999) and the
Vocational Cognitive Rating Scale (VCRS; Greig et al.,
2004). Ratings were performed by the vocational
specialists, who were trained to good reliability by the
instrument developers. Evaluations were based upon
direct observation of the participant at work and upon
brief interviews with the supervisor. Almost all partici-
pants had obtained their job with the assistance of the
vocational specialist so that the employer was aware that
the participant was involved in a vocational rehabilita-
tion program. Three participants obtained work on their
own and did not want the vocational specialist to
interview their supervisor. They did agree to being
observed discretely while at work, and their ratings were
not blind to condition. However, these scores were used
Fig. 1. Participant flow chart.
20 M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
only for giving work feedback and were not an outcome
measure for comparison by condition.
5. Outcome measures
5.1. Work hours
Work hours were based on payroll records and
verified weekly. Work hours compensated by transi-
tional funds were differentiated from competitive
employment hours, which were compensated by the
employer. Total hours worked refers to all work hours
regardless of transitional or competitive payment
source. Hours spent in groups or in cognitive remedia-
tion were not included as work hours, even though
participation in those activities was compensated.
5.2. Employment rates
Cumulative rates of employment were calculated by
condition. Employment rates were also calculated for
each 13-week period, representing quarterly intervals
over two years.
Work hours and employment rates are objective
measures that are particularly resistant to experimenter
bias. Blinding was not relevant since these outcomes
were not dependent on ratings.
After obtaining written informed consent, which
conformed to the standards of the local institutional
review board, research psychologists performed
diagnostic assessments, neuropsychological testing,
clinical ratings, and collected demographic informa-
tion. Following stratification, participants were ran-
domly assigned to the vocational rehabilitation only
(VOC), or neurocognitive enhancement+vocational
rehabilitation conditions (NET+VOC). Randomiza-
tion was in blocks of 8 generated from a randomiza-
tion website by a statistical assistant not associated
with the study, who worked at a different facility. The
assistant kept the randomization tables and would be
called by the project director when assignment was to
7.1. Vocational rehabilitation (VOC)
The vocational rehabilitation condition was com-
prised of the work services provided by the CMHC
with the addition of weekly groups (described below)
led by the research staff. The CMHC vocational
program is a hybrid transitional and supported
employment program that derives its essential features
from the Individual Placement and Support model
(IPS) plus transitional funds to facilitate rapid job
placement in community-based sites. Job specialists
determined when transitional funds, provided by the
State of Connecticut, were necessary to quickly place a
participant in community-based employment. These
decisions were made independently and without input
from the research team. Transitional funds were used
Mean SDMean SD
Age at 1st hospitalization
Lifetime # hospitalization
aNo significant differences by condition.
cNeurocognitive enhancement therapy.
21M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
when employers were willing to hire a participant if the
client could perform the job adequately. Usually transi-
tional funds were used at the beginning of a participant's
involvement, but sometimes they were used later on, in a
second job attempt. These jobs were incommunity-based
competitive settings where participants performed the
same jobs as their non-mentally ill co-workers, the only
difference being the payment source during the transi-
tional period. In all other respects, these jobs are
consistent with competitive employment defined by the
US Department of Labor and the supported employment
literature in that these jobs 1) paid minimum wage or
higher; 2) were in a mainstream, socially integrated
and 4) were not set aside or contracted with a social
service agency (Razzano et al., 2005).
led groups: a work support group and a lifestyles group.
of work-related issues. The lifestyles group focused on
social concerns (including how to handle their newly
earned income), but without structured exercises or
planned activities in order to preserve the distinction of
the VOC condition as a non-cognitive intervention.
Participants were paid minimum wage for each hour-
long group attended. Groups were offered weekly for
the first 12 months of their participation.
7.1.1. Treatment integrity of the VOC condition
(Perepletchikova et al., 2007):
Apart from the deliberate deviation of transitional
funds, the vocational program was committed to
adhering to the principles of the IPS model. This
modified IPS program received a score of 62 out of 75
(83%; “Fair” rating) on the IPS fidelity scale when
rated by three independent raters including Gary Bond
Ph.D. (personal communication), who interviewed the
program staff and vocational specialists. The rating
was lowered primarily by the use of transitional funds
and by the fact that job specialists were not members
of the primary treatment teams. Following the guide-
lines of supported employment, job specialists and
participants developed a work plan collaboratively,
with participants' interests, preferences and experience
taken into consideration. On-site coaching and all other
conventional services were provided throughout the
24 months of participation. Typical jobs were entry-
level service positions such as washing dishes at a
restaurant, serving meals in a retirement home, and
processing clothes in a department store. Vocational
specialists were supervised weekly by their own
program directors, and the research staff met with
them at least once a week to review their activity. Their
job coaching hours were recorded and compared
between the two conditions (reported in results).
The PI met weekly with research staff members, who
were doctoral level clinical psychologists, to review
group activities, and the PI occasionally attended the
groups. Careful attention was paid to preserving the
distinction between these groups and the cognitively
based groups in the other condition by frequent reviews
of the guiding principles and format. Attendance was
carefully monitored. Those in the VOC condition had a
mean of 26.6 (15.6) work support groups and a mean of
21.9 (14.7) life-style groups for a combined mean of
48.6 (28.8) groups for the year and these rates were
compared with those of the NET+VOC condition (see
7.2. Neurocognitive enhancement therapy (NET)+VOC
Participants in the NET+VOC condition received
identical employment services to participants in the
VOC condition. Participants in the NET+VOC condi-
tion received up to 10 h per week of computerized
cognitive exercises and attended two cognitively
focused, research staff-led groups per week focused on
work feedback and social information-processing. Job
specialists attended the work feedback group and
provided participants with specific feedback from
WBI and VCRS evaluations which was then used to
formulate individual weekly work goals.
Cognitive remediation exercises (Table 2) were
drawn from two sources: CogRehab, (Bracy, 1995)
originally developed for people with compromised brain
function and modified according to our specifications
for people with schizophrenia, and Sci-Learn, developed
by the Scientific Learning Corporation in conjunction
with one of the authors (BEW) (Scientific Learning,
2003). The exercises required cognitive abilities often
compromised in schizophrenia (e.g., attention, lan-
guage, memory, executive function), were deliberately
made very easy at the start of treatment and followed a
standard sequence and progression of difficulty. Parti-
cipants graduated to new tasks after achieving a
prescribed performance level, or when their perfor-
mance remained unchanged over 8 half-hour training
sessions. In this way, tasks were neither too boring nor
too challenging, and the intervention was adjusted to
each participant's pattern of cognitive strengths and
weaknesses. Patients were paid minimum wage for each
hour of cognitive remediation.
group exercises designed by Ben-Yishay et al. (1985) for
22M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
patients with traumatic brain injury. Each week, one
participant was responsible for preparing an oral
presentation on a work-related topic (e.g., “The people I
work with”) while non-presenting members asked ques-
tions and provided feedback and constructive criticism.
The social information-processing group demanded
attention, memory, and problem solving, as well as affect
recognition, empathy, and verbal communication skills.
As in the VOC condition, participants received minimum
wage for each group attended, and the groups were
available for the first 12 months of participation.
7.2.1. Treatment integrity of the NET+VOC condition
because it was computer based, the exercises themselves
were the same for all participants. However, a guiding
principle was to optimize the balance between challenge
and frustration. For that reason, occasional alterations in
would be made in order to preserve that optimal balance.
Staff was vigilant to participant's engagement in the
activity. If a participant began to fall asleep or appeared to
be responding randomly, staff would intervene. In that
encouraged a high level of training intensity and duration,
even thoughwe were concernedthat it might conflictwith
of neuroplasticity that call for intense and repetitive
practice in order to remediate impaired neurocognitive
of cognitive training, over an average of 28 weeks (sd
SD=17.0), for an average weekly training of 3.6 h (sd
SD=1.9). As with any treatment trial, there was a wide
(more than 70 h) of the intervention. Six participants
(15.8%) trainedfor 10horfewer;6(15.8%) trainedfor 11
to 64 h; 19 (50%) trained for 71 to 165.75 h; 7 (18.4%)
trained for 212 to 359.5 h.
The PI met weekly with research staff to review
group activities and occasionally observed the groups.
Cognitive training tasks
Training tasksComponent cognitive processes
Simple visual random small
Complex attention and response inhibition
Simultaneous multiple attention
PAT-alternate by signal
Reverse digits auditory
Reverse digits visual
Verbal memory categorizing
Planning and strategy
X xxx xx
23M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
The group principles and procedures had been devel-
oped and manualized by the PI in an earlier study (Bell
et al., 2005), and stafffrom the earlier study were among
those in the current study. Adherence to these
procedures was reinforced weekly, and written material
from the meetings (such as WBI feedback graphs and
participant presentation outlines) were checked for
conformity to the principles and completeness. The
social information-processing group included ratings on
each participant's performance in the group (data not
included in this report) which served as another check
on the quality of the groups. Attendance was also
recorded. Participants had a mean of 28.1 (14.8) work
feedback sessions and a mean of 20.9 (14.8) social
information information-processing sessions for a
combined mean of 49.0 (26.5) for the year. Conditions
did not differ significantly on number of groups
attended (df=70, t=.07, p=n.s.).
8. Data analyses
Except when specified, statistical analyses were
performed using SPSS 12.0 for Windows. Chi-square
analyses were employed for categorical data and t-tests
for violations of normality and homogeneity of variance.
Transformations were performed where necessary using
Blom normalization procedures in SPSS. Because job
specialists were not blind to condition, intensity of job
coaching over the one-year period was compared by
condition to test for the possibility that one condition
received more intensive services than the other. Hours
coached by a job specialist were calculated as both the
total hours of job coaching and as the ratio of coached
work hours to total work hours.
Work outcome measures were analyzed by a
statistical consultant using SAS version 9 for Windows.
Employment rates were compared by condition over
eight quarters using repeated measures, mixed model
analyses with a binary dependent variable indicating the
presence or absence of work activity (work activity —
yes/no). Analyses were performed separately for rates of
total community-based work activity (transitional and
competitive payment combined), and rates of commu-
nity-based work activity paid exclusively by the
employer (competitive payment). Rates of employment
(total and competitive payment) were also examined
cumulatively for the two-year period.
The statistical consultant compared hours worked by
condition across quarters over the course of the two-year
study using mixed models, repeated measures ANO-
VAs. Analyses were performed separately for total hours
of community-based work activity (transitional and
competitive payment combined), and hours of commu-
nity-based work activity paid exclusively by the
employer (competitive payment). Because of significant
skew introduced by non-working participants in each
quarter, analyses were repeated using ranked data as a
nonparametric check of the results. Except where
corrected for multiple comparisons, alpha was set at
.05 and all tests were two-tailed.
There were no significant differences between
conditions in demographic characteristics (Table 1) or
previous work experience (Table 3).
Cumulative transitional and competitive employment
rates reveal that 95% of participants in NET+VOC and
100% in VOC went to work within the first 13 weeks
(Fig. 2a). Cumulatively, conditions did not differ on
rates of total employment, but the NET+VOC group
achieved significantly higher cumulative competitive
Participant work experience at intakea
# Full time jobs held for one or more years
No full time employment
Three or more jobs
Longest full time job (years)
No full time employment
Two to four years
Five to ten years
N Ten years
# Years since last full time employment
No full time employment
Six to ten years
Eleven to fifteen years
Sixteen to twenty years
N Twenty years
aNo significant differences by condition.
24M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
employment rates (Fig. 2b) by the sixth quarter
(χ2=3.57, df=1 pb.05). The absolute difference in
risk of never being competitively employed between the
two conditions was 22.3% (95%CI — .26% to 44.84%)
and the Number Needed to Treat to see an effect is 5.
During the active intervention in year-one, the NET+
VOC and VOC conditions did not differ significantly in
total hours worked (NET+VOC Mean (SD)=245.5
(250.7); VOC=346.4 (299.3)), total competitive hours
worked (NET+VOC Mean (SD)=89.1 (196.6);
VOC=101.9 (216.7)), number of jobs held (NET+
VOC Mean (SD)=1.35 (.75); VOC=1.54 (.56)) and
duration of jobs held in weeks (NET+VOC Mean
(SD)=20.3 (16.0); VOC=22.05 (15.6)). Hours of job
coaching did not differ by condition (NET+VOC
Mean (SD)=46.2 (43.8); VOC=52.8 (41.9)) nor did
the intensity of job coaching as a percentage of number
of hours worked in which there was on on-site job
coaching (NET+VOC=25%; VOC=22%).
Mixed model repeated measures ANOVA results for
total hours worked at quarterly intervals over two years
(Table 4), however, showed a significant time by
condition interaction (F(1,572)=10.00, pb.001). The
finding for the time by condition interaction remained
Fig. 2. Cumulative rates of employment.
Hours and rates of employment by condition over 8 quarters
Q1 Q2 Q3 Q4Q5Q6Q7 Q8
Total hours (transitional and competitive)a
NET+VOC 59.38 (64.67)
Quarterly rates in percent (combined)c
Quarterly rates in percent (competitive)d
aTotal hours time×condition interaction, pb.001.
bCompetitive hours show significant linear trend by condition for Q5 to Q8, pb.05.
cQuarterly rates show a significant time×condition interaction for Q5 to Q8, pb.005.
dQuarterly rates of competitive employment do not differ significantly by condition.
25M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
7.74, pb.005), indicating that skew from non-working
participants did not influence the results. Post-hoc
comparison of hours in the 12 months of follow-up
show a significant difference (pb.05) favoring NET+
VOC. However, the standardized effect size for this
model repeated measures analyses for competitive
(employer-funded) work hours over two years were not
significant. Competitive work hours did show a sig-
(F(1,70)=3.72, pb.05) with results favoring NET+
Mixed models analyses of quarterly employment
rates over two years showed a significant interaction
by condition for year two (t(71)=2.91, pb.005) when
transitional and competitive community-based work
activity were combined. In the final quarter, 44.7%
in the NET+VOC condition were working compared
with 23.5% of VOC. By the 8th quarter (Table 4), the
absolute difference in risk of unemployment was
21.2% (95%CI — .08 to 42.5%) and the Number
Needed to Treat to see an effect is 5. Strictly competitive
(employer-funded) employment rates did not show a
significant interaction by condition over two years.
Results support the hypothesis that NET enhanced
vocational outcomes. Participants in NET+VOC
achieved more total hours worked during the 12 month
during the follow-up period, and achieved higher
cumulative rates of competitive employment in the last
three quarters. Patients in the VOC only condition
It is particularly encouraging that patients who had
received NET were significantly better able to sustain
employment throughout the second year.
The vocational program at the CMHC was a
combination of transitional funds and IPS services. It
was the same for both conditions and was not an
independent variable. The NET training was the inde-
pendent variable, but with its groups and computer
computer-based training, it was multifaceted. Thus, any
and their interactions must be complex and speculative.
However, we offer some comments about the treatment
elements and their possible influence on results based
upon our observation of our participants' experience.
Cumulative employment rates (transitional and compe-
in this study tried to work and that they did so within the
first 13 weeks. Thanks to the availability of transitional
funds, even the most impaired had the opportunity to be
employedinthe community ina jobof their choosing with
appropriate supports. Given the opportunity, most took
advantage of it. These participants were not screened for
“work readiness”. Following the IPS model, all that was
required of them was an initial willingness to work. We
believe that this high response to appropriate opportunity
supports the survey literaturethat many people with severe
in the NET+VOC condition were somewhat slower to go
to work initially, probably because they were engaged in
cognitive training. Any non-specific effects of additional
staff attention during computer training in the first few
months of the intervention would have been offset for the
VOC only condition by the non-specific benefits of more
Patients in both conditions received approximately
1 h of on-site job coaching for every 4 h worked as well
as a weekly group discussion focused on work
experiences. This ratio may be more intensive than is
generally the case in IPS programs (personal commu-
nication with Gary Bond). It may be a consequence of
the availability of transitional funds that allowed
everyone to work; including the most impaired who
needed continual coaching.
Patients in the NET+VOC condition had a potential
advantage in that their weekly group included their job
coach. They received regular and consistent feedback
about their work performance and their cognitive
functioning on the job with specific goal-setting based
upon that feedback. Because of rapid job engagement,
this feedback (as well as feedback from on-the-job
coaching received by all subjects) could begin early in
the rehabilitation program, thus maximizing the time
period in which the participant could benefit from these
Those in the NET+VOC condition received an
intense dose of cognitive remediation. To our knowl-
edge, this is the highest dose thus far reported in the
published literature. We deliberately sought to achieve a
high intensity and duration of training because our
approach is based upon animal and human experimental
literature on experienced-based neuroplastic brain
changes, which require intensive, repetitive training for
results (Wexler and Bell, 2005). We also were concerned
that negative results not be attributable to insufficient
training dose. Naturally, a few participants did not take
advantage of the training, but most did. Elsewhere, (Bell
et al., 2001, 2007, 2005; Greig et al., 2007) we have
reported that this training led to greater cognitive
26M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
improvements compared with those who received only
work services on pre-post neuropsychological measures
of executive function and working memory.
Participants were paid for their time in cognitive
cognitive training adherence or delayed rapidity of job
attainment. We noticed that a number of participants
began to think of the cognitive training as an agreeable
part-time job, and we were sometimes concerned that
cognitive training might be substituting for competitive
employment. Quarterly rates of employment rose more
only participants' rates declined. While lower rates in the
first three quarters for NET+VOC may be attributable to
to indicate that participants gained something from their
year of training that allowed them to sustain employment
and to achieve and maintain competitive employment
even after cognitive remediation and weekly support
groups ended. By contrast, the decline for VOC only
participants in the second year would seem to suggest
someother stressor) thatledtojobdiscontinuationandno
further job attainment, despite on-going assistance from
NET+VOC included a social information informa-
tion-processing group that may have served therapeutic
as well as cognitive training purposes. In this group,
participants prepared and delivered presentations on
topics related to their work life. In addition to the
cognitive aspects of the exercise, we observed that this
process helped many participants organize and give
voice to their on-going experience of struggle and
recovery. Lysaker et al. (2002, 2006) have described
how patients' narrative about their illness and recovery
can change through rehabilitation and that interventions
that address this narrative process may improve
functional outcomes. Thus, this group may have had
therapeutic effects beyond the intended training targets.
The results of this study support the hypothesis that
cognitive interventions when combined with compre-
hensive rehabilitation may enhance functional out-
comes, a similar finding to that of Hogarty et al. (2004,
2006) who used cognitive training and other rehabilita-
tion methods to enhance cognitive function, “gistful
learning” and social competence. Although time by
condition interaction shows a powerful effect for NET+
VOC, the effect sizes for total hours and rates of
employment are modest. Cognitive training has more
powerful effects on proximal end-points such as
neuropsychological test performance (McGurk et al.,
2007a), but more distal outcomes such as employment
likely have many other moderating or mediating factors
in press), we found that the greatest benefits from
training occurred for those with the poorest community
NET may only be needed for those whose impairments
make it unlikely that they would attain functional goals
a cost utility analysis regarding its effects on such a
subgroup within the schizophrenia spectrum.
The strengths of this study are the 100% follow-up at
two years and the objectivity of its outcome measures
(hours and rates of employment). Its limitations include
the complexity of the intervention and the fact that staff
was not blind to condition. Despite our efforts to
equalize attention by having two support groups for the
VOC only condition to match the two groups in the
NET+VOC condition, we cannot know what other
biasing processes might have been at work. Regarding
the vocational specialists, they were from a separate
agency and were evaluated by their supervisor accord-
ing to their clients' achievements without regard to
condition. They made their own decisions about
transitional funds, job coaching and job seeking
strategies. Our check on intensity of job support
indicates that NET+VOC and VOC only participants
received similar intensity of services. We do not believe
that the vocational specialists were aware of, or
particularly invested in, our cognitive training hypoth-
eses. Nevertheless, we cannot know for certain that bias
was not present.
This study is now the third, along with the VA study
(Bell et al., 2005) and that of McGurk et al. (2005) to
demonstrate that cognitive training combined with work
services may yield better vocational outcomes. More-
over, the current study addresses a number of metho-
dological concerns raised by previous studies in regard
to study size, length, intensity and integrity of cognitive
training, type of vocational services, and integrity of the
active control condition. The mechanisms for this effect
are as yet undetermined, and dismantling studies that
unpack the intervention may be needed to understand
essential therapeutic elements and their interaction. The
social information information-processing group and
the work feedback and goal-setting group were
cognitively based activities, but they were not traditional
cognitive remediation. It could be that these group
experiences caused the main effects alone or that they
were necessary but insufficient without the computer-
based exercises to cause the main effects. Hogarty et al.
27 M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
(2004, 2006) found that improvements in processing
speed and neurocognition from cognitive training in the
first six months of their two two-year intervention
persisted for the next two and a half years and mediated
subsequent social cognition and social adjustment
outcomes. Wykes et al. (2007b) found a similar
mediating effect of cognitive training on functional
outcome in a study of young, early onset patients with
schizophrenia. More studies of this kind may help shape
future interventions by indicating how cognitive
remediation should be related to other rehabilitation
interventions that may translate cognitive gains into
meaningful functional outcomes.
Role of funding source
Rehabilitation Research and Development Service.
Morris Bell and Bruce Wexler made substantial contributions to
the conception, design, analysis, interpretation, drafting and final
approval of this article. Wayne Zito and Tamasine Greig contributed to
the collection, analysis and interpretation of data as well as
contributing important intellectual content to the article and giving
final approval of this version.
Conflict of interest
Authors have no relevant financial interest in this manuscript and
no conflict of interest.
There are no acknowledgements.
American Psychiatric Association, & Task Force on DSM-IV, 1994.
Diagnostic and Statistical Manual of Mental Disorders, 4th ed.
American Psychiatric Association, Washington, DC.
Bell, M.D., Lysaker, P.H., Beam-Goulet, J.L., Milstein, R.M.,
Lindenmayer, J.P., 1994. Five-component model of schizophrenia:
assessing the factorial invariance of the positive and negative
syndrome scale. Psychiatry Research 52 (3), 295–303.
Bell, M., Bryson, G., Lysaker, P., 1997a. Positive and negative affect
recognition in schizophrenia: a comparison with substance abuse
and normal control subjects. Psychiatry Research 73 (1–2), 73–82.
Bell, M.D., Greig, T.C., Kaplan, E., Bryson, G., 1997b. Wisconsin
card sorting test dimensions in schizophrenia: factorial, predictive,
and divergent validity. Journal of Clinical & Experimental
Neuropsychology: Official Journal of the International Neuropsy-
chological Society 19 (6), 933–941.
Bell, M., Bryson, G., Greig, T., Corcoran, C., Wexler, B.E., 2001.
Neurocognitive enhancement therapy with work therapy: effects
on neuropsychological test performance. Archives of General
Psychiatry 58 (8), 763–768.
Bell, M.D., Bryson, G.J., Greig, T.C., Fiszdon, J.M., Wexler, B.E.,
2005. Neurocognitive enhancement therapy with work therapy:
productivity outcomes at 6- and 12-month follow-ups. Journal of
Rehabilitation Research and Development 42 (6), 829–838.
Bell, M., Fiszdon, J.M., Greig, T., Wexler, B.W., Bryson, G., 2007.
Neurocognitive enhancement therapy with work therapy in schizo-
phrenia: a six month follow-up of neuropsychological performance.
Bell, M.D., Zito, W., Greig, T., Wexler, B.E., in press. Neurocognitive
Enhancement Therapy and Competitive Employment in Schizo-
phrenia: Effects on Clients with Poor Community Functioning.
American Journal of Psychiatric Rehabilitation.
Ben-Yishay, Y., Rattok, J.A., Lakin, P., Piasetsky, E., Ross, B., Silver,
S., et al., 1985. Neuropsychological rehabilitation: quest for a
holistic approach. Seminars in Neurology 5, 252–259.
Benton, A.L., Hamsher, K., deSivian, A., 1994. Multilingual Aphasia
Examination. AJA Associates, Iowa City, IA.
Bracy, O., 1995. CogReHab Software. Psychological Software
Services, Indianapolis, Indiana.
Brandt, J., Benedict, R.H.B., 2001. Hopkins Verbal Learning Test—
revised. Professional Manual.
Bryson, G., Bell, M.D., Lysaker, P., Zito, W., 1997. The work behavior
severe mental illness. Psychiatric Rehabilitation Journal 20 (4),
Bryson, G.J., Greig, T., Bell, M., Kaplan, E., 1999. The work behavior
inventory: prediction of future work success of people with
schizophrenia. Psychiatric Rehabilitation Journal 23, 113–117.
Davis, H., 1999. Tower of London. Colorado Assessment Tests,
University of Colorado.
First, M.B.S., R.L., Gibbon, M., Williams, J.B. (1996). Structured
clinical interview for DSM-IV Axis I disorders—patient edition
(SCID-I/P, version 2.0). New York: Biometrics Research Depart-
ment, New York State Psychiatric Institute.
Gorham, D., 1950. The use of the proverbs test for differentiating
schizophrenics from normals. Journal of Consulting Psychology
Greig, T.C., Nicholls, S., Bryson, G.J., Bell, M.D., 2004. The
vocational cognitive rating scale: a scale for the assessment of
cognitivefunctioning at work for clients with severemental illness.
Journal of Vocational Rehabilitation 21, 71–81.
Greig, T.C., Zito, W., Wexler, B.E., Bell, M.D., 2007. Neurocognitive
enhancement therapy and supported employment: effects on
neuropsychological tests performance at one year [abstract].
Schizophrenia Bulletin 33, 559.
Heaton, R., 1981. The Wisconsin Card Sorting Test manual.
Psychological Assessment Resources, Inc, Odessa, FL.
Hogarty, G.E., Flesher, S., Ulrich, R., Carter, M., Greenwald, D.,
Pogue-Geile, M., et al., 2004. Cognitive enhancement therapy
for schizophrenia: effects of a 2-year randomized trial on
cognition and behavior. Archives of General Psychiatry 61 (9),
Hogarty, G.E., Greenwald, D.P., Eack, S.M., 2006. Durability and
mechanism of effects of cognitive enhancement therapy. Psychia-
tric Services 57 (12), 1751–1757.
Kay, S.R., Fiszbein, A., Opler, L.A., 1987. The positive and negative
syndrome scale (PANSS) for schizophrenia. Schizophrenia
Bulletin 13 (2), 261–276.
Kurtz, M.M., Moberg, P.J., Gur, R.C., Gur, R.E., 2001. Approaches to
cognitive remediation of neuropsychological deficits in schizophre-
nia: a review and meta-analysis. Neuropsychology Review 11 (4),
28M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29
Loong, J., 1991. The Manual for the Continuous Performance Test. Download full-text
Wang Neuropsychological Laboratory, San Louis Obispo, CA.
Lysaker, P.H., Clements, C.A., Plascak-Hallberg,C.D., Knipscheer, S.J.,
Wright, D.E., 2002. Insight and personal narratives of illness in
schizophrenia. Psychiatry 65 (3), 197–206.
Lysaker, P.H., Taylor, A., Miller, A., Beattie, N., Strasburger, A.,
Davis, L.W., 2006. The scale to assess narrative development:
association with other measures of self and readiness for recovery
in schizophrenia spectrum disorders. Journal of Nervous & Mental
Disease 194 (3), 223–225.
Marengo, J., Harrow, M., C., R., 1980. A Manual for Scoring Abstract
and Concrete Responses to Verbal Tests. Microfilm Publications,
McGurk, S.R., Mueser, K.T., Pascaris, A., 2005. Cognitive training
and supported employment for persons with severe mental illness:
one-year results from a randomized controlled trial. Schizophrenia
Bulletin 31 (4), 898–909.
2007a. A meta-analysis of cognitive remediation in schizophrenia.
American Journal of Psychiatry 164, 1791–1802.
McGurk, S.R., Mueser, K.T., Feldman, K., Wolfe, R., Pascaris, A.,
2007b. Cognitive training for supported employment: 2–3 year
outcomes of a randomized controlled trial. American Journal of
Psychiatry 164 (3), 437–441.
Perepletchikova, F., Treat, T.A., Kazdin, A.E., 2007. Treatment
integrity in psychotherapy research: analysis of the studies and
examination of the associated factors. Journal of Consulting and
Clinical Psychology 75 (6), 829–841.
Pilling, S., Bebbington, P., Kuipers, E., Garety, P., Geddes, J.,
Martindale, B., et al., 2002. Psychological treatments in schizo-
phrenia: Ii. Meta-analyses of randomized controlled trials of social
skills training and cognitive remediation [see comment] Psycho-
logical Medicine 32 (5), 783–791.
Razzano, L.A., Cook, J.A., Burke-Miller, J.K., Mueser, K.T., Pickett-
Schenk, S.A., Grey, D.D., et al., 2005. Clinical factors associated
with employment among people with severe mental illness:
findings from the employment intervention demonstration pro-
gram. Journal of Nervous & Mental Disease 193 (11), 705–713.
Reitan, R., Wolfson, D., 1985. The Halstead-Reitan Neuropsychological
Scientific Learning. (2003). Www.Hypomania.Scilearn.Com/alp.
Twamley, E.W., Jeste, D.V., Bellack, A.S.,2003.A review of cognitive
training in schizophrenia. Schizophrenia Bulletin 29 (2), 359–382.
Wechsler, D., 1997a. Wais-iii Manual: Wechsler Adult Intelligence
ScaleIIIi. Psychological Corporation, San Antonio, TX.
Wechsler, D., 1997b. The Wechsler Memory Scale—Third Edition.
Psychological Corporation, San Antonio, TX.
Wexler, B.E., Bell, M.D., 2005. Cognitive remediation and vocational
rehabilitation for schizophrenia. Schizophrenia Bulletin 31 (4),
M.J., Green, M.F., 1997. Normal neurocognitive performance after
extended practice in patients with schizophrenia. Schizophrenia
Research 26 (2–3), 173–180.
2007a. Cognitive remediation therapy in schizophrenia: randomised
controlled trial. British Journal of Psychiatry 190, 421–427.
Wykes, T., Newton, E., Landau, S., Rice, C., Thompson, N., Frangou,
S., 2007b. Cognitive remediation therapy (CRT) for young early
onset patients with schizophrenia: an exploratory randomized
controlled trial. Schizophrenia Research 94, 221–230.
29M.D. Bell et al. / Schizophrenia Research 105 (2008) 18–29