Implementing Evidence-Based Practices for People With Schizophrenia
Robert E. Drake1,2, Gary R. Bond3, and SusanM. Essock4
2Dartmouth Psychiatric Research Center, Dartmouth Medical
School, Lebanon, NH;3Department of Psychology, Indiana
University-Purdue University Indianapolis, Indianapolis, IN;
4Department of Psychiatry, Columbia University, and New York
State Psychiatric Institute, New York, NY
Over the last decade, a consensus has emerged regarding
a set of evidence-based practices for schizophrenia that ad-
dress symptom management and psychosocial functioning.
Yet, surveys suggest that the great majority of the popula-
tion of individuals with schizophrenia do not receive
evidence-based care. In this article, we review the empirical
literature on implementation of evidence-based practices
for schizophrenia patients. We first examine lessons
learned from implementation studies in general medicine.
We then summarize the implementation literature specific
to schizophrenia, including medication practices, psychoso-
cial interventions, information technology, and state- and
federal-level interventions. We conclude with recommenda-
tions for future directions.
Key words: evidence-based practices/schizophrenia/
Mental Health,1the President’s New Freedom Commis-
sion Report,2Schizophrenia Patient Outcomes Research
Project (TMAP),4and several other systematicefforts5–10
have identified a variety of evidence-based interventions
for persons with schizophrenia. Examples of effective
interventions include systematic approaches to medica-
prevention programs, and supported employment.
Despite robust evidence on effective interventions, ep-
idemiologic and clinical surveys have established that
individuals with schizophrenia in the United States are
unlikely to receive these effective treatments.11Epidemi-
ologic data from the National Comorbidity Study in the
tal illnesses received no treatment in the past year, 25%
received clearly inadequate treatment, and only 15% re-
ceived minimally adequate (far short of evidence-based)
treatment.12The Schizophrenia PORT study similarly
found that patients in 2 large state public mental health
programs were unlikely to receive most of the indicated
evidence-based practices.13More recently, the 2005 Na-
tional Survey on Drug Use and Health14found that only
8.5% of adults who reported both serious psychological
distress and a substance use disorder received any treat-
ment (again far short of evidence-based treatment) for
both problems in the past year. Several recent studies in-
dicate that quality of care may be worsening rather than
improving.15–17Thus, even as researchers continue to de-
velop more effective interventions for the treatment of
schizophrenia, the preponderance of individuals with
this disorder, perhaps as many as 95%, receive either
no care or less than optimal care. Previous articles in
this special section have reviewed the epidemiology of
services and the problems of engagement and retention.
In this article, we review efforts to implement effective
interventions for schizophrenia in routine mental health
treatment settings and offer suggestions for narrowing
the gap between science and practice.
Implementation Research in General Medicine
Shojania and Grimshaw18summarized the general med-
ical literature on quality improvement efforts with 4 con-
clusions: First, the standard approach of passive diffusion
of research (ie, publication of research findings in profes-
sional journals), including dissemination of findings on
effective interventions, has little or no impact on routine
practice. Second, more complex efforts to synthesize re-
search evidence in the form of systematic reviews and dis-
seminated guidelines also have little or no effect on
practice. Third, adopting total quality management/con-
tinuous quality improvement techniques from industry
has produced modest but disappointing results. Quality
improvement, popularized by Deming,19is a process that
1To whom correspondence should be addressed; Psychiatric
Research Center, 2 Whipple Place, Lebanon, NH 03766, tel: 603-
Schizophrenia Bulletin vol. 35 no. 4 pp. 704–713, 2009
Advance Access publication on June 2, 2009
? The Author 2009. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved.
For permissions, please email: email@example.com.
focuses on training, education, and using data to enhance
the performance of an organization.20,21In mental health
treatment settings, quality improvement has taken the
form of field-based supervision and systematic review
of patient outcomes.22–24Fourth, current attempts at
complete systems reengineering using information tech-
nology have produced mixed results, including many
prominent successes, which need to be understood in
greater detail, as discussed below.
Disease Management Systems and the Chronic Care
Health maintenance organizations have widely adopted
disease management systems that emphasize activating
patients for self-management, ensuring that clinical teams
have the relevant expertise and access to evidence-based
guidelines, and monitoring patients without face-to-face
contacts.25Wagner’s26–28chronic care model represents
the most clearly articulated and widely studied approach
to disease management. It explicitly aims to combine the
best available implementation strategies. The Wagner
model includes 6 components: (1) addressing ‘‘health care
organization’’ at the level of leadership, financing, and
removing barriers; (2) improving access to ‘‘community
resources,’’ such as self-help, self-management classes, and
nurse educator services; (3) enhancing ‘‘self-management
plans, identify barriers, and solve problems; (4) establishing
evidence-based guidelines into registries, flow charts, and
ning, services, outcomes, and algorithms. Many disease
include only a portion of Wagner’s model. The evidence
diabetes, asthma, cardiovascular disease, and depression.29
Implementing Medication Guidelines for Patients With
Since the original Schizophrenia PORT study in the
1990s, several efforts have targeted implementing effec-
tive interventions, or evidence-based practices, for
schizophrenia patients on a large scale rather than as
small demonstrations. Because changing medication
practice involves a different set of providers and mecha-
nisms from changing psychosocial practice, we consider
implementing medication guidelines first.
Antipsychotic treatments are a mainstay of treatment
for schizophrenia, and most people with schizophrenia
who are in treatment have access to antipsychotic med-
ication. In the original Schizophrenia PORT study of ad-
herence to recommendations in routine practice, the
likelihood of a person hospitalized with schizophrenia re-
ceiving an antipsychotic medication was high (89%), but
the quality of the antipsychotic prescribing frequently de-
viated from the evidence-based recommendations (only
62%of inpatients and 29% of outpatients were prescribed
antipsychotic medication either within the recommended
dosage range or with chart documentation justifying the
dosage prescribed).13Because prescribing medications is
an essential component of treatment of schizophrenia
and is relatively easy to implement (compared, eg, with
mounting assertive community treatment teams or mul-
payers, and researchers have focused considerable
efforts on improving prescribers’ adherence to treatment
The Texas Medication Algorithm Project
The largest of these efforts, the TMAP, began in 1996
with the intent to develop, implement, and evaluate an
algorithm-driven treatment approach for adults with
major psychiatric disorders treated within the public
mental health sector in Texas (http://www.dshs.state.tx.
several elements of Wagner’s chronic care model, includ-
ing patient education, systematic assessments, and clear
guidelines. The project encompassed 4 phases: (1) crea-
tion of evidence-informed algorithms via consensus con-
ferences that clearly indicated (via a flow diagram) the
sequence in which the various medication alternatives
should be considered (eg, ‘‘Consider clozapine before
polypharmacy’’), (2) a feasibility trial using the algo-
rithms with early adopters to estimate their clinical im-
pact and determine the resources needed to implement
them, (3) a prospective comparison of the clinical out-
comes and economic costs of using the algorithms vs
treatment as usual in outpatient public mental health
clinics, and (4) a broad implementation of the algorithms
throughout the public mental health system in Texas,
called the Texas Implementation of Medication Algo-
rithms.The goalsofTMAP were simple butfar-reaching:
to encourage measurement-based medicine to optimize
patients’ outcomes. By combining sequential treatment
algorithms with standardized measurements of symp-
toms and functioning at each visit, the prescriber could
track progress or lack thereof over time to determine
whether a medication change was indicated.
A wide variety of stakeholders followed TMAP closely.
Patients, family members, payers, and clinicians were con-
cerned about wide variations in prescribing practices and
lauded TMAP for seeking to reduce unwarranted varia-
tion and ensure access to evidence-based decisions regard-
ing medication management. Many other states, counties,
and systems of care modeled algorithm-driven interven-
tions after TMAP, even before results of the prospective
comparison study were available.
Evidence-Based Practices for Schizophrenia Patients
In the prospective comparison study, after 3 months of
treatment, patients with schizophrenia who received
treatment in the sites that were trained and staffed to
use the TMAP algorithms had greater improvement in
symptoms (about 3 points on the 18-item Brief Psychiat-
ric Rating Scale, after transformations of the scores to
adjust for baseline differences and other factors known
to affect health outcomes) than did patients in the com-
parison sites, but this difference in transformed scores
disappeared over time.4Patients in both algorithm sites
test scores measuring cognitive functioning, with the
patients in the algorithm sites showing greater improve-
ment that was sustained as of the final (9 mo) measure-
ment of cognitive functioning. Whether this difference in
measures of cognition translated into differences in func-
tioning in everyday life is not known. Because the impact
of the score transformations used was so profound (eg,
using raw scores reported in table 2 of the 2004 report,
after 3 mo of treatment the symptom scores for the algo-
rithm sites improved by about 5 points compared with
improvements of 6 points at comparison sites that
were implementing an algorithm for a disorder other
than schizophrenia and improvements of about 6 points
at other comparison sites), TMAP analyses may best be
considered unfinished. Further exploration of this valu-
able dataset could provide much needed information on,
eg, whether patients of prescribers who became more
guideline adherent over time showed greater improve-
ments in outcomes than did patients of prescribers
who remained less adherent. Given the large impact in
ity analyses showing the impact of various transforma-
tions could inform interpretation.
By design, the TMAP intervention included compo-
nents to support and encourage the use of the algorithms
in addition to the algorithms themselves. The TMAP
investigators recognized that passive diffusion of algo-
rithms was unlikely to change practice. The intervention
package at a site included the provision of a clinical co-
ordinator to work with physicians and families, treat-
ment manuals, expert consultations through conference
ized assessments of symptoms and side effects, as well as
the enhanced record keeping procedures that accompa-
nied use of the algorithms.31,32The understandable con-
founding of these intervention components means that
one cannot know the extent to which any of the individ-
ual components were responsible for TMAP’s findings.
What at first blush looks like an expensive intervention
(training across sites, implementing measurement tools
for prescribers to use to monitor changes over time,
and placement of a clinician in each site to work with
physicians to help them follow the guidelines) for a mod-
est impact may be too pessimistic a conclusion. No
doubt, adherence to the guidelines varied across prescrib-
ers, even within the algorithm sites. A stronger test of the
payoff of changing prescriber behavior to more closely
follow the treatment algorithm would be to determine
whether patients of psychiatrists who followed the algo-
rithm more closely had better outcomes than patients of
psychiatrists who followed the algorithms less closely.
This remains a challenging analysis for a multitude of
reasons. Nevertheless, knowing whether and to what ex-
tent efforts to enhance adherence to guidelines improves
patient outcomes is critical for policy makers who must
decide if they should go to the effort of implementing and
sustaining the use of such algorithms.
The zeitgeist surrounding TMAP prompted public and
private payers to identify questionable patterns of pre-
scribing and to intervene and monitor the impact of
such interventions. The Psychiatric Clinical Knowledge
Enhancement System in New York provides detailed cur-
rent medication regimens and histories at the patient, psy-
chiatrist, ward, and facility levels and has been used to
decrease rates of antipsychotic polypharmacy.33,34In an-
other effort aimed at increasing prescriber adherence
to evidence-based guidelines, a statewide longitudinal
project in Kentucky examined the impact of prescriber
training and systematic monitoring of medication prac-
tices, using Medication Management Approaches in
Psychiatry (MedMAP) (mentalhealth.samhsa.gov/cmhs/
communitysupport/toolkits/community/), which is an
adaptation of the TMAP schizophrenia module. The
Kentucky Project used the MedMAP Fidelity Scale35as
a quality improvement tool and documented modest im-
The last decade has also seen the emergence of phar-
macymanagementcompanies thatcontractwith employ-
ers to review patterns of utilization, establish preferred
drug formularies to steer patients toward/away from par-
concerns about the medication regimens used. The extent
to which steering is influenced by cost vs quality concerns
typically cannot be determined because the full list of
such managed care firms’ algorithms typically is propri-
etary, making it difficult for the payer or a neutral third
party to evaluate the attention paid to cost vs quality of
care by such algorithms. Firms may use polypharmacy as
an example of a clinically questionable practice their
guidelines target but offer few other specifics. While
physicians can be offended by receiving ‘‘Dear Pre-
scriber’’ letters from such pharmacy management firms
and suspect that the goal is cost containment as opposed
to quality improvement, one also can imagine situations
data to flag clinically questionable situations for review
(eg, no use of an antipsychotic medication following
R. E. Drake et al.
a hospitalization for schizophrenia). Transparency via
independent review and broad opportunity for public
comment helps quell understandable concerns that algo-
rithms could be promoting cost savings at the expense of
improving clinical outcomes. The business of promoting
adherence to particular practices is rife with controversy
as well as essential for improving care.37
Research on Implementing Psychosocial Interventions
uals with schizophrenia has a long history, dating back to
the 1970s with the failed effort to promote the use of com-
pitals.38In the late 1970s, the National Institute of Mental
Program principles to promote community integration of
individuals with serious mental illness.39Many of these
projects produced disappointing results, in part due to
poor model specification,40,41inadequate implementation
andinadequate leadership.44Manyprojectslaunched dur-
ing this era had naive assumptions about the minimum
requirements for effective implementation. The challenges
of implementing complex psychosocial interventions are
much greater than those for medication interventions.45
One positive outcome from the Community Support
of the need to define adherence to program models, which
in turn required clearly defined models. Building on the
work on treatment integrity in the psychotherapy litera-
ture,46,47researchers began developing fidelity measures,
defined as methods to assess adherence to the standards
of a program model.48Among the earliest of these efforts
nity treatment model.49–53Assertive community treatment
was the most clearly defined of the psychosocial models at
the time.54In the 1990s, supported employment began ac-
cumulating a strong evidence base, and a fidelity scale was
developed and validated for this model as well.55,56For
supported employment, fidelity has been found to be as-
sociated with higher competitive employment rates.57–63
For assertive community treatment, the correlation
between fidelity and improved outcomes has been less
onthe systematic studyofthe implementation process.In
2006, a new journal was established devoted to this en-
terprise (http://www.implementationscience.com). Ac-
cordingly, the literature on strategies and barriers to
implementing evidence-based practices has expanded
dramatically.45,67With regard to schizophrenia, recent
projects employing systematic implementation strategies
include the National Implementing Evidence-Based
Practices Project, the Mental Health Treatment Study,
the Department of Veterans Affairs (VA) dissemination
of supported employment, several learning communities,
and a variety of implementation studies outside the
In 1998, a national panel of experts convened by the
Robert Wood Johnson Foundation recommended that
5 psychosocial practices and systematic medication man-
ter.8The panel also recommended the development of
a systematic approach to dissemination. The National
Implementing Evidence-Based Practices Project was
launched to address the aforementioned deficiencies.68–72
The investigators hypothesized that implementation of
evidence-based practices in routine settings minimally
required comprehensive, user-friendly information about
the practices and their implementation—resources gener-
the first phase of the National Evidence-Based Practices
sisting of a variety of materials to facilitate practice imple-
mentation, such as practitioner workbooks, research
articles, introductory and instructional video, and Power-
Point lectures. The toolkits aimed at multiple stakehold-
ers, assuming that success depended on active support
from the state mental health authority, agency leadership,
practitioners, consumers, and family members.
The investigatorsalso hypothesized that,in addition to
the toolkits, another necessary component for successful
implementation was systematic training and consulta-
tion. Thus, the project researchers developed a training-
elements71: (1) consultation to the state mental health au-
ter administrators, (3) a kickoff presentation to the
provider agency, (4) provision of the evidence-based
practice toolkit to the agency, (5) initial skill training
for practitioners, (6) ongoing consultation to sites, and
cluded 2 steps: (a) fidelity assessments based on daylong
site visits by a pair of independent fidelity assessors who,
guided by fidelity scale criteria, conduct structured inter-
views, observe team meetings and interventions, and re-
viewmedicalrecordsand (b)fidelity reviewmeetingsheld
with agency leaders in which assessors give oral and writ-
ten feedback on quality of implementation.73
The second phase of the project was a field test of 5
psychosocial evidence-based practices in 53 sites in 8
ing clinical outcomes would have been prohibitively ex-
pensive; therefore, the study focused on fidelity scores at
baseline and 6-month intervals. Findings from this pro-
ject are now appearing in the literature.24,73–84
Overall, 29 (55%) sites showed high-fidelity implemen-
tation at 2-year follow-up.79Most sites achieving high
Evidence-Based Practices for Schizophrenia Patients
fidelity did so within the first year of implementation.
treatment and supported employment) were more readily
implemented than others. Based on qualitative data col-
lected within each site, several factors influenced quality
of implementation, including the state mental health
authority’s provision of leadership, funding, and practice
standards76,80,85; skilled mentoring by trainer-consul-
pervision at the site level24,74–77,81; systematic monitoring
of fidelity and outcomes73,78; and staff turnover.83
Mental Health Treatment Study
Insurance (SSDI) beneficiaries with schizophrenia or af-
supported employment, systematic medication manage-
ment, and other behavioral services. A nurse care coordi-
individualized and evidence-based. The experimental
group also receives supplemental insurance to cover
riences of the National Implementing Evidence-Based
Practices Project, a quality management team is monitor-
ing fidelity and providing technical assistance to the sites.
Many of the same themes have emerged from this project
as in earlier implementation studies. Barriers include the
maze of state bureaucratic regulations even in the face of
a well-funded project, the pivotal role of site leadership,
the challenges of integrating supported employment
with mental health treatment services, and staff turnover.
Department of VA
Like an earlier dissemination of the assertive community
Administration of the Department of VA to disseminate
supported employment are ambitious in their national
Organizational barriers include lack of
resources for supported employment, fragmentation of
services (with individual clinicians working indepen-
dently), and VA traditions and policies not aligned
with evidence-based supported employment. In addition,
longstanding sheltered and transitional work programs
within the VA conflict with the principles of supported
employment. This project depends on mostly remote su-
technical assistance and quality improvement efforts.90
Learning Community Approaches
care organizations have adopted a learning community
approach to quality, forming networks among key staff
from these organizations that share common values and
beliefs and are actively engaged in learning together
from each other.91Learning communities are especially
suited to promoting sustained adherence to a practice as
well as continuous quality improvement by creating
a culture of peer accountability and sharing among par-
ticipating organizations. One such application to schizo-
phrenia has been a network of state and local leaders
aimed at disseminating evidence-based supported em-
ployment.92First established in 2002, the network has
now grown to include participants from 10 states and
the District of Columbia. Using familiar strategies
such as annual meetings, newsletters, development of
videos, on-site trainings, bimonthly teleconferences,
and sharing data, this network has stimulated interest
in identifying innovative implementation strategies suc-
cessful in one state that are adapted for use in others.93,94
Program evaluation data suggest growing access to sup-
ported employment and achievement of site-level com-
petitive employment rates exceeding 40% for quarterly
A similar project in New York State, the Wellness
Self-management Program (a variation of Illness Man-
agement and Recovery95), also uses learning collabora-
tives.96Teams from participating sites meet to describe
implementation experiences and hear suggestions from
colleagues as well as a trainer-facilitator.
Wide-scale dissemination of evidence-based practices has
been attempted in Canada,97various European coun-
tries,98–100Australia,101and Japan.102Common barriers
found in these projects have included funding issues, lack
of cooperation among different services (eg, vocational
rehabilitation, mental health, and substance abuse treat-
ment), staff turnover, and insufficient time allocated for
Both the Institute of Medicine103and the New Freedom
Commission2recommended using modern information
technology to improve the quality of mental health
care in America. The United States lags behind other
Westernized countries in the use of health information
technology,104and the US mental health system lags be-
hind other areas of American medicine.11Several recent
nology can improve quality of health care.105,106A simple
example of psychiatric knowledge enhancement systems
is the use of programs that check for medication interac-
tions and incorrect dosages. Other common components
based practices; increasing practitioners’ awareness of
R. E. Drake et al.
evidence-based treatments and algorithms and of patients’
comes and side effects; and monitoring programs and sys-
systems, which address all these components, are just
emerging in mental health. The Veterans Administration
has adopted a clinically oriented electronic medical record
ical prompts, monitoring of care, and review of outcomes;
the system has recently added a patient portal (http://
www.myhealth.va.gov/). Enhancements to the Veterans
several European Union countries, a computerizedsystem
tocompare their perspectivesongoalsand negotiate plans
has demonstrated improvements in satisfaction, reduced
unmet need, and increased quality of life.110
Numerous innovative efforts are underway.111Deegan
support system that is used to facilitate shared decisions
during medication visits. Similar systems are being cre-
ated and tested within the Department of VA to support
Dartmouth PsychiatricResearchCenterto supporttreat-
ment planning, smoking cessation, employment services,
co-occurring disorders treatments, and care of comorbid
medical illnesses. The New York State Office of Mental
Health also is developing a consumer portal to support
decision making around medications.
Comprehensive electronic decision support systems
must be integrated into real-world contexts to be use-
ful.105,106Deegan et al112established a recovery resource
center within a routine community mental health center
by providing semiprivate computer kiosks, visual and
oral information, and peer supports. Patients and clini-
cians achieved high levels of participation and satisfac-
tion. Of course, much of current community mental
health work is done in the community rather than in clin-
ics. Demonstrating that electronic resources can be deliv-
ered in other forms, such as through health buddies, in
peer support centers, on the web, and on portable infor-
mation systems that case managers and other clinicians
carry with them, will be important.113
The Role of Government
In the United States, the role of state government in pro-
moting and inhibiting the growth of evidence-basedprac-
tices is enormous if not decisive.114–117The reasons are
obvious: State agencies, in conjunction with federal agen-
cies (notably the Centers for Medicare and Medicaid
Services) to a large extent determine what services are
funded or not funded. In most states, the state mental
health authority has sponsored statewide initiatives
to promote evidence-based practices. States have ag-
gressively promoted adoption of assertive community
treatment,76,80,118supported employment,93family psy-
choeducation,43integrated dual disorders treatment,119
and medication management approaches.30,33,36A pop-
ular but ineffective approach has been the sponsorship
of statewide conferences.120,121To address the enormous
costs of face-to-face meetings, some states are exploring
web-based options for training and ongoing supervision.
A comprehensive approach to enhancing broad dissem-
ination has often been centered in the formation of tech-
nical assistance centers providing consultation, training,
and fidelity monitoring.24,118,122,123
authority critically impacts the development of evidence-
based practices, most research to date has been anecdotal.
health authorityactions, suchasthedesignationofa point
person within the state agency responsible for dissemina-
tion, the establishment of a technical assistance center,
state-level policies and regulations aligned to support the
evidence-based practices, and provision of financial incen-
for the evidence-based practices in each state.85
To date, most studies of statewide implementation have
and other Hawthorne effects abound. Much less is known
regarding sustaining a statewide initiative, especially in the
face of the frequent leadership changes in senior staff
munity agencies, decisions about adopting evidence-based
practices are influenced by judgments about the perceived
riskiness of the uptake, anticipated resource availability,
and exposure to evidence.124,125Early adopters are willing
see the risks associated with adopting as lower and more
manageable than nonadopters.124,125
is driven in part by staff turnover126; hence, e-learning
ble can also help deal with this challenge.
for servicesresearch helped launch TMAP andother stud-
ies of the effectiveness of various treatment approaches.
That set-aside is long gone, but, as this article is being
written, the current version of the federal economic
stimulus package contains $400 million to the Agency
Evidence-Based Practices for Schizophrenia Patients
to stimulate comparative effectiveness research. As the
largest single payer for health care in the nation, the Fed-
eral government can make significant investments in nar-
rowing the quality chasm between what is known and
what is practiced. In theory, NIMH generates new knowl-
edge, and the Substance Abuse and Mental Health Serv-
ices Administration helps such knowledge make its way to
real people in real-world settings by promoting ways
across this chasm. Knowing how to promote such uptake
and where one’s efforts are best invested are themselves
pressing research issues. The Social Security Administra-
tion also has shown an interest in promoting evidence-
to identify a package of evidence-based interventions
to help SSDI beneficiaries with mental illness to return
to work.127The work of this panel led to the funding of
the Mental Health Treatment Study described earlier.
The Social Security Administration has also sought to in-
centivize employment services through its Ticket to Work
program.128The President’s New Freedom Commission
was an inspirational product of the executive branch of
the federal government, even if the executive branch
also required that the Commission’s recommendations
be revenue neutral. Revenue neutrality, especially in the
short term, may be incompatible with implementing
evidence-based practices unless a comparable amount of
ineffective services can be discontinued.
Summary and Conclusions
Implementation research in schizophrenia care has
followed the pattern observed in general health care18:
moving gradually from passive diffusion to system reen-
gineering based on complex electronic records, decision
supports, and Wagner’s26chronic care model. Simple
implementation efforts are often fruitless and waste
resources, while traditional continuous quality improve-
cessful. Complex reengineering of systems is needed.
However, public mental health systems are currently
mired in financing constraints (eg, Medicaid regulations
are not aligned with evidence-based practices), economic
survival problems (state budget problems have eroded
lems (currently many states have no electronic medical
records while others have electronic medical records
than quality of services and clinical outcomes). Public
mental health systems need better alignment between ev-
medical records to monitor process and outcomes, and
a systemic commitment to quality. The research commu-
effective services in order to improve the care of schizo-
phrenia in the United States.
1. U.S. Surgeon General. Mental Health: A Report of the Sur-
geon General—Executive Summary. Rockville, MD: U.S.
Department of Health and Human Services, Substance
Abuse and Mental Health Services Administration, Center
for Mental Health Services, National Institutes of Health,
National Institute of Mental Health; 1999.
2. New Freedom Commission on Mental Health. Achieving the
Promise: Transforming Mental Health Care in America. Final
Report. Rockville, MD: Substance Abuse and Mental
Health Services Administration, 2003. DHHS Pub No
3. Lehman AF, Kreyenbuhl J, Buchanan RW, et al. The
Schizophrenia Patient Outcomes Research Team (PORT):
updated treatment recommendations 2003. Schizophr Bull.
4. Miller AL, Crismon ML, Rush AJ, et al. The Texas Medica-
tion Algorithm Project: clinical results for schizophrenia.
Schizophr Bull. 2004;30:627–647.
5. Canadian Psychiatric Association Working Group. Clinical
practice guidelines: treatment of schizophrenia. Can J Psy-
6. Chambless D, Baker M, Baucom D, et al. Update on empir-
ically validated therapies, II. Clin Psychol. 1998;51:3–16.
7. Cook JA. Blazing trails: using evidence-based practice and
stakeholder consensus to enhance psychosocial rehabilita-
tion services in Texas. Psychiatr Rehabil J. 2004;27:305–318.
8. Drake RE, Merrens MR, Lynde DW, eds. Evidence-Based
Mental Health Practice: A Textbook. New York, NY: WW
Norton & Company; 2005.
9. National Institute for Health and Clinical Excellence. NICE
Clinical Guideline 82: Schizophrenia: Core Interventions in
the Treatment and Management of Schizophrenia in Adults
in Primary and Secondary Care 2009; www.nice.org.uk/
CG082. Accessed April 11, 2009.
10. Thornicroft G, Susser E. Evidence-based psychotherapeutic
interventions in the community care of schizophrenia. Br J
11. Mojtabai R, Fochtman L, Chang S, Kotov K, Craig TJ,
Bromet E. Unmet need for care in schizophrenia. Schizophr
Bull. 2009; doi:10.1093/schbul/sbp045.
12. Wang PS, Demler O, Kessler RC. Adequacy of treatment
for serious mental illness in the United States. Am J Public
13. Lehman AF, Steinwachs DM. Patterns of usual care for
schizophrenia: initial results from the Schizophrenia Patient
Outcomes Research Team (PORT) client survey. Schizophr
14. SAMHSA. 2005 National Survey on Drug Use and Health
2006;Rockville, MD: Substance Abuse and Mental Health
Services Administration, Office of Applied Studies.
15. Cunningham P, McKenzie K, Taylor EF. The struggle to
provide community-based care to low-income people with
serious mental illness. Health Aff. 2006;25:694–705.
16. NAMI. Grading the States: A Report on America’s Health
Care System for Serious Mental Illness 2006;Arlington, VA:
National Alliance on Mental Illness.
R. E. Drake et al.
17. Young GJ, Mohr DC, Meterko M, Siebert M, McGlynn G.
Psychiatrists’ self reported adherence to evidence-based pre-
scribing practices in the treatment of schizophrenia. Psy-
chiatr Serv. 2006;57:130–132.
18. Shojania KG, Grimshaw JM. Evidence-based quality im-
provement: the state of the science. Health Aff. 2005;
19. Deming WE. Out of Crisis. Cambridge, MA: Center for
Advanced Engineering Study, Massachusetts Institute of
20. Nadler DA. Feedback and Organization Development. Using
Data Based Methods. Reading, MA: Addison-Wesley, 1977.
21. Shannon HS, Robson LS, Sale JE. Creating safer and
healthier workplaces: role of organizational factors and
job characteristics. Am J Ind Med. 2001;40:319–334.
22. Gowdy E, Rapp CA. Managerial behavior: the common
denominators of successful community based programs.
Psychosoc Rehabil J. 1989;13:31–51.
23. Harkness DR, Hensley H. Changing the focus of social work
supervision: effects on client satisfaction and generalized
contentment. Soc Work. 1991;36:506–512.
24. Rapp CA, Etzel-Wise D, Marty D, et al. Evidence-based
practice implementation strategies: results of a qualitative
study. Community Ment Health J. 2008;44:213–224.
25. Todd WE, Nash D. Disease Management: A Systems Ap-
proach to Improving Patient Outcomes. San Francisco, CA:
26. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J,
Bonomi A. Improving chronic illness care: translating evi-
dence into action. Health Aff. 2001;20:64–78.
27. Wagner EH, Glasgow RE, Davis C, et al. Quality improve-
ment in chronic illness care: a collaborative approach. Jt
Comm J Qual Improv. 2001;27:63–80.
28. Wagner EH, Bennett SM, Austin BT, Greene SM, Schaefer
JK, Vonkorff M. Finding common ground: patient-
centeredness and evidence-based chronic illness care. J
Altern Complement Med. 2005;11(Suppl 1):S7–S15.
29. Coleman K, Mattke S, Perrault PJ, Wagner EH. Untangling
practice redesign from disease management. How do we best
care for the chronically ill? Annu Rev Public Health.
30. Miller AL, Chiles JA, Chiles JK, Crismon ML, Rush AJ,
ShonSP.The Texas Medication
(TMAP) schizophrenia algorithms. J Clin Psychiatry.
31. Toprac MG, Dennehy EB, Carmody TJ, et al. Implementa-
tion of the Texas Medication Algorithm Project Patient and
Family Education Program. J Clin Psychiatry. 2006;
32. Toprac MG, Rush AJ, Conner TM, et al. The Texas Medi-
cation Algorithm Project Patient and Family Education Pro-
gram: a consumer-guided initiative. J Clin Psychiatry.
33. Finnerty M, Altmansberger R, Bopp J, et al. Using state ad-
ministrative and pharmacy databases to develop a clinical
decision support tool for schizophrenia guidelines. Schizophr
34. Uttaro T, Finnerty M, White T, Gaylor R, Shindelman L.
Reduction of concurrent antipsychotic prescribing practices
through the use of PSYCKES. Adm Policy Ment Health.
35. Taylor AC, Bond GR, Tsai J, et al. A scale to evaluate qual-
ity of medication management: development and psycho-
February 27, 2009]. Adm Policy Ment Health. 2009; doi:
36. Howard PB, El-Mallakh P, Miller AL, et al. Prescriber fidel-
ity to a medication management evidence-based practice in
the treatment of schizophrenia. Psychiatr Serv. In press.
37. Tanenbaum SJ. Evidence-based practice as mental health
policy: three controversies and a caveat. Health Aff.
38. Fairweather GW. The Fairweather lodge: a twenty-five year
retrospective. New Dir Ment Health Serv. 1980;7:3–100.
39. Turner JC, TenHoor WJ. The NIMH community support
program: pilot approach to a needed social reform. Schiz-
ophr Bull. 1978;4:319–348.
40. Brekke JS. What do we really know about community sup-
port programs? Strategies for better monitoring. Hosp Com-
munity Psychiatry. 1988;39:946–952.
41. Noble JH. The Benefits and Costs of Supported Employment
for People With Mental Illness and With Traumatic Brain In-
jury in New York State. Buffalo, NY: Research Foundation
of the State University of New York; 1991.
42. Rosenheck R, Neale M, Leaf P, Milstein R, Frisman L.
Multisite experimental cost study of intensive psychiatric
community care. Schizophr Bull. 1995;21:129–140.
43. McFarlane WR, McNary S, Dixon L, Hornby H, Cimett E.
Predictors of dissemination of family psychoeducation in
community mental health centers in Maine and Illinois. Psy-
chiatr Serv. 2001;52:935–942.
44. Backer T, Liberman R, Kuehnel T. Dissemination and
adoption of innovative psychosocial interventions. J Consult
Clin Psychol. 1986;54:111–118.
45. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F.
Implementation Research: A Synthesis of the Literature.
Tampa, FL: University of South Florida; 2005.
46. Moncher FJ, Prinz RJ. Treatment fidelity in outcome stud-
ies. Clin Psychol Rev. 1991;11:247–266.
47. Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the in-
tegrity of a psychotherapy protocol: assessment of adherence
and competence. J Consult Clin Psychol. 1993;61:620–630.
48. Bond GR, Evans L, Salyers MP, Williams J, Kim HK. Mea-
surement of fidelity in psychiatric rehabilitation. Ment
Health Serv Res. 2000;2:75–87.
49. Brekke JS. The model-guided method for monitoring pro-
gram implementation. Eval Rev. 1987;11:281–299.
50. Essock SM, Kontos N. Implementing assertive community
treatment teams. Hosp Community Psychiatry. 1995;46:
51. McGrew JH, Bond GR, Dietzen LL, Salyers MP. Measur-
ing the fidelity of implementation of a mental health pro-
gram model. J Consult Clin Psychol. 1994;62:670–678.
52. Teague GB, Drake RE, Ackerson TH. Evaluating use of
continuous treatment teams for persons with mental illness
and substance abuse. Psychiatr Serv. 1995;46:689–695.
53. Teague GB, Bond GR, Drake RE. Program fidelity in asser-
tive community treatment: development and use of a mea-
sure. Am J Orthopsychiatry. 1998;68:216–232.
54. McGrew JH, Bond GR. Critical ingredients of assertive
community treatment: judgments of the experts. J Ment
Health Adm. 1995;22:113–125.
55. Becker DR, Swanson S, Bond GR, Merrens MR. Evidence-
NH: Dartmouth Psychiatric Research Center, 2008.
properties [publishedonline aheadof print
Evidence-Based Practices for Schizophrenia Patients
56. Bond GR, Becker DR, Drake RE, Vogler KM. A fidelity
scale for the individual placement and support model of sup-
ported employment. Rehabil Couns Bull. 1997;40:265–284.
57. Becker DR, Smith J, Tanzman B, Drake RE, Tremblay T.
Fidelity of supported employment programs and employ-
ment outcomes. Psychiatr Serv. 2001;52:834–836.
58. Becker DR, Xie H, McHugo GJ, Halliday J, Martinez RA.
What predicts supported employment program outcomes?
Community Ment Health J. 2006;42:303–313.
59. Catty J, Lissouba P, White S, et al. Predictors of employment
for people with severe mental illness: results of an interna-
tional six-centre RCT. Br J Psychiatry. 2008;192:224–231.
60. Gowdy EA, Carlson LS, Rapp CA. Practices differentiating
high-performing from low-performing supported employ-
ment programs. Psychiatr Rehabil J. 2003;26:232–239.
61. Hayward BJ, Schmidt-Davis H. Longitudinal Study of the Vo-
cational Rehabilitation Services Program Final Report: VR
Services and Outcomes (Report to the Rehabilitation Services
Administration Under ED Contract No. HR92022001). Re-
search Triangle Park, NC: RTI International; 2003.
62. McGrew J, Griss M. Concurrent and predictive validity of
two scales to assess the fidelity of implementation of sup-
ported employment. Psychiatr Rehabil J. 2005;29:41–47.
63. McGrew JH. IPS fidelity survey of 17 supported employ-
ment programs in Indiana: final report to SECT Center.
Indianapolis, IN: Indiana University-Purdue University
64. Burns T, Catty J, Dash M, Roberts C, Lockwood A,
Marshall M. Use of intensive case management to reduce time
and meta-regression. Br Med J. 2007;335:336.
65. Bond GR, Salyers MP. Prediction of outcome from the Dart-
mouth ACT Fidelity Scale. CNS Spectr. 2004;9:937–942.
66. McHugo GJ, Drake RE, Teague GB, Xie H. Fidelity to as-
sertive community treatment and client outcomes in the New
Hampshire Dual Disorders Study. Psychiatr Serv. 1999;
67. Bhattacharyya O, Reeves S, Zwarenstein M. What is imple-
mentation research? Rationale, concepts and practices. Res
Soc Work Pract. In press.
68. Drake RE, Goldman HH, Leff HS, et al. Implementing
evidence-based practices in routine mental health service
settings. Psychiatr Serv. 2001;52:179–182.
69. Mueser KT, Torrey WC, Lynde D, Singer P, Drake RE.
Implementing evidence-based practices for people with se-
vere mental illness. Behav Modif. 2003;27:387–411.
70. Torrey WC, Drake RE, Dixon L, et al. Implementing
evidence-based practices for persons with severe mental ill-
ness. Psychiatr Serv. 2001;52:45–50.
71. Torrey WC, Finnerty M, Evans A, Wyzik PF. Strategies for
leading the implementation of evidence-based practices. Psy-
chiatr Clin North Am. 2003;26:883–897.
72. Torrey WC, Lynde DW, Gorman P. Promoting the imple-
mentation of practices that are supported by research: the
National Implementing Evidence-Based Practice Project.
Child Adolesc Psychiatr Clin N Am. 2005;14:297–306.
73. Bond GR, Drake RE, McHugo GJ, Rapp CA, Whitley R.
National Evidence-Based Practices Project Research Group.
Strategies for improving fidelity in the National Evidence-
Based Practices Project. Res Soc Work Pract. In press.
74. Bond GR, McHugo GJ, Becker DR, Rapp CA, Whitley R.
Fidelity of supported employment: lessons learned from the
National Evidence-Based Practices Project. Psychiatr Reha-
bil J. 2008;31:300–305.
75. Brunette MF, Asher D, Whitley R, et al. Implementation of
integrated dual disorders treatment: a qualitative analysis of
facilitators and barriers. Psychiatr Serv. 2008;59:989–995.
76. Mancini AD, Moser LL, Whitley R, et al. Assertive commu-
nity treatment: facilitators and barriers to implementation
in routine mental health settings. Psychiatr Serv. 2009;60:
77. Marshall T, Rapp CA, Becker DR, Bond GR. Key factors
for implementing supported employment. Psychiatr Serv.
78. Marty D, Rapp CA, McHugo G, Whitley R. Factors influ-
encing consumer outcome monitoring in implementation of
evidence-based practices: results from the National EBP
Implementation Project. Adm Policy Ment Health. 2008;
79. McHugo GJ, Drake RE, Whitley R, et al. Fidelity outcomes
in the National Implementing Evidence-Based Practices Pro-
ject. Psychiatr Serv. 2007;58:1279–1284.
80. Moser LL, DeLuca NL, Bond GR, Rollins AL. Implement-
ing evidence based psychosocial practices: lessons learned
from statewide implementation of two practices. CNS
81. Rapp CA, Etzel-Wise D, Marty D, et al. Barriers to
evidence-based practice implementation: results of a qualita-
tive study. Community Ment Health J. In press.
82. Whitley R, Gingerich S, Lutz W. Facilitators and barriers to
the implementation of the illness management and recovery
program in community mental health settings. Psychiatr
83. Woltmann EM, Whitley R, McHugo GJ, et al. The role of
staff turnover in the implementation of evidence-based prac-
tices in mental health care. Psychiatr Serv. 2008;59:732–737.
84. Woltmann E, Whitley R. The role of staffing stability in the
implementation of integrated dual disorders treatment: an
exploratory study. J Ment Health. 2007;16:697–701.
85. Finnerty MT, Rapp CA, Bond GR, Lynde DW, Goldman
HH. The State Health Authority Yardstick (SHAY)
[published online ahead of print March 21, 2009]. Commu-
nity Ment Health J. 2009; doi: 10.1007/s10597-009-9181-z.
86. Frey W, Azrin S, Goldman HH, et al. The Mental Health
Treatment Study. Psychiatr Rehabil J. 2008;31:306–312.
87. Resnick SG, Rosenheck RA. Scaling up the dissemination of
evidence-based mental health practice to large systems and
long-term time frames. Psychiatr Serv. In press.
88. Resnick SG, Rosenheck R. Dissemination of supported em-
ployment in Department of Veterans Affairs. J Rehabil R D.
89. Rosenheck RA, Mares AS. Dissemination of supported em-
ployment for homeless veterans with psychiatric and/or ad-
diction disorders: two-year client outcomes. Psychiatr Serv.
90. Bond GR. Modest implementation efforts, modest fidelity,
and modest outcomes. Psychiatr Serv. 2007;58:334.
sons from research. Qual Saf Health Care. 2002;11:345–351.
92. Drake RE, Becker DR, Goldman HH, Martinez RA. Best
practices: the Johnson & Johnson—Dartmouth Community
Mental Health Program: disseminating evidence-based prac-
tice. Psychiatr Serv. 2006;57:302–304.
93. Becker DR, Lynde D, Swanson SJ. Strategies for state-wide
implementation of supported employment: the Johnson &
R. E. Drake et al.
Johnson—Dartmouth Community Mental Health Program. Download full-text
Psychiatr Rehabil J. 2008;31:296–299.
94. Becker DR, Baker SR, Carlson L, et al. Critical strategies
for implementing supported employment. J Vocat Rehabil.
95. Mueser KT, Corrigan PW, Hilton DW, et al. Illness man-
agement and recovery: a review of the research. Psychiatr
96. Institute for Healthcare Improvement. The Breakthrough
Series: IHI’s Collaborative Model for Achieving Break-
through Improvement. IHI Innovation Series White Paper
(www.IHI.org). Boston, MA: Institute for Healthcare Im-
97. Corbie `re M, Bond GR, Goldner E, Ptasinski T. The fidelity
of supported employment implementation in Canada and
the United States. Psychiatr Serv. 2005;56:1444–1447.
98. Boyce M, Secker J, Floyd M, Schneider J, Slade J. Factors
influencing the delivery of evidence-based supported em-
ployment in England. Psychiatr Rehabil J. 2008;31:360–366.
99. Rinaldi M, Perkins R. Implementing evidence-based sup-
ported employment. Psychiatr Bull. 2007;31:244–249.
100. van Erp NH, Giesen FB, van Weeghel J, et al. A multisite
study of implementing supported employment in The
Netherlands. Psychiatr Serv. 2007;58:1421–1426.
101. Waghorn G, Collister L, Killackey E, Sherring J. Challenges
to implementing evidence-based supported employment in
Australia. J Vocat Rehabil. 2007;27:29–37.
102. Ito J, Oshima I, Nishio M, Kuno E. Initiative to build
a community-based mental health system including assertive
community treatment for people with severe mental illness in
Japan. Am J Psychiatr Rehabil. In press.
103. Institute of Medicine. Improving the Quality of Health Care
for Mental and Substance-Use Conditions. Washington, DC:
National Academies Press, 2006.
On the front lines of care: primary care doctors’ office
systems, experiences, and views in seven countries. Health
105. Dorr D, Bonner L, Cohen A, et al. Informatics systems to
promote improved care for chronic illness: a literature re-
view. J Am Med Inform Assoc. 2007;14:156–163.
106. Garg AX, Adhikari NK, McDonald H, et al. Effects of com-
puterized clinical decision support systems on practitioner
performance and patient outcomes: a systematic review.
J Am Med Assoc. 2005;293:1223–1238.
107. Drake RE, Teague GB, Gersing K. State mental health au-
thorities and informatics. Community Ment Health J. 2005;41:
108. Owen RR. Department of Veterans Affairs Quality Enhance-
ment Research Initiative Annual Report. Little Rock, AR:
Center for Mental Healthcare and Outcomes Research; 2008.
109. Chinman M, Hassell J, Magnabosco J, Nowlin-Finch N,
Marusak S, Young AS. The feasibility of computerized pa-
tient self-assessment at mental health clinics. Adm Policy
Ment Health. 2007;34:401–409.
110. Priebe S, McCabe R, Bullenkamp J, et al. Structured patient-
clinician communication and 1-year outcome in community
mental healthcare. Br J Psychiatry. 2007;191:420–426.
Dartmouth partnership on shared decision making. Psychiatr
112. Deegan PA, Rapp C, Holter M, Riefer M. Best practices:
a program to support shared decision making in an
outpatient psychiatric medication clinic. Psychiatr Serv.
113. Ruland CM. Handheld technology to improve patient care:
evaluating a support system for preference-based care plan-
ning at the bedside. J Am Med Inform Assoc. 2002;9:192–201.
114. Ganju V. Implementation of evidence-based practices in
state mental health systems: implications for research and ef-
fectiveness studies. Schizophr Bull. 2003;29:125–131.
115. Isett KR, Burnam MA, Coleman-Beattie B, et al. The state
policy context of implementation issues for evidence-based
practices in mental health. Psychiatr Serv. 2007;58:914–921.
116. Isett KR, Burnam MA, Coleman-Beattie B, et al. The role of
state mental health authorities in managing change for the
implementation of evidence-based practices. Community
Ment Health J. 2008;44:195–211.
117. Rapp CA, Bond GR, Becker DR, Carpinello SE, Nikkel
RE, Gintoli G. The role of state mental health authorities
in promoting improved client outcome through evidence-
based practice. Community Ment Health J. 2005;41:347–363.
118. Bjorklund RW, Monroe-DeVita M, Reed D, Toulon A,
Morse G. Washington State’s initiative to disseminate and
119. Wilson DC, Crisanti AS. Psychometric properties of the
Dual-Disorder Treatment Fidelity Scale: inter-rater reliability
and concurrent validity [published online ahead of print
October 11, 2008]. Community Ment Health J. 2008; doi:
120. Davis D, Thomson O’Brien M, Freemantle N, Wolf FM,
Mazmanian P, Taylor-Vaisey A. Impact of formal continu-
ing medical education: do conferences, workshops, rounds,
and other traditional continuing education activities change
physician behaviors or health care outcomes? J Am Med
121. Salyers MP, Rollins AL, McGuire AB, Gearhart T. Barriers
and facilitators in implementing illness management and
recovery for consumers with severe mental illness: trainee
perspectives [published online ahead of print]. Adm Policy
Ment Health. 2009;36:102–111.
122. Biegel DE, Swanson S, Kola LA. The Ohio Supported Em-
ployment Coordinating Center of Excellence. Adm Soc.
123. Salyers MP, McKasson M, Bond GR, McGrew JH, Rollins
AL, Boyle C. The role of technical assistance centers in
implementing evidence-based practices: lessons learned.
Am J Psychiatr Rehabil. 2007;10:85–101.
124. Panzano PC, Roth D. The decision to adopt evidence-based
and other innovative mental health practices: risky business?
Psychiatr Serv. 2006;57:1153–1161.
125. Seffrin B, Panzano PC, Roth D. What gets noticed: how
barrier and facilitator perceptions relate to the adoption
and implementation of innovative mental health practices.
Community Ment Health J. 2008;44:475–484.
126. Massatti RR, Sweeney HA, Panzano PC, Roth D. The de-
adoption of innovative mental health practices (IMHP):
why organizations choose not to sustain an IMHP. Adm
Policy Ment Health. 2007;35:50–65.
127. Aron L, Burt M, Wittenburg D. Recommendations to the So-
cial SecurityAdministration on the Design of the MentalHealth
Treatment Study. Washington, DC: Urban Institute; 2005.
128. Livermore GA, Goodman N, Wright D. Social Security dis-
ability beneficiaries: characteristics, work activity, and use of
services. J Vocat Rehabil. 2007;27:85–93.
Evidence-Based Practices for Schizophrenia Patients