Evidence-based Practice Implementation Strategies:
Results of a Qualitative Study
Charles A. Rapp Æ Æ Diane Etzel-Wise Æ Æ Doug Marty Æ Æ
Melinda Coffman Æ Æ Linda Carlson Æ Æ Dianne Asher Æ Æ
Jennifer Callaghan Æ Æ Rob Whitley
Received: 30 November 2006/Accepted: 21 September 2007/Published online: 1 November 2007
? Springer Science+Business Media, LLC 2007
based practices of supported employment and integrated dual diagnosis treatment.
Using qualitative research methods, the study uncovered eight strategies that con-
tributed to successful implementation in six sites.
This study reports on the strategies used to implement the evidence-
Mental health administration
Evidence-based practice ? Implementation strategies ?
Evidence is overwhelming that people with severe mental illness infrequently
receive effective services in community mental health centers (CMHCs) (U.S.
Department of Health and Human Services 1999; President’s New Freedom
Commission on Mental Health 2003; Lehman et al. 1998; West et al. 2003;
Tashjian et al. 1989). Torrey and Gorman (2005) note that lack of knowledge
regarding successful implementation of proven interventions in CMHCs inhibits the
development of effective service provision. The National Implementing Evidence-
Based Practice (EBP) Project was designed to explore the facilitating conditions,
barriers, and strategies that affected implementation of EBPs in CMHCs (McHugo
et al. in press). The project involved 53 sites in eight states with each CMHC
implementing one of the following EBPs: Supported employment (SE), integrated
dual diagnosis treatment (IDDT), family psychoeducation, illness self-management,
or assertive community treatment. The implementation intervention lasted 2 years
with the first year being devoted to implementation and the second to sustaining.
C. A. Rapp (&) ? D. Etzel-
Wise ? D. Marty ? M. Coffman ? L. Carlson ? D. Asher ? J. Callaghan ? R. Whitley
School of Social Welfare, The University of Kansas, Twente Hall, Lawrence, KS 66044, USA
Community Ment Health J (2008) 44:213–224
A previous report described the barriers to implementation of IDDT and SE in
five CMHCs in Kansas (Rapp et al. in press). Three significant barriers emerged: (1)
Deficits in skills and role performance by front-line supervisors; (2) resistance by
front-line practitioners (3) failure of other agency personnel (e.g., medical staff) to
adequately fulfill new responsibilities. This article focuses on strategies employed to
overcome these barriers and achieve successful implementation outcomes in the five
Kansas CMHCs involved in the project.
The Kansas’ Department of Social and Rehabilitation Services (SRS), the umbrella
agency for the state mental health authority, issued a request for applications from
CMHCs interested in implementing either IDDT or SE. The SRS Commissioner of
Mental Health selected five CMHCs according to the guidelines presented by the
National Project oversight committee (one site offered both EBPs). This included a
mix of rural and urban sites and commitment of agency leadership. Each CMHC
created a leadership team consisting of the CMHC executive director, Community
Support Services director, program leader, consumers, families, and a state
representative. On the ground, the implementation team consisted of a program
leader (PL) and 3–6 direct service staff. This team was assisted by a Consultant and
Trainer (CAT) assigned to the site as the principle support for implementation.
Implementation data were collected over 2 years by independent implementation
monitors (IMs) during monthly site visits where trainings, leadership meetings,
team meetings, and everyday activities were systematically observed. The IM also
conducted structured interviews with the PL and CAT every 6 months. The CAT
also made written observations of the implementation process on a monthly basis.
IMs and CATs met regularly to share observations, notes and impressions. In order
to enhance trustworthiness and credibility (Lincoln and Guba 1985), IMs
participated in monthly conference calls with the National Coordinating Centre
(Dartmouth Psychiatric Research Centre) and attended annual meetings for training
and feedback. Similarly, IMs submitted examples of collected data to the National
Centre on a monthly basis for review of appropriate scope and quality.
Formal fidelity reviews were conducted at 6-monthly intervals from baseline to
24 months by the IM and CAT. These reviews assessed the CMHCs degree of
adherence to the practice model of the EBP. This practice model was given to the
leadership team of each CMHC as a gold standard to which they should aspire to
reach (Bond et al. 2000). Detailed protocols directed the reviewers inquiry. Fidelity
scores were based on interviews with consumers, staff and administrators,
observation of practice and team meetings, and reviews of a sample of case
214 Community Ment Health J (2008) 44:213–224
records. This information was then used to rate the individual fidelity items on a
five-point scale. The CAT and the IM independently rated levels of practice fidelity
before a composite score was calculated.
Firstly the raw data for each site was organized into three categories: Facilitating
conditions, strategies, and barriers. Facilitators referred to pre-intervention condi-
tions assisting EBP implementation. Strategies referred to intentional actions
assisting EBP implementation. Barriers referred to activity inhibiting EBP
implementation. Secondly, themes were developed within 26 a priori deductively
formulated categories such as ‘‘staffing,’’ ‘‘attitude,’’ and ‘‘engagement.’’ A theme
was defined as a thread of activity or condition that was related to the category and
considered ‘‘salient, prominent, conspicuous, or non-ignorable.’’ Key stakeholders
involved with the theme were also identified. These coding processes occurred
within Atlas-ti qualitative software (Atlas.ti 2002). On completion of the
intervention (24 months), the IM wrote a comprehensive site reports describing
the process and outcome of implementation (24 months).
For this paper, the six site reports were analyzed inductively by three
investigators, who were blinded to the titles of the 24 a priori themes described
in the reports. The goal was to develop a set or category of emergent strategies that
adequately organized and accounted for the data in its local context. This was done
iteratively as perceived flaws and inadequacies in a category led to ongoing
reformulations of the emergent schemata (Strauss and Corbin 1990). Once a
provisional coding guide was established, one site was selected and two analysts
independently re-coded data relevant to strategies. In conjunction with the primary
investigator, the results were compared. Through ongoing discussion, codes were
modified and a coding guide established. This was then applied to the data of all six
sites. Part of this process involved returning to the raw data or the IMs/CATs in
order to clarify, inform, and elaborate on ambiguous data.
Five strategies emerged from the data (see in Table 1). These are described
separately in this section.
Evidence-based practices impose a new set of expectations on front-line staff. These
include increased documentation, the acquisition, and use of new skills (e.g.,
motivational interviewing), and more sophisticated time-management. There were
varying degrees of acceptance/resistance among staff to these new expectations,
ranging from enthusiasm to overt hostility. However positive actions by front-line
Community Ment Health J (2008) 44:213–224215
supervisors and upper-management tended to overcome hostility to new expecta-
tions. In the beginning, supervisors in five of the six sites resisted significant pieces
of the EBP and appeared to be weak leaders. In each case, expectations of front-line
staff were not set and/or not monitored and enforced. Implementation was sluggish
until new supervisors were hired. In contrast, one site had a tradition of expectation-
setting and concomitant staff enthusiasm to meet these expectations. This site had
quicker and more successful implementation outcomes.
In cases of practitioner resistance to new EBP expectations, a combination of
education and persuasion were used first to engender acceptance. PLs and CATs
provided information, research results, and case vignettes to practitioners. They met
individually with practitioners to address concerns. In conjunction with upper
administration, they would convey enthusiasm about the EBP. This was generally
effective with those who were simply skeptical about the EBP. However, this
strategy was ineffective on those who were opposed outright to the EBP. Progress
was only made with such recalcitrant individuals when the supervisor and/or upper
management stated ‘‘get in line or take another position.’’ Some practitioners quit,
others transferred to other positions in the agency (often quite successfully), others
were counseled to find other employment and some were fired. New hires were
people who accepted the expectations and were judged to have the necessary skills
to develop in the post. Without upper management intervention as the ‘‘ultimate
enforcer,’’ implementation of the EBP’s would likely have been slower and less
The clarity and conscientious monitoring of expectations seemed to go together.
PLs who set behavioral standards for practice (e.g., % of time in community) and
outcomes (e.g., number of consumer employed) were also most likely to closely
monitor performance through the use of data, field supervision, review of
documentation and team meetings. They shared performance information, provided
feedback, and strategized avenues of improvement. They also sought to alter wider
barriers to performance like agency productivity policies. Similarly, they induced
cooperation from others in the agency, for example getting psychiatrists more
closely involved with IDDT. In general, things moved slower for IDDT than SE, as
the supervisor was often learning the practice at the same time as the front line staff.
Table 1 Strategies employed
1 Instituting expectations
2 Unique role of upper management
Championing the EBP
Changing structure and policy
3 Creating intra-agency synergy
Integrative structure: Leadership teams
Integrative structure: EBP team meetings
4 Managing information: Fidelity measurement
5 Training and consultation
216 Community Ment Health J (2008) 44:213–224
Unique Role of Upper Management: Championing The Ebp
In each site, a champion of the EBP was present. In two sites, the principal
champion was the executive director of the agency. One of these (SE) was located in
a large urban CMHC, the other (IDDT) a small, rural one. The urban center’s
executive championed the practice by frequently expressing commitment to high
fidelity implementation. The executive communicated confidence that the CMHC
would succeed and her unwillingness to accept second-best, asking pertinent
questions and ensuring relevant answers (e.g., ‘‘Why can’t we do this?,’’ ‘‘How are
we going to do it?’’). The director modeled the ‘‘can-do’’ approach by making
supportive decisions, for example transferring, upgrading or adding positions. When
agency personnel became distracted by other demands, she would re-focus the team
back to the EBP.
The rural site executive drove the entire implementation. This executive attended
and was active in the Leadership Team meetings, frequently expressing commit-
ment to the EBP, confidence in staff, and expectations that implementation would
be a success. The executive also made supportive decisions such as reallocation of
resources and adjustment of policies. When interest in IDDT waned among staff, the
executive reestablished expectations and took measures to boost staff confidence,
for example attending staff training as a participant. Additionally, the Executive
Director, the Community Support Services (CSS) Director and the Substance Abuse
Director all visited an out-of-state recommended IDDT program along with the
CAT to learn more about successful implementation. At one point, the executive
director presented the results of a fidelity review to the Board of County
Commissioners in order to sustain support for implementation.
In three sites (2-SE; 1-IDDT), the executive director was only occasionally
involved but authority for implementation was delegated to the CSS Director, who
became the principle champion. Each of these sites had rather weak project
supervisors and some front-line practitioners who were hostile toward implemen-
tation. In two cases, the CSS Directors instigated or supported the transfer or firing
of recalcitrant workers, as well as the replacement of supervisors with more able and
committed people. In one site, the CSS director was instrumental in making many
positive changes, but was unwilling to assertively intervene with an under-
performing supervisor. This project attained the lowest fidelity score of the six sites.
In all cases, EBP champions allocated necessary resources, set expectations, and
helped remove obstacles to implementation. They became ‘‘partners’’ with the CAT
to ensure successful implementation.
Unique Role of Upper Management: Structural and Policy Change
Two categories of structure/policy were targeted by the EBP Project. First both
IDDT and SE fidelity ratings contained items that were structural in nature, in
contrast to service or clinical items. In general, structural items refer to the
organization of service delivery for the EBP practice. Examples for SE would
include fidelity items concerning caseload size, integration with mental health
Community Ment Health J (2008) 44:213–224217
services, and specification of a vocational specialist’s job. For IDDT, examples
would include the composition of the multidisciplinary team and the expectation
that services are delivered in an outreach mode. IDDT contains three structural
items and SE contains six. Second, sites confronted obstacles to effective EBP
services that emanated from their agency, catchment area or state system. In some
cases, structural/policy remedies were pursued.
The Kansas SE-EBP sites started the project already high in fidelity to the six
structural items, with scores of 3.6, 3.8, 4.0 (Max = 5, Min = 1). These scores
further increased during the 2 years of the project to 4.6; 4.6; 5.0. Biggest gains
occurred during the initial 6 months. These changes were reflective of upper
management’s commitment to implementation, as only they had the authority to
make the necessary changes. In fact, once notified of fidelity criteria, all three sets of
upper management initiated the necessary changes. For example, in all three SE
sites, a decision was quickly made that employment specialists should be assigned
to specific case management teams. Their caseload was simultaneously limited to
consumers from that team, and they were required to attend all case management
team meetings. Other facilitative policy changes were: Allowing indefinite follow-
along supports, discontinuing pre-vocational service and entry to SE based solely on
expressed desire to work.
The three IDDT structural items at baseline averaged 4.0, 2.25, 3.5. The scores
increased over the 2 years of the project to: 5.0, 4.25, 4.0. For one site, the biggest
improvement occurred during the initial 6 months. This was the only site with an
executive director who was integrally involved in EBP implementation from day
one. The other two sites saw most increase between 6 and 12 months. The most
notable improvement in both concerned the structuring of a multidisciplinary team
that included an integrated substance abuse specialist. Other facilitative policy
changes were: Additional supervisory meetings that included the psychiatrist and
were focused on DD consumer, removal of limits on substance abuse counselors’
time to serve consumers, the assignment of substance abuse specialist to team
meetings and reduced caseloads to 15:1. The fact that each agency already had an
SE program where no agency had an IDDT program at baseline may explain the
time differential in making structural changes by EBP. Two-thirds of the sites
adjusted their standards for billable hours by lowering them for the EBP personnel.
This allowed employment specialists to increase their time for job development and
for IDDT workers to attend on-going training to develop IDDT skills.
No Program is an Island: Creating Intra-Agency Synergy
For a given EBP to reach high fidelity, agency personnel, beyond the staff targeted
for EBP implementation, are required to practice in certain ways. Three strategies
were used by the six EBP sites to ensure other personnel fulfilled this requirement.
The first was education and training. This involved activities such as general
orientation to the practice and training specific to the behaviors necessary for
successful EBP implementation. For IDDT, this included training medical staff on
best-practice prescribing and follow-up consultations for dually diagnosed
218 Community Ment Health J (2008) 44:213–224
consumers, and educating substance abuse staff on IDDT philosophy and methods.
For SE this included showing how work can be effective ‘‘therapy’’ for many
consumers (a special training was designed for case managers) and educating
therapists on the benefits of work and the research that work does not necessarily
increase stress for consumers. Often the education and training occurred through the
joint activity of CATs, PLs, and upper management; they made positive comments
about the EBP and shared relevant information widely.
The second strategy was consultation and modeling. Here the CAT or PL would
intervene in a particular situation where the EBP practitioner was having difficulty
influencing other agency personnel to practice in a way consistent with the EBP. For
example, specific case managers, medical staff or therapists who would not support
a consumer’s desire to work were targeted for intervention. On occasion, the CAT
would model complex skill sets. For example, in SE employment specialists were
each assigned to a specific case management team to enhance integration. While the
specialists dutifully attended, they were often passive. The employment specialists,
the CM’s, and CM supervisor were not clear on their role and contribution in the
meeting. The CAT or supervisor attended the case management team meetings and
modeled appropriate behaviors; for example intimating ideas about work being a
solution to a particular consumer problem or asking if consumers had ever talked
about work. The IDDT CATs ability to problem-solve during case reviews made a
strong impression on many practitioners and aided in the acceptance of IDDT.
The third strategy involved structural changes to enhance on-going participation
of agency personnel. In many cases, they were made members of the leadership
team. In IDDT, the integrated treatment team was created (or altered) to involve
psychiatrists, nurses, psychosocial staff, substance abuse counselors, and SE
workers. In SE, one site integrated the benefits’ specialists in team meeting. At
times, this involved changing meeting times or altering the structure so that
participation could occur. As participation increased, knowledge and skills
increased, and more consistent agency-wide practice occurred.
Integrative Structure: Leadership Teams
The National EBP Project recommended the creation of a steering committee at
each site. In Kansas, they were called ‘‘leadership teams.’’ These teams brought a
variety of key actors together to improve EBP implementation. Membership at all
sites included the agency executive director (or designee), CSS director, team
supervisor, a representative of the direct service workers, consumer representative,
state field support service person and CAT. At IDDT sites, a substance abuse
counselor was part of this team as was the Director of Substance Abuse services. In
one IDDT site, the psychiatrist was a member. For two of these sites, the Director of
SA services also served as co-program leader of the implementation team. The SE
sites all had the Vocational Rehabilitation Chief from the region on the team. In one
site, all members of the SE team were members of the leadership team. In no case,
were the sites successful in recruiting a family member for regular involvement.
Community Ment Health J (2008) 44:213–224219
Leadership teams oversaw EBP implementation. The CAT led the meetings for
the first year, then an agency person took over. As such, they monitored
implementation through frequent reports by the PL and CAT, and fidelity reviews.
They made decisions concerning training, ensuring the necessary EBP structural
features were adopted such as changes regarding job duties, caseload size and
documentation. They also ensured adequate resources were available for imple-
mentation, as well as aligning other agency practices to be consistent and supportive
of the EBP. Finally, they formulated solutions to obstacles as they arose. For
12 months, the leadership team met monthly, before moving to bi-monthly or
There seemed to be a common evolution to the leadership teams. Teams were
constituted 4–6 months before implementation kick-off. The first phase served as a
principal mechanism by which the CAT could educate staff about the EBP, thereby
boosting the agency’s commitment to change. The second phase focused on
planning and sponsoring the kick off events, and training of staff. The third phase
was dominated by structural and personnel changes. The fourth phase focused on
overcoming barriers to implementation. The final phase was preoccupied with
developing the mechanisms for sustaining the EBP. These ‘‘phases’’ should not be
conceptualized as homogenous time-periods, but rather points of emphasis during
The leadership teams were a critical component of successful implementation. As
all the key stakeholders were involved in the team and the team was the prime
decision-making mechanism, decisions were easily supported and defended agency-
wide. Decisions made were decisions executed, in a timely and efficient manner.
This stands in contrast to the more typical pattern of a recommendation being
passed up various levels of the organization whereby each level has de facto veto
power, leading to costly and time-consuming revision and indecision.
Integrative Structure: Ebp Team Meetings
While the leadership team was the core mechanism for administrative decision-
making, team meetings in conjunction with group supervision (Rapp and Goscha
2006) were the key integrative mechanism at the service delivery level. The central
task of these meetings was the generation of promising ideas to work more
effectively with clients through the skilled use of EBP practices. This was done
through reviews of case situations where progress has not been occurring. Typically
each case situation consumes 20–30 min and 2–4 cases can be covered during a 2-h
Group supervision was a critical vehicle for improving EBP fidelity. It was the
setting where supervisors and CAT’s helped direct services workers apply the
clinical skills to the myriad of idiosyncratic case situations. All supervisors were
trained and tutored in leading group supervision and the CAT modeled the practice.
The consistent presence of the CATS in team meetings was vitally important in this
regard. Two SE supervisors (the two that replaced earlier ones) quickly became
proficient in providing the necessary structure and expectations for group
220 Community Ment Health J (2008) 44:213–224
supervision. By 24 months, each EBP supervisor was operating well-run sessions,
though hitches were encountered on the way at some sites. One SE supervisor took
the better part of the 24-months to develop sufficient discipline. For example, case
reviews often did not follow the guidelines and therefore evolved into unfocused
unproductive discussions. In other cases, group supervision time would be
consumed discussing administrative matters. This was in a large part due to the
team leader’s inability to provide expectations and structure for the team meeting.
Managing Information: Fidelity Measurement
‘‘Managing information’’ refers to the design and use of information (data) for
monitoring, goal-setting, and evaluation, with the aim of instigating management
action and program improvement. Fidelity measures developed for SE and IDDT
were the principle means for assessing the implementation of the key elements of
the service. As previously stated, fidelity was assessed by the CAT and IM every
6 months from baseline to 24 months. The CAT would write a report after each
assessment based on this data that included fidelity ratings, supporting evidence and
recommendations for next steps. This was then presented to the leadership team and
staff. The ratings were trusted by the agencies as accurate, were used to set goals,
celebrate successes, and make plans for improving the scores.
The fidelity measures required information that was not previously collected at
the CMHC. For IDDT, examples would include consumers in the action or relapse
prevention stage receiving substance abuse counseling, and those participating in
self-help groups. For SE, examples would include employment specialist time in
community, number of job development contacts, date of first employer contact, and
type and location of employment. In this way, the fidelity measures provoked
systematic data collection on critical service variables not previously collected.
There is evidence that the collection of data, under certain conditions, can generate
energy around the activity being measured (Nadler 1977; Taylor et al. 1984; Taylor
1987). A few supervisors were adept at using this information to monitor
components, giving appropriate feedback to staff to improve performance.
Training and Consultation
Training and consultation was a core component of the implementation intervention
package. The CATS devoted 2 years to working with each EBP site, visiting for
consultation bi-monthly during the first year. These diminished to monthly visits
during the second year. Training employed lectures, power-point slides, role-
playing, discussion, and application of methods to specific case situations. In IDDT,
emphasis was placed on assessing stages of change, motivational interviewing
skills, substance abuse counseling skills, and other IDDT related assessments. The
CAT also supported study groups where IDDT practitioners could practice skills. In
team meetings, the CAT would help practitioners present case situations using the
stages of change model. The CAT began field mentoring when practitioners
Community Ment Health J (2008) 44:213–224221
conducted IDDT case management. The CAT would observe, give feedback, and at
times model the IDDT skills.
In SE, emphasis was placed on using the vocational profile, follow-along
supports, individualized job search and job development. Training was provided on
using motivational interviewing as a means of engagement. Job development, the
work one does with an employer to help a consumer obtain a job, was new and
uncomfortable for many practitioners. The CAT arranged for a special workshop by
a renowned expert in the area. Even with this extra training and consultation, job
development skills remained difficult for some practitioners to master. The CAT
began to do mentoring where she would go with practitioners as they did job
development with employers in the field. The CAT would observe, give feedback,
and model the job development skills. It became clear that the CAT nor the
supervisor had a clear or complete picture of the practitioners job development work
in the field. The field mentoring was a major breakthrough in teaching skills and
increasing the comfort level of employment specialists. Although naturally reluctant
at first to have their practice observed, after a short period of time most perceived of
it as helpful.
After our training in job development, our supervisor set the expectation that
we should be doing job development and set a certain number of contacts that
we should be having with employers each month. I didn’t really do it. I didn’t
like to do job development and did not feel comfortable doing it. My
supervisor said she wanted to help us be more effective with our job
development and would be going out regularly with us to talk with employers.
At first, I was really upset and did not want to do it. After a while, though, I
realized that I was feeling more comfortable with talking with employers and
that my supervisor was really helping me do it better. I really liked it when I
started getting clients jobs through my contact with employers. I realize now
how helpful it was and how much better I am at job development.
For both SE and IDDT, CATs devoted time to instruction in proper documen-
tation. This included work on goal-setting and treatment planning and doing quality
assessments. Part of the impetus concerning good practice in this regard was to
ensure that the EBP service would meet Medicaid documentation standards and
thereby be reimbursable.
The CATs also used visits to ‘‘model’’ programs in other states. One SE site was
convinced that the impoverished area they served prevented consumers from
obtaining employment. Representatives from the site visited an SE program in
Hartford, Connecticut which helped dissuade them from their perspective. For
IDDT, representatives of all three sites visited an exemplary program in Ohio.
Supervisors attended all of the practitioner training and a 2-day supervisor
training. The supervisor training covered the following topics: (a) Managing by
consumer outcomes (b) how to use data to help supervise (c) methods for enhancing
fidelity (d) giving feedback (e) group supervision methods for team meetings (f)
creating a reward—Based environment (g) transfer of training & learning to actual
practice. The CATS did a lot of modeling and job coaching in running a team
meeting, practicing skills with front-line staff and conducting effective case
222 Community Ment Health J (2008) 44:213–224
reviews. Each EBP had quarterly project leader meetings (Supervisor Forum) with
the CAT to review programs, share innovations and problem-solve.
The analysis uncovered specific tools (e.g., fidelity measurement), mechanisms
(e.g., leadership teams, focused team meetings) and methods (e.g., field mentoring,
modeling, and group supervision) that contributed to successful implementation of
the EBPs in Kansas. The findings provide further specificity in the behaviors
necessary. For example, rather than ‘‘leadership,’’ expectations could include
instituting and enforcing expectations, willingness to transfer staff, making EBP-
specific structural changes, etc. The overall impression from the data was that it was
the confluence of tools, methods, mechanisms that produced successful EBP
implementation rather than any one factor.
This analysis and a previous one on barriers (Rapp et al. in press) also
underscores the critical role of quality supervision that seems to transcend specific
methods or tools. Although the role of program leaders (front-line supervisors) is
rarely studied in the implementation literature, the context and requirements of EBP
implementation may be somewhat unique. Both IDDT and SE have clinical
elements that direct service staff are to carry out with skill. This direct service staff
are virtually all B.A. level with limited experience with people with psychiatric
disabilities and who receive very little training prior to assuming their responsi-
bilities. In this situation, the onus of implementation of clinical skills falls on the
supervisor who teaches, guides, monitors, and enforces skill development and use.
Within this context, group supervision and field mentoring become important
mechanisms for discharging this responsibility.
The positive influence of fidelity measurement had on implementation is
consistent with the decades of research demonstrating that the collection and
feedback of information affects human behavior and system change in desired
directions (Alvero et al. 2001; Balcazar et al. 1985; Nadler 1977; Taylor et al. 1984).
In this project, the fidelity reviews were the basis for agencies setting goals. As Locke
et al. (1981) state: ‘‘The beneficial effect of goal setting on task performance is one
of the most robust and replicable findings in the psychological literature’’ (p. 145).
The findings suggest a framework for future EBP implementation research. The
organizational schema would specify the people and behaviors necessary for
implementation success. It would also include the role of ‘‘auxiliary’’ agency staff
such as case managers, therapists, and psychiatrists, as well as staff directly
responsible for implementation. Measurement would focus on the degree to which
these people meet behavioral expectations. This study is exploratory and limited to
one state’s implementation efforts with only two EBPs. It is likely that other states
and other practices may suggest a different array of tools, mechanisms, and methods
that contribute to success.
Kansas School of Social Welfare and Greg McHugo from the Dartmouth College Psychiatric Research
The authors would like to thank Professor Edward Canda from The University of
Community Ment Health J (2008) 44:213–224223
Center for their assistance with this paper. Developed by The University of Kansas School of Social Download full-text
Welfare Office of Mental Health Research and Training through a contract with the Kansas Department
of Social and Rehabilitation Services.
Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness and essential
characteristics of performance feedback in organizational settings (1985–1998). Journal of
Organizational Behavior Management, 21(1), 3–29.
Atlas.ti (2002). Atlas (version 2.0) [computer software] Berlin, Germany: GmbH.
Balcazar, F. E., Hopkins, B. L., & Suarez, Y. (1985). A critical objective review of performance
feedback. Journal of Organizational Behavior Management, 7, 65–89.
Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. K. (2000). Measurement of fidelity in
psychiatric rehabilitation. Mental Health Service Research, 2(2), 75–87.
Lehman, A. F., Steinwachs, D. M., & co-investigators. (1998). Patterns of usual care for schizophrenia:
Initial results from the schizophrenia patient outcomes research team (PORT) client survey.
Schizophrenia Bulletin, 24, 11–20.
Lincoln, S. L., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park: Sage Publications.
Locke, E., Shaw, K., Saari, L., & Latham, G. (1981). Goal setting and task performance: 1969–1980.
Psychological Bulletin, 90(1), 125–152.
McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., Goldman, H. H., Lutz,
W., & Finnerty, M. (in press). Fidelity outcomes in the National Evidence-Based Practice Project.
Nadler, D. A. (1977). Feedback and organization development. Using data based methods. Reading, MA:
President’s New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming
mental health care in America. DHHS Publication No. SMA-03-3832, Rockville, MD.
Rapp, C. A., & Goscha, R. (2006). The strengths model: Case management with people with psychiatric
disabilities (2nd ed.). New York: Oxford University Press.
Rapp, C. A., Etzel-Wise, D., Marty, D., Coffman, M., Carlson, L., Asher, D., Callaghan, J., & Holter, M.
(in press). Barriers to Evidence-Based Practice Implementation: Results of a Qualitative Study.
Community Mental Health Journal.
Strauss, S., & Corbin, J. (1990). Basis of qualitative research. Newbury Park, CA: Sage.
Tashjian, M. D., Hayward, B. J., Stoddard, S., & Kraus, L. (1989). Best practice study of vocational
rehabilitation services to severely mentally ill persons. Washington, DC: Washington Rehabilitation
Services Administration, US Department of Education.
Taylor, M. S. (1987). The effects of feedback on the behavior of organizational personnel. Administration
in Social Work, 11(3/4) Fall/Winter, 191–203.
Taylor, M. S., Fisher, C. D., & Ilgen, D. R. (1984). Individuals’ reaction to performance feedback in
organizations: Control theory perspective. In K. M. Rowland, & G. R. Ferris (Eds.), Research in
personnel and human resource management (Vol. 2, p. 810124). Greenwich, CT: JAI.
Torrey, W. C., & Gorman, P. G. (2005). Closing the gap between what services are and what they could
be. In R. E. Drake, M. R. Merrans, & D. W. Lynde (Eds.), Evidence-based mental health practice
(pp. 167–188). New York: W.W. Norton.
U.S. Department of Health, Human Services (DHHS). (1999). A report of the surgeon general.
Washington, DC: Author, Wang, P. S., Demler, O., & Kessler, R. C. (2002).
West, J., Olfson, M., Narrow, W. E., Rae, D., Wilk Marcus, S., & Reier, D. A. (2003). The treatment in
routine psychiatric practice. Paper presented at the Academy of Health Annual Meeting, Nashville,
224 Community Ment Health J (2008) 44:213–224