T130. DEVELOPMENT OF A STATEWIDE PROGRAM EVALUATION PROGRAM IN COORDINATED SPECIALTY CARE PROGRAMS

Abstract

Background
Since 2016, the Pennsylvania (PA) state Office of Mental Health and Substance Abuse Services (OMHSAS) has partnered with the University of Pennsylvania to develop and implement a statewide program evaluation (PE) of Pennsylvania’s coordinated specialty care (CSC) programs serving individuals with first episode psychosis (FEP). PA-FEP-PE is unique in establishing a computerized PE battery and complementary fidelity scale capable of measuring the delivery of evidence-based care and clinical and service outcomes across a variety of CSC models delivered by both academic and community-based programs.

Methods
Nine CSC programs serve youth age 12–34 who have experienced onset of early psychosis between 12–24 months prior to admission. Services are offered for a minimum of 2 years. The PA-FEP-PE outcomes battery, computerized in REDCap, is composed of measures selected for Domain Coverage, Clinical Utility, and Reliability/Validity (based on inclusion in the PhenX toolkit), as well as Practical Utility,Low Burden and High Utility to Multiple Stakeholders. The battery is administered at admission, and at 6-, 12-, 18- and 24-month follow-up and was implemented 1/1/17.The PA Fidelity Scale, which combines the First-Episode Psychosis Services Fidelity Scale and the OnTrackNY fidelity scale, is based on chart review and an annual in-person site visit using standardized interview protocols with pre-designated staff members. Ratings are on 5-point subscales according to operational criteria for adherence to a CSC model in 3 key domains: Program Structure, Staff Roles and Responsibilities, and Process and Clinic Flow. Total possible score is 100. The first annual fidelity assessment was completed in 7/18. Results of outcome measure collection and fidelity assessment are summarized in quarterly reports, delivered to OMHSAS and to each site, providing program evaluation data in aggregate and for each site individually.

Results
Between 1/1/17-6/30/18, 908 referrals were received, of whom 327 participants (mean age=21.2 SD=4.7; 35% Female; 48% Caucasian, 37% African-American, 14% Asian/Multi-racial/Other) were enrolled. Time to screening was <1 week (mean days=1.8, SD=4.4) and to eligibility determination approximately 1 week (mean days=8.9, SD=10.9). Age at psychosis onset was 20.3 years (SD=4.7), and an average of 11.3 (SD=16.5) months lapsed between symptom onset and admission. The majority (74%) of participants had hospitalizations prior to enrollment. At admission, participants showed moderate severity of psychiatric symptoms, and serious impairment in global role and social functioning. Among participants completing 6-month follow-up (n=67) by 7/30/18, several significant improvements were observed, including decreased hospitalizations, suicidal ideation and attempts, substance use, overall psychopathology, psychosis and depression symptoms, and increased employment and school enrollment, global role and social function, self-rated quality of life, and satisfaction with mental health services. Fidelity assessment indicated that all 9 programs provide essential CSC components, including pharmacotherapy, psychotherapy, case management, supported employment and education services, 24/7 crisis services, family/caregiver involvement, outreach and psychoeducation, and treatment and discharge planning. Scores ranged from 73–97%.

Discussion
The state of PA has developed a comprehensive PE model that monitors both CSC fidelity and outcomes. Early data demonstrate promising clinical and service utilization. Further, the PA-FEP-PE model offers disparate CSC programs a collaborative learning network for ongoing quality improvement based on program outcomes and fidelity feedback.