Susan Esp’s research while affiliated with Boise State University and other places

Background Implementation and hybrid effectiveness–implementation trials aspire to speed the translation of science into practice by generating crucial evidence for improving the uptake of effective health interventions. By design, they pose unique recruitment and retention challenges due to their aims, units of analysis, and sampling plans, which typically require many clinical sites (i.e. often 20 or more) and participation by individuals who are related across multiple levels (e.g. linked organizational leaders, clinicians, and patients). In this article, we present a new multilevel, theory-informed, and relationship-centered framework for conceptualizing recruitment and retention in implementation and hybrid effectiveness–implementation trials which integrates and builds on prior work on recruitment and retention strategies in patient-focused trials. We describe the framework’s application in the Working to Implement and Sustain Digital Outcome Measures hybrid type III trial, which occurred in part during the COVID-19 pandemic. Methods Recruitment for the Working to Implement and Sustain Digital Outcome Measures trial occurred from October 2019 to February 2022. Development of recruitment and retention strategies was guided by a newly developed multilevel framework, which targeted the capability, opportunity, and motivation of organizational leaders, clinicians, patient-facing administrative staff, and patients to engage in research. A structured assessment guide was developed and applied to refine recruitment and retention approaches throughout the trial. We describe the framework and its application amid the onset of the COVID-19 pandemic which required rapid adjustments to address numerous barriers. Results The Working to Implement and Sustain Digital Outcome Measures trial enrolled 21 outpatient clinics in three US states, incorporating 252 clinicians and 686 caregivers of youth (95% of patient recruitment target) across two distinct phases. Data completion rates for organizational leaders and clinicians averaged 90% over five waves spanning 18 months, despite the onset of the COVID pandemic. Caregiver completion rates of monthly follow-up assessments ranged from 80%–88% across 6 months. This article presents the multilevel framework, assessment guide, and strategies used to achieve recruitment and retention targets at each level. Conclusion We conducted a multi-state hybrid type III effectiveness–implementation trial that maintained high recruitment and retention across all relevant levels amid a global pandemic. The newly developed multilevel recruitment and retention framework and assessment guide presented here, which integrates behavioral theory, a relationship-focused lens, and evidence-based strategies for participant recruitment and retention at multiple levels, can be adapted and used by other researchers for implementation, hybrid, and multilevel pragmatic trials as well as other implementation studies.

Sensation seeking is a significant predictor of alcohol‐related consequences among high school students. High school seniors report the highest prevalence rates of alcohol use and binge drinking among students in high school. Among high sensation‐seeking students, higher rates of Protective Behavioral Strategies (PBS) use are associated with fewer alcohol‐related consequences. To date, however, there is no research investigating which specific PBS are related to lower levels of alcohol‐related consequences in this high‐risk group of students. The current study examined the frequency of use and effectiveness of 20 PBS measured by the PBS Scale‐20 among high sensation‐seeking high school seniors ( N = 77). “Go‐zone” methodology was used to classify PBS items into one of four zones (i.e., used frequently, high effectiveness; used infrequently, high effectiveness; used frequently, low effectiveness; used infrequently; low effectiveness). Four PBS were identified as “go‐zone” strategies (e.g., used frequently; high effectiveness) and five strategies were identified as used infrequently but highly effective. Findings from the current study extend the literature on PBS use as a buffer against alcohol‐related consequences among high sensation‐seeking high school students. Implications for school mental health professionals are discussed. Trial Registration: ClinicalTrials.gov identifier: NCT03613818

Background Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI’s hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI’s effect on implementation climate, which in turn will (3) mediate LOCI’s effect on MBC fidelity. Methods Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders’ implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. Results LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI’s effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI’s improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI’s effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI’s effect on MBC fidelity during the same period (pm = 0.71, p = .045). Conclusions LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. Trial registration ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Background Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers’ implementation of evidence-based psychosocial interventions—(1) audit and feedback and (2) active learning. Methods Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. Results In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ²=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ²=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. Conclusions Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. Trial registration ClinicalTrials.gov NCT04096274. Registered on 19 September 2019.

Purpose Despite significant interest in improving behavioral health therapists’ implementation of measurement-based care (MBC)—and widespread acknowledgment of the potential importance of organization-level determinants—little is known about the extent to which therapists’ use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation. Methods Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes. Results Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC. Conclusions There is important variation in therapists’ attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Using a cross‐sectional design, we examined protective behavioral strategies (PBS) as a moderator of the relationship between sensation seeking and hazardous drinking and alcohol‐related consequences among high school seniors (N = 212). Hierarchical regression analyses indicated sensation seeking was a significant predictor of binge drinking (β = 0.65, p < 0.001), prepartying (β = 0.71, p < 0.001), gaming (β = 0.75, p < 0.001), and alcohol‐related consequences (β = 0.69, p < 0.001). Further, PBS moderated these relationships such that among high sensation seeking adolescents, PBS use was associated with better outcomes, including lower levels of binge drinking (β = −0.37, p < 0.01), prepartying (β = −0.44, p < 0.01), gaming (β = −0.31, p < 0.05), and alcohol‐related consequences (β = −0.53, p < 0.001). We discuss counseling implications, including assessment and harm reduction strategies focusing on PBS to reduce hazardous drinking among high sensation seeking adolescents.

Background Among high school students, seniors report the highest levels of alcohol use and hazardous drinking. Technology-based interventions are a promising approach for reducing alcohol use among this age group. Objectives: This randomized controlled trial investigated the efficacy of the eCHECKUP TO GO, an online personalized feedback intervention, on reducing alcohol use among high school seniors across the academic year (ClinicalTrials.gov Identifier NCT03613818). Drinker risk status, with students classified as high- or low-risk drinkers based on baseline reports binge drinking, was examined as a moderator of intervention effects. Method: Participants (N = 311) were recruited from two high schools. Class periods were randomized to the intervention condition or an assessment-only control condition. Participants completed online surveys at baseline, 30-day, and 6-month assessments. Results: Intervention effects were moderated by drinker risk status. Among high-risk drinkers, those in the intervention condition reported a significant reduction in weekly drinking quantity and estimated blood alcohol concentration (eBAC) at the 30-day follow-up relative to students in the control condition. By 6 months, reductions in alcohol outcomes among high-risk students in the intervention condition were sustained, but high-risk students in the control condition also reported reductions in weekly drinking. In contrast, for eBAC, intervention effects were significantly greater among students in the intervention condition relative to the control condition at 6 months. Conclusions: Results support the efficacy of the eCHECKUP TO GO for decreasing short-term alcohol use among seniors who are high-risk drinkers and provide preliminary evidence that reductions are sustained across the academic year.