ArticlePublisher preview available

Teacher Competence of Delivery of BEST in CLASS as a Mediator of Treatment Effects

DOI:10.1007/s12310-017-9224-5
Authors:
Kevin S. Sutherland
Kevin S. Sutherland
Kevin S. Sutherland
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Maureen Conroy at University of Florida
Maureen Conroy
Bryce D McLeod at Virginia Commonwealth University
Bryce D McLeod
James Algina at University of Florida
James Algina
Show all 5 authorsHide
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

This study investigated if training and practice-based coaching in an evidence-based program was associated with higher observed treatment integrity (adherence and competence) and if these treatment integrity components were associated with teacher report of child behavioral outcomes in the BEST in CLASS efficacy trial. Participants were 462 children (M = 4.32 years, SD = 0.53; 65% male; 17.0% Caucasian, 66.0% African-American, 5.0% Hispanic, and 12.0% other) identified as having problem behavior and their 185 teachers (M = 12.09 years teaching experience; 99% female; 47.0% Caucasian, 48.0% African-American, 1.0% Asian/Pacific Islander, 3.0% Hispanic and 1.0% other). Teachers and focal children were randomly assigned to the intervention (teacher n = 92, children n = 230) or control condition (teacher n = 93, child n = 232). Results of a multilevel mediation analysis indicated that the BEST in CLASS intervention had a positive effect on teacher report of child problem behavior (SSIS-RS) and externalizing problems (C-TRF), as well as having a positive effect on teachers’ adherence and competence of delivery of the intervention. There was an indirect effect through competence of delivery for externalizing problems, but not problem behavior. No indirect effects for adherence were found. Implications of these findings and directions for future research are discussed.
Figures - available from: School Mental Health
This content is subject to copyright. Terms and conditions apply.
  • A preview of this full-text is provided by Springer Nature.
  • Learn more
Preview content only
Content available from School Mental Health
This content is subject to copyright. Terms and conditions apply.
ORIGINAL PAPER
Teacher Competence of Delivery of BEST in CLASS
as a Mediator of Treatment Effects
Kevin S. Sutherland
1
•Maureen A. Conroy
2
•Bryce D. McLeod
1
•James Algina
2
•
Eleanor Wu
1
Published online: 9 August 2017
ÓSpringer Science+Business Media, LLC 2017
Abstract This study investigated if training and practice-
based coaching in an evidence-based program was asso-
ciated with higher observed treatment integrity (adherence
and competence) and if these treatment integrity compo-
nents were associated with teacher report of child behav-
ioral outcomes in the BEST in CLASS efficacy trial.
Participants were 462 children (M=4.32 years,
SD =0.53; 65% male; 17.0% Caucasian, 66.0% African-
American, 5.0% Hispanic, and 12.0% other) identified as
having problem behavior and their 185 teachers
(M=12.09 years teaching experience; 99% female;
47.0% Caucasian, 48.0% African-American, 1.0% Asian/
Pacific Islander, 3.0% Hispanic and 1.0% other). Teachers
and focal children were randomly assigned to the inter-
vention (teacher n=92, children n=230) or control
condition (teacher n=93, child n=232). Results of a
multilevel mediation analysis indicated that the BEST in
CLASS intervention had a positive effect on teacher report
of child problem behavior (SSIS-RS) and externalizing
problems (C-TRF), as well as having a positive effect on
teachers’ adherence and competence of delivery of the
intervention. There was an indirect effect through compe-
tence of delivery for externalizing problems, but not
problem behavior. No indirect effects for adherence were
found. Implications of these findings and directions for
future research are discussed.
Keywords Treatment integrity Teacher delivery
Problem behavior Evidence-based program
Introduction
Prevalence rates suggest that between 5 and 30% of chil-
dren entering early childhood programs exhibit chronic
problem behaviors (e.g., social, emotional or behavioral
challenges; Barbarin, 2007; Brauner & Stephens, 2006)
that impact their learning and ability to benefit from early
educational experiences (Feil et al., 2005). Such early onset
problem behaviors in young children are associated with
later learning problems (Bierman et al., 2013; Hamre &
Pianta, 2001), negative relationships with teachers (Bulot-
sky-Shearer, Bell, & Dominquez, 2012), and elevated risk
for identification of an emotional/behavioral disorder
(EBD) (Bierman et al., 2013, Qi & Kaiser, 2003; Webster-
Stratton, 1997). Though some children who initially enter
early childhood programs exhibiting problem behavior
learn and adjust to school and classroom behavioral
expectations within the first several months of school, some
children continue to exhibit frequent and increasingly
intense problem behaviors.
Given the poor outcomes for children with early onset
chronic problem behavior, classroom-based programs
focused upon improving their social, emotional, learning,
and behavioral skills have been developed and evaluated
(Greenberg, 2010). Several of these programs have
&Kevin S. Sutherland
kssuther@vcu.edu
Maureen A. Conroy
mconroy@coe.ufl.edu
Bryce D. McLeod
bmcleod@vcu.edu
James Algina
algina@ufl.edu
Eleanor Wu
egwu@vcu.edu
1
Virginia Commonwealth University, Richmond, VA, USA
2
University of Florida, Gainesville, FL, USA
123
School Mental Health (2018) 10:214–225
https://doi.org/10.1007/s12310-017-9224-5
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... However, competence may be more important to assess as it shows higher-order skills and a level of mastery of the practices contained within an EBI (Sutherland et al., 2013). Also, evidence suggests that teacher competence may be important in promoting positive child social-emotional outcomes (Sutherland et al., 2018). For gauging the success of teacher training and coaching efforts, assessing competence helps establish whether teachers may have mastered the delivery of the practices (e.g., skillfulness in delivery of each practice, responsiveness to child needs) contained within an EBI (e.g., Sutherland et al., 2018). ...
... Also, evidence suggests that teacher competence may be important in promoting positive child social-emotional outcomes (Sutherland et al., 2018). For gauging the success of teacher training and coaching efforts, assessing competence helps establish whether teachers may have mastered the delivery of the practices (e.g., skillfulness in delivery of each practice, responsiveness to child needs) contained within an EBI (e.g., Sutherland et al., 2018). Despite the importance of assessing competence, only a few observational and self-report measures designed to assess the quality of teacher-delivered practices exist (Sanetti et al., 2020). ...
... An assumption of EBIs is that high quality delivery of the practices account for the influence of those programs on child outcomes (Durlak et al., 2011;Wanless & Domitrovich, 2015). Yet to our knowledge only one study in the early childhood literature has evaluated integrity-outcome relations (see Sutherland et al., 2018). Findings from the broader education literature are mixed. ...
Development and Initial Psychometrics of a Generic Treatment Integrity Measure Designed to Assess Practice Elements Targeting Social, Emotional, and Behavioral Outcomes in Early Childhood Settings
Article
  • Oct 2021
Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment integrity measure for early childhood settings (TIMECS), an observational measure designed to address existing measurement gaps related to treatment integrity with tier 2 interventions in the early childhood field. To assess the preliminary score reliability (interrater) and validity (construct, discriminant) of the TIMECS, live observations (N = 650) in early childhood classrooms from 54 teachers (92.6% female, 7.4% male; 61.1% White) and 91 children (M age = 4.53 years, SD = .44; 45.1% female, 54.9% male; 45.1% Black) at risk for EBDs were scored by 12 coders using the TIMECS and an observational measure designed to assess teacher–child interactions. Teachers also self-reported on the quality of the teacher–child relationship. Interrater reliability (intraclass correlation coefficients, ICC [2,2]) for the quantity (i.e., adherence) item scores had a mean of .81 (SD = .07; range from .68 to .95), and the quality (i.e., competence) item scores had a mean of .69 (SD = .08; range from .52 to .80). Scores on the TIMECS Quantity and Quality items and scales showed evidence of construct validity, with the magnitude of the correlations suggesting that the quantity and quality items assess distinct components of treatment integrity. A TIMECS quantity scale also showed promise for intervention evaluation research by discriminating between teachers who had and had not been trained in a specific evidence-based intervention targeting social and behavioral skills in early childhood. The findings support the potential of the TIMECS to assess treatment integrity of teacher-delivered practices designed to address child social and behavioral outcomes of children at risk for EBDs in early childhood settings.
View
... Delivery of EBPs as intended within elementary schools is essential if positive outcomes such as reducing risk for the development of EBDs in children, are to be achieved Sutherland, Conroy, McLeod et al., 2018;Webster-Stratton et al., 2008). With upward of 70% to 80% of services for children with mental health problems delivered in schools (Duong et al., 2021), the education sector is unsurpassed in its potential to reach a broad population of children. ...
... To develop the conceptual model for this project (see Figure 2), we drew on relevant and impactful implementation models from the education sector (Domitrovich et al., 2010;Han & Weiss, 2005;Lyon & Bruns, 2019a, 2019b, research on implementation and sustainment in school settings (e.g., Forman et al., 2013;Owens et al., 2014); Sutherland, Conroy, McLeod et al., 2018; and the broader implementation literature (Chambers et al., 2013;Proctor et al., 2011). First, we use the DSF (Chambers et al., 2013) to conceptualize the three levels of factors that influence sustainment outcomes in school contexts. ...
Study Protocol: Multi-Level Determinants of Implementation and Sustainment in the Education Sector
Article
  • May 2022
Evidence-based programs (EBPs) delivered in elementary schools show great promise in reducing risk for emotional and behavioral disorders (EBDs). However, efforts to sustain EBPs in school face barriers. Improving EBP sustainment thus represents a priority, but little research exists to inform the development of sustainment strategies. To address this gap, the Sustaining Evidenced-Based Innovations through Multi-level Implementation Constructs (SEISMIC) project will (a) determine if malleable individual, intervention, and organizational factors predict EBP treatment fidelity and modifications during implementation, sustainment, or both; (b) assess the impact of EBP fidelity and modifications on child outcomes during implementation and sustainment; and (c) explore the mechanisms through which individual, intervention, and organizational factors influence sustainment outcomes. This protocol article describes SEISMIC, which builds upon a federally funded randomized clinical trial evaluating BEST in CLASS, a teacher-delivered program for K to Grade 3 children at risk for EBDs. The sample will include 96 teachers, 384 children, and 12 elementary schools. A multi-level, interrupted time series design will be used to examine the relationship between baseline factors, treatment fidelity, modifications, and child outcomes, followed by a mixed-method approach to elucidate the mechanisms that influence sustainment outcomes. Findings will be used to create a strategy to improve EBP sustainment in schools.
View
... Previous reports in primary and secondary education have shown that intervention programs that aims to improve students' ASC are more effective when they are implemented by the teachers themselves than by support professionals (Campbell et al., 2015;Desimone & Hill, 2017;Sutherland et al., 2018). From this point of view, the change in teaching behavior has been reported to be a mediator between teacher training interventions and changes in students' ASC (Bruhn et al., 2014;Hagermoser et al., 2012). ...
... The main finding was that changes in teaching strategies for ASC stimulation mediated the effect of the intervention on students' ASC, thus confirming hypothesis of the study based on the literature indicating that a determinant factor in the impact of student change is change in the teacher (Bruhn et al., 2014;Hagermoser et al., 2012). In line with these findings are the conclusions of the studies by Desimone and Hill (2017), Sutherland et al. (2018), and Schütze et al. (2017), in which they report that participating in the experimental group increases the teachers' ability to implement strategies and these changes in the teacher lead to changes in the students. This further explains the results of a study where it was shown that teacher references perceived by students are positively related to their ASC (Lohbeck & Philipp, 2020). ...
Promoting ASC in the primary education classroom: The role of teacher training
Article
  • Jul 2023
The purpose of this research was to study the effect of the changes in the strategies to stimulate self-concept used by teachers, that received training, in the academic self-concept of its students. A quasi-experimental design with pre- and post-application measures was used. A total of 36 secondary school teachers and 814 students participated. The program consisted of four months of workshop sessions and accompaniment in the classroom. It was found that changes in the teaching strategies mediated the effect of the intervention on students’ self-concept. In addition, the change in the students’ type of self-concept depended on the change in the teachers’ specific strategies. In conclusion, the training program designed for stimulating academic self-concept was effective in promoting strategies in teachers that increased student’s self-concept. Thus, it can be concluded that the intervention is an approachable methodology to support the development of self-concept, potentially impacting students’ academic success. The findings contribute to the design of future interventions in school contexts for the improvement of students’ self-concept. Free e-prints https://www.tandfonline.com/eprint/FN8RIDEPH2CSHEEJAIGS/full?target=10.1080/21683603.2022.2105997
View
... Teachers' competence, moreover, allows them to develop supporting relations with students, allowing them to strengthen their abilities with intrinsic motivation, work collaboratively and fostering learning outcomes (Jennings and Greenberg, 2009). Leighton and Bustos Gómez (2018), encouraged future studies to pursue teacher competence as a mediator, while a recent study did use teacher competence as a mediator (Sutherland et al., 2018). Since it was conducted in a school setting, therefore, this study is applying it to Pakistan's higher education. ...
... Moreover, teachers take advantage of available professional prospects to develop their EI (Dolev and Leshem, 2017), hence, maximizing their learning outcomes and fostering institutional performance (Thurlings and den Brok, 2017;Iqbal et al., 2019). Additionally, teachers' competence as mediators (Sutherland et al., 2018) argues the possessors are the obligor of persistent EI development and learning improvements (Jennings and Greenberg, 2009;García, 2016). ...
Teachers’ Adoption of Emotions-Based Learning Outcomes: Significance of Teachers’ Competence, Creative Performance, and University Performance
Article
Full-text available
  • Jun 2022
Studies have revealed that emotion-based learning outcomes are scarce when teachers’ competence and creative performance are neglected, further university performance in relation to teachers’ emotion-based learning outcomes is disregarded in literature so far. Based on the Attributional theory of achievement motivation and emotion, the purpose of this empirical study is to investigate the effects of Emotional Intelligence (EI) on learning outcomes (social, cognitive, self-growth outcomes, and satisfaction with university experience) of academicians in Pakistan’s higher education institutions (HEIs). This study also examines the mediating role of teacher competence (personal assessment) and creative performance (Creative self-efficacy and leadership/supervisor support) in a relationship between EI and learning outcomes. Furthermore, this study ascertained the relationship between learning outcomes and organizational performance (OP) of HEIs. This study used a sample frame of 237 academic professionals from Pakistani HEIs, the hypothesized associations were ascertained using the partial least squares structural equation modeling method (PLS-SEM). The findings disclose that EI has a positive and significant influence on learning outcomes. Furthermore, an indirect relation between EI and learning outcomes is established through teacher competence and creative performance while the relationship between learning outcomes and OP is established also. Results of the considered study reinforce the academic understanding of EI and propose how academicians of HEIs can value their competence and creative performance which in turn enhances learning outcomes and OP. There is a lack of studies in HEIs that investigate the relationship between EI, teacher competence, creative performance, learning outcomes, and OP. This is one of the initial researches that not only empirically examine the interface of EI, learning outcomes, and OP of HEIs’ academicians but also enlightens comprehensions into the prevailing literature by immediate investigation of the mediating role of teacher competence and creative performance in fundamental association.
View
... In the current study, adherence was not directly related to reductions in student problem behavior, while teacher competence of delivery of BEST in CLASS practices was trending toward significance; the associations between teacher competence and adherence and reductions in problem behavior appear to operate via student responsiveness to teachers' delivery of the intervention practices. This is not surprising, as adherence has been found to be associated with positive child outcomes (see Durlak & DuPre, 2008), and previous research on BEST in CLASS (Sutherland et al., 2018b) has found that teacher competence of delivery is associated with positive child treatment outcomes. Our findings thus highlight the potential importance of student responsiveness in intervention effectiveness, and the role played by adherence and teacher competence of delivery. ...
Examining the Role of Student Responsiveness in Treatment Effects of a Tier 2 Program Targeting Reductions in Problem Behavior
Article
Student responsiveness's role in promoting intervention outcomes for students who exhibit problem behavior is understudied. Due to the relational nature of many interventions delivered by teachers that target social, emotional, or behavioral outcomes of students in classrooms, it is essential to assess how responsive students are to teachers' attempts to engage them in the intervention, particularly for students with problem behaviors that may impede teachers' attempts to engage these students in intervention effectively. In the current study, we combine samples from four randomized controlled trials to examine the relationship between student outcomes and teacher attempts to deliver BEST in CLASS, a Tier 2 intervention, via student responsiveness. Delivery of BEST in CLASS and student responsiveness were assessed through direct observations and teachers' reported measures. Results suggest that teacher adherence and competence in delivering BEST in CLASS practices was associated with reductions in problem behavior from pretest to post-test via student responsiveness. Limitations of the current study and implications for future research are discussed.
View
... In some cases, this paper estimated missing data using the method described by Hozo et al. (2005) to calculate the mean and estimate the standard deviation (Wan et al., 2014). Missing data for three studies meant that it was impossible to obtain or Table 1 Overview of Studies Included in the Systematic Review and the Meta-analysis (Studies that contributed to the analysis are shown in bold) *The following papers between brackets are one study using the same sample but reporting different outcomes (Baker-Henningham & Walker, 2018;Baker-Henningham et al., 2012), (Baker-Henningham et al., 2009a, 2009b, (Conroy et al., 2018(Conroy et al., , 2019Sutherland et al., 2018aSutherland et al., , 2018b, (Hoogendijk et al., 2018(Hoogendijk et al., , 2020, (LoCasale-Crouch et al., 2018;Williford et al., 2017), (Vancraeyveldt et al., 2015a(Vancraeyveldt et al., , 2015b, (Veenman et al., 2017(Veenman et al., , 2019 (Achenbach & Rescorla, 2000); CTRS, Conners Teacher Rating Scale (Goyette et al., 1978); DPICS, Dyadic Parent Child Inter-active Coding System (Eyberg & Robinson, 1981); EBP, Externalizing Behavior Problems; ECBI, Eyberg & Pincus (Eyberg, 1999); MOOSES, Multi-Option Observation System for Experimental Studies (Tapp et al., 1995); MPROX, Most Proximal; OREVS, Observer Rating of Ecobehavioral Variables Scale (Chandler et al., 1999); PBLIND, Probably Blinded; PBQ, Preschool Behavior Questionnaire (Behar, 1977); PRO, Prosocial; SCP, Social Competence Performance Checklist (Stoiber, 2004); SDQ, Strengths and Difficulties Questionnaire (Goodman & Goodman, 2009); SESBI, Sutter-Eyberg Student Behavior Inventory (Rayfield et al., 1998); SSBS-2, School Social Behavior Scales-Second Edition (Merrell & Gimpel, 1998); SSIS, Social Skills Improvement System (Gresham & Elliott, 2007); STRS, Student-Teacher Relationship Scale (Pianta, 2001); TAS, Teacher use of Appropriate Strategies; TCIDOS, Teacher-Child Interaction Direct Observation System (Sutherland et al., 2013); TOCA-C, Teacher Observation of Classroom Adaptation-Checklist (Koth et al., 2009); TPOT, Teacher-Pupil Observation Tool (Martin et al., 2010); WMCSC, Walker-McConnell Scale of Social Competence & School Adjustment (Walker & McConnell, 1988) a MPROX; b PBLIND calculate an effect size; these studies were included in the review but excluded from the meta-analysis. ...
Systematic Review and Meta-Analysis of the Effectiveness of Teacher Delivered Interventions for Externalizing Behaviors
Article
Full-text available
This systematic review and meta-analysis explores the effectiveness of teacher interventions supporting children with externalizing behaviors based on teacher and child outcomes. A systematic search was conducted using 5 electronic databases. From 5714 papers, 31 papers that included interventions delivered directly to teachers and aimed to benefit either teachers and/or children with externalizing behaviors were included. The review focused on qualified teachers working with children aged 2–13. The results of the current meta-analysis revealed a positive effect of teacher intervention on teacher and child outcomes, including the increased use of teacher-appropriate strategies, as well as significant and moderate improvements in teacher–child closeness, and small reductions in teacher–child conflict. For child outcomes, the interventions reduced externalizing behavior problems and ADHD symptoms and enhanced prosocial behavior. Only one fully blinded analysis for conduct problems was possible and revealed a moderate but significant reduction in favor of intervention. These findings provide evidence to support the role of teacher interventions for both teachers and children with externalizing behaviors. Future research should include more PBLIND measurements so that MPROX findings can be confirmed. More research should be done to evaluate the influence of teacher interventions on teachers’ well-being.
View
View
View
Teacher Burnout and Supporting Teachers of Students with Emotional and Behavioral Disorders
Article
  • Feb 2023
This article will provide a brief reflection on Garwood’s summary of research on special education teacher burnout and fidelity of implementation in delivery of evidence-based behavioral interventions. Subsequently, we provide a conceptual and empirical summary of key issues for supporting teachers of and students with emotional and behavioral disorders (EBDs) through the lens of BEST in CLASS (a Tier 2 intervention supporting teacher’s use of evidence-based practices with students with or at-risk for EBD). This summary will (a) outline the theoretical structure that supports how BEST in CLASS may improve teacher–student relationships and reduce teacher burnout, (b) demonstrate the influence of BEST in CLASS on teacher burnout in a sample of elementary school teachers and discuss findings, (c) propose that researchers consider burnout within the context of dynamic classroom systems, and (d) link these suggestions to theoretical frameworks. We conclude with a discussion of Garwood’s call to action and implications for future research.
View
Exploring profiles of coaches' fidelity to Double Check's Motivational Interviewing-embedded coaching: Outcomes associated with fidelity
Article
Motivational interviewing (MI) is applied in a variety of clinical and coaching models to promote behavior change, with increasing interest in its potential to optimize school-based implementation fidelity. Yet there has been less consideration of fidelity indicators for MI-embedded coaching and their associations with outcomes. We leveraged exisiting data from 151 teachers across 18 schools, who were part of a larger 39 middle school randomized controlled trial of a teacher coaching model, to explore profiles of fidelity and the associations between fidelity and outcomes. We conducted latent profile analysis (LPA) to examine profiles of four components of fidelity (i.e., adherence, dosage, quality, and teachers' responsiveness). Next, we examined whether observed teacher practices and student behaviors varied across fidelity profiles. Because coaches and independent coders reported adherence, we also examined the reliability of retrospective coach adherence ratings. Results indicated that coaches show promise as a reliable rater of adherence. The LPA indicated that there were two (high and lower) fidelity profiles. Statistically significantly fewer instances of student non-cooperation were observed in classrooms where the teacher was engaged in high fidelity coaching, reflecting a large effect size. Moderate-sized, but non-statistically significant, effects also emerged for teacher opportunities to respond and reactive behavior management. We identify concrete areas to ensure that reliability can be achieved in other contexts. Future directions are also considered regarding fidelity measurement and how to optimize coaching.
View
The Effectiveness and Micro-costing Analysis of a Universal, School-Based, Social–Emotional Learning Programme in the UK: A Cluster-Randomised Controlled Trial
Article
Full-text available
  • Jun 2016
There are a growing number of school-based interventions designed to promote children’s social and emotional learning. One such intervention, PATHS (Promoting Alternative Thinking Strategies), was evaluated in a randomised controlled trial involving 5074 pupils aged 4–6 years at baseline in 56 primary schools across a large city in the UK. The programme was implemented for two academic years. The primary outcome measure was the teacher-rated Strengths and Difficulties Questionnaire (SDQ). A secondary measure was the PATHS Teacher Rating Scale (PTRS). Observations of child and teacher behaviours were undertaken in a third of intervention and control schools using the Teacher–Pupil Observation Tool (T-POT). Regarding fidelity, dose and adherence were measured via weekly logs completed by teachers, and a semi-structured questionnaire completed by PATHS coaches was used as a global measure of fidelity (capturing adherence, dose and quality). A cost-consequence analysis examined programme costs from a multi-agency public sector perspective. At 1 year post-baseline, there were no statistically significant differences between the programme and control groups on the SDQ subscales or the SDQ total difficulties and impact scores. There were statistically significant differences favouring the programme group for six out of 11 subscales on the secondary outcome measure (PTRS). At 2 years post-baseline, there were no statistically significant differences between the groups on either measure. Fidelity, according to the global measure, was relatively strong, and there was no relationship between fidelity and treatment effects. The average cost of PATHS was £12,666 per school or £139 per child. The study, which was fully powered and independent of the programme developer, shows no statistically significant effect of the programme on child behaviour or emotional well-being. Trial registration site and number: www.controlled-trials.com: ISRCTN 32534848.
View
View
Studying Program Implementation Is Not Easy but It Is Essential
Article
Full-text available
This study offers a commentary on the articles contained in the special issue of Prevention Science, "Readiness to implement Social- Emotional Learning interventions." The commentary also puts these articles into current context by summarizing important findings in implementation research and listing some priorities for future work.
View
Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination
Chapter
  • Jan 2017
There is substantial variability in the implementation of evidence-based interventions across the United States, which leads to inconsistent access to evidence-based prevention and treatment strategies at a population level. Increased dissemination and implementation of evidence-based interventions could result in significant public health gains. While the availability of evidence-based interventions is increasing, study of implementation, adaptation, and dissemination has only recently gained attention in public health. To date, insufficient attention has been given to the issue of fidelity. Consideration of fidelity is necessary to balance the need for internal and external validity across the research continuum. There is also a need for a more robust literature to increase knowledge about factors that influence fidelity, strategies for maximizing fidelity, methods for measuring and analyzing fidelity, and examining sources of variability in implementation fidelity.
View
Reducing child problem behaviors and improving teacher-child interactions and relationships: A randomized controlled trial of BEST in CLASS
Article
Research has consistently linked early problem behavior with later adjustment problems, including antisocial behavior, learning problems and risk for the development of emotional/behavioral disorders (EBDs). Researchers have focused upon developing effective intervention programs for young children who arrive in preschool exhibiting chronic problem behaviors; however, Tier-2 interventions that can be delivered by teachers with fidelity in authentic settings are lacking. This study examined the effect of BEST in CLASS, a Tier-2 intervention delivered by teachers, on child problem behavior, teacher-child interactions and teacher-child relationships using a cluster randomized controlled trial design. Participants were 465 children (3–5 year olds) identified at risk for the development of EBDs and their 185 teachers from early childhood programs located in two southeastern states. Significant effects were found across both teacher reported (ES ranging from 0.23 to 0.42) and observed child outcomes (ES ranging form 0.44–0.46), as well as teacher-child relationships (ES ranging from 0.26 to 0.29) and observed teacher-children interactions (ES ranging from 0.26 to 0.45), favoring the BEST in CLASS condition. Results suggest the promise of BEST in CLASS as a Tier-2 intervention for use in authentic early childhood classroom contexts and provide implications for future research on transactional models of teacher and child behavior.
View
Development and Initial Psychometrics for a Therapist Competence Instrument for CBT for Youth Anxiety
Article
  • Dec 2016
Therapist competence is an important component of treatment integrity. This article reports on the development and initial psychometric assessment of the Cognitive-Behavioral Treatment for Anxiety in Youth Competence Scale (CBAY-C), an observational instrument designed to capture therapist limited-domain competence (i.e., competence in the delivery of core interventions and delivery methods found in a specific psychosocial treatment program) in the delivery of the core practice elements in individual cognitive-behavioral treatment (ICBT) for youth anxiety. Treatment sessions (N = 744) from 68 youth participants (M age = 10.60 years, SD = 2.03; 82.3% Caucasian; 52.9% male) of the same ICBT program for youth anxiety from (a) an efficacy study and (b) an effectiveness study were independently scored by 4 coders using observational instruments designed to assess therapist competence, treatment adherence, treatment differentiation, alliance, and client involvement. Interrater reliability—intraclass correlation coefficients (2,2)—for the item scores averaged 0.69 (SD = 0.11). The CBAY-C item, scale, and subscale (Skills, Exposure) scores showed evidence of validity via associations with observational instruments of treatment adherence to ICBT for youth anxiety, theory-based domains (cognitive-behavioral treatment, psychodynamic, family, client centered), alliance, and client involvement. Important to note, although the CBAY-C scale, subscale, and item scores did overlap with a corresponding observational treatment adherence instrument independently rated by coders, the degree of overlap was moderate, indicating that the CBAY-C assesses a distinct component of treatment integrity. Applications of the instrument and future research directions discussed include the measurement of treatment integrity and testing integrity-outcome relations.
View
View
View
View
Supporting Implementation of Evidence-Based Practices Through Practice-Based Coaching
Article
In active implementation science frameworks, coaching has been described as an important competency “driver” to ensure evidence-based practices are implemented as intended. Empirical evidence also has identified coaching as a promising job-embedded professional development strategy to support implementation of quality teaching practices. The purpose of the present article is to describe a coaching framework designed to support early childhood practitioners to implement evidence-based teaching practices with fidelity. We explicate the key components of the coaching framework, provide theoretical and empirical rationales for each component, and describe how it was operationalized for use as a coaching protocol in several studies. The studies focused on supporting preschool teachers of young children with or at risk for disabilities to implement social-emotional, behavioral, and instructional teaching practices with fidelity. For this special issue, we offer recommendations for future research and considerations for wider scale application and situate each article in the context of coaching and the coaching framework described in this article.
View