Figure 1 - uploaded by Matthew Chinman
Content may be subject to copyright.
The 10 steps of the Getting To Outcomes (GTO) Framework. Source: Reprinted from Wandersman, Chinman, Imm, & Kaftarian (2000), with permission from Elsevier. 

The 10 steps of the Getting To Outcomes (GTO) Framework. Source: Reprinted from Wandersman, Chinman, Imm, & Kaftarian (2000), with permission from Elsevier. 

Source publication
Article
Full-text available
Demands on community-based prevention programs for performance accountability and positive outcomes are ever increasing in the face of constrained resources. Relatively little is known about how technical assistance (TA) should be structured to benefit community-based organizations and to lead to better outcomes. In this study, data from multiple s...

Citations

... The synthesis affirmed the significance of provider-recipient relationships to TA, noting trust, collaboration, and a strengths-based orientation as most commonly reported relationship attributes. When TA providers establish rapport with recipients, recipients view providers as trusting, respectful, patient, and motivating, underscoring the importance of the recipient-provider relationship [17][18][19]. Collaborative TA relationships are positively associated with implementation-related outcomes including implementation adherence [19,20], and high-quality team functioning-a proximal outcome linked to implementation effectiveness [21]. ...
Article
Full-text available
Background Technical assistance (TA) is a tailored approach to capacity building that is commonly used to support implementation of evidence-based interventions. Despite its widespread applications, measurement tools for assessing critical components of TA are scant. In particular, the field lacks an expert-informed measure for examining relationship quality between TA providers and recipients. TA relationships are central to TA and significantly associated with program implementation outcomes. The current study seeks to address the gap in TA measurement tools by providing a scale for assessing TA relationships. Methods We utilized a modified Delphi approach involving two rounds of Delphi surveys and a panel discussion with TA experts to garner feedback and consensus on the domains and items that compose the TA Engagement Scale. Results TA experts represented various U.S. organizations and TA roles (e.g., provider, recipient, researcher) with 25 respondents in the first survey and 26 respondents in the second survey. The modified Delphi process resulted in a scale composed of six domains and 22 items relevant and important to TA relationships between providers and recipients. Conclusion The TA Engagement Scale is a formative evaluation tool intended to offer TA providers the ability to identify strengths and areas for growth in the provider-recipient relationship and to communicate about ongoing needs. As a standard measurement tool, it lends a step toward more systematic collection of TA data, the ability to generate a more coherent body of TA evidence, and enables comparisons of TA relationships across settings.
... Further, while several other determining factors shifted substantially in their prioritization after the onset of the COVID-19 pandemic, stakeholder input remained steady (i.e., appeared to be resistant to disruption). This is an important finding because multiple sources of evidence support the utility of TA provision incorporating feedback from, or partnering with, stakeholders (Hunter et al., 2009;Jensen et al., 2023;Le et al., 2016;Rushovich et al., 2015). One group of scholars has even argued that "relationships (workers with community, agency-to-agency, and among local residents) create pathways for action [emphasis ours]" (Kavanagh et al., 2022). ...
Article
Full-text available
It is important to use evidence-based programs and practices (EBPs) to address major public health issues. However, those who use EBPs in real-world settings often require support in bridging the research-to-practice gap. In the US, one of the largest systems that provides such support is the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) Technology Transfer Center (TTC) Network. As part of a large external evaluation of the Network, this study examined how TTCs determine which EBPs to promote and how to promote them. Using semi-structured interviews and pre-testing, we developed a “Determinants of Technology Transfer” survey that was completed by 100% of TTCs in the Network. Because the study period overlapped with the onset of the COVID-19 pandemic, we also conducted a retrospective pre/post-pandemic comparison of determinants. TTCs reported relying on a broad group of factors when selecting EBPs to disseminate and the methods to do so. Stakeholder and target audience input and needs were consistently the most important determinant (both before and during COVID-19), while some other determinants fluctuated around the pandemic (e.g., public health mandates, instructions in the funding opportunity announcements). We discuss implications of the findings for technology transfer and frame the analyses in terms of the Interactive Systems Framework for Dissemination and Implementation.
... The Interactive Systems Framework (ISF), developed by Wandersman and colleagues (2008), provides a conceptual logic model for how coordinated implementation support systems help to narrow the research-practice gap (Hunter et al., 2009a;Katz & Wandersman, 2016;Mitchell et al., 2004). The ISF consists of three interconnected systems: the synthesis and translation system, the support system, and the delivery system. ...
Article
Full-text available
The research-practice gap between evidence-based intervention efficacy and its uptake in real-world contexts remains a central challenge for prevention and implementation science. Providing technical assistance (TA) is considered a crucial support mechanism that can help narrow the gap. However, empirical measurement of TA strategies and their variation is often lacking. The current study unpacks the black box of TA, highlighting different TA strategies, amounts, and their relation to intervention characteristics. First, we qualitatively categorized interactions between TA providers and implementers. Second, we explored how characteristics of implementing organizations and the intervention related to variations in the amount of TA delivered. Using data spanning six years, we analyzed over 10,000 encounters between TA providers and implementers. Content analysis yielded four distinct strategies: Consultation (27.2%), Coordination Logistics (24.5%), Monitoring (16.5%), and Resource Delivery (28.2%). Organizations with prior experience required less monitoring and resource delivery. Additionally, characteristics of the intervention were significantly associated with the amount of consultation, monitoring, coordination logistics, and resource delivery provided. The specific features of the intervention showed significant variation in their relation to TA strategies. These findings provide initial insights into the implications of intervention characteristics in determining how much of which TA strategies are needed to support implementations in real-world settings.
... The synthesis a rmed the signi cance of provider-recipient relationships to TA, noting trust, collaboration, and a strengths-based orientation as most commonly reported relationship attributes. When TA providers establish rapport with recipients, recipients view providers as trusting, respectful, patient, and motivating, underscoring the importance of the recipient-provider relationship (17)(18)(19). Collaborative TA relationships are positively associated with implementation-related outcomes including implementation adherence (19,20), and high-quality team functioning -a proximal outcome linked to implementation effectiveness (21). ...
Preprint
Full-text available
Background: Technical assistance (TA) is a tailored approach to capacity building that is commonly used to support implementation of evidence based interventions. Despite its widespread applications, measurement tools for assessing critical components of TA are scant. In particular, the field lacks a robust measure for examining relationship quality between TA providers and recipients. TA relationships are central to TA and significantly associated with program implementation outcomes. The current study seeks to address the gap in TA measurement tools by providing a scale for assessing TA relationships. Methods: We utilized a modified Delphi approach involving two rounds of Delphi surveys and a panel discussion with TA experts to garner feedback and consensus on the domains and items that compose the TA Engagement Scale. Results: TA experts represented various U.S. organizations and TA roles (e.g., provider, recipient, researcher) with 25 respondents in the first survey and 26 respondents in the second survey. The modified Delphi process resulted in a scale composed of six domains and 22 items relevant and important TA relationships between providers and recipients. Conclusion: The TA Engagement Scale is a formative evaluation tool intended to offer TA providers the ability to identify strengths and areas for growth in the provider-recipient relationship and to communicate about ongoing needs. As a standard measurement tool, it lends a step toward more systematic collection of TA data, the ability to generate a more coherent body of TA evidence, and enables comparisons of TA relationships across settings.
... For example, some definitions reflect a general aim for TA (e.g., "to build the capacity of individuals or organizations" [32]), while other definitions offer a more specific aim (e.g., "to facilitate knowledge and skill acquisition" [44]). In terms of implementation, some TA definitions encompass a variety of processes or activities (e.g., a "multi-tiered approach" [32]; "different types of activities including community-friendly manuals, on-site consultation, regional workshops, train-thetrainers models, and interactive Web-based systems" [43]). Others use less specific language (e.g., "support to help…" [36]; "tailored or targeted support to…" [38]). ...
Article
Full-text available
Background Although the benefits of evidence-based practices (EBPs) for advancing community outcomes are well-recognized, challenges with the uptake of EBPs are considerable. Technical assistance (TA) is a core capacity building strategy that has been widely used to support EBP implementation and other community development and improvement efforts. Yet despite growing reliance on TA, no reviews have systematically examined the evaluation of TA across varying implementation contexts and capacity building aims. This study draws on two decades of peer-reviewed publications to summarize the evidence on the evaluation and effectiveness of TA. Methods Guided by Arksey and O’Malley’s six-stage methodological framework, we used a scoping review methodology to map research on TA evaluation. We included peer-reviewed articles published in English between 2000 and 2020. Our search involved five databases: Business Source Complete, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PsycInfo, and PubMed. Results A total of 125 evaluation research studies met the study criteria. Findings indicate that publications have increased over the last two decades, signaling a growth in the recognition and reporting of TA. Technical assistance is being implemented across diverse settings, often serving socially vulnerable and under-resourced populations. Most evaluation research studies involved summative evaluations, with TA outcomes mostly reported at the organizational level. Only 5% of the studies examined sustainability of TA outcomes. This review also demonstrates that there is a lack of consistent standards regarding the definition of TA and the level of reporting across relevant TA evaluation categories (e.g., cadence of contact, and directionality). Conclusions Advances in the science and practice of TA hinge on understanding what aspects of TA are effective and when, how, and for whom these aspects of TA are effective. Addressing these core questions requires (i) a standard definition for TA; (ii) more robust and rigorous evaluation research designs that involve comparison groups and assessment of direct, indirect, and longitudinal outcomes; (iii) increased use of reliable and objective TA measures; and (iv) development of reporting standards. We view this scoping review as a foundation for improving the state of the science and practice of evaluating TA.
... Budgetconstrained TA providers often use group-based regional trainings and webinars that are not individualized or evidence-based [16]. Even individualized TA typically does not determine if coalitions are selfdiagnosing correctly or addressing relevant capacities [17,18]. Coalitions struggling to implement EBPs are often disconnected and do not reach out for help [19]. ...
Article
Full-text available
Background Over 5000 community anti-drug coalitions operating in the USA serve as a cornerstone of federal drug prevention. These coalitions, however, have demonstrated effectiveness in preventing substance use only when they use technical assistance (TA) and implement evidence-based programs (EBPs). The absence of TA and EBP implementation by coalitions is a key research-to-practice gap. The Coalition Check-Up TA system is designed to fill this gap by supporting community coalition implementation of EBPs. Existing TA models for evidence-based coalition approaches are resource intensive and coalition model specific. The Coalition Check-Up is a lower cost strategy that works with a variety of types of coalitions to support sustainable implementation of EBPs. This study protocol describes a hybrid type 3 effectiveness-implementation trial applying Wandersman’s Interactive Systems Framework to test the effects of the Coalition Check-Up on coalition EBP implementation capacity and outcomes. The Interactive Systems Framework outlines how the prevention support system—especially TA—bolsters EBP dissemination and implementation. Methods Using a cluster randomized controlled design, this trial will test the overall effectiveness of the Coalition Check-Up, including how it contributes to EBP implementation and prevention of youth substance use. The first aim is to estimate the impact of the Coalition Check-Up on coalitions’ capacity to do their work. We will recruit 68 anti-drug coalitions for random assignment to the Coalition Check-Up or “TA as usual” condition. We will evaluate whether the Coalition Check-Up improves coalition capacity using measures of coalition member responses about team processes, coalition network composition, and collaborative structure. Our second aim is to estimate the impact of the Coalition Check-Up on implementation of EBPs, and our third aim is to estimate the impact of the Coalition Check-Up on youth substance use. Discussion This project will clarify how the Coalition Check-Up, a scalable approach to TA due to its low cost, affects coalition capacity to support EBP implementation. Analyses also provide insight into causal pathways from the prevention support system to the prevention delivery system outlined by the Interactive Systems Framework. Results will build the evidence-base for how to support community coalitions’ sustainable implementation of evidence-based prevention programs and policies. Trial registration Clinicaltrials.gov registration number NCT04592120 . Registered on October 19, 2020.
... To date there is no standard agreement on the definition or essential elements of what comprises a TA intervention (9,10). A content analysis of working definitions found three overarching common elements: (1) capacity building (11)(12)(13), (2) quality implementation (14), and 3) quality improvement (QI) (15). However, the field lacks consensus on how to effectively implement TA interventions (16). ...
Article
Full-text available
Introduction: The Accountable Health Communities (AHC) Model was designed to address the health-related social needs of Centers for Medicare & Medicaid Services beneficiaries. Bridge organizations across the AHC Model have identified lack of technical assistance and peer planning as potential barriers to Model success, particularly around patient navigation. The technical assistance and peer planning literature lacks an organizing, conceptual framework, but implementation science frameworks could serve as useful guides. The Strengthening Peer AHC Navigation (SPAN) research protocol seeks to fill this gap and will apply three implementation science frameworks, Consolidated Framework for Implementation Research, Intervention Mapping, and the Expert Recommendations for Implementing Change compilation, to develop a multi-level quality improvement intervention and evaluate the impact of peer planning on Model outcomes. The aims of the SPAN study are to implement and evaluate a novel multi-level quality improvement intervention to improve AHC implementation and navigation milestones through structured peer planning and to provide successful technical assistance for the AHC Model. Methods and Analysis: The quality improvement intervention is outlined in four Tasks: (1) Assessment – to conduct an assessment of each bridge organization's current implementation, needs, and readiness in AHC Model navigation activities; (2) Planning – to engage in a peer planning approach to build capacity for AHC Model navigation activities; (3) Implementation with technical assistance – Co-creation of a quality improvement protocol for AHC Model navigation activities; and (4) Evaluation – measure the impact of the peer planning and technical assistance approach. Alongside the development and implementation of the quality improvement intervention, this protocol describes a mixed method, convergent parallel study design which will be used to evaluate whether the quality improvement intervention will lead to better outcomes. Tasks will be replicated with five bridge organizations participating in the AHC Model. Discussion: This research protocol provides a framework that can be used to conduct structured peer planning with technical assistance for social needs programs. This study will provide data on both implementation and outcomes which eventually may impact healthcare cost and utilization.
... Currently, there are several community-based systems that use an RPF-focused approach to guide the selection, implementation, and/or evaluation of preventive interventions for youth (Chinman et al., 2008;Hunter et al., 2009;Orwin et al., 2012;Osgood et al., 2013;Greenberg et al., 2015). One of the most widely used systems to prevent health and behavior problems in youth is Communities That Care (CTC; see www.com munitiesthatcare.net), a prevention service delivery system that incorporates the community diagnosis model. ...
Article
Many effective community health service delivery systems implemented in the USA assess risk and protective factors (RPFs) for youth problem behaviors in a community, and report these data back to local coalitions for prevention planning. This study examined whether community prevention coalitions in Chile and Colombia perceived these reports of RPFs-based on the results of the Communities That Care Youth Survey-to be understandable, valid, useful, and worth disseminating. Thematic content analysis was used to analyze qualitative data collected from 7 focus groups with 75 coalition members. Results indicated heterogeneity between and within coalitions in terms of partici-pants' understanding of RPFs. However, most participants found reports of RPFs to be easy to understand, thorough, 'true' to their communities, and useful for diagnosing and prioritizing needs, action planning, and mobilizing others in their communities. Findings suggest the viability of preventive systems that rely on community-level RPF data, for use in Latin America.
... Often proposed factors and interventions are themselves unclear, a related but distinct issue (Section 3, Appendix A). The evidence base is an important facet of programme theory as well [7,8,14,28,29] both in terms of articulating how components of the logic model are evidenced [14,28] as well as identifying the source and strength of the evidence [28,29] (recognising that even where logic models are evidence-based, this understanding can guide evaluation [30]). ...
... Often proposed factors and interventions are themselves unclear, a related but distinct issue (Section 3, Appendix A). The evidence base is an important facet of programme theory as well [7,8,14,28,29] both in terms of articulating how components of the logic model are evidenced [14,28] as well as identifying the source and strength of the evidence [28,29] (recognising that even where logic models are evidence-based, this understanding can guide evaluation [30]). ...
... Further research is needed to investigate whether expert facilitation or technical assistance can encourage engagement in programme theory and iterative development and use of the programme theory diagrams to support constructive dialogue and exchange of tacit knowledge between stakeholders, and to reduce the cognitive burden associated with diagram construction. Facilitation or technical assistance could lead to more substantial improvements in developing factors and causeeffect chains that are less reliant on tacit knowledge, link to the evidence base and build a robust evaluation framework [29]. This is theorised in the literature to lead to improvements in team functioning and buy-in to the improvement project as well as aiding the spread of success to other environments and initiatives [22,23,28]. ...
Article
Full-text available
Background: Despite criticisms that many quality improvement (QI) initiatives fail due to incomplete programme theory, there is no defined way to evaluate how programme theory has been articulated. The objective of this research was to develop, and assess the usability and reliability of scoring criteria to evaluate programme theory diagrams. Methods: Criteria development was informed by published literature and QI experts. Inter-rater reliability was tested between two evaluators. About 63 programme theory diagrams (42 driver diagrams and 21 action-effect diagrams) were reviewed to establish whether the criteria could support comparative analysis of different approaches to constructing diagrams. Results: Components of the scoring criteria include: assessment of overall aim, logical overview, clarity of components, cause-effect relationships, evidence and measurement. Independent reviewers had 78% inter-rater reliability. Scoring enabled direct comparison of different approaches to developing programme theory; action-effect diagrams were found to have had a statistically significant but moderate improvement in programme theory quality over driver diagrams; no significant differences were observed based on the setting in which driver diagrams were developed. Conclusions: The scoring criteria summarise the necessary components of programme theory that are thought to contribute to successful QI projects. The viability of the scoring criteria for practical application was demonstrated. Future uses include assessment of individual programme theory diagrams and comparison of different approaches (e.g. methodological, teaching or other QI support) to produce programme theory. The criteria can be used as a tool to guide the production of better programme theory diagrams, and also highlights where additional support for QI teams could be needed.
... Process category 2: Educate at all levels Strategy 5: Training and technical assistance [75][76][77] We provided multiple trainings for prescribing medical providers, behavioral health providers, and general clinic staff on the SUD treatments. Trainings were tailored to specific staff roles within the organization and were as follows: ...
Article
Full-text available
Background Millions of people with substance use disorders (SUDs) need, but do not receive, treatment. Delivering SUD treatment in primary care settings could increase access to treatment because most people visit their primary care doctors at least once a year, but evidence-based SUD treatments are underutilized in primary care settings. We used an organizational readiness intervention comprised of a cluster of implementation strategies to prepare a federally qualified health center to deliver SUD screening and evidence-based treatments (extended-release injectable naltrexone (XR-NTX) for alcohol use disorders, buprenorphine/naloxone (BUP/NX) for opioid use disorders and a brief motivational interviewing/cognitive behavioral –based psychotherapy for both disorders). This article reports the effects of the intervention on key implementation outcomes. Methods To assess changes in organizational readiness we conducted pre- and post-intervention surveys with prescribing medical providers, behavioral health providers and general clinic staff (N = 69). We report on changes in implementation outcomes: acceptability, perceptions of appropriateness and feasibility, and intention to adopt the evidence-based treatments. We used Wilcoxon signed rank tests to analyze pre- to post-intervention changes. ResultsAfter 18 months, prescribing medical providers agreed more that XR-NTX was easier to use for patients with alcohol use disorders than before the intervention, but their opinions about the effectiveness and ease of use of BUP/NX for patients with opioid use disorders did not improve. Prescribing medical providers also felt more strongly after the intervention that XR-NTX for alcohol use disorders was compatible with current practices. Opinions of general clinic staff about the appropriateness of SUD treatment in primary care improved significantly. Conclusions Consistent with implementation theory, we found that an organizational readiness implementation intervention enhanced perceptions in some domains of practice acceptability and appropriateness. Further research will assess whether these factors, which focus on individual staff readiness, change over time and ultimately predict adoption of SUD treatments in primary care.