[Show abstract][Hide abstract] ABSTRACT: Classrooms are unique and complex work settings in which teachers and students both participate in and contribute to classroom processes. This article describes the measurement phase of a study that examined the social ecology of urban classrooms. Informed by the dimensions and items of an established measure of organizational climate, we designed the Student Climate Survey (n = 53 items) to assess student psychological climate in third through eighth grades. We administered the survey to 621 students at three time points within one school year in 69 classrooms within eight urban schools. A multidimensional item response theory (IRT) analysis based on a full-information item bifactor model revealed 18 items that loaded on a primary factor and demonstrated good criterion and predictive validity. Opportunities for the Student Climate Survey to advance our contextual understanding of urban classrooms and inform intervention are discussed.
Full-text · Article · Feb 2015 · The Journal of Early Adolescence
[Show abstract][Hide abstract] ABSTRACT: Effective evaluation of treatment requires the use of measurement tools producing reliable scores that can be used to make valid decisions about the outcomes of interest. Therapist-rated treatment outcome scores that are obtained within the context of empirically supported treatments (ESTs) could provide clinicians and researchers with data that are easily accessible and complimentary to existing instrumentation. We examined the psychometric properties of scores from the Therapist Perception of Treatment Outcome: Youth Antisocial Behavior (TPTO:YAB), an instrument developed to assess therapist judgments of treatment success among families participating in an EST, Multisystemic Therapy (MST), for youth with antisocial behavior problems. Data were drawn from a longitudinal study of MST. The initial 20-item TPTO:YAB was completed by therapists of 111 families at midtreatment and 163 families at treatment termination. Rasch model dimensionality analyses provided evidence for 2 dimensions reflecting youth- and caregiver-related aspects of treatment outcome, although a bifactor analyses suggested that these dimensions reflected a single more general construct. Rasch analyses were also used to assess item and rating scale characteristics and refine the number of items. These analyses suggested items performed similarly across time and that scores reflect treatment outcome in similar ways at mid and posttreatment. Multilevel and zero-order analyses provided evidence for the validity of TPTO:YAB scores. TPTO:YAB scores were moderately correlated with scores of youth and caregiver behaviors targeted in treatment, adding support to its use as a treatment outcome measurement instrument. (PsycINFO Database Record (c) 2015 APA, all rights reserved).
No preview · Article · Feb 2015 · Psychological Assessment
[Show abstract][Hide abstract] ABSTRACT: Given the significant rates and deleterious consequences of childhood sexual abuse (CSA), identifying effective primary prevention approaches is a clear priority. There is a growing awareness that childcare professionals (e.g., teachers, childcare personnel, clergy) are in a unique position to engage in prevention efforts due to high accessibility to children and expertise in child development. However, CSA prevention programs targeting childcare professionals have received insufficient attention. The goal of this study was to conduct an independent multi-site controlled evaluation of an existing CSA prevention program, Stewards of Children, offered through both in-person and web-based formats. This study included 352 childcare professionals recruited from children's advocacy centers across three states. Participants were randomly assigned to one of three conditions: (1) in-person training, (2) web-based training, or (3) waitlist control. Dependent variables included CSA knowledge, CSA attitudes, and self-reported CSA preventive behaviors. Results indicated that Stewards impacted knowledge, attitudes, and preventive behaviors. No differences were found between training modalities (i.e., in-person versus web-based) on knowledge and preventive behaviors. Results indicate that brief trainings for childcare professionals may impact CSA prevention efforts.
[Show abstract][Hide abstract] ABSTRACT: One challenge to research on the implementation of effective psychosocial treatments is how to define and measure fidelity at the program level. The purpose of this study was to evaluate an approach to defining, measuring, and observing over time fidelity at the program level for Multisystemic Therapy (MST). For this study, program fidelity was conceptualized as a program’s performance on several key areas identified in the literature as contributing to sustainability. A composite index, the MST Program Performance Index (PPI), was developed using data from the MST quality assurance system. The PPI included indicators of treatment adherence, treatment completion, program operations, program capacity, clinical supervisor leadership, and stakeholder relationships. A PPI score was calculated for 496 MST teams every 6 months for a two year period, during which time, the teams served over 25,000 young people. The predictive validity of the PPI score was supported by both client- and team-level outcomes. Specifically, youth treated by teams with lower PPI scores were more likely to be rearrested during their course of treatment. Likewise, teams with lower PPI scores were more likely to close during the time period covered by the study. Analysis of scores over the two year period showed substantial within-team variability in the PPI; however, the scores did not follow a linear pattern of change. In summary, preliminary evidence suggests the PPI may be a useful tool to index program-level fidelity for comparative purposes and as an additional tool for a decision support system. (PsycINFO Database Record (c) 2014 APA, all rights reserved)
Preview · Article · Jan 2014 · Zeitschrift für Psychologie
[Show abstract][Hide abstract] ABSTRACT: Juvenile offenders with substance use problems are at high risk for deleterious long-term outcomes. This study evaluated the capacity of a promising vocational and employment training program in the building sector (i.e., Community Restitution Apprenticeship-Focused Training, CRAFT) to mitigate such outcomes through enhanced employment and education. Participants were 97 high-risk juvenile offenders (mean age=15.8years) randomized to CRAFT versus education as usual (EAU) intervention conditions. Multi-method procedures measured employment, education, substance use, mental health, and criminal outcomes through a 30-month post-baseline follow-up. CRAFT was significantly more effective than EAU at increasing rates of youth employment and GED attendance. Intervention effects were not observed, however, for months employed, hours worked, or hourly wage. Measures of youth substance use, mental health symptoms, and criminal activity showed no favorable or iatrogenic effects. The potential of CRAFT was modestly supported, and suggestions were made for future research.
No preview · Article · Aug 2013 · Journal of substance abuse treatment
[Show abstract][Hide abstract] ABSTRACT: The effects of three increasingly intensive training methods on therapist use, knowledge, and implementation adherence of contingency management (CM) with substance abusing adolescents were evaluated. Ten public sector substance abuse or mental health provider organizations were randomized to one of three training conditions: workshop and resources (WS+), WS+and computer assisted training (WS+/CAT), or WS+/CAT and supervisory support (WS+/CAT/SS). Across conditions, 161 therapists participated in the training experiences, and measures were obtained at baseline and 2-month intervals for 12months following workshop participation. Across training conditions, therapists reported increased CM use, knowledge, and implementation adherence through the 12-month follow-up. The findings show that community-based practitioners are amenable to the adoption of evidence-based treatments when provided access to useful resources. Moreover, high quality workshops in combination with resource access can increase knowledge of the evidence-based treatment and might enhance intervention adherence to a level needed to improve youth outcomes.
No preview · Article · Aug 2013 · Journal of substance abuse treatment
[Show abstract][Hide abstract] ABSTRACT: Objective:
This study evaluated the accuracy of youth, caregiver, therapist, and trained raters relative to treatment experts on ratings of therapist adherence to a substance abuse treatment protocol for adolescents.
Adherence ratings were provided by youth and caregivers in an ongoing trial evaluating a Contingency Management (CM) intervention for youth in juvenile drug court. These ratings were compared to those provided by therapists and trained raters, and each rater type was compared to ratings provided by CM treatment experts. Data were analyzed using item-response-theory-based Many-Facet Rasch Models.
Relative to treatment experts, youth and caregivers were significantly more likely to endorse the occurrence of CM components. In contrast, therapists and trained raters were much more consistent with treatment experts. In terms of practical significance, youth and caregivers each had a 97% estimated probability of indicating that a typical treatment component had occurred. By comparison, the probability was 31%, 19%, and 26% for therapists, trained raters, and treatment experts, respectively.
Youth and caregivers were highly inaccurate relative to treatment experts, whereas, therapists and trained raters were generally consistent with treatment experts. The implications of these findings for therapist adherence measurement are considered.
Full-text · Article · May 2013 · Journal of Consulting and Clinical Psychology
[Show abstract][Hide abstract] ABSTRACT: Abstract Adherence to antiretroviral medication for the treatment of HIV is a significant predictor of virologic suppression and is associated with dramatic reductions in mortality and morbidity and other improved clinical outcomes for pediatric patient populations. Effective strategies for addressing adherence problems in youth infected with HIV are needed and require significant attention to the complex interplay of multiple, interacting causal risk factors that lead to poor self-care. Within the context of a pilot randomized trial, we evaluated the feasibility and initial efficacy of a multisystemic therapy (MST) intervention adapted to address HIV medication adherence problems against a usual care condition that was bolstered with a single session of motivational interviewing (MI). For 34 participating youth, health outcomes (viral load [VL] and CD4 count) were obtained from approximately 10 months pre-baseline through approximately 6 months post-baseline and self-reported medication adherence outcomes were obtained quarterly from baseline through 9 months post-baseline. Using mixed-effects regression models we examined within- and between-groups differences in the slopes of these outcomes. Feasibility was supported, with a 77% recruitment rate and near-maximal treatment and research retention and completion rates. Initial efficacy also was supported, with the MST condition but not the MI condition demonstrating statistically and clinically significant VL reductions following the start of treatment. There was also some support for improved CD4 count and self-reported medication adherence for the MST but not the MI condition. MST was successfully adapted to improve the health outcomes of youth poorly adherent to antiretroviral medications. Replication trials and studies designed to identify the mechanisms of action are important next steps.
[Show abstract][Hide abstract] ABSTRACT: A prospective multi-site study examined organizational climate and structure effects on the behavior and functioning of delinquent youth with and without co-occurring substance treated with an evidence-based treatment for serious antisocial behavior (i.e., Multisystemic Therapy). Participants were 1979 youth treated by 429 therapists across 45 provider organizations in North America. Results of Mixed Effects Regression Models showed some aspects of climate and structure had no effects, some had similar effects, and some had slightly differential and sometimes counter-intuitive effects on the outcomes of these youth. Implications are considered for research to increase the array and availability of effective treatments for youth with co-occurring substance use across service sectors.
[Show abstract][Hide abstract] ABSTRACT: Debate continues about the extent to which postulated mechanisms of action of cognitive behavior therapies (CBT), including standard CBT (i.e., Beckian cognitive therapy [CT]) and acceptance and commitment therapy (ACT) are supported by mediational analyses. Moreover, the distinctiveness of CT and ACT has been called into question. One contributor to ongoing uncertainty in this arena is the lack of time-varying process data. In this study, 174 patients presenting to a university clinic with anxiety or depression who had been randomly assigned to receive either ACT or CT completed an assessment of theorized mediators and outcomes before each session. Hierarchical linear modeling of session-by-session data revealed that increased utilization of cognitive and affective change strategies relative to utilization of psychological acceptance strategies mediated outcome for CT, whereas for ACT the mediation effect was in the opposite direction. Decreases in self-reported dysfunctional thinking, cognitive "defusion" (the ability to see one's thoughts as mental events rather than necessarily as representations of reality), and willingness to engage in behavioral activity despite unpleasant thoughts or emotions were equivalent mediators across treatments. These results have potential implications for the theoretical arguments behind, and distinctiveness of, CT and ACT.
[Show abstract][Hide abstract] ABSTRACT: The primary purpose of this study was to test a relatively efficient strategy for enhancing the capacity of juvenile drug courts (JDC) to reduce youth substance use and criminal behavior by incorporating components of evidence-based treatments into their existing services.
Six JDCs were randomized to a condition in which therapists were trained to deliver contingency management in combination with family engagement strategies (CM-FAM) or to continue their usual services (US). Participants included 104 juvenile offenders (average age = 15.4 years; 83% male; 57% White, 40% African American, 3% Biracial). Eighty-six percent of the youths met criteria for at least 1 substance use disorder, and co-occurring psychiatric diagnoses were highly prevalent. Biological and self-report measures of substance use and self-reported delinquency were assessed from baseline through 9 months postrecruitment.
CM-FAM was significantly more effective than US at reducing marijuana use, based on urine drug screens, and at reducing both crimes against persons and property offenses. Such favorable outcomes, however, were not observed for the self-report measure of substance use. Although some variation in outcomes was observed between courts, the outcomes were not moderated by demographic characteristics or co-occurring psychiatric disorders.
The findings suggest that JDC practices can be enhanced to improve outcomes for participating juvenile offenders. A vehicle for promoting such enhancements might pertain to the development and implementation of program certification standards that support the use of evidence-based interventions by JDCs. Such standards have been fundamental to the successful transport of evidence-based treatments of juvenile offenders.
Full-text · Article · Feb 2012 · Journal of Consulting and Clinical Psychology
[Show abstract][Hide abstract] ABSTRACT: This study examined the system-level effects of implementing a promising treatment for adolescent substance abuse in juvenile drug courts (JDCs). Six JDCs were randomized to receive training in the experimental intervention (contingency management-family engagement [CM-FAM]) or to continue their usual services (US). Participants were 104 families served by the courts, 51 therapists, and 74 JDC stakeholders (e.g., judges, prosecutors, defense attorneys). Assessments included repeated measurements of CM-FAM implementation by therapists and therapist and stakeholder perceptions of incentive-based interventions and organizational characteristics. Results revealed greater use of CM and family engagement techniques among CM-FAM relative to US therapists. In addition, therapists and stakeholders in the CM-FAM condition reported more favorable attitudes toward the use of incentives and greater improvement on several domains of organizational functioning relative to US counterparts. Taken together, these findings suggest that JDC professionals are amenable to the adoption and implementation of a treatment model that holds promise for improving youth outcomes.
Full-text · Article · Dec 2011 · Journal of substance abuse treatment
[Show abstract][Hide abstract] ABSTRACT: To extend the reach, transparency, and accountability for the implementation and outcomes of effective treatments in routine care, more clarity is needed about what happens in treatment. We attempt to (a) clarify terminology to describe and measure psychological treatment, and (b) consider what treatment adherence instruments can tell us about what happens in treatment. We reviewed the content of 11 adherence instruments for 14 evidence-based treatments for disruptive behavior problems in youth identified in an ongoing review of adherence measurement methods used in psychosocial treatment studies from 1980 - 2008. Item number, content, and level of detail varied widely. Implications are considered for the definition of effective treatments and design and testing of strategies to measure and monitor their delivery.
Full-text · Article · Dec 2011 · Clinical Psychology Science and Practice
[Show abstract][Hide abstract] ABSTRACT: A better understanding of clinicians’ attitudes toward evidence-based treatments (EBT) will presumably enhance the transfer of EBTs for substance-abusing adolescents from research to clinical application. The reliability and validity of two measures of therapist attitudes toward EBT were examined: the Evidence-Based Practice Attitude Scale (Aarons, 20041.
Aarons , G. A. ( 2004 ). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research , 6 , 61 – 74 . [CrossRef], [PubMed]View all references), and Attitudes Toward Psychotherapy Treatment Manuals Scale (Addis & Krasnow, 20003.
Addis , M. E. , & Krasnow , A. D. ( 2000 ). A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology , 68 , 331 – 339 . [CrossRef], [PubMed], [Web of Science ®], [CSA]View all references). Participants included 543 public sector, master's-level mental health and substance abuse therapists who treat adolescents. Factor analyses generally corroborated factor structures of the instruments found previously. Beliefs that EBTs negatively affect treatment process were associated with relatively low openness to new treatments and with beliefs that EBTs do not produce positive outcomes.
[Show abstract][Hide abstract] ABSTRACT: The current study investigated relations among ethnic similarity in caregiver-therapist pairs of youth participating in Multisystemic Therapy, therapist adherence, and youth long-term behavioral and criminal outcomes. Participants were 1,979 youth and families treated by 429 therapists across provider organizations in 45 sites. Relations were found, independently, and in the presence of ethnic similarity, between adherence and reductions in youth externalizing and internalizing behavior problems 1-year posttreatment and youth criminal charges 4 years posttreatment. Relations between ethnic similarity and outcomes were found only for reductions in youth externalizing behavior problems and not when adherence was included in the model. Adherence ratings were higher, however, in ethnically similar caregiver-therapist pairs, and evidence was found that this increased adherence predicted slightly better outcomes for youth. Implications for further research and clinical practice are considered.
No preview · Article · Mar 2011 · Journal of Emotional and Behavioral Disorders
[Show abstract][Hide abstract] ABSTRACT: Using data from a recent randomized clinical trial involving juvenile drug court (JDC), youth marijuana use trajectories and the predictors of treatment nonresponse were examined. Participants were 118 juvenile offenders meeting diagnostic criteria for substance use disorders assigned to JDC and their families. Urine drug screen results were gathered from weekly court visits for 6 months, and youth reported their marijuana use over 12 months. Semiparametric mixture modeling jointly estimated and classified trajectories of both marijuana use indices. Youth were classified into responder versus nonresponder trajectory groups based on both outcomes. Regression analyses examined pretreatment individual, family, and extrafamilial predictors of nonresponse. Results indicated that youth whose caregivers reported illegal drug use pretreatment were almost 10 times as likely to be classified into the nonresponder trajectory group. No other variable significantly distinguished drug use trajectory groups. Findings have implications for the design of interventions to improve JDC outcomes.
Preview · Article · Dec 2010 · Journal of substance abuse treatment
[Show abstract][Hide abstract] ABSTRACT: Implementation science in mental health is informed by other academic disciplines and industries. Conceptual and methodological territory charted in psychotherapy research is pertinent to two elements of the conceptual model of implementation posited by Aarons and colleagues (2010)--implementation fidelity and innovation feedback systems. Key characteristics of scientifically validated fidelity instruments, and of the feasibility of their use in routine care, are presented. The challenges of ensuring fidelity measurement methods are both effective (scientifically validated) and efficient (feasible and useful in routine care) are identified as are examples of implementation research attempting to balance these attributes of fidelity measurement.
Full-text · Article · Oct 2010 · Administration and Policy in Mental Health and Mental Health Services Research
[Show abstract][Hide abstract] ABSTRACT: A randomized trial assessed the effectiveness of a 2-level strategy for implementing evidence-based mental health treatments for delinquent youth.
A 2 x 2 design encompassing 14 rural Appalachian counties included 2 factors: (a) the random assignment of delinquent youth within each county to a multisystemic therapy (MST) program or usual services and (b) the random assignment of counties to the ARC (for availability, responsiveness, and continuity) organizational intervention for implementing effective community-based mental health services. The design created 4 treatment conditions (MST plus ARC, MST only, ARC only, control). Outcome measures for 615 youth who were 69% male, 91% Caucasian, and aged 9-17 years included the Child Behavior Checklist and out-of-home placements.
A multilevel, mixed-effects, regression analysis of 6-month treatment outcomes found that youth total problem behavior in the MST plus ARC condition was at a nonclinical level and significantly lower than in other conditions. Total problem behavior was equivalent and at nonclinical levels in all conditions by the 18-month follow-up, but youth in the MST plus ARC condition entered out-of-home placements at a significantly lower rate (16%) than youth in the control condition (34%).
Two-level strategies that combine an organizational intervention such as ARC and an evidence-based treatment such as MST are promising approaches to implementing effective community-based mental health services. More research is needed to understand how such strategies can be used effectively in a variety of organizational contexts and with other types of evidence-based treatments.
Preview · Article · Aug 2010 · Journal of Consulting and Clinical Psychology
[Show abstract][Hide abstract] ABSTRACT: This nonexperimental study used mixed-effects regression models to examine relations among supervisor adherence to a clinical supervision protocol, therapist adherence, and changes in the behavior and functioning of youths with serious antisocial behavior treated with an empirically supported treatment (i.e., multisystemic therapy [MST]) 1 year posttreatment. Participants were 1,979 youths and families treated by 429 clinicians across 45 provider organizations in North America. Four dimensions of clinical supervision were examined. Mixed-effects regression model results showed that one dimension, supervisor focus on adherence to treatment principles, predicted greater therapist adherence. Two supervision dimensions, Adherence to the Structure and Process of Supervision and focus on Clinician Development, predicted changes in youth behavior. Conditions required to test hypothesized mediation by therapist adherence of supervisor adherence effects on youth outcomes were not met. However, direct effects of supervisor and therapist adherence were observed in models including both of these variables.
Full-text · Article · Jul 2009 · Journal of Consulting and Clinical Psychology