Efficacy vs effectiveness trial results of an indicated "model" substance abuse program: implications for public health.

Pacific Institute for Research and Evaluation, Chapel Hill, NC, USA.
American Journal of Public Health (Impact Factor: 4.23). 01/2007; 96(12):2254-9. DOI: 10.2105/AJPH.2005.067462
Source: PubMed

ABSTRACT The US Department of Education requires schools to choose substance abuse and violence prevention programs that meet standards of effectiveness. The Substance Abuse and Mental Health Services Agency certifies "model" programs that meet this standard. We compared findings from a large, multisite effectiveness trial of 1 model program to its efficacy trial findings, upon which the certification was based.
1370 high-risk youths were randomized to experimental or control groups across 9 high schools in 2 large urban school districts. We used intent-to-treat and on-treatment approaches to examine baseline equivalence, attrition, and group differences in outcomes at the end of the program and at a 6-month follow-up.
Positive efficacy trial findings were not replicated in the effectiveness trial. All main effects were either null or worse for the experimental than for the control group.
These findings suggest that small efficacy trials conducted by developers provide insufficient evidence of effectiveness. Federal agencies and public health scientists must work together to raise the standards of evidence and ensure that data from new trials are incorporated into ongoing assessments of program effects.

Download full-text


Available from: Hyunsan Cho, Jun 27, 2014
1 Follower
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This study examined outcomes associated with the Family Check-Up (FCU), an adaptive, tailored, family-centered intervention to enhance positive adjustment of middle school youth and prevent problem behavior. The FCU intervention model was delivered to families in 3 public middle schools. The study sample comprised 377 families, and participants were randomly assigned to receive either the intervention or school as usual. Participation in the intervention was relatively high, with 38% of the families receiving the FCU. Participation in the intervention improved youth self-regulation over the 3 years of the study. Self-regulation skills, defined as effortful control, predicted both decreased depression and increased school engagement in high school, with small to medium effect sizes. The results have implications for the delivery of mental health services in schools that specifically target family involvement and parenting skills.
    School Mental Health 06/2010; 2(2):82-92. DOI:10.1007/s12310-009-9025-6
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This study explores the possibility that any drug prevention program might be considered ;;evidence-based'' given the use of data analysis procedures that optimize the chance of producing statistically significant results by reanalyzing data from a Drug Abuse Resistance Education (DARE) program evaluation. The analysis produced a number of statistically significant differences between the DARE and control conditions on alcohol and marijuana use measures. Many of these differences occurred at cutoff points on the assessment scales for which post hoc meaningful labels were created. Our results are compared to those from evaluations of programs that appear on evidence-based drug prevention lists.
    Evaluation Review 05/2009; 33(4):396-414. DOI:10.1177/0193841X09334711 · 1.20 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper describes evidence-based kernels, fundamental units of behavioral influence that appear to underlie effective prevention and treatment for children, adults, and families. A kernel is a behavior-influence procedure shown through experimental analysis to affect a specific behavior and that is indivisible in the sense that removing any of its components would render it inert. Existing evidence shows that a variety of kernels can influence behavior in context, and some evidence suggests that frequent use or sufficient use of some kernels may produce longer lasting behavioral shifts. The analysis of kernels could contribute to an empirically based theory of behavioral influence, augment existing prevention or treatment efforts, facilitate the dissemination of effective prevention and treatment practices, clarify the active ingredients in existing interventions, and contribute to efficiently developing interventions that are more effective. Kernels involve one or more of the following mechanisms of behavior influence: reinforcement, altering antecedents, changing verbal relational responding, or changing physiological states directly. The paper describes 52 of these kernels, and details practical, theoretical, and research implications, including calling for a national database of kernels that influence human behavior.
    Clinical Child and Family Psychology Review 10/2008; 11(3):75-113. DOI:10.1007/s10567-008-0036-x · 4.75 Impact Factor