Content uploaded by Michael Gass
Author content
All content in this area was uploaded by Michael Gass
Content may be subject to copyright.
Private Therapeutic Programs, p. 1
The evidence base for
private therapeutic schools, residential programs, and wilderness therapy programs
Ellen Behrens, Ph. D., Canyon Research & Consulting
John Santa, Ph. D., Montana Academy
Michael Gass, Ph. D., University of New Hampshire
Members of the Standing Research Committee of
The National Association of Therapeutic Schools and Programs
In the past 25 years adolescent risk factors have dramatically increased. Approximately one-third
of public high school students fail to graduate (National Center for Higher Education
Management Systems, 2006). Drug use is rampant in middle and high school. More children
have been diagnosed with mood, anxiety, attention, substance, and behavior disorders than ever
before (Substance Abuse and Mental Health Services Administration, 2006). Alarmingly large
percentages of youth participate in high-risk behaviors, such as carrying weapons and attempting
suicide (Centers for Disease Control and Prevention, 2007).
Unfortunately during this period of heightened need, mental health systems have experienced an
unprecedented decline in intensive services for high-risk youth. This decrease in the availability of
adequate mental health services has been driven by a paradigm shift to crisis stabilization and
medication management designed to manage care and contain costs for insurance companies.
These real and palpable problems have led to a rapid growth of private therapeutic programs.
Private therapeutic schools, residential programs, and wilderness therapy programs are aimed at
serving the complex needs of struggling adolescents and their desperate families who have not been
helped by palliative remedies offered by outpatient therapy or short-term psychiatric
hospitalization. Enrollment in these programs is typically preceded by complex and intensive
academic, legal, substance abuse, behavioral, emotional, and familial problems when the
adolescents’ caregivers (e.g., parents, guardians, teachers, therapists) are unable to provide adequate
support. In most cases, youth treated in private therapeutic programs return home after receiving
individual and family therapy designed to facilitate their return. The National Association of
Therapeutic Schools and Programs (NATSAP) (www.natsap.org) is the primary professional
association serving private therapeutic schools, residential programs, and wilderness therapy
programs. Its member programs typically consist of “brick and mortar” programs (e.g., therapeutic
residential programs) and outdoor-based therapeutic programs. Most NATSAP member programs
are independently owned and licensed by appropriate state agencies.
The Evidence
In the past 10 years, a growing body of evidence has accumulated documenting the effectiveness of
private therapeutic schools, residential programs, and wilderness therapy programs. This body of
research is comprised of studies led by university faculty with the oversight of federally recognized
institutional review boards. Collaborative studies have been conducted among NATSAP member
programs as well as at individual NATSAP programs.
There are five major research initiatives that, combined together, provide a growing base of
research support for private therapeutic programs. These research initiatives are particularly
ambitious: many use large samples, longitudinal designs, multiple research sites, multiple reporters,
and “gold standard” outcome measures.
Private Therapeutic Programs, p. 2
Initiative # 1
The Outdoor Behavior Healthcare Research Cooperative (OBHRC) (www.obhrc.org) is a
research collaborative comprised of several wilderness therapy programs who are NATSAP
member organizations. It is currently located at the University of New Hampshire and is operated
by several research scientists. In the last decade, these researchers have generated over 100
published research studies based on outdoor programs addressing the needs of youth.
Two foundational studies within this initiative systematically explored youth outcomes in outdoor
programs. The first was conducted at seven OBHRC wilderness therapy programs
(http://obhrc.org/publications.php). Using the Youth Outcome Questionnaire (YOQ, Burlingame,
Wells, & Lambert, 1995), a commonly used measure of outcomes in mental health treatment, Keith
Russell, Ph.D. (2002, 2003a) found statistically and clinically significant reductions of behavioral
and emotional symptoms of youth immediately following treatment. A one year follow up study
with a random sample of these youth found that, on average, the gains made during the program
had been maintained.
Mean scores on Y-OQ at Admission, Discharge, and 12 Months After Discharge for
Adolescent Self Reports and Parent Assessments (Russell, 2002a, p. 29)
Rater
Sample Size
Admission
Discharge
12 Months After
Discharge
Adolescent
621
71.80
50.58
37.70
Parent
560
99.04
55.10
42.84
In a second foundational study within this initiative, Russell (2005) followed the same sample of
youth two years later and found they had maintained therapeutic progress initiated by treatment.
Furthermore, according to youth self-report data, these youth continued to improve after leaving the
program. In fact, the majority of parents and youth indicated they were doing well and parents
believed that their son or daughter could not have begun their recovery without the initial impact of
the wilderness treatment.
As with many initial quantitative studies designed to determine basic effectiveness of programming,
Dr. Russell’s work did not include various control groups that would help determine more precisely
the reasons and generalizability of the strong positive effects obtained with wilderness intervention.
Qualitatively, parents obviously did not need control groups to attest to the degree of positive
changes produced by several weeks of wilderness intervention. Dr. Russell’s work begins to
empirically document these positive effects.
Initiative # 2
Ellen Behrens, Ph.D. (2006) conducted another major, IRB-approved research initiative at nine
“brick-and-mortar” NATSAP programs owned by Aspen Education Group. Generated by repeated
surveys of nearly 1000 youth and their parents, these results were presented at the American
Psychological Association Annual Conference in 2006. Employing standardized measures of
psychosocial functioning developed by Achenbach (2001), the study found strong positive effects
of treatment on internalizing problems (i.e., depression, anxiety, attention), problematic external
behaviors (i.e., aggression, rule breaking), and overall functioning. In addition, youth academic
functioning and youth family relationships improved significantly during treatment. A study that
Private Therapeutic Programs, p. 3
followed the youth for 12 months after treatment found that the positive treatment effects were
maintained (Behrens, 2007; Behrens & Satterfield, 2007).
Problem Severity
0
10
20
30
40
50
60
70
80
90
100
Admission Discharge 3 Mo Post 6 Mo Post 12 Mo Post
Parent
Youth
These data are perhaps most striking when considered in terms of changes in youth level of
functioning. By the end of residential treatment and one year after treatment, the majority of the
youth demonstrated clinically significant improvement by shifting from the abnormal (or clinical)
range to the normal range of behavioral and psychological functioning. These results are especially
remarkable when considered in the context of other evidence-based treatments. For example, the
maintenance of gains after private residential treatment is more favorable than those reported with
two of the most highly acclaimed evidenced-based treatments for youth (i.e., Multi-systemic
Therapy and Functional Family Therapy) (see Fonagy, Target, Cottrell, Phillips & Kurtz, 2002;
U.S. Department of Health and Human Services, 2001).
Percentage of Youth in normal, moderate, and severe ranges of functioning at admission,
discharge, and one year after discharge
0
20
40
60
80
100
Admission Discharge 12 Mo Post
Normal
Moderate
Severe
Private Therapeutic Programs, p. 4
Initiative # 3
In collaboration with the University of Arkansas, Sarah Lewis, Ph.D. coordinated a program
evaluation initiative at Aspen Education Group’s wilderness therapy programs (Dixon, Leen-
Feldner, Ham, Feldner, Lewis, in press; Lewis, Rogers, Dixon, Barreto, Leen-Feldner, & Daniels,
2007; Rogers, Dixon, Barreto, Farrell, Daniels & Lewis, 2007; Rogers, et. al., 2007). This
longitudinal outcome study used the Treatment Outcome Package (TOP) questionnaire (Krause,
Seligman, & Jordan, 2005) to measure changes in participating youth. The study confirmed the
findings generated by the evaluation initiatives of Russell (2003a, 2005): wilderness therapy was
associated with statistically significant positive changes in overall functioning. Youth in the
study experienced significant decreases in suicidal ideation, anxiety, depression, substance abuse,
social conflict, sleep disruption, violence, as well as an overall reduction in externalizing
behaviors such as impulsivity, defiance, and hostility. Furthermore, these youth demonstrated
improvements in work and academic functioning during the follow-up portion of the study.
Initiative #4
Michael Gass, Ph.D. and colleagues (2009) provided a summary of preliminary analyses of the
NATSAP Outcomes Research Project. The Project involves systematic data collection from more
than 33 NATSAP programs examining the status of participants at the beginning of treatment in
residential or wilderness settings, at the conclusion of treatment, and one-year post discharge. This
large-scale study has currently collected more than 1200 participant surveys using either the Youth
Outcome (Burlingame et al., 1995) or Achenbach (Achenbach, 2001) assessments of psychiatric
and behavioral symptoms. Preliminary analysis indicates strong positive effects of program
intervention with large statistically significant reductions in both psychiatric and behavioral
symptoms from admission to discharge. The data collection process is ongoing and later analyses
will allow for a more detailed and prescriptive assessment of the effect of different types of
program intervention on a range of specific presenting problems. This large-scale database is
located at the University of New Hampshire and will be available to investigators who have various
research questions.
Initiative #5
A number of other published, well-designed studies have been conducted at single NATSAP
member programs. For example, one study conducted at The Menninger Residential Treatment
Program with a sample of 123 youth found that parents and youth reported a significant decline in
problems from admission to 3 months after completing the program, and these gains from treatment
lasted up to 12 months after completing the program (Leichtman, Leichtman, Barber, & Neese,
2001).
Another published study, conducted at Alpine Academy, employed a strong control group design
and found families reported significant improvement in child behavior, parental effectiveness, and
parent–child relationships when compared with similar difficulties in families who were referred
for the service but not served (Lewis, 2005). These changes were maintained on assessments three
months after discharge.
Wediko Children's Services, another NATSAP member program, has participated in numerous
research projects with Jack Wright, Ph.D. of Brown University and Audry Zakriski, Ph.D. of
Connecticut College. These studies examined adaptive and contextually based behavior for youth
Private Therapeutic Programs, p. 5
in residential treatment (Wright & Zakriski, 2003; Zakriski, Wright, & Parad, 2006; Zakriski,
Wright, & Underwood, 2005).
Nick Hong, Ph.D. and his colleagues (Hong & Santa, 2007; Hong & McKinnon, 2009; Hong, 2010)
at Montana Academy conducted a series of studies demonstrating marked increases in parent
ratings of their child’s maturity over the course of treatment, and these gains persisted one year
after discharge. They also demonstrated that over the course of treatment school performance
improved markedly, psychiatric and behavioral symptoms dissipated, and parents increased positive
warmth and decreased negative emotional control of their children.
Joanna Bettmann, Ph.D. has produced a series of valuable studies on the positive influences of
wilderness therapy on attachment issues for both adolescents and adults (Bettmann, 2007; Bettmann
& Jasperson, 2008; Bettmann, Demong, & Jasperson, 2008). Her work demonstrates how
wilderness therapy programs can aid in the treatment of insecure attachment cycles.
Conclusions
Private therapeutic schools, residential programs, and wilderness therapy programs possess a solid
and growing research base. This collective body of research demonstrates that participating youth
improve significantly during treatment and these improvements continue after youth return home.
These findings are based on several different research programs of study: studies that were large
scale, multi-center, and longitudinal, conducted by nationally recognized university researchers,
and reviewed by federally recognized institutional review boards. Further research of course is
needed, and encouraged, that will include a variety of control conditions aimed at refining the
explanations of the powerful treatment effects that have been revealed.
References and Recommended Reading
Achenbach, T.M. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington:
University of Vermont.
Attarian, A., Ewert, A. Hollenhorst, S., Russell, K. & Voigt, A. (2006). Evolving adventure
pursuits on public lands: Emerging challenges for management and public policy.
Journal of Parks Recreation Administration, 24, 2, 125-140. [authors acknowledge equal
contribution].
Behrens, E. (2006, August). Report of findings from a multi-center, longitudinal study of
youth outcomes in residential treatment. Paper presented at the American
Psychological Association Annual Convention, New Orleans, LA.
Behrens, E. (2007). Longitudinal Outcomes in Private, Regulated Residential Programs:
How are the youth one year later? Paper presented at the semi-annual conference of
the Independent Educational Consultants Association, Boston, MA.
Behrens, E., & Satterfield, K. (2007). Longitudinal family and academic outcomes in
residential schools: How students function in two important areas of their lives.
Journal of Therapeutic Schools & Programs, 2, 1, 81-94.
Bettmann, J.E. (2007). Changes in adolescent attachment relationships as a response to
wilderness treatment. Journal of the American Psychoanalytic Association, 55 (1),
259-265.
Private Therapeutic Programs, p. 6
Bettmann, J., & Jasperson, R. (2008). Adults in wilderness treatment: A unique application of
attachment theory and research. Clinical Social Work Journal, 36(1), 51-61.
Bettmann, J., Demong, E. & Jasperson, R. (2008). Treating adolescents with adoption and
attachment issues in wilderness therapy settings. Journal of Therapeutic Schools and
Programs, 3(1), 116-137.
Burlingame, G. M., Wells, M. G., & Lambert, M. J. (1995). The Youth Outcome Questionnaire.
Stevenson, MD: American Professional Credentialing Services.
Centers for Disease Control and Prevention. (2007). Trends in the prevalence of selected risk
behaviors for all students, National YRBSS: 1991-2007. Retrieved May 10, 2010 from
http://www.cdc.gov/HealthyYouth/yrbs/pdf/yrbs07_us_summary_trend_all.pdf.
Dixon, L.J., Leen-Feldner, E.W., Ham, L.S., Feldner, M.T., Lewis, S.F. (in press). Alcohol use
motives among traumatic event-exposed, treatment-seeking adolescents: Associations
with posttraumatic stress. Addictive Behaviors.
Fonagy, P., Target, M., Cottrell, D., Phillips, J., & Kurtz, Z. (2002). What works for whom? A
critical review of treatment of children and adolescents. Guilford: New York.
Gass, M.; Zelov, R.; & Young, M. (2009). The 2009 annual report of the NATSAP Research
Database. Bethesda, MD: The National Association for Therapeutic Schools and
Programs.
Gassner, M., Kahlid, A. & Russell, K.C. (2006). Investigating the long-term impact of adventure
education: A retrospective study of Outward Bound Singapore's classic 21-day challenge
course. In Paisley, K., Bunting, C.J., & Young, A.B. (Eds.) Research in Outdoor
Education, (Vol. 8) (pp.75-94). Cortland, NY: Coalition for Education in the Outdoors.
Gassner, M. & Russell, K. C. (2008). Relative impact of course components at Outward Bound
Singapore: a retrospective study of long-term outcomes. Journal of Adventure Education
and Outdoor Learning, 8, 2, 133-156.
Gillis, H. L. & Gass, M. A. (2010). Residential treatment of juveniles in a sex offender program
using an adventure-based behavior management program: A matched group design.
Journal of Child Sex Abuse, 19, 20-34.
Gillis, H. L., Gass, M. & Russell, K. C. (2008). The effectiveness of Project Adventure's behavior
management program for male offenders in residential treatment. Residential Treatment
for Children and Youth, 25, 3, 227-247.
Harper, N. & Russell, K. C. (2008). Family involvement and outcome in adolescent wilderness
treatment: A mixed-methods evaluation. International Journal of Child and Family
Welfare, 1, 19-36.
Harper, N., Russell, K.C., Cooley, R. & Cupples, J. (2007). Catherine Freer Wilderness Therapy
Expeditions: An exploratory case study of adolescent wilderness therapy, family
functioning, and the maintenance of change.
Child and Youth Care Forum, 36, 111-129.
Private Therapeutic Programs, p. 7
Hong, N.N. & Santa, J. (2007). Family functioning in the lives of adolescents in therapeutic
boarding schools. Symposium presentation at the Annual Meeting of the National
Association of Therapeutic Schools and Programs, La Jolla, California.
Hong, N.N. & McKinnon, J. (2009). Beyond symptomatology: Maturation in adolescents.
Symposium presentation at the Annual Meeting of the National Association of
Therapeutic Schools and Programs, Austin, Texas.
Hong, N.N. (2010). A Longitudinal Study of Change in the Quality of Adolescent Relationships
Symposium presentation at the Annual Meeting of the National Association of
Therapeutic Schools and Programs, La Jolla, California.
Kraus, D. R., Seligman, D. A., & Jordan, J. R. (2005). Validation of a behavioral health treatment
outcome and assessment tool designed for naturalistic settings: The Treatment Outcome
Package. Journal of Clinical Psychology, 61, 285 – 314.
Leichtman, M., Leichtman, M. L., Barber, C. C., & Neese, D. T. (2001). Effectiveness of
intensive short-term residential treatment with severely disturbed adolescents. American
Journal of Orthopsychiatry, 71, 2, 227-235.
Lewis, R.E. (2005). The effectiveness of Families First services: An experimental study. Children
and Youth Services Review, 27, 499-509.
Lewis, S. F. (2009, January). Evaluating Treatment Outcomes in a Sample of Adolescents: A
Prospective Study. Paper presented at the 2nd annual Aspen Education Group symposium
for referring professionals. Atlanta, GA.
Lewis, S. F., Rogers, S. M., Dixon, L. J., Barreto, C. J., Leen-Feldner, E. W., & Daniels, S. G.
(2007, October). Treatment-Resistant Adolescents: Outcome Data from a Promising
Therapeutic Intervention. Paper presented at the 54
th
Annual Convention of the
Southeastern Psychological Association, Charlotte, NC.
National Center for Higher Education Management Systems. (2006). Public high school
graduation rates. Retrieved May 14, 2010 from
http://www.higheredinfo.org/dbrowser/index.php?submeasure=36&year=2006&level=na
tion&mode=data&state=0
Rogers, S. M., Dixon, L. J., Barreto, C. J., Farrell, H. J., Daniels, S. G., & Lewis, S. F. (2007,
October). Outdoor Behavioral Healthcare: A Case Demonstration of Effectiveness.
Poster presented at the 54
th
Annual Convention of the Southeastern Psychological
Association, Charlotte, NC.
Rogers, S. M., Dixon, L. J., Barreto, C. J., Farrell, H. J., Lewis, S. F., Leen-Feldner, E. W., &
Daniels, S. G. (2007, October). Adolescent Substance Use and Abuse: The Role of
Gender and Treatment-Resistance. Poster presented at the 54
th
Annual Convention of the
Southeastern Psychological Association, Charlotte, NC.
Russell, K. C. (2000). Exploring how the wilderness therapy process relates to outcomes.
Journal
of Experiential Education, 23, 3, 170-176.
Private Therapeutic Programs, p. 8
Russell, K. C. (2001). What is wilderness therapy? Journal of Experiential Education, 24, 2, 70-
79.
Russell, K. C. (2002). A longitudinal assessment of treatment outcomes in outdoor
behavioral healthcare. Technical Report 28, Idaho Forest Wildlife and Range
Experiment Station, Moscow, ID. 35 pp.
Russell, K.C. (2003a). Assessing treatment outcomes in outdoor behavioral healthcare using the
Youth Outcome Questionnaire. Child and Youth Care Forum. 32, 6, 355-381.
Russell, K.C. (2003b). A nation-wide survey of outdoor behavioral healthcare programs for
adolescents with problem behaviors. Journal of Experiential Education, 25, 3, 322-331.
Russell, K.C. (2005). Two years later: A qualitative assessment of youth-well-being and the role
of aftercare in outdoor behavioral healthcare treatment. Child and Youth Care Forum, 34,
3, 209-239.
Russell, K.C. (2006). Evaluating the effects of the Wendigo Lake Expedition program for young
offenders. Journal of Juvenile Justice and Youth Violence, 4, 2, 185-203.
Russell, K.C. (2008). Adolescence substance use treatment: Service delivery, research on
effectiveness, and emerging treatment alternatives. Journal of Groups in Addiction and
Recovery, 2, 2-4, 68-96.
Russell, K.C. and Farnum, J. (2004). A concurrent model of the wilderness therapy process.
Journal of Adventure Education and Outdoor Learning, 4, 1, 39-55.
Russell, K. C., Gillis, H. L., & Lewis, T. G. (2008). A five-year follow-up of a nationwide survey
of outdoor behavioral healthcare programs. Journal of Experiential Education, 31, 1, 55-
77.
Russell, K. C. & Harper, N. (2006). Incident monitoring in outdoor behavioral healthcare
programs: A four-year summary of restraint, runaway, injury, and illness rates. Journal of
Therapeutic Schools and Programs, 1, 1, 70-90.
Russell, K. C., & Phillips-Miller, D. (2002). Perspectives on the wilderness therapy process and
its relation to outcome. Child and Youth Care Forum, 31, 6, 415-437.
Russell, K.C. & Sibthorp, J. (2004). Hierarchical linear modeling of treatment outcomes in
outdoor behavioral healthcare. Journal of Experiential Education, 27, 2, 176-191.
Substance Abuse and Mental Health Services Administration. (2006). Results from the 2005
National Survey on Drug Use and Health: National Findings (Office of Applied Studies,
NSDUH Series H-30, DHHS Publication No. SMA 06-4194). Rockville, MD.
U.S. Department of Health and Human Services. (2001) Youth violence: A report of the Surgeon
General. [Full text online] Retrieved may 17, 2010 from
http://www.surgeongeneral.gov/library/youthviolence
.
Private Therapeutic Programs, p. 9
White, D.D. , Caulkins, M., & Russell, K.C. (2006). The role of physical exercise in wilderness
therapy for troubled adolescent women. Journal of Experiential Education, 29,1, 18-37.
Wright, J. C., & Zakriski, A. L. (2003). When syndromal similarity obscures functional
dissimilarity: Distinctive evoked environments of externalizing and mixed syndrome
boys. Journal of Consulting and Clinical Psychology, 71, 3, 516-527.
Zakriski, A. L., Wright, J. C., & Parad, H. W. (2006). Intensive short-term residential treatment: A
contextual evaluation of the "stop-gap" model. Brown University Child & Adolescent
Behavior Letter, 22, 6.
Zakriski, A. L., Wright, J. C., & Underwood, M. K. (2005). Gender similarities and differences in
children's social behavior: Finding personality in contextualized patterns of adaptation.
Journal of Personality and Social Psychology, 88, 5, 844.