Multisystemic treatment: a meta-analysis of outcome studies.
ABSTRACT Multisystemic treatment (MST) is a family- and home-based therapeutic approach that has been found to be effective in treating antisocial youths and that has recently been applied to youths with serious emotional disturbances. In light of the increasing dissemination of MST, this review examines the effectiveness of MST by quantifying and summarizing the magnitude of effects (treatment outcomes) across all eligible MST outcome studies. Included in a meta-analysis were 7 primary outcome studies and 4 secondary studies involving a total of 708 participants. Results indicated that across different presenting problems and samples, the average effect of MST was d = .55; following treatment, youths and their families treated with MST were functioning better than 70% of youths and families treated alternatively. Results also showed that the average effect of MST was larger in studies involving graduate student therapists (i.e., efficacy studies; d = .81) than in studies with therapists from the community (i.e., effectiveness studies; d = .26). In addition, MST demonstrated larger effects on measures of family relations than on measures of individual adjustment or peer relations.
- [Show abstract] [Hide abstract]
ABSTRACT: The current investigation examined the effectiveness of Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) in treating child traumatic stress when implemented in community settings on a state-wide level. Seventy-two youths (ages 7-16 years) with a history of documented trauma (sexual or physical abuse, traumatic loss, domestic or community violence) and symptoms of Posttraumatic Stress Disorder (PTSD) received an average of 10 sessions delivered in a state-contracted mental health agency. PTSD symptoms and internalizing and externalizing behavior problems were assessed at pre-treatment and then at 3-, 6-, 9-, and 12-months after intake. Piecewise hierarchical linear modeling revealed that symptoms of PTSD, as well as internalizing and externalizing problems, decreased significantly over the six months after intake (pretreatment, 3-month, 6-month assessments), and these gains were maintained over the next 6 months (6, 9, and 12 month assessments). Symptoms of externalizing symptoms increased somewhat during the follow-up period, but this change was not statistically significant. These findings suggest that TF-CBT can be implemented effectively in community settings. Treatment outcomes were similar to those reported in efficacy trials of TF-CBT delivered in specialty clinic settings. Improvements in PTSD symptoms and internalizing and externalizing problems were maintained up to one year after treatment began, although the changes in externalizing symptoms were the least stable.Psychological Trauma Theory Research Practice and Policy 09/2014; 6(5):555-562. · 0.89 Impact Factor
- Child and Youth Care Forum 10/2014; 43(5):593-605. · 1.25 Impact Factor
Technical Report: An Audit of the Gangs in Niagara Falls, New York[Show abstract] [Hide abstract]
ABSTRACT: This report relies on official crime data, law enforcement intelligence, census data, and ethnographic interviews to assess the nature and extent of the gang problem in Niagara Falls, NY. It finds that street gangs are firmly entrenched on the streets of Niagara Falls, and that they are fueled by high poverty rates, a vibrant underground illicit economy, and the dissemination of gang culture across geographical locations. We also identify that the city of Niagara Falls has experienced escalating rates of violence that contrast national trends. We examine the extent to which street gangs may be involved in localized violence and note that interpersonal violence appears to be a growing problem in the city, but gangs are not yet involved in patterns of contagious murder. They are, however, heavily involved in drug dealing and may increasingly be demonstrating a willingness to use excessive forms of violence (guns) to resolve interpersonal disputes. Moreover, gangs and gang members appear to evolve and become more sophisticated as they remain active in street life. This report concludes with a discussion of viable solutions for dealing with street gangs in Niagara Falls.
Joumal of Family Psychology
2004. Vol. 18, No. 3, 41l—4l9
Copyright 2004 by the American Psychological Association
0893-3200/04/$12.00 D01: l0.l037l0893~3200.l8.3.4ll
Multisystemic Treatment: A Meta-Analysis of Outcome Studies
Nicola M. Curtis and Kevin R. Ronan
Charles M. Borduin
University of Missouri—Columbia
Multisystemic treatment (MST) is a family- and home-based therapeutic approach that has
been found to be effective in treating antisocial youths and that has recently been applied to
youths with serious emotional disturbances. In light of the increasing dissemination of MST,
this review examines the effectiveness of MST by quantifying and summarizing the magni-
tude of effects (treatment outcomes) across all eligible MST outcome studies. Included in a
meta-analysis were 7 primary outcome studies and 4 secondary studies involving a total of
708 participants. Results indicated that across different presenting problems and samples, the
average effect of MST was d = .55; following treatment, youths and their families treated
with MST were functioning better than 70% of youths and families treated alternatively.
Results also showed that the average effect of MST was larger in studies involving
graduate student therapists (i.e., efficacy studies; d = .81) than in studies with therapists
from the community (i.e., effectiveness studies; d = .26). In addition, MST demonstrated
larger effects on measures of family relations than on measures of individual adjustment
or peer relations.
keywords: antisocial behavior, community—based treatment, juvenile offenders, multisystemic
treatment, treatment adherence, youths
Antisocial behavior in youths represents a complex and
pervasive clinical problem, with large numbers of antisocial
youth coming to the attention of mental health, social wel-
fare, and youth justice systems throughout the western
world each year (Kazdin, 1987; Rutter, Giller, & Hagell,
1998). Recent prevalence statistics from the United States,
the United Kingdom, and New Zealand indicate that anti-
social behaviors manifest in up to 15% of young people
(Fergusson, Horwood, & Lynskey, 1997; Ofﬁce of Juvenile
Justice and Delinquency Prevention, 1999).
Treatment approaches have typically focused on different
aspects of the range of dysfunction (e.g., individual or
family factors) found in antisocial youths. Despite a large
number of available treatments, few have demonstrated
sustained effectiveness in the amelioration of serious and
pervasive antisocial behavior (Kazdin, 2000). However,
leading reviewers (e.g., Bums, Hoagwood, & Mrazek,
1999; Elliott, 1998; Farrington & Welsh, 1999; Kazdin &
Weisz, 1998; U.S. Public Health Service, 2001) of empiri-
cally supported child and adolescent treatments have noted
that multisystemic treatment (MST) was effective across
various replications, problems, therapists, and settings.
MST is a treatment model that emphasizes recognized
Nicola M. Curtis and Kevin R. Ronan, School of Psychology,
Massey University, Palmerston North, New Zcaland; Charles M.
Borduin, Department of Psychological Sciences, University of
Correspondence concerning this article should be addressed to
Nicola M. Curtis or Kevin R. Ronan, School of Psychology,
Massey University, Private Bag ll-222, Palmerston North, New
Zealand. E—mail: email@example.com or firstname.lastname@example.org
255 Curtis, N. M., Ronan, K. R., & Borduin, C. M. (2004). Multisystemic treatment: A‘
meta—analysis of outcome studies. Journal of Family Psychology,
risk factors associated with antisocial behavior. MST is
underpinned by both Bronfenbrenner’s (1979) theory of
social ecology and the empirically validated determinants of
antisocial behavior in youth (Borduin &; Schaeffer, 1998;
Lahey, Mofﬁtt; & Caspi, 2003; Loeber, Farrington, &
Waschbusch, 1998). MST interventions target the indi-
vidual, family, peer, school, and community factors iden-
tiﬁedgas contributing to and maintaining problematic
behavior. Interventions aim to empower parents to facil-
itate pragmatic changes in the youth's and the family’s
natural environments (home and other community—based
Ongoing evaluation of outcomes has been an important
feature in the development of MST. Serious juvenile of-
fenders have been the primary focus of MST outcome
studies conducted to date (Borduin et a1., 1995; Henggeler,
Melton, & Smith, 1992; Henggeler et al., 1997). Other
studies have used MST in comorbid populations (i.e.,
delinquency comorbid with substance abuse and/or se-
vere emotional disturbance) (llenggeler, Pickrel, & Bron—
dino, 1999; Henggeler, Rowland, et al., 1999). A quality
assurance system has been implemented in more recent
outcome studies of MST in an attempt to ensure that
treatment ﬁdelity is maintained in the absence of the
treatment developers and across a range of applied
Despite the encouraging research outcomes that have
been reported for MST in recent narrative reviews (Borduin,
1999; Borduin, Schaeffer, & Ronis, 2003; Brown, Borduin,
& Henggeler, 2001), no systematic quantitative review of
this body of research has been conducted. Given the increas-
ing dissemination of MST in the United States and abroad,
412 CURTIS, RONAN. AND BORDUIN
it would be useful to provide quantitative summary data of
overall treatment effectiveness‘ in the form of a meta-
analysis. The present review examines the effectiveness of
MST by quantifying and summarizing the magnitude of
effects (treatment outcomes) across all eligible MST out-
come studies. Categorical variables that may account for
variation in treatment outcomes are also examined.
Literature Review Procedures
The time frame of the literature search spans the time
period from 1986 (when controlled outcome research on
MST began) through 2003. Ninety—one articles were iden-
tiﬁed from studies listed in the Psychological Literature and
Educational Resources Information Center databases using
the keywords multisystemic therapy and multisystemic
treatment crossed with treatment outcomes, juvenile delin-
quency, antisocial behavior, and family relations. In addi-
tion, the recent tables of contents of journalsz most likely to
publish studies on MST were manually searched.
Inclusion of studies in the meta-analysis required (a)
identiﬁcation of the treatment approach as MST, including
documented adherence to the MST treatment principles
(Henggeler, Schoenwald, Borduin, Rowland, & Cunning-
ham, 1998); (b) random assignment of participants to MST
and one or more control groups; (c) a clinical sample in
which youths or their parents/caregivers manifested antiso-
cial behavior (deﬁned as social rule violations, acts against 1
others, or both) and/or psychiatric symptoms; (d) use of
both pretreatment and posttreatrnent assessment measures
and/or follow—up assessment measures; and (d) use of test
statistics suitable for meta-analysis (means, standard devi-
ations, and/or Fisher’s F ratios).
V Search Outcome
Overall, seven primary outcome studies containing a total
of 708 participants and 35 MST therapists met inclusionary
criteria and were included in this meta—analysis. These stud-
ies were all published in peer-reviewed joumals between
1987 and 2002. One other primary study was not included
in the meta-analysis because insufﬁcient test statistics were
reported (Henggeler et al., 1986). Four secondary studies
(i.e., studies reporting secondary analyses of data from
primary outcome studies) were also included (Brown,
Henggeler, Schoenwald, Brondino, & Pickrel, 1999;
Henggeler et al., 1991; Henggeler, Clingempeel, Brondino,
& Pickrel, 2002; Schoenwald, Ward, Henggeler, & Row-
land, 2000). Three other secondary studies were not in-
cluded because insufﬁcient test statistics were reported (i.e.,
Henggeler, Melton, Smith, Schoenwald, & Hanley, 1993;
Schoenwald, Ward, Henggeler, Pickrel, & Patel, 1996) or
data were reported from a small subsample of a later pri-
mary study (i.e., Scherer, Brondino, Henggeler, Melton, &
Effect sizes. To derive information about the magnitude
of the differences between treatment groups, we expressed
comparisons in terms of a standardized measure of effect
size, the d index (Cohen, 1977). The d index is deﬁned for
present purposes as the difference between the mean change
scores of two groups divided by the average or common
standard deviation of the groups. This calculation results in
a measure of the degree to which two groups differ in
standard deviation units.
Effect sizes were calculated from three primary studies
that used an alternative treatment as the control condition
and from four primary studies that included a usual services
control group. Effect sizes from secondary studies (n = 4)
were only included when the outcomes were not reported in
the related primary study. In studies in which means and
standard deviations were not reported (n = 2), effect sizes
(d indexes) were estimated by converting reported F values
(Cooper, 1998). ' ’ ‘
Correction for bias. Effect sizes based on small sam-
ples tend to inﬂate the population values that they estimate
and must be reduced (Lipsey, 1992). To account for small
sample bias (N < 30), the effect size of one study (Borduin,
Henggeler, Blaske & Stein, 1990) was adjusted using the
correction procedure recommended by Hedges (1991). The
speciﬁc weighting coefﬁcient used for effect size adjust-
ments in this studyvwas 1 — (3/4n, + 4n,, - 9), where n, is
the sample size for the treatment group and nc is the sample
size for the ‘control group (Hedges & Olkin, 1985). No other
attempts were made to adjust for sample size at the time of
measurement (as recommended by Lipsey, 1992).
Conﬁdence levels. Ninety-ﬁve percent confidence inter-
vals (Cls) are reported for the overall effect size estimate.
The standard error was estimated by applying the following
formula to each study’s effect size (Hedges & Olkin, 1985):
1 Although all of the MST outcome studies were conducted in’
community settings with real-world clinical samples, three of the
seven studies (Borduin, Henggeler, Blaske, & Stein, 1990; Bor-
duin et al., 1995', Brunk, Henggeler, & Whelan, 1987) involved
graduate students as therapists and thus should probably be clas-
siﬁed as “efﬁcacy” (rather than “effectiveness”) studies (see Bick-
man & Noser, 1999). Even so, for economy of expression, we use
the term eﬁectiveness throughout the article when referring to the
overall outcomes of MST.
2 Recent volumes of the following journals were hand searched-
Joumal of the American Academy of Child and Adolescent Psy-
chiatry, Joumal of Consulting and Clinical Psychology, Journal of
Counseling Psychology, Psychological Bulletin, Journal of Child
and Family Studies, American Journal of Psychiatry, Journal of
Emotional and Behavioral Disorders, Mental Health Services Re-
search, International Journal of Offender Therap)’. and Compar-
MULTISYSTEMIC TREATMENT 413
CI = [d - l.96(SE)] to [d + 1.96(SE)],
where SE = 1/V2w,.
l n, : nc dz
w,-= -~ and v,-= + .
vi ntnc 2(n:+nc)
Statistical power. The power of a statistical test is de-
ﬁned as the probability that it will yield a true effect that is
statistically signiﬁcant (i.e., reducing the likelihood of mak-
ing a Type 11 error; Cohen, 1988). In a meta-analysis, a
power survey estimates the proportion of studies that should
yield a statistically signiﬁcant effect (Borenstein, Rothstein,
& Cohen, 1997). Statistical power was calculated based on
sample sizes, Cohen’s (1988) power tables, and effect size
conventions (d = .20 for small effects, at = .50 for medium
effects, and d = .80 for large effects).
Homogeneity of eﬁect sizes. The Q, statistic was com-
puted to test for homogeneity among primary study out-
comes. This statistic evaluates whether all studies have the
same population effect size (i.e., whether the variation in
effect sizes is no greater than would be expected due to
errors in sampling or measurement; Hedges & Olkin, 1985).
Q, is distributed as a chi-square variable with K — 1 degrees
of freedom, where K equals the number of effect sizes. If the
Q, is not signiﬁcant, the reviewer can assume that the effect
sizes reported for the group of studies are homogeneous. If
the Q, is signiﬁcant, the effect sizes are considered to be
heterogeneous, and the reviewer should try to determine
which studies (or effect sizes) might be included in
further subsets of the studies. In other words, the studies
can be partitioned into groups of effect sizes according to
the theoretical or practical importance of the grouping
Assuming that the Q, is signiﬁcant and that studies can be
partitioned into meaningful groups, two other tests are then
used to evaluate possible differences between the groups.
First, the Q,, statistic (which has an approximate chi—square
distribution with p — 1 degrees of freedom, where p equals
the number of categories or groups) is used to test whether
the average effect sizes from the groupings are homoge-
neous (Cooper, 1998). If the Q,, is not signiﬁcant, then the
average d indexes are considered homogeneous and the
grouping factor does not explain variance in effects beyond
that associated with sampling error. If Qb exceeds the crit-
ical value (i.e., is signiﬁcant), then the grouping factor is a
signiﬁcant contributor to variance in effect sizes. However,
(2,, can only be interpreted correctly in conjunction with a
second statistic, Qw. The Q, statistic (distributed as a chi-
square with K — p degrees of freedom) is used to provide an
estimate of within-class homogeneity. As recommended by
Lipsey and Wilson (2001), a correctly speciﬁed grouping
variable (i.e., categorical moderator) that accounts for the
heterogeneity among effect sizes across studies is achieved
when the value of Q,, is signiﬁcant (i.e., mean d differs
between and/or among groups) and the value of Qw is not
signiﬁcant (i.e., no heterogeneity remains urunodeled given
the moderator and the conditional variances/weights that
quantify random subject sampling).
Characteristics of Participants
All studies were conducted in the United States, and the
primary studies were funded through local, state, and/or
federal mental health agencies (n = 6) or by a research
center of excellence (n = 1). Study sample sizes ranged
from 16 to 176, with a median of 116. The youths ranged in
age from 8.3 to 17.6 years (Mdn = 14.8), 70% were male,
and 81% lived with at least one biological parent. Fifty-four
percent (n = 380) of the youths were African American,
45% (n = 319) were Caucasian, 0.7% (n = 5) were His-
panic American, and 0.5% (n = 4) were Asian American.
Fifty—nine percent (n = 415) of the youths were classiﬁed as
chronic, at-risk, and/or violent juvenile offenders; 17% (n =
118) were classiﬁed as substance abusers; 16% (n = 116)
required emergency psychiatric hospitalization (presenting
problems included suicidal ideation, homicidal ideation, and
psychosis); 6% (n = 43) were classiﬁed as abused (includ-
ing physical abuse and psychological trauma) and/or ne-
glected (including abandonment/lack of supervision and
inadequate care); and 2% (n = 16) were classiﬁed as sexual
offenders. Eighty-four percent (n = 593) of the youths had
been arrested previously. Insufﬁcient information was pro-
vided in the primary studies to derive an overall socioeco-
nomic score according to Ho1lingshead’s (1975) criteria.
However, on the basis of information regarding parental
education, employment status, single-parent status, and me-
dian income, it appears that the samples in most studies
were drawn from disadvantaged populations.
Characteristics of Treatments and Therapists in
Primary MST Studies
MST was compared with a range of usual services in four
studies and with other treatment programs in the remaining
three studies (see Table 1). Usual services were provided
through (a) juvenile justice agencies, (b) a community men-
tal health center, (c) an outpatient substance abuse treatment
program, and (d) an inpatient psychiatric hospital. Youths
assigned to juvenile justice agencies were monitored for
school attendance and were seen weekly, fortnightly, or
monthly by probation ofﬁcers for up to 6 months; these
youths were also referred to other social service agencies
(i.e., substance abuse treatment agencies, community men-
tal health agencies) as necessary. Youths in the community
mental health comparison group received family or individ-
ual counseling, social skills training, and/or vocational
training. The outpatient substance abuse service offered
adolescent group therapy. Youths in the inpatient hospital-
ization group were provided with crisis stabilization, psy-
chiatric evaluation, and intensive individualized care.
Across studies, youths in usual services conditions received
414 CURTIS, RONAN, AND BORDUIN
Clinical Population, Comparison Condition, and Mean Effect Size for Multisystemic Treatment Outcome Studies
Study N Population Comparison condition d SD
1. Brunk et al. (1987) 43 Abusing/neglectful parents Parent training 1.32 0.65
2. Borduin et al. (1990) 16 Juvenile sexual offenders Individual therapy 1.08 0.23
3. I-lenggeler et al. (1991) Same sample as Studies 4 and 5 0.64 0.33
4. Henggeler et al. (1992) 84 Violent and chronic juvenile offenders Individual therapy 0.37 0.13
5. Borduin et al. (1995) 176 Violent and chronic juvenile offenders Individual therapy 0.66 0.43
6. Henggeler et al. (1997) 155 Violent and chronic juvenile offenders Usual services 0.27 0.25
7. Henggeler, Pickrel, et al. (1999) 118 Substance abusing juvenile offenders Usual services 0.25 0.08
8. Henggeler, Rowland, et al. (1999) 116 Psychiatrically disturbed adolescents Usual services 0.19 0.92
9. Brown et al. (1999) Same sample as Study 7 0.60 0.36
10. Schoenwald, Ward, et al. (2000) Same sample as Study 9 0.52 0.22
11. Henggeler, Clingempeel, et al. (2002) Same sample as Study 7 0.15 0.02
Note. The d for each primary outcome study is an average effect across multiple outcome measures. The d for each secondary analysis
of data is an average effect across multiple outcome measures not reported in primary outcome studies.
an average of 20 more hours of services than did youths in
The three comparison treatment programs included par—
ent training (n = 1 study) and individual therapy'(n = 2
studies). Parent training consisted of weekly group sessions
in which caregivers received instruction on human devel-
opment, behavioral management techniques, and positive
parent— child interactions. Individual therapy for the youths
included an eclectic blend of psychodynamic, client-
centered, and behavioral approaches in which therapists
focused on personal, family, and academic issues and pro-
vided encouragement for behavior change. Across studies,
youths in individual therapy conditions received an average
of 6.3 hr more treatment than did youths in MST. Treatment
completion rates ranged from 76% to 100% for MST and
from 56% to 100% for other treatment conditions. The
average treatment completion rate over primary and second-
ary studies was 86% for MST and 78% for other treatments.
MST was most often conducted in family homes and
other community settings (e.g., schools). MST was of brief
duration, averaging approximately 40 hr over 15 weeks for
up to 24 weeks. Ninety—seven percent of the MST therapists
were either current graduate students (with a bachelor’s or
master’s degree) or had earned a terminal master’s degree.
MST therapists had 1 to 15 years of clinical experience in
social work, pastoral counseling, psychology, or other re-
lated mental health ﬁelds. MST therapists received 3 to 6
days of intensive didactic and experiential training and
attended additional quarterly booster training sessions.
Among MST therapists, 43% (n = 15) were male, 37%
(n = 13) were African American, and 63% (n = 22) were
Caucasian. MST therapists received an average of 1.7 hr of
supervision each week.
A multiagent, multimethod assessment approach was
taken in six of seven studies, with a total of 23 different
. outcome measures being used (M = 6.4 per study; see Table
2). The one study that did not include multiple assessment
measures (Borduin et al., 1990) determined treatment out-
comes from re-arrest data obtained from juvenile court,
adult court, and state police records.
Ultimate goals, which are common to all treatments of
juvenile offenders, were assessed across studies and in-
cluded changes in (a) the rate, frequency, and seriousness of
adolescent criminal activity; (b) days incarcerated; (c) days
absent from school; (d) alcohol and marijuana use; and (e)
days in mandated out-of—home placements (including hos-
pitalization). Ultimate outcomes were typically assessed at
posttreatrnent (i.e., treatment completion) and follow—up
assessments, the- latter of which were conducted from 12
weeks to 4 years following treatment completion.
Therapist adherence to the MST treatment protocol was
assessed with the 26-item MST Adherence‘ Measure
(Henggeler & Borduin, 1992) in three of the studies
(I-lenggeler, Pickrel, et al., 1999; Henggeler et al., 1997',
Henggeler, Rowland, et al., 1999). Items on the measure
assess six factors that reﬂect (a) therapist adherence to the
MST treatment principles, (b) the degree to which therapy
sessions were nonproductive, (c) problem-solving efforts of
the therapist and family, (d) therapist attempts to change
family interactions, (e) lack of therapeutic direction in ses-
sions, and (f) the degree of family therapist consensus. The
measure was administered to families and therapists follow-
ing randomly selected sessions during the 4th and 8th weeks
Magnitude of Effects
Effect sizes ranged in magnitude from -0.02 to 5.79. As
recommended by Cooper (1998), both signiﬁcant and non-
signiﬁcant d index values were included in the analyses in
an attempt to minimize bias and maximize conﬁdence in
any conclusions that were drawn. There were a total of 101
d index values, and 6 of the 7 studies had multiple indexes.
Mean Eﬁect Size for Domain and Source of Outcome Measure
Effect size (d)
Outcome Domain (variable) Relevant studies
Individual .28 2, 4, 6, 7, 8, 9, 10
Youth symptoms .43 J, 2, 4, 9, 10
Parent symptoms .33 l 2, 9, 10
Youth behavior problems .341 2, 7, 9
Hospitalization .52 J, 11
Family .57 2, 4, 7, 9, 10
Self-reported family relations
Adaptabilitylcohesion .31 T 2, 7, 9, 10
Parental monitoring .60 T 2
Stress 1.01 i 4
Observed family interactions .76 2, 4
Conﬁict-hostility .621, 2, 4
Overall family supportiveness .841 2, 4
Parental effectiveness .941‘ 2, 4
Verbal activity .221 2, 4
Youth noncompliance .92 J, 2, 4
Peer relations .11 2, 7, 9, 10
Aggression .021 2, 7, 9
Bonding .08 T 2, 7, 9
Social maturity .071 2, 9
Social competence .281 7, 10
Association with deviant peers 31 l 9, 10
School attendance .54 3, 10
Ultimate outcomes (criminal activity) .50 1, 2, 6, 7, 8, 9, 10
Number of arrests for all crimes .551 1, 2, 7, 8, 9, 10
Number of arrests for substance abuse crimes .291 6, 7, 10
Seriousness of arrests 1.01 l 2
Days incarcerated .55 1, 7, 9
Self-reported delinquency .07 l 8, 9
Self-reported drug use .64 L 5
Note. Upward and downward arrows indicate a res
pective increase or decrease in the associated
domain. 1 = Borduin et al. (1-990); 2 = Borduin et al. (1995); 3 = Brown et al. (1999); 4 = Brunk
et al. (1987); 5 = Henggeler et al. (1991); 6 = Henggeler, Clingem
peel, et al. (2002); 7 = I-Ienggeler
et al. (1992); 8 = Henggeler, Pickrel et al. (1999); 9 = Henggeler et al. (1997); 10 = Henggeler,
Rowland et al. (1999); 11 = Schoenwald, Ward, et al. (2000).
These d index values were averaged to yield one d index per
The average effect of MST across the seven primary and
four secondary outcome studies was d = .55 (Mdn = .52).
The signiﬁcance of this effect size was tested by computing
a 95% conﬁdence interval, which ranged from d = .40 to
d = .70. As the lower limit of the 95% conﬁdence interval
is well above zero, the mean d index value is assumed to be
signiﬁcantly different» from zero (Shadish & Haddock,
1994). Of note, for 6 of the 101 d indexes (6%), improve-
ments in the control group exceeded those in the MST
U3 is a measure of distribution overlap that provides
another method of interpreting the d index (Cohen, 1988).
U3 tells the percentage of people in the lower mean group
who are surpassed by the average person in the higher mean
group. In the present case, a d of .55 equates to a U3 of .70,
meaning that the average participant in the MST condition
surpassed 70% of the control condition participants on the
measures of instrumental and ultimate outcomes.
The d of .55 calculated in this study can be categorized as
a moderate effect (see Cohen, 1988). On the basis of the
overall comparison between MST (n = 361) and control (n
= 347) groups, the power to detect a moderate effect size
was .57 (Mdn = .73). Thus, on average, investigators had a
57% chance of detecting an effect size in the moderate
range. In terms of detecting a moderate effect, power failed
to reach the 80% criterion for design sensitivity (Cohen,
1988). Accordingly, as there is an increased likelihood of
making a Type II error (i.e., rejecting the null hypothesis
416 CURTIS, RONAN, AND BORDUIN
when it is true), results of this analysis should be interpreted
with some caution.
Relationship Between Effect Size and Treatment
The effect sizes achieved in the different instrumental
domains (individual, family, peer) that are targeted in the
MST model were examined. A comparison of average effect
sizes on measures of individual adjustment (d = .28) ‘versus
measures of peer relations (d = .11) did not reveal a
signiﬁcant difference (Z = -0.92, ns). However, measures
of family relations demonstrated a larger average effect size
(d = .57) than did measures of individual adjustment (d =
.28) or measures of peer relations (d = .11; Zs > -2.13,
ps < .03). Table 2 summarizes the effect sizes for various
domains and subdomains.
Categorical Moderator Analyses
The homogeneity analysis for the effect sizes in the
present review revealed there was more variability in the
combined d indexes than would be expected due to sam-
pling error or other sources of expected error, Q,(af =1) =
11.73, p < .05. Potential moderator variables were then
tested to identify sources of heterogeneity among studies.
Studies were initially grouped into mutually exclusive
categories on the basis of the target population (i.e., violent
and chronic juvenile offenders vs. all other youth popula-
tions, including psychiatrically disturbed youths, substance-
abusing juvenile offenders, juvenile sexual offenders, and
abused or neglected youths). Three of the seven studies (i.e.,
Borduin et al., 1995; Henggeler et al., 1992, 1997) used
MST with populations of violent and chronic juvenile of-
fenders. The average effect size achieved in these studies
(d = .44, CI = i .19) was compared with the average effect
size achieved by the remaining studies.(d = .38, CI = i
.27). The analyses of differences in effect sizes across
‘studies, Q,,(df = 1) = .11, ns, and within studies, Q,,,(df =
5) = 11.62, p < .05, indicated that the type of target
population did not moderate treatment effects.
Given that target population did not account for the
heterogeneity between studies, we then examined whether
differences in study conditions (i.e., efﬁcacy vs. effective-
ness conditions) might moderate treatment effects. Al-
though all of the MST outcome studies were conducted in
community settings, three of the seven studies (i.e., Borduin
et a1., 1990, 1995; Brunk, Henggeler, & Whelan, 1987) used
closely supervised graduate students as therapists. Such
conditions are more characteristic of “efﬁcacy” studies than
“effectiveness” studies (see Bickrnan & Noser, 1999). That
is, more control was exercised over the treatment conditions
than in the remaining four studies (Henggeler et al., 1992;
Henggeler, Pickrel, & Brondino, 1999; Henggeler et al.,
1997; Henggeler, Rowland et al., 1999). The average effect
size achieved in more controlled studies with graduate stu-
dent therapists (d = .81, CI = *_*' .33) was compared with
the average effect size achieved in community-based studies
(d = .26, CI = i .06). The computed values of the tests of
differences in effect sizes across studies, Q,,(df == 1) = 8.74,
p < .05, and within studies, Q,,(df = 5) = 2.99, ns,
indicated that the effect sizes were not homogeneous across
groups. Thus, the study conditions variable may account for
the heterogeneity among studies.3
The primary objective of this review was to establish an
estimate of the overall effectiveness of MST in treating
antisocial and associated behaviors in a range of youth and
family populations. Across both instrumental and ultimate
outcome measures, youths and their families treated with
MST were functioning better and offending less than 70%
of their counterparts who received alternative treatment or
services. Speciﬁcally, MST was found to be relatively ef-
fective in reducing emotional and behavioral problems in
individual family members, in improving parent—youth and
overall family relations, in decreasing youth aggression
toward peers and involvement with deviant peers, and in
reducing youth criminality. Follow—up data suggest that
treatment effects were sustained for up to 4 years.
This review found that MST demonstrated larger effects
on measures of family relations than on measures of indi-
vidual adjustment or peer relations. This ﬁnding is consis-
tent with the emphasis that MST places on family interven-
tions (Henggeler & Borduin, 1990) and with previous
studies of change processes in MST showing that improve-
ments in family relations predicted decreases in individual
problems (i.e., symptoms, delinquent behavior) and in de-
linquent peer afﬁliation (Huey, Henggeler, Brondino, &
Pickrel, 2000; Mann, Borduin, Henggeler, & Blaske, 1990).
The larger-observed effect of MST on family relations
measures than on other measures was also likely due to the
fact that the comparison conditions (i.e., usual services or an
alternate treatment) typically focused on the individual ad-
olescent and not on the family. Indeed, two of the studies
included in this review (Borduin et al., 1995; Henggeler et
al., 1992) showed that, in contrast to MST, the individually
focused comparison conditions led to a deterioration in
family relations over the course of treatment. This deterio-
ration in family relations has also been observed in other
studies of individually focused child and adolescent treat-
ments (e.g., Szapocznik et al., 1989) and is consistent with
3 Another key difference between the “efficacy” and “effective-
ness” studies pertained to the nature of the control groups that were
used. More speciﬁcally, all of the efficacy studies included an
altemative treatment control group (either individual therapy or
parent training), whereas all of the effectiveness studies included a
usual services control group. Inasmuch as study condition (efficacy
vs. effectiveness) was confounded with the type of control group
(altemative treatment vs. usual services) in these studies, it is not
possible to ascertain that study condition per se accounted for the
heterogeneity in effect sizes. It is possible that moderator effects
can be attributed to both the type of study condition and the type
of comparison group.
MULTISYSTEMIC TREATMENT 417
the systemic perspective that child misbehavior often serves
a functional purpose (e.g., by uniting parents who are oth-
erwise in conﬂict) in the family (Hoffman, 1981; Minuchin,
1985). From this perspective, treatments that focus primar-
ily on improving the individual child’s behavior may desta-
bilize the family system by removing the child from his or
her central position in family (or marital) conﬂicts.
The results of this review also indicate that treatment
effect sizes in MST outcome studies were not moderated by
the type of target population (broadly deﬁned as violent and
chronic juvenile offenders vs. other populations of youths)
that participated in MST. MST was originally developed
and validated with populations of serious and violent juve-
nile offenders and only more recently has been extended to
populations of youths with substance abuse problems or
serious emotional disturbances. Although the results of this
review suggest that MST appears to be a promising ap-
proach for populations other than violent and chronic juve-
nile offenders, additional studies evaluating the effective-
ness of MST with these other populations will be needed
before more deﬁnitive conclusions can be drawn and before
dissemination efforts would be justiﬁed.
Another issue highlighted by this review pertains to the
dissemination of efﬁcacious treatments to community set-
tings. The results indicate that treatment effects in MST
outcome studies might have been moderated by differences
in study conditions (i.e., efﬁcacy vs. effectiveness condi-
tions). It is entirely possible that the involvement of the
MST developers as clinical supervisors in the efﬁcacy stud-
ies contributed to the higher effect sizes that were observed
in those studies (d = .81) versus the effectiveness studies
(d = .26). In fact, Schoenwald, Henggeler, Brondino, and
Rowland (2000) have noted that ongoing quality assurance
procedures (e.g., therapist and supervisor adherence proto-
cols) are indispensable when disseminating MST to com-
munitysettings. It is also important to note that most of the
recent MST clinical trials have emphasized ongoing evalu-
ation of various components of the MST quality assurance
process. In particular, several studies have examined deter-
minants (e.g., supervision by MST-trained supervisors, or-
ganizational support for the MST model) of therapist ﬁdel-
ity to the MST model (llenggelerlet al., 1997; Henggeler,
Schoenwald, Liao, Letourneau, & Edwards, 2002; Huey et
al., 2000; Schoenwald & Hoagwood, 2001). It remains to be
seen whether MST effectiveness studies will be able to
demonstrate results that are comparable to those that have
been obtained in MST efﬁcacy studies.
In addition to improving ﬁdelity to the MST model in
community settings, researchers of MST effectiveness
should broaden their assessment of instrumental outcomes
in each of the systems pertinent to the goals of MST. For
example, although a common goal of MST is to increase
youth involvement with prosocial peers, the assessment of
change in peer afﬁliations has been limited to measures of
association with deviant peers. Measures that directly assess
involvement with prosocial peers would be informative in
future studies. Similarly, assessment of relevant cognitive
processes such as attributional style or bias would be a
valuable addition to the assessment of individual behavior
problems and psychiatric symptoms. Broader assessment of
other areas that are frequent targets of MST interventions
might include measures of performance in school (e.g.,
grades, achievement levels) and participation in extracur-
ricular activities (e.g., sports teams, church groups, recre-
ation center activities).
Research is also needed to evaluate the processes that
MST incorporates to facilitate changes in youths and their -
families. Although investigators have begun to identify im-
portant moderators (e.g., treatment ﬁdelity; Henggeler et al.,
1997) and mediators (e.g., improved peer relations; Huey et
al., 2000; family engagement; Schaeffer & Borduin, 2003)
of MST outcomes, greater understanding of speciﬁc mech-
anisms of change is required to enable community—based
agencies to apply targeted and cost-effective treatment ap-
proaches such as MST.
Findings of the present review must be interpreted in the
context of the following methodological limitations. First,
given the relatively small number of outcome studies that
were available for inclusion in the review, the conclusions
should be considered tentative. Nevertheless, a total of 708
participants helped to offset the limited number of separate
investigations. Second, some of the subcategories of effects
were based on a small number (i.e., subgroup) of studies,
thus limiting the generalizability of the ﬁndings. Third, the
confounding of study condition (efﬁcacy vs. effectiveness)
with type of control group (altemative treatment vs. usual
services) in the moderator analysis clouds the interpretation
of the results. This interpretive ambiguity cannot be re-
solved in the present study. Finally, it is prudent to acknowl-
edge the potential for bias in the current review due to the
pooling of sample studies (Hedges & Olkin, 1985). How-
ever, it is also the case that a more conservative approach to
pooling effect sizes strengthens conﬁdence in the results.
In conclusion, as an empirically established treatment for
violent and chronic juvenile offenders, MST appears to be
worthy of wider implementation and continued evaluation.
The overarching objective of MST (i.e., empowering par-
ents to facilitate pragmatic changes in the youth’s and the
family’s natural environments) appears to be robust with
this population. More empirical support is required before
MST can be considered an effective treatment of substance
abuse in adolescents or an effective community—based al-
ternative to the hospitalization of youths presenting psychi-
References marked with one or two asterisks indicate primary
studies or secondary analysis studies, respectively, in the meta-
Bickman, L., &. Noser, K. (1999). Meeting the challenges in the
delivery of child and adolescent mental health services in the
next millennium: The continuous quality improvement approach.
Applied and Preventive Psychology, 8, 247-255.
Borduin, C. M. (1999). Multisystemic treatment of criminality and
violence in adolescents. Journal of the American Academy of
Child and Adolescent Psychiatry, 38, 242-249.
418 CURTIS, RONAN. AND BORDUIN
*Borduin, C. M., Henggeler, S. W., Blaske, D. M., & Stein, R.
(1990). Multisystemic treatment of adolescent sexual offenders.
International Journal of Oﬁender Therapy and Comparative
Criminology, 35, 105-114.
*Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W.,
Fucci, B. R., Blaske, D. M., & Williams, R. A. (1995). Multi-
systemic treatment of serious juvenile offenders: Long-term pre-
vention of criminality and violence. Joumal of Consulting and
Clinical Psychology, 63, 569-578.
Borduin, C..M., & Schaeffer, C. M. (1998). Violent offending in
adolescence: Epidemiology, correlates, outcomes, and treatment.
In T. P. Gullotta, G. R. Adams, & R. Montemayer (Eds.),
Delinquent violent youth: Theory and interventions (pp. 144-
174). Newbury Park, CA: Sage.
Borduin, C. M., Schaeffer, C. M., & Ronis, S. T. (2003). Multi-
systemic treatment of serious antisocial behavior in adolescents.
In C. A. Essau (Ed), Conduct and oppositional deﬁant disor-
ders: Epidemiology, risk factors, and treatment (pp. 299-318).
Mahwah, NJ : Erlbaum.
Borenstein, M., Rothstein, H., & Cohen, J. (1997). Power preci-
sion. Hillsdale, NJ: Erlbaum.
Bronfenbrenner, U. (1979). The ecology of human development:
Experiments by nature and design. Cambridge, MA:' Harvard
Brown, T. L., Borduin, C. M., & l-lenggeler, S. W. (2001). Treat-
ing juvenile offenders in community settings. In J. B. Ashford,
B. D. Sales, & W. H. Reid (Eds.), Treating adult and juvenile
offenders with special needs (pp. 445-464). Washington, DC:
American Psychological Association.
**Brown, T. L., Henggeler, S. W., Schoenwald, S. K., Brondino,
M. J., & Piclcrel, S. G. (1999). Multisystemic treatment of sub-
stance abusing and dependent juvenile delinquents: Effects on
school attendance at posttreatment and 6-month followup. Chil-
dren's Services: Social Policy, Research and Practice, 2, 81-93.
*Brunk, M., I-Ienggeler, S. W., & Whelan, J. P. (1987). Compar-
ison of multisysternic therapy and parent training in the brief
treatment of child abuse and neglect. Journal of Consulting and
Clinical Psychology, 55 171-178.
Burns, B. J., Hoagwood, K., & Mrazek, P. J. (1999). Effective
treatment for mental disorders in children and adolescents. Clin-
ical Child and Family Psychology Review, 2, 199-254.
Cohen, J. (1977). Statistical power analysis for the behavioral
sciences. New York: Academic Press. .
Cohen, J. (1988). Statistical power analysis for the behavioral
sciences (2nd .ed.). Hillsdale, NJ: Erlbaum.
Cooper, H. (1998). Synthesizing research: A guide for literature
reviews (3rd ed.). London: Sage. ’
Elliott, D. S. (Series Ed.). (1998). Blueprints for violence preven-
tion. Boulder, CO: University of Colorado, Center for the Study
and Prevention of Violence, Blueprint Publications.
Farrington, D. P., & Welsh, B. C. (1999). Delinquency prevention
using family-based interventions. Children and Society, 13, 287-
Fergusson, D., Horwood, J., & Lynskey, M. (1997). Children and
adolescents. In P. M. Ellis & S. C. D. Collings (Eds.), Mental
health in New Zealand from a public health perspective. Wel-
lington, New Zealand: Ministry of Health.
Hedges, L. V. (1991). Statistical considerations. In H. Cooper &
L. V. Hedges (Eds.), The handbook of research synthesis (pp.
29-40). New York: Russell Sage.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-
analysis. San Diego, CA: Academic Press.
Henggeler, S. W., & Borduin, C. M. (1990). Family therapy and
beyond: A multisystemic approach to treating the behavior prob-
lems of children and adolescents. Paciﬁc Grove, CA: Brooksl
Henggeler, S. W., & Borduin, C. M. (1992). Multisystemic therapy
adherence scales. Unpublished instrument, Department of Psy-
chiatry and Behavioral Sciences, Medical University of South
**I-ienggeler, S. W., Borduin, C. M., Melton, G. B., Mann, B. J.,
Smith, L. A., Hall, J. A., Cone, L., & Fucci, B. R. (1991). Effects
of multisystemic therapy on drug use and abuse in serious
juvenile offenders: A progress report from two outcome studies.
Family Dynamics of Addiction Quarterly, 1, 40-51.
**Henggeler, S. W., Clingempeel, W. G., Brondino, M. J., &
Pickrcl, S. G. (2002). Four-year follow-up of multisystemic
therapy with substance-abusing and substance-dependent juve-
nile offenders. Joumal of the American Academy of Child and
Adolescent Psychiatry, 41, 868-874.
*Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family
preservation using multisystemic therapy: An effective alterna-
tive to incarcerating serious juvenile offenders. Journal of Con-
sulting and Clinical Psychology, 60, 953-961.
Henggeler, S. W., Melton, G. B., Smith, L. A., Schoenwald, S. K.,
& Hanley, J. (1993). Family preservation using multisystemic
therapy: Long-term follow-up to a clinical trial with serious
juvenile offenders. Journal of Child and Family Studies, 2,
*Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999).
Multisystemic treatment of substance-abusing and dependent
delinquents: Outcomes, treatment ﬁdelity, and transportability.
Mental Health Services Research, 1, 171-184.
*Hengge1er, S. W., Rowland, M. D., Pickrel, S. G., Miller, S. L.,
Cunningham, P. B., Santos, A. B., Schoenwald, S. K., Randall,
J., & Edwards, J. E. (1997). Investigating family-based alterna-
tives to institution-based mental health services for youth: Les-
sons learned from the pilot study of a randomized field trial.
Journal of Clinical Child Psychology, 26, 226-233.
Hengge1er,_S. W., Rodick, J. D., Borduin, C. M., Hanson, C. L.
Watson, S‘. M., & Urey, J . R. (1986). Multisystemic treatment of
juvenile offenders: Effects on adolescent behavior and family
interaction. Developmental Psychology, 22, 132-141.
*Henggeler,' S. W., Rowland, M. D., Randall, J., Ward, D. M.,
Piclcrel, S. G., Cunningham, P. B., Miller, S. L., Edwards, J.,
Zealberg, J., Hand, L. D., & Santos, A. B. (1999). Home-based
rnultisystemic therapy as an alternative to the hospitalization of
youth in psychiatric crisis: Clinical outcomes. Joumal of the
American Academy of Child and Adolescent Psychiatry, 38,
Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland,
M. D., & Cunningham, P. B. (1998). Multisystemic treatment of
antisocial behavior in children and adolescents. New York:
Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letoumeau,
E. J ., & Edwards, D. L. (2002). Transporting efficacious treat-
ments to ﬁeld settings: The link between supervisory practices
and therapist ﬁdelity in MST programs. Joumal of Child and
Adolescent Psychology, 31, 155-167.
Hoffman, L. (1981). Foundations of family therapy. New York:
Hollingshead, A. B. (1975). The four-factor index of social status.
Unpublished manuscript, Yale University, New Haven, CT.
Huey, S. J., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G.
(2000). Mechanisms of change in multisysternic therapy: Reduc-
ing delinquent behavior through therapist adherence and im-
MULTISYSTEMIC TREATMENT 419
proved family and peer functioning. Journal of Consulting and
Clinical Psychology, 68, 451-467.
Kazdin, A. E. (1987). Treatment of antisocial behavior in children:
Current status and future directions. Psychological Bulletin, 102,
Kazdin, A. E. (2000). Treatments for aggressive and antisocial
children. Child and Adolescent Psychiatric Clinics of North
America, 9, 841-858.
Kazdin, A. 1-3., & Weisz, J. R. (1998). Identifying and developing
empirically supported child and adolescent treatments. Jountal of
Consulting and Clinical Psychology, 66, 19-36.
Lahey, B. B., Mofﬁtt, T. E., & Caspi, A. (Eds.). (2003). Causes of
conduct disorder and juvenile delinquency. New York: Guilford
Lipsey, M. W. (1992). Juvenile delinquency treatment: A meta-
analytic enquiry into the variability of effects. In T. D. Cook, H.
Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light,
T. A. Louis, & F. Mostellor (Eds), Meta-analysis for explana-
tion: A casebook (pp. 83-127). New York: Russell Sage.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta—analysis.
Thousand Oaks, CA: Sage.
Loeber, R., Farrington, D. P., & Waschbusch, D. A. (1998).
Serious and violent juvenile offenders. In R. Loeber & D. P.
Farrington (Eds), Serious and violent juvenile offenders: Risk
factors and successﬁil interventions (pp. 13-29). Thousand
Oaks, CA: Sage.
Mann, B. J., Borduin, C. M., Henggeler, S. W., & Blaske, D. M.
(1990). An investigation of systemic conceptualizations of
parent-child coalitions and symptom change. Journal of Consult-
ing and Clinical Psychology, 58, 336-344.
Minuchin, P. P. (1985). Families and individual development:
Provocations from the field of family therapy. Child Develop-
ment, 56, 289-302.
Ofﬁce of Juvenile Justice and Delinquency Prevention, U.S. De-
partment of Justice. (1999). Delinquency cases in juvenile courts,
1999. Washington, DC: Author.
Rutter, M., Giller, H., & Hagell, A. (1998). Antisocial behavior by
young people. Cambridge, MA: Cambridge University Press.
Schaeffer, C. M., & Borduin, C. M. (2003). Moderators and
mediators of therapeutic change in multisystemic treatment of
serious juvenile offenders. Unpublished manuscript.
Scherer, D. G., Brondino, M. J., Henggeler, S. W., Melton, G. B.,
& I-lanley, J. H. (1994). Multisystemic family preservation ther-
apy: Preliminary ﬁndings from a study of rural and minority
serious adolescent offenders. Joumal of Emotional and Behav-
ioural Disorders, 2, 198-206.
Schoenwald, S. K., Henggeler, S. W., Brondino, M. l., & Row-
land, M. D. (2000). Multisystemic therapy: Monitoring treatment
ﬁdelity. Family Process, 39, 83-103.
Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, trans-
portability, and dissemination of interventions: What matters
when? Psychiatric Services, 52, 1190-1197.
Schoenwald, S. K., Ward, D. M., Henggeler, S. W, Pickrel, S. G.,
& Patel, H. (1996). Multisystemic therapy treatment of substance
abusing and dependent adolescent offenders: Costs of reducing
incarceration, inpatient, and residential placement. Journal of
Child and Family Studies, 5, 431-444.
**Schoenwa1d, S. K., Ward, D. M., Henggeler, S. W., & Rowland,
M. (2000). MST versus hospitalization for crisis stabilization of
youth: Placement outcomes 4 months post-referral. Mental
Health Services Research, 2, 3-12.
Shadish, W. R., & Haddock, C. K. (1994). Combining estimates of
effect size. In H. Cooper & L. V. Hedges (Eds.). The handbook
of research synthesis (pp. 261-231). New York: Russell Sage.
Szapocznik, 1., Rio, A., Murray, E., Cohen, R., Scopetta, M.,
Rivas-Vasquez, A., Hervis, 0., Posada, V., & Kurtines, W.
(1989). Structural family versus psychodynamic child therapy for
problematic Hispanic boys. Journal of Consulting and Clinical
Psychology, 57, 571-578.
U.S. Public Health Service. (2001). Youth violence: A report of the
Surgeon General. Washington, DC: Author.
Received March 4, 2003
Revision received January 2, 2004
Accepted February 14, 2004 I