Content uploaded by Terje Ogden
Author content
All content in this area was uploaded by Terje Ogden on Dec 27, 2017
Content may be subject to copyright.
(This is a sample cover image for this issue. The actual cover is not yet available at this time.)
This article appeared in a journal published by Elsevier. The attached
copy is furnished to the author for internal non-commercial research
and education use, including for instruction at the authors institution
and sharing with colleagues.
Other uses, including reproduction and distribution, or selling or
licensing copies, or posting to personal, institutional or third party
websites are prohibited.
In most cases authors are permitted to post their version of the
article (e.g. in Word or Tex form) to their personal website or
institutional repository. Authors requiring further information
regarding Elsevier’s archiving and manuscript policies are
encouraged to visit:
http://www.elsevier.com/copyright
Author's personal copy
A randomized trial of group parent training: Reducing child conduct problems
in real-world settings
John Kjøbli
*
, Silje Hukkelberg, Terje Ogden
The Norwegian Center for Child Behavioral Development, University of Oslo, P.O. Box 7053, Majorstuen, 0306 Oslo, Norway
article info
Article history:
Received 1 May 2012
Received in revised form
20 November 2012
Accepted 20 November 2012
Keywords:
Conduct problems
Aggressive behavior
Children
Parent management training
Effectiveness study
Randomized controlled trial
abstract
Objective: Group-based Parent Management Training, the Oregon model (PMTO, 12 sessions) was
delivered by the regular staff of municipal child and family services. PMTO is based on social interaction
learning theory and promotes positive parenting skills in parents of children with conduct problems. This
study examined the effectiveness of the group-based training intervention in real world settings both
immediately following and six months after termination of the intervention.
Methods: One hundred thirty-seven children (3e12 years) and their parents participated in this study.
The families were randomly assigned to group-based training or a comparison group. Data were collected
from parents and teachers.
Results: The caregiver assessments of parenting practices and child conduct problems and caregiver and
teacher reported social competence revealed immediate and significant intervention effects. Short- and
long-term beneficial effects were reported from parents, although no follow-up effects were evident on
teacher reports.
Conclusions: These effectiveness findings and the potential for increasing the number of families served
to support the further dissemination and implementation of group-based parent training.
Ó2012 Elsevier Ltd. All rights reserved.
Children with conduct problems are at risk for a broad range of
current and future mental health problems, as well as being
marginalized from work and other social arenas in adolescence and
adulthood (Moffitt, 2006). In spite of extensive evidence supporting
the use of parent training interventions (Lundahl, Risser, & Lovejoy,
2006;Reyno & McGrath, 2006), these and other evidence-based
interventions have not been widely disseminated and imple-
mented (Kerner, Rimer, & Emmons, 2005;Weisz & Gray, 2008). One
way to meet this challenge is to offer group-based parent training
to families, which deliver the same number of treatment hours but
require fewer sessions than individually delivered parent training
(Ogden & Hagen, 2008). This modification may increase the reach of
the intervention. Therefore, in an effort to increase the availability
of training and thereby benefit public health, a group-based version
of Parent Management Training, the Oregon model (PMTO) was
developed in Norway. To have beneficial public impact, the inter-
vention must produce positive effects in both the short and long
term (Glasgow, Vogt, & Boles, 1999). Thus, the present study tested
the effectiveness of group-based PMTO in a randomized controlled
trial immediately following and six months after completion of the
intervention.
PMTO is directly derived from the social interaction learning
(SIL) model (Patterson, 1982). According to the model, aggressive
and aversive behaviors occur if they are functional in changing or
controlling the behavior of other family members (e.g., parents and
siblings). Research has shown that this training process may begin
when children are toddlers (Shaw, Owens, Giovannelli, & Winslow,
2001), and parentechild conflict has been found to be a particularly
strong predictor for early onset conduct problems (Ingoldsby et al.,
2006). In support of the SIL model, negative parenting practices
have been found to predict child conduct problems (e.g., DeGarmo
& Forgatch, 2005). These findings provide a solid theoretical and
empirical rationale for targeting parenting practices when aiming
to reduce and prevent conduct problems.
The aim of PMTO is to increase the positive interactions parents
have with their children by using positive teaching strategies
(e.g., encouragement of child pro-social behaviors). Another aim is to
help parents use mild forms of negative consequences (i.e., removal
of privilege or timeout) for deviant behaviors and rule breaking.
Parents are trained in five basic skills or core components of PMTO
throughout the intervention: encouragement, positive involvement,
monitoring, discipline and problem solving (Ogden & Hagen, 2008).
*Corresponding author. Tel.: þ47 23205840.
E-mail address: john.kjobli@atferdssenteret.no (J. Kjøbli).
Contents lists available at SciVerse ScienceDirect
Behaviour Research and Therapy
journal homepage: www.elsevier.com/locate/brat
0005-7967/$ esee front matter Ó2012 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.brat.2012.11.006
Behaviour Research and Therapy 51 (2013) 113e121
Author's personal copy
Two reviews of interventions for children and youth withconduct
problems recognized PMTO as “well established”(Brestan & Eyberg,
1998;Eyberg, Nelson, & Boggs, 2008). Moreover, randomized
controlled trials published after the release of these reviews have
documented positive short- and long-term outcomes of PMTO, both
in Norway (Hagen, Ogden, & Bjornebekk, 2011;Ogden & Hagen,
2008) and in the US (Forgatch, Patterson, DeGarmo, & Beldavs,
2009). Interventions building on the same principles as PMTO,
such as The Incredible Years Parent Training program (IY), Triple P
and ParenteChild Interaction Therapy (PCIT) have also provided
evidence for the positive effects of parent training. A recent meta-
analysis of Triple P revealed that parent training had positive
outcomes on child conduct problems and parenting practices
(Nowak & Heinrichs, 2008). The studies included in this meta-
analysis were conducted in Australia, Asia, the US and Europe. The
Incredible Years Parent Training program has been rigorously tested
with positive outcomes in the US (e.g., Brotman et al., 2008;Webster-
Stratton, Reid, & Hammond, 2004) and in Europe (e.g., Bywater et al.,
2009;Larsson et al., 2009;Scott, 2005). Studies conducted in the US
and in Australia have indicated that PCIT is an effective intervention
for child conduct problems (e.g., Nixon, Sweeney, Erickson, & Touyz,
2003;Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998).
Although there is evidence to suggest that parent training
effectively reduces child conduct problems, relatively few
randomized controlled trials have investigated the long-term
effectiveness of such interventions. In a meta-analysis by Mccart,
Priester, Davies, and Azen (2006) findings showed that the post-
treatment effect size for parent training was .47, while no follow-
up effect size was computed because only four studies included
in the analysis collected such data. A meta-analysis byLundahl et al.
(2006) also revealed that few follow-up studies exist. The authors
did, however, include 21 studies using follow-up data, and analyses
revealed a moderate effect size (ES ¼.42) immediately after
treatment and a small effect size (ES ¼.21) at follow-up. Based on
these findings, the investigators suggested that, although the
immediate outcomes provided support for parent training, the
attenuation of follow-up effects indicated that the sustainability of
outcomes may be best secured by continuing-care models which
help parents maintain their skills and adjust them to the children’s
developmental changes.
The positive results of parent training have primarily been found
at home with parents (Lundahl et al., 2006). Whether parent
training produces positive outcomes in day-care or school settings
is less clear, because research findings have been mixed and con-
flicting. While some studies have not found child conduct problems
to improve in day-care of school after parent training (Drugli &
Larsson, 2006;Kjøbli & Ogden, 2012;Taylor & Biglan, 1998),
others have reported positive generalization effects (Forgatch et al.,
2009;Ogden & Hagen, 2008;Webster-Stratton et al., 2004).
Although the SIL model postulates that parenting practices
constitute the most proximal foundation for persistent conduct
problems, it also suggests that preschool and elementary school
settings can amplify processes already initiated in the family,
leading to new and often more serious forms of antisocial behavior
(Patterson, 1982). Behavior problematic children have been found
to often interact coercively with teachers, to receive less support
and teaching, and more criticism in the school context (Snyder,
2002). Therefore, even though the cause of conduct problems
may originate from coercion in the family, it may not always be
enough to address parenting practices when the problems have
spread to the school or preschool setting. In other words, the effects
of parent training interventions do not necessarily generalize from
one setting to the other (Taylor & Biglan, 1998).
Findings have been mixed with regard to whether the format
(i.e., group-based vs. individually delivered) of parent management
training intervention influences the outcomes. In an early study of
IY, Webster-Stratton (1984) reported similar outcomes in child
behaviors when comparing individually based and group based
parent training. In a comparable study, Chadwick, Momcilovic,
Rossiter, and Stumbles (2001) directly compared group and indi-
vidually based parenting interventions in a randomized trial. At the
termination of treatment, the parents in individual treatment re-
ported less frequent behavior problems and fewer management
problems than the parents in the group intervention or those in
a control group receiving no intervention. However, there were no
significant differences between the three groups in the absolute
severity of behavior problems at the follow-up.
On the other hand, Cunningham, Bremner, and Boyle (1995)
found in a randomized trial that parents in group treatment
reported greater improvements in behavior problems at home and
better maintenance of these gains at a 6-month follow-up,
compared to individual parent training. In contrast, Lundahl et al.
(2006) concluded their meta-analysis of parent management
training interventions by reporting that individual interventions
resulted in similar treatment effects relative to group treatments,
although findings were in favor of individual parent training in one
of three outcomes. Interestingly, when delivered to disadvantaged
families, findings suggested that individual parent training had
more beneficial outcomes relative to group based parent training.
According to Chadwick et al. (2001), some of the advantages of
individual interventions are the increased possibilities of resched-
uling sessions and conducting a more detailed and accurate func-
tional analysis of the child’s behavior. Group treatment allows
fewer possibilities for providing each family individual attention.
On the other hand, Cunningham et al. (1995) found that solving
problems in large groups yielded a wider range of child manage-
ment options. Large group discussions usually provided a wider
perspective on common child management difficulties. Second,
proposing solutions, formulating personal goals, and describing
homework successes in a group may enhance parent commitment
and adherence.
Even though empirical studies have documented the efficacy of
group based parent training (e.g., Webster-Stratton et al., 2004), the
above mixed findings suggest that it is not necessarily a given that
group-based parent training will produce positive outcomes. We
have found it important to evaluate the effectiveness of group
based PMTO in Norwegian regular services before any direct
comparison between individual and group-based parent training is
made.
Aims
The present study evaluated the effectiveness of group-based
PMTO immediately following and six months after the interven-
tion. The study was conducted in a sample (N¼137) of parents
seeking help for emerging or already developed child conduct
problems in real world settings. The main aim of the present study
was to examine whether the intervention would have positive
effects on child conduct problems, social competence, and parenting
practices, compared to regular services, immediately following and
six months after the intervention. The second aim was to examine
whether the same effects would emerge on parental mental health
and child internalizing problems. Parental mental distress was
included as an outcome variable because it has been found to be
associated with the outcomes of parent training interventions
(e.g., Lundahl et al., 2006) and parent training trials have found
positive effects on parental mental health (Patterson, DeGarmo, &
Forgatch, 2004;Shaw, Connell, Dishion, Wilson, & Gardner, 2009).
Assessments of child internalizing problems was included, as
such problems share many risk factors (e.g., ineffective parenting
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121114
Author's personal copy
practices) with conduct problems and findings from randomized
trials have shown that parent training may reduce internalizing
problems (Larsson et al., 2009;Webster-Stratton & Herman, 2008).
Methods
The study was designed as a pretest, posttest and follow up
parallel group randomized trial with an approximately 50:50
allocation ratio between the PMTO and comparison groups.
Participants
Calculations of statistical power indicated that approximately
140 families would be needed to identify differences between the
PMTO and comparison groups. One hundred fifty-three families
were assessed for eligibility (see Fig. 1). One hundred thirty-seven
families agreed to participate in the pretest assessment. The fami-
lies were recruited from 11 agencies situated in different munici-
palities in Norway. The participating parents had contacted an
agency because of a child (3e12 years) who was either at an early
stage of problem behavior development or had developed conduct
problems. To match the regular procedures for referral or intake to
municipal child and family services, no formal screening proce-
dures were used as part of this study; the intervention was offered
based on the practitioners’clinical judgments.
Children between the ages of 3 and 12 who exhibited conduct
problems were eligible to participate in the study. Children were
excluded if they (a) were diagnosed with autism, (b) had been
exposed to documented sexual assaults, (c) were mentally
retarded, or (d) had parents with serious mental health problems or
severe mental retardation (see Fig. 1). No child was excluded from
this study because of these restrictions.
Seventy-seven (56.2%) of the children in the sample scored
above the cutoff for conduct problems on the Eyberg Intensity scale
(based on the suggested Norwegian cutoff scores for children in
potential need of psychiatric treatment for conduct problems at the
90th percentile, Reedtz et al., 2008) at the pre assessment. This
finding indicated that a considerable number of the participating
children had already developed conduct problems in the clinical
range by the time of intake.
The 137 children in this study ranged from 2 to 12 years of age at
intake (M¼8.56, SD ¼2.35), and 50 (36.5%) were girls. One
hundred and thirteen children were in school, 21 were in kinder-
garten and three were cared for at home. The average age of the
reporting parent was 37.42 years (SD ¼6.34). Of the participating
children, 66 (48.2%) lived with both biological parents, 21 (15.3%)
with parents who were married to or cohabiting with another
adult, and 50 (36.5%) lived with single parents (divorced, separated,
or never married). The proportion of single parents in the study was
markedly higher than in the general population (20.1%, Statistics
Randomized (N =137)
Allocation
Follow-up
Post-Assessment
Assessed (n = 59)
Lost to follow-up (n = 6)
Refused to participate (n = 6)
Analyzed with ITT (n = 65) Analyzed with ITT (n = 72)
Analysis
Enrollment
Excluded (n = 16)
Not meeting inclusion criteria
(n = 0)
Refused to participate (n= 16)
Other reasons (n = 0)
Assessed (n = 66)
Lost to posttest (n = 6)
Discontinued intervention (n = 8,
i.e., participated less than 50%)
Refused to participate (n = 6)
Assessed (n = 60)
Lost to posttest (n = 5)
Discontinued intervention (n= 0)
Refused to participate (n= 5)
Assessed (n = 64)
Lost to follow-up (n = 8)
Refused to participate (n = 8)
Assessed for eligibility (N = 153)
Randomized to BPT (n = 72)
Received intervention (n = 67)
Did not receive intervention (n = 5)
Reason: Did not want to receive
intervention
Randomized to comparison group
(n = 65)
Received intervention (n = 32)
Did not receive intervention (n = 33)
Reasons: Agencies did not provide
services or families did not ask for
services
Fig. 1. Flowchart for the PMTO effectiveness study. ITT ¼Intent-to-treat.
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121 115
Author's personal copy
Norway, 2012). The average gross annual family income was
509,609 Norwegian Kroner (SD ¼347.701), which is approximately
$83,542. The parent self-report revealed that 37 (27%) parents had
a college or higher university degree, 83 (60.6%) had finished high
school, and 17 (12.4%) had completed junior high school or
elementary school. Most of the parents had a Norwegian back-
ground (126 or 92%), while one (.7%) was from another western
European country and 8 (7.3%) reported “other”ethnicity.
Procedures
The project was approved by the regional ethics board. The
families were recruited from January 2008 to May 2009. The eligible
families were informed of the study and agreed to participate by
signing a written informed consent document. The participating
families were assessed before (pretest, T1), immediately after
(posttest, T2), and six months after (follow up, T3) completion of
PMTO. The assessments took place at the agencies where the inter-
ventions were offered. If the parents agreed (all provided consent),
the children’s teachers were informed of the study and asked to
complete questionnaires about the child’s behavior. The family
assessment sessions lasted for about 1 h. The parents completed
questionnaires about family demographics and parent and child
behavior. For budgetary and logistic reasons, only the parent who
reported having spent the most time with the child was asked to
complete the questionnaires. The parents were not offered any
economic compensation for participating in the study.
After completing the pretest, the families were randomly
assigned to PMTO or the comparison condition (see Fig. 1). The
random assignment procedure was computer generated by a staff
member who did not recruit families and was monitored by the
principal investigator of the study. The local interventionists were
informed of which families should receive PMTO or regular practice
upon completion of the randomization. No interim analyses were
conducted during the trial. The recruitment of families ended when
the participating agencies had reached 142 families.
Interventions
The aim of PMTO is to promote effective parenting skills to
reduce and prevent the further escalation of child conduct prob-
lems. Throughout the intervention, parents are encouraged to learn
and role-play core parenting practices. PMTO as tested in this study
was delivered to groups of families, with a maximum number of 16
participants (the caregivers of eight children) in each group. The
intervention is manual and principle-based (Askeland, Solholm,
Jørgensen, & Pettersen, 2006) and is directly derived from
individually delivered PMTO (Ogden & Hagen, 2008) and the PMTO
intervention for single mothers and stepfamilies developed at the
Oregon Social Learning Center (Forgatch & DeGarmo, 1999). The
group-based PMTO program consists of 12 weekly sessions lasting
2.5 h (30 h in total), which is about the same number of hours as full
scale PMTO therapy (M¼28 h, Ogden & Hagen, 2008).
The parent groups were led by two group leaders, at least one of
whom had to be a PMTO therapist. All of the groups in this study
were led by two PMTO therapists. Eleven pairs of interventionists
participated in the PMTO condition (ten pairs had one group and
one pair had two groups). There were 11 parent groups, which
ranged in size from 4 to 13 families (M¼6.52). The PMTO therapists
were recruited from child service agencies and child psychiatric
services. To ensure the integrity of the intervention, the training
was face-to-face and required the interventionists to complete
a standardized training that lasted for two days (Askeland et al.,
2006). The training consisted of lectures, role-play exercises, and
home assignments. Following the training program, the PMTO
therapists took part in group-based supervision meetings every 6
months to discuss, problem solve, and role-play challenges that had
emerged during the intervention sessions.
The comparison group was offered PMTO after the termination
of the study period (i.e., T3) and was free to seek and receive any
available intervention offered in regular services except for closely
related interventions based on the same principles (e.g., BPT or
individual PMTO). None of the practitioners who offered inter-
vention in the comparison group were trained in or practiced
PMTO. Because some families chose to receive PMTO instead of
regular services after the study period, and because some agencies
did not offer any other type of services or interventions, 33 (51%)
families in the comparison group did not receive any intervention.
The other families received school-based psychological counseling
(n¼12) or counseling from public health nurses (n¼5), social
workers in the child welfare services (n¼4), or other professionals.
Measures
The parent report outcome measures included parenting prac-
tices, child conduct problems, child social competence, child
internalizing problems, and parental distress. The teacher reports
included child conduct problems, child social competence, and
child internalizing problems.
Parenting practices were measured with the Parenting Practices
Interview (PPI), which is a widely used instrument (e.g., Reid,
Webster-Stratton, & Beauchaine, 2001) that has previously been
translated and used in randomized trials in Norway (Kjøbli &
Ogden, 2012;Larsson et al., 2009). The following scales were
used to measure parenting skills in the present study: (1) Harsh for
Age (9 items, e.g., “Sometimes it takes getting really angry with
children to teach them a lesson”), with Cronbach’s alphas of .74
(T1), .58 (T2) and .61 (T3); (2) Harsh Discipline (14 items, e.g., “Give
your child a spanking”), with Cronbach’s alphas of .86 (T1), .88 (T2)
and .89 (T3); (3) Inconsistent Discipline (6 items, e.g., “How often is
your child successful in getting around the rules you have set?”),
with Cronbach’s alphas of .78 (T1), .67 (T2) and .72 (T3); (4)
Appropriate Discipline (16 items, e.g., “Get your child to correct the
problem or make up for his/her mistake”), with Cronbach’s alphas
of .82 for T1, .82 for T2 and .78 for T3, respectively; (5) Positive
Parenting (15 items, e.g., “Praise or compliment your child”), with
Cronbach’s alphas of .77 (T1) and .81 (T2) and .79 (T3); and (6) Clear
Expectations (3 items, e.g., “I have made clear rules or expectations
for my child about chores”), with Cronbach’s alphas of .72 (T1), .63
(T2) and .48 (T3).
The parent reported child outcomes were measured with the
Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999), the
Home and Community Social Behavior Scales (HCSBS; Merrell &
Caldarella, 2002), and the Child Behavior Check List (CBCL;
Achenbach, 1991). The ECBI is widely used and consists of 36 items
(e.g., “Has temper tantrums”) that have been translated to Norwe-
gian and standardized in a national sample of children (Reedtz et al.,
2008). The instrument consists of an intensityscale that indicates the
frequency of conduct problems (7-point Likert scale items) and
a problem scale that indicates whether the reporting parent views
the behaviors of the child as problematic (scored as 1) or not prob-
lematic (scored as 0). Cronbach’s alphas for the Intensity scale were
.93atT1,.93atT2and.92atT3.Cronbach’s alphas for the Problem
scale were .88 (T1), .90 (T2), and .90 (T3), respectively. The HCSBS
inventory consists of 64 items (rated on a 5-point Likert scale) that
indicate both conduct problems (32 items, e.g., “Gets into fights”)
and social competence (32 items, e.g., “Is accepting of peers”). The
scale was translated into Norwegian for this project, and Cronbach’s
alphas for the conduct problems pretest and posttest were .94 (T1),
.94 (T2), and .94 (T3). Cronbach’s alphas for the social competence
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121116
Author's personal copy
pretest and posttest were .95 (T1), .96 (T2) and .96 (T3), respectively.
The CBCL was used to measure child anxiety and depression only,
whereas ECBI and HCSBS were used to measure conduct problems.
The CBCL has been validated and standardized in a Norwegian study
(Nøvik, 1999). The Anxious/Depressed Scale (14 items, item example
“Worr ie s”) consists of 3-point Likert scale items to which the
respondent can answer “0”if the item is never/seldom true of the
child, “1”if the item is sometimes or somewhat true, and “2”if it is
often or always true. Cronbach’s alphas for this scale were .85 (T1),
.85 (T2) and .82 (T3).
The teacher reported child outcomes were measured with the
School Social Behavior Scales (SSBS; Merrell & Caldarella, 2002)and
the Teacher Report Form (TRF; Achenbach , 1991). The SSBS inventory
was translated into Norwegian and consists of 64 items (5-point
Likert scale) that indicate both conduct problems (32 items, e.g.,
“Insults peers”) and social competence (32 items, e.g., “Cooperates
with other students”). Cronbach’s alphas for the conduct problems
were .94 at T1, T2 and T3, but .95 (T1), .96 (T2) and .96 (T3) for social
competence. The TRF was used to measure child anxiety and
depression only, whereas SSBS was used to measure conduct prob-
lems. The TRF has been validated and standardized in a Norwegian
study (Lurie, 2006). The Anxious/Depressed Scale had 18 items
scored on a 3-point Likert scale. An item example is “Afraid of
mistakes,”and Cronbach’s alphas were .88 for T1, .89 for T2, and .85
for T3, respectively. Parental mental distress was assessed with the
Symptom Check Liste5 (SCL-5; e.g., “Feeling fearful”), which is
a short form of SCL-25 that has been validated in a Norwegian
sample (Tambs & Moum, 1993). Cronbach’s alphas for this list were
.87 (T1), .89 (T2), and .90 (T3), respectively.
The parent reported adherence to PMTO was measured after the
3rd, 8th and 12th sessions with an instrument developed by the
Norwegian research group conducting this study. The measure
consisted of 44 items (scored on a 5-point Likert scale) that indi-
cated the degree to which the PMTO therapists covered the topics
and the core components of the intervention (e.g., “We have
practiced how to give my child good directions”and “We practiced
how to use timeout”). Cronbach’s alpha for all of the items on the
instrument was .87.
Analytical procedures
The analyses were performed in PASW (formerly SPSS) version
18. T-tests (independent sample) and chi-squared tests were
carried out to examine differences between the PMTO and
comparison groups at baseline. Linear mixed models (LMM) were
used in intent-to-treat (ITT) analyses to examine the effects of
intervention. That is, all of the randomized cases were included
regardless of whether they had received intervention or partici-
pated in all of the assessments. LMMs have the advantage of using
all of the available data in addition to accounting for the correlation
between repeated measurements from the same subject, modeling
time effects, and handling missing data more appropriately than
traditional ANOVAs. In contrast to traditional ANOVAs where only
complete cases are included or the last observation carried forward
procedure is applied, LMM takes all available observations (i.e., the
direct likelihood method) into account in the analyses. LMM uses
correlated residual structures to account for repeated observations
within individuals and provides the tools necessary to estimate
fixed and random effects in one model (West, 2009). In the present
study, the effect of treatment (PMTO vs. comparison group), group
and time and the group time interaction on the outcomes was
investigated as a fixed effect, but the latter indicated differential
change between the groups over time. Each outcome measure
was analyzed in separate unconstrained models. The caregivers in
the PMTO condition were nested within the groups. Therefore,
additional models including the group level as a fixed factor were
run to examine if group nesting influenced the outcomes.
To estimate the magnitude of the intervention effects, we
calculated the effect sizes (Cohen’sd) based on the t-tests from the
LMMs. By convention, effect sizes less than .20 are regarded as
small, .50 as moderate, and .80 as large (Cohen, 1992). A p-value
<.05 was considered significant.
Results
Attrition
The 137 participants demonstrated a high attendance rate at
pre-assessment (T1). One hundred twenty-six participants (92%)
also participated in the post-assessment (T2), while 123 (89.8%)
participated in the follow-up (T3). Of the participants who
completed all of the assessments, 64 (52%) were randomized to the
PMTO group and 59 (48%) to the comparison group. Moreover,
there were no indications of systematic attrition in the study. The
T1 comparisons (t-tests and chi squared tests) between the attrition
group and those who completed all of the assessments showed no
differences in the demographic characteristics (child age, child
gender, ethnicity, number of siblings, single-parent household,
education, family income, or parent age) and outcome variables
(i.e., parenting practices and child behaviors).
Baseline comparisons
To test for differences between the intervention conditions at
Time 1, the participants who completed all of the assessments were
compared in terms of demographic characteristics and outcome
variables. No significant differences in the demographic variables or
the parent reports of the children’s health, attendance in special
class or the number of children in the family emerged.
Group nesting
We used additional models to examine the influence of group
nesting in the PMTO condition. The findings revealed that the
group time interaction only had a significant effect on Appropriate
discipline (F(20, 67) ¼2.03, p<.02), indicating that group nesting
did not influence any other outcomes in this study. Because group
nesting had a significant effect on Appropriate discipline, group
nesting and the interaction term of group nesting time was
included as a fixed factor in the main effect analysis. The inclusion of
group nesting in the model did not noticeablychange the outcome of
Appropriate discipline and thus was not included in the final model.
Intervention dosage (attendance)
Dosage was measured by the interventionists’and parents’
reports of the registered number of sessions the families had
completed. Dosage reports were available for 65 of the families in
the PMTO group, while only 24 reports were available in the
comparison group. The families in the PMTO group received
significantly more hours of intervention than the comparison group
(t(103) ¼7.61, p¼.00). The PMTO group received an average of
25.65 h of intervention (SD ¼7.98), while the comparison group
received 6.71 h (SD ¼15.29). Five families in the PMTO group did
not receive intervention, while eight received less than 50% of the
intervention (i.e., less than 15 h).
To examine whether the differences between the PMTO group
and the comparison group resulted from 33 families (51%) in the
comparison group not receiving intervention, additional LMMs
were run to determine whether receiving or not receiving
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121 117
Author's personal copy
intervention in the comparison group predicted different slopes
(intervention time). No LMMs were significant at the .05 level,
indicating that the families in the comparison group had similar
outcomes irrespective of whether they received intervention or
not. This finding indicated that the differences between the inter-
vention group and the comparison group would have been similar
regardless of whether the families in the comparison group
received intervention or not.
Adherence
The parent reported practitioner adherence to PMTO was found
to be high, with a mean score of 4.84 (SD ¼.18) out of a maximum
score of 5 (the sum-score for the adherence scale was divided by
the number of items).
Intervention effects
As shown in Table 1, 6 of the 14 Intervention Time effects were
significant, indicating different slopes in the PMTO and the
comparison group from T1 to T3 on these outcomes. When
analyzing the intervention effects at T2, 8 of the 14 outcomes were
significantly in favor of PMTO (see Table 1). The parents in the
PMTO group reported that the children presented significantly
fewer problems on the Eyberg Intensity and Eyberg Problem scales
at T2. The children in the group condition scored higher on Merrell
social competence on both the parent and teacher reports. The
parents in the PMTO group reported higher Positive Parenting and
Clear Expectations scores and lower Harsh Discipline scores than
the parents in the comparison group. The parents in the group
condition also reported lower levels of parental mental distress
(SCL-5) at T2. At T3, 7 of the 14 outcomes were significantly in favor
of the group treatment format. The parents in the PMTO group
reported that the children presented significantly fewer problems
on the Eyberg Intensity and Merrell externalizing scales. The parent
reports found that the children in the PMTO group scored higher on
Merrell social competence. The effect on Merrell social competence
was not, however, significant at T3 on the teacher reports. The
parents in the PMTO group reported higher Positive Parenting
scores and lower Harsh and Inconsistent Discipline scores than the
parents in the comparison group. The PMTO parents also reported
lower levels of mental distress (SCL-5) at T3.
Additional LMM analyses (not shown) of child gender and age
effects (i.e., that girls and boys or younger and older children had
differential effects) revealed that neither variable influenced the
intervention outcomes in any of the analyses.
Effect sizes
The effect sizes are reported in Table 2 and include all of the
outcome measures, regardless of whether the differences between
the intervention and the comparison group were statistically
significant or not. The magnitude of the effects ranged from small to
large, with the largest effects obtained on parenting practices
(harsh discipline and positive parenting).
Discussion
The present study was a randomized effectiveness trial of
group-based parent training in a sample of 137 families with chil-
dren who had emerging or developed conduct problems. The ITT
results showed that the group-based treatment had a positive
impact on parent ratings of child externalizing behavior, social
competence, parental mental health and parenting practices both
at the termination of treatment and six months later (as measured
by ECBI-intensity, HCSBS, SCL-5 and PPI-harsh discipline and
positive parenting, respectively). The group-based training exhibi-
ted an immediate significant positive effect on clear parent
expectations (PPI) and teacher reported social competence (SSBS),
but no difference between treatment conditions was found at
follow up six months later. On the other hand, the group treatment
seemed to have a delayed positive effect on parent reported child
externalizing problems (HSCBS) and consistent discipline (PPI). In
line with previous studies (Ogden & Hagen, 2008), no significant
effects were found for any of the internalizing outcome variables,
indicating that the group intervention did not have any effect on
the children’s symptoms of anxiety and depression. In sum, the
most encouraging changes were found in the primary intervention
domains of parenting practices, child externalizing behavior and
social competence.
The findings showed that the effects were in favor of group-
based PMTO for 8 of the 14 outcomes at T2, while 7 of the 14
outcomes were significant at T3. The effect sizes (Cohen’sd)were
robust, with the largest effect sizes registered for self-reports of
harsh discipline and positive parenting and the latter measure
increasing from .88 to .99 between the post assessment
and follow up (see Table 2). The effect sizes compare well
with the average effect sizes found in the Lundahl et al. (2006)
Table 1
Estimated means, standard errors, F-values and p-values.
Variables Intervention group Comparison group Omnibus test Post-test Follow-up test
T1 Mean (SE) T2 Mean (SE) T3 Mean (SE) T1 Mean (SE) T2 Mean (SE) T3 Mean (SE) Fptptp
Parent reports
Eyberg intensity 135.96 (3.68) 114.85 (3.46) 111.09 (3.21) 133.64 (3.88) 122.85 (3.68) 120.28 (3.48) 4.21 .02 2.42 .02 2.73 .01
Eyberg problem 16.97 (.85) 9.94 (.99) 8.79 (.98) 17.15 (.89) 12.65 (1.06) 11.53 (1.08) 2.26 .11 1.96 .05 1.78 .08
Merrell externalizing 80.38 (2.59) 72.13 (2.50) 65.21 (2.28) 81.57 (2.73) 75.91 (2.66) 73.65 (2.48) 2.81 .06 .85 .40 2.27 .03
Merrell social competence 100.15 (2.62) 113.24 (2.70) 115.19 (2.76) 99.68 (2.76) 102.29 (2.87) 106.76 (3.00) 5.54 .01 3.32 .00 2.18 .03
CBCL anxiety/depression 8.57 (.67) 6.22 (.61) 6.20 (.59) 8.25 (.71) 7.02 (.65) 6.32 (.65) 1.24 .29 1.53 .13 .55 .58
SCL-5 9.79 (.46) 8.28 (.48) 8.03 (.45) 9.27 (.49) 9.08 (.51) 8.93 (.50) 2.71 .07 2.16 .03 2.09 .04
Harsh discipline 31.61 (1.15) 24.57 (1.04) 24.56 (.99) 30.24 (1.21) 29.23 (1.10) 29.01 (1.08) 14.64 .00 5.07 .00 4.46 .00
Inconsistent discipline 19.31 (.60) 17.34 (.51) 16.57 (.50) 19.80 (.63) 18.82 (.54) 18.52 (.55) 1.92 .15 1.26 .21 1.96 .05
Appropriate discipline 73.44 (1.56) 74.78 (1.54) 75.40 (1.55) 70.31 (1.65) 71.43 (1.64) 68.61 (1.69) 2.02 .14 .10 .92 1.58 .12
Positive parenting 66.48 (1.25) 76.72 (1.33) 74.77 (1.30) 66.44 (1.32) 67.51 (1.41) 65.63 (1.42) 19.67 .00 5.13 .00 5.49 .00
Clear expectations 16.63 (.39) 18.34 (.27) 17.79 (.28) 15.59 (.41) 15.50 (.29) 16.41 (.32) 7.30 .00 3.27 .00 .56 .58
Teacher reports
Merrell externalizing 79.03 (3.53) 73.29 (3.46) 72.46 (3.32) 66.63 (3.75) 67.55 (3.70) 65.86 (3.51) 1.85 .16 1.87 .06 1.51 .13
Merrell social competence 93.13 (3.62) 100.77 (3.57) 99.08 (3.27) 105.60 (3.83) 103.62 (3.80) 104.96 (3.49) 3.80 .03 2.76 .01 1.78 .08
TRF anxiety/depression 8.26 (.77) 8.63 (.81) 6.97 (.72) 6.84 (.82) 6.79 (.87) 6.95 (.76) 1.76 .18 .49 .62 1.32 .19
Note: Analyzed with linear mixed method. N¼137.
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121118
Author's personal copy
meta-analysis. In the current sample, the average effect size
immediately after intervention was .30 for conduct problems and
.51 for parenting practices. At follow up, the average effect size
was .39 for conduct problems and .49 for parenting practices.
Eight effect sizes decreased during the follow up period, while six
increased. Contrary to the findings in the meta-analysis from
Lundahl et al. (2006), the positive effect sizes did not generally
decline over time.
Although an indication of a positive generalization from family to
school or the kindergarten context was found at the end of the
intervention, these effects were not sustained at the follow up. The
positive short-term effect of the group training on teacher rated
social competence was not present at the follow up, and at no time
point did the teacher ratings of externalizing behavior differ between
the intervention conditions. The teachers and parents rated the
children similarly at intake (see Table 1), which indicates that the
lack of teacher reported effects were not caused by teachers’low
ratings of children prior to the group treatment (i.e., floor effect). The
findings from this study and others (e.g., Drugli & Larsson, 2006;
Kjøbli & Ogden, 2012) suggest that the effect of parent training
interventions does not necessarily transfer to schools and kinder-
gartens. Children with conduct problems have been found to interact
coercively with teachers, receive less support, and more criticism in
the school context (Snyder, 2002). This suggests that adding teacher
interventions to parent training could improve child outcomes in the
school and daycare setting. Correspondingly, Webster-Stratton et al.
(2004) have showed additive treatment effects on child conduct
problems when combining teacher and parent training compared to
parent training alone.
Limitations of the study and future directions
One of the limitations of this study was the constraint imposed
by the sample size. Although the sample provided adequate power
to test for intervention effects, the power was not ideal for testing
the impact of potential moderators (e.g., gender and age). The
LMMs that included gender and age in the models should therefore
be interpreted with caution, as the sample size increases the like-
lihood for type II errors (i.e., failure to reject the null hypothesis).
The parent reported adherence to group-based parenting was
measured with high average ratings; the scores came close to
demonstrating a ceiling effect. It would be valuable to compare this
measure with other sources of information, such as observational
data (Schoenwald et al., 2010), in a validation study. A reliance on
one source of information can inflate correlations between vari-
ables, and the parent reported adherence could be biased because
of positive attitudes toward the interventionist (Breitenstein
et al., 2010).
Although the participants represented the whole range of
income levels, a limitation of the study is that the sample was
predominantly from middle to upper middle class families. This
bias may have inflated outcomes as the study by Lundahl et al.
(2006) showed that group based parent training had less benefi-
cial outcomes on socially disadvantaged families.
The intervention tracking process for the comparison group was
not optimal and could preferably have been more systematic and
detailed. Moreover, although the parents in the comparison group
were free to seek and receive other services, to the best of our
knowledge about half of the participants in the comparison group
did not receive any intervention at all. Some participants may not
have looked for an alternative intervention because they were told
that they could receive PMTO after the study period ended. The
effects may have turned out differently if the design of the study
had included a comparison group where all of the participants
received some form of intervention. A meta-analysis by Weisz,
Jensen-Doss, and Hawley (2006) that included 32 randomized
controlled trials showed that usual care sometimes outperformed
evidence based interventions. This indicates that future studies
should track and describe the contents of usual care or services to
reveal the effective components and procedures of those
approaches (Weisz & Gray, 2008). A direct comparison of individ-
ually based and group-based parent management training would
also be relevant in comparing the effectiveness of the two training
formats.
Practical implications and conclusion
Norway has developed a two level parent training model in
which parents may first ask the municipal child and family services
for help with their children’s disruptive behavior problems. The
parents are offered BPT (approximately 5 sessions) carried out by
trained practitioners in the public child service system (Kjøbli &
Ogden, 2012) without the need for any formal diagnosis. Those
who do not benefit from first level services or have children who
initially have more serious behavior problems can receive second
level full scale parent training from qualified PMTO therapists in the
specialist services (Hagen et al., 2011;Ogden & Hagen, 2008). PMTO
has typically been offered to individual families at this second level.
Only recently has PMTO in Norway been adapted to a group format
as commonly applied in programs such as the Incredible Years
(Webster-Stratton,1984) and Triple P (Sanders, 2008). The potential
reach of the PMTO program (Glasgow et al., 1999) is increased by
offering it to groups of parents in place of individually delivered
parent training.
This study indicates that group-based parent training is effective
in reducing conduct problems in real-world settings up to six
months after the intervention, which makes it likely that the
positive findings in this study reflect the true effects of PMTO in
Norway. Consistent with findings from the individually delivered
PMTO trial (Ogden & Hagen, 2008) and the BPT trial (Kjøbli &
Ogden, 2012), a relatively high number of single parents partici-
pated in this study. This is encouraging, given previous research
that has found single parenthood to be a risk factor for child
conduct problems (DeGarmo & Forgatch, 2005). However, few
families with low SES or an immigrant status were recruited for this
trial. As documented by Bjørknes, Jacobsen, and Nærde (2011),
additional recruitment strategies are needed to reach ethnic
minority groups. The findings from the current study indicate that
Table 2
Effect sizes (Cohen’sd) at posttest and follow-up.
Variables Posttest (T2) Follow-up (T3)
Cohen’sd95% CI Cohen’s d 95% CI
Parent reported outcomes
Eyberg intensity .42 .08e.76 .47 .13e.81
Eyberg problem .34 .00e.68 .31 .03e.65
Merrell externalizing .15 .19e.48 .39 .05e.73
Merrell social competence .57 .22e.91 .38 .04e.72
CBCL anxiety/depression .26 .08e.60 .10 .24e.43
SCL-5 .37 .03e.71 .36 .02e.70
Parent reports of parenting practices
Harsh discipline .87 .51e1.22 .77 .42e1.11
Inconsistent discipline .22 .12e.56 .34 .00e.68
Appropriate discipline .02 .32e.36 .27 .07e.61
Positive parenting .88 .52e1.23 .95 .59e1.30
Clear expectations .56 .22e.90 .10 .24e.43
Teacher reported outcomes
Merrell externalizing .32 .02e.66 .26 .08e.60
Merrell social competence .47 .13e.81 .31 .03e.65
TRF anxiety/depression .11 .23e.44 .23 .11e.57
Note: Cohen’sdwas calculated using the formula: d¼tffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n
t
þn
c
n
t
n
c
n
t
þn
c
n
t
þn
c
2
s:
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121 119
Author's personal copy
the same may be the case with low SES families. Still, most of the
families eligible for participation took part in the intake assess-
ment, and the attendance rate was high. This indicates that the
parents who were offered participation in the group-based inter-
vention to a large extent accepted and completed the group
training. The positive outcomes and the potential reach of this
intervention are encouraging contributions to the efforts of
reducing the prevalence of child conduct problems at the pop-
ulation level. Future studies should examine the impact of parent
training on the prevalence of conduct problems, as doing so could
help answer the ultimate question of whether parent management
training is beneficial to public health.
Acknowledgments
We are grateful to the following people who worked extensively
toward the coordination of the study, data collection, data
management and statistical analyses: Trine Staer, Terje Christian-
sen, Roar Solholm, Bjørn Arild Kristiansen and Torbjørn Torsheim.
Special thanks go to the families, agencies and interventionists for
their contributions to this study.
References
Achenbach, T. M. (1991). Integrative guide for the 1991 CBCL/4-18, YSR and TRF
profiles. Burlington: University of Vermont, Department of Psychiatry.
Askeland, E., Solholm, R., Jørgensen, E., & Pettersen, R. (2006). Handbook of PMTO
parent group. Oslo: The Norwegian Center for Child Behavioral Development.
Bjørknes, R., Jacobsen, R., & Nærde, A. (2011). Recruiting ethnic minority groups to
evidence-based parent training. Who will come and how? Children and Youth
Services Review, 33,351e357.
Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010).
Implementation fidelity in community-based interventions. Research in Nursing
and Health, 33,164e173 .
Brestan, E. V., & Eyberg, S. M. (1998). Effective psychosocial treatments of conduct-
disordered children and adolescents: 29 years, 82 studies, and 5,272 kids.
Journal of Clinical Child Psychology, 27,180e189.
Brotman, L. M., Gouley, K. K., Huang, K. Y., Rosenfelt, A., O’Neal, C., Klein, R. G., et al.
(2008). Preventive intervention for preschoolers at high risk for antisocial
behavior: long-term effects on child physical aggression and parenting prac-
tices. Journal of Clinical Child and Adolescent Psychology, 37, 386e396.
Bywater, T., Hutchings, J., Daley, D., Whitaker, C., Yeo, S. T., Jones, K., et al. (2009).
Long-term effectiveness of a parenting intervention for children at risk of
developing conduct disorder. British Journal of Psychiatry, 195,318e324.
Chadwick, O., Momcilovic, N., Rossiter, R., & Stumbles, E. (2001). A randomized trial
of brief individual versus group parent training for behaviour problems in
children with severe learning disabilities. Behavioural and Cognitive Psycho-
therapy, 29,151e167.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112,155e159.
Cunningham, C. E., Bremner, R., & Boyle, M. (1995). Large group community-based
parenting programs for families of preschoolers at risk for disruptive behavior
disorders - utilization, cost-effectiveness, and outcome. Journal of Child
Psychology and Psychiatry and Allied Disciplines, 36,1141e1159.
DeGarmo, D. S., & Forgatch, M. S. (2005). Early development of delinquency within
divorced families: evaluating a randomized preventive intervention trial.
Developmental Science, 8, 229e239.
Drugli, M. B., & Larsson, B. (2006). Children aged 4-8 years treated with parent
training and child therapy because of conduct problems: generalisation effects
to day-care and school settings. European Child and Adolescent Psychiatry, 15,
392e399.
Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial
treatments for children and adolescents with disruptive behavior. Journal of
Clinical Child & Adolescent Psychology, 37,215e237.
Eyberg, S. M., & Pincus, D. (1999). Eyberg child behavior inventory and Sutter-Eyberg
student behavior inventory: Professional manual. Odessa, FL: Psychological
Assessment Resources.
Forgatch, M. S., & DeGarmo, D. S. (1999). Parenting through change: an effective
prevention program for single mothers. Journal of Consulting and Clinical
Psychology, 67,711
e724.
Forgatch, M. S., Patterson, G. R., DeGarmo, D. S., & Beldavs, Z. G. (2009). Testing the
Oregon delinquency model with 9-year follow-up of the Oregon Divorce Study.
Development and Psychopathology, 21,637e660.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact
of health promotion interventions: the RE-AIM framework. American Journal of
Public Health, 89, 1322e1327.
Hagen, K. A., Ogden, T., & Bjornebekk, G. (2011). Treatment outcomes and mediators
of parent management training: a one-year follow-up of children with conduct
problems. Journal of Clinical Child and Adolescent Psychology, 40,165e178.
Ingoldsby, E. M., Shaw, D. S., Winslow, E., Schonberg, M., Gilliom, M., & Criss, M. M.
(2006). Neighborhood disadvantage, parentechild conflict, neighborhood peer
relationships, and early antisocial behavior problem trajectories. Journal of
Abnormal Child Psychology, 34, 303e319.
Kerner, J., Rimer, B., & Emmons, K. (2005). Introduction to the special section on
dissemination edissemination research and research dissemination: how can
we close the gap? Health Psychology, 24,443e446.
Kjøbli, J., & Ogden, T. (2012). A randomized effectiveness trial of brief parent
training in primary care settings. Prevention Science. http://dx.doi.org/10.1007/
s11121-012-0289-y.
Larsson, B., Fossum, S., Clifford, G., Drugli, M. B., Handegard, B. H., & Morch, W. T.
(2009). Treatment of oppositional defiant and conduct problems in young
Norwegian children: results of a randomized controlled trial. European Child
and Adolescent Psychiatry, 18,42e52.
Lundahl, B., Risser, H. J., & Lovejoy, M. C. (2006). A meta-analysis of parent training:
moderators and follow-up effects. Clinical Psychology Review, 26,86e104.
Lurie, J. (2006). Teachers’perceptions of emotional and behavioral problems in 6e12
year old Norwegian school children. Trondheim: Barnevernets Utviklingssenter i
Midt-Norge.
Mccart, M. R., Priester, P. E., Davies, W. H., & Azen, R. (2006). Differential effec-
tiveness of behavioral parent-training and cognitive-behavioral therapy for
antisocial youth: a meta-analysis. Journal of Abnormal Child Psychology, 34,
527e543.
Merrell, K. W., & Caldarella, P. (2002). Home and community social behavior Scales:
User’s guide. Eugene, OR: Assessment-Intervention Resources.
Moffitt, T. E. (2006). Life-course-persistent versus adolescence-limited antisocial
behavior. In D.Cicchetti., & D. J. Cohen (Eds.), Developmental psychopathology:
Risk, disorder and adaptation (2nd ed.) (pp. 570e598). New Jersey: John Wiley &
Sons.
Nixon, R., Sweeney, L., Erickson, D., & Touyz, S. (2003). Parentechild interaction
therapy: a comparison of standard and abbreviated treatments for opposi-
tional defiant preschoolers. Journal of Consulting and Clinical Psychology, 71(2),
251e260.
Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of Triple P-
Positive Parenting Program using hierarchical linear modeling: effectiveness
and moderating variables. Clinical Child and Family Psychology Review, 11,
114 e144.
Nøvik, T. S. (1999). Validity and use of the child behavior checklist in Norwegian
children and adolescents: An epidemiological and clinical study. Oslo: Centre for
Child and Adolescent Psychiatry, Department Group of Psychiatry, University
of Oslo.
Ogden, T., & Hagen, K. A. (2008). Treatment effectiveness of Parent Management
Training in Norway: a randomized controlled trial of children with conduct
problems. Journal of Consulting and Clinical Psychology, 76, 607e621.
Patterson, G. R. (1982). A social learning approach: Coercive family process, Vol. 3.
Eugene, OR: Castalia Publishing Company.
Patterson, G. R., DeGarmo, D., & Forgatch, M. S. (2004). Systematic changes in
families following prevention trials. Journal of Abnormal Child Psychology, 32,
621e633.
Reedtz, C., Bertelsen, B., Lurie, J., Handegard, B. H., Clifford, G., & Morch, W. T. (2008).
Eyberg Child Behavior Inventory (ECBI): Norwegian norms to identify conduct
problems in children. Scandinavian Journal of Psychology, 49,31e38.
Reid, M. J., Webster-Stratton, C., & Beauchaine, T. P. (2001). Parent training in head
start: a comparison of program response among African American, Asian
American, Caucasian, and Hispanic mothers. Prevention Science, 2, 209e227.
Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training efficacy for child
externalizing behavior problemsea meta-analytic review. Journal of Child
Psychology and Psychiatry and Allied Disciplines, 47,99e111.
Sanders, M. R. (2008). Triple P-positive parenting program as a public health
approach to strengthening parenting. Journal of Family Psychology, 22,506e517.
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., &
Southam-Gerow, M. A. (2010). Toward the effective and efficient measurement
of implementation fidelity. In Administration and policy in mental health and
mental health services research.
Schuhmann, E., Foote, R., Eyberg, S., Boggs, S., & Algina, J. (1998). Efficacy of parente
child interaction therapy: interim report of a randomized trial with short-term
maintenance. Journal of Clinical Child Psychology, 27(1), 34e45.
Scott, S. (2005). Do parenting programmes for severe child antisocial behaviour
work over the longer term, and for whom? One year follow-up of a multi-centre
controlled trial. Behavioural and Cognitive Psychotherapy, 33, 403e421.
Shaw, D. S., Connell, A., Dishion, T. J., Wilson, M. N., & Gardner, F. (2009).
Improvements in maternal depression as a mediator of intervention effects
on early childhood problem behavior. Development and Psychopathology, 21,
417e439.
Shaw, D. S., Owens, E. B., Giovannelli, J., & Winslow, E. B. (2001). Infant and toddler
pathways leading to early externalizing disorders. Journal of the American
Academy of Child & Adolescent Psychiatry, 40,36e43.
Snyder, J. (2002). Reinforcement and coercion mechanisms in the development of
antisocial behavior: peer relationships. In J. J. Snyder, J. B. Reid, & G. R. Patterson
(Eds.), Antisocial behavior in children and adolescents: a developmental analysis
and model for intervention (pp. 101e145). Washington, DC: American Psycho-
logical Association.
Statistics Norway. (2012). Inntekt for par med barn og enslige forsørgere [Incomes for
couples with children and single parents]. http://statbank.ssb.no/statistikkbanken
Accessed 19 March).
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121120
Author's personal copy
Tambs, K., & Moum, T. (1993). How well can a few questionnaire items indicate
anxiety and depression? Acta Psychiatrica Scandinavica, 87, 364e367.
Taylor, T. K., & Biglan, A. (1998). Behavioral family interventions for improving child-
rearing: a review of the literature for clinicians and policy makers. Clinical Child
and Family Psychology Review, 1,41e60.
Webster-Stratton, C. (1984). Randomized trial of two parent-training programs for
families with conduct-disordered children. Journal of Consulting and Clinical
Psychology, 52,666e678.
Webster-Stratton, C., & Herman, K. C. (2008). The impact of parent behavior-
management training on child depressive symptoms. Journal of Counseling
Psychology, 55,473e484.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with
early-onset conduct problems: intervention outcomes for parent, child, and
teacher training. Journal of Clinical Child and Adolescent Psychology, 33,105e124.
Weisz, J. R., & Gray, J. S. (2008). Evidence-based psychotherapy for children and
adolescents: data from the present and a model for the future. Child and
Adolescent Mental Health, 13,54e65.
Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth
psychotherapies versus usual clinical care: a meta-analysis of direct compari-
sons. American Psychologist, 61,671e689.
West, B. T. (2009). Analyzing longitudinal data with the linear mixed models
procedure in SPSS. Evaluation & the Health Professions, 32, 207e228.
J. Kjøbli et al. / Behaviour Research and Therapy 51 (2013) 113e121 121