Predictability of oppositional defiant
disorder and symptom dimensions in children
and adolescents with ADHD combined type
M. Aebi1*, U. C. Mu ¨ller1, P. Asherson2, T. Banaschewski3, J. Buitelaar4, R. Ebstein5, J. Eisenberg5,
M. Gill6, I. Manor7, A. Miranda8, R. D. Oades9, H. Roeyers10, A. Rothenberger11, J. Sergeant12,
E. Sonuga-Barke10,13, M. Thompson13, E. Taylor2, S. V. Faraone14and H.-C. Steinhausen1,15,16
1Department of Child and Adolescent Psychiatry, University of Zurich, Switzerland;2MRC Social Genetic Developmental and Psychiatry
Centre, Institute of Psychiatry, London, UK;3Department of Child and Adolescent Psychiatry and Psychotherapy, Central Institute of Mental
Health, J 5, Mannheim, Germany;4Department of Psychiatry, Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands;
5Department of Psychology, Hebrew University, Jerusalem, Israel;6Department of Psychiatry, School of Medicine, Trinity College Dublin,
Dublin, Republic of Ireland;7Geha MHC, Petach-Tikva, Israel;8Department of Developmental and Educational Psychology, University of
Valencia, Valencia, Spain;9Clinic for Child and Adolescent Psychiatry and Psychotherapy, University of Duisburg-Essen, Essen, Germany;
10Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium;11Department of Child and Adolescent
Psychiatry, University of Go ¨ttingen, Go ¨ttingen, Germany;12Department of Clinical Neuropsychology, Vrije Universiteit, Amsterdam, The
Netherlands;13School of Psychology, University of Southampton, Southampton, UK;14Departments of Psychiatry and of Neuroscience and
Physiology, SUNY Upstate Medical University, Syracuse, NY, USA;15Aalborg Psychiatric Clinic, Aarhus University Hospital, Denmark;
16Child and Adolescent Clinical Psychology, University of Basel, Switzerland
Background. Oppositional defiant disorder (ODD) is frequently co-occurring with attention deficit hyperactivity
disorder (ADHD) in children and adolescents. Because ODD is a precursor of later conduct disorder (CD) and
affective disorders, early diagnostic identification is warranted. Furthermore, the predictability of three recently
confirmed ODD dimensions (ODD-irritable, ODD-headstrong and ODD-hurtful) may assist clinical decision making.
Method. Receiver-operating characteristic (ROC) analysis was used in order to test the diagnostic accuracy of the
Conners’ Parent Rating Scale revised (CPRS-R) and the parent version of the Strength and Difficulties Questionnaire
(PSDQ) in the prediction of ODD in a transnational sample of 1093 subjects aged 5–17 years from the International
Multicentre ADHD Genetics study. In a second step, the prediction of three ODD dimensions by the same parent
rating scales was assessed by backward linear regression analyses.
Results. ROC analyses showed adequate diagnostic accuracy of the CPRS-R and the PSDQ in predicting ODD in this
ADHD sample. Furthermore, the three-dimensional structure of ODD was confirmed by confirmatory factor analysis
and the CPRS-R emotional lability scale significantly predicted the ODD irritable dimension.
Conclusions. The PSDQ and the CPRS-R are both suitable screening instruments in the identification of ODD. The
emotional lability scale of the CPRS-R is an adequate predictor of irritability in youth referred for ADHD.
Received 16 February 2009; Revised 12 August 2009; Accepted 10 November 2009; First published online 12 April 2010
Key words: Attention deficit hyperactivity disorder, Conners’ Parent Rating Scale Revised, emotional lability,
irritability, oppositional defiant disorder, Strength and Difficulties Questionnaire.
Conduct disorders (CDs) and oppositional defiant
disorders (ODDs) are leading causes of referral for
youth mental health services. Whereas CD criteria are
related to a consistent pattern of rule breaking and
antisocial behaviour, ODD encompasses parenting
and anger-related problems. After the introduction of
ODD to the major classification systems, criticism has
been raised regarding the distinction of ODD from
normal behaviour in adolescence and from milder
forms of CD. Thus, high symptom overlap has been
found for both disorders (Frick et al. 1992). However,
in the meantime ODD has been established as a sep-
arate disorder due to its differentiation from normal
behaviour (Keenan & Wakschlag, 2004), its persistence
into adolescence (Maughan et al. 2004), its psychiatric
co-morbidity (Simonoff et al. 1997; Greene et al. 2002;
Maughan et al. 2004) and its continuity with emotional
* Address for correspondence: Dr M. Aebi, University of Zurich,
Department of Child and Adolescent Psychiatry, Neptunstrasse 60,
8032 Zurich, Switzerland.
Psychological Medicine (2010), 40, 2089–2100.
f Cambridge University Press 2010
disorders after controlling for CD (Nock et al. 2007).
Furthermore, sex differences indicate a less consistent
role of ODD in the development of CD and antisocial
behaviour in girls (Rowe et al. 2002; Moffitt et al. 2008).
Finally, twin studies suggest a different contribution of
gene and environmental factors for ODD rather than
CD (Dick et al. 2005; Hudziak et al. 2005).
ODD is highly co-morbid with attention deficit
hyperactivity disorder (ADHD; Angold et al. 1999;
Egger & Angold, 2006) and several studies have
pointed to ADHD as a precursor of persistent and
serious CD (Loeber et al. 1995; Mannuzza et al. 2004).
Furthermore, independently from ADHD, ODD has
been found to be a significant mediator for the devel-
opment of CD (Lahey et al. 2002; Burke et al. 2005; van
Lier et al. 2007; Biederman et al. 2008b) and is, there-
fore, presumed to have a pivotal role in the develop-
ment of later serious antisocial behaviour. An early
and reliable identification of ODD in ADHD referred
youth may contribute a significant improvement for
the assessment of subtypes and courses of antisocial
behaviour (Moffitt, 1993; Moffitt et al. 2008).
Parent and teacher rating scales have been found to
be useful and reliable instruments for assessing behav-
ioural problems in children and adolescents. The
Conners’ Parent Rating Scale (CPRS; Conners et al.
1998) and the Strength and Difficulties Questionnaire
(SDQ; Goodman, 1997, 2001) are two of the most
common rating scales and have been translated into
diverse languages. Both of these instruments also in-
clude specific scales to screen for ODD (Conners, 1997;
Goodman et al. 2000b; Goodman, 2001). The CPRS and
related versions have been used in previous studies as
screening instruments for various mental disorders
and as outcome parameters in treatment studies deal-
ing with externalizing behaviour problems, including
ADHD (for an overview, see Gianarris et al. 2001). So
far, the CPRS revised oppositional scale (CPRS-R OPP)
has not yet been tested in terms of its predictive val-
idity for ODD (Collett et al. 2003).
In comparison with the Conners’ Parent Rating
Scale revised (CPRS-R), the SDQ is of more recent
origin and is a shorter instrument for screening the
most important mental disorders in childhood and
adolescence. The SDQ addresses five narrowband
syndromes: emotional symptoms; conduct problems;
hyperactivity; peer problems; prosocial behaviour.
A computer algorithm has been developed for the
inattention, anxious-depressed or any psychiatric
disorder. The predictions from the algorithm of the
multi-informant SDQ have been found to correlate
with clinical diagnoses of CD/ODD in referred sub-
(Goodman et al. 2000c; Mathai et al. 2004). High
sensitivity in the detection of clinical CD/ODD has
been established (86–93%), whereas specificity was
only modest, indicating that the SDQ was over-
including subjects in these samples. On the other
hand, in a community sample, a smaller number of
subjects (68.2%) with Internet interview-based di-
agnosis of CD/ODD (Development and Well-Being
Assessment; Goodman et al. 2000a) were rated as
having a probable diagnosis of CD/ODD based on the
SDQ (Goodman et al. 2000b). Due to the high rate of
false positives, the SDQ seems to be more suitable for
screening rather than for confirmation of diagnoses in
A recentstudy based
Multicentre ADHD Genetics (IMAGE) sample has
analysed CPRS-R and the parent version of the SDQ
(PSDQ) in the identification of conduct problems
(Christiansen et al. 2008). This study found that the
CPRS-R OPP and the PSDQ conduct problem scales
(PSDQ CP) yielded the best discrimination of pure
ADHD, ODD and CD. However, the prediction of
ODD as a separate disorder apart from CD has not yet
been analysed in this study. Therefore, the present
study is a first step aimed at the assessment of the
diagnostic accuracy of the CPRS-R and the PSDQ in
the prediction of ODD in an ADHD-referred sample.
These analyses will include the establishment of cut-
off scores. The performance of these instruments in
clinical practice is important given the high prevalence
rates of ADHD and its co-occurrence with ODD in
mental health services. Sound assessments of ODD
will contribute favourably to clinical decision making.
Reflecting the heterogeneous nature of ODD (Lahey
et al. 1999; Burke et al. 2005), Stringaris & Goodman
(2009b) defined three a priori dimensions of oppos-
itionality, which were labelled ODD-irritable, ODD-
headstrong and ODD-hurtful based on the DSM-IV
criteria for ODD. The authors found different associ-
ations with other disorders in a large community
sample of youth aged 5–16 years using parent and
teacher information from a structured Internet-based
diagnostic interview (Development and Well-Being
Assessment; Goodman et al. 2000a). The ODD-irritable
whereas the ODD-headstrong dimension was related
to ADHD and all three dimensions were related to
CD. In a 3-year follow-up study, the longitudinal pre-
diction of these ODD dimensions was tested after
controlling for initial psychopathology in a com-
munity sample (Stringaris & Goodman, 2009a). ODD-
irritable was found to be a predictor of generalized
anxiety disorders and mood disorders, whereas ODD-
headstrong was the sole predictor of ADHD. Not as
expected, among all three dimensions, only the head-
strong dimension was found to be associated with the
2090 M. Aebi et al.
outcome of CD. However, the hurtful dimension was
predicting aggressive CD symptoms. In conclusion,
these findings suggest that ODD is a complex problem
that may require differential clinical interventions ac-
cording to the predominant dimension.
Based on these findings, the second aim of the
present study was to test the predictive power of the
CPRS-R and the SDQ for the irritable, headstrong and
hurtful dimensions of ODD. The performance of the
instruments in these domains may be clinically im-
portant in children and adolescents with ADHD, in-
dependently from the presence of ODD. It has been
shown that irritability is associated with early age
of onset and persistence of major depression (Fava
et al. 2009) and that irritability in combination with
hyperarousal is a core symptom of paediatric bipolar
disorders and severe mood dysregulation (SMD;
Brotman et al. 2006). Particularly in children and ado-
lescents with ADHD, who often show an impaired
affect regulation (Braaten & Rosen, 2000), the assess-
ment of irritability dimension may be of clinical im-
portance for the prevention of future affective and
2009a). Furthermore, an early and reliable assessment
of the hurtful dimension may be helpful for the
identification of callous unemotional features in sub-
jects with early onset and chronic persistent antisocial
behaviour (Moffitt, 1990, 1993). Finally, the assessment
of the headstrong dimension may be important for the
evaluation of parent counselling needs, because these
items predominantly refer to parenting problems.
Prior to testing the predictability of the ODD di-
mensions, the substructure of ODD was analysed
in the present sample with children and adolescents
referred for ADHD by confirmatory factor analysis. In
contrast to the procedure used in the study by
Stringaris & Goodman (2009a,b), the item ‘often de-
liberately annoys people’ was assigned to the ODD-
hurtful dimension because, in a previous study, this
item was most strongly correlated with spiteful be-
haviour (Speltz et al. 1999). Thus, there is some face
validity that this item belongs to the hurtful rather
than the headstrong dimension. In a final step of the
analyses, the accuracy of the CPRS-R and the PSDQ in
addressing these separate dimensions was tested both
in subjects with and without ODD.
The IMAGE study comprises 3229 offspring from 1187
fathers and 1341 mothers. Probands participating in
the present study were European Caucasians aged
5–17 years who had been recruited in 12 child and
countries: Belgium; Germany; Switzerland; Holland;
Republic of Ireland; Israel; Spain; UK. Entry criteria
for probands were a clinical diagnosis of ADHD based
on DSM-IV criteria and access to one or both biological
parents and one or more full siblings for DNA collec-
tion and clinical assessment. Exclusion criteria apply-
ing to both probands and siblings included autism,
epilepsy, IQ<70, brain disorders and any genetic or
medical disorder associated with externalizing behav-
iours that might mimic ADHD.
The original sample of 1401 probands has been re-
stricted to 1225 subjects with ADHD combined type.
Furthermore 91 (7%) were excluded due to missing
information on DSM-IV ODD criteria and another 31
(3%) subjects due to more than 10% missing items in
the CPRS-R or the PSDQ. Thus, the final sample con-
sisted of 1093 probands with a mean age of 10.8
(S.D.=2.8) years. A total of 956 subjects were male
(87.5%) and 726 (66.4%) subjects from the present
sample fulfilled DSM-IV criteria of ODD based on the
Parental Account of Childhood Symptoms (PACS)
interview (see below).
Diagnoses of ADHD and co-morbid disorders were
based on a standardized, semi-structured interview
with the parents (PACS; Taylor et al. 1986; Chen &
Taylor, 2006). The PACS was developed for assessing
ADHD and the most common child psychiatric dis-
order according to DSM-IV with good inter-rater re-
liability, predictive and discriminant validity and has
been used in a number of epidemiological, genetic and
interventional studies (Taylor et al. 1991; Leung et al.
1996; Chen & Taylor, 2006). The diagnoses of ADHD,
ODD and CD were based on an algorithm that is
appropriate for symptom count, age, time interval
and impairment according to DSM-IV criteria. The
diagnosis of ODD was considered irrespective of
the presence of CD. The interview was administered
by skilled interviewers after advanced training.
Translation and back translation procedures were
used for validation of the non-English versions of
The long form of the CPRS-R, consisting of 80 items,
was used in the present study. The CPRS-R is a re-
liable, accurate and relatively brief measure of par-
ental perceptions of children’s disruptive behaviour.
Adequate psychometric properties have been con-
firmed (Conners, 1997; Conners et al. 1998). The seven
syndrome scales (cognitive problems, oppositional,
hyperactivity-impulsivity, anxious-shy, perfectionism,
and the two subscales of the Conners Global Index
Predictability of ODD dimensions2091
(CGI; restless-impulsive, emotional lability) were
included in the present study.
The SDQ is a brief behavioural screening question-
naire valid for 4- to 16-year-olds. There are versions for
adolescents (starting from 11 years onwards), parents
and teachers. The SDQ consists of five syndrome
hyperactivity, peer problems and prosocial behaviour)
and can be obtained free via the Internet (http://
www.sdqinfo.com). Adequate psychometric proper-
ties of the scales have been documented (Goodman,
To study the diagnostic accuracy in the prediction of
ODD, receiver-operating characteristic (ROC) analyses
were performed separately for each CPRS-R syndrome
scale including the two CGI subscales and the ADHD
index scale. Furthermore, the PSDQ scales were in-
cluded in the ROC analyses. To compare different
scales within the same sample, a critical z ratio was
calculated using a formula correcting for the non-
independence of the scales (Hanley & McNeil, 1983).
Finally, the optimal cut-off score for the best scales
was established: Efficiency (EFF) was calculated by
the sum of true positives and true negatives. In order
to correct EFF for independence of the base rate (P) in
the sample and to take into account the rate of a posi-
tive test result (Q), a quality index of efficiency was
calculated using the following formula (Kraemer,
In addition, the proposed computer algorithm for
the identification of possible and probable CD/ODD
cases was compared with the results based on the cut-
off score analyses.
Before testing the predictability of the three ODD
dimensions, their validity was analysed by the use of
confirmatory factor analysis including all symptoms
accounting for ODD in the PACS. Each symptom was
rated as present or absent according to the corre-
sponding PACS algorithm. Due to the dichotomous
nature of the items, weighted least square confirma-
tory factor analysis (CFA) of the tetrachoric correlation
matrix of the DSM-IV criteria was used to test the
three-factor model and a conventional one-factor
model of ODD (Brown, 2006). Three different rec-
ommended goodness of fit indicators (GFIs; Hair et al.
2006) have been assessed using AMOS 16 software (SPSS
Inc., USA), i.e. the root mean square residual (RMR)
as indicator of the unexplained co-variances of the
model, the root mean square error of approximation
(RMSEA), which includes a parsimony correction and
the comparative fit index (CFI) for evaluating the
hypothesized model compared with a null model.
Acceptance of any model was based on the following
cut-offs: RMR<0.05, RMSEA<0.08 and CFI>0.95
(Hu & Bentler, 1999; Marsh et al. 2004). x2difference
for nested models was used when comparing the
three-factor model with the DSM-IV related one-factor
model of ODD.
Subsequently, backward linear regression analyses
were performed separately for the CPRS-R (with and
without inclusion of the index scales) and the PSDQ in
order to predict the ODD-irritable, ODD-headstrong
and ODD-hurtful dimensions in the entire ADHD
An overview of the means and standard deviations of
the CPRS-R scores and the PSDQ scores is available on
request to the corresponding author. Internal consist-
ency as measured by Cronbach’s a was 0.88 for the
CPRS-R oppositional scale and 0.66 for the PSDQ CP.
The scores of the two scales were strongly correlated
Table 1 shows the results of the ROC analyses for all
CPRS-R syndrome scales and the PSDQ scales for
predicting ODD. The CPRS-R oppositional scale
showedthe best prediction
(AUC)=0.77] compared with all remaining CPRS-R
scales. The PSDQ CP showed the best prediction
(AUC=0.73) in contrast to the remaining SDQ prob-
lem scales. The CPRS-R oppositional scale was su-
perior when compared with the SDQ CP scale
(z=2.248, p=0.014). There were no gender differences
in the prediction of ODD by the CPRS-R OPP (boys
AUC=0.76; girls AUC=0.79; z=x0.63, p=0.263)
and for the PSDQ CP (boys AUC=0.73; girls
AUC=0.75; z=x0.34, p=0.367).
The results of the cut-off analyses are shown in
Table 2. For the CPRS-R OPP, a cut-off score of 15–16
was established based on the quality index of ef-
ficiency (dQ=0.40). In total, 73% of the subjects were
classified correctly by this score. Sensitivity, specificity
and positive and negative predictive power ranged
between 0.58 and 0.80. For the PSDQ CP, the optimal
cut-off score was 5 (dQ=0.34). The corresponding
sensitivity and specificity scores were in a similar
range between 0.55 and 0.79. In addition, the point-
biserial correlation coefficients were 0.44 (p<0.001)
between ODD and CPRS-R OPP and 0.38 (p<0.001)
between ODD and the PSDQ CP.
As can be seen from Table 2, the proposed computer
algorithmfor the SDQ
CD/ODD resulted in equivalent results as those
2092 M. Aebi et al.
observed for the quality index efficiency score of 0.40
(sensitivity=0.73, specificity=0.55). Finally, the corre-
sponding computer algorithm for probable CD/ODD,
which considers the social impact of the symptoms,
showed quite comparable efficiency with a reduced
sensitivity score (0.61) when compared with the
specificity score (0.75).
In the second part of the analyses, the three-factor
structure of the ODD was tested in the entire ADHD
sample by confirmatory factor analysis with weighted
least square statistics for the parameter estimation.
The factor structure and parameter estimates are
shown in Fig. 1. Whereas the comparative fit indicator
value was close to an acceptable level (CFI=0.947), the
other two GFIs suggested that the model had an ex-
cellent fit to the data (RMR=0.006 and RMSEA=
0.041). The three dimensions as latent factors were
correlated moderately to strongly. In particular, the
irritable and the headstrong dimension showed
a strong correlation of 0.89. However, compared
with the three-factor solution, a single factor model of
ODD showed a decreased fit (x2difference for nested
models=60.24, degrees of freedom=3, p<0.001)
and according to the CFI an unacceptable fit to the
present data (RMR=0.010, RMSEA=0.064 and CFI=
(probability level of F for entry=0.001 and for re-
moval=0.01) were performed including the entire
ADHD sample: first, for the CPRS-R problem scales;
second, for all the CPRS problem and index scales;
third, for the PSDQ. The results for the prediction of
ODD-irritable, ODD-headstrong and ODD-hurtful are
shown in Table 3 for the CPRS-R and in Table 4 for the
PSDQ. All tested regression models were highly sig-
nificant. The ODD-irritable dimension was predicted
most successfully by the CPRS-R (R=0.507 only for
problem scales; R=0.524 for all scales) and the PSDQ
(R=0.436) compared with the prediction of the ODD-
headstrong (CPRS-R R=0.449, PSDQ R=0.389) and
ODD-hurtful dimensions (CPRS-R R=0.410, PSDQ
ODD-irritable was positively and most strongly
predicted by the CRPS-R OPP, positively by the
Table 1. Receiver-operating characteristic analysis findings with AUC of the CPRS-R and
the PSDQ problem syndrome scales
CGI: Emotional lability
AUC, Area under the curve; CPRS-R, Conners’ Parent Rating Scale revised; PSDQ,
parent version of the Strength and Difficulties Questionnaire; ADHD, attention
deficit hyperactivity disorder; CGI, Conners Global Index; S.E., standard error.
All scales showed significant deviance of AUC from random prediction
(AUC=0.5) except the PSDQ hyperactivity scale (p=0.07).
Predictability of ODD dimensions2093
CPRS-R anxiety scale (CPRS-R ANX) and negatively
by the CPRS-R hyperactivity scale (CPRS-R HYP).
However, when all CPRS problem and index scales
were included in the analyses, the anxiety scale
was replaced by the CPRS-R emotional-lability index
(CPRS-R EL), whereas the CPRS-R OPP and CPRS-R
HYP remained as significant predictors of the irritable
dimension. In fact, the latter model led to a slightly
increased predictive power compared with the first
model (R=0.524 v. R=0.507), whereas the CPRS-R EL
had stronger impact in the regression model than the
CPRS-R ANX (b=0.22 v. 0.09). Furthermore, similar
results were found for the PSDQ when an emotional
problem scale was included. However, next to the
PSDQ CP and PSDQ EP, the PSDQ prosocial scale
was also identified as a significant negative predictor
of the ODD-irritable dimension. Compared with the
CPRS-R, the predictive power by the PSDQ model was
The ODD-headstrong dimension was strongly pre-
dicted by the CPRS-R OPP and less strongly by the
CPRS-R ANX and by the CPRS-R perfectionism scale,
whereas the ODD-hurtful dimension was only pre-
dicted by the CPRS-R OPP. These results were stable
and independent of inclusion of the additional CPRS-
R index scales. Both, the ODD-headstrong and ODD-
hurtful dimensions were predicted by the PSDQ CP
and PSDQ prosocial scale.
The first part of the present study dealt with testing
the diagnostic accuracy of two common parent rating
scales for predicting ODD in a sample of ADHD
Table 2. Cut-off score analyses of the CPRS-R oppositional scale and the parent version of the PSDQ CP by a dQ
Computer algorithmBase rates SE SPPPPNPPEFFdQ
CPRS-R oppositional scale
PSDQ computer algorithm for CD/ODD
0.68 0.790.550.780.570.71 0.34 0.11 0.35
0.490.61 0.750.830.50 0.660.32 0.03 0.27
CPRS-R, Conners’ Parent Rating Scale revised; PSDQ CP, parent version of the Strength and Difficulties Questionnaire
conduct problem scales; dQ, quality efficiency indicator; SP, specificity; SE, sensitivity; PPP, positive predictive power;
NPP, negative predictive power; EFF, efficiency; LR+, likelihood ratio of a positive test; LRx, likelihood ratio of a negative
test; CD, conduct disorder; ODD, oppositional defiant disorder.
2094M. Aebi et al.
referred youth. Second, after confirming the three-
dimensional ODD structure in the present sample,
the diagnostic accuracy of the CPRS-R and PSDQ in
the prediction of these three dimensions of ODD was
Diagnostic accuracy was tested by ROC, leading to
the calculation of the AUC. This measure of excellence
in the prediction of diagnoses should be interpreted as
follows: poor (50–0.70); moderate to fair (0.70–0.80);
good (0.80–0.90); excellent (0.90–1.00). Accordingly,
the AUC for CPRS-R OPP (0.77) and PSDQ CP (0.73)
indicate an acceptable convergence of these scales
with the diagnosis of ODD. These results are quite
comparable with the diagnostic accuracy of the Child
Behaviour Checklist aggressive behaviour scale in
a pure ADHD sample (Biederman et al. 2008a) and
in a mixed ADHD sample with unreferred controls
(Hudziak et al. 2004).
In comparison with the present findings, higher
AUC based on parental ratings have been reported in
the prediction of various psychiatric disorders other
than ODD, e.g. for obsessive compulsive disorders
(Hudziak et al. 2006) and for ADHD (Chen et al. 1994).
Furthermore, a better diagnostic accuracy has also
been found in the study by Christiansen et al. (2008) in
the prediction of CD in ADHD subjects by the PSDQ
CP and the CPRS-R OPP in a smaller subsample of the
IMAGE study. The differences in diagnostic accuracy
may be partly due to sample and rater effects. The
assessment of CD may be superior because CD
symptoms differentiate more strongly than ODD
symptoms from normal behaviour.
In the present study, a cut-off score of 15/16 on the
CPRS-R oppositional problem scale and a cut-off score
of 4 on the PSDQ CP in the detection of ODD were
found by quality efficiency statistics. For the CPRS-R,
raw scores of 15/16 correspond to T scores of 66–73 in
boys and to 70–75 in girls. On the other hand a cut-off
score of T=65 has been recommended for screening
for ODD (Conners, 1997). Whereas this lower cut-off
score may be accurate in clinical settings, the same
score will be over-inclusive in an ADHD sample and
in particular for girls. However, the PSDQ computer
algorithm for possible ODD/CD seems to work well in
subjects with or without co-morbid ADHD.
Before addressing the prediction of the ODD
dimensions, the three-factor structure of ODD was
tested by using CFA. In contrast with previous studies,
a slightly different item composition was used by
attaching one item to the hurtful rather than the
headstrong dimension. The GFI results of the CFA
convincingly show that a three-factor structure of
ODD is more appropriate than a single general
factor of ODD. However, the latent factor structure
was highly correlated (Fig. 1). (Spearman correlations:
irritable – headstrong r=0.450; irritable – hurtful r=
0.410; headstrong – hurtful r=0.346). Nevertheless,
the present results show that ODD is a heterogeneous
construct including three related but distinct dimen-
sions. This finding may have nosological implications
for the upcoming DSM-V criteria. Furthermore,
the strong correlation of ODD-irritable and ODD-
headstrong may have its origins in the present
ADHD sample. Thus, emotional self-regulation defi-
cits (Barkley, 1997) and delay aversion in ADHD
(Castellanos et al. 2006; Sonuga-Barke et al. 2008) may
strongly affect both ODD-irritable and ODD-head-
Based on these analyses, potential predictors of
these three dimensions were analysed. Overall, the
prediction models based on the CPRS-R were slightly
better than those based on the PSDQ but both instru-
ments were adequate in the prediction of the ODD
dimensions. However, approximately 75–80% of
the variance remained unexplained in all prediction
models. An improved diagnostic assessment of ODD
dimensions seems feasible. All ODD dimensions were
significantly predicted by the CPRS-R OPP and the
PSDQ CP. Thus, both scales are non-specific for the
assessment of ODD dimensions. In addition, the PSDQ
prosocial scale was inversely correlated with opposi-
tionality. Again, the PSDQ prosocial scale predicted all
three dimensions and did not show a distinct profile
for the three ODD dimensions. As expected, the PSDQ
emotional problem scale was a significant predictor of
Often loses temper
Is often touchy or
easily annoyed by others
Is often angry and resentful
Often argues with adults
Often blames others for his
or her mistakes or misbehaviours
Often actively defies or refuses to
comply with adult’s requests or rules
Is often spiteful or
Fig. 1. Confirmatory factor analysis of the eight DSM-IV
oppositional defiant disorder (ODD) criteria. Standardized
regression weights and correlations between the three ODD
factors: ODD-irritable; ODD-headstrong; ODD-hurtful.
Predictability of ODD dimensions2095
ODD-irritable. In contrast with previous studies
(Stringaris & Goodman, 2009a,b) the CPRS-R ANX
significantly predicted ODD-headstrong. However,
this was not true for the CPRS-R EL, which obviously
is more specific in the prediction of ODD-irritability.
Both, the CPRS-R and the PSDQ are suitable
screening instruments for ODD-irritability. The pres-
ent results suggest consideration of both the CPRS-R
OPP and the CPRS-R EL scales for the assessment
of ODD-irritability. However, the CPRS-EL consists
of three items only and the item ‘temper outbursts’ is
also part of the CPRS-R OPP. Diagnostic accuracy
of the CPRS-R EL may be improved by considering
additional items reflecting DSM-IV ODD-irritable cri-
teria. However, there is sufficient evidence that
emotional problem scales need to cover stress-related
Table 3. Prediction of ODD dimensions by the CPRS-R problem and index scales based
on backward linear regression analyses
Models and predictors
CPRS-R oppositional behaviour
CPRS-R oppositional behaviour
CPRS-R emotional labile
CPRS-R oppositional behaviour
CPRS-R oppositional behaviour
ODD, Oppositional defiant disorder, CPRS-R, Conners’ Parent Rating Scale
revised; b=standardized regression coefficent.
aIncluding CPRS-R problem scales only.
bIncluding CPRS-R problem and index scales.
Table 4. Prediction of ODD dimensions by the PSDQ scales based on backward linear
Models and predictors
PSDQ conduct problems
PSDQ emotional problems
PSDQ conduct problems
PSDQ conduct problems
ODD, Oppositional defiant disorder; PSDQ, parent version of the Strength and
Difficulties Questionnaire; b=standardized regression coefficent.
2096 M. Aebi et al.
and emotional symptoms of ODD when evaluating
ODD-irritability. As a consequence, more adequate
ODD-irritable assessment may help to administer ap-
propriate prevention programmes for stress-related
Recently, the role of irritability in ADHD with co-
morbid ODD has been addressed in the context of
SMD (Brotman et al. 2006). Next to abnormal mood,
the diagnostic criteria of SMD include symptoms that
are similar to ADHD (e.g. distractibility, pressured
speech) and a markedly increased reactivity to nega-
tive emotional stimuli (similar to ODD-irritable).
Furthermore, Waschbusch et al. (2002) found increased
anger expression and increased heart rate after mild
provocation in a sample that was co-morbid for
ADHD/ODD but not in ADHD or ODD-only subjects.
Thus, the present results indicate that the construct
of SMD is related to the ODD-irritable dimension in
A previous study has found support for two separ-
ate but correlated constructs of ODD against adults
and ODD against peers (Taylor et al. 2006). Further
studies may test ODD dimensions in combination
with the target of oppositional behaviour. It may be
assumed that the headstrong dimension is associated
with coercive parent–child interactions (Granic &
Patterson, 2006) and may, therefore, be restricted pre-
dominantly to adults, whereas irritable and hurtful
behaviours are more strongly associated with tem-
peramental factors and may be independent of the
Some limitations of the present findings should be
mentioned. First, the present results were based on a
referred ADHD sample and may not generalize to
other community and clinical samples with different
base rates and characteristics of ODD. Second, the
subjects were recruited from several mental health
clinics and the sample may be biased by a referral bias.
Third, the results were based on Caucasian subjects
only and can hardly be generalized to females because
the sample consisted mostly of male subjects. Finally,
the present findings are based on parental ratings
of ODD only. Multi-informant diagnostic criteria
might shed further light on the prediction of these
However, the use of an ADHD-referred sample
does not necessarily restrict conclusions dealing with
ODD assessment. Due to the definition of ADHD as a
precondition for inclusion into the present study, the
validity of the CPRS-R and the PSDQ was confirmed
in a sample at risk for serious antisocial behaviour
(Loeber et al. 1995; Mannuzza et al. 2004; Moffitt et al.
2008). Both the frequent co-morbidity of these two
disorders and the increased risk for later CD and
antisocial personality disorder development require a
more specific treatment programme as compared with
subjects referred for pure ADHD (Biederman et al.
In summary, both the PSDQ, including the rec-
ommended computer algorithm, and the CPRS-R with
the suggested cut-off scores can be recommended
for clinical assessment of ODD. In clinical practice,
lower cut-off scores may be chosen to increase sensi-
tivity and by taking into account the higher costs for
missing true cases. However, additional assessments
may be necessary regarding onset, duration and im-
pact of the symptoms to improve diagnostic efficiency.
For clinicians, the three dimensions of ODD can be
helpful for a better understanding of the disorder.
Accordingly, the CPRS-R EL scale may help to detect
irritability symptoms in ADHD subjects. These pro-
cedures may be important for treatment planning
because next to ADHD therapy additional training
of emotional skills or stress prevention is useful.
However, the diagnostic assessment of ODD dimen-
sions with the present rating scales is still limited and
further studies involving other diagnostic instruments
Declaration of Interest
Tobias Banaschewski, Jan Buitelaar, Joseph Seargent,
Margaret Thompson and Hans-Christoph Steinhausen
have been consultants, speakers or members of advis-
ory boards of various companies including Bioproject,
Janssen-Cilag, Medice, Novartis, Organon, Pfizer,
Servier, Shire and/or UCB. Robert D. Oades, Herbert
Roeyers, Joseph Seargent, Edmund Sonuga-Barke,
Aribert Rothenberger, Hans-Christoph Steinhausen
and Margaret Thompson have received research
grants from Eli Lilly, Janssen-Cilag, National Institute
for Mental Health (USA), and/or UCB.
Angold A, Costello EJ, Erkanli A (1999). Comorbidity.
Journal of Child Psychology and Psychiatry 40, 57–87.
Barkley RA (1997). Behavioral inhibition, sustained
attention, and executive functions: constructing
a unifying theory of ADHD. Psychological Bulletin 121,
Biederman J, Ball SW, Monuteaux MC, Kaiser R,
Faraone SV (2008a). CBCL clinical scales discriminate
ADHD youth with structured-interview derived diagnosis
of oppositional defiant disorder (ODD). Journal of Attention
Disorders 12, 76–82.
Biederman J, Petty CR, Dolan C, Hughes S, Mick E,
Monuteaux MC, Faraone SV (2008b). The long-term
longitudinal course of oppositional defiant disorder
Predictability of ODD dimensions2097
and conduct disorder in ADHD boys: findings from a
controlled 10-year prospective longitudinal follow-up
study. Psychological Medicine 38, 1027–1036.
Braaten EB, Rosen LA (2000). Self-regulation of affect
in attention deficit-hyperactivity disorder (ADHD)
and non-ADHD boys: differences in empathic
responding. Journal of Consulting and Clinical Psychology
Brotman MA, Schmajuk M, Rich BA, Dickstein DP,
Guyer AE, Costello EJ, Egger HL, Angold A, Pine DS,
Leibenluft E (2006). Prevalence, clinical correlates, and
longitudinal course of severe mood dysregulation in
children. Biological Psychiatry 60, 991–997.
Brown T (2006). Confirmatory Factor Analysis for Applied
Research. The Guilford Press: New York.
Burke JD, Loeber R, Lahey BB, Rathouz PJ (2005).
Developmental transitions among affective and behavioral
disorders in adolescent boys. Journal of Child Psychology and
Psychiatry 46, 1200–1210.
Castellanos FX, Sonuga-Barke EJ, Milham MP, Tannock R
(2006). Characterizing cognition in ADHD: beyond
executive dysfunction. Trends in Cognitive Science 10,
Chen W, Taylor E (2006). Parental account of children’s
symptoms (PACS), ADHD phenotypes and its
application to molecular genetic studies. In
Attention-Deficit/Hyperactivity Disorder and the
Hyperkinetic Syndrome: Current Ideas and Ways Forward
(ed. R. D. Oades), pp. 3–20. Nova Science Publishing Inc.:
Chen WJ, Faraone SV, Biederman J, Tsuang MT
(1994). Diagnostic accuracy of the Child Behavior
Checklist scales for attention-deficit hyperactivity
disorder: a receiver-operating characteristic
analysis. Journal of Consulting and Clinical Psychology
Christiansen H, Chen W, Oades RD, Asherson P,
Taylor EA, Lasky-Su J, Zhou K, Banaschewski T,
Buschgens C, Franke B, Gabriels I, Manor I, Marco R,
Muller UC, Mulligan A, Psychogiou L, Rommelse NN,
Uebel H, Buitelaar J, Ebstein RP, Eisenberg J, Gill M,
Miranda A, Mulas F, Roeyers H, Rothenberger A,
Sergeant JA, Sonuga-Barke EJ, Steinhausen HC,
Thompson M, Faraone SV (2008). Co-transmission of
conduct problems with attention-deficit/hyperactivity
disorder: familial evidence for a distinct disorder. Journal
of Neural Transmission 115, 163–175.
Collett BR, Ohan JL, Myers KM (2003). Ten-year review of
rating scales. VI: scales assessing externalizing behaviors.
Journal of the American Academy of Child and Adolescent
Psychiatry 42, 1143–1170.
Conners CK (1997). Conners’ Rating Scales-Revised: Technical
Manual. Multi Health Systems: North Tonawanda,
Conners CK, Sitarenios G, Parker JD, Epstein JN (1998).
The revised Conners’ Parent Rating Scale (CPRS-R): factor
structure, reliability, and criterion validity. Journal of
Abnormal Child Psychology 26, 257–268.
Dick DM, Viken RJ, Kaprio J, Pulkkinen L, Rose RJ (2005).
Understanding the covariation among childhood
externalizing symptoms: genetic and environmental
influences on conduct disorder, attention deficit
hyperactivity disorder, and oppositional defiant
disorder symptoms. Journal of Abnormal Child Psychology
Egger HL, Angold A (2006). Common emotional and
behavioral disorders in preschool children: presentation,
nosology, and epidemiology. Journal of Child Psychology
and Psychiatry 47, 313–337.
Fava M, Hwang I, Rush AJ, Sampson N, Walters EE,
Kessler RC (2009). The importance of irritability as a
symptom of major depressive disorder: results from the
National Comorbidity Survey Replication. Molecular
Psychiatry. Published online: 10 March 2009.
Frick PJ, Lahey BB, Loeber R, Stouthamer-Loeber M,
Christ MA, Hanson K (1992). Familial risk factors to
oppositional defiant disorder and conduct disorder:
parental psychopathology and maternal parenting.
Journal of Consulting and Clinical Psychology 60, 49–55.
Gianarris WJ, Golden CJ, Greene L (2001). The Conners’
Parent Rating Scales: a critical review of the literature.
Clinical Psychology Review 21, 1061–1093.
Goodman R (1997). The Strengths and Difficulties
Questionnaire: a research note. Journal of Child Psychology
and Psychiatry 38, 581–586.
Goodman R (2001). Psychometric properties of the
strengths and difficulties questionnaire. Journal of the
American Academy of Child and Adolescent Psychiatry 40,
Goodman R, Ford T, Richards H, Gatward R,
Meltzer H (2000a). The Development and Well-Being
Assessment: description and initial validation of an
integrated assessment of child and adolescent
psychopathology. Journal of Child Psychology and
Psychiatry 41, 645–655.
Goodman R, Ford T, Simmons H, Gatward R, Meltzer H
(2000b). Using the Strengths and Difficulties Questionnaire
(SDQ) to screen for child psychiatric disorders in a
community sample. British Journal of Psychiatry 177,
Goodman R, Renfrew D, Mullick M (2000c). Predicting type
of psychiatric disorder from Strengths and Difficulties
Questionnaire (SDQ) scores in child mental health clinics
in London and Dhaka. European Child and Adolescent
Psychiatry 9, 129–134.
Granic I, Patterson GR (2006). Toward a comprehensive
model of antisocial development: a dynamic systems
approach. Psychological Review 113, 101–131.
Greene RW, Biederman J, Zerwas S, Monuteaux MC,
Goring JC, Faraone SV (2002). Psychiatric comorbidity,
family dysfunction, and social impairment in referred
youth with oppositional defiant disorder. American Journal
of Psychiatry 159, 1214–1224.
Hair JF, Black WC, Babin BE, Anderson RE, Tatham RL
(2006). Multivariate Data Analysis (6th edn). Prentice-Hall:
Upper Saddle River, NJ.
Hanley JA, McNeil BJ (1983). A method of comparing the
areas under receiver operating characteristic curves
derived from the same cases. Radiology 148, 839–843.
2098M. Aebi et al.
Hu L, Bentler PM (1999). Cutoff criteria for fit indexes in
covariance structure analysis: conventional criteria
versus new alternatives. Structural Equation Modeling 6,
Hudziak JJ, Althoff RR, Stanger C, van Beijsterveldt CE,
Nelson EC, Hanna GL, Boomsma DI, Todd RD (2006).
The Obsessive Compulsive Scale of the Child Behavior
Checklist predicts obsessive-compulsive disorder: a
receiver operating characteristic curve analysis. Journal of
Child Psychology and Psychiatry 47, 160–166.
Hudziak JJ, Copeland W, Stanger C, Wadsworth M (2004).
Screening for DSM-IV externalizing disorders with the
Child Behavior Checklist: a receiver-operating
characteristic analysis. Journal of Child Psychology and
Psychiatry 45, 1299–1307.
Hudziak JJ, Derks EM, Althoff RR, Copeland W, Boomsma
DI (2005). The genetic and environmental contributions to
oppositional defiant behavior: a multi-informant twin
study. Journal of the American Academy of Child and
Adolescent Psychiatry 44, 907–914.
Keenan K, Wakschlag LS (2004). Are oppositional defiant
and conduct disorder symptoms normative behaviors in
preschoolers? A comparison of referred and nonreferred
children. American Journal of Psychiatry 161, 356–358.
Kraemer HC (1992). Evaluating Medical Tests. Objective and
Quantitative Guidelines. Sage Publications, Inc.: Newbury
Lahey BB, Loeber R, Burke J, Rathouz PJ, McBurnett K
(2002). Waxing and waning in concert: dynamic
comorbidity of conduct disorder with other disruptive and
emotional problems over 7 years among clinic-referred
boys. Journal of Abnormal Psychology 111, 556–567.
Lahey BB, Waldman ID, McBurnett K (1999). Annotation:
the development of antisocial behavior: an integrative
causal model. Journal of Child Psychology and Psychiatry 40,
Leung PW, Luk SL, Ho TP, Taylor E, Mak FL,
Bacon-Shone J (1996). The diagnosis and prevalence of
hyperactivity in Chinese schoolboys. British Journal of
Psychiatry 168, 486–496.
Loeber R, Green SM, Keenan K, Lahey BB (1995). Which
boys will fare worse? Early predictors of the onset of
conduct disorder in a six-year longitudinal study. Journal of
the American Academy of Child and Adolescent Psychiatry 34,
Mannuzza S, Klein RG, Abikoff H, Moulton JL, 3rd (2004).
Significance of childhood conduct problems to later
development of conduct disorder among children with
ADHD: a prospective follow-up study. Journal of Abnormal
Child Psychology 32, 565–573.
Marsh HW, Kit-Tai H, Zhonglin W (2004). In search of the
golden rules: comment on hypothesis-testing approaches
to setting cutoff values for fit indexes and dangers in
overgeneralizing Hu and Bentler’s (1999) findings.
Structural Equation Modeling 11, 320–341.
Mathai J, Anderson P, Bourne A (2004). Comparing
psychiatric diagnoses generated by the Strengths and
Difficulties Questionnaire with diagnoses made by
clinicians. Australian and New Zealand Journal of Psychiatry
Maughan B, Rowe R, Messer J, Goodman R, Meltzer H
(2004). Conduct disorder and oppositional defiant disorder
in a national sample: developmental epidemiology. Journal
of Child Psychology and Psychiatry 45, 609–621.
Moffitt TE (1990). Juvenile delinquency and attention deficit
disorder: boys’ developmental trajectories from age 3 to
age 15. Child Development 61, 893–910.
Moffitt TE (1993). Adolescence-limited and life-course-
persistent antisocial behavior: a developmental taxonomy.
Psychological Review 100, 674–701.
Moffitt TE, Arseneault L, Jaffee SR, Kim-Cohen J,
Koenen KC, Odgers CL, Slutske WS, Viding E (2008).
Research review: DSM-V conduct disorder: research
needs for an evidence base. Journal of Child Psychology and
Psychiatry 49, 3–33.
Nock MK, Kazdin AE, Hiripi E, Kessler RC (2007). Lifetime
prevalence, correlates, and persistence of oppositional
defiant disorder: results from the National Comorbidity
Survey Replication. Journal of Child Psychology and
Psychiatry 48, 703–713.
Rowe R, Maughan B, Pickles A, Costello EJ, Angold A
(2002). The relationship between DSM-IV oppositional
defiant disorder and conduct disorder: findings from the
Great Smoky Mountains Study. Journal of Child Psychology
and Psychiatry 43, 365–373.
Simonoff E, Pickles A, Meyer JM, Silberg JL, Maes HH,
Loeber R, Rutter M, Hewitt JK, Eaves LJ (1997). The
Virginia Twin Study of Adolescent Behavioral
Development. Influences of age, sex, and impairment
on rates of disorder. Archives of General Psychiatry 54,
Sonuga-Barke EJ, Sergeant JA, Nigg J, Willcutt E (2008).
Executive dysfunction and delay aversion in attention
deficit hyperactivity disorder: nosologic and diagnostic
implications. Child and Adolescent Psychiatric Clinics of North
America 17, 367–384, ix.
Speltz ML, McClellan J, DeKlyen M, Jones K (1999).
Preschool boys with oppositional defiant disorder:
clinical presentation and diagnostic change. Journal of the
American Academy of Child and Adolescent Psychiatry 38,
Stringaris A, Goodman R (2009a). Longitudinal outcome of
youth oppositionality: irritable, headstrong, and hurtful
behaviors have distinctive predictions. Journal of the
American Academy of Child and Adolescent Psychiatry 48,
Stringaris A, Goodman R (2009b). Three dimensions of
oppositionality in youth. Journal of Child Psychology and
Psychiatry 50, 216–223.
Taylor E, Sandberg S, Thorley G, Giles S (1991). The
Epidemiology of Childhood Hyperactivity. Oxford University
Press: New York.
Taylor E, Schachar R, Thorley G, Wieselberg M (1986).
Conduct disorder and hyperactivity: I. Separation of
hyperactivity and antisocial conduct in British child
psychiatric patients. British Journal of Psychiatry 149,
Taylor TK, Burns GL, Rusby JC, Foster EM (2006).
Oppositional defiant disorder toward adults and
oppositional defiant disorder toward peers: initial
Predictability of ODD dimensions 2099
evidence for two separate constructs. Psychological Download full-text
Assessment 18, 439–443.
van Lier PA, van der Ende J, Koot HM, Verhulst FC
(2007). Which better predicts conduct problems?
The relationship of trajectories of conduct problems with
ODD and ADHD symptoms from childhood into
adolescence. Journal of Child Psychology and Psychiatry 48,
Waschbusch DA, Pelham Jr, WE, Jennings JR, Greiner AR,
Tarter RE, Moss HB (2002). Reactive aggression in boys
with disruptive behavior disorders: behavior, physiology,
and affect. Journal of Abnormal Child Psychology 30, 641–656.
2100M. Aebi et al.