University of Iowa9-2008
Further validation of the IDAS: Evidence
of Convergent, Discriminant, Criterion,
and Incremental Validity
Michael W. O’Hara†
Elizabeth A. McDade-Montez∗∗
∗University of Iowa, email@example.com
†University of Iowa
‡University of Iowa
∗∗University of Iowa
††University of Iowa
‡‡University of Iowa
§University of Iowa
Author Posting. Copyright c ?American Psychological Association, 2008. This article may
not exactly replicate the final version published in the APA journal. It is not the copy of
record. It is posted here by permission of the APA for personal use, not for redistribution.
This paper is posted at Iowa Research Online.
Further Validation of the IDAS 1
In press, Psychological Assessment
Running Head: FURTHER VALIDATION OF THE IDAS
Further validation of the IDAS:
Evidence of Convergent, Discriminant, Criterion, and Incremental Validity
David Watson, Michael W. O’Hara, Michael Chmielewski, Elizabeth A. McDade-Montez,
Erin Koffel, Kristin Naragon, & Scott Stuart
University of Iowa
Further Validation of the IDAS 2
We explicated the validity of the Inventory of Depression and Anxiety Symptoms (IDAS;
Watson et al., 2007) in two samples (306 college students, and 605 psychiatric patients). The
IDAS scales showed strong convergent validity in relation to parallel interview-based scores on
the Clinician Rating version of the IDAS (IDAS-CR); the mean convergent correlations were .51
and .62 in the student and patient samples, respectively. With the exception of Well-Being, the
scales also consistently demonstrated significant discriminant validity. Furthermore, the scales
displayed substantial criterion validity in relation to DSM-IV mood and anxiety disorder
diagnoses in the patient sample. We identified particularly clear and strong associations between
(a) major depression and the IDAS General Depression, Dysphoria and Well-Being scales; (b)
panic disorder and IDAS Panic; (c) posttraumatic stress disorder and IDAS Traumatic Intrusions;
and (d) social phobia and IDAS Social Anxiety. Finally, in logistic regression analyses, the
IDAS scales showed significant incremental validity in predicting several DSM-IV diagnoses
when compared against the Beck Depression Inventory-II (Beck, Steer, & Brown, 1996) and
Beck Anxiety Inventory (Beck & Steer, 1990).
Keywords: major depression, anxiety disorders, convergent validity, discriminant validity,
Further Validation of the IDAS 3
Further validation of the IDAS:
Evidence of Convergent, Discriminant, and Criterion Validity
Traditional self-report measures of depression—such as the Beck Depression Inventory-II
(BDI-II; Beck, Steer, & Brown, 1996) and the Center for Epidemiological Studies Depression
Scale (CES-D; Radloff, 1977)—have been valuable clinical research tools for more than 40 years
(for a recent review, see Joiner, Walker, Pettit, Perez, & Cukrowicz, 2005). At the same time,
however, the accumulating research also has exposed some limitations of these instruments,
thereby establishing the need for alternative measures (Joiner et al., 2005). Watson et al. (2007)
created the Inventory of Depression and Anxiety Symptoms (IDAS) to complement these
traditional measures and to address their limitations.
The IDAS differs from these older instruments in two basic ways. First, these traditional
measures originally were created to yield a single overall index of symptom severity. These total
scores are valuable in many contexts; nevertheless, this focus on global dysfunction ignores the
heterogeneous and multidimensional nature of depressive symptoms, and it hampers the
identification of meaningful subtypes (Ingram & Siegle, 2002; Joiner et al., 2005). In contrast,
the IDAS was specifically designed to contain multiple scales assessing specific symptoms of
depression (e.g., insomnia, suicidality, appetite loss).
Second, extensive evidence has established that these depression measures are very
strongly associated with symptoms of anxiety (e.g., Clark & Watson, 1991; Mineka, Watson, &
Clark, 1998; Watson, 2005). Consequently, the original IDAS item pool contained a broad range
of anxiety-related symptoms. The inclusion of these items facilitated the development of
depression scales with good discriminant validity, and also eventually led to the creation of
complementary anxiety scales (e.g., social anxiety, panic).
Development and Preliminary Validation of the IDAS
Further Validation of the IDAS 4
Development of the IDAS
An initial pool of 180 items was subjected to a series of analyses in a large undergraduate
sample (see Watson et al., 2007, Study 1); this yielded a revised pool of 169 items. Next, this
revised set of items was administered to large samples of college students, psychiatric patients,
and community adults (Watson et al., 2007, Study 2). Data from these three samples were
subjected to separate series of principal factor analyses. Ten specific content factors emerged in
all three samples and were used to create corresponding scales. Five of these scales represent
specific symptoms of major depression: Insomnia, Lassitude (which includes items reflecting
fatigue, lack of energy, and hypersomnia), Suicidality, Appetite Loss, and Appetite Gain. In
addition, three scales—Panic, Social Anxiety, and Traumatic Intrusions—assess specific types of
anxiety symptoms. The other two content scales assess feelings of high energy and positive
affect (Well-Being) and anger/hostility (Ill Temper).
A large, nonspecific factor also emerged in all three solutions; this dimension represents
the core (and largely nonspecific) emotional and cognitive symptoms of depression and anxiety
(see Watson et al., 2007, Table 1). The IDAS Dysphoria scale was created to capture the nature
and scope of this diverse factor. It contains single items assessing depressed mood, anhedonia,
worry, worthlessness, guilt, and hopelessness, as well as two items apiece tapping psychomotor
disturbance (one reflecting retardation, the other agitation) and cognitive problems.
Although the Dysphoria scale is broad and non-specific in its content, it is narrower in
scope than most traditional measures of depression, such as the BDI-II. Watson et al. (2007)
therefore created an expanded measure that more closely resembles these older measures and
that includes a comprehensive range of depression-related content. Thus, the 20-item General
Depression scale includes all 10 Dysphoria items, as well as two items apiece from Suicidality,
Lassitude, Insomnia, Appetite Loss, and Well-Being (these items are reverse-keyed).
Further Validation of the IDAS 5
Reliability evidence. Watson et al. (2007) reported a variety of psychometric data in the
three scale development samples and in five new samples (high school students, college students,
young adults, postpartum women, and psychiatric patients; see Watson et al., Study 3). For
instance, they present evidence indicating that the scales all are internally consistent, with
coefficient alphas typically exceeding .80 (see Watson et al., 2007, Table 3). In addition, they
report one-week retest correlations in a sample of 250 psychiatric patients ranging from .72 (Ill
Temper) to .84 (General Depression).
Validity evidence. Watson et al. (2007) also presented various types of data to establish the
validity of the scales. For instance, the 10 specific scales generally have low to moderate
correlations with one another (typically in the |.20| to |.50| range; see Watson et al., 2007, Table
4), demonstrating that these symptom dimensions can be distinguished from one another.
Moreover, item-level factor analyses established that the specific scales do an excellent job of
capturing the underlying target dimensions.
The Current Research
Overview of the Research
The primary goal of this study is to explicate the convergent, discriminant, criterion and
incremental validity of the IDAS scales. To do so, we present additional data from two of the
Study 3 samples—college students and psychiatric patients—previously reported in Watson et al.
1 In addition, we evaluate the comparative and incremental validity of the IDAS scales
against the BDI-II and the Beck Anxiety Inventory (BAI; Beck & Steer, 1990). In the sections
that follow, we briefly summarize relevant data reported earlier by Watson et al. (2007) and then
describe how the current results augment and extend them.
Convergent and Discriminant Validity
Further Validation of the IDAS 6
Prior evidence. Watson et al. (2007) reported several types of evidence to establish
convergent and discriminant validity. For example, they demonstrated that the IDAS General
Depression and Dysphoria scales were very strongly correlated with the BDI-II (rs ranged from
.81 to .83; see Watson et al., 2007, Table 6), whereas the IDAS Panic scale was highly related to
the BAI (r = .79 and .78 in Studies 2 and 3, respectively).
Moreover, Watson et al. (2007) correlated eight of the IDAS scales with corresponding
symptom composites from the Interview for Mood and Anxiety Symptoms (IMAS; Kotov,
Gamez, & Watson, 2005), a semi-structured instrument that was administered approximately 6
weeks later. The mean convergent correlation was .50, which indicates good convergent validity
(see Watson et al., 2007, Table 7). Next, a classic test of discriminant validity is that each of the
convergent correlations should be higher than any of the other values in its row or column of the
heteromethod block (Campbell & Fiske, 1959). Watson et al. conducted significance tests
comparing the convergent correlations to each of the 14 discriminant correlations in the same
row or column of the heteromethod block. Overall, 106 of these 112 comparisons (94.6%) were
significant, which offers encouraging evidence of discriminant validity.
The current study. The current study extends this evidence by examining convergent and
discriminant relations between self-report and interview-based measures of all 11 non-
overlapping IDAS scales (i.e., General Depression is excluded because it shares items with other
scales). Because no suitable interview-based measure existed (i.e., one that assessed all of the
IDAS dimensions), we created the Clinician Rating version of the IDAS (IDAS-CR), which
consists of a single clinician rating for each of the 11 non-overlapping IDAS scales. IDAS-CR
data are available in both samples. Based on the earlier results reported by Watson et al. (2007),
we expected the IDAS scales to show strong convergent validity and significant discriminant
validity in relation to their IDAS-CR counterparts.
Further Validation of the IDAS 7
Prior evidence. Watson et al. (2007) examined the criterion validity of the IDAS scales by
correlating them with clinicians’ ratings on the widely used Hamilton Rating Scale for
Depression (HRSD; Hamilton, 1960) in a postpartum sample (see their Table 9); for comparison
purposes, they also reported parallel results for the BDI-II and BAI. Three aspects of these data
were noteworthy. First, consistent with previous research (Beck & Steer, 1993; Clark & Watson,
1991), the results demonstrated strong associations between self-rated and clinician-rated
symptoms. Second, all of the IDAS scales were significantly correlated with the HRSD (rs
ranged from |.30| to |.67|), which establishes some degree of criterion validity for each of them.
Third, the two general IDAS scales—General Depression (r = .67) and Dysphoria (r = .64)—had
correlations with the HRSD that were comparable to those of the BDI-II (r = .62) and BAI (r =
.64), thereby demonstrating a similar level of criterion validity.
The current study. Although these HRSD results are encouraging, Watson et al. (2007) did
not report any data relating the IDAS scales to formal DSM-IV diagnoses of major depression
and the anxiety disorders. Accordingly, a basic goal of the current research was to relate the
IDAS scales to DSM-IV diagnoses that were assessed using the Structured Clinical Interview for
DSM-IV (SCID-IV; First, Spitzer, Gibbon, & Williams, 1997) in the patient sample.
Based on the data reported in Watson et al. (2007), we expected that the IDAS General
Depression and Dysphoria scales would display correlates that are very similar to those of the
BDI-II. All of these scales tap variance that is strongly related to the general distress/negative
affectivity dimension that lies at the heart of the unipolar mood and anxiety disorders (see Clark
& Watson, 1991; Mineka et al., 1998; Watson, 2005; Watson et al., 2007). All of these scales,
therefore, should be significantly associated with a broad range of diagnoses, but should display
particularly strong links to distress-based disorders such as major depression and generalized
Further Validation of the IDAS 8
anxiety disorder (GAD; see Watson, 2005). We expected that the other IDAS scales would show
weaker—but significant—associations with depression and GAD.
In addition, the three anxiety scales of the IDAS should show more specific associations
with corresponding DSM-IV anxiety disorders. That is, one would expect to observe specific
associations between (a) IDAS Panic and DSM-IV panic disorder, (b) IDAS Social Anxiety and
social phobia, and (c) IDAS Traumatic Intrusions and PTSD.
Multivariate analyses. We also report multivariate logistic regression analyses in which
the self-report scales are examined together in relation to the DSM-IV diagnoses. These analyses
have two basic goals. First, given that the IDAS scales are significantly correlated and,
therefore, not completely independent, we were interested in identifying which of them showed
unique, incremental power in relation to each DSM-IV disorder. Second, we used these analyses
to examine the incremental validity of the IDAS scales in relation to the BDI-II and BAI.
Participants and Procedure
College student sample. The participants were 307 students enrolled in an introductory
psychology course at the University of Iowa. They participated in partial fulfillment of a course
research exposure requirement. They were assessed in small-group sessions. The sample
consisted of 194 women and 112 men (the sex of one participant was unknown); it included 272
Whites (88.6%), 13 Asian Americans, (4.2%), 4 African Americans (1.3%), and 18 participants
(5.9%) whose racial status was either unknown or from another category. Because of missing
data, we subsequently report results based on data from 303 students.
Psychiatric patient sample. Watson et al. (2007, Study 3) report findings from a sample of
337 psychiatric patients. We present data here on an expanded sample of 605 patients. The
participants (age range = 18-83, M = 41.8 years) were recruited from the Community Mental
Further Validation of the IDAS 9
Health Center of Mideastern Iowa, the Adult Psychiatry Clinic at the University of Iowa Hospital
and Clinics, and the Seashore Psychology Clinic in the Department of Psychology at the
University of Iowa. Patients at these sites were individually approached and asked if they were
interested in participating in a research study. They were assessed in small group sessions and
were paid for their participation. The sample consisted of 386 women and 217 men (the sex of
two participants was unknown); it included 544 Whites (89.9%), 12 African Americans (2.0%),
10 native Americans (1.7%). 9 Asian Americans (1.5%), 12 multiracial participants (2.0%), and
18 respondents (3.0%) whose racial status was either unknown or from another category. We
subsequently report results based on data from 605 (IDAS-CR analyses) and 575 (SCID-IV
IDAS. All participants completed the final 64-item version of the IDAS; they indicated the
extent to which they had experienced each symptom “during the past two weeks, including
today” on a 5-point scale ranging from not at all to extremely. The IDAS contains the 10-item
Dysphoria scale; 8-item measures of Well-Being and Panic; 6-item measures of Suicidality,
Lassitude, and Insomnia; 5-item measures of Social Anxiety and Ill Temper; a 4-item Traumatic
Intrusions scale; and 3-item measures of Appetite Loss and Appetite Gain. Finally, as discussed
earlier, it also includes the 20-item General Depression scale, which contains all 10 Dysphoria
items, as well as two items apiece from the Suicidality, Lassitude, Insomnia, Appetite Loss, and
(reverse-keyed) Well-Being scales. Coefficient alphas for the scales in these samples are
reported in Watson et al. (2007, Table 3).
BDI-II and BAI. Participants in the patient sample also completed the BDI-II (Beck et al.,
1996) and BAI (Beck & Steer, 1990). The BDI-II is one of the most widely used and best
validated self-report measures of depression (see Joiner et al., 2005). The BDI-II consists of 21
Further Validation of the IDAS 10
items, each of which is rated on a 4-point scale ranging from 0 to 3; thus, total scores can range
from 0 to 63. For each item, respondents choose the option that best characterizes how they have
been feeling “during the past two weeks, including today.” The BDI-II had a coefficient alpha of
.93 in the patient sample.
The BAI assesses 21 affective and somatic symptoms of anxiety that are rated on a 4-point
scale (0 = not at all, 3 = severely/I could barely stand it). Respondents indicate to what extent
they have been bothered by each symptom “during the past week, including today.” The BAI
had a coefficient alpha of .94 in the patient sample.
IDAS-CR. Participants in both samples were rated on the IDAS-CR. The interviewers were
trained staff members who had masters’ level training in clinical/counseling psychology or
As noted earlier, the IDAS-CR consists of a series of ratings representing each of the 11
non-overlapping IDAS scales (i.e., General Depression is not assessed). Each rating is made on a
3-point scale (absent, subthreshold, present). In order to make these ratings, the clinicians asked
a standard initial probe question, as well as several standard follow up questions, for each
symptom. In addition, the clinicians were free to ask additional questions to ensure the
individual received a proper rating on the dimension. For the IDAS-CR Dysphoria rating, for
example, the interviewers began with the standard probe question, “"Did you feel sad, depressed,
or down over the past two weeks?" They then asked a number of follow up questions, including
“Have you felt inadequate?”, “Have you had trouble concentrating?” and “Have you found
yourself worrying much of the time?” The interviewers also clarified whether or not reported
symptoms had been present “more days than not” over the past two weeks, and whether they had
(a) been noticed by others or (b) interfered with the patients’ day-to-day activities.
Further Validation of the IDAS 11
To assess interrater reliability, the interviews were audiotaped; 56 (student sample) and 76
(patient sample) of them were scored independently by a second interviewer (due to audiotape
problems, some intraclass correlations in the patient sample are based on an N of only 75).
Intraclass correlations in the student sample ranged from .65 (Ill Temper) to .95 (Insomnia), with
a mean value of .83 and a median value of .87. Intraclass correlations in the patient sample
ranged from .74 (Well-Being) to .99 (Appetite Gain), with a mean value of .90 and a median
value of .89. Intraclass correlations in this range indicate good to excellent interrater reliability
(see Cicchetti, 1994).
SCID-IV. The patients were interviewed using the mood disorders, anxiety disorders, and
substance use disorders modules of the SCID-IV. This last class was included primarily as a
further test of discriminant validity (i.e., we expected that the IDAS scales would relate much
more weakly to substance use disorders than to the mood and anxiety disorders). Because we
had no differential predictions regarding individual substance use disorders, we collapsed them
into a single overall category (i.e., “any substance use disorder”). This left 10 diagnoses for
further consideration. Table 1 presents prevalence data for these disorders in this sample.
To assess interrater reliability, the SCID-IV interviews were audiotaped, and 76 of them
were scored independently by a second interviewer (because of audiotape problems, the actual N
ranged from 74 to 76 across various disorders). The resulting kappas are reported in Table 1.
Nine diagnoses showed good to excellent interrater reliability; the kappas ranged from .70
(GAD) to .95 (major depression), with a median value of .86. Accordingly, these diagnoses were
retained for subsequent analyses. In contrast, however, the interrater reliability for agoraphobia
(κ = .46) was unacceptably low; it therefore was dropped from further consideration.
Further Validation of the IDAS 12
Student sample. We first evaluate the convergent and discriminant validity of the IDAS
scales by examining correlations between them and parallel interview-based ratings on the
IDAS-CR. Table 2 presents these correlations in the college student sample in the form of a
heteromethod block (Campbell & Fiske, 1959). Looking first at convergent validity, all of the
IDAS scales were significantly related to their IDAS-CR counterparts, with coefficients ranging
from .30 (Well-Being) to .62 (Dysphoria). The mean convergent correlation was .51, which
reflects a strong level of convergent validity; these results are particularly impressive given that
the IDAS-CR consists of a series of single ratings. Thus, consistent with previous research (e.g.,
Beck et al., 1988; Clark & Watson, 1991; Watson et al., 2007), our results demonstrate
substantial associations between self-report and interview-based symptom measures.
Having said that, however, we also must acknowledge that the convergent correlation for
Well-Being (r = .30) is substantially lower than that for any other scale (the next lowest
coefficient is r = .42 for Ill Temper). In this regard, some of our interviewers commented that
they found it particularly challenging to rate well-being/positive affectivity (this rating also
showed a relatively low level of interrater reliability in this sample, with an intraclass correlation
of .69), given that they were much more used to evaluating dysfunction and psychopathology
than healthy psychological functioning.
We turn now to discriminant validity. In discussing these data, it is important to emphasize
that valid measures of depression and anxiety should not be completely independent of one
another, but rather should be significantly interrelated (Clark & Watson, 1991; Watson, 2005).
Thus, the key issue here is the pattern of the correlations; that is, whether purported measures of
the same construct (e.g., IDAS Lassitude vs. IDAS-CR Lassitude) correlate more highly than
purported measures of different constructs (e.g., IDAS Lassitude vs. IDAS-CR Insomnia).
As noted earlier, a classic test of discriminant validity is that each of the convergent
Further Validation of the IDAS 13
correlations should be higher than any of the other values in its row or column of the
heteromethod block (Campbell & Fiske, 1959). In part because of its low convergent
correlation, Well-Being clearly failed this test. In contrast, however, the other 10 scales easily
met this criterion. We further quantified these relations by conducting significance tests (using
the Williams modification of the Hotelling test for two correlations involving a common
variable; see Kenny, 1987) comparing these convergent correlations to each of the 20
discriminant coefficients in the same row or column of the block; this yields a total of 200 tests
of discriminant validity across these 10 symptom dimensions. Overall, 199 of these 200
comparisons (99.5%) were significant (p < .05, 1-tailed), which offers strong evidence of
discriminant validity. The single exception was that the convergent coefficient for Ill Temper (r
= .42) did not significantly exceed the .35 correlation between IDAS Ill Temper and IDAS-CR
Dysphoria (z = 1.09, n.s.).
Patient sample. Table 3 presents parallel data from the psychiatric patient sample. It is
interesting to note that the self-report and interview-based measures showed a stronger overall
level of convergence in this sample. The convergent coefficients ranged from .52 (Well-Being)
to .71 (Appetite Loss), with a mean value of .62; this reflects a very strong level of convergent
validity, particularly when one considers the limitations of using single IDAS-CR ratings.
In part because of these stronger convergent correlations, the patient data also yielded
clearer evidence of discriminant validity. As before, we conducted significance tests comparing
each of the convergent correlations to all of the other values in its row or column of the
heteromethod block. The results indicated that 219 of the 220 comparisons (99.5%) were
significant (p < .05, 1-tailed); the sole exception was that the convergent correlation for Well-
Being (r = .62) was not significantly higher than the -.49 correlation between IDAS-CR Well-
being and IDAS Dysphoria (z = 0.86, n.s.).
Further Validation of the IDAS 14 Download full-text
Summary. Overall, these data offer encouraging evidence of convergent and discriminant
validity. All of the convergent correlations were significant and at least moderate in magnitude,
with mean coefficients of .51 and .62 in the student and patient data, respectively. With the
exception of Well-Being, the scales also consistently showed adequate discriminant validity;
indeed, 399 of the 400 individual comparisons involving the other 10 scales were significant (p <
.05, 1-tailed) across the two samples. Thus, our data demonstrate that specific symptom
dimensions—such as lassitude, insomnia, suicidality and panic—can be distinguished from one
another across methods.
Preliminary analyses. To examine the criterion validity of the scales, we first report point
biserial correlations with the SCID-IV diagnoses; in these analyses, diagnoses are scored as
0=absent, 1=present, so that positive correlations indicate that higher scores on a scale are
associated with an increased likelihood of receiving the diagnosis.
Preliminary analyses revealed that three diagnostic categories—specific phobia, dysthymic
disorder, and any substance use disorder—had consistently weak associations with all of the self-
report scales. In fact, the strongest correlations were only .16 (IDAS Panic and the BAI) for
specific phobia, -.07 (IDAS Well-Being) for dysthymic disorder, and .16 (IDAS Ill Temper) for
any substance use disorder. Moreover, mean-level comparisons (to be described in more detail
subsequently) yielded 39 null or small effects, only 3 moderate effects (Panic and the BAI with
specific phobia; Ill Temper with substance use), and no large effects. Consequently, these
disorders will not be considered further.
Correlational analyses. Table 4 presents point biserial correlations with the six remaining
disorders. Several aspects of these data are noteworthy. First, Table 4 displays many moderate
to strong associations; thus, our results again demonstrate the substantial criterion validity of