Content uploaded by Courtney J Bolstad
Author content
All content in this area was uploaded by Courtney J Bolstad on Sep 29, 2021
Content may be subject to copyright.
Dreaming
Factor Analysis and Validation of the Disturbing Dreams and Nightmare
Severity Index
Courtney J. Bolstad, Erica Szkody, and Michael R. Nadorff
Online First Publication, September 27, 2021. http://dx.doi.org/10.1037/drm0000178
CITATION
Bolstad, C. J., Szkody, E., & Nadorff, M. R. (2021, September 27). Factor Analysis and Validation of the Disturbing Dreams
and Nightmare Severity Index. Dreaming. Advance online publication. http://dx.doi.org/10.1037/drm0000178
Factor Analysis and Validation of the Disturbing Dreams
and Nightmare Severity Index
Courtney J. Bolstad
1
, Erica Szkody
1
, and Michael R. Nadorff
1, 2
1
Department of Psychology, Mississippi State University
2
Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine
The Disturbing Dream and Nightmare Severity Index (DDNSI) has been used
widely in research and clinical practice without psychometric evidence supporting its
use. The present study aimed to explore and confirm the factor structure of the
DDNSI as well as to test the measure’s construct validity and invariance between
groups based on sex and race. In all, 2 samples of U.S. undergraduate participants
(N = 614 and N = 606) provided data on nightmares (i.e., DDNSI, Nightmare Effects
Survey, Nightmare Frequency Questionnaire, Nightmare Distress Questionnaire, and
Trauma-Related Nightmare Survey) and related psychopathology (e.g., symptoms of
insomnia, depression, posttraumatic stress disorder, and anxiety). Exploratory and
confirmatory factor analyses found the 5 original items of the DDNSI to load onto a
single latent factor. The DDNSI was found to be a valid measure of nightmare fre-
quency and distress, as it was significantly correlated with the Nightmare Frequency
Questionnaire and the Nightmare Distress Questionnaire, and the DDNSI was able
to differentiate between nightmares and psychopathology. Multiple group analysis
invariance testing found that the latent structure of the DDNSI was comparable
between sex (male vs. female) and race (White vs. Black). Though this research
comes nearly 2 decades after the initial creation and use of the DDNSI, it provides a
foundation for the scientific rigor of previous and future studies on nightmares using
the DDNSI.
Keywords: nightmare assessment, psychometrics, nightmares, Disturbing Dreams and
Nightmare Severity Index, disturbing dreams
Supplemental materials: https://doi.org/10.1037/drm0000178.supp
Courtney J. Bolstad https://orcid.org/0000-0003-2297-2778
No funding was received for conducting this study. The authors also report no potential conflicts of
interest. Data will be made available as needed, and those interested in obtaining the data may contact
the corresponding author via email for access.
Correspondence concerning this article should be addressed to Courtney J. Bolstad, Department
of Psychology, Mississippi State University, P.O. Box 6161, Mississippi State, MS 39762, United States.
Email: cjb905@msstate.edu
1
Dreaming
©2021 American Psychological Association
ISSN: 1053-0797 https://doi.org/10.1037/drm0000178
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Nightmares, or distressing dreams that cause startled awakenings, are clinically
relevant in behavioral sleep medicine and the treatment of psychological disorders.
Nightmares have been found to be related to symptoms of depression (Nadorff et
al., 2011,2013), anxiety (Nadorff, Porter, et al., 2014;Nielsen et al., 2000), posttrau-
matic stress disorder (PTSD; Bryant et al., 2010;Mellman et al., 1995;Ohayon et al.,
2000), dissociative disorders (Agargun et al., 2003), borderline personality disorder
(Claridge et al., 1998;Hartmann et al., 1981), psychosis (Hartmann et al., 1981;
Michels et al., 2014), and suicidality (Nadorff, Anestis et al., 2014;Sjöström et al.,
2009;Tanskanen et al., 2001). Despite these significant clinical implications, night-
mares are often not reported nor assessed (Nadorff et al., 2015). A notable barrier
to adoption has been the lack of well-validated nightmare screening measures.
Nightmare Assessment Measures
When deciding to assess nightmares, there are several measures to choose
from. These include the Nightmare Distress Questionnaire (NDQ; Belicki, 1985,
1992), Nightmare Effects Survey (NES; Krakow et al., 2000), Nightmare Frequency
Questionnaire (NFQ; Krakow, Schrader et al., 2002), Trauma-Related Nightmare
Survey (TRNS; Cranston et al., 2017), Nightmare Proneness Scale (Kelly, 2018),
Nightmare Experience Scale (Kelly & Mathe, 2019), Cognitive Appraisal of Night-
mares (Gieselmann et al., 2020), Nightmare Disorder Index (Dietch et al., 2020),
and Disturbing Dreams and Nightmare Severity Index (DDNSI; Krakow, Schrader,
et al., 2002). Despite all of these measures assessing nightmares, there are significant
differences between them. For instance, some examine just frequency (e.g., NFQ),
whereas others focus on the severity or effects of the nightmares (e.g., NDQ and
NES). By looking at just part of the nightmare experience, these measures may not
properly assess the full extent of a nightmare problem. Another limitation of the lit-
erature is that although several nightmare measures exist, many have little, if any,
evidence in support of their psychometric soundness, and studies that have
attempted to validate these measures, such as the NDQ, do not support the initial
structure of the measures (Stieger & Kuhlmann, 2018). Thus, there is little clarity in
the literature about which measures perform validly and should be used both in
research and clinical practice.
The Disturbing Dreams and Nightmare Severity Index
The DDNSI (Krakow et al., 2001) is a self-report measure consisting of five
items that assess the frequency, quantity, severity, and intensity of disturbing
dreams and nightmares as well as the frequency of nightmare-related awakenings.
Two additional items ask about frequency (i.e., never, yearly, monthly, weekly) and
duration (i.e., number of months or years) of disturbing dreams and nightmares but
are not included in the total score (see Supplement 1 in the online supplemental
materials for full measure). Because it assesses both the frequency and effects of
nightmares, it is an ideal measure to assess the full nightmare experience. The
DDNSI is an adaptation and expansion of the NFQ (Krakow, Schrader, et al.,
2002), and the development of the DDNSI is sometimes miscredited to a study using
the NFQ with survivors of sexual assault who had PTSD (Krakow, Schrader, et al.,
2 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
2002) or to a study on sleep dynamic therapy with Cerro Grande Fire evacuees
(Krakow, Melendrez, et al., 2002). However, the first mention of the DDNSI
appears in an abstract by Krakow and colleagues (2001) as the Nightmare Severity
Index, which only names the measure. To the best of our knowledge, there is no
published report on how the DDNSI was developed or validated prior to its use in
research and clinical settings.
The DDNSI has been widely used in research on nightmares, being cited
approximately 50 times since its initial development in 2002 (see Supplement 2 in
the online supplemental materials). The DDNSI is also often used as a measure that
new nightmare measures are correlated with to determine convergent validity
(Kelly & Mathe, 2019;Kelly & Yu, 2019). The DDNSI has demonstrated adequate
internal consistency in many samples (
a
= .73 in Hom et al., 2018 to
a
= .93 in
Nadorff et al., 2013), yet other psychometric properties of the DDNSI have yet to
be examined directly.
Given the widespread use of the DDNSI in research and practice and lack of
complete, sound psychometric evidence supporting the validity of the measure,
the purpose of the present study was to explore and confirm the factor structure of
the DDNSI and to establish its construct validity against other nightmare meas-
ures and measures of other related, yet different constructs. In addition, because
there are significant racial and sex differences in normal sleep and sleep disturban-
ces between Whites and Blacks (Petrov & Lichstein, 2016;Ruiter et al., 2011)as
well as males and females (Bjorvatn et al., 2010;Mallampalli & Carter, 2014), we
examined whether the latent structure of the DDNSI was comparable across these
groups.
Methods: Study 1
Participants and Procedure
Data were collected in 2016. Participants included 614 undergraduate students
from a large, public land-grant university in the southern United States. Participants
completed the research for credits required by multiple psychology courses. Partici-
pants logged into an online recruitment website (Sona Systems), where they were
shown all of the studies for which they met the inclusion criteria. The study was
advertised as “A Validation of a New Measure of Bad Dreams and Nightmares”
and took approximately 30 min to complete. Participants who were interested read
the informed consent document, indicated consent by clicking to take part in the
study, and answered questions on the SONA website regarding demographics,
insomnia symptoms, trauma, alcohol use, impulsivity, depression symptoms, suici-
dality, and nightmares. This study was deemed exempt by the Mississippi State Uni-
versity Institutional Review Board (13-008). Participants were on average 20 years
of age (range: 18–52; SD = 3.36), primarily female (62.6%), single (65.3%), and Cau-
casian (60.2%). To increase the validity of the responses, we eliminated participants
who responded randomly or in a patterned fashion (e.g., all 0s, 01230123) using
responses to the Center for Epidemiologic Studies Depression Scale, as this mea-
sure has reverse-coded items.
DDNSI VALIDATION 3
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Measure
The DDNSI (Krakow et al., 2001)isafive-item self-report measure that
assesses the frequency, quantity, severity, and intensity of disturbing dreams and
nightmares as well as the frequency of nightmare-related awakenings (see
Supplement 1 in the online supplemental materials for the full measure and two
items that are not included in the total score). The measure is scored by adding
item scores, which provides a total score range from 0 to 37. Generally, scores
above 10 are thought to be indicative of nightmare disorder, although the basis for
this cutoff is unclear. Participants who reported “never”experiencing disturbing
dreams and/or nightmares on the first item of the DDNSI were coded to receive
scores of 0 on all other DDNSI items, and these participants were included in the
data analysis.
Data Analysis
Missing data were handled with imputation through expectation-maximization,
in which the measure was compared with other sleep measures within the data and
then these relationships were maximized to obtain parameter estimates (Bennett,
2001). Exploratory factor analysis (EFA) on the DDNSI items was conducted. A
maximum-likelihood analysis was conducted using SPSS 27.0 with extraction values
set to eigenvalues over 1 (Kaiser, 1960). A criterion level of .40 for factor loadings
indicated an adequate fit. Items below a minimum factor loading threshold of .40
were dropped from the measure. Final EFA results consist of items only above this
threshold.
Results: Study 1
Results of the EFA are shown in Table 1. Kaiser–Meyer–Olkin measure of
sampling adequacy was .85, over the recommended value of .6, and Bartlett’s test of
sphericity was significant,
x
2
(21) = 868.53, p,.001. Eigenvalues indicated a single
solution with factors accounting for 67.38% of the variance. Parallel analysis sug-
gested a model with two factors and one component, and visual review of the scree
plot suggested one factor (Figure 1). An unrotated EFA was conducted and found
Table 1
Factor Loadings
Item
EFA factor load-
ings (Study 1)
CFA factor load-
ings (Study 2)
How often do you have disturbing dreams and/or
nightmares? .743 .871
How many nights in a week do you have
disturbing dreams or nightmares? .715 .848
On average, do your nightmares wake you up? .682 .251
How would you rate the severity of your
disturbing dreams and/or nightmares? .863 .456
How would you rate the intensity of your
disturbing dreams and/or nightmares? .843 .366
Note.
x
2
(2) = 7.996, p= .018. EFA = exploratory factor analysis; CFA = confirmatory factor analysis.
4 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
poor model fit when constrained to two factors (
x
2
= .133, p= .715), and thus a sin-
gle-factor model was retained. Internal consistency of the final scale was examined
using Cronbach’s
a
(.87 in the current study), and no increases in
a
would have been
achieved by deleting any other items.
Methods: Study 2
Participants and Procedure
Data were collected in 2013. Participants included 606 undergraduate students
from the same university as Study 1. The Study 2 procedure was consistent with the
procedure used in Study 1, although the specific measures included in the survey dif-
fered. Participants completed the following measures in addition to a demographic
questionnaire. Participants in Study 2 were on average 20 years of age (range:
18–52; SD = 2.28), primarily female (63.7%), single (65.2%), and Caucasian
(71.8%).
Measures
Convergent Validity Measures
Nightmare Effects Survey. The NES is an 11-item self-report measure used to
assess the adverse impact of nightmares in various life domains (Krakow et al.,
2000). Respondents use a 5-point Likert scale to report the degree their nightmares
affect each life domain (0 = not at all to 4 = a great deal). Answers are then summed
to prove a total score, and higher scores indicate greater adverse consequences of
nightmares on daily functioning. The NES has been found to have good reliability
Figure 1
Scree Plot Depicting the Eigenvalues of the Principal Components and Factor
Analysis, Which Show a One-Factor Solution
DDNSI VALIDATION 5
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
(
a
= .90; Krakow et al., 2000), and a Spanish version of the NES has been validated
(Martínez et al., 2005). The Cronbach’s
a
in the current study was .92.
Nightmare Distress Questionnaire. The NDQ is a 13-item self-report measure
that assesses one’s distress due to their experiencing nightmares as well as one’s
interest in receiving nightmare treatment (Belicki, 1985,1992). Items are rated on
a 5-point Likert scale, and responses are summed to provide a total score.
Research on the Spanish and German versions of the NDQ has found a three-fac-
tor structure to the measure (Böckermann et al., 2014;Martínez et al., 2005). The
NDQ has been found to have adequate reliability (
a
= .83 to .88; Belicki, 1992).
Although studies examining the Spanish and German versions of the NDQ sup-
port the validity of the measure, Stieger and Kuhlmann (2018) found only two of
the three NDQ subscales to be valid measures of nightmare distress. The Cron-
bach’s
a
in the current study was .90.
Nightmare Frequency Questionnaire. The NFQ is a two-item measure assessing
the number of nights respondents experienced nightmares and their actual number
of nightmares weekly, monthly, or yearly in the previous 3 months (Krakow,
Schrader, et al., 2002). Responses were converted to nights and nightmares per
week and then summed for a total score. This scoring method is in line with the
standard scoring of the first two items of the DDNSI, which approximate the two
NFQ items. Because the NFQ served as a basis of the DDNSI, the total score of the
NFQ was used to determine the convergent validity of the DDNSI. The individual
items of the NFQ have been found to have adequate test–retest reliability (Krakow,
Schrader, et al., 2002). The German version of the NFQ has shown adequate validity
(Schmid et al., 2017), though the validity of the English version of the measure
remains unexplored. The Cronbach’s
a
in the current study was .88.
Trauma-Related Nightmare Survey. The TRNS assesses several characteristics of
sleep and chronic nightmares using 16 items with Likert, dichotomous, categorical,
and open-ended response methods (Cranston et al., 2017;Davis & Wright, 2007).
The TRNS individual item responses are used independently, and no total score is
derived from the measure due to the variability of response formats (Cranston et
al., 2017). The TRNS items have been found to have adequate reliability and valid-
ity (Cranston et al., 2017;Davis & Wright, 2007). For the present study, the Items 6,
8, 9, 10, and 11 as listed in Cranston and colleagues’(2017) article were used to
determine the convergent validity of the DDNSI. Item 8 was asked using two differ-
ent questions (i.e., “Approximately how many nightmares have you experienced in
the past month per week?”and “Approximately how many nightmares have you
experienced in the past month per month [if less than one per week]?”). To obtain a
consistent measure of nightmare frequency, responses to the first item were multi-
plied by 4 to obtain a monthly nightmare frequency score for individuals who
endorsed weekly nightmares. The Cronbach’s
a
in the current study was .78.
Divergent Validity Measures
Pittsburgh Revision of the Taylor Manifest Anxiety Scale. The Pittsburgh revision of
the Taylor Manifest Anxiety Scale (TMAS; Bendig, 1956) is a self-report measure
composed of 20 statements regarding one’s personality as an anxious person. This
measure is a revision of the 50-item TMAS (Taylor, 1953). Respondents report
whether each statement is true or false of their personality. Item responses are then
summed to provide a total score ranging from 0 to 20, with higherscores indicating a
6 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
more anxious personality. The Pittsburgh revision of the TMAS has been found to
have adequate reliability (
a
= .76; Bendig, 1956). The Cronbach’s
a
in the current
study was .85.
Specific Loss of Interest and Pleasure Scale. The Specific Loss of Interest and
Pleasure Scale (SLIPS; Winer et al., 2014) is a 23-item self-report measure used to
assess recent (i.e., past 2 weeks) changes in anhedonia. The SLIPS uses a 4-point
Likert scale ranging from 0 (no loss of interest or pleasure)to3(never any interest or
pleasure) regarding 23 specific activities or interactions. To score the SLIPS,
responses of 3 are recoded to 0 to account for trait anhedonia. Item scores are then
summed to provide a total score ranging from 0 to 46, with higher scores indicating
more severe changes in anhedonia in the past 2 weeks. The SLIPS has been found to
be reliable and a valid measure of anhedonia (Winer et al., 2014). The Cronbach’s
a
in the current study was .94.
Center for Epidemiologic Studies Depression Scale. The Center for Epidemiologi-
cal Studies Depression Scale (Radloff, 1977) is a 20-item self-report questionnaire
that measures depressive symptoms over the past week. The Center for Epidemio-
logical Studies Depression Scale has been found to have sufficient reliability and va-
lidity in the general population (Radloff, 1977). Items are rated on a 4-point scale
(0 = less than 1 day to 3 = 5–7 days). Scores are then summed to provide a total score
ranging from 0 to 60, with a score of 16 or more being indicative of clinically signifi-
cant depressive symptoms. The Cronbach’s
a
in the current study was .89.
Insomnia Severity Index. The Insomnia Severity Index (ISI) measures self-
reported insomnia severity over the past 2 weeks using seven items (Bastien et al.,
2001). The ISI uses a 5-point Likert scale with corresponding scores from 0 to 4. To
obtain a total score, item scores are summed for a total score range between 0 and
28. Cutoff scores are as follows: 8 to 14 = subthreshold insomnia, 15 to 21 = moder-
ate insomnia, and 22 or above = severe insomnia. Previous research has found the
ISI to have adequate psychometric properties (Bastien et al., 2001;Savard et al.,
2005). The Cronbach’s
a
in the current study was .84.
Posttraumatic Stress Disorder Checklist–Civilian Version. The Posttraumatic Stress
Disorder Checklist–Civilian Version (PCL-C) measures symptoms of PTSD over
the past month (Weathers et al., 1993). The PCL-C includes 17 self-report items,
which are rated on a 5-point Likert scale, ranging from 1 (not at all)to5(extremely).
A total score is obtained by adding all item responses, for a total score range from
17 to 85. Scores greater than 50 are suggestive of clinically significant posttraumatic
stress symptoms. The PCL-C has adequate psychometric properties (Weathers et
al., 1993). The Cronbach’s
a
in the current study was .93.
Data Analysis
Missing data were handled with expectation-maximization as described in
Study 1. Structural equation modeling was conducted using AMOS 27.0 to conduct
a confirmatory factor analysis (CFA) with items found in Study 1. Model fit was
examined with the standardized root mean square residual (SRMR) in combination
with the comparative fit index (CFI); in combination, SRMR values less than or
equal to .08 and CFI values greater than or equal to .90 indicate good model fit(Hu
& Bentler, 1999). Invariance testing was conducted using multiple group analysis
(MGA) according to DCFI, DRMSEA, and DSRMR cutoffs established by Putnick
DDNSI VALIDATION 7
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
and Bornstein (2016; i.e., DCFI ,.02, DRMSEA ,.02, and DSRMR ,.03). Metric
(i.e., factor loadings are not significantly different across groups, allowing for direct
comparisons among path coefficients and correlations), scalar (i.e., intercepts are
not significantly different across groups, allowing for direct mean comparisons), and
residual invariance (i.e., error terms are not significantly different across groups)
were tested. Lastly, convergent and divergent validity were examined using correla-
tions between scales and items as discussed in the Method section earlier.
Results: Study 2
A CFA was performed loading the five items from the EFA onto the single
latent variable indicated in the EFA. The model demonstrated good model fit
(CFI = .99, root mean square error of approximation (RMSEA) = .07, SRMR =
.01). An MGA between sex (i.e., male vs. female) demonstrated configural, met-
ric, scalar, and residual invariance (see Table 2 for fit indices) and for race
(i.e., White vs. Black; see Table 3 for fit indices). Thus, regardless of sex group
membership, factor loadings are similar across the comparison, correlations coef-
ficients can be directly compared between groups, means of the constructs repre-
sent the same scale and may be directly compared, and error terms were not
significantly different between groups.
Items from the CFA were summed with a Cronbach’s
a
of .78 in the current
study. Pearson correlations demonstrating convergent and divergent validity are
shown in Table 4. The strength of the correlations was examined using Fisher’sRto
Z transformations to determine whether the DDNSI correlated more strongly with
the nightmare measures than measures of other related constructs. The DDNSI had
the strongest correlation with the NDQ (r= .68), which was significantly stronger
than the correlations with symptoms of manifest anxiety, anhedonia, PTSD, insom-
nia, and depression, p,.01. The correlation with the NES (r= .54) was significantly
stronger than the correlation with manifest anxiety, depressive, insomnia symptoms
(p,.01) but did not significantly differ from the correlation with PTSD symptoms
(r= .48, p= 17). Further, even when the trauma item was removed, correlations
Table 2
Invariance Testing for Sex
Model
x
2
(df) CFI
RMSEA
[90% CI] SRMR
Model
compared
D
x
2
(Ddf)DCFI DRMSEA DSRMR Decision
Model 1:
Configural
invariance 9.43 (4) .99 .047 [.000, .088] .007 Accept
Model 2:
Metric
invariance 10.94 (8) .99 .025 [.000, .057] .020 Model 1 1.51 (4) .00 .02 .02 Accept
Model 3:
Scalar
invariance 26.50 (14) .99 .039 [.014, .061] .026 Model 2 15.56 (6) .00 .00 .00 Accept
Model 4:
Residual
invariance 47.41 (22) .99 .044 [.027, .061] .034 Model 3 20.91 (8) .00 .01 .08 Accept
Note. CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR =
standardized root mean square residual.
8 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
between the PCL-C (with no trauma item) and the DDNSI (r=.47) were still not sig-
nificantly different. The correlation with the NFQ (r= .68) was significantly stronger
than the correlations with manifest anxiety, anhedonia, PTSD, insomnia, and
depressive symptoms, p,.01.
Lastly, correlations with Items 6 (r= .36) and 10 (r= .31) of the TRNS were not
significantly larger than the correlation between the DDNSI and symptoms of mani-
fest anxiety, anhedonia, depression, or insomnia and were significantly weaker than
the correlation of the DDNSI with symptoms of PTSD. The DDNSI correlations
with the TRNS Items 9 (r= .51) and 11 (r= .57) were significantly stronger than the
Table 3
Invariance Testing for Race
Model
x
2
(df) CFI
RMSEA
(90% CI) SRMR
Model
compared
D
x
2
(Ddf)DCFI DRMSEA DSRMR Decision
Model 1:
Configural
invariance 9.64 (4) .99 .050 (.005, .091) .009 Accept
Model 2: Metric
invariance 13.89 (8) .99 .036 (.000, .067) .018 Model 1 4.25 (4) .01 .01 .06 Reject
Model 3: Scalar
invariance 24.70 (14) .99 .037 (.009, .060) .014 Model 2 10.81 (6) .00 .00 .01 Accept
Model 4:
Residual
invariance 47.28 (22) .98 .045 (.027, .063) .016 Model 3 22.58 (8) .00 .01 .01 Accept
Note. CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR =
standardized root mean square residual.
Table 4
Correlations for Both Convergent and Divergent Validity
Measure 1 2 3 4567891011121314
Convergent
1. DDNSI —
2. NES .54 —
3. NDQ .68 .71 —
4. NFQ .67 .42 .46 —
5. TRNS 6 .36 .12 .26 .16 —
6. TRNS 8 .46 .33 .36 .51 .23 —
7. TRNS 9 .58 .42 .41 .63 .13 .57 —
8. TRNS 10 .31 .26 .27 .43 .13 .41 .59 —
9. TRNS 11 .58 .43 .53 .31 .32 .36 .39 .16 —
Divergent
10. P-TMAS .39 .40 .43 .29 .20 .18 .22 .11 .30 —
11. SLIPS .31 .41 .33 .27 .11 .20 .30 .21 .23 .62 —
12. CES-D .38 .45 .43 .28 .12 .22 .28 .16 .30 .72 .67 —
13. ISI .40 .43 .45 .31 .20 .25 .33 .15 .37 .49 .40 .52 —
14. PCL-C .48 .56 .57 .37 .30 .33 .30 .21 .46 .69 .65 .69 .58 —
Note. All correlations were significant at p,.001. DDNSI = The Disturbing Dreams and Nightmare
Severity Index measure as found in the confirmatory factor analysis of the current study (M= 6.67,
SD = 4.55, range: 0–26); NES = Nightmare Effects Survey (M= 6.04, SD = 7.06, range: 0–32); NDQ =
Nightmare Distress Questionnaire (M= 12.64, SD = 8.77, range: 0–44); NFQ = Nightmare Frequency
Questionnaire (M= 1.52, SD = 2.09, range: 0–14); TRNS = Trauma-Related Nightmare Survey; P-
TMAS = Pittsburgh revision of the Taylor Manifest Anxiety Scale (M= 7.65, SD = 4.82, range: 0–19);
SLIPS = Specific Loss of Interest and Pleasure Scale (M= 4.78, SD = 7.28, range: 0–40); CES-D =
Center for Epidemiological Studies Depression Scale (M= 12.75, SD = 9.76, range: 0–54); ISI =
Insomnia Severity Index (M= 6.50, SD = 4.79, range: 0–24); PCL-C = Posttraumatic Stress Disorder
Checklist–Civilian Version (M= 29.14, SD = 11.72, range: 0–77).
DDNSI VALIDATION 9
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
correlations with symptoms of manifest anxiety, anhedonia, depression, and insom-
nia (p,.01), though not when compared with PTSD symptoms. Finally, the correla-
tion of Item 8 of the TRNS (r= .46) was only significantly stronger than the
correlation with anhedonia.
Discussion
Summary
Through the use of EFA and CFA, the present study found the five original
items of the DDNSI to load onto a single latent factor. The DDNSI was found to be
valid, as it correlated with several other nightmare measures. Specifically, the
DDNSI appears to be a measure of nightmare frequency and distress, as it was sig-
nificantly correlated with the NFQ and NDQ. The DDNSI does not appear to be a
valid assessment of the broader effects of nightmares, however, as it failed to signifi-
cantly correlate with the NES. In addition, correlations between the DDNSI and
TRNS items were mixed, as some correlations were not significantly different, sig-
nificantly stronger, or significantly weaker than the correlations with the measures
of psychopathology. These complex findings may be due to the lack of variability of
the individual TRNS items, which can lead to weaker correlations, or the fact that
the DDNSI is positioned to ask about broader qualities of nightmares than the TRNS
itemsindividually.HadtheTRNSitemsbeensummedthencorrelatedwiththe
DDNSI, we may have found greater support for the convergent validity of the
DDNSI. However, the TRNS is not typically used in this manner. Further, the DDNSI
was found to significantly correlate with measures assessing symptoms of anxiety,
depression, anhedonia, insomnia, and PTSD. Most of these correlations were weaker
than those between the DDNSI and other nightmare measures, which supports that
the DDNSI has divergent validity. The significant correlations between the DDNSI
and measures assessing symptoms of psychopathology suggest that experiencing night-
mares may be transdiagnostic. Finally, MGA invariance testing found that the latent
structure of the DDNSI was comparable across sex (male vs. female) and race (White
vs. Black).
Implications
To the best of our knowledge, the present study represents the first published
examination of the validity of the DDNSI, though it has been widely used in both
research and clinical practice for nearly 2 decades. Therefore, our findings bolster
the findings of previous studies that used the DDNSI and support further use of the
measure in research studies. Further, our findings support the use of the DDNSI in
clinical practice to assess nightmare frequency and distress, though not to differenti-
ate between these two constructs. The use of a single measure to assess both night-
mare frequency and distress may reduce the burden on both patients and
practitioners, compared with the use of two separate measures (e.g., the NFQ and
NDQ). On the other hand, the confounding of nightmare distress and frequency on
the DDNSI limits the measure’s ability to tease these constructs apart, which has
been found to be important for the relation between nightmares and psychopathol-
ogy (Speed et al., 2018).
10 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Strengths and Limitations
Similar to all research, the present study has both strengths and limitations. An
important strength of the present study is the use of such rigorous tests of divergent
validity. Commonly, divergent validity is ascertained using measures that are vastly
different from the measure of interest. The present study set a high standard for di-
vergent validity by using measures of psychopathology that often correlate with
nightmares.
Limitations of the present study include the use of an undergraduate sample,
though this sample reported a high prevalence of nightmares (see Table 4 for aver-
age scores), which makes the sample ideal for research on nightmares and bolsters
generalization to the clinical population. The present study is also limited by the use
of retrospective self-report measures, as these measures may be biased by partici-
pants’memory. The use of dream logs would have made the present study stronger,
and future research may consider the use of dream logs in similar studies. The use of
retrospective self-report measures is a common practice in clinical work, however,
so our use of these measures may help to generalize our findings to practice. The
present findings are also limited by the lack of psychometric testing of some meas-
ures used in the present study. Specifically, the validity of the NFQ, NES, NDQ,
Pittsburgh revision of the TMAS is questionable (see Methods: Study 2 section).
Certainly, additional psychometric analysis of these measures is warranted, just as
the present study has completed for the DDNSI. Although these measures lack firm
evidence of their validity, some versions of these measures have demonstrated
adequate validity, and our stringent divergent validity testing attenuates the limita-
tion of using these measures. Finally, the present study did not conduct diagnostic
assessments of nightmare disorder in participants, and therefore, we were unable to
determine a DDNSI cutoff score that is indicative of nightmare disorder. Generally,
a score .10 has been used to indicate the existence of nightmare disorder, though
future research is necessary to verify this threshold.
References
Agargun, M. Y., Kara, H., Özer, Ö. A., Selvi, Y., Kiran, U., & Kiran, S. (2003). Nightmares and dissocia-
tive experiences: The key role of childhood traumatic events. Psychiatry and Clinical Neurosciences,
57(2), 139–145. https://doi.org/10.1046/j.1440-1819.2003.01093.x
Bastien, C. H., Vallières, A., & Morin, C. M. (2001). Validation of the Insomnia Severity Index as an
outcome measure for insomnia research. Sleep Medicine,2(4), 297–307. https://doi.org/10.1016/
S1389-9457(00)00065-4
Belicki, K. (1985). The assessment and prevalence of nightmare distress. Sleep Research,14, 145.
Belicki, K. (1992). The relationship of nightmare frequency to nightmare suffering with implications for
treatment and research. Dreaming,2(3), 143–148. https://doi.org/10.1037/h0094355
Bendig, A. W. (1956). The development of a short form of the Manifest Anxiety Scale. Journal of Con-
sulting Psychology,20(5), Article 384. https://doi.org/10.1037/h0045580
Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Jour-
nal of Public Health,25(5), 464–469. https://doi.org/10.1111/j.1467-842X.2001.tb00294.x
Bjorvatn, B., Grønli, J., & Pallesen, S. (2010). Prevalence of different parasomnias in the general popu-
lation. Sleep Medicine,11(10), 1031–1034. https://doi.org/10.1016/j.sleep.2010.07.011
Böckermann, M., Gieselmann, A., & Pietrowsky, R. (2014). What does nightmare distress mean? Facto-
rial structure and psychometric properties of the Nightmare Distress Questionnaire (NDQ).
Dreaming,24(4), 279–289. https://doi.org/10.1037/a0037749
DDNSI VALIDATION 11
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Bryant, R. A., Creamer, M., O'Donnell, M., Silove, D., & McFarlane, A. C. (2010). Sleep disturbance
immediately prior to trauma predicts subsequent psychiatric disorder. Sleep,33(1), 69–74. https://
doi.org/10.1093/sleep/33.1.69
Claridge, G., Davis, C., Bellhouse, M., & Kaptein, S. (1998). Borderline personality, nightmares, and
adverse life events in the risk for eating disorders. Personality and Individual Differences,25(2),
339–351. https://doi.org/10.1016/S0191-8869(98)00052-X
Cranston, C. C., Miller, K. E., Davis, J. L., & Rhudy, J. L. (2017). Preliminary validation of a brief mea-
sure of the frequency and severity of nightmares: The Trauma-Related Nightmare Survey. Journal
of Trauma and Dissociation,18(1), 88–99. https://doi.org/10.1080/15299732.2016.1191578
Davis, J. L., & Wright, D. C. (2007). Randomized clinical trial for treatment of chronic nightmares in
trauma-exposed adults. Journal of Traumatic Stress,20(2), 123–133. https://doi.org/10.1002/jts.20199
Dietch, J. R., Taylor, D. J., Pruiksma, K., Wardle-Pinkston, S., Slavish, D. C., Messman, B., Estevez, R.,
Ruggero, C. J., & Kelly, K. (2020). The Nightmare Disorder Index: Development and initial valida-
tion in a sample of nurses. Sleep,44(5), Article zsaa254. https://doi.org/10.1093/sleep/zsaa254
Gieselmann, A., Elberich, N., Mathes, J., & Pietrowsky, R. (2020). Nightmare distress revisited: Cogni-
tive appraisal of nightmares according to Lazarus’transactional model of stress. Journal of Behav-
ior Therapy and Experimental Psychiatry,68, Article 101517. https://doi.org/10.1016/j.jbtep.2019
.101517
Hartmann, E., Russ, D., van der Kolk, B., Falke, R., & Oldfield, M. (1981). A preliminary study of the
personality of the nightmare sufferer: Relationship to schizophrenia and creativity? The American
Journal of Psychiatry,138(6), 794–797. https://doi.org/10.1176/ajp.138.6.794
Hom, M. A., Stanley, I. H., Spencer-Thomas, S., & Joiner, T. E. (2018). Exposure to suicide and suicide
bereavement among women firefighters: Associated suicidality and psychiatric symptoms. Journal
of Clinical Psychology,74(12), 2219–2237. https://doi.org/10.1002/jclp.22674
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conven-
tional criteria versus new alternatives. Structural Equation Modeling,6(1), 1–55. https://doi.org/10
.1080/10705519909540118
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psycho-
logical Measurement,20(1), 141–151. https://doi.org/10.1177/001316446002000116
Kelly, W. E. (2018). The Nightmare Proneness Scale: A proposed measure for the tendency to experi-
ence nightmares. Sleep and Hypnosis,20(2), 120–127. https://doi.org/10.5350/Sleep.Hypn.2017.19
.0143
Kelly, W. E., & Mathe, J. R. (2019). A brief self-report measure for frequent distressing nightmares:
The Nightmare Experience Scale (NExS). Dreaming,29(2), 180–195. https://doi.org/10.1037/
drm0000106
Kelly, W. E., & Yu, Q. (2019). Convergent, discriminant, and incremental validity of the Nightmare
Proneness Scale. Dreaming,29(1), 100–111. https://doi.org/10.1037/drm0000097
Krakow, B., Hollifield, M., Schrader, R., Koss, M., Tandberg, D., Lauriello, J., McBride, L., Warner,
T. D., Cheng, D., Edmond, T., & Kellner, R. (2000). A controlled study of imagery rehearsal for
chronic nightmares in sexual assault survivors with PTSD: A preliminary report. Journal of Trau-
matic Stress,13(4), 589–609. https://doi.org/10.1023/A:1007854015481
Krakow, B. J., Melendrez, D. C., Johnston, L. G., Clark, J. O., Santana, E. M., Warner, T. D., Hollifield,
M. A., Schrader, R., Sisley, B. N., & Lee, S. A. (2002). Sleep dynamic therapy for Cerro Grande
Fire evacuees with posttraumatic stress symptoms: A preliminary report. The Journal of Clinical
Psychiatry,63(8), 673–684. https://doi.org/10.4088/JCP.v63n0804
Krakow, B., Melendrez, D., Santana, E., Johnson, L., Sisley, B., & Hollifield, M. (2001). Prevalence and
timing of sleep disturbance in Cerro Grande firestorm victims. Sleep,24, A394–A395.
Krakow, B., Schrader, R., Tandberg, D., Hollifield, M., Koss, M. P., Yau, C. L., & Cheng, D. T. (2002).
Nightmare frequency in sexual assault survivors with PTSD. Journal of Anxiety Disorders,16(2),
175–190. https://doi.org/10.1016/S0887-6185(02)00093-2
Mallampalli, M. P., & Carter, C. L. (2014). Exploring sex and gender differences in sleep health: A Soci-
ety for Women’s Health Research Report. Journal of Women’s Health,23(7), 553–562. https://doi
.org/10.1089/jwh.2014.4816
Martínez, M. P., Miro, E., & Arriaza, R. (2005). Evaluation of the distress and effects caused by night-
mares: A study of the psychometric properties of the Nightmare Effects Survey. Sleep and Hypno-
sis,7(1), 29–41.
Mellman, T. A., David, D., Kulick-Bell, R., Hebding, J., & Nolan, B. (1995). Sleep disturbance and its
relationship to psychiatric morbidity after Hurricane Andrew. The American Journal of Psychiatry,
152(11), 1659–1663. https://doi.org/10.1176/ajp.152.11.1659
Michels, F., Schilling, C., Rausch, F., Eifler, S., Zink, M., Meyer-Lindenberg, A., & Schredl, M. (2014).
Nightmare frequency in schizophrenic patients, healthy relatives of schizophrenic patients, patients
at high risk states for psychosis, and healthy controls. International Journal of Dream Research,
7(1), 9–13. https://doi.org/10.11588/ijodr.2014.1.11819
12 BOLSTAD, SZKODY, AND NADORFF
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Nadorff, M. R., Anestis, M. D., Nazem, S., Claire Harris, H., & Samuel Winer, E. (2014). Sleep disorders
and the interpersonal-psychological theory of suicide: Independent pathways to suicidality? Journal
of Affective Disorders,152–154(1), 505–512. https://doi.org/10.1016/j.jad.2013.10.011
Nadorff, M. R., Fiske, A., Sperry, J. A., Petts, R., & Gregg, J. J. (2013). Insomnia symptoms, nightmares,
and suicidal ideation in older adults. The Journals of Gerontology. Series B, Psychological Sciences
and Social Sciences,68(2), 145–152. https://doi.org/10.1093/geronb/gbs061
Nadorff, M. R., Nadorff, D. K., & Germain, A. (2015). Nightmares: Under-reported, undetected, and
therefore untreated. Journal of Clinical Sleep Medicine,11(7), 747–750. https://doi.org/10.5664/jcsm
.4850
Nadorff, M. R., Nazem, S., & Fiske, A. (2011). Insomnia symptoms, nightmares, and suicidal ideation in
a college student sample. Sleep,34(1), 93–98. https://doi.org/10.1093/sleep/34.1.93
Nadorff, M. R., Porter, B., Rhoades, H. M., Greisinger, A. J., Kunik, M. E., & Stanley, M. A. (2014).
Bad dream frequency in older adults with generalized anxiety disorder: Prevalence, correlates, and
effect of cognitive behavioral treatment for anxiety. Behavioral Sleep Medicine,12(1), 28–40.
https://doi.org/10.1080/15402002.2012.755125
Nielsen, T. A., Laberge, L., Paquet, J., Tremblay, R. E., Vitaro, F., & Montplaisir, J. (2000). Develop-
ment of disturbing dreams during adolescence and their relation to anxiety symptoms. Sleep,23(6),
727–736. https://doi.org/10.1093/sleep/23.6.1
Ohayon, M. M., Shapiro, C. M., Ohayon, M. M., & Shapiro, C. M. (2000). Sleep disturbances and psychi-
atric disorders associated with posttraumatic stress disorder in the general population. Comprehen-
sive Psychiatry,41(6), 469–478. https://doi.org/10.1053/comp.2000.16568
Petrov, M. E., & Lichstein, K. L. (2016). Differences in sleep between black and white adults: An update
and future directions. Sleep Medicine,18,74–81. https://doi.org/10.1016/j.sleep.2015.01.011
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The
state of the art and future directions for psychological research. Developmental Review,41,71–90.
https://doi.org/10.1016/j.dr.2016.06.004
Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general popula-
tion. Applied Psychological Measurement,1(3), 385–401. https://doi.org/10.1177/014662167700100306
Ruiter, M. E., Decoster, J., Jacobs, L., & Lichstein, K. L. (2011). Normal sleep in African-Americans
and Caucasian-Americans: A meta-analysis. Sleep Medicine,12(3), 209–214. https://doi.org/10.1016/
j.sleep.2010.12.010
Savard, M. H., Savard, J., Simard, S., & Ivers, H. (2005). Empirical validation of the Insomnia Severity
Index in cancer patients. Psycho-Oncology,14(6), 429–441. https://doi.org/10.1002/pon.860
Schmid, C., Kröner-Borowik, T., Hansen, K., Weßlau, C., & Steil, R. (2017). German versions of the
Nightmare Effects Survey (NES) and the Nightmare Frequency Questionnaire (NFQ): Psychomet-
ric properties in a sample of adult chronic nightmare sufferers. Dreaming,27(3), 232–250. https://
doi.org/10.1037/drm0000052
Sjöström, N., Hetta, J., & Waern, M. (2009). Persistent nightmares are associated with repeat suicide
attempt: A prospective study. Psychiatry Research,170(2–3), 208–211. https://doi.org/10.1016/j
.psychres.2008.09.006
Speed, K. J., Drapeau, C. W., & Nadorff, M. R. (2018). Differentiating single and multiple suicide
attempters: What nightmares can tell us that other predictors cannot. Journal of Clinical Sleep Med-
icine,14(5), 829–834. https://doi.org/10.5664/jcsm.7116
Stieger, S., & Kuhlmann, T. (2018). Validating psychometric questionnaires using experience-sampling
data: The case of nightmare distress. Frontiers in Neuroscience,12, Article 901. https://doi.org/10
.3389/fnins.2018.00901
Tanskanen, A., Tuomilehto, J., Viinamäki, H., Vartiainen, E., Lehtonen, J., & Puska, P. (2001). Night-
mares as predictors of suicide. Sleep,24(7), 844–847. https://doi.org/10.1038/srep44756
Taylor, J. A. (1953). A personality scale of manifest anxiety. Journal of Abnormal and Social Psychol-
ogy,48(2), 285–290. https://doi.org/10.1037/h0056264
Weathers, F. W., Litz, B. T., Herman, D. S., Huska, J. A., & Keane, T. M. (1993). The PTSD checklist
(PCL): reliability, validity, and diagnostic utility. In The Annual Convention of the International
Society for Traumatic Stress Studies. San Antonio, TX.
Winer, E. S., Veilleux, J. C., & Ginger, E. J. (2014). Development and validation of the Specific Loss of
Interest and Pleasure Scale (SLIPS). Journal of Affective Disorders, 152–154(1), 193–201. https://doi
.org/10.1016/j.jad.2013.09.010
DDNSI VALIDATION 13
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
A preview of this full-text is provided by American Psychological Association.
Content available from Dreaming
This content is subject to copyright. Terms and conditions apply.
