Content uploaded by Holly G Prigerson
Author content
All content in this area was uploaded by Holly G Prigerson on Jan 16, 2024
Content may be subject to copyright.
RESEARCH REPORT
World Psychiatry 20:1 - February 2021 1
Validation of the newly proposed DSM criteria for prolonged grief
disorder and the PG-13-Revised (PG-13-R) scale
Holly G. Prigerson1,2, Paul A. Boelen3,4, Jiehui Xu1, Kirsten V. Smith5, Paul K. Maciejewski1,2,6
1Cornell Center for Research on End-of-Live Care, Weill Cornell Medicine, New York, NY, USA; 2Division of Geriatrics and Palliative Medicine, Department of Medicine, Weill
Cornell Medicine, New York, NY, USA; 3Department of Clinical Psychology, Faculty of Social Sciences, Utrecht University, Utrecht, The Netherlands; 4ARQ Research, ARQ Na-
tional Psychotrauma Centre, Diemen, The Netherlands; 5Department of Exper imental Psychology, University of Oxford, Oxford, UK; 6Depar tment of Radiology, Weill Cornell
Medicine, New York, NY, USA
Although the concept of pathological grief dates back at least as far as Freud’s Mourning and Melancholia, there has been opposition to its
recognition as a distinct mental disorder. Resistance has been overcome by evidence demonstrating that distinctive symptoms of prolonged grief
disorder (PGD) – an attachment disturbance featuring yearning for the deceased, loss of meaning and identity disruption – can endure, prove
distressing and disabling, and require targeted treatment. In acknowledgement of this evidence, the DSM Steering Committee has recently voted
to include PGD as a new mental disorder in the DSM. We tested the validity of the newly proposed DSM criteria for PGD and of an adapted
version of our PG-13 scale, the PG-13-Revised (PG-13-R), designed to map onto these criteria, using data from investigations conducted at Yale
University (N=270), Utrecht University (N=163) and Oxford University (N=239). Baseline assessments were performed at 12-24 months post-loss;
follow-up assessments took place 5.3-12.0 months later. Results indicated that the PG-13-R grief symptoms represent a unidimensional construct,
with high degrees of internal consistency (Cronbach’s alpha=0.83, 0.90 and 0.93, for Yale, Utrecht and Oxford, respectively). e DSM PGD
diagnosis was distinct from post-traumatic stress disorder (phi=0.12), major depressive disorder (phi=0.25) and generalized anxiety disorder
(phi=0.26) at baseline. Temporal stability was remarkable for this diagnosis (r=0.86, p<0.001). Kappa agreement between a PG-13-R threshold
symptom summary score of 30 and the DSM symptom criterion for PGD was 0.70-0.89 across the datasets. Both the DSM PGD diagnosis and
the PG-13-R symptom summary score at baseline were signicantly associated (p<0.05) with symptoms and diagnoses of major depressive dis-
order, post-traumatic stress disorder and/or generalized anxiety disorder, suicidal ideation, worse quality of life and functional impairments at
baseline and at follow-up, in the Yale, Utrecht and Oxford datasets. Overall, the newly proposed DSM criteria for PGD and the PG-13-R both
proved reliable and valid measures for the classication of bereaved individuals with maladaptive grief responses.
Key words: Prolonged grief disorder, DSM, PG-13-R, ICD-11, pathological grief, bereavement, post-traumatic stress disorder
(World Psychiatry 2021;20:1-11)
Although the concept of pathological grief dates back at least
as far as Freud’s Mourning and Melancholia1, there has been
public and professional opposition to its recognition as a men-
tal disorder2-5. For example, a 2015 international online survey of
public attitudes revealed that approximately 25% of respondents
did not endorse the position that grief could be a mental disor-
der2. More recently, an online survey on public opinion in Chi-
na found that about 40% of participants did not agree that grief
could be a mental disorder, even under circumstances such as
threat of harm to self or others4. Concerns about “pathologizing”
grief are reported to be rooted in the belief that all grief is normal
and an expected response to the death of a loved one. us, a
diagnosis of pathological grief is considered to be tantamount to
stigmatizing, medicalizing and/or pathologizing love2,4.
Himself wary of pathologizing grief, Freud conceptualized
mourning (grief) as a normal, natural reaction to loss of a loved
one, and even deemed working through grief as necessary to be-
reavement adjustment – the hard, often painful, work a mourner
must do to withdraw emotional attachment to the deceased
person. In fact, Freud considered medical interference in “grief
work” to be “inadvisable if not even harmful”1. By contrast, he
considered melancholia (i.e., depression) the pathological re-
sponse to bereavement, and noted that this condition, not grief,
posed a risk for suicide, and warranted medical attention.
Research over the past quarter century has shown not only
that a small but substantial proportion of grief reactions can
be severe, disabling, and endure beyond normal expectations,
but that they may respond only to specialist treatment. Speci-
cally, studies have documented that certain grief symptoms are
distinct from those of bereavement-related depression6-9, have
idiosyncratic neurobiological10 and clinical11-13 correlates, can
persist unabated for months or even years8,14, prove distressing
and dysfunctional14-16, and may only respond to targeted inter-
vention17,18. us, there exists a substantial and mounting body
of evidence in support of a psychiatric syndrome of maladaptive
grief.
e ICD-11 Workgroup on Stress-Associated Disorders found
the available evidence for prolonged grief disorder (PGD) suf-
ficiently compelling to recommend its recognition as a new
mental disorder19. e DSM-5 had included “persistent com-
plex bereavement disorder” (PCBD) in Section III (i.e., among
“conditions for further study”). In response to the ICD’s inclu-
sion of PGD and the accumulated evidence, the DSM Steering
Committee convened a workshop in June 2019. An invited panel
of researchers presented their data to the Committee, who con-
cluded that these data supported moving the disorder to Section
II (i.e., among recognized mental disorders). A provisional PGD
criteria set was then drafted, and the researchers were tasked
with using the best data available to inform the parameters of the
PGD diagnostic algorithm, and then to evaluate that algorithm’s
reliability and validity. e researchers submitted their reports,
which found the same PGD diagnostic algorithm to be optimal.
e Steering Committee then posted that PGD algorithm online
on the American Psychiatric Association’s website and opened a
2 World Psychiatry 20:1 - Februar y 2021
period for public commentary between April and May 2020. Af-
ter reviewing the research reports and submitted comments, the
DSM Steering Committee voted to approve the inclusion of the
proposed criteria set for PGD in Section II (see Table1).
In order to be sensitive to the concern expressed in the public
commentary about pathologizing normal grieving and diagnos-
ing a grief-related disorder “too soon” after the death, the newly
proposed DSM PGD criteria specify that 12 months must elapse
since the death. is time frame contrasts with the ICD-11 diag-
nostic guidelines for PGD, requiring a period of 6 months20. Un-
like the PCBD criteria, the DSM criteria for PGD acknowledge the
possibility of delayed onset of symptoms at or beyond 12 months
post-loss. Furthermore, the PGD criteria require that three of eight
C criteria (compared to PCBD’s six of 12) be met for a diagnosis,
and focus more on “yearning for” and preoccupation with the de-
ceased person and less on “preoccupation with the circumstances
of the death” – the latter of which could be captured by a post-trau-
matic stress disorder (PTSD) diagnosis. Lastly, the PGD diagnosis
allows for fewer combinations of symptoms to meet the criteria
compared to the PCBD diagnosis. An empirical analysis of the
performance of these newly proposed DSM criteria for PGD has
not yet been published, nor has the psychometric performance
of a scale that maps onto these diagnostic criteria been evaluated.
e PG-13 scale22 was introduced in the process of developing
PGD diagnostic criteria proposed for inclusion in the DSM-5 and
ICD-118. e scale contains 13 items that can be used for the dual
purposes of assessing grief intensity continuously on a dimen-
sional scale and of diagnosing PGD according to the proposed
criteria. Items in the PG-13 are a subset of those in the Inventory
of Complicated Grief - Revised (ICG-R)23, which is a revision of
the Inventory of Complicated Grief (ICG)7. Included items were
those that we found to be informative and unbiased with respect
to gender, relationship to the decedent, and time from loss in
item response theory-based item analysis, and which mapped
onto our criteria for PGD proposed in 20098.
e present paper has two primary objectives. First, it aims to
introduce and validate the PG-13-R, a revised version of the PG-
13 scale that corresponds to the newly proposed DSM criteria for
PGD. Second, it aims to validate these new DSM criteria for PGD.
Data from the US (the Yale Bereavement Study), the Netherlands
(the Utrecht Bereavement Study), and the UK (the Oxford Grief
Study) were used to evaluate the psychometric properties of the
PG-13-R, determine its agreement with the proposed DSM cri-
teria for PGD, assess the PG-13-R and DSM criteria’s predictive
validity, and establish a threshold PG-13-R score to identify syn-
dromal level PGD.
METHODS
Datasets and measures
Data to evaluate the performance of PG-13-R items and the
newly proposed DSM criteria for PGD came from the Yale Be-
reavement Study, the Utrecht Bereavement Study, and the
Oxford Grief Study. In the Yale Bereavement Study, communi-
ty-based bereaved individuals were recruited for a eld trial of
consensus criteria for PGD8. In the Utrecht Bereavement Study,
community-based bereaved subjects were enrolled by mental
health care providers to examine the role of cognitive behavioral
factors in bereavement adjustment24. In the Oxford Grief Study, a
community-based bereaved sample was recruited to investigate
loss-related memories, appraisals and coping strategies relevant
to the development and maintenance of PGD25.
Across datasets, participants with at least one assessment
at 12-24 months post-loss were included. Participants without
Table 1 Proposed DSM criteria for prolonged grief disorder
A. The death, at least 12 months ago, of a person who was close to the bereaved (for children and adolescents, at least 6 months ago).
B. Since the death, there has been a grief response characterized by one or both of the following, to a clinically significant degree, nearly every day or more
often for at least the last month:
1. Intense yearning/longing for the deceased person
2. Preoccupation with thoughts or memories of the deceased person (in children and adolescents, preoccupation may focus on the circumstances of the death)
C. As a result of the death, at least 3 of the following 8 symptoms have been experienced to a clinically significant degree since the death, including nearly
every day or more often for at least the last month:
1. Identity disruption (e.g., feeling as though part of oneself has died)
2. Marked sense of disbelief about the death
3. Avoidance of reminders that the person is dead (in children and adolescents, may be characterized by efforts to avoid reminders)
4. Intense emotional pain (e.g., anger, bitterness, sorrow) related to the death
5. Difficulty with reintegration into life after the death (e.g., problems engaging with friends, pursuing interests, planning for the future)
6. Emotional numbness (i.e., absence or marked reduction in the intensity of emotion, feeling stunned) as a result of the death
7. Feeling that life is meaningless as a result of the death
8. Intense loneliness (i.e., feeling alone or detached from others) as a result of the death
D. The disturbance causes clinically significant distress or impairment in social, occupational, or other important areas of functioning.
E. The duration and severity of the bereavement reaction clearly exceeds expected social, cultural, or religious norms for the individual’s culture and context.
F. The symptoms are not better explained by major depressive disorder, posttraumatic stress disorder, or another mental disorder, or attributable to the physi-
ological effects of a substance (e.g., medication, alcohol) or another medical condition.
©2020 American Psychiatric Association, all rights reserved. Reprinted with permission. These are the criteria as of August 6, 2020. Final approval is pending
World Psychiatry 20:1 - February 2021 3
complete responses to the DSM proposed PGD symptom items
were excluded (total missing rate ~5%), resulting in sample sizes
of N=270 (Yale), N=163 (Utrecht) and N=239 (Oxford), for a to-
tal of N=672. In participants with more than one assessment, the
rst evaluation within the time frame was used for item evalu-
ation and threshold sensitivity analysis. e average time post-
loss for the rst assessment (T1) was 16.7±2.6 months for the
Yale study, 16.3±3.7 months for the Utrecht study, and 14.1±1.7
months for the Oxford study. Participants’ next available assess-
ment (T2) was used for predictive external validity analysis, with
a time lag of 7.4±2.0, 12.0±0 (xed by design), and 5.3±1.3 months
after T1 for Yale (N=48), Utrecht (N=90) and Oxford (N=35) sub-
jects, respectively. All studies were approved by each university’s
institutional review board.
All three studies assessed the 10 symptom items included
in both the newly proposed DSM criteria for PGD and the PG-
13-R (yearning, preoccupation, identity disruption, disbelief,
avoidance, intense emotional pain, diculty with reintegration,
emotional numbness, feeling that life is meaningless, and in-
tense loneliness). ese items (questions Q3 through Q12 in the
PG-13-R) were rated using a 5-point Likert scale ranging from “1
= not at all” to “5 = overwhelmingly”. In the PG-13-R, the symp-
tom items are accompanied by three gatekeeper items exploring
whether the respondent had lost a signicant other (Q1), how
long ago the death occurred (Q2), and impairment associated
with the above symptoms (Q13) (see Figure1).
In the Yale study, the occurrence of post-traumatic stress dis-
order (PTSD), major depressive disorder (MDD), generalized
anxiety disorder (GAD) and panic disorder was further explored
using the Structured Clinical Interview for DSM-IV Axis I Dis-
orders (SCID-I)26; suicidal ideation was assessed using the Yale
Evaluation of Suicidality (YES)27; and quality of life in eight do-
mains (physical functioning, role-physical, bodily pain, general
health, vitality, social functioning, role-emotional, and mental
health) was evaluated using the SF-12 Health Survey28.
In the Utrecht study, PTSD symptoms were assessed using
the PTSD Symptom Scale Self-Report (PSS-SR)29, and depressive
symptoms by the Beck Depression Inventory (BDI-II)30. In the
Oxford study, mental health problems were assessed using the
Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5)31, the
Patient Health Questionnaire (PHQ-9)32 and the Work and Social
Adjustment Scale (WSAS)33.
Statistical analysis
e item performance of the PG-13-R symptom items (Q3-
Q12) was evaluated within each dataset at T1. This included
inspection of item means and variances, percentage of syndro-
mal-level responses (score of 4 or 5), and item-total correlations.
Cronbach’s alpha of the PG-13-R symptom items was used to
evaluate the internal consistency (reliability) of the scale.
A principal components factor analysis was conducted for
each dataset at T1 to evaluate the dimensionality of the grief
symptoms (Q3-Q12) construct. In each dataset, the eigenvalues
obtained from actual PG-13-R symptom item data were com-
pared with those obtained from simulated random data (parallel
analysis)34.
e external validity of the 10-item PG-13-R symptom score
at T1, not including the impairment item (Q13), was assessed by
its associations with other concurrent (T1, concurrent validity)
and follow-up (T2, predictive validity) psychological and behav-
ioral health measures within each dataset, including measures of
depression, post-traumatic stress, suicidality, quality of life and
functional impairments. Associations with dichotomous vari-
ables were estimated as odds ratios (ORs) using logistic regres-
sion; associations with continuous variables were evaluated with
Pearson’s correlation coecients.
The summed PG-13-R score for the symptom items may
range from 10 to 50. e optimal threshold was the symptom
score that had the highest degree of agreement (kappa statistic)
with fulllment of B and C symptom criteria for PGD according
to DSM within each dataset. e median maximum-agreement
threshold score across the datasets was taken to be the overall
optimal PG-13-R symptom threshold score.
e associations between the dichotomous PG-13-R diagnos-
tic threshold score plus the three gatekeeper criteria (i.e., loss,
timing, impairment) as well as the DSM PGD diagnosis with the
mental and behavioral health outcomes at baseline and follow-
up were estimated as ORs using logistic regression.
Phi coecients were used to determine associations between
PGD and other diagnosed mental disorders (e.g., MDD, PTSD,
GAD in the Yale data). Pearson’s correlation coecients were
used to determine stability of PGD and these other mental disor-
ders between T1 and T2.
Statistical analyses for the Yale, Utrecht and Oxford studies
were performed using SAS (version 9.4), R (version 3.6.2), and
SPSS (version 24), respectively.
RESULTS
Table2 summarizes the demographic characteristics of the
three study samples. The Yale sample was older (mean age:
61.8±13.5 years) than the Utrecht (mean age: 56.2±13.3 years)
and Oxford (mean age: 46.9±13.3 years) ones. All three samples
were primarily female (73.0 to 79.1%), and most survived a death
from natural causes (compared to unnatural causes such as sui-
cide or homicide or accidental) (>90%). e Yale and Oxford
samples had higher levels of educational attainment (college or
above >60%) than the Utrecht sample (college or above <40%).
e mean scores for each PG-13-R symptom item at T1 are
presented in Table3. They ranged from 1.3 to 2.9 in the Yale
study; from 1.9 to 3.8 in the Utrecht study; and from 1.8 to 3.2
in the Oxford study. In general, most item means were located
around the center of the range, which is an indication of desira-
ble variability. e avoidance (Q7) and preoccupation (Q4) items
were infrequent in the Yale study, where mean scores in general
were low. Variances for most items across the datasets were rea-
sonably high, conrming the scale’s discriminating ability.
4 World Psychiatry 20:1 - Februar y 2021
Figure 1 PG-13-Revised (by H.G. Prigerson, J. Xu and P.K. Maciejewski)
Across studies, the PG-13-R symptom items cohered well
(Cronbach’s alpha=0.83 for Yale, 0.90 for Utrecht, 0.93 for the
Oxford study) (see Table3). is analysis revealed that the dele-
tion of the avoidance item in each of the three datasets result-
ed in either the same or an improved overall Cronbach’s alpha
(deleted alpha=0.84, 0.91, 0.93 for the Yale, Utrecht and Oxford,
respectively). Similarly, while all the other items had high item-
total correlations (r ≥ 0.50, 0.56 and 0.69 for the three datasets,
respectively), the avoidance item was an exception, with lower
item-total correlations (r=0.25, 0.33, 0.52, respectively).
As illustrated in Figure2, principal components factor analysis
in combination with parallel analysis for each dataset supported
the conclusion that the PG-13-R grief symptoms represent a uni-
dimensional construct. In fact, in each dataset, a single factor
emerged whose eigenvalue was substantially larger than 1 and
greater than would be expected by chance. is primary factor
explained 40.3%, 53.5% and 61.8% of the variance in the Yale,
Utrecht and Oxford studies, respectively.
World Psychiatry 20:1 - February 2021 5
Table 2 Sample characteristics for the three bereavement studies
Yale Study Utrecht Study Oxford Study
(N=270) (N=163) (N=239)
Age, years (mean±SD) 61.8±13.5 56.2±13.3 46.9±13.3
Time from loss, months (mean±SD) 16.7±2.6 16.3±3.7 14.1±1.7
Gender, N (%)
Male 67 (24.9) 44 (27.0) 50 (20.9)
Female 202 (75.1) 119 (73.0) 189 (79.1)
Highest education, N (%)
Primary/secondary school 103 (38.3) 102 (62.6) 55 (23.0)
College/university 166 (61.7) 61 (37.4) 184 (77.0)
Relationship to the deceased, N (%)
Partner/spouse 219 (83.6) 128 (78.5) 71 (29.7)
Other 43 (16.4) 35 (21.5) 168 (70.3)
Cause of death, N (%)
Natural 251 (94.0) 151 (92.6) 218 (91.2)
Unnatural 16 (6.0) 12 (7.4) 21 (8.8)
Results in Table4 support the external validity of the PG-13-R
symptom score, not including the impairment item (Q13). PG-
13-R symptom scores at T1 were signicantly associated with
PTSD, MDD and/or GAD diagnoses or symptomatology and
suicidal ideation, both concurrently (p<0.001) and predictively
(p<0.05), in the Yale, Utrecht and Oxford data. PG-13-R symptom
scores were signicantly associated with poorer role-emotional
and mental health domains of quality of life both concurrently
and predictively in the Yale data (p<0.005), and with work and
social adjustment diculties both concurrently and predictively
in the Oxford data (p<0.001).
PG-13-R symptom threshold scores of 29, 32 and 30 maxi-
mized agreement with meeting DSM symptom criteria for PGD
in the Yale (kappa=0.77), Utrecht (kappa=0.86), and Oxford (kap-
pa=0.89) study data, respectively. Overall, a symptom threshold
score of 30 optimized agreement with meeting DSM symptom
criteria for PGD across the three datasets (kappa ≥0.70 across the
datasets).
Results in Table5 illustrate that using a PG-13-R symptom
threshold score of 30 in combination with the impairment crite-
rion demonstrated excellent external validity. e prevalence of
PGD using the PG-13-R score ≥30 at T1, including impairment,
was 6.3%, 16.6% and 11.3% for the Yale, Utrecht and Oxford
samples, respectively. e PG-13-R threshold-based diagnoses
of PGD at T1 were signicantly (p<0.05) associated with PTSD,
MDD and/or GAD diagnoses or symptomatology and suicidal-
ity in the Yale, Utrecht and Oxford data, concurrently and pre-
dictively (except for suicidality in the Utrecht study, where the
association was signicant only concurrently). PG-13-R thresh-
old-based diagnoses of PGD were signicantly associated with
poorer role-emotional and mental health domains of quality of
life both concurrently and predictively in the Yale data (p<0.05),
and with work and social adjustment diculties both concur-
rently and predictively in the Oxford data (p≤0.001).
Results in Table6 illustrate that the DSM diagnosis of PGD
demonstrated excellent external validity. The prevalence of
PGD using DSM criteria at T1 was 4.4%, 15.3% and 10.9% for the
Yale, Utrecht and Oxford samples, respectively. DSM diagnoses
of PGD at T1 were signicantly (p<0.05) associated with PTSD,
MDD and/or GAD diagnoses or symptomatology concurrently
and predictively in the Yale, Utrecht and Oxford data. Interest-
ingly, in the Yale sample, DSM diagnoses of PGD were signi-
cantly associated with suicidality predictively (at T2) but not
concurrently (at T1). DSM diagnoses of PGD were signicantly
associated with poorer vitality, role-emotional and mental health
domains of quality of life both concurrently and predictively in
the Yale data (p<0.05), and with work and social adjustment dif-
culties both concurrently and predictively in the Oxford data
(p≤0.001).
In the Yale data (T1, N=270), the DSM PGD diagnosis was
found to be distinct from PTSD (phi=0.12), MDD (phi=0.25) and
GAD (phi=0.26). Temporal stability (T1, T2 correlation; N=48)
was greatest for DSM PGD (r=0.86, p<0.001), signicant for MDD
(r=0.31, p=0.030), and not signicant for GAD (r=–0.07, p=0.653).
We could not estimate the temporal stability for PTSD because
no participants with T2 data met criteria for PTSD at T1 (and
only one study participant met criteria for PTSD at T2).
DISCUSSION
Results of analyses of data from independent Yale, Utrecht
and Oxford bereavement studies suggest that both the PG-13-R
and the DSM PGD diagnostic criteria possess desirable perfor-
6 World Psychiatry 20:1 - Februar y 2021
Table 3 PG-13-R item performance and scale internal consistency
Yale Study (N=270)
Alpha=0.83
Utrecht Study (N=163)
Alpha=0.90
Oxford Study (N=239)
Alpha=0.93
PGD-13-R symptom item Rate
Score
(mean±SD)
Deleted
alpha
Corrected
item-total
correlation Rate
Score
(mean±SD)
Deleted
alpha
Corrected
item-total
correlation Rate
Score
(mean±SD)
Deleted
alpha
Corrected
item-total
correlation
Q3 Yearning 35.2% 2.9±1.3 0.81 0.59 68.1% 3.8±0.9 0.89 0.65 34.7% 3.1±1.2 0.92 0.75
Q4 Preoccupation 2.6% 1.3±0.8 0.82 0.53 26.4% 2.9±0.9 0.88 0.72 36.4% 3.2±1.2 0.92 0.74
Q5 Identity disruption 22.6% 2.2±1.4 0.81 0.58 42.3% 3.1±1.3 0.88 0.71 33.9% 2.7±1.4 0.92 0.76
Q6 Disbelief 6.3% 1.5±1.0 0.82 0.50 27.0% 2.9±1.2 0.89 0.56 33.9% 2.8±1.3 0.92 0.69
Q7 Avoidance 2.6% 1.3±0.7 0.84 0.25 5.5% 1.9±1.0 0.91 0.33 11.7% 1.8±1.2 0.93 0.52
Q8 Intense emotional pain 10.7% 2.1±1.0 0.82 0.51 49.7% 3.4±1.0 0.88 0.75 26.8% 3.0±1.1 0.92 0.74
Q9 Difficulty with reintegration 9.3% 1.8±1.1 0.82 0.52 26.4% 2.7±1.2 0.89 0.67 17.6% 2.1±1.3 0.92 0.76
Q10 Emotional numbness 7.4% 1.5±1.0 0.82 0.50 16.6% 2.4±1.1 0.88 0.70 21.8% 2.4±1.2 0.92 0.76
Q11 Life is meaningless 16.3% 2.0±1.2 0.81 0.61 39.3% 3.1±1.1 0.88 0.76 18.8% 2.1±1.3 0.92 0.80
Q12 Intense loneliness 33.3% 2.8±1.3 0.81 0.61 51.5% 3.4±1.1 0.89 0.65 26.4% 2.5±1.3 0.92 0.76
World Psychiatry 20:1 - February 2021 7
mance characteristics. e symptoms were uniformly higher
in the Utrecht sample, which is unsurprising given that this
sample was recruited via mental health professionals. Across
all three datasets, the preoccupation item was infrequently re-
ported at syndromal levels. is was most noticeable in the Yale
data, where syndromal level preoccupation was found in <3%
of the sample. Such low prevalence is an undesirable property
for a “gatekeeper” item, which suggests that it might be prefer-
Figure 2 Eigenvalues from principal components factor analysis for PG-13-R symptom items and comparison to eigenvalues from parallel
analysis (median of 100 replications of simulated random data) for the three studies
Table 4 Concurrent and predictive validity of PG-13-R symptom score (excluding impairment)
PG-13-R symptom score (sum of 10 items) at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N % OR p N % OR p
Post-traumatic stress disorder (PTSD) 270 1.5 1.23 0.007 48 2.1 n.e.
Major depressive disorder (MDD) 270 5.9 1.16 <0.001 48 4.2 n.e.
Generalized anxiety disorder (GAD) 270 3.3 1.24 <0.001 48 6.3 1.26 0.032
PTSD, MDD or GAD 270 8.1 1.18 <0.001 48 8.3 1.57 0.033
Yale Evaluation of Suicidality (YES):
atleast one positive response
269 17.5 1.18 <0.001 48 18.8 1.13 0.032
Yale Study N mean±SD r p N mean±SD r p
SF-12: Physical functioning 269 5.1±1.3 –0.10 0.109 48 4.7±1.7 0.10 0.518
SF-12: Role-physical 270 3.5±0.8 –0.12 0.048 48 3.3±0.9 –0.05 0.715
SF-12: Bodily pain 270 4.5±0.9 –0.24 <0.001 48 4.4±1.0 –0.10 0.513
SF-12: General health 270 3.6±1.0 –0.25 <0.001 48 3.6±1.1 –0.21 0.162
SF-12: Vitality 270 2.6±1.3 –0.42 <0.001 48 2.4±1.3 –0.23 0.110
SF-12: Social functioning 270 4.3±1.0 –0.41 <0.001 48 4.4±1.0 –0.13 0.373
SF-12: Role-emotional 270 3.6±0.7 –0.45 <0.001 48 3.6±0.7 –0.42 0.003
SF-12: Mental health 270 7.4±2.0 –0.60 <0.001 48 7.3±2.1 –0.61 <0.001
Utrecht Study N mean±SD r p N mean±SD r p
PSS-SR 158 31.4±8.4 0.77 <0.001 85 26.3±6.5 0.68 <0.001
BDI-II 153 34.6±8.8 0.75 <0.001 82 31.1±7.8 0.53 <0.001
BDI-II: Suicidality (item 9) 161 1.2±0.4 0.34 <0.001 90 1.2±0.4 0.29 0.005
Oxford Study N mean±SD r p N mean±SD r p
PCL-5 239 23.5±17.8 0.78 <0.001 35 20.7±16.8 0.53 0.001
PHQ-9 239 8.9±7.1 0.68 <0.001 35 7.8±7.1 0.60 <0.001
PHQ-9: Suicidality (item 9) 239 0.4±0.8 0.52 <0.001 35 0.3±0.8 0.55 0.001
WSAS 237 12.8±9.4 0.77 <0.001 35 11.5±9.7 0.64 <0.001
OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated
8 World Psychiatry 20:1 - Februar y 2021
Table 5 Concurrent and predictive validity of prolonged grief disorder (PGD) diagnosis using PG-13-R symptom threshold score of 30 and
including impairment
PG-13-R threshold score-based diagnosis of PGD at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N OR p N OR p
Post-traumatic stress disorder (PTSD) 270 54.00 0.001 48 n.e.
Major depressive disorder (MDD) 270 18.98 <0.001 48 n.e.
Generalized anxiety disorder (GAD) 270 15.26 <0.001 48 28.00 0.014
PTSD, MDD or GAD 270 20.77 <0.001 48 63.00 0.002
Yale Evaluation of Suicidality (YES):
atleast one positive response
269 3.71 0.012 48 9.25 0.028
Yale Study N r p N r p
SF-12: Physical functioning 269 –0.05 0.433 48 0.10 0.509
SF-12: Role-physical 270 –0.08 0.216 48 0.03 0.857
SF-12: Bodily pain 270 –0.24 <0.001 48 0.00 0.992
SF-12: General health 270 –0.17 0.006 48 –0.14 0.351
SF-12: Vitality 270 –0.29 <0.001 48 –0.20 0.183
SF-12: Social functioning 270 –0.34 <0.001 48 0.00 0.992
SF-12: Role-emotional 270 –0.38 <0.001 48 –0.31 0.034
SF-12: Mental health 270 –0.30 <0.001 48 –0.38 0.007
Utrecht Study N r p N r p
PSS-SR 158 0.48 <0.001 85 0.39 <0.001
BDI-II 153 0.47 <0.001 82 0.39 <0.001
BDI-II: Suicidality (item 9) 161 0.18 0.024 90 0.19 0.070
Oxford Study N r p N r p
PCL-5 239 0.51 <0.001 35 0.58 <0.001
PHQ-9 239 0.45 <0.001 35 0.59 <0.001
PHQ-9: Suicidality (item 9) 239 0.54 <0.001 35 0.79 <0.001
WSAS 237 0.49 <0.001 35 0.52 0.001
OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated
able to have only “yearning” in the B criterion for PGD in the
DSM.
The weakest performing item across all the datasets was
“avoidance of reminders that the deceased is dead”. Item-total
correlations for this item were the lowest of all items examined,
and Cronbach’s alpha improved in the Yale and Utrecht datasets
when the avoidance item was removed. It may be the case that
avoidance is more a function of fear, with roots in psychological
trauma, than a function of grief, with roots in an attachment dis-
turbance. Alternately, there may be a need to revise the item to
focus on what aspect of the loss is avoided (e.g., avoidance of re-
minders of the death as an event may be more a traumatic stress
response, while avoidance of reminders that the deceased is tru-
ly gone may be the most relevant to disturbed grief). Future stud-
ies are needed to conrm whether the avoidance item should be
retained, revised or discarded.
In accordance with the high internal consistency of the PG-
13-R symptom items, factor analyses revealed that the scale is
unidimensional. These results are consistent with those re-
ported for the Inventory of Complicated Grief7 and its Dutch
version35, and for the original PG-138 and its Swedish36, Chi-
nese37, Portuguese38 and many other translated versionse.g.,39.
ough some studies have found multiple factors in this set of
grief symptoms40, these exceptions occurred only in highly co-
morbid treatment-seeking and treatment-receiving samples and
a military family study, not in community-based samples. e
preponderance of evidence supports the unidimensional nature
of PGD symptomatology as found in the three studies examined
here.
Because the Yale data alone included structured clinical in-
terviews that yielded diagnoses of mental disorders, only these
data could be used to assess PGD’s overlap with other disorders
and to compare diagnostic stability over time. e results dem-
onstrated minimal overlap between PGD and competing diag-
World Psychiatry 20:1 - February 2021 9
noses (i.e., PTSD, MDD and GAD) (phi=0.12-0.26), suggesting its
distinctness from mental disorders already included in Section
II of the DSM. In addition, the PGD diagnosis proved remark-
ably stable between the T1 and T2 assessments approximately
7.4 months apart (r=0.86, p<0.001) and much more stable than
MDD (r=0.31, p=0.030) or GAD (r=–0.07, p=0.653). ese results
suggest that PGD lls a diagnostic gap left open by other mental
disorders secondary to bereavement. Furthermore, they show
that PGD is likely not to remit with the passage of time and to
require specialized treatment.
With respect to concurrent and predictive validity, we rst
sought to determine if the intensity of PGD symptoms alone
(excluding impairment, the DSM criterion D) would predict
distress and dysfunction. e PG-13-R symptom score proved
to be highly predictive of both concomitant and future distress
and dysfunction, indicating that the severity of these symptoms
themselves is pathological even without “stacking the deck” by
requiring the fulllment of an impairment criterion.
Next, we sought to determine the threshold score of these
symptoms that optimized agreement with meeting the B and C
symptom criteria for PGD in the DSM. We found that the PG-
13-R symptom score of 30 was the optimal threshold score across
the three datasets. Finally, we sought to evaluate and compare
the concurrent and predictive validity of diagnoses for PGD us-
ing the PG-13-R threshold diagnostic score, and, separately, us-
ing the DSM criteria B and C, each in combination with meeting
the impairment criterion. Results indicated that both performed
extremely well in predicting substantial current and future mala-
daptive behaviors and outcomes.
A strength of this study was the use of three independent
community-based bereavement cohort samples. A possible
weakness was the fact that the wording for the PG-13-R ques-
tions was slightly different in the three studies. The Utrecht
sample was uniformly more distressed than the Yale and Oxford
samples, which is understandable given that Utrecht partici-
pants were recruited via mental health care providers, who are
Table 6 Concurrent and predictive validity of newly proposed DSM diagnostic criteria for prolonged grief disorder (PGD)
DSM diagnosis for PGD at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N OR p N OR p
Post-traumatic stress disorder (PTSD) 270 7.73 0.087 48 n.e.
Major depressive disorder (MDD) 270 10.25 0.001 48 n.e.
Generalized anxiety disorder (GAD) 270 14.00 0.001 48 43.00 0.008
PTSD, MDD or GAD 270 10.13 <0.001 48 129.00 0.002
Yale Evaluation of Suicidality (YES):
atleast one positive response
269 1.61 0.486 48 19.00 0.017
Yale Study N r p N r p
SF-12: Physical functioning 269 0.00 0.965 48 0.05 0.737
SF-12: Role-physical 270 –0.02 0.805 48 0.15 0.316
SF-12: Bodily pain 270 –0.14 0.024 48 0.03 0.828
SF-12: General health 270 –0.09 0.134 48 –0.25 0.086
SF-12: Vitality 270 –0.20 0.001 48 –0.31 0.032
SF-12: Social functioning 270 –0.32 <0.001 48 –0.05 0.760
SF-12: Role-emotional 270 –0.28 <0.001 48 –0.38 0.008
SF-12: Mental health 270 –0.19 0.002 48 –0.45 0.001
Utrecht Study N r p N r p
PSS-SR 158 0.48 <0.001 85 0.39 <0.001
BDI-II 153 0.47 <0.001 82 0.39 <0.001
BDI-II: Suicidality (item (9) 161 0.20 0.011 90 0.19 0.070
Oxford Study N r p N r p
PCL-5 239 0.48 <0.001 35 0.58 <0.001
PHQ-9 239 0.43 <0.001 35 0.59 <0.001
PHQ-9: Suicidality (item 9) 239 0.54 <0.001 35 0.79 <0.001
WSAS 237 0.48 <0.001 35 0.52 0.001
OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated
10 World Psychiatry 20:1 - February 2021
more likely to encounter distressed bereaved individuals. e
Yale and Utrecht samples were predominantly comprised of
widowed persons, which was not the case for the Oxford sam-
ple (~80% to ~30%, respectively). With respect to ethnicity, all
three samples nearly entirely consisted of people of Caucasian
ethnicity.
In conclusion, three independent community-based sam-
ples showed that the PG-13-R is a reliable tool for assessing grief
symptoms on a dimensional scale. A PG-13-R symptom score
of 30 or greater identies syndromal-level PGD symptomatol-
ogy. e dimensional PG-13-R symptom score, the diagnosis
of PGD using the PG-13-R threshold symptom score of 30 plus
the impairment criterion, and the diagnosis of PGD using newly
proposed DSM criteria all predict enduring distress and dysfunc-
tion. us, the PG-13-R and the newly proposed DSM criteria for
PGD appear to be reliable and valid measures for the classica-
tion of bereaved individuals with maladaptive grief responses.
Future research is needed to conrm their psychometric perfor-
mance in more ethnically diverse samples.
ACKNOWLEDGEMENTS
is work was supported by grants from the US National Cancer Institute (nos.
CA197730 and CA218313), the US National Institute of Minority Health and
Health Disparities (no. MD007652), the US National Institute of Nursing Re-
search (no. NR018693), the US National Institute on Aging (no. AG049666), the
US National Institute of Mental Health (no. MH121886), the US National Center
for Advancing Translational Science (no. TR002384), the Wellcome Trust (no.
200796), the National Institute for Health Research (NIHR) Biomedical Re-
search Centre, based at Oxford University Hospitals National Health System
(NHS) Trust (no. NIHR-INF-0085), and the Oxford Health NIHR Biomedical
Research Centre. e views expressed are those of the authors and not neces-
sarily those of the supporting institutions. e authors are grateful to A. Ehlers
for her support and supervision of the Oxford Grief Study. A smart pdf version
of the PG-13-R is available at https://endoife.weill.cornell.edu/sites/default/
les/le_uploads/pg-13-r.pdf.
REFERENCES
1. Freud S. Mourning and melancholia. In: Rickman J (ed). A general selection
from the works of Sigmund Freud. New York: Doubleday, 1957:124-40.
2. Breen LJ, Penman EL, Prigerson HG et al. Can grief be a mental disorder?: An
exploration of public opinion. J Nerv Ment Dis 2015;203:569-73.
3. Bambauer KZ, Prigerson HG. e Stigma Receptivity Scale and its asso-
ciation with mental health service use among bereaved older adults. J Nerv
Ment Dis 2006;194:139-41.
4. Tang S, Chow AYM, Breen LJ et al. Can grief be a mental disorder? An on-
linesurvey on public opinion in mainland China. Death Stud 2020;44:
152-9.
5. Johnson JG, First MB, Block S et al. Stigmatization and receptivity to mental
health services among recently bereaved adults. Death Stud 2009;33:691-
711.
6. Prigerson HG, Frank E, Kasl SV et al. Complicated grief and bereavement-
related depression as distinct disorders: preliminary empirical validation in
elderly bereaved spouses. Am J Psychiatry. 1995;152:22-30.
7. Prigerson HG, Maciejewski PK, Reynolds CF 3rd et al. Inventory of Compli-
cated Grief: a scale to measure maladaptive symptoms of loss. Psychiatry Res
1995;59:65-79.
8. Prigerson HG, Horowitz MJ, Jacobs SC et al. Prolonged grief disorder: psy-
chometric validation of criteria proposed for DSM-V and ICD-11 PLoS Med
2009;6:e1000121.
9. Boelen PA, van den Bout J. Complicated grief, depression, and anxiety as dis-
tinct postloss syndromes: a conrmatory factor analysis study. Am J Psychia-
try 2005;162:2175-7.
10. Kakarala SE, Roberts KE, Rogers M et al. e neurobiological reward system
in prolonged grief disorder (PGD): a systematic review. Psychiatry Res Neu-
roimaging 2020;303:111135.
11. Boelen PA, Reijntjes A, Djelantik AAAMJ et al. Prolonged grief and depres-
sion after unnatural loss: latent class analyses and cognitive correlates. Psy-
chiatry Res 2016;240:358-63.
12. Johnson JG, Zhang B, Greer JA et al. Parental control, partner dependency,
and complicated grief among widowed adults in the community. J Nerv
Ment Dis 2007;195:26-30.
13. Wright AA, Keating NL, Balboni TA et al. Place of death: correlations with
quality of life of patients with cancer and predictors of bereaved caregivers’
mental health. J Clin Oncol 2010;28:4457-64.
14. Maciejewski PK, Zhang B, Block SD et al. An empirical examination of the
stage theory of grief. JAMA 2007;297:716-23.
15. Prigerson HG, Bierhals AJ, Kasl SV et al. Traumatic grief as a risk factor for
mental and physical morbidity. Am J Psychiatry 1997;154:616-23.
16. Maciejewski PK, Maercker A, Boelen PA et al. “Prolonged grief disorder”
and “persistent complex bereavement disorder”, but not “complicated grief”,
are one and the same diagnostic entity: an analysis of data from the Yale Be-
reavement Study. World Psychiatry 2016;15:266-75.
17. Reynolds CF 3rd, Miller MD, Pasternak RE et al. Treatment of bereavement-
related major depressive episodes in later life: a controlled study of acute and
continuation treatment with nortriptyline and interpersonal psychotherapy.
Am J Psychiatry 1999;156:202-8.
18. Shear K, Frank E, Houck PR et al. Treatment of complicated grief: a rand-
omized controlled trial. JAMA 2005;293:2601-8.
19. Maercker A, Brewin CR, Bryant RA et al. Diagnosis and classication of dis-
orders specically associated with stress: proposals for ICD-11. World Psy-
chiatry 2013;12:198-206.
20. World Health Organization. ICD-11 guidelines. https://gcp.network/en/.
21. Boelen PA, Lenferink LIM. Comparison of six proposed diagnostic criteria
sets for disturbed grief. Psychiatry Res (in press).
22. Prigerson HG, Maciejewski PK. Prolonged Grief Disorder (PG-13) scale. Bos-
ton: Dana-Farber Cancer Institute, 2008.
23. Prigerson HG, Jacobs S. Traumatic grief as a distinct disorder: a ration-
ale, consensus criteria, and a preliminary empirical test. In: Stroebe MS,
Hansson RO, Stroebe W et al (eds). Handbook of bereavement research:
consequences, coping, and care. Washington: American Psychological As-
sociation, 2001:613-45.
24. Boelen PA, de Keijser J, Smid G. Cognitive-behavioral variables mediate
the impact of violent loss on post-loss psychopathology. Psychol Trauma
2015;7:382-90.
25. Smith KV, Ehlers A. Cognitive predictors of grief trajectories in the first
months of loss: a latent growth mixture model. J Consult Clin Psychol
2020;88:93-105.
26. First MB, Spitzer RL, Gibbon M et al. Structured Clinical Interview for DSM-
IV Axis I Disorders, Non-Patient Version (SCID-V-NP). New York: Biometrics
Research Department, New York State Psychiatric Institute, 1995.
27. Latham AE, Prigerson HG. Suicidality and bereavement: complicated grief
as psychiatric disorder presenting greatest risk for suicidality. Suicide Life
reat Behav 2004;34:350-62.
28. Ware JE, Sherbourne CD. e MOS 36-item short-form health survey (SF-
36). I. Conceptual framework and item selection. Med Care 1992;30:473-
83.
29. Engelhard IM, Arntz A, van den Hout MA. Low specicity of symptoms on
the post-traumatic stress disorder (PTSD) symptom scale: a comparison of
individuals with PTSD, individuals with other anxiety disorders and indi-
viduals without psychopathology. Br J Clin Psychol 2007;46:449-56.
30. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II.
San Antonio: Psychological Corporation, 1996.
31. Weathers F, Litz B, Keane T et al. e PTSD Checklist for DSM-5 (PCL-5).
www.ptsd.va.gov.
32. Kroencke K, Spitzer R, Williams J. e PHQ-9: validity of a brief depression
severity measure. J Gen Int Med 2001;16:606-13.
33. Mundt JC, Marks IM, Shear MK et al. The Work and Social Adjustment
Scale: a simple measure of impairment in functioning. Br J Psychiatry
2002;180:461-4.
34. Kabaco R. Determining the dimensionality of data: a SAS Macro for parallel
analysis. SAS Users Group International 28, Seattle, March 2003.
35. Boelen PA, Van Den Bout J, De Keijser J et al. Reliability and validity of the
Dutch version of the Inventory of Traumatic Grief (ITG). Death Studies
2003;27:227-47.
World Psychiatry 20:1 - February 2021 11
36. Pohlkamp L, Kreicbergs U, Prigerson HG et al. Psychometric properties of
the Prolonged Grief Disorder-13 (PG-13) in bereaved Swedish parents. Psy-
chiatry Res 2018;267:560-5.
37. He L, Tang S, Yu W et al. e prevalence, comorbidity and risks of prolonged
grief disorder among bereaved Chinese adults. Psychiatry Res 2014;219:347-
52.
38. Delalibera M, Coelho A, Barbosa A. Validação do instrumento de aval-
iação do luto prolongado para a população portuguesa. Acta Med Port
2011;24:935-42.
39. Morina N, Rudari V, Bleichhardt G et al. Prolonged grief disorder, depression,
and posttraumatic stress disorder among bereaved Kosovar civilian war sur-
vivors: a preliminary investigation. Int J Soc Psychiatry 2010;56:288-97.
40. Fisher JE, Mauro C, Cozza SJ et al. Examination of factor structure of the
inventory of complicated grief (ICG) in a sample of bereaved military fam-
ily members with persistent and elevated grief. Int J Methods Psychiatr Res
2017;26:e1571.
DOI:10.1002/wps.20823
