Korean version of the delirium rating scale-revised-98: reliability and validity.
ABSTRACT The aims of the present study were 1) to standardize the validity and reliability of the Korean version of Delirium Rating Scale-Revised-98 (DRS-R98-K) and 2) to establish the optimum cut-off value, sensitivity, and specificity for discriminating delirium from other non-delirious psychiatric conditions.
Using DSM-IV criteria, 157 subjects (69 delirium, 29 dementia, 32 schizophrenia, and 27 other psychiatric patients) were enrolled. Subjects were evaluated using DRS-R98-K, DRS-K, Mini-Mental State Examination (MMSE-K), and Clinical Global Impression-Severity (CGI-S) scale.
DRS-R98-K total and severity scores showed high correlations with DRS-K. They were significantly different across all groups (p=0.000). However, neither MMSE-K nor CGI-S distinguished delirium from dementia. All DRS-R98-K diagnostic items (#14-16) and items #1 and 2 significantly discriminated delirium from dementia. Cronbach's alpha coefficient revealed high internal consistency for DRS-R98-K total (r=0.91) and severity (r=0.89) scales. Interrater reliability (ICC between 0.96 and 1) was very high. Using receiver operating characteristic analysis, the area under the curve of DRS-R98-K total score was 0.948 between the delirium group and all other groups and 0.873 between the delirium and dementia groups. The best cut-off scores in DRS-R98-K total score were 18.5 and 19.5 between the delirium and the other three groups and 20.5 between the delirium and dementia groups.
We demonstrated that DRS-R98-K is a valid and reliable instrument for assessing delirium severity and diagnosis and discriminating delirium from dementia and other psychiatric disorders in Korean patients.
- SourceAvailable from: Paula T Trzepacz[show abstract] [hide abstract]
ABSTRACT: The DRS-R-98, a 16-item clinician-rated scale with 13 severity items and 3 diagnostic items, was validated against the Cognitive Test for Delirium (CTD), Clinical Global Impression scale (CGI), and Delirium Rating Scale (DRS) among five diagnostic groups (N=68): delirium, dementia, depression, schizophrenia, and other. Mean and median DRS-R-98 scores significantly (P<0.001) distinguished delirium from each other group. DRS-R-98 total scores correlated highly with DRS, CTD, and CGI scores. Interrater reliability and internal consistency were very high. Cutoff scores for delirium are recommended based on ROC analyses (sensitivity and specificity ranges: total, 91%-100% and 85%-100%; severity, 86%-100% and 77%-93%, respectively, depending on the cutoffs or comparison groups chosen). The DRS-R-98 is a valid measure of delirium severity over a broad range of symptoms and is a useful diagnostic and assessment tool. The DRS-R-98 is ideal for longitudinal studies.Journal of Neuropsychiatry 02/2001; 13(2):229-42. · 2.40 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: To cross-validate the Delirium Rating Scale (DRS). Cross-sectional. Geriatric medicine and geriatric psychiatry assessment units and consultation services. A total of 104 older patients on the above services. Mini-Mental State Examination (MMSE) score, Barthel Index score, DRS score, Blessed Dementia Scale score, clinical diagnoses using DSM-III-R criteria. The mean DRS score was highest in the delirium group. Cronbach's alpha was .90, and inter-rater reliability of total scores was .91 (intra-class correlation). Receiver Operating Characteristic curve analysis showed that the area under the curve for the DRS was significantly higher than the MMSE as a test for delirium. At its published cutpoint of 10, the sensitivity of the DRS is .82 and the specificity is 94. The value at which the sensitivity of the DRS is .90 is 8, at which specificity is .82. The DRS appears to a feasible instrument. In a sample with a high proportion of delirious patients, it has acceptable measurement properties when used by expert observers.Journal of the American Geriatrics Society 08/1996; 44(7):839-42. · 3.98 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: The Delirium Rating Scale (DRS) has been shown to be a valid instrument for identifying and grading the severity of delirium in patients admitted to a general hospital for medical or surgical treatment. However, its accuracy in identifying delirium among elderly patients admitted to a psychiatric hospital for evaluation and treatment of psychiatric illness has not been previously addressed. The DRS was administered to 791 elderly patients who were consecutively admitted to a psychogeriatric unit; 70 met DSM-III-R criteria for delirium. A DRS threshold score of > or = 10 correctly identified delirious patients with a sensitivity of 94% and a specificity of 82%. Both psychosis and cognitive impairment appeared to falsely elevate the DRS score in this population.Journal of Neuropsychiatry 01/1994; 6(1):30-5. · 2.40 Impact Factor
Print ISSN 1738-3684 / On-line ISSN 1976-3026
30 Copyright © 2011 Korean Neuropsychiatric Association
Korean Version of the Delirium Rating Scale-Revised-98:
Reliability and Validity
Yanghyun Lee1* , Jian Ryu2, Jinyoung Lee3, Hwi-Jung Kim4,
Im Hee Shin5, Jeong-Lan Kim6 and Paula T. Trzepacz7,8,9,10
1Department of Psychiatry, School of Medicine, Kyungpook National University, Daegu, Korea
2MAYA Mental Hospital, Yeongcheon, Korea
3Department of Psychiatry, Daegu Veterans Hospital, Daegu, Korea
4Angang Chung-ang Hospital, Gyeongju, Korea
5Department of Medical Statistics, College of Medicine, Catholic University of Daegu, Daegu, Korea
6Department of Psychiatry, College of Medicine, Chungnam National University, Daejeon, Korea
7Lilly Research Laboratories, Neurosciences, Indianapolis, IN, USA
8Department of Psychiatry and Behavioral Sciences, University of Mississippi Medical School, Jackson, MS, USA
9Department of Psychiatry, Tufts University School of Medicine, Boston, MA, USA
10Department of Psychiatry, Indiana University Medical School, Indianapolis, IN, USA
ObjectiveaaThe aims of the present study were 1) to standardize the validity and reliability of the Korean version of Delirium Rating Scale-
Revised-98 (DRS-R98-K) and 2) to establish the optimum cut-off value, sensitivity, and specificity for discriminating delirium from oth-
er non-delirious psychiatric conditions.
MethodsaaUsing DSM-IV criteria, 157 subjects (69 delirium, 29 dementia, 32 schizophrenia, and 27 other psychiatric patients) were en-
rolled. Subjects were evaluated using DRS-R98-K, DRS-K, Mini-Mental State Examination (MMSE-K), and Clinical Global Impression-
Severity (CGI-S) scale.
ResultsaaDRS-R98-K total and severity scores showed high correlations with DRS-K. They were significantly different across all groups
(p=0.000). However, neither MMSE-K nor CGI-S distinguished delirium from dementia. All DRS-R98-K diagnostic items (#14-16) and
items #1 and 2 significantly discriminated delirium from dementia. Cronbach’s alpha coefficient revealed high internal consistency for
DRS-R98-K total (r=0.91) and severity (r=0.89) scales. Interrater reliability (ICC between 0.96 and 1) was very high. Using receiver oper-
ating characteristic analysis, the area under the curve of DRS-R98-K total score was 0.948 between the delirium group and all other groups
and 0.873 between the delirium and dementia groups. The best cut-off scores in DRS-R98-K total score were 18.5 and 19.5 between the de-
lirium and the other three groups and 20.5 between the delirium and dementia groups.
ConclusionaaWe demonstrated that DRS-R98-K is a valid and reliable instrument for assessing delirium severity and diagnosis and
discriminating delirium from dementia and other psychiatric disorders in Korean patients.
Psychiatry Investig 2011;8:30-38
Key Wordsaa Delirium, Psychiatric status rating scales, Validation studies.
Delirium is an acute psychiatric syndrome characterized
with impairment of consciousness with a prominent distur-
bance of attention and deficits of other cognitive domains, per-
ception, language, thought, motor behavior, sleep-wake cycle,
and affective control. Delirium is a common problem among
hospitalized medically ill patients, affecting roughly 25% of this
population.1 Delirium is associated with significant morbidi-
ty and increased mortality (between 11% and 65% during hos-
Received: July 3, 2010 Revised: September 13, 2010 Accepted: September 20, 2010 Available online: December 13, 2010
Correspondence: Yanghyun Lee, MD, PhD
Department of Psychiatry, Mungyeong Jeil General Hospital, 188 Mojeon-dong, Mungyeong 745-882, Korea
Tel: +82-54-550-7977, Fax: +82-54-555-4034, E-mail: email@example.com
*Yanghyun Lee, move to Department of Psychiaty, Mungyeong Jeil General Hospital, 188 Mojeon-dong, Mungyeong 745-882, Korea
Tel: +82-54-550-7977, Fax: +82-54-555-4034
cc This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which
permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
online © ML Comm
Y Lee et al.
pital admission), and length of hospitalization as compared to
non-delirious patients.2 Because it is frequently under- or mis-
diagnosed and associated with poor outcomes, a precise and
reliable instrument for the diagnosis and repeated evaluation
of delirium is needed.
The Delirium Rating Scale (DRS) objectively measures symp-
toms severity of delirium.3 The DRS is widely used and has been
translated into eleven languages other than English (French,
Chinese, Dutch, Spanish, Italian, Swedish, Japanese, German,
Thai, Indian, and Korean).4 The DRS was validated to differ-
entiate delirium from other psychiatric disorders, and compris-
ed a broader range of symptoms of delirium than cognitive tests
such as the Mini-Mental Status Examination (MMSE). Many
studies have demonstrated that the DRS is a useful tool for as-
sessment of delirium, showing high validity and reliability.5,6
The Korean version of DRS (DRS-K) has also been demonstrat-
ed as a valid tool which significantly discriminates delirium from
dementia and schizophrenia, with sensitivity and specificity
similar to the original DRS. It is reliable with high internal cor-
relation and interrater reliability.7
Limitations of the DRS include: certain items (e.g., tempo-
ral onset) are difficult to rate for repeated serial assessments dur-
ing an episode of delirium; hypoactivity and hyperactivity are
measured in the same item; and only one item rates all aspects
of cognitive functions because the DRS was intended to be used
in conjunction with bedside cognitive tests. Further, the DRS
lacks items for language and thought process abnormalities.
As a result, the Delirium Rating Scale-Revised-98 (DRS-R98) was
created to compensate for the shortcomings of the DRS, espe-
cially for use in phenomenological and longitudinal research.4
The DRS-R98 has been translated into Dutch,8 Spanish,9 Ger-
man, Portuguese,10 Japanese,11 Danish, Hebrew, Lithuanian,
Greek, Norwegian, both traditional12 and modern Chinese,
The objectives of the present study were to evaluate the va-
lidity and reliability of the Korean version of Delirium Rating
Scale-Revised-98 (DRS-R98-K) and to establish the optimal
cut-off values, sensitivity, and specificity for discriminating de-
lirium from other non-delirious psychiatric conditions.
All delirious patients were recruited from medical wards of
two university hospitals. Patients for dementia, schizophrenia,
and other psychiatric disorders groups were recruited from
inpatient psychiatric and general medical units or outpatient
facilities. Diagnoses were made according to the diagnostic cri-
teria of Diagnostic and Statistical Manual of Mental Disorders
4th edition (DSM-IV).14 The consultation service psychiatrist
referred patients diagnosed with delirium to the research team.
Control patients were referred to the research team from their
treating outpatient or inpatient service psychiatrist. Dementia
and schizophrenia patients were chosen as control groups be-
cause of overlapping symptoms with delirium.2,15 Other psy-
chiatric disorder patients did not have significant cognitive
dysfunction or psychotic symptoms and typically had mild to
moderate depression or anxiety disorder. Following explana-
tion of the procedures and rationale for the study, verbal con-
sent was obtained from the patient or the proxy decision-
maker. Because of the non-invasive nature of the study, ethics
committee approved the use of verbal consent. Patients exclud-
ed if they refused evaluation or were on mechanical ventilation.
This study was approved by the Institutional Review Board and
Ethics Committee of the Kyungpook National University and
The original English version of the DRS-R98 scale was trans-
lated into Korean with permission from the copyright owner.
This translation was performed by a native Korean who is a pro-
fessor in the department of English literature who lived in
the USA. for several years. Then the translation was reviewed
and revised by three experts in psychiatry and delirium. This
revised version was back-translated by a bilingual person who
was fluent in both languages. The back-translated version was
compared with the original version by the three above-men-
tioned experts. This Korean version was also reviewed and re-
vised by a professor in the department of Korean to ensure that
it was sensitive to the Korean culture and environment. Then, four
expert psychiatrists who had vast experience with the diagno-
sis and treatment of delirium and were not involved with this
research reviewed this Korean version from a user’s point of view.
Finally, taking into consideration the experts’ opinions, the au-
thors produced the final Korean version.
Korean version of Delirium Rating Scale-Revised-98
The DRS-R98 proved to be a valid and reliable instrument for
the assessment of delirium severity and distinguish patients with
delirium from non-delirious patients including dementia.4 It
was originally developed for use by psychiatrically trained clini-
cians. Similarly, the DRS-R98-K is intended for these clinicians.
The scale is composed of two sections. The first section is the se-
verity scale, which consists of 13 items that are scored from 0 to 3
according to a Likert scale that is anchored by descriptions of phe-
nomenology (Table 3). The second section consists of three diag-
nostic items that are scored from 0 to 2 or 3. These items include
temporal onset of symptoms, fluctuation of symptom severity,
32 Psychiatry Investig 2011;8:30-38
DRS-R98K: Reliability and Validity
and physical disorder. The total scale is comprised of 16 items.
Copies of this scale can be obtained from Dr. Trzepacz.
Korean version of Delirium Rating Scale
The DRS-K was performed in all groups in order to compare
with DRS-98R-K for validity. The scale is commonly used for
the evaluation of delirium in Korea.7 The DRS-K has shown
significant differences in the total score between delirium, de-
mentia, and schizophrenia patients as well as high sensitivity
and specificity. Cronbach’s alpha coefficient of this scale was
0.88 and interrater reliability was 0.98 (p<0.0001). The origi-
nal English version correlated highly with the DRS-R98 Eng-
Korean version of Mini-Mental State Examination and
Clinical Global Impression-Severity scale
The Korean version of Mini-Mental State Examination
(MMSE-K), which was standardized by Park et al,16 is a com-
monly used instrument for the bedside evaluation of cognitive
dysfunction in Korea. The CGI-S17 is a commonly used glob-
al rating of disease severity based on a clinician’s total experi-
ence in evaluating patients with a particular disorder and is
rated as a Likert scale from 1 (normal, not at all ill) to 7 (among
the most extremely ill patients) points. The CGI-S was used for
the comparison of disease severity across groups.
All subjects were rated by one of the two primary raters (JR,
JL) who are extensive experienced psychiatrists with admin-
istering these scales. Primary raters were each compared to an-
other trained psychiatrist (YL) who independently scored a
subgroup of the delirium and control patients (i.e., blinded to
each other’s ratings) to calculate interrater reliability between
pairs of raters (JR-YL, JL-YL). Some of the delirious subjects
(n=41) were retested by the same rater after the delirious symp-
toms resolved to determine whether the DRS-R98-K was sen-
sitive to change in delirium status per DSM-IV criteria. All av-
ailable information including medical records, nursing records,
and interviews with care-givers or family members were re-
viewed to obtain data for proper rating. The rating of DRS-R98-
K was based on a 24-hour period due to the fluctuating nature
of delirium symptoms.
Data were analysed using SPSS software version 14. All rat-
ing scale scores by the primary raters (JR, JL) were used for sta-
tistical analysis. The scores of the secondary rater (YL) were
only used for interrater reliability analysis. All demographic
data and scale scores are expressed in means and standard de-
viations. Statistical significance was set as alpha level 0.05. Chi-
square tests were used for the comparison of the categorical
Concurrent validity was established by calculating Spear-
man’s correlation coefficients between the DRS-R98-K and
other tests (DRS-K, MMSE-K, and CGI-S) in delirium pati-
ents as well as for the whole sample. Predictive validity was
assessed by comparing total and severity scale scores of DRS-
R98-K with the scores of DRS-K, MMSE-K, and CGI-S among
groups with post hoc (LSD) pairwise comparisons to determine
where the differences existed. Boxplots were graphed for the dis-
tribution of scale scores, quartiles, and median scores with Krus-
kal-Wallis test for comparison.
Normality test was done for all variables using the Shapiro-
Wilk p value. One-way ANOVA was used to compare the scale
score between groups when the data was normally distribut-
ed (DRS-K, Shapiro-Wilk p=0.056-0.650). For the variables
that were not normally distributed, Kruskal-Wallis test was
used (DRS-R98-K total, Shapiro-Wilk p=0.000-0.514; DRS-
R98-K severity, Shapiro-Wilk p=0.000-0.477; CGI-S, Shapiro-
Wilk p=0.000-0.041; MMSE-K, Shapiro-Wilk p=0.002-0.123).
Clinically, delirium and dementia are difficult to differenti-
ate and commonly co-morbid.18,19 It is well known that mor-
tality risk is increased in the comorbidity.20 Therefore, dis-
criminant analysis was conducted to determine which items
of the DRS-R98-K could be helpful for differential diagnosis
between delirium and dementia and then, to determine how
well the current classification functions of discriminant anal-
ysis using these items of DRS-R98-K could predict clinical
group classification between the two groups by DSM-IV. Op-
timal cut-off scores for the DRS-R98-K were determined from
the receiver operating characteristic (ROC) curve and sensi-
tivity and specificity.
Internal consistency of DRS-R98-K was assessed using Cr-
onbach’s alpha coefficient, including for after each item had been
deleted. Interrater reliability was calculated using the intraclass
correlation (ICC) between pairs of raters. The DRS-R98-K scor-
es before and after treatment were compared by paired t-test to
All 157 subjects were enrolled and comprised four groups:
69 delirium, 29 dementia, 32 schizophrenia, and 27 other psy-
chiatric patients (e.g., depression). Half (n=75, 47.8%) of all
patients in the study were men, and the mean age was 59.9
years (range 16-91; SD=15.7). Demographic data of each group
are summarized in Table 1.
There was a significant difference in sex of the groups
(χ2=28.09, df=3, p<0.0001). The majority of patients were fe-
Y Lee et al.
male in each group, except delirium. However, there were no
significant differences in DRS-K score and DRS-R98-K se-
verity and total scores between sexes, except in the other psy-
chiatric group where scores are well into the non-delirious range
(The p values of DRS-R98-K total, DRS-R98-K severity, and
DRS-K scores in each group, respectively; In delirium group,
p=0.208, p=0.246, p=0.194; In dementia group, p=0.631,
p=0.487, p=0.805; In schizophrenia group, p=0.169, p=0.185,
p=0.276; In other group, p=0.007, p=0.004, p=0.003). There was
also a significant difference in mean age of the groups where de-
mentia patients were significantly older than the other groups
and schizophrenia patients were the youngest (F=27.26, df=
3,153, p<0.0001), though the age ranges were similarly broad
in each group. The age of the delirium group was also signifi-
cantly different from the other three groups, while there was
no difference between other psychiatric and schizophrenia gro-
ups. Age was not considered to influence the rating scores be-
cause there was no correlation between the rating score and age
for any of the scales. In addition, analysis of covariance (AN-
COVA) was conducted to evaluate the possibility of an age ef-
fect on the comparison of rating scores between groups. Age
was not a significant covariate for the rating scale scores (DRS-
R98-K total score, p=0.379; DRS-R98-K severity score, p=0.301;
DRS-K, p= 0.433); therefore, it is not considered to have a large
effect on group comparisons of the rating scores.
In the delirium group, the strongest correlations were be-
tween the DRS-R98-K total and severity scales and the DRS-
K scale (r=0.831, r=0.811, respectively). Correlations between
the DRS-R98-K total and severity and the MMSE-K scales
were strong but in the moderate range (r=-0.657, r=-0.672, re-
spectively) while with the CGI-S was also moderate but the
lowest of the scale comparisons (r=0.532, r=0.552, respective-
ly). All were statistically significant (p=0.000). These results
demonstrate that the DRS-R98-K rating scale is a validated in-
strument for delirium. In the all-subjects comparison, all cor-
relations were also significant. Strengths of correlation paral-
leled the pattern for the delirium only correlations.
Mean scores for all rating scales were significantly different
across groups at the p=0.000 or p=0.009 level (Table 1). Post
hoc analysis demonstrated that DRS-R98-K total score was
significantly higher in the delirium group compared to all oth-
er groups. The dementia group was also significantly differ-
ent from the other three groups, though mean rating scale
scores did not differ between the schizophrenia and other
psychiatric group. The DRS-R98-K severity score was also
significantly different among groups (p<0.001) where the de-
lirium group had the highest mean score, followed by demen-
tia, schizophrenia, and other psychiatric groups. DRS-K scores
were significantly different across groups (p<0.001) where de-
lirium group was significantly higher than in any other group,
where the dementia group was significantly higher than the
schizophrenia and other psychiatric groups that were similar
to each other.
MMSE-K score was significantly different among groups,
with the lowest scores recorded in delirium and dementia
groups (no significant differences between them), whereas
mean scores were in the mildly impaired/borderline normal
range for the schizophrenia and other psychiatric groups.
Mean CGI-S ratings, used to compare illness severity, scored
in the “4” point range (moderately ill) for all groups suggest-
ing good clinical comparability despite statistical significant
differences among groups, largely accounted for by the delir-
ium group. Nonetheless, the CGI-S score was used as a covari-
ate to explore the possibility of impact of overall disease se-
Table 1. Demographic and rating scale scores in four diagnostic groups, mean±SD (range)
Age, mean±SD (range)63.4±14.5 (23-84)
Sex, female (%) 20 (29)
a17 Alzheimer’s, 7 vascular, 2 mixed, 3 dementia not otherwise specified (NOS), b18 depressive disorder, 1 bipolar disorder, 2 panic disorder, 2
generalized anxiety disorder, 2 adjustment disorder, 1 undifferentiated somatoform disorder, 1 alcohol dependence, †MMSE-K was assessed
in 65 of the 69 delirium patients because of uncooperativeness, ‡CGI-S score was calculated in 66 of the 69 delirium patients because of in-
complete data. *Kruskal-Wallis Test, p=0.000 among groups, **One-way analysis of variance, p=0.000 among groups, ***Kruskal-Wallis Test,
p=0.009 among groups. DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98, DRS-K: Korean version of Delirium Rating Scale,
MMSE-K: Korean version of Mini Mental Status Examination, CGI-S: Clinical Global Impression-Severity
34 Psychiatry Investig 2011;8:30-38
DRS-R98K: Reliability and Validity
verity on DRS-R98-K differences among groups using the
ANCOVA. CGI-S was a statistically significant covariate be-
cause it significantly correlated with all scale scores but there
was no interaction between it and the other scores. After ad-
justment of the covariate, rating scale scores were again com-
pared. DRS-R98-K total and severity scores were statistically
significantly different among groups, and pairwise compari-
sons revealed significant differences for both among all groups.
Findings were essentially the same on all scales for group dif-
ferences as described above.
Boxplots were graphed to show the scale score distribution
with medians and quartiles (Figure 1). Medians and middle
quartiles of DRS-R98-K total score of the delirium group did
not overlap with those of the other groups. However, DRS-R98-
K severity distribution of the delirium group slightly overlapped
with that of the dementia group. MMSE-K score showed sub-
stantial overlap between the delirium and dementia group and
between the schizophrenia and other psychiatric group. CGI-S
scores were significantly different but there was substantial over-
lap across groups.
Sensitivity and specificity
ROC curve analysis was used to identify the threshold of
DRS-R98-K total and severity score and DRS-K scores that
could distinguish patients with delirium from those with all
other diagnoses, and particularly from those with dementia
(Figure 2). ROC curve analysis showed that the area under the
curve (AUC) of DRS-R98-K total score between the delirium
group and all other groups was 0.948 (SE=0.016), and that the
AUC between the delirium and dementia group was 0.873
(SE=0.037). The best cut-off scores were 18.5 and 19.5 in DRS-
R98-K total score between the delirium and other three groups,
and 20.5 in those between the delirium and dementia group
Figure 1. Boxplots of Korean version of Delirium Rating Scale-Revised-98 (DRS-R98-K) total, DRS-R98-K severity, Korean version of Mini-Men-
tal State Examination (MMSE-K), and Global Impressio-Severity (CGI-S) scores for each of the four diagnostic groups. Median scores are denot-
ed by the horizontal lines within the 50th percentile boxes; tails denotes the 25th percentiles and circles and asterisks are outliers. Significant dif-
ferences between delirium and each other group were noted for the DRS-R98-K and DRS-K.
Y Lee et al.
The AUC of DRS-R98-K severity score between the deliri-
um and other three groups was 0.894 (SE=0.024), and the AUC
between the delirium and dementia group 0.763 (SE=0.05). The
best cut-off points for the DRS-R98-K severity scale were 14.5
and 15.5 between delirium and all other groups, and 19.5 be-
tween the delirium and dementia group.
The AUC of DRS-K recorded the highest value, 0.999 (SE=
0.001) between the delirium and other three groups, and 0.997
(SE=0.003) between delirium and dementia group. The cut-off
score of DRS-K was 12.5 between the delirium and other three
groups, and 14.5 between the delirium and dementia group.
Discriminant analysis was applied to determine which items
were the most effective at differentially diagnosing between
delirium and dementia. Five of the 16 items were statistically
significant for discriminating delirium from dementia. Item 15
(‘fluctuation of symptom severity’, F=112.66, p=0.000, Canoni-
cal coefficient=0.633) showed the greatest discriminating pow-
er, followed by item 16 (‘physical disorder’, F=111.67, p=0.000,
Canonical coefficient=0.408), item 2 (‘perceptual disturbanc-
es and hallucinations’, F=99.84, p=0.000, Canonical coefficient=
0.468), item 14 (‘temporal onset of symptoms’ , F=84.43, p=0.000,
Canonical coefficient=0.349), and item 1 (‘sleep-wake cycle dis-
turbance’, F=72.16, p=0.000, Canonical coefficient=0.282). And
also, it is possible to set discriminant function as D=0.282×i-
tem16) with statistical significance (Wilks’ Lambda=0.203,
p=0.000). The concordance rate between the DSM-IV diagnosed
delirium group and the group ascertained by applying these five
items from discriminant analysis was 98%, showing that those
five items can be enough to distinguish delirium from dementia
with high discriminant power.
Cronbach’s alpha coefficient was 0.91 for the DRS-R98-K
total scale and 0.89 for the severity scale, indicating good in-
ternal consistency. After the correction for the effect of each
item removal from the scale, the alpha coefficients ranged
Table 2. Sensitivity and specificity for DSM-IV delirium diagnosis
at various cut-off scores (delirium N=69; dementia N=29; all other
total other groups
ScalesCut-off Sensitivity Specificity
Delirium vs. 17.59680
66 Delirium vs.
DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98,
DRS-K: Korean version of Delirium Rating Scale
Table 3. Alpha coefficients for DRS-R98-K total and severity scales
when each item is removed from the scale
Sleep-wake cycle disturbance
Perceptions and hallucinations
Lability of affect
Thought process abnormalities
Temporal onset of symptoms
Fluctuation of symptom severity
DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98,
N/A: not applicable
36 Psychiatry Investig 2011;8:30-38
DRS-R98K: Reliability and Validity
from 0.90 to 0.92 for the DRS-R98-K total scale and from
0.87 to 0.91 for the DRS-R98-K severity scale, indicating high
internal consistency and reliability of this scale (Table 3).
Interrater reliability was measured using the ICC between
two primary raters and one secondary rater. When the pri-
mary rater JR was compared with the secondary rater YL for
the DRS-R98-K total score (n=20), ICC=0.987; for the DRS-
R98-K severity score (n=20), ICC=0.986; and for the DRS-K
score (n=20), ICC=0.978. When the other primary rater JL was
compared with the secondary rater YL for the DRS-R98-K
total score (n=18), ICC=0.969; for the DRS-R98-K severity
score (n=18), ICC=0.965; for the DRS-K score (n=18), ICC=
0.965. These data indicate excellent interrater reliability.
DRS-R98-K pre and post treatment
Forty-four of the delirious subjects were reassessed after
delirium treatment and 41 of them no longer met DSM-IV
criteria for delirium. Mean scores for DRS-R98-K total (pre,
27.7±7.2; post, 9.2±4.0), DRS-R98-K severity (pre, 22.2±7.2;
post, 6.5±3.6), and DRS-K (pre, 21.4±3.7; post, 6.1±2.9) du-
ring pre- and post-treatment were significantly improved
(p=0.000) and well below cut-off scores from ROC analyses.
Figure 2. ROC curve of DRS-R98-K total and severity score between the delirium group and other three groups and between the delirium
and dementia groups. ROC: receiver operating characteristic, DRS-R98-K: Korean version of the Delirium Rating Scale-Revised-98.
0.2 0.40.6 0.8 1.0
DRS-R98-K total in delirium vs. others
DRS-R98-K severity in delirium vs. others
DRS-R98-K total in delirium vs. dementia
0.2 0.40.6 0.81.0
DRS-R98-K severity in delirium vs. dementia
Y Lee et al.
Mean MMSE-K scores also improved (pre, 16.6±6.7; post,
23.8±4.4; p=0.000). The DRS-R98-K is responsive to change
in clinical status as anchored by DSM-IV.
This study demonstrates that the DRS-R98-K is a reliable
and valid instrument for diagnosis and severity assessment
of delirium in Korean patients. Both DRS-R98-K total and se-
verity scores showed high correlations with Korean versions
of validated instruments that assess some or many delirium
symptoms, the DRS-K and MMSE-K. DRS-R98-K total and
severity scores showed higher correlations with the DRS-K
than with MMSE-K and CGI-S scores. This finding is sup-
ported by the report that the admission MMSE score have
12% influence of the variance on the DRS-R98 score.21 This re-
sult demonstrates that this instrument is specific to delirium
and that it also shares common characteristics with MMSE-
K, in measuring cognitive functions and with the CGI-S, in as-
sessing disease severity. Our findings for validity, reliability, in-
ternal consistency and response to change are consistent with
other validation studies of the DRS-R98 in other countries.
DRS-R98-K significantly distinguishes delirium patients
from dementia, schizophrenia, and other psychiatric groups
based on mean and median score comparisons. However,
there is some overlap of DRS-R98-K scores distributions be-
tween the delirium and dementia groups as expected from
overlapping phenomenology. The score distribution in the
delirium group of our study is similar with the results of the
original study,4 whereas the score distribution in our demen-
tia group was somewhat higher than theirs [DRS-R98 total
score 13.9±4.2 (range 9-22), DRS-R98 severity score 12.4±
3.5 (range 8-20)]. The greater overlap in score distribution in
dementia in our study may have resulted from inclusion of
some end-stage dementia patients that raised the mean more
than the median. The overlap between the two groups did not
affect the diagnosis of each disease by discriminant analysis.
As expected, the diagnostic three items (items 14, 15, 16) as well
as item 1 (‘sleep-wake cycle disturbance’) and item 2 (‘percep-
tual disturbances and hallucinations’) were revealed as the im-
portant items for the differential diagnosis between delirium
and dementia. Other studies have found similar high differ-
entiation for the diagnostic items, though a different pattern for
the other items and different statistical methods were em-
CGI-S was clinically comparable across groups though sta-
tistically different. When data for DRS-K and DRS-R98-K to-
tal and severity scales were reanalyzed using CGI-S as a co-
variate, the findings did not differ. Thus, slight difference in
overall illness severity among groups could not explain the group
differences in DRS-K and DRS-R98-K severity and total scale
findings. Unlike the delirium specific scales, the MMSE-K did
not discriminate between delirium and dementia groups nor be-
tween the schizophrenia and other psychiatric group.
Our optimum cut-off scores (18.5 and 19.5) on the DRS-R98-
K total scale between delirium and the other three groups
were slightly higher than scores found by Trzepacz et al (15.25 &
17.75). Similarly, our cut-off score between delirium and de
mentia groups was 20.5 whereas theirs was 17.75. The cut-off
score of 14.5 and 15.5 for DRS-R98-K severity between delir-
ium and other groups in our study was similar to the score of
15.25 in Trzepacz et al’.s study, while the cut-off score of 19.5
Table 4. Cut-off scores based on ROC analysis in studies across countries
Delirium vs. all other other diagnosis
Trzepacz et al.
et al. (2007)
Franco et al.
Huang et al.
Lim et al.
Lee et al.
ROC: receiver operating characteristic, DRS-R-98: Korean version of Delirium Rating Scale-Revised-98
Delirium vs. Dementia
77 15.392 92
38 Psychiatry Investig 2011;8:30-38
DRS-R98K: Reliability and Validity
between delirium and dementia groups in our study was
higher than their score of 15.25. These findings are likely be-
cause the mean scores of DRS-R98-K total and severity scale
in our dementia group were high due to the enrollment of
some end-stage dementia patients into the group, demon-
strating that the severity of dementia might affect the cut-off
points. Our cut-off scores are more similar to those reported
in the Portuguese DRS-R98 validation (20.1) where more se-
vere dementia cases were included than in some other stud-
ies.10 The optimum cut-off scores do vary somewhat across
countries (Table 4) and may be affected by cultural differenc-
es in addition to dementia severity range.
One limitation of our study was that the sex distribution am-
ong groups was not even, though the comparison of scale
scores between sexes did not differ, except in the other psychi-
atric group where scores were in the non-delirious range. An-
other limitation was the age difference among groups. It was
predicted because of the characteristics of the peak age of
onset of each disease. It was not considered to affect the score
of the rating scales due to the lack of correlation between age
and score within groups. Next obstacle was the possibility
that the test-retest results might be counterbalanced by the
rater’s expectation or the fact of clinical improvement.
This study included completely different subjects from an-
other validation study13 of DRS-R98 carried out by other re-
searchers in Korea. Compared to the previous study, this study
enrolled more patients and end-stage dementia patients whom
we could meet in the clinical field. Furthermore, our study put
some great efforts to define the cut-off score in order to differ-
entiate delirium and dementia, so these cut-off scores can be
used in bedside effectively and easily. The most influential items
for making a distinction between delirium and dementia were
revealed by discriminant analysis in the study. Taken together
our data reveal that the DRS-R98-K is a valid and reliable instru-
ment for diagnosing and evaluating the severity of delirium.
This research was supported by Kyungpook National University Re-
search Fund, 2003.
1. Brown TM, Boyle MF. Delirium. BMJ 2002;325:644-647.
2. Trzepacz PT, Meagher DJ. Neuropsychiatric Aspects of Delirium. In:
Yudofsky SC, Hales RE, editors. APPI Textbook of Neuropsychiatry
and Clinical Neurosciences. 5th ed. Washington, DC: American Psy-
chiatric Publishing, Inc.; 2008, p.445-517.
3. Trzepacz PT, Baker RW, Greenhouse J. A symptom rating scale for de-
lirium. Psychiatry Res 1988;23:89-97.
4. Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Valida-
tion of the Delirium Rating Scale-revised-98: comparison with the de-
lirium rating scale and the cognitive test for delirium. J Neuropsychiatry
Clin Neurosci 2001;13:229-242.
5. Rockwood K, Goodman J, Flynn M, Stolee P. Cross-validation of the De-
lirium Rating Scale in older patients. J Am Geriatr Soc 1996;44:839-842.
6. Rosen J, Sweet RA, Mulsant BH, Rifai AH, Pasternak R, Zubenko GS. The
Delirium Rating Scale in a psychogeriatric inpatient setting. J Neuropsy-
chiatry Clin Neurosci 1994;6:30-35.
7. Lee Y, Ryu J, Kim H-J, Lee S. Standardization of the Korean Version of De-
lirium Rating Scale. J Korean Soc Biol Ther Psychiatry 2003;9:181-190.
8. de Rooij SE, van Munster BC, Korevaar JC, Casteelen G, Schuurmans
MJ, van der Mast RC, et al. Delirium subtype identification and the
validation of the Delirium Rating Scale--Revised-98 (Dutch version) in
hospitalized elderly patients. Int J Geriatr Psychiatry 2006;21:876-882.
9. Franco JG, Mejía MA, Ochoa SB, Ramírez LF, Bulbena A, Trzepacz P. [De-
lirium rating scale-revised-98 (DRS-R-98): Colombian adaptation of the
Spanish version]. Actas Esp Psiquiatr 2007;35:170-175.
10. de Negreiros DP, da Silva Meleiro AM, Furlanetto LM, Trzepacz PT.
Portuguese version of the Delirium Rating Scale-Revised-98: reliability
and validity. Int J Geriatr Psychiatry 2008;23:472-477.
11. Kato M, Kishi Y, Okuyama T, Trzepacz PT, Hosaka T. Japanese version
of the Delirium Rating Scale-Revised-98 (DRS-R98-J): reliability and
validity. Psychosomatics 2010;51:425-431.
12. Huang MC, Lee CH, Lai YC, Kao YF, Lin HY, Chen CH. Chinese version
of the Delirium Rating Scale-Revised-98: reliability and validity. Com-
pr Psychiatry 2009;50:81-85.
13. Lim KO, Kim SY, Lee YH, Lee SW, Kim JL. A validation study for the Ko-
rean version of Delirium Rating Scale-Revised-98 (K-DRS-98). J Korean
Neuropsychiatr Assoc 2006;45:518-526.
14. American Psychiatric-Association. Diagnostic and Statistical Manual
of Mental Disorders, Text Revision, DSM-IV-TR. 4th ed. Washington,
DC: American Psychiatric Association; 2000.
15. Ross CA, Peyser CE, Shapiro I, Folstein MF. Delirium: phenomenologic
and etiologic subtypes. Int Psychogeriatr 1991;3:135-147.
16. Park JH, Park YN, Ko HJ. Modification of the Mini-Mental State Ex-
amination for use with the elderly in a non-western society; Part II. cut-
off points and their diagnosis validities. Int J Geriatr Psychiatry 1991;
17. Guy W. Clinical global impression (CGI). In: ECDEU Assessment Man-
ual for Psychopharmacology (revised). Rockville, MD: National Institute
of Mental Health, 1976, p.217-221.
18. Fick DM, Agostini JV, Inouye SK. Delirium superimposed on dementia:
a systematic review. J Am Geriatr Soc 2002;50:1723-1732.
19. Fick DM, Kolanowski AM, Waller JL, Inouye SK. Delirium superim-
posed on dementia in a community-dwelling managed care popula-
tion: a 3-year retrospective study of occurrence, costs, and utilization. J
Gerontol A Biol Sci Med Sci 2005;60:748-753.
20. Bellelli G, Frisoni GB, Turco R, Lucchi E, Magnifico F, Trabucchi M. De-
lirium superimposed on dementia predicts 12-month survival in elder-
ly patients discharged from a postacute rehabilitation facility. J Geron-
tol A Biol Sci Med Sci 2007;62:1306-1309.
21. Franco JG, Valencia C, Bernal C, Ocampo MV, Trzepacz PT, Pablo J, et
al. Relationship between cognitive status at admission and incident de-
lirium in older medical inpatients. J Neuropsychiatry Clin Neurosci 2010;
22. Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Delir-
ium vs dementia symptoms: Delirium Rating Scale-Revised-98 (DRS-
R-98) and Cognitive Test for Delirium (CTD) item comparisons. Psycho-