Content uploaded by Nikol Figalová
Author content
All content in this area was uploaded by Nikol Figalová on May 13, 2021
Content may be subject to copyright.
Running head: PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 1
Psychometric Properties of the Perceived Stress Scale in an Adult Psychiatric Inpatient
Sample
Nikol Figalová1, João Silva2, & Miroslav Charvát1
1 Department of Psychology, Palacký University Olomouc
2 Faculty of Psychology, University of Lisbon
Author Note
Nikol Figalová https://orcid.org/0000-0001-7618-4852
Miroslav Charvát https://orcid.org/0000-0003-3413-4501
Nikol Figalová is now at the Department of Clinical and Health Psychology, Ulm
University, Ulm, Germany.
The data and the analysis script are publicly available at
http://dx.doi.org/10.17632/k8zk3xhcjf.2. We have no known conflict of interest to disclose. The
study is based on an unpublished Master’s dissertation of Figalová (2019).
Correspondence concerning this article should be addressed to Nikol Figalová, Faculty of
Engineering, Computer Science and Psychology, Institute of Psychology and Education,
Deptartment of Clinical & Health Psychology, Albert-Einstein-Allee 41, 89081 Ulm, Germany.
Email: Nikol.figalova@uni-ulm.de
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 2
Abstract
Despite the fact that the Perceived Stress Scale (PSS) is often used in the psychiatric population,
only few studies reported its psychometric properties in such samples. This study aims to bridge
this gap. We administered the 10-item version of the PSS to a sample of psychiatric inpatients (n
= 153) and evaluated its psychometric properties. Using the confirmatory factor analysis, we
found that a bifactor model was the best fit. The scale showed excellent internal consistency (α =
.91 and ω = .93 for the bifactor model). Item analysis discovered strong inter-item correlations,
and indicated that item 9 had relatively low factor loading and item-total correlation. Women
obtained a higher score of perceived stress than men. Our findings suggest that the scale works
differently in the psychiatric sample than in the general population, and that the PSS might be
omitting some of the important aspects of the perceived stress construct.
Keywords: Perceived Stress Scale, Reliability, Validity, Factor Structure, Psychiatric Inpatient
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 3
The Perceived Stress Scale (PSS), developed by Cohen, Kamarck, and Mermelstein
(1983), is a self-report scale designed to measure the degree to which individuals appraise
situations in their lives as stressful. The items of the PSS are developed to evaluate how
unpredictable, uncontrollable, and overloaded respondents find their lives (Cohen & Williamson,
1988). Originally, the scale consists of 14 items. Two shorter versions, derived from the original
scale, exist: the four-item PSS-4, and the 10-item PSS-10. The three forms of the PSS were
previously compared (Andreou et al., 2011; Cohen & Williamson, 1988; Lesage, Berjot, &
Deschamps, 2012; Leung, Lam, & Chan, 2010). Authors consensually report comparable or
higher internal consistency, better factor structure, and higher sensitivity for the PSS-10
compared to both PSS-14 and PSS-10. This phenomenon was also confirmed in the Czech
version of the scale (Figalová & Charvát, 2021a). Therefore, the PSS-10 is the most often used
version of the scale.
Psychometric properties of the PSS were previously reported in a wide variety of
samples, including general adult population (Andreou et al., 2011; Cohen & Janicki-Deverts,
2012; Figalova & Charvat, 2021a), students (Örücü & Demir, 2009; Ramírez & Hernández,
2007; Roberti, Harrington, & Storch, 2006), teachers (Reis, Hino, & Añez, 2010), policewomen
(Wang et al., 2011), or adults with a physical illness (Golden-Kreutz, Browne, Frierson, &
Andersen, 2004; Lee, Chung, Suh, & Jung, 2015; Leung, Lam, & Chan, 2010). Furthermore, two
studies reporting data obtained in a psychiatric sample were identified. Authors of the first study
(Jovanovic & Gavrilov-Jerkovic, 2015) administered the PSS-10 to a sample including 157
outpatients, diagnosed with depressive disorders (36.90%), mixed anxiety and depression
(28.70%), anxiety disorders (26.80%), and other (7.60%). The authors suggested that the
structure of the PSS was best represented by a bifactor model, comprising two specific factors
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 4
plus a general factor. The scale exhibited good internal consistency (α = .86), positive
correlations with measures of depression (r = .73), anxiety (r = .61), stress (r = .71), negative
affect (r = .78), and negative correlations with positive affect (r = -.68) and life satisfaction (r = -
.55). In the second study, Hewitt, Flett, and Mosher (1992) administered the PSS-14 to a sample
of 96 psychiatric patients (76 outpatients, 20 inpatients), diagnosed mostly with depressive
disorders, schizophrenia, marital/family problems, alcoholism, and adjustment disorders.
Authors suggested that a two-factor solution (which accounted for 46.60% of the variance) was
most appropriate. The scale exhibited good internal consistency (α = .80) and positive correlation
with the Beck Depression Inventory (r = .57).
Despite the fact that the PSS is the most widely used measure of perceived stress in the
English-speaking countries and has been translated into more than 30 languages (Cohen &
Janicki-Deverts, 2012), we found only two studies reporting the psychometric properties of the
PSS in a psychiatric sample. Nevertheless, the PSS is often used with psychiatric patients. For
example, Candrian et al. (2008) administered the PSS to patients with major medical depression
who underwent a medical treatment with antidepressants, and Mugrabi et al. (2020) used the PSS
to assess changes in perceived stress induced by psychiatric treatment in patients with psychiatric
disorders. However, if an instrument is to be used in a very specific sample (e.g., psychiatric
population), it is important to evaluate its psychometric properties in such samples. Otherwise,
the external validity of an instrument may be low and the results biased. Item 9 of the PSS-10
may serve as an example. Respondents are asked: „In the last month, how often have you been
angered because of things that were outside of your control? “. While a person from a general
adult population usually controls the majority of things in his or her life, psychiatric inpatients
are in a different position. They are often hospitalised against their will, live in a highly
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 5
controlled environment, have to follow the rules of the institution, and have even to share their
personal space with other patients. Therefore, it is inappropriate to expect that this item would
generate the same results under such different circumstances.
Even though the results of other psychometric studies seem promising, it might be
misleading to generalise these results to a psychiatric population. Therefore, the goal of this
study is to evaluate the psychometric properties of the Czech version of the PSS-10 in a
population of psychiatric inpatients. We aim to assess its usability in psychiatric samples for
other authors who consider employing this instrument in their studies, as well as for healthcare
professionals who use the PSS in their practice.
Method
Sample
The participants had to meet the following inclusion criteria: 1. psychiatric hospital
inpatient; 2. primary psychiatric diagnosis; 3. lucidity and vigilance on a level necessary for the
administration of the PSS-10; 4. signed an informed consent prior to the administration.
We collected the data in two institutions, the Psychiatric Hospital Opava (n = 79) and the
Psychiatric Hospital Kroměříž (n = 74), Czech Republic. The PSS-10 was administered to N =
153 psychiatric hospital inpatients, out of which 56.2% were women (n = 86). The age of the
respondents ranged from 16 to 71 years (M = 41.94, SD = 12.49). The average time of
hospitalization was 36 days (SD = 27.16, five outliers with hospitalization ranging from 126 to
344 days were removed from the calculation). For future analyses, respondents were also divided
into two groups according to the length of hospitalization. The first group consisted of 74
patients hospitalized for less than one month (M = 14.51 days, SD = 9.40), the second group
consisted of 79 patients hospitalized for more than one month (M = 68.39 days, SD = 43.00). The
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 6
educational attainment and primary diagnosis of the respondents (according to the International
Classification of Diseases-10; World Health Organisation, 1993) are presented in Table 1.
Table 1
Characteristics of the sample by educational attainment and primary diagnosis
Variable
n
(N=153)
%
Level of education
Secondary school or less (9 years or less)
29
19.0
Practical high school (12 years)
56
36.6
High school (13 years)
41
26.8
University (16 years or more)
25
16.3
Not specified
2
1.3
Primary diagnosis
F10–F19 Mental and behavioural disorders due to psychoactive substance use
79
51.6
F20-F29 Schizophrenia, schizotypal and delusional disorders
11
7.2
F30-F39 Mood [affective] disorders
13
8.5
F40-F48 Neurotic, stress-related and somatoform disorders
34
22.2
F60-F69 Disorders of adult personality and behaviour
14
9.2
Other
2
1.3
This research was approved by the ethical committee of both institutions. All participants
had to sign an informed consent prior to the test administration, and agree to share the necessary
medical information with the researcher. All participants were informed that their participation is
voluntary and can be terminated at any time. All data were anonymized and processed according
to the General Data Protection Regulation (GDPR).
Data from Figalova and Charvat (2021b) representing the general adult population of the
Czech Republic were used to compare the psychiatric sample presented in this paper with the
general population. The sample of the general population consisted of 1725 adults aged 18–91
years (M = 44.32, SD = 12.83), of whom 56.9% were women, mostly with university education
(70.7%). For further specification of the sample, see Figalova & Charvat (2021a).
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 7
Instrument
Participants were asked to provide basic demographic characteristics. Furthermore, we
administered the 10-item Czech version of the Perceived Stress Scale (Figalová & Charvát,
2021a). The questions of the scale ask about feelings and thoughts during the last month.
Respondents report how often they felt in a certain way on a 5-point scale (from 0 = never to 4 =
very often). The PSS consists of both negatively stated items (measuring perceived distress) and
positively stated items (measuring perceived self-efficacy). The total score is obtained by
reversing the scores for the positively stated items and then summing all items across the scale
(Cohen, Kamarck, & Mermelstein, 1983). The data collection was performed either in group or
individually. In some cases, participants required assistance, such as reading the items aloud by
the researcher. We did not administer any further instruments alongside the PSS, as this was
evaluated as demanding and potentially stressful for the participants.
Analytic strategy
We asked participants to fill the PSS-10 in a paper-pencil form. We did not include
protocols with three or more missing items in the analysis (in total 33 protocols were omitted).
The confirmatory factor analysis (CFA) and reliability analyses were performed in the lavaan
package (Rosseel, 2012) in the RStudio. For the CFA, we used the standard settings of the
function ‘cfa’ with standardized estimates. We set the items as ordered; hence the weighted least
square mean and variance adjusted (WLSMV) estimator was used. We report robust (scaled) test
statistics. We compared the one-factor model, two-factor model, and bifactor model of the PSS-
10. To evaluate the internal consistency, we computed both Cronbach’s α and McDonald’s ω.
The item analysis was performed with a focus on an item’s mean, inter-item correlation, item-
total correlation, and Cronbach’s α if the item was deleted. Descriptive analyses and known-
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 8
group difference analyses were performed using SPSS, version 26. As the data of different
subsamples do not show normal distribution and/or homoscedasticity, we used non-parametric
methods to compare these subsamples.
Results
Confirmatory factor analysis
Results of the CFA are presented in Table 2. We compared the one-factor, two-factor,
and bifactor model. The AGFI statistic, as well as the incremental fit indices (NFI, TLI, CFI,
IFI), were satisfactory in all three models (>.90, Hooper, Coughlan, & Mullen, 2008). The SRMR
values should be below .05 to indicate a good fit, however, values as high as .08 are deemed
acceptable (Hooper, Coughlan, & Mullen, 2008). The observed SRMR values suggested that the
bifactor model was the best fit for the data, and the two-factor model was an acceptable fit.
Moreover, the threshold of an RMSEA suggesting good fit is generally considered to be below
.07 (Steiger, 2007). In this sample, only the bifactor model was close to this threshold.
Table 2
Results of the CFA comparing the one-factor, two-factor, and bifactor model
Absolute Fit Indices
Incremental Fit Indices
Model
χ2
df
p
AGFI
SRMR
RMSEA
NFI
TLI
CFI
IFI
one-factor
170.15
3
35
.00
0
.954
.086
.159
.91
2
.90
8
.92
8
.929
two-factor
77.406
34
.00
0
.984
.051
.092
.96
0
.97
0
.97
7
.977
bifactor
44.588
25
.00
9
.990
.033
.072
.97
7
.98
1
.99
0
.990
Note. χ2 = chi-square; df = degrees of freedom; AGFI = Adjusted good fit index; SRMR = Standardized root mean square of residuals; RMSEA
= Root mean square of approximation, NFI = Normed fit index; TLI = Tucker-Lewis index; CFI = Comparative fit index, IFI = Incremental fit
index.
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 9
Table 3 presents the factor loading of the PSS-10 items in all three models we compared.
The observed factor loading was acceptable in case of the one-factor and two factor models, only
the factor loading of item 9 was slightly below the recommended threshold (.60, Awang, 2014).
For the bifactor model, we observed low factor loading of the items on the group factors, but the
factor loadings on the general factor was satisfactory. The low loading of the items on the group
factors is not problematic if the scores for these factors are not reported individually (DeMars,
2013). The correlation between factors in the two-factor model was r = .75. The factors in the
bifactor model were set as orthogonal.
Table 3
Factor loading of the PSS-10 items
One-factor
model
Two-factor
model
Bifactor model
Item
PS
PI
NI
PI
NI
PS
PSS_1
.680
.715
.319
.634
PSS_2
.707
.735
.193
.705
PSS_3
.729
.762
.288
.699
PSS_4r
.764
.814
.335
.718
PSS_5r
.786
.835
.359
.730
PSS_6
.585
.625
.304
.551
PSS_7r
.754
.798
.527
.641
PSS_8r
.737
.781
.543
.619
PSS_9
.532
.571
.799
.364
PSS_10
.865
.932
.327
.854
Note. PS = Perceived Stress; PI = Positively Stated Items; NI = Negatively Stated Items
Reliability
We computed both Cronbach’s α and the McDonald’s ω to assess internal consistency.
The total values for the full scale are presented in Table 4. The internal consistency was very
good in all three models.
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 10
Table 4
Internal consistency
One-factor
Two-factor
Bifactor
Cronbach’s α
.91
.91
.91
McDonald’s ω
.89
.90
.93
Furthermore, we carried an item analysis to evaluate the quality of the items and how
these behave in a sample consisting of psychiatric inpatients. Table 5 shows the results. No
attenuation effect was observed. All items had satisfactory item-total correlation and the overall
internal consistency of the scale would not increase by removal of any of the items.
Table 5
Item analysis
Item
M
SD
Response option
Item-Total
Correlatio
n
α if
Item
Delete
d
0
1
2
3
4
PSS_1
2.49
1.13
.05
.15
.29
.29
.22
.61
.88
PSS_2
2.44
1.12
.06
.13
.31
.31
.19
.64
.88
PSS_3
2.71
1.08
.04
.08
.29
.31
.27
.65
.88
PSS_4r
1.75
1.19
.18
.24
.32
.18
.08
.66
.88
PSS_5r
1.95
1.14
.12
.23
.33
.24
.09
.68
.88
PSS_6
2.16
1.13
.07
.23
.33
.24
.14
.54
.89
PSS_7r
1.83
.99
.09
.27
.40
.20
.04
.62
.88
PSS_8r
1.93
1.04
.09
.23
.40
.22
.07
.60
.88
PSS_9
2.33
1.19
.08
.16
.30
.26
.20
.48
.89
PSS_1
0
2.28
1.39
.13
.18
.27
.13
.29
.79
.87
Note. M = Mean; SD = Standard Deviation; Item-Total Correlation = Corrected Item-Total Correlation; α if Item Deleted = Cronbach's Alpha if
item deleted
Table 6 presents the inter-item correlation. The upper part of the table shows Pearson
correlations, the bottom part of the table shows polychoric correlations. The observed values
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 11
ranged from r = .15 to r =.62 for Pearson correlation, and from ρ = .15 to ρ = .67 for polychoric
correlation. Cohen, Swerdlik, and Phillips (1996) recommends that the ideal range of inter-item
correlation is between .20 and .40. With an exception of the correlation between items 8r and 9,
which was under the suggested threshold, all items fit within or above this threshold. The high
observed correlations are further discussed in the Discussion section.
Table 6
Inter-item correlation
PSS_
1
PSS_
2
PSS_
3
PSS_4
r
PSS_5
r
PSS_
6
PSS_7
r
PSS_8
r
PSS_
9
PSS_1
0
PSS_1
.49
.56
.41
.38
.37
.38
.36
.44
.53
PSS_2
.53
.49
.41
.49
.41
.41
.42
.36
.61
PSS_3
.61
.54
.48
.46
.34
.44
.37
.45
.58
PSS_4r
.45
.44
.53
.58
.34
.61
.56
.32
.57
PSS_5r
.42
.52
.50
.62
.38
.59
.62
.27
.58
PSS_6
.41
.45
.39
.36
.41
.28
.34
.39
.59
PSS_7r
.43
.45
.49
.67
.64
.30
.62
.18
.47
PSS_8r
.40
.46
.41
.61
.67
.37
.68
.15
.47
PSS_9
.48
.40
.50
.34
.29
.43
.20
.15
.53
PSS_1
0
.58
.67
.64
.63
.64
.65
.53
.53
.58
Descriptive statistics and known-group differences
The total score of the scale ranged from 5 to 38, the average score of the whole sample
was M = 21.88 (SD = 8.10). The average score of women was M = 24.41 (SD = 6.89), the
average score of men was M = 18.63 (SD = 8.43). No relationship between age and total score
was observed (r = -.05, p = .532). A weak negative correlation was observed between the time of
hospitalization and total score (r = -.318, p < .001).
A Mann-Whitney test indicated that the score of perceived stress was higher for women
(Mdn = 24) than for men (Mdn = 18), U (nwomen = 86, nmen = 67) = 1750.00, z = -4.16, p < .001. A
large effect size was observed (η2= .11, d = 0.71). Furthermore, respondents with a primary
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 12
diagnosis from the group of Neurotic, stress-related and somatoform disorders (Mdn = 29) had a
higher score of perceived stress compared to respondents with diagnoses from other categories
(Mdn = 21). A Mann-Whitney test indicated that this difference was statistically significant, U
(nstress = 34, nother = 119) = 919.00, z = -4.85, p <.001. A large effect size was observed (η2 = .15,
d = 0.85).
To compare the sample of psychiatric inpatients with the general adult population, we
used data shared by Figalova & Charvat (2021b). Respondents recruited from the psychiatric
inpatient population reported a higher score of perceived stress (Mdn = 22) than respondents
recruited from the general adult population (Mdn = 18). A Mann-Whitney test indicated that this
difference was statistically significant, U (npsychiatric = 153, ngeneral = 1725) = 94969.50, z = -5.76,
p < .001. A small effect size was observed (η2 = .02, d = 0.27).
Discussion
The aim of this study was to evaluate the psychometric properties of the Czech
translation of the 10-item Perceived Stress Scale (PSS-10) in a population of psychiatric
inpatients. The sample consists of N = 153 respondents. The respondents were hospitalised in a
psychiatric hospital with various mental and behavioural disorders, classified accordingly to the
International Classification of Diseases-10 (World Health Organisation, 1993). The average time
of hospitalisation was 36 days.
The construct validity of the PSS was examined using confirmatory factor analysis
(CFA). The fit of a one-factor, two-factor, and bifactor model was compared and several
absolute and incremental fit indices were reported. The χ2 statistic indicated a poor fit for all
three compared models. However, the χ2 statistic is very sensitive to sample size and is no longer
used as a basis for an acceptance or rejection of a model (Vandenberg, 2006). Considering the
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 13
RMSEA absolute fit indices, only the value of the bifactor model reached the threshold for a good
fit (RMSEA = .072). However, values below .10 are generally considered to be tolerable (Steiger,
2007), suggesting that the two-factor model could also be a good fit. Furthermore, the SRMR
values indicating good fit should be below .05, however, values as high as .08 are deemed
acceptable (Hooper, Coughlan, & Mullen, 2008). The SRMR value observed in the present study
suggests the bifactor model was the best fit for the data (SRMR = .033), and that the two-factor
model had an acceptable fit (SRMR = .051). Overall, the one-factor model did not fit the data
well. The two-factor model could be considered acceptable, although it was clearly inferior to the
bifactor model in all observed indices. The bifactor model fit data the best. This finding is in line
with the reports of other authors who compared the one-factor, two-factor, and bifactor model
(Figalova & Charvat, 2021a; Jovanovic & Gavrilov-Jerkovic, 2015).
The factor loading of the items was overall satisfactory. However, the factor loading of
item 9 (.532 and .571 for the one-factor and two-factor model, respectively) was slightly below
the recommended threshold (.60; Awang, 2014). This was also observed previously in a sample
of Czech general population (Figalova & Charvat, 2021a). We believe that the reason might be
an imperfect translation of this item, or cultural differences in responses to this item. Moreover,
bias could also arise from the fact that this item asks participants how often they have been
angered because of things that were outside of their control. This question might provide
misleading results if asked respondents living in a highly controlled environment of a psychiatric
hospital. Furthermore, we found low factor loading of most of the items on group factors in the
bifactor model. Therefore, reporting individual scores for the separate “subscales” formed by
positively and negatively stated items is not recommended (DeMars, 2013). This finding
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 14
supports the recommendation of Cohen and Williamson (1988), who claim that only a single
total score of perceived stress should be obtained while using the PSS.
We used both Cronbach’s α and McDonald’s ω to evaluate the internal consistency of the
PSS-10. While Cronbach’s α is the most commonly used measure of internal consistency,
McDonald’s ω is more appropriate for multidimensional data and should therefore be preferred
(Dunn et al., 2014). All the observed values suggested very good internal consistency. This
agrees with other authors, who also generally reported good or very good internal consistency of
the scale.
We have also conducted an item analysis in order to evaluate how each item of the PSS-10
behaves in the studied sample. We found that all items are of a good quality. The only potential
problem arose when we analysed the inter-item correlation. Cohen et al. (1996) recommends that
the ideal range of inter-item correlation is between .20 and .40, suggesting that while the items
are reasonably homogenous, they contain sufficiently unique variance. However, a considerable
number of inter-item correlation values reported in this study exceeded the recommended upper
threshold. It could suggest that the items might be too homogenous, not capturing the entire
bandwidth of the construct. This finding might be also interesting in relation to convergent
validity of the PSS. A number of previous authors reported strong and very strong correlations
between the PSS and measures of anxiety (Figalova & Charvat, 2021a; Pbert et al., 1992; Remor,
2006; Roberti et al., 2006). We presume that the PSS might omit some of the important aspects
of the perceived stress, and this might be accented even more in a sample of psychiatric
inpatients. We recommend that this issue be addressed directly in a future study, and potentially
a new method is developed to address the full complexity of perceived stress.
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 15
We observed higher scores of perceived stress amongst women compared to men in the
psychiatric inpatient sample. Similar gender effect was previously observed in several studies on
different populations (Andreou et al., 2011; Lesage et al., 2012; Leung et al., 2010; Remor,
2006). The study by Hewitt et al. (1992) on psychiatric population also reported higher levels of
perceived stress amongst women. Moreover, gender differences were found in the Czech general
adult population, signifying that women have a higher score of perceived stress than men
(Figalova & Charvat, 2021a). Contrary, Jovanovic and Gavrilov-Jerkovic (2015) did not observe
any gender differences in perceived stress in a Serbian psychiatric sample. We believe that the
results of the present study support the good quality of the Czech version of the PSS, as the
translation seems to work in a similar manner as the original, English language version.
However, the gender difference effect size between the Czech general adult population (d = 0.34)
and Czech psychiatric inpatient population (d = 0.71) is relatively large. This difference may be
accounted for by the clinical nature of the present sample. The observed difference in total score
between the Czech general adult population and the Czech psychiatric sample support this
thought. Overall, our findings suggest that the PSS indeed works differently for the general adult
population and for the psychiatric inpatient population.
Interestingly, we observed a large effect size in differences between respondents with a
primary diagnosis from the group of Neurotic, stress-related and somatoform disorders and
respondents with diagnoses from other categories (d = 0.85). We believe this finding supports the
idea that the PSS really measures, at least up to a point, stress-related symptoms.
The present study has several limitations. First, the sample size was relatively small and a
convenience sampling method was used. Second, a relatively small number of psychiatric
diagnoses is represented in the sample. In order to generalise the findings of this study, we
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 16
recommend using larger sample size and quota sampling to obtain a representative sample of
psychiatric inpatient population. Third, no further self-report measures were administered
alongside the PSS. Although this was necessary in order not to expose participants to a
potentially stressful situation, collecting more data to assess convergent validity would be
extremely useful. Nevertheless, the diagnosis of the patient can be, up to a certain point, linked to
criterion validity.
To summarize, our findings suggest that the scale works differently in the psychiatric
inpatient sample and general adult sample. Moreover, our results suggest that the PSS might be
omitting some of the important aspects of the perceived stress construct. Therefore, we
recommend caution when using the PSS to assess the level of perceived stress, especially when
used in the very specific context of psychiatric inpatients. A new tool to assess perceived stress,
that would include more aspects of the complex and multifaceted perceived stress phenomena,
would be beneficiary not only for psychiatric inpatient samples, but presumably for researchers
studying all populations.
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 17
Literature
Andreou, E., Alexopoulos, E. C., Lionis, C., Varvogli, L., Gnardellis, C., Chrousos, G. P., &
Darviri, C. (2011). Perceived Stress Scale: Reliability and Validity Study in Greece.
International Journal of Environmental Research and Public Health, 8(8), 3287–3298.
https://doi.org/10.3390/ijerph8083287
Awang, Z. (2014). A handbook on SEM for academicians and practitioners: the step by step
practical guides for the beginners. Bandar Baru Bangi, MPWS Rich Resources.
Cohen, R. J., Swerdlik, M. E., & Phillips, S. M. (1996). Psychological testing and assessment:
An introduction to tests and measurement. Mayfield Publishing Co.
Cohen, S., & Janicki-Deverts, D. (2012). Who’s Stressed? Distributions of Psychological Stress
in the United States in Probability Samples from 1983, 2006, and 2009. Journal of Applied
Social Psychology, 42(6), 1320–1334. doi: 10.1111/j.1559-1816.2012.00900.x
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A Global Measure of Perceived
Stress. Journal of Health and Social Behavior, 24(4), 385. doi: 10.2307/2136404
Cohen, S., & Williamson, G. (1988). Perceived stress in a probability sample of the United
States. In S. Spacapan & S. Oskamp (Eds.), The social psychology of health: Claremont
Symposium on Applied Social Psychology, 31–67. Newbury Park, CA: SAGE.
Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: guidelines for
determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.
https://doi.org/10.21427/D7CF7R
Candrian, M., Schwartz, F., Farabaugh, A., Perlis, R. H., Ehlert, U., & Fava, M. (2008).
Personality disorders and perceived stress in major depressive disorder. Psychiatry
Research, 160(2), 184–191. https://doi.org/10.1016/j.psychres.2007.06.014
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 18
DeMars, C. E. (2013). A Tutorial on Interpreting Bifactor Model Scores. International Journal
of Testing, 13(4), 354–378. doi: 10.1080/15305058.2013.799067
Dunn, T. J., Baguley, T., & Brunsden, V. (2013). From alpha to omega: A practical solution to
the pervasive problem of internal consistency estimation. British Journal of
Psychology, 105(3), 399–412. doi: 10.1111/bjop.12046
Figalová, N. (2019). Translation and psychometric properties of the Czech version of the
Perceived Stress Scale. (Unpublished master’s thesis). Olomouc: Palacky University.
Figalova, N., & Charvat, M. (2021a). The Perceived Stress Scale: Reliability and Validity Study
in the Czech Republic. Ceskoslovenska Psychologie, 65(1), 46-59.
Figalova, N., & Charvát, M. (2021b). The Perceived Stress Scale: Reliability and Validity Study
in the Czech Republic (3) [Data file, analysis script, and Czech version of the scale].
Retreived from http://dx.doi.org/10.17632/kmkhx3tzjx.3
Golden-Kreutz, D. M., Browne, M. W., Frierson, G. M., & Andersen, B. L. (2004). Assessing
Stress in Cancer Patients. Assessment, 11(3), 216–223. doi: 10.1177/1073191104267398
Hewitt, P. L., Flett, G. L., & Mosher, S. W. (1992). The Perceived Stress Scale: Factor structure
and relation to depression symptoms in a psychiatric sample. Journal of Psychopathology
and Behavioral Assessment, 14(3), 247–257. doi: 10.1007/bf00962631
Jovanovic, V., & Gavrilov-Jerkovic, V. (2015). More than a (negative) feeling: Validity of the
perceived stress scale in Serbian clinical and non-clinical samples. Psihologija, 48(1), 5-
18. doi:10.2298/PSI1501005J
Lee, E.-H., Chung, B. Y., Suh, C.-H., & Jung, J.-Y. (2014). Korean versions of the Perceived
Stress Scale (PSS-14, 10 and 4): psychometric evaluation in patients with chronic
disease. Scandinavian Journal of Caring Sciences, 29(1), 183–192. doi: 10.1111/scs.12131
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 19
Lesage, F.-X., Berjot, S., & Deschamps, F. (2012). Psychometric properties of the French
versions of the Perceived Stress Scale. International Journal of Occupational Medicine
and Environmental Health, 25(2). doi: 10.2478/s13382-012-0024-8
Leung, D. Y., Lam, T.-H., & Chan, S. S. (2010). Three versions of Perceived Stress Scale:
validation in a sample of Chinese cardiac patients who smoke. BMC Public Health, 10(1).
doi: 10.1186/1471-2458-10-513
Mugrabi, F., Rozner, L., & Peles, E. (2020). The mindfulness trait and high perceived stress
changes during treatment in patients with psychiatric disorders. Current Psychology, 1–2.
https://doi.org/10.1007/s12144-020-01108-1
Örücü, M. Ç., & Demir, A. (2009). Psychometric evaluation of perceived stress scale for Turkish
university students. Stress and Health, 25(1), 103–109. https://doi.org/10.1002/smi.1218
Pbert, L., Doerfler, L. A., & Decosimo, D. (1992). An evaluation of the perceived stress scale in
two clinical populations. Journal of Psychopathology and Behavioral Assessment, 14(4),
363–375. doi: 10.1007/bf00960780
Ramírez, M. T. G., & Hernández, R. L. (2007). Factor Structure of the Perceived Stress Scale
(PSS) in a Sample from Mexico. The Spanish Journal of Psychology, 10(1), 199–206. doi:
10.1017/s1138741600006466
Reis, R. S., Hino, A. A. F., & Añez, C. R. R. (2010). Perceived Stress Scale. Journal of Health
Psychology, 15(1), 107–114. doi: 10.1177/1359105309346343
Remor, E. (2006). Psychometric Properties of a European Spanish Version of the Perceived
Stress Scale (PSS). The Spanish Journal of Psychology, 9(1), 86–93. doi:
10.1017/s1138741600006004
PERCEIVED STRESS SCALE IN A PSYCHIATRIC SAMPLE 20
Roberti, J. W., Harrington, L. N., & Storch, E. A. (2006). Further Psychometric Support for the
10-Item Version of the Perceived Stress Scale. Journal of College Counseling, 9(2), 135–
147. doi: 10.1002/j.2161-1882.2006.tb00100.x
Rosseel, Y. (2012). lavaan: an R Package for Structural Equation Modeling. Journal of
Statistical Software, 48(2). doi: 10.18637/jss.v048.i02
Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation
modeling. Personality and Individual Differences, 42(5), 893–
898. https://doi.org/10.1016/j.paid.2006.09.017
Vandenberg, R. J. (2006). Introduction: Statistical and Methodological Myths and Urban
Legends. Organizational Research Methods, 9(2), 194–201. doi:
10.1177/1094428105285506
Wang, Z., Chen, J., Boyd, J. E., Zhang, H., Jia, X., Qiu, J., & Xiao, Z. (2011). Psychometric
Properties of the Chinese Version of the Perceived Stress Scale in Policewomen. PLoS
ONE, 6(12). doi: 10.1371/journal.pone.0028610
World Health Organization. (1993). The ICD-10 classification of mental and behavioural
disorders: diagnostic criteria for research (Vol. 2). World Health Organization.
