Psychometric Properties of the Malay Schizotypal Personality Questionnaire: Measurement Invariance and Latent Mean Comparisons in Malaysian Adults

Article (PDF Available)inAsia-Pacific Psychiatry 10(1) · May 2017with 84 Reads
DOI: 10.1111/appy.12293
Abstract
Introduction. The Schizotypal Personality Questionnaire (SPQ) is a widely-used self-report measurement instrument for the assessment of schizotypal personality traits. However, the factor structure of the SPQ has been a matter of some debate. As a contribution to this debate, we examined the factor structure of the SPQ in Malaysian adults. Method. A total of 382 Malaysian adults completed a Malay translation of the SPQ. Confirmatory factory analysis was used to examine the fit of 3- and 4-factor solutions for the higher-order dimensionality of the SPQ. Ethnic invariance for the best-fitting model was tested at the configural, metric, and scalar levels, and a multivariate analysis of variance was used to examine sex and ethnicity differences in domain scores. Results. The 4-factor model provided a better fit to the data than did the 3-factor model. The 4-factor model also demonstrated partial measurement invariance across ethnic groups. Latent mean comparisons for sex and ethnicity revealed a number of significant differences for both factors, but effect sizes were small. Discussion. The 4-factor structure of the SPQ received confirmatory support and can be used in Malay-speaking populations.
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
ORIGINAL ARTICLE
Psychometric properties of the Malay Schizotypal Personality
Questionnaire: Measurement invariance and latent mean
comparisons in Malaysian adults
David Barron MRes
1
|Kevin D. Morgan PhD
1
|Tony Towell PhD
1
|Jas L. Jaafar PhD
2
|
Viren Swami PhD
3,4
1
Department of Psychology, University of
Westminster, London, UK
2
Department of Educational Psychology and
Counselling, University of Malaya, Kuala
Lumpur, Malaysia
3
Department of Psychology, Anglia Ruskin
University, Cambridge, UK
4
Centre for Psychological Medicine, Perdana
University, Serdang, Malaysia
Correspondence
David Barron, Department of Psychology,
Faculty of Science and Technology, University
of Westminster, 115 New Cavendish Street,
London W1W 6UW, UK.
Email: davidsbarron1@outlook.com
Abstract
Introduction: The Schizotypal Personality Questionnaire (SPQ) is a widely used selfreport
measurement instrument for the assessment of schizotypal personality traits. However, the
factor structure of the SPQ has been a matter of some debate. As a contribution to this debate,
we examined the factor structure of the SPQ in Malaysian adults.
Method: A total of 382 Malaysian adults completed a Malay translation of the SPQ.
Confirmatory factory analysis was used to examine the fit of 3and 4factor solutions for the
higherorder dimensionality of the SPQ. Ethnic invariance for the bestfitting model was tested
at the configural, metric, and scalar levels, and a multivariate analysis of variance was used to
examine sex and ethnicity differences in domain scores.
Results: The 4factor model provided a better fit to the data than did the 3factor model. The
4factor model also demonstrated partial measurement invariance across ethnic groups. Latent
mean comparisons for sex and ethnicity revealed a number of significant differences for both
factors, but effect sizes were small.
Discussion: The 4factor structure of the SPQ received confirmatory support and can be
used in Malayspeaking populations.
KEYWORDS
confirmatory factor analysis, measurement invariance, Schizotypal Personality Questionnaire,
schizotypy
1|INTRODUCTION
Schizotypy describes a clustered personality dimension that includes
suspiciousness, magical thinking, unusual perceptual experiences, and
social anhedonia (Nelson, Seal, Pantelis, & Phillips, 2013). One well
established measure that assesses schizotypal personality is the
Schizotypal Personality Questionnaire (SPQ; Raine, 1991). The SPQ
was designed to have one subscale measure each of the 9 symptoms
of schizotypal personality disorder (Raine, 1991), namely, no close
friends, constricted affect, ideas of reference, odd beliefs and magical
thinking, unusual perceptual experiences, odd or eccentric behavior,
odd speech, suspiciousness, and excessive social anxiety.
Raine et al. (1994) grouped the 9 subscales into 3 higherorder
domains: cognitiveperceptual, interpersonal, and disorganized.
Subsequent exploratory and confirmatory factor analyses have
suggested this latent structure has acceptable fit (Badcock & Dragović,
2006; Chen, Hsiao, & Lin, 1997; Claridge et al., 1996; Reynolds, Raine,
Mellingen, Venables, & Mednick, 2000; Rossi & Daneluzzo, 2002). In
addition, findings with the 3factor model suggested measurement
invariance across some cultures, religious affiliations, psychopathy,
and sex (Reynolds et al., 2000). In general, women score higher on
the cognitiveperceptual dimension and men score higher on the
negative and disorganized dimensions (Fossati, Raine, Carretta,
Leonardi, & Maffei, 2003; Raine, 2006).
The authors declare that there are no conflict of interests regarding the
publication of this paper. No funding was received for this project.
Received: 7 October 2016 Revised: 18 April 2017 Accepted: 5 June 2017
DOI: 10.1111/appy.12293
AsiaPacific Psychiatry. 2017;e12293.
https://doi.org/10.1111/appy.12293
© 2017 John Wiley & Sons Australia, Ltdwileyonlinelibrary.com/journal/appy 1of7
However, structural fit indices of the 3factor model through
confirmatory factor analysis (CFA) in some studies have been
problematic (eg, Bora & Baysan Arabaci, 2009). This has led some
scholars to suggest that the SPQ may be better suited to a 4factor
structure. For example, Stefanis et al. (2004) proposed a 4factor model
comprising cognitiveperceptual, paranoid, negative, and disorganized
dimensions. Improved fit of this structure over alternative solutions,
including the 3factor structure, has since been obtained in multiple
populations (Bora & Arabaci, 2009; Cicero, 2016; Compton, Goulding,
Bakeman, & McClureTone, 2009; FonsecaPedrero et al., 2014).
The SPQ has also been translated into several languages, including
Chinese (Chen et al., 1997), Turkish (Şener, Bora, Tekin, & Özaşkınlı,
2006), Spanish (Fumero, Santamaría, & Navarrete, 2009), and Greek
(Stefanis et al., 2004). One limitation of previous research is the fact
that very few translational studies have examined fit of the proposed
4factor model of the SPQ. For example, recent findings with the
SPQ and SPQBrief (Raine & Benishay, 1995) have favored a 3factor
solution with Chinese samples (Ma et al., 2015; Yu, Bernardo, & Zaroff,
2015), but in each of these cases the fit of the 4factor model has not
been investigated. However, while the 3factor model has been shown
to provide adequate fit, it is possible that the 4factor model provides
better fit and should therefore be favored.
A second limitation of previous work is that the fact that there has
been limited work across samples varying in ethnicity. Establishing
crossethnic measurement invariance is important because variability
in the dimensionality of the SPQ may limit betweengroup score
comparisons. Recent measurement invariance evidence for the SPQ
has been found with multiethnic Greek (Tsaousis, Zouraraki,
Karamaouna, Karagiannopoulou, & Giakoumaki, 2015) and Pacific
Islander (Cicero, 2016). Further, Barron, Swami, Towell, Hutchinson,
and Morgan (2015) found evidence of measurement invariance for
the SPQ with a UKbased British White population and an African
Caribbean sample resident in Trinidad. This work provides
measurement invariance support across both ethnicity and cultural
settings, but much more work is needed in this regard.
Here, we examined the higherorder factorial structure of the SPQ in
an ethnically diverse Malaysian sample. There are a number of reasons
why this is important and meaningful. First, clinical schizotypal research
is limited in Malaysia, with an absence of nonclinical personality
dimensional research (Chee & Salina, 2014). Therefore, translation of
the SPQ into Bahasa Malaysia (Malay) will help stimulate schizotypal
research in a previously neglected national context. As high ratings of
schizotypy may constitute a risk factor for schizophrenia and other
psychotic illnesses, this would not only add to the limited psychosis
literature in this population but, through the identification of people at
risk for psychosis, be of substantial clinical benefit and contribute to real
improvements in quality of life (Razali & Yahya, 1995; Salleh, 1994).
Second, Malaysia is ethnically heterogeneous, with a Malay
majority and large ethnic minority populations of Chinese, Indians,
and other indigenous ethnic groups (Department of Statistics Malaysia,
2010). This provides an opportunity to examine the factor structure of
the SPQ in different ethnic groups within the same national context
(as opposed to different ethnic groups in different national contexts;
eg, Barron et al., 2015). Furthermore, by examining latent mean
comparisons between ethnic groups, we are able to examine the
extent to which the phenomenology of schizotypy may be influenced
by ethnocultural factors. For example, Malays are more likely than Chi-
nese in Malaysia to believe in supernatural agents (eg, ghosts, demons,
and possession by spirits) as precursors of mental illness (Razali, Khan,
& Hasanah, 1996; Swami, Furnham, Kannan and Sinniah, 2008). There-
fore, we hypothesized that Malays will have higher SPQ ratings than
Chinese, emphasized particularly in the positive domains. Further, we
expected women to score higher on the positive dimension and men
to score higher on the negative and disorganized dimensions.
2|METHOD
2.1 |Participants
There were 382 undergraduate participants; 195 (51%) Malay and 187
(49%) Chinese individuals recruited from a national university in Kuala
Lumpur, the capital city of Malaysia. Kuala Lumpur is ethnically hetero-
geneous, with large groups of Malays and Chinese making up just
under 90% of the estimated 7 million residents of the metropolitan
population. The mean age of participants was 20.47 years (SD = 1.46)
for the Malay subsample with 124 (63.6%) women and 71 (36.4%)
men, and 20.69 years (SD = 1.40) for the Chinese subsample with
108 (57.8%) women and 79 (42.2%) men. There was no significant
difference in age between subsamples, t
380
= 1.60, P= .111,
d= 0.16. All participants selfreported as not having a history of mental
health problems relating to psychosis.
2.2 |Measures
2.2.1 |Schizotypal Personality Questionnaire
The SPQ (Raine, 1991) is a selfreport scale consisting of 74 items, with
a dichotomous response format (Yes/No). The subscales measured by
the SPQ are ideas of reference, excessive social anxiety, odd beliefs
or magical thinking, unusual perceptual experience, odd or eccentric
behavior, no close friends, odd speech, constricted affect, and suspi-
ciousness. Each Yes response counts as 1 point and subscale scores
were computed as the total score for all items associated with each
subscale. Therefore, the present study did not evaluate the observed
lowerorder domains. Instead, and similar to previous SPQ factorial
investigations (eg, Cicero, 2016), the latent, higherorder factors were
evaluated. The 2 most prominent models proposed in the literature,
the 3factor (Raine et al., 1994) and 4factor (Stefanis et al., 2004),
were assessed (see Figure 1 for latent factorial structure).
2.3 |Procedure
Ethics approval for this study was obtained from all participating
institutions' ethics committees. The SPQ was translated from English
into Bahasa Malaysia (Malay) using the backtranslation technique
(for details, see Brislin, 1970). Survey package dissemination was
undertaken via a paperandpencil format in classroom settings. All
participants were invited to complete the anonymous survey material
voluntarily and, to minimize risk of coercion, participants were told that
they were free not to participate with no risk to their grades. Two
research assistants and the fourth author ensured that all participants
2of7 BARRON ET AL.
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
completed the survey materials individually. All participants took part
in the study on a voluntary basis with no monetary incentives offered
to the participants for completion of the survey. All participants pro-
vided written informed consent and were verbally debriefed once they
had returned their questionnaires.
2.4 |Data analysis
Confirmatory factor analyses were conducted using Analysis of
Moment Structures (AMOS 21; Arbuckle, 2012) to examine the factor
structure of the SPQ. Confirmatory factor analysis was deemed appro-
priate for use as the sample size was above the accepted 200
(Boomsma & Hoogland, 2001), and there were at least 5 participants
per model parameter for both models under examination (Bentler &
Chou, 1987). Standard goodnessoffit indices were selected a priori
to assess the measurement models. The normed model chisquare
(χ
2normed
) is reported with lower values of the overall model chisquare
indicating goodnessoffit. Good fit cutoff metric recommendations for
χ
2normed
range from 5.0 (Wheaton, Muthen, Alwin, & Summers, 1977)
to 2.0 (Tabachnick & Fidell, 2013). The SteigerLind root mean square
error of approximation (RMSEA) and its 90% confidence interval pro-
vide a correction for model complexity. The RMSEA values close to
.06 indicate good fit, with values ranging to .10 representing mediocre
fit (Hu & Bentler, 1999). The standardized root mean square residual
(SRMR) assesses the mean absolute correlation residual and is a bad-
nessoffit index: The smaller the SRMR, the better the model fit. A
cutoff value for SRMR is recommended to be close toor < .09
(Hu & Bentler, 1999, p. 27). The comparative fit index (CFI) measures
the proportionate improvement in fit by comparing a target model with
a more restricted, nested baseline model. The CFI reflects a goodness
offit index and is recommended to close toor > .95 for adequate fit
(Hu & Bentler, 1999, p. 27). The Akaike information criterion (AIC)
provides a measure to compare nonhierarchical factor structures, with
the lowest AIC value being preferred. Even so, these recommended cut-
off values should be considered subjective guidelines (Heene, Hilbert,
Draxler, Ziegler, & Bühner, 2011; Marsh et al., 2011). To determine if
the bestfitting model was invariant across ethnicity, we tested for
invariance at the configural (ie, whether an unconstrained model had
adequate fit for both ethnic groups individually), metric (ie, whether
the magnitude of factor loadings is the same; lack of significance
between unconstrained and fully constrained models is indicative of
metric invariance), and scalar (ie, whether the intercept of the regres-
sion relating each item to its factor is the same) levels (Chen, 2007).
For scalar invariance, significant Δχ
2
values (P< .008) and model fit
changes (ie, ΔCFI .010 and ΔRMSEA .015 or ΔSRMR .010) indicate
intercept noninvariance (Chen, 2007). Finally, a multivariate analysis of
variance was used to examine sex and ethnicity differences with the
domains for the model of best fit.
3|RESULTS
3.1 |Data quality analysis
Internal consistency for each of the 9 subscales was assessed using
Cronbach alpha. A Cronbach alpha coefficient of between .65 and
.70 is considered as minimally acceptable, .70 to .80 as respectable,
and >.80 as reflecting very good internal reliability (DeVellis, 2012).
As one subscale (odd beliefs or magical thinking) fell below .65 and 2
subscales (ideas of reference and unusual perceptual experiences)
had questionable reliability, item deletion was implemented to improve
alpha. Table 1 shows itemrest correlations (internal item conver-
gence), which should be above .30 (Field, 2005), and the change in
alpha if items are deleted. Five items were deleted in total from the
SPQ (ideas of reference, item no. 19; odd beliefs or magical thinking,
item nos. 3, 39, and 47; unusual perceptual experiences, item no. 40).
Table 2 shows the revised metrics following item deletion and
Cronbach alpha for the remaining subscales. All itemrest correlations
were above .30 except item no. 10, but as deleting this item had min-
imal improvement in overall reliability for this subscale, it was not
FIGURE 1 The measurement models under examination: A, The 4factor model (Stefanis et al., 2004). B, The 3factor model (Raine et al., 1994).
Factors: Cog P, cognitiveperceptual; Pn, paranoid; Neg, negative; Dis, disorganized; Int P, interpersonal. Subscales: OboMT, odd beliefs or magical
thinking; UPE, unusual perceptual experiences; IoR, ideas of reference; Sus, suspiciousness; ESA, excessive social anxiety; NCF, no close friends;
CA, constricted affect; OoEB, odd or eccentric behavior; OS, odd speech
BARRON ET AL.3of7
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
removed. Further, while deleting item no. 30 marginally improves reli-
ability for the odd beliefs or magical Thinking subscale, it was not
removed as Tabachnick and Fidell (2013) recommend no subscale
should have fewer than 3 items.
3.2 |Confirmatory factor analysis
First, CFA was conducted where all lowerorder domains loaded onto
the multidimensional 4factor model proposed by Stefanis et al.
(2004). This model had moderatetogood fit: χ
2
(19,
N = 382) = 73.619, χ
2normed
= 3.875, CFI = .961, RMSEA = .087 with
90% CI = .066 to .108, SRMR = .058, AIC = 125.619. Next, CFA was
conducted on Raine et al. (1994) proposed multidimensional 3factor
model. Fit indices were χ
2
(23, N = 382) = 96.000, χ
2normed
= 4.174,
CFI = .948, RMSEA = .091 with 90% CI = .073 to .111, SRMR = .064,
AIC = 140.000. This model fits the data adequately well, but to a lesser
degree than that of the 4factor solution across all metrics. Using the
AIC of the models as a comparative measure of fit, the Stefanis et al.
(2004) model had the better fit. Therefore, the present data appear
to be better suited to the Stefanis et al. (2004) 4factor model,
although the 3factor model also had adequate fit.
As there were over the recommended 100 observations per ethnic
subsample (Kline, 2015) to test for measurement invariance across
ethnicity, we performed multiplegroup analyses with the bestfitting,
4factor model. The unconstrained model had adequate fit for both
ethnic subsamples individually, χ
2
(38, N = 382) = 105.550,
χ
2normed
= 2.778, CFI = .953, RMSEA = .068 with 90% CI = .053 to
.084, SRMR = .051 (see Table 3), suggesting configural invariance
between the subsamples. Differences between the unconstrained
and fully constrained model were not significant, indicating that the
structure of the model achieved metric invariance across ethnicity,
Δχ
2
(2) = 11.240, P= .339. Finally, scalar invariance was evaluated,
where all itemfactor intercepts were constrained equally across
ethnicity and evaluated against the factor loading invariance model.
However, according to the changes to the fit indices and Δχ
2
, intercept
invariance was not supported, Δχ
2
(9) = 32.528, P< .001. To identify
TABLE 1 Correlations between items and the rest of items in its own
scale and Cronbach αof domains if item is deleted
Itemrest
a
αif deleted
b
α
Ideas of Reference .66
1 .264 .638
10 .223 .646
19 .179 .659
28 .408 .605
37 .350 .619
45 .279 .635
53 .349 .619
60 .459 .592
63 .442 .596
Odd beliefs or magical thinking .64
3 .161 .648
12 .485 .545
21 .554 .517
30 .260 .623
39 .251 .626
47 .197 .636
55 .490 .544
Unusual perceptual experiences .68
4 .453 .627
13 .426 .634
22 .339 .654
31 .339 .654
40 .031 .697
48 .326 .656
56 .381 .645
61 .477 .623
64 .322 .657
a
Itemrest, itemrest correlation between item and sum of the other items
in its own domain.
b
αif deleted, the values of the overall alpha if item is not in the calculation.
TABLE 2 Modified subscales, reflecting itemrest correlations,
Cronbach αof domains if item is deleted, and Cronbach αof all
domains
Itemrest
a
αif deleted
b
α
Ideas of Reference .66
1 .320 .641
10 .236 .659
28 .395 .622
37 .351 .633
45 .308 .643
53 .339 .636
60 .470 .601
63 .404 .619
Odd beliefs or magical thinking .68
12 .501 .583
21 .542 .553
30 .316 .691
55 .485 .594
Unusual perceptual experiences .69
4 .453 .648
13 .427 .655
22 .335 .675
31 .351 .673
48 .301 .683
56 .372 .668
61 .508 .636
64 .323 .678
Excessive social anxiety ––.83
Odd or eccentric behavior ––.82
No close friends ––.76
Odd speech ––.75
Constricted affect ––.76
Suspiciousness ––.79
The hyphens are acknowledgment that the box is deliberately blank.
a
Itemrest, itemrest correlation between item and sum of the other items
in its own domain.
b
αif deleted, the values of the overall alpha if item is not in the calculation.
4of7 BARRON ET AL.
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
which intercepts were not invariant, modification indices were
assessed. Examination of the modification indices revealed that the
significant increase in χ
2
value and fit indices was due to a lack of scalar
invariance of 2 indicators: Unusual Perceptual Experiences and
Suspiciousness. Relaxing these 2 constraints yielded substantial and
statistically significant improvement in fit as compared to the full scalar
invariance model, Δχ
2
(2) = 20.515, P< .001. This partial scalar
invariance model was then evaluated against the full metric invariance
model. With the changes to the fit indices and Δχ
2
, partial intercept
invariance was evident, Δχ
2
(7) = 12.013, P= .100. Full metric/scalar
invariance is not necessary for further tests of invariance and
substantive analysis (eg, latent factor comparisons) to be meaningful,
provided that at least one item (other than the one fixed at unity to
define the scale of each latent construct) is invariant (Byrne, Shavelson,
& Muthén, 1989; Hong, Malik, & Lee, 2003). Thus, we continued with
the analyses on the basis of partial scalar invariance.
3.3 |Betweengroup differences
We investigated sex and ethnicity differences in scoring with the 4
domains of the bestfitting model. A 2way multivariate analysis of
variance was conducted, with the 4 dimension scores as dependent
variables and sex (women versus men) and ethnicity (Malay versus
Chinese), respectively, as independent variables. The ethnicity × sex
interaction was significant, F
4375
= 2.53, P= .040, Wilk Λ= .97,
η
p2
= .03. A series of followup oneway analyses of variance
(ANOVAs) indicated a significant difference in the cognitivepercep-
tual domain only, F
1378
= 5.77, P= .017, η
p2
= .02. Within this
domain, using Bonferronicorrected (α= .05/6 = .008) independent
samples ttests, female Malay participants had significantly higher
scores than female Chinese, t
230
= 3.39, P< .001, d= 0.45, male
Malay participants, t
193
= 3.48, P< .001, d= 0.52, and male
Chinese participants, t
201
= 2.93, P= .004, d= 0.42. There were
no further statistically significant comparisons within this domain
(all Ps > .008).
A statistically significant omnibus main effect was obtained for
ethnicity, F
4375
= 3.23, P= .013, Wilk Λ= .97, η
p2
= .03. Followup
1way ANOVAs indicated a significant difference in the paranoid
domain only, with Malay participants (M = 13.81, SD = 5.22) having
significantly higher scores than Chinese participants (M = 12.13,
SD = 6.19), F
1380
= 6.92, P= .009, η
p2
= .02. However, the effect size
of this difference was small. There was also a significant omnibus main
effect of sex, F
4375
= 14.41, P< .001, Wilk Λ= .87, η
p2
= .13. Oneway
ANOVAs indicated a significant sex differences in the disorganized,
cognitiveperceptual, and negative domains and total schizotypy (see
Table 4).
4|DISCUSSION
This study showed that the SPQ is suited to preexisting factor struc-
tures for use with a multiethnic Malayspeaking sample. The 3factor
solution (Raine et al., 1994) had adequate fit for the present data, but
Stefanis et al. (2004) 4factor structure had better fit. The latter
retained all latent domains (disorganized, cognitiveperceptual,
paranoid, and negative) as reported by Stefanis et al. (2004), though
a number of lowerorder items were removed to improve internal
consistency. Our support for a 4factor structure is consistent with
findings in Western sites (eg, Compton et al., 2009), but data from
nonWestern sites have been limited to primarily evaluating 1or
3factor solutions (eg, Yu et al., 2015). Importantly, measurement
invariance for the model of best fit was achieved in the Malaysian
sample for both Malay and Chinese adults.
Regarding group differences at the latent mean level, there was a
significant ethnicity by sex interaction, with female Malay participants
having significantly higher scores than male and female Chinese and
male Malay participants on the cognitiveperceptual domain. One
possible explanation for this finding is that there are basic differences
in personality that put Malay women at higher risk of schizotypy along
the cognitiveperceptual domain. For example, there is some evidence
that, in the Malaysian context, Malay women in particular have very
high scores on dimensions of selfeffacement (Abdullah, 1993),
selfconsciousness (McCrae & Terracciano, 2005), indecisiveness
(Swami, Sinniah, et al., 2008), and possibly lower scores on selfesteem
(Swami, 2012). Differences in such endogenous traits that emerge
within culturally circumscribed environments may account for the
present findings.
Regarding ethnocultural influences, we hypothesized that Malay
participants would have significantly higher scores than Chinese
TABLE 3 Model fit indices and tests of measurement invariance for the 4factor structure of the SPQ across ethnicity
Model χ
2
Mdf Mχ
2normed
RMSEA (90% CI) SRMR CFI
Malay (n = 195) 37.228 19 1.959 .070 (.036, .104) .051 .967
Chinese (n = 187) 68.318 19 3.596 .118 (.089, .149) .072 .943
Configural invariance 105.550 38 2.778 .068 (.053, .084) .051 .953
Metric invariance 117.813 48 2.454 .062 (.048, .076) .070 .952
Scalar invariance 150.341 57 2.638 .066 (.053, .079) .068 .935
Partial scalar invariance 129.826 55 2.360 .060 (.047, .073) .069 .948
Abbreviations: CFI, comparative fit index; RMSEA, root mean square error of approximation; SPQ, Schizotypal Personality Questionnaire; SRMR, standard-
ized root mean square residual.
TABLE 4 Intrapopulation univariate analyses of variances of domains
in a 4factor structure
Domains
Men Mean,
SD
Women
Mean, SD FdfPη
p2
Cognitiveperceptual 4.04 (3.34) 4.94 (2.94) 7.11 1 .008 .02
Negative 16.81 (9.35) 13.35 (6.40) 19.00 1 <.001 .05
Disorganized 6.87 (4.70) 5.14 (3.28) 17.68 1 <.001 .05
Paranoid 13.40 (6.88) 12.69 (4.91) 1.81 1 .179 .01
BARRON ET AL.5of7
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
participants, particularly in the positive domains. Indeed, this was
reflected in the paranoid but not the cognitiveperceptual domain.
The former finding may reflect the extent to which the phenomenol-
ogy of schizotypy is influenced by Malays' greater tendency to believe
in supernatural agents as precursors of mental illness (Razali et al.,
1996; Swami, Furnham, et al., 2008). Further, sex differences within
the domains reflected previous findings (Bora & Baysan Arabaci,
2009; Fossati et al., 2003; Raine, 2006), with women scoring
significantly higher on the cognitiveperceptual domain and men
scoring significantly higher on the negative and disorganized domains.
Limitations of this work should be recognized. First, while our
sample contained the 2 largest ethnic groups in Malaysia, findings here
may not be representative of other ethnic groups in Malaysia, such as
Malaysian Indians. Further, our sample consisted of undergraduate
students, thus potentially limiting generalizability to the general
Malaysian population. For example, Zhang and Brenner (2016)
highlighted the importance of using both community and undergradu-
ate samples when examining the factor structure of the SPQ, as
bestfitting factor solutions may diverge between community and
student samples even within the sample cultural context. Therefore,
future work in developing the SPQ, and by extension schizotypal
research, in Malaysia should aim to include more heterogeneous
sampling, particularly from community settings. Further, through our
data quality analysis, a number of the items were identified as having
poor internal consistency within the lowerorder factors and had to
be removed. While this removal procedure strengthened the reliability
of our subscales, the potential for crosssample comparisons is
weakened with item removal. Future work with a larger sample
would allow for the exploration of items with an exploratorythen
confirmatory approach at item level. While this could potentially
move the SPQ away from a schizotypal personality disorder
symptomatology basis, the benefit would be a reliable measurement
tool for Malayspeaking populations.
In conclusion, the present study suggests that both the 3and
4factor models of the SPQ have adequate fit in a Malaysian student
sample. Of the 2 models, however, the 4factor model had better fit
incides and should be favored by scholars working with Malay
speaking samples. It is hoped that stimulation of nonclinical
schizotypal personality research, clinical schizotypal research, and
by extension schizophrenia research, will be further generated in
Malaysia. With regards to the SPQ, further refinement of the
structure and knowledge regarding the SPQ, in Malaysia and in other
different national settings, will continue to advance this measure-
ment tool, allowing it to be used in community studies and in parallel
with endophenotypes for the early detection of schizophrenia.
REFERENCES
Abdullah, M. M. (1993). Konsep malu dan segan orang Melayu [Self
consciousness and the shame concept among Malay people]. In A.
H. Othman (Ed.), Psikologi melayu (pp. 244294). Kuala Lumpur,
Malaysia: Dewan Bahasa dan Pustaka.
Arbuckle, J. (2012). AMOS 21 reference guide. Meadville, PN: Amos
Development Corporation.
Badcock, J. C., & Dragović, M. (2006). Schizotypal personality in mature
adults. Personality and Individual Differences,40,7785. https://doi.
org/10.1016/j.paid.2005.06.015
Barron, D., Swami, V., Towell, T., Hutchinson, G., & Morgan, K. D. (2015).
Examination of the factor structure of the Schizotypal Personality
Questionnaire among British and Trinidadian adults. BioMed Research
International.2015, article ID 258275. doi:https://doi.org/10.1155/
2015/258275
Bentler, P. M., & Chou, C. (1987). Practical issues in structural modeling.
Sociological Methods and Research,16,78117. https://doi.org/
10.1177/0049124187016001004
Boomsma, A., & Hoogland, J. J. (2001). The robustness of LISREL modeling
revisited. In R. Cudeck, S. du Toit, & D. Sörbom (Eds.), Structural equa-
tion models: Present and future. A festschrift in honor of Karl Jöreskog
(pp. 139168). Scientific Software International: Chicago, IL.
Bora, E., & Arabaci, L. (2009). Confirmatory factor analysis of schizotypal
personality traits in university students. Turkish Journal of Psychiatry,
20, 339345.
Bora, E., & Baysan Arabaci, L. (2009). Effect of age and gender on
schizotypal personality traits in the normal population. Psychiatry and
Clinical Neurosciences,63, 663669. https://doi.org/10.1111/j.1440
1819.2009.02011.x
Brislin, R. W. (1970). Backtranslation for crosscultural research. Journal of
CrossCultural Psychology,1, 185216. https://doi.org/10.1177/
135910457000100301
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equiva-
lence of factor covariance and mean structures: The issue of partial
measurement invariance. Psychological Bulletin,105, 456466.
https://doi.org/10.1037/00332909.105.3.456
Chee, K. Y., & Salina, A. A. (2014). A review of schizophrenia research in
Malaysia. Medical Journal of Malaysia,69,4654.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measure-
ment invariance. Structural Equation Modeling,14, 464504. https://
doi.org/10.1080/10705510701301834
Chen, W. J., Hsiao, C. K., & Lin, C. C. (1997). Schizotypy in community sam-
ples: the threefactor structure and correlation with sustained
attention. Journal of Abnormal Psychology,106, 649654. https://doi.
org/10.1037/0021843X.106.4.649
Cicero, D. C. (2016). Measurement invariance of the Schizotypal Personal-
ity Questionnaire in Asian, Pacific Islander, White, and multiethnic
populations. Psychological Assessment,28, 351361. https://doi.org/
10.1016/j.psychres.2011.05.021
Claridge, G., McCreery, C., Mason, O., Bentall, R., Boyle, G., Slade, P., &
Popplewell, D. (1996). The factor structure of schizotypaltraits: A
large replication study. British Journal of Clinical Psychology,35, 103
115. https://doi.org/10.1111/j.20448260.1996.tb01166.x
Compton, M. T., Goulding, S. M., Bakeman, R., & McClureTone, E. B.
(2009). Confirmation of a fourfactor structure of the Schizotypal Per-
sonality Questionnaire among undergraduate students. Schizophrenia
Research,111,4652. https://doi.org/10.1016/j.schres.2009.02.012
Department of Statistics Malaysia. (2010). Population distribution and basic
demographic characteristic report 2010. Putrajaya, Malaysia: Depart-
ment of Statistics.
DeVellis, R. F. (2012). Scale development: theory and applications. London:
Sage Publications.
Field, A. (2005). Discovering statistics with SPSS. London: Sage Publications.
FonsecaPedrero, E., Fumero, A., Paino, M., de Miguel, A., OrtuñoSierra, J.,
LemosGiráldez, S., & Muñiz, J. (2014). Schizotypal Personality Ques-
tionnaire: New sources of validity evidence in college students.
Psychiatry Research,219, 214220. https://doi.org/10.1016/j.
psychres.2014.04.054
Fossati, A., Raine, A., Carretta, I., Leonardi, B., & Maffei, C. (2003). The
threefactor model of schizotypal personality: Invariance across age
and gender. Personality and Individual Differences,35, 10071019.
https://doi.org/10.1016/S01918869(02)003148
Fumero, A., Santamaría, C., & Navarrete, G. (2009). Predisposición al
consumo de alcohol y drogas en personas vulnerables a la esquizofrenia
6of7 BARRON ET AL.
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
[Predisposition to alcohol and drugs in people vulnerable to schizophre-
nia]. Revue Neurologique,49,812.
Heene, M., Hilbert, S., Draxler, C., Ziegler, M., & Bühner, M. (2011). Masking
misfit in confirmatory factor analysis by increasing unique variances: A
cautionary note on the usefulness of cutoff values of fit indices. Psycho-
logical Methods,16, 319336. https://doi.org/10.1037/a0024917
Hong, S., Malik, M. L., & Lee, M.K. (2003). Testing configural, metric, scalar,
and latent mean invariance across genders in sociotropy and autonomy
using a nonWestern sample. Educational and Psychological Measure-
ment,63, 636654. https://doi.org/10.1177/0013164403251332
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance
structure analysis: Conventional criteria versus new alternatives.
Structural Equation Modeling,6,155. https://doi.org/10.1080/
10705519909540118
Kline, R. B. (2015). Principles and practice of structural equation modeling.
New York, NY: Guilford.
Ma, W.F., Lane, H.Y., Chiang, L.C., Wu, P.L., Yang, S.J., & Tsai, G. E.
(2015). Assessing the construct validity of the Chineseversion
Schizotypal Personality QuestionnaireBrief on male and female under-
graduate students. Journal of Nursing Research,23, 162166. https://
doi.org/10.1097/JNR.0000000000000068
Marsh, H. W., Nagengast, B., Morin, A. J., Parada, R. H., Craven, R. G., &
Hamilton, L. R. (2011). Construct validity of the multidimensional struc-
ture of bullying and victimization: An application of exploratory
structural equation modeling. Journal of Educational Psychology,103,
701732. https://doi.org/10.1037/a0024122
McCrae, R. R., & Terracciano, A. (2005). Personality profiles of cultures:
Aggregate personality traits. Journal of Personality and Social Psychology,
89, 407425. https://doi.org/10.1037/00223514.89.3.407
Nelson, M. T., Seal, M. L., Pantelis, C., & Phillips, L. J. (2013). Evidence of a
dimensional relationship between schizotypy and schizophrenia: A
systematic review. Neuroscience and Biobehavioral Reviews,37,
317327. https://doi.org/10.1016/j.neubiorev.2013.01.004
Raine, A. (1991). The SPQ: A scale for the assessment of schizotypal
personality based on DSMIIIRcriteria. Schizophrenia Bulletin,17,
555564. https://doi.org/10.1093/schbul/17.4.555
Raine, A. (2006). Schizotypal personality: Neurodevelopmental and psycho-
social trajectories. Annual Review of Clinical Psychology,2, 291326.
https://doi.org/10.1146/annurev.clinpsy.2.022305.095318
Raine, A., & Benishay, D. (1995). The SPQB: A brief screening instrument
for schizotypal personality disorder. Journal of Personality Disorders,9,
346355. https://doi.org/10.1521/pedi.1995.9.4.346
Raine, A., Reynolds, C., Lencz, T., Scerbo, A., Triphon, N., & Kim, D. (1994).
Cognitiveperceptual, interpersonal, and disorganized features of
schizotypal personality. Schizophrenia Bulletin,20, 191201. https://
doi.org/10.1093/schbul/20.1.191
Razali, M. S., & Yahya, H. (1995). Compliance with treatment in schizophre-
nia: A drug intervention program in a developing country. Acta
Psychiatrica Scandinavica,91, 331335. https://doi.org/10.1111/
j.16000447.1995.tb09790.x
Razali, S. M., Khan, U. A., & Hasanah, C. I. (1996). Belief in supernatural
causes of mental illness among Malay patients: Impact on treatment.
Acta Psychiatrica Scandinavica,94, 229233. https://doi.org/10.1111/
j.16000447.1996.tb09854.x
Reynolds, C. A., Raine, A., Mellingen, K., Venables, P. H., & Mednick, S. A.
(2000). Threefactor model of schizotypal personality: Invariance across
culture, gender, religious affiliation, family adversity, and
psychopathology. Schizophrenia Bulletin,26, 603618. https://doi.org/
10.1093/oxfordjournals.schbul.a033481
Rossi, A., & Daneluzzo, E. (2002). Schizotypal dimensions in normals and
schizophrenic patients: A comparison with other clinical samples.
Schizophrenia Research,54,6775. https://doi.org/10.1016/S0920
9964(01)00353X
Salleh, M. R. (1994). The burden of care of schizophrenia in Malay families.
Acta Psychiatrica Scandinavica,89, 180185. https://doi.org/10.1111/
j.16000447.1994.tb08089.x
Şener, A., Bora, E., Tekin, I., & Özaşkınlı, S. (2006). Şizotipal Kişilik Ölçeğinin
üniversite öğrencilerinde geçerlik ve güvenirliği [The reliability and
validity of Schizotypal Personality Questionnaire in college students].
Klinik Psikofarmakoloji Bülteni,16,8492.
Stefanis, N. C., Smyrnis, N., Avramopoulos, D., Evdokimidis, I., Ntzoufras, I.,
& Stefanis, C. N. (2004). Factorial composition of selfrated schizotypal
traits among young males undergoing military training. Schizophrenia
Bulletin,30, 335350.
Swami, V. (2012). Further examination of the psychometric properties of
the Malay Rosenberg SelfEsteem Scale. In S. de Wals, & K. Meszaros
(Eds.), Handbook on the psychology of selfesteem (pp. 371380).
Hauppauge, NY: Nova Science Publishers.
Swami, V., Furnham, A., Kannan, K., & Sinniah, D. (2008). Beliefs about
schizophrenia and its treatment in Kota Kinabalu, Malaysia. Interna-
tional Journal of Social Psychiatry,54, 164179. https://doi.org/
10.1177/0020764007084665
Swami, V., Sinniah, D., Subramaniam, P., Pillai, S. K., Kannan, K., &
ChamorroPremuzic, T. (2008). An exploration of the indecisiveness
scale in multiethnic Malaysia. Journal of CrossCultural Psychology,39,
309316. https://doi.org/10.1177/0022022108315544
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics. Boston,
MA: Pearson Education.
Tsaousis, I., Zouraraki, C., Karamaouna, P., Karagiannopoulou, L., &
Giakoumaki, S. G. (2015). The validity of the Schizotypal Personality
Questionnaire in a Greek sample: Tests of measurement invariance
and latent mean differences. Comprehensive Psychiatry,62,5162.
https://doi.org/10.1016/j.comppsych.2015.06.003
Wheaton, B., Muthen, B., Alwin, D., & Summers, G. (1977). Assessing
reliability and stability in panel models. Sociological Methodology,8,
84136. https://doi.org/10.2307/270754
Yu, J., Bernardo, A. B., & Zaroff, C. M. (2015). Chinese version of the
Schizotypal Personality Questionnaire: Factor structure replication
and invariance across sex. AsiaPacific Psychiatry,8, 226237. https://
doi.org/10.1111/appy.12215
Zhang, L. C., & Brenner, C. A. (2016). The factor structure of the Schizotypal
Personality Questionnaire in undergraduate and community samples.
Journal of Personality Disorders,115. https://doi.org/10.1521/pedi_
2016_30_233
How to cite this article: Barron D, Morgan KD, Towell T,
Jaafar JL, Swami V. Psychometric properties of the Malay
Schizotypal Personality Questionnaire: Measurement invari-
ance and latent mean comparisons in Malaysian adults. Asia
Pacific Psychiatry. 2017;e12293. https://doi.org/10.1111/
appy.12293
BARRON ET AL.7of7
bs_bs_banner
Offi cial jour nal of the
Pacifi c Rim Colle ge of Psychi atr ist s
  • ... For instance, we have little information about the factorial structure of SPQ-B scores and its possible variation across countries, particularly non-Western countries. Moreover, as previous studies have demonstrated with the SPQ, alternative models (e.g., Barron et al., 2017;Preti et al., 2015) may better explain the latent structure of SPQ-B scores. Thus, it is important to gather new information about the validity of this tool through cross-cultural research and collaborative multinational studies. ...
    Article
    The Schizotypal Personality Questionnaire-Brief (SPQ-B) was developed with the aim of examining variations in healthy trait schizotypy, as well as latent vulnerability to psychotic-spectrum disorders. No previous study has studied the cross-cultural validity of the SPQ-B in a large cross-national sample. The main goal of the present study was to analyze the reliability and the internal structure of SPQ-B scores in a multinational sample of 28,426 participants recruited from 14 countries. The mean age was 22.63 years (SD = 7.08; range 16-68 years), 37.7% (n = 10,711) were men. The omega coefficients were high, ranging from 0.86 to 0.92 for the total sample. Confirmatory factor analysis revealed that SPQ-B items were grouped either in a theoretical structure of three first-order factors (Cognitive-Perceptual, Interpersonal, and Disorganized) or in a bifactor model (three first-order factors plus a general factor of schizotypal personality). In addition, the results supported configural but not strong measurement invariance of SPQ-B scores across samples. These findings provide new information about the factor structure of schizotypal personality, and support the validity and utility of the SPQ-B, a brief and easy tool for assessing self-reported schizotypal traits, in cross-national research. Theoretical and clinical implications for diagnostic systems, psychosis models, and cross-national mental health strategies are derived from these results.
  • Article
    Full-text available
    Introduction: Schizotypal personality represents genetic underpinning of schizophrenia spectrum disorders; hence, it provides conceptual models for understanding psychosis as well as a scheme for high-risk group identification. The study of structure of schizotypal personality is largely confined to western societies, whereas its assessment in varied socio-cultural groups is highly required. Aim: To study the factor structure of schizotypal personality in an Indian population. Materials and Methods: A sample of 492 college students (age, Mean= 21.3, SD= 2.61) filled the Hindi translated version of 74-items Schizotypal Personality Questionnaire (SPQ). Confirmatory Factor Analysis (CFA) technique was used to test the fitness (consonance of the modeled relationships, among latent and observed variables, in the sampled population) of different schizotypal personality models, that is, two-, three-, four-, bi-, and uni-factor models. The best-fit model was also studied for Measurement Invariance (MI) across gender groups. Results: Three, four, and bi-factor models adequately fitted the data Whereas, four-factor model was the best good-fit model. It also showed partially strong MI across gender groups. The internal consistency of total SPQ was 0.90 and of subscales ranged from 0.62 to 0.78. Men scored higher on several schizotypal facets but lower on social anxiety as compared to women. Conclusion: The factor structure of schizotypal personality in India is similar to that reported elsewhere in the world. Thus, the present study supports the generalisation of schizotypal personality construct to the Indian people.
  • Article
    Full-text available
    The heterogeneity of schizotypal traits, suggested in previous research, was further investigated in a sample of subjects (N = 1095) administered a composite questionnaire consisting of a large number of published scales the majority of which were designed to measure psychotic characteristics. Factor analysis confirmed the four components previously indicated in our work with the same instrument; namely, 'aberrant perceptions and beliefs', 'cognitive disorganization', 'introvertive anhedonia ' and ' asocial behaviour '. This structure was maintained regardless of whether or not the analysis included scales from the Eysenck Personality Questionnaire, which might otherwise have been held to explain the variance. ' Aberrant perceptions and beliefs '-reminiscent of the positive symptoms of schizophrenia-was the strongest component; but given the multidimensional nature of the data, together with the pattern of factor loadings and intercorrelations for the sees involved, it was concluded that the broader term 'psychosis proneness' or 'psychoticism' (in a non-Eysenckian sense) might be a better descriptor of the clinical and personality domain sampled.
  • Article
    The prevailing theoretical model of the Schizotypal Personality Questionnaire (SPQ) is a three-factor model based on subscale-level analyses. However, recent item-level factor analyses of the SPQ suggest a four- or five-factor model. To examine the factor structure of the SPQ and how this structure may differ between undergraduate and community samples, the authors conducted exploratory and confirmatory item-level factor analyses of this measure on undergraduate (N = 1,850) and community participants (N = 1,464). A clear three-factor solution was found in the community sample, whereas a somewhat equivocal four-factor solution was found in the undergraduate sample. Both structures displayed gender invariance. This is the first study to address the issues of undergraduate sample generalizability and gender invariance in an item-level exploratory factor analysis of the SPQ. Given the disparate findings in the samples, this study indicates the importance of using both community and undergraduate samples when examining the factor structure of the SPQ.
  • Article
    Objective: The patients with schizotypal personality disorder and many nonaffected relatives of patients with schizophrenia shares many neurophysiological, neurocognitive and neuroanatomic features of schizophrenia. The concept of schizotypy is useful to investigate the characteristics of liability for schizophrenia. Schizotypy has several distinct dimensions similar to schizophrenia. There is an impact of gender on schizotypal features. This study investigates the psychometric characteristics of Turkish version of Schizotypal Personality Questionnaire (SPQ). Method: The sample of the study consists of 388 medical students. The internal consistency and %10 cut-off scores were studied. Seventy-one students were readministered the scale to measure the test-retest reliability. The Magical Ideation Scale (MIS) was also administered to 86 students besides SPQ with the aim of testing concurrent validity. The factor structure of the scale was also examined. Results: The mean score of the scale was 25.9 and 10% high cut-off score was 42. The Crohnbach alpha coefficient of the scale was 0.91, the Cronbach coefficients for the subscales ranged from 0.66 to 0.83. Test- retest scores were highly correlated (r=0.84). The correlation coefficients of MIS with total score of SPQ and cognitive perceptual schizotypy score were 0.49 and 0.74 respectively. Males had significantly higher scores on the interpersonal schizotypy subscale. Factor analysis of the items and subscales supported the construct validity of the scale. Conclusions: SPQ is a reliable and valid instrument for studying several dimensions of schizotypy in young adults.
  • Chapter
    Full-text available
    Previous studies have not fully examined the psychometric properties of the Malay Rosenberg Self-Esteem Scale (RSES). To overcome this limitation, the present study conducted a thorough examination of the reliability and validity of a newly-developed translation of the RSES into Bahasa Malaysia (Malay). A total of 800 participants from Malaysia completed the Malay RSES, along with scales measuring their satisfaction with life, subjective happiness, Big Five personality factors, and demographics. A sub-sample of 120 participants also completed the RSES again after five weeks. Results showed that the Malay RSES could be reduced to a single dimension, although contrary to previous findings, one item loaded negatively. Internal consistency for the full Malay RSES was adequate and test-retest reliability was good after five weeks. Results also showed that self-esteem was significantly correlated with satisfaction with life, subjective happiness, and the Big Five factors of Extraversion and Neuroticism. These results are discussed in relation to the limitations of previous studies that have developed and used Malay versions of the RSES.
  • Article
    Full-text available
    Introduction: The Schizotypal Personality Questionnaire (SPQ) is a self-report measure assessing symptoms of schizotypy. The SPQ has been used in both normative and clinical samples and has much theoretical and empirical support. A three-factor structure of the SPQ, derived on the basis of work in schizophrenia, consisting of Cognitive-Perceptual, Interpersonal, and Disorganized factors, has been well replicated. The present study aimed to (i) validate this three-factor structure in the Chinese version of the SPQ in a sample of individuals of Chinese ethnicity, and (ii) test for invariance across sex. Methods: A total of 209 (99 males) undergraduate university students (Mage = 19.5, SD = 1.6) were administered the SPQ. Results: Confirmatory factor analysis indicated a better fit between the data and the three-factor model compared with a one-factor model. Multigroup confirmatory factor analysis also found strong measurement invariance across sex. Discussion: The current results add to a growing body of literature evidencing cross-cultural validity of the SPQ and its invariance across sex. Research and clinical implications of the current results are discussed.
  • Article
    Full-text available
    The Schizotypal Personality Questionnaire (SPQ; Raine, 1991) is one of the most commonly used self-report measures of schizotypal personality traits. Previous work has found that the SPQ has a 3- or 4-factor structure, but most of this work was with White participants. Little is known about the psychometric properties of the scale in Pacific Islander populations, and some evidence suggests scores may differ between White and Asian participants. The current study included 398 Asian, 293 White, 159 Pacific Islander, and 308 multiethnic nonclinical participants. A 4-factor model fit the data well, and this factor structure displayed configural and metric invariance, suggesting that the factor structure is the same across these diverse groups. However, results provided mixed evidence for scalar invariance, suggesting the scale may lack scalar invariance in these populations. Follow-up analyses revealed that the questionable scalar invariance was related to the intercepts of the Ideas of Reference and Suspiciousness subscales in the White sample. This suggests that mean comparisons among ethnic groups involving the Ideas of Reference and Suspiciousness subscales are not appropriate. (PsycINFO Database Record (c) 2015 APA, all rights reserved).