Additional services for Psychological Medicine:
Email alerts: Click here
Subscriptions: Click here
Commercial reprints: Click here
Psychoticlike experiences in a community sample of 8000 children aged
9 to 11 years: an item response theory analysis
K. R. Laurens, M. J. Hobbs, M. Sunderland, M. J. Green and G. L. Mould
Psychological Medicine / Volume 42 / Issue 07 / July 2012, pp 1495 1506
DOI: 10.1017/S0033291711002108, Published online: 17 October 2011
Link to this article: http://journals.cambridge.org/abstract_S0033291711002108
How to cite this article:
K. R. Laurens, M. J. Hobbs, M. Sunderland, M. J. Green and G. L. Mould (2012). Psychoticlike experiences in a community
sample of 8000 children aged 9 to 11 years: an item response theory analysis. Psychological Medicine, 42, pp 14951506
Request Permissions : Click here
Downloaded from http://journals.cambridge.org/PSM, IP address: 220.127.116.11 on 21 Jan 2013
Psychotic-like experiences in a community sample of
8000 children aged 9 to 11 years: an item response
K. R. Laurens1,2,3*, M. J. Hobbs4, M. Sunderland4, M. J. Green1,2and G. L. Mould3
1Research Unit for Schizophrenia Epidemiology, School of Psychiatry, University of New South Wales, Sydney, Australia
2Schizophrenia Research Institute, Sydney, Australia
3Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, King’s College London, UK
4Clinical Research Unit for Anxiety and Depression, School of Psychiatry, University of New South Wales, Sydney, Australia
Background. Psychotic-like experiences (PLEs) in the general population are common, particularly in childhood, and
may constitute part of a spectrum of normative development. Nevertheless, these experiences confer increased risk
for later psychotic disorder, and are associated with poorer health and quality of life.
Method. This study used factor analytic methods to determine the latent structure underlying PLEs, problem
behaviours and personal competencies in the general child population, and used item response theory (IRT) to assess
the psychometric properties of nine PLE items to determine which items best represented a latent psychotic-like
construct (PSY). A total of 7966 children aged 9–11 years, constituting 95% of eligible children, completed self-report
Results. Almost two-thirds of the children endorsed at least one PLE item. Structural analyses identified a
unidimensional construct representing psychotic-like severity in the population, the full range of which was well
sampled by the nine items. This construct was discriminable from (though correlated with) latent dimensions
representing internalizing and externalizing problems. Items assessing visual and auditory hallucination-like
experiences provided the most information about PSY; delusion-like experiences identified children at more severe
levels of the construct.
Conclusions. Assessing PLEs during middle childhood is feasible and supplements information concerning
internalizing and externalizing problems presented by children. The hallucination-like experiences constitute
appropriate items to screen the population to identify children who may require further clinical assessment or
monitoring. Longitudinal follow-up of the children is required to determine sensitivity and specificity of the PLE
items for later psychotic illness.
Received 24 May 2011; Revised 31 August 2011; Accepted 6 September 2011; First published online 17 October 2011
Key words: Delusions, development, hallucinations, latent construct, psychopathology, questionnaires, screening,
Hallucinations and delusions are cardinal diagnostic
features of psychotic illness, but epidemiological
studies of population cohorts indicate that psychotic
symptoms are not pathognomonic for psychotic dis-
order. Rather, psychotic symptoms are distributed
along a continuum in the population: lifetime preva-
lence of psychotic disorders in the general population
(3%; Perala et al. 2007) is exceeded by the median
prevalence of subclinical psychotic-like experiences
(PLEs: 5–8%; van Os et al. 2009), with marked vari-
ation across countries (0.8–31%; Nuevo et al. 2010) and
higher rates among youth (van Os et al. 2009).
Nevertheless, subclinical psychotic symptoms or PLEs
in the general population are associated with signifi-
cantly poorer health status (Nuevo et al. 2010) and
lower quality of life (van Os et al. 2000). They also
share genetic, sociodemographic, environmental and
biological risk factors with schizophrenia (Kelleher &
Cannon, 2011). Although psychotic symptoms present
in many conditions during development that do not
progress to psychotic disorder (Arango, 2011), several
prospective cohort studies indicate that juvenile
presentation of psychotic symptoms increases risk for
* Address for correspondence: Dr K. R. Laurens, Department of
Forensic and Neurodevelopmental Sciences (Box P023), Institute of
Psychiatry, King’s College London, De Crespigny Park, London SE5
Psychological Medicine (2012), 42, 1495–1506.
f Cambridge University Press 2011
the development of psychotic disorders in adulthood.
For example, children reporting psychotic symptoms
at age 11 years during diagnostic interview by a clin-
ician had a 16-times greater risk of a schizophreniform
diagnosis at age 26 years (Poulton et al. 2000), and
adolescents self-reporting auditory hallucinations on
questionnaire at age 14 years experienced a two-
(females) to five-times (males) greater risk of non-
affective psychoses at age 21 years (Welham et al.
2009). By contrast, another prospective cohort study
indicated that adolescents’ self-reports of auditory and
visual hallucinations on questionnaire (assessed at age
11–18 years) predicted past-year diagnoses of de-
pressive, anxiety and substance use disorders, but not
psychotic disorders, 8 years later (assessed at age
19–26 years), although sample size and longitudinal
sample attrition rendered the study underpowered to
detect a relationship between hallucinations and later
psychotic disorder (Dhossche et al. 2002). Screening
community samples of children for the presence of
PLEs may thus offer a means of identifying indivi-
duals at risk for psychosis during the pre-prodromal
period, prior to the marked deterioration of function-
ing that immediately precedes transition into illness,
and improve prospects for preventive intervention
(Keshavan et al. 2011).
Methodological issues, particularly study cohort
and design factors, contribute to variation in rates of
psychotic symptoms reported in the general popu-
lation (Linscott & van Os, 2010). Assessment of self-
reported experiences by questionnaire or lay interview
may elicit greater false positives than clinical inter-
view (van Os et al. 2009). However, a previous study
indicated that apparently ‘false-positive’ cases (i.e.
those self-reporting definite or probable PLEs subse-
quently rated as clinically not relevant by clinicians)
nevertheless experienced a 25-fold increased risk of
psychotic disorder after 3 years (Bak et al. 2003).
Arguably, a greater rate of false positives than false
negatives is the preferred outcome of general popu-
lation screening to identify cases that require detailed
clinical assessment and/or closer monitoring over
time. A recent study of a small community sample of
youth without psychotic disorder (aged 11–13 years)
indicated variable criterion validity among seven
self-report PLE questionnaire items in terms of their
specificity, sensitivity and positive (PPP) and negative
predictive power (NPP) for definite psychotic symp-
toms as verified by diagnostic interview (Kelleher et al.
2011). The best-performing screening item, which as-
sessed auditory hallucination-like experiences (‘Have
you ever heard voices or sounds that no-one else could
hear?’; endorsed by y16%), had high PPP and NPP
for any clinically verified psychotic symptom at
interview (100% and 88% respectively). Items that
assessed visual hallucinations (‘Have you ever seen
things other people could not see?’; 82% PPP and
80% NPP) and paranoid thoughts (‘Have you ever
thought that people are following or spying on you?’;
80% PPP and 79% NPP) also performed well. Poorest
performance was observed for ‘Have you ever had
messages sent just to you through TV or radio?’ (40%
PPP and 71% NPP).
In the general adult population, correlated but
separable dimensions (latent constructs) underlie
PLEs, depression and mania (Krabbendam et al. 2004);
and a psychotic-like dimension is distinct from the
well-established internalizing and externalizing psy-
chopathology dimensions (Markon, 2010). During
adolescence, PLEs and depression are co-occurring
phenomena that do not predict one another over time
(Wigman et al. 2011). The aim of the present paper was
to examine the latent construct(s) underlying PLEs in
the general child population, and to determine the
extent to which a psychotic-like construct (PSY) was
differentiable from childhood internalizing and ex-
ternalizing symptom dimensions. The study further
used item response theory (IRT) to determine the
psychometric properties of each of nine self-report
items used previously to screen community samples
of children aged 9–11 years for PLEs (Laurens et al.
2007, 2011). Unlike classical methods, IRT accounts
explicitly for individual variation in latent construct
severity, and in the way each questionnaire item
measures the latent (or unobservable) construct.
Each PLE item was evaluated for how it assessed
(or represented) the latent construct in the general
child population, and indices of the overall function-
ing of the PLE questionnaire were also determined.
Children aged 9–11 years were recruited from 73
primary schools located in the Greater London area
were sampled to span the range of socio-economic
disadvantage represented in London schools (as in-
dexed by the percentage of enrolled children eligible
to receive free school meals), and to encompass both
state and religious schools of varying enrolment size.
A total of 7966 children (49% female) completed
questionnaires, constituting 95% of eligible children
(4% of parents, and 1% of children, refused the child’s
participation; no information was available concern-
ing reason for refusal). The mean age of the child
sample was 10 years 5 months (S.D.=9 months), with
96% of the sample aged between 9.00 and 11.99 years.
School-level data, rather than individual-specific data,
1496K. R. Laurens et al.
were used to index socio-economic status and ethnic
diversity in the sample. On average, 31% of children
(range 2–62%) in participating schools were eligible to
receive free school meals (cf. Greater London average
of 25% and England average of 16%; Office for
National Statistics). As is characteristic of inner-city
London, school enrolment was ethnically diverse:
on average, 25% of children were of white British
ethnicity (cf. Greater London average of 37%; England
average 80%), with the remainder of black (45%),
Asian (11%), other white (9%) or other (10%) ethni-
PLE items included five questions adapted from the
Diagnostic Interview Schedule for Children (Costello
et al. 1982), and an additional four items to assess a
broader range of PLEs (Laurens et al. 2007, 2008, 2011).
These items are reproduced in full in Table 1. Each
PLE item was rated on a three-choice response scale:
‘0=not true’, ‘1=somewhat true’ or ‘2=certainly
Internalizing and externalizing psychopathology
The Strengths and Difficulties Questionnaire (SDQ;
Goodman, 1997) is a widely used screening instru-
ment assessing problem behaviours and personal
competencies during childhood, with established
reliability and validity (Goodman, 2001). The SDQ
comprises 25 items assessing four domains of child-
hood psychopathology [Emotional Symptoms (ES),
Conduct Problems (CP), Hyperactivity-Inattention
(H-I), and Peer Relationship Problems (PRP)] in ad-
dition to personal strengths [Prosocial Behaviour
(PB)]. The five subscales each comprise five items,
with each item rated on a three-choice response scale:
‘0=not true’, ‘1=somewhat true’ or ‘2=certainly
true’. Adequate psychometric properties for the self-
report SDQ have been established down to age 8 years
(Muris et al. 2004). Typically, a five-factor model of the
SDQ corresponding to the five subscales is derived
(Stone et al. 2010), although an alternative three-factor
model comprising internalizing (encompassing the
ES and PRP items), externalizing (encompassing the
CP and H-I items) and PB may be more appropriate
in epidemiological (non-clinical) samples (Goodman
et al. 2010).
Table 1. Descriptive statistics, prevalence (detailed by response option), and item response theory (IRT) parameter estimates for the nine
psychotic-like experience (PLE) items. Items are rank ordered from the most discriminating item to the least (parameter a); item
administration order within the questionnaire is indicated by the item number assigned to each item
prevalences (%) IRT parameters
Mean (S.D.) NTSTCTab1
9. Have you ever seen something or someone that other
people could not see? (seen things)
4. Have you ever heard voices that other people could not
hear? (heard voices)
5. Have you ever felt that you were under the control of some
special power? (controlled)
8. Do you have any special powers that other people don’t
have? (special powers)
7. Have you ever felt as though your body had been changed
in some way that you could not understand? (body
2. Have you ever believed that you were being sent special
messages through the television? (special messages)
3. Have you ever thought that you were being followed or
spied upon? (spied upon)
6. Have you ever known what another person was thinking
even though that person wasn’t speaking? (read minds)
1. Some people believe that their thoughts can be read. Have
other people ever read your thoughts? (thoughts read)
0.83 (0.84)45.3 26.128.6 2.00
0.50 (0.73) 64.6 21.1 14.31.520.571.61
0.63 (0.81) 58.320.920.8 1.380.33 1.29
0.73 (0.80) 49.229.0 21.8 1.29
0.40 (0.67)69.9 19.810.3 1.250.862.14
0.96 (0.80)34.235.8 30.01.23
0.84 (0.77)38.7 38.7 22.61.12
0.52 (0.66)18.104.22.168 0.850.37 2.95
S.D., Standard deviation; NT, not true; ST, somewhat true; CT, certainly true; a, discrimination parameter; b1and b2, difficulty
IRT analysis of child psychotic-like experiences 1497
Children further indicated their sex, and their dates
of birth and questionnaire completion (to derive the
child’s exact age at assessment).
The Joint South London and Maudsley and the
Institute of Psychiatry National Health Service (NHS)
Research Ethics Committee granted ethical permission
for the study. Questionnaires were completed inde-
pendently and anonymously by children in the class-
room, with items read aloud to children by a
researcher. All enrolled children present in class on
the day of questionnaire administration were eligible
to participate, excepting children judged by their class
teacher to be insufficiently proficient in English to
complete questionnaires (y0.3% of enrolled children).
Information concerning the research was issued to
all parents and teachers at least 2 weeks prior to the
research session, allowing sufficient time for children
or parents to withdraw consent for participation in the
A three-tiered method was used to evaluate the
psychometric properties of the PLE items within an
IRT framework. First, the latent structure underlying
the PLE and the SDQ items was assessed. This deter-
mined whether unidimensional (rather than multi-
dimensional) IRT methods were appropriate for
evaluating the PLE items. Furthermore, simultaneous
assessment of the structure of the PLE and the SDQ
items provided an index of the discriminant validity
of the PLE questionnaire. Second, the IRT properties
of each of the PLE items were evaluated to show
how each item assessed a latent PSY. Third, the test
characteristics of the PLE questionnaire were calcu-
lated to provide overall indices of how the question-
naire assessed the latent construct.
Examining the latent structure underlying the PLE and
Using a robust weighted least squares estimator in
the Mplus version 6.0 statistical software package
(Muthe ´n & Muthe ´n, 1998–2010), exploratory and con-
firmatory factor analyses (EFA, CFA) were conducted
to evaluate the latent structure underlying the PLE
and SDQ items. Initially, a quartimin rotator was
used to extract one to six factors to explain the co-
occurrence of the questionnaire items (Jennrich &
Sampson, 1966). The Tucker–Lewis index (TLI; Tucker
& Lewis, 1973) and the comparative fit index (CFI;
Bentler, 1990) were used to assess the relative fit of the
factor models in comparison to a baseline model, and
the root mean square error of approximation (RMSEA;
Steiger, 1990) was used to assess the discrepancy be-
tween the estimated and observed polychoric corre-
lations of the items. Based on the findings of the EFA
and the existing psychometric literature concerning
the SDQ, a series of CFAs were specified. Detail re-
garding the various EFA and CFA models examined
is provided in the Results section and in the online
Determining the IRT parameters of the PLE items
The Samejima (1969) IRT graded response model was
used to evaluate how each of the PLE items assessed
the latent construct underlying these items (PSY). An
example of how the Samejima IRT model describes the
statistical relationship between the child, PSY and a
PLE item is illustrated in Fig. 1. Specific detail of the
likelihood that endorsement of a PLE item will be
associated with particular levels of construct severity
can be gleaned from the relationship plotted in the
figure. IRT uses two main indices to describe this
statistical relationship. The discrimination (or ‘a’)
parameter describes the ability of each of the PLE
items to distinguish between similar degrees of PLE
severity. Two difficulty (or ‘b’) parameters are derived
for the three-response PLE items: b1indexes the point
along the latent construct at which individuals with
latent severity equal to or greater than this point of
severity have a 50% or greater likelihood of selecting
the first response option (‘not true’); b2indexes the
point on the latent construct at which the probability
of endorsing the third (‘certainly true’) response
option is 50%. Multilog version 7.03 (Thissen et al.
2003) was used to calculate the discrimination and
difficulty parameters of each of the PLE items.
Item information functions (IIFs) were derived to
index the precision of each PLE item in capturing
PSY severity, with peaked item information curves
providing greater precision of measurement of the
underlying latent construct than items with flatter
curves. The IIFs were estimated using marginal maxi-
mum likelihood methods (Thissen et al. 2003).
Evaluating the test information function (TIF) for the
The TIF plots the precision of measurement associated
with PSY severity. Larger values on the function indi-
cate levels on the latent construct where more precise
measurement or more information is gathered by
the nine-item scale, whereas smaller values indicate
points on the latent construct where the scale provides
little information or precision of measurement. As a
result, TIFs are generally humped (peaked) curves.
1498K. R. Laurens et al.
The TIF was estimated using marginal maximum
likelihood methods (Thissen et al. 2003).
Prevalence of the PLE item responses
The prevalence of each of the PLE item response cat-
egories is provided in Table 1. The prevalence of a
‘certainly true’ response on each PLE item ranged
from 9% to 35%. Overall, 66% of children reported at
least one ‘certainly true’ response across the nine PLE
items included in the questionnaire.
IRT analysis of the psychometric properties of the
Latent structure underlying the PLE and SDQ items
The model fit indices of the EFAs and CFAs on the
PLE and SDQ items are provided as online sup-
plementary material (Table A1). The EFAs revealed
that four or five latent constructs explain the co-
occurrence of the PLE and SDQ items (eigenvalues:
6.49, 4.10, 2.57, 1.65, 1.19 and 1.07 respectively). On the
basis of the EFA, and guided by the extant psycho-
metric literature on the SDQ (Goodman et al. 2010;
Stone et al. 2010), four CFAs were specified. Although
unlikely to provide an ideal fit for the data, model 1
used a single factor to explain the co-occurrence of the
PLE and SDQ items. This most parsimonious model,
which assumed a single undifferentiated psycho-
pathology construct in the population, was used as the
baseline standard model. Model 2 explained the co-
occurrence of the questionnaire items with reference
to six first-order factors, and provided a better fit of
the data than model 1. This included one factor that
explained the co-occurrence of the PLE items (PSY),
and the remaining factors reflected the established
SDQ subscales (ES, PRP, CP, H-I, PB). However, the
ES and PRP subscales, and the CP and H-I subscales,
exhibited high correlations (0.74 and 0.78 respect-
ively). This implied that two broader dimensions
of psychopathology underlay these SDQ subscales.
Accordingly, two subsequent confirmatory models
were specified: model 3 explained the co-variation
of the ES and PRP subscales, and the CP and H-I sub-
scales, using two correlated hierarchical constructs
that were conceptualized as internalizing and exter-
nalizing psychopathology respectively (Krueger, 1999;
Goodman et al. 2010; Kessler et al. 2011; Lahey et al.
2011). Both constructs covaried with the PSY and
PB dimensions. Finally, model 4 used two first-order
factors, again conceptualized as internalizing and
externalizing psychopathology, to explain the co-
occurrence of their respective SDQ items. Again, these
constructs covaried with the PSY and PB dimensions.
Models 2, 3 and 4 each provided similar and accept-
able fits of the data, according to the definitions of
Hu & Bentler (1998). Based on the parsimony of the
model, and given the magnitude of the relationship
between the hierarchical and first-order latent con-
structs in model 3 (loadings ranged between 0.77 and
0.96), model 4 (CFI=0.86, TLI=0.92, RMSEA=0.05)
was accepted as the best explanation of the covariation
Probability of reporting the response option
Latent construct (θj)
Fig. 1. Category response functions of a hypothetical
psychotic-like experience (PLE) item i that was measured
using three response categories (i1, i2, i3). This item had a
discrimination parameter (a) of 1 and difficulty parameters
(b1and b2) of x1 and 1. The vertical axis of this figure indexes
the likelihood (ranging from 0 to 1) that a child will report
that they have experienced the respective response option
(i.e. ‘not true’, ‘somewhat true’ and ‘certainly true’). The
horizontal axis indexes the distribution of the latent
construct, PSY severity, measured by the PLE items. This
latent psychological construct is typically referred to as theta
(hj), and is measured on a standard normal metric. Each of
the three trace lines represents one of the three response
options that were used to assess the PLE items. The trace line
labelled i1, which represents the first response option for PLE
item i (‘not true’), has a greater likelihood of being endorsed
by children with lower degrees of PSY severity, and this
likelihood decreases with increasing PSY severity. In other
words, if a child has a low degree of PSY severity, it is more
likely that they will report that they have not experienced the
respective PLE item by answering ‘not true’ on the PLE item.
As would be expected, the trace line i2, which represents the
second response option for PLE item i (‘somewhat true’),
peaks in the mid-range of PSY severity, and the trace line i3,
which represents the third response option for PLE item i
(‘certainly true’), peaks in the high range of PSY severity.
Thus, if a child experienced an average or a high degree of
PSY severity, it would be expected that they would endorse
the second and third response categories of the PLE item
IRT analysis of child psychotic-like experiences 1499
of the PLE and SDQ items in this London community
The standardized factor loadings in model 4 for
each PLE and SDQ item are detailed in Fig. 2. All were
significant at the p<0.05 level. Of particular interest
to the current investigation, factor loadings on PSY
ranged from 0.46 to 0.74, indicating that all items
loaded on the one factor within the moderate to strong
range. Items with the strongest loadings were ‘seen
things’ and ‘heard voices’ (each 0.74). The items with
weakest loadings on the factor were ‘read minds’ and
‘thoughts read’ (0.57 and 0.46 respectively).
Item response characteristics
The IRT parameters for each of the nine PLE items,
ranked by the most discriminating (i.e. largest a
values) to the least discriminating, are presented in
Table 1. In support of the CFA, the most discriminat-
ing items along the latent construct are ‘seen things’
and ‘heard voices’, with a parameter values of 2.00
and 1.86 respectively. Thus, these are the two items
best able to discriminate children with higher levels of
PSY severity from children with lower levels of PSY
severity. The least discriminating items were ‘read
0.46 Thoughts read
0.57 Special messages
0.63 Spied upon
0.74 Heard voices
0.74 Seen things
Best with adults
5 . 0
Kind to kids
Fig. 2. Four-factor model derived from confirmatory factor analysis (CFA) of the psychotic-like experiences (PLEs)
and Strengths and Difficulties Questionnaire (SDQ) items, indicating standardized factor loadings for each item and
inter-construct correlations. PSY, Psychotic-like construct; INT, internalizing construct; EXT, externalizing construct;
PRO, prosocial construct.
1500K. R. Laurens et al.
minds’ and ‘thoughts read’, with a parameter values
of 1.12 and 0.85 respectively. This indicates that these
items do not discriminate as effectively between chil-
dren at high and low levels of PSY severity relative to
the other items.
The category response functions of the PLE items
are presented in Fig. 3, providing a graphical rep-
resentation of the a, b1and b2IRT parameters pres-
ented in Table 1. The most discriminating items, ‘seen
things’ and ‘heard voices’, are those with the steepest
curves. Furthermore, it can be seen that these two
items discriminate along the full range of the under-
lying latent construct. That is, the likelihood of en-
dorsing the ‘not true’ response option peaks at lower
levels of the construct; as PSY severity level moves
towards the mean, the likelihood of endorsing the
‘somewhat true’ response option peaks; and finally,
as the severe range is approached, the likelihood of
endorsing the ‘certainly true’ response option peaks.
These curves confirm that these two items are par-
ticularly good indicators of PSY severity in children
aged between 9 and 11 years.
The majority of the remaining items tend to dis-
criminate between children showing higher levels
of PSY severity, particularly ‘thoughts read’, ‘special
messages’, ‘controlled’ and ‘special powers’. This is
indicated by the response option trace lines crossing
consistently at the upper level of the latent construct
(i.e. indicating the point at which an option becomes
more likely than another; as indexed by the positive b1
and b2IRT parameters). For the item ‘special powers’,
the likelihood of endorsing the ‘somewhat true’
–4 –3 –2 –1 01234
Special messages Spied upon
Heard voices Controlled Read minds
Body changed Special powers Seen things
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
–4 –3 –2 –1 01234
Fig. 3. Matrix plot of the category response functions for each of the nine psychotic-like experience (PLE) items, presented
according to their order of administration on the questionnaire. The estimated latent construct (hj) is plotted along the horizontal
axis, and the probability of endorsing each response option (ranging from 0 to 1) is indicated along the vertical axis. The ‘not
true’ response option is shown by the dotted line, the ‘somewhat true’ response option by the short-dashed line, ‘certainly true’
response option by the long-dashed line.
IRT analysis of child psychotic-like experiences 1501
option offers no discriminating ability relative to ‘not
true’ and ‘certainly true’ response options because
the likelihood of endorsing ‘somewhat true’ never
exceeds the likelihood of endorsing the other response
options along the full range of the construct.
Fig. 4 presents the IIFs for each of the nine PLE
items. These curves indicate the amount of infor-
mation provided about the latent construct by each
item (indicated on the vertical axis). The items ‘seen
things’ and ‘heard voices’ provide the most infor-
mation about the latent construct, and this information
is greatest around the mean levels of the construct
(i.e. their peaks occur within the mean range). The
other items provide a moderate amount of information
about the construct, at slightly higher than mean levels
on PSY severity (i.e. their peaks are distributed above
mean levels). The items ‘thoughts read’ and ‘read
minds’ provide the least information about the con-
struct, as indicated by their relatively low, flat curves.
Test information characteristics
Fig. S1 in the online supplementary material indicates
that the PLE questionnaire provided the most infor-
mation (and the highest precision of measurement)
around 0.5 S.D. above the mean on the latent PSY.
Thus, the scale provides more information for slightly
more severe cases than for less severe cases. However,
the wide TIF indicates that the PLE items have rela-
tively good measurement precision between x2 and
2 S.D. from the mean of the latent construct, thus
providing information about the majority of the
This study explored the latent structure underlying
items assessing PLEs, internalizing and externalizing
psychopathology, and personal competencies in a
general child population aged 9–11 years. The nine
–3 –2 –1 0123
Special messages Spied upon
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
–3 –2 –1 0123
Body changed Special powers Seen things
Fig. 4. Matrix plot of the item information functions (IIFs) for the nine psychotic-like experience (PLE) items in children aged
9 to 11 years. For each function, the horizontal axis indicates the distribution of the latent construct (hj) in scale scores
(ranging from x3 to 3), and the vertical axis indicates the item information available at a specific scale score. The items are
presented in order of administration on the questionnaire.
1502K. R. Laurens et al.
self-report PLE items constitute a simple checklist
to elicit information from the general paediatric
population concerning a range of hallucination- and
delusion-like experiences that correspond to symp-
toms reported by individuals with psychotic illnesses,
particularly schizophrenia. CFAs indicated a best-
fitting four-factor model encompassing PSY, inter-
nalizing, externalizing and prosocial dimensions.
Thus, PLEs and internalizing and externalizing
difficulties in children cannot be understood as re-
flecting a single undifferentiated psychopathology,
but instead represent separable but correlated dimen-
sions that might share some aetiological influences
(Lahey et al. 2011). The PLE items each loaded on the
single PSY, and together were able to provide infor-
mation about the majority of the population, with
measurement precision achieved between x2 and 2
S.D. from the mean of the latent construct.
This study further examined the psychometric
properties of each PLE item. The IRT model indicates
how the probability of endorsing a particular item
varies as a function of underlying latent construct
severity. Among the nine PLE items, the questions
assessing visual and auditory hallucination-like ex-
periences were the most capable of assessing the latent
construct. These two questionnaire items previously
also showed strongest criterion validity for psychotic
symptoms elicited by diagnostic interview among
11–13-year-olds (Kelleher et al. 2011). These two
most discriminating items provided most information
around mean (mid) levels of PSY severity, whereas the
remaining items provided most information at slightly
above mean levels, thus tapping a more severe ex-
pression of PSY. These variable item parameters imply
that PLE items may be used in different capacities
in research and clinical settings. For example, items
tapping mid-levels of a latent psychological construct
may be most appropriate for community screening,
where the interest lies in determining population
variation in mental health vulnerability. That is, the
items ‘seen things’ and ‘heard voices’ offer the best
means of identifying a child with illness vulnerability
with expediency. Thus, they might be used as initial
screening questions within the general population to
best discriminate children with and without PLEs. The
other scale items, which discriminated better at the
higher levels of the construct (i.e. detected children
with higher levels of PSY severity), could be used to
distinguish the upper severity levels of the construct
among children who screen positive on the ‘seen
things’ and ‘heard voices’ items. The item with the
poorest item parameters was ‘thoughts read’. In its
current form, this item would be the first candidate to
remove if seeking to shorten the scale because of the
relatively little information it provides regarding the
latent PSY severity and its relatively poor discrimi-
nating ability. However, this item might be retained if
the purpose of the tool is to serve as a clinical checklist
identifying a range of experiences. Revision of the item
wording might improve its IRT parameters. The re-
maining items all provided useful and discriminating
characteristics in this population of children aged
between 9 and 11 years.
An advantage of IRT over classical test theory is
that the psychometric properties of the questionnaire
and the individual items are relatively sample inde-
pendent (Embretson & Reise, 2000; Streiner, 2010).
Thus, the PLE items may be expected to perform
similarly among other paediatric samples of similar
age. However, the nine PLE items constituted a
brief checklist to assess a range of hallucination-
and delusion-like experiences only. Longer and more
comprehensive assessment instruments have been
used with general population samples of adolescents
and adults to assess a broader range of positive, and
also negative, PLEs (e.g. the Community Assessment
of Psychic Experiences;
Factor analysis of such instruments reveals multiple
(correlated) latent dimensions underlying both posi-
tive and negative PLEs that relate differentially to de-
pressive symptoms (Armando et al. 2010; Barragan
et al. 2011). Such instruments are not readily utilized
with children. The unidimensional PSY underlying
PLEs in the present study may reflect the relatively
brief checklist of items assessed and/or a relatively
undifferentiated latent PSY prior to adolescence.
An important strength of the current study was the
sampling of 95% of the total eligible population of
children aged 9–11 years, thereby assessing the full
spectrum of experience distributed within the general
population. Within the sample, 78% of children lived
in inner-city London communities categorized among
the most deprived 11% of all English local authorities
according to the The English Indices of Deprivation 2007
(Noble et al. 2008), and among the lowest scoring 5%
of English local authorities on the Local Index of Child
Well-Being (Bradshaw et al. 2009). These communities
are characterized by an increased incidence of schizo-
phrenia (Kirkbride et al. 2007) and a high prevalence of
self-reported PLEs among the general adult popu-
lation (e.g. >20%; Morgan et al. 2009). Thus, elevated
rates of positive endorsement of PLE items were
anticipated in this sample of 9–11-year-olds (Laurens
et al. 2008, 2011). The most common experience
(‘heard voices’) was reported as ‘certainly true’ by
approximately a third of children, whereas almost
two-thirds of children reported a ‘certainly true’ re-
sponse to at least one of the nine PLE items included in
the scale. High rates of self-reported PLEs in child-
hood may also imply that subthreshold experiences of
IRT analysis of child psychotic-like experiences1503
this nature are not intrinsically pathological in child-
hood, but rather, that such experiences may constitute
part of the spectrum of normative development.
Investigation of the factors contributing to persistence
versus discontinuity of these experiences from child-
hood is needed, as has recently been examined in
adolescent populations (De Loore et al. 2011; Mackie
et al. 2011). Furthermore, the utility of the present
study is inherently limited in providing no indication
of whether the latent construct assessed using the
PLE items is sensitive or specific in predicting later
transition to psychotic, relative to other psychiatric,
illnesses in later adolescence or adulthood. This will
require longitudinal tracking of the children.
Whether elevated PLE prevalence in childhood is
part of normative development, or predicts later psy-
chotic disorder, is pertinent to current debate con-
cerning the existence of a psychosis continuum in the
general population. Several recent works have em-
phasized the need to distinguish the apparent pheno-
typic continuum of psychotic-like experiences in the
general population from a discontinuous population
structure comprising individuals with and without
psychotic disorders (David, 2010; Lawrie et al. 2010;
Linscott & van Os, 2010). Kaymaz & van Os (2010)
propose that, despite the phenotypic continuum of
subthreshold psychotic experiences, a latent cate-
gorical structure may distinguish two groups in the
population: that is, one categorical group comprising
individuals who present psychotic experiences in the
context of cognitive and motivational impairments
(associated with high probability of need for care), and
the other group presenting psychotic experiences of
potentially different origin (associated with lower
likelihood of need for care). Thus, PLEs might confer
risk of later psychotic illness only in the context of
additional psychosis risk markers and/or the absence
of protective factors. The high prevalence of PLE en-
dorsement among this sample of 9–11-year-old chil-
dren inevitability limits the specificity of these items in
predicting later psychotic disorder, and implies that,
alone, self-reported PLEs at this age may have limited
utility as risk markers for psychosis. We have pro-
posed previously that PLEs, presented in conjunction
with other putative antecedents of schizophrenia
(e.g. social, emotional or behavioural problems, and
delays/abnormalities in motor and language devel-
opment), might confer greater specificity and sensi-
tivity for predicting later schizophrenia (Laurens et al.
2007, 2008, 2011). Recent data indicate that children
presenting a combination of PLEs and other putative
antecedents of schizophrenia display abnormalities of
brain structure and function that are similar to those
observed in patients with schizophrenia (Cullen et al.
2010; Laurens et al. 2010; MacManus et al. 2011).
The present study identifies child self-report ques-
tionnaire items assessing clinically subthreshold PLEs
that might be used in longitudinal epidemiological
research to differentiate developmental trajectories to
psychotic and other psychiatric illnesses. The nine PLE
items may be used in prospective cohort studies be-
ginning in childhood to ascertain the stability of PLEs
as children transition into adolescence, and to charac-
terize the potentially normative developmental aspect
of PLEs. Among the nine items, those assessing visual
and auditory hallucination-like experiences are of
greatest use for population screening to identify chil-
dren who may merit more detailed clinical assessment
(including ascertainment of the presence of other
risk markers for schizophrenia) and/or longitudinal
monitoring for the persistence of these PLEs into ado-
lescence and young adulthood.
Supplementary material accompanies this paper on
the Journal’s website (http://journals.cambridge.org/
We thank the children and caregivers who partici-
pated in this study, and the researchers and students
who contributed to data collection. The research was
supported by funding awarded to K.R.L. from a
National Institute for Health Research Career Devel-
opment Fellowship (CDF/08/01/015), a Bial Foun-
dation Research Grant (36/06), a National Alliance
for Researchon Schizophrenia
(NARSAD) Young Investigator Award (2005), and the
British Medical Association Margaret Temple Award
for schizophrenia research (2006). M.J.G. is supported
by the Australian Research Council (FT0991551).
for Health Research Specialist Biomedical Research
Centre (BRC) for Mental Health at the South London
and Maudsley NHS Foundation Trust and the Insti-
tute of Psychiatry, King’s College London, UK.
the National Institute
Declaration of Interest
Arango C (2011). Attenuated psychotic symptoms
syndrome: how it may affect child and adolescent
psychiatry. European Child and Adolescent Psychiatry 20,
1504K. R. Laurens et al.
Armando M, Nelson B, Yung AR, Ross M, Birchwood M,
Girardi P, Fiori Nastro P (2010). Psychotic-like experiences
and correlation with distress and depressive symptoms in
a community sample of adolescents and young adults.
Schizophrenia Research 119, 258–265.
Bak M, Delespaul P, Hanssen M, de Graaf R, Vollebergh W,
van Os J (2003). How false are ‘false’ positive psychotic
symptoms? Schizophrenia Research 62, 187–189.
Barragan M, Laurens KR, Navarro JB, Obiols JE (2011).
Psychotic-like experiences and depressive symptoms in a
community sample of adolescents. European Psychiatry 26,
Bentler PM (1990). Comparative fit indexes in structural
models. Psychological Bulletin 107, 238–246.
Bradshaw J, Bloor K, Huby M, Rhodes D, Sinclair I, Gibbs I
(2009). Local Index of Child Well-Being: Summary Report.
Department for Communities and Local Government:
Costello A, Edelbrock C, Kalas R, Kessler M, Klaric S
(1982). Diagnostic Interview Schedule for Children: Child
Version. National Institute of Mental Health: Rockville,
Cullen AE, Dickson H, West SA, Morris RG, Mould GL,
Hodgins S, Murray RM, Laurens KR (2010).
Neurocognitive performance in children aged 9–12 years
who present putative antecedents of schizophrenia.
Schizophrenia Research 121, 15–23.
David AS (2010). Why we need more debate on whether
psychotic symptoms lie on a continuum with normality.
Psychological Medicine 40, 1935–1942.
De Loore E, Gunther N, Drukker M, Feron F, Sabbe B,
Deboutte D, van Os J, Myin-Germeys I (2011).
Persistence and outcome of auditory hallucinations in
adolescence: a longitudinal general population study of
1800 individuals. Schizophrenia Research 127, 252–256.
Dhossche D, Ferdinand R, Van der Ende J, Hofstra MB,
Verhulst F (2002). Diagnostic outcome of self-reported
hallucinations in a community sample of adolescents.
Psychological Medicine 32, 619–627.
Embretson S, Reise S (2000). Item Response Theory for
Psychologists. Lawrence Erlbaum Associates, Inc.:
Goodman A, Lamping DL, Ploubidis GB (2010). When to
use broader internalising and externalising subscales
instead of the hypothesised five subscales on the
Strengths and Difficulties Questionnaire (SDQ): data from
British parents, teachers and children. Journal of Abnormal
Child Psychology 38, 1179–1191.
Goodman R (1997). The Strengths and Difficulties
Questionnaire: a research note. Journal of Child Psychology
and Psychiatry 38, 581–586.
Goodman R (2001). Psychometric properties of the
Strengths and Difficulties Questionnaire. Journal of the
American Academy of Child and Adolescent Psychiatry 40,
Hu L, Bentler PM (1998). Fit indices in covariance structure
modeling: sensitivity to underparamterized model
misspecification. Psychological Methods 3, 424–453.
Jennrich RI, Sampson PF (1966). Rotation for simple
loadings. Psychometrika 31, 313–323.
Kaymaz N, van Os J (2010). Extended psychosis
phenotype – yes: single continuum – unlikely.
Psychological Medicine 40, 1963–1966.
Kelleher I, Cannon M (2011). Psychotic-like experiences in
the general population: characterizing a high-risk group
for psychosis. Psychological Medicine 41, 1–6.
Kelleher I, Harley M, Murtagh A, Cannon M (2011). Are
screening instruments valid for psychotic-like
experiences? A validation study of screening questions for
psychotic-like experiences using in-depth clinical
interview. Schizophrenia Bulletin 37, 352–361.
Keshavan MS, Delisi LE, Seidman LJ (2011). Early and
broadly defined psychosis risk mental states. Schizophrenia
Research 126, 1–10.
Kessler RC, Ormel J, Petukhova M, McLaughlin KA,
Green JG, Russo LJ, Stein DJ, Zaslavsky AM,
Aguilar-Gaxiola S, Alonso J, Andrade L, Benjet C,
de Girolamo G, de Graaf R, Demyttenaere K, Fayyad J,
Haro JM, Hu C, Karam A, Lee S, Lepine JP, Matchsinger
H, Mihaescu-Pintia C, Posada-Villa J, Sagar R, Ustun TB
(2011). Development of lifetime comorbidity in the World
Health Organization World Mental Health Surveys.
Archives of General Psychiatry 68, 90–100.
Kirkbride JB, Fearon P, Morgan C, Dazzan P, Morgan K,
Murray RM, Jones PB (2007). Neighbourhood variation in
the incidence of psychotic disorders in Southeast London.
Social Psychiatry and Psychiatric Epidemiology 42, 438–445.
Krabbendam L, Myin-Germeys I, De Graaf R, Vollebergh
W, Nolen WA, Iedema J, Van Os J (2004). Dimensions of
depression, mania and psychosis in the general
population. Psychological Medicine 34, 1177–1186.
Krueger RF (1999). The structure of common mental
disorders. Archives of General Psychiatry 56, 921–926.
Lahey BB, Van Hulle CA, Singh AL, Waldman ID, Rathouz
PJ (2011). Higher-order genetic and environmental
structure of prevalent forms of child and adolescent
psychopathology. Archives of General Psychiatry 68, 181–189.
Laurens KR, Hodgins S, Maughan B, Murray RM, Rutter
ML, Taylor EA (2007). Community screening for
psychotic-like experiences and other putative antecedents
of schizophrenia in children aged 9–12 years. Schizophrenia
Research 90, 130–146.
Laurens KR, Hodgins S, Mould GL, West SA,
Schoenberg PL, Murray RM, Taylor EA (2010).
Error-related processing dysfunction in children aged
9 to 12 years presenting putative antecedents of
schizophrenia. Biological Psychiatry 67, 238–245.
Laurens KR, Hodgins S, Taylor EA, Murray RM (2011). Is
earlier intervention for schizophrenia possible? Identifying
antecedents of schizophrenia in children aged 9–12 years.
In Schizophrenia: The Final Frontier – A Festscrift for Robin
M. Murrary (ed. A. S. David, P. McGuffin and S. Kapur),
pp. 19–32. Psychology Press: London.
Laurens KR, West SA, Murray RM, Hodgins S (2008).
Psychotic-like experiences and other antecedents of
schizophrenia in children aged 9–12 years: a comparison of
ethnic and migrant groups in the United Kingdom.
Psychological Medicine 38, 1103–1111.
Lawrie SM, Hall J, McIntosh AM, Owens DG, Johnstone
EC (2010). The ‘continuum of psychosis’: scientifically
IRT analysis of child psychotic-like experiences 1505
unproven and clinically impractical. British Journal of Download full-text
Psychiatry 197, 423–425.
Linscott RJ, van Os J (2010). Systematic reviews of
categorical versus continuum models in psychosis:
evidence for discontinuous subpopulations underlying
a psychometric continuum. Implications for DSM-V,
DSM-VI, and DSM-VII. Annual Review of Clinical
Psychology 6, 391–419.
Mackie CJ, Castellanos-Ryan N, Conrod PJ (2011).
Developmental trajectories of psychotic-like experiences
across adolescence: impact of victimization and substance
use. Psychological Medicine 41, 47–58.
MacManus D, Laurens KR, Walker EF, Brasfield JL, Riaz M,
Hodgins S (2011). Movement abnormalities and psychotic-
like experiences in childhood: markers of developing
schizophrenia? Psychological Medicine. Published online
11 July 2011. doi:10.1017/S0033291711001085.
Markon KE (2010). Modeling psychopathology structure: a
symptom-level analysis of Axis I and II disorders.
Psychological Medicine 40, 273–288.
Morgan C, Fisher H, Hutchinson G, Kirkbride J, Craig TK,
Morgan K, Dazzan P, Boydell J, Doody GA, Jones PB,
Murray RM, Leff J, Fearon P (2009). Ethnicity, social
disadvantage and psychotic-like experiences in a healthy
population based sample. Acta Psychiatrica Scandinavica
Muris P, Meesters C, Eijkelenboom A, Vincken M (2004).
The self-report version of the Strengths and Difficulties
Questionnaire: its psychometric properties in 8- to
13-year-old non-clinical children. British Journal of Clinical
Psychology 43, 437–448.
Muthe ´n L, Muthe ´n B (1998–2010). Mplus User’s Guide.
Muthe ´n & Muthe ´n: Los Angeles, CA.
Noble M, McLennan D, Wilkinson K, Whitworth A,
Barnes H, Dibben C (2008). The English Indices of
Deprivation 2007. Department for Communities and Local
Government: London, UK.
Nuevo R, Chatterji S, Verdes E, Naidoo N, Arango C,
Ayuso-Mateos JL (2010). The continuum of psychotic
symptoms in the general population: a cross-national
study. Schizophrenia Bulletin. Published online 13
September 2010. doi:10.1093/schbul/sbq099.
Perala J, Suvisaari J, Saarni SI, Kuoppasalmi K, Isometsa E,
Pirkola S, Partonen T, Tuulio-Henriksson A, Hintikka J,
Kieseppa T, Harkanen T, Koskinen S, Lonnqvist J (2007).
Lifetime prevalence of psychotic and bipolar I disorders
in a general population. Archives of General Psychiatry 64,
Poulton R, Caspi A, Moffitt TE, Cannon M, Murray R,
Harrington H (2000). Children’s self-reported psychotic
symptoms and adult schizophreniform disorder: a 15-year
longitudinal study. Archives of General Psychiatry 57,
Samejima F (1969). Estimation of Latent Ability Using a
Response Patter of Graded Scores. Psychometric Monograph
No. 17. Psychometric Society: Richmond, VA.
Stefanis NC, Hanssen M, Smirnis NK, Avramopoulos DA,
Evdokimidis IK, Stefanis CN, Verdoux H, van Os J
(2002). Evidence that three dimensions of psychosis have a
distribution in the general population. Psychological
Medicine 32, 347–358.
Steiger JH (1990). Structural model evaluation and
modification: an internal estimation approach.
Multivariate Behavioral Research 25, 173–180.
Stone LL, Otten R, Engels RC, Vermulst AA, Janssens JM
(2010). Psychometric properties of the parent and teacher
versions of the Strengths and Difficulties Questionnaire for
4- to 12-year-olds: a review. Clinical Child and Family
Psychology Review 13, 254–274.
Streiner DL (2010). Measure for measure: new developments
in measurement and item response theory. Canadian
Journal of Psychiatry 55, 180–186.
Thissen D, Chen W-H, Bock RD (2003). Multilog (Version 7).
Scientific Software International: Lincolnwood, IL.
Tucker LR, Lewis C (1973). A reliability coefficient for
maximum likelihood factor analysis. Psychometrika 38,
van Os J, Hanssen M, Bijl RV, Ravelli A (2000). Strauss
(1969) revisited: a psychosis continuum in the general
population? Schizophrenia Research 45, 11–20.
van Os J, Linscott RJ, Myin-Germeys I, Delespaul P,
Krabbendam L (2009). A systematic review and
meta-analysis of the psychosis continuum: evidence
for a psychosis proneness-persistence-impairment
model of psychotic disorder. Psychological Medicine 39,
Welham J, Scott J, Williams G, Najman J, Bor W,
O’Callaghan M, McGrath J (2009). Emotional and
behavioural antecedents of young adults who screen
positive for non-affective psychosis: a 21-year birth
cohort study. Psychological Medicine 39, 625–634.
Wigman JT, Lin A, Vollebergh WA, van Os J,
Raaijmakers QA, Nelson B, Baksheev G, Yung AR (2011).
Subclinical psychosis and depression: co-occurring
phenomena that do not predict each other over time.
Schizophrenia Research 130, 277–281.
1506 K. R. Laurens et al.