A. Sorkin et al., in Proc: 16th Annual Medicine Meets Virtual Reality (MMVR), 2008
The distortion of reality perception in
schizophrenia patients, as measured in
Anna SORKIN1 Ph.D, Daphna WEINSHALL1,2 Ph.D., Avi PELED3,4 M.D.
?Interdisciplinary Center for Neural Computation, Hebrew University of Jerusalem,
?School of Computer Science and Engineering, Hebrew University of Jerusalem, Israel?
3Institute for Psychiatric Studies, Sha’ar Menashe Mental Health Center, Israel
4 Rappaport Faculty of Medicine, Technion-Israel Institute of Technology, Israel
Abstract. Background: Virtual Reality is an interactive three-dimensional
computer generated environment. Providing a complex and multi-modal
environment, VR can be particularly useful for the study of complex cognitive
functions and brain disorders. Here we used a VR world to measure the distortion
in reality perception in schizophrenia patients. Methods: 43 schizophrenia patients
and 29 healthy controls navigated in a VR environment and were asked to detect
incoherencies, such as a cat barking or a tree with red leaves. Results: Whereas the
healthy participants reliably detected incoherencies in the virtual experience, 88%
of the patients failed in this task. The patients group had specific difficulty in the
detection of audio-visual incoherencies; this was significantly correlated with the
hallucinations score of the PANSS. Conclusions: By measuring the distortion in
reality perception in schizophrenia patients, we demonstrated that Virtual Reality
can serve as a powerful experimental tool to study complex cognitive processes.
Keywords. Schizophrenia, reality perception, audio-visual incoherency
The term Virtual Reality describes systems in which the user becomes fully immersed
in an artificial, three-dimensional world generated by a computer. The sensation of
presence is typically achieved through the use of a head-mounted display (HMD). A
motion tracker continuously measures the position and orientation of the user's head
and allows the image-generating computer to adjust the scene representation to the
current view. As a result, the viewer can look around and walk through the surrounding
virtual environment in a similar fashion to the real world.
As the main motivation for this study, we hypothesize that by using VR
technology, one will be able to design complex interactive tasks that will challenge
multiple modalities simultaneously in a natural way. In this way VR may become a
particularly useful tool for the study of complex disorders such as schizophrenia.
Specifically, we chose in this study to measure the distortion in one’s reality perception
in a complex realistic VR Environment, because this deficiency is a common serious
manifestation of schizophrenia . Though it is difficult to find any cognitive task that
schizophrenia patients perform adequately as a group [2-4], there are always
individuals that fall within the normal range. Therefore we were particularly interested
in seeing how many patients will demonstrate impairment and what fraction will
We used a detection paradigm in order to measure abnormal reality perception. A
subject is required to detect various incoherent events inserted into a normal virtual
environment. Everything is possible: a guitar may sound like a trumpet, causing an
audio-visual incoherency; a passing lane may be pink and a house may stand on its
roof, resulting in visual-visual incoherencies of color and location respectively (see
43 schizophrenia patients (23 in-patients and 20 out-patients) and 29 healthy controls
matched by age, education level and gender were recruited for the study. Mean age was
32.6 (SD=8.5), with an average of 11.1 (SD=1.8) years of schooling; 19% were
All patients underwent a psychiatric interview with a senior psychiatrist (author
AP). The diagnosis of schizophrenia was established according to the DSM-IV-TR
criteria, and symptoms severity was assessed using the Positive and Negative
Syndromes Scale (PANSS) . The study was approved by the Sha'ar Menashe Mental
Health Center Review Board, and informed consent was obtained from all participants
after the nature of the study was fully explained to them. All subjects volunteered and
1.2. Experimental Design and Procedure
Subjects sat comfortably in a reclining chair, wearing a Head Mounted Display (HMD)
containing the audio and visual devices and a position tracker. Subjects navigated
along a predetermined path through a residential neighborhood, shopping centers and a
street market. Apart from the incoherencies which were deliberately planted, the virtual
environment was designed to resemble the real world as closely as possible.
Whenever the path traversed an incoherent event, progress was halted and a one
minute timer appeared, during which the subject had to detect the incoherency.
Response included marking the whereabouts of the incoherent event by a mouse click,
and an accompanying verbal explanation to be recorded. A number of external
observers, blind to the purpose of the experiment and the assignment to patient vs.
control group, determined correctness of an explanation. We gave no examples before
the test as guidelines, and no feedback indicating correct or incorrect response. (See
http://www.cs.huji.ac.il/~daphna/movies/vr_inconsistencies/Demo_best.swf - a movie
demonstration of the virtual world.)
We created three categories of incoherent events: sound (Figure 1A), color (Figure
1B) and location (Figure 1C). The virtual world contained 50 incoherencies: 16
involving color, 18 concerning location and 16 related to sound. Three incoherencies
were excluded from the final analysis: two due to the high miss rate (?25%) among the
control subjects, and one due to repeated reports of its being confusing. This resulted in
14 incoherencies of color, 17 - location, 16 – sound, for a total of 47.
Figure 1. Examples from the virtual world used in the experiment, illustrating the 3 types of incoherencies.
A. incoherent sound: a guitar emitting trumpet sounds; B. incoherent color: pink crossing and a red cloud; C.
incoherent location – a giraffe grazing in a local store.
2.1. Detection Rates
The histogram of detection rates is shown in Figure 2A. The control subjects detected
incoherencies very well, with an accuracy level of 96% on average (SD=4) (left panel).
In general, the patient group (right panel) differed significantly from the controls.
Normal detection rates (defined as the mean of the control group ±2.5 SD, including
roughly 99% of the normal population) are shown in red for each category, whereas
blue bars indicate the number of subjects that performed below normal. For example,
the normal range for total detection rates is 87-100%. The upper plot shows that all but
one of the control subjects performed in this range. Among the patients only 6 subjects
(red bars) performed in the normal range, whereas 37 subjects (blue bars) had lower
detection rates. The patients group exhibited the most difficulty in the sound category:
30 patients performed below the normal range, and 19 had detection rates below 50%,
compared to the location category, where only 10 patients detected less than 50% of the
2.2. The ‘Gap’ Phenomenon
While analyzing the data, we noticed that many patients exhibited specific categorical
deficiencies unlike the control group, which showed similar detection rates in all three
categories (Figure 2B, left plot). We therefore defined and quantified the notion of gap,
which indicates whether the subject’s detection rate in one category has differed
significantly from his/her best category (a significant difference is a difference
exceeding the mean±2.5SD of the control group). Thus a gap is measured relative to
individual performance levels. Almost half of the patients (20 out of 43) demonstrated
this phenomenon, showing specific difficultly in one or two categories (see Figure 2B).
Figure 2. Histogram of detection rates among the control and patient groups.
A. The horizontal axis represents detection rates, while the vertical axis shows the number of subjects
obtaining each score. The red bars indicate performance in the normal range (defined as the mean of the
control group ±2.5 SD), and the blue bars – performance outside the normal range. B. Comparison of
individual detection rates among sound, color and location categories. Left: controls, middle: patients with
gap in the sound category only, right: patients with gap in the sound and color categories.
2.3. Various divisions of the patient group
Based on the similarity in detection rates the patient group could be divided into two
major sub-groups: (1) The uniform group (23 subjects) – patients whose detection rates
in all three categories were similar. (2) Gap (20 subjects) – the group of patients having
specific difficulty in one or two categories.
The uniform group could be further divided into: i) uniform normal: patients
performing at normal levels (N=5 subjects); ii) uniform fair: patients with good
detection rates (50-87%) but below the normal range (N=10 subjects); and finally iii)
uniform poor: patients with poor uniform performance below 50% (N=8 subjects).
In the gap group the majority of patients had a difficulty in the sound category. 16
patients (37%) had a specific difficulty in detecting audio-visual incoherencies: 7
patients had difficulty in the sound category only (Figure 2B, middle plot), 7 patients
had difficulty in the sound and color categories as compared to the location category
(Figure 2B, right plot), and two patients had difficulty in the sound and location
categories. Four additional patients exhibited other specific difficulties. In contrast,
only one of the control subjects exhibited a gap in the sound category.
The detection rates discriminate well between the control and patient populations.
Nearly all control subjects, 96.5% (28 out of 29), belong to the uniform normal group.
In contrast, while 5 patients belong to the uniform normal group and thus cannot be
distinguished from the control group, the remaining 88% of the patients show a
significantly different detection profile.
2.4. Symptom Analysis
We found a number of significant correlations (Spearman’s r?0.3, t?2.02, df=41,
p<0.05) between the detection rates and the PANSS scores in the patient group: i) The
‘hallucinations’ score was correlated with low total and sound detection rates. ii)
‘Difficulty in abstract thinking’ showed correlation with low total, sound and color
detection rates (correlation: Spearman’s r?0.3885, t?2.7, df=41, p<0.01). In addition,
reaction time showed negative correlation with age.
3. Summary and Discussion
Our results show that Virtual Reality can be readily used with schizophrenia patients,
allowing for the measurement of some complex deficiencies that they experience. Thus
the Virtual Reality experimental environment promises to become a particularly
beneficial tool for the characterization of complex brain disorders such as
schizophrenia, where the complex nature of the syndrome may manifest itself
differently in different complex multi-modal tasks.
In this experiment we designed an environment in which we can challenge one’s
reality perception, using a detection task of incoherent visual and audio-visual stimuli.
This very simple task distinguished very well between the control and patient
populations: 88% of the patients differed significantly from the control group. The
patient group showed the greatest difficulty in detecting audio-visual incoherencies,
where poor performance correlated with the presence of hallucinations. Interestingly,
we observed that most effective were events involving auditory stimuli, where the
object and sound matched overall but were used under the wrong circumstances, as in
adults laughing like babies or a civilian airplane emitting bombing sounds.
The task has additional advantages: it is short - taking only half an hour, and it can
be self-administrated requiring only minimal non-professional assistance. The analysis
of individual incoherencies revealed that some incoherencies present greater difficulty
for the patient group but not for the controls. Therefore, the set of incoherencies may be
further improved to shorten the duration of the test, and to increase the discriminability
of the patient population.
It would be interesting to compare the performance of schizophrenia patients in
this incoherencies detection task with standard cognitive tests. Previous studies show
that in some tests less than 40% of schizophrenia patients are impaired [6, 7], while in
others only 11%-55% of the schizophrenia patients perform in the normal range [8-10].
In an extensive study Palmer et al.  aimed to explore the prevalence of
neuropsychological (NP) normal subjects among the schizophrenia population. In their
study, the proportion of impaired patients in different cognitive dimensions varied in
the range 9%-67%. In comparison, our results in the incoherencies detection task seem
very promising, with 88% of the patients demonstrating impairment. These results
should be replicated with a larger group of patients and controls and confirmed with
additional comparison groups, including groups consisting of patients with different
Acknowledgments Download full-text
The authors thank the staff of the Hesed and Emuna hostel in Jerusalem and its
director Hannah Rosenthal for their help and encouragement.
 David AS., 2004. The cognitive neuropsychiatry of auditory verbal hallucinations: an overview. Cognit
 MATRICS: Measurement and Treatment Research to Improve Cognition in Schizophrenia. 2004.
MATRICS: Provisional Consensus
http://www.matrics.ucla.edu/provisional-MATRICS-battery.shtml. Accessed Oct 22, 2006.
 Palmer, B.W., Heaton, R.K., Paulsen, J.S., Kuck, J., Braff, D., Harris, M.J., Zisook, S. and Jeste, D.V.,
1997. Is it possible to be schizophrenic yet neuropsychologically normal?. Neuropsychology 11, pp.
 Green Michel F., 2006. “Cognitive Impairment and Functional Outcome in Schizophrenia and Bipolar
Disorder” . J Clin Psychology 67 (suppl 9):3-8.
 Kay, S.R., Fiszbein, A. and Opler, L.A., 1987, The Positive and Negative Syndrome Scale (PANSS) for
schizophrenia. Schizophr. Bull. 13, 261-276.
 Goldberg, T.E. and Gold, J.M., 1995. Neurocognitive functioning in patients with schizophrenia. In:
Bloom, F.E. and Kupfer, D.J., Editors, 1995. Psychopharmacology: The Fourth Generation of Progress,
Raven Press, New York, pp. 1245–1257.
 Braff DL, Heaton R, Kuck J, Cullum M, Moranville J, Grant I, Zisook S., 1991. The generalized pattern
of neuropsychological deficits in outpatients with chronic schizophrenia with heterogeneous Wisconsin
Card Sorting Test results. Arch Gen Psychiatry 48:891–898?
 Torrey, E. E, Bowler, A. E., Taylor, E. H., & Gottesman, I. I., 1994. Schizophrenia and manic-
depressive disorder. New York: Basic Books.
 Strauss, B. S., & Silverstein, M. L., 1986. Luria-Nebraska measures in neuropsychologically
nonimpaired schizophrenics: A comparison with normal subjects. International Journal of Clinical
Neuropsychology, 8, 35-38.
 Bryson, G. J., Silverstein, M. L., Nathan, A., & Stephen, L.,1993. Differential rate of
neuropsychological dysfunction in psychiatric disorders: Comparison between alstead-Reitan and
Luria-Nebraska batteries. Perceptual and Motor Skills, 76, 305-306.
Cognitive Battery. Available at: