Scanning of Scenes in Autism and Schizophrenia 1
Running Head: SCANNING OF SCENES IN AUTISM AND SCHIZOPHRENIA
Orienting to Social Stimuli Differentiates Social Cognitive Impairment in Autism and
Noah Sasson, Ph.D.¹, Naotsugu Tsuchiya, Ph.D.², Robert Hurley, M.A.¹, Shannon M.
Couture, M.A.3, David L. Penn, Ph.D.3, Ralph Adolphs, Ph.D.² and Joseph Piven, M.D.¹
¹University of North Carolina at Chapel Hill, Neurodevelopmental Disorders Research
²California Institute of Technology, Division of Humanities and Social Sciences
3University of North Carolina at Chapel Hill, Department of Psychology
Correspondence should be addressed to: Joseph Piven, M.D., Neuroscience Hospital,
University of North Carolina-Chapel Hill, CB#3366, Chapel Hill, NC 27599-3366;
email: Joe_Piven@med.unc.edu, telephone: (919) 843-8641, fax: (919) 966-7080
Scanning of Scenes in Autism and Schizophrenia 2
Context. Both autism and schizophrenia feature impairments in aspects of social
cognition that may be related to amygdala dysfunction, but the exact nature and
specificity of these impairments remain unclear.
Objective. To compare the visual scanning patterns and emotion judgments of
individuals with schizophrenia, individuals with autism and controls on a task that is
well-characterized with respect to amygdala functioning.
Design. A group comparison on a task in which participants view a series of statically
presented complex social scenes where faces were either included or digitally erased.
Setting. The University of North Carolina at Chapel Hill or the participant’s home.
Participants. 10 adult subjects with autism, 10 with schizophrenia, and 10 typically
developing controls. Participants did not differ in age or I.Q.
Results. Controls increased their viewing time on the face region when faces were
present to a significantly greater degree than both the autism and schizophrenia groups.
While both the control and the schizophrenia groups oriented to face regions faster when
faces were present, the autism group showed no such distinction. The schizophrenia
group exhibited a delay in orienting to face regions across both conditions.
Conclusions. The findings indicate that while individuals with autism or schizophrenia
do not fixate faces in complex social scenes as much as controls, only the autism group
fails to orient to faces more rapidly based upon the presence of facial emotional
information. Autism and schizophrenia therefore share a common abnormality in
utilizing facial information when assessing the emotional content of a social scene, but
differ in the ability to seek out socially relevant information from complex stimuli.
Scanning of Scenes in Autism and Schizophrenia 3
Orienting to Social Stimuli Differentiates Social Cognitive Impairment in Autism and
Both autism and schizophrenia are characterized by pervasive social dysfunction
that impairs the ability to initiate and maintain reciprocal interaction (1). In recent years, a
great deal of attention has been devoted to uncovering specific deficits in social cognition
that may contribute to this impairment. The term social cognition generally refers to the
perception, processing and interpretation of information related to social interaction (2).
A range of social cognitive deficits have been reported for both autism and schizophrenia
(3-5), particularly in theory of mind (6-8), facial affect recognition (9-10), and the
perception of social cues (11-13). Although reports of these impairments are often
remarkably similar for both disorders, they have typically been investigated
independently with little attempt to compare the groups. To this point, only three studies
have contrasted social cognitive functioning in schizophrenia and autism: two reported
similar impairments in autism and schizophrenia on varying measures of theory of mind
(14-15), while a third found evidence for greater impairment in autism than schizophrenia
for facial affect recognition (16).
Investigations aimed at characterizing the neural correlates of social cognitive
dysfunction in autism and schizophrenia also tend to occur in parallel and without direct
comparison, despite overlap in the neural structures that are implicated in both disorders.
The amygdala, a neural region that has been a primary focus of investigations regarding
social cognitive impairments in autism and schizophrenia, has been identified as a critical
Scanning of Scenes in Autism and Schizophrenia 4
structure for threat detection, social appraisal, and the recognition of affect, particularly
negative emotions such as fear (17; for reviews, see 18-20). Structurally, the amygdala
has been reported to be small in schizophrenia (21-23) and increased in volume in autism,
although some studies find no differences in volume (for a review, see 24). Functionally,
a number of fMRI studies demonstrate decreased amygdala activation in both disorders in
response to emotionally-laden stimuli (25-27). Behavioral evidence is further suggestive
of abnormal amygdala involvement in autism and schizophrenia, as subjects of both
disorders have been shown to be impaired in the making of complex social judgments
(28-30) and in the recognition of negative affect, particularly fear (10, 31-33).
Taken together, these findings suggest that amygdala dysfunction may contribute
to the similar behavioral impairments exhibited by individuals with schizophrenia and
autism in emotional and social processing. However, to date, no study has attempted to
compare the social cognitive performance of individuals with schizophrenia to
individuals with autism on a task that is well-characterized with respect to amygdala
functioning. The present study addresses this need by comparing two clinical groups
hypothesized to exhibit amygdala dysfunction, (i.e., autism and schizophrenia), with
those of controls as they participate in an established task previously determined to
implicate amygdala functioning. In this task, subjects judge the primary emotion being
portrayed in a series of static social scene images depicting complex social scenes with
faces either present or digitally erased. Adolphs and Tranel (34) first used this task with
amygdala-damaged patients and found that the accuracy of their judgments for negative
emotions did not improve to the same degree as controls when faces were present relative
to when they were absent.
Scanning of Scenes in Autism and Schizophrenia 5
Given that this task has been effectively employed with amygdala-lesioned
patients, and given that both autism and schizophrenia have reported amygdala
dysfunction, it was hypothesized that both disorders would exhibit similar social
cognitive impairments on these stimuli, most notably in a reduced ability to process
information from faces. It was predicted that, similar to results reported by Adolphs and
Tranel (34) for amygdala-damaged patients, performance of the two clinical groups
would not improve to the same degree as controls when faces were included compared to
when they were absent. To test this, we obtained two dependant measures: the accuracy
of subjects in judging the emotional content of these stimuli, and their eye movements
while doing so. The inclusion of eye-tracking not only enables a direct comparison of the
three groups on visual attention to social stimuli, but also allows for an examination of
the potential role played by the amygdala in social orientation. Adolphs, et al. (35)
recently clarified the relationship between the amygdala and affect recognition by
determining that it subserves the direction of attention to emotionally relevant areas of
the face. This finding suggests that the amygdala may help orient attention to socially
meaningful information and highlights the need for more broadly delineating the role of
the amygdala in social orientation. The use of eye-tracking in this study addresses this
need by examining perceptual mechanisms that may contribute to abnormal social
cognitive performance in two distinct clinical populations hypothesized to exhibit
amygdala dysfunction. Specific attention was given to fixation on faces of the scenes and
latency to fixate social regions of interest. Consistent with studies that have found
abnormal visual scanning of social stimuli in autism and schizophrenia (12-13, 33, 36-
37), both clinical groups were predicted to exhibit abnormal attention to social scenes
Scanning of Scenes in Autism and Schizophrenia 6
compared to controls. However, because impairments in orienting to social stimuli are a
well-characterize feature of autism from a very early age (38-40), it was hypothesized
that this variable might differentiate the two clinical groups.
Thirty individuals (10 with autism, 10 with schizophrenia and 10 typically
developing controls) participated in this study. Demographic variables can be found in
Table 1. The three groups did not differ statistically in chronological age or in full-scale
IQ on the Wechsler Abbreviated Scale of Intelligence (41). Individuals with autism were
recruited through referrals from local clinicians and through the North Carolina Autism
Subject Registry for participation in a larger study investigating the neuropsychological
characteristics of the disorder. DSM-IV diagnoses were confirmed using the Autism
Diagnostic Observational Schedule, Revised (42) and the Autism Diagnostic Interview,
Revised (43). Individuals with schizophrenia were recruited from the Schizophrenia
Treatment and Evaluation Program (STEP) at UNC Hospitals. Diagnoses were confirmed
using the Structured Clinical Interview for DSM-IV (SCID-P) and via chart review. All
were on a stabilized treatment of anti-psychotic medication at the time of testing, with a
mean Chlorpromazine equivalent dosage of 409.3mg (SD: 278.4) (dosage information
was not available for one subject). All were experiencing minimal symptoms based on
the Positive and Negative Syndrome Scale (PANSS; 44) at the time of testing (mean
positive, 7.5 (SD: 2.1); mean negative, 10.2 (SD: 4.2)). The group had been ill for a mean
of 4.2 years (SD: 3.1). Typically-developing individuals with no history of mental illness
or neurological impairment were recruited from the local community as controls for a
Scanning of Scenes in Autism and Schizophrenia 7
broader study of the neuropsychological features of autism. They were matched on age
and IQ with the autism subjects. This study was approved by the human subjects
committee at the University of North Carolina at Chapel Hill, and all participants signed
Stimuli and Task
The social scenes task (34) is comprised of static, high-quality black and white
images depicting a range of basic emotions. These images were originally presented to a
population of typically developing individuals for emotional assessment. Stimuli
producing a high reliability response rate were then included in the task. Because the task
was originally developed for use in patients with amygdala damage, who demonstrate
specific deficits in the processing of negative emotions, the bulk of images used in the
task were drawn from this affective category. We utilized a subset of social scenes that
maintained the task’s emphasis on negative emotions. Modal responses of the twelve
images used in this study defined the emotions of the scenes in the following way: four
were angry, three were afraid, two were sad, two were happy and one was surprise. Each
image varied in the number of people and objects in the scene, and all included both
facial expressions and non-face visual cues (i.e., body posture and gestures) that
conferred emotional information.
The duration of the social scenes task lasted approximately 15 minutes and
occurred within the context of a larger battery of cognitive assessment and
neuropsychological measures. Subjects sat approximately 56cm from a 1024 horizontal X
768 vertical video monitor and viewed stimuli subtending a horizontal visual angle of
14.2° and a vertical visual angle of 10.7°. Before the task began, the subject was fitted
Scanning of Scenes in Autism and Schizophrenia 8
with a head-mounted eyetracking system and presented with a brief procedure for
calibrating the point of regard data. Each subject was then shown social scene images one
at a time on a computer monitor, first in a block with the face region digitally removed
(the rest of the image remains unaltered) followed by a block in which the face region
was included. Images were displayed for three seconds each, at which point the seven
emotion choices (happy, surprised, afraid, angry, sad, disgusted and neutral) appeared at
the bottom of the screen. The image continued to be displayed until the subject verbally
selected the emotion he or she believed was being depicted in the scene. An experimenter
manually recorded the response and then triggered the presentation of the subsequent
image. After the completion of the first block, the second block was then presented. It
differed from the first only in that all face regions contained within the image were now
Eye-movement and point of regard data were recorded with a head-mounted
ISCAN series RK-464 remote infrared pupil-corneal reflection eye imaging system (45).
The eyetracking system transmitted to a host computer in real time at a 60HZ data stream
representing the subject’s point of regard within the stimulus scene. Each three second
epoch of eye movement data corresponding to the time between onset of a social scene
image and the display of the seven emotion choices at the bottom of the screen was
manually inspected for tracking integrity and served as our eyetracking window of
interest. In between the display of each image, a crosshair appeared at the center of the
screen in order to ensure that all scanpaths began at the same point for each subject. Eye
Scanning of Scenes in Autism and Schizophrenia 9
blinks resulted in missing data, but because the groups did not differ on total number of
eye blinks (F (2, 29) = .08, p = .92), these were excluded from subsequent data analyses.
The use of a mobile head-mounted eyetracker and a laptop computer enabled
testing to occur in either the home of the subject or in a laboratory on the campus of
UNC-Chapel Hill. Measures were taken to ensure that the testing environment (e.g.,
lighting, volume, etc.) was similar for each subject regardless of the testing location.
Although a chinrest was used in order to minimize head motion and establish near-
equivalent testing conditions for each subject, both head movement and minor differences
in the physical setup occasionally occurred, causing the scene camera of the eyetracker to
capture slightly different recording dimensions for each subject. In order to account for
these artifacts, the scene camera recording was later reviewed and a computer software
program (PFTrack, version 2.0) was used to track the edges of the laptop screen
throughout the testing session. A computer program was then devised to mathematically
adjust for the movement of the scene so that point-of-regard data for all subjects referred
to the same 1024 x 768 space, thereby enabling quantitative comparisons to be made.
Both the first author and a research assistant manually inspected the integrity of each
subject scanpath to ensure that the resulting output matched up with the point of regard
videos that were recorded during the testing session.
The groups were first compared on the accuracy of their emotional judgments
during the face absent and face present conditions. Next, fixation patterns to social
elements of the scene (i.e., faces and bodies) were analyzed. Because the area occupied
by face and body ROI is different and varies across image sets, comparing the proportion
Scanning of Scenes in Autism and Schizophrenia 10
of time spent on these regions is problematic. In order to compare the relative fixation
duration within ROIs across different ROI types and across different images, we created a
‘normalized region of interest (ROI) value’ for each ROI on each image. The area inside
the ROI is given an ROI value of ‘1’, while the area outside the ROI is given a ‘0’.
Across all pixels, the ROI value is then z-transformed to have the 0 mean and the unit
standard deviation. In this way, the larger area the ROI occupies, the smaller is the ROI
value assigned for each pixel. This technique therefore allows us to compensate for the
difference in the size of ROIs and directly compare the proportion of time spent on faces
and bodies across all images.
We also analyzed the location of the first saccade and the latency of the first
fixation onto the face ROI. The differential latency across groups and conditions (i.e.,
face present/absent) was analyzed by looking at the temporal evolution of normalized
ROI values (Figure 4). Normalized ROI values were used in order to adjust for the
inconsistency in latency values that occur as a result of the size and location of ROIs
varying across images. This process provided information about social orientation by
determining when subjects in each group started to look at faces in the scenes, and the
degree to which orientation was greater when the faces were present compared to when
they were absent.
Correlations between time spent on social regions and accuracy of emotion
judgments were also analyzed.
Group comparisons in the accuracy of emotional judgments
Scanning of Scenes in Autism and Schizophrenia 11
Behavioral responses were analyzed using a repeated measures ANOVA with
face condition (absent vs. present) as the within-subjects factor and group (autism vs.
schizophrenia vs. controls) as the between-subjects factor. The three groups did not differ
significantly on the overall accuracy of their emotional judgments (F (2, 27) = 1.41, ns).
All groups performed more accurately in the judgments when the face was present
relative to when it was absent (F (1, 27) = 14.83, p < .01), and a non-significant
interaction between face condition and group (F (2, 27) = .04, ns) suggests that the degree
of improvement in emotion accuracy between the face-absent and face-present conditions
did not differ between the three groups. Additionally, group behavioral responses did not
significantly differ on each of the five depicted emotions (happy, surprise, angry, afraid
and sad) in either the face condition.
Group comparisons in visual scanpaths
Figure 1 shows examples of visual scanpaths for an individual in each group on a
social scene image in both face conditions. The line overlaying the image represents the
scanpath of an individual subject as he examines the image. Figure 2 (left and center
columns) shows the mean fixation density for an image over 3 seconds across all subjects
in each group. Subtraction of fixation density maps between face conditions reveals that
all three groups spent more time on the facial ROI when faces were present, but that the
control group demonstrated the largest incremental gain between the face-absent and
face-present conditions (Figure 2, right column).
Across all images, the groups did not differ on either total number of fixations (F
(2, 27) = .86, ns) or total number of saccades (F (2, 27) = 1.55, ns). Fixation duration to
face regions was characterized by calculating the mean of the normalized face ROI
Scanning of Scenes in Autism and Schizophrenia 12
values over the three seconds of image inspection time (Figure 3, left column). Group
differences in normalized face ROI values were assessed using a repeated measures
ANOVA with face condition (absent vs. present) as the within-subjects factors and group
(autism vs. schizophrenia vs. controls) as the between-subjects condition. A significant
main effect emerged for face condition (F (1, 27) = 177.22, p < .01), demonstrating that
all three groups increased fixation duration to face regions when faces were present
compared to when they were absent. A significant interaction between condition and
group was also found (F (2, 27) = 6.58, p < .01), indicating that the groups differed in the
degree to which they increased their fixation duration to the face region when the face
was present. Tukey post hoc tests revealed that controls spent significantly greater
fixation duration on faces in the face-present condition than the autism group (-.96, p =
.01 ), and a trend level effect in the same direction was found compared to the
schizophrenia group (-66, p = .09). No differences, however, emerged between the
schizophrenia and autism groups (-.29, p = .60). Analysis on the total time spent visually
inspecting the face regions of the social scenes revealed a similar pattern of results: while
all three groups spent greater time inspecting face regions when faces were present
compared to when they were absent (F (1, 27) = 220.33, p < .01), the control group
increased gaze time on the face region in the face-present condition to a greater degree
than either the autism and schizophrenia group (F (2, 27) = 6.79, p < .01).
Despite comparable proportion of gaze time between faces and bodies for each group
(see Table 2), faces resulted in dramatically greater normalized ROI values relative to
bodies for all groups (F (1, 27) = 502.56, p < .01, see Figure 3). An ANOVA on
normalized ROI values for bodies revealed a main effect for condition (F (1, 27) = 37.69,
Scanning of Scenes in Autism and Schizophrenia 13
p < .01), indicating that across all groups, fixation duration on bodies was significantly
greater when faces were absent. The group X condition interaction was not significant (F
(2, 27) = .732, p = .49), suggesting that fixation duration on bodies decreased to the same
degree for each group in the face present condition relative to the face absent condition.
Spatio-temporal characteristics of initial fixation patterns
Next, we concentrated on the analysis of the fixation pattern during the initial
inspection period (<1 sec) because we hypothesized that the attentional orienting system
would be crucially involved in the early phase of scene examination. A main effect of
condition indicated that all three groups fixated first to a face region more often when
faces were present (F (1, 27) = 4.38, p < .05). However, the groups differed in how much
they increased the percentage of their first fixations to a face region in the face-present
condition (F (2, 27) = 3.88, p < .05). Tukey post hoc tests revealed that the percentage of
time the first fixation by controls was on the face region in the face-present condition was
significantly greater than the in the schizophrenia group (-36.62, p < .01) and a trend-
level effect in the same direction was found compared to the autism group (-18.33, p =
.10). Additionally, the autism group maintained a trend-level advantage relative to the
schizophrenia group (-18.29, p = .10).
The mean latency of the first fixation on face ROI was tabulated in Table 2 and
hint at interesting group x condition differences. When the face was present, the autism
and the control groups rapidly oriented to face ROIs (<0.50 sec) while the schizophrenia
group was slower (>0.63 sec). When faces were absent, the latency to face ROI increased
dramatically in controls, less so in schizophrenia, and not at all in autism. The log-
transformed latency submitted to a 3-way ANOVA (group X condition X trial) revealed
Scanning of Scenes in Autism and Schizophrenia 14
significant main effects for condition (F (1, 27) = 15.03, p < .01) and trial (F (11, 297) =
14.42, p < .01), a trend level main effect for group (F (2, 27) = 2.84, p < .01), and a trend-
level group x condition interaction (F (2, 27) = 3.23, p = .055).
Group differences in the percentage and the latency of first fixation on face
regions led us to analyze the temporal evolution of normalized ROI values in order to
more thoroughly investigate the timing of social orientation for each group in each
condition. Of particular interest was estimating when subjects began to increase their
fixation probability on the face ROI when the face was present compared to when it was
absent. Note that using normalized ROI values compensated for the variability of the size
of face ROI across images (i.e., reducing the main effect of trials). We then discarded all
the trials where subjects’ fixation started on a face ROI in order to include only those
trials in which orientation was required.
As can be seen in Figure 4 (top), while both the control and schizophrenia groups
orient to the face region quicker when the face is present (green and blue solid lines) than
when the face is absent (broken lines), the autism group orients to the face region at the
same speed whether the face is present or not (red lines). We estimated the point in time
the normalized ROI values diverged, using a running paired t-test. We determined the
first of the three consecutive points with P < 0.05, then lowpass filtered P-values, and
finally interpolated the latency as the crossing time where P-values first became P<0.05.
The latency for the differential normalized ROI values were 0.20, 0.68, and 1.03 sec for
the control, schizophrenia, and autism groups (Figure 4, bottom). We also estimated
when the difference in differential normalized ROI values across the groups emerged,
using a running-one-way ANOVA (the same running-t-test procedure as above was
Scanning of Scenes in Autism and Schizophrenia 15
performed). The differential normalized ROI values diverged at 0.22 sec across the
groups (Figure 4, bottom), because the control subjects started discriminating face
conditions at 0.20 sec, while the other two groups did not. To check the robustness of our
finding, we analyzed the latency without removing any trials: 0.17, 0.45, and 0.80 sec for
the control, schizophrenia, and autism groups, indicating no qualitative difference and the
same rank order. To exclude a possibility that the difference across the groups emerged
due to the accuracy of the fixation, we blurred the face ROI using a Gaussian kernel with
standard deviation of 1º visual angle. The normalized ROI values were created for each
image and the differential latency was estimated. Again, there was no qualitative
difference and the rank order remained the same (0.17, 0.69, and 1.03 sec for the control,
schizophrenia, and autism groups).
While the autism group and the control group orient to the face ROI at the same
speed (mean latency ~ 0.50 sec), the schizophrenia group does so at a much slower rate
(median latency of the first fixation was > 0.63 sec). Despite this delay in overall
orientation latency, the schizophrenia group begins to exhibit a fixation duration
advantage for faces in the face-present condition relative to the face-absent condition
much earlier (0.68 sec) than the autism group (1.03 sec). In other words, only the autism
group is failing to modulate orientation based on the presence of the face. Both the
control and schizophrenia groups orient to the face ROI quicker when the face is present;
the autism group does not. All three groups eventually demonstrate a fixation duration
advantage for faces in the face present condition relative to the face absent condition
(Figure 2 and 3), but only the control group does so immediately (0.2 sec) following the
onset of image presentation (Figure 4).
Scanning of Scenes in Autism and Schizophrenia 16
Correlations between visual scanning patterns and emotional judgments
Overall accuracy of emotional judgments was not significantly associated with
amount of scanning time on the face region for the control and schizophrenia groups in
either the face condition. This pattern of results did not differ for any of the five emotion
types. A significant negative correlation, however, was found between gaze time on the
face region and overall emotional accuracy for the autism group in the face-present
condition only (r² = -.71, p < .05), indicating that greater amounts of time spent by the
autism group scanning the face region when the face was present was associated with
worse performance on judgments of emotion. This suggests that prolonged gaze time on
the face in autism may be indicative of difficulty in decoding facial emotion. Gaze time
on body regions was not significantly associated with emotion accuracy for any of the
three groups in either the face absent or the face present condition.
Analysis of visual attention patterns to social scene images revealed provocative
differences between the autism, schizophrenia and control groups. While all groups spent
a similar proportion of their gaze time on the face region when the face was absent,
controls increased their gaze time to faces in the face-present condition to a greater
degree than the autism and the schizophrenia groups. This effect extended to the location
of first fixation: the control group increased the percentage of time their first fixation was
on a face region of a scene when the faces were included to a greater degree than the
other two groups. The clinical groups, however, did not differ from each other. This
similarity in abnormal scanning behavior suggests that individuals with autism and
Scanning of Scenes in Autism and Schizophrenia 17
schizophrenia may not utilize facial information to the same extent as controls when
assessing the emotional content of a complex social scene.
An important distinction between the autism and schizophrenia groups emerged
when temporal evolution analyses were conducted to examine latencies to fixate faces in
the images. Only the autism group failed to orient to faces more rapidly when faces were
present relative to when they were absent. This suggests that in autism, social orientation
may not be modulated by the presence of emotional information, even in the context of
an emotion recognition paradigm. The failure of the autism group to orient to face
regions faster when faces were included may reflect attentional mechanisms driven more
by the general presence of a face region rather than the quality of meaningful information
it contains. This finding is consistent with impairments found in autism from a very early
age in social attention and orientation (39-41), and suggests that abnormalities in social
orienting may be a primary deficit in autism that persists throughout the lifespan. In this
study, the impairment was also specific to the autism group: even though the
schizophrenia group demonstrated a shared impairment in fixating face regions in the
social scenes, they did not exhibit the same failure to modulate orientation speed based
on the presence of the face. In contrast, the schizophrenia group was slower to orient to
face regions relative to the other two groups, although it is impossible for the present
study to determine whether the use of anti-psychotic medication contributed to this effect.
Despite this generalized orientation delay, the schizophrenia group modulated
their latency to fixate faces in the scenes based on the presence of the social information
much sooner than did the autism group. This distinction demonstrates the profitability of
employing temporal evolution analyses for examining visual scanning behavior in autism
Scanning of Scenes in Autism and Schizophrenia 18
and schizophrenia: while gaze time on faces did not differentiate the two clinical groups,
inspection of their latency to orient to faces did. Although a number of studies
independently examining social cognitive impairments in autism and schizophrenia hint
at significant overlap at both the behavioral and neural levels, the social orientation
differences found between the groups here suggest that the underlying mechanisms
subserving social cognitive dysfunction in the two disorders may differ.
Even with these abnormalities in visual scanning, the autism and schizophrenia
groups exhibited intact behavioral performance in the recognition of emotions portrayed
in the scenes. This may suggest that the social scenes task is not as sensitive a measure
for revealing emotion recognition deficits in autism and schizophrenia as those used in
other studies. The emotional displays included here were only of simple emotions, often
exaggeratedly portrayed for dramatic effect, and their depiction may not have been subtle
enough to elicit group differences. Nevertheless, the behavioral performance by the two
clinical groups in this study differs importantly from results previously reported for
amygdala-damaged patients on the same task, whose emotional accuracy only improved
minimally, if at all, when the face region was present (34). While this suggests that
neither autism nor schizophrenia exhibit impairments in emotion recognition associated
with amygdala damage previously found for this task, the abnormalities found for both
groups in social fixation and orientation are similar to those reported for a patient with
bilateral amygdala lesions (35). The role of the amygdala in directing attention to
emotionally-relevant information provides a potentially provocative explanation for both
the failure of the clinical groups to exhibit normal levels of facial attention and the
finding that the social orientation latency of the autism group did not differ as a function
Scanning of Scenes in Autism and Schizophrenia 19
of the presence or absence of the face. The findings here are consistent with research
indicating abnormal visual attention to social stimuli in autism (12-13, 33) and
schizophrenia (36-37), and highlight the need for further studies, particularly those using
neuroimaging techniques, to assess the degree to which amygdala dysfunction may
underlie deficits in social orientation in both disorders.
Future research should also continue to directly compare autism and
schizophrenia in order to elucidate why two heterogeneous disorders with differing
etiologies and developmental pathways often, in adulthood, exhibit similar social
cognitive profiles. Contrasting performance across clinical groups with overlapping
features not only provides greater insight into areas of similarity, but also helps illuminate
meaningful differences in phenomenology and neural circuitry between the disorders.
The common practice of simply comparing clinical groups with normal populations may
therefore limit the amount of information to be gained in a study by failing to highlight
clinical characteristics specific to the disorder being investigated. By revealing a shared
impairment in facial fixation but an important distinction in social orienting, the current
study exemplifies the benefits of this approach by demonstrating how a direct comparison
of autism and schizophrenia can expose both commonalities and dissociations that refine
our understanding of the social cognitive deficits that characterize each disorder.
Scanning of Scenes in Autism and Schizophrenia 20
n =10) Schizophrenia (
n =10) Control (
Age23.0 (5.27)28.1 (5.07) 22.4 (6.26)
IQ107.8 (17.15)98.5 (12.99)108.1 (21.57)
Scanning of Scenes in Autism and Schizophrenia 21
Variable Autism Schizophrenia Controls F
Mean Fixations per Image
Mean Saccades per Image
% of Gaze Time on Face
% of Gaze Time on Body
% of 1st Fixation on Face
Mean Latency to Face (ms)
35.01 (17.90) 27.47 (24.26)
513 (249) 734 (233)
37.50 (15.84) 0.71
Mean Fixations per Image
Mean Saccades per Image
% of Gaze Time on Face
% of Gaze Time on Body
% of 1st Fixation on Face
Mean Latency to Face (ms)
42.51 (23.70) 24.22 (20.89)
500 (274) 631 (193)
60.84 (10.43) 9.09*
Note. N = 30. Values marked with an asterisk (*) are significant at p < .01.
Scanning of Scenes in Autism and Schizophrenia 22
Figure 1. Visual Scanpath Examples from a Control Subject, a Subject with Autism, and
a Subject with Schizophrenia
Scanning of Scenes in Autism and Schizophrenia 23
Figure 2. Fixation behavior by each of the three groups on a social scene image. Higher-
contrast spots indicate areas of more intense fixation; the right column shows the
difference in fixation between the two face conditions.
Scanning of Scenes in Autism and Schizophrenia 24
Figure 3. Normalized face (left column) and body (right column) ROI values for each
group. White bars indicate the face absent condition and black bars indicate the face
present condition. The bottom row shows the difference in normalized values between
the face present and face absent conditions.
Scanning of Scenes in Autism and Schizophrenia 25
Figure 4. Normalized face ROI values plotted by time for each group (autism, red;
normal, green; schizophrenia, blue) in the face-absent (broken lines) and the face-present
(solid lines) conditions. The bottom chart shows the difference between the with-face and
without-face conditions. The blur above and below the solid line indicate one standard
error. Images in which fixation began on a face region were excluded.
Scanning of Scenes in Autism and Schizophrenia 26
This project was funded by grants from the National Institutes of Mental Health
(STAART grant U54 MH66418; J. Piven), the Cure Autism Now Foundation (R.
Adolphs), the National Alliance For Autism Research/Autism Speaks (R. Adolphs), and
Johnson and Johnson Pharmaceutical Research and Development, LLC, USA (D. Penn).
Noah Sasson was supported by National Institute of Child Health and Human
Development Grant T32-HD40127. We are grateful to Monica Stubbs, Ellen Cohen,
Morgan Parlier, Shannon Gallagher and Todd Corl for their help collecting and coding
these data, Eden Kung for his programming assistance, and Grace Baranek for her help
recruiting participants. We would also like to thank all the individuals who participated in
Scanning of Scenes in Autism and Schizophrenia 27
1. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental
disorders (4th ed.). Washington, DC: Author.
2. Brothers L. (1990) The social brain: a project for integrating primate behaviour and
neuropsychology in a new domain. Concepts in Neuroscience, 1, 27-51.
3. Abdi Z, Sharma T. (2004). Social cognition and its neural correlates in Schizophrenia
and Autism. CNS Spectrums, 9, 335-343.
4. Pinkham AE, Penn DL, Perkins DO, Lieberman J. (2003). Implications for the neural
basis of social cognition for the study of schizophrenia. Am J Psychiatry , 160,
5. Pelphrey K, Adolphs R, Morris JP. (2005). Neuroanatomical substrates of social
cognition dysfunction in autism.Mental Retardation and Developmental
Disabilities Research Reviews, 10, 259-271.
6. Baron-Cohen S. (1995). Mindblindness. Boston: MIT Press/Bradford Books.
7. Corcoran R. (2001). Theory of mind and schizophrenia. In PW Corrigan & DL Penn
(Eds.), Social cognition and schizophrenia (pp. 149-174). Washington, DC:
American Psychological Association.
8. Yirmiya N, Erel O, Shaked M, Solomonica-Levi D. (1998). Meta-analyses
comparing theory of mind abilities of individuals with autism, individuals with
mental retardation, and normally developing individuals. Psychol Bull, 124, 283-
9. Celani G, Battacchi MW, Arcidiacono L. (1999). The understanding of the emotional
Scanning of Scenes in Autism and Schizophrenia 28
meaning of facial expressions in people with autism. J Autism Dev Disord, 29, 57-
10. Kohler CG, Brennan AR. (2004). Recognition of facial emotions in schizophrenia.
Current Opinion in Psychiatry, 17, 81-86.
11. Archer J, Hay, DC, Young, AW. (1994). Movement, face processing, and
schizophrenia: Evidence of a differential deficit in expression analysis. Br J Clin
Psychol , 33, 517-528.
12. Klin A, Jones W, Schultz R, Volkmar F, Cohen D. (2002). Defining and Quantifying
the Social Phenotype in Autism. Am J Psychiatry , 159, 895-908.
13. Klin A, Jones W, Schultz R, Volkmar F, Cohen D. (2002). Visual fixation patterns
during viewing of naturalistic social situations as predictors of social competence
in individuals with autism. Arch Gen Psychiatry, 59, 809-816.
14. Craig JS, Hatton C, Craig FB, Bentall RP. (2004). Persecutory beliefs, attributions
and theory of mind: Comparison of patients with paranoid delusions, Asperger’s
syndrome and healthy controls. Schizophr Res, 69, 29-33.
15. Pilowsky T, Yirmiya N, Arbelle S, Mozes T. (2000). Theory of mind abilities in
children with schizophrenia, children with autism, and normally developing
children. Schizophr Res, 42, 145-155.
16. Bolte S, Poustka F. (2003). The recognition of facial affect in autistic and
schizophrenic subjects and their first-degree relatives. Psychol Med, 33, 907-915.
17. Adolphs R, Tranel D, Damasio H, Damasio AR. (1995). Fear and the human
amygdala. J Neurosci, 15, 5879-5891.
18. Adolphs R. (2001). The neurobiology of social cognition. Curr Opin Neurobiol, 11,
Scanning of Scenes in Autism and Schizophrenia 29
19. Amaral DG. (2003). The amygdala, social behavior, and danger detection. Ann N Y
Acad Sci, 1000, 337-347.
20. Green MJ, Phillips ML. (2004). Social threat perception and the evolution of
paranoia. Neurosci Biobehav Rev, 28, 333-342.
21. Joyal CC, Laakso MP, Tiihonen J, Syvalahti E, Vilkman H, Laakso A,Alakare B,
Räkköläinen V, Salokangas RKR, Hietala J.. (2003). The amygdala and
schizophrenia: A volumetric magnetic resonance imaging study in first-episode
neuroleptic-naïve patients. Biol Psychiatry, 54, 1302-1304.
22. Nelson MD, Saykin AJ, Flashman LA, Riordan HJ. (1998). Hippocampal volume
reduction in schizophrenia as assessed by magnetic resonance imaging. Arch Gen
Psychiatry, 55, 433-440.
23. Wright IC, Rabe-Hesketh S, Woodruff PWR, David AS, Murray RM, & Bullmore
ET. (2000). Meta-analysis of regional brain volumes in schizophrenia. Am J
Psychiatry , 157, 16-25.
24. Brambilla P, Hardan A, di Nemi SU, Perez J, Soares JC, Barale F. (2003). Brain
anatomy and development in autism: Review of structural MRI studies. Brain Res
Bull, 61, 557-569.
25. Baron-Cohen S, Ring H, Wheelwright S, Bullmore E, Brammer M, Simmons A,
Williams SC. (1999). Social intelligence in the normal and autistic brain: An
fMRI study. Eur J Neurosci, 11, 1891-1998.
26. Critchley HD, Daly EM, Bullmore ET, Williams,SCR, Van Amelsvoort T, Robertson
DM, Rowe A, Phillips M, McAlonan G, Howlin P, Murphy DG. (2000). The
Scanning of Scenes in Autism and Schizophrenia 30
functional neuroanatomy of social behaviour: Changes in cerebral blood flow
when people with autistic disorder process facial expressions. Brain, 123, 2203-
27. Taylor SF, Liberzon I, Decker LR, Koeppe RA. (2002). A functional anatomic study
of emotion in schizophrenia. Schizophr Res, 58, 159-172.
28. Adolphs R, Sears L, Piven J. (2001). Abnormal processing of social information from
faces in autism. Journal of Cognitive Neuroscience, 13, 232-240.
29. Baron-Cohen S, Wheelwright S, Jolliffe T. (1997). Is there a "language of the eyes"?
Evidence from normal adults and adults with autism or Asperger syndrome.
Visual Cognition, 4, 311-331.
30. Oguz K, Rita E, Miklosne P, Szabolcs K, Zoltan J. (2003). The relationship between
“theory of mind” and IQ in patients with schizophrenia. Psychiatria Hungarica,
31. Capps L, Yirmiya N, Sigman MD. (1992). Understanding of simple and complex
emotions in non-retarded children with autism. J Child Psychol Psychiatry , 33,
32. Evangeli M, Broks P. (2000) Face processing in schizophrenia: Parallels with the
effects of amygdala damage. Cognitive Neuropsychiatry, 5, 81-104.
33. Pelphrey KA, Sasson NJ, Reznick JS, Paul G, Goldman BD, Piven, J. (2002). Visual
scanning of faces in autism. J Autism Dev Disord, 32, 249-261.
34. Adolphs R, Tranel D. (2003). Amygdala damage impairs emotion recognition from
scenes only when they contain facial expressions. Neuropsychologia, 41, 1281-
Scanning of Scenes in Autism and Schizophrenia 31
35. Adolphs R, Gosselin F, Buchanan TW, Tranel D, Schyns P, Damasio AR. (2005). A
mechanism for impaired fear recognition after amygdala damage. Nature, 433,
36. Phillips ML, David AS. (1997). Visual scanpaths are abnormal in deluded
schizophrenics. Neuropsychologia, 35, 99-105.
37. Williams LM, Loughland CM, Gordon E, Davidson D. (1999). Visual scanpaths in
schizophrenia. Is there a deficit in face recognition? Schizophr Res, 40, 189-199.
38. Dawson G, Meltzoff AN, Osterling J, Rinaldi J, Brown E. (1998). Children with
autism fail to orient to naturally occurring social stimuli. J Autism Dev Disord, 28,
39. Dawson G, Toth K, Abbott R, Osterling J, Munson J, Estes A, Liaw J. (2004). Early
social attention impairments in autism: Social orienting, joint attention, and
attention to distress. Dev Psychol, 40, 271 - 283.
40. Swettenham J, Baron-Cohen S, Cox A, Baird G, Drew A, Charman T, Rees L,
Wheelwright S. (1998). The frequency and distribution of spontaneous attention
shifts between social and nonsocial stimuli in autistic, typically developing, and
nonautistic developmentally delayed infants. J Child Psychol Psychiatry , 39,
41. Wechsler D. (1997). Wechsler Adult Intelligence Scale—3rd Edition (WAIS-3®) San
Antonio, TX: Harcourt Assessment.
42. Lord C, Rutter M, Goode S, Heemsbergen J, Mawhood L, Schopler E. (1989).
Autism diagnostic observation schedule: A standardized observation of
communicative and social behavior. J Autism Dev Disord, 19, 185-212.
Scanning of Scenes in Autism and Schizophrenia 32 Download full-text
43. Le Couteur A, Rutter M, Lord C, Rios P, Robertson S, Holdgrafer M, McLennan J.
(1989). Autism Diagnostic Interview: A Standardized Investigator-Based
Instrument. J Autism Dev Disord, 19, 363-387.
44. Kay SR., Opler LA, Fiszbein A. In: Positive and Negative Syndrome Scale (PANSS)
Rating Manual, Multihealth Systems, Inc., Toronto, Canada (1992).
45. ISCAN Inc., Cambridge, MA, USA.