ArticlePDF Available

Brain responses to biological motion predict treatment outcome in young adults with autism receiving Virtual Reality Social Cognition Training: Preliminary findings


Abstract and Figures

Autism Spectrum Disorder (ASD) is characterized by remarkable heterogeneity in social, communication, and behavioral deficits, creating a major barrier in identifying effective treatments for a given individual with ASD. To facilitate precision medicine in ASD, we utilized a well-validated biological motion neuroimaging task to identify pretreatment biomarkers that can accurately forecast the response to an evidence-based behavioral treatment, Virtual Reality-Social Cognition Training (VR-SCT). In a preliminary sample of 17 young adults with high-functioning ASD, we identified neural predictors of change in emotion recognition after VR-SCT. The predictors were characterized by the pretreatment brain activations to biological vs. scrambled motion in the neural circuits that support (a) language comprehension and interpretation of incongruent auditory emotions and prosody, and (b) processing socio-emotional experience and interpersonal affective information, as well as emotional regulation. The predictive value of the findings for individual adults with ASD was supported by a regression-based multivariate pattern analysis with cross validation. To our knowledge, this is the first pilot study that shows neuroimaging-based predictive biomarkers for treatment effectiveness in adults with ASD. The findings have potentially far-reaching implications for developing more precise and effective treatments for ASD.
Content may be subject to copyright.
Brain responses to biological motion predict treatment outcome in
young adults with autism receiving Virtual Reality Social Cognition
Training: Preliminary ndings
Y.J. Daniel Yang
, Tandra Allen
, Sebiha M. Abdullahi
, Kevin A. Pelphrey
Fred R. Volkmar
, Sandra B. Chapman
Autism and Neurodevelopmental Disorders Institute, The George Washington University and Children's National Health System, Washington, DC 20052,
Child Study Center, Yale University School of Medicine, New Haven, CT 06520, USA
Center for BrainHealth, The University of Texas at Dallas, Dallas, TX 75235, USA
article info
Article history:
Received 10 November 2016
Received in revised form
2 March 2017
Accepted 27 March 2017
Available online 29 March 2017
Virtual reality
Emotion recognition
Theory of mind
Predictive biomarker
Biological motion
Autism Spectrum Disorder (ASD) is characterized by remarkable heterogeneity in social, communication,
and behavioral decits, creating a major barrier in identifying effective treatments for a given individual
with ASD. To facilitate precision medicine in ASD, we utilized a well-validated biological motion neu-
roimaging task to identify pretreatment biomarkers that can accurately forecast the response to an
evidence-based behavioral treatment, Virtual Reality-Social Cognition Training (VR-SCT). In a preliminary
sample of 17 young adults with high-functioning ASD, we identied neural predictors of change in
emotion recognition after VR-SCT. The predictors were characterized by the pretreatment brain activa-
tions to biological vs. scrambled motion in the neural circuits that support (a) language comprehension
and interpretation of incongruent auditory emotions and prosody, and (b) processing socio-emotional
experience and interpersonal affective information, as well as emotional regulation. The predictive
value of the ndings for individual adults with ASD was supported by regression-based multivariate
pattern analyses with cross validation. To our knowledge, this is the rst pilot study that shows
neuroimaging-based predictive biomarkers for treatment effectiveness in adults with ASD. The ndings
have potentially far-reaching implications for developing more precise and effective treatments for ASD.
©2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND
license (
1. Introduction
One of the major barriers in therapy for individuals with Autism
Spectrum Disorder (ASD) (APA, 2013) is the difculty to identify
appropriate and effective treatments for a given individual with
ASD. On the one hand, there is a remarkable variation and het-
erogeneity within the spectrum (Hahamy, Behrmann, &Malach,
2015; Lombardo et al., 2016), which makes it difcult for a single
treatment to t all individuals with ASD. On the other hand, people
often need to spend considerable amount of resources (e.g., time,
money) in trying out various treatment protocols before they are
able to identify the most appropriate intervention. This problem is
particularly severe in adults with ASD, where intervention research
has been very limited. To facilitate the tting process and reduce
potential waste of resources, it is crucial to develop objective pre-
dictors for treatment outcome in ASD, especially for adults with
ASD, which would directly accelerate the long-term goal of preci-
sion medicine (Insel, 2014) in ASD.
In this research, we used a well-validated biological motion
functional magnetic resonance imaging (fMRI) paradigm (Kaiser
et al., 2010), which robustly engages the neural circuits support-
ing both socio-emotional and socio-cognitive components of social
information processing, to identify pretreatment predictive bio-
markers that can accurately forecast the response to an evidence-
based behavioral intervention in young adults with ASD. The bio-
logical motion videos feature an adult engaging in children's games
and social actions (e.g., waving, pat-a-cake, and peek-a-boo). Prior
research has shown that social orienting to biological motion is
evolutionarily well-conserved and fundamental to adaptive social
*Corresponding author. Autism and Neurodevelopmental Disorders Institute,
The George Washington University and Children's National Health System, 2300 I
St. NW, Washington, DC 20052, USA.
E-mail address: (Y.J.D. Yang).
Contents lists available at ScienceDirect
Behaviour Research and Therapy
journal homepage:
0005-7967/©2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (
Behaviour Research and Therapy 93 (2017) 55e66
engagement (Heberlein &Adolphs, 2004; Johnson, 2006; Simion,
Regolin, &Bulf, 2008; Vallortigara, Regolin, &Marconato, 2005;
Yang, Rosenblau, Keifer, &Pelphrey, 2015). The biological motion
fMRI task has revealed key brain regions implicated in core ASD
decits (Allison, Puce, &McCarthy, 2000; Kaiser et al., 2010; McKay
et al., 2012; Yang et al., 2015), including the ventrolateral prefrontal
cortex (vlPFC), ventromedial prefrontal cortex (vmPFC), posterior
superior temporal sulcus (pSTS), amygdala, and fusiform gyrus
(FFG). These regions are implicated in various functions. Generally
speaking, vlPFC, vmPFC, and amygdala are more closely related to
socio-emotional processing and emotion regulation (Etkin, Buchel,
&Gross, 2015; Kanske, Heissler, Schonfelder, Bongers, &Wessa,
2011; Phelps &LeDoux, 2005), while pSTS and FFG are more
closely related to socio-cognitive processing and social information
integration (Deen, Koldewyn, Kanwisher, &Saxe, 2015; Saggar,
Shelly, Lepage, Hoeft, &Reiss, 2014; Yang et al., 2015). Recently,
research has successfully applied the biological motion fMRI task in
identifying predictive biomarkers for treatment outcome in young
children with ASD receiving Pivotal Response Treatment (Yang
et al., 2016), which marks the rst evidence that neuroimaging-
based task can effectively predict treatment effectiveness in ASD.
This adds to the likelihood that the biological motion fMRI task may
be used to identify neuropredictive biomarkers in young adults
with ASD.
The treatment approach investigated in this research is Virtual
Reality-Social Cognition Training (VR-SCT) (Kandalaft, Didehbani,
Krawczyk, Allen, &Chapman, 2013), which is a short-term trial
and consisted of 5-week treatment: two 1-h sessions per week
with a total of 10 sessions. Recent research highlights the potential
benets of using Virtual Reality (VR) as an effective tool in training
social skills for individuals with ASD (Bellani, Fornasari, Chittaro, &
Brambilla, 2011; Kandalaft et al., 2013; Maskey, Lowry, Rodgers,
McConachie, &Parr, 2014; Parsons &Mitchell, 2002; Wainer &
Ingersoll, 2011). Reviews on VR studies suggest that there are
several advantages of using VR environments to train social skills
(Bellani et al., 2011; Parsons &Mitchell, 2002). Specically, VR can
simulate real-world contexts in a safe, non-threatening setting in
which participants can practice commonly encountered social in-
teractions (Bellani et al., 2011; Kandalaft et al., 2013; Parsons,
Mitchell, &Leonard, 2005). VR also affords the user the opportu-
nity to be immersed into the training by promoting engagement
and a sense of presence within the simulated experience (Wallace
et al., 2010). As presented in the previous studies (Didehbani,
Allen, Kandalaft, Krawczyk, &Chapman, 2016; Kandalaft et al.,
2013), the format of immersive role-play in VR-SCT can afford the
participant a variety of opportunities to become engaged in
training, while reducing social anxiety and allowing for a dynamic
practice experience without negative real-world social conse-
quences. It is safe to try and fail, because scenarios are controlled by
a clinician and allow for repeated practice using targeted social
strategies. As well, the technology itself supports the realism of
immersive role-play conversation by allowing a clinician to change
his/her appearance, voice or even the physical setting of the con-
versation, which are key elements that in-person treatment can
hardly provide.
Furthermore, VR and computer technologies as a training
method are highly motivating platforms for many individuals with
ASD (Chen &Bernard-Opitz, 1993; Moore &Calvert, 2000; Parsons
&Mitchell, 2002). The ability to customize and practice dynamic
social scenarios across multiple training sessions is also a strength
of VR, which facilitates opportunities for generalization of social
skills learned in VR to everyday life interactions (Bellani et al., 2011;
Didehbani et al., 2016; Parsons &Cobb, 2011; Tzanavari,
Charalambous-Darden, Herakleous, &Poullis, 2015). Generaliza-
tion of trained skills to the real-world was also reported in a pre-
vious study (Maskey et al., 2014), whereby gradual exposure to a
specic anxiety was presented in a visual context through VR,
which combined with cognitive behavioral therapy resulted in
participants reporting reduced anxiety in their everyday life.
Similarly, in the previous VR-SCT study (Kandalaft et al., 2013),
participants reported that their social functioning has continued to
improve several months after the end of training. For adolescents
and young adults with ASD, previous research has also shown that
VR offers the exibility to target social skills in isolation, such as
social appropriateness with spatial proximity and knowing what to
say in a job interview (Cheng, Moore, McGrath, &Fan, 2005;
Parsons, Mitchell, &Leonard, 2004; Smith et al., 2014; Trepagnier,
Olsen, Boteler, &Bell, 2011), or to target multiple skills in one
platform (Kandalaft et al., 2013), including emotion recognition,
theory of mind, and social functioning collectively.
The main principles of the VR-SCT intervention utilized in this
study were based on prior VR-SCT studies (Didehbani et al., 2016;
Kandalaft et al., 2013). In these studies, VR-SCT involved a semi-
manualized structured prompt used by the clinicians for all par-
ticipants. Clinicians used a scripted personality and response style
for each character they played that further promoted standardiza-
tion across participants. Even though the prompts were structured
and repeated across scenarios, participantsindividual responses
allowed for exible real-time responses by both the clinician and
the participant. Similarly, in the current study, a manualized
approach was used to engage participants in a conversation and the
participants partially customized their experience through their
own responses. As shown in these prior studies (Didehbani et al.,
2016; Kandalaft et al., 2013), the role-play method has been uti-
lized in both young adult and pediatric populations with similar
improvements, regardless of the age group, in emotion recognition
and theory of mind. Overall, VR-SCT offers an engaging, interactive,
and individualized platform for training and improving socio-
emotional and socio-cognitive abilities for individuals with ASD.
In this study, we investigated whether a pre-treatment biolog-
ical motion fMRI task could predict therapeutic response to VR-SCT
in young adults with ASD. The biological motion fMRI task was
chosen also because it measures key socio-emotional and socio-
cognitive processing, which correspond to the treatment targets
of VR-SCT. Linking to the biological motion fMRI task and VR-SCT,
we utilized two separate behavioral tasks to measure behavioral
changes in emotional and cognitive aspects of social information
processing, respectively: for the emotional component, we
measured behavioral changes in emotion-recognition ability, while
for the cognitive component, we measured behavioral changes in
theory-of-mind ability. As VR-SCT has been demonstrated to
ACS-SP Advanced Clinical Solutions-Social Perception
ADOS Autism Diagnostic Observation Schedule
ASD Autism Spectrum Disorder
BIO Biological Motion
BOLD Blood Oxygen Level Dependent
CDT Cluster-Dening Threshold
GLM General Linear Model
LOSO Leave-One-Subject-Out
MRI Magnetic Resonance Imaging
MVPA Multivariate Pattern Analyses
SCR Scrambled Motion
SRS Social Responsiveness Scale
VR-SCT Virtual Reality-Social Cognition Training
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6656
improve emotion recognition and theory of mind in prior research
(Didehbani et al., 2016; Kandalaft et al., 2013), we expected that
there will be similar behavioral improvement on average (one-
tailed). Moreover, central to the aim of the current study, we hy-
pothesized that the biological motion fMRI task will be able to
identify pretreatment neurobiological markers that can predict
change in emotion recognition and theory of mind, respectively.
2. Methods
2.1. Participants
Study participants included 17 young adults with a primary
diagnosis of ASD (Mage ¼22.50 years, SD ¼3.89; 2 females, 15
males), recruited from two research sites: Yale Child Study Center
(YCSC) for 7 participants, and Center for BrainHealth at The Uni-
versity of Texas at Dallas (CBH-UTD) for the other 10 participants. A
recent neuroprediction study involving young children with autism
receiving Pivotal Response Treatment (PRT) (Yang et al., 2016)
shows that the effect size as measured by the linear bivariate cor-
relation between pretreatment brain activities and behavioral
changes induced by treatment is about r¼0.54e0.81. Accordingly,
for power ¼0.80 and
¼0.05, G*Power (Faul, Erdfelder, Buchner, &
Lang, 2009) suggests that it requires at least 17 participants to be
sufciently powered to detect similar neuropredictive effects,
while assuming that the neuropredictive effect sizes are similar
across different age groups (young children vs. young adults) and
different treatment approaches (PRT vs. VR-SCT).
IQ was measured using the Wechsler Abbreviated Scale of In-
telligence (WASI) (Wechsler, 1999, 2011). In terms of full-scale IQ,
all participants were relatively high-functioning (IQ 80; M
IQ ¼109.65, SD ¼13.32). All participants met DSM-V (APA, 2013)
diagnostic criteria for ASD as determined by the results of a gold-
standard diagnostic instrument, the Autism Diagnostic Observa-
tion Schedule (Gotham, Risi, Pickles, &Lord, 2007; Hus &Lord,
2014; Lord et al., 2000)dadministered by research-reliable clini-
cians and licensed clinical psychologists. Unfortunately, the ADOS
subdomain scores were not available for two participants because
they were evaluated by psychologists involved in other projects,
although their autism diagnosis was re-conrmed and current
before they were included in this project. Pretreatment clinical
behavioral characterization was based on the self-reported Social
Responsiveness Scale, 2nd edition (SRS-2) (Constantino, 2012)(M
total raw ¼82.41, SD ¼33.43), which assesses ASD symptom
severity in ve domains: social awareness, social cognition, social
communication, social motivation, and restricted interests and re-
petitive behavior. Comprehensive demographics and characteriza-
tion information are provided in Table 1. This is a pretest-posttest
treatment-only study and all of the 17 participants were assigned
to receive VR-SCT intervention. The study is registered at Clinical- (ID: NCT02139514; NCT02922400).
Inclusion criteria for all participants included being within the
ages of 18e40 years, high-functioning (IQ 80), having a diagnosis
for ASD, and having a mean length of utterance greater than 5
words (as required by the intervention method). Exclusion criteria
for all participants included not being uent in English (the inter-
vention has not been validated in other languages), signicant
hearing loss, and a history of signicant head trauma or serious
psychiatric illness (other than ASD). Ten adults (beyond the 17
participants) were excluded: ve did not meet inclusion criteria
(one was >40 years of age, one had IQ <80, three were non-
spectrum), three were unavailable (two lived too far away, one
had schedule conicts), and two were discontinued after treatment
began due to either the therapist being unavailable or the partici-
pant being unable to make the treatment visits. All of the remaining
17 participants passed the MRI (Magnetic Resonance Imaging)
safety screening, including being free of any metal implants and
having no evidence of claustrophobia. All of the 17 participants
were right-handed. Written informed consent was obtained from
each participant. The Human Investigations Committee (HIC) at
Yale University and the Institutional Review Board (IRB) at the
University of Texas at Dallas approved this study.
2.2. Treatment approach: Virtual RealityeSocial Cognition Training
2.2.1. Technology
VR-SCT training technology, Virtual Gemini, was developed by
programmers at Center for BrainHealth, using an Unreal
and transmitted over the Internet. The platform included facial
emotion tracking using Faceshift Studio
software, which displayed
the participant and clinician's facial movements in real time.
server was utilized to transmit audio as well as Morph-
Vox (Screaming_Bee, 2005) was utilized for voice-modulation
software. VR-SCT ran on a standard Windows computer (mini-
mum specications of a Mobile core i7, 750 m, 4 gigabytes of RAM),
a web cam, and a headphone with built-in microphone.
2.2.2. Training
VR-SCT is a strategy-based immersive role-play intervention
program designed to strengthen socio-emotional processing and
socio-cognitive reasoning abilities in both children and young
adults with ASD. As documented in previous research studies
(Didehbani et al., 2016; Kandalaft et al., 2013), VR-SCT has been
shown to specically target and improve emotion recognition and
theory of mind as well as executive function and daily social
function. Based upon previous research ndings (Didehbani et al.,
2016; Kandalaft et al., 2013), we expected to see improvements in
emotion recognition and theory of mind after VR-SCT in the current
study. For the purposes of the study, sessions took place at both
YCSC and CBH-UTD. The coach clinician conducted the training and
hosted each session from CBH-UTD via the Internet. Across both
sites, participants arrived for the session and were set up on the
computer by the coach clinician or research staff. Once a participant
logged into the virtual platform, he/she independently interacted
with the coach clinician online. During the training session, both
the clinician and participant interacted entirely through virtual
Table 1
Participants demographics and pretreatment autism symptom severity prole
Variable Mean (S.D.) Range
Pretreatment Age (years) 22.50 (3.89) 18.06e31.08
Gender, male (0 ¼f, 1 ¼m) 0.88 (0.33) e
Full-scale IQ 109.65 (13.32) 88e131
Handedness (1 ¼right, 0 ¼ambi., 1¼left) 1.00 (0.00) e
ADOS Module 4 (n¼15)
SA Domain 10.73 (3.63) 7e19
RRB Domain 0.93 (1.03) 0e3
Total 11.67 (3.85) 7e20
Pretreatment SRS-2 self-reported raw scores
Social awareness 9.59 (3.22) 5e15
Social cognition 14.35 (5.79) 6e28
Social communication 25.41 (11.77) 6e47
Social motivation 15.18 (7.72) 5e27
Restricted interests and repetitive behavior 17.88 (7.78) 6e30
Total 82.41 (33.43) 33e142
Note. SA, Social Affect; RRB, Restricted and Repetitive Behaviors.
Unfortunately, the ADOS subdomain scores were not available for two partici-
pants because they were evaluated by psychologists involved in other projects,
although their autism diagnosis was re-conrmed and current before they were
included in this project.
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e66 57
avatar characters (see Fig. 1 for sample)
For every participant, the training program lasts for ve weeks,
with two 1-h sessions per week and thus 10 h in total. VR-SCT
presented hierarchical socio-emotional and socio-cognitive stra-
tegies that increased in complexity over the course of the 10 ses-
sions. The rst three sessions targeted learning three core social
strategies (recognizing others, responding to others, self-assertion)
and the remaining seven sessions focused on integrating all stra-
tegies together across varied and complex social situations (e.g.,
dealing with confrontation, job interview, blind date). Each session
allowed for multiple conversations and practicing the same social
objective for that day, so as to build a dynamic learning opportunity
of the social strategies and to encourage generalization to real-life
At the beginning of each session, a social learning objective was
presented and reviewed with the participant by a coach clinician.
After being given a social prompt, (e.g., You will be meeting a new
neighbor at the apartment building), the participant engaged in a
semi-structured live conversation with a confederate clinician
posing as a conversational partner. The confederate clinician could
change avatar appearance and modify his/her voice to quickly
change from one character to the next. Each character, played by
the confederate, had a unique pre-determined conversational
opening and emotional style (e.g., pleasant demeanor and easy-
going, or a rush and acting rude style). However, the responses
were dynamic and individualized as each participant partially
determined the outcome based upon his/her response. Following
each practice conversation, the coach and participant engaged in
feedback discussion of the conversation that included self-ratings.
The participant was then given additional conversational oppor-
tunities to integrate the discussed feedback into subsequent
2.2.3. Treatment targets of VR-SCT
The strategy-based program was presented in a top-down
fashion, to strengthen socio-emotional and social-cognitive abili-
ties of recognizing others, responding to others, and self-assertion.
The rst strategy of recognizing others targeted (a) ltering and
blocking social distractions, (b) identifying key and relevant social
cues, and (c) inferring social meaning in other's expressions. The
next strategy of responding to others built on the social perceptions
of the other person to generate meaningful connections by (a)
considering one's own social emotional cues and how they are
being conveyed, (b) formulating clear and direct responses that
relate to the situation or the other person, and (c) building back-
and-forth conversation to allow deeper conversation beyond the
surface level. Finally, engagement of socio-emotional and social-
cognitive control processes was further facilitated by the nal
self-assertion strategy of (a) considering multiple perspectives, (b)
considering possible outcomes and reecting on past mistakes, and
(c) applying new knowledge to new situations.
2.3. Primary clinical outcome: changes in emotion recognition and
theory of mind
Treatment effectiveness was measured by behavioral changes in
two distinct domains of social abilities: emotion recognition (tap-
ping change in socio-emotional processing abilities) and theory of
mind (tapping change in socio-cognitive processing abilities),
2.3.1. Emotion recognition
The Advanced Clinical Solutions for WAIS-IV and WMS-IV Social
Perception Subtest (ACS-SP) (Kandalaft et al., 2012; Pearson, 2009),
administered by trained research staff in our research centers, was
utilized to measure emotion recognition abilities. Three subscales
are generated from the subtest tasks: (a) SP-Affect Naming, a mea-
sure of face emotion recognition; (b) SP-Prosody, a measure of vocal
affect recognition; (c) SP-Pairs, a measure of non-literal language
interpretation. Across ACS-SP scores, average internal consistency
has been reported as r¼0.69e0.81, testeretest stability coefcient
as corrected r¼0.60e0.70, and inter-scorer agreement from 0.98 to
0.99. Normative scaled scores are available for all ACS-SP subtests.
Treatment effectiveness on emotion recognition is modeled as the
change scores of the ACS-SP scaled scores, that is, post minus pre,
such that positive (or negative) delta change scores indicate in-
crease (or decrease) in emotion recognition abilities. In the prior
pilot study involving VR-SCT and young adults with autism
(Kandalaft et al., 2013), emotion recognition changed signicantly
from pretreatment (M¼7.63, SD ¼3.42) to posttreatment
(M¼9.63, SD ¼3.78), t(7) ¼2.83, p¼0.03, Cohen's d
2.3.2. Theory of mind
The Social Attribution Task, also known as the triangles task
(Abell, Happe, &Frith, 2000) was administered to measure a per-
son's abilities of theory of mind. Videos were adapted from a pre-
vious study (Heider &Simmel, 1944), in which participants were
asked to narrate the movements of triangles presented in six
separate brief videos. Narratives were recorded, transcribed, and
double-scored by two blind raters. Based upon the scoring criteria
established in previous research (Abell et al., 2000), participants'
narratives were rst scored on accuracy and attribution, respec-
tively, and the two scores were then summed up to derive a total
Fig. 1. The computer set-up and example screenshots of a virtual reality training
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6658
score. The accuracy and attribution aspects were scored separately
using a 4-point Likert scale (0e3 point scale) for each video, with 18
as the maximum possible score across all 6 videos for both accuracy
and attribution, respectively. More points were awarded when the
participant stated descriptions that were accurate to the nature of
the video (for the accuracy aspect), or when more mentalizing or
emotional words were utilized to describe the movement of the
triangles (for the attribution aspect). The triangles task has been
shown to have a high test-retest reliability of r¼0.76 to 0.88 and
concurrent validity r¼0.78 to 0.93 (Hu, Chan, &McAlonan, 2010).
The order of the videos was randomized and participants were
presented with different sets of videos at pre- and post-
intervention testing. In the prior pilot study involving VR-SCT and
young adults with autism (Kandalaft et al., 2013), theory of mind
changed signicantly from pretreatment (M¼12.63, SD ¼4.93) to
posttreatment (M¼15.38, SD ¼4.81), t(7) ¼3.45, p¼0.01, Cohen's
(Lakens, 2013)¼0.56.
2.4. fMRI imaging task
We measured the pretreatment BOLD (blood oxygen level
dependent) responses using a well-established biological motion
fMRI task (Bjornsdotter, Wang, Pelphrey, &Kaiser, 2016; Kaiser
et al., 2010; Ventola et al., 2015; Yang et al., 2016; Yang et al.,
2017). We selected this paradigm because it measures the neural
activities of two key components of social information processing
(Kaiser et al., 2010), namely, socio-emotional and socio-cognitive
processing, which correspond to the treatment targets of VR-SCT
and the two behavioral measures of emotion recognition and the-
ory of mind, respectively. Before the treatment, the participants
were scanned while viewing coherent and scrambled point-light
displays of biological motion created from motion capture data.
The coherent biological motion displays featured an adult male
actor performing movements relevant to early childhood experi-
ences, such as playing pat-a-cake (Klin, Lin, Gorrindo, Ramsay, &
Jones, 2009), and contained 16 points corresponding to major
joints. The scrambled motion animations were created by selecting
all the 16 points from the biological motion displays and randomly
plotting their trajectories on a black background (see
Supplementary Material 1 for sample fMRI stimuli). Thus, the
coherent and scrambled displays contained the same local motion
information, but only the coherent displays contained the cong-
uration of a person (Johansso, 1973). During the MRI scan, stimuli
were presented using E-Prime 2.0 software (Psychological Software
Tools, Pittsburgh, PA, USA). Six coherent biological motion clips
(BIO) and six scrambled (SCR) motion clips were presented once
each in an alternating-block design (time per block, ~24 s). The
experiment began with a 20-s xation period and ended with a 16-
sxation period. The total duration was 328 s. The movies were
presented without audio. The participants were asked to watch the
videos and reminded to remain still and alert. The imaging task and
stimuli are available from the authors upon reasonable request.
2.5. Imaging acquisition and processing
Scanning was performed on a Siemens MAGNETOM 3 T Tim Trio
scanner at the Yale Magnetic Resonance Research Center (for YCSC
participants) or a Philips 3 T MR system (for CBH-UTD participants)
within one week before the treatment began. For each YCSC
participant, a structural MRI image series was acquired with a 12-
channel head coil, a high-resolution T1-weighted MPRAGE
sequence, and the following parameters: 176 slices; TR ¼2530 ms;
TE ¼3.31 ms; ip angle ¼7 deg; slice thickness ¼1.00 mm; voxel
size ¼11mm
; matrix ¼256 256. Afterwards, BOLD T2*-
weighted functional MRI images were acquired using the
following parameters: 164 vol; TR ¼2000 ms; TE ¼30 ms; ip
angle ¼90
; slice thickness ¼4.00 mm; voxel size ¼33mm
matrix ¼64 64; number of slices per volume ¼34. For each CBH-
UTD participant, a structural MRI image series was acquired with an
8-channel head coil, a high-resolution T1-weighted MPRAGE
sequence, and the following parameters: 176 slices; TR ¼7.730 ms;
TE ¼3.53 ms; ip angle ¼7 deg; slice thickness ¼1.00 mm; voxel
size ¼11mm
; matrix ¼256 256. Afterwards, BOLD T2*-
weighted functional MRI images were acquired using the
following parameters: 164 vol; TR ¼2000 ms; TE ¼30 ms; ip
angle ¼90
; slice thickness ¼4.00 mm; voxel size ¼33mm
matrix ¼64 64; number of slices per volume ¼34. The site
variable (YCSC vs. CBH-UTD; dummy-coded and mean-centered)
was included as a covariate of no interest across all neuroimaging
analyses. Although each site used a different scanner, controlling
for site has statistically controlled for the differences due to the two
sites, including using two different scanners, head coils, etc.
The T1-weighted MPRAGE structural scan was segmented by
SPM12 into gray matter, white matter (WM), and cerebrospinal
uid (CSF) images. This method is highly accurate and has reduced
bias, comparable to manual measurement (Malone et al., 2015).
The fMRI data were processed using FSL (Jenkinson, Beckmann,
Behrens, Woolrich, &Smith, 2012) v5.0.9 and the participant-level
preprocessing steps followed a standardized processing streamd-
ICA-AROMA (ICA-based strategy for Automatic Removal of Motion
Artifacts) (Pruim et al., 2015). This consisted of the following
sequence: (a) motion correction using MCFLIRT; (b) interleaved
slice timing correction; (c) BET brain extraction; (d) grand mean
intensity normalization for the whole 4D data set; (e) spatial
smoothing with 5 mm FWHM; (f) data denoising with ICA-AROMA
(Pruim et al., 2015), which uses a robust set of theoretically moti-
vated temporal and spatial features to remove motion-related
spurious noise; (g) nuisance regression using time-series for WM
and CSF signal to remove residual, physiological noise, and nally,
(h) high-pass temporal ltering (100 s). The rst 4 s were discarded
to establish T1 equilibrium. Registration of the fMRI data was per-
formed using both the subject's structural scan and then the
Montreal Neurological Institute (MNI152) standard brain. Pre-
processed data were then pre-whitened using FSL's FILM to remove
time series autocorrelation.
To model the BIO and SCR conditions, the timing of the corre-
sponding blocks was convolved with the default gamma function
(phase ¼0 s, standard deviation ¼3 s, mean lag ¼6 s) with tem-
poral derivatives. The participant-level contrast of interest is
BIO >SCR, which served as inputs for the subsequent mass uni-
variate, whole-brain, group-level GLM analyses and multivariate
pattern analyses.
2.6. Mass univariate group-level GLM analyses
We conducted mass univariate voxel-wise GLM analyses across
the whole brain to identify clusters where pretreatment BOLD
activation to the contrast of BIO >SCR predicted treatment effec-
tiveness. The analyses were conducted using mixed-effects
modeling with FSL's FLAME (FMRIB's Local Analysis of Mixed Ef-
fects) 1 þ2 inference algorithm, which provides highly accurate
estimation of group-level results that are generalizable to the
population. Consistent with our prior neuropredictive research
(Yang et al., 2016), we employed a stringent cluster-dening
threshold (CDT) of Z>2.33, p<0.01, while correcting for multi-
ple comparisons at a cluster-level threshold of p<0.05. Information
about the surviving clusters was reported, including number of
voxels in the cluster, the anatomical regions covered by the clusters
based on the Desikan-Killiany atlas (Desikan et al., 2006), the co-
ordinates of the peak voxels within each of the anatomical regions,
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e66 59
and the Z-statistics associated with the peak voxels. Site, age, IQ,
sex, and pretreatment autism symptom severity using the SRS total
raw scores were mean-centered and controlled for as covariates of
no interest in all group-level univariate GLM analyses. This was to
ensure that the results could be generalized to different sites, ages,
IQ levels, sexes, and levels of pretreatment autism symptom
severity. The pretreatment autism symptom severity was not
signicantly correlated with either pretreatment emotion recog-
nition, r¼0.39, p¼0.13, or pretreatment theory of mind, r¼0.17,
p¼0.51, which could be due to a number of possible reasons (e.g.,
the generality-specicity differences, autism symptom severity
encompassing some different domains, small sample size, differ-
ence in responders/coders, etc.).
2.7. Meta-analytical reverse inference
To understand the functional relevance of the surviving clusters,
we performed a quantitative reverse inference using NeuroSynth
( The NeuroSynth dataset v0.6 (July
2015 release) contains activation data for over 11,406 studies and
feature information for over 3,300 term-based features. The term-
based features were derived from the abstracts of articles in the
NeuroSynth database. For each feature, the database stores the
whole-brain, reverse inference, meta-analysis map, P(Term jActi-
vation), that is, the likelihood that a feature term is used in a study
given the presence of reported activation (Yarkoni, Poldrack,
Nichols, Van Essen, &Wager, 2011). Each surviving cluster was
decoded with NeuroSynth, which computed the voxel-wise Pear-
son correlation between the cluster image le and the meta-
analytical image le associated with each of the 3,300 feature
terms. The top 10 psychological functional terms with the highest
positive correlation were retained and reported, while we omitted
non-functional terms, such as (but not limited to) those describing
an anatomical region (e.g., frontal operculum, vlpfc), a technique/
method/task (e.g., signal task, nogo), a population (e.g., speakers),
or being relatively generic (e.g., reference, difculty, conveyed, etc.).
2.8. Multivariate pattern analyses (MVPA)
To guard against data over-tting and to gain understanding of
how different voxels in the neuropredictive network derived from
the mass univariate GLM analyses worked together in predicting
change in emotion recognition, we utilized regression-based
Multivariate Pattern Analyses (MVPA) (Haxby, Connolly, &
Guntupalli, 2014). In MVPA, the samples were divided into
training and testing data sets, which constitute a cross validation
framework in which the predictive model is rst trained with the
training set and then used to predict the regression labels of the
sample in the testing set. This type of cross validation provides
approximately unbiased estimates of effects, generalizable to new
samples, helping to minimize the likelihood that the results over-t
the data. Moreover, in contrast to the mass univariate voxel-wise
GLM analyses, MVPA draws on the multivariate information
across many voxels comprising neural networks, which may cap-
ture how the voxels or regions work together to achieve complex
functions. All these characteristics render MVPA well suited for
establishing robust predictive biomarkers. MVPA has been applied
to fMRI data to establish predictive biomarkers for treatment
response or long-term outcome in a number of neuropsychiatric or
neurocognitive disorders, such as depression (van Waarde, Scholte,
van Oudheusden, Verwey, Denys, &van Wingen, 2015), dyslexia
(Hoeft et al., 2011), social anxiety disorder (Mansson et al., 2015;
Whiteld-Gabrieli et al., 2015), panic disorder (Hahn et al., 2015),
and more recently autism spectrum disorder (Yang et al., 2016).
MVPA were performed using the Pattern Recognition for
Neuroimaging Toolbox (Schrouff et al., 2013) (PRoNTo) v2.0 in
Matlab and followed several steps. First, each participant's pre-
treatment Z-statistic BIO >SCR contrast image (up-sampled to the
standard MNI152 space using trilinear interpolation) was inputted
into the MVPA. The surviving clusters derived from the univariate
analysis as a network was used as an analytical mask, while
treatment effectiveness was entered as the regression target. Sec-
ond, PRoNTo computed a linear kernel (that is, dot product) be-
tween the voxel intensities within the mask for each pair of the
input images, thereby generating a 17 17 similarity matrix, which
served as the input feature set for the subsequent machine learning
algorithm. Third, we used kernel ridge regression (KRR) (Chu, Ni,
Tan, Saunders, &Ashburner, 2011) as the multivariate regression
method. This is the dual-form formulation of ridge regression and
solves regression problems with high dimensional data in a
computationally efcient way. Cross validation was based on a
leave-one-subject-out (LOSO) framework with mean-centered
features across training images. We selected LOSO (which is equal
to 17-fold cross-validation with our sample) because a larger
number of folds may reduce bias of the estimates, even at the cost
of increasing variance of the estimates, and should provide more
accurate estimates of neural predictability, especially when sample
sizes are small. For each fold, one input image was left out and
served as the testing set. The KRR machines were trained to asso-
ciate treatment effectiveness with the multivariate information in
the remaining sample of 16 participants. The trained KRR machines
were then used to predict treatment effectiveness in the left-out
image. This step was repeated for each of the 17 folds. Across all
folds, predictive accuracy was calculated as the Pearson's correla-
tion coefcient (r), coefcient of determination (R
), and normal-
ized mean squared error (nMSE) between predicted and actual
treatment effectiveness. Fourth, the signicance of the predictive
accuracy statistics was evaluated using a permutation test, con-
sisting of 50,000 iterations. In each iteration, the regression targets
were randomly permuted across all participants and the cross-
validation procedure was repeated. The p-values of r,R
, and
nMSE were then calculated as the proportion of all permutations
where r,R
, and nMSE were greater than (or less than, in the case of
nMSE) or equal to the obtained r,R
, and nMSE, respectively.
3. Results
3.1. Primary clinical outcome
As illustrated in Fig. 2(A) and as hypothesized, VR-SCT signi-
cantly improved emotion recognition in terms of the ACS-SP scaled
scores from pretreatment (M¼11.41, SD ¼4.42) to posttreatment
(M¼12.94, SD ¼3.51),
¼1.53, S.D. of
¼2.72, t(16) ¼2.32,
p¼0.03 (two-tailed), Cohen's d
(Lakens, 2013)¼0.37. In addition,
as shown in Fig. 2(B), VR-SCT marginally improved theory of mind
in terms of the total scores from the triangles task from pretreat-
ment (M¼19.41, SD ¼3.89) to posttreatment (M¼20.35,
SD ¼3.84),
¼0.94, S.D. of
¼1.95, t(16) ¼1.99, p¼0.06 (two-
tailed), Cohen's d
(Lakens, 2013)¼0.24. This effect was signi-
cant one-tailed, p¼0.03, and therefore also as hypothesized. The
change scores of emotion recognition were not signicantly
correlated with those of theory of mind, r(15) ¼-0.22, p¼0.40,
suggesting that they were tapping improvement in distinctively
different domains. The raw data for individual participants were
reported in Supplementary Material 2.
3.2. Mass univariate GLM analyses
For change in emotion recognition, as illustrated in Fig. 3 and as
hypothesized, the whole-brain mass univariate GLM analyses of the
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6660
pre-treatment brain BOLD responses to BIO vs. SCR on the change in
emotion recognition from baseline to treatment endpoint revealed
a network (758 voxels) of two distinct clusters of neuropredictive
activities. Cluster#1 has 319 voxels and included primarily the left
posterior superior temporal sulcus, left superior temporal gyrus,
and left middle temporal gyrus. Cluster#2 has 439 voxels and
included primarily the right insula, right orbitofrontal cortex, and
right inferior frontal gyrus. The scatterplot in Fig. 3 also illustrates
the form of the neuropredictive relationship between the change in
emotion recognition (y-axis) vs. pretreatment BIO >SCR activation
(x-axis) for each network and the estimated effect size in terms of
Pearson's r¼0.71. It should be noted that the data points in the
scatterplot were retrieved from the surviving neuropredictive
network; they serve the purpose of illustration only and were not
the results of a separate analysis. As can be seen, greater levels of
pretreatment activation in the network were positively correlated
with improvement of emotion recognition brought about by VR-
SCT. There was no region that showed negative correlation
between pretreatment activation and change in emotion recogni-
tion. Table 2 lists the peak signicance, peak coordinates, spatial
extent, and anatomical locations encompassed by each predictive
cluster within the network.
To further interpret the possible functions of each distinct
cluster in the neuropredictive networks for predicting change in
emotion recognition, we conducted a NeuroSynth-based (http:// reverse inference analysis. As seen in Table 3,
cluster#1 correlates with language processing, meaning compre-
hension and integration, semantic inference, and resolution of
meaning conicts, such as interpretation of incongruent auditory
emotions and prosody (Mitchell, 2006). Cluster#2 correlates with
socio-emotional experience, interpersonal affective information,
emotional regulation (Nummenmaa, Hirvonen, Parkkola, &
Hietanen, 2008; Walter et al., 2009), and social information pro-
cessing (Yang et al., 2015). The image les from this analysis are
available at that inter-
ested readers may independently decode the image les with
Fig. 2. Treatment effectiveness quantied as the changes in (A) emotion regulation and (B) theory of mind. In each panel, left: The black lines indicate each patient's change in
the behavioral measure from pretreatment to posttreatment, while the red line denotes the group mean; right: the mean and the 95% condence interval (CI) of
, the change score
(that is, post minus pre). (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)
Fig. 3. Neural prediction of change in emotion recognition induced by VR-SCT using biological motion fMRI and univariate GLM. Greater pretreatment BOLD activations to
biological vs. scrambled motion in these regions were associated with more improvement in emotion recognition due to VR-SCT. The scatterplot illustrated the relationship between
pretreatment BOLD activations and actual changes in emotion recognition (post minus pre), with a horizontal reference line at y¼0 indicating no change from pretreatment to
posttreatment (that is, post ¼pre). The regression lines and the 95% condence intervals were plotted. The results were based on cluster-dening threshold of Z>2.33, p<0.01 and
cluster-level threshold of p<0.05. Site, IQ, age, sex, and pretreatment autism symptoms severity were included as covariates of no interest. pSTS, posterior superior temporal sulcus;
MTG, middle temporal gyrus; STG, superior temporal gyrus; SMG, supramarginal gyrus; IFG, inferior frontal gyrus; OFC, orbitofrontal cortex; VR-SCT, Virtual Reality-Social Cognition
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e66 61
NeuroSynth through links within the NeuroVault website.
For change in theory of mind, contrary to our hypothesis, the
neuropredictive analysis revealed no surviving clusters. However,
to avoid possible Type II errors and the premature conclusion that
biological motion fMRI cannot predict treatment outcome in terms
of change in theory of mind, we followed a procedure recom-
mended in the literature (Lieberman &Cunningham, 2009)by
analyzing the uncorrected results with voxel-level threshold
Z>3.29, p<0.001, and minimal extent k¼20 voxels. The analysis
revealed three small distinct clusters of neuropredictive activities,
which were reported in Supplementary Material 3 for readers
Because the neuropredictive network for change in emotion
recognition is implicated in verbal and non-verbal socio-emotional
processing, the nding suggests that individuals who demonstrate
greater activity in brain regions implicated in socio-emotional
processing at baseline show greater treatment response. Here,
there is a question whether the pretreatment activities in these
regions are specic to predicting change in emotion recognition, or
also correlated with pretreatment abilities of emotion recognition.
To address this question, we conducted additional analyses to
examine the neural correlates of pretreatment levels of emotion
recognition. As shown in Supplementary Material 4, the pretreat-
ment levels of emotion recognition were correlated with brain re-
gions that are distinct from the neuropredictive network forchange
in emotion recognition. This suggests that the neuropredictive re-
gions are specic to treatment outcome and separable from the
regions that are associated with variance in emotion recognition
capacities at the pretreatment time point. The corresponding
neural correlates for pretreatment levels of theory of mind were
reported in Supplementary Material 5. Compared to
Supplementary Material 3, the nding also suggests that there exist
distinct pretreatment neural correlates for the longitudinal change
in theory of mind brought out by VR-SCT versus the cross-sectional
pretreatment variance in theory of mind.
3.3. Multivariate pattern analyses with cross validation
To guard against the possibility of data over-tting in the mass
univariate analyses and to gain an understanding of how the voxels
comprising the univariate clusters worked together in predicting
change in emotion recognition, we applied regression-based
Multivariate Pattern Analyses (MVPA) of pretreatment BOLD re-
sponses to the contrast of BIO >SCR with leave-one-subject-out
(LOSO) cross-validation in the voxels comprising the univariate
clusters. As shown in Table 4, the neuropredictive network con-
sisting of the two clusters survived cross validationdthe multi-
variate pattern information from this brain network signicantly
predicted change in emotion recognition (r¼0.79, p¼0.001;
¼0.62, p¼0.002; nMSE ¼0.24, p¼0.001). Fig. 4(A) shows the
weight map (i.e. model parameters) in the representative slices of
this network derived from the multivariate modeling of pretreat-
ment images predicting change in emotion recognition. Fig. 4(B)
shows the scatterplot of actual versus predicted treatment
response on emotion recognition. As estimated via the LOSO
framework, each of the points in this plot was derived from a
separate training set, and for a new unseen patient (testing set), the
remaining 16 participantsdata were used as the training set. Thus,
the correlation is not a standard correlation derived from a single
set of participants. Rather, each point reects different combina-
tions of training and testing sets. Finally, Fig. 4(C) shows the line
plot of actual versus predicted treatment response on emotion
recognition for each of the 17 folds.
In addition, we conducted MVPA analyses with comparison/
control ROIs that we did not expect to be predictive of treatment
outcome in ASD. The inferior occipital gyrus and middle occipital
gyrus were selected because: (a) they respond strongly to a range of
visual stimuli including the SCR and BIO stimuli used here, and (b)
the occipital gyri were shown to be neuropredictive of treatment
effectiveness in a markedly different neuropsychiatric condition,
social anxiety disorder (Doehrmann et al., 2013). Furthermore, we
conducted MVPA with the whole brain (including the neuro-
predictive network) to evaluate the specicity of our ndings to the
network of these univariate clusters. As shown in Table 4, neither
the comparison ROIs nor the whole brain analysis were predictive
of treatment outcome (ps>0.05).
3.4. Demographic and behavioral ndings
To evaluate whether fMRI provides unique information con-
cerning the prediction of response to VR-SCT on emotion recogni-
tion and theory of mind, we examined how a host of demographic
and pretreatment ASD symptom severity measures predict changes
in emotion recognition and theory of mind, respectively. We ran
correlation analyses between each of the measures listed in Table 1
and the delta change in emotion recognition and that in theory of
mind. As seen in Supplementary Material 6, no measure showed a
signicant correlation, ps>0.11. This suggests that the fMRI
Table 2
Neural prediction of change in emotion recognition due to VR-SCT using biological
motion fMRI and mass univariate GLM.
Cluster N
Local Maxima
Anatomical Region xyzZ
#1 319 L Banks of the STS 58 50 14 3.58
L Inferior temporal gyrus 54 56 2 3.64
L Middle temporal gyrus 54 56 2 3.64
L Superior temporal gyrus 58 50 14 3.58
L Supramarginal gyrus 56 50 16 2.77
#2 439 R Insula 42 0 14 3.25
R Lateral orbitofrontal cortex 42 28 8 3.60
R Pars opercularis 52 10 2 3.54
R Pars orbitalis 44 28 8 3.72
R Precentral gyrus 54 8 0 3.07
R Superior temporal gyrus 52 8 6 4.30
Note. The coordinates are in MNI152 space, mm. L, Left; R, Right; STS, Superior
temporal sulcus; VR-SCT, Virtual Reality-Social Cognition Training. The analysis was
corrected, with voxel-level threshold Z>2.33, p<0.01, and cluster-level threshold
p<0.05. Greater pretreatment activation to the social perception contrast: bio-
logical motion vs. scrambled motion in these regions was associated with greater
improvement in emotion recognition brought out by VR-SCT. Site, IQ, age, sex, and
pretreatment autism symptoms severity were included as covariates of no interest.
Table 3
Reverse inference analysis of the neuropredictive clusters for change in emotion recognition using NeuroSynth.
Cluster Top 10 NeuroSynth-decoded feature terms
#1 Conicting (0.120), integration (0.107), lexical (0.104), sentence (0.100), verbs (0.085), linguistic (0.081), meaning (0.077), audiovisual (0.075),
comprehension (0.072), reading (0.072)
#2 Musical (0.079), interpersonal (0.046), painful (0.046), monitor (0.041), regulating (0.037), nociceptive (0.034), social (0.034), affective (0.033),
emotions (0.031), sad (0.030)
Note. Numbers within the parentheses are correlation coefcients between the surviving clusters and the meta-analysis maps of the feature terms in NeuroSynth.
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6662
measure provided unique advantage over behavioral measures on
predicting behavioral responses to VR-SCT.
4. Discussion
As hypothesized, the biological motion fMRI task successfully
identied brain regions in which pretreatment brain activations
during passively viewing biological motion versus scrambled mo-
tion predicted change in verbal and non-verbal emotion recogni-
tion in a study of young adults with high-functioning autism, who
received an evidence-based VR-SCT behavioral intervention.
Specically, the key brain regions are implicated in functions sup-
porting (a) language-related comprehension and meaning inte-
gration, as well as interpretation of incongruent auditory emotions
and prosody, and (b) socio-emotional experience processing,
interpersonal affective information processing, and emotional
regulation. These two groups of functions are closely related to
verbal and non-verbal emotion recognition, respectively. As such, it
is possible that the regions may be involved in processing the socio-
emotional aspect of the biological motion stimuli, that is, how the
social emotions associated with adult's actions in children's games
(e.g., waving, pat-a-cake, peek-a-boo) are decoded and interpreted.
Table 4
Predictive accuracy of the univariate neuropredictive network for change in emotion recognition, as estimated by MVPA with cross validation.
Mask N
nMSE p
Neuropredictive network 758 0.79 0.001 0.62 0.002 0.24 0.001
Inferior occipital gyrus 1930 -0.24 0.67 0.06 0.50 1.04 0.58
Middle occipital gyrus 5368 -0.08 0.46 0.01 0.83 0.88 0.44
Whole brain 228,453 -0.65 0.92 0.43 0.08 0.90 0.88
Note. Predictive accuracy was indicated by Pearson's correlation coefcient (r), coefcient of determination (R
), and normalized mean squared error (nMSE) between pre-
dicted and actual change in emotion recognition. Signicance (p-value) was determined with a random permutation test (50,000 iterations). Signicant regions and statistics
were displayed in bold. Cross validation was based on a 17-fold leave-one-subject-out (LOSO) framework.
Fig. 4. Predictive accuracy of the univariate neuropredictive clusters, as estimated by MVPA with cross validation, for predicting change in emotion recognition due to VR-
SCT. (A): Weight map showing the relative weights derived from the multivariate modeling of pretreatment response to biological motion that contributed to the prediction of
change in emotion recognition (that is, post minus pre) at representative slices (MNI152 mm space). (B): Scatterplot illustrating actual and predicted changes in emotion recog-
nition, with a horizontal reference line at y¼0 indicating no change from pretreatment to posttreatment (that is, post ¼pre). The regression lines and the 95% condence intervals
were plotted. Cross validation was based on a 17-fold leave-one-subject-out (LOSO) framework. (C): Line plot illustrating actual and predicted changes in emotion recognition for
each of the 17 folds used in the cross-validation framework. The results were based on the neuropredictive network estimated with cluster-dening threshold of Z>2.33, p<0.01
and cluster-level threshold of p<0.05. VR-SCT, Virtual Reality-Social Cognition Training.
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e66 63
Importantly, the results were supported by regression-based
MVPA with a standard LOSO cross-validation framework, which
suggests that the brain activities within the neuropredictive net-
works may serve as robust predictive biomarkers, generalizable to
new, unseen participants. Moreover, we found that there existed
distinctively different regions that are associated with variance in
emotion recognition capacities at the pretreatment time point,
which suggests that the neuropredictive regions may be specicto
predicting treatment outcome. To our knowledge, the current
ndings provide the rst evidence of neuroimaging-derived pre-
dictive biomarkers in young adults with ASD. The predictive bio-
markers identied in this research may be interpreted as the
pretreatment neurobiological readiness to respond to a specic
treatment, VR-SCT, in the domain of emotion recognition.
For change in theory of mind, although the corrected results
were not as hypothesized and revealed no surviving regions, the
exploratory uncorrected analyses identied several neuro-
predictive regions for change in theory of mind. These regions need
to be interpreted with caution. First, they were uncorrected in
nature and due to skipping the procedure of correcting for multiple
comparisons, there is a greater chance of Type I error here. Second,
they were based on a very high voxel-level threshold and had
relatively small extant. Thus, they may be ill-suited to be analyzed
by MVPA because the small extant would render the results unre-
liable. Nonetheless, the exploratory analyses here raised the pos-
sibility that fMRI can predict change in not only emotion
recognition but also theory of mind, and there likely exist separable
neuropredictive regions for change in emotion recognition and
theory of mind, respectively.
Compared to the previous neuropredictive study involving a 16-
week Pivotal Response Treatment and young children with autism,
and using the change in SRS total raw scores as the measure of
treatment effectiveness (Yang et al., 2016), the current study has a
number of similarities and differences. First, in terms of similarities,
both studies used the same biological motion fMRI task (Kaiser
et al., 2010) as the pretreatment neural predictor. This supports
the usability of the task across a wide range of age, treatment
modality, and outcome measures. Both studies also recruited high-
functioning individuals with autism as participants, included both
male and female participants, and employed the same data
analytical pipeline and cluster thresholding. Second, in terms of
differences, the current study involved a 5-week VR-SCT inter-
vention and young adults with autism, and used changes in
emotion regulation and theory of mind, respectively, as the treat-
ment outcome variables, while the SRS total raw scores measured
trait-like (6-month tendency) autism symptom severity and was
not suitable for a 5-week intervention.
Given the differences, particularly on the variable of treatment
effectiveness, we did not expect to identify the same neural pre-
dictive biomarkers in the current study. Specically, the cluster#1
in the current study, predicting change in emotion recognition, was
around the left pSTS region. In contrast, the cluster#1 in the pre-
vious study (Yang et al., 2016), predicting change in autism symp-
tom severity, was around the right pSTS region. The left pSTS region
was known for its role in language processing and resolution of
conicting meaning, which is arguably closely relevant to verbal
emotion recognition, while the verbal behavioral measure of
emotion recognition in this project consisted of prosody under-
standing (SP-Prosody) and conicts between literal and non-literal
meanings (SP-Pairs). In contrast, the right pSTS region was impli-
cated in temporal and spatial sensory information integration,
which is arguably closely related to the core symptoms of autism
(Yang et al., 2015). Moreover, the cluster#2 in the current study was
near the right insula but more dorsal and extended to the pars
opercularis of the inferior frontal gyrus, while the cluster#3 in the
previous study was also near the right insula but more ventral and
extended to the temporal pole. These two clusters are spatially
close and located to the same hemisphere. This region at large is
generally known for socio-emotional processing and emotion
recognition, which is arguably closely aligned with non-verbal
behavioral measure of emotion recognition in the current project
(SP-Affect Naming), in which the participants were asked to identify
the expressed emotion from a given list when presented various
faces. Relatedly, difculty in interpretation of facial expressions of
emotion is also frequently cited as one of the main characteristics
associated with the core symptoms of autism (Eack, Mazefsky, &
Minshew, 2015).
There has been a signicant barrier in identifying and predicting
which treatments might be benecial for a given individual with
ASD, before the treatment is prescribed and delivered. Joining a
recent discovery of predictive biomarkers in young children with
ASD (Yang et al., 2016), our ndings advanced the eld one more
step forward toward the goal of targeted, personalized treatment
for individuals with ASD. Although more research is needed, it is
promising that fMRI techniques may provide guidelines and sug-
gestions for possible treatment(s) for those who are most likely to
immediately benet from the treatment(s). For those who would be
otherwise unlikely to immediately benet from the treatment, the
current ndings also raise the question as to whether increasing
pretreatment activations through other types of pretreatment in-
terventions, may theoretically lead to better response to treatment.
Accordingly, more research is needed to investigate methods that
would increase the pretreatment activation in these individuals
and test the hypothesis as to whether they could become more
ready to respond to treatment. For example, in adults with ASD,
oxytocin has been shown to increase brain activation, particularly
in regions similar to the predictive biomarkers identied in this
research such as inferior frontal gyrus (Domes, Kumbier, Heinrichs,
&Herpertz, 2014). It is thus possible that administration of intra-
nasal oxytocin at pretreatment may increase brain readiness to
respond to treatment in ASD (e.g., increasing learning rate of socio-
emotional and cognitive training) for those who would not be able
to immediately benet from the behavioral intervention alone. This
possibility may be tested in future research.
4.1. Limitations
Several limitations should be considered regarding this
research. First, the neuropredictive ndings were limited to one
single treatment-only group in a pretest-posttest design. Future
work should conduct randomized controlled trials to further
establish these ndings. A waitlist control group will generate
additional insight into the robustness and specicity of the neural
predictive biomarkers (they should have no predictive abilities in
the absence of an ongoing treatment) as well as further evaluation
of the test-retest reliability of the behavioral measures within the
5-week span of the current study. Second, the size of our pre-
liminary sample (N¼17) is relatively small. The neuropredictive
ndings need to be further tested for reproducibility in an inde-
pendent, larger sample. A larger sample would also increase the
statistical power, which is needed to detect smaller sizes of effect.
On the contrary, the non-signicant neuropredictive results on
theory of mind (after controlling for multiple comparisons) should
not be interpreted as no predictive potential in this ability. The
uncorrected exploratory analyses still revealed several regions of
neuropredictive activities for change in theory of mind.
Third, the primary clinical outcomes are limited to changes in
emotion recognition and theory of mind, respectively. Although
these two abilities are among the most basic abilities in socio-
emotional and socio-cognitive processing (Baron-Cohen, Leslie, &
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6664
Frith, 1985; Gallese, Keysers, &Rizzolatti, 2004) that underlie a
wide range of social skills and are also central to our understanding
of ASD decits (Uljarevic &Hamilton, 2013), there is a need for
future research to include other ASD-related measures, such as
interaction behaviors (Rice &Redcay, 2016) and conversation skills
(Scattone, 2008). Future studies are also needed to establish the
minimal clinically important difference (MCID) in the change
scores in these behavioral measures, and include naturalistic
measures to test whether the behavioral effects can generalize to
real-life daily functioning at the 5-week posttreatment endpoint.
Fourth, all participants were high-functioning and it remains un-
clear whether the ndings may apply to all individuals with autism.
The VR-SCT has an inclusion criterion of IQ 80 and future research
needs to investigate other treatment approaches for those who may
be cognitively impaired. Fifth, while the ndings suggest that cli-
nicians may potentially one day use brain activity patterns to
identify individuals with autism who would benet from a given
treatment, wide use of such an approach might depend on clini-
ciansability to gather the data using imaging methods robust to
motion or relatively inexpensive, such as functional near-infrared
spectroscopy (fNIRS).
Finally, while the current paper focused exclusively on testing
the use of fMRI as a forecasting tool to facilitate subject selection to
inform future treatment design, it did not address the question of
why VR-SCT works on the brain level. The latter question concerns
the neural mechanisms of change and can be answered by
comparing brain activations before and after VR-SCT, or analyzing
what brain changes track with behavioral changes induced by VR-
SCT. This question requires a full consideration well beyond the
scope of the current paper and should be further pursued in future
research and analyses.
5. Conclusions
Despite the limitations, for the rst time in the eld of ASD, we
provide evidence that treatment effectiveness at the individual
level in adults with ASD in the domain of verbal and non-verbal
emotion recognition can be accurately predicted by the pretreat-
ment fMRI activations using biological motion task in brain circuits
implicated in (a) language comprehension and integration, and
processing incongruent auditory emotions and prosody, and (b)
socio-emotional experience processing, interpersonal affective in-
formation processing, and emotional regulation. Relative to chil-
dren with ASD, intervention research for adults with ASD is very
limited. This study offers a key direction toward increasing the
effectiveness of intervention for adults with ASD and extends the
ndings that the biological motion fMRI task can be used to predict
treatment outcome from young children with ASD to young adults
with ASD. Our results thus open a new avenue for important future
research and should potentially accelerate progress toward devel-
oping more precise and effective treatments for individuals with
ASD across the lifespan.
Conict of interest
The authors declare no competing nancial interests.
We thank the participants and their families included in this
study for their time and participation and the research assistants in
our research centers, making this research possible. This work was
supported by the Harris Professorship at Yale Child Study Center to
KAP, Autism Speaks Meixner Postdoctoral Fellowship in Trans-
lational Research (#9284) to DY, a gift from the Autism
SocietyeNorthwestern Pennsylvania to DY, and the Yale University
Biomedical High Performance Computing Center (NIH grants
RR19895 and RR029676-01). Additionally, we thank the Rees-Jones
Foundation, Vin and Caren Prothro Foundation, and the Crystal
Charity Ball for their generous support of Center for BrainHealth's
research. We also thank Jeffrey Spence for feedback with statistical
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://
Abell, F., Happe, F., & Frith, U. (2000). Do triangles play tricks? Attribution of mental
states to animated shapes in normal and abnormal development. Cognitive
Development, 15,1e16.
Allison, T., Puce, A., & McCarthy, G. (2000). Social perception from visual cues: Role
of the STS region. Trends in Cognitive Sciences, 4, 267e278.
APA. (2013). Diagnostic and statistical manual of mental disorders : DSM-5 (5th ed.).
Washington, D.C: American Psychiatric Publishing.
Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985). Does the autistic child have a
theory of mind ?.Cognition, 21,37e46.
Bellani, M., Fornasari, L., Chittaro, L., & Brambilla, P. (2011). Virtual reality in autism:
State of the art. Epidemiology and Psychiatric Sciences, 20,235e238.
Bjornsdotter, M., Wang, N., Pelphrey, K., & Kaiser, M. D. (2016). Evaluation of
quantied social perception circuit activity as a neurobiological marker of
autism spectrum disorder. JAMA Psychiatry, 73,614e621.
Chen, S. H., & Bernard-Opitz, V. (1993). Comparison of personal and computer-
assisted instruction for children with autism. Mental Retardation, 31, 368e376.
Cheng, Y. F., Moore, D., McGrath, P., &Fan, Y. L. (2005). Collaborative virtual envi-
ronment technology for people with autism. 5th IEEE International Conference
on Advanced Learning Technologies, Proceedings, 247e248.
Chu, C., Ni, Y., Tan, G., Saunders, C. J., & Ashburner, J. (2011). Kernel regression for
fMRI pattern prediction. Neuroimage, 56, 662e673.
Constantino, J. N. (2012). Social responsiveness scale (2nd ed.). Torrance, CA: Western
Psychological Services.
Deen, B., Koldewyn, K., Kanwisher, N., & Saxe, R. (2015). Functional organization of
social perception and cognition in the superior temporal sulcus. Cerebral Cortex,
25, 4596e4609.
Desikan, R. S., Segonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., et al.
(2006). An automated labeling system for subdividing the human cerebral
cortex on MRI scans into gyral based regions of interest. Neuroimage, 31,
Didehbani, N., Allen, T., Kandalaft, M., Krawczyk, D., & Chapman, S. (2016). Virtual
reality social cognition training for children with high functioning autism.
Computers in Human Behavior, 62, 703e711.
Doehrmann, O., Ghosh, S. S., Polli, F. E., Reynolds, G. O., Horn, F., Keshavan, A., et al.
(2013). Predicting treatment response in social anxiety disorder from functional
magnetic resonance imaging. JAMA Psychiatry, 70,87
Domes, G., Kumbier, E., Heinrichs, M., & Herpertz, S. C. (2014). Oxytocin promotes
facial emotion recognition and amygdala reactivity in adults with asperger
syndrome. Neuropsychopharmacology, 39, 698e706.
Eack, S. M., Mazefsky, C. A., & Minshew, N. J. (2015). Misinterpretation of facial
expressions of emotion in verbal adults with autism spectrum disorder. Autism,
19, 308e315.
Etkin, A., Buchel, C., & Gross, J. J. (2015). The neural bases of emotion regulation.
Nature Reviews Neuroscience, 16, 693e700.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (20 09). Statistical power analyses
using G*Power 3.1: Tests for correlation and regression analyses. Behavior
Research Methods, 41,1149e116 0 .
Gallese, V., Keysers, C., & Rizzolatti, G. (20 04). A unifying view of the basis of social
cognition. Trends in Cognitive Sciences, 8, 396e403.
Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The autism diagnostic observation
schedule: Revised algorithms for improved diagnostic validity. Journal of Autism
and Developmental Disorders, 37,613e627.
Hahamy, A., Behrmann, M., & Malach, R. (2015). The idiosyncratic brain: Distortion
of spontaneous connectivity patterns in autism spectrum disorder. Nature
Neuroscience, 18, 302e309.
Hahn, T., Kircher, T., Straube, B., Wittchen, H. U., Konrad, C., Strohle, A., et al. (2015).
Predicting treatment response to cognitive behavioral therapy in panic disorder
with agoraphobia by integrating local neural information. JAMA Psychiatry, 72,
Haxby, J. V., Connolly, A. C., & Guntupalli, J. S. (2014). Decoding neural represen-
tational spaces using multivariate pattern analysis. Annual Review of Neurosci-
ence, 37, 435e456, 37.
Heberlein, A. S., & Adolphs, R. (2004). Impaired spontaneous anthropomorphizing
despite intact perception and social knowledge. Proceedings of the National
Academyof Sciences of the United States of America, 101, 7487e7491.
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e66 65
Heider, F., & Simmel, M. (1944). An experimental study of apparent behavior.
American Journal of Psychology, 57,243e259.
Hoeft, F., McCandliss, B. D., Black, J. M., Gantman, A., Zakerani, N., Hulme, C., et al.
(2011). Neural systems predicting long-term outcome in dyslexia. Proceedings of
the National Academyof Sciences of the United States of America, 108,361e366.
Hu, Z., Chan, R. C., & McAlonan, G. M. (2010). Maturation of social attribution skills
in typically developing children: An investigation using the social attribution
task. Behavioral and Brain Functions, 6,10.
Hus, V., & Lord, C. (2014). The autism diagnostic observation schedule, module 4:
Revised algorithm and standardized severity scores. Journal of Autism and
Developmental Disorders, 44, 1996e2012.
Insel, T. R. (2014). The NIMH research domain criteria (RDoC) project: Precision
medicine for psychiatry. The American Journal of Psychiatry, 171, 395e397.
Jenkinson, M., Beckmann, C. F., Behrens, T. E., Woolrich, M. W., & Smith, S. M. (2012).
Fsl. Neuroimage, 62, 782e790.
Johansso, G. (1973). Visual-perception of biological motion and a model for its
analysis. Perception &Psychophysics, 14,201e211.
Johnson, M. H. (2006). Biological motion: A perceptual life detector? Current Biology,
Kaiser, M. D., Hudac, C. M., Shultz, S., Lee, S. M., Cheung, C., Berken, A. M., et al.
(2010). Neural signatures of autism. Proceedings of the National Academyof Sci-
ences of the United States of America, 107, 21223e21228.
Kandalaft, M. R., Didehbani, N., Cullum, C. M., Krawczyk, D. C., Allen, T. T.,
Tamminga, C. A., et al. (2012). The wechsler ACS social perception subtest: A
preliminary comparison with other measures of social cognition. Journal of
Psychoeducational Assessment, 30, 455e465.
Kandalaft, M. R., Didehbani, N., Krawczyk, D. C., Allen, T. T., & Chapman, S. B. (2013).
Virtual reality social cognition training for young adults with high-functioning
autism. Journal of Autism and Developmental Disorders, 43,34e44.
Kanske, P., Heissler, J., Schonfelder, S., Bongers, A., & Wessa, M. (2011). How to
regulate Emotion? Neural networks for reappraisal and distraction. Cerebral
Cortex, 21,1379e1388.
Klin, A., Lin, D. J., Gorrindo, P., Ramsay, G., & Jones, W. (2009). Two-year-olds with
autism orient to non-social contingencies rather than biological motion. Nature,
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative
science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4,
Lieberman, M. D., & Cunningham, W. A. (2009). Type I and type II error concerns in
fMRI research: Re-balancing the scale. Social Cognitive and AffectiveNeuroscience,
4, 423e428.
Lombardo, M. V., Lai, M. C., Auyeung, B., Holt, R. J., Allison, C., Smith, P., et al. (2016).
Unsupervised data-driven stratication of mentalizing heterogeneity in autism.
Scientic Reports, 6, 35333.
Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Jr., Leventhal, B. L., DiLavore, P. C., et al.
(2000). The autism diagnostic observation schedule-generic: A standard mea-
sure of social and communication decits associated with the spectrum of
autism. Journal of Autism and Developmental Disorders, 30, 205e223.
Malone, I. B., Leung, K. K., Clegg, S., Barnes, J., Whitwell, J. L., Ashburner, J., et al.
(2015). Accurate automatic estimation of total intracranial volume: A nuisance
variable with less nuisance. Neuroimage, 104, 366e372.
Mansson, K. N., Frick, A., Boraxbekk, C. J., Marquand, A. F., Williams, S. C.,
Carlbring, P., et al. (2015). Predicting long-term outcome of Internet-delivered
cognitive behavior therapy for social anxiety disorder using fMRI and support
vector machine learning. Translational Psychiatry, 5, e530.
Maskey, M., Lowry, J., Rodgers, J., McConachie, H., & Parr, J. R. (2014). Reducing
specic phobia/fear in young people with autism spectrum disorders (ASDs)
through a virtual reality environment intervention. PLoS One, 9,e100374.
McKay, L. S., Simmons, D. R., McAleer, P., Marjoram, D., Piggot, J., & Pollick, F. E.
(2012). Do distinct atypical cortical networks process biological motion infor-
mation in adults with Autism Spectrum Disorders? Neuroimage, 59, 1524e1533.
Mitchell, R. L. C. (2006). How does the brain mediate interpretation of incongruent
auditory emotions? The neural response to prosody in the presence of con-
icting lexico-semantic cues. European Journal of Neuroscience, 24,3611e3618.
Moore, M., & Calvert, S. (200 0). Brief report: Vocabulary acquisition for children
with autism: Teacher or computer instruction. Journal of Autism and Develop-
mental Disorders, 30, 359e362.
Nummenmaa, L., Hirvonen, J., Parkkola, R., & Hietanen, J. K. (2008). Is emotional
contagion special? An fMRI study on neural systems for affective and cognitive
empathy. Neuroimage, 43,571e580.
Parsons, S., & Cobb, S. (2011). State-of-the-art of virtual reality technologies for
children on the autism spectrum. European Journal of Special Needs Education,
Parsons, S., & Mitchell, P. (2002). The potential of virtual reality in social skills
training for people with autistic spectrum disorders. Journal of Intellectual
DisabilityResearch, 46, 430e443.
Parsons, S., Mitchell, P., & Leonard, A. (2004). The use and understanding of virtual
environments by adolescents with autistic spectrum disorders. Journal of
Autism and Developmental Disorders, 34,449e466.
Parsons, S., Mitchell, P., & Leonard, A. (2005). Do adolescents with autistic spectrum
disorders adhere to social conventions in virtual environments? Autism, 9,
Pearson. (2009). Advanced clinical soultions (ACS) for the WAIS-IV and WMS-IV. San
Antonio, TX: Pearson.
Phelps, E. A., & LeDoux, J. E. (20 05). Contributions of the amygdala to emotion
processing: From animal models to human behavior. Neuron, 48,175e187.
Pruim, R. H., Mennes, M., van Rooij, D., Llera, A., Buitelaar, J. K., & Beckmann, C. F.
(2015). ICA-aroma: A robust ICA-based strategy for removing motion artifacts
from fMRI data. Neuroimage, 112, 267e277.
Rice, K., & Redcay, E. (2016). Interaction matters: A perceived social partner alters
the neural processing of human speech. Neuroimage, 129, 480e488.
Saggar, M., Shelly, E. W., Lepage, J. F., Hoeft, F., & Reiss, A. L. (2014). Revealing the
neural networks associated with processing of natural social interaction and the
related effects of actor-orientation and face-visibility. Neuroimage, 84,648e656.
Scattone, D. (2008). Enhancing the conversation skills of a boy with Asperger's
Disorder through Social Stories and video modeling. Journal of Autism and
Developmental Disorders, 38, 395e400.
Schrouff, J., Rosa, M. J., Rondina, J. M., Marquand, A. F., Chu, C., Ashburner, J., et al.
(2013). PRoNTo: Pattern recognition for neuroimaging toolbox. Neuro-
informatics, 11,319e337.
Screaming_Bee. (2005). MorphVox (3.4.18 ed.) (In).
Simion, F., Regolin, L., & Bulf, H. (2008). A predisposition for biological motion in the
newborn baby. Proceedings of the National Academyof Sciences of the United
States of America, 105, 809e813.
Smith, M. J., Ginger, E. J., Wright, K., Wright, M. A., Taylor, J. L., Humm, L. B., et al.
(2014). Virtual reality job interview training in adults with autism spectrum
disorder. Journal of Autism and Developmental Disorders, 44, 2450e2463.
Trepagnier, C. Y., Olsen, D. E., Boteler, L., & Bell, C. A. (2011). Virtual conversation
partner for adults with autism. Cyberpsychology Behavior and Social Networking,
Tzanavari, A., Charalambous-Darden, N., Herakleous, K., &Poullis, C. (2015). Effec-
tiveness of an Immersive Virtual Environment (CAVE) for Teaching Pedestrian
Crossing to Children with PDD-NOS. 15th Ieee International Conference on
Advanced Learning Technologies (Icalt 2015), 423e427.
Uljarevic, M., & Hamilton, A. (2013). Recognition of emotions in autism: A formal
meta-analysis. Journal of Autism and Developmental Disorders, 43,1517e1526.
Vallortigara, G., Regolin, L., & Marconato, F. (2005). Visually inexperienced chicks
exhibit spontaneous preference for biological motion patterns. PLoS Biology, 3,
Ventola, P., Yang, D. Y., Friedman, H. E., Oosting, D., Wolf, J., Sukhodolsky, D. G., et al.
(2015). Heterogeneity of neural mechanisms of response to pivotal response
treatment. Brain Imaging and Behavior, 9,74e88.
van Waarde, J. A., Scholte, H. S., van Oudheusden, L. J., Verwey, B., et al. (2015).
A functional MRI marker may predict the outcome of electroconvulsive therapy
in severe and treatment-resistant depression. Molecular Psychiatry, 20,
Wainer, A. L., & Ingersoll, B. R. (2011). The use of innovative computer technology
for teaching social communication to individuals with autism spectrum disor-
ders. Research in Autism Spectrum Disorders, 5,96e107.
Wallace, S., Parsons, S., Westbury, A., White, K., White, K., & Bailey, A. (2010). Sense
of presence and atypical social judgments in immersive virtual environments.
Responses of adolescents with Autism Spectrum Disorders. Autism, 14,199e213.
Walter, H., von Kalckreuth, A., Schardt, D., Stephan, A., Goschke, T., & Erk, S. (2009).
The temporal dynamics of voluntary emotion regulation. PLoS One, 4, e6726.
Wechsler, D. (1999). Wechsler abbreviated scale of intelligence (WASI). San Antonio,
TX: Pearson.
Wechsler, D. (2011). Wechsler abbreviated scale of intelligence (WASI-II) (2nd ed.).
San Antonio, TX: Pearson.
Whiteld-Gabrieli, S., Ghosh, S. S., Nieto-Castanon, A., Saygin, Z., Doehrmann, O.,
Chai, X. J., et al. (2016). Brain connectomics predict response to treatment in
social anxiety disorder. Molecular Psychiatry, 21, 680e685.
10.1038/mp.2015.109. published online 11 August 2015.
Yang, D., Pelphrey, K. A., Sukhodolsky, D. G., Crowley, M. J., Dayan, E., Dvornek, N. C.,
et al. (2016). Brain responses to biological motion predict treatment outcome in
young children with autism. Translational Psychiatry, 6, e948.
Yang, D. Y., Rosenblau, G., Keifer, C., & Pelphrey, K. A. (2015). An integrative neural
model of social perception, action observation, and theory of mind. Neurosci-
ence and BiobehavioralReviews, 51, 263e275.
Yang, Y. J., Sukhodolsky, D. G., Lei, J., Dayan, E., Pelphrey, K. A., & Ventola, P. (2017).
Distinct neural bases of disruptive behavior and autism symptom severity
in boys with autism spectrum disorder. Journal of NeurodevelopmentalDisorders,
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011).
Large-scale automated synthesis of human functional neuroimaging data. Na-
ture Methods, 8, 665e670.
Y.J.D. Yang et al. / Behaviour Research and Therapy 93 (2017) 55e6666
... Acceptability, enjoyment, and high motivation in using HMDs and computer-technology platforms have been reported in many studies, including children with autism Newbutt et al., 2016;Newbutt et al., 2020). In addition, several researchers have described VR as having advantages over other forms of training skills for autistic individuals (Yang et al., 2017). The possibility of simulating real-world environments with reduced stress or feeling of anxiety for negative real-world consequences when practicing various situations is seen as the major possible advantage. ...
... However, several studies also showed the essential role of left pSTS in ASD. For example, the left pSTS activation could predict the changes in emotion recognition in adults with ASD receiving virtual reality social cognition training (Yang et al. 2017). Another study on neurotypical populations found that children tend to recruit bilateral pSTS, but adults would show right-lateralized pSTS activation during the facial processing task (Hildesheim et al. 2020). ...
Full-text available
Following the published behavioral and cognitive results of this single-blind parallel sham-controlled randomized clinical trial, the current study aimed to explore the impact of intermittent theta burst stimulation (iTBS), a variant of excitatory transcranial magnetic stimulation, over the bilateral posterior superior temporal sulci (pSTS) on white matter macro/microstructure in intellectually able children and adolescents with autism. Participants were randomized and blindly received active or sham iTBS for 4 weeks (the single-blind sham-controlled phase). Then, all participants continued to receive active iTBS for another 4 weeks (the open-label phase). The clinical results were published elsewhere. Here, we present diffusion magnetic resonance imaging data on potential changes in white matter measures after iTBS. Twenty-two participants in Active-Active group and 27 participants in Sham-Active group underwent multi-shell high angular resolution diffusion imaging (64-direction for b=2000 & 1000 s/mm², respectively) at baseline, week 4, and week 8. With longitudinal fixel-based analysis, we found no white matter changes following iTBS from baseline to week 4 (a null treatment by time interaction and a null within-group paired comparison in the Active-Active group), nor from baseline to week 8 (null within-group paired comparisons in both Active-Active and Sham-Active groups). As for the brain-symptoms relationship, we did not find baseline white matter metrics associated with symptom changes at week 4 in either group. Our results raise the question of what the minimal cumulative stimulation dose required to induce the white matter plasticity is.
... The changes in social cognitive outcomes were the same across the sample. These findings demonstrate that while our virtual reality social cognition training was originally designed to support individuals with autism spectrum disorder (Kandalaft et al., 2013;Didehbani et al., 2016;Yang et al., 2017), this program appears to also be effective at supporting social learners with neurodiverse backgrounds, including those without any formal diagnosis. In addition, the location effect on the SAT score was not influenced by participants' diagnoses. ...
Full-text available
Low immersion virtual reality (LIVR) is a computer-generated, three-dimensional virtual environment that allows for authentic social interactions through a personal avatar, or digital representation of oneself. Lab-based delivery of LIVR social skills intervention has been shown to support social learning through controlled, targeted practice. Recent remote technological advancements allow LIVR-based social skills training to potentially overcome accessibility barriers by delivering to youth in their home. This study investigated the impact of 10-h of Charisma™ Virtual Social Training (CHARISMA-VST), a LIVR-based intervention, on social skill changes in children and adolescents who struggle socially via either in-person or remote training protocols. Specifically, the aims examined both the impact of training location (in-person vs remote access) and diagnosis (parent report of autism spectrum disorder (ASD) diagnosis versus parent report of other non-ASD diagnosis) on objective measures of social skill change following CHARISMA-VST. Researchers delivered the CHARISMA-VST via Charisma 1.0, a customized virtual gaming environment. Sixty-seven participants (49 males, 18 females) between the ages of 9–17, with parent reported social challenges, completed 10, 1-h CHARISMA-VST sessions during which nine social cognitive strategies were taught and then practiced within a LIVR environment with interspersed social coaching. Four social cognitive domains were measured pre-post training: emotion recognition, social inferencing, social attribution, and social self-schemata. Results revealed improvements in emotion recognition, social inferencing, social attribution, and social self-schemata with medium to large effect sizes following the CHARISMA-VST. There was no moderating effect of training location on emotion recognition, social inferencing, and social self-schemata, suggesting comparable gains whether participants accessed the technology in their own homes or from a school or specialty center. There was no moderating effect of ASD versus non-ASD diagnosis on performance measures, suggesting CHARISMA-VST may be effective in improving social skills in individuals beyond its initially designed use focused on individuals with ASD. These encouraging findings from this pilot intervention study provide some of the first evidence of potential new virtual technology tools, as exemplified by CHARISMA-VST, to improve one of the most important aspects of human behavior—social skills and human connectedness in youth with a range of social competency challenges.
... J. D. Yang et al., 2018). Also, pre-treatment brain activity predicted treatment response (Yang, Allen, et al., 2017). An RCT by Frolli et al. (2022) compared VR-based (3D-videos) and traditional emotion recognition training (cardboard images) in children with ASD. ...
Full-text available
Background Virtual reality (VR) technologies are playing an increasingly important role in the diagnostics and treatment of mental disorders. Objective To systematically review the current evidence regarding the use of VR in the diagnostics and treatment of mental disorders. Data source Systematic literature searches via PubMed (last literature update: 9th of May 2022) were conducted for the following areas of psychopathology: Specific phobias, panic disorder and agoraphobia, social anxiety disorder, generalized anxiety disorder, posttraumatic stress disorder (PTSD), obsessive-compulsive disorder, eating disorders, dementia disorders, attention-deficit/hyperactivity disorder, depression, autism spectrum disorder, schizophrenia spectrum disorders, and addiction disorders. Eligibility criteria To be eligible, studies had to be published in English, to be peer-reviewed, to report original research data, to be VR-related, and to deal with one of the above-mentioned areas of psychopathology. Study evaluation For each study included, various study characteristics (including interventions and conditions, comparators, major outcomes and study designs) were retrieved and a risk of bias score was calculated based on predefined study quality criteria. Results Across all areas of psychopathology, k = 9315 studies were inspected, of which k = 721 studies met the eligibility criteria. From these studies, 43.97% were considered assessment-related, 55.48% therapy-related, and 0.55% were mixed. The highest research activity was found for VR exposure therapy in anxiety disorders, PTSD and addiction disorders, where the most convincing evidence was found, as well as for cognitive trainings in dementia and social skill trainings in autism spectrum disorder. Conclusion While VR exposure therapy will likely find its way successively into regular patient care, there are also many other promising approaches, but most are not yet mature enough for clinical application. Review registration PROSPERO register CRD42020188436. Funding The review was funded by budgets from the University of Bonn. No third party funding was involved.
... To achieve this effect, including text, speech, and other information superimposed into the real-world scene, it is necessary to integrate virtual information into the real-world scene in real time and output and display the final result. e database puts the user information collected in real time back into the virtual image, and the virtual image changes through the real-time user data [18]. is allows the virtual image to change with the user's dynamic behavior. ...
Full-text available
Since the Belt and Road Initiative was proposed, it has received extensive attention from many aspects of the international community. To disseminate the ideas, development status, and construction achievements of the Belt and Road Initiative to the outside world has become a key and difficult problem in the development process of the initiative. Among different types of film and television, virtual reality technology should be the best interpretation of film communication. The film would bring a wider space for the development of the film by means of real reproduction and virtual reality and make the film’s record form more diversified. The continuous development and innovation of new media technology enable virtual reality to pass through panoramic lenses. As the most powerful technical support for the movie, it can even be converted into the user’s eyes to serve the movie so that the presentation of the movie has more new possibilities. In this context, the article aims to interpret the visualization practice and dissemination effects of digital Chinese films with virtual reality technology. The article used the mathematical expression of the visual SLAM problem to improve the process from the world coordinate system to the pixel coordinate system. The system is constructed, and the approximate calculation of the function is carried out. The constructed system was used in the experiments. This article conducts a survey of Google’s average popularity and a comparison of the voxelization process. The experimental data showed that the performance of Sponza in this branch of the sparse octree is 88.48 ms at the resolution of 100,000, which is weaker than 80.34 ms in the Dragon scene. The performance of the cascaded voxel texture is 6.889 ms, which is also weaker than Dragon’s 5.727 ms. The structure of the rules of the cascaded voxel texture scheme made it smoother to assist with the GPU, and the speed was the fastest in both the Dragon scene and the Sponza scene. The experimental content was well completed.
... Learning emotion recognition in VEs could remove such emotional barriers and obstacles for the ASD population, as VR training programs have been proved to be particularly helpful for them regarding emotion recognition improvement. For example, previous studies have reported enhanced behavioral performance [61], as well as neural predictors of change [143,152], in emotion recognition and theory of mind in participants with ASD after they completed the VR-SCT program. Similar results were obtained by Ke and Im [139], who observed a consistent and obvious increase in ASD children's perceptions of emotion from facial expression and posture cues after a VR-based social interaction program, and Ip, et al. [138], who documented significant improvements in emotion recognition in over 100 school-aged ASD children after training completion in a CAVE-like immersive VR-enabled system. ...
Full-text available
Virtual reality (VR) technology gains theoretical support from rehabilitation and pedagogical theories and offers a variety of capabilities in educational and interventional contexts with affordable products. VR is attracting increasing attention in the medical and healthcare industry as it provides fully interactive three-dimensional simulations of real-world settings and social situations, which are particularly suitable for cognitive and performance training including social and interaction skills. The worldwide rising trend in the prevalence of autism spectrum disorder calls for innovative and efficacious techniques for assessment and treatment. The article offers a summary of current perspectives and evidence-based applications of VR technology as an educational and intervention tool for individuals with autism spectrum disorder, with a primary focus on social communication including social functioning, emotion recognition, and speech and language. Technology- and design-related limitations as well as the disputes over the application of virtual reality to autism research and therapy are discussed and future directions of this emerging field are highlighted with regards to application expansion and improvement, technology enhancement, and the development of brain-based research and theoretical models.
Full-text available
Technological change is at the core of all major disruptions in human history, and revolutions, wars, and general development are regularly connected to some sort of technological change. However, not all development is beneficial. While technology has fueled great innovations and rapid development, the notion of sustainable development has gained prominence as we now experience serious social, economic, and environmental challenges. This book examines whether technology can be used to fix the very problems caused by technology, as the various chapters examine different aspects related to how technology has brought us where we are today (which some will say is the best place humanity’s been at according to a range of metrics), and whether technology helps or hinders us in our efforts to solve the challenges we currently face. The issues discussed cover the three sustainability dimensions and include topics such as the materiality of AI, technology in education, AI for gender equality, innovation and the digital divide, and how technology relates to power, the political system, and capitalism. The chapters all build on the theoretical backdrop of technological change, sustainable development, and the UN’s Sustainable Development Goals are actively used throughout this book, both to examine how these goals capture or overlook central elements of sustainable development, and also to facilitate and create a common framework of engagement between the chapters. This book provides a novel combination of traditional theories that are explored through different case studies, providing the ground for a better understanding of how and when technology can –and cannot –be the enabler of sustainable development. It is thus an important resource for students of all disciplines, technologists, and those developing and applying new technologies. It is also a valuable resource for politicians and regulators attempting to harness the power of technology for good, while limiting its negative potential.
Full-text available
Objective We aimed to explore the impact of using virtual reality technology to intervene in and encourage the developmental behavior areas of cognition, imitation, and social interaction in children with autism spectrum disorder. Methods Forty-four children with autism spectrum disorder were divided randomly into an intervention group and a control group, with each group consisting of 22 participants. Incorporating conventional rehabilitation strategies, virtual reality technology was used with the intervention group to conduct rehabilitation training in areas including cognition, imitation, and social interaction. The control group was provided conventional/routine clinical rehabilitation training. The children's cognitive development was evaluated before and 3 months after intervention. Results After intervention, the developmental abilities of both groups of children in the areas of cognition, imitation, and social interaction were improved over their abilities measured before the intervention ( P < 0.05). However, post-intervention score differences between the two groups demonstrated that the intervention group levels were better than the control group levels only in the areas of cognition and social interaction ( P < 0.05). Conclusion Combining virtual reality with conventional rehabilitation training improved the cognitive and social development of children with autism spectrum disorder and supported the goal of improving the rehabilitation effect.
Full-text available
Background: Intensification of summer and winter forage cropping is an urgent requirement for reducing feeding costs for the beef and dairy cow industry in Kyushu, Japan. Based on the expectation of progression of global warming, the yield following quadruple cropping of triple maize crops with a winter barley crop was evaluated in a previous 2 years trial. Materials and Methods: Forage quality of the previous triple (spring, summer and autumn) maize with winter barley cropping was determined for 2 years in the region. Winter barley, sown in inter-rows in mid-November, was harvested in early to mid-March while spring maize, sown in early to mid-March, was harvested in early to mid-June, summer maize, sown from early to mid-June, was harvested in late August and autumn maize, sown from late August to early-September, was harvested in mid-November. Results: Yield of winter barley was around 900-930 g DM m⁻², spring maize was 1440-1800 g DM m⁻² with 41-59% ear DM and summer maize was 820-1170 g DM m⁻² with 41-44% ear DM. However, the autumn maize yield varied greatly from 330-810 g DM m⁻² with 33-56% ear DM due to typhoon and cool weather damage. In vitro DM digestibility (IVDMD) in maize ears was higher than 75% and in leaf blades was higher than 60%, while for barley leaf blade, it was higher than 80%; the values for stems were lower than leaf blades for all crops of both species. The IVDMD was negatively correlated with Neutral Detergent Fiber (NDF) and Acid Detergent Fiber (ADF) concentration (r = -0.790 and -0.856, respectively) at the 1% probability level. By estimating Total Digestible Nutrient (TDN) concentration from ADF concentration based on regression equations, the TDN yield was 2750-2870 g DM m⁻² year⁻¹ over 2 years. Conclusion: Triple maize with winter barley cropping can produce 4090-4330 g DM m⁻² year⁻¹ with 66-67% TDN concentration in the region, which is recommended as a potential system for intensive ley farming. © 2017 Yingkui Yang et al. This is an open access article distributed under the terms of the creative commons attribution License, which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.
Full-text available
Background Disruptive behavior in Autism Spectrum Disorder (ASD) is an important clinical problem, but its neural basis remains poorly understood. The current research aims to better understand the neural underpinnings of disruptive behavior in ASD, while addressing whether the neural basis is shared with or separable from that of core ASD symptoms. Methods Participants consisted of 48 male children and adolescents: 31 ASD (7 had high disruptive behavior) and 17 typically-developing (TD) controls, well-matched on sex, age, and IQ. For ASD participants, autism symptom severity, disruptive behavior, anxiety symptoms, and ADHD symptoms were measured. All participants were scanned while viewing biological motion (BIO) and scrambled motion (SCR). Two fMRI contrasts were analyzed: social perception (BIO>SCR) and Default Mode Network (DMN) deactivation (fixation>BIO). Age and IQ were included as covariates of no interest in all analyses. Results First, the between-group analyses on BIO>SCR showed that ASD is characterized by hypoactivation in the social perception circuitry, and ASD with high or low disruptive behavior exhibited similar patterns of hypoactivation. Second, the between-group analyses on fixation>BIO showed that ASD with high disruptive behavior exhibited more restricted and less DMN deactivation, when compared to ASD with low disruptive behavior or TD. Third, the within-ASD analyses showed that (a) autism symptom severity (but not disruptive behavior) was uniquely associated with less activation in the social perception regions including the posterior superior temporal sulcus and inferior frontal gyrus; (b) disruptive behavior (but not autism symptom severity) was uniquely associated with less DMN deactivation in the medial prefrontal cortex (MPFC) and lateral parietal cortex; and (c) anxiety symptoms mediated the link between disruptive behavior and less DMN deactivation in both anterior cingulate cortex (ACC) and MPFC, while ADHD symptoms mediated the link primarily in ACC. Conclusions In boys with ASD, disruptive behavior has a neural basis in reduced DMN deactivation, which is distinct and separable from that of core ASD symptoms, with the latter characterized by hypoactivation in the social perception circuitry. These differential neurobiological markers may potentially serve as neural targets or predictors for interventions when treating disruptive behavior versus core symptoms in ASD.
Full-text available
Autism spectrum disorders (ASDs) are common yet complex neurodevelopmental disorders, characterized by social, communication and behavioral deficits. Behavioral interventions have shown favorable results—however, the promise of precision medicine in ASD is hampered by a lack of sensitive, objective neurobiological markers (neurobiomarkers) to identify subgroups of young children likely to respond to specific treatments. Such neurobiomarkers are essential because early childhood provides a sensitive window of opportunity for intervention, while unsuccessful intervention is costly to children, families and society. In young children with ASD, we show that functional magnetic resonance imaging-based stratification neurobiomarkers accurately predict responses to an evidence-based behavioral treatment—pivotal response treatment. Neural predictors were identified in the pretreatment levels of activity in response to biological vs scrambled motion in the neural circuits that support social information processing (superior temporal sulcus, fusiform gyrus, amygdala, inferior parietal cortex and superior parietal lobule) and social motivation/reward (orbitofrontal cortex, insula, putamen, pallidum and ventral striatum). The predictive value of our findings for individual children with ASD was supported by a multivariate pattern analysis with cross validation. Predicting who will respond to a particular treatment for ASD, we believe the current findings mark the very first evidence of prediction/stratification biomarkers in young children with ASD. The implications of the findings are far reaching and should greatly accelerate progress toward more precise and effective treatments for core deficits in ASD.
Full-text available
Individuals affected by autism spectrum conditions (ASC) are considerably heterogeneous. Novel approaches are needed to parse this heterogeneity to enhance precision in clinical and translational research. Applying a clustering approach taken from genomics and systems biology on two large independent cognitive datasets of adults with and without ASC (n = 694; n = 249), we find replicable evidence for 5 discrete ASC subgroups that are highly differentiated in item-level performance on an explicit mentalizing task tapping ability to read complex emotion and mental states from the eye region of the face (Reading the Mind in the Eyes Test; RMET). Three subgroups comprising 45–62% of ASC adults show evidence for large impairments (Cohen’s d = −1.03 to −11.21), while other subgroups are effectively unimpaired. These findings delineate robust natural subdivisions within the ASC population that may allow for more individualized inferences and accelerate research towards precision medicine goals.
Full-text available
Virtual reality appears to be a promising and motivating platform to safely practice and rehearse social skills for children with Autism Spectrum Disorders (ASD). However, the literature to date is subject to limitations in elucidating the effectiveness of these virtual reality interventions. This study investigated the impact of a Virtual Reality Social Cognition Training to enhance social skills in children with ASD. Thirty children between the ages of 7–16 diagnosed with ASD completed 10, 1-h sessions across 5 weeks. Three primary domains were measured pre-post: emotion recognition, social attribution, attention and executive function. Results revealed improvements on measures of emotion recognition, social attribution, and executive function of analogical reasoning. These preliminary findings suggest that the use of a virtual reality platform offers an effective treatment option for improving social impairments commonly found in ASD.
Importance: Autism spectrum disorder (ASD) is marked by social disability and is associated with dysfunction in brain circuits supporting social cue perception. The degree to which neural functioning reflects individual-level behavioral phenotype is unclear, slowing the search for functional neuroimaging biomarkers of ASD. Objective: To examine whether quantified neural function in social perception circuits may serve as an individual-level marker of ASD in children and adolescents. Design, setting, and participants: The cohort study was conducted at the Yale Child Study Center and involved children and adolescents diagnosed as having ASD and typically developing participants. Participants included a discovery cohort and a larger replication cohort. Individual-level social perception circuit functioning was assessed as functional magnetic resonance imaging brain responses to point-light displays of coherent vs scrambled human motion. Main outcomes and measures: Outcome measures included performance of quantified brain responses in affected male and female participants in terms of area under the receiver operating characteristic curve (AUC), sensitivity and specificity, and correlations between brain responses and social behavior. Results: Of the 39 participants in the discovery cohort aged 4 to 17 years, 22 had ASD and 30 were boys. Of the 75 participants in the replication cohort aged 7 to 20 years, 37 had ASD and 52 were boys. A relative reduction in social perception circuit responses was identified in discovery cohort boys with ASD at an AUC of 0.75 (95% CI, 0.52-0.89; P = .01); however, typically developing girls and girls with ASD could not be distinguished (P = .54). The results were confirmed in the replication cohort, where brain responses were identified in boys with ASD at an AUC of 0.79 (95% CI, 0.64-0.91; P < .001) and failed to distinguish affected and unaffected girls (P = .82). Across both cohorts, boys were identified at an AUC of 0.77 (95% CI, 0.64-0.86) with corresponding sensitivity and specificity of 76% each. Additionally, brain responses were associated with social behavior in boys but not in girls. Conclusions and relevance: Quantified social perception circuit activity is a promising individual-level candidate neural marker of the male ASD behavioral phenotype. Our findings highlight the need to better understand effects of sex on social perception processing in relation to ASD phenotype manifestations.