Available via license: CC BY 4.0
Content may be subject to copyright.
Behav. Sci. 2023, 13, x. https://doi.org/10.3390/xxxxx www.mdpi.com/journal/behavsci
Article
Virtual Reality Training of Social Skills in Adults with Autism
Spectrum Disorder: An Examination of Acceptability, Usabil-
ity, User Experience, Social Skills, and Executive Functions.
Panagiotis Kourtesis 1,2 ┼*, Evangelia-Chrysanthi Kouklari 3,4┼, Petros Roussos 1, Vasileios Mantas4, Katerina Pa-
panikolaou3 , Christos Skaloumpakas5,6, and Artemios Pehlivanidis4
1 Department of Psychology, National and Kapodistrian University of Athens, Athens, Greece
2 Department of Psychology, University of Edinburgh, Edinburgh, United Kingdom
3 Department of Child Psychiatry, Aghia Sophia Children’s Hospital, School of Medicine, National and Ka-
podistrian University of Athens, Athens, Greece
4 1st Department of Psychiatry, Eginition Hospital, School of Medicine, National and Kapodistrian University
of Athens, Athens, Greece
5 Department of Child Psychiatry, P. & A.Kyriakou Children's Hospital
6 Habilis, R&D Team, Athens, Greece
┼ Authors share First Authorship due to equal contribution
* Correspondence: pkourtesis@psych.uoa.gr
Abstract: Poor social skills in autism spectrum disorder (ASD) are associated with reduced inde-
pendence in daily life. Current interventions for improving the social skills of individuals with ASD
fail to represent the complexity of real-life social settings and situations. Virtual reality (VR) may
facilitate social skills training in social environments and situations proximal to real life, however,
more research is needed for elucidating aspects such as the acceptability, usability, and user expe-
rience of VR systems in ASD. Twenty-five participants with ASD attended a neuropsychological
evaluation and three sessions of VR social skills training, incorporating 5 social scenarios with three
difficulty levels for each. Participants reported high acceptability, system usability, and user expe-
rience. Significant correlations were observed between performance in social scenarios, self-reports,
and executive functions. Working memory and planning ability were significant predictors of func-
tionality level in ASD and the VR system’s perceived usability respectively. Yet, performance in
social scenarios was the best predictor of usability, acceptability, and functionality level. Planning
ability substantially predicted performance in social scenarios, postulating an implication in social
skills. Immersive VR social skills training in individuals with ASD appears an appropriate service,
yet an error-less approach, which is adaptive to the individual’s needs, should be preferred.
Keywords: Virtual Reality, Training, Autism, Social Skills, , Executive Functions, Acceptability, Us-
ability, User Experience, Prompts.
1. Introduction
Autism Spectrum Disorder (ASD) is a lifelong complex neurodevelopmental disor-
der that significantly impairs individuals’ verbal and nonverbal communications, social
interactions and behaviours (i.e. exhibition of restricted interests, repetitive and unusual
sensory-motor behaviours) (Diagnostic and Statistical Manual of Mental Disorders
[DSM]-fifth Edition; [1]). Prevalence estimates of ASD have increased over time as a recent
systematic review [2] reported a global prevalence (ranging within and across regions)
with a median prevalence of 100/10,000. ASD presents a striking sex difference as males
are more likely to be affected relative to females (3:1 ratio) [3]. The ASD etiology is sug-
gested to be multifactorial as both genetic and non-genetic factors (e.g., prenatal/
Citation: To be added by editorial
staff during production.
Academic Editor: Firstname Last-
name
Received: date
Revised: date
Accepted: date
Published: date
Copyright: © 2023 by the authors.
Submitted for possible open access
publication under the terms and
conditions of the Creative Commons
Attribution (CC BY) license
(https://creativecommons.org/license
s/by/4.0/).
Behav. Sci. 2023, 13, x FOR PEER REVIEW 2 of 37
perinatal) may play a crucial role to the manifestation of the disorder (see [4] for a review).
Since its first depiction, ASD is now regarded as a spectrum that spans from very mild to
severe [5] as symptoms manifest differently in each individual based on their functionality
level (level 1-requiring support, level 2-requiring substantial support; level 3-requiring
very substantial support). Nevertheless, several individuals with ASD (not all) require
some kind of support throughout their life [5]. Even individuals with high-functioning
ASD, similar to other individuals on the mild and lower ends of the spectrum, present
social skills deficits across the lifespan (up to adulthood). Adults with ASD are likely to
experience problems in social and everyday life functioning due to a lack of ecological
training and intervention programs during childhood and adolescence [6].
1.1. Social Skills and Executive Functions in ASD
Adults with ASD have been found to experience social isolation, loneliness and social
anxiety (e.g., [7]) due to their deficient social skills such as atypical gaze/poor eye contact,
less conversational involvement, appropriate affect, reduced verbal fluency (e.g., [8], [9]),
poor understanding of social cues, and difficulties in initiating and maintaining social
conversation/communication [10]. The social skills deficits in individuals with high-func-
tioning ASD are mainly attributed to impairments in cognitive components such as Exec-
utive Functions (EF) (e.g.,[11]) or cognitive processing speed (e.g.,[12]). Indeed impaired
EF is another salient characteristic of the spectrum [13] which refers to high-order, goal-
directed cognitive processes that control behaviour, thought and emotions. The EF con-
struct is seen as an umbrella term including abilities such as inhibition, working memory,
and planning (not an exhaustive list; see [14] and [15] for a more detailed EF discussion).
Two recent meta-analyses [16], [17] demonstrated a broad EF impairment in ASD as def-
icits have been consistently found in several EF aspects (e.g., inhibition, working memory,
cognitive flexibility & planning) across the life span.
To implement effective interventions, research over the last decade was needed to
identify which EF aspects contribute to the manifestation of social skills in ASD ([11], [18]
as it is suggested that higher-order cognitive regulation is required for social interactions
[19]. EF has been proposed to support the processing and manipulation of information
from one’s and others’ perspectives, to facilitate socially interactive and communicative
skills [20]. Such associations are understudied in adulthood in ASD. Limited evidence
from childhood and adolescence has shown that performance-based measures of EF (e.g.,
auditory attention, inhibition-switching) relate to social deficits in ASD (e.g., [21], [22])
while ratings-based EF such as initiation, cognitive flexibility and working memory were
found related to adaptive social skills in ASD [23], [24]. A recent study [18] also demon-
strated significant associations between ratings-based EF (self-monitoring) and selective
social skills (social inferencing and social knowledge) in ASD. It should be noted though
that all aforementioned studies, despite their findings, did not use in vivo measures of
social functioning or a naturalistic context of assessment. Social skills finally have been
theoretically proposed to also depend on social cognition aspects such as mental
state/emotion recognition [25] but as these aspects are not consistently associated with
social impairment in ASD [26], the extent to which socio-cognitive abilities associate with
the social difficulties in ASD has been debated over the years. Given these potential asso-
ciations among social cognition and social skills, EF and social skills, and EF and social
cognition (e.g., [27]–[29]), it has been suggested that EF may contribute to social skills both
directly and indirectly [30]. Social cognition aspects are likely to partially mediate the as-
sociation between EF and social skills; perhaps no single cognitive mechanism in ASD can
explain the various social difficulties as argued [31], as there may be several factors po-
tentially contributing to social skills (e.g., poor emotion regulation) that could also explain
the social and behavioral problems in ASD (e.g., [32]).
1.2. Assessment, Training, and Intervention in ASD
Assessment of ASD impairments is critical for identifying potential difficulties and
weaknesses when implementing interventions. For example, widely used measures of
Behav. Sci. 2023, 13, x FOR PEER REVIEW 3 of 37
social functioning include the Social Responsiveness Scale (a measure of general social
ability; [33], [34]), Reading the Mind in the Eyes test (a measure of mental state/emotion
recognition; [35]), and the Autism Diagnostic Observation Schedule (a measure of social
interaction, communication and play [36], [37]). Taking into consideration the tremendous
impact of the aforementioned cognitive and social impairments on the everyday life of
individuals with ASD, suitable intervention and training programs are needed [38]. Tar-
geting cognitive deficits, cognitive training exercises in adults with ASD are usually im-
plemented to enhance performance through repeated practice on EF tasks (e.g., [39], [40]).
Cognitive training exercises encompass various intervention methods such as pen-and-
paper tasks, downloadable tools, and logical games. Given the EF contribution to several
aspects of social functioning, targeting specific EF aspects is thought to improve the effec-
tiveness of training interventions in ASD [41]. However, it should be noted that cognitive
training studies in ASD have been designed in recent years and thus their limited and
mixed results as well as their lack of ecological validity are under ongoing discussion (e.g.,
[42]–[44]).
When it comes to social skills, several different strategies have been used in training
and intervention programs to enhance social functioning (usually social interaction and
communication) in adults with ASD. For example, strategies such as social stories and
social scripts, behavioural modelling and role-playing demonstrations, video modelling
and self-modelling (e.g., [45]) in the context of didactic lessons to enhance conversational
skills, developing friendships, appropriate use of humour, dating, handling embarrassing
feedback and peer pressure (e.g., [46]) have been used in ASD. Most psychosocial inter-
vention and training programs in ASD however are thought to yield limited benefits [47]
because of their limited ecological validity, which does not permit a generalization of the
outcomes to everyday life [48], [49]. The limitations of the aforementioned methods are
thought to likely arise because of the ASD literature’s tendency of examining social
(and/or cognitive) deficits as isolated and individual features and not by evaluating how
they manifest in real-life contexts in which outcomes are influenced by relational dynam-
ics as well [41], [50]. For that reason, computing technology with more naturalistic set-ups
and role plays is a significantly effective training and intervention medium for individuals
with ASD [51].
1.3. Ecological Validity, Virtual Reality Assessments and Interventions
Ecological validity refers to the verisimilitude (i.e., the likeness to everyday life) and
veridicality (i.e., the association between the observed and real-life performance) of a neu-
ropsychological tool, which subsequently allows the generalization to everyday life [52].
In contrast with the paper-and-pencil or computerized approaches, which incorporate
static and simplistic testing and training environments and stimuli, immersive virtual re-
ality (VR) facilitates the attainment of enhanced ecological validity and pleasantness [53].
Immersive VR neuropsychological tools may thus contribute to the understanding of eve-
ryday functionality (e.g., [54], [55]) and improve everyday physical and cognitive func-
tioning (e.g., [56]–[58]). In the context of VR interventions in ASD, immersive VR technol-
ogy facilitates the creation of simulated environments that can be used to help individuals
with ASD improve social skills, communication, and behaviour [59]–[62]. These interven-
tions aim to provide individuals with ASD with a safe and controlled environment in
which to practice and develop skills, as well as to reduce anxiety and stress associated
with real-world interactions [60]. VR interventions can include activities such as role-play-
ing social scenarios, virtual social skills training, and virtual exposure therapy. However,
the effective implementation of immersive VR for research and clinical purposes requires
technological competence [63]. An inappropriate conceptualization of VR training may
render ramifications and compromise its otherwise beneficial outcomes [62].
Nevertheless, several VR applications have efficaciously been implemented for as-
sessment and intervention purposes. The VR Everyday Assessment Lab assesses every-
day memory (prospective and episodic), attention (visuospatial and auditory), and EF
Behav. Sci. 2023, 13, x FOR PEER REVIEW 4 of 37
(planning and multitasking), and has been found a valid and substantially more pleasant
testing experience [53], which indicates everyday functionality of adults [54], [55]. The
ClinicaVR: Classroom-CPT is a VR classroom that examines selective and sustained atten-
tion, and inhibition, which has been validated in children and adolescents [64]. Regarding
interventions in ASD, there is preliminary evidence postulating its feasibility for being
adopted in clinical and educational environments [59], [65]. Also, the use of social stories
in VR has been evaluated by clinicians for implementation in clinical and educational set-
tings for training the social skills of children with ASD [66]. Preliminary evidence sug-
gests that VR software may improve the conversational [61], problem-solving, and com-
munication skills of children with ASD [67]. After a VR training protocol, children with
ASD showed significant improvements in emotion expression and regulation and socio-
emotional reciprocity [68]. Comparably two more studies [69], [70] reported substantial
enhancements in social skills in children with ASD after attending VR-based training ses-
sions. It is important however to underline that VR interventions in ASD are still consid-
ered an emerging field and more research is needed to fully understand their efficacy,
usability, the provided user experience, as well as their acceptability by individuals with
ASD [59], [60], [62]. Furthermore, the relationship between performance in VR social sce-
narios and cognitive functioning has not yet been investigated. Finally, while there are
several VR applications used in children and/or adolescents with ASD, none of the afore-
mentioned VR applications was designed for and implemented in adults with ASD.
1.4. VRESS
The VR Enhancement of Social Skills (VRESS) was developed in line with the guidelines
for developing VR software for research and clinical applications in the field of psychol-
ogy [71], which they have been found to produce VR software that meets the criteria of
the American Academy of Clinical Neuropsychology (AACN) and the National Academy
of Neuropsychology (NAN) [72]. The VRESS incorporates social scenarios that are exem-
plary of adult activities and common in daily life, such as renewing your subscription at
the gym, selecting a movie and buying a ticket at the cinema, browsing the available op-
tions and purchasing a smartphone at the phone store, attending a seminar class and in-
teracting with the instructor and the co-students, and attending a job interview and re-
sponding to the interviewers’ questions. The social scenarios were designed in line with
the guidelines of Gray and Garand [73] for providing social stories that provide to indi-
viduals with ASD (i.e., the learners), a visual representation and a description of a situa-
tion or activity to prepare and instruct them on what to be expected, as well as the under-
lying reasons of this matter. Thus, the social scenarios of VRESS are rather descriptive
than directive. The social stories were designed for individuals with ASD to comprehend
and apply the intricacies of interpersonal communication to interact more appropriately
and effectively. The social stories approach provides the opportunity for people with ASD
to identify the context, discuss their motives, comprehend the amplifiers or the obstacles
and improve their social skills [73], [74].
1.5. Research Aims
In sake of clarity, we provide the description of the terminology that pertains to the
research aims:
• Usability: the capacity of a system to provide a condition for its users to per-
form the tasks safely, effectively, and efficiently while enjoying the experi-
ence.
• User Experience: how a user interacts with and experiences a product, sys-
tem or service.
• Acceptability: the quality of being satisfactory and able to be agreed to or
approved of being a software for a specific purpose.
This study thus aims to:
Behav. Sci. 2023, 13, x FOR PEER REVIEW 5 of 37
1) Evaluate the usability and user experience of an immersive VR training soft-
ware of social skills (i.e., VRESS) in adults with ASD. Moreover, this study
strives to
2) Examine the acceptability of the VR training software of social skills as a so-
cial service (i.e., by a service user’s point of view) that may be prescribed
and/or offered by clinicians, educators, and social workers to adults with
ASD for training and improving their everyday social skills.
3) Investigate the relationships between cognitive functioning (i.e., aspects of
social cognition and EF), independence/functionality level of individuals
with ASD, performance in the VR social scenarios, as well as the acceptabil-
ity, usability, and user experience ratings.
2. Materials and Methods
2.1. VRESS Scenarios and Interface
The VRESS software runs on SteamVR and is compatible with every VR headset that
runs on the SteamVR platform (e.g., HTC Vive series, HTC Vive Pro series, Oculus Rift
series, and Varjo VR series; see here for an exhaustive list). VRESS encompasses five social
scenarios: 1) being at the gym; 2) buying a smartphone at the phone store; 3) going to the
cinema to watch a movie; 4) attending a seminar class; 5) attending a job interview. Each
scenario has three different difficulty levels: 1) easy; 2) moderate; and 3) difficult. Thus,
the five scenarios have three diverse versions (i.e., per difficulty level), which means that
there are a total of fifteen diverse scenarios in VRESS. The difficulty level is determined
by the complexity of the scenario in terms of how many social tasks the users have to
perform and how many 3D characters they need to interact with (e.g., just buying a ticket
or discussing with friends about which film they should watch and then buying tickets
for everyone). Furthermore, given that visual sensitivity (e.g., to intense light) [75] and
agoraphobia and/or social phobia [76] symptoms are highly prevalent in ASD, the diffi-
culty may further be modulated by defining the intensity of lights and the density of the
population of 3D Non-Player Characters (NPCs; i.e., 3D characters that the user does not
interact with them) in the virtual environment. VRESS provides a distinct User Interface
(UI) to the operator’s (e.g., clinician, researcher, social worker, or an educator) laptop/PC,
which is not visible to the immersed user. Thus, beyond rendering the virtual environ-
ment that the user is immersed in it, VRESS provide a UI to the operator, which allows
them to control the VR experience.
Figure 1. Central User Interface for Selecting the Social Scenario (Left) and Describing the Require-
ments to the User (Right).
Behav. Sci. 2023, 13, x FOR PEER REVIEW 6 of 37
There are two types of UIs. There is a central UI (see Figure 1), which appears when
the VRESS application starts, that provides the operator with the available scenarios and
their difficulty levels, as well as the description of each scenario level that has to be given
to the users/trainees (i.e., the individuals with ASD) for understanding the social situation
and the social tasks that they need to perform. This central UI also appears when the
user/trainee completes a social scenario, thus, the operator requires to select and com-
mence the next social scenario that the user/trainee has to perform. While the user/trainee
is immersed in a scenario, another UI appears on the operator’s screen (see Figure 2). This
UI enables the operator to control which 3D character the user/trainee should interact with
(note that there are 1-4 interactable 3D characters per scenario, while the other characters
are just bystanders). Also, it allows the operator to manage how the interacting 3D char-
acter will respond by opting for one of the available responses. Furthermore, the opera-
tor may control the 3D character facial expressions that correspond to diverse emotional
states (e.g., neutral, angry, enthusiastic, sad, happy, confused, disappointed, or surprised),
as well as define the gaze direction of the 3D character (e.g., looking at the trainee, straight,
or down). As mentioned above, using this UI, the operator may also control the intensity
of the lighting, and the density of the NPCs’ population (i.e., how many bystanders will
populate the virtual environment) in the virtual environment. Finally, given that social
anxiety is associated with increased heart rate [77] and the atypical eye contact [78] that
are common in ASD, this UI permits the operator to monitor the user’s/trainee’s gaze (i.e.,
where the trainee is looking at, e.g., at the 3D character’s torso or eyes, mouth, and nose)
and heart rate.
Figure 2. User Interface during the Scenario for Selecting the Interacting 3D Character (Top Left)
and Controlling the Interacting 3D Character’s Responses (Middle Left) and Emotions (Bottom Left),
the Lighting’s Intensity and NPC’s Population Density (Top Right) in the Virtual Environment, as
well as for Observing User’s Gaze (Middle Right) and Heart Rate (Bottom Right).
2.1.1. Gym
In this scenario, the trainee is at the gym (see Figure 3). In the easy mode, the trainee
has to ask the gym instructor how they may operate the running treadmill. At the medium
level, the trainee has to ask another person (i.e., a co-athlete) at the gym, and then from
the gym instructor how they may operate the running treadmill. Finally, at the difficult
level, on top of asking around about how to operate the running treadmill, the trainee has
to renew their subscription to the gym and bargain the increased fee.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 7 of 37
Figure 3. The Instructor (Top), Main Area (Middle), and Reception Desk (Bottom) of the Gym.
2.1.2. Phone Store
At the phone store (see Figure 4), the trainee has to buy a smartphone that costs up
to €200. At the easy level, the examinee has just to browse the available options offered by
the customer service person. At the moderate level, while the trainee is instructed that
should buy a specific model of a brand, they have to be open to a special offer for a
smartphone with better technological specifications and a lower price. At the hard level,
the trainee has to browse all the available options and bargain based on a lower price that
they found online.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 8 of 37
Figure 4. The Customer Service (Top), Main Area (Middle), and Cashier (Bottom) of the Store
2.1.3. Cinema
At the cinema (see Figure 5), the trainee has to select a movie and buy a ticket for this
movie. At the easy level, while having a specific movie in mind, the trainee arrives late at
the cinema, and they need to browse their options (e.g., next projection or another movie)
and buy a ticket. At the moderate level, while having an appointment with a friend, the
examinee arrives late, and they need to apologize and then buy tickets for the movie. At
the difficult level, while having an appointment with a friend and another person (a friend
of the friend), they need to meet them, introduce themselves, discuss finding the way to
the cinema, and film genres that they like, and then choose a movie, and finally buy tickets
for everybody.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 9 of 37
Figure 5. The Friends (Top), Main Area (Middle), and Tickets’ Desk (Bottom) of the Cinema.
2.1.4. Classroom
In this scenario, the trainee has to attend a seminar class (see Figure 6). At the easy
level, the trainee has to attend a 3 mins lecture by the instructor on how to find reliable
information on the internet. The trainee has to respond to the instructor’s question, where
they have to share their opinion on Wikipedia. At the moderate level, on top of the afore-
mentioned interaction, the trainee has to interact with their co-students during the break
and ask them about their presentation. At the difficult level, the trainee has also to apolo-
gize to a co-student for making a mistake, which may undermine the reliability of their
co-project and presentation.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 10 of 37
Figure 6. The Co-students (Top), Main Area (Middle), and Lecturer (Bottom) of the Classroom.
2.1.5. Interview
In this scenario, the trainee has to attend a job interview at the offices of an IT com-
pany (see Figure 7). At the easy level, the trainee is required to convince the team leader
to hire them as IT assistant. At the moderate level, the trainee needs to convince both the
team leader and the HR manager to hire them as IT assistant. Finally, at the difficult level,
there is one more person in the waiting room, with whom, the trainee has to initiate a
discussion and extract information that may assist them with getting the job. Then, the
trainee has to use this information for convincing both the team leader and the HR man-
ager that they are the best candidate for this position.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 11 of 37
Figure 7. The HR Manager (Top), Main Area (Middle), and Team Leader (Bottom) of the Office.
2.2. Neuropsychological Assessment
2.2.1. Reading the Mind in the Eyes Test (adult version)– Mental state/emotion recogni-
tion
This test [35] measures participants’ mental state/emotion recognition ability. It in-
cludes 36 pictures of the eyes (only) of different people which participants are asked to
look at carefully and then choose which one of the four available options around each
picture best describes what that person may be feeling/thinking. Successful performance
requires participants to correctly attribute the emotional or mental state of each picture.
One point was awarded for each correct answer. Scores range from 0 to 36. The Reading
Behav. Sci. 2023, 13, x FOR PEER REVIEW 12 of 37
the Mind in the Eyes has been used in hundreds of studies to date and has been found to
have good test-retest reliability [79], [80].
2.2.2. Tower of London - Planning
The Tower of London [81] was used to measure participants' planning skills. This test
includes two identical wooden boards, one for the researcher and another for the partici-
pant. Each board has three wooden beams on which there are three wooden balls: one
green, one red and one blue. Participants are asked to reproduce a series of patterns using
the wooden balls only with a certain number of moves each time. Participants have to
complete 12 planning problems in total: two 2-move planning problems; two 3-move
planning problems; four 4-move planning problems and four 5-move planning problems.
To complete all planning problems successfully, participants must follow two rules.
Firstly, each planning problem must be completed in a specific number of moves and sec-
ondly, participants are allowed to remove only one ball from each beam at a time. The
number of planning problems completed successfully (adhering to the rules) was rec-
orded. One point was given for each successful completion and 0 points if participants
failed. This test has been the most commonly used measure of planning across the lifespan
[82], and presents good test-retest reliability [83].
2.2.3. Digits Recall - Verbal Working Memory
For verbal working memory, the forward and backward digit span subtests from
WAIS-III [84] were administered. Participants have to recall and repeat sequences of ran-
dom numbers back to the researcher in the same order (e.g., "Please listen carefully and then
repeat the following sequence of numbers back to me in the exact same order: 67893”). Each num-
ber sequence is read at a rate of one number per second. In the backward digit span sub-
test, participants have to repeat the sequence of numbers in the reverse order (e.g., “1236”
will be repeated as “6321”). In the case of two successfully repeated trials within each
block, the examiner proceeds with the next one. Participants were awarded 1 point for
each correct trial. Digit span has been extensively researched and is considered to be a
highly reliable and valid measure of working memory [84], [85].
2.2.4. Stroop Test - Inhibition
The Stroop test [86] is a widely used measure of word-colour interference with two
conditions. In the congruent condition, the colour of the ink and the printed name of the
colour are similar (e.g., the colour name “yellow” is printed in yellow ink) whereas, in the
incongruent condition, the colour of the ink and the printed colour word do not match.
The ability to inhibit the cognitive interference occurring when the processing of a partic-
ular characteristic of a stimulus impedes the processing of a simultaneous second feature
of the stimulus is known as the Stroop effect. This test assesses participants’ ability to
produce a contradicting response as they are asked to read the colour of the ink in which
different colour words are printed, instead of reading the colour word. The response time
(in seconds) was recorded. Stroop test has been found to present high test-retest reliability
[87].
2.3. Questionnaires
2.3.1. Demographics and IT Skills
The participants then provided their demographic data: age in years, sex, education
in years, VR experience, computing experience, and gaming experience, by responding to
a custom questionnaire. VR, computing, and gaming experience were calculated by add-
ing scores from two questions (6-item Likert Scale) for each one. The first question was
regarding the participants’ ability (e.g., 5 - highly skilled) to operate a VR system, a com-
puter, or a game respectively. Comparably, the second question was pertinent to the fre-
quency (e.g., 4 - once a week) of operating a VR system, a computer, or a game respec-
tively. This method of providing a composite score of ability and frequency has been seen
as an effective approach for evaluating the experience of an individual in using a techno-
logical medium [71], [88].
Behav. Sci. 2023, 13, x FOR PEER REVIEW 13 of 37
2.3.2. Service User Technology Acceptability Questionnaire
The Service User Technology Acceptability Questionnaire (SUTAQ) is a valid and
reliable tool for evaluating the acceptability of a technological mean in a target population,
which uses or will use this telehealth/telemedicine service [89]. The survey includes 22
questions, rated on a scale of 1 to 6, indicating the level of agreement with the statements
provided. The survey is divided into 5 sections, each containing between 3 and 9 ques-
tions. The addition of the subscores then formulates a Total Score.
2.3.3. User Experience Questionnaire
The short version of the User Experience Questionnaire (UEQ) is a valid tool for eval-
uating the subjective opinion of users towards the user experience that a technological
product facilitates [90]. The UEQ is made up of 26 items that are organized into 6 catego-
ries. Each item includes a pair of terms with opposite meanings (e.g., "efficient" and "inef-
ficient"). Participants rate each item on a 7-point Likert scale, with responses ranging from
-3 (completely agree with the negative term) to +3 (completely agree with the positive
term). Half of the items begin with the positive term and the other half begin with the
negative term, and they are presented in a randomized order. The addition of all the re-
sponses forms a total score, representing the overall user experience.
2.3.4. System Usability Scale
The System Usability Scale (SUS) is a simple and efficient tool for assessing the usa-
bility of a system [91]. It is made up of a 10-question survey that utilizes a five-point Likert
scale for participant responses, ranging from "Strongly Agree" to "Strongly Disagree." The
responses are combined to create a Total Score, which reflects the usability of the system
[92]. The SUS can be used to evaluate a wide range of products and services, such as hard-
ware, software, mobile devices, websites, and applications [92].
2.3.5. Cybersickness in Virtual Reality Questionnaire
The Cybersickness in Virtual Reality Questionnaire (CSQ-VR) is a questionnaire that
evaluates the symptoms and severity of cybersickness, which has been shown a strong
structural and construct validity [93], and a convergent validity against other cybersick-
ness measurements [94]. It assesses different sub-types of cybersickness symptoms, such
as nausea, disorientation, and oculomotor. It consists of 6 questions, which are presented
on a 7-point Likert Scale, ranging from "1 - absent feeling" to "7 - extreme feeling". The
CSQ-VR produces a Total Score, calculated by adding all the responses.
2.4. Participants
Twenty-five (25) adults (19 males/6 females) with an official diagnosis of ASD, aged
between 19 and 52 years [M(SD) = 29.96(9.77)], were recruited to participate in the present
study. Participants were all either high or moderate functioning (functionality levels 1 and
2 according to DSM-5; [1]), had fluent phrase speech, normal intelligence and held an of-
ficial ASD diagnosis based on DSM-5 criteria [1] by psychiatrists of multidisciplinary
teams with an extended clinical and research experience among adults with neurodevel-
opmental disorders (further see [95]–[97]). Exclusion criteria included the presence of
acute psychopathology requiring urgent psychiatric treatment as well as Full Scale Intel-
ligence Quotient (FSIQ) below 70. Ethical approval for the study was obtained by the hos-
pital’s ethics board and all participants provided the researchers with written informed
consent. All participants were compensated for their participation in this project.
2.5. Procedures
Every participant in this study attended first a neuropsychological session, where
their cognitive functioning was assessed. Also, three VR sessions were attended by the
participants, where they were immersed and performed the VR social scenarios per diffi-
culty level. At the end of the last VR session, they responded to the questionnaires.
2.5.1. Neuropsychological Session
Behav. Sci. 2023, 13, x FOR PEER REVIEW 14 of 37
Participants were assessed in the mental state/emotion recognition test and EF
measures across one appointment (60 mins) by a researcher. During the neuropsycholog-
ical assessment, the mental state/emotion recognition test was addressed first whereas the
order of the EF tasks was randomised across participants. Participants’ responses were
scored at the end of each session. Breaks were included when necessary.
2.5.2. VR Sessions
Participants were immersed in the VRESS by using an HTC Vive Pro Eye headset,
which substantially exceeds the recommended hardware criteria for avoiding or alleviat-
ing any cybersickness symptomatology [63]. HTC Vive Pro Eye integrates an eye-tracker
with a 120Hz refresh rate and a tracking accuracy of 0.5°-1.1°. There were three VR ses-
sions per participant, corresponding to the three difficulty levels: easy, moderate, and dif-
ficult. Five scenarios were performed in each session. The order of the five scenarios was
counterbalanced between the participants (i.e., a complete counterbalance was achieved
for every five participants). The order of the scenarios was then the same for the partici-
pant in each difficulty-level session. The three different sessions had a week gap between
them. At the beginning of every VR session, a demonstration of how to properly use and
handle the headset and controllers was provided to all participants. The participants per-
formed the social scenarios in a standing or a sitting position (see Figure 8), respectively
to the scenario’s requirement (e.g., classroom and interview required a sitting position).
At the end of the third VR session, the participants responded to the questionnaires (see
subsection 2.3 above), followed by a debriefing session, where the research aims were ex-
plained to them.
Figure 8. Participants Performing the VR Social Scenarios in a Standing or Sitting Position. Images
are blurred to prevent the identification of participants.
2.5.3 Performance Evaluation in the VR Social Scenarios
The researcher who conducted the VR sessions was also scoring the performance of
the participants. As mentioned above (see subsection 2.1), the researcher, who was the
operator of the VRESS, controlled the VR experience. During each scenario, the
Behav. Sci. 2023, 13, x FOR PEER REVIEW 15 of 37
participants had to interact with the 3D characters in the virtual environment by simply
talking to them. The operator of VRESS (i.e., the researcher), then was choosing the re-
sponse of the 3D character. The response was the most appropriate to what the participant
said to the 3D character. In case of an inappropriate social interaction by the participant
(e.g., saying something irrelevant, being silent, repeating the same thing, or making a faux
pas), the operator was providing a prompt to the participant to assist them with reacting
appropriately. If the participant was again not behaving in consistency with the social
situation’s demands, then the operator was opting for a response for the 3D character that
will continue the social scenario’s storyline. The performance of the participants was eval-
uated by two overall scores, the Task Completion Score and the Prompts Score. The task com-
pletion was calculated by the number of social tasks/interactions that were correctly per-
formed in each social scenario. The participants received 2 points when they performed
all the social tasks/interactions efficiently, 1 point when they performed half or more than
the half social interactions correctly, and 0 points when they performed appropriately less
than half social interactions. A total score for task completion was calculated per diffi-
culty level (i.e., the addition of all the points gathered in the five social scenarios of this
difficulty level). An overall task completion score was formed by adding the total scores
per difficulty level. Equally, a prompts’ score was calculated. The number of prompts,
which were given to the participants in each social scenario, was noted. The addition of
all prompts per difficulty level formulated a corresponding total score for each difficulty
level. The overall prompts’ score was formulated by the sum of all total scores of each
difficulty level.
2.6. Statistical Analyses
Descriptive statistical analysis was performed to provide an overview of the sample.
Pearson’s correlational analyses were performed to investigate the relationships between
cognitive functions, performance in VR social scenarios, and acceptability, usability, and
user experience ratings. Kendal’s Tau correlational analyses were conducted to inspect
the associations with the functionality level of individuals with ASD (i.e., 1- high-func-
tioning and 2- moderate functioning; dichotomous variable). Generalized regression anal-
yses were performed to inspect the ability of the performance variables to predict the
functionality level of individuals with ASD. Linear regression analyses were used to ex-
amine the predictors of acceptability, usability, user experience, and the number of
prompts. The R language [98] on R Studio [99] was used for performing the analyses.
The best-Normalize R package [100] was used to transform and centralize the data since the
continuous variables violated the normality assumption. The distribution of the continu-
ous data was then normal. For performing the respective analyses, the psych (correlational
analyses) [101], the ggplot2 (plots) [102], and the stats (regression analyses) [98] R packages
were used.
3. Results
3.1. Descriptive Statistics
3.1.1. Demographic Information
The descriptive statistics of the population are displayed in Table 1. The age of par-
ticipants seems to extend to the whole spectrum of early adulthood (i.e., 20-39 years) and
the early half of middle adulthood (i.e., 40-59 years) albeit the population is predomi-
nantly representative of the former. The education level of the participants indicates that
the majority had a university (undergraduate), college, or professional post-high school
education. Furthermore, participants experienced none to very mild cybersickness symp-
toms, postulating that the cybersickness did not interfere with performance or user expe-
rience metrics. The VR experience of the population was relatively low. However, the
computing experience appears to be on the upper tier of the possible scores, indicating
that the participants were experienced in using computers in their daily life. Yet, the
Behav. Sci. 2023, 13, x FOR PEER REVIEW 16 of 37
gaming experience was balanced, suggesting that the sample consisted equally of both
gamers and non-gamers.
3.1.2. Performance on Neuropsychological Tests and Social Scenarios
Regarding the performance on the social scenarios of VRESS, the descriptive statistics
for the task completion score indicate a ceiling effect (i.e., the vast majority of participants
received a high score, which is close to or exactly at the maximum possible score) and a
limited variance. On the other hand, the number of prompts required to efficaciously per-
form the social interactions in every scenario appears to have a greater range and variance,
postulating that it can be a better discriminator of the performance differences among
participants. Finally, regarding the performance on neuropsychological tests, the descrip-
tive statistics indicate an intermediate (e.g., Digit Span Backward) or upper intermediate
performance on emotional recognition and EF tests. However, the correct responses on
the Stroop test reveal a ceiling effect, while the participants’ response times show a greater
variance and range on this test.
Table 1. Descriptive Statistics of the Sample
Variables
Mean (SD)
Range
Maximum Score
Sex (Female/Male)
6/19
-
-
ASD Functionality Level (1/2)
14/11
-
-
Age
29.96 (9.76)
19 – 52
-
Education
15.88 (2.26)
12 – 20
-
Cybersickness
7.52 (2.04)
6 – 14
42
VR Experience
3.48 (1.38)
2 – 6
12
Computing Experience
8.96 (2.38)
3 – 12
12
Gaming Experience
6.68 (3.13)
2 – 12
12
Acceptability
104.28 (21.07)
49 – 127
132
User Experience
126.00 (26.33)
78 – 180
180
Usability
77.12 (12.08)
54 – 98
100
Task Completion Score
27.56 (1.82)
24 – 30
30
Prompts’ Score
11.68 (4.38)
6 – 20
-
RTMIE
25.72 (5.21)
8 – 33
36
Digit Span Forward
9.88 (2.33)
4 – 14
16
Digit Span Backward
7.52 (2.88)
2 – 13
14
Tower of London
7.88 (2.06)
3 – 11
12
Stroop – Correct Responses
48.24 (4.01)
30 – 50
50
Strop – Response Time*
65.44 (24.99)
36 – 159
-
RTMIE = Reading The Mind in the Eyes; *measured in seconds
Behav. Sci. 2023, 13, x FOR PEER REVIEW 17 of 37
Figure 9. Percentages of the Responses per Scores’ Quartile; Quartiles were defined by the maxi-
mum possible score of each questionnaire divided by four. Fourth quartile = highest possible scores,
first quartile = lowest possible scores.
3.1.3. Acceptability, User Experience, and Usability Ratings
As Table 1 and Figure 9 illustrate, the vast majority of participants reported very high
acceptability of using immersive VR training as a social/health service. 68% of the re-
sponses were at the highest quartile postulating substantially high acceptability. Notably,
92% of the participants’ responses had an overall score above the medium scores of
SUTAQ (i.e., 66), which indicates a very high rate of acceptability [103]. For user experi-
ence, the majority of responses were in the third quartile (see Figure 9), while all the re-
sponses were above the medium score. These scores postulate a high to very high user
experience [90], [104]. Comparably, 100% of the respondents gave scores in the third and
fourth quartile of the possible scores (see Figure 9), which indicates good to excellent us-
ability [92]. Likewise, considering both the mean and standard deviation of the usability
scores (see Table 1), the usability score of VRESS postulates a very good to excellent usa-
bility rating [92].
3.2. Pearson’s and Kendall's Tau Correlations
3.2.1. Demographics Correlations with Self-Reports and Performance
Demographic information of participants showed no significant associations with the
acceptability, usability, and user experience ratings (see Table 2); however, significant cor-
relations were observed with the performance on social scenarios and neuropsychological
tests (see Table 3). Specifically, the participant’s age was positively correlated with the
correct responses on the Stroop test, yet no other correlations were detected. Similarly, the
educational level of the participants revealed positive associations only with the Digit
Span scores, Forward and Backward recall, respectively. Participants' experience in using
VR systems showed no significant correlations with any of the performance metrics. How-
ever, both computing and gaming experience were substantially correlated with the per-
formance on RTMIE, postulating that individuals with higher experience in using com-
puters and/or playing video games are better at recognizing the emotional/mental states
of others. In line with this finding, computing experience was also associated with the
overall task completion score in VR social scenarios. Finally, the experience of playing
video games was negatively associated with response time in the Stroop task.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 18 of 37
Table 2. Pearson’s Correlations between Demographics and Self-Reports
Age
Education
VR XP
Computing XP
Gaming XP
Acceptability
Pearson's r
0.345
-0.044
0.071
0.213
-0.141
p-value
0.091
0.834
0.736
0.306
0.503
User Experience
Pearson's r
0.351
-0.340
-0.061
0.096
-0.183
p-value
0.085
0.096
0.771
0.647
0.382
Usability
Pearson's r
0.119
0.031
0.269
0.310
0.169
p-value
0.572
0.884
0.193
0.131
0.420
XP = Experience; *p <.05, ** p <.01, ***p <.001.
Table 3. Pearson’s Correlations between Demographics and Performance Metrics
Age
Education
VR XP
Computing XP
Gaming XP
RTMIE
Pearson's r
0.059
0.372
0.276
0.427*
0.503*
p-value
0.780
0.067
0.181
0.033
0.010
DS Forward
Pearson's r
-0.064
0.412*
0.281
0.331
0.348
p-value
0.760
0.040
0.173
0.106
0.088
DS Backward
Pearson's r
0.152
0.413*
0.108
0.195
0.237
p-value
0.469
0.040
0.607
0.349
0.255
ToL
Pearson's r
0.206
0.349
0.356
0.393
0.349
p-value
0.323
0.088
0.081
0.052
0.087
Stroop CR
Pearson's r
0.411*
-0.049
0.100
0.267
0.193
p-value
0.041
0.815
0.635
0.197
0.354
Stroop RT
Pearson's r
0.037
-0.227
-0.340
-0.380
-0.483*
p-value
0.860
0.276
0.097
0.061
0.015
Prompts
Pearson's r
-0.064
-0.096
0.347
-0.169
-0.115
p-value
0.760
0.647
0.059
0.419
0.585
Task Completion
Pearson's r
0.177
0.206
-0.392
0.468*
0.196
p-value
0.396
0.324
0.053
0.018
0.349
XP = Experience; *p <.05, ** p <.01, ***p <.001.
3.2.2. Self-Reports, Performance Metrics, and ASD Functionality Level
The functionality level of individuals with ASD revealed significant correlations only
with the usability rating, the number of prompts required to perform the social scenarios,
the performance on Digit Span Forward recall, and the response time on the Stroop test
(see Table 4). Specifically, a higher functionality level is associated with higher ratings in
the system’s perceived usability, requiring fewer prompts for performing social tasks,
having a greater verbal working memory span and faster inhibition. Moreover, substan-
tial positive associations were detected between acceptability, usability, and user experi-
ence (see Table 5), postulating that higher usability of a VR system facilitates a better user
experience and increased acceptability as a digital social/health service.
Table 4. Kendall's Tau Significant Correlations with ASD Functionality Level
Usability
Prompts
DS Forward
Stroop RT
ASD
Functionality Level
Kendall's Tau B
0.488**
-0.406*
0.416*
-0.365*
p-value
0.005
0.021
0.021
0.033
DS = Digit Span; RT = Response time; *p <.05, ** p <.01, ***p <.001.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 19 of 37
Furthermore, performance in social scenarios and performance on neuropsychologi-
cal tests were significantly correlated with self-reports on acceptability, usability, and user
experience. Requiring more prompts to perform the social interactions in the scenarios
was associated with lower acceptability and the system’s perceived usability, as well as
with less social tasks completion. Equally, a larger task completion score was correlated
to a higher system’s perceived usability. Usability also revealed positive correlations
with both Digit Span scores (i.e., Forward and Backward recall; greater working memory
span) and Tower of London (i.e., better planning ability), and a negative correlation with
the response time in the Stroop test (i.e., faster inhibition). Finally, the number of prompts
required for performing the social tasks showed substantial negative associations with the
Digit Span Forward recall and the Tower of London, postulating that greater working
memory and planning ability respectively assist with performing social interactions with-
out requiring support and/or reminders.
Table 5. Pearson’s Correlations between Self-Reports and Performance Metrics
Acceptability
User Experience
Usability
Prompts
Task Completion
Acceptability
Pearson's r
-
-
-
-
-
p-value
-
-
-
-
-
User Experience
Pearson's r
0.534**
-
-
-
-
p-value
0.006
-
-
-
-
Usability
Pearson's r
0.693***
0.486*
-
-
-
p-value
< .001
0.014
-
-
-
Prompts
Pearson's r
-0.451*
-0.200
-0.757***
-
-
p-value
0.024
0.339
< .001
-
-
Task Completion
Pearson's r
0.366
0.272
0.523**
-0.635***
-
p-value
0.072
0.189
0.007
< .001
-
RTMIE
Pearson's r
-0.076
-0.158
0.004
-0.014
0.107
p-value
0.716
0.452
0.987
0.947
0.611
DS Forward
Pearson's r
0.387
0.004
0.628***
-0.452*
0.285
p-value
0.056
0.986
< .001
0.023
0.167
DS Backward
Pearson's r
0.228
0.072
0.477**
-0.299
0.207
p-value
0.273
0.733
0.016
0.146
0.321
ToL
Pearson's r
0.354
0.001
0.685***
-0.499*
0.262
p-value
0.083
0.995
< .001
0.011
0.206
Stroop CR
Pearson's r
0.039
0.145
0.182
-0.187
0.370
p-value
0.852
0.490
0.383
0.370
0.069
Stroop RT
Pearson's r
-0.203
0.032
-0.569**
0.313
0.118
p-value
0.330
0.879
0.003
0.128
0.576
RTMIE = Reading the mind in the eyes test.; DS = Digit Span; ToL = Tower of London; CR = Correct responses; RT = Response
time; *p <.05, ** p <.01, ***p <.001.
3.3. Linear Regression and Generalized Linear Models
3.3.1. ASD Functionality Level
Three models were found for predicting the functionality level of individuals with
ASD (see Table 6). All models were significantly better than the null model. The models
showed high R2, indicating that they explain 26% - 30% of the variance of functionality
level. While all predictors showed a large β coefficient, the number of prompts had the
highest one, suggesting that requiring more prompts substantially predicts a lower
Behav. Sci. 2023, 13, x FOR PEER REVIEW 20 of 37
functionality level (see Figure 10). Similarly, reduced working memory capacity and
slower inhibition respectively predict a lower functionality level in ASD.
Table 6. Best Generalized Linear Models For Predicting ASD Functionality Level.
Predictor
χ2
p-value (χ2)
β coefficient
p-value (β)
R2
Prompts
6.22
0.01*
-1.25
0.03*
0.30
DS Forward
5.83
0.02*
1.22
0.04*
0.28
Stroop RT
5.30
0.02*
-1.09
0.04*
0.26
DS = Digit Span; RT = Response time; *p <.05, ** p <.01, ***p <.001.
Figure 10. Best Generalized Linear Model For Predicting ASD Functionality Level.
3.3.2. Performance in VR Social Scenarios
Considering that the task completion showed a ceiling effect and a reduced range of
scores and variance (see subsubsection 3.1.2), while the number of prompts did not suffer
from a ceiling effect and had a long-range and rich variance of scores, the number of
prompts was preferred as an indicator of performance on VR social scenarios. Only the
Digit Span Forward and the Tower of London were significant predictors of the number
of prompts. The model with the Digit Span Forward as a predictor was significantly better
than the null model, explained the 20% of the variance of the number of prompts, and had
a relatively large β coefficient [F(1,23) = 5.91, p = 0.02, R² = 0.20; β = -0.46, p = 0.02], sug-
gesting that the lower verbal working memory span predicts that a greater number of
prompts is required for performing efficiently the social tasks in the VR scenarios. How-
ever, the score on the Tower of London was the best predictor of the number of prompts
(see Figure 11). The model showed that the planning ability explains 25% of the variance
of the number of prompts, and had a slightly larger β coefficient, which indicates that
higher planning ability predicts that is required a smaller number of prompts for effi-
ciently interacting and completing the VR social scenarios.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 21 of 37
Figure 11. Best Linear Regression Model For Predicting Prompts’ Number.
3.3.3. Service User’s Acceptability and User Experience
The model with the number of prompts as a predictor of service users’ acceptability
was the only one that was significantly better than the null model (see Figure 12). The
model showed that the number of prompts explained 22% of the variance of acceptability
ratings, and had a relatively large β coefficient, which indicates that individuals with ASD,
who required more prompts for performing the social scenarios, provided lower accepta-
bility ratings. Comparably, the only model for predicting user experience ratings, that was
substantially better than the null model, was the one with the system’s perceived usability
rating as a predictor (see Figure 13). The model explained the 25% variance of user’s ex-
perience rating, and had a large β coefficient, postulating that the individuals who per-
ceived that the VR system has higher usability, reported a higher user experience.
Figure 12. Best Linear Regression Model For Predicting Service User’s Acceptability.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 22 of 37
Figure 13. Best Linear Regression Model For Predicting User’s Experience.
3.3.4. System’s Perceived Usability
For predicting the system’s perceived usability, four models with a single predictor
were detected, which were significantly better than the null model (see Table 7). All pre-
dictors had a large (e.g., task completion) to a very large (e.g., number of prompts) β co-
efficient, postulating that individuals with a better working memory capacity, planning
ability, and/or performance on VR social scenarios, perceived a higher system’s usability.
The overall task completion in the VR social scenarios, the Digit Span Forward recall (i.e.,
verbal working memory capacity), and the Tower of London (i.e., planning ability) ex-
plained 27%, 39%, and 47% of the variance of system’s perceived usability ratings respec-
tively. However, the best model was the one with the number of prompts as a predictor
(see Table 7 and Figure 14). The model explained 57% of the variance of the usability rat-
ings, postulating that the individuals with ASD, who required fewer prompts for effi-
ciently performing the VR social scenarios, perceived a higher system’s usability.
Table 7. Linear Regression: Best Models For Predicting System’s Perceived Usability.
Predictor
F
p-value (F)
β coefficient
p-value (β)
R2
Prompts
30.81
< 0.001***
-0.79
< 0.001***
0.57
ToL
20.37
< 0.001***
0.69
< 0.001***
0.47
DS Forward
14.98
< 0.001***
0.67
< 0.001***
0.39
Task Completion
8.64
0.01**
0.52
0.01**
0.27
DS = Digit Span; ToL = Tower of London; *p <.05, ** p <.01, ***p <.001.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 23 of 37
Figure 14. Best Linear Regression Model For Predicting System’s Perceived Usability.
4. Discussion
The present study aimed firstly to assess the usability, user experience, and accepta-
bility of an immersive VR social skills training software (i.e., VRESS) in adults with ASD.
Results showed that in terms of the system’s ratings, the VRESS software exhibited a rel-
atively high performance with positive evaluations, as average scores were close to the
high edge of the possible scores of questionnaires. Secondly, the examination of the asso-
ciations between mental state/emotion recognition, EF, functionality level of individuals
with ASD, performance in VR social scenarios, and the self-reported ratings revealed sev-
eral statistically significant relations. Furthermore, the regression models’ (single predic-
tor) analyses revealed significant predictors of several aspects. The performance in VR
social scenarios (i.e., the number of prompts required to perform efficiently the social
tasks) was the best significant predictor of ASD functionality level, as well as the ratings
of the VR system’s perceived usability, and VR social skills’ training acceptability. Inhibi-
tion speed (i.e., the response time on the Stroop task) was also a significant predictor of
ASD functionality level. Working memory (i.e., performance on the Digit Span Forward
task) was the second-best predictor of ASD functionality level and a significant predictor
of the VR system’s perceived usability. Finally, the planning ability (i.e., performance on
the Tower of London test) was the second-best predictor of the VR system’s perceived
usability and the best predictor of performance in VR social scenarios. Overall, the results
of this study offered interesting insights into the utility and feasibility of VR social skills
training in ASD, the possible implication of EF in social skills, and the importance of social
skills in ASD severity.
4.1. VR Training of Social Skills in Adults with ASD
Based on the authors of the SUTAQ, UXQ, and SUS recommendations for interpret-
ing their scores for technological interventions’ acceptability [103], quality of the user ex-
perience facilitated by the software [90], [104], and system’s usability [92], the VRESS
showed very high acceptability, user experience, and usability, as rated by the participants
with ASD. High acceptability suggests that this software [103], which also facilitates re-
mote intervention and training of social skills, will probably be preferred by adults with
ASD. Likewise, the very high usability indicates that the VR software requires a small
Behav. Sci. 2023, 13, x FOR PEER REVIEW 24 of 37
amount of effort from the user/trainee [92]. In VRESS, the user had only to speak to the
3D characters by using the microphone of the headset and navigating in the virtual envi-
ronment by pressing a single large button on the controller (either left or right). Hence,
from the trainees’ part, only a single button was required to be used, while the rest were
controlled and operated by the researcher (see subsection 2.1. for details). Finally, the high
user experience postulates that the VR software offered a highly pleasant and immersive
experience to the trainee [90], [104]. Given that providing a therapeutic process, which is
perceived as pleasant and positive by the patients, enhances the engagement and commit-
ment to therapy, as well as the effect size of the therapeutic outcomes [105], the high user
experience of VRESS suggests that it may achieve comparable positive outcomes.
Furthermore, given that there is a scarcity of robust evidence supporting the feasibil-
ity and acceptability of implementing immersive VR interventions in populations with
ASD [59], [60], [62], the results of this study provide substantial evidence that the imple-
mentation of immersive VR social skills training in ASD is feasible and acceptable by
adults with ASD. However, it should be noted that the VRESS was developed in line with
the guidelines for developing VR software for psychological sciences [71], which also lead
to VR software which meets the criteria of AACN and NAN [72]. For this reason, beyond
the high ratings in terms of acceptability, user experience, and usability, the participants
experienced minimal to absent symptoms of cybersickness, which indicates that VRESS is
a VR software that meets the health and safety criteria. Finally, since the VRESS was de-
signed specifically for individuals with ASD, the observed high ratings of acceptability,
user experience, and usability, highlight the necessity of developing VR software which
considers the highly prevalent cognitive and behavioural symptoms in ASD. However, a
downside was that the usability and acceptability of VRESS were significantly predicted
by the performance in social scenarios. This finding indicates that the negative feeling that
was experienced when the participants performed poorly, influenced them to rate VRESS
with lower scores, while the positive feeling of accomplishment influenced more positive
scores. Both error correction and errorless learning have been seen as effective in ASD
[106], however, the results of this study suggest that an errorless approach in VR social
skills training may result in even higher acceptability and perceived usability. Thus, in-
stead of receiving prompts from the operator/researcher of VRESS, which may be per-
ceived as external corrections, the VR system may provide in-game guidance to promote
an errorless completion of social tasks, while making the trainees feel that they completed
them without external assistance (e.g., with the help of the researcher). Thus, an errorless
approach should be preferred in a future iteration of VRESS.
4.2. Demographics’ Role in Cognition
4.2.1. Executive Functions
Results showed that verbal working memory was correlated with the participants’
education. The relationship between digit span scores and education is not surprising,
considering that the majority of academic tasks involve reading and lectures, which rely
heavily on verbal working memory. Working memory plays an important role in educa-
tional attainment as it is consistently found to predict academic success [107], [108]. In-
volved in the maintenance and processing of information [109], working memory signifi-
cantly associates with broad reading, comprehension and mathematical abilities [110]–
[112]. In terms of inhibition, Stroop response time was shown to significantly associate
with gaming experience ratings and usability. Findings of faster inhibition relating to
higher perceived usability scores could suggest that the ability to suppress automatic re-
sponses/ignore distractions faster allowed participants to better use and interact with the
software. The significant association between gaming experience and inhibition response
time is in line with previous evidence showing that video gamers generally demonstrate
faster reaction times and fewer errors relative to non-gamers [113]. Action video gamers
were also found to have faster visual and auditory information processing; thus they
Behav. Sci. 2023, 13, x FOR PEER REVIEW 25 of 37
presented faster response times than non-gamers [114]. Indeed, practising tasks which
rely on inhibition and working memory -such as videogames- may lead to improved per-
formance on similar tasks [115].
4.2.2. Mental State/Emotion Recognition
With regards to the mental state/emotion recognition ability, it was not found signif-
icantly related to performance in VR social scenarios but associated only with computing
and gaming experience variables. Previous evidence suggests that individuals with ASD
present difficulties in recognition of mental states/emotions (e.g., [116]–[118]), but there
are limited and mixed findings regarding its association with social competence (e.g.,
[26]). Generally, as already discussed in the Introduction, socio-cognitive abilities (such as
the recognition of mental states/emotions) do not present consistent associations with so-
cial skills in ASD. Our results showed that in adults with ASD, plausibly, it is other cog-
nitive functions (such as EF) that are more strongly implicated in the expression of social
skills. Considering though that there could be the case that no single cognitive construct
can explain the whole variance of social difficulties in ASD, further research is needed to
shed more light on this association. Future studies may also take into consideration other
emotional and relational factors that could potentially contribute to social skills. For ex-
ample, individuals with ASD may have difficulty regulating their emotions (emotional
regulation) or sharing others’ feelings (e.g., empathy) which can make it challenging for
them to respond appropriately to social cues and situations. Accordingly, low self-esteem,
negative interpersonal relationships or even low social motivation may also play a role in
shaping social skills of individuals with or without ASD. Finally, the correlations between
mental state/emotion recognition, gaming experience, and computing experience reveal
that individuals who had more experience with video games were more able to recognize
mental states/emotions in the RTMIE test. Due to their interactive nature, modern video
games offer realistic cinematics and compelling avatars with complex facial expressions
which may enhance gamers’ ability to attribute and recognize emotions and mental states
in real life contexts.
4.2. Executive Functions and Social Skills
Gollwitzer’s implementation intention pertains to the formulation of an effective plan
of action, which incorporates the associations between a cue with the intended action (e.g.,
if I encounter the X then I will do the Y) [119]. Correspondingly, the planning ability is an
executive function which requires thinking about the future and respectively organising
and prioritising future actions for achieving the desired goal(s) [15], [120]. In everyday
life, planning defines when and where an action will take place, and updates/prioritises
the plan of action based on the acquired information (e.g., I received a notification for my
overdue subscription to the gym, so, I need to renew it this evening) [120]. As a result, an
impaired planning ability is highly prevalent in clinical populations with reduced every-
day functionality [121], [122], as well as in ASD [123], [124]. In this study, planning ability
was measured by the Tower of London test, which requires individuals to generate an
explicit plan of action, including all the necessary steps, towards achieving their goal [81].
Planning ability was found to be the best predictor of performance in VR social scenarios
performance (i.e., the number of prompts). Comparable to everyday life, the VR social
scenarios required participants to plan/implement strategies of how to move their bodies,
modulate the tone of their voices, express their thoughts and perspectives, and decide to
which person and how they should interact with them for achieving the respective social
goals (e.g., choosing a film and buying tickets for it). Participants with ASD who presented
lower planning abilities experienced more difficulties in performing the required tasks in
these social scenarios. On the other hand, participants with ASD, who had better planning
ability, required fewer prompts to perform the social tasks in VRESS suggesting that their
planning ability has assisted them with performing social interactions without requiring
support. These results and interpretation are in line with the findings of studies in children
Behav. Sci. 2023, 13, x FOR PEER REVIEW 26 of 37
with fetal alcohol spectrum disorders [121] and 22q11 deletion syndrome [122], where
planning ability was a significant predictor of social skills. Note that, comparable to indi-
viduals with ASD [123], [124], individuals with these syndromes frequently have im-
paired social skills and planning abilities [121], [122]. Taken together with the results of
this and previous studies, planning skills are likely to facilitate social interactions as indi-
viduals need to plan and monitor their own and others’ actions to adjust their responses
and behaviour. Successful social interactions thus require not only the manipulation of
one’s and others’ perspectives or the processing of social cues (i.e., working memory) but
may also need planning abilities to select behavioural decisions and strategies. It should
be noted at this point that social strategies may involve conscious planning as discussed
above, but of course social behaviours may also manifest unconsciously (particularly in
everyday life) as they are often based on previous interactions or emotional experiences.
In line with a review of studies on working memory impairments in ASD, where
lower scores in verbal working memory were associated with greater problems in adap-
tive behaviour [125], in this study, verbal working memory was correlated with the per-
formance in VR social scenarios (i.e., the number of prompts). Performance in situations
such as the social interactions presented in VRESS scenarios places high demands on pro-
cessing which in turn demands increased controlled attentional processing by the execu-
tive system of working memory. Participants with ASD, who had higher digit span scores,
required fewer prompts to perform the social tasks in VRESS, suggesting that working
memory may facilitate social interactions without individuals needing support and/or re-
minders. Cognitive structures such as the recognition and understanding of others’
thoughts, beliefs and mental/emotional states during social interactions require a heavier
load of working memory [126] as individuals have to actively maintain and manipulate
personal perspectives and new, complex information of external social cues. Accordingly,
social interactions could be considered as a dual task (i.e., based on one having to balance
personal perspectives with those of the people interacting with) and for that reason re-
quire working memory mechanisms [127]. Taking all these together, it is likely that par-
ticipants with ASD who have lower working memory abilities required more prompts to
complete the social scenarios because effective social cognition and social interaction are
not possible unless one can maintain and process perspectives, social cues, and commu-
nicative strategies effectively. Nevertheless, working memory ability was not a significant
predictor of performance in VR social scenarios, suggesting that its implication in social
skills may be secondary and/or moderating. Indeed, this interpretation of our findings
agrees with the findings of a recent study, where a moderating role of working memory,
between verbal ability and social skills, was observed during early schooling years where
the acquisition of social skills is crucial [128].
4.3. Predictors of Functionality Level in ASD
Our results indicated that the ASD functionality level was related to and predicted
by inhibition and verbal working memory supporting previous evidence that has pin-
pointed a link between EF and the later severity features/symptoms in ASD [129]. Gener-
ally, impaired EF have been proposed to underlie core severity symptoms of the spectrum
[125], [130]. In line with this evidence, our results suggest that executive functions are
central to ASD and highlight their importance as a crucial domain for support and train-
ing/intervention. It should be noted at this point though that less attention has been gen-
erally given to the examination of potential cognitive factors which may be crucial for the
implementation of timely and effective interventions in ASD. Future longitudinal studies
can further clarify whether executive functions have prognostic significance in adults with
ASD.
Most importantly though, the performance in VR social scenarios (i.e., number of
prompts) was found to be the best significant predictor of ASD functionality level. Im-
paired social and communication skills are core features of ASD, which is common across
the spectrum regardless of the functionality level [131]–[134]. Although some of the best
Behav. Sci. 2023, 13, x FOR PEER REVIEW 27 of 37
predictors of ASD severity/functionality in childhood are language level [135] or IQ [136],
the severity of social and communication skills have been found to associate with [131],
[134], or differ across [132], [133], the diverse functionality levels within ASD diagnosis.
Observing the performance in VR social scenarios as a significant predictor of ASD func-
tionality level is thus aligned with the findings of the aforementioned studies. However,
it should be noted that the results of this study indicated that social skills were not just a
significant predictor, but the best predictor of functionality level in ASD. Given that the
participants of this study were either diagnosed with functionality levels 1 and 2 (i.e., high
and moderate functioning respectively) based on DSM-5 [1], this outcome postulates that
social skills may potentially serve as a central indicating factor of functionality in high and
moderate functioning adults with ASD. Notably, the social scenarios of VRESS benefit
from enhanced ecological validity, which allows the depiction of everyday functionality
[52], [72]. Thus, this outcome may be also attributed to the enhanced ecological validity of
VRESS social scenarios, which encompass the complexity and the demands of social con-
texts and situations in daily life.
4.4. Limitations and Future Studies
The findings of the present study should be interpreted considering its limitations.
The present sample of adults with ASD may not represent the more general population of
the spectrum. Participants’ average age was approximately 30 years, being mostly repre-
sentative of early adulthood (i.e., 20-39 years old). Future studies are thus required to es-
tablish whether these results can be replicated across younger children, adolescents,
and/or older adults. Furthermore, the current study was not a randomised controlled trial
(RCT) study to effectively examine the efficiency of VRESS in improving the social skills
of individuals with ASD. Future studies should hence consider conducting an RCT exper-
imental protocol, also incorporating a control group, to scrutinize the efficiency of the VR
interventions in enhancing the social skills of adults with ASD. Finally, the VRESS did not
offer an errorless learning approach, which our results showed may be beneficial for
adults with ASD. Future iterations of VRESS should facilitate an errorless learning ap-
proach for improving its efficacy. Finally, future VR studies are needed to identify more
potential prognostic markers of cognitive and social functioning in ASD.
5. Conclusions
The VRESS appears an appropriate VR social skills training system, which facilitates
a high acceptability, usability, and user experience in individuals with ASD, without in-
ducing adverse symptoms. These positive outcomes pertaining to VRESS also support the
effectiveness and feasibility of implementing VR social skills training in individuals with
ASD. Furthermore, executive functions were found to be implicated in the social skills of
adults with ASD. Finally, social skills were seen as the best indicator of the severity/func-
tionality level of adults with ASD.
Author Contributions: Conceptualization, K.P., V.M., C.S., and A.P methodology, P.K., E.K., C.S.,
and A.P; software, P.K. and C.S.; validation, P.K., E.K., P.R., V.M., K.P., C.S., and A.P; formal anal-
ysis, P.K. and P.R.; investigation, P.K. and E.K.; resources, K.P., C.S., V.M., and A.P; data curation,
P.K., E.K., and P.R.; writing—original draft preparation, P.K. and E.K.; writing—review and editing,
P.K., E.K., P.R., V.M., K.P., C.S., and A.P; visualization, P.K. and P.R.; supervision, P.R., K.P., and
A.P; project administration, P.K. and C.S.; funding acquisition, C.S.. All authors have read and
agreed to the published version of the manuscript.
Funding: The VRESS project is co-financed by the European Regional Development Fund of the
European Union and Greek national funds through the Operational Program Competitiveness, En-
trepreneurship, and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code:
T1EDK-01248).
Institutional Review Board Statement: The study was conducted in accordance with the Declara-
tion of Helsinki and approved by the Ethics Committee of Eginition Hospital (117/16.03.2020).
Behav. Sci. 2023, 13, x FOR PEER REVIEW 28 of 37
Informed Consent Statement: Informed consent was obtained from all subjects involved in the
study.
Data Availability Statement: The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to ethical approval requirements.
Acknowledgements: We would like to thank the participants for their attendance and commitment
to this research project. Also, we deeply thank Omega Technology for developing VRESS and al-
lowing us to use it in this study.
Conflicts of Interest: The authors declare no conflict of interest.
References
[1] American Psychiatric Association, Diagnostic and statistical manual of mental disorders (5th ed). Author, 2013. doi:
10.1176/appi.books.9780890425596.
[2] J. Zeidan et al., “Global prevalence of autism: A systematic review update,” Autism Research, vol. 15, no. 5, pp. 778–
790, May 2022, doi: 10.1002/aur.2696.
[3] R. Loomes, L. Hull, and W. P. L. Mandy, “What Is the Male-to-Female Ratio in Autism Spectrum Disorder? A
Systematic Review and Meta-Analysis,” Journal of the American Academy of Child & Adolescent Psychiatry, vol. 56, no. 6,
pp. 466–474, Jun. 2017, doi: 10.1016/j.jaac.2017.03.013.
[4] P. Chaste and M. Leboyer, “Autism risk factors: genes, environment, and gene-environment interactions,” Dia-
logues in Clinical Neuroscience, vol. 14, no. 3, pp. 281–292, Sep. 2012, doi: 10.31887/DCNS.2012.14.3/pchaste.
[5] C. Lord, M. Elsabbagh, G. Baird, and J. Veenstra-Vanderweele, “Autism spectrum disorder,” The Lancet, vol. 392,
no. 10146, pp. 508–520, Aug. 2018, doi: 10.1016/S0140-6736(18)31129-2.
[6] R. Keller et al., “Real-Life Social-Skills Training and Motor-Skills Training in Adults With Autism Spectrum Dis-
order: The Con-Tatto Project Walking Down the Francigena Route,” Frontiers in Psychiatry, vol. 13, 2022, [Online]. Avail-
able: https://www.frontiersin.org/articles/10.3389/fpsyt.2022.846619
[7] A. J. McVey et al., “A Replication and Extension of the PEERS® for Young Adults Social Skills Intervention: Ex-
amining Effects on Social Skills and Social Anxiety in Young Adults with Autism Spectrum Disorder,” Journal of Autism
and Developmental Disorders, vol. 46, no. 12, pp. 3739–3754, Dec. 2016, doi: 10.1007/s10803-016-2911-5.
[8] K. E. Morrison, A. E. Pinkham, D. L. Penn, S. Kelsven, K. Ludwig, and N. J. Sasson, “Distinct profiles of social skill
in adults with autism spectrum disorder and schizophrenia,” Autism Research, vol. 10, no. 5, pp. 878–887, May 2017, doi:
10.1002/aur.1734.
[9] A. B. Ratto, L. Turner-Brown, B. M. Rupp, G. B. Mesibov, and D. L. Penn, “Development of the Contextual As-
sessment of Social Skills (CASS): A Role Play Measure of Social Skill for Individuals with High-Functioning Autism,”
Journal of Autism and Developmental Disorders, vol. 41, no. 9, pp. 1277–1286, Sep. 2011, doi: 10.1007/s10803-010-1147-z.
[10] O. Shtayermman, “Peer Victimization in Adolescents and Young Adults Diagnosed with Asperger’s Syndrome:
A Link to Depressive Symptomatology, Anxiety Symptomatology and Suicidal Ideation,” Issues in Comprehensive Pedi-
atric Nursing, vol. 30, no. 3, pp. 87–107, Jan. 2007, doi: 10.1080/01460860701525089.
[11] G. Iarocci and E. Gardiner, “Executive Functions and the Developing Social Competence of Children With Autism
Spectrum Disorder,” in Executive Functions in Children’s Everyday Lives: A Handbook for Professionals in Applied Psychology,
M. J. Hoskyn, G. Iarocci, and A. R. Young, Eds. Oxford University Press, 2017, p. 0. doi: 10.1093/ac-
prof:oso/9780199980864.003.0010.
[12] S. M. Haigh, J. A. Walsh, C. A. Mazefsky, N. J. Minshew, and S. M. Eack, “Processing Speed is Impaired in Adults
with Autism Spectrum Disorder, and Relates to Social Communication Abilities,” Journal of Autism and Developmental
Disorders, vol. 48, no. 8, pp. 2653–2662, Aug. 2018, doi: 10.1007/s10803-018-3515-z.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 29 of 37
[13] E. A. Demetriou, M. M. DeMayo, and A. J. Guastella, “Executive Function in Autism Spectrum Disorder: History,
Theoretical Models, Empirical Findings, and Potential as an Endophenotype,” Frontiers in Psychiatry, vol. 10, 2019,
[Online]. Available: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00753
[14] M. D. Lezak, Neuropsychological assessment, 3rd ed. New York, NY, US: Oxford University Press, 1995, pp. xviii,
1026.
[15] A. Miyake, N. P. Friedman, M. J. Emerson, A. H. Witzki, A. Howerter, and T. D. Wager, “The Unity and Diversity
of Executive Functions and Their Contributions to Complex ‘Frontal Lobe’ Tasks: A Latent Variable Analysis,” Cognitive
Psychology, vol. 41, no. 1, pp. 49–100, Aug. 2000, doi: 10.1006/cogp.1999.0734.
[16] E. A. Demetriou et al., “Autism spectrum disorders: a meta-analysis of executive function,” Molecular Psychiatry,
vol. 23, no. 5, pp. 1198–1204, May 2018, doi: 10.1038/mp.2017.75.
[17] C. L. E. Lai et al., “Meta-analysis of neuropsychological measures of executive functioning in children and adoles-
cents with high-functioning autism spectrum disorder,” Autism Research, vol. 10, no. 5, pp. 911–939, May 2017, doi:
10.1002/aur.1723.
[18] V. C. Fong and G. Iarocci, “The Role of Executive Functioning in Predicting Social Competence in Children with
and without Autism Spectrum Disorder,” Autism Research, vol. 13, no. 11, pp. 1856–1866, Nov. 2020, doi:
10.1002/aur.2350.
[19] N. R. Riggs, L. B. Jahromi, R. P. Razza, J. E. Dillworth-Bart, and U. Mueller, “Executive function and the promotion
of social–emotional competence,” Journal of Applied Developmental Psychology, vol. 27, no. 4, pp. 300–309, Jul. 2006, doi:
10.1016/j.appdev.2006.04.002.
[20] E. S. Nilsen and A. M. Fecica, “A model of communicative perspective-taking for typical and atypical populations
of children,” Developmental Review, vol. 31, no. 1, pp. 55–78, Mar. 2011, doi: 10.1016/j.dr.2011.07.001.
[21] L. M. Freeman, J. Locke, E. Rotheram-Fuller, and D. Mandell, “Brief Report: Examining Executive and Social Func-
tioning in Elementary-Aged Children with Autism,” Journal of Autism and Developmental Disorders, vol. 47, no. 6, pp.
1890–1895, Jun. 2017, doi: 10.1007/s10803-017-3079-3.
[22] L. Kenworthy, D. O. Black, B. Harrison, A. della Rosa, and G. L. Wallace, “Are Executive Control Functions Re-
lated to Autism Symptoms in High-Functioning Children?,” Child Neuropsychology, vol. 15, no. 5, pp. 425–440, Aug. 2009,
doi: 10.1080/09297040802646983.
[23] C. E. Pugliese, L. Anthony, J. F. Strang, K. Dudley, G. L. Wallace, and L. Kenworthy, “Increasing Adaptive Behav-
ior Skill Deficits From Childhood to Adolescence in Autism Spectrum Disorder: Role of Executive Function,” Journal of
Autism and Developmental Disorders, vol. 45, no. 6, pp. 1579–1587, Jun. 2015, doi: 10.1007/s10803-014-2309-1.
[24] L. Gilotty, L. Kenworthy, L. Sirian, D. O. Black, and A. E. Wagner, “Adaptive Skills and Executive Function in
Autism Spectrum Disorders,” Child Neuropsychology, vol. 8, no. 4, pp. 241–248, Dec. 2002, doi: 10.1076/chin.8.4.241.13504.
[25] S. Baron-Cohen, A. M. Leslie, and U. Frith, “Does the autistic child have a ‘theory of mind’ ?,” Cognition, vol. 21,
no. 1, pp. 37–46, 1985, doi: https://doi.org/10.1016/0010-0277(85)90022-8.
[26] D. Alkire, K. A. McNaughton, H. A. Yarger, D. Shariq, and E. Redcay, “Theory of mind in naturalistic conversa-
tions between autistic and typically developing children and adolescents,” Autism, p. 13623613221103700, Jun. 2022, doi:
10.1177/13623613221103699.
[27] E.-C. Kouklari, T. Thompson, C. P. Monks, and S. Tsermentseli, “Hot and Cool Executive Function and its Relation
to Theory of Mind in Children with and without Autism Spectrum Disorder,” Journal of Cognition and Development, vol.
18, no. 4, pp. 399–418, Aug. 2017, doi: 10.1080/15248372.2017.1339708.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 30 of 37
[28] E.-C. Kouklari, S. Tsermentseli, and C. P. Monks, “Developmental trends of hot and cool executive function in
school-aged children with and without autism spectrum disorder: Links with theory of mind,” Development and Psycho-
pathology, vol. 31, no. 2, pp. 541–556, 2019, doi: 10.1017/S0954579418000081.
[29] A. Gentil-Gutiérrez et al., “Executive Functions in Children and Adolescents with Autism Spectrum Disorder in
Family and School Environment,” International Journal of Environmental Research and Public Health, vol. 19, no. 13, 2022,
doi: 10.3390/ijerph19137834.
[30] C. Hughes, J. Dunn, and A. White, “Trick or Treat?: Uneven Understanding of Mind and Emotion and Executive
Dysfunction in ‘Hard-to-manage’ Preschoolers,” The Journal of Child Psychology and Psychiatry and Allied Disciplines, vol.
39, no. 7, pp. 981–994, 1998, doi: 10.1111/1469-7610.00401.
[31] H. Tager-Flusberg, “Evaluating the Theory-of-Mind Hypothesis of Autism,” Curr Dir Psychol Sci, vol. 16, no. 6,
pp. 311–315, Dec. 2007, doi: 10.1111/j.1467-8721.2007.00527.x.
[32] C. A. Mazefsky et al., “The Role of Emotion Regulation in Autism Spectrum Disorder,” Journal of the American
Academy of Child & Adolescent Psychiatry, vol. 52, no. 7, pp. 679–688, Jul. 2013, doi: 10.1016/j.jaac.2013.05.006.
[33] J. N. Constantino, “Social Responsiveness Scale,” in Encyclopedia of Autism Spectrum Disorders, F. R. Volkmar, Ed.
New York, NY: Springer New York, 2013, pp. 2919–2929. doi: 10.1007/978-1-4419-1698-3_296.
[34] T. P. Bruni, “Test Review: Social Responsiveness Scale–Second Edition (SRS-2),” Journal of Psychoeducational As-
sessment, vol. 32, no. 4, pp. 365–369, Jul. 2014, doi: 10.1177/0734282913517525.
[35] S. Baron-Cohen, S. Wheelwright, J. Hill, Y. Raste, and I. Plumb, “The ‘Reading the Mind in the Eyes’ Test Revised
Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High-functioning Autism,” Journal of
Child Psychology and Psychiatry, vol. 42, no. 2, pp. 241–251, 2001, doi: https://doi.org/10.1111/1469-7610.00715.
[36] K. Papanikolaou et al., “Using the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation
Schedule-Generic for the Diagnosis of Autism Spectrum Disorders in a Greek Sample with a Wide Range of Intellectual
Abilities,” Journal of Autism and Developmental Disorders, vol. 39, no. 3, pp. 414–420, Mar. 2009, doi: 10.1007/s10803-008-
0639-6.
[37] C. Lord et al., “The Autism Diagnostic Observation Schedule—Generic: A Standard Measure of Social and Com-
munication Deficits Associated with the Spectrum of Autism,” Journal of Autism and Developmental Disorders, vol. 30, no.
3, pp. 205–223, Jun. 2000, doi: 10.1023/A:1005592401947.
[38] Y. Dandil, K. Smith, E. Kinnaird, C. Toloza, and K. Tchanturia, “Cognitive Remediation Interventions in Autism
Spectrum Condition: A Systematic Review,” Frontiers in Psychiatry, vol. 11, 2020, [Online]. Available: https://www.fron-
tiersin.org/articles/10.3389/fpsyt.2020.00722
[39] S. M. Eack et al., “Cognitive Enhancement Therapy for Adults with Autism Spectrum Disorder: Results of an 18-
month Feasibility Study,” Journal of Autism and Developmental Disorders, vol. 43, no. 12, pp. 2866–2877, Dec. 2013, doi:
10.1007/s10803-013-1834-7.
[40] M. Miyajima et al., “The effects of cognitive remediation therapy using the frontal/executive program for autism
spectrum disorder,” Int J Psychiatry Med, vol. 51, no. 3, pp. 223–235, Apr. 2016, doi: 10.1177/0091217416651254.
[41] A. Pasqualotto, N. Mazzoni, A. Bentenuto, A. Mulè, F. Benso, and P. Venuti, “Effects of Cognitive Training Pro-
grams on Executive Function in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review,”
Brain Sciences, vol. 11, no. 10, 2021, doi: 10.3390/brainsci11101280.
[42] J. Au, E. Sheehan, N. Tsai, G. J. Duncan, M. Buschkuehl, and S. M. Jaeggi, “Improving fluid intelligence with
training on working memory: a meta-analysis,” Psychonomic Bulletin & Review, vol. 22, no. 2, pp. 366–377, Apr. 2015, doi:
10.3758/s13423-014-0699-x.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 31 of 37
[43] J. Karbach and P. Verhaeghen, “Making Working Memory Work: A Meta-Analysis of Executive-Control and
Working Memory Training in Older Adults,” Psychological Science, vol. 25, no. 11, pp. 2027–2037, 2014, doi:
10.1177/0956797614548725.
[44] M. Melby-Lervåg, T. S. Redick, and C. Hulme, “Working Memory Training Does Not Improve Performance on
Measures of Intelligence or Other Measures of ‘Far Transfer’: Evidence From a Meta-Analytic Review,” Perspect Psychol
Sci, vol. 11, no. 4, pp. 512–534, Jul. 2016, doi: 10.1177/1745691616635612.
[45] E. A. Laugeson and R. Ellingsen, “Social Skills Training for Adolescents and Adults with Autism Spectrum Dis-
order,” in Adolescents and Adults with Autism Spectrum Disorders, F. R. Volkmar, B. Reichow, and J. C. McPartland, Eds.
New York, NY: Springer New York, 2014, pp. 61–85. doi: 10.1007/978-1-4939-0506-5_4.
[46] A. Gantman, S. K. Kapp, K. Orenski, and E. A. Laugeson, “Social Skills Training for Young Adults with High-
Functioning Autism Spectrum Disorders: A Randomized Controlled Pilot Study,” Journal of Autism and Developmental
Disorders, vol. 42, no. 6, pp. 1094–1103, Jun. 2012, doi: 10.1007/s10803-011-1350-6.
[47] L. Bishop-Fitzpatrick, N. J. Minshew, and S. M. Eack, “A Systematic Review of Psychosocial Interventions for
Adults with Autism Spectrum Disorders,” Journal of Autism and Developmental Disorders, vol. 43, no. 3, pp. 687–694, Mar.
2013, doi: 10.1007/s10803-012-1615-8.
[48] K. Bottema-Beutel, H. Park, and S. Y. Kim, “Commentary on Social Skills Training Curricula for Individuals with
ASD: Social Interaction, Authenticity, and Stigma,” Journal of Autism and Developmental Disorders, vol. 48, no. 3, pp. 953–
964, Mar. 2018, doi: 10.1007/s10803-017-3400-1.
[49] J. A. Gates, E. Kang, and M. D. Lerner, “Efficacy of group social skills interventions for youth with autism spec-
trum disorder: A systematic review and meta-analysis,” Clinical Psychology Review, vol. 52, pp. 164–181, Mar. 2017, doi:
10.1016/j.cpr.2017.01.006.
[50] D. Bolis, J. Balsters, N. Wenderoth, C. Becchio, and L. Schilbach, “Beyond Autism: Introducing the Dialectical
Misattunement Hypothesis and a Bayesian Account of Intersubjectivity,” Psychopathology, vol. 50, no. 6, pp. 355–372,
2017, doi: 10.1159/000484353.
[51] R. Beaumont and K. Sofronoff, “A multi-component social skills intervention for children with Asperger syn-
drome: The Junior Detective Training Program,” Journal of Child Psychology and Psychiatry, vol. 49, no. 7, pp. 743–753,
Jul. 2008, doi: 10.1111/j.1469-7610.2008.01920.x.
[52] N. Chaytor and M. Schmitter-Edgecombe, “The Ecological Validity of Neuropsychological Tests: A Review of the
Literature on Everyday Cognitive Skills,” Neuropsychology Review, vol. 13, no. 4, pp. 181–197, Dec. 2003, doi:
10.1023/B:NERV.0000009483.91468.fb.
[53] P. Kourtesis, S. Collina, L. A. A. Doumas, and S. E. MacPherson, “Validation of the Virtual Reality Everyday
Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Valid-
ity,” Journal of the International Neuropsychological Society, vol. 27, no. 2, pp. 181–196, 2021, doi:
10.1017/S1355617720000764.
[54] P. Kourtesis, S. Collina, L. A. A. Doumas, and S. E. MacPherson, “An ecologically valid examination of event-
based and time-based prospective memory using immersive virtual reality: the effects of delay and task type on every-
day prospective memory,” Memory, vol. 29, no. 4, pp. 486–506, Apr. 2021, doi: 10.1080/09658211.2021.1904996.
[55] P. Kourtesis and S. E. MacPherson, “An ecologically valid examination of event-based and time-based prospective
memory using immersive virtual reality: The influence of attention, memory, and executive function processes on real-
world prospective memory,” Neuropsychological Rehabilitation, pp. 1–26, Dec. 2021, doi: 10.1080/09602011.2021.2008983.
[56] Y.-Y. Liao, I.-H. Chen, Y.-J. Lin, Y. Chen, and W.-C. Hsu, “Effects of Virtual Reality-Based Physical and Cognitive
Training on Executive Function and Dual-Task Gait Performance in Older Adults With Mild Cognitive Impairment: A
Behav. Sci. 2023, 13, x FOR PEER REVIEW 32 of 37
Randomized Control Trial,” Frontiers in Aging Neuroscience, vol. 11, 2019, [Online]. Available: https://www.fron-
tiersin.org/articles/10.3389/fnagi.2019.00162
[57] S. Arlati et al., “A Social Virtual Reality-Based Application for the Physical and Cognitive Training of the Elderly
at Home,” Sensors, vol. 19, no. 2, 2019, doi: 10.3390/s19020261.
[58] L. Zając-Lamparska et al., “Effects of virtual reality-based cognitive training in older adults living without and
with mild dementia: a pretest–posttest design pilot study,” BMC Research Notes, vol. 12, no. 1, p. 776, Nov. 2019, doi:
10.1186/s13104-019-4810-2.
[59] R. Bradley and N. Newbutt, “Autism and virtual reality head-mounted displays: a state of the art systematic re-
view,” Journal of Enabling Technologies, vol. 12, no. 3, pp. 101–113, Jan. 2018, doi: 10.1108/JET-01-2018-0004.
[60] P. Mesa-Gresa, H. Gil-Gómez, J.-A. Lozano-Quilis, and J.-A. Gil-Gómez, “Effectiveness of Virtual Reality for Chil-
dren and Adolescents with Autism Spectrum Disorder: An Evidence-Based Systematic Review,” Sensors, vol. 18, no. 8,
2018, doi: 10.3390/s18082486.
[61] N. Stewart Rosenfield, K. Lamkin, J. Re, K. Day, L. Boyd, and E. Linstead, “A Virtual Reality System for Practicing
Conversation Skills for Children with Autism,” Multimodal Technologies and Interaction, vol. 3, no. 2, 2019, doi:
10.3390/mti3020028.
[62] N. Glaser and M. Schmidt, “Systematic Literature Review of Virtual Reality Intervention Design Patterns for In-
dividuals with Autism Spectrum Disorders,” International Journal of Human–Computer Interaction, vol. 38, no. 8, pp. 753–
788, May 2022, doi: 10.1080/10447318.2021.1970433.
[63] P. Kourtesis, S. Collina, L. A. A. Doumas, and S. E. MacPherson, “Technological Competence Is a Pre-condition
for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Re-
view and Meta-Analysis,” Frontiers in Human Neuroscience, vol. 13, 2019, doi: 10.3389/fnhum.2019.00342.
[64] P. Nolin et al., “ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children
and adolescents,” Computers in Human Behavior, vol. 59, pp. 327–333, Jun. 2016, doi: 10.1016/j.chb.2016.02.023.
[65] N. Newbutt, C. Sung, H.-J. Kuo, M. J. Leahy, C.-C. Lin, and B. Tong, “Brief Report: A Pilot Study of the Use of a
Virtual Reality Headset in Autism Populations,” Journal of Autism and Developmental Disorders, vol. 46, no. 9, pp. 3166–
3176, Sep. 2016, doi: 10.1007/s10803-016-2830-5.
[66] P. Ghanouni, T. Jarus, J. G. Zwicker, J. Lucyshyn, K. Mow, and A. Ledingham, “Social Stories for Children with
Autism Spectrum Disorder: Validating the Content of a Virtual Reality Program,” Journal of Autism and Developmental
Disorders, vol. 49, no. 2, pp. 660–668, Feb. 2019, doi: 10.1007/s10803-018-3737-0.
[67] J. F. Herrero and G. Lorenzo, “An immersive virtual reality educational intervention on people with autism spec-
trum disorders (ASD) for the development of communication skills and problem solving,” Education and Information
Technologies, vol. 25, no. 3, pp. 1689–1722, May 2020, doi: 10.1007/s10639-019-10050-0.
[68] H. H. S. Ip et al., “Enhance emotional and social adaptation skills for children with autism spectrum disorder: A
virtual reality enabled approach,” Computers & Education, vol. 117, pp. 1–15, Feb. 2018, doi:
10.1016/j.compedu.2017.09.010.
[69] F. Ke, J. Moon, and Z. Sokolikj, “Virtual Reality–Based Social Skills Training for Children With Autism Spectrum
Disorder,” J Spec Educ Technol, vol. 37, no. 1, pp. 49–62, Mar. 2022, doi: 10.1177/0162643420945603.
[70] S. N. V. Yuan and H. H. S. Ip, “Using virtual reality to train emotional and social skills in children with autism
spectrum disorder,” London Journal of Primary Care, vol. 10, no. 4, pp. 110–112, Jul. 2018, doi:
10.1080/17571472.2018.1483000.
[71] P. Kourtesis, D. Korre, S. Collina, L. A. A. Doumas, and S. E. MacPherson, “Guidelines for the Development of
Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual
Behav. Sci. 2023, 13, x FOR PEER REVIEW 33 of 37
Reality Everyday Assessment Lab (VR-EAL), a Neuropsychological Test Battery in Immersive Virtual Reality,” Frontiers
in Computer Science, vol. 1, 2020, doi: 10.3389/fcomp.2019.00012.
[72] P. Kourtesis and S. E. MacPherson, “How immersive virtual reality methods may meet the criteria of the National
Academy of Neuropsychology and American Academy of Clinical Neuropsychology: A software review of the Virtual
Reality Everyday Assessment Lab (VR-EAL),” Computers in Human Behavior Reports, vol. 4, p. 100151, 2021, doi:
https://doi.org/10.1016/j.chbr.2021.100151.
[73] C. A. Gray and J. D. Garand, “Social Stories: Improving Responses of Students with Autism with Accurate Social
Information,” Focus on Autistic Behavior, vol. 8, no. 1, pp. 1–10, Apr. 1993, doi: 10.1177/108835769300800101.
[74] R. Samuels and J. Stansfield, “The effectiveness of social storiesTM to develop social interactions with adults with
characteristics of autism spectrum disorder,” British Journal of Learning Disabilities, vol. 40, no. 4, pp. 272–285, Dec. 2012,
doi: 10.1111/j.1468-3156.2011.00706.x.
[75] A. Bakroon and V. Lakshminarayanan, “Visual function in autism spectrum disorders: a critical review,” Clinical
and Experimental Optometry, vol. 99, no. 4, pp. 297–308, Jul. 2016, doi: 10.1111/cxo.12383.
[76] G. Joshi et al., “Psychiatric Comorbidity and Functioning in a Clinically Referred Population of Adults with Au-
tism Spectrum Disorders: A Comparative Study,” Journal of Autism and Developmental Disorders, vol. 43, no. 6, pp. 1314–
1325, Jun. 2013, doi: 10.1007/s10803-012-1679-5.
[77] L. Rösler, S. Göhring, M. Strunz, and M. Gamer, “Social anxiety is associated with heart rate but not gaze behavior
in a real social interaction,” Journal of Behavior Therapy and Experimental Psychiatry, vol. 70, p. 101600, Mar. 2021, doi:
10.1016/j.jbtep.2020.101600.
[78] A. Senju and M. H. Johnson, “Atypical eye contact in autism: Models, mechanisms and development,” Neurosci-
ence & Biobehavioral Reviews, vol. 33, no. 8, pp. 1204–1214, Sep. 2009, doi: 10.1016/j.neubiorev.2009.06.001.
[79] E. G. Fernández-Abascal, R. Cabello, P. Fernández-Berrocal, and S. Baron-Cohen, “Test-retest reliability of the
‘Reading the Mind in the Eyes’ test: a one-year follow-up study,” Molecular Autism, vol. 4, no. 1, p. 33, Sep. 2013, doi:
10.1186/2040-2392-4-33.
[80] M. Vellante et al., “The ‘Reading the Mind in the Eyes’ test: Systematic review of psychometric properties and a
validation study in Italy,” Cognitive Neuropsychiatry, vol. 18, no. 4, pp. 326–354, Jul. 2013, doi:
10.1080/13546805.2012.721728.
[81] T. Shallice, D. E. Broadbent, and L. Weiskrantz, “Specific impairments of planning,” Philosophical Transactions of
the Royal Society of London. B, Biological Sciences, vol. 298, no. 1089, pp. 199–209, Jan. 1997, doi: 10.1098/rstb.1982.0082.
[82] Y.-K. Chang, C.-L. Tsai, T.-M. Hung, E. C. So, F.-T. Chen, and J. L. Etnier, “Effects of Acute Exercise on Executive
Function: A Study With a Tower of London Task,” Journal of Sport and Exercise Psychology, vol. 33, no. 6, pp. 847–865,
Dec. 2011, doi: 10.1123/jsep.33.6.847.
[83] L. Köstering, K. Nitschke, F. K. Schumacher, C. Weiller, and C. P. Kaller, “Test–retest reliability of the Tower of
London Planning Task (TOL-F).,” Psychological Assessment, vol. 27, pp. 925–931, 2015, doi: 10.1037/pas0000097.
[84] D. Wechsler, Wechsler Adult Intelligence Scale-Third Edition Manual. The Psychological Corporation, 1997.
[85] D. J. Siegel, N. J. Minshew, and G. Goldstein, “Wechsler IQ profiles in diagnosis of high-functioning autism,”
Journal of Autism and Developmental Disorders, vol. 26, no. 4, pp. 389–406, Aug. 1996, doi: 10.1007/BF02172825.
[86] J. R. Stroop, “Studies of interference in serial verbal reactions.,” Journal of Experimental Psychology, vol. 18, no. 6, p.
643, 1935, doi: 10.1037/h0054651.
[87] G. P. Strauss, D. N. Allen, M. L. Jorgensen, and S. L. Cramer, “Test-Retest Reliability of Standard and Emotional
Stroop Tasks: An Investigation of Color-Word and Picture-Word Versions,” Assessment, vol. 12, no. 3, pp. 330–337, Sep.
2005, doi: 10.1177/1073191105276375.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 34 of 37
[88] S. P. Smith and S. Du’Mont, “Measuring the effect of gaming experience on virtual environment navigation tasks,”
in 2009 IEEE Symposium on 3D User Interfaces, Mar. 2009, pp. 3–10. doi: 10.1109/3DUI.2009.4811198.
[89] S. P. Hirani et al., “Quantifying beliefs regarding telehealth: Development of the Whole Systems Demonstrator
Service User Technology Acceptability Questionnaire,” Journal of Telemedicine and Telecare, vol. 23, no. 4, pp. 460–469,
2017, doi: 10.1177/1357633X16649531.
[90] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Design and evaluation of a short version of the user experience
questionnaire (UEQ-S),” International Journal of Interactive Multimedia and Artificial Intelligence, 4 (6), 103-108., 2017, doi:
10.9781/ijimai.2017.445.
[91] J. Brooke, “SUS: A Retrospective,” J. Usability Studies, vol. 8, no. 2, pp. 29–40, Feb. 2013.
[92] A. Bangor, P. Kortum, and J. Miller, “Determining What Individual SUS Scores Mean: Adding an Adjective Rating
Scale,” J. Usability Studies, vol. 4, no. 3, pp. 114–123, May 2009.
[93] P. Kourtesis, S. Collina, L. A. A. Doumas, and S. E. MacPherson, “Validation of the Virtual Reality Neuroscience
Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions Without the Presence of Pertinent Adverse
Symptomatology,” Frontiers in Human Neuroscience, vol. 13, 2019, doi: 10.3389/fnhum.2019.00417.
[94] P. Kourtesis, J. Linnell, R. Amir, F. Argelaguet, and S. E. MacPherson, “Cybersickness in Virtual Reality Question-
naire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ,” Virtual Worlds, vol. 2, no. 1, pp. 16–35, 2023,
doi: 10.3390/virtualworlds2010002.
[95] A. Pehlivanidis et al., “Lifetime co-occurring psychiatric disorders in newly diagnosed adults with attention deficit
hyperactivity disorder (ADHD) or/and autism spectrum disorder (ASD),” BMC Psychiatry, vol. 20, no. 1, p. 423, Aug.
2020, doi: 10.1186/s12888-020-02828-1.
[96] A. Papaioannou et al., “Differences in Performance of ASD and ADHD Subjects Facing Cognitive Loads in an
Innovative Reasoning Experiment,” Brain Sciences, vol. 11, no. 11, 2021, doi: 10.3390/brainsci11111531.
[97] V. Mantas, A. Pehlivanidis, K. Papanikolaou, V. Kotoula, and C. Papageorgiou, “Strategic decision making and
prediction differences in autism,” PeerJ, vol. 10, p. e13328, 2022, doi: 10.7717/peerj.13328.
[98] R Core Team, R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical
Computing, 2022. [Online]. Available: https://www.R-project.org/
[99] RStudio Team, RStudio: Integrated Development Environment for R. Boston, MA: RStudio, PBC, 2022. [Online]. Avail-
able: http://www.rstudio.com/
[100] R. A. Peterson and J. E. Cavanaugh, “Ordered quantile normalization: a semiparametric transformation built for
the cross-validation era,” Journal of Applied Statistics, vol. 47, no. 13–15, pp. 2312–2327, 2020, doi:
10.1080/02664763.2019.1630372.
[101] W. Revelle, psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, Illinois: Northwest-
ern University, 2022. [Online]. Available: https://CRAN.R-project.org/package=psych
[102] H. Wickham, ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016. [Online]. Available:
https://ggplot2.tidyverse.org
[103] A. Torbjørnsen, M. C. Småstuen, A. K. Jenum, E. Årsand, and L. Ribu, “The Service User Technology Acceptability
Questionnaire: Psychometric Evaluation of the Norwegian Version,” JMIR Hum Factors, vol. 5, no. 4, p. e10255, Dec.
2018, doi: 10.2196/10255.
[104] M. Schrepp, A. Hinderks, and J. Thomaschewski, “Construction of a Benchmark for the User Experience Ques-
tionnaire (UEQ),” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, pp. 40–44, Jun. 2017, doi:
10.9781/ijimai.2017.445.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 35 of 37
[105] A. H. Lequerica and K. Kortte, “Therapeutic Engagement: A Proposed Model of Engagement in Medical Rehabil-
itation,” American Journal of Physical Medicine & Rehabilitation, vol. 89, no. 5, 2010, [Online]. Available: https://jour-
nals.lww.com/ajpmr/Fulltext/2010/05000/Therapeutic_Engagement__A_Proposed_Model_of.10.aspx
[106] J. B. Leaf, J. H. Cihon, J. L. Ferguson, C. M. Milne, R. Leaf, and J. McEachin, “Comparing Error Correction to
Errorless Learning: A Randomized Clinical Trial,” The Analysis of Verbal Behavior, vol. 36, no. 1, pp. 1–20, Jun. 2020, doi:
10.1007/s40616-019-00124-y.
[107] T. Packiam. Alloway, S. Elizabeth. Gathercole, A.-Marie. Adams, Catherine. Willis, Rachel. Eaglen, and Emily.
Lamont, “Working memory and phonological awareness as predictors of progress towards early learning goals at
school entry,” British Journal of Developmental Psychology, vol. 23, no. 3, pp. 417–426, Sep. 2005, doi:
10.1348/026151005X26804.
[108] S. E. Gathercole, S. J. Pickering, C. Knight, and Z. Stegmann, “Working memory skills and educational attainment:
evidence from national curriculum assessments at 7 and 14 years of age,” Applied Cognitive Psychology, vol. 18, no. 1, pp.
1–16, Jan. 2004, doi: 10.1002/acp.934.
[109] N. Unsworth and R. W. Engle, “On the division of short-term and working memory: An examination of simple
and complex span and their relation to higher order abilities.,” Psychological Bulletin, vol. 133, pp. 1038–1066, 2007, doi:
10.1037/0033-2909.133.6.1038.
[110] B. Carretti, E. Borella, C. Cornoldi, and R. De Beni, “Role of working memory in explaining the performance of
individuals with specific reading comprehension difficulties: A meta-analysis,” Learning and Individual Differences, vol.
19, no. 2, pp. 246–251, Jun. 2009, doi: 10.1016/j.lindif.2008.10.002.
[111] P. Peng, J. Namkung, M. Barnes, and C. Sun, “A meta-analysis of mathematics and working memory: Moderating
effects of working memory domain, type of mathematics skill, and sample characteristics.,” Journal of Educational Psy-
chology, vol. 108, pp. 455–473, 2016, doi: 10.1037/edu0000079.
[112] P. Peng et al., “A meta-analysis on the relation between reading and working memory.,” Psychological Bulletin, vol.
144, pp. 48–76, 2018, doi: 10.1037/bul0000124.
[113] C. Shawn Green, M. A. Sugarman, K. Medford, E. Klobusicky, and D. Bavelier, “The effect of action video game
experience on task-switching,” Computers in Human Behavior, vol. 28, no. 3, pp. 984–994, May 2012, doi:
10.1016/j.chb.2011.12.020.
[114] C. S. Green, A. Pouget, and D. Bavelier, “Improved Probabilistic Inference as a General Learning Mechanism with
Action Video Games,” Current Biology, vol. 20, no. 17, pp. 1573–1579, Sep. 2010, doi: 10.1016/j.cub.2010.07.040.
[115] L. Stockdale, R. G. Morrison, R. Palumbo, J. Garbarino, and R. L. Silton, “Cool, callous and in control: superior
inhibitory control in frequent players of video games with violent content,” Social Cognitive and Affective Neuroscience,
vol. 12, no. 12, pp. 1869–1880, Dec. 2017, doi: 10.1093/scan/nsx115.
[116] O. Golan and S. Baron-Cohen, “Systemizing empathy: Teaching adults with Asperger syndrome or high-func-
tioning autism to recognize complex emotions using interactive multimedia,” Development and Psychopathology, vol. 18,
no. 2, pp. 591–617, 2006, doi: 10.1017/S0954579406060305.
[117] M. V. Lombardo, J. L. Barnes, S. J. Wheelwright, and S. Baron-Cohen, “Self-Referential Cognition and Empathy in
Autism,” PLOS ONE, vol. 2, no. 9, p. e883, Sep. 2007, doi: 10.1371/journal.pone.0000883.
[118] M. Losh et al., “Neuropsychological Profile of Autism and the Broad Autism Phenotype,” Archives of General Psy-
chiatry, vol. 66, no. 5, pp. 518–526, May 2009, doi: 10.1001/archgenpsychiatry.2009.34.
[119] P. Gollwitzer, “Implementation Intentions: Strong Effects of Simple Plans,” American Psychologist, vol. 54, pp. 493–
503, Jul. 1999, doi: 10.1037/0003-066X.54.7.493.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 36 of 37
[120] R. Morris and G. Ward, The cognitive psychology of planning. Psychology Press, 2004. [Online]. Available:
https://doi.org/10.4324/9780203493564
[121] A. M. Schonfeld, B. Paley, F. Frankel, and M. J. O’Connor, “Executive Functioning Predicts Social Skills Following
Prenatal Alcohol Exposure,” Child Neuropsychology, vol. 12, no. 6, pp. 439–452, Dec. 2006, doi:
10.1080/09297040600611338.
[122] K. Kiley-Brabeck and C. Sobin, “Social Skills and Executive Function Deficits in Children With the 22q11 Deletion
Syndrome,” Applied Neuropsychology, vol. 13, no. 4, pp. 258–268, Dec. 2006, doi: 10.1207/s15324826an1304_7.
[123] J. Sinzig, D. Morsch, N. Bruning, M. H. Schmidt, and G. Lehmkuhl, “Inhibition, flexibility, working memory and
planning in autism spectrum disorders with and without comorbid ADHD-symptoms,” Child and Adolescent Psychiatry
and Mental Health, vol. 2, no. 1, p. 4, Jan. 2008, doi: 10.1186/1753-2000-2-4.
[124] L. M. E. Olde Dubbelink and H. M. Geurts, “Planning Skills in Autism Spectrum Disorder Across the Lifespan: A
Meta-analysis and Meta-regression,” Journal of Autism and Developmental Disorders, vol. 47, no. 4, pp. 1148–1165, Apr.
2017, doi: 10.1007/s10803-016-3013-0.
[125] S. Kercood, J. A. Grskovic, D. Banda, and J. Begeske, “Working memory and autism: A review of literature,”
Research in Autism Spectrum Disorders, vol. 8, no. 10, pp. 1316–1332, Oct. 2014, doi: 10.1016/j.rasd.2014.06.011.
[126] M. Brüne, “Theory of mind and the role of IQ in chronic disorganized schizophrenia,” Schizophrenia Research, vol.
60, no. 1, pp. 57–64, Mar. 2003, doi: 10.1016/S0920-9964(02)00162-7.
[127] A. Baddeley, S. Della Sala, C. Papagno, and H. Spinnler, “Dual-task performance in dysexecutive and nondysex-
ecutive patients with a frontal lesion.,” Neuropsychology, vol. 11, pp. 187–194, 1997, doi: 10.1037/0894-4105.11.2.187.
[128] Y. Tang, P. L. Harris, H. Zou, J. Wang, and Z. Zhang, “The relationship between emotion understanding and social
skills in preschoolers: The mediating role of verbal ability and the moderating role of working memory,” European Jour-
nal of Developmental Psychology, vol. 18, no. 4, pp. 593–609, Jul. 2021, doi: 10.1080/17405629.2020.1854217.
[129] L. Kenny, S. J. Cribb, and E. Pellicano, “Childhood Executive Function Predicts Later Autistic Features and Adap-
tive Behavior in Young Autistic People: a 12-Year Prospective Study,” Journal of Abnormal Child Psychology, vol. 47, no.
6, pp. 1089–1099, Jun. 2019, doi: 10.1007/s10802-018-0493-8.
[130] L. Van Eylen, B. Boets, J. Steyaert, J. Wagemans, and I. Noens, “Executive functioning in autism spectrum disor-
ders: influence of task and sample characteristics and relation to symptom severity,” European Child & Adolescent Psy-
chiatry, vol. 24, no. 11, pp. 1399–1417, Nov. 2015, doi: 10.1007/s00787-015-0689-1.
[131] L. Kjellmer, Å. Hedvall, E. Fernell, C. Gillberg, and F. Norrelgen, “Language and communication skills in pre-
school children with autism spectrum disorders: Contribution of cognition, severity of autism symptoms, and adaptive
functioning to the variability,” Research in Developmental Disabilities, vol. 33, no. 1, pp. 172–180, Jan. 2012, doi:
10.1016/j.ridd.2011.09.003.
[132] S. Williams White, K. Keonig, and L. Scahill, “Social Skills Development in Children with Autism Spectrum Dis-
orders: A Review of the Intervention Research,” Journal of Autism and Developmental Disorders, vol. 37, no. 10, pp. 1858–
1868, Nov. 2007, doi: 10.1007/s10803-006-0320-x.
[133] A. S. Carter, N. O. Davis, A. Klin, and F. R. Volkmar, “Social Development in Autism.,” Handbook of autism and
pervasive developmental disorders: Diagnosis, development, neurobiology, and behavior, Vol. 1, 3rd ed., pp. 312–334, 2005.
[134] C. Hilton, K. Graver, and P. LaVesser, “Relationship between social competence and sensory processing in chil-
dren with high functioning autism spectrum disorders,” Research in Autism Spectrum Disorders, vol. 1, no. 2, pp. 164–173,
Apr. 2007, doi: 10.1016/j.rasd.2006.10.002.
Behav. Sci. 2023, 13, x FOR PEER REVIEW 37 of 37
[135] R. Luyster, K. Lopez, and C. Lord, “Characterizing communicative development in children referred for Autism
Spectrum Disorders using the MacArthur-Bates Communicative Development Inventory (CDI),” Journal of Child Lan-
guage, vol. 34, no. 3, pp. 623–654, 2007, doi: 10.1017/S0305000907008094.
[136] E. Waizbard-Bartov et al., “Trajectories of Autism Symptom Severity Change During Early Childhood,” Journal of
Autism and Developmental Disorders, vol. 51, no. 1, pp. 227–242, Jan. 2021, doi: 10.1007/s10803-020-04526-z.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual au-
thor(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
