U.S. Government work not protected by U.S. copyright
Doctoral Colloquium Paper—Enhancing Brain
Plasticity and Cognition Utilizing Immersive
Technology and Virtual Reality Contexts for
Cassondra M. Eng
Carnegie Mellon University
Dominic M. Calkosz
Carnegie Mellon University
Sebastian Y. Yang
Carnegie Mellon University
Nathan C. Williams
Logic and Computation
Carnegie Mellon University
Erik D. Thiessen
Carnegie Mellon University
Anna V. Fisher
Carnegie Mellon University
Abstract—This work-in-progress paper examines the effects of
immersive virtual experiences on cognition and neuroplasticity.
Study 1 examined the separate and combined effects of physically-
active and cognitively-demanding immersive gameplay on
executive function and associated neural substrates. Results
indicated that cognition and neuroplasticity—the building of new
brain connections—increase when learning novel skills via active
gameplay. Study 2 devised an experimental design to reproduce
Study 1 in virtual reality to examine whether the findings of
enhanced cognition and neuroplasticity generalize across virtual
contexts and development. Incorporating neuroimaging measures
into virtual experiences may identify the underlying mechanisms
for behavioral changes in learning.
Index terms—executive function, neuroplasticity, exergames
Exergames are video games that promote active, whole-body
experiences by requiring participants to interact with the virtual
environment through body movements. As the use of digital
games in childhood vastly increases, effective designs for
enriching cognitive development raise great interest. Despite the
practical applications of exergames in youth, carefully
controlled experimental studies are needed to clarify the unique
efficacy of exergames on executive function (EF) in youth. This
paper 1) assesses the efficacy of exergames on EF by comparing
the effects of exergame (combined cognitive + exercise)
immersion to cognitive immersion, exercise immersion, and a
control group in 4- to 5-year old children in Study 1 2) examines
the effects of exergame-induced changes on neuroplasticity with
noninvasive neuroimaging and 3) proposes an experimental
design to reproduce Study 1 in virtual reality (VR) to examine
whether the findings generalize across contexts and ages.
EF is an umbrella term for cognitive processes that subserve
goal-directed behavior that enable individuals to plan, focus
attention, remember instructions, and multitask successfully .
Environmental enrichment is critical in shaping the development
of EF by providing increased quantity and quality of multimodal
input to the prefrontal cortex (PFC), the prominent brain area
linked to EF . The combination of exercise and cognitive
stimulation through exergame play could potentially foster EF
by immersing children in an enriching environment that
promotes increased quantity (kinesthetic, auditory, visual) and
quality (increasing challenge, individual adaptability, matching
feedback) of multimodal input.
Fostering EF development is important because EF predicts
math and reading competence throughout all school years, and
remains critical for success throughout life in occupations,
interpersonal relationships, and mental and physical health .
Children with worse EF at ages 3 to 11 exhibit poorer health,
lower academic achievement, earn less, and commit more
crimes 30 years later than those with better EF as children
(controlling for IQ, sex, and socioeconomic status); therefore,
providing contexts that foster EF early on in development could
contribute to closing the achievement gap .
Exergames are relatively inexpensive, require a small
amount of space, and are perceived as enjoyable by children .
Research on exergames and EF focus primarily on elderly
populations with findings that exergames improve EF over and
above single component cognitive or exercise immersion alone,
suggesting that the combination of exercise and cognitive
immersion through exergame play affects neuroplasticity
additively . While there are well-documented beneficial
effects of exergames on EF in elderly populations , research
on the effects of exergames on EF in youth is scarce .
A concern of EF training more generally is how the skills
transfer to real-world contexts and the mechanisms underlying
The research reported here was supported in part by a training grant from the
Institute of Education Sciences (R305B150008). Opinions expressed do not
represent the views of the U.S. Department of Education.
successful learning paradigms. Study 1 aims to fill these gaps by
(1) including adult ratings of children’s EF to assess if skills
children learn from training transfer to EF-related behavior in a
real-world educational context (2) elucidating the effects of
exergame-induced changes on neuroplasticity with functional
near-infrared spectroscopy (fNIRS): a neuroimaging technique
that uses low-levels of light to measure changes in cerebral blood
volume and oxygenation. fNIRS is ideal for imaging studies
with children because it is noninvasive and does not require
participants to remain immobile in a confined environment like
traditional neuroimaging methods do. It was hypothesized that
exergame training would improve children’s performance on
neurocognitive EF tasks, EF-related behaviors in the classroom,
and neuroplasticity in the PFC compared to the control groups.
The second study proposes a work-in-progress experimental
design to reproduce Study 1 in VR to examine whether the
findings generalize across contexts and ages.
II. STUDY 1
Fifty children were recruited from a pre-primary school in a
Northeastern city in the United States ages 4 to 5 (M=
4.89, SD=7 months). The experimental protocol was approved
by the University Institutional Review Board. Signed consent
was obtained from the parents of participants. This study utilized
a randomized block design by classroom, age, and sex. Then,
within each block, subjects were randomly assigned to one of
the four conditions: Exergame (n=14), Exercise (n=12),
Cognitive (n=11), or Control (n=13).
A. Procedure and Measures
A between-subjects design was implemented with two
testing phases: pretest and posttest. Neuroimaging utilizing
fNIRS, performance on EF tasks, and teacher ratings of EF were
assessed at pretest. The day after pretest, children in the
Exergame, Cognitive, and Exercise Conditions trained for 2
consecutive days, 20 mins/day, for 40 mins total. Participants
returned to the laboratory the following day after training and
neuroimaging and EF assessments were reassessed at posttest.
Teachers and research assistants who administered the pre- and
posttest assessments were condition-blind. Children in the
Control Condition continued their typical activities and just
participated in the pre- and posttest assessments.
The Flanker Task presented 5 fish in a row facing either right
or left . The task goal was to make a response in accordance
with the direction of the middle fish while ignoring the
surrounding fish, which either faced the opposite or same
direction of the target fish for incongruent and congruent trials,
respectively. This task involves multiple EF skills: following
rules, paying attention, and inhibiting distraction from the
flanking fish. Children completed 50 trials, with the primary
outcome measure being mean accuracy on the incongruent trials.
The Day-Night Task presented children with cards of either
a moon or a sun . A block of 16 congruent trials was shown
first in which children were instructed to say “day” or “night”
when shown a sun or moon card, respectively. Next, a block of
16 incongruent trials were shown in which children were
instructed to do the opposite and say “day” for moon cards and
“night” for sun cards. This task also involves multiple EF skills:
following rules, task switching, and inhibiting prepotent
responses. The main outcome measure was accuracy for the
The Behavior Rating Inventory of Executive Function-
Preschool Version (BRIEF) is a clinical questionnaire that
assessed EF as manifested in the everyday behavior of
preschool-aged children . Teachers completed items
assessing Inhibitory Control, Flexibility, Working Memory, and
Planning with response options ranging from “never” to “often”
as a problem for the child (e.g., “Is impulsive”).
Resting-state functional connectivity—correlated activity in
the absence of a task—has been found to predict cognitive
performance, educational attainment, and even household
income . Brain connectivity was collected with a continuous
wave real-time TechEn, Inc. fNIRS system. There were 4 light
sources and 8 detectors positioned on the PFC, with 45 channel-
pair connections. Inscapes , a movie paradigm that features
abstract shapes without a social narrative and a validated
measure of resting-state functional connectivity for children,
was administered at pre- and posttest.
B. Immersive Training Sessions
Each condition was programmed in Unity Technologies: a
cross-platform game engine that permitted customization to
carefully control the features of the experimental conditions. The
immersive conditions were “gamified” versions of the Flanker
Task, meaning game interface design patterns and features were
implemented, but the main goal of the task remained unchanged
. Each condition was projected onto a wall with a non-slip
game step mat (144 x 96 x 48 in) with one left arrow and one
right arrow for children to interact with the game. Each condition
included evidence-based approaches that enhance learning: a
narrative, player feedback, the anticipation of a competitor, and
a computational algorithm that provided incremental challenge
by continuously adapting the difficulty level based on
performance . For each correct trial, the difficulty advanced
by decreasing the allotted response time by 500 ms. For each
incorrect trial (i.e., children step in the wrong direction or take
too long), difficulty decreased by increasing the allotted
response time by 500 ms. Training was administered to
participants in the same room each day, with a hypothesis-blind
experimenter present during the entire session.
The Exergame Condition (combined exercise + cognitive
immersion) was identical to the Flanker Task, but instead of
pressing left and right arrows on a computer keyboard, children
responded by stepping left or right on the physical game mat’s
arrows depending on the direction that the central target was
facing. The Cognitive Condition (cognitive immersion) was
identical to the Exergame, except participants sat on the mat and
pressed left or right with their hands, rather than stepping. The
Exercise Condition (exercise immersion) was identical to the
Exergame, except that the central target was not surrounded by
Preliminary analyses examined condition differences of age,
sex, and pretest assessments. Neither age nor sex differed
significantly between the groups (ps>0.11). At pretest, there
were no significant differences between groups on the Flanker
Task, the Day-Night Task, BRIEF, or brain connectivity
(ps>0.41). Neuroimaging analyses were carried out using the
NIRS Brain AnalyzIR toolbox . Pearson correlation
coefficients were computed between all 45 channel-pairs at pre-
and posttest. A Fisher's r-to-Z transformation was then applied
to normalize the variance of the correlation values. To account
for variability in the hemodynamic response function, the
temporal and dispersion derivatives were estimated and
discarded. The estimated coefficients for all conditions were
submitted to a robust weighted mixed effects model, with time
modeled as a fixed effect and subject as a random effect. To
assess the changes in brain connectivity over time between
conditions, a t-contrast of ‘Posttest’ versus ‘Pretest’ was carried
out by group, and the false discovery rate (FDR) correction was
used to control for multiple comparisons via multiple channel-
pairs. Out of 45 channel-pairs, functional connectivity
significantly changed from pre- to posttest in 26 for the
Exergame group, 4 in the Exercise group, 3 in the Cognitive
group, and 1 in the Control group (Fig. 1).
Fig. 1. Warmer colors indicate greater changes in connectivity strength, while
no connections indicate no significant changes from pre- to posttest.
One-way ANOVAs were conducted with difference scores
from pre- to posttest as dependent variables and Condition as the
explanatory variable. Planned contrasts based on a priori
hypotheses were conducted if a significant main effect of
condition was found. Only children in the Exergame Condition
exhibited improvement on the transfer Day-Night EF Task and
teacher ratings of EF (Fig. 2). These findings demonstrate that
improvements from Exergame immersion transferred to teacher
ratings of EF in addition to a novel EF task, while the Cognitive,
Exercise, and Control groups did not. Furthermore, to examine
the association between the behavioral EF assessments and PFC
connectivity changes, changes in each EF task were standardized
using Z-scores and averaged together to create an EF composite
variable (M=-.002; SD=.79). The changes in PFC connectivity
were positively associated with children’s EF composite scores,
r(49)=.41, 95%CI [.17, .98], p=.004, supporting the
neuroplasticity hypothesis of exergames.
Fig. 2. Changes in (A) Flanker (B) Day-Night (C) BRIEF (D) PFC
Connectivity. ***p < .001 **p < .01 *p < .05.
Although conservative interpretations should be drawn due
to the small sample size, the present findings extend the literature
in several ways. Exergame immersion exhibited the greatest
gains on EF and associated PFC connectivity, suggesting that the
combination of exercise and cognitive immersion drives the
effects on EF. The results support the hypothesis that Exergame
immersion is capable of fostering EF development and
neuroplasticity in children as young as 4 to 5-years old.
However, an open question is whether the findings generalize
across immersive virtual contexts and development.
IV. STUDY 2
This is a proposed experimental design to reproduce Study 1
in VR to test the extent to which the results of enhanced EF and
neuroplasticity can be generalized to both a distinct immersive
virtual context and across development.
A. Study 2 Procedure and Measures
Participants recruited for this study are youth ages 13-18, as
the effects of immersive VR on preschool-aged children are still
unknown. Evidence has emerged that the PFC remains plastic
throughout the lifespan, contrary to previous beliefs that the PFC
develops up into late adolescence . A between-subjects design
is implemented with two testing phases: pretest and posttest.
Neuroimaging utilizing fNIRS, standardized EF tasks, and self-
reported ratings of EF are assessed at pre- and posttest. Youth in
the Exergame, Cognitive, Exercise, and Control Conditions train
for 2 consecutive days, 30 mins/day, for 60 mins total.
Participants return to the laboratory the following day after
training to be reassessed at posttest. A Polar OH1, Wireless
Physical Activity Monitor records physical activity. The
immersive environments are experimentally modified versions
of Beat Saber, a VR rhythm game developed and published by
Beat Games, with meticulous manipulations to the amount of
physical activity and cognitive demands. Participants use
commercially available VR headsets. The handheld controllers
are used as light sabers as players slice targets, avoid incoming
obstacles with matching auditory, visual, and performance
feedback to provide an enriching multimodal experience.
B. Study 2 Immersive Training Sessions
The iterations of Beat Saber used in this study present
participants with a stream of approaching blocks. Each block is
colored red or blue, which is sliced with corresponding red or
blue sabers. Participants respond by moving their arms left,
right, up, or down depending on whether the direction of the
arrow on the target block is facing left, right, up, or down. Across
conditions, participants play 1 practice song on easy difficulty,
followed by 8 songs on normal difficulty. Song order is
randomly assigned and the same order is used across conditions.
The Exergame (high physical activity + high cognitive
demands) is modified with a rule-switch: for every other song,
participants hit the blocks in the opposite direction of the arrows.
This mixed-rule condition places heavy cognitive demands as it
requires working memory, attention, and cognitive flexibility to
override previously learned rules and prepotent responses to
directionality (Fig. 3A). The Cognitive Condition (low physical
activity + high cognitive demands) is identical to the Exergame
except participants sit in an adjustable chair, to control for height
so energy is not expended reaching (Fig. 3B). The Exercise
Condition (high physical activity + low cognitive demands) is
identical to the Exergame except there is no rule-switch between
songs (Fig. 3C). The Control Condition (low physical activity +
low cognitive demands) is identical to the Exercise Condition in
that there is no rule-switch between songs, except participants sit
in an adjustable chair, rather than standing and playing (Fig. 3D).
Unlike Study 1, Study 2 has an active control group because
having a typically-developing control group is important with
studies involving very young children to control for maturation
Pilot data (within-subjects) indicates participants expend less
energy in the Cognitive and Control conditions indicated by
changes in heart rate, and participants find the Exergame and
Cognitive Conditions more cognitively taxing than the Exercise
and Control Conditions indicated by average reaction time and
self-reports. It is hypothesized that youth randomly assigned to
exergame play in VR will exhibit improved performance on EF
tasks, self-reported measures of EF, and increased PFC
connectivity more so than single component cognitive or
exercise immersion in VR, and changes in EF performance will
be associated with increased connectivity in the PFC.
Fig. 3A. Exergame Immersion
↑ Cognitive ↑Exercise
Fig. 3B. Cognitive Immersion
↑ Cognitive ↓ Exercise
Fig. 3C. Exercise Immersion
↓ Cognitive ↑ Exercise
Fig. 3D. Control
↓ Cognitive ↓ Exercise
Fig. 3. (A) Exergame (B) Cognitive (C) Exercise (D) Control Conditions
These findings would 1) provide evidence that changes in
cognition and neuroplasticity can be induced across ages
through applying cognitive principles in VR 2) validate the
concept replicability of Study 1 in that the combination of
cognitive and exercise stimulation in a virtual environment
modulates the effects on EF and increased functional
connectivity in the PFC and 3) increase the ecological validity
by carrying out a carefully controlled experimental design with
a widely-available, popular VR game. Exergames can easily be
implemented in everyday lifestyles and have high potential to
improve essential EF skills that are crucial for academic success,
as well as prevent and remediate symptoms of EF deficits
associated with neurobehavioral disorders. These preliminary
findings and proposed study inform both theoretical and
practical avenues for fostering EF across development in
immersive virtual environments.
We thank Melissa Pocsai, Kalpa Anjur, Suanna Moron,
Emery Noll, Kristy Zhang, and Elizabeth Fulton for their help
collecting and coding data, Eileen Lee, Bridget Tan, and Adrian
Mester for assisting in the Exergame design and task
development, Dr. Frank Fishburn for assisting with the fNIRS
data preprocessing and analysis, and Professor Sarah Pickett for
composing the music and sound effects for the games. We also
thank The Children’s School at Carnegie Mellon University who
made Study 1 possible.
 J. R. Best and P. H. Miller, “A developmental perspective on executive
function,” Child Dev., vol. 81, no. 6, pp. 1641–1660, 2010.
 D. M. Werchan and D. Amso, “A novel ecological account of prefrontal
cortex functional development,” Psychol. Rev., vol. 124, no. 6, pp. 720–
 C. Blair and R. P. Razza, “Relating effortful control, executive function,
and false belief understanding to emerging math and literacy ability in
kindergarten,” Child Dev., vol. 78, no. 2, pp. 647–663, 2007.
 A. Diamond and K. Lee, “Interventions shown to aid executive function
development in children 4 to 12 years old,” Sci., vol. 333, no. 6045, pp.
 J. R. Best, “Effects of physical activity on children’s executive function:
Contributions of experimental research on aerobic exercise,” Dev. Rev.,
vol. 30, no. 4, pp. 331–351, 2010.
 R. Monteiro-Junior et al., “Exergames: Neuroplastic hypothesis about
cognitive improvement and biological effects on physical function of
institutionalized older persons,” Neu. Reg. Res., vol. 2, no. 2, pp. 201–204,
 E. Stanmore et al., “The effect of active video games on cognitive
functioning in clinical and non-clinical populations: A meta-analysis of
randomized controlled trials,” Neuro. Biobeh. Rev., vol. 78, pp. 34-43,
 J. R. Best, “Exergaming in Youth: Effects on Physical and Cognitive
Health.,” Z. Psychol., vol. 221, no. 2, pp. 72–78, 2013.
 M. R. Rueda et al., “Development of attentional networks in childhood,”
Neuropsych., vol. 48, no. 8, pp. 1029–1040, 2004.
 C. L. Gerstadt, Y. J. Hong, and A. Diamond, “The relationship between
cognition and action: performance of children 3 1/2-7 years old on a
stroop- like day-night test,” Cog., vol. 53, no. 2, pp. 129–153, 1994.
 G. A. Gioia, P. K. Isquith, S. C. Guy, L. Kenworthy, and I. S. Baron,
“Behavior rating inventory of executive function: BRIEF,” in Child
Neuropsych., vol. 6, no. 3, pp. 235–238, 2000.
 X. Shen et al., “Resting-state connectivity and its association with
cognitive performance, educational attainment, and household income in
the uk biobank,” Biol. Psychiatry Cogn. Neurosci. Neuroimaging, vol. 3,
no. 10, pp. 878–886, 2018.
 T. Vanderwal, C. Kelly, J. Eilbott, L. C. Mayes, and F. X. Castellanos,
“Inscapes: A movie paradigm to improve compliance in functional
magnetic resonance imaging,” NeuroIm., vol. 122, pp. 222–232, 2015.
 S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From game design
elements to gamefulness: defining ‘gamification’,” in Proc. 15th Inter.
Acad. MindTrek Conf., vol. 11, Sep. 2011, pp. 9–15.
 S.A. Ambrose, M. W. Bridges, M. DiPietro, M. W. Lovett, and M. K.
Norman, How learning works: Seven research-based principles for smart
teaching, San Francisco, CA, USA: John Wiley & Sons, 2010.
 H. Santosa, X. Zhai, F. Fishburn, and T. Huppert, “The NIRS Brain
AnalyzIR toolbox,” Algor., vol. 11., no. 5., pp. 73–106, 2018.