Conference PaperPDF Available

Pilot results from a virtual reality executive function task


Abstract and Figures

Impairments in executive functions (EF) negatively impact the capacity for independent living, damaging personal autonomy, and diminishing quality of life. Virtual environments offer an ecologically valid way to evaluate a person's ability to carry out tasks that depend on EFs. The purpose of this pilot study was to evaluate the feasibility of a virtual reality office task, the Assessim Office (AO), in persons with Multiple Sclerosis and Traumatic Brain Injury, to evaluate performance of patient groups relative to each other and to healthy controls on the AO, and to explore the relationship between patient's performance on AO tasks and neuropsychological measures of EF.
Content may be subject to copyright.
Pilot results from a virtual reality executive
function task
D Krch, O Nikelshpur, S Lavrador, ND Chiaravalloti
Kessler Foundation Research Center
West Orange, NJ, USA
and University of Dentistry and Medicine of New Jersey
Newark, NJ, USA
S Koenig, A Rizzo
USC Institute for Creative Technologies
Playa Vista, CA, USA
AbstractImpairments in executive functions (EF)
negatively impact the capacity for independent living,
damaging personal autonomy, and diminishing quality of
life. Virtual environments offer an ecologically valid way
to evaluate a person’s ability to carry out tasks that
depend on EFs. The purpose of this pilot study was to
evaluate the feasibility of a virtual reality office task, the
Assessim Office (AO), in persons with Multiple Sclerosis
and Traumatic Brain Injury, to evaluate performance of
patient groups relative to each other and to healthy
controls on the AO, and to explore the relationship
between patient’s performance on AO tasks and
neuropsychological measures of EF.
Keywords virtual reality, executive functions, traumatic
brain injury, multiple sclerosis
Executive Functioning (EF) is a multifaceted construct that
supports the formation, maintainenance and shifting of mental
sets. Although EF is a widely recognized term, it is often
conceptualized and operationalized differently among
researchers. Within clinical neuropsychology, there are a
handful of skills and processes that are widely recognized as
elemental within EF. Skills include planning and reasoning,
organizing, and problem solving; processes include working
memory, selective attention, attentional vigilance, divided
attention and inhibition of irrelevant information [1, 2]. Skills
and processes are integrated into complex goal-directed and
purposive behaviors that are requisite for successful execution
of daily life functions. EF impairments negatively impact the
capacity for independent living by decreasing personal
autonomy, impeding return to employment and community,
and diminishing quality of life [3, 4]. The accurate evaluation
of these impairments is important in order to assist clinicians in
treatment planning.
The traditional approach to evaluating EF impairments is
through paper and pencil neuropsychological evaluations.
However, patients who are expected to perform poorly due to
self-reported difficulties with daily activities may actually
perform within normal limits on standardized
neuropsychological tests of executive functioning [5-7].
Indeed, studies have demonstrated that the relationship
between performance on paper and pencil EF tasks and
performance in activities of daily living is weak [8, 9]. These
discrepancies suggest that standardized paper and pencil
neuropsychological tests may not adequately reproduce the
complexity and dynamic nature of real-life situations, resulting
in limited meaningfulness, practicality, and generalizability to
activities of daily living for patient populations. Ecologically
valid tools must be capable of taxing multiple executive
processes simultaneously (e.g., increasing stressors and
distractions) to be more predictive of real-world performances
Virtual environments allow for the creation of testing
instruments that are able to adequately detect dysfunction in
specific cognitive domains while providing greater ecological
validity than some traditional neurocognitive batteries [12].
Research has demonstrated that performance using VR-based
measures of learning and memory are correlated with
neuropsychological measures of memory ability [13, 14] and
similar findings have been documented between VR-based
measures of EF and neuropsychological measures of EF [e.g.,
15]. Further, studies investigating computer-based cognitive
assessment instruments have shown that virtual reality can be
used as an effective tool to predict everyday task performance
in individuals with neurological impairment [13, 15, 16].
Therefore, further research into the development of such
virtual-based testing instruments is particularly helpful for
improved elucidation of traditionally complex cognitive
The application of virtual environments to the evaluation
and treatment of EF impairments has been gradually
increasing over the last decade. EF skills and processes
targeted through existing studies include prospective memory,
planning, organizing, problem solving, attention shifting, and
sense of presence [2, 15, 17-26]. Some of these efforts
produced isolated publications, which served largely to
establish feasibility and usability of specific virtual EF
evaluation tools. However, there are three virtual
environments (i.e., Virtual Action Planning - Supermarket
(VAP-S); Virtual Mall (VMall); Virtual Library Test (VLT))
wherein the research has moved beyond ascertaining
This work was supported by funding from the National Multiple Sclerosis
Society (RG 4607-A-3) and the National Institute on Disability and
Rehabilitation Research (Northern NJ Traumatic Brain Injury Model System
Grant H133A070037).
feasibility, and the investigators have sought to establish
ecological and construct validity [15, 20-26]. The VAP-S is a
supermarket developed to assess cognitive planning ability. It
has been established as effective in differentiating various
patient groups (i.e., mild cognitive impairment, Parkinson’s
disease, schizophrenia, and stroke) from healthy controls
(HCs) [20-22, 27]. The VMall is a virtual supermarket
environment, utilized to evaluate and treat deficits in planning,
organizing, multitasking and problem solving. The VMall has
been found to demonstrate ecological and construct validity as
an evaluation tool in stroke [23, 25] and to be effective in
improving complex everyday activities in acquired and
traumatic brain injury [24, 26]. The VLT evaluates
prospective memory performance in a virtual library setting.
Findings supported the convergent and discriminant validity
of the VLT applied in a TBI population [15].
Overall, studies to date have demonstrated strong
preliminary evidence of the advantage of using virtual
environments for the evaluation and treatment of EF
impairments. EF is multifaceted, however, and existing studies
have not yet addressed all of the skills and processes
comprising EF. The Assessim Office (AO) was devised to
complement the existing body of work and to capture
elements of EF not yet addressed in current virtual
environments. The AO evaluates performance on tasks of
selective and divided attention, complex problem solving,
working memory, and prospective memory [28]. The goal of
this initial pilot work was to evaluate the feasibility of using a
virtual reality-based evaluation tool in individuals with
traumatic brain injury (TBI) and multiple sclerosis (MS), to
assess the performance of the patient groups relative to each
other and to healthy controls, and to explore the relationship
between patients’ performance on the AO and on standardized
EF measures. Subsequent studies will evaluate the ecological
and construct validity of the AO.
A. Participants
The study sample included seven individuals with TBI
(age = 40.1±15.3; education = 14.3±1.8; 3 male), five
individuals with MS (age = 51.0±8.9; education = 14.6±2.0; 1
male), and seven HCs (age = 31.1±14.1; education = 16±1.2; 4
male), for a total of 19 participants. There were no significant
group differences in age or education. Both patient groups
were enrolled in clinical trials evaluating the effectiveness of
behavioral interventions in improving cognitive function and
completed a comprehensive neuropsychological evaluation as
part of their participation in each respective study. HCs were
recruited specifically for evaluating their performance on the
AO for comparison to that of the patient populations.
B. Measures: Assessim Office
The Assessim Office (AO) task is a comprehensive VR
framework for the assessment of cognitive functions,
developed in collaboration with the University of Southern
California’s Institute for Creative Technologies and the Kessler
Foundation Research Center. The application is based on the
Assessim Framework and provides a range of realistic tasks for
the assessment of cognitive abilities. Although the AO
environment takes place in a work setting, the aim of the
application is to assess EF in a complex functional
environment. The combination of several tasks of different
priorities (e.g. rule-based decision task, reaction time task,
divided attention task) is designed to simulate challenging
scenarios that are similar to the demands that are placed on the
cognitive system in a real-world work setting.
The developed framework encompasses a virtual office
environment and the software infrastructure to rapidly
implement cognitive tasks anywhere in the virtual scenario.
The AO was developed using iterative design and testing [28];
through more than 20 iterations, we tested and adjusted input
devices, user interfaces, task difficulties, data collection
algorithms and consumer instructions (See Koenig, et al.,
2012 for a thorough review of the development process).
In order to facilitate standardization of the administration
of AO tasks, manualized instructions were created to
accompany the software and assist the examiner. The manual
includes a list of questions frequently asked by participants,
along with the standardized responses to be given in different
situations. For example, if the participant asks, “Can I use the
shredder”; the examiner would respond, “Remember to focus
on completing the tasks on your priority list. These tasks are
time-sensitive and should be completed as quickly as
possible”. The manual also provides cues for common
confusions. For example, if the participant becomes lost in the
office environment, initial cueing is, “Are you looking for
something?” Depending on the participant’s response,
additional detailed cueing is provided to ensure that AO is
measuring the participant’s ability to carry out EF tasks
instead of spatial orientation.
C. Measures: Neuropsychological Assessment
TBI and MS participants were evaluated with a
comprehensive battery of standardized paper and pencil tests
designed to measure various domains of cognition, including,
but not limited to attention, memory, speed of information
processing, and intellectual and executive functions. Only the
measures of complex attention and executive function were
examined for the purpose of this study (Table 1).
Neuropsychological Construct
WAIS-III Letter Number Sequencing
(LNS) [29]
Working memory, mental control
WAIS-III Digit Span (Backward) [29]
Working memory, complex
Delis-Kaplan Executive Function System
(D-KEFS) Trails Condition 4 [30]
Executive function (set switching,
flexibility of thinking)
D-KEFS Color Word Test (Inhibition and
Inhibition Switching Conditions) [30]
Executive function (response
inhibition, flexibility of thinking)
Wechsler Abbreviated Scale of
Intelligence (WASI; Block Design and
Matrix Reasoning) [31]
Executive function (complex
problem solving, abstract
D. Procedures
All studies were approved by the Institutional Review
Board (IRB) of Kessler Foundation. All participants signed an
informed consent approved by the Kessler Foundation IRB
prior to enrollment and participation in the respective study.
Participants in clinical samples received the AO as part of a
comprehensive neuropsychological evaluation. HCs were
recruited for the sole purpose of comparing their data to patient
populations and did not undergo any other neuropsychological
All participant groups (i.e., MS, TBI, HC) were
administered the AO on an IBM PC-compatible computer with
a standard 24-inch LCD monitor and plug-and-play stereo
desktop speakers. Participants were seated at least 50 cm away
from the screen and navigated the virtual environment and
completed virtual tasks using the left and right keys of a two-
key mouse.
Participants worked in the virtual office environment,
where they were seated at the virtual desk equipped with a
computer monitor, a keyboard, a counting device to track the
budget, a document tray, and a file folder. The virtual office
environment also included other desks, two printers, and a
conference room with a projector screen (See Fig. 1 for a
screen shot of the AO environment). Prior to beginning the
experimental task, participants were oriented to the location of
key objects in the virtual office environment. They were able to
practice navigating through the environment, and were trained
on which tasks they would be asked to carry out during their
workday. Multiple attempts to review screen shots and task
instructions were given as necessary. Participants were told
their workday would last approximately 15 minutes.
Participants were asked to complete the following tasks
during the workday: 1) respond to emails, 2) decide whether to
accept or reject real estate offers based on specific criteria, 3)
print the real estate offers that met specific criteria
(independent of whether the offer was accepted or rejected), 4)
retrieve printed offers from the printer and deliver them to a
file box located on participants’ desk, and 5) ensure that the
conference room projector light remained on at all times. Each
of the tasks was intended to reflect specific EF skills and
processes. In addition to evaluating the targeted task behaviors,
off-task behaviors were tallied and evaluated for the presence
of inattentiveness and perseverative behaviors. See Table 2 for
AO tasks and their corresponding EF constructs.
E. Data Analysis
Data were analyzed using SPSS software, version 18. Due
to small sample sizes, the data were not normally distributed,
necessitating the use of nonparametric analyses. Group
differences in performance on AO tasks were evaluated using
Mann-Whitney U tests. Spearman’s rank order correlations
were conducted among the AO tasks and performance on
standardized neuropsychological tasks, separately for the MS
and the TBI groups.
A. Qualitative Evaluation of Feasibility in MS and TBI
Participants with MS and TBI reported to have understood
the task instructions, were able to navigate through the virtual
environment and maneuver around the virtual obstacles using
the mouse, and could complete the tasks in the AO. Qualitative
feedback from participants after completing the VR Office Test
revealed that distractors (e.g., phones ringing in the
background) made completing the assigned tasks much more
challenging. Clinical evaluation of behaviors during AO
administration revealed that despite endorsement by the
participants that task instructions were fully understood,
behaviors suggested superficial comprehension at times.
Specifically, it appeared that the criteria for real estate
decisions and whether or not to print the real estate offers were
too complex, resulting in these tasks depending too heavily on
working memory.
B. TBI vs. HC Performance on the AO
Individuals with TBI made significantly fewer correct real
estate decisions than HCs, U=4.50, p=.007, r=.69 (For
reference, Cohen’s conventional effect sizes: small = .10,
moderate = .30, large = .50). Individuals with TBI incorrectly
printed declined real estate offers significantly less often than
HCs, U=5.50; p=.011, r=.68. A trend towards significance was
seen in correct delivery of the printed offers to the file box,
wherein individuals with TBI were less likely to file offers
relative to HCs (U=9.00, p=.053, r=.54). The lack of other
significant results may be due to the limited sample size of
this pilot study. See Table 3 for a summary of median and
range values for TBI and HC groups.
EF Construct
Selective attention (EF process)
Complex problem-solving (EF skill)
with working memory component
(EF process)
Prospective memory (EF process)
Divided attention (EF process)
Figure 1. Virtual office environment rendered in the Unity game engine
C. MS vs. HC Performance on the AO
Individuals with MS made significantly fewer correct real
estate decisions than HCs, U=4.00, p=0.030, r=.64.
Individuals with MS failed to turn the projector light back on
significantly more times than HCs, U=2.50; p=.010, r =.72.
The following trends toward significance were also observed:
participants with MS correctly responded to fewer emails than
HCs (U=7.00, p=.106, r=.51); HCs incorrectly printed
declined offers more often than participants with MS (U=6.00,
p=.073, r=.55). Again, the lack of significant results may have
been due to the limited sample size. See Table 4 for a summary
of median and range values for MS and HC groups.
D. TBI vs. MS Performance on the AO
TBI participants viewed instructions a significantly greater
number of times than MS participants (U=5.00, p=.048,
r=.63). MS participants tended to fail to turn the projector
back on more frequently than TBI participants, though not to a
significant degree (U=7.50, p=.106, r=.50). See Table 5 for a
summary of group comparisons.
E. Correlations Among AO and Neuropsychological (NP)
Measures in TBI
In the TBI sample, a correlation between the real estate
decision task and the NP measure of complex problem solving
(WASI Matrix Reasoning) approached significance.
Performance on an AO measure of selective attention
(responding to emails) correlated significantly with a NP
measure of response inhibition and flexibility of thinking (D-
KEFS Color Word Test Inhibition/Switching). Printing real
estate offers, an AO measure of complex problem solving with
a working memory component, correlated significantly with
performance on NP measures of cognitive flexibility (D-KEFS
Trail Making Test Condition 4 raw score and set loss errors).
An AO measure of divided attention (turning back on the
projector light) also correlated significantly with the NP
measures of cognitive flexibility (D-KEFS Trail Making Test
Condition 4 raw score and set loss errors). See Table 6 for a
summary of select correlations.
Digit Span Backward
Letter Number
D-KEFS Trails 4 raw
D-KEFS Trails 4 set
loss errors
D-KEFS Trails 4 all
D-KEFS Color Word
D-KEFS Color Word
WASI Block Design
WASI Matrix
D-KEFS = Delis-Kaplan Executive Function System; WASI = Wechsler
Abbreviated Scale of Intelligence; ECR = Emails correctly replied; CDRE =
Correct decision real estate offers; DOIP = Declined (real estate) offers,
incorrectly printed; PODF = Printed offers delivered to file box; PLM =
Projector light missed; *p<.05, **p<.01
Emails correctly replied
4 (1-8)
6 (0-8)
Correct decision, real estate offers
7 (0-9)
12 (5-15)
Declined offers incorrectly printed
0 (0-1)
2 (0-5)
Printed offers delivered to file box
2 (0-4)
4 (0-15)
Projector light missed
3 (0-6)
2 (0-5)
Redundant clicks
0 (0-75)
1 (0-8)
Emails correctly replied
4 (0-5)
6 (0-8)
Correct decision, real estate offers
7 (1-9)
12 (5-15)
Declined offers incorrectly printed
0 (0-2)
2 (0-5)
Printed offers delivered to file box
0 (0-6)
4 (0-15)
Projector light missed
6 (3-6)
2 (0-5)
Redundant clicks
8 (0-28)
1 (0-8)
# times requested to view instructions
Emails correctly replied
Correct decision, real estate offers
Declined offers incorrectly printed
Projector light missed
Redundant clicks
F. Correlations Among AO and NP Variables in MS
In the MS sample, performance on the real estate decision
task (AO measure of the complex problem solving) correlated
significantly only with a NP measure of complex attention and
working memory (WAIS-III Digit Span Backward). In
addition, responding to emails correlated significantly with a
measure of inhibition and cognitive flexibility (D-KEFS Color
Word Inhibition/ Switching). No other significant correlations
were found between AO and neuropsychological performance
in this sample. See Table 7 for a summary of select
The primary aims of this pilot study were to evaluate the
feasibility of using a virtual reality-based evaluation tool with
TBI and MS populations, to assess the performance of these
patient populations relative to HCs, and to explore the
relationship between patients’ performance on the AO and on
standardized EF measures.
The qualitative feasibility evaluation revealed that
individuals with MS and TBI were able to tolerate engaging in
a virtual environment; they had minimal difficulty
maneuvering the virtual environment with a mouse, and they
understood instructions for the simpler tasks. However, some
of the tasks were decidedly too complex and an unintended
consequence of this complexity was too great a demand on the
working memory system of participants. Despite the fact that
participants were allowed and encouraged to ask to review the
decision task criteria at any point during administration,
participants often failed to ask for assistance, possibly due to
failing to remember that this was an option. Rather, they
proceeded throughout the virtual workday either guessing
haphazardly at the decision tasks or perseveratively responding
in the same fashion to the decision tasks. This approach was
evidenced by the fact that both individuals with TBI and MS
exhibited significantly fewer errors on the printing decision
task; inspection of the raw data revealed that patient
populations simply didn’t print any of the real estate offers,
suggesting that they avoided responding when they weren’t
sure how to do so correctly. Although this response pattern
renders the data uninterpretable, it underscores the capability of
virtual tasks to achieve an extremely high level of difficulty,
reflecting the complexity of various real-life demands.
Ecologically valid tools must be capable of taxing multiple
executive processes simultaneously (e.g., increasing stressors
and distractions) to be more predictive of real-world
performances [9-11]. It appears that the AO is certainly
capable of accomplishing this goal. Subsequent iterations of
the AO will take this complexity into consideration when
determining the appropriateness of difficulty level for target
patient populations.
Quantitative evaluation of the data revealed that the AO
was able to successfully distinguish TBI subjects from HCs on
measures of selective and divided attention, problem solving,
and prospective memory. Specifically, individuals with TBI
performed overall more poorly on all the tasks in the AO in
comparison to HCs. Although these results weren’t always
supported by statistically significant p-values, effect sizes were
often large, suggesting that failure to reach significance was
more likely a product of small sample size than the lack of an
effect. The only variable that failed to provide useful clinical
information was the printing decision task, which was rendered
uninterpretable as described above. Impaired performance on
the tasks within the AO is consistent with deficits documented
on standardized neuropsychological measures [7, 32-36].
Quantitative evaluation of the MS data revealed that the
AO successfully differentiated the MS participants’
performance from that of HCs on all of the targeted EF
constructs, with the exception of the printing decision task.
Such findings were supported by statistically significant results
on two tasks, paired with a trend toward significance on two
other measures. Again, the only exception was the
uninterpretable printing decision task. Impairments in problem
solving and selective and divided attention, as evidenced on the
AO, are functions reported in the literature to be compromised
in MS [37, 38].
Comparison of patient groups on AO tasks was conducted
to explore the differential sensitivity in the TBI and MS
populations. Participants with TBI requested to review the real
estate criteria instructions more frequently than participants
with MS. The MS group had a slightly greater tendency to miss
turning the projector back on relative to the TBI group. Other
than these findings, the two clinical groups were similarly
impaired on AO tasks. A differential sensitivity analysis with a
larger subject sample may provide richer information regarding
distinct patterns of performance between different patient
Exploration of the relationship between performance on
AO tasks and standardized neuropsychological EF tasks was
Digit Span Backward
Letter Number
D-KEFS Trails 4 raw
D-KEFS Trails 4 set
loss errors
D-KEFS Trails 4 all
D-KEFS Color Word
D-KEFS Color Word
WASI Block Design
WASI Matrix
D-KEFS = Delis-Kaplan Executive Function System; WASI = Wechsler
Abbreviated Scale of Intelligence; ECR = Emails correctly replied; CDRE =
Correct decision real estate offers; DOIP = Declined (real estate) offers,
incorrectly printed; PODF = Printed offers delivered to file box; PLM =
Projector light missed; *p<.05
conducted for each patient group separately. For participants
with TBI, each AO task showed a signficant relationship or
trend toward significance with one to two neuropsychological
measures. In contrast, for participants with MS, significant
findings or trends toward significance were seen between only
two AO tasks (correct decision on real estate offers, printed
offers delivered to the file box) and standardized NP measures.
D-KEFS Trails 4 raw score was the standardized test that was
most frequently associated with AO tasks, whereas Letter
Number Sequencing was the only standardized test to not
relate strongly to any AO measure. In general, there were
striking patterns that arose between AO and standardized
measures, but given the large number of correlations that were
performed, it is unclear whether the significant findings and
other trends toward significance observed were due to chance.
Given the small sample sizes, it is difficult to draw conclusions
in either direction.
The current study demonstrated that the AO was well
tolerated by our TBI and MS samples and that performance by
the clinical samples on the AO was distinct from that of HCs.
Overall, patient performance was poorer than that of HCs
across all AO tasks. While these differences were not always
supported by significant p-values, strong effect sizes suggest
that an adequately powered study with greater sample size
would demonstrate more robust differences between patient
groups and HCs. Evaluation of the relationship between
performance on AO tasks and neuropsychological tests of EF
revealed that there were more significant relationships
demonstrated within the TBI group as compared with the MS
group. Given the small sample size, these correlations are
inconclusive. However, it should be noted that the rationale for
utilizing a virtual reality approach relies on it’s ability to mimic
real-life, complex, and dynamic situations; given the criticism
that paper and pencil tests fail to achieve these characteristics,
it is hypothesized that strong and systematic correlations would
not necessarily emerge if a larger sample were to be used.
Indeed, Renison and colleagues found that their VLT only
related to verbal fluency, Zoo Map, and the Modified Six
Elements Test [15]. To confirm this hypothesis, we will
evaluate these relationships once a larger sample has been
One limitation of the current study is that we were unable
to evaluate the ecological validity of the AO by comparing
performance on AO tasks to self-report of EF impairments in
daily life [e.g., Dysexecutive Questionnaire (DEX); Frontal
Systems Behavior Scale (FrSBe)] or performance on
functional instruments [e.g., the Multiple Errands Test
(MET); Executive Functions Performance Test (EFPT)] [5, 39-
41]; these tests were not included in the currrent testing battery.
Given that these self-report measures and functional
instruments have been reported to relate to performance on the
VAP-S, the VMall, and the VLT [15, 21, 26], evaluating for
the presence of such a relationship with AO tasks would be an
important next step.
Deficits in executive function have been shown to
contribute to obstacles in community integration and return to
employment, which in turn negatively impact quality of life.
Therefore, continued development of the AO is expected to
lead to improved ability to evaluate EF skills and processes not
currently captured by existing virtual environment evaluation
[1] Suchy, Y., Executive functioning: Overview, assessment, and
research issues for non-neuropsychologists. Ann Behav Med,
2009. 37: p. 106-16.
[2] Carelli, L., F. Morganti, P.L. Weiss, R. Kizony, and G. Riva. A
virtual reality paradigm for the assessment and rehabilitation of
executive function deficits post stroke: Feasibility study. in Virtual
Rehabilitation. 2008.
[3] Crépeau, F. and P. Scherzer, Predictors and indicators of work
status after traumatic brain injury: A meta-analysis. Neuropsychol
Rehabil, 1993. 3(1): p. 5-35.
[4] Worthington, A. and J. Waller, Rehabilitation of everyday living
skills in the context of executive disorders, in The rehabilitation of
executive disorders: a guide to theory and practice, M. Oddy and
A. Worthington, Editors. 2009, Oxford University Press Inc.: New
[5] Alderman, N., P.W. Burgess, C. Knight, and C. Henman,
Ecological validity of a simplified version of the multiple errands
shopping test. J Int Neuropsychol Soc, 2003. 9: p. 31-44.
[6] Norris, M. and R. Tate, he Behavioral Assessment of the
Disexecutive Syndrome (BADS): Ecological, concurrent and
construct validity. Neuropsychol Rehabil, 2000. 10: p. 33-45.
[7] Ord, J.S., K.W. Greve, K.J. Bianchini, and L.E. Aguerrevere,
Executive dysfunction in traumatic brain injury: the effects of
injury severity and effort on the Wisconsin Card Sorting Test. J
Clin Exp Neuropsychol, 2010. 32(2): p. 132-40.
[8] Chaytor, N., M. Schmitter-Edgecombe, and R. Burr, Improving the
ecological validity of executive functioning assessment. Arch Clin
Neuropsychol, 2006. 21(3): p. 217-27.
[9] Manchester, D., N. Priestley, and H. Jackson, The assessment of
executive functions: Coming out of the office. Brain Inj 2004.
18(11): p. 1067-81.
[10] Burgess, P.W., et al., The case for the development and use of
‘ecologically valid’ measures of executive function in experimental
and clincial neuropsychology. J Int Neuropsychol Soc, 2006.
[11] Chan, R.C., D. Shum, T. Toulopoulou, and E.Y. Chen, Assessment
of executive functions: review of instruments and identification of
critical issues. Arch Clin Neuropsychol, 2008. 23(2): p. 201-16.
[12] Rizzo, A., M.T. Schultheis, K.A. Kerns, and C.A. Mateer, Analysis
of assets for virtual reality applications in neuropsychology.
Neuropsychological Rehabilitation, 2004. 14(1/2): p. 207-239.
[13] Parsons, T.D. and A.A. Rizzo, Initial validation of a virtual
environment for assessment of memory functioning: Virtual reality
cognitive performance assessment test. Cyberpsychology and
Behavior, 2008. 11: p. 17-25.
[14] Matheis, R.J., et al., Is learning and memory different in a virtual
environment? Clin Neuropsychol, 2007. 21(1): p. 146-61.
[15] Renison, B., J. Ponsford, R. Testa, B. Richardson, and K.
Brownfield, The Ecological and Construct Validity of a Newly
Developed Measure of Executive Function: The Virtual Library
Task. Journal of the International Neuropsychological Society,
2012. 18: p. 440-450.
[16] Flanagan, S.R., J.B. Cantor, and T.A. Ashman, Traumatic Brain
injury: future assessment tools and treatment prospects.
Neuropsychiatric Disease and Treatment, 2008. 4(5): p. 877-92.
[17] Lo Priore, C., G. Castelnuovo, D. Liccione, and D. Liccione,
Experience with V-STORE: Considerations on presence in virtual
environments for effective neuropsychological rehabilitation of
executive functions. CyberpPsychology and Behavior, 2003. 6(3):
p. 281-7.
[18] Cao, X., A.-S. Douguet, P. Fuchs, and E. Klinger. Designing an
ecological virtual task in the context of executive functions: a
preliminary study. in International Conference Series on
Disability, Virtual Reality and Associated Technologies. 2010.
Valparaiso, Chile.
[19] McGeorge, P., et al., Using Virtual Environments in the
Assessment of Executive Dysfunction. Presence, 2001. 10(4): p.
[20] Josman, N., A. Elbaz-Scheniderman, E. Klinger, and E. Shevil,
Using virtual reality to evaluate executive functioning among
persons with schizophrenia: A validity study. Schizophrenia Res.,
2009. 115: p. 270-77.
[21] Werner, P., S. Rabinowitz, E. Klinger, A.D. Korczyn, and N.
Josman, Use of the Virtual Action Planning Supermarket for the
Diagnosis of Mild Cognitive Impairment. Dementia and Geriatric
Cognitive Disorders, 2009. 27: p. 301-309.
[22] Klinger, E., I. Chemin, S. Lebreton, and R.M. Maríe, Virtual
Action Planning in Parkinson’s Disease: a control study.
Cyberpsychol Behav, 2006. 9(3): p. 342-7.
[23] Rand, D., N. Katz, and P.L. Weiss, Evaluation of virtual shopping
in the VMall: Comparison of post-stroke participants to healthy
control groups. Disabil Rehabil, 2007. 29: p. 1710-9.
[24] Rand, D., P.L. Weiss, and N. Katz, Training multitasking in a
virtual supermarket: A novel intervention after stroke. Am J Occup
Ther, 2009. 63: p. 535-42.
[25] Rand, D., S. Basha-Abu Rukan, P.L. Weiss, and N. Katz,
Validation of the Virtual MET as an assessment tool for executive
functions. Neuropsychol Rehabil, 2009. 19(4): p. 583-602.
[26] Jacoby, M., et al., Effectiveness of executive functions training
within a virtual supermarket for adults with traumatic brain
injury: a pilot study. IEEE Trans Neural Syst Rehabil Eng, 2013.
21(2): p. 182-90.
[27] Josman, N., et al. Performance within a virtual supermarket and
its relationship to executive functions in post-stroke patients. in
Proc Int Workshop Virt Rehabil. 2006. New York.
[28] Koenig, S.T., et al., User-centered development of a virtual reality
cognitive assessment, in International Conference Series on
Virtual Reality and Associated Technologies, P.S.a.E. Klinger,
Editor 2012: Laval, France.
[29] Wechsler, D., Weschsler Adult Intelligence Scale-III, 1997, The
Psychological Corporation: San Antonio, TX.
[30] Delis, D.C., E. Kaplan, and J.H. Kramer, eds. Delis-Kaplan
Executive Function System. 2000, The Psychological Corporation:
San Antonio, Texas.
[31] Wechsler, D., ed. Wechsler abbreviated scale of intelligence
manual (WASI). 1999, Harcourt Assessment: San Antonio, TX.
[32] Cazalis, F., et al., fMRI study of problem-solving after severe
traumatic brain injury. Brain Injury, 2006. 20(10): p. 1019-28.
[33] Chan, R.C., E.Y. Chen, E.F. Cheung, R.Y. Chen, and H.K.
Cheung, Problem-solving ability in chronic schizophrenia. A
comparison study of patients with traumatic brain injury. Eur Arch
Psychiatry Clin Neurosci, 2004. 254(4): p. 236-41.
[34] Vas, A.K., S.B. Chapman, L.G. Cook, A.C. Elliott, and M.
Keebler, Higher-order reasoning training years after traumatic
brain injury in adults. J Head Trauma Rehabil, 2011. 26(3): p.
[35] Rochat, L., J. Ammann, E. Mayer, J.M. Annoni, and M. Van der
Linden, Executive disorders and perceived socio-emotional
changes after traumatic brain injury. J Neuropsychol, 2009. 3((Pt
2)): p. 213-27.
[36] Robertson, R.H. and R.G. Knight, Evaluation of social problem
solving after traumatic brain injury. Neuropsychol Rehabil, 2008.
18(2): p. 236-50.
[37] Rao, S.M., G.J. Leo, L. Bernardin, and F. Unverzagt, Cognitive
dysfunction in multiple sclerosis. I. Frequency, patterns, and
prediction. Neurology, 1991. 41(5): p. 685-91.
[38] Nebel, K., et al., Activity of attention related structures in multiple
sclerosis. Brain Res, 2007. 1151: p. 150-60.
[39] Grace, J. and P.F. Malloy, eds. Frontal Systems Behavior Scale.
2001, Psychological Assessment Resources, Inc.: Lutz, FL.
[40] Wilson, B.A., N. Alderman, P.W. Burgess, H. Emslie, and J.J.
Evans, Behavioral Assessment of the Dysexecutive Syndrome.
1996, Bury St. Edmund, UK: Thames Valley Test Company.
[41] Baum, M.C., T. Morrison, M. Hahn, and D. Edwards, Executive
Function Performance Test Manual. 2003, St Louis: Program in
Occupational Therapy Washington University School of Medicine.
... Another VR office task, known as Assessim Office (AO), was implemented by Krch et al. to evaluate several performances on realistic tasks of selective and divided attention, complex problem solving, working memory and prospective memory (Krch et al., 2013). Participants are seated at least 50 cm away from the computer screen and are immersed in the virtual office environment (rendered in the Unity game engine), in which they must navigate and complete virtual tasks using both keys of the mouse. ...
... To facilitate the administration of AO tasks, the authors created an instruction manual that includes many questions frequently asked by participants, with standardized responses and hints for common confusions (e.g., if the participant is lost in the virtual office, the initial cue is "Are you looking for something?") (Krch et al., 2013). The validation study showed that AO could be an exciting solution for evaluating executive impairments in patients with MS and TBI compared to healthy controls (Krch et al., 2013). ...
... (Krch et al., 2013). The validation study showed that AO could be an exciting solution for evaluating executive impairments in patients with MS and TBI compared to healthy controls (Krch et al., 2013). The findings suggested a significant difference between patients with MS and the control group on all executive tasks of AO, except for the printing decision task (working memory). ...
Full-text available
Introduction Executive dysfunctions constitute a significant public health problem: their high impact on everyday life makes it a priority to identify early strategies for evaluating and rehabilitating these disorders in a real-life context. The ecological limitation of traditional neuropsychological tests and several difficulties in administering tests or training in real-life scenarios have paved the way to use Virtual Reality-based tools to evaluate and rehabilitate Executive Functions (EFs) in real-life. Objective This work aims to conduct a systematic review to provide a detailed description of the VR-based tools currently developed for the evaluation and rehabilitation of EFs. Methods We systematically searched for original manuscripts regarding VR tools and EFs by looking for titles and abstracts in the PubMed, Scopus, PsycInfo, and Web of Science databases up to November 2021 that contained the following keywords “Virtual Reality” AND “Executive function * .” Results and Conclusion We analyzed 301 articles, of which 100 were included. Our work shows that available VR-based tools appear promising solutions for an ecological assessment and treatment of EFs in healthy subjects and several clinical populations.
... It is important to note that not all clinical populations are good candidates for HMDs due to increased susceptibility to simulator sickness and/or hypersensitivity to audiovisual stimuli. Computer monitor displays were utilized in thirteen studies (13/23, 57%) (Dockree et al., 2004;Knight et al., 2006;Pollak et al., 2009;Potvin et al., 2011;Schnabel and Kydd, 2012;Krch et al., 2013;Faria et al., 2016;Uitvlugt et al., 2016;Remington et al., 2019;Banire et al., 2020;Biss et al., 2020;Park et al., 2020;Pretus et al., 2020). One study (1/23, 4%) utilized a goggle system, the NordicNeuroLab goggle system for the presentation of distraction stimuli (Harrington et al., 2019). ...
... Thirteen articles combined both auditory and visual distractions for attention and memory assessment (13/23, 57%) (Knight et al., 2006;Parsons et al., 2007;Adams et al., 2009;Nolin et al., 2009;Potvin et al., 2011;Schnabel and Kydd, 2012;Krch et al., 2013;Díaz-Orueta et al., 2014;Parsons and Carlew, 2016;Ansado et al., 2018;Ouellet et al., 2018;Kratz et al., 2020;Yeh et al., 2020). Auditory distractions included background music, broadcasted news, chatter, cars, horns, pencils dropping, and bells from a cash register. ...
... effect occurred for group (p < 0.05), and condition (p < 0.05) for actions correctly retrieved (Potvin et al., 2011). Krch et al. (2013) created the Assessim Office to assess executive function in persons with TBI and multiple sclerosis (MS) in an office setting with relevant and irrelevant distractions. The Assessim Office was able to distinguish between persons with TBI, MS, and healthy controls in the areas of selective and divided attention, problem solving, and prospective memory. ...
Full-text available
Introduction: Deficits in the cognitive domains of attention and memory leave a large impact on everyday activities that are not easily captured in the clinical environment. Therefore, clinicians are compelled to utilize assessment tools that elicit everyday functioning that include real-world contexts and distractions. As a result, the use of computer-assisted assessment has emerged as a tool for capturing everyday functioning in a variety of environments. The purpose of this scoping review is to map how virtual reality, augmented reality, and computer-based programs have implemented distractions for clinical populations. Methods: A scoping review of peer reviewed publications was conducted by searching Pubmed, PsychInfo, Web of Science, Rehabdata, and Scopus databases (1960-October 20, 2020). Authors completed hand-searches for additional published and unpublished studies. Results: Of 616 titles screened, 23 articles met inclusion criteria to include in this review. Primary distraction display modalities included computer monitor displays ( n = 12) and head mounted displays (HMD) ( n = 7). While computer-assisted assessments included distractions, no systematic approach was utilized to implement them. Primary distractions included both auditory and visual stimuli that were relevant to the task and/or simulated environment. Additional distraction characteristics emerged including location, timing, and intensity that can contribute to overall noticeability. Conclusion: From this review, the authors examined the literature on the implementation of distractions in simulated programming. The authors make recommendations regarding identification, measurement, and programming with suggestions that future studies examining metrics of attention to implement distraction in measurable and meaningful ways. Further, the authors propose that distraction does not universally impact performance negatively but can also enhance performance for clinical populations (e.g. additional sensory stimuli to support focused attention).
... A number of examples illustrate efforts to enhance the ecological validity of assessment and rehabilitation by designing VR scenarios that are "replicas" of relevant archetypic functional environments. This has included the creation of virtual cities (Brown et al., 1998;Costas, Carvalho, & de Aragon, 2000;Gamito et al., 2016), supermarkets (Cromby et al., 1996;Josman et al., 2014;Levy et al., 2015); homes (Koenig, 2012a;Rose, Attree, Brooks, & Andrews, 2001); kitchens (Christiansen et al., 1998;Davies et al., 1998;Foloppe, Richard, Yamaguchi, Etcharry-Bouyx, & Allain, 2015;Wall, Cumming, Koenig, Pelecanos, & Copland, 2017), school environments (Coleman, Marion, Rizzo, Turnbull, & Nolty, 2019;Parsons et al., 2019;Rizzo et al., 2006;Stanton, Foreman, & Wilson, 1998), workspaces/offices Krch, Koenig, Lavrador, Rizzo, & Chiaravalloti, 2013;Matheis et al., 2007;McGeorge et al., 2001); rehabilitation wards (Brooks et al., 1999) and even a virtual beach (Elkind, Rubin, Rosenthal, Skoff, & Prather, 2001). From these efforts, recent reviews have provided support for the impact of ecologically-relevant Clinical VR applications on real world treatment outcomes in both clinical psychology (Morina et al., 2015) and in rehabilitation (Howard, 2017). ...
... In this regard, an ideal match exists between the stimulus delivery assets of VR simulation systems and the requirements of any clinical assessment and intervention procedure. This can be seen as a core asset whether one is testing constructspecific cognitive processes (e.g., selective attention performance contingent on varying levels of stimulus intensity and distraction) (M€ uhlberger et al., 2016;Rizzo et al., 2006), to the complex targeting of more molar functional behaviors (e.g., planning and initiating the steps to function within a complex office or home setting) (Keefe et al., 2016;Krch et al., 2013), to the precise titration of anxiety activating content in the service of pacing exposure therapy for the treatment of phobias or PTSD Rothbaum et al., 1995. ...
Systematic Representative Design and Clinical Virtual Reality Sharon Mozgai, Arno Hartholt & Albert “Skip” Rizzo* University of Southern California Institute for Creative Technologies Los Angeles, California *contact author The authors of the paper, “Causal Inference in Generalizable Environments: Systematic Representative Design”, boldly announce their core point in the opening line of the abstract stating that, “Causal inference and generalizability both matter.” While a surface glance might suggest this to be a simple notion, a closer examination reveals the complexity of what they are proposing. This complexity is apparent when one considers that the bulk of human experimental research has always been challenged in its inability to concurrently deliver on both of these aims. This is no slight on the tens of 1000’s of human researchers and behavioral scientists who have devoted long careers to highly controlled human psychological and social science laboratory research. Rather, it reflects the sheer enormity of the challenges for conducting human studies designed to specify human function with physics-informed lab methods, while at the same time producing results that lead to enhanced understanding and prediction of how people will operate in the complex and ever-changing contexts that make up everyday life. At the core of this issue is a methodological and philosophical challenge that is relevant to all areas of human subjects’ research, beyond the social science focus of the Miller et al. (2020) paper. It is our aim to discuss the central topics in their paper through the lens of our own work using Virtual/Augmented Reality and Virtual Human simulation technologies for clinical and training applications. The Miller et al. (2020) piece is dense with diverse theoretical viewpoints in support of their analysis of the causal inference vs. generalization dilemma and proposes a remedy that integrates these aims by leveraging mixed reality technologies. The authors engage the reader in a tour de force of classical (and current) theory and research aimed at dissecting this timeless challenge in the field of experimental psychology in the social sciences. This challenge can be stated simply in terms of goals that are synergistic in the ideal, but competing in the pragmatic: How does one conduct solid scientific human research under controlled laboratory conditions, which at the same time creates generalizable knowledge about thinking, feeling, behaving, and interacting in everyday life? To address this challenge, the authors propose the application of simulation technology as a new approach that optimizes representativeness to the organism-in-situation, to which they aim to generalize. This is not a newly recognized challenge. Over the last century, psychology has strived to establish its place in the “hard” sciences via the rigorous application of the scientific method aiming to measure, understand, and modify (or treat) human cognition, emotion, behavior, and social interaction under highly controlled experimental conditions. However, such laboratory-based research conditions are often lacking in context and bears little resemblance to the dynamic stimulus complexity of the everyday world. Not only does reality richly vary from moment-to-moment and place-to-place, but since humans are not clones, they also vary dramatically in genetically endowed capabilities, limitations, and predispositions. Thus, while the knowledge generated under very constrained laboratory-based conditions, where the rich diversity of human response and function is reduced to mean values, can describe phenomenon with strong internal consistency (to the highly controlled and reality-constrained/limited experimental setting), it often produces limited generalizability for understanding and/or predicting human function in everyday life. The Behaviorist movement strived to apply rigor to this experimental challenge by maintaining a militant focus on observable behavior as the only variable of interest in any attempt to study humans in a scientific fashion. And in some sense that view has endured, whether explicitly stated or implicit in the value that researchers place on behavioral measures in counterpoint to what some view as the fuzzy and variably-biased nature of introspective self-report data. However, the relevance of verbal declarations of perspective, intent, cognitive appraisal/analysis, and emotional expression in everyday life cannot be denied or discounted, and thus the various waves of cognitive-behavior approaches have succeeded in “sneaking the mind back into psychology”, to the dismay of old school behaviorists. However, the good news as Miller et al. (2020) clearly state, is that recent advances in modern simulation technologies (Virtual Reality (VR), Augmented Reality (AR), Virtual Human (VH) intelligent agents, etc.), have now provided new opportunities for creating research tools that aim to support a more predictive and ecologically relevant analysis of human function in everyday life. VR simulations can now present research participants with highly controlled, systematically deliverable stimulus presentations/challenges while they are immersed within the context of a functionally relevant simulation of the “real” world. AR provides similar opportunities for human research via its capacity to insert and naturally embed stimuli within a user’s perception of an everyday context. Moreover, the birth of intelligent VH agents that are capable of engaging users in credible interactions within VR/AR environments and give researchers the ability to create experimental manipulations that would be difficult or even impossible to systematically control in real life, have the potential to revolutionize how research is conducted in psychology and the social sciences. Miller et al. (2020) propose that such technological capability will support their view of Systematic Representative Design (SRD) as “…a synthesis of two major designs: representative design (Brunswik, 1943; Brunswik, 1955a, 1955b) and classic experimental or systematic designs (Shadish et al., 2002) with roots in Wundt (1902).” SRD is described as an approach that supports an integrated human research methodology that can provide both valid cause-effect results, along with the added value of supporting inferences that enhance the generalizability of research findings to everyday life (GEL). Moreover, this theoretical framework can be seen to underlie the emergence of VR/AR/VH clinical research and practice (Rizzo & Koenig, 2017). Such a perspective formalizes the conceptual drivers that inspired early researchers in the use of these technologies for therapeutic purposes since the mid-90’s when the vision for Clinical VR use drove significant excitement and pioneering research, in spite of the immaturity of the technology. (Note: for the rest of this paper the term VR will be used to generally include AR and VH, unless otherwise specified.
... A summary of previous research utilizing VR technology to study cognitive abilities is presented in Table 1. Among the features that make VR an appealing tool is the ability to create dynamic two/three-dimensional virtual environments (VEs) wherein users navigate through realistic spaces and interact with various stimuli as they would in a real-world situation (Krch et al., 2013;Lalonde, Henry, Drouin-Germain, Nolin, & Beauchamp, 2013). From a research and more applied perspective (i.e., rehabilitation), this creates opportunities to develop scenarios with high ecological validity, while the researcher retains a great deal of experimental control (Josman et al., 2014). ...
... Assess cognitive changes in Alzheimer's disease • Allain et al., 2014• Cushman et al., 2008• Kessels, van Doormaal, & Janzen, 2011• Pengas et al., 2012• Plancher, Tirard, Gyselinck, Nicolas, & Piolino, 2012• Widmann, Beinhoff, & Riepe, 2012 Assess cognitive changes in Mild Cognitive Impairment (MCI) • Plancher et al., 2012• Weniger et al., 2011 Assess cognitive changes in traumatic brain injury (TBI) • Krch et al., 2013 Assess cognitive changes in stroke • Josman et al., 2014 Assess knowledge transfer between real and virtual environments • Koenig et al., 2011• Ruddle et al., 1997• Wallett et al., 2011 Develop rehabilitation tools for cognitive or motor deficits • Cameirao et al., 2010• Klinger et al., 2013• Tost et al., 2009 Assess navigation abilities • Cushman et al., 2008• Cutmore et al., 2000• Kessels et al., 2011• Moffat, Zonderman, & Resnick, 2001• Richardson et al., 1999• Taillade et al., 2013• Zancada-Menendez et al., 2015 Investigate everyday multitasking abilities • Law, Trawley, Brown, Stephens, & Logie, 2013 Determine the value of VR for cognitive training • Anguera et al., 2013• Legault et al., 2013 Assess age-related cognitive decline • Anguera et al., 2013• Craik & Bialystok, 2006• Cushman et al., 2008• Moffat et al., 2001• Plancher et al., 2010• Taillade et al., 2013• Zancada-Menendez et al., 2015 on previous research, we expected all participants to perform slightly worse in the VE (Allain et al., 2014;Cushman, Stein, & Duffy, 2008). Navigational deficits are common in healthy older adults (Anguera et al., 2013;Klencklen, Despres, & Dufour, 2012). ...
Background: Virtual reality applications to assist older adult with cognitive and functional decline are fast growing. However, such technological developments face limitations such as due to limited constructs and ecological validity. This study was aimed at investigating age-related changes in functional abilities and their associated cognitive underpinnings during task performance in virtual and real environments. Method: Twenty-two younger adults (university students) and 22 older adults (aged 58–74) performed a multiple errands task twice, once in the “Discoveries” section of the National Museum of Scotland and once in the same room as a virtual environment. Accuracy and distance traveled were measured in both groups. Cognitive and daily living abilities were recorded in older adults using standard and novel questionnaires. Results: The testing environment had a significant effect on how efficient individuals performed the task. Older and younger adults’ performance was alike but older adults relied on more cognitive resources. Older adults struggled in the virtual but not in the real environment. Younger but not older adults could transfer knowledge between environments. Conclusion: The use of technology to assist frail older adults and those affected by dementia is growing rapidly. For these novel tools to be theoretically valid, they need to incorporate knowledge of the challenges they pose to these vulnerable groups. Here we present evidence of such challenges and their cognitive underpinnings. This theory may be considered by future applications aimed at enhancing functional abilities in these populations.
... Se espera que un escenario basado en tareas ecoló gicamente relevantes sea más sensible a déficits cognitivos en individuos con daño cerebral y pueda predecir el rendimiento cognitivo en entornos rea les con mayor precisión. En agosto de 2013 se pre sentaron los resultados de un estudio piloto que mostraba diferencias de rendimiento entre pacien tes con traumatismo craneoencefálico y controles en las medidas de atención selectiva y dividida, re solución de problemas y memoria prospectiva [64]. En el caso de la esclerosis múltiple, los datos reve laron que AssesSim Office diferenciaba con éxito el rendimiento entre esclerosis múltiple y controles. ...
... Entorno de la oficina virtual de AssesSim Office [tomada con permiso de[64]). ...
Full-text available
La valoración de las funciones ejecutivas es un aspecto central de la evaluación neuropsicológica. De cara a una evaluación fiable, la estrategia tradicionalmente seguida para la evaluación de las funciones ejecutivas ha sido su atomización en diferentes subprocesos cognitivos. No obstante, en la práctica clínica, frecuentemente resulta artificiosa la disgregación de un proceso cognitivo global y complejo como éste en infinidad de subcomponentes relacionados. Así, a menudo, las pruebas diseñadas de acuerdo con este paradigma resultan de poco valor para procedimientos clínicos (diagnóstico, diseño de la rehabilitación) por la pobre correspondencia con la realidad clínica del sujeto o paciente. Los objetivos del presente trabajo son revisar el concepto de validez ecológica aplicado a la evaluación de las funciones ejecutivas y hacer una revisión crítica de la valoración de las funciones ejecutivas mediante paradigmas de multitarea como medio de incrementar la validez ecológica y valor predictivo del desempeño funcional del sujeto. Tras un recorrido histórico por la (escasa) validez ecológica de los tests monotarea y la apuesta por un paradigma multitarea para la evaluación de las funciones ejecutivas, se recogen minuciosamente los tests multitarea existentes hasta la fecha (con sus respectivas ventajas e inconvenientes). Se concluye con recomendaciones concretas sobre cómo desarrollar tests multitarea en el futuro, atendiendo a parámetros concretos relacionados con entorno, tareas, objetivos, reglas y puntuaciones.
... In another study, neurological patients completed a number of office-based tasks in a virtual office (Assessim Office), receiving manualized cueing if they became disoriented or lost attentional set for the tasks in the face of distracters (e.g. phone ringing) (Krch et al., 2013). Examples of tasks included printing real estate documents, making real estate decisions, or shutting off a projector light. ...
... Examples of tasks included printing real estate documents, making real estate decisions, or shutting off a projector light. For patients with traumatic brain injury (TBI), poor performance on the VR office tasks was significantly correlated with poor performance on a number of executive neuropsychological measures, while for MS patients the only significant correlation was with visual reasoning (Krch et al., 2013). ...
Full-text available
Background: Virtual reality (VR) technology has demonstrated usefulness in diagnosis, education, and training. Studies supporting use of VR as a therapeutic treatment in medical rehabilitation settings remain limited. This study examines the use of VR in a treatment capacity, and whether it can be effectively integrated into neurorehabilitation. Objective: To determine whether immersive VR treatment interventions improve executive dysfunction in patients with brain injury and whether performance is stronger on a VR version of the Stroop than traditional Stroop formats. Methods: 15 patients with brain injury admitted to day neurorehabilitation. Outcome measures: reaction time, inhibition, and accuracy indices on VR Stroop; Automated Neuropsychological Assessment Metrics (ANAM) Stroop, Delis-Kaplan Executive Function System Stroop, Golden Stroop, and Woodcock-Johnson, 3rd Edition (WJ-III): Pair Cancellation. Results: Participants demonstrated significantly reduced response time on the word-reading condition of VR Stroop and non-significantly reduced response time on the interference condition. Non-significant improvements in accuracy and inhibition were demonstrated on the color-naming condition of VR Stroop. Significantly improved accuracy under time pressure was found for the ANAM, after VR intervention. Conclusion: Implementation of immersive VR interventions during neurorehabilitation is effective in improving specific executive functions and information processing speed in brain-injured patients during the subacute period.
... This may further explain the complete lack of research on VR as an adjuvant for CR or ET for inducing everyday functional improvements in persons with MS. Of note, VR programs have been created for cognitive assessment (including in persons with MS [114,115]), and might be conducive for application in rehabilitation settings. Such approaches are particularly exciting for application with established CR and ET approaches in eventual clinical trials on cognition and everyday functioning in this population. ...
Full-text available
Cognitive impairment is a common and detrimental consequence of multiple sclerosis (MS) and current rehabilitation methods are insufficient. Cognitive rehabilitation (CR) and exercise training (ET) are the most promising behavioral approaches to mitigate cognitive deficits, but effects are small and do not effectively translate to improvements in everyday function. This article presents a conceptual framework supporting the use of virtual reality (VR) as an ideal, common adjuvant traditional CR and ET in MS. VR could strengthen the effects of CR and ET by increasing sensory input and promoting multisensory integration and processing during rehabilitation. For ET specifically, VR can also help incorporate components of CR into exercise sessions. In addition, VR can enhance the transfer of cognitive improvements to everyday functioning by providing a more ecologically valid training environment. There is a clear interest in adding VR to traditional rehabilitation techniques for neurological populations, a stronger body of evidence of this unique approach is needed in MS. Finally, to better understand how to best utilize VR in rehabilitation for cognitive deficits in MS, more systematic research is needed to better understand the mechanism(s) of action of VR with CR and ET.
... The authors found that together with risk-taking propensity, their combined contributions could predict risky driving performance in VR driving tasks. In [24], an office scenario was developed to evaluate the performances in separate tasks addressing selective and divided attention, problem-solving capabilities, working memory components and prospective memory. A certain amount of flexibility was a fundamental requirement of this study to allow the user to switch among the various tasks depending on his or her particular needs (replying to emails, printing, etc.). ...
Brain injury can cause a variety of physical effects and cognitive deficits. Although it has not yet been systematically adopted in clinical settings, virtual reality promises to be an excellent therapeutic tool for regaining both locomotor and neurological capacities. This work presents the design and implementation of VR\(^2\) (Virtual Reality Rehabilitation), a customizable rehabilitation framework intended to enable the creation of motivating rehabilitation scenarios based on an ecologically valid semi-immersive system. Following the implementation phase, a study to test the acceptability of VR\(^2\) in a group of subjects with cerebral lesions was conducted to investigate the usability of the framework. The group consisted of 11 people from 22 to 70 years of age, who were divided into two groups depending on the chronicity of disorder. The adequacy of the interface between patient and system was verified through questionnaires containing subjective questions, which revealed good overall acceptance and enjoyment of the tool. Moreover, to obtain early results useful for tuning the overall system in preparation for rigorous clinical trials, a set of preliminary cognitive tests concerning the rehabilitation protocol was conducted within the same group. Although the preliminary findings are promising and reveal a positive trend in neurocognitive investigations, the system should undergo clinical trials before being used in real clinical settings.
Esta publicación es de máxima importancia en el ámbito educativo, puesto que es un medio idóneo y actualizado para mejorar el nivel de nuestros alumnos, erradicar el fracaso escolar y establecer planes estratégicos para mejorar la educación, incorporando la neuropsicología en los centros educativos. El conocimiento del cerebro tiene gran importancia porque es clave que los psicólogos y educadores conozcan cómo evoluciona el sistema a nivel neuropsicológico sobre el que intervienen cuando se educa (García-Moreno, 2014) y autores como Howard-Jones (2011), proponen la comprensión de la significación educativa de los descubrimientos científicos; en este sentido la neuropsicología educativa favorece la prevención, el desarrollo y la atención específica a cada uno de los alumnos (Martín-Lobo, 2012). En esta línea de actuación, tal y como dice Carew y Magsamen (2010), la colaboración entre neuropsicólogos, neurocientíficos y educadores dará el fruto de una mejor educación de niños y jóvenes para una sociedad mejor preparada para el futuro, tal y como ya se está llevando a cabo en diferentes centros educativos a nivel nacional e internacional
Full-text available
Virtual reality (VR) technology offers new opportunities for the development of innovative neuropsychological assessment and rehabilitation tools. VR-based testing and training scenarios that would be difficult, if not impossible, to deliver using conventional neuropsychological methods are now being developed that take advantage of the assets available with VR technology. If empirical studies continue to demonstrate effectiveness, virtual environment applications could provide new options for targeting cognitive and functional impairments due to traumatic brain injury, neurological disorders, and learning disabilities. This article focuses on specifying the assets that are available with VR for neuropsychological applications along with discussion of current VR-based research that serves to illustrate each asset. VR allows for the precise presentation and control of dynamic multi-sensory 3D stimulus environments, as well as providing advanced methods for recording behavioural responses. This serves as the basis for a diverse set of VR assets for neuropsychological approaches that are detailed in this article. We take the position that when combining these assets within the context of functionally relevant, ecologically valid virtual environments, fundamental advancements can emerge in how human cognition and functional behaviour is assessed and rehabilitated.
Traumatic brain injury (TBI) is widespread and leads to death and disability in millions of individuals around the world each year. Overall incidence and prevalence of TBI are likely to increase in absolute terms in the future. Tackling the problem of treating TBI successfully will require improvements in the understanding of normal cerebral anatomy, physiology, and function throughout the lifespan, as well as the pathological and recuperative responses that result from trauma. New treatment approaches and combinations will need to be targeted to the heterogeneous needs of TBI populations. This article explores and evaluates the research evidence in areas that will likely lead to a reduction in TBI-related morbidity and improved outcomes. These include emerging assessment instruments and techniques in areas of structural/chemical and functional neuroimaging and neuropsychology, advances in the realms of cell-based therapies and genetics, promising cognitive rehabilitation techniques including cognitive remediation and the use of electronic technologies including assistive devices and virtual reality, and the emerging field of complementary and alternative medicine.
Conference Paper
Brain damage is a major cause of disability that often leads to deficits in executive functions with dramatic consequences on activities of daily living. While rehabilitation approaches of the dysexecutive syndrome are still limited, virtual reality has shown its potential to propose innovative intervention strategies based on ecologically valid functional tasks. The Therapeutic Virtual Kitchen (TVK) was designed as an assessment tool to help therapists and their patients with brain injury. In the TVK, some ecological virtual tasks may be configured by the therapists for patients' assessment and rehabilitation. The purpose of this paper is to validate the feasibility of the TVK with healthy subjects and patients with brain injury in Kerpape Rehabilitation Center.
Heterogeneity of studies reporting data about predictive and indicative variables of return to work after traumatic brain injury led to a lack of clarity and reliability. A meta-analysis was performed to integrate and organise the results of studies reporting data about the relation between variables that are conceptually meaningful in rehabilitation and a dichotomous criteria of work status: employed or unemployed. The large quantity of variables were grouped chronologically according pre-, peri and post-traumatic phases. The results indicate that the highest and most reliable correlations were between unemployment and executive dysfunctions, emotional disturbances, deficits in activities of daily living, and less vocational rehabilitation services. Other variables, such as anosmia and post-coma activity level, show high correlations but are still to be corroborated.
Issues of validity, and in particular ecological validity, are a current concern for tests examining disorders of executive abilities related to frontal systems dysfunction. The Behavioural Assessment of the Dysexecutive Syndrome (BADS, Wilson et al., 1996) was developed in response to the need for better neuropsychological tests in this area. The present study examines the validity of the BADS, along with six other commonly used tests of executive ability in two groups of participants with either neurological disorder (n = 36) or without brain damage (n = 37). The BADS and most of its subtests correlate significantly with the standard executive tests indicating that it possesses adequate concurrent validity. In terms of construct validity, it is comparable to standard executive tests in discriminating between neurological and non-brain-damaged participants. The ecological validity of the BADS is superior to standard executive tests in terms of predicting competency in role functioning.