ArticlePDF Available

Abstract and Figures

The purpose of this study was to establish ecological validity and initial construct validity of the Virtual Reality (VR) version of the Multiple Errands Test (MET) (Shallice & Burgess, 1991; Fortin et al., 2003) based on the NeuroVR software as an assessment tool for executive functions. In particular, the MET is an assessment of executive functions in daily life, which consists of tasks that abide by certain rules and is performed in a shopping mall-like setting where items need to be bought and information needs to be obtained. The study population included three groups: post-stroke participants (n = 5), healthy, young participants (n = 5), and healthy, older participants (n = 5). Specific objectives were (1) to examine the relationships between the performance of three groups of participants in the Virtual Multiple Errands Test (VMET) and at the traditional neuropsychological tests employed to assess executive functions and (2) to compare the performance of post-stroke participants to those of healthy, young controls and older controls in the VMET and at the traditional neuropsychological tests employed to assess executive functions.
Content may be subject to copyright.
Simona Raspelli*
Federica Pallavicini
Applied Technology for
Neuropsychology Laboratory
Istituto Auxologico Italiano
Milan, Italy
Laura Carelli
Department of Human Sciences
University of Bergamo
Bergamo, Italy
Department of Neurology and
Laboratory of Neuroscience
‘‘Dino Ferrari’’ Center
University of Milan
IRCCS Istituto Auxologico Italiano
Milan, Italy
Francesca Morganti
Department of Human Sciences
University of Bergamo
Bergamo, Italy
Elisa Pedroli
Pietro Cipresso
Applied Technology for
Neuropsychology Laboratory
Istituto Auxologico Italiano
Milan, Italy
Barbara Poletti
Barbara Corra
Davide Sangalli
Vincenzo Silani
Department of Neurology and
Laboratory of Neuroscience
‘‘Dino Ferrari’’ Center
University of Milan
IRCCS Istituto Auxologico Italiano
Milan, Italy
Giuseppe Riva
Applied Technology for
Neuropsychology Laboratory
Istituto Auxologico Italiano
Milan, Italy
Department of Psychology
Catholic University of Milan
Milan, Italy
Validating the Neuro VR-Based
Virtual Version of the Multiple
Errands Test: Preliminary Results
The purpose of this study was to establish ecological validity and initial construct valid-
ity of the virtual reality version of the Multiple Errands Test based on NeuroVR soft-
ware as an assessment tool for executive functions. In particular, the Multiple Errands
Test is an assessment of executive functions in daily life which consists of tasks that
abide by certain rules and is performed in a shopping mall-like setting where there are
items to be bought and information to be obtained. The study population included
three groups: post-stroke participants (n¼9), healthy young participants (n¼10),
and healthy older participants (n¼10). The general purpose of the study was investi-
gated through the following specific objectives: (1) to examine the relationships
between the performance of three groups of participants in the Virtual Multiple
Errands Test (VMET) and in the traditional neuropsychological tests employed to
assess executive functions; and (2) to compare the performance of post-stroke partici-
pants to those of healthy young and older controls in the Virtual Multiple Errands Test
and in the traditional neuropsychological tests employed to assess executive functions.
Correlations between Virtual Multiple Errands Test variables and some traditional ex-
ecutive functions measures provide preliminary support for the ecological and con-
struct validity of the VMET; further performance obtained at the Virtual Multiple
Errands Test provided a distinction between the clinical and healthy population, and
between the two age control groups. These results suggest a possible future applica-
tion of such an ecological approach for cognitive assessment and rehabilitation of
stroke patients and elderly population with age-related cognitive decline.
1 Introduction
The term executive functions is an umbrella term comprising a wide range
of cognitive processes and behavioral competencies which include verbal rea-
soning, problem solving, planning, sequencing, the ability to sustain attention,
resistance to interference, utilization of feedback, multitasking, cognitive flexi-
bility, and the ability to deal with novelty (Chan, Shum, Toulopoulou, & Chen,
Impairments of executive functions are common in neurological patients, in
particular, those with frontal lobe damage consequent to traumatic brain injury
and stroke (Baddeley & Wilson, 1988; Burgess, Veitch, Costello, & Shallice,
2000; Shallice & Burgess, 1991). Stroke is a common cause of death, and also
Presence, Vol. 21, No. 1, Winter 2012, 31–42
ª2012 by the Massachusetts Institute of Technology *Correspondence to
Raspelli et al. 31
represents a very important cause of disability worldwide.
It is well known that stroke affects both motor and cog-
nitive aspects of human functioning, even if it is only in
recent years that studies focused on the distribution of
neuropsychological impairments in stroke patients, to-
gether with their role in predicting functional outcomes
(Nys et al., 2007). Nys et al. highlighted a disorder in ex-
ecutive functioning, abstract reasoning, verbal memory,
and/or language in 60–70% of the patients.
Individuals who suffer from executive function impair-
ments present different problems, such as those of start-
ing and stopping activities or the inability of mental and
behavioral shifts; and demonstrate difficulties in activities
of daily living (ADL) and instrumental activities of daily
living (IADL; Shallice & Burgess, 1991; Chevignard
et al., 2000; Fortin, Godbout, & Braun, 2003). In par-
ticular, McDowd, Filion, Pohl, Richards, and Stiers
(2003) highlighted the role of attentional functioning,
particularly divided attention and switching attention, in
functional outcome and successful rehabilitation after
stroke. Attentional abilities are not only a pure mental
process, but play an important role in daily life function-
ing and in the rehabilitative process.
Studies performed in recent years revealed that deficits
in executive function in the healthy elderly population
appear to be associated with aging of the prefrontal cor-
tex (Raz, 2000).
The assessment and rehabilitation of executive func-
tions has been generally performed under typical clinical
or laboratory settings, usually via pen and paper tasks
rather than being presented in an actual or simulated
manner. This aspect, together with the fact that oppor-
tunities for choices and decision-making are less available
to patients within the clinical setting, makes these condi-
tions unsatisfactory (Burgess et al., 2006; Lo Priore, Cas-
telnuovo, & Liccione, 2003).
Indeed, the lack of ecological validity has been an im-
portant criticism for experimental tasks and traditional
neuropsychological tests (Goldstein, 1996; Sbordone,
1996). Although many patients with frontal lobe lesions
have been found to perform equally well, compared to
controls, on traditional neuropsychological tests, they
still experienced a lot of difficulty in everyday life activ-
ities (Shallice & Burgess, 1991). Conventional experi-
mental tasks demand relatively simple responses to single
events. On the contrary, more complex multi-step tasks
in daily life may require a more complicated series of
responses: goal setting and subgoals setting, prioritiza-
tion of subgoals, triggering prospective memory to initi-
ate subtasks when the conditions for them become ripe,
and inhibition of irrelevant and inappropriate actions to
different subtasks (Chan et al., 2008).
For these reasons, increasing the ecological validity
of neuropsychological assessment is important, since
this will increase the likelihood that a patient’s cognitive
and behavioral responses will replicate the response that
would occur in real-life situations (Burgess et al., 2006).
Regrettably, the assessment of executive function
during typical daily life activities is still difficult (Rand,
Rukan, Weiss, & Katz, 2009). Laboratory-based
evaluations which simulate real-life tasks, such as the Be-
havioral Assessment of Dysexecutive Syndrome (BADS;
Wilson, Alderman, Burgess, Emslie, & Evans, 1996) do
exist. However, although the BADS has good validity
(Wilson, Evans, Emslie, Alderman, & Burgess, 1998)
and was recently found to predict function in a chronic
schizophrenic sample (Katz, Tadmor, Felzen, &
Hartman-Maeir, 2007), it still does not measure per-
formance during real-life tasks, but rather during simu-
lated life tasks. Functional instruments have been devel-
oped to assess executive function in real-life. One
example is the Multiple Errands Test (MET; Alderman,
Burgess, Knight, & Henman, 2003; Burgess et al.;
Knight, Alderman, & Burgess, 2002; Shallice & Bur-
gess, 1991), which is performed at a real shopping mall
or in a hospital environment and involves the comple-
tion of various tasks, rules to adhere to, and a specified
time frame. The assessment of executive functions in
real-life settings has the advantage of giving a more
accurate estimate of the patient’s deficits than is
possible within laboratory conditions (Burgess et al.).
However, it is time-consuming and not always feasible
in typical clinical settings (Chevignard et al., 2000;
Fortin et al., 2003). Even the simplified versions of
the MET, adapted especially to be performed in a
hospital setting or in a nearby shopping mall, are time-
consuming, primarily since patients must be taken to
the setting where the assessment will be carried out and
should be able to walk independently in order to per-
form the assessment.
The application of virtual reality and the use of simu-
lated environments, perceived by the user as comparable
to real-world objects and situations, in the assessment
and rehabilitation of executive functions may help to
tackle the issues of ecological validity discussed earlier
and overcome these limits (Chan et al., 2008; Rizzo &
Kim, 2005).
Virtual reality’s potential for rehabilitation assessment
and intervention in general, and for cognitive rehabilita-
tion specifically, is due to a number of unique attributes
(Brooks & Rose, 2003; Riva et al., 2009; Rizzo & Kim,
2005): besides the opportunity for experiential, active
learning which encourages and motivates the participant,
the ability to objectively measure behavior in challenging
but safe and ecologically-valid environments, while
maintaining strict experimental control over stimulus
delivery and measurement (Rizzo, Buckwalter, & Van
der Zaag, 2002).
However, as underlined by Chan et al. (2008), this
advanced technology should be carefully implemented in
clinical setting, in particular with regard to those patients
who are not familiar with computerized tests or who are
anxious in undertaking a computerized test or in being
tested in a semi-enclosed environment (Browndyke
et al., 2002; Wiechmann & Ryan, 2003).
Rand, Rukan, Weiss, and Katz (2009) have developed
a first version of the Virtual Multiple Errands Test
(VMET) as an assessment tool for executive functions,
within the virtual mall (Rand, Katz, Shahar, Kizony, &
Weiss, 2005), a functional virtual environment currently
consisting of a large supermarket which was pro-
grammed via GestureTek’s IREX video capture virtual
reality system. It was developed to provide post-stroke
participants with the opportunity to engage in a com-
plex, everyday task of shopping, in which their weak
upper extremity and executive functions deficits can be
trained. The VMall has been shown to provide an inter-
esting and motivating task which can offer post-stroke
participants an opportunity to practice different shop-
ping tasks without leaving the treatment room (Rand,
Katz, & Weiss, 2007); in this study, the VMall was found
to be sensitive to differences in the shopping time and
number of mistakes between post-stroke participants,
healthy young participants, and healthy older partici-
pants on a simple shopping task of four items (the Four-
Item Test; Carelli et al., 2009; Raspelli et al., 2010).
The purpose of the current study was to establish eco-
logical validity and initial construct validity of the VR
version of the MET (Shallice & Burgess, 1991; Fortin
et al., 2003), based on NeuroVR software as an assess-
ment tool for executive functions. As opposed to
GestureTek’s IREX, which includes a camera which films
the user and displays his or her image within the virtual
environment and where the interaction is done using
active movements, in NeuroVR, the participant is able
to freely navigate in the various aisles with the aid of a
joypad and to collect products (by pressing a button
placed on the right side of the joypad), after having
selected them with the viewfinder.
The study population included three groups: post-
stroke participants, healthy young participants, and
healthy older participants. The specific aims were to
examine the relationships between the performance of
three groups of participants in the VMET and at the tra-
ditional neuropsychological tests employed to assess ex-
ecutive functions and to compare the performance of
post-stroke participants to those of healthy young and
older controls in the VMET and in the traditional
neuropsychological tests employed to assess executive
2 Method
2.1 Participants
A total of 29 participants in three groups were
included in the study, 9 post-stroke individuals and 20
healthy people in two age groups. The 9 stroke partici-
pants ranged in age from 50 to 70 years (mean age 62
years; SD 7.83). In addition, 20 healthy participants vol-
unteered to participate in this study, including 10 young
participants with an age range between 20 and 30 years
(mean age 26 years; SD 1.95) and 10 older participants
with an age range between 50 and 70 years (mean age
55 years; SD 6.03). All groups were fully independent in
activities of daily living and instrumental activities of
Raspelli et al. 33
daily living. Their demographic data are presented in
Table 1.
Patients were excluded from the study who had a severe
cognitive impairment (MMSE 18/30) (M. F. Folstein,
S. E. Folstein, & McHugh, 1975), a severe motor impair-
ment which does not allow performance of the dual task
procedure (Rand, Katz, Shahar, Kizony, & Weiss, 2005),
auditory language comprehension difficulties (score at the
ENB Token Test 26.5; Mondini, Mapelli, Vestri, &
Bisiacchi, 2003), visual recognition impairments (score
on Street’s Completion Test 2.25/14; Spinnler &
Tognoni, 1987), and spatial hemi-inattention and neglect
as assessed by the Star Cancellation Test within the
Behavioural Inattention Test (BIT; Wilson, Cockburn, &
Halligan, 1987). Patients also underwent an exhaustive
traditional neuropsychological assessment.
Control subjects were excluded from the study who
had a cognitive impairment (MMSE 24/30) (Folstein
et al., 1975), a motor impairment which does not allow
performance of the dual task procedure (Rand, Katz
et al., 2005), auditory language comprehension difficul-
ties (score at the ENB Token Test 26.5; Mondini
et al., 2003), visual recognition impairments (score on
Street’s Completion Test 2.25/14; Spinnler & Tog-
noni, 1987), and spatial hemi-inattention and neglect as
assessed by the Star Cancellation Test within the BIT
(Wilson et al., 1987).
2.2 Instruments
2.2.1 The neuropsychological evaluation. A
neuropsychological evaluation was conducted both for
inclusion criteria and data collecting referred to patients’
cognitive profile.
In particular, the following neuropsychological tests
were employed: the Mini-Mental State Evaluation (M.
F. Folstein et al., 1975), to assess the general cognitive
level; the Star Cancellation Test within the BIT (Wilson
et al., 1987), for visuo-spatial assessment; the Token
Test within the Brief Neuropsychological Examination
(ENB; Mondini et al., 2003), for auditory language
comprehension difficulties; and Street’s Completion
Test, for object recognition and denomination
(Spinnler & Tognoni, 1987).
As for the assessment of executive functions, the Test
of Attentional Performance (TEA) (Zimmerman &
Fimm, 1992) was employed with specific subtests to
evaluate the ability to increase, waiting for a high priority
stimulus, the level of attention and ability to keep it (a
state of alertness); to pay attention to and to elaborate
different information present at the same time (divided
attention); to sustain selective attention (sustained atten-
tion); to control for input from different sensorial chan-
nel (intermodal comparison); to explore the visual field
(visual exploration); to change the attentive focus (flexi-
bility); to repress an inadequate reaction (go/no go); to
focus attention, that is, the ability to reject irrelevant
aspects of the stimulus (spatial incompatibility); to con-
tinuously control for the information flow through short
term memory (working memory), and to shift the visual
attentive focus without eye movements (attention shift).
In addition, for the assessment of executive functions,
the Stroop Colour-Word Test (Stroop, 1935) was
employed to evaluate frontal and inhibition abilities; the
Iowa Gambling Task (Bechara et al., 1994), to evaluate
the functional integrity of the orbito-frontal areas,
through the simulation, in real time, of the personal abil-
ity in decision-making relative to the uncertainty of the
premises and of their outcome, like a reward or a punish-
ment; finally, the Dysexecutive Questionnaire (DEX;
Wilson et al., 1998), to evaluate executive functions in
everyday life.
The Activities of Daily Living (ADL) and Instrumental
Activities of Daily Living (IADL) Tests (Katz, Ford,
Moskowitz, Jackson, & Jaffe, 1963) were employed to
assess activities of daily living, such as having a bath or
getting dressed; and instrumental activities of daily liv-
ing, such as using the telephone, doing some shopping,
Table 1. Population Characteristics, Mean (SD)
Patients (n¼9) 62 (7.83) 12.78 (3.56)
Adult control
group (n¼10)
55 (6.03) 14 (2.1)
Young control
group (n¼10)
26 (1.94) 17.1 (1.44)
and keeping house, while the State and Trait Anxiety
Index (STAI; Spielberger, Gorsuch, & Lushene, 1970)
and the Beck Depression Inventory (BDI; Beck, Ward,
Mendelson, Mock, & Erbaugh, 1961) were used to eval-
uate the level of state and trait anxiety and of depression.
2.2.2 The Virtual Multiple Errands Test. The
virtual environment employed in this study is a super-
market developed via NeuroVR
software and displayed
on a desktop monitor. It consists of a Blender-based
that enables active exploration of a virtual
supermarket where users are requested to select and buy
various products presented on shelves (see Figure 1).
The user enters the supermarket and is presented with
icons of the various items to be purchased.
With the aid of a joypad, the participant is able to
freely navigate in the various aisles (using the up-down
joypad arrows), and to collect products (by pressing a
button placed on the right side of the joypad), after hav-
ing selected them with the viewfinder. The virtual super-
market contains products grouped into the main grocery
categories including beverages, fruits and vegetables,
breakfast foods, hygiene products, frozen foods, garden
products, and animal products. Signs at the top of each
section indicate the product categories as an aid to navi-
The original procedure of the Multiple Errands Test
(MET; Shallice & Burgess, 1991) was modified to be
adapted to the virtual scenario of the supermarket. It
consists of some tasks (to buy some products from
a shop and to obtain some information) that are
performed in a mall-like setting or shopping center and
abide by certain rules (e.g., to carry out all tasks but in
any order; not to go into the same aisle more than
once; not to buy more than two items per category of
In order to evaluate the VMET usability, a Likert scale
ranging from 1 to 10 points was created to assess the
dimensions of knowledge of technological means (com-
puter; video games; joystick, and virtual reality); usability
of the interface (difficulties during the experience; in
using the joystick; in selecting products from aisle and in
learning to move in the supermarket), and environmen-
tal content (difficulties in recognizing products on the
aisle; evaluation of the products’ organization in the
supermarket; visibility of the signs at the top of each sec-
2.3 Procedure
Participants were included in the study after a pre-
liminary neuropsychological evaluation.
According to this preliminary evaluation and the
exclusion criteria described above, participants who were
deemed suitable for the study underwent a more exhaus-
Figure 1. Two screen shots of the virtual supermarket.
Raspelli et al. 35
tive neuropsychological assessment in order
to obtain an accurate overview of their cognitive func-
tion. Moreover, participants were asked to complete
the Virtual Multiple Errands Test (VMET) procedure af-
ter a training session. Indeed, a training period of about
15 min was first provided in a smaller version of the vir-
tual supermarket environment in order to familiarize
participants with both the navigation and shopping
Two sessions of about 60 min were scheduled for
each patient; during the first session they underwent
the complete neuropsychological assessment, while
during the second session (held the following day) the
VMET procedure within the virtual supermarket and
TEA were administered. The order of administration of
the two most time-consuming tests, VMET and TEA,
was counterbalanced in the sample so that for half of
the participants VMET was administered before TEA
and for the other half TEA was administered before
2.3.1 Outcome Measures. Neuropsychological
tests scores were recorded and corrected for age, educa-
tion level, and gender.
While completing the VMET procedure, the time of
execution, total errors, partial task failures, inefficiencies,
rule breaks, strategies, and interpretation failures were
recorded. These are defined as follows.
Errors. Errors are task failures that are omissions
(defined as failing to attempt the task). For errors in
executing the tasks, the scoring range was from 11
(the subject has correctly done the 11 tasks) to 33
(the subject has totally omitted the 11 tasks). In par-
ticular, the scoring scale for each task failure was
from 1 to 3 (1 ¼the participant performed the task
100% correctly as indicated by the test; 2 ¼the par-
ticipant performed aspects of the task, but the task
was not completed 100% accurately; 3 ¼the partici-
pant totally omitted the task).
Inefficiencies. Inefficiencies are defined as a failure
to do more than one thing in one place when that is
the only place to accomplish that task. Examples of
the eight inefficiencies are not grouping tasks or not
reading the instructions. For inefficiencies, the
scoring range was from 8 (great inefficiencies) to 32
(no inefficiencies). In particular, the scoring scale for
each inefficiency was from 1 to 4 (1 ¼always; 2 ¼
more than once; 3 ¼once; 4 ¼never).
Rule Breaks. Rule breaks are defined as anything
that violates the rules listed in the MET task list.
Examples of the eight rule breaks are entering an
area more than once or speaking to the examiner
when not necessary. For rule breaks, the scoring
range was from 8 (a large number of rule breaks) to
32 (no rule breaks). In particular, the scoring scale
for each rule break was from 1 to 4 (1 ¼always; 2 ¼
more than once; 3 ¼once; 4 ¼never).
Strategies. Examples of the 13 strategies are plan-
ning before starting the tasks and marking off the
tasks completed. The scoring range was from 13
(good strategies) to 52 (no strategies). In particular,
the scoring scale for each strategy was from 1 to 4
(1 ¼always; 2 ¼more than once; 3 ¼once; 4 ¼
Interpretation Failures. Interpretation failures
offer insight into the type of errors and interpreta-
tion failures experienced by the subject in the testing
situation. An example of the three interpretation
failures is thinking that the tasks all had to be done
in the order presented in the task list. The scoring
range was from 3 (a large number of interpretation
failures) to 6 (no interpretation failures). In particu-
lar, the scoring scale for each interpretation failure
was from 1 to 2 (1 ¼yes; 2 ¼no).
Partial Task Failures. For partial task failures, the
scoring range was from 8 (no errors) to 16 (a large
number of errors). As for partial task failures, the
eight specific items, with a scoring range from
1 (yes) to 2 (no) for each one were: searched for
item in the correct area; maintained task objective to
completion; maintained sequence of the task; di-
vided attention between components of task and
components of other VMET tasks; organized mate-
rials appropriately throughout task; self-corrected
upon errors made during the task; no evidence of
perseveration; and sustained attention throughout
the sequence of the task (not distracted by other
3 Results
Data analysis was carried out using SPSS for Win-
dows, version 17.0. Due to the small group sample size,
nonparametric statistics were used. Specifically, Pearson’s
correlation coefficients were used to examine the rela-
tionships between the various scores of the neuropsycho-
logical tests employed to assess executive functions and
the scores of the VMET for each group separately.
The comparison of the scores of the neuropsychologi-
cal tests employed to assess executive functions and
VMET between the post-stroke participants and both
groups of healthy controls was performed using the
Kruskal–Wallis procedure; in the case of significant dif-
ferences, the Mann–Whitney procedure was used to
determine the source of significance between each pair in
the groups.
3.1 Descriptive Statistics
Table 2 shows the descriptive statistics with regard
to the neuropsychological evaluation conducted for
inclusion criteria for the three examined groups: patients
and the two control groups (young and adult).
3.2 Correlations
With the aim of analyzing the relationships among
the scores at VMET and at traditional tools for the mea-
surement of executive functions within the patients
group and the control groups, correlations analyses were
carried out.
For patients, the following correlations emerged as
significant between the VMET variables and the reaction
times in some tests of the TEA.
The time of execution of the VMET with the test
for the state of alert with warning sign (r¼.762,
p¼.028); the test of the intermodal comparison
(r¼.81, p¼.007); and the test of audio divided
attention (r¼.71, p¼.047).
Total errors in VMET with the test of incompatibil-
ity (with time reaction as the measure; r¼.75, p¼
Table 2. Descriptive Statistics for the Three Groups, Mean (SD)
Patients (n¼9) 27.86 (1.62) 53.44 (1.13) 3.89 (2.54) 5 (.00) 10.71 (12.43) 47.5 (15.34) 39 (11.13) 6 (.00) 8 (.00)
Adult control
group (n¼10)
28.92 (1.26) 54.00 (.00) 6.35 (2.5) 5 (.00) 6.2 (5.47) 40.3 (8.09) 36.4 (8.38) 6 (.00) 7.96 (.89)
Young control
group (n¼10)
29.7 (.92) 54.00 (.00) 8.55 (2.95) 5 (.00) 6.5 (8.4) 42.9 (14.38) 37.4 (13.57) 6 (.00) 8 (.00)
Raspelli et al. 37
Inefficiencies in VMET with the test of attention
shift with valid stimulus (r¼.67, p¼.045) and
without valid stimulus (r¼.73, p¼.026).
For the control group made of adult participants, sig-
nificant correlations emerged among some of the VMET
variables; in particular, inefficiencies and rule breaks (r¼
.76, p¼.01) and between the time of execution and
total errors (r¼.678, p¼.031).
Finally, for the control group made of young partici-
pants, significant correlations emerged between interpre-
tation failures and visual exploration with noncritical
stimulus (r¼.63, p¼.04) and the time of execution
(r¼.71, p¼.023) and among some of the VMET
variables and in particular between the time of execution
and inefficiencies (r¼.76, p¼.013), between ineffi-
ciencies and rule breaks (r¼.82, p¼.004), and inter-
pretation errors (r¼.81, p¼.005).
3.3 Kruskal–Wallis and Mann–Whitney
With the aim of analyzing the differences among
the examined groups in VMET and in tests measuring
executive functions, and in order to study the construct
validity, the Kruskal–Wallis Test was first performed;
finally, the Mann–Whitney Test for the direction of the
results was performed.
Table 3 shows the emerged significant results.
4 Discussion
The construct validity of the VMET has been dem-
onstrated by the significant correlations that emerged
among the VMET and different tests employed for the
measurement of executive functions within the groups
of patients and healthy subjects. More specifically, sig-
nificant correlations are those between the VMET and
the scores of tests of TEA which measure executive func-
tions within the different groups: intermodal compari-
son, spatial incompatibility, and divided attention.
These tests are very sensitive to the correct ability to
reject trivial aspects of stimuli (spatial incompatibility
test) and to pay attention to and elaborate on different
information present at the same time (divided attention
test); these are all basic components of executive
A result opposite to expectations was the lack of signif-
icant correlations between variables of the VMET and
the traditional tests measuring executive functions in the
examined groups, the Stroop Test and the Iowa Gam-
bling Task.
With regard to the lack of correlation with the Iowa
Gambling Task, it is important to underline that differ-
ent processes are involved into the two tests: while the
VMET requires adherence to specific rules, the Iowa
Gambling Task, which represents a test of affective deci-
sion-making, simulates in real time the ability to make
decisions with uncertainty of premises and results, and
lack of rules.
The lack of correlation between the VMET and the
Stroop Test, but also between the VMET and the Test
of Flexibility (TEA) in the different experimental groups
could be explained considering that in the VMET proce-
dure, there are nonconflicting instructions. In the future,
the possibility to introduce supermarket announcements
can be considered, in order to introduce information
conflicting with the main task.
Finally, significant differences emerged among the
three groups on some measures of the VMET and the
other tests traditionally employed for the assessment of
executive functions. As expected, patients made the
greater number of errors, followed by adult and young
control subjects (Raz, 2000). Moreover, patients
employed the longest time for executing the VMET.
This result could be interpreted considering the Likert
scores for the assessment of the usability of the VMET.
Indeed, contrary to patients and to adult control sub-
jects, the 42% of the young control subjects had previ-
ously used virtual reality systems.
Moreover, 33% of patients had no previous knowledge
of computers, while 28% of adult control subjects
showed knowledge between 7 and 8 on the Likert score.
Among those patients who had previously used a com-
puter, the level of knowledge was low (less than 5).
Finally, patients showed the slowest reaction time in
some tests of TEA battery, followed by the control
groups, specifically in working memory, intermodal
Table 3. Kruskal–Wallis and Mann–Whitney Tests*
Kruskal–Wallis H
Adult control
Young control
group v
(p) Mann–Whitney
Time VMET 1102 (920) 645.00 (189) 433.7 (102.06) 13.6 .001 Patients <adults <
Errors VMET 17.67 (4.09) 13.8 (1.81) 13.6 (1.26) 9.72 .008 Patients <adults <
Mean intermodal
comparison TEA
511.67 (75.72) 427.8 (93.53) 421.5 (76.42) 7.6 .022 Patients <adults <
Mean flexibility TEA 1401.44 (574.87) 803.8 (150.25) 660.05 (143.25) 13.7 .001 Patients <adults <
Mean spatial
incompatibility TEA
556 (81.48) 533.4 (87.12) 437.1 (56.63) 10.3 .006 Patients <adults <
Mean attention shift
with nonvalid stimulus TEA
534.56 (225.22) 421.5 (85.29) 302.5 (37.34) 12.09 .002 Patients <adults <
Mean attention shift with
valid stimulus TEA
424.56 (110.53) 342 (49.22) 277.5 (34.47) 13.45 .001 Patients <adults <
Mean working memory TEA 910.11 (338.93) 596.6 (143.66) 566.8 (109.01) 7.65 .022 Patients <adults <
*For the patients, the adult control group, and the young control group, values given are mean (SD).
Raspelli et al. 39
comparison, flexibility, spatial incompatibility, attention
shift, and working memory tests (see Table 3).
5 Conclusions
Given the above mentioned results, it is possible to
conclude that the VMET appeared sensible both to brain
damage and aging, as already observed in previous stud-
ies with the original version of the MET.
The results of the present study may have important
implications for rehabilitation of patients with cerebral
lesions following stroke with regard to the possible eco-
logical usefulness of the VR-based tools. Indeed, the lit-
erature shows many studies on damage in executive
functions following stroke, employing functional mag-
netic resonance (fMRI), magnetic resonance (MRI), and
clinical and functional measurements. However, little
attention has been focused on ecological issues in the
rehabilitation processes of stroke patients.
However, due to the small sample size, the obtained
results should be considered preliminary: as an assess-
ment tool, the VMET should be analyzed with regard
to its temporal stability, namely test–retest reliability
and criterion validity, before being ready for clinical
application and for being applied to different clinical
To conclude, this study provides preliminary data sup-
porting the ecological and construct validity of the
VMET as an assessment tool of executive functions and
its role in differentiating between stroke patients and
controls and between different age groups, with regard
to the healthy population.
The authors thank Professor Patrice L. Weiss, Rachel Kizony,
and Noomi Katz of the Department of Occupational Therapy of
the University of Haifa (Israel) and of the Research Institute for
the Health & Medical Professions of the Ono Academic College
of Kiryat Ono (Israel), since the results of the study are part of
an ongoing collaboration between our research groups. The
work in preparing this paper was supported by the project
‘‘Immersive Virtual Telepresence (IVT) for Experiential Assess-
ment and Rehabilitation,’’ IVT2010, RBIN04BC5C.
Alderman, N., Burgess, P. W., Knight, C., & Henman, C.
(2003). Ecological validity of a simplified version of the Mul-
tiple Errands Shopping Test. Journal of the International
Neuropsychological Society, 9, 31–44.
Baddeley, A. D., & Wilson, B. (1988). Frontal amnesia and
the dysexecutive syndrome. Brain and Cognition, 7,
Bechara, A., Dama
´sio, A. R., Dama
´sio, H., & Anderson, S. W.
(1994). Insensitivity to future consequences following dam-
age to human prefrontal cortex. Cognition, 50(1–3), 7–15.
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., &
Erbaugh, J. (1961). An inventory for measuring depression.
Archives of General Psychiatry, 4, 561–571.
Brooks, B. M., & Rose, F. D. (2003). The use of virtual reality
in memory rehabilitation: Current findings and future direc-
tions. Neurorehabilitation, 18, 147–157.
Browndyke, J. N., Albert, A. L., Malone, W., Schartz, P., Paul,
R. H., Cohen, R. A., et al. (2002). Computer-related anxi-
ety: Examining the impact of technology-specific affect on
the performance of a computerized neuropsychological
assessment measure. Applied Neuropsychology, 9(4),
Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates,
L. M. A., Dawson, D. R., Anderson, N. D., Gilbert, S. J.,
Dumontheil, I., & Channon, S. (2006). The case for the de-
velopment and use of ‘‘ecologically valid’’ measures of execu-
tive function in experimental and clinical neuropsychology.
Journal of the International Neuropsychological Society, 12,
Burgess, P. W., Veitch, E., Costello, A., & Shallice, T. (2000).
The cognitive and neuroanatomical correlates of multitask-
ing. Neuropsychologia, 38, 848–863.
Carelli, L., Morganti, F., Poletti, B., Corra
`, B., Weiss, P. L.,
Kizony, R., Silani, V., & Riva, G. (2009). A NeuroVR based
tool for cognitive assessment and rehabilitation of post-
stroke patients: Two case studies. Studies in Health Technol-
ogy Informatics, 144, 243–247.
Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H.
(2008). Assessment of executive functions: Review of instru-
ments and identification of critical issues. Archives of Clinical
Neuropsychology, 23, 201–216.
Chevignard, M., Pillon, B., Pradat-Diehl, P., Taillefer, C.,
Rousseau, S., Le Bras, C., & Dubois, B. (2000). An ecologi-
cal approach to planning dysfunction: Script execution.
Cortex, 36, 649–669.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975).
‘‘Mini-mental state’’: A practical method for grading the
mental state of patients for the clinician. Journal of Psychiat-
ric Research, 12(3), 189–198.
Fortin, S., Godbout, L., & Braun, C. M. J. (2003). Cognitive
structure of executive deficits in frontal lesioned head trauma
patients performing activities of daily living. Cortex, 39(2),
Goldstein, G. (1996). Functional considerations in neuropsy-
chology. In R. J. Sbordone & C. J. Long (Eds.), Ecological
validity of neuropsychological testing (pp. 75–89). Delray
Beach, FL: GR Press/St. Lucie Press.
Katz, S., Ford, A. B., Moskowitz, R. W., Jackson, B. A., &
Jaffe, M. W. (1963). Studies of illness in the aged. The index
of ADL: A standardized measure of biological and psychoso-
cial function. JAMA, 185, 914–919.
Katz, N., Tadmor, I., Felzen, B., & Hartman-Maeir, A. (2007).
The Behavioral Assessment of the Dysexecutive Syndrome
(BADS) in schizophrenia and its contribution to functional
outcome. Neuropsychological Rehabilitation, 17,192205.
Knight, C., Alderman, N., & Burgess, P. W. (2002). Develop-
ment of a simplified version of the Multiple Errands Test for
use in hospital settings. Neuropsychological Rehabilitation,
12, 231–255.
Lo Priore, C., Casterlnuovo, G., Liccione, D., & Liccione, D.
(2003). Experience with VSTORE: Considerations on pres-
ence in virtual environments for effective neuropsychological
rehabilitation of executive functions. CyberPsychology and
Behavior, 6(3), 281–287.
McDowd, J. M., Filion, D. L., Pohl, P. S., Richards, L. G., &
Stiers, W. (2003). Attentional abilities and functional out-
comes following stroke. Journal of Gerontology, 58, 45–53.
Mondini, S., Mapelli, D., Vestri, A., & Bisiacchi, P. (2003).
Esame Neuropsicologico Breve. Milan, Italy: Raffaello Cortina
Nys, G. M. S., van Zandvoort, M. J. E., de Kort, P. L. M., Jan-
sen, B. P. W., de Haan, E. H. F., & Kappelle, L. J. (2007).
Cognitive disorders in acute stroke: Prevalence and clinical
determinants. Cerebrovascular Disease, 23, 408–416.
Rand, D., Katz, N., Shahar, M., Kizony, R., & Weiss, P. L.
(2005). The virtual mall: A functional virtual environment
for stroke rehabilitation. Annual Review of Cybertherapy and
Telemedicine (ARCTT): A Decade of VR, 3, 193–198.
Rand, D., Katz, N., & Weiss, P. L. (2007). Evaluation of vir-
tual shopping in the VMall: Comparison of post-stroke par-
ticipants to healthy control groups. Disability and Rehabili-
tation, 13, 1–10.
Rand, D., Rukan, S. B., Weiss, P. L., & Katz, N. (2009). Vali-
dation of the virtual MET as an assessment tool for executive
functions. Neuropsychological Rehabilitation, 19(4), 583–
Raspelli, S., Carelli, L., Morganti, F., Poletti, B., Corra, B.,
Silani, V., & Riva, G. (2010). Implementation of the Multi-
ple Errands Test in a NeuroVR-supermarket: A possible
approach. Annual Review of CyberTherapy and Telemedicine,
8, 92–95.
Raz, N. (2000). Aging of the brain and its impact on cognitive
performance: Integration of structural and functional find-
ings. In F. I. M. Craik & T. A. Salthouse (Eds.), The hand-
book of aging and cognition (2nd ed.) (pp. 1–90). Mahwah,
NJ: Lawrence Erlbaum.
Riva, G., Carelli, L., Gaggioli, A., Gorini, A., Vigna, C., Algeri,
D., Repetto, C., Raspelli, S., Corsi, R., Faletti, G., & Vezza-
dini, L. (2009). NeuroVR 1.5 in practice: Actual clinical
applications of the open source VR system. Studies in Health
Technology and Informatics, 144, 57–60.
Rizzo, A. A., Buckwalter, J. C., & Van der Zaag, C. (2002).
Virtual environment applications in clinical neuropsychol-
ogy. In K. Stanney (Ed.), The handbook of virtual environ-
ments (pp. 1027–1064). New York: Lawrence Erlbaum
Associates, Inc.
Rizzo, A. A., & Kim, G. (2005). A SWOT analysis of the field
of virtual rehabilitation and therapy. Presence: Teleoperators
and Virtual Environments, 14(2), 119–146.
Sbordone, R. J. (1996). Ecological validity: Some critical issues
for the neuropsychologist. In R. J. Sbordone & C. J. Long
(Eds.), Ecological validity of neuropsychological testing (pp.
15–41). Delray Beach, FL: GR Press/St. Lucie Press.
Shallice, T., & Burgess, P. W. (1991). Deficits in strategy appli-
cation following frontal lobe damage in man. Brain, 114,
Spielberger, C. D., Gorsuch, R. L., & Lushene, R. (1970). The
State Trait Anxiety Inventory (STAI). Test manual for form
X(R. E. Lazzari & P. Pancheri, Trans.). Palo Alto, CA: Con-
sulting Psychologists Press.
Spinnler, H., & Tognoni, G. (1987). Standardizzazione e tara-
tura italiana di test neuropsicologici. Italian Journal of Neu-
rological Sciences, 8(6), 20–120.
Stroop, J. R. (1935). Studies of interference in serial
verbal reaction. Journal of Experimental Psychology, 18,
Wiechmann, D., & Ryan, A. M. (2003). Reactions to compu-
terized testing in selection contexts. International Journal of
Selection and Assessment, 11(2–3), 215–229.
Raspelli et al. 41
Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., &
Evans, J. J. (1996). Behavioural assessment of the dysexecutive
syndrome. London: Harcourt Assessment.
Wilson, B., Cockburn, J., & Halligan, P. W. (1987). The
Behavioural Inattention Test. Bury St. Edmunds, UK:
Thames Valley Test Company.
Wilson, B. A., Evans, J. J., Emslie, H., Alderman, N., & Bur-
gess, P. (1998). The development of an ecologically valid
test for assessing patients with a dysexecutive syndrome.
Neuropsychological Rehabilitation, 8, 213–228.
Zimmerman, P., & Fimm, B. (1992). Test Batterie zur Auf-
merksamkeitspru¨fung (TAP).Wu¨rselen: Psytest.
... Overall, we found 38 articles on 31 different tools. 4 out 31 tools (12.9%) appeared in more than one study, with 3 tools each appearing in 2 distinct studies (Denmark et al., 2017;Hadad et al., 2012;Jansari et al., 2014;Lajeunesse et al., 2019;Nir-Hadad et al., 2017;Potvin et al., 2011) and one in 3 studies (Raspelli et al., 2010(Raspelli et al., , 2011(Raspelli et al., , 2013. One tool was used in 2 studies involving the same sample (Banville & Nolin, 2012;Banville et al., 2010) and the remaining large majority (83.87%) was used only in one study. ...
... Raspelli and colleagues (Raspelli et al., 2010(Raspelli et al., , 2011(Raspelli et al., , 2013 modified the original Multiple Errands Test (MET) and developed a new virtual version (Virtual Reality based Version of the Multiple Errands Test, VMET) to assess executive functions by using a shopping task in post-stroke patients. The virtual scenario was developed via NeuroVR1 software and was displayed on a desktop monitor. ...
... Moreover, they were required to get different information (e.g., the supermarket closing time) and follow specific rules (i.e. to go to the same aisle only once or to buy only 2 items per product category). Before the beginning of the task, participants took a 15-minute period to freely move in a training environment representing a food market (Raspelli et al., 2011) or a smaller version of the target virtual supermarket (Raspelli et al., 2013) in order to get familiar with navigation (the presence of a familiarization phase before the task was not specified in Raspelli et al., 2010). ...
Full-text available
Individuals with acquired brain injury (ABI) commonly present with impairments in cognitive abilities. As these competencies seem to be predictive of patients’ abilities to reintegrate into the everyday settings, it is crucial to assess them properly. However, previous research has indicated that patients may perform relatively well on standard tests of cognitive functioning, but may nonetheless encounter significant difficulties in organizing and executing everyday tasks. In order to overcome this issue, virtual reality (VR) methods have been introduced in clinical practice with the aim of creating assessments that simulate real-world activities and thus, provide a clearer picture of patients’ functioning in everyday settings. This review offers an overview of VR assessment tools described in the scientific literature between 2010 and 2019. Overall, 38 relevant records describing 31 different tools were found. Among these tools, 16 assessed executive functions and prospective memory, while the other 15 assessed visuo-spatial abilities. Although promising results have been reported, our analysis indicated that about half of the tools deliver tasks that differ from everyday activities, thus limiting the generalizability of patients’ performance to the real-world. Moreover, a variety of methodological shortfalls related to study Internal and External Validity have been highlighted, which hamper the possibility of drawing definite recommendations on tool choice. These limitations suggest the importance of putting considerable efforts into the improvement or development of VR tools for patients with ABI for both research and clinical purposes, considering the great potential of this form of assessment.
... Several research showed that assessment in these reallife scenarios provides a more accurate estimate of the patient's impairment, showing difficulties invisible to traditional measurements (Rand et al., 2009;Armstrong et al., 2013;Cipresso et al., 2013a;Parsons, 2015). Finally, several studies showed the feasibility of VR-based tool as assessment for EFs in neurological [e.g., traumatic brain injury, Parkinson's disease (PD), and multiple sclerosis (MS)] and psychiatric (i.e., schizophrenia and obsessive-compulsive disorder) populations (Josman et al., 2009;Raspelli et al., 2011;Aubin et al., 2018;Liu et al., 2019;Słyk et al., 2019). ...
Full-text available
Executive dysfunctions constitute a significant public health problem due to their high impact on everyday life and personal independence. Therefore, the identification of early strategies to assess and rehabilitate these impairments appears to be a priority. The ecological limitations of traditional neuropsychological tests and the numerous difficulties in administering tests in real-life scenarios have led to the increasing use of virtual reality (VR) and 360° environment-based tools for assessing executive functions (EFs) in real life. This perspective aims at proposing the development and implementation of Executive-functions Innovative Tool 360° (EXIT 360°), an innovative, enjoyable, and ecologically valid tool for a multidimensional and multicomponent evaluation of executive dysfunctions. EXIT 360° allows a complete and integrated executive functioning assessment through an original task for EFs delivered via a mobile-powered VR headset combined with eye tracker (ET) and electroencephalograms (EEG). Our tool is born as a 360°-based instrument, easily accessible and clinically usable, that will radically transform clinicians’ and patient’s assessment experience. In EXIT 360°, patients are engaged in a “game for health,” where they must perform everyday subtasks in 360° daily life environments. In this way, the clinicians can obtain quickly more ecologically valid information about several aspects of EFs (e.g., planning, problem-solving). Moreover, the multimodal approach allows completing the assessment of EFs by integrating verbal responses, reaction times, and physiological data (eye movements and brain activation). Overall, EXIT 360° will allow obtaining simultaneously and in real time more information about executive dysfunction and its impact in real life, allowing clinicians to tailor the rehabilitation to the subject’s needs.
... To further corroborate this point, it should be noted that considering each index within each single task would not have allowed us to capture the ecological added value of these platforms. In fact, researchers in the SG field usually consider performances in terms of global indices and not task by task (e.g., Raspelli et al., 2011;Cipresso et al., 2014;Ouellet et al., 2018). Second, we believe that each of the analyzed indices provides different information on participant performance. ...
Full-text available
Background: Smart Aging is a serious game (SG) platform that generates a 3D virtual reality environment in which users perform a set of screening tasks designed to allow evaluation of global cognition. Each task replicates activities of daily living performed in a familiar environment. The main goal of the present study was to ascertain whether Smart Aging could differentiate between different types and levels of cognitive impairment in patients with neurodegenerative disease. Methods: Ninety-one subjects (mean age = 70.29 ± 7.70 years)—healthy older adults (HCs, n = 23), patients with single-domain amnesic mild cognitive impairment (aMCI, n = 23), patients with single-domain executive Parkinson's disease MCI (PD-MCI, n = 20), and patients with mild Alzheimer's disease (mild AD, n = 25)—were enrolled in the study. All participants underwent cognitive evaluations performed using both traditional neuropsychological assessment tools, including the Mini-Mental State Examination (MMSE), Montreal Overall Cognitive Assessment (MoCA), and the Smart Aging platform. We analyzed global scores on Smart Aging indices (i.e., accuracy, time, distance) as well as the Smart Aging total score, looking for differences between the four groups. Results: The findings revealed significant between-group differences in all the Smart Aging indices: accuracy ( p < 0.001), time ( p < 0.001), distance ( p < 0.001), and total Smart Aging score ( p < 0.001). The HCs outperformed the mild AD, aMCI, and PD-MCI patients in terms of accuracy, time, distance, and Smart Aging total score. In addition, the mild AD group was outperformed both by the HCs and by the aMCI and PD-MCI patients on accuracy and distance. No significant differences were found between aMCI and PD-MCI patients. Finally, the Smart Aging scores significantly correlated with the results of the neuropsychological assessments used. Conclusion: These findings, although preliminary due to the small sample size, suggest the validity of Smart Aging as a screening tool for the detection of cognitive impairment in patients with neurodegenerative diseases.
... Since its development, multiple versions of the MET have been published including hospital versions (e.g., Knight et al., 2002;Dawson et al., 2009;Morrison et al., 2013), shopping mall versions (e.g., Alderman et al., 2003), virtual reality versions (e.g., Rand et al., 2009;Raspelli et al., 2011;Cipresso et al., 2014;Kizony et al., 2017) and most recently a home version (Burns et al., 2019). The MET-Home is the first version to be published that can be used across different sites without adaptation, greatly enhancing its utility for clinicians and researchers. ...
Full-text available
The Multiple Errands Test (MET) is an ecologically valid assessment that characterizes how executive dysfunction manifests in everyday activities. Due to the naturalistic nature of this assessment, clinicians and researchers have had to develop site-specific versions resulting in numerous published versions and making it difficult to establish standard psychometric properties. The aim of this study was to develop a standardized, community version of the MET designed to be used in large department stores meeting set criteria that would not require site specific modifications. This paper reports on the development, content validity, feasibility, and inter-rater reliability of a Big-Store MET, and the performance of healthy participants on this test. Items were selected to match previously published versions in relation to quantity and complexity. Content validity was established by having experts (n = 4) on the MET review the proposed Big-Store version and evaluate the task consistency with previously published versions. To assess feasibility of administration, and inter-rater reliability, a convenience sample of 14 community dwelling adults, self-reporting as healthy, were assessed by two trained raters. We found the Big-Store MET to be feasible to deliver (completed within 30 min, scores show variability, acceptable to participants in community environment) and inter-rater reliability to be very high (ICCs = 0.92–0.99) with the exception of frequency of strategy use. This study introduces the Big-Store MET to the literature, establishes its preliminary validity and reliability thus laying the foundation for a standardized, community-based version of the MET.
Full-text available
Introduction: Over the last decades, interactive technologies appeared a promising solution in the ecological evaluation of executive functioning. We have developed the EXecutive-functions Innovative Tool 360° (EXIT 360°), a new instrument that exploits 360° technologies to provide an ecologically valid assessment of executive functioning. Aim: This work wanted to evaluate the convergent validity of the EXIT 360°, comparing it with traditional neuropsychological tests (NPS) for executive functioning. Methods: Seventy-seven healthy subjects underwent an evaluation that involved: (1) a paper-and-pencil neuropsychological assessment, (2) an EXIT 360° session, involving seven subtasks delivered by VR headset, and (3) a usability assessment. To evaluate convergent validity, statistical correlation analyses were performed between NPS and EXIT 360° scores. Results: The data showed that participants had completed the whole task in about 8 min, with 88.3% obtaining a high total score (≥12). Regarding convergent validity, the data revealed a significant correlation between the EXIT 360° total score and all NPS. Furthermore, data showed a correlation between the EXIT 360° total reaction time and timed neuropsychological tests. Finally, the usability assessment showed a good score. Conclusion: This work appears as a first validation step towards considering the EXIT 360° as a standardized instrument that uses 360° technologies to conduct an ecologically valid assessment of executive functioning. Further studies will be necessary to evaluate the effectiveness of the EXIT 360° in discriminating between healthy control subjects and patients with executive dysfunctions.
Neuropsychological testing aims to measure individu-als' cognitive abilities (e.g. memory, attention), analysing their performance on specific behavioural tasks. Most neuropsychological tests are administered in the so-called 'paper-and-pencil' modality or via computerised protocols. The adequacy of these procedures has been recently questioned , with more specific concerns about their ecological validity, i.e. the relation between test scores observed in the laboratory setting and the actual everyday cognitive functioning. In developing more ecological tasks, researchers started to implement virtual reality (VR) technology as an administration technique focused on exposing individuals to simulated but realistic stimuli and environments, maintaining at the same time a controlled laboratory setting and collecting advanced measures of cognitive functioning. This systematic review aims to present how VR procedures for neuropsy-chological testing have been implemented in the last years. We initially explain the rationale for supporting VR as an advanced assessment tool, but we also discuss the challenges and risks that can limit the widespread implementation of this technology. Then, we systematised the large body of studies adopting VR for neuropsychological testing, describing the VR tools' distribution amongst different cognitive functions through a PRISMA-guided systematic review. The systematic review highlighted that only very few instruments are ready for clinical use, reporting psychometric proprieties (e.g. validity) and providing normative data. Most of the tools still need to be standardised on large cohorts of participants, having published only limited data on small samples up to now. Finally, we discussed the possible future directions of the VR neuropsychological test development linked to technological advances. K E Y W O R D S cognitive assessment, neuropsychology, PRISMA, test, virtual reality, VR
The merging of immersive technologies and cognition has been around for a while. However, it is only in the last decade or so that immersive technologies' contributions in the areas of cognitive assessments and interventions have gathered recognition. This chapter covers findings from published research in cognition-based assessments and interventions using the immersive technologies of virtual reality (VR) and augmented reality (AR). The role of immersive technologies in cognition is critically evaluated to inform all its stakeholders about its potential for use in the future.
As the healthcare industry continues to expand, it must utilize technology to ensure efficiencies are maintained. Healthcare needs to move in a direction where computational methods and algorithms can relieve the routine work of medical doctors, leaving them more time to carry out more important and skilled tasks such as surgery. Computational Methods and Algorithms for Medicine and Optimized Clinical Practice discusses some of the most interesting aspects of theoretical and applied research covering complementary facets of computational methods and algorithms to achieve greater efficiency and support medical personnel. Featuring research on topics such as healthcare reform, artificial intelligence, and disease detection, this book will particularly appeal to medical professionals and practitioners, hospitals, administrators, students, researchers, and academicians.
Zusammenfassung. Exekutive Dysfunktionen treten besonders bei Erkrankungen, die zu einer Schädigung des präfrontalen und/oder orbitofrontalen Kortex oder subkortikaler Strukturen (insbesondere des Nucleus caudatus oder des Thalamus) führen, bzw. bei Schädigungen von Faserverbindungen zwischen relevanten kortikalen und subkortikalen Hirnstrukturen eines exekutiven Funktionsnetzwerks auf. Jede Untersuchung bei Verdacht auf exekutive Dysfunktion sollte mindestens je ein Verfahren zum Arbeitsgedächtnis und Monitoring, zur kognitiven Flüssigkeit und Flexibilität sowie zum planerischen und problemlösenden Denken umfassen. Die Verhaltensbeobachtung des Patienten und die umfassende Befragung der Angehörigen sind bei exekutiven Dysfunktionen zentrale Bestandteile der neuropsychologischen Untersuchung. Sie sind dabei vor allem für die Einschätzung der Auswirkungen möglicher Funktionsstörungen auf die Aktivitäten und die Teilhabe des Patienten von Relevanz. Für die Therapie von exekutiven Dysfunktionen ist eine sorgfältige Diagnostik Voraussetzung, da aufgrund der Verschiedenartigkeit der Symptome die Therapie spezifisch auf das jeweilige Defizit zugeschnitten sein muss. Exekutive Dysfunktionen können sehr vielgestaltig, teilweise sogar scheinbar gegensätzlich sein, daher wird die Art der Intervention durch die spezifischen Symptome bestimmt. Die Durchsicht und Bewertung der Therapiestudien nach den Evidenzkriterien für den Zeitraum von 2007 bis 2017 ergibt folgende Therapieempfehlungen für Patienten mit exekutiven Dysfunktionen: Kognitiv übende Verfahren allgemein sollen eingesetzt werden (Empfehlungsstärke A). Training des Arbeitsgedächtnisses soll eingesetzt werden (Empfehlungsstärke A). Therapieansätze zum planerischen Denken sollen eingesetzt werden (Empfehlungsstärke A). Kognitive Trainingsansätze mehrerer Funktionsbereiche unter Einschluss der Exekutivfunktionen sollen eingesetzt werden (Empfehlungsstärke A). Therapieansätze zum Problemlösetraining sollten eingesetzt werden (Empfehlungsstärke B). Zu den Ansätzen zur Modifikation und Manipulation der Umwelt liegen im aktuellen Suchzeitraum anders als in der Vorgängerleitlinie nur wenige Studien mit geringer Evidenzstärke vor, daher kann für den Untersuchungszeitraum keine Empfehlung ausgesprochen werden (Empfehlung offen). Pharmakologische Therapieansätze finden zunehmend Beachtung, aufgund ihrer Unterschiedlichkeit kann aktuell keine Empfehlung gegeben werden (Empfehlung offen).
Bei der vorliegenden Leitlinie handelt es sich um eine Überarbeitung bzw. Aktualisierung (aktueller Suchzeitraum: 2007–2017) der bereits existierenden Leitlinie „Diagnostik und Therapie von exekutiven Dysfunktionen bei neurologischen Erkrankungen“ (AWMF-Registernummer: 030/125). Die Art der Interventionen, die evaluiert werden, hat sich in den letzten 10 Jahren verändert. Therapieansätze, die in der Vergangenheit häufig Gegenstand von Studien waren, werden nicht mehr erforscht, andere Ansätze sind hinzugekommen. Ehemals gut beforschte und in der Vorgängerversion der Leitlinie als wirksam bewertete Interventionen können trotz aktuell veränderter Forschungsaktivität weiter in der klinischen Praxis Verwendung finden. Kaum noch untersucht wurden der Einsatz externer Reize und das Self-Awareness-Training und überhaupt nicht mehr das Trainieren von Shifting-Prozessen. Hinzugekommen sind Studien zur Impulskontrolle/Selbstregulation und Studien, die Realisierungen von Virtual Reality oder Serious Games benutzen, außerdem mehrere pharmakologische Studien, die die Wirkung unterschiedlicher Substanzen untersuchen. Weiterhin haben wir eine größere Gruppe von Studien identifizieren können, die integrierte Trainingsprogramme zu mehreren kognitiven Funktionsbereichen evaluieren, von denen ein Bereich die Exekutivfunktionen (EF) waren. Intensiv untersucht wurden Effekte von Arbeitsgedächtnis-Trainingsverfahren, von Problemlöseverfahren und Ziel-Management-Trainingsverfahren. Grundsätzlich hat sich die methodische Qualität der Studien im Referenzzeitraum nochmals deutlich verbessert. Hinsichtlich der Diagnostik exekutiver Dysfunktionen sind die Neuerungen insgesamt überschaubarer, interessante potenzielle Weiterentwicklungen ergeben sich jedoch aus der Kombination neurowissenschaftlicher (z. B. nicht invasiver Stimulationsverfahren) mit klassischen neuropsychologischen Vorgehensweisen sowie aus der fortschreitenden technischen Weiterentwicklung zum Beispiel im Sinne der virtuellen Realität.
The Index of ADL was developed to study results of treatment and prognosis in the elderly and chronically ill. Grades of the Index summarize over-all performance in bathing, dressing, going to toilet, transferring, continence, and feeding. More than 2,000 evaluations of 1,001 individuals demonstrated use of the Index as a survey instrument, as an objective guide to the course of chronic illness, as a tool for studying the aging process, and as an aid in rehabilitation teaching. Of theoretical interest is the observation that the order of recovery of Index functions in disabled patients is remarkably similar to the order of development of primary functions in children. This parallelism, and similarity to the behavior of primitive peoples, suggests that the Index is based on primary biological and psychosocial function, reflecting the adequacy of organized neurological and locomotor response.
( This reprinted article originally appeared in the Journal of Experimental Psychology, 1935, Vol 18, 643–662. The following abstract of the original article appeared in PA, Vol 10:1863.) In this study pairs of conflicting stimuli, both being inherent aspects of the same symbols, were presented simultaneously (a name of one color printed in the ink of another color—a word stimulus and a color stimulus). The difference in time for reading the words printed in colors and the same words printed in black is the measure of the interference of color stimuli on reading words. The difference in the time for naming the colors in which the words are printed and the same colors printed in squares is the measure of the interference of conflicting word stimuli on naming colors. The interference of conflicting color stimuli on the time for reading 100 words (each word naming a color unlike the ink-color of its print) caused an increase of 2.3 sec or 5.6% over the normal time for reading the same words printed in black. This increase is not reliable, but the interference of conflicting word stimuli on the time for naming 100 colors (each color being the print of a word which names another color) caused an increase of 47.0 sec or 74.3% of the normal time for naming colors printed in squares.… (PsycINFO Database Record (c) 2012 APA, all rights reserved)
critically evaluate some of the issues facing neuropsychologists with respect to the topic of ecological validity / examines several of the assumptions widely held by many neuropsychologists, and introduces a theoretical model to obtain an operational estimate of the ecological validity of neuropsychological test data, using a vector analysis approach definition of ecological validity / an operational estimate of the ecological validity of neuropsychological test data (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Organizations are increasingly using computerized tests (e.g., multimedia, web-based, computer adaptive testing) in selection systems. A 2 (mode of presentation: paper-and-pencil – computerized)×2 (technical level of the job: high technical job–low technical job)×2 (selection decision: rejected or selected) between subjects design was used to assess proposed relationships between reactions to tests, their antecedents, and their consequences. While test-takers' post-test perceptions did not significantly differ as a result of mode of administration, computer anxiety and experience with computing were important factors in performing successfully. Significant relationships were found between post-feedback reactions and test-takers' intentions. The discussion highlights implications for implementing computerized selection tools.