Content uploaded by Amine Choukou
Author content
All content in this area was uploaded by Amine Choukou on Jul 25, 2023
Content may be subject to copyright.
Immersive VR for Training Cognition in Seniors
with Neurological Disorders
Mohamed-Amine Choukou(B)
College of Rehabilitation Sciences, University of Manitoba, Winnipeg, MB R3E 0T6, Canada
amine.choukou@umanitoba.ca
Abstract. The need to deliver rehabilitation services remotely has increased dra-
matically since the COVID-19 pandemic hit. Over the last decade, there has been
a surge in the use of virtual reality (VR) as an in-home rehabilitative tool. VR
undoubtedly offers the user an unequalled opportunity to immerse in the task at
hand entirely. Clients with neurologic conditions need continuous cognitive train-
ing, but receiving such care at home without a therapist is challenging. Programs
projected on screens are inadequately designed for use in non-conventional set-
tings such as the home environment due to a lack of technological literacy in most
cases and the inability of a person with a neurological condition to operate technol-
ogy efficiently, particularly in the case of altered dexterity. Home setting entails
user-friendly portable devices with as less client involvement as possible. This
paper presents an immersive VR platform developed for the in-home neurocog-
nitive rehabilitation of seniors with neurologic conditions. This paper introduces
a newly developed immersive VR-based neurocognitive program to be adminis-
tered in the homes of seniors with neurological disorders. The setup consists of a
commercial headset with two wireless hand-held controllers and two base stations
on a tripod each. The setup supports a comprehensive program introduced by the
researcher, consisting of 24 sessions of about 20 min each. The program is to be
completed at home in a self-paced manner over a 12-week training period. The
paper also discusses the benefits of the training program and the challenges for
seniors with neurologic conditions to engage in immersive VR.
Keywords: Virtual reality ·rehabilitation ·neurologic disorders
1 Problem Statement
Clients with neurologic conditions need repetitive and task-oriented training as a nec-
essary component of their neurorehabilitation program to produce desired neuroplastic
changes within the brain to improve motor deficits and promote functional recovery [1].
However, many people who require neurorehabilitation are discharged from the hospital
with unfulfilled needs. Telerehabilitation is promising in that regard. Telerehabilitation
is the remote delivery of rehabilitation services made possible by various technologies,
such as wearables and digital platforms. Telerehabilitation technologies enable remote
consultation, evaluation, assessment, prevention, monitoring and coaching, intervention,
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023
Q. Gao and J. Zhou (Eds.): HCII 2023, LNCS 14042, pp. 522–530, 2023.
https://doi.org/10.1007/978-3-031-34866-2_37
Immersive VR for Training Cognition in Seniors 523
supervision, and education. Beyond consultation, emerging technologies enable remote
replication of many characteristics of a person-to-person therapeutic session for clients
and clinicians. Under exceptional circumstances such as COVID-19, in-home telere-
habilitation services have proven to be the most effective way to continue therapeutic
regimens. However, in-home telerehabilitation, as it has been implemented recently, has
only relied on basic information and communication technology, primarily teleconfer-
encing platforms and tablet-based applications. More elaborated emerging technologies
such as virtual and mixed reality platforms are profoundly meaningful in the context
of neurorehabilitation. Technology offers the ability not only to provide better quality
training but also to practice at home independently or with the help of family caregivers.
Traditional rehabilitation is tedious, costly [2,3], labour-intensive, and requires strict
adherence, with most traditional stroke therapy sessions lacking the ability to provide
enough repetitions required for critical neuroplastic changes to be made [1]. Further-
more, geographical, logistical, and financial barriers to such specialized rehabilitation
programs may prevent clients, such as clients with stroke, from accessing adequate and
necessary care within the critical first 3–6-month time frame [4]. Geographic location
is a significant barrier for those who live rurally or remotely [5], especially seniors
who can no longer drive a car and have no one among their close contacts to help with
transportation. Many impediments, such as living far from metropolitan centres, lack
of transportation or unaffordability of services, are all solid reasons for someone need-
ing rehabilitation to give up. The quality of roads and climate conditions (e.g., winter)
make it even more complicated to travel and seek in-person healthcare services, as most
neuro-clinics are located in urban areas [3]. Clients who are deterred from attending
their rehabilitation appointments will therefore decrease their rehabilitation potential
[2]. This situation is not what patients, their families, and the community want. We
wish people to recover from injuries and diseases, return home safely, and resume their
usual activities to improve inclusion and social participation. An emotional burden and
anxiety related to transportation also exist. There are also financial barriers to engaging
in neurocognitive therapy [6].
The intensity of rehabilitation and the many barriers individuals may face in accessing
these programs have created a need for accessible outpatient rehabilitation services that
can be delivered safely and effectively to ensure equitable recovery for clients. VR-based
training is an alternative to conventional (in-person) rehabilitation that uses information
and communication technologies to provide rehabilitation services [7]. The expanding
literature on TR for stroke has indicated its effectiveness for various recovery domains,
including motor, cognitive, and affective [4,7]. The technology available varies across the
literature to include the use of telephones, smartphones, computers, video conferencing,
and virtual reality (i.e., immersive, semi-immersive, and non-immersive) [8,9].
Immersive VR has been introduced to cognitive training as gamified platforms that
certainly stimulate cognition, but that is, in most cases, not related to real-life tasks. How-
ever, in healthcare applications, there is a lack of realism in the simulated environment
that, even though gamified, remains unrelated to natural daily tasks, which may affect
motivation, adherence, and transfer of training for VR to real life. This paper introduces
a novel VR-based approach to training cognition in a VR environment in seniors with
524 M.-A. Choukou
neurological disorders. Immersive VR is presented as a pragmatic approach to deliv-
ering cognition-focused training programs in the homes of seniors with neurological
disorders. The paper also discusses the benefits of the training program and its support-
ing technology and the challenges for seniors with neurologic conditions to engage in
immersive VR.
2 Development of an Immersive VR Environment to Stimulate
Cognition in Seniors with Neurological Disorders
This paper introduces an Immersive VR platform that replicates an existing smart apart-
ment [10] in a virtual format (Link) (Fig. 1) and a novel training program developed
by the author of this paper that enables cognitive training in the simulated VR environ-
ment, with activities for clients to complete. The activities are incremental and resemble
scenarios we encounter during our daily lives inside the home, which involve memory,
attention, orientation, and decision-making. Training in the proposed VR environment
progresses throughout 24 sessions. The client is provided with a list of tasks before
wearing the headset, for which they can be assisted by a family caregiver. The user is
asked to complete the session without additional assistance beyond reading the task, on
paper or screen, before beginning a given session. The setup also can record activity logs
by the end of each session and record the information on a server or share it via email to
the program manager (therapists or researchers, depending on the use case). The latter
can use the information to provide the client with feedback post-session. Email, phone
Fig. 1. Main daily tasks performed in the Immersive VR environment
Immersive VR for Training Cognition in Seniors 525
calls, or a videoconferencing app are recommended in the case of remote training to
optimize the connection between the client and the therapist.
The setup consists of a commercial headset, two wireless hand-held controllers,
and two base stations on a tripod. The system is designed to be set up in a safe
room for the program to be self-administered and self-paced, such as in a research
facility, hospital, or the client’s home, for seated and standing use.
3 Development of a VR-Based Cognitive Training Program
The VR training program consists of five training modules with a total of 24 sessions
available on the platform. A person familiar with VR setups and with no cognitive or
physical disability would be able to complete a session within approximately 20–25 min
in a row. For safety reasons, most tasks are to be performed in a seated position. Only
some tasks require the user to stand up and return to the seat without any pivoting
needed (e.g., grabbing a towel from the towel rail). The first four modules contain
five incremental sessions each, and the last includes four entire day-like sessions. Each
module has specific objective(s):
•Module 1: Familiarity with the environment and technology
•Module 2: Task management and prioritization
•Module 3: Financial management
•Module 4: Task and financial management
•Module 5: Entire day
Module 1: Familiarity with the environment and technology
The first module is used to introduce the client to immersive VR. On a technical level,
the goal is to make it possible to manipulate the material. The activities involve putting
on the helmet, learning the basic commands, and getting comfortable with moving with
the technology. On the emotional level, this module intends to reassure the user, increase
confidence in the technology used, and break the “wow effect”. To do so, the development
of the sessions of this module capitalizes on the provision of optimal conditions to provide
the client with the possibility of discovering the environment at the desired pace and
to provide the evaluator or therapist with the opportunity to assess the client’s comfort
level with the technology. The five sessions of this module are incremental, in the sense
that tasks are added progressively from one session to another to arrive at session 5 with
a virtually entire day of basic tasks.
Module 2: Task management and prioritization
The second module has the same structure as the first module. Still, it focuses on task
management, the way the client will prioritize actions based on common sense and
preferences and the management of the interruptions of a task in progress. The activities
will be added progressively so that the client can complete an entire day by the module’s
last session. The goal is to manage tasks, identify priorities and sometimes even interrupt
a task in progress to meet an immediate need. The kind of interaction can be, for example:
receiving a message on your landline phone while completing lunch, placing an order if
the medicine bottle seems almost empty, retrieving a delivery despite a task in progress,
526 M.-A. Choukou
calling a maintenance specialist if there is a breakage, choosing a new laundry detergent
if there is none left, etc. For this module, the session scripts will help the client follow
the expected procedures.
Module 3: Financial management
The third module takes the same form as the previous ones, aiming to develop budget
awareness and financial management skills. Activities are added progressively so that
by the module’s last session, the client can complete an entire day. This module aims
to introduce the concept of financial management. For the five sessions of this module,
a total amount of $100 is proposed. Throughout each session, the clients will have to
make financial decisions and ensure they do not exceed $100. If they do, they must be
able to explain their financial decisions in the questionnaire to be answered at the end
of the module since every financial decision must be factored in as part of the financial
management tasks. For example, if the client has no detergent left for the laundry and
they have to choose between three types of detergent, they must remember the price and
deduct it from the total finances for this module ($100). This module will require the
client to remember the expenses made during the five sessions. Clients can build their
own memorizing strategy by writing down the expenses made right before the end of a
session. Clients are free to follow whatever method they are comfortable with.
Module 4: Task and financial management
The fourth module follows the form of the previous modules but aims to give more
autonomy to the client so that they can carry out the tasks thanks to the practices carried
out. The activities will all have already been carried out beforehand; this module prepares
the client for the last module by developing the skills required to understand and manage
contingencies and to be able to adapt to various situations. In the fourth module, activities
are also added progressively so that by the last module session, the client can complete
an entire day. However, the sessions will be more complex, mixing interactions and
financial situations. Also, since the activities have all been covered in previous modules,
the scripts will not be detailed.
Module 5: Entire day
The fifth module intends to get closer to the reality of activities that are naturally and
commonly performed over an entire day. This module is considered comprehensive and
intensive for someone with cognitive impairments to complete correctly. It incorpo-
rates the lessons from prior modules, allowing to check whether the client can respond
adequately and effectively to complex sessions without forgetting entire tasks or disre-
specting the sequence of actions underpinning a given task. In this module, the client will
experience complex situations, as each of the four sessions requires dealing with unex-
pected daily life situations and financial situations over a virtually entire day. Hence, the
evaluator will examine the skills gained throughout the program to assess the recovery
of cognitive abilities.
3.1 VR Senarios Structure
The scenarios attempt to reproduce some of the sets of knowledge related to
healthy/ecological habits in everyday life. The purpose of the practice in immersive VR
Immersive VR for Training Cognition in Seniors 527
is to help the user manage to perform daily life activities by making adequate choices and
decisions based on common sense and preferences (what most people do, for example:
brushing teeth in the morning, taking a coffee as part of the breakfast, making the bed in
the morning, etc.). To do so, the scenarios are based on tools used to allocate assistive
devices. These tools include two main categories of activity: activities of daily living
and domestic activities. The distinction between these two types is that activities of daily
living encompass more basic personal care, and domestic life activities relate to more
elaborate actions that a person must perform independently. It is critical to remember
that some categories cannot be evaluated in an immersive environment in the same way
they can be assessed in real life. We cannot, for example, assess mobility, hygiene, eating
habits, and so on in the same way. Memory, among other cognitive functions, will most
likely be stimulated when the user is engaged in the VR environment. The scenarios were
created based on the assumption that these cognitive skills can be stimulated and taking
into account technological constraints and the feasibility of simulating activities of daily
living and domestic activities. The scenarios are built incrementally to allow redundancy,
re-learning, task repetition, and progression in day-to-day activity management.
Each session included several household tasks requiring the user to move around the
virtual apartment using the left and right wireless hand-held controllers to interact with
the spaces and objects. As the sessions progressed, the tasks became more complex,
necessitating complex tasks within a given activity or more steps to complete within the
same activity. The data log is saved and transmitted to a dedicated server owned and
secured by the university. The information is made available on a server and emailed
immediately to the program manager following each practice (therapist, researcher).
3.2 Procedures
Documentation and Support
The tools made available to the clients are written material that explains the program and
guides the clients throughout all the program steps. However, the clients need help from
a family member to reach the training objectives. A support person (caregiver) must
be present to ensure that the client can use the setup (headset, controller, laptop) and
not develop bad training habits, which would instill bad practices in real life. Feedback
is essential, and thanks to the monitoring of recorded actions, the program manager
(therapist, researcher) can provide feedback post-session in addition to the caregiver
support.
Controlled Environment
The client’s health and safety are the priority in this program. The client is asked to take
enough time to properly prepare for each practice, physically, mentally and in terms of
room space and VR setup. The caregiver is also invited to help by frequently checking
on the client’s safety and comfort.
Technological Constraints
It is important to remember that some actions cannot be evaluated in virtual reality in the
same way as in everyday life. For example, mobility in VR does not resemble mobility in
the real world in everyday life. When designing the scenarios, the skills needed to move
in the space were identified as a technology constraint that is not part of the evaluation
528 M.-A. Choukou
criteria. Instead, the goal of the evaluations is to understand the cognitive skills used
and the client’s decision in response to various situations. For this reason, the clients
are advised to consult with the program manager to seek informed feedback instead of
self-evaluating themselves based on their perceived performance and evolution in the
program as they would while playing a ‘game’.
3.3 VR Training Platform as an Evaluation – and Data Transfer – Tool
The success of this program cannot be achieved without the active involvement of the
program manager in the therapy. However, the program manager should evaluate the use
remotely by assessing his performance. The program supports and encourages program
managers to monitor the clients by accessing log activity (Fig. 2) after each session. The
information is available on a server and emailed following each practice. The therapist or
researcher can then compare the client’s performance to the planned activity and detect
the anomalies that would refer to cognitive malfunctioning (e.g. an attention deficit, error
in sequence recognition).
Time (H-mm-ss) Log text
17-27-33 Alarm stopped
17-27-49 Bed done
17-28-59 Put on outfit
17-29-39 Put dirty clothes in washer
17-30-16 Put detergent in machine
17-30-23 Washer Started
17-30-28 Washer done
17-33-02 Put clothes in dryer
17-33-06 Dryer Started
17-33-11 Dryer done
17-33-23 Brushed teeth
17-33-28 Took meds
17-34-03 Clothes folded
17-34-31 Started Coffee
17-38-06 Put coffee on table
17-38-25 Coffee consumed
17-38-36 Put dirty mug in dishwasher
17-38-38 Dishwasher started
17-38-38 Dishwasher stopped
17-44-53 Turned stove on
17-45-29 Turned stove off
17-46-49 Dishwasher started
17-48-46 Dishwasher stopped
Fig. 2. An example of activity log
Immersive VR for Training Cognition in Seniors 529
4 Benefits and Challenges of Using Immersive VR for Training
Cognition in Seniors with Neurological Disorders
Finding a satisfactory technical solution to a complex problem is not easy, especially
when it comes to stimulating and evaluating cognition in seniors with neurologic condi-
tions. Many stakeholders must be mobilized to allow this neurocognitive program to run
successfully. The proposed in-home neurocognitive training program presents a benefit
to clients, therapists and researchers:
4.1 Benefits and Challenges for the Clients
The training program offers a progressive practice and allows repeating tasks as needed
and going back and forth as the program evolves. These design features are an asset for
keeping clients safe, engaged and motivated till recovering as many cognitive abilities
as possible after a traumatic (e.g., motor vehicle accident, fall) or non-traumatic brain
injury (e.g. after a stroke). The client can engage in the training program in the comfort
of their residence and practice at their desired pace.
4.2 Benefits and Challenges for Clinicians
This program offers an opportunity for remote coaching accompanied by personal-
ized and constructive feedback built upon tangible information (Fig. 2). The program
also implies better remote communication between a therapist and a person with a
neurological disorder.
4.3 Benefits and Challenges for Researchers
This program will provide a novel opportunity to test the feasibility, safety and effec-
tiveness of training and evaluating cognition remotely in seniors with neurological
disorders. The presented neurocognitive program will be introduced as a standalone
telerehabilitation service after a series of case studies and larger trials.
5 Conclusions
This paper introduced a new immersive VR-based neurocognitive program designed for
the in-home training of seniors with neurological disorders and discussed the benefits
and challenges of implementing the proposed telerehabilitation program. The proposed
VR solution is to be introduced as an in-home telerehabilitation program that is a con-
venient way to increase access to rehabilitation services at home for patients who would
otherwise be unable to get them. Receiving care at home would also cut down the
cost of transportation and lodging in some cases, which would benefit both individu-
als and communities. The VR-based training program supports the clients by enabling
remote training, which is particularly interesting for the chronic phase where clients
are discharged home without a full recovery. The program also benefits therapists with
opportunities for semi-supervised training while evaluating their clients and providing
530 M.-A. Choukou
informed feedback. Despite the completeness of the current version, small-scale clinical
testing needs to occur before deciding on larger-scale implementation. However, several
factors suggest that the solution responds well to the targeted issues, namely stimulating
cognition in a meaningful, safe and user-friendly way. There is a need to evaluate the fit
of the proposed program to the needs and preferences of training duration and frequency.
One can legitimately inquire whether in-home telerehabilitation can outperform on-
site rehabilitation. Based on the solution proposed in this paper, in-home telerehabilita-
tion provides access to services that are not currently available, such as remote evaluation,
semi-supervised therapeutic sessions, and progress monitoring. In-home telerehabilita-
tion can be introduced as adjunctive or standalone therapy to existing inpatient and
outpatient programs. However, in the absence of the option of providing neurocognitive
rehabilitation as an outpatient program, in-home telerehabilitation fills a major gap in cur-
rent healthcare service offerings. It may eventually allow for cost savings for the clients
and the community. In both circumstances, telerehabilitation encourages inclusion and
social engagement, whereas preventing patients from obtaining therapy disadvantages
them, with apparent consequences for their well-being and social participation. The
value of in-home telerehabilitation is a sure bet. So, why is it not used more?
References
1. Lohse, K.R., et al.: Virtual reality therapy for adults post-stroke: a systematic review and
meta-analysis exploring virtual environments and commercial games in therapy. PLoS ONE
9(3), e93318 (2014)
2. RHIHub: Barriers to Health Promotion and Disease Prevention in Rural Areas. 2018 [Cited
10 September 2022]. https://www.ruralhealthinfo.org/toolkits/health-promotion/1/barriers
3. Stroke, H.A.: 2017 Stroke Report (2017)
4. Chen, J., et al.: Effects of home-based telerehabilitation in patients with stroke: a randomized
controlled trial. Neurology 95(17), e2318–e2330 (2020)
5. Nelson, S.E., Wilson, K.: Understanding barriers to health care access through cultural safety
and ethical space: indigenous people’s experiences in Prince George. Canada. Soc. Sci. Med.
218, 21–27 (2018)
6. Zaman, M.S., Ghahari, S., McColl, M.A.: Barriers to accessing healthcare services for people
with Parkinson’s disease: a scoping review. J. Parkinsons Dis. 11(4), 1537–1553 (2021)
7. Cramer, S.C., et al.: Efficacy of home-based telerehabilitation vs in-clinic therapy for adults
after stroke: a randomized clinical trial. JAMA Neurol. 76(9), 1079–1087 (2019)
8. Wiley, E., Khattab, S., Tang, A.: Examining the effect of virtual reality therapy on cognition
post-stroke: a systematic review and meta-analysis. Disabil. Rehabil. Assist. Technol. 17(1),
50–60 (2022)
9. Bui, J., Luauté, J., Farnè, A.: Enhancing upper limb rehabilitation of stroke patients with
virtual reality: a mini review. Front. Virtual Reality 2(2021)
10. Choukou, M.-A., et al.: Designing the University of Manitoba technology for assisted living
project (TALP): a collaborative approach to supporting aging in place. In :Proceedings of the
21st Congress of the International Ergonomics Association (IEA 2021), pp. 223–228 (2021)
