Content uploaded by Aviv Elor
Author content
All content in this area was uploaded by Aviv Elor on Jul 18, 2022
Content may be subject to copyright.
Designing Spellcasters from Clinician Perspectives
A Customizable Gesture-Based Immersive Virtual Reality Game for Stroke Rehabilitation
JARED DUVAL∗,University of California Santa Cruz, USA
RUTUL THAKKAR∗,University of California Santa Cruz, USA
DELONG DU, University of California Santa Cruz, USA
KASSANDRA CHIN, University of California Santa Cruz, USA
SHERRY LUO, University of California Santa Cruz, USA
AVIV ELOR, University of California Santa Cruz, USA
MAGY SEIF EL-NASR, University of California Santa Cruz, USA
MICHAEL JOHN, University of California Santa Cruz, USA
Fig. 1. Screenshot of Spellcasters, featuring spell-casting gestures and virtual environment
Developing games is time-consuming and costly. Overly clinical therapy games run the risk of being boring, which defeats
the purpose of using games to motivate healing in the rst place [
10
,
23
]. In this work, we adapt and repurpose an existing
immersive virtual reality (iVR) game, Spellcasters, originally designed purely for entertainment for use as a stroke rehabilitation
game—which is particularly relevant in the wake of COVID-19, where telehealth solutions are increasingly needed [
4
]. In
preparation for participatory design sessions with stroke survivors, we collaborate with 14 medical professionals to ensure
Spellcasters is safe and therapeutically valid for clinical adoption. We present our novel VR sandbox implementation that
allows medical professionals to customize appropriate gestures and interactions for each patient’s unique needs. Additionally,
we share a co-designed companion app prototype based on clinicians’ preferred data reporting mechanisms for telehealth. We
discuss insights about adapting and repurposing entertainment games as serious games for health, features that clinicians
value, and the potential broader impacts of applications like Spellcasters for stroke management.
CCS Concepts: • Human-centered computing →Accessibility design and evaluation methods;User studies.
∗Co-First Authors
Authors’ addresses: Jared Duval, University of California Santa Cruz, 1156 High Street, Santa Cruz, USA, JDuval@ucsc.edu; Rutul Thakkar,
University of California Santa Cruz, 1156 High Street, Santa Cruz, USA, rmthakka@ucsc.edu; Delong Du, University of California Santa
Cruz, 1156 High Street, Santa Cruz, USA, dedu@ucsc.edu; Kassandra Chin, University of California Santa Cruz, 1156 High Street, Santa Cruz,
USA, kamchin@ucsc.edu; Sherry Luo, University of California Santa Cruz, 1156 High Street, Santa Cruz, USA, yluo78@ucsc.edu; Aviv Elor,
University of California Santa Cruz, 1156 High Street, Santa Cruz, USA, aelor@ucsc.edu; Magy Seif El-Nasr, University of California Santa
Cruz, 1156 High Street, Santa Cruz, USA, MSeifeln@ucsc.edu; Michael John, University of California Santa Cruz, 1156 High Street, Santa
Cruz, USA, MBJohn@ucsc.edu.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page.
Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
© 2022 Copyright held by the owner/author(s).
1936-7228/2022/4-ART
https://doi.org/10.1145/3530820
ACM Trans. Access. Comput.
2 • Duval and Thakkar, et al.
Additional Key Words and Phrases: Stroke Rehabilitation, Digital Therapeutics, Therapy, Immersive Virtual Reality, Game
Design, Games for Health, Serious Games
1 INTRODUCTION
According to The National Institute of Neurological Disorders and Stroke (NINDS), a component of the U.S.
National Institutes of Health (NIH), more than 800,000 people suer a stroke each year in the United States alone,
and approximately two-thirds of these individuals survive and require rehabilitation [
60
]. Stroke is a leading
cause of serious long-term disability [
80
]. Moreover, stroke management market size was valued at 30.1 billion
American dollars in 2019 and is expected to witness 6.3% compound annual growth rate from 2020 to 2026 [
79
].
In the wake of COVID 19, telehealth solutions have become increasingly relevant for delivering scalable remote
healthcare solutions to curb the spread of the virus [
4
,
5
,
86
]. Stroke physical and occupational rehabilitation
typically requires long-term and consistent intervention [
45
] with many challenges to keep stroke survivors
motivated [
51
]—a unique challenge that games have successfully navigated in many contexts [
64
,
69
,
83
] including
mental health (e.g., [55]), physical health (e.g., [6, 30, 37, 54, 57, 66]), and speech therapy (e.g., [24]).
Potential benets of repurposing existing entertainment games for therapy include reduced costs and de-
velopment times with some promise for appropriating already-fun mechanics [
23
]. During a global pandemic,
when telehealth solutions are needed quickly, appropriating technology to serve a new purpose is particularly
relevant. In this work, we explore the adaption and redesign of Spellcasters, an immersive virtual reality (iVR)
game where wizards cast spells by making gestures with their wand (VR controller). The original implementation
of Spellcasters is designed purely for entertainment, but the gesture-based spellcasting mechanic is intriguing
because it shares many commonalities with upper limb rehabilitation exercises, including repetitions, accurate
movements, and varying range of motions that can be measured using motion tracking. Placing stroke survivors
in VR has risks, so we begin our participatory approach by collaborating with 14 medical professionals to
validate that Spellcasters is safe and medically vetted before co-designing with stroke survivors. To this end, we
co-design an open environment for physical therapists and occupational therapists to create custom gestures for
their patients to "cast" and collect preferences on features for optimizing Spellcasters for telehealth, including
performance monitoring and goal setting. In this work, to protect stroke survivors from contracting the COVID
19 virus during the pandemic and from using untested non-specialized software, we focus on designing elements
of Spellcasters that will be primarily used by medical professionals—namely a custom gesture creation sandbox
for dening individualized stroke exercises (described in Section 4.1) and a companion app for remote monitoring
and administration (described in Section 4.5). These developments could be generalized to other game domains
beyond casting spells in future participatory work with stroke survivors.
The contributions of this work are 3-fold: (1) We contribute to a growing body of work that advocates for scalable
telehealth games for aordable and equitable access to healthcare, (2) we provide insights from collaborating
with medical professionals on how serious games for health can support custom and adaptable physical therapy
curriculum remotely, and (3) we share auto-ethnographic reections on adapting existing games for therapy and
including medical stakeholders in remote participatory design sessions.
2 BACKGROUND
This section introduces what traditional stroke management and rehabilitation entail, related works using VR for
stroke rehabilitation we are inspired by, and an argument for serious games for health as a potential solution
space for scalable, customizable, and equitable healthcare delivery.
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 3
2.1 Stroke Rehabilitation
Stroke survivors often experience a range of impairments, including loss of balance, cognitive deciencies, pain,
weakness, and paralysis—resulting in challenges performing everyday activities such as bathing, eating, walking,
and cooking [
76
]. In most cases, these eects of stroke are present on one side of the body, known as hemiparesis
[
11
]. Even though rehabilitation does not reverse brain damage, it can substantially help people achieve the best
possible long-term outcome [
45
]. For some survivors, rehabilitation will be an ongoing process to maintain and
rene skills for months or years after the stroke [
45
]. Stroke survivors tend to avoid using impaired limbs—a
behavior called learned non-use. However, the repetitive use of impaired limbs encourages brain plasticity and
helps reduce disabilities [
85
]. In practice, physical and occupational therapy emphasizes performing isolated
movements, repeatedly changing from one kind of movement to another, and rehearsing complex movements
that require a combination of coordination and balance. A recent trend in stroke rehabilitation emphasizes the
eectiveness of engaging in goal-directed activities, such as playing games to promote coordination [60].
There have been many software interventions for stroke rehab (e.g., [
3
,
15
,
17
,
27
,
47
,
58
,
66
,
72
]). Many input
modalities have been explored including motion tracking (e.g., using the wii controller [
3
]), 3d sensors such as the
Kinect or the Intel Real Sense (e.g., [
58
,
70
,
84
]), keyboards for ne motor control [
44
], shape changing robots [
47
],
webcams that track colored objects [
3
], smartphones that track performance [
32
], and VR (e.g., [
15
,
16
,
27
,
46
,
71
]).
Many of these software interventions are games—we discuss the benets of using serious games for therapy in
the next section. We present more details on VR for stroke rehabilitation in section 2.3.
2.2 Serious Games for Health
A serious game for health is a game created to entertain and achieve health goals [
39
]. Games are motivators and
make tedious repetition engaging through gameplay [
3
,
11
]. Many serious games have led to improved health
outcomes (e.g.[
49
]). Video games improved 69% of psychological therapy outcomes, 59% of physical therapy
outcomes, 50% of physical activity outcomes, 46% of clinician skills outcomes, 42% of health education outcomes,
42% of pain distraction outcomes, and 37% of disease self-management outcomes [
65
]. Learnability, exibility,
and robustness are core paradigms to creating usable and valuable serious games for health [
21
]—the ability to
customize serious games for health for the heterogeneous needs of stroke survivors is integral to our work.
2.2.1 Customizable Serious Games for Health. It is important that therapy game be customizable because players
have individual abilities [
13
], therapy goals [
45
], and motivations [
52
], similar to the concepts of player archetypes
[
59
]. We found Alankus et al.’s insights on multimodal inputs, feedback, and breadth of games inspiring in the
stroke rehab games context [
3
]. We are particularly interested in what medical practitioners nd important for
custom stroke rehabilitation in this work. There are various types of motions that therapy games designed for
upper extremity should cover; these include: shoulder abduction and adduction, shoulder exion and extension,
shoulder internal and external rotation, elbow exion and extension, wrist rotation, exion and extension, hand
and nger exion and extension, grasp, move, release and reaching [
3
]. 3D depth sensors can track most of these
motions [
84
] as well as VR devices [
73
] by using the standard motion controller. Not all of these exercises are
accessible to each stroke survivors’ abilities [
45
]. In this work, we are interested in leveraging these sensing
abilities in ways that are customizable and adaptable to the individual needs of each player. The aordances of
scalable technology allow adaptive therapy experiences for telehealth, described below.
2.2.2 Telehealth and games. Benets to telehealth interventions such as VR stroke rehab games include scalability
[
20
], increased access [
35
], customization [
20
], and rich data-driven insights based on large data sets and articial
intelligence [
25
]—an approach common in game user research, called telemetry [
22
]. In this work, we are
particularly interested in the reporting features clinicians are interested in to make informed insights into their
patient’s progress both at home and outside the clinical setting. Telehealth aords interactive contact, allowing
ACM Trans. Access. Comput.
4 • Duval and Thakkar, et al.
for day-to-day tracking of improvement and modication of recovery plans. The benets of using telehealth may
boost the eciency of stroke therapy with more prompt and regular evaluations and better consistency across
the healthcare chain.
2.2.3 Telehealth and COVID 19. Due to the COVID 19 pandemic, many inpatient rehabilitation facilities and
services have emergency preparedness plans in place to curb the spread of the virus, including cancellation
of non-required therapies such as physical, speech, and occupational therapy [
56
]. Medical professionals and
patients who have looked towards telehealth opportunities have been met with complex barriers, including
limited options and lack of insurance coverage [56].
It can be challenging to include people with disabilities for participatory work generally [
82
], but there is a
specic added risk during a global pandemic. Co-Design sessions should be valuable to all parties [
8
], but they
disrupt everyday life and require participants to invest their precious time. In addition to general guidelines that
limit in-person contact, populations of people with disabilities often have medical needs that place them at higher
risk from the COVID 19 virus [
5
]. In light of quarantine, designers are employing creative methodologies to carry
out remote design work that is usually done in situ [
86
] (e.g., using games to educate the public about COVID 19
and collect data [
50
]). Many of these technologies are not accessible to people with disabilities [
4
]. To this end,
in this work, we focus on co-designing Spellcasters with medical professionals to ensure its medical safety and
ecacy before working with stroke survivors after the COVID 19 vaccine is widely administered. We concentrate
on designing an intuitive environment for medical professionals to create custom and adaptive gestures for their
patients’ needs as well as tools for tracking and reporting on progress for telehealth. We argue that drawing
inspiration from existing games developed for entertainment provides some insurance that the game will have
entertaining mechanics without risking the health of stroke survivors or subjecting them to inaccessible remote
protocols. The groundwork for the rehabilitative custom gesture system we develop in this work could be utilized
and expanded to domains and themes beyond spell-casting in magical worlds, based on future participatory
work with stroke survivors. If, however, stroke survivors enjoy the magical domain, we can contribute further
anecdotal examples on the value of appropriating from entertainment games for therapy through this design
choice that protects the health and well-being of stroke survivors in the current pandemic climate.
2.3 Virtual Reality for Stroke Rehabilitation
VR for physical rehabilitation with stroke has seen an extensive exploration over the past decade due to the
potential to use gaming to motivate and guide users through repetitive therapy exercises. Immersive virtual
reality (iVR) refers to the sensation of being physically present in a non-physical environment. The perception is
generated by enveloping the user of the VR system with visuals, sounds, or other stimuli that create a complete
and engaging experience. Non-immersive virtual reality, unlike iVR, delivers the identical picture to both users’
eyes. As a result, people experience this picture in just two dimensions: height and breadth, but fully iVR
technology oer a digital image in three dimensions: height, width, and depth. Non-immersive VR games have
been explored as early as 2007 with systems such as Microsoft Kinect [
58
], PlayStation [
34
], and Nintendo Wii [
72
],
demonstrating feasibility in tracking patient progress and improving compliance through these motion-based
controllers. Multiple reviews have suggested that VR-supported mediums for stroke rehabilitation can be eective
in improving patient outcomes compared to traditional stroke therapy due to the ability to simulate controlled
interactive environments for exercise guidance and quantitative data capture [
18
,
48
,
58
]. Moreover, hundreds
of studies throughout the past decade support that VR is useful for motor rehabilitation with many reporting
signicant improvements in compliance and or recovery [12, 14, 27, 30, 42, 58, 72].
In 2021, the consumer market saw widespread adoption of iVR, enabling full-body movement and 360-degree
viewing of the virtual world through head-mounted display (HMD) systems. These systems are becoming
increasingly immersive, accurate at capturing human motion, and are projected to reach 30 million sales per
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 5
year by 2023 [
67
]. More recently, the academic community has begun investigating the usage of iVR HMDs for
post-stroke rehabilitation. Many studies have suggested that iVR HMDs can signicantly improve post-stroke
standardized upper-extremity motor tests. However, existing evidence is limited as there is a greater need in
more studies to investigate the non-pharmacological therapeutic pathway of iVR for people after stroke [62].
There has been a growing interest in translating these motor rehabilitation exercises into iVR games for
post-stroke. Project Star Catcher has demonstrated that iVR can benet treatments such as Constraint-Induced
Movement Therapy with stroke survivors by increasing compliance up to 40% when compared to traditional
methods and providing an accessible medium for capturing patient success metrics with HMD motion capture
[
27
,
29
]. Project Buttery, an iVR experience inspired by Mirror Visual Feedback Therapy, has explored the potential
to engage patients for long-term treatment with immersive virtual environments, which is vital because stroke
rehabilitation can span years [28, 63]. REINVENT applied neurofeedback systems to iVR games with promising
pilot results suggesting feasible and safe usage for severe stroke upper limb motor recovery [
74
]. Additionally,
some studies have begun testing commercial entertainment-based iVR exercise games (e.g., Beat Saber) for
users with chronic stroke and found that long-term gameplay can improve patient results for standardized
upper extremity motor function tests [
31
]. While many iVR solutions exist for stroke rehabilitation and suggest
promising results, there is an inherent need for more validation within the academic community to investigate
iVR HMD based design and clinician needs [
62
]. Thus, in this paper, we examine the usage of gesture-based
exercises for an iVR HMD experience from clinician perspectives by repurposing an entertainment-based game
for stroke rehabilitation as a precursor to participatory work with stroke survivors.
3 METHOD
The primary goal of this work was to redesign and adapt an entertainment-based exergame for stroke rehabilitation.
In this work, we focus on the intuitive software medical professionals will use for creating custom rehabilitative
gestures for their patients’ unique and heterogeneous needs. Based on our literature review and initial interviews
with physical therapists, we dened three research questions that drove our development and evaluation of
Spellcasters:
RQ1:
How do clinicians want to customize the therapy exercises towards an accessible primary spell-casting game
mechanic (gesture creation)?
RQ2: What data, visualizations, and reports are clinicians interested in for clinical decision making?
RQ3: How can Spellcasters support the telehealth needs of the COVID 19 era?
These contributions aim to ensure that Spellcasters is safe and medically vetted before engaging in future
participatory work with Stroke Survivors for prototyping iterations. The research questions above are exploratory
and aord a qualitative and iterative Research through Design approach [36, 88].
3.1 Soware
Due to the COVID 19 pandemic restrictions, we employed a fully virtualized human subjects protocol completed
online over Zoom
1
, a video-based teleconferencing platform with screen sharing capabilities. Spellcasters is an
immersive VR game that requires a head-mounted display (HMD) system, but not all medical professionals
have access to these devices. Consequently, we employed separate procedures for those with and without
supported VR HMD systems, described below in Section 3.3. We used Google Sheets
2
to analyze and transcribe the
recordings, grouped by interview question (included in our supplementary materials and live at https://tinyurl.
com/Spellcasters-Supplementary). A shared executable le of the game’s build was given to all participants who
1https://zoom.us/
2https://www.google.com/sheets/about/
ACM Trans. Access. Comput.
6 • Duval and Thakkar, et al.
had access to a VR system during the interviews. Additionally, a mockup interface was shared through Figma
3
,
an industry-standard software for rapid prototyping of user interfaces, to iterate and evaluate a companion app
described in the Spellcasters Section 4.
3.2 Participants
The participants recruited in this study consisted of physical and occupational therapists with experience in
post-stroke rehabilitation. Recruitment was conducted by reaching out to medical professionals in rehabilitation
leadership positions (e.g., chapter presidents of the United States American Physical Therapy Association
4
)
followed by Snowball recruiting [
40
] and posting recruitment iers on social media groups for rehabilitation.
Through this process, 14 medical professionals were recruited to participate in this study, with corresponding
demographics illustrated in Table 1.
Sex Role State VR HMD Access Companion App
P1 M Physical Therapist Ohio ✓X
P2 M Physical Therapist Kansas X X
P3 M Physical Therapist California X X
P4 M Physical Therapist Minnesota ✓X
P5 M Physical Therapist California X X
P6 F Physical Therapist Michigan X X
P7 M Physical Therapist California X X
P8 F Physical Therapist Virginia X X
P9 F Occupational Therapist Ohio ✓X
P10 F Physical Therapist New York X ✓
P11 F Physical Therapist New York X ✓
P12 F Occupational Therapist Massachusetts X ✓
P13 F Occupational Therapist Washington D.C. X ✓
P14 M Occupational Therapist Washington D.C. X ✓
Totals: 7 Female : 7 Male 10 Physical : 4 Occupational 11 Unique 3 Self VR Access 5 Self Companion Access
Table 1. Participant Demographics.
3.3 Procedures
With the consent of the participants, all virtual interviews were recorded for post-analysis, with each session
lasting between one to two hours (with all materials used in the procedure of evaluating Spellcasters shared
at https://tinyurl.com/Spellcasters-Supplementary). Sessions began with a set of preliminary semi-structured
interview questions inspired by [
22
] to understand each medical professional’s experience and practices related
to stroke recovery. We included questions related to their openness, experience, and expectations for physical
therapy games (RQ1). We asked how they currently collect data, communicate with insurance providers, and set
goals (RQ2). Pre-surveys were concluded with questions on how COVID 19 has changed their practice if they
have adopted telehealth and reections on if and how they measured progress made by patients outside of their
appointments (RQ3).
After the preliminary interview, participants experienced Spellcasters either directly with their own VR HMD
or indirectly by seeing a mirror of the research teams’ HMD view. In both cases, gameplay and user interaction
were mirrored in video using the Zoom screen sharing feature so everyone could see the game and record the
3https://www.gma.com/
4https://www.apta.org/
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 7
gameplay for later analysis (including game audio). Medical professionals without VR HMDs were asked to
instruct the researcher on how to play similar to a Think Aloud protocol [22] (RQ1).
As the Research through Design process progressed, it became clear that medical professionals would benet
from a companion app, described in Section 4.5 (RQ2,RQ3). Subsequently, the procedure was adapted to explore
this interaction for Spellcasters, where participants tested a companion app designed in Figma using the Think
Aloud protocol [
22
]. Participants freely explored the app during this phase, provided initial impressions, discussed
confusing elements, and shared design recommendations. We were particularly interested in the data visualizations
medical professionals were interested in (RQ2), so we asked participants to describe how they interpret each
graph, asked them if there was a more appropriate format they prefer, and if there were any elements that could
be added to make the graphs more intuitive. We iterated on the Figma prototype between sessions to incorporate
each participants’ feedback. We continued this cascading iterative process until a critical mass of participants
could understand the graphs and found them intuitive and valuable.
Finally, participants were asked a set of closing semi-structured interview questions [
22
] (RQ1,RQ2,RQ3),
provided at https://tinyurl.com/Spellcasters-Supplementary, so that we could evaluate the prototypes and recruit
more participants. After each session, the game and companion app prototypes were iterated based on observations
and feedback. The highly iterative Research through Design approach led to many insights towards answering
our research questions.
After completing all research through design sessions, medical professionals were asked to complete a follow-
up survey. 8 out of 14 completed the survey. We included the Spellcasters trailer in the survey to highlight the
iterative updates to the game since their playtest. The Figma prototype was also linked in the survey so the
participants could experience the most up-to-date version. The survey consisted of 9 Likert questions asking
participants to rate how much they disagreed or agreed with the 9 statements.
All of our transcriptions and summaries are available in the supplemental materials for transparency here:
https://tinyurl.com/Spellcasters-Supplementary. We sorted responses for each question by each participant in a
document table that includes summaries of each question, giving equal weight to all feedback. In our results
section, we organized these summaries into themes inductively, taking careful precautions to reduce bias by
including both positive and negative feedback from our participants.
3.4 Ethics
This research was reviewed and approved by the institutional ethics review board. An important consideration
was COVID 19 and the added risk many stroke survivors would face if they participated in this research—both
in-person and remotely. In-person protocols would be hazardous because they would place stroke survivors at
risk of infection. However, remote participation is also dangerous because the software many of us have come to
rely on during the pandemic is not accessible [
5
] and without medical supervision, playtesting, and co-designing
Spellcasters before it is medically vetted could also result in injury. Our institutional ethics board mandated
that we rst work with medical professionals before designing and testing with stroke survivors, which we
agree protects stroke survivors from unnecessary risk during the pandemic. We believe including people with
disabilities early and often in the design process is critical, which we will discuss in the next paragraph. However,
given the circumstances of the pandemic, we decided to work exclusively with medical professionals and leave
the game world open-ended for future participatory work with stroke survivors once the vaccine has been widely
administered.
Over a decade ago, ASSETS scholars called for the use of a critical disability lens while designing and developing
assistive technology for disabled individuals [
53
]. This call has only strengthened in the proceeding years, with
an emphasis on allowing for more co-design and co-research with disabled people [
7
,
75
,
87
]. To summarize the
concern, the majority of assistive technology devices and applications are rooted in medical discourse. That is,
ACM Trans. Access. Comput.
8 • Duval and Thakkar, et al.
disability is an inherent problem in the body and must be "xed" or "normalized" by intervention. Using a more
socially-oriented lens, such as those found in disability studies, emphasizes the social context and environment as
creating disability by denying access to particular body congurations [
38
,
68
,
78
]. Given this concern about the
discourses that inuence the design of assistive technologies, it has become even more important for researchers
to acknowledge the needs of the disabled individuals the technology is meant to help. We intend to include stroke
survivors in the design process for the gameplay surrounding casting the gestures made by medical professionals
in future work. We chose magic wands because the possibility space remains open for stroke survivors to choose
further game directions. The Results section provides evidence that supports the magical spell-casting domain
we have appropriated from the pure entertainment version of the game. However, if stroke survivors imagine
other domains during our participatory design sessions, the gesture recognition system and environment for
medical professionals to create custom exercises could easily be generalized to new domains.
Designing serious games for medical use is complicated due to the ethical responsibility that it entails. Specic
game mechanics may seem fun to play, yet a poorly designed game may cause more harm when used practically
with impaired users. As designers, we need to conduct research and trials to ensure our design suits our target
audience. While designing Spellcasters, we have worked closely with medical practitioners to evaluate what set
of features are needed and valuable for stroke survivors. From our research, we found that strokes are aecting
increasingly younger populations. One medical practitioner (PT8) expressed that they treat stroke patients as
young as ve. Given this concern, our team studied pre-existing games and stories such as; Waltz of the Wizard
and Harry Potter—popular themes among younger generations. With the help of physical therapists, we can
design our spells to resemble traditional therapy while maintaining an engaging experience. We envision this
game supporting rather than replacing traditional physical therapy. With this game, we hope to help engage
and motivate the stroke survivors to do their rehabilitation exercises prescribed to them more often because
consistency and compliance are critical to their recovery.
4 SPELLCASTERS
Spellcasters
5
is an immersive virtual experience designed in the Unreal Game Engine
6
(v4.24.3) that was repurposed
from an entertainment-based exergame to a medically informed therapy game for stroke survivors. Spellcasters
was originally a game purely for entertainment where two teams of 5 wizards competed in a magical duel. Wizards
had various roles on their team, such as tank, support, and melee—each with a corresponding spellbook and spells.
Both teams had a pool of lives, and the last team standing won the round. The entertainment version of Spellcasters
was developed in Unity and used an o-the-shelf machine learning-based gesture recognition system. When the
stroke rehabilitation version of Spellcasters project began, approximately a year after the entertainment version
was completed, we realized the underlying gesture products were no longer supported and were dicult to train
with new gestures. We found no alternative products that would meet our needs, so we decided to build an intuitive
gesture creation system so medical professionals could create custom therapy exercises. The novelty of Spellcasters
lies in this custom rehabilitative gesture system we have designed, described in Section 4.1. Around the same
time, we met with industry leaders developing specialized VR hardware for physical rehabilitation that required
the additional processing power of Unreal Engine. To keep future potential partnership opportunities open, we
chose to rebuild a rehabilitation version of Spellcasters in Unreal Engine—creating a custom gesture system and
removing the competitive multiplayer features from the original game (for now). Reusing design documents,
art styles, media, and drawing inspiration from the entertainment version of the game made development
much smoother—even in a new game engine. Our focus turned to make Spellcasters medically vetted while the
pandemic made participatory work with stroke survivors unsafe. To this end, we worked on designing and
5Spellcasters Trailer: https://tinyurl.com/Spellcasters-Trailer
6https://www.unrealengine.com/
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 9
implementing the gesture creation system and an accessible spell-casting experience. We left the game world an
open sandbox so stroke survivors could inspire future development directions when participatory work is safe or
when teleconferencing tools become more accessible.
We have access to the full source code through the serious games masters program at the University of
California Santa Cruz—a rare opportunity which we discuss in Section 6. The onset of transitioning the game for
therapy purposes began with obvious accessibility updates based on heuristics [
19
] and tacit knowledge from
the research team, including simplied and more legible user interface elements and multimodal audio, visual,
and haptic feedback. For example, instead of a text-based menu, we created a magical oce space where virtual
objects represent the menu options (e.g., wizard hats for jumping to various levels and a magic spellbook with
game options, such as audio settings and accessibility feature toggles). Another example is the addition of the
fairy companion, a guide who contextually gives instructions and hints via subtitled verbal instructions. We
reduced the amount of text, made text larger, and oered alternatives to text in the form of contextual cues and
instructions given by the fairy companion who serves as a guide in the game. Early in the process, we began
interviewing physical therapists and occupational therapists working with stroke survivors to collect functional
requirements for the core therapeutic spellcasting mechanic. The medical professionals wanted the ability to
create custom spell gestures for the idiosyncratic needs of their patients with 6 degrees of freedom (Section
4.1; Related to: RQ1), the ability to set which hand is used for casting for patients with hemiparesis and isolate
movement (Section 4.3; Related to: RQ1), the ability to play seated or standing for safety (Section 4.3; Related to:
RQ1,RQ3), and the ability use data-driven insights to customize rehabilitation curricula, insurance reports, and
manage patients (Section 4.5; Related to: RQ2,RQ3). Many of these requirements echo the framework presented
by Saini et al, including varying levels of diculty, precise direction, display feedback, and time limitations [
70
].
A playable build of Spellcasters can be found at https://tinyurl.com/Spellcasters-Build.
4.1 Custom Gesture Creation
Fig. 2. Process of creating custom exercises for stroke survivors as magical spells in a virtual environment including sphere
placement, reward selection, and repetition seing.
We developed a novel custom gesture creation system that allows clinicians to create exercises in the form of
3D-enabled magical spells. These spells can be customized using an intuitive point and click mechanic capable
of recording therapeutically relevant variables, including scale, shape, direction, and depth, depending on the
motion the clinicians want the patient to perform (depicted in Figures 2 and 3). For example, a clinician may
design a small circle by placing the spheres close together—useful for wrist rotation exercises. In contrast, a larger
circle would require the stroke survivor to use their whole arm for an external rotation exercise. This feature
gives clinicians creative freedom and endless possibilities for designing and customizing their gestures. To create
a spell, a clinician must decide and place a series of collision spheres, which creates a specic shape. The order
that the clinicians place the spheres determines the sequence in which the stroke survivor needs to connect
them to complete the gesture (Shown in Figure 3). The clinician can set repetition counts of these gestures and
what this spell does for patients (Shown in Figure 2). Each spell eect helps provide a sense of purpose to each
patient’s successful attempt to perform a gesture. It also shows the patient’s progress during the session as one
ACM Trans. Access. Comput.
10 • Duval and Thakkar, et al.
Fig. 3. Screenshot of how spells can be customized in Spellcasters by clinicians using scale and depth
Fig. 4. On the le, a screenshot of the spellbook with goal progress, and to the right, the contextual support of the non
player fairy character, subtitles, and video pop-up demonstrating the mechanic
can see and count the number of trees or owers a patient might have planted by the end of the session. These
repetition counts are shown on the spellbook value, indicating how many times the patients need to do them
successfully, shown in Figure 4.
Using a participatory approach during our design sessions with medical professionals, we co-designed a set of
predened gestures that Spellcasters will support by default. The resulting 18 predened spell gestures include a
horizontal line, vertical line, diagonal line, rectangle, square, triangle, semi-circle arcs, circles, and innity symbol.
Medical professionals helped us inductively sort these gestures into relevant non-mutually exclusive themes,
including ‘Extending Arm,’ ‘Rotation,’ ‘Internal Rotation,’ ‘External Rotation,’ ‘Crossing Mid-line,’ and ‘Raising
Arm.’ This design feature is related to RQ1. The spellbook, shown in Figure 4, is stocked with these predened
spells and displays a subset of relevant themes.
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 11
Fig. 5. Screenshot of Spellcasters, stroke survivors perform exercises by tracing magical spells in a virtual environment
Fig. 6. Screenshot of rewards in Spellcasters that clinicians can assign for each spell for stroke survivors, who will get confei
and fireworks on completion of a set of exercises
4.2 Gesture Tracing
For a player to cast a spell, they begin by ipping through pages of the spellbook until they nd one they would
like to cast—the active spellbook page signals to the custom gesture recognition system, which guides the player
on how to trace the shape that appears in front of them. Tracing the spell requires the stroke survivor to point
their wand and contact the depth-sensitive spheres in the correct order. Using a participatory approach, we
co-designed an initial set of user experience features to make gesture tracing accessible to the stroke survivor,
including haptic feedback for when the player starts to veer o the path, spheres that indicate order by growing
and shrinking as the player progresses through the points, thick green lines with arrows between the points to
illustrate the ‘target threshold zone’ and direction, verbal feedback from the fairy, sound eects for successful
and failed attempts, and a tracking line that visually traces the stroke survivors path from the tip of the wand.
Many of these features can be seen in Figures 4 and 5. Additionally, we created contextual videos that pop up and
demonstrate to players the various aspects of the spell-tracing mechanic, shown in Figure 4. Once the gesture is
successfully traced, patients can cast the spell by pressing a button to point and shoot at a given location. If the
patient wants to cast the spell somewhere other than where they are located in the scene, they can press a button
to teleport to the desired location and then cast the spell. For patients who do not wish to use the teleportation
mechanic, some spells summon the various creatures they can interact with, so Spellcasters can be experienced
completely from one location. Before participatory work with stroke survivors, some preliminary spells include
summoning plants and trees to decorate the garden, animal summoning spells, spells to feed animals, and animal
interaction spells such as a ball to play fetch with the dog. A new spell is required for every interaction to
encourage players to repeat the therapeutic motions. Figure 6 shows some of the current possible spell outcomes
and the reward system’s confetti particle eects when a complete set of clinician-set repetitions is completed.
The spellbook keeps track of whether the patient has completed a gesture successfully or not. When a sphere
is skipped or not connected correctly, the attempt will be recorded as an incomplete attempt. If the patient is not
moving for a while, the game plays a sound to inform the player that the spell is timed out, the traced line turns
red, the controller provides haptic feedback, and the attempt is logged as incomplete. However, the game allows
the patient to continue tracing after hearing the ‘incomplete’ sound eect. In such a case, if the patient connects
the spheres in the correct sequence, it will be recorded as a successful attempt.
ACM Trans. Access. Comput.
12 • Duval and Thakkar, et al.
4.3 Isolated Movement Therapy
Strokes often co-occur with hemiparesis, aecting one side of the body [
11
], so providing the option to play
the game with either hand is essential. Stroke survivors may rst play with their stronger arm when initially
learning the mechanics. The spellbook has a swap hand button to quickly move the wand to either side without
swapping game controllers.
As discussed in Section 4.1 some spells, such as the wrist rotations, should be performed without moving the
shoulder or elbow, so we include a feature in the companion app (Described in Section 4.5) that allows medical
professionals to pre-record messages and instructions that will remind patients to isolate their movements or
remind them not to use a compensation strategy.
4.4 Levels
Spellcasters provides two dierent tutorials: One for the medical professionals and one for the stroke survivors.
The tutorial for the medical professionals provides a walk-through of creating a custom spell gesture and the
gesture tracing mechanic that the stroke survivors will use. The gesture tracing walk-through is benecial for
medical professionals because it allows them to test their spells and ensure they are appropriate for individual
stroke survivors. Spellcasters is designed with multi-sensory feedback (including haptics) to be more accessible to
stroke survivors who may have co-occurring disabilities such as vision or hearing impairments. The built-in
nonplaying character, the fairy, uses closed-captioned dialog to support accessibility. The soundscape is highly
customizable, so players can independently adjust the dialog, background music, audio cues, and sound eects.
The accessibility features were iterated throughout the playtesting process. For example, the pop-up video
tutorials demonstrating the various mechanics were introduced after our ninth participant.
Beyond both tutorials, there are two sandbox levels: One that allows players to maintain a forest garden and
one with animals that the player can interact with. The rationale behind separating the levels is to provide one
experience with less sensory overload. The second level allows the player to feed animals, play, fetch, call the
animals, and pet the animals.
4.5 Companion App
As the playtests and co-design sessions progressed, it became clear that it would not be convenient for medical
professionals to access logs, reports, and patient management tasks within VR due to the (lack of) aordances
for text entry. Additionally, a busy medical professional is less likely to wear a headset for quick adjustments
than logging into a web app. Therefore, we rapidly iterated on the design of a companion web app using Figma
for medical professionals to remotely control the game client, see data visualizations, generate reports, share
resources with other medical professionals, and manage patients. The rst Figma prototype can be found at
this link: https://tinyurl.com/SpellcastersCompanionLowFi. The nal version can be seen at this link: https:
//tinyurl.com/SpellcastersCompanionHighFi. To facilitate rapid iterations of the companion app, we used dummy
data to populate the prototype. Based on how clinicians guide us on what graphs are shown and how they are
formatted, we can update Spellcasters to provide appropriate data to replace the dummy data in the deployed
product.
We worked closely with participants to make intuitive data visualizations. These visualizations track variables
such as accuracy, velocity, time spent in-game doing exercises compared to time spent doing other activities,
and range of motion. These visualizations and the overall design of the prototype were updated between
each participant using a cascading iterative protocol. We provided two versions of the same visual in many
iterations—the original and the updated one based on the previous participant’s feedback—and asked the new
participant to choose between them. The updated visualization was always chosen. The trickiest visualization we
nalized was the goal tracking graph. The goal tracking graph represents goals that are both set in the game
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 13
Fig. 7. Screenshots of companion web app prototype.
and external stroke rehabilitation goals. For example, an in-game goal might be to achieve a certain accuracy
percentage or number of repetitions, while an external goal might be to walk a certain number of steps or cook a
certain number of meals—all goals must be quantiable to graph. Our original goal visualization was a spider
graph, but many participants had never seen one or did not nd them intuitive. In the end, we included a primary
multi-line graph that provides an overview of how close each goal was met week-by-week as well a stacked line
graph that breaks down each goal and a table with raw data, shown in Figure 8.
Clinicians can also write notes and set reactions to inform and motivate patients. Clinicians have access to
various spells created by themselves or other clinicians, helping them reduce the need to create them frequently.
They can assign these to their patients along with repetition counts.
4.6 Usage and Seing
While Spellcasters takes place in a VR environment that provides an immersive experience for stroke survivors
to practice depth-sensitive gestures, a concerning drawback of VR HMDs is that they wrap around our eyes,
inhibiting users from seeing the real world. To reduce the chances of experiencing motion sickness and increased
safety, stroke survivors can play while sitting in a chair, and the game’s teleportation mechanic is not required
to play. For example, we created spells that call distant animals to the player location. However, we have also
enabled teleportation in the game so players can move around in the world if they wish, without needing to
leave their chairs or get motion sick. Stroke survivors can also play while standing in one location—the gesture
system will adjust to their standing height. Given the immersive nature of VR, clinicians have conrmed their
interest, during our interviews with them, in using Spellcasters primarily in a supervised environment such as a
rehabilitation facility. Once stroke survivors have made enough progress, dened and measured by their clinician,
they can use Spellcasters as an at-home rehabilitation tool, possibly under the supervision of a caretaker, which
we discuss in Section 5.3.
Another design consideration for choosing VR is the hardware cost and the prevalence of availability in
rehabilitation clinics and homes. While VR is expensive hardware, the cost of consumer devices is continually
becoming more aordable. For example, at the time of writing this, the Oculus Quest 2 is a standalone headset
ACM Trans. Access. Comput.
14 • Duval and Thakkar, et al.
Fig. 8. Example Goal visualizations in Companion Web app.
that supports hand tracking [
81
] and is powerful enough to run Spellcasters—available for $299. The headset does
not require an expensive VR-ready computer, does not need complicated tracking towers, and is a standalone,
ready-to-use device. The hand tracking features that are becoming available are valuable alternatives for stroke
survivors who cannot hold a controller or do not have the dexterity to use the buttons due to the higher System
Usability Scale (SUS) of hand tracking [
81
]. We argue that the cost of these devices is a worthy investment in an
expensive healthcare climate.
5 RESULTS
For each of the 14 cascading iterative Research through Design [
36
,
88
] sessions, we began with a semi-structured
interview, followed by a playtest, and then a semi-structured design debrief to get feedback and iterate. Summaries
of all the responses and transcriptions are included in the https://tinyurl.com/Spellcasters-Supplementary. We
share qualitative quotes from these sessions in the relevant sections below as they relate to our research questions.
The follow-up survey responses are also available in the supplementary materials, and the outcomes are presented
in the following sections, organized by their related research questions.
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 15
Count of Medical Professional Responses to Statement
0
1
2
3
4
5
Strongly Disagree Disagree Neutral Agree Strongly Agree
I would be Interested in Adopting Spellcasters in my own
Practice
Count of Medical Professional Responses to Statement
0
2
4
6
Strongly Disagree Disagree Neutral Agree Strongly Agree
Playing Spellcasters Would Likely Lead to Improved Health
Outcomes
Count of Medical Professional Responses to Statement
0
1
2
3
4
Strongly Disagree Disagree Neutral Agree Strongly Agree
The Spell Gesture Creation is Intuitive
Count of Medical Professional Responses to Statement
0
2
4
6
Strongly Disagree Disagree Neutral Agree Strongly Agree
The Spell Gesture Creation is Useful
Fig. 9. Results related to RQ1 including exercises that should be supported, interest in adopting Spellcasters, the likelihood of
the game leading to improved health outcomes, intuitiveness, and usefulness of the gesture creation system using 5-point
Likert scales of degree on the agreement to statements
5.1 Spellcasters (RQ1)
Throughout the iterative process, Spellcasters was updated to include recommendations made by medical profes-
sionals, including adding arrows to the gestures to make the direction of tracing clear, the inclusion of larger
shapes for a range of motion exercises, and inclusion of small shapes for wrist exercises. While some clinicians
prefer more presets to save their time, all clinicians found the custom gesture creation useful, as can be seen
in Figure 9. Clinicians were excited about the ability to add depth to the gestures, citing its usefulness in many
dierent exercise contexts: "I feel like you got everything covered—crossing the midline, shoulder rotations,
extensions, etc.” (P11), "I think it is complete—I do not have anything else that I would want to customize" (P10),
and "Yeah, I think that is fantastic because you can do as small and as big as you want to make the patient do."
(P6). Clinicians found the game format appropriate: "I think there is a lot that can be done with it. I think my
patients are going to want to try it" (P6), "I think people would like to have this in their toolbelt to make therapy
more exciting" (P3), and "It is like playing a game—they would get excited’ ’(P13). P1 suggested we make the
game support multiplayer. In terms of usability and accessibility, P6, P8, P10, P12, and P14 are concerned with
some stroke survivor’s ability to hold the controller or press the trigger button, which is why we will use the
hand tracking feature in the Quest 2 for future testing with stroke survivors. P11 said, "I love the armations that
you added. I like the reworks and confetti. I think the spellbook looks good—I feel like it is readable and I can
understand what is going on", and P14 suggested numbering the spheres. All 14 medical professionals mentioned
Spellcasters is a potentially valuable tool for other populations, including pediatric populations and those with
spinal cord injury.
ACM Trans. Access. Comput.
16 • Duval and Thakkar, et al.
Count of Medical Professional Responses to Statement
0
2
4
6
Strongly Disagree Disagree Neutral Agree Strongly Agree
I would be Interested in Adopting the Spellcasters
Companion Web App
Count of Medical Professional Responses to Statement
0
1
2
3
4
Strongly Disagree Disagree Neutral Agree Strongly Agree
The Companion App's Data Visualizations are Appropriate
Fig. 10. Results related to RQ2 including interest in adopting the companion web app and its data visualizations using 5-point
Likert scales of degree on the agreement to statements
5.2 Companion App (RQ2)
As sessions progressed, it became clear that an external tool would be helpful to clinicians to manage their patients
and track their progress. The development of the companion app was prompted by P8, who said, "We only have
45 minutes with the patient, so we cannot really spend 15 minutes creating the exercises—have more presets,
and a save feature so we can reuse exercises". All but 1 clinician who interacted with the companion app stated
they would be interested in adopting it into their practice, shown in Figure 10. Our initial low and high-delity
prototypes can be found in the supplementary materials, which illustrate rapid iterations and improvement based
on clinician feedback.In the end, all clinicians found the visualizations appropriate, which can be seen in Figure
10. Beyond the visualizations, a reoccurring theme was that fast, easy-to-digest facts were preferred (e.g., P14
said, "I cannot spend much time looking at graphs, so a quick stats interface would be better"). There were many
iterations on the mock data visualizations, such as for P11, who had never seen a spider/radar chart before, so we
changed it to a grouped bar graph, which was clear to the following participants. Bar graphs were, by far, the
preferred format.
5.3 Telehealth (RQ3)
Telehealth has become an even more critical part of our healthcare system today due to COVID 19. Based on our
conversation with clinicians, we have found that none of them have used any telehealth games or applications
beyond video conferencing tools and assigned videos (P4) due to a lack of options. The software clinicians
mentioned they currently use includes Bluestream (3 mentions), Zoom (1 mention), and Doxy.me (1 mention).
Clinicians typically use this software to watch their patients remotely on a screen and provide them with
instructions. Observation through a screen is problematic because "It is much harder to get people to do telehealth
with its current technology—they need many other cues than the limited visual and audio cues we have now" (P3).
Because of the limited set of activities they can do while remote, clinicians have suggested that patients have been
eager to go back to their traditional in-person system. 4 clinicians have never done telehealth and continued to
oer in-person care during the COVID 19 pandemic. P8 serves many patients from a lower socioeconomic status
and is concerned with telehealth because of their limited access to good digital resources. These conversations
conrmed our agreement with our institutional ethical review board—we need to wait to work with stroke
survivors until after the vaccine is widely administered or after teleconference tools are made more accessible.
Spellcasters has the potential to serve as a telehealth option as access to VR becomes more common—and
clinicians tend to agree that Spellcasters could be useful in this context (shown in Figure 11). P5 indicated that "If
Spellcasters can help them stay more consistent with their program, then sure—really consistency is the name
of the game, so anything that can help somebody be more consistent is going to be a win." P3 said, "I believe
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 17
Count of Medical Professional Responses to Statement
0
1
2
3
4
Strongly Disagree Disagree Neutral Agree Strongly Agree
Spellcasters Could be Useful for Telehealth
Count of Medical Professional Responses to Statement
0
1
2
3
4
5
Strongly Disagree Disagree Neutral Agree Strongly Agree
Spellcasters is Safe for Stroke Survivors when Supervised
Count of Medical Professional Responses to Statement
0
1
2
3
4
5
Strongly Disagree Disagree Neutral Agree Strongly Agree
Spellcasters is Safe for Stroke Survivors who are
Unsupervised
Fig. 11. Results related to RQ3 including usefulness as a telehealth solution and the safety of using the game while supervised
vs. unsupervised using 5-point Likert scales on the degree of agreement to statements
that if you can tie it to patient adherence, then you can correlate that to improvement—and that goes to taking
it at home with you and doing it more often." Safety was one of our primary concerns because, in the home
context, stroke survivors may be practicing without a present medical professional, but as can be seen in Figure
11, medical professionals tend to believe the game would be safe (more so with supervision). Many indicated this
is due to the ability to play the entire game while seated.
6 DISCUSSION
In this section, we begin by discussing our autoethnographic insights [
26
] about converting Spellcasters from a pure
entertainment game into a serious game for health for stroke rehabilitation. Next, we present our interpretations
on our three research questions that drove our Research Through Design iterative process [
36
,
88
]. Namely, the
importance of being able to customize therapy mechanics for the personal needs of stroke rehabilitation, the
potential benets of using game telemetry to inform scalable and equitable data-driven healthcare solutions, and
games as an option for future telehealth opportunities, which is particularly relevant in the wake of COVID 19.
Finally, we discuss the limitations and future directions of our work.
6.1 Converting Entertainment Games to Therapy Games
At the project’s onset, we were hopeful that converting an entertainment game into a serious game for health
would reduce development time and oer some promise for translating mechanics already proven to be fun
and engaging. We ultimately redeveloped the game from the ground up and drastically altered many original
core features, including the multiplayer magical duels, and created an entirely new gesture system. Still, the
time and eort were signicantly reduced than if we started from scratch because the shared vision was clear
from the onset. However, these changes come with some inherent risk: the original qualities that made the
original Spellcasters fun could be lost in translation. Some of the old features, namely the multiplayer competitive
ACM Trans. Access. Comput.
18 • Duval and Thakkar, et al.
experience, may come back based on future design work with stroke survivors (or they might lead us down an
entirely dierent path). Because Spellcasters is an open sandbox world, we now have an opportunity to co-design
with stroke survivors using the simple spell-casting mechanic to aord a multiplicity of design directions—from
a narrative adventure to multiplayer duels to tending magical gardens. Most of our investment, and the main
novel contribution of this work, is the intuitive custom rehabilitative gesture creation system paired with a
companion app, which could be extended to support many dierent domains other than a magical world if
stroke survivors show interest during future participatory design sessions. To provide this essential resource to
stroke survivors as quickly as possible with the restraints of the pandemic, we chose to draw inspiration from an
existing entertainment game rather than use Participatory Design with stroke survivors. While the gold standard
would have been to enlist stroke survivors as co-designers at the onset, the COVID 19 pandemic restricted our
options, so we chose to investigate this higher research question of whether there is value in repurposing existing
entertainment games into therapy games. While this work can not thoroughly answer this more meta-level
question, our experience so far has been that the shared vision and available resources (existing source code,
design documents, aesthetic style, existing mechanics) made our development much more time-ecient than
exploring many possibility spaces. Our results from clinicians are promising in that they believe stroke survivors
will love the magical world and spell-casting domain.
6.2 Custom Therapy (RQ1)
Healthcare does not follow a "one shoe ts all" model—everyone has individual goals, needs, abilities, and
preferences [
33
]. Stroke medical professionals employ numerous strategies for motivating their patients based on
their patient’s health, environmental factors, and personal factors [
61
]. The primary mechanic in Spellcasters
is making gestures to cast spells—and one of our primary contributions in this work is designing an intuitive
gesture creation system that medical professionals can use to customize Spellcasters for their patients. From the
feedback we collected, this is by far the most valuable feature in the game (Section 5.1). The gesture creation
system is also where most of the development eort went. We prioritized this mechanic because we believe
the core mechanic in a serious game for health is central to the success of the game—it should be accessible,
customizable, lead to improved health outcomes, intuitive, engaging, and data-driven—a tall order. We plan to
work with stroke survivors in the future to ensure it is genuinely accessible, intuitive, and engaging.
6.2.1 Future Work. As discussed in Section 3.4, Spellcasters currently follows an overly clinical model because
we have not yet incorporated input from stroke survivors. We have made Spellcasters customizable from the
perspective of clinicians (which is very important), but therapy should also be customizable from the players’
perspective. A research question that will drive our future work is: How can Spellcasters holistically support stroke
survivors?
6.3 Companion App (RQ2)
Companion apps create added value towards long-term engagement in games for health because they can visualize
progress and medical information, increase the perceived value of compliance with sustained use, and can help
embed the training routine in daily practice [
43
]. As the iterative design process progressed with Spellcasters, it
became clear that medical professionals were interested in quantitative insights and data visualizations from
the game but did not think it would be convenient to wear a headset and launch the game to access them.
Additionally, the aordance of VR is not as suitable for patient management as traditional web browsers. Once
we introduced the companion app Figma prototype, medical professionals were highly enthusiastic (Section 5.2).
Instead of focusing on the app’s actual implementation details, we were primarily focused on designing the data
visualizations, information organization, intuitive navigation, and desired features the companion app would
support—Figma was highly eective because we could rapidly iterate between each session.
ACM Trans. Access. Comput.
Designing Spellcasters from Clinician Perspectives • 19
6.3.1 Future Work. Medical professionals were highly interested in community-based sharing of sets of spell
gestures and communication with their patients within the game. We think the social aordances of a companion
app for serious games for health are highly interesting. A 2-part research question that will drive our future work
is: How do social aordances in the Spellcasters Companion app aect the gameplay experience and relationship
between stroke survivors and their clinicians. We plan to study the impact of the companion app on the gameplay
experience and relationships with clinicians by conducting a comparative study where one group of stroke
survivors has access to the companion app while another does not. Each group would take a player experience
inventory [2] and an inventory on the clinician-patient relationship [41] for comparison.
A benet to using serious games for health is the added ability to collect rich data using game telemetry [
22
].
This data can train machine learning models to better support players by predicting when they need support, more
accurately sensing their therapy-based mechanics and standardized metrics for developing a therapy curriculum.
We are interested in exploring how machine learning can support stroke survivors who play Spellcasters. A
research question that will drive our future work is: How can machine learning support stroke survivors who play
Spellcasters.
6.4 Telehealth (RQ3)
Many medical professionals are becoming experts at meeting with their patients remotely—our rst participant,
without prompt, said he needed to optimize his screen share in Zoom for videos, indicating that he was well-versed
with the software’s advanced features. While telehealth has been around for almost 50 years [
9
], the COVID
19 pandemic has created an explosion of need for more telehealth options [
56
]. Most medical practitioners felt
Spellcasters would be useful and safe in a clinical setting because the stroke survivor would be supervised (Section
5.3). However, a few of the participants indicated that they are hesitant to recommend Spellcasters for at-home
use while unsupervised. If a stroke survivor injured themself, there might not be anyone around to respond. It
helps that Spellcasters can be played while seated, but more work remains into investigating how safe VR is for
unsupervised stroke survivors (and if the game will require a supervisor).
6.4.1 Future Work. Based on our </