Content uploaded by Jared Duval
Author content
All content in this area was uploaded by Jared Duval on Jan 28, 2021
Content may be subject to copyright.
fi
fi fi
fi
fi
fi
fi
fi
fi fi
fl
fi
fi
fi
fi
fi
fi
fi
fi fi
fi
fi
fi fi
fi
fi
fl
fi
fi
fi
fi
fi
fl
fi
fi fi
fl
fi
fi
fi
fi
Approaches for creating therapy games
Abstract
Therapy games have the potential to offer people with disabilities a cost-effective, personalized, data-driven, connected, and motivating context for otherwise tedious and repetitive therapy. The paramount challenge in creating therapy games is creating a motivating experience with mechanics that translate into improved health outcomes---a wicked problem. To this end, I use research through design to explore multiple approaches to the co-creation of therapy games for various populations, including children with speech impairments, adults with developmental disabilities, children with Sensory-Based Motor Disorder (SBMD), and stroke survivors. I have collaborated on 3 therapy games, which serve as case studies where I explore identifying best practices, unique insights, and suggestions for future therapy game creators. Specifically, I discuss game-first versus therapy-first approaches, closed-game systems versus more open-ended playful systems, and potential future research directions.
... In another similar study, Ramamurthy et al. proposed a therapy robot, "Buddy," allowing children to practice assigned exercises at home (Ramamurthy & Li, 2018). Many studies have incorporated serious games as an intervention tool for automatic speech therapy (Anjos et al., 2017;Bílková et al., 2020;Duval, 2020;Duval et al., 2018;Ramamurthy & Li, 2018). One of the studies incorporated augmented reality to build a serious game using tongue detection (Bílková et al., 2020). ...
... Desolda, Lanzilotti, Piccinno, & Rossano, 2021;Duval, 2020;Mahmut, Nicola, & Stoicu-Tivadar, 2018;Ng et al., 2020Ng et al., , 2018V. Robles-Bykbaev et al., 2017V. ...
This paper presents a systematic literature review of published studies on AI-based automated speech therapy tools for persons with speech sound disorders (SSD). The COVID-19 pandemic has initiated the requirement for automated speech therapy tools for persons with SSD making speech therapy accessible and affordable. However, there are no guidelines for designing such automated tools and their required degree of automation compared to the conventional speech therapy given by Speech Language Pathologists (SLPs). In this systematic review, we followed the PRISMA framework to address four research questions: 1) what types of SSD do AI-based automated speech therapy tools address, 2) what is the level of autonomy achieved by such tools, 3) what are the different modes of intervention, and 4) how effective are such tools in comparison with the conventional mode of speech therapy. An extensive search was conducted on digital libraries to find research papers relevant to our study from 2007 to 2022. The results show that AI-based automated speech therapy tools for persons with SSD are increasingly gaining attention among researchers. Articulation disorders were the most frequently addressed SSD based on the reviewed papers. Further, our analysis shows that most researchers proposed fully automated tools without considering the role of other stakeholders. Our review indicates that mobile-based and gamified applications were the most frequent mode of intervention. The results further show that only a few studies compared the effectiveness of such tools compared to the conventional mode of speech therapy. Our paper presents the state-of-the-art in the field, contributes significant insights based on the research questions, and provides suggestions for future research directions.
... However, 12 studies out of 24 studies did not address any specific SSD (see Figure 9). These studies proposed automated tools for a generalized SSD population and experimented without specifying any particular SSD (Das & Saha, 2017;Desolda, Lanzilotti, Piccinno, & Rossano, 2021;Duval, 2020;Mahmut, Nicola, & Stoicu-Tivadar, 2018;Ng et al., 2020Ng et al., , 2018V. Robles-Bykbaev et al., 2017V. ...
... In another similar study, Ramamurthy et al. proposed a therapy robot, "Buddy," allowing children to practice assigned exercises at home (Ramamurthy & Li, 2018). Many studies have incorporated serious games as an intervention tool for automatic speech therapy (Anjos et al., 2017;Bílková et al., 2020;Duval, 2020;Duval et al., 2018;Ramamurthy & Li, 2018). One of the studies incorporated augmented reality to build a serious game using tongue detection (Bílková et al., 2020). ...
This paper presents a systematic literature review of published studies on AI-based automated speech therapy tools for persons with speech sound disorders (SSD). The COVID-19 pandemic has initiated the requirement for automated speech therapy tools for persons with SSD making speech therapy accessible and affordable. However, there are no guidelines for designing such automated tools and their required degree of automation compared to human experts. In this systematic review, we followed the PRISMA framework to address four research questions: 1) what types of SSD do AI-based automated speech therapy tools address, 2) what is the level of autonomy achieved by such tools, 3) what are the different modes of intervention, and 4) how effective are such tools in comparison with human experts. An extensive search was conducted on digital libraries to find research papers relevant to our study from 2007 to 2022. The results show that AI-based automated speech therapy tools for persons with SSD are increasingly gaining attention among researchers. Articulation disorders were the most frequently addressed SSD based on the reviewed papers. Further, our analysis shows that most researchers proposed fully automated tools without considering the role of other stakeholders. Our review indicates that mobile-based and gamified applications were the most frequent mode of intervention. The results further show that only a few studies compared the effectiveness of such tools compared to expert Speech-Language Pathologists (SLP). Our paper presents the state-of- the-art in the field, contributes significant insights based on the research questions, and provides suggestions for future research directions.
... Developing games is time-consuming and costly. Overly clinical therapy games run the risk of being boring, which defeats the purpose of using games to motivate healing in the irst place [10,23]. In this work, we adapt and repurpose an existing immersive virtual reality (iVR) game, Spellcasters, originally designed purely for entertainment for use as a stroke rehabilitation gameÐwhich is particularly relevant in the wake of COVID-19, where telehealth solutions are increasingly needed [4]. ...
Developing games is time-consuming and costly. Overly clinical therapy games run the risk of being boring, which defeats the purpose of using games to motivate healing in the first place [10, 23]. In this work, we adapt and repurpose an existing immersive virtual reality (iVR) game, Spellcasters, originally designed purely for entertainment for use as a stroke rehabilitation game—which is particularly relevant in the wake of COVID-19, where telehealth solutions are increasingly needed [4]. In preparation for participatory design sessions with stroke survivors, we collaborate with 14 medical professionals to ensure Spellcasters is safe and therapeutically valid for clinical adoption. We present our novel VR sandbox implementation that allows medical professionals to customize appropriate gestures and interactions for each patient's unique needs. Additionally, we share a co-designed companion app prototype based on clinicians' preferred data reporting mechanisms for telehealth. We discuss insights about adapting and repurposing entertainment games as serious games for health, features that clinicians value, and the potential broader impacts of applications like Spellcasters for stroke management.
... Interaction within such environments has been found to yield a plethora of benefits for a variety of health related applications stimulate mental, behavioral, and physical outcomes [13]- [19]. Yet, the challenge in utilizing virtual environments as games for health comes in finding a balance between therapy and entertainment, maintaining player engagement, and ensuring an accessible and positive user experience [20], [21]. ...
In the United States, social anxiety disorder is one of the most prevalent health challenges with psychological symptoms that impact multiple areas of life; however, one of the barriers to seeking treatment is the phobia itself. Therefore, one possible solution to this issue is to make exposure therapy more attractive and accessible. This paper explores the viability of gamification of treatment solutions through a serious game that affords users to access social exposure therapy to improve the at-home exercise and relapse prevention experience. We aim to simulate real-life interactions during a time when many individuals are socially isolated due to the global COVID-19 pandemic through a safe platform for gamified public speaking. Thus, we present a game designed towards assisting the reduction of the pervasiveness in social anxiety.
Physical training can be frustrating and hard, especially for those who experience additional challenges to access and control their proprioceptive senses. In the context of designing for children with Sensory-based Motor Disorder, we designed and deployed a series of Training Technology Probes to be used in circus training. Here we focus on how these were used, tested, and appropriated by children and instructors during a six-week circus training course. Through these explorations, we identified a range of potential benefits from using their functions in training. We present the Physical Training Technology Probes and the benefits they brought to training. We show how the technology functions helped children focus and provided feedback related to posture and balance. Furthermore, their open-ended designs and versatile options for use were crucial in exploring their contributions to training, and in how they helped foster creative engagement with technology and training. Our work contributes towards understanding the specific requirements when designing for the target group, and more generally contributes with design strategies for technology support for skill training.
SpokeIt is a mobile serious game for health designed to support speech articulation therapy. Here, we present SpokeIt as well as 2 preceding speech therapy prototypes we built, all of which use a novel offline critical speech recognition system capable of providing feedback in real-time. We detail key design motivations behind each of them and report on their potential to help adults with speech impairment co-occurring with developmental disabilities. We conducted a qualitative within-subject comparative study on 5 adults within this target group, who played all 3 prototypes. This study yielded refined functional requirements based on user feedback, relevant reward systems to implement based on user interest, and insights on the preferred hybrid game structure, which can be useful to others designing mobile games for speech articulation therapy for a similar target group.
A lack of intrinsic motivation to practice speech is attributed to tedious and repetitive speech curriculums, but mobile games have been widely recognized as a valid motivator for jaded individuals. SpokeIt is an interactive storybook style speech therapy game that intends to turn practicing speech into a motivating and productive experience for individuals with speech impairments as well as provide speech therapists an important diagnostic tool. In this paper, I discuss the novel intellectual contributions SpokeIt can provide such as an offline critical conversational speech recognition system, and the application of therapy curriculums to mobile platforms, I present conducted research, and consider exciting future work and research directions.
Children with speech impairments often find speech curriculums tedious, limiting how often children are motivated to practice. A speech therapy game has the potential to make practice fun, may help facilitate increased time and quality of at-home speech therapy, and lead to improved speech. We explore using conversational real-time speech recognition, game methodologies theorized to improve immersion and flow, and user centered approaches to design an immersive interactive speech therapy solution. Our preliminary user evaluation showed that compared to traditional methods, children were more motivated to practice speech using our system.
The concept of playification has recently been proposed as an extension of, or alternative to, gamification. We present a playification design project targeting the re-design of physiotherapy rehabilitative sessions for elderly inpatients. The menial and repetitive nature of the physical exercises targeted for design might seem ideal for shallow widespread gamification approaches that add external rewards to entice usage. In the PhySeEar project, we introduced a "third agent" instead, in the form of technology that would take over some of the work typically carried out by the physiotherapist. This technological intervention triggered the emergence of playfulness, when inpatients and the therapist re-signified the ongoing activity by engaging in playful role-taking, such as blaming the technology for mistakes, or for sensitivity to the inpatient's inaccurate movements. Based on the experiences from this project, we discuss some of the major differences between playification and gamification
Serious Games (SGs) are gaining an ever increasing interest for education and traning. Exploiting the latest simulation and visualization technologies, SGs are able to contextualize the player’s experience in challenging, realistic environments, supporting situated cognition. However, we still miss methods and tools for effectively and deeply infusing pedagogy and instruction inside digital games. After presenting an overview of the state of the art of the SG taxonomies, the paper introduces the pedagogical theories and models most relevant to SGs and their implications on SG design. We also present a schema for a proper integration of games in education, supporting different goals in different steps of a formal education process. By analyzing a set of well-established SGs and formats, the paper presents the main mechanics and models that are being used in SG designs, with a particular focus on assessment, feedback and learning analytics. An overview of tools and models for SG design is also presented. Finally, based on the performed analysis, indications for future research in the field are provided.
The usefulness and effectiveness of specific serious games in the medical
domain is often unclear. This is caused by a lack of supporting
evidence on validity of individual games, as well as a lack of publicly
available information. Moreover, insufficient understanding of design
principles among the individuals and institutions that develop or apply a medical
serious game compromises their use. This article provides the first
consensus-based framework for the assessment of specific medical serious
games. The framework provides 62 items in 5 main themes, aimed at assessing
a serious game’s rationale, functionality, validity, and data safety. This
will allow caregivers and educators to make balanced choices when applying a
serious game for healthcare purposes. Furthermore, the framework provides
game manufacturers with standards for the development of new, valid serious
games.
People using wheelchairs face barriers in their daily lives, many of which are created by people who surround them. Promoting positive attitudes towards persons with disabilities is an integral step in removing these barriers and improving their quality of life. In this context, persuasive games offer an opportunity of encouraging attitude change. We created a wheelchair-controlled persuasive game to study how embodied interaction can be applied to influence player attitudes over time. Our results show that the game intervention successfully raised awareness for challenges that people using wheelchairs face, and that embodied interaction is a more effective approach than traditional input in terms of retaining attitude change over time. Based on these findings, we provide design strategies for embodied interaction in persuasive games, and outline how our findings can be leveraged to help designers create effective persuasive experiences beyond games.
What if every part of our everyday life was turned into a game? The implications of “gamification.”
What if our whole life were turned into a game? What sounds like the premise of a science fiction novel is today becoming reality as “gamification.” As more and more organizations, practices, products, and services are infused with elements from games and play to make them more engaging, we are witnessing a veritable ludification of culture.
Yet while some celebrate gamification as a possible answer to mankind's toughest challenges and others condemn it as a marketing ruse, the question remains: what are the ramifications of this “gameful world”? Can game design energize society and individuals, or will algorithmicincentive systems become our new robot overlords?
In this book, more than fifty luminaries from academia and industry examine the key challenges of gamification and the ludification of culture—including Ian Bogost, John M. Carroll, Bernie DeKoven, Bill Gaver, Jane McGonigal, Frank Lantz, Jesse Schell, Kevin Slavin, McKenzie Wark, and Eric Zimmerman. They outline major disciplinary approaches, including rhetorics, economics, psychology, and aesthetics; tackle issues like exploitation or privacy; and survey main application domains such as health, education, design, sustainability, or social media.
Objective:
The purpose of this study was to explore the design of a health game that aims to both support tobacco-related health literacy and a tobacco-free life in early adolescence and to meet adolescents' expectations.
Materials and methods:
Data were collected from adolescents using an open-ended questionnaire (n = 83) and focus groups (n = 39) to obtain their view of a health game used for tobacco-related health education. The data were analyzed using thematic analysis. A group of experts combined the adolescents' views with theoretical information on health literacy and designed and produced the first version of the game. Adolescents (session 1, n = 16; session 3, n = 10; and session 4, n = 44) and health promotion professionals (session 2, n = 3) participated in testing the game. Feedback from testing sessions 3 and 4 was analyzed using descriptive statistics.
Results:
Adolescents pointed out that the health game needs to approach the topic of tobacco delicately and focus on the adolescents' perspective and on the positive sides of a tobacco-free life rather than only on the negative consequences of tobacco. The adolescents expected the game to be of high quality, stimulating, and intellectually challenging and to offer possibilities for individualization. Elements from the adolescents' view and theoretical modelling were embedded into the design of a game called Fume. Feedback on the game was promising, but some points were highlighted for further development.
Conclusion:
Investing especially in high-quality design features, such as graphics and versatile content, using humoristic or otherwise stimulating elements, and maintaining sufficiently challenging gameplay would promote the acceptability of theory-based health games among adolescents.