Project

Gesture Therapy

Goal: The goal is the development of a virtual reality-based motor neurorehabilitation platform for the upper limb, i.e. the Gesture Therapy.

Importantly, this is more a research line than a project, and in addition to this technological target, the project or research line cover research from the neuroscientific first principles of neurorehabilitation e.g. including fMRI neuroimaging studies to understand brain reorganization after injury, to pure technological developments.

Date: 31 December 2007

Updates
0 new
1
Recommendations
0 new
0
Followers
0 new
2
Reads
0 new
54

Project log

Jesús Joel Rivas
added 2 research items
Data from multiple sensors can boost the automatic recognition of multiple affective states in a multilabel and multimodal recognition system. At any time, the streaming from any of the contributing sensors can be missing. This work proposes a method for dealing with a missing sensor in a multilabel and multimodal automatic affective states recognition system. The proposed method, called Hot Deck using Conditional Probability Tables (HD-CPT), is incorporated into a multimodal affective state recognition system for compensating the loss of a sensor using the recorded historical information of the sensor and its interaction with the other available sensors. In this work, we consider a multilabel classifier, named Circular Classifier Chain, for the automatic recognition of four states: tiredness, anxiety, pain, and engagement; combined with a multimodal classifier based on three sensors: fingers pressure, hand movements, and facial expressions; which was adapted for coping with the problem of a missing sensor in a virtual rehabilitation platform for post-stroke patients. A dataset of five post-stroke patients who attended ten longitudinal rehabilitation sessions was used for the evaluation. The inclusion of HD-CPT compensated for the loss of one sensor with results above those obtained with only the remaining sensors available. HD-CPT prevents the system from collapsing when a sensor fails, providing continuity of operation with results that attenuate the loss of the sensor. The proposed method HD-CPT can provide robustness for the naturalistic everyday use of an affective states recognition system.
Data from multiple sensors can boost the automatic recognition of multiple affective states in a multilabel and multi-modal recognition system. At any time, the streaming from any of the contributing sensors can be missing. This work proposes a method for dealing with a missing sensor in a multilabel and multimodal automatic affective states recognition system. The proposed method, called Hot Deck using Conditional Probability Tables (HD-CPT), is incorporated into a multimodal affective state recognition system for compensating the loss of a sensor using the recorded historical information of the sensor and its interaction with the other available sensors. In this work, we consider a multilabel classifier, named Circular Classifier Chain, for the automatic recognition of four states: tiredness, anxiety, pain, and engagement; combined with a multimodal classifier based on three sensors: fingers pressure, hand movements, and facial expressions; which was adapted for coping with the problem of a missing sensor in a virtual rehabilitation platform for post-stroke patients. A dataset of five post-stroke patients who attended ten longitudinal rehabilitation sessions was used for the evaluation. The inclusion of HD-CPT compensated for the loss of one sensor with results above those obtained with only the remaining sensors available. HD-CPT prevents the system from collapsing when a sensor fails, providing continuity of operation with results that attenuate the loss of the sensor. The proposed method HD-CPT can provide robustness for the naturalistic everyday use of an affective states recognition system.
Felipe Orihuela-Espina
added a project goal
The goal is the development of a virtual reality-based motor neurorehabilitation platform for the upper limb, i.e. the Gesture Therapy.
Importantly, this is more a research line than a project, and in addition to this technological target, the project or research line cover research from the neuroscientific first principles of neurorehabilitation e.g. including fMRI neuroimaging studies to understand brain reorganization after injury, to pure technological developments.
 
Felipe Orihuela-Espina
added an update
Gesture therapy (GT) is both a rehabilitation concept and a platform for supporting this concept. As a concept, GT is a virtual reality based motor rehabilitation therapy which favours the three pillars of rehabilitation (repetition, feedback and motivation) by challenging the patient to fulfill daily tasks in a safe virtual environment. The tasks are presented in the form of short serious games. As a platform, GT provides the physical and virtual elements to realise the concept.
This is more a research line than a project, and as a research line it has received funding from several projects. The research line has been active since 2008 more or less, and over 20 people have participated in it throguhout the years.
For further information, please visit the platform website: http://robotic.inaoep.mx/~foe/blog/
 
Felipe Orihuela-Espina
added 16 research items
Background: Gesture Therapy is an upper limb virtual reality rehabilitation-based therapy for stroke survivors. It promotes motor rehabilitation by challenging patients with simple computer games representative of daily activities for self-support. This therapy has demonstrated clinical value, but the underlying functional neural reorganization changes associated with this therapy that are responsible for the behavioral improvements are not yet known. Objective: We sought to quantify the occurrence of neural reorganization strategies that underlie motor improvements as they occur during the practice of Gesture Therapy and to identify those strategies linked to a better prognosis. Methods: Functional magnetic resonance imaging (fMRI) neuroscans were longitudinally collected at 4 time points during Gesture Therapy administration to 8 patients. Behavioral improvements were monitored using the Fugl-Meyer scale and Motricity Index. Activation loci were anatomically labelled and translated to reorganization strategies. Strategies are quantified by counting the number of active clusters in brain regions tied to them. Results: All patients demonstrated significant behavioral improvements (P < .05). Contralesional activation of the unaffected motor cortex, cerebellar recruitment, and compensatory prefrontal cortex activation were the most prominent strategies evoked. A strong and significant correlation between motor dexterity upon commencing therapy and total recruited activity was found (r2 = 0.80; P < .05), and overall brain activity during therapy was inversely related to normalized behavioral improvements (r2 = 0.64; P < .05). Conclusions: Prefrontal cortex and cerebellar activity are the driving forces of the recovery associated with Gesture Therapy. The relation between behavioral and brain changes suggests that those with stronger impairment benefit the most from this paradigm.
We explore the use of a virtual rehabilitation platform as the interaction means for physical activation and cognitive stimulation of elders. A usability evaluation of actual and projected use of the tool suggests that this could be feasible to perform. Elders perceived the use of the evaluated tool as useful (93.75/100), easy to use (93.75/100) and pleasurable to use (91.66/100) during an actual activation and stimulation session. Previous experience on the use of computers by the participants did not significantly impact on their usability perception for most of the included factors, with the sole exception being the perception of anxiety. This is an encouraging result to reuse and adapt technologies from “close” domains (e.g., virtual rehabilitation). In addition, this can reduce development times and cost, and facilitate knowledge transfer into the domain of physical activation and cognitive stimulation of elders.
The use of serious games, those not intended for leisure, out of their intended context may have consequences for evoked engagement and for the facilitation of transversal reutilization of developed games. However, the effectiveness and usability of serious games out of their intended context must be questioned. Here, we present two case studies in virtual rehabilitation. In a first experiment, a classical leisure game (i.e. pong) is compared in its ability to propose a valid alternative to games specifically developed for virtual rehabilitation use in terms of compliance with principles required in motor rehabilitation such as movement repetition, feedback, task significance and motivation. The second experiment explores the complementary situation in which a virtual rehabilitation game set is exported as a plausible tool for the cognitive stimulation of elders. Not unexpectedly, our results suggest that transfer of games across realms is, in some cases, a feasible enterprise but their effectiveness and usability may be compromised. In particular, we found that importing pong to a virtual rehabilitation environment is not a valid alternative in the sense that it does not obey rehabilitation principles. However, exporting serious games across related domains, i.e. from virtual rehabilitation to cognitive stimulation, has encountered a high degree of acceptance by the receiving community of users, being perceived as useful and easy to use, and we further establish that this acceptance cannot be explained by expertise in using technology. Qualitative assessment of usersÕ responses points to the interaction interface as a co-responsible factor for this acceptance of the serious virtual environment. These findings may have consequences for the future development of serious games both in terms of reusing ideas and alerting about possible difficulties during the migration process.
Jesús Joel Rivas
added a research item
The automatic recognition of multiple affective states can be enhanced if the underpinning computational models explicitly consider the interactions between the states. This work proposes a computational model that incorporates the dependencies between four states (tiredness, anxiety, pain, and engagement) known to appear in virtual rehabilitation sessions of post-stroke patients, to improve the automatic recognition of the patients’ states. A dataset of five stroke patients which includes their fingers’ pressure (PRE), hand movements (MOV) and facial expressions (FAE) during ten sessions of virtual rehabilitation was used. Our computational proposal uses the Semi-Naive Bayesian classifier (SNBC) as base classifier in a multiresolution approach to create a multimodal model with the three sensors (PRE, MOV, and FAE) with late fusion using SNBC (FSNB classifier). There is a FSNB classifier for each state, and they are linked in a circular classifier chain (CCC) to exploit the dependency relationships between the states. Results of CCC are over 90% of ROC AUC for the four states. Relationships of mutual exclusion between engagement and all the other states and some co-occurrences between pain and anxiety for the five patients were detected. Virtual rehabilitation platforms that incorporate the automatic recognition of multiple patient’s states could leverage intelligent and empathic interactions to promote adherence to rehabilitation exercises.
Felipe Orihuela-Espina
added a research item
Background. Adaptation and customization are two related but distinct concepts that are central to virtual rehabilitation if this motor therapy modality is to succeed in alleviating the demand for expert supervision. These two elements of the therapy are required to exploit the flexibility of virtual environments to enhance motor training and boost therapy outcome. Aim. The chapter provides a non-systematic overview of the state of the art regarding the evolving manipulation of virtual rehabilitation environments to optimize therapy outcome manifested through customization and adaptation mechanisms. Methods. Both concepts will be defined, aspects guiding their implementation reviewed, and available literature suggesting different solutions discussed. We present "Gesture Therapy", a platform realizing our contributions to the field and we present results of the adaptation techniques integrated into it. Less explored additional dimensions such as liability and privacy issues affecting their implementation are briefly discussed. Results. Solutions to implement decision-making on how to manipulate the environment are varied. They range from predefined system configurations to sophisticated artificial intelligence (AI) models. Challenge maintenance and feedback personalization is the most common driving force for their incorporation to virtual rehabilitation platforms. Conclusions. Customization and adaptation are the main mechanisms responsible for the full exploitation of the potential of virtual rehabilitation environments, and the potential benefits are worth pursuing. Despite encouraging evidence of the many solutions proposed thus far in literature, none has yet proven to substantially alter the therapy outcome. In consequence, research is still on going to equip virtual rehabilitation solutions with efficacious tailoring elements.
Jesús Joel Rivas
added a research item
Virtual rehabilitation platforms may tailor the rehabilitation tasks to the patients' needs if they could recognize the patient's affective state. Affective states recognition systems can enhance their performance if they receive data coming from different sensors of human behaviour. In this work, we propose a late Fusion using Semi-Naive Bayesian classifier (FSNB) as a multimodal affective states recognition system to infer four states: tiredness, anxiety, pain, and motivation, from observable metrics of fingers pressure, hand movements, and facial expressions of post-stroke patients. Data streams were recorded from 5 post-stroke patients while they attended virtual rehabilitation therapies along 10 sessions over 4 weeks, manifesting the aforementioned states spontaneously. Recognition rates of the FSNB classifier were over 90% (with a standard deviation of around ± 0.06) of AUC for the four states. These results represent contributions for enhancing the development of affective states recognition systems in virtual rehabilitation.
Patrick Heyer
added 6 research items
Unacted posture conveys cues about people’s attentional dis- position. We aim to identify robust markers of attention from posture while people carry out their duties seated in front of their computers at work. Body postures were randomly captured from 6 subjects while at work using a Kinect, and self-assessed as attentive or not attentive. Robust postural features exhibiting higher discriminative power across classification exercises with 4 well-known classifiers were identified. Av- erage classification of attention from posture reached 76.47%±4.58% (F- measure). A total of 40 postural features were tested and those proxy of head tilt were found to be the most stable markers of attention in seated conditions based upon 3 class separability criteria. Unobtrusively moni- toring posture of users while working in front of a computer can reliably be used to infer attentional disposition from the user. Human-computer interaction systems can benefit from this knowledge to customize the experience to the user changing attentional state.
Given its virtually algorithmic process, the Fugl-Meyer Assessment (FMA) of motor recovery is prone to automatization reducing subjectivity, alleviating therapists’ burden and collaterally reducing costs. Several attempts have been recently reported to achieve such automatization of the FMA. However, a cost-effective solution matching expert criteria is still unfulfilled, perhaps because these attempts are sensor-specific representation of the limb or have thus far rely on a trial and error strategy for building the underpinning computational model. Here, we propose a sensor abstracted representation. In particular, we improve previously reported results in the automatization of FMA by classifying a manifold embedded representation capitalizing on quaternions, and explore a wider range of classifiers. By enhancing the modeling, overall classification accuracy is boosted to 87% (mean: 82% ± 4.53:) well over the maximum reported in literature thus far 51.03% (mean: 48.72 ± std: 2.10). The improved model brings automatic FMA closer to practical usage with implications for rehabilitation programs both in ward and at home.
Attribution of attention from observable body pos- ture is plausible, providing additional information for affective computing applications. We previously reported a promissory 69.72 ± 10.50 (μ ± σ) of F-measure to use posture as a proxy for attributed attentional state with implications for affective comput- ing applications. Here, we aim at improving that classification rate by reweighting votes of raters giving higher confidence to those raters that are representative of the raters population. An increase to 75.35 ± 11.66 in F-measure was achieved. The improvement in predictive power by the classifier is welcomed and its impact is still being assessed.
Patrick Heyer
added a research item
Motor dexterity assessment is regularly performed in rehabilitation wards to establish patient status and automatization for such routinary task is sought. A system for automatizing the assessment of motor dexterity based on the Fugl-Meyer scale and with loose restrictions on sensing technologies is presented. The system consists of two main elements: 1) A data representation that abstracts the low level information obtained from a variety of sensors, into a highly separable low dimensionality encoding employing t-distributed Stochastic Neighbourhood Embedding, and, 2) central to this communication, a multi-label classifier that boosts classification rates by exploiting the fact that the classes corresponding to the individual exercises are naturally organized as a network. Depending on the targeted therapeutic movement class labels i.e. exercises scores, are highly correlated-patients who perform well in one, tends to perform well in related exercises-; and critically no node can be used as proxy of others - an exercise does not encode the information of other exercises. Over data from a cohort of 20 patients, the novel classifier outperforms classical Naive Bayes, random forest and variants of support vector machines (ANOVA: p < 0.001). The novel multi-label classification strategy fulfills an automatic system for motor dexterity assessment, with implications for lessening therapist's workloads, reducing healthcare costs and providing support for home-based virtual rehabilitation and telerehabilitation alternatives.