Conference Paper

Dynamic Movement Monitoring - Algorithms for Real Time Exercise Movement Feedback

Authors:
If you want to read the PDF, try requesting it from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... In order to tackle this problem, the mechanism of a monitoring algorithm while doing a movement execution can be advantageous in these situations. Such strategy is often applied in exergames where therapeutic exercises have to be executed as interaction method while playing a game [18]. For instance, Tsilidi and Pachoulakis [19] developed an exergame for rehabilitation patients of the carpal tunnel syndrome. ...
Conference Paper
Scary climbing situations can appear when climbing on big walls. Some stones may be slippery due to humidity or smoothness and distances between belaying bolts can be large whereby a deep fall is risked. In such situations, the stress level increases, climbing performance decreases and an ascent might even be aborted. Furthermore, hazardous accidents can happen when the concentration is disturbed by stress and fear. Moreover, a beginner has to overcome some hurdles such as learning movements and belaying techniques as well as overcoming acrophobia or fear of falling down. We try to facilitate these hurdles with augmentation of the human in Mixed Reality (MR) superhuman sport climbing. In this paper, we present the development of a MR climbing system that tackles these hurdles by allowing a user to climb in safe height at ground level in reality while seeing his or her own body visualised in a much more demanding situation (e.g. in the middle of a big wall) in a Virtual Environment (VE) through a HMD. Additionally an audience can be integrated into the VE via a collaboration with an operator of a virtual drone. The drone operator has the ability to provide additional feedback, e.g. point to a certain stone with a virtual laser and can communicate with the climber via microphone. Hence, if a climber gets stuck in a climbing route, the drone operator can suggest the next move in the VE.
Article
Full-text available
We report on a Kinect-based, augmented reality, real-time physiotherapy platform tailored to Parkinson’s disease (PD) patients. The platform employs a Kinect sensor to extract real-time 3D skeletal data (joint information) from a patient facing the sensor (at 30 frames per second). In addition, a small collection of exercises practiced in traditional physiotherapy for PD patients has been implemented in the Unity 3D game engine. Each exercise employs linear or circular movement patterns and poses very light-weight processing demands on real-time computations. During an exercise, trainer instruction demonstrates correct execution and Kinect-provided 3D joint data are fed to the game engine and compared to exercise-specific control routines to assess proper posture and body control in real time. When an exercise is complete, performance metrics appropriate for that exercise are computed and displayed on screen to enable the attending physiotherapist to fine-tune the exercise to the abilities/needs of an individual patient as well as to provide performance feedback to the patient. The platform can operate in a physiotherapist’s office and, following appropriate validation, in a home environment. Finally, exercises can be parameterized meaningfully, depending on the intended purpose (motor assessment versus plain exercise at home).
Article
Full-text available
The adverse effects of improper posture and unhealthy computer usage habits on human physiology have been well documented. Prominent among these, the Carpal Tunnel Syndrome (CTS) is a medical condition where the median nerve passing through the wrist’s carpal tunnel is compressed causing pain, numbness and tingling in affected parts of the hand. Depending on the severity of the symptoms, treatment may include physiotherapy and/or surgery. The present paper discusses possibilities that technology-driven physiotherapy based on serious games can augment a more traditional physiotherapy exercise curriculum. Accordingly, we present a Unity3D game called Roller Ball, the scenario of which combines CTS-specific physiotherapy exercises in a natural scenario-based way to guide a ball across a bridge in a 3D scene. The game employs the Leap Motion sensor, whose detailed wrist and hand (including fingers) tracking abilities make it a promising hardware platform for rehabilitation oriented exercises intended for patients suffering from CTS.
Article
Full-text available
This paper presents an intelligent Kinect and fuzzy inference system based e-rehabilitation system. The Kinect can detect the posture and motion of the patients while the fuzzy inference system can interpret the acquired data on the cognitive level. The system is capable to assess the initial posture and motion ranges of 20 joints. Using angles to describe the motion of the joints, exercise patterns can be developed for each patient. Using the exercise descriptors the fuzzy inference system can track the patient and deliver real-time feedback to maximize the efficiency of the rehabilitation. The first laboratory tests confirm the utility of this system for the initial posture detection, motion range and exercise tracking.
Article
Full-text available
This article describes a novel approach to realtime motion assessment for rehabilitation exercises based on the integration of comprehensive kinematic modeling with fuzzy inference. To facilitate the assessment of all important aspects of a rehabilitation exercise, a kinematic model is developed to capture the essential requirements for static poses, dynamic movements, as well as the invariance that must be observed during an exercise. The kinematic model is expressed in terms of a set of kinematic rules. During the actual execution of a rehabilitation exercise, the similarity between the measured motion data and the model is computed in terms of their distances, which are then used as inputs to a fuzzy interference system to derive the overall quality of the execution. The integrated approach provides both a detailed categorical assessment of the overall execution of the exercise and the degree of adherence to individual kinematic rules.
Article
Full-text available
Interactive systems are being developed with the intention to help in the engagement of patients on various therapies. Amid the recent technological advances, Kinect™ from Microsoft (Redmond, WA) has helped pave the way on how user interaction technology facilitates and complements many clinical applications. In order to examine the actual status of Kinect developments for rehabilitation, this article presents a systematic review of articles that involve interactive, evaluative, and technical advances related to motor rehabilitation. Systematic research was performed in the IEEE Xplore and PubMed databases using the key word combination "Kinect AND rehabilitation" with the following inclusion criteria: (1) English language, (2) page number >4, (3) Kinect system for assistive interaction or clinical evaluation, or (4) Kinect system for improvement or evaluation of the sensor tracking or movement recognition. Quality assessment was performed by QualSyst standards. In total, 109 articles were found in the database research, from which 31 were included in the review: 13 were focused on the development of assistive systems for rehabilitation, 3 in evaluation, 3 in the applicability category, 7 on validation of Kinect anatomic and clinical evaluation, and 5 on improvement techniques. Quality analysis of all included articles is also presented with their respective QualSyst checklist scores. Research and development possibilities and future works with the Kinect for rehabilitation application are extensive. Methodological improvements when performing studies on this area need to be further investigated.
Conference Paper
Full-text available
Particularly in sports or physical rehabilitation, users have to perform body movements in a specific manner for the exercises to be most effective. It remains a challenge for experts to specify how to perform such movements so that an automated system can analyse further performances of it. In a user study with 10 participants we show that experts' explicit estimates do not correspond to their performances. To address this issue we present MotionMA, a system that: (1) automatically extracts a model of movements demonstrated by one user, e.g. a trainer, (2) assesses the performance of other users repeating this movement in real time, and (3) provides real-time feedback on how to improve their performance. We evaluated the system in a second study in which 10 other participants used the system to demonstrate arbitrary movements. Our results demonstrate that MotionMA is able to extract an accurate movement model to spot mistakes and variations in movement execution.
Conference Paper
Full-text available
In the framework of our Reactive Virtual Trainer (RVT) project, we are developing an Intelligent Virtual Agent (IVA) capable to act similarly to a real trainer. Besides presenting the physical exercises to be performed, she keeps an eye on the user. She provides feedback whenever appropriate, to in- troduce and structure the exercises, to make sure that the exercises are per- formed correctly, and also to motivate the user. In this paper we talk about the corpora we collected, serving a basis to model repetitive exercises on high level. Then we discuss in detail how the actual performance of the user is com- pared to what he should be doing, what strategy is used to provide feedback, and how it is generated. We provide preliminary feedback from users and out- line further work.
Conference Paper
Full-text available
Games for rehabilitation are developing rapidly in recent years. It has been shown that utilization of therapy and gaming technology affects positively on the patients' physical and mental condition. However, to this day there are only few playable games for Parkinson's disease patients. This paper presents the development process of WuppDi! – a collection of five motion-based games for Parkinson's disease patients, aimed at supporting their exercises routines in various playful environments. We describe the game design challenges for Parkinson's disease patients and our solutions used in the games. Finally, we present the results of a conducted field test showing a very positive motivational effect among the majority of the patients but also highlighting remaining issues and technical difficulties, which can be beneficial for the future development in this field.
Conference Paper
Full-text available
The last decade has witnessed a surge of interest in new sensing and monitoring devices for healthcare, with implantable in vivo monitoring and intervention devices being key developments in this area. Permanent implants combined with wearable monitoring devices could provide continuous assessment of critical physiological parameters for identifying precursors of major adverse events. Open research issues in this area are predominantly related to novel sensor interface design, practical and reliable distributed com- puting environments for multi-sensory data fusion, but the concept itself is deemed to have further impact in many other areas. This position paper describes a scenario from the UbiMon [1] project, which is aimed at investigating healthcare delivery by combining wearable and implantable sensors.
Article
Full-text available
In recent years, a growing number of occupational therapists have integrated video game technologies, such as the Nintendo Wii, into rehabilitation programs. 'Wiihabilitation', or the use of the Wii in rehabilitation, has been successful in increasing patients' motivation and encouraging full body movement. The non-rehabilitative focus of Wii applications, however, presents a number of problems: games are too difficult for patients, they mainly target upper-body gross motor functions, and they lack support for task customization, grading, and quantitative measurements. To overcome these problems, we have designed a low-cost, virtual-reality based system. Our system, Virtual Wiihab, records performance and behavioral measurements, allows for activity customization, and uses auditory, visual, and haptic elements to provide extrinsic feedback and motivation to patients.
Article
Full-text available
To explore the attitudes and beliefs of stroke patients identified by professionals as having either "high" or "low" motivation for rehabilitation. Qualitative study with semistructured interviews. The stroke unit of an inner city teaching hospital. 22 patients with stroke who were undergoing rehabilitation; 14 with high motivation for rehabilitation and eight with low motivation. All patients thought rehabilitation was important for recovery. High motivation patients were more likely to view rehabilitation as the most important means of recovery and to accord themselves an active role in rehabilitation. These patients were also more likely to understand rehabilitation and in particular to understand the specialist role of the nursing staff. Many patients reported independence at home as a personal goal, though few low motivation patients related this goal to success in rehabilitation. Information from professionals about rehabilitation, favourable comparisons with other stroke patients, and the desire to leave hospital had a positive effect on motivation. Conversely, overprotection from family members and professionals, lack of information or the receipt of "mixed messages" from professionals, and unfavourable comparisons with other patients had a negative effect. There are some differences in beliefs between stroke patients identified as having low or high motivation for rehabilitation. These beliefs seem to be influenced by the environment in which the patient is rehabilitated. Professionals and carers should be made aware of the ways in which their behaviour can positively and negatively affect motivation.
Article
Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking its specific motion, and use the correlation between the system’s output with the user’s input to determine the selection. The resulting interaction has compelling properties, as users are guided by motion feedback, and only need to copy a presented motion. Motion correlation has been explored in earlier work but only recently begun to feature in holistic interface designs. We provide a first comprehensive review of the principle, and present an analysis of five previously published works, in which motion correlation underpinned the design of novel gaze and gesture interfaces for diverse application contexts. We derive guidelines for motion correlation algorithms, motion feedback, choice of modalities, overall design of motion correlation interfaces, and identify opportunities and challenges identified for future research and design.
Conference Paper
In weight training, correct exercise execution is crucial for maximizing its effectiveness and for preventing injuries. However, given the complexity of these movements, it is a challenge for trainees to know whether they are performing the exercise correctly. Considering the fact that wrong moves may result in life long injuries, it is important to design systems that can detect incorrect performances automatically. In this paper, we present a workflow to detect performance anomalies from only observations of the correct performance of an exercise by the trainee. We evaluated our algorithm on a benchmark data set for the biceps curl exercise, and also evaluated our system with a publicly available dataset, and showed that our method detects unseen anomalies in weight lifting exercises with 98 percent accuracy.
Article
Background and objective: Interactive systems for rehabilitation have been widely investigated for motivational purposes. However, more attention should be given to the manner in which user movements are recognized and categorized. This paper aims to evaluate the efficacy of using a clinically-related gesture recognition tool, based on the international biomechanical standards (ISB) for the reporting of human joint motion, for the development of an interactive augmented reality (AR) rehabilitation system -mirrARbilitation. Methods: This work presents an AR rehabilitation system based on ISB standards, which enables the system to interact and to be configured according to therapeutic needs. The Kinect(TM) skeleton tracking technology was exploited and a new movement recognition method was developed to recognize and classify biomechanical movements. Further, our mirrARbilitation system provides exercise instructions while simultaneously motivating the patient. The system was evaluated on a cohort of 33 patients, physiotherapists, and software developers when performing shoulder abduction therapy exercises. Tests were performed in three moments: (i) users performed the exercise until they feel tired without the help of the system, (ii) the same however using the mirrARbilitation for motivation and guidance, and (iii) users performed the exercise again without the system. Users performing the movement without the help of the system worked as baseline reference. Results: We demonstrated that the percentage of correct exercises, measured by the movement analysis method we developed, improved from 69.02% to 93.73% when users interacted with the mirrARbilitation. The number of exercise repetitions also improved from 34.06 to 66.09 signifying that our system increased motivation of the users. The system also prevented the users from performing the exercises in a completely wrong manner. Finally, with the help of our system the users' worst result was performing 73.68% of the rehabilitation movements correctly. Besides the engagement, these results suggest that the use of biomechanical standards to recognize movements is valuable in guiding users during rehabilitation exercises. Conclusion: The proposed system proved to be efficient by improving the user engagement and exercise performance outcomes. The results also suggest that the use of biomechanical standards to recognize movements is valuable in guiding users during rehabilitation exercises.
Conference Paper
We propose in this paper new techniques for correction and parameterization of motion capture sequences containing upper-body exercises for physical therapy. By relying on motion capture sequences we allow therapists to easily record new patient-customized exercises intuitively by direct demonstration. The proposed correction and parameterization techniques allow the modification of recorded sequences in order to 1) correct and modify properties such as alignments and constraints, 2) customize prescribed exercises by modifying parameterized properties such as speed, wait times and exercise amplitudes, and 3) to achieve real-time adaptation by monitoring user performances and updating the parameters of each exercise for improving the therapy delivery. The proposed techniques allow autonomous virtual therapists to improve the whole therapy process, from exercise definition to delivery.
Article
Most formal rehabilitation facilities are situated in a hospital or care center setting, which may not always be conveniently accessible for patients, especially those in geographically isolated areas. Home-based rehabilitation has potential to offer greater accessibility and thus increase consistent uptake. In addition, the exercise performed in conventional rehabilitation contexts may be insufficient to ensure the patient's speedy recovery, with complimentary rehabilitation exercises at home required to make a difference. The goal is to provide effective home-based rehabilitation offering outcomes similar to those obtained through hospital-based rehabilitation under the supervision of an occupational therapist. This paper presents the development of a Kinect-based system for ensuring home-based rehabilitation using a Dynamic Time Warping (DTW) algorithm and fuzzy logic. The ultimate goal is to assist patients in conducting safe and effective home-based rehabilitation without the immediate supervision of a physician.
Article
presence of a professional may cause adverse event or lead to secondary injury. In this paper, we describe our development of a Kinect-based system - KHRD using Dynamic Time Warping (DTW) algorithm and fuzzy logic for ensuring home-based rehabilitation. The KHRD allows a patient perform a prescribed exercise with the presence of a professional. The exercise performed will then be recorded as a base for evaluating the patient's rehabilitation exercise at home. The outcomes of the evaluation can be used as a reference for the patient to validate his/her exercise and to prevent adverse events. A summary report of the outcomes may also be uploaded to a cloud setting for physicians to monitor the patient's progress and adjust the prescription. B. Kinect
Conference Paper
In this paper, we describe the design and evaluation of an interactive home-based rehabilitation visualisation system used by a wide variety of ages (users in our studies were aged from 47-89) to undertake rehabilitation in the home following knee replacement surgery. We present the rehabilitation visualization system and the results of a randomized controlled study in which we investigated the usability and feasibility of the system in the home. We found that our users were able to use the system successfully for their rehabilitation with improved rehabilitation outcomes after 6 weeks when compared to the current rehabilitation care. Finally we highlight the lessons learned which will benefit prospective designers of home rehabilitation technology in ensuring successful home evaluations in clinical rehabilitation.
Conference Paper
YouMove is a novel system that allows users to record and learn physical movement sequences. The recording system is designed to be simple, allowing anyone to create and share training content. The training system uses recorded data to train the user using a large-scale augmented reality mirror. The system trains the user through a series of stages that gradually reduce the user's reliance on guidance and feedback. This paper discusses the design and implementation of YouMove and its interactive mirror. We also present a user study in which YouMove was shown to improve learning and short-term retention by a factor of 2 compared to a traditional video demonstration.
Conference Paper
Musculoskeletal disorders are a globally significant health problem affecting millions. Physiotherapy, including prescribed exercises performed independently by patients in their homes, is a key treatment for many sufferers. However, many patients fail to complete home exercises, prolonging recovery periods or accelerating decline. Pervasive health technologies, capable of monitoring users in their homes, are ideally suited to address this problem. This paper describes user research with a group of three physiotherapists and eleven current physiotherapy patients to understand the problems and user needs underlying non-compliance with home exercise regimes. The research adopted a speed dating approach and culminated with six insights and design recommendations relating to the form and type of feedback that should be used in such systems, to how scheduling and therapist-patient communication systems should be designed and to the role of privacy.
Conference Paper
Patients' willingness to play is essential for rehabilitation games. This study (1) identified patients' general expectations for rehabilitation games; and (2) evaluated two newly developed low-cost rehabilitation games through surveys. Patients were found to expect low-cost systems (< $100), ease of use, interesting game contents, proven clinical efficiency, and access to rehabilitation games without prescription although welcoming therapists to follow their progress. House of quality analysis performed for the two developed rehabilitation games based on the patient expectations identified the need to improve reliability and precision of the low-cost hardware as well as to demonstrate clinical benefits.
Conference Paper
This paper describes a novel real-time Multimedia Rehabilitation Environment for the rehabilitation of patients with Parkinson’s Disease (PD). The system integrates two well known physical therapy techniques, multimodal sensory cueing and BIG protocol, with visual and auditory feedback to created an engaging mediated environment. The environment has been designed to fulfill the both the needs of the physical therapist and the patient. KeywordsParkinson’s Disease–Physical Therapy–Mediated Rehabilitation–Sensory Cueing–Multimodal Feedback–Virtual Environment
Article
This article tackles several problems faced by professionals in physiotherapy: the performance of the rehabilitation exercises by the patients, the control of the course of the illness and the patient's ignorance about whether or not he is properly performing the exercises. We propose a solution based on the use of the Wii Controller to control the exercise movements, along with software that provides the patient with an easy, intuitive and interactive control system. Finally, web services are used to allow the remote monitoring of the treatment by physiotherapy professionals.
Pt Viz: Towards a Wearable Device for Visualizing Knee Rehabilitation Exercises
  • S Ananthanarayan
  • M Sheh
  • A Chien
  • H Profita
  • K Siek
Ananthanarayan, S., Sheh, M., Chien, A., Profita, H., and Siek, K. (2013). Pt Viz: Towards a Wearable Device for Visualizing Knee Rehabilitation Exercises. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1247-1250.
Freehand interaction with leap motion controller for stroke rehabilitation
  • M Khademi
  • H Mousavi Hondori
  • A Mckenzie
  • L Dodakian
  • C V Lopes
  • S C Cramer
Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C. V., and Cramer, S. C. (2014). Freehand interaction with leap motion controller for stroke rehabilitation. Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems -CHI EA '14, (November 2015):1663-1668.
Rehabilitative Guidelines after Total Knee Arthroplasty: A
  • J B Mistry
  • R D Elmallah
  • A Bhave
  • M Chughtai
  • J J Cherian
  • T Mcginn
  • S F Harwin
  • M A Mont
Mistry, J. B., Elmallah, R. D., Bhave, A., Chughtai, M., Cherian, J. J., McGinn, T., Harwin, S. F., and Mont, M. A. (2016). Rehabilitative Guidelines after Total Knee Arthroplasty: A Review. Journal of Knee Surgery, 29(3):201-217.
Knieoperationen : Große regionale Unterschiede bei der Versorgung mit künstlichen Kniegelenken
  • A Raten
Raten, A. (2013). Knieoperationen : Große regionale Unterschiede bei der Versorgung mit künstlichen Kniegelenken.
Qualitative activity recognition of weight lifting exercises
  • E Velloso
  • A Bulling
  • H Gellersen
  • W Ugulino
  • H Fuks
Velloso, E., Bulling, A., Gellersen, H., Ugulino, W., and Fuks, H. (2013b). Qualitative activity recognition of weight lifting exercises. Proceeding AH '13 Proceedings of the 4th Augmented Human International Conference, pages 116-123.
A Kinect-based rehabilitation exercise monitoring and guidance system
  • W Zhao
  • H Feng
  • R Lun
  • D D Espy
  • M A Reinthal
Zhao, W., Feng, H., Lun, R., Espy, D. D., and Reinthal, M. A. (2014a). A Kinect-based rehabilitation exercise monitoring and guidance system. Proceedings of the IEEE International Conference on Software Engineering and Service Sciences, ICSESS, pages 762-765.