Chapter

Automatic Segmentation of Therapeutic Exercises Motion Data with a Predictive Event Approach

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

We propose a novel approach for detecting events in data sequences, based on a predictive method using Gaussian processes. We have applied this approach for detecting relevant events in the therapeutic exercise sequences, wherein obtained results in addition to a suitable classifier, can be used directly for gesture segmentation. During exercise performing, motion data in the sense of 3D position of characteristic skeleton joints for each frame are acquired using a RGBD camera . Trajectories of joints relevant for the upper-body therapeutic exercises of Parkinson’s patients are modelled as Gaussian processes. Our event detection procedure using an adaptive Gaussian process predictor has been shown to outperform a first derivative based approach.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Abstract. The Perceptive Workbench endeavors to create a spontaneous and unimpeded interface between the physical and virtual worlds. Its vision-based methods for interaction constitute an alternative to wired input devices and tethered tracking. Objects are recognized and tracked when placed on the display surface. By using multiple infrared light sources, the object's 3-D shape can be captured and inserted into the virtual interface. This ability permits spontaneity, since either preloaded objects or those objects selected at run-time by the user can become physical icons. Integrated into the same vision-based interface is the ability to identify 3-D hand position, pointing direction, and sweeping arm gestures. Such gestures can enhance selection, manipulation, and navigation tasks. The Perceptive Workbench has been used for a variety of applications, including augmented reality gaming and terrain navigation. This paper focuses on the techniques used in implementing the Perceptive Workbench and the system's performance.
Article
Full-text available
This paper addresses the problem of segmenting perception in physical robots into meaningful events along time. In structured environments this problem can be approached using domain-specific techniques, but in the general case, as when facing unknown environments, this becomes a non-trivial problem. We propose a dynamical systems approach to this problem, consisting of simultaneously learning a model of the robot’s interaction with the environment (robot and world seen as a single, coupled dynamical system), and deriving predictions about its short-term evolution. Event boundaries are detected once synchronization is lost, according to a simple statistical test. An experimental proof of concept of the proposed framework is presented, simulating a simple active perception task of a robot following a ball. The results reported here corroborate the approach, in the sense that the event boundaries are correctly detected. Keywordsevent segmentation–anticipative systems–active perception–cognitive robotics
Article
Full-text available
This paper presents a systematic analysis of twenty four performance measures used in the complete spectrum of Machine Learning classification tasks, i.e., binary, multi-class, multi-labelled, and hierarchical. For each classification task, the study relates a set of changes in a confusion matrix to specific characteristics of data. Then the analysis concentrates on the type of changes to a confusion matrix that do not change a measure, therefore, preserve a classifier’s evaluation (measure invariance). The result is the measure invariance taxonomy with respect to all relevant label distribution changes in a classification problem. This formal analysis is supported by examples of applications where invariance properties of measures lead to a more reliable evaluation of classifiers. Text classification supplements the discussion with several case studies.
Article
Full-text available
This paper makes the case for a spotting computation scheme which gives rise to a new classification methodology for processing real world data by surveying algorithms developed under the Real World Computing (RWC) program and related work in Japan. A spotting function has the segmentation-free characteristic which ignores gracefully most real world input data which do not belong to a task domain. Some members of the family of spotting methods have been developed under the RWC program. This paper shows how some spotting methods rise to the challenge of the case made for them. The common computational structure amongst spotting methods suggests an architecture for spotting computation.
Article
Full-text available
Hand and face gestures are modeled using an appearance-based approach in which patterns are represented as a vector of similarity scores to a set of view models defined in space and time. These view models are learned from examples using unsupervised clustering techniques. A supervised teaming paradigm is then used to interpolate view scores into a task-dependent coordinate system appropriate for recognition and control tasks. We apply this analysis to the problem of context-specific gesture interpolation and recognition, and demonstrate real-time systems which perform these tasks
Article
Full-text available
Abstract The GPML toolbox provides a wide range of functionality for Gaussian process (GP) inference and prediction. GPs are specified by mean and covariance functions; we offer a broad library of simple mean and covariance functions and mechanisms to compose more complex ones. Several likelihood functions are supported including Gaussian and heavy-tailed for regression as well as others suitable for classification. Finally, a range of inference methods is provided, including exact inference, Expectation Propagation, Laplace‘s method and variational inference dealing with non-Gaussian likelihoods and FITC for dealing with large regression tasks. The package has a modular design, enabling simple addition of new functionality.
Article
Full-text available
This paper describes the identification of nonlinear dynamic systems with a Gaussian process (GP) prior model. This model is an example of the use of a probabilistic non-parametric modelling approach. GPs are flexible models capable of modelling complex nonlinear systems. Also, an attractive feature of this model is that the variance associated with the model response is readily obtained, and it can be used to highlight areas of the input space where prediction quality is poor, owing to the lack of data or complexity (high variance). We illustrate the GP modelling technique on a simulated example of a nonlinear system.
Article
Full-text available
A method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Finally, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction
Article
Full-text available
We present two real-time hidden Markov model-based systems for recognizing sentence-level continuous American sign language (ASL) using a single camera to track the user's unadorned hands. The first system observes the user from a desk mounted camera and achieves 92 percent word accuracy. The second system mounts the camera in a cap worn by the user and achieves 98 percent accuracy (97 percent with an unrestricted grammar). Both experiments use a 40-word lexicon
Article
Full-text available
In this paper a new and general stochastic approach to find and identify dynamic gestures in continuous video streams is presented. Hidden Markov Models (HMMs) are used to solve this combined problem of temporal segmentation and classification in an integral way. Basically, an improved normalized Viterbi algorithm allows to continuously observe the output scores of the HMMs at every time step. Characteristic peaks in the output scores of the respective models indicate the presence of gestures. Our experiments in the domain of hand gesture spotting provided excellent recognition results and very low temporal detection delays. 1. Introduction Human gestures appearing in a natural environment are movements rather than static postures. Consequently, a vision-based gesture recognition requires the classification of image sequences. Some previous works already demonstrated that a HMM-based recognition can cope with the variability and complexity of gestures or other human movements (e. g. ...
Conference Paper
We give a basic introduction to Gaussian Process regression models. We focus on understanding the role of the stochastic process and how it is used to define a distribution over functions. We present the simple equations for incorporating training data and examine how to learn the hyperparameters using the marginal likelihood. We explain the practical advantages of Gaussian Process and end with conclusions and a look at the current trends in GP work.
Article
In this paper, we introduce a hand gesture recognition system to recognize continuous gesture before stationary background. The system consists of four modules: a real time hand tracking and extraction, feature extraction, hidden Markov model (HMM) training, and gesture recognition. First, we apply a real-time hand tracking and extraction algorithm to trace the moving hand and extract the hand region, then we use the Fourier descriptor (FD) to characterize spatial features and the motion analysis to characterize the temporal features. We combine the spatial and temporal features of the input image sequence as our feature vector. After having extracted the feature vectors, we apply HMMs to recognize the input gesture. The gesture to be recognized is separately scored against different HMMs. The model with the highest score indicates the corresponding gesture. In the experiments, we have tested our system to recognize 20 different gestures, and the recognizing rate is above 90%.
Conference Paper
Vision-based interfaces pose a tempting alternative to physical interfaces. Intuitive and multi-purpose, these interfaces could allow people to interact with computer naturally and effortlessly. The existing various vision-based interfaces are hard to apply in reality since it has many environmental constraints. In this paper, we introduce a vision-based game interface which is robust in varying environments. This interface consists of three main modules: body-parts localization, pose classification and gesture recognition. Firstly, body-part localization module determines the locations of body parts such as face and hands automatically. For this, we extract body parts using SCI-color model, human physical character and heuristic information. Subsequently, pose classification module classifies the positions of detected body parts in a frame into a pose according to Euclidean distance between the input positions and predefined poses. Finally, gesture recognition module extracts a sequence of poses corresponding to the gestures from the successive frames, and translates that sequence into the game commands using a HMM. To assess the effectiveness of the proposed interface, it has been tested with a popular computer game, Quake II, and the results confirm that the vision-based interface facilitates more natural and friendly communication while controlling the game.
Conference Paper
Gesture spotting is the challenging task oflocating the start and end frames ofthe video stream that correspond to a gesture ofinter- est, while at the same time rejecting non-gesture motion patterns. This paper proposes a new gesture spotting and recognition algorithm that is based on the continuous dynamic programming (CDP) algorithm, and runs in real-time. To make gesture spotting efficient a pruning method is proposed that allows the system to evaluate a relatively small num- ber ofhypotheses compared to CDP. Pruning is implemented by a set ofmodel-dependent classifiers, that are learned f rom training examples. To make gesture spotting more accurate a subgesture reasoning process is proposed that models the fact that some gesture models can falsely match parts ofother longer gestures. In our experiments, the proposed method with pruning and subgesture modeling is an order ofmagnitude faster and 18% more accurate compared to the original CDP algorithm.
Article
The CAVE is a new virtual reality interface. In its abstract design, it consists of a room whose walls, ceiling and floor surround a viewer with projected images. Its design overcomes many of the problems encountered by other virtual reality systems and can be constructed from currently available technology. Suspension of disbelief and viewer-centered perspective, are often used to describe such systems. The main topics are head-mounted display (HMD), binocular omni-oriented monitor (BOOM), and audio-visual experience automatic virtual environment (CAVE). The article concludes that the CAVE is a nonintrusive easy-to-learn high-resolution virtual reality interface. It is superior to other virtual reality paradigms across many issues, particularly in field-of-view, visual acuity and lack of intrusion. Moreover, it is open to limited use for collaborative visualization.
Article
The synchronization of dynamical systems is a method that allows two systems to have identical state trajectories, appart from an error converging to zero. This method consists in an appropriate unidirectional coupling from one system (drive) to the other (response). This requires that the response system shares the same dynamical model with the drive. For the cases where the drive is unknown, Chen proposed in 2002 a method to adapt the response system such that synchronization is achieved, provided that (1) the response dynamical model is linear with a vector of parameters, and (2) there is a parameter vector that makes both system dynamics identical. However, this method has two limitations: first, it does not scale well for complex parametric models (e.g., if the number of parameters is greater than the state dimension), and second, the model parameters are not guaranteed to converge, namely as the synchronization error approaches zero. This paper presents an adaptation law addressing these two limitations. Stability and convergence proofs, using Lyapunov's second method, support the proposed adaptation law. Finally, numerical simulations illustrate the advantages of the proposed method, namely showing cases where the Chen's method fail, while the proposed one does not.
Article
We present a clinimetric assessment of the Movement Disorder Society (MDS)-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS). The MDS-UDPRS Task Force revised and expanded the UPDRS using recommendations from a published critique. The MDS-UPDRS has four parts, namely, I: Non-motor Experiences of Daily Living; II: Motor Experiences of Daily Living; III: Motor Examination; IV: Motor Complications. Twenty questions are completed by the patient/caregiver. Item-specific instructions and an appendix of complementary additional scales are provided. Movement disorder specialists and study coordinators administered the UPDRS (55 items) and MDS-UPDRS (65 items) to 877 English speaking (78% non-Latino Caucasian) patients with Parkinson's disease from 39 sites. We compared the two scales using correlative techniques and factor analysis. The MDS-UPDRS showed high internal consistency (Cronbach's alpha = 0.79-0.93 across parts) and correlated with the original UPDRS (rho = 0.96). MDS-UPDRS across-part correlations ranged from 0.22 to 0.66. Reliable factor structures for each part were obtained (comparative fit index > 0.90 for each part), which support the use of sum scores for each part in preference to a total score of all parts. The combined clinimetric results of this study support the validity of the MDS-UPDRS for rating PD.
Conference Paper
An intelligent robot requires natural interaction with humans. Visual interpretation of gestures can be useful in accomplishing natural human-robot interaction (HRl). Previous HRI researches were focused on issues such as hand gesture, sign language, and command gesture recognition. However, automatic recognition of whole body gestures is required in order to operate HRI naturally. This can be a challenging problem because describing and modeling meaningful gesture patterns from whole body gestures are complex tasks. This paper presents a new method for spotting and recognizing whole body key gestures at the same time on a mobile robot. Our method is simultaneously used with other HRI approaches such as speech recognition, face recognition, and so forth. In this regard, both of execution speed and recognition performance should be considered. For efficient and natural operation, we used several approaches at each step of gesture recognition; learning and extraction of articulated joint information, representing gesture as a sequence of clusters, spotting and recognizing a gesture with HMM. In addition, we constructed a large gesture database, with which we verified our method. As a result, our method is successfully included and operated in a mobile robot.
Conference Paper
Complex human motion (e.g. dance) sequences are typically analyzed by segmenting them into shorter motion sequences, called gestures. However, this segmentation process is subjective, and varies considerably from one choreographer to another. Dance sequences also exhibit a large vocabulary of gestures. In this paper, we propose an algorithm called hierarchical activity segmentation. This algorithm employs a dynamic hierarchical layered structure to represent human anatomy, and uses low-level motion parameters to characterize motion in the various layers of this hierarchy, which correspond to different segments of the human body. This characterization is used with a naive Bayesian classifier to derive choreographer profiles from empirical data that are used to predict how particular choreographers segment gestures in other motion sequences. When the predictions were tested with a library of 45 3D motion capture sequences (with 185 distinct gestures) created by 5 different choreographers, they were found to be 93.3% accurate.
A design framework for 3D spatial gesture interfaces
  • D Y Kwon
D. Y. Kwon (2008), A Design Framework for 3D Spatial Gesture Interfaces, PhD, ETH, Switzerland.