ArticlePDF Available

Abstract and Figures

The new generation of videogame interfaces such as Microsoft's Kinect opens the possibility of implementing exercise programs for physical training, and of evaluating and reducing the risks of elderly people falling. However, applications such as these might require measurements of joint kinematics that are more robust and accurate than the standard output given by the available middleware. This paper presents a method based on particle filters for calculating joint angles from the positions of the anatomical points detected by PrimeSense's NITE software. The application of this method to the measurement of lower limb kinematics reduced the error by one order of magnitude, to less than 10º, except for hip axial rotation, and it was advantageous over inverse kinematic analysis, in ensuring a robust and smooth solution without singularities, when the limbs are out-stretched and anatomical landmarks are aligned.
No caption available
… 
Content may be subject to copyright.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Note. This article will be published in a forthcoming issue of
the Journal of Applied Biomechanics. The article appears here
in its accepted, peer-reviewed form, as it was provided by the
submitting author. It has not been copyedited, proofread, or
formatted by the publisher.
Section: Technical Note
Article Title: Correction of Joint Angles From Kinect for Balance Exercising and
Assessment
Authors: Helios De Rosario,
1,2
Juan Manuel Belda-Lois,
1,2
Francisco Fos,
1
Enrique Medina,
1
Rakel Poveda-Puente,
1
Michael Kroll
3
Affiliations:
1
Instituto de Biomecánica de Valencia, Valencia, Spain.
2
CIBER de
Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain.
3
Deutsche
Sporthochschule Köln, Cologne, Germany.
Journal: Journal of Applied Biomechanics
Acceptance Date: June 21, 2013
©2013 Human Kinetics, Inc.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Correction of joint angles from Kinect for balance exercising and
assessment
Helios De Rosario,
1,2
Juan Manuel Belda-Lois,
1,2
Francisco Fos,
1
Enrique Medina,
1
Rakel
Poveda-Puente,
1
Michael Kroll
3
1
Instituto de Biomecánica de Valencia, Valencia, Spain
2
CIBER de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Spain
3
Deutsche Sporthochschule Köln, Cologne, Germany
Funding: This work has been undertaken within the framework of the iStoppFalls
project, which has received funding from the European Community (grant agreement FP7-
ICT-2011-7-287361) and the Australian Government.
Conflict of interest disclosure: No conflict of interest exists for any of the authors of
this paper.
Correspondence address: Helios De Rosario. Instituto de Biomecánica de Valencia,
Universitat Politècnica de Valencia, Edificio 9C, Camino de Vera s/n, E-46022, Valencia,
Spain. Tel: +34 963879160. Fax: +34 963879169. E-mail: helios.derosario@ibv.upv.es
Word count: 1,996 words from the introduction through the discussion
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Abstract
The new generation of videogame interfaces such as Microsoft’s Kinect opens the
possibility of implementing exercise programs for physical training, and of evaluating and
reducing the risks of elderly people falling. However, applications such as these might
require measurements of joint kinematics that are more robust and accurate than the standard
output given by the available middleware. This paper presents a method based on particle
filters for calculating joint angles from the positions of the anatomical points detected by
PrimeSense’s NITE software. The application of this method to the measurement of lower
limb kinematics reduced the error by one order of magnitude, to less than 10º, except for hip
axial rotation, and it was advantageous over inverse kinematic analysis, in ensuring a robust
and smooth solution without singularities, when the limbs are out-stretched and anatomical
landmarks are aligned.
Keywords: particle filter, human movement analysis, video games
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Introduction
The benefit of ―exergames‖ in stimulating sensorimotor learning processes
1
has
encouraged the creation of therapy programs based on new-generation videogames, using
motion capture devices that reduce the distance between biomechanical research, clinical
practice, and home-based interventions.
Besides Nintendo’s Wii,
24
Microsoft’s Kinect is a popular videogame platform used
for balance-recovery programs.
57
In addition to motivating and guiding the user, Kinect can
also be used to monitor the exercises for assessment and feedback purposes. A successful
application, however, requires a trade-off between the amount and quality of motion data, and
the accuracy that the sensors can provide. Many studies have used the OpenNI interface to
the Kinect sensor, and PrimeSense’s NITE toolbox.
8
This solution yields position errors of 1-
to-10 cm,
9,10
which can be acceptable for the proposed applications. Microsoft’s Software
Development Kit, which uses ―randomized decision forests,
11
has similar margins of
error.
1214
There are few studies about the errors of joint rotations, which are also given by
modern versions of those interfaces. The only published study that we have found reported
mean errors of less than 10°;
15
however, NITE’s documentation warns about the important
noise of joint orientations, and their indetermination when limb segments are aligned.
8
This paper investigates an alternative resolution of the inverse kinematic problem for
the ―iStoppFalls‖ balance-training and assessment program,
16
using particle filters (PF). This
technique is normally used to analyze complex images, formed by point clouds and volumes.
The present study will explore its application to the analysis of higher-level, simpler
positional data provided by NITE, and test whether it can improve the angle estimations
given by that middleware.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Materials and methods
Skeleton and PF model
Since the program of exercises focused on the motion of limbs, the model took the
trunk as the root segment, from which the arms and legs stemmed as two-segment linkages
(Figure 1, left). Three rotational degrees of freedom were assigned to trunk, shoulders and
hips, and one to elbows and knees.
Trunk displacement was irrelevant for the purposes of the study. Thus, the 3D
positions of the remaining 12 joints were represented by their distance to the trunk joint, with
their 36 coordinates gathered in vector y
t
for each instant t. Human joint rotations are usually
represented as Euler sequences,
17,18
but for computational purposes, we used vector θ
t
, which
contained the 19 coordinates of the joint relative attitude vectors.
19
Given the body segments
length, the relation between ―error-free‖ measurements of y
t
and θ
t
was determined by the
direct kinematic model:
),(
tt
θy K
(1)
A PF was used to calculate θ
t
from y
t
. Both vectors were considered random
variables, whose dynamic behavior was modeled by a Markovian Stochastic Process:
,|~
1tt
f θθθ
(2)
.|~
tt
g θyy
(3)
The values of θ
t
were obtained recursively as the average of 512 particles
512
1i
t
i
θ
,
which were propagated by the ―sample-resample‖ technique, as follows. An initial population
of particles with fixed values
0
θ
i
was first defined. In each later instant t, the a priori
distribution of θ
t
was estimated by sampling the values
t
i
θ
~
out of (2), given
1t
i
θ
. Then,
the a posteriori distribution was estimated by assigning weights to
t
i
θ
~
after (3) and the
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
observed value of y
t
. Then
t
i
θ
were resampled with replacement from
t
i
θ
~
, according to
those weights. This algorithm ensures that particles remain in regions of high probability,
avoids accumulation of errors over time, and gives stable results.
20
Model training
For the sake of simplicity and computational efficiency, the functions defined in (2)
and (3) were based on multivariate normal distributions. Their means were defined on
theoretical grounds, such that
1
|
t
f θθ
described a random walk, and
was centered
around the value of the direct kinematic model:
,|
11
tt
fE θθθ
(4)
).(|
tt
gE θθy K
(5)
Limb lengths, required by
)(θK
, were obtained by an initial calibration, through
measuring distances between joints in the first static instants.
1
|
t
f θθ
was trimmed to
retain the possible values of θ
t
within the ranges of motion of healthy adults.
21,22
The covariance matrices of both distributions (Σ
θ
, Σ
y
) were obtained experimentally
with a volunteer. The work was approved by the ethical committee of the Universitat
Politècnica de València, and the subject gave informed consent to participate.
Two 10-second static measurements were taken with Kinect, in order to estimate the
covariance of
. Medium-to-high correlations were observed between the variations
of some coordinates of y
t
; therefore, a full matrix was defined for Σ
y
.
Then, cyclic rotations around the different axes of each joint, starting in the ―T-
posture,‖ were recorded separately in 10-second measurements, using the Kinescan-IBV
photogrammetry system with 10 high-resolution cameras, and 42 markers that facilitated the
calculation of joint positions from a well-defined kinematic model (Figure 1, right).
23
Thus,
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
the size of the difference θ
t
- θ
t-1
was considered to model the variability of
1
|
t
f θθ
. Σ
θ
was assumed diagonal.
The stochastic behavior of the model was assumed symmetric. This property was
forced by averaging the variances of the left and right joints for both covariance matrices.
Comparison of joint rotations
The joint rotation errors of the NITE and PF algorithms were compared for the main
movements that must be done by the user during the ―iStoppFalls‖ exercises: ―back knee‖
(alternate knee flexion while standing), ―sit+front knee‖ (sit and alternate leg elevation), ―side
hip‖ (alternate lateral leg elevation while standing), and ―step‖, based on the Otago
strengthening program,
24
plus ―near tandem‖ (keep standing with one foot in front of the
other) from the ―Quick-Screen‖ fall risk assessment.
25
These exercises were done by the volunteer and recorded simultaneously by Kinect
and Kinescan-IBV. In each exercise, the subject started in the ―T-posture,to calibrate limb
lengths and set the initial value of θ
0
. In all cases, the exercise was done with the right-hand
side of the body, in order to evaluate whether the results varied depending on the level of
motion.
Overall, there were three sets of results: (1) the ―gold standard‖ of Kinescan-IBV, (2)
the values obtained by the NITE algorithm from Kinect data, and (3) the results of the PF.
The output of NITE, given as rotation matrices, was transformed into attitude vectors for
comparison. As data analysis revealed frequent axial ―flips‖ of the limbs, as if an axial
rotation of 180° along their long axes had been added, an opposite rotation was introduced at
such discontinuities, in order to avoid error inflation.
The errors of the two Kinect-based results were modeled as motion artifacts added to
the ―correct‖ movement represented by the gold standard.
26
This resulted in the ―error attitude
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
vectors‖
j
t
A
ε
, with different values for each joint j and for each algorithm A. The analysis
was focused on hips and knees, the joints of interest for the target application.
These errors were represented as Euler angles for the standard sequence of lower
limbs (flexion-abduction-axial rotation). For a more concise comparison of the total amount
of error in each joint, the difference between the modules of
j
t
A
ε
was also calculated at each
instant:
.
j
t
PFj
t
NITEj
t
εε
(8)
These variables were a measure of the ―improvement‖ provided by the PF. To verify
that such an improvement was significant, a Wilcoxon signed-rank test was applied to the
distributions of
j
t
ε
. Their values were also calculated with a four-times smaller or larger set
of particles (128 or 2048), in order to evaluate the potential impact of changing the ratio
between the variability of the model and the number of particles. All the calculations were
done with GNU Octave.
27
Results
Standard deviations of
1
|
t
f θθ
and
were between 1° and 6°, and between
6 and 40 mm, respectively, with the remarkable exception of the error in the position of the
hand, which was over 150 mm in the camera plane (Table 1).
The absolute values of the Euler angle errors had a great dispersion. Their
distributions were mainly concentrated between 1° and 10°, but NITE errors were one order
of magnitude greater (tens of degrees) for hip abduction and for knee flexion of both sides
(Figure 2). When all the measures were taken together, the NITE average error for right hip
flexion was also greater than the corresponding PF error, although both distributions had the
same order of magnitude (Table 2). On the other hand, the PF did not improve the error of
NITE in hip axial rotation, and in fact, it was substantially increased for the left-hand side.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Considering the set of all measurements, the improvement provided by the PF to hips
and knees (
H
t
ε
and
K
t
ε
, respectively) was clearly positive for the right-hand side, but small
for the left-hand side (Table 2). Wilcoxon’s test showed that the average of those variables
was significantly positive (W>10, P<.001) in all cases, except for the left hip, where the (also
positive) difference was not significant (W=1.35, P=.176).
When the individual exercises were considered, PF errors were often similar to NITE
errors, although there are relatively more cases when NITE errors were greater and fewer
cases when PF errors were greater. For side hip and tandem, the PF performed generally
better, especially for the right-hand side. On the other hand, left hip axial rotation presented
greater PF errors in all exercises. The values of
H
t
ε
and
K
t
ε
were small in most cases, but
again, tended to be positive. Their interquartile ranges were fully positive in 8 cases, and only
slightly negative in 1 out of 20 cases (left
K
t
ε
for the side hip exercise).
The average improvement in hip rotations (but not of knee rotations) increased with
the number of particles (Figure 3). However, even the gain in hips (around 6° from 128 to
2048 particles) was small compared with the wide dispersion of errors that had been
observed.
Discussion
This study presents an alternative to the methods used by Kinect middleware to
calculate joint rotations. NITE errors often exceeded 10° (even after correcting accidental
―flips‖) except for hip flexion, unlike in previous studies.
15
A possible explanation for this
discrepancy is that the gold standard used by other authors was obtained by an inverse
kinematic analysis of a simple skeleton model, formed by one-dimensional lines linking
joints. That model has the same problems of indeterminations and singularities reported for
NITE algorithms, and could mask NITE errors. The gold standard of the present study was
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
obtained from measurements with a greater number of markers and therefore, could be
considered more valid.
The error of the PF was typically below 10°, except for hip axial rotation. This was
comparable with NITE errors and one order of magnitude lower for hip abduction/adduction
and knee flexion. The rotations calculated by the PF were more reliable for joints in motion.
PF and related techniques are often employed for motion tracking, using data from
inertial sensors or more complex optical data from marker-less motion capture. This study
demonstrated that they can also be used with simpler, high-level data such as the position of a
small number of anatomical landmarks. In addition to reducing errors, PF have the advantage
of giving smooth angle trajectories, coherent with skeletons of fixed anthropometry. To
obtain those properties with inverse kinematics, the analysis should be complicated with
nonlinear optimization techniques.
28
A limitation of this study is that it was conducted with just one subject in a laboratory.
The intended application of this technology involves measuring many people in various home
environments, such that the dispersions of
1
|
t
f θθ
and
may be larger than the
values observed in the experiment. However, the PF might still converge if there are a
sufficient number of particles.
29
The results show that hip errors are more sensitive to the
number of particles, especially if the degree of motion is small. The required computational
resources grow proportionally with the number of particles, but other mathematical
approaches could be attempted to counteract that problem, such as Unscented Kalman
Filters.
30
Better performance could also be obtained by using a state space based on positions,
instead of joint angles, whose effect on the observed posture is accumulated across the
kinematic chains.
31
This study was limited to the analysis of lower limb angles, as they were the ones of
interest for the balance-assessment and training exercises considered in the present research
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
framework. For less specific applications, the methodology could be easily applied to the
analysis of upper limbs.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
References
1. Di Tore S, D’Elia F, Aiello P, Carlomagno N, Sibilio M. Didactics, movement and
technology: new frontiers of the human-machine interaction. J Hum Sport Exerc -
University of Alicante. 2012;7(1Proc):S178S184.
2. Pigford T, Andrews AW. Feasibility and Benefit of Using the Nintendo Wii Fit for
Balance Rehabilitation in an Elderly Patient Experiencing Recurrent Falls. J Student
Phys Ther Res. 2010;2(1):1220.
3. Young W, Ferguson S, Brault S, Craig C. Assessing and training standing balance in older
adults: A novel approach using the ―Nintendo Wii‖ Balance Board. Gait Posture.
2011;33(2):303305.
4. Agmon M, Perry CK, Phelan E, Demiris G, Nguyen HQ. A pilot study of Wii Fit
exergames to improve balance in older adults. J Geriatr Phys Ther. 2011;34(4):161
167.
5. Garcia JA, Felix Navarro K, Schoene D, Smith ST, Pisan Y. Exergames for the elderly:
towards an embedded Kinect-based clinical test of falls risk. Stud Health Technol
Inform. 2012;178:5157.
6. Lloréns R, Alcañiz M, Colomer C, Navarro MD. Balance recovery through virtual stepping
exercises using Kinect skeleton tracking: a follow-up study with chronic stroke
patients. Stud Health Technol Inform. 2012;181:108112.
7. Smith ST, Schoene D. The use of exercise-based videogames for training and rehabilitation
of physical function in older adults: current practice and guidelines for future
research. Aging Health. 2012;8(3):243252.
8. PrimeSense Inc. Prime Sensor
TM
NITE 1.3 Algorithms notes. 2010.
http://pr.cs.cornell.edu/humanactivities/data/NITE.pdf. Accessed January 25, 2013.
9. Chang C-Y, Lange B, Zhang M, et al. Towards pervasive physical rehabilitation using
Microsoft Kinect. In: 2012 6th International Conference on Pervasive Computing
Technologies for Healthcare (PervasiveHealth); 2012:159162.
10. Schonauer C, Pintaric T, Kaufmann H, Jansen - Kosterink S, Vollenbroek-Hutten M.
Chronic pain rehabilitation with a serious game using multimodal input. In: 2011
International Conference on Virtual Rehabilitation (ICVR); 2011:18.
11. Shotton J, Fitzgibbon A, Cook M, et al. Real-time human pose recognition in parts from
single depth images. In: 2011 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR); 2011:12971304.
12. Clark RA, Pua Y-H, Fortin K, et al. Validity of the Microsoft Kinect for assessment of
postural control. Gait Posture. 2012;36(3):372377.
13. Obdrzalek S, Kurillo G, Ofli F, et al. Accuracy and robustness of Kinect pose estimation
in the context of coaching of elderly population. In: 2012 Annual International
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Conference of the IEEE Engineering in Medicine and Biology Society (EMBC);
2012:11881193.
14. Samejima I, Maki K, Kagami S, Kouchi M, Mizoguchi H. A body dimensions estimation
method of subject from a few measurement items using KINECT. In: 2012 IEEE
International Conference on Systems, Man, and Cybernetics (SMC); 2012:33843389.
15. Fernández-Baena A, Susin A, Lligadas X. Biomechanical Validation of Upper-Body and
Lower-Body Joint Movements of Kinect Motion Capture Data for Rehabilitation
Treatments. In: 2012 4th International Conference on Intelligent Networking and
Collaborative Systems (INCoS); 2012:656661.
16. Wieching R, Kaartinen N, De Rosario H, et al. A New Approach for Personalised falls
risk prediction and Prevention: Tailored Exercises. Unobtrusive Sensing & Advanced
Reasoning. J Aging Phys Act. 2012;20(S):S120S124.
17. Wu G, Siegler S, Allard P, et al. ISB recommendation on definitions of joint coordinate
system of various joints for the reporting of human joint motionpart I: ankle, hip,
and spine. J Biomech. 2002;35(4):543548.
18. Wu G, Van der Helm FCT, (DirkJan) Veeger HEJ, et al. ISB recommendation on
definitions of joint coordinate systems of various joints for the reporting of human
joint motionPart II: shoulder, elbow, wrist and hand. J Biomech. 2005;38(5):981
992.
19. Woltring HJ. 3-D attitude representation of human joints: A standardization proposal. J
Biomech. 1994;27(12):13991414.
20. Cappe O, Godsill SJ, Moulines E. An Overview of Existing Methods and Recent
Advances in Sequential Monte Carlo. Proc IEEE. 2007;95(5):899924.
21. Anon. Range of Motion Evaluation Chart. Washington State Department of Social &
Health Services; 2003. http://www.dshs.wa.gov/pdf/ms/forms/13_585a.pdf. Accessed
January 25, 2013.
22. Soucie JM, Wang C, Forsyth A, et al. Range of motion measurements: reference values
and a database for comparison studies. Haemophilia. 2011;17(3):500507.
23. Page A, De Rosario H, Mata V, Atienza C. Experimental Analysis of Rigid Body Motion.
A Vector Method to Determine Finite and Infinitesimal Displacements From Point
Coordinates. J Mech Des. 2009;131(3):031005.
24. Robertson MC, Campbell AJ. Otago Exercise Programme to Prevent Falls in Older
Adults. Wellington: Accident Compensation Corporation; 2003.
25. Tiedemann A, Lord SR, Sherrington C. The Development and Validation of a Brief
Performance-Based Fall Risk Assessment Tool for Use in Primary Care. J Gerontol A
Biol Sci Med Sci. 2010;65A(8):896903.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
26. De Rosario H, Page A, Besa A, Mata V, Conejero E. Kinematic description of soft tissue
artifacts: quantifying rigid versus deformation components and their relation with
bone motion. Med Biol Eng Comput. 2012;50(11):11731181.
27. Eaton JW, Bateman D, Hauberg S. GNU Octave Manual Version 3. Godalming, UK:
Network Theory Limited; 2008.
28. Schwarz LA, Mkhitaryan A, Mateus D, Navab N. Human skeleton tracking from depth
data using geodesic distances and optical flow. Image Vis Comput. 2012;30(3):217
226.
29. 1. Crisan D, Doucet A. A survey of convergence results on particle filtering methods for
practitioners. IEEE Trans Signal Process. 2002;50(3):736746.
doi:10.1109/78.984773.
30. Larsen ABL, Hauberg S, Pedersen KS. Unscented Kalman Filtering for Articulated
Human Tracking. In: Heyden A, Kahl F, eds. Image Analysis. Lecture Notes in
Computer Science. Springer Berlin Heidelberg; 2011:228237.
31. Hauberg S, Sommer S, Pedersen KS. Natural metrics and least-committed priors for
articulated tracking. Image Vis Comput. 2012;30(67):453461.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Figure 1 Left: skeleton model in the reference posture (null rotation of all joints). Right: set
of markers used for motion capture with marker sets of body segments highlighted.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Figure 2 NITE vs. particle filter (PF) errors in Euler angles of hips and knees for all the
exercises. Absolute values in degrees (axes in logarithmic scales). Points in the right bottom
sectors represent instants where NITE error was higher than PF error, and vice versa.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Figure 3 Mean and standard error of the difference between particle filter and NITE joint
orientation errors (
j
t
ε
) for 128, 512, and 2048 particles. Values in degrees.
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Table 1. Standard deviations (SD) used to model the distributions of the particle filter.
SD (f(θ|θ
t
)) (º)
SD (g(y|θ
t
)) (mm)
x
y
z
x
y
z
Trunk
1.60
2.44
1.16
Shoulder
2.88
5.64
2.8
9.1
6.7
12.7
Elbow
2.44
29.8
25.2
20.3
Hand
151.5
181.7
14.5
Hip
2.76
3.28
3.36
6.0
6.5
10.1
Knee
1.31
8.8
16.8
35.1
Foot
19.8
25.2
38.1
Correction of Joint Angles From Kinect for Balance Exercising and Assessment‖ by De Rosario H et al.
Journal of Applied Biomechanics
© 2013 Human Kinetics, Inc.
Table 2. Summary of joint rotation errors associated with the NITE and particle filter
algorithms. Mean values are given for each joint angle and the difference of attitude errors (
H
t
ε
,
K
t
ε
), together with the interquartile ranges (25th and 75th percentiles), given the
strong non-normality of the distributions. All values in degrees.
Hip flex.
Hip abd.
Hip rot.
Knee flex.
NITE
PF
NITE
PF
NITE
PF
H
t
ε
NITE
PF
K
t
ε
Right-hand side
Back knee
mean
7.7
5.0
1.6
1.7
6.3
12.3
-2.9
10.4
10.1
0.3
p25
4.6
2.8
0.4
0.6
3.0
9.7
-8.6
6.3
1.7
-9.5
p75
10.7
6.8
2.4
2.7
9.3
14.7
2.6
13.3
16.7
10.6
Sit + front knee
mean
11.1
8.1
8.3
3.5
8.9
19.9
-4.2
8.5
8.9
-0.4
p25
3.1
3.0
6.4
2.1
2.7
18.1
-10.3
5.5
4.3
-5.6
p75
17.6
12.1
9.5
5.0
11.9
22.7
1.6
11.9
13.2
4.0
Side hip
mean
46.1
6.8
36.9
2.2
28.2
14.7
56.5
37.3
2.8
34.5
p25
13.2
5.2
12.7
0.7
21.6
7.0
20.6
3.7
1.4
1.1
p75
55.8
8.2
65.9
2.9
32.0
20.0
110.5
30.5
4.0
26.6
Step
mean
7.8
6.4
3.8
3.5
6.9
12.1
-1.7
8.0
5.3
2.8
p25
2.9
4.7
1.1
2.1
2.8
8.0
-4.7
5.7
3.4
0.2
p75
11.7
8.2
4.7
4.6
10.9
16.1
3.0
10.4
7.0
5.3
Tandem
mean
9.2
4.2
12.6
4.3
12.5
5.9
14.4
31.8
4.0
27.9
p25
2.9
2.6
12.1
3.6
4.6
1.8
6.0
29.0
1.4
23.3
p75
10.6
5.6
13.9
5.2
13.3
10.6
18.9
37.0
4.9
34.6
All measures
mean
16.6
6.2
13.0
3.2
12.8
12.7
13.1
19.3
5.6
13.7
p25
3.7
3.8
2.1
1.4
4.0
6.8
-3.9
5.9
2.1
-0.1
p75
14.4
8.1
13.5
4.6
18.0
17.8
18.3
17.7
7.2
21.8
Left-hand side
Back knee
mean
2.8
6.0
20.9
4.1
5.2
14.9
4.2
6.9
3.8
3.1
p25
1.1
4.8
20.0
2.4
2.3
5.3
-1.8
3.4
1.5
-1.1
p75
4.3
7.2
21.5
5.2
8.1
22.6
11.4
9.5
5.5
6.6
Sit + front knee
mean
13.5
6.7
7.3
3.4
5.3
6.6
6.4
5.8
6.8
-1.0
p25
4.9
1.5
5.2
1.9
2.6
4.5
0.4
2.9
4.4
-4.0
p75
17.8
10.0
9.4
4.6
6.3
8.6
9.3
6.4
8.2
1.9
Side hip
mean
3.6
3.5
20.5
2.8
5.8
24.3
-1.9
3.2
3.1
0.1
p25
0.5
1.8
21.1
1.5
3.9
11.3
-12.8
1.2
1.0
-2.5
p75
4.2
4.9
26.7
4.1
7.4
38.9
9.0
5.1
5.1
3.0
Step
mean
7.4
5.4
4.5
5.0
5.2
14.1
-6.4
7.7
6.5
1.2
p25
6.2
2.8
3.0
3.1
2.2
6.3
-9.5
7.0
5.1
-1.5
p75
9.1
6.2
5.9
6.8
7.6
15.8
1.0
10.0
8.0
3.6
Tandem
mean
5.9
7.0
19.8
3.2
11.7
17.9
5.0
22.5
12.1
10.4
p25
4.3
3.9
15.7
0.8
5.8
10.0
-1.0
12.8
4.1
6.8
p75
6.5
9.3
24.7
6.1
10.1
18.5
11.3
31.1
19.8
12.6
All measures
mean
6.9
5.6
13.0
3.8
6.7
15.8
0.1
9.4
6.7
2.7
p25
3.1
2.8
4.9
1.8
3.0
6.6
-6.4
3.2
3.3
-2.0
p75
8.9
7.3
23.2
5.3
8.5
19.5
8.4
10.8
8.1
5.7
Note: flex.: flexion-extension error; abd.: abduction-adduction error; rot.: axial rotation error.
... The challenge of living with dementia and managing everyday life is of varying intensity according to the symptoms at different stages. This can have a more pronounced effect upon both professional caregivers and relatives providing care than other kinds of care for chronically ill patients [14]. The relatives of people with dementia often have to cope with considerable physical, psychological, emotional, and financial burdens [11,12,64]. ...
... This phenomenon has been characterized using the notion of "hidden patients". It is also sometimes referred to, in advanced cases, as "caregiver overload" [13,14]. In our own work we extend upon this literature by examining how the deployed system was able to ameliorate certain aspects of caregiver overload. ...
... A male and female 3-D model and an underlying biomechanical model were provided for the Kinect-based motion detection. Recognition of the movements detected by the Kinect is reflected in their visual representation, providing comparable and measurable movement definitions and movement sequences [14]. ...
Conference Paper
This paper presents the outcomes of an exploratory field study that examined the social impact of an ICT-based suite of exergames for people with dementia and their caregivers. Qualitative data was collected over a period of 8 months, during which time we studied the daily life of 14 people with dementia and their informal and professional caregivers. We focus on the experiential aspects of the system and examine its social impact when integrated into the daily routines of both people with dementia themselves and their professional and family caregivers. Our findings indicate that relatives were able to regain leisure time, whilst people with dementia were able to recapture certain aspects of their social and daily activities that might otherwise have been lost to them. Results suggest that the system enhanced social-interaction, invigorated relationships, and improved the empowerment of people with dementia and their caregivers to face daily challenges.
... (Treleaven & Wells, 2007;Sims et al., 2012), national surveys of the general population (Wells et al., 2015), motor performance (Lim et al., 2015;Sevick et al., 2016;Taha et. al., 2016), posture/balance training (Dutta et al., 2014;Mentiplay et al., 2013;Oh et al., 2014;Saenz-deUrturi & Garcia-Zapirain Soto, 2016) and rehabilitation (Galna et al., 2014;Mobini et al., 2015;De Rosario et al., 2014;Shapi'i et al., 2015). Advantages of 3D scanning represent a rapid raw data collection, a wide variety of digital shape outputs that can extend to 2D or 3D format, an electronic achieving of scans, which could be utilized in future analysis with improved software, a construction, and comparison of composite shape models, etc. (Wells et al., 2015;Šimenko & Čuk, 2016). ...
Conference Paper
Full-text available
With the progress of technology, new digital shape-analysis tools are being developed for use in several different fields. Innovation and market demand has pushed developers to create a portable 3D scanner. The aim of this research was to perform a comparison of a new portable measuring system for digital measurement of anthropometric dimensions of the body, with the system of manual anthropometry. The results show that the Coefficient of determination (R2) was in 7 measurements over 90%, in 6 measurements over 80%, and in 2 measurements above 74.9%. Cronbach Alpha results of compared variables were all over 90%, which show very strong expected correlations. No significant bias between measurement techniques was shown as Bland-Altman plots showed a good agreement between measurement techniques with a small number of outliers. Results provide high validity and accuracy of the new portable scanner when correctly used. However, methods of 3D body scanning and classical anthropometry should not be regarded as interchangeable as there are differences in initial body positions due to the implementation of measurement protocols. Further work is recommended to make the two methods more interchangeable, with the possible usage of corrective coefficients.
... Its measurement errors have also been carefully investigated [39][40][41][42][43]. These results clearly support the use of the Kinect sensor for the assessment of gait and balance performance [44][45][46][47][48][49][50][51][52][53][54][55][56]. It should be noted that many similar RGB-depth (RGB-D) sensor devices are already available. ...
Article
Full-text available
A stable posture requires the coordination of multiple joints of the body. This coordination of the multiple joints of the human body to maintain a stable posture is a subject of research. The number of degrees of freedom (DOFs) of the human motor system is considerably larger than the DOFs required for posture balance. The manner of managing this redundancy by the central nervous system remains unclear. To understand this phenomenon, in this study, three local inter-joint coordination pattern (IJCP) features were introduced to characterize the strength, changing velocity, and complexity of the inter-joint couplings by computing the correlation coefficients between joint velocity signal pairs. In addition, for quantifying the complexity of IJCPs from a global perspective, another set of IJCP features was introduced by performing principal component analysis on all joint velocity signals. A Microsoft Kinect depth sensor was used to acquire the motion of 15 joints of the body. The efficacy of the proposed features was tested using the captured motions of two age groups (18–24 and 65–73 years) when standing still. With regard to the redundant DOFs of the joints of the body, the experimental results suggested that an inter-joint coordination strategy intermediate to that of the two extreme coordination modes of total joint dependence and independence is used by the body. In addition, comparative statistical results of the proposed features proved that aging increases the coupling strength, decreases the changing velocity, and reduces the complexity of the IJCPs. These results also suggested that with aging, the balance strategy tends to be more joint dependent. Because of the simplicity of the proposed features and the affordability of the easy-to-use Kinect depth sensor, such an assembly can be used to collect large amounts of data to explore the potential of the proposed features in assessing the performance of the human balance control system.
... Even though the sample size used in our study is similar or slightly higher than in other related studies, 14,15,23,25 it would be necessary to increase the sample size to improve the accuracy of the correction model and to reinforce the validity of the results. ...
Article
Introduction The joint range of motion (ROM) is an important clinical parameter used to assess the loss of functionality resulting from joint bleedings in people with haemophilia. These episodes require a close follow‐up and, to decrease patients' hospital dependence, telemedicine tools are needed. Therefore, this study is aimed to analyse the validity of the Microsoft Kinect V2 sensor with corrected angle measurement to be used in the monitoring of elbow ROM in people with haemophilia. Methods A convenience sample of 10 healthy controls (CG) and 10 patients with haemophilia with elbow arthropathy (HG) participated in this study. Full ROM of elbow joints was measured in the frontal view with a 10‐degree sweep using: (a) a clinical goniometer; (b) the Kinect V2; (c) the Kinect V2 with angle correction; and (d) using a photograph. Bland‐Altman graphs (mean and 95% Limits of Agreement [LOA]) and Wilcoxon test were used to determine differences between measurements and groups. Results The angle‐corrected Kinect V2 measurement removed the skew in the original data, reducing the average errors from 7.9° (LoA = −10.3°; 26.0°; CG) and 9.5° (LoA = −7.9°; 26.9°; HG) to −0.1° (LoA = −8.1°; 7.9°; CG) and −0.7° (LoA = −10.7°; 9.3°; HG). Conclusions These error levels allow the use of Kinect V2 in the clinical practice. Kinect V2 with angle correction can complement the classical goniometry allowing an efficient and touchless measurement of ROM.
... Currently, a MS Kinect (3) is used to detect the movements of the participant when interacting with the system. Recognition of the movements detected by the Kinect is reflected in their visual representation, providing comparable and measurable movement definitions and movement sequences [16]. To simplify the interaction with the overall system, a tablet (not in the figure) and a PlayStation 3 Buzzer (6) were used. ...
Conference Paper
Dementia not only affects the cognitive capabilities, especially memory and orientation, but also physical capabilities, which are associated with a decrease of physical activities. Here, ICT can play a major role to improve health, quality of life and wellbeing in older adults suffering from dementia and related stakeholders, such as relatives, professional and informal caregivers. The aim of the presented system is to increase physical and cognitive capabilities of people with dementia and their caregivers to support them in daily life activities, reduce the strain of the caregivers and improve both their wellbeing.
... Microsoft Kinect is used in the field of anthropometrics (Sameijma et al., 2012;Xu et al., 2013;Clarkson et al., 2016;Zhang et al., 2015), gait analysis (Springer & Seligman, 2016;Pfister et al., 2014;Motiian et al., 2015;Prochazka et al., 2015;Cippitelli et al., 2015), motor performance (Lim et al., 2015;Sevick et al., 2016;Taha et. al., 2016), posture/balance training (Dutta et al., 2014;Mentiplay et al., 2013;Oh et al., 2014;Saenz-de-Urturi & Garcia-Zapirain Soto, 2016) and rehabilitation (Galna et al., 2014;Mobini et al., 2015;De Rosario et al., 2014;Shapi'i et al., 2015). Reliability of instruments in clinical and sport application differ, therefore the goal of this research was to initially determine: ...
Conference Paper
Full-text available
Background: Microsoft Kinect is used in the field of anthropometry (Sameijma et al.,). Reliability of instruments in clinical and sport application differ, therefore the goal of this research was to initially determine the protocol of validation of a new measuring instrument for digital measurement of anthropometric dimensions of the body (structural and metric). Reliability of results in this paper was tested on three classically and digitally measured anthropometric variables, i.e. height, left forearm length and left lower leg length. Methods: Male and female employees of the Technology Park Zagreb (N=52) volunteered for this research. Subjects were wearing their everyday clothes. Among 471 assessed variables (3 + ((26 * 6)) * 3) three variables from a set of classically measured anthropometric dimensions were extracted-height, length of left forearm and length of left lower leg. Classical measurements were conducted through standard IBP protocols, a Standardized protocol for digital measurement (DM-I) was produced. Data were analyzed by Statistica 12 for Windows operating system. Mean, standard deviation, range, variability coefficient, skewness and kurtosis were used as descriptive parameters, as well as Pearson correlation coefficient, Spearman-Brown alpha, Cronbach`s alpha and Spearman-Brown (standardized) alpha. Results: Classically and digitally measured height in average results do not differ significantly, while for lengths of the left forearm and the left lower leg do indicate significant differences (lower values). The differences could be attributed to different reference points used in two measurement methods. Measures of internal consistency (reliability) for digitally measured variables: height of the body, length of left forearm and length of left lower leg demonstrate high reliability (Cronbach alpha, the standardized alpha 0.995 to 0.997) and the average inter-item correlation (0.973 to 0.985), indicates a high internal consistency between items related to digitally measured height. Reliability was slightly lower for digitally measured length of the left forearm and lower leg due to greater differentiation in average inter-item correlations coefficients. Conclusions: Digital measurements with Kinect are not appropriate for clinical trials demanding high precision. There is no statistical evidence that could differentiate distances of examinee from Kinect sensor in order to define optimal distance (as long as subject stands within Kinects range. Small errors occur due to clothing, possibly due to illumination, and sensor height and distance, which is in line with previous research.
Chapter
Standardized measurement protocol for assessing anthropometric dimensions of human body is precisely predefined by International Biological Programme (IBP), [16]. Objective of this research was to produce, compare, validate and standardize protocol for digital measurement (DM-I) using Kinect sensor in order to economize future large scale research. Results in selected variables revealed that classically and digitally measured parameters, e.g. height, in average results do not differ significantly, while e.g. for lengths of the left forearm and the left lower leg do indicate lower values. Different reference points used in two measurement methods, i.e. anthropometric points (IBP) and Kinect points, represent similar, but not identical representation of human body. Measures of internal consistency (reliability) for assessed digitally measured variables demonstrated high reliability, but inappropriateness for clinical trials demanding extremely high precision. Since reliability of instruments in clinical and sport application differ, broad spectrum of useful specific diagnostic tools and instruments may be produced based on results assessed in this research.
Conference Paper
In this work, a low-budget markerless gait analysis application that is aimed at orthopedic care was built and optimized for use in developing countries and small practices. The application utilizes Microsoft Kinect to detect and track body joints and then calculates nine gait parameters that are important for performing a gait assessment. The measurements of the hip flexion/extension, hip abduction/adduction and knee flexion/extension followed the graphs of standard gait pattern. Also they were consistent and homogeneous among all ten participants. Due to the low tracking accuracy for both the ankle and the foot, measurements of ankle plantar flexion and dorsiflexion didn’t follow the trajectory of the standard graph. Moreover, measurements from participants were sparsely distributed across their average value.
Article
Full-text available
Kinect-based physical rehabilitation grows significantly as a mechanism for clinical assessment and rehabilitation due to its flexibility, low-cost and markerless system for human action capture. It is also an approach to provide convenience for for patients’ exercises continuation at home. In this paper, we discuss a review of the present Kinect-based physiotherapy and assessment for rehabilitation patients to provide an outline of the state of art, limitation and issues of concern as well as suggestion for future work in this approach. The paper is constructed into three main parts. The introduction was discussed on physiotherapy exercises and the limitation of current Kinect-based applications. Next, we also discuss on Kinect Skeleton Joint and Kinect Depth Map features that being used widely nowadays. A concise summary with significant findings of each paper had been tabulate for each feature; Skeleton Joints and Depth Map. Afterwards, we assemble a quite number of classification method that being implemented for activity recognition in past few years. © 2018 Institute of Advanced Engineering and Science. All rights reserved.
Article
Full-text available
This paper presents an intelligent Kinect and fuzzy inference system based e-rehabilitation system. The Kinect can detect the posture and motion of the patients while the fuzzy inference system can interpret the acquired data on the cognitive level. The system is capable to assess the initial posture and motion ranges of 20 joints. Using angles to describe the motion of the joints, exercise patterns can be developed for each patient. Using the exercise descriptors the fuzzy inference system can track the patient and deliver real-time feedback to maximize the efficiency of the rehabilitation. The first laboratory tests confirm the utility of this system for the initial posture detection, motion range and exercise tracking.
Article
Full-text available
Di Tore S, D'Elia F, Aiello P, Carlomagno N, Sibilio M. Didactics, movement and technology: new frontiers of the human-machine interaction. J. Hum. Sport Exerc. Vol. 7, No. Proc1, pp. S178-S184, 2012. The current focus on the use of new technologies and media for teaching-learning purposes has led to an intensifying interest in the properties and peculiarities of educational videogames. Creating a learning environment within a video game might be an opportunity to capitalize and use in a constructive way the time that more and more teens spend playing video games. In light of this, it is particularly interesting the continuous evolution of computer videogames known as "exergames", a term derived from the joining of the words "exercise" and "games". Exergames are video games which encourage the emotional involvement and the sense of presence through interactions based on devices that allow a greater involvement of the body respect to typical controllers such as joysticks, joy pads, keyboards and mousses. These games involve the player in some form of exercise and physical activity through video games and specific feedback and data collection devices. The aim of this study, which is currently under development, is therefore to design an active and engaging learning environment in order to assess whether it is suitable to stimulate and facilitate the learning processes, particularly focusing on the functions of coordination and sensorimotor integration, through the typical interaction of exergames and exploiting the properties and peculiarities of the Microsoft Kinect system as a device to acquire data on the body movements of the player.
Conference Paper
Full-text available
The use of Virtual Reality technology for developing tools for rehabilitation has attracted significant interest in the physical therapy arena. This paper presents a comparison of motion tracking performance between the low-cost Microsoft Kinect and the high fidelity OptiTrack optical system. Data is collected on six upper limb motor tasks that have been incorporated into a game-based rehabilitation application. The experiment results show that Kinect can achieve competitive motion tracking performance as OptiTrack and provide “pervasive” accessibility that enables patients to take rehabilitation treatment in clinic and home environment.
Conference Paper
New and powerful hardware like Kinect introduces the possibility of changing biomechanics paradigm, usually based on expensive and complex equipment. Kinect is a markerless and cheap technology recently introduced from videogame industry. In this work we conduct a comparison study of the precision in the computation of joint angles between Kinect and an optical motion capture professional system. We obtain a range of disparity that guaranties enough precision for most of the clinical rehabilitation treatments prescribed nowadays for patients. This way, an easy and cheap validation of these treatments can be obtained automatically, ensuring a better quality control process for the patient's rehabilitation.
Conference Paper
This paper describes a method to estimate a set of body dimensions of a subject using the Microsoft KINECT sensor. Principal Component Analysis(PCA) of the AIST anthropometric database shows first and second principal components are interpreted as scale and degree of obesity respectively. A few body dimensions were chosen by linear multiple regression analysis. Applying the resulting estimator to a given subject using the KINECT sensor to obtain joint positions and 3 body dimensions by analyzing depth image, we can estimate a set of body dimensions for the subject. The system is described and experimental results are shown.
Article
In this paper, we present a method for human full-body pose estimation from depth data that can be obtained using Time of Flight (ToF) cameras or the Kinect device. Our approach consists of robustly detecting anatomical landmarks in the 3D data and fitting a skeleton body model using constrained inverse kinematics. Instead of relying on appearance-based features for interest point detection that can vary strongly with illumination and pose changes, we build upon a graph-based representation of the depth data that allows us to measure geodesic distances between body parts. As these distances do not change with body movement, we are able to localize anatomical landmarks independent of pose. For differentiation of body parts that occlude each other, we employ motion information, obtained from the optical flow between subsequent intensity images. We provide a qualitative and quantitative evaluation of our pose tracking method on ToF and Kinect sequences containing movements of varying complexity.
Article
In articulated tracking, one is concerned with estimating the pose of a person in every frame of a film. This pose is most often represented as a kinematic skeleton where the joint angles are the degrees of freedom. Least-committed predictive models are then phrased as a Brownian motion in joint angle space. However, the metric of the joint angle space is rather unintuitive as it ignores both bone lengths and how bones are connected. As Brownian motion is strongly linked with the underlying metric, this has severe impact on the predictive models. We introduce the spatial kinematic manifold of joint positions, which is embedded in a high dimensional Euclidean space. This Riemannian manifold inherits the metric from the embedding space, such that distances are measured as the combined physical length that joints travel during movements. We then develop a least-committed Brownian motion model on the manifold that respects the natural metric. This model is expressed in terms of a stochastic differential equation, which we solve using a novel numerical scheme. Empirically, we validate the new model in a particle filter based articulated tracking system. Here, we not only outperform the standard Brownian motion in joint angle space, we are also able to specialise the model in ways that otherwise are both difficult and expensive in joint angle space.
Article
Recent studies involving the linguistic analysis of texts has shown the importance of the interrelationships across the various strata of language. A network based collaborative system represents a major step forward in carrying out such cross-stratum analysis. Two vital prerequisites for developing this system include facilities for retrieval of text resources and a mechanism for capturing complex linguistic information. In this paper, a database framework for collaborative retrieval of complex linguistic information is presented. The framework provides a platform for combining both syntactic and semantic processes of parallel multilingual corpora leading to pattern visualization, speech analysis, speech-to-text transcription and statistical analysis.