MAGNI Dynamics: A Vision-Based Kinematic and Dynamic Upper-Limb Model for Intelligent Robotic Rehabilitation

Abstract

This paper presents a home-based robot-rehabilitation instrument, called "MAGNI Dynamics", that utilized a vision-based kinematic/dynamic module and an adaptive haptic feedback controller. The system is expected to provide personalized rehabilitation by adjusting its resistive and supportive behavior according to a fuzzy intelligence controller that acts as an inference system, which correlates the user's performance to different stiffness factors. The vision module uses the Kinect's skeletal tracking to monitor the user's effort in an unobtrusive and safe way, by estimating the torque that affects the user's arm. The system's torque estimations are justified by capturing electromyographic data from primitive hand motions (Shoulder Abduction and Shoulder Forward Flexion). Moreover, we present and analyze how the Barrett WAM generates a force-field with a haptic controller to support or challenge the users. Experiments show that by shifting the proportional value, that corresponds to different stiffness factors of the haptic path, can potentially help the user to improve his/her motor skills. Finally, potential areas for future research are discussed, that address how a rehabilitation robotic framework may include multisensing data, to improve the user's recovery process.

Full text

MAGNI Dynamics: A Vision-Based Kinematic and
Dynamic Upper-Limb Model for Intelligent Robotic
Rehabilitation
Alexandros Lioulemes, Michail Theofanidis, Varun Kanal, Konstantinos Tsiakas, Maher Abujelala, Chris
Collander, William B. Townsend, Angie Boisselle, Fillia Makedon
Abstract—This paper presents a home-based robot-rehabilitation
instrument, called ”MAGNI1Dynamics”, that utilized a vision-based
kinematic/dynamic module and an adaptive haptic feedback
controller. The system is expected to provide personalized
rehabilitation by adjusting its resistive and supportive behavior
according to a fuzzy intelligence controller that acts as an inference
system, which correlates the user’s performance to different stiffness
factors. The vision module uses the Kinect’s skeletal tracking to
monitor the user’s effort in an unobtrusive and safe way, by estimating
the torque that affects the user’s arm. The system’s torque estimations
are justified by capturing electromyographic data from primitive
hand motions (Shoulder Abduction and Shoulder Forward Flexion).
Moreover, we present and analyze how the Barrett WAM generates
a force-field with a haptic controller to support or challenge the
users. Experiments show that by shifting the proportional value,
that corresponds to different stiffness factors of the haptic path, can
potentially help the user to improve his/her motor skills. Finally,
potential areas for future research are discussed, that address how
a rehabilitation robotic framework may include multisensing data, to
improve the user’s recovery process.
Keywords—Human-robot interaction, kinect, kinematics,
dynamics, haptic control, rehabilitation robotics, artificial
intelligence.
I. INTRODUCTION
Ajob or traffic accident, a misfortune even or an
unforeseen stroke can lead to brain or musculoskeletal
injuries, that impact motor and cognitive functions. Modern
Alexandros Lioulemes is with the HERACLEIA Human-Centered
Computing Laboratory, Department of Computer Science and Engineering,
The University of Texas at Arlington, USA, Institute of Informatics
and Telecommunications, (N.C.S.R.) Demokritos, Athens, Greece, Barrett
Technology LLC, Boston, MA (e-mail: alexandros.lioulemes@mavs.uta.edu).
Michail Theofanidis and Konstantinos Tsiakas are with the HERACLEIA
Human-Centered Computing Laboratory, Department of Computer
Science and Engineering, The University of Texas at Arlington,
USA, Institute of Informatics and Telecommunications, (N.C.S.R.)
Demokritos, Athens, Greece (e-mail: michail.theofanidis@mavs.uta.edu,
konstantinos.tsiakas@mavs.uta.edu).
Maher Abujelala, Chris Collander and Fillia Makedon are with the
HERACLEIA Human-Centered Computing Laboratory, Department of
Computer Science and Engineering, The University of Texas at Arlington,
USA (e-mail: maher.abujelala@mavs.uta.edu, chris.collander@mavs.uta.edu,
makedon@uta.edu).
William B. Townsend is with the Barrett Technology LLC, Boston, MA
(e-mail: wt@barrett.com).
Angie Boisselle is with the Cook Children’s Healthcare System, Fort Worth,
TX (e-mail: aboisselle@att.net).
Varun Kanal is with the HERACLEIA Human-Centered Computing
Laboratory, Department of Computer Science and Engineering, The University
of Texas at Arlington, USA.
1MAGNI is the God of strength in Norse mythology
physical rehabilitation has proven to be instrumental in
the ability to partially or fully heal patients with impaired
motor capabilities. During the last two decades, the use
of robotic instruments for upper-limb rehabilitation has
increased as robot-based rehabilitation provides an accurate
evaluation of motor recovery and automates simple tasks
that burden caregivers. Nowadays, as the number of people
that require physical rehabilitation has increased, the need
has arisen to create low-cost home-based robotic instruments
that are simple, acceptable and provide easy monitoring,
smart assessment, and adaptable training [1]. However, an
element that is poorly designed in the current rehabilitation
robotics systems is the incorporation of the user in the robot’s
control loop, to provide personalized and adaptable training.
To achieve that we present an accurate and safe motion
analysis system that does not rely on wearable sensors [2].
The system estimates the user’s force/torque state which can
be used as a visual feedback to the robot’s control loop
system. Moreover, the system can be used by physicians and
occupational therapists to monitor the physical state of the
patient’s upper-limb, while performing repetitive exercises. To
prove the intellectual merit of the algorithm, we validate our
previous kinematic and dynamic estimation system [3] by
utilizing electromyographic signals from the Delsys [4] device
and by correlating the electromyographic values with torque
estimations.
Due to the increasing number of senior patients that require
physical rehabilitation, there is a growing need for therapist
and nurses that are able to provide home-based assistance and
training. In our work, we proposed to automate the physical
rehabilitation by building a robotic system that introduces
home-based robotic instruments. In Fig. 1, we envision a
home-based rehabilitation system that consists of a robotic
arm, a monitor system, such as a depth camera, and a virtual
reality exergame system that displays instructions, as well as
allows communication with the therapists.
In the field of rehabilitation robotics, there are two
types of robots developed for upper extremity exercises, the
end-effector and the powered exoskeletons devices [5]. The
end-effector robots were the first devices to be implemented
and tested in stroke rehabilitation research due to their
straightforward design. On the other hand, the powered
exoskeletons carry the distinct advantage of enabling both
accurate measurements of the torques that affect each joint,
as well as the precise recording and monitoring of motion
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
158International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

Fig. 1 Proposed home-based robotic rehabilitation system
trajectories in joint space [6]. Unfortunately, the powered
exoskeletons constrain the user’s range of motion due to their
complex configuration. In our work, we utilize the Barrett
WAM manipulator [20] for its advanced mechanical structure
and high haptic resolution in conjunction with the Kinect
skeleton tracker [21] to achieve highly dynamic adaptation,
force feedback and torque sensing to deliver an unobtrusive,
safe and guided physical therapy system.
As seen from Fig. 1, the system is consisted of the following
components: The RGBD camera sensor that provides skeletal
tracking information, which is fed into the proposed vision
system. Note that this system is thoroughly analyzed in our
past work [3], [23]. Furthermore, the estimated torque (τe)of
the user is passed to a fuzzy controller. The fuzzy controller
acts as a high intelligence system that shifts the gains (KP)of
the haptic controller, according to some abstract rules that have
been defined by the therapist in a linguistic manner, and the
performance of the user. As a result, the fuzzy intelligence
system adjusts the control input signal (τr) of the robot to
provide adaptive/assistive training.
II. RELATED WORK
Joint torques are of main importance for physicians and
occupational therapists to analyze the effects of rehabilitation
and to obtain an indicator of patient’s functional capacity to
perform a motion [7]. A joint’s strength is assessed through
the measurement of the maximal joint torque, which represents
the resultant action of all muscles crossing the joint. Manual
muscle testing (MMT) is a measure of upper and lower
body strength that occupational and physical therapists often
complete as part of a clinical evaluation and to measure
progress in therapy. MMT is a graded scale (typically on
a scale from zero to five) that is used to assess patients
with neurological or orthopedic impairments [8]. A score of
zero indicates that there is not any muscle contraction to five
indicates that strong pressure can be applied. Many issues arise
because MMT can be subjective based on many factors. The
validity and reliability of MMT are dependent upon a variety
of factors including training of the therapist; the patients
diagnosis, pain level, and other physiologic issues; which
muscle is tested; the position of the patient; hand placement
of therapist during testing; and variability between therapists
[9].
The rehabilitation therapists may change the parameters of
the exercise or activities (commonly referred to as grading)
between or during treatment sessions, based on confounding
patient factors such as pain or fatigue [10]. For example, the
therapist may change the number of repetitions, the number
of sets, and/or the amount of resistance given to the patient.
These parameters may remain consistent over time or need
to be changed during each session based on the patients
performance and muscle fatigue. Multiple researchers have
attempted to generate models for muscle fatigue based on
joint torques and muscle contraction levels. For example, the
authors in [11] utilized electromyographic data and derived
an analytical muscle model, taking into account physiological
and anatomical data, to estimate the joints’ torque. This model
helps them to generate joint torques and stiffness values while
the user is interacting with a rehabilitation instrument.
One simple exercise in rehabilitation is to repetitively
follow pre-described trajectories to help users strengthen their
weakened muscles or regain motor control. A haptic path can
be defined as a virtual tunnel that uses force feedback to help
users move through that path or constrain them from deviating
in other directions. The authors in [12] use gait trajectories
to help users while doing exoskeleton gait training on the
treadmill. They proposed a haptic controller designed to be
’assist-as-needed’ system, which can apply suitable forces on
the patient’s leg to help him move on the desired trajectory.
Similarly, in upper limb rehabilitation, [13]-[15] tracking the
performance and progress of the users, can be achieved by
comparing their measured trajectories with the Dynamic Time
Warping (DTW) algorithm [25]. The literature has shown that
haptic feedback/guidance can help the users improve their
tracing abilities by following a prescribed trajectories [16],
[17]. This haptic feedback can be by probing the user’s hand
through the path or by providing perpendicular forces that
prevent the user’s hand to deviate from the desired path.
A considerable amount of research has been conducted
to implement a robotic rehabilitation system that adapts
its behavior according to the patient’s performance and
physiological state. Rajibul et. al. [18], have presented
preliminaries studies in developing a fuzzy logic intelligent
system for autonomous post-stroke upper-limb rehabilitation.
In their work, an intelligent system estimates the muscle
fatigue of the patient and tunes the control parameters
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
159International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

Fig. 2 Robot-based Rehabilitation system
to generate different haptic effects. Badesa et. al. [19],
have incorporated multisensory data in the control loop to
adaptively and dynamically change in real-time the therapy.
The aforementioned results demonstrate the potential to create
a fuzzy system that adapts the robot’s behavior and delivers
personalized rehabilitation sessions. Similarly, in our work,
we incorporated a fuzzy logic module that controls the haptic
forces which are exerted upon the user.
The following sections of the paper are dedicated to
the thorough analysis and representation of the proposed
intelligent rehabilitation system. In Section III we recap the
algorithm that is used by the proposed vision system and we
justify the correctness of each estimation. Moreover, in Section
IV, we present the developed haptic force-field impedance
controller and we test its application with an chronic stroke
patient and an unimpaired user. Lastly, we provide some future
discussion on how to make this robotic framework capable of
making decisions that would improve the user’s performances
over time.
III. HUMAN ARM KINEMATIC AND DYNAMIC VISION
SYSTEM
A. Algorithm
At this point, we would like to recap our previous work [23]
that used a biomechanical model and the Kinect depth camera
to reconstruct upper-limb kinematics and dynamics in the
joint space. Algorithm 1 presents the steps for the extraction
of the human arm kinematics and dynamics from a Kinect
camera. As an input, the system must capture the person
who is performing the exercise with the Kinect, according to
the configuration that Fig. 3 suggests. Once the trajectory of
the subject’s arm has been captured, the Kinect passes the
cartesian positions of the chest, shoulder, elbow and wrist
frames to the first unit of the system.
The system then applies a median filter to the Kinect data
to eliminate any abnormal behavior from the skeleton tracking
algorithm of the Kinect. The result of this module produces
a smooth cartesian trajectory that is used by the Inverse
Kinematics Solver (IK Solver) to provide an estimation of
the angles of the human arm joints. Afterward, the system
produces the first estimation in joint space. In the next
iteration, the system makes sure that all data in joint space are
characterized by a polynomial profile function. This happens
because the motion of all rigid bodies, such as our Kinematic
Model, must be expressed with a polynomial function that
can produce a second, third or even forth derivative (jerk)
[24]. Lastly, as an output, the system provides an estimation
of the torques that affect the subject’s arm with the RNE
method. Interested readers can read our previous work, which
explains in great detail the formulation of the forward and
inverse kinematics equations [3].
Algorithm 1 Steps to calculate human arm dynamics using
Kinect camera system and a robotic arm
1: INPUT1: A sequence {Pt}N
t=1 of frames recordings from Kinect,
where each Pt=(PWt,P
Et,P
St,P
Ct)consists of the cartesian
position of the wrist, elbow, shoulder and chest.
2: INPUT2: Import user height and weight and extract
anthropomorphic data for the human body segments for
the length and mass of the upper and lower section.
3: INPUT3: Add the external forces f
B
robot from the robotic arm
that are exerted to the user’s wrist.
4: Reconstruct a raw model from the captured {Pt}N
t=1 frames
according to the proposed kinematic model using the
Homogeneous transformations of our previous work [3].
5: Apply a moving median filter to the raw position data.
6: Utilize the proposed Inverse Kinematics (IK Solver) to generate
an estimation of the joint angles {θr(t)}N
t=1 and r=[1, ..., 4]
7: Apply a higher-order polynomials to the θr(t)in order to fit the
joints estimation θe(t)=a0+a1t+a2t2+a3t3+a4t4+a5t5
to the trajectory sequence (exercise).
8: Generate estimated angles: {θe(t)}N
t=1 and e=[1, ..., 4]
9: Recreate the kinematic model according to the forward kinematic
equations as per [3].
10: Apply the Recursive Newton-Euler [22] dynamics algorithm
(RNE) and propagate the external force f
B
robot from the robotic
arm to the user wrist joint.
11: OUTPUT: Export the human arm joint velocity, acceleration and
torque profiles (q, ˙q, ¨q)for the recorded trajectory sequence with
the applied forces of the robotic arm.
B. Experimental Setup
In order to fully validate that the torque estimation derived
by our biomechanical model is correct, we conducted a series
of experiments that involves primitive arm movements that
isolate the shoulder axis and muscle activations. Fig. 4 shows
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
160International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

Fig. 3 4 DoF Kinematic Model of the Human Arm
Fig. 4 Muscles to sensor placement
the Delsys sensors placement in the user’s arm and the muscles
area that are associated.
Our initial goal is to correlate the joints’ frame placement,
according to Fig. 3, with the muscles that are triggered and
move the shoulder at each axis. For this reason, sensor 3 has
been placed on the Lateral Deltoid muscle, sensor 4 has been
placed to the Anterior Deltoid muscle area, connecting to the
clavicle, and sensor 1 and 2 were placed to the biceps and
triceps respectively. The exercise that is first chosen is the
shoulder abduction (Fig. 5a). This allows the first frame of
the shoulder to rotate along axis z1in the positive direction.
The second exercise is the shoulder forward flexion (Fig. 5b)
that allows the second shoulder frame to rotate along axis z2.
From the experimental results, in Fig. 5a, it is obvious
that the first exercise triggers the third sensor more which
correlates the deltoid’s muscle movement. The torque values
of the frame 1 at the beginning are close to 11 N/m and when
the shoulder is fully abducted they reach 36 N/m. For the
frame 2, the absolute torque values increased slightly exactly
like the correspondence muscle contraction (sensor 4).
The second experimental results (Fig. 5b) show the opposite
torque value estimation which corresponds with the muscles’
activation. The torque values of the second frame are increased
from 9 N/m to 34 N/m relatively as sensor 4 jumps. The frame
1 torque values show some discrepancy but this is caused
because of the axis zis crossed while the user is flexing
forward his arm. Also, sensor 3 is reacting to this motion as
the deltoid muscle is triggered slightly. It should be mentioned
that is difficult to isolate the muscle’s activation at the shoulder
as they are wrapped together to help the shoulder’s rotation to
the three axes.
C. Experimental Analysis
To analyze the electromyographic (EMG) data, the collected
signal was first filtered. Filtering was done in 3 stages: High
Pass filter, Low Pass filter and Notch Filter. A butterworth
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
161International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

(a) Shoulder Abduction
(b) Shoulder Forward Flexion
Fig. 5 Experimental results for the Shoulder Abduction and Shoulder Forward Flexion Motions
filter was used to design these filters. The corner frequency of
the high-pass filter was 10 Hz while the corner frequency of
the low-pass filter was 500 Hz and the frequency of the notch
filter was 50 Hz. This process removed any noise below 10
Hz, above 500 Hz and at 50 Hz.
After the filtering process, the peaks of the EMG were
found. These peaks were used to find a relationship between
the torque extracted from the Kinect data and the EMG
extracted from the Delsys. We used the inbuilt peak detection
function in MATLAB to detect the peaks. Furthermore, both
the EMG and the torque data were downsampled to 1 HZ,
resulting in one data point per second. This was done for
EMG data too. Peak data at each second was calculated as
the mean of the EMG peaks for 500 ms on either side of the
second mark.
Lastly, the relationship was found by using Kendalls
Rank Correlation method. This is a nonparametric correlation
method. It operates by assigning ranks to each datapoint and
calculating the concordant and the discordant pairs. Consider
a data point in a set, any data point below the considered one
is assumed to be a concordant pair if the rank for the new data
TABLE I
CORRELATION VALUES BETWEEN TORQUE AND ELECTROMYOGRAPHIC
SIGNAL
Exercise Sensor Tau P
Abduction EMG3 1 5.51E-07
EMG4 -0.82222 3.58E-04
Forward
Flexion
EMG3 -1 4.96E-05
EMG4 1 4.96E-05
point is smaller than the rank for the considered data point. It
is a discordant pair if the rank for the new data point is greater
than the initial data point. Kendalls correlation calculates τby
using the following formulae:
τ=D−D
D+D(1)
where Care the Concordant Pairs and Dare the Discordant
Pairs. This yields a value between -1 and 1 where -1 indicates a
strong negative correlation and ’+1’ indicates a strong positive
correlation. 0 indicates no correlation.
Figs. 6a and 6b show the opposite correlation of the
torque values estimated by our biomechanical model and
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
162International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

(a) Shoulder Abduction
(b) Shoulder Forward Flexion
Fig. 6 Experimental Analysis for the Shoulder Abduction and Shoulder Forward Flexion Motions
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
163International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

the electromyographic filtered signal after the analysis. These
results confirm our hypotheses for torque estimation per axes
with the isolated muscle to electromyographic data analysis.
Specifically, in Table I we can see that the correlation values
for the shoulder abduction motion give τ=1and τ=
−0.82222 for the EMG3 and EMG4 respectively. This means
that the torque1value has linear increasing rate such as
the EMG3 signal. On the other hand, torque2value follows
closely the linear decreasing of the signal. An analogous trend
is observed in the shoulder forward flexion EMG and torque
correlation graph (Fig. 6b), because the increments rates are
opposite. Thus, we can justify the torque values and claim
that our biomechanical model can be used for shoulder torque
estimation in rehabilitation exercises.
IV. HAPTIC PATH
In this work, the robotic arm can help guide the user
to follow a precise trajectory as dictated by previously
recorded exercises done by a physical/occupational therapist.
The user can attempt to perform the prescribed exercise and if
he/she deviates from the prescribed trajectory, an appropriate
correctional force is applied by the robotic arm to guide
him/her back to the correct trajectory. Besides spatial, the
deviation can also be temporal, i.e. the user performs the
exercise much slower or much faster than the therapist. When
either of the two deviation types occurs, an error-correction
force is applied to bring the patients hand position closer to
the prescribed trajectory.
A. Haptic Forces
In order to assist the user to stay close to a prescribed path
in the 3D space, a force-field (Fig. 8a) is rendered from the
given start position p
Bstart to the end target position p
Bend
of the path, as Fig. 8b depicts. If the user deviates from the
given path, a perpendicular force will be applied in order to
push the users arm to stay close to the path. At each moment,
the robot’s end-effector position p
Btsearches for the closest
point at the haptic path. The direction and magnitude of the
force in the end-effector position p
Bt, is calculated by the
p
BNpoint and the absolute distance dtrespectively.
The haptic path has been reconstructed with the use of an
impedance control mechanism that controls the position of the
robot’s end-effector ( p
Bt) at the corresponding trajectory point
(p
BNN). The impedance control aims to increase or decrease
the compliance (stiffness) of the robot in order to allow the
user to deviate more or less from the predefined trajectory.
This stiffness values (K) constrains the user to the trajectory
and acts as the spring constant. The force generated (ft)is
equivalent to ft=K×dt. The proportional gain (P) that
represents the stiffness of the force-field of the impedance
controller, behaved similarly to the K spring constant value.
By changing the P value we are able to bring the patient’s
hand closer to the therapist’s prerecorded trajectory.
B. Haptic Control
The control chart of the proposed control system can
be found in Fig. 7. The Barrett WAM robot is directly
interacting with patients arm τp. All motion parameters that
associate the kinematics of the robot are measured with
internal sensors. In our case, the measurements are provided
through the Barrett WAM’s Puck sensor that operates in
500 Hz. The forward kinematics of the robot is used to
calculate the actual end-effector position, which is fed into the
visual interface implemented in the Unity 3D game engine.
This provides a visual feedback about the end-effector’s
trajectory as well as the start position pstart and target
position ptarget that defines the haptic path. This information
is used to calculate the nearest neighbor point pNN on
the path and the tangential vector fassist/resist by means
of the end-effector position. The transposed Jacobian JT(q)
is used to calculate the corresponding joint torques τthat
accelerates the robot. Additionally, the compensation model
τcomp which is consisting of the friction, gravity and spring
compensation module, provides the necessary torque to keep
the arm stationary.
C. Haptic Experiments
In order to test the compliance of the impedance controller,
we recruited one chronic stroke patient and we conducted
three experiments with different proportional values. Then,
we analyzed the effects of the haptic controller by using the
Dynamic Time Warping method to derive spatial or temporal
error deviations in the user’s cartesian trajectory. Figs. 9a and
9b illustrates the Cartesian position in the plane during the
haptic path exercise. In particular, the desired trajectory is
shown with red targets to the stroke patient (Virtual Exercise)
(Fig. 2) and is represented by the red line in Fig. 9a. The
stroke patient was instructed to perform each exercise (Haptic
path) with the best of his abilities and try to reach all the red
virtual targets.
D. Haptic Response
Three exercises were performed with the stroke patient with
small breaks of 5 minutes (Fig. 9a). In the first, exercise (A) the
stroke patient was unassisted (P= 50) and his error deviation
was error =4.9114. At the second execution (B), the stiffness
value of the impedance controller was (P= 100) and the
stroke patient’s error trajectory deviation from the prescribed
path was error =0.65122. Finally, we increased the robot’s
assistance (C) with (P= 800) and he managed to execute the
exercise correctly (error =0.22548).
Similar experiments were conducted with an unimpaired
user. In Fig. 9b the user’s performance did not change
drastically as he manages to control the motion of his hand
successfully and his error deviation is getting better as long
the robotic arm constrains him to the prescribed trajectory.
It is clear that when the applied rendered forces constrain the
users to the prerecorded exercises, the error deviation is getting
smaller. This phenomenon implies that patient will be able to
increase hand coordination and improve motor skills with the
passage of time.
V. CONCLUSIONS AND FUTURE WORK
In this work, we presented an unobtrusive home-based
rehabilitation system that consists of an RGBD camera, an
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
164International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

Fig. 7 Control chart of the impedance haptic path controller implemented in the Barrett WAM robot
(a) Prescribed exercise represented by a haptic path
(b) Prescribed exercise represented by a haptic path
Fig. 8 Prescribed exercise represented by a haptic path
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
165International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

(a) Stroke patient (b) Unimpaired user
Fig. 9 Error deviation for three difference proportional values: A) P = 50; B) P = 100; C)P=800
end-effector robotic arm, an intelligent control module that
allows therapists to change the input parameters, and a haptic
controller that adjusts the control input signal of the robot.
Particularly, we validate our previous proposed vision-based
system with electromyographic signals from the Delsys
device for primitive motions. Furthermore, we investigate the
developed haptic controller’s response in exercises performed
with a chronic stroke patient and an unimpaired user.
In the future, we plan to integrate our home-based robotic
rehabilitation system with multisensory data coming from
physiological sensors, such as the Microsoft Band. These data
will estimate the user’s arousal and muscle fatigue in real-time
using a hierarchical fuzzy logic controller. It is estimated
that such a multisensing upper-limb intelligent rehabilitation
system will be able to adapt the robot’s behavior and deliver
personalized rehabilitation sessions.
ACKNOWLEDGMENT
This work is supported in part by the National Science
Foundation under award numbers 1338118. Any opinions,
findings, and conclusions or recommendations expressed
in this publication are those of the author(s) and do
not necessarily reflect the views of the National Science
Foundation.
REFERENCES
[1] Sivan, Manoj, et al. ”Home-based Computer Assisted Arm Rehabilitation
(hCAAR) robotic device for upper limb exercise after stroke: results of
a feasibility study in home setting.” Journal of neuroengineering and
rehabilitation 11.1 (2014): 1.
[2] Mukhopadhyay, Subhas Chandra. ”Wearable sensors for human activity
monitoring: A review.” IEEE Sensors Journal 15.3 (2015): 1321-1330.
[3] Theofanidis Michail, Lioulemes Alexandros, and Makedon Fillia. ”A
Motion and Force Analysis System for Human Upper-limb Exercises.”
International Conference on PErvasive Technologies Related to Assistive
Environments,(PETRA), Corfu Island Greece. 2016.
[4] Delsys, Inc. http://www.delsys.com/, Accessed on 03/22/2017.
[5] H. S. Lo, S. Q. Xie, ”Exoskeleton robots for upper-limb rehabilitation:
State of the art and future prospects, Medical Engineering & Physics”,
Volume 34, Issue 3, Pages 261-268, April 2012,.
[6] G. Maxime, et al. ”A robotic device as a sensitive quantitative tool
to assess upper limb impairments in stroke patients: a preliminary
prospective cohort study.”Journal of rehabilitation medicine44.3 (2012):
210-217.
[7] Ba. Laurent, et al. ”Joint torque variability and repeatability during
cyclic flexion-extension of the elbow.” BMC sports science, medicine
and rehabilitation 8.1 (2016): 1.
[8] Cuthbert, Scott C., and George J. Goodheart. ”On the reliability and
validity of manual muscle testing: a literature review.” Chiropractic &
osteopathy 15.1 (2007)
[9] Jepsen, Jrgen, et al. ”Manual strength testing in 14 upper limb muscles
A study of inter-rater reliability.” Acta Orthopaedica Scandinavica 75.4
(2004): 442-448.
[10] Toemen, Angela, Sarah Dalton, and Fiona Sandford. ”The intra-and
inter-rater reliability of manual muscle testing and a hand-held
dynamometer for measuring wrist strength in symptomatic and
asymptomatic subjects.” Hand Therapy 16.3 (2011): 67-74.
[11] Osu, Rieko, and Hiroaki Gomi. ”Multijoint muscle regulation
mechanisms examined by measured human arm stiffness and EMG
signals.” Journal of neurophysiology 81.4 (1999): 1458-1468.
[12] Banala, Sai K., Suni K. Agrawal, and John P. Scholz. ”Active Leg
Exoskeleton for gait rehabilitation of motor-impaired patients.” In 2007
IEEE 10th International Conference on Rehabilitation Robotics, pp.
401-407. IEEE, 2007.
[13] Abujelala, Maher, Alexandros Lioulemes, Paul Sassaman, and
Fillia Makedon. ”Robot-aided rehabilitation using force analysis.” In
Proceedings of the 8th ACM International Conference on PErvasive
Technologies Related to Assistive Environments, p. 97. ACM, 2015.
[14] Phan, Scott, Alexandros Lioulemes, Cyril Lutterodt, Fillia Makedon, and
Vangelis Metsis. ”Guided physical therapy through the use of the barrett
wam robotic arm.” In Haptic, Audio and Visual Environments and Games
(HAVE), 2014 IEEE International Symposium on, pp. 24-28. IEEE, 2014.
[15] Saraee,Elham, Margrit Betke. ”Dynamic Adjustment of Physical
Exercises Based on Performance Using the Proficio Robotic Arm.” In
Proceedings of the 8th ACM International Conference on PErvasive
Technologies Related to Assistive Environments. ACM, 2016.
[16] Liu, J., J. L. Emken, S. C. Cramer, and D. J. Reinkensmeyer. ”Learning
to perform a novel movement pattern using haptic guidance: slow
learning, rapid forgetting, and attractor paths.” In 9th International
Conference on Rehabilitation Robotics, 2005. ICORR 2005., pp. 37-40.
IEEE, 2005.
[17] Feygin, David, Madeleine Keehner, and R. Tendick. ”Haptic guidance:
Experimental evaluation of a haptic training method for a perceptual
motor skill.” In Haptic Interfaces for Virtual Environment and
Teleoperator Systems, 2002. HAPTICS 2002. Proceedings. 10th
Symposium on, pp. 40-47. IEEE, 2002.
[18] Huq, Rajibul, et al. ”Development of a fuzzy logic based intelligent
system for autonomous guidance of post-stroke rehabilitation exercise.”
Rehabilitation Robotics (ICORR), 2013 IEEE International Conference
on. IEEE, 2013.
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
166International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973

[19] Badesa, Francisco Javier, et al. ”Dynamic Adaptive System for
Robot-Assisted Motion Rehabilitation.” IEEE Systems Journal 10.3
(2016): 984-991.
[20] Barrett Technology, LLC. http://www.barrett.com/products-arm.htm,
Accessed on 03/22/2017.
[21] Microsoft Kinect. https://developer.microsoft.com/en-us/windows/kinect/develop,
Accessed on 03/22/2017.
[22] L. Ferrajoli and A. De Luca. A modified newton-euler method for
dynamic computations in robot fault detection and control. Proceedings
- IEEE International Conference on Robotics and Automation, pages
33593364, 2009.
[23] Lioulemes Alexandros, Michail Theofanidis, and Fillia Makedon.
”Quantitative analysis of the human upper-limb kinematic model for
robot-based rehabilitation applications.” IEEE Conference on Automation
Science and Engineering (CASE), Fort Worth TX. 2016.
[24] Craig, John J. Introduction to robotics: mechanics and control. Vol. 3.
Upper Saddle River: Pearson Prentice Hall, 2005.
[25] Gattupalli, S., Lioulemes, A., Gieser, S., N., Sassaman, P., Athitsos,
V., Makedon F., ”MAGNI: A Real-Time Robot-Aided Game-Based
Tele-Rehabilitation System”., Universal Access in Human-Computer
Interaction. 10th International Conference, UAHCI 2016, Held as Part
of HCI International 2016, Toronto, ON, Canada, July 17-22, 2016.
World Academy of Science, Engineering and Technology
International Journal of Biomedical and Biological Engineering
Vol:11, No:4, 2017
167International Scholarly and Scientific Research & Innovation 11(4) 2017 ISNI:0000000091950263
Open Science Index, Biomedical and Biological Engineering Vol:11, No:4, 2017 waset.org/Publication/10006973