Multi-disciplinary Design and In-Home Evaluation of Kinect-Based Exercise Coaching System for Elderly

Conference Paper · August 2015with 219 Reads
DOI: 10.1007/978-3-319-20913-5_10
Abstract
Physical activity is recognized as one of the most effective measures to reduce risk of injury and to improve the quality of life in elderly. Many of the elderly however lack the motivation, confidence and skills to engage in regular exercise activity. One of the promising approaches is semi-automated coaching that combines exercise monitoring and interaction with a health coach. To gain a better understanding of the needs and challenges faced by the elderly when using such systems, we developed Kinect-based interactive exercise system to encourage healthy behavior and increase motivation to exercise. We present the multi-disciplinary design process and evaluation of the developed system in a home environment where various real-world challenges had to be overcome.
adfa, p. 1, 2011.
© Springer-Verlag Berlin Heidelberg 2011
Multi-disciplinary Design and In-Home Evaluation of
Kinect-based Exercise Coaching System for Elderly
Gregorij Kurillo
1
, Ferda Ofli
1*
, Jennifer Marcoe
2
, Paul Gorman
2
, Holly Jimison
3
,
Misha Pavel
3
, Ruzena Bajcsy
1
1
University of California at Berkeley, Berkeley, CA
{gregorij, fofli, bajcsy}@eecs.berkeley.edu
2
Oregon Health and Science University, Portland, OR
{marcoej, gormanp}@ohsu.edu
3
Northeastern University, Boston, MA
{h.jimison, m.pavel}@neu.edu
Abstract. Physical activity is recognized as one of the most effective measures
to reduce risk of injury and to improve the quality of life in elderly. Many of the
elderly however lack the motivation, confidence and skills to engage in regular
exercise activity. One of the promising approaches is semi-automated coaching
that combines exercise monitoring and interaction with a health coach. To gain a
better understanding of the needs and challenges faced by the elderly when using
such systems, we developed Kinect-based interactive exercise system to encour-
age healthy behavior and increase motivation to exercise. We present the multi-
disciplinary design process and evaluation of the developed system in a home
environment where various real-world challenges had to be overcome.
Keywords: Gerontechnology; Interactive Exercise; Kinect; Health Coaching
1 Introduction
Growing ageing population in the United States is having significant implications on
the current healthcare system as the elderly face neurodegenerative conditions which
may reduce the level of independence and increase the risk of falls and injury. There
are currently almost 40 million persons aged 65 years or older living in the US, while
the number is expected to increase to 72.1 million by 2030 [1]. By improving the quality
of independent living through increased physical activity these challenges can be par-
tially mitigated [2]. Many elderly, however, lack access to exercise facilities, or the
skills and motivation to perform exercise at home.
To improve the health behavior and to overcome the lack of motivation in the general
population, various forms of computer-assisted coaching and “gamification” of activity
monitoring have been investigated; initially in the academic space and later on, with
the introduction of affordable motion sensing, also in the commercial space. The suc-
cess of interactive exercise products, as pointed out by Sinclair et al. [3], by and large
*
F. Ofli is now with Qatar Computing Research Institute (QCRI), Doha, Qatar.
depends on two interrelated dimensions: (1) effectiveness, which relates to achieving
exercise goals, and (2) attractiveness, which refers to the level of engagement for the
user to retain required duration and level of exercise. Many of the commercial products
have focused on the attractiveness aspect, while the academic field has tried to examine
the effectiveness of these technologies in exercise training and rehabilitation. Many of
the commercially available systems, however, are targeting different demographics,
such as younger users, and are as such less applicable for most older adults as they do
not offer appropriate type and level of exercise and fail to provide appropriate safety
considerations. Furthermore, the feedback provided by such systems may be overly-
engaging. In addition, the interaction modality may entail of complex user interfaces,
which may not be easy to use for elderly with reduced sensory and cognitive functions
[4]. Although several interactive systems for exercise in elderly have been presented in
research (e.g., [5, 6]), majority of the works focused on short-term and controlled in-
laboratory evaluations. A comprehensive review of the research literature on interactive
exercise in older adults can be found in [7, 8].
2 Background
The goal of this research was to develop an interactive exercise coaching system for
elderly that would be integrated with the semi-automated coaching framework at the
Oregon Center for Aging & Technology (ORCATECH) Living Lab
, which is focused
on exploring technologies to support independent living of elderly. The coaching plat-
form comprises of unobtrusive sensing of participant’s behaviors in combination with
artificial intelligence tools that aid the coach to send individualized messages to the
participants [9]. Originally, the participants were encouraged to exercise alongside
YouTube videos, however the system was not able to track individual’s exercise habits
or provide feedback on the performance that could be used to close the loop of the
health-coaching support. To achieve the interactive component for the health coaching,
we considered several different solutions, including wearable devices and 2D cameras.
After the release of Kinect for Xbox 360 (Microsoft, Redmond, WA) and accompany-
ing Kinect SDK, we decided to use the Kinect as it offered unobtrusive, low-cost and
relatively reliable way of measuring human motion kinematics. Although several com-
mercial applications for exercise have been developed to date, one of the challenges is
how this technology can be introduced in homes of elderly.
3 Methods
3.1 Design Process
The design of the Kinect-based exercise system architecture followed participatory de-
sign concepts by engaging the computer scientists and researchers with the health
coaches and caregivers during the interactive software development process over the
http://www.orcatech.org
last three years. We also took into account user feedback at several stages of the project.
The interactions among the members of the team included the following:
Identifying the requirements and objectives of the architecture;
Researching the needs and expectations of the targeted population;
Defining basic functionality of the exercise system;
Determining what data should be collected by the system;
Determining accuracy of the Kinect measurements;
Determining the conditions for home deployment;
Defining general user interaction flow with the system;
Selecting exercises appropriate for elderly users and the Kinect;
Recording exercise videos and defining movement features related to exercises;
Testing and modifying the prototype system at several stages;
Collecting and integrating user feedback;
Resolving various technical issues related to the deployment and maintenance;
Running in-home pilot studies with health-coaching support;
Discussing and evaluating various forms of data analysis.
We have approached the goals of this project in two stages. Our first prototype de-
ployment was primarily focused on understanding better the user needs and technical
challenges related to the exercise monitoring, user interfaces, and the use of Kinect
technology in homes of elderly users. We therefore installed the prototype exercise sys-
tem with 12 basic exercises into the homes of six independently-living elderly individ-
uals for an informal evaluation study. The system and results are described in details in
our prior publication [10]. The lessons learned from this study were then used to make
considerable improvements to the exercise system and evaluate it in an 18-week long
deployment in 7 homes of elderly users. In this paper, we thus focus on the second stage
of the design and evaluation. For completeness, we briefly describe some of the find-
ings from the first stage of the project while further details can be found in our referred
publications [10, 11].
3.2 Design Objectives
The primary goal of this research was to integrate an automated exercise coaching with
semi-automated health coaching of elderly in order to improve their fitness level in
terms of standard measures of fitness, such as flexibility, strength, balance, and endur-
ance [12]. Table 1 summarizes the design objectives for the development of the exercise
system and provides brief overview of identified issues from Phase 1 (described in [10])
and how they were addressed in Phase 2.
3.3 Implementation
In this section we describe the design, implementation, and setup of the Kinect-based
exercise system. For completeness we briefly refer to some of the findings and lessons
learned from Phase 1 while providing more details on the final version of the system
used in the last pilot study.
Table 1. Design Objectives for the Kinect-based Exercise Coaching System
Objective
Phase 1
[10]
Identified Issues Phase 2
Unobtrusive, low-mainte-
nance, and low-cost sensor
Microsoft Kinect 1 Space required for
the camera
Microsoft Kinect 1;
camera installed in
living room
Standalone, turn-key system,
minimum maintenance
All-in-one computer Large footprint;
complex interaction
Small footprint PC
connected to TV
Age-appropriate UI Basic UI Information clutter;
difficult to see text
on buttons
UI design based on
recommendations for
elderly users
Easy interaction with UI Wireless mouse &
keyboard
Difficult to use at
large distance
Use of wireless Pow-
erPoint remote
Ability to record interaction
with UI
None Need to understand
interaction issues
Timings of screens
interactions
Inclusion of age-appropriate
exercises
12 exercises Users desired more
exercises for variety
40+ exercises
Exercises grouping based on
fitness level
None Some find existing
exercises too easy
3 groups with up to 3
difficulty levels
Ability to record raw kine-
matic measurements
Yes None Yes
Real-time in-exercise feed-
back to encourage and correct
users’ performance
Video feedback, au-
dio & text cues
Users could not see
what Kinect was re-
cording
Video feedback, 3D
Kinect feedback, au-
dio & text cues
Summary of exercises to in-
form users of their overall
performance
Performance
measures
Difficult to under-
stand the meaning
Repetition counts,
summary statistics
Collection of subject-reported
data on health status
None Collected only dur-
ing phone contact
with the coach
Integration of pre-
and post-exercise
survey
Integration of the system with
the health-coaching
None Health coach did not
have access to data
Health coaching da-
tabase integration
Software. The exercise software was implemented in C++ with support from open
source 3D library Ogre (ogre3d.org) for graphics, MyGUI (mygui.info) for UI, Mi-
crosoft Kinect SDK for data acquisition, and MySQL for database management.
Kinect System. Our exercise system is based on Microsoft Kinect camera [13]
which was originally developed for the gaming console Xbox 360. The Kinect is a
depth-sensing system (combining RGB and infrared cameras) that provides 3D recon-
struction of the scene and segmentation of human blobs with real-time estimation of
the 3D location of 20 joints. The accuracy of the pose reconstruction depends on various
factors including orientation of the body, self-occlusions, interference with other ob-
jects, etc. During the planning stages, we examined the accuracy of the Kinect tracking
alongside a motion capture system to identify the exercises where the tracking was ro-
bust and to determine the accuracy of joint estimation [11]. One of the challenges of
using the Kinect camera was its limited field of view which requires users to be posi-
tioned between 1.8 m and 4 m. This can be particular challenging in smaller and clut-
tered homes. In addition, the pose estimation becomes less reliable when users are
seated or turned sideways. These limitations posed several constraints on the system
setup and exercise selection.
Movement Analysis. The real-time movement analysis during the exercise was per-
formed by first extracting measurement primitives from the skeletal data, such as joint
angles, relative angles to the vertical/horizontal plane, distances, absolute positions, etc.
These features were chosen manually based on the goals of specific exercise. The goals
were defined in consultation with the health coach. The selected measurement primi-
tives were then used to evaluate the performance of the exercise (e.g. how high person
can reach), to support repetition counting, and to trigger feedback alerts. More details
on the implementation can be found in [10].
Feedback and Visualization. During the development and testing phase, we exam-
ined different options on how to provide effective feedback during exercise. The pos-
sible feedback modalities included video, skeletal data, 3D graphics, 2D overlays, tex-
tual messages, and auditory feedback. During initial in-laboratory testing, we examined
three different options as shown in Fig. 1. Informal usability assessment with coaches
and several elderly users suggested a preference for full-screen video mode (Fig. 1a)
which was further refined as shown in Fig. 2 (left). After the first pilot study [10], the
feedback collected from six users indicated that although the participants liked the
video guidance they were confused about what the camera sees and they were not al-
ways able to relate their movement to the movement of the coach. Therefore, we de-
cided to include a mirrored human figure as captured by the Kinect depth sensor next
to the video of the coach as shown in Fig. 2 (right). This element provided more intui-
tive way for a subject to relate their movement to the exercise performance. Addition-
ally, we replaced all the videos with high definition recordings of the coach that were
integrated into the 3D environment for more attractive overall appearance.
The visual feedback in the exercise software also included several informational el-
ements that were displayed as 2D overlaid graphics and text. In the first version, we
included performance bars which indicated how well the user is performing a particular
exercise based on the measurement primitives. The users, however, found the perfor-
mance bars difficult to understand and map to their own movements. In the second
version, we instead decided to report the performance in terms of the number of accom-
plished repetitions. The current repetition count was indicated by a large numerical
counter and corresponding number of yellow stars on the bottom of the screen. When
the user first started the exercise, gray stars were shown while their number corre-
sponded to the number of repetitions of the previous session. This information was
Fig. 1. Different in-exercise feedback options that were considered in early design stages.
intended to encourage the user to try to reach or exceed previous performance. The
number of yellow stars increased as user performed more repetitions.
The feedback also included auditory and textual messages triggered by the perfor-
mance evaluation. For example, if the subject were to sit tall in a particular exercise,
the system would trigger an alert whenever the user started slouching. In the first ver-
sion of the system, the messages were shown under the performance bar measure (Fig.
2, left). In the second version, we tried to reduce the clutter on the screen and created a
separate messaging panel to display feedback messages (Fig. 2, right). In addition to
the corrective messages, the system also included several general encouraging mes-
sages (e.g., “Good job!”, “Keep up the good work!”) and exercise-specific messages
that would remind the users for correct performance (e.g. in Leg Lifts exercise: “Kick
one foot up, then the other.”). These messages were displayed randomly. The main
exercise screen also included a countdown clock with a graphical display and a numer-
ical counter. As opposed to the first version, where the exercise would finish after com-
pleting 10 repetitions, we limited the exercise duration to 45 seconds as recommended
by the health coach.
User Interface (UI) and Navigation. Since the initial pilot study was primarily fo-
cused on testing the feasibility of collecting exercise data at home, the user interface
was relatively simple. Although we considered several different modalities to control
the software (e.g., speech, gestures, presenter remote), we decided to use a wireless
mouse and keyboard since the participants were familiar with these devices. In general
the users found the software to be easy to use, however some participants reported that
they were not able to read the text on the screen from the distance and had difficulty
controlling the application in Windows environment.
Our focus in the second phase was to improve the user experience, especially since
the system was intended to be used over a much longer time period. As recommended
by guidelines for design of software for elderly users [4], we implemented the following
improvements to the original interface:
Large fonts for text messages and button labels;
Familiar icons on buttons (e.g., video controls used icons similar to VCR);
Consistent positioning of buttons with similar functions;
Simple graphical elements;
Fig. 2. Comparison of the in-exercise feedback screen between the initial [10] and the final
version of the exercise software.
Color scheme with good contrast;
Improved text-to speech (offered by the new Windows 8 platform);
Minimal textual information on each screen;
Overall reduction of screen clutter; information organized into display panels;
Linear screen interaction flow;
Fig. 3 shows several example screenshots from the updated software with the fol-
lowing interaction flow. From the main screen (Fig. 3a), the user is able to view help
and safety videos, start a new session or complete unfinished session. Next, the user is
presented with a survey of five questions about their general health and goals for the
day (Fig. 3b). The user is able to skip a specific question if they prefer not to answer it.
On the exercise selection screen users can select between three different exercise groups
with various difficulty levels (Fig. 3c). Once the exercise group and level are selected,
the user is prompted to perform optional warm-up which includes only the video play-
back of the coach without any feedback on the performance. Next, depending on se-
lected preference, the full instructional video on benefits of the exercise or a short 10-
second preview is displayed (Fig. 3d). Afterwards, the user performs the exercise for
45 seconds with the real-time feedback and accompanying video (Fig. 3e). At any time,
the user can review instructions, skip to the next exercise, or exit the session. Once the
exercise is completed, a bar chart showing current and past repetition counts is dis-
played. After completing all the exercises, the session summary screen is shown, dis-
playing the summary statistics of the particular exercise session compared to the past
performance (Fig. 3f). Demo video can be viewed at: http://tinyurl.com/KinectExercise
To simplify the navigation, we implemented support for a 3-button wireless Power-
Point remote with large buttons (Kensington, K72441AM). Two buttons on the remote
were used to change the selection on the screen (which was highlighted) back and forth
while the third button was used to confirm the current selection.
Exercises. The original system included 12 exercises (e.g., Heel Drags, Lateral
Stepping, Leg Extensions, Cops and Robbers, Buddha’s Prayer, etc. [10]) focused on
improving balance, flexibility, and strength. Based on the feedback collected from the
first pilot study, we included several variations of these exercises to provide variety for
users of different capabilities. The final system included about 40 exercises which were
Fig. 3. Software interaction flow: (a) home screen, (b) daily survey, (c) exercise selection,
(d) exercise preview/instructions, (e) in-exercise feedback, (f) session summary.
grouped into three groups (i.e., full body, upper body with core & lower body) and
arranged into three difficulty levels, each containing between 6 to 15 exercises.
System Setup. In the first pilot study, we used all-in-one computer with large screen
which provided easy setup and portability. We found, however, that in some homes it
was difficult to find sufficient space; therefore, the physical setup often had to be im-
provised to achieve the required distance for the Kinect. For easy interaction, we con-
figured the system as a ‘turn-key’ system. A small desktop PC running Windows 8.1
was connected to participant’s existing TV set using HDMI connection to transmit
video and audio signals from the computer. To simplify the process of switching be-
tween the regular channels and PC input, we installed an HDMI switch. The PC was
configured to be always on and to boot into the desktop without requiring users to log
in. Remote connection to the PC was enabled via TeamViewer software
(teamviewer.com) for administration of the system. For protection of privacy all the
data saved on the PC were stored in temporary MySQL tables which were copied
nightly to the external server. This process however turned out to cause occasional data
loss as some of the homes experienced outages of network and power due to weather
and construction.
4 Results
In this section we present results from 18-week study in the homes of 7 elderly individ-
uals ranging from 77 to 96 years of age (mean age: 83.2). Baseline data on physical
fitness (e.g., Berg Balance Test, Senior Fitness Test, etc.), general health, and physical
activity were collected prior to the deployment of the Kinect system and subsequently
every four weeks. As part of the Living Lab enrollment other quantitative data were
collected during the study, such as sleep data, in-home motion sensors, cognitive
games, etc. Subjects were also in contact weekly with a health coach who provided
guidance on the exercise regimen and collected feedback on the system usage and any
technical issues. The study protocols were approved by the Oregon Health and Science
University IRB.
Fig. 4. Daily exercise adherence during the eighteen-week study. The color of the patches de-
notes the type and level of exercise (UB – Upper Body, LB – Lower Body, FB –Full Body).
W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 W17 W18
Subject 1
Subject 2
Subject 3
Subject 4
Subject 5
Subject 6
Subject 7
Weeks
Exercises Group
No Session
UB - Level 1
UB - Level 2
UB - Level 3
LB - Level 1
LB - Level 2
LB - Level 3
FB - Level 1
FB - Level 2
FB - Level 3
Fig. 5. Comparison of post-exercise survey replies for subjects #3 and #5. The charts show
mean (*), standard deviation (Δ), and minimal/maximal response values per week.
Fig. 4 shows the exercise program adherence over the course of 18 weeks. The sub-
jects were instructed to exercise 3-5 times a week. From the 7 subjects who were en-
rolled, four subjects performed the exercises somewhat regularly. Subject #1 got ill
early on and never returned to the exercise. Subject #2 exercised regularly, however
due to the internet connectivity issues, we were unable to recover the data of the second
portion of the study. Subjects #3, #4, and #5 completed most of the exercise sessions.
Subject #6 performed exercises intermittently but later on stopped using the system due
to holidays and travel. Subject #7 initially used the system but found it was not as useful
to him as he was already involved regularly in Tai Chi and riding exercise bike. Fig. 4
also shows the type of exercise sessions the subjects performed. Most of the subjects
performed full body exercise sessions. Subjects #2 and #4 were both able to increase
the exercise level after a few weeks. Subject #3 was on the other hand alternating be-
tween the different exercise groups, which was also the general recommendation.
Fig. 5 shows the results of the post-exercise survey compared between subjects #3
and #5 who had the most completed sessions. The subjects’ response data reflect their
exercise habits. Subject #3 reported to be pain free most of the time and very motivated
to exercise. On the other hand, subject #5 reported pain and low motivation, in partic-
ular in weeks 9 and 10 after which the subject took a break from the exercise. Daily
survey responses could be in general used by the health coach or an automated system
to provide appropriate intervention to increase the motivation of the user.
Fig. 6 shows the raw Kinect skeleton output for six sample frames captured during
the exercise Shallow Squats. The skeleton configurations are shown for every 30 frames
corresponding to the time interval of 1 second. As mentioned previously, the skeletal
data were used to extract the measurement primitives for repetition counting and feed-
back. Fig. 7 shows the number of completed repetitions for the same exercise over the
course of the study for subjects, #4 and #5, who had performed this exercise in majority
of their sessions. Note that this exercise was included only in the Full Body - Level 1
and Lower Body – Level 2 exercise groups. For both subjects we can see the trend of
an overall increase in the number of repetitions over time which is likely due to im-
proved endurance.
5 Discussion and Conclusion
Based on the findings from our Phase 1 study we have successfully improved the exer-
cise system to achieve the objectives for long-term use that were summarized in Table
1. Majority of the changes in the design, additional exercises, and overall system per-
formance were well-accepted by the participants. There were several minor technical
issues that were identified and corrected during the first two weeks after the installation.
These included changes to the scripts that suppressed various system pop-ups and al-
ways put the exercise software in the foreground of the desktop. We also noticed that
some users were either double clicking or holding the button on the remote for a longer
time period which sometimes resulted in multiple confirmations. These navigation is-
sues were resolved by a subsequent software update. Overall, the wireless remote was
easy to use for the participants after the initial issues were resolved. Only one user
experienced a failure of the remote during the course of the study.
Since the users only had the wireless remote to control the system, we were not able
to use the login mechanisms that would allow for the encryption of the hard drive in
order to protect the privacy of the data in case the computer was stolen. Instead, all the
data were stored in memory tables and subsequently copied to the remote server each
night. This arrangement however created another technical challenge because some of
the homes experienced internet and power outages that were not anticipated to happen
Fig. 6. Raw skeleton sequence recorded for the exercise Shallow Squats (Subject #3)
Fig. 7. Number of completed repetitions for the exercise Shallow Squats for subjects #4
and #5.
-0.2
0
0.2
#90
#120
#150
#180
#210
0
0.5
1
x(m)
Frame
z(m)
W1 W2 W3 W4 W5 W6 W7 W8 W9 W10 W11 W12 W13 W14 W15 W16 W17 W18
0
5
10
15
20
Weeks
Repetition Count
Subject #4
Subject #5
at such frequency. Although the implementation worked for most users, several ses-
sions from one of the users were lost in the process. For the future studies, we plan to
investigate other mechanisms to ensure data privacy and security while providing a
robust data collection regardless of the internet connectivity.
From the users’ perspective, the biggest challenge was switching the TV setting from
their regular cable channel to the PC input. Initially the users were instructed how to do
that via their TV remote; however, some were not able to remember the steps. To re-
solve this issue, we installed for some users a physical HDMI switch that would allow
them to more easily switch the inputs at their convenience.
On the software side, one of the common issues reported by the participants was the
lack of or incorrect repetition counting in some of the more complex exercises. As re-
ported previously [10], the real-time analysis is sensitive to various factors, which in-
clude camera position, inclusion of other objects in the scene, orientation of the user
with respect to the camera, type of chair, etc. We are currently working on more robust
methods to perform the analysis and repetition counting while using the collected da-
taset for benchmarking.
The feedback collected from the interviews with the participants revealed that lack
of exercising was primarily due to reasons unrelated to the system itself, such as illness,
scheduling, low motivation, etc. The subjects did express hope that any technical issues
would be resolved in the future, such as more reliable exercise recognition and issues
with the TV setup. Overall, the subjects who did use the system on regular basis pro-
vided mostly positive impressions, such as:
“I was excited to exercise, but should stick with every other day, I did it 2 days in
a row and was sore.”
“I don't exercise that much, but try to complete the video 4x/week.”
“I exercise right before bed, I have seen that it helps me sleep better.”
“Coach very encouraging and that makes me want to do it.”
"Good program. Instructions well done; I like the bar chart, makes me feel better
to exercise and helps me see what I need to work on… Feedback could be better,
feels canned. Delays and technical issues would be great if not there.”
Due to the limited space, we have shown only a small subset of the results collected
during the time the participants used the exercise system. Future analyses will include
comparison of the exercise performance with the clinical measures that were collected
before and during the study. Since the dataset also includes raw skeletal data, we are
planning to further investigate how to quantify the exercise performance in terms of
standard fitness measures, such as flexibility, balance, strength, and endurance [12].
Furthermore, we will analyze the strategies that the participants used to exercise by
comparing their data to the data of the coach in the video. Such temporal analysis could
quantify how closely the participants were following the movement of the coach or if
they have developed their own strategy for each exercise. The results of the analysis
will be important for implementing a more effective feedback in the future. Further-
more, we will investigate how the exercise system could be used in a closed-loop semi-
automated coaching.
Acknowledgements
The authors would like to thank Štěn Obdržálek, Alex Triana, and Kavan Sikand for
contributions to software development; Sue Scott of Renewable Fitness for providing
the exercises and her assistance in designing the exercise program; and Edmund Seto
for contributions to the study design. This research was supported by the National Sci-
ence Foundation (NSF) under Grant No. 1111965.
References
1. Department of Health & Human Services, "Administration on Aging," URL:
http://www.aoa.gov. Accessed: 1 February 2015.
2. F. Sun, I. Norman and A. While, "Physical activity in older people: A systematic review,"
BMC Public Health, vol. 13, no. 1, pp. 17, 2013.
3. J. Sinclair, P. Hingston and M. Masek, "Considerations for the Design of Exergames," in Proc.
of the 5th Inter. Conf. on Computer Graphics and Interactive Techniques in Australia and
Southeast Asia (GRAPHITE '07), Perth, Australia, 2007.
4. J. Doyle, D. Kelly and B. Caulfield, "Design considerations in Therapeutic Exergaming," in
Proc. of 5th Inter. Conf. on Pervasive Computing Technologies for Healthcare (Perva-
siveHealth) and Workshops, 2011.
5. S. Ganesan and L. Anthony, "Using the Kinect to encourage older adults to exercise: A pro-
totype," in CHI’12 Extended Abstracts on Human Factors in Computing Systems, 2012.
6. Y. Pisan, J. G. Marin and K. F. Navarro, "Improving lives: Using Microsoft Kinect to predict
the loss of balance for elderly users under cognitive load," in Proc. of the 9th Australasian
Conf. on Interactive Entertainment: Matters of Life and Death, 2013.
7. L. H. Larsen, L. Schou, H. H. Lund and H. Langberg, "The physical effect of exergames in
healthy elderly – A systematic review," Games for Health Journal, vol. 2, no. 4, pp. 205-212,
2013.
8. K. J. Miller, B. S. Adair, A. J. Pearce, C. M. Said, E. Ozanne and M. M. Morris, "Effectiveness
and feasibility of virtual reality and gaming system use at home by older adults for enabling
physical activity to improve health-related domains: A systematic review," Age and Ageing,
vol. 43, no. 2, pp. 188-195, 2013.
9. H. B. Jimison and M. Pavel, "Integrating Computer-Based Health Coaching Into Elder Home
Care," in Assistive Technology Research Series, Volume 21: Technology and Aging, IOS
Press, 2008, pp. 122-129.
10. F. Ofli, G. Kurillo, S. Obdrzalek, R. Bajcsy, H. Jimison and M.Pavel, "Design and Evaluation
of an Interactive Exercise Coaching System for Older Adults: Lessons Learned," IEEE Jour-
nal of Biomedical and Health Informatics. Accepted, 2015.
11. Š. Obdržálek, G. Kurillo, F. Ofli, R. Bajcsy, E. Seto, H. Jimison and M. Pavel, "Accuracy and
Robustness of Kinect Pose Estimation in the Context of Coaching of Elderly Population," in
Proc. of 34th Inter. Conf. of the IEEE Engineering in Medicine and Biology Society
(EMBEC), San Diego, CA, 2012.
12. C. J. Caspersen, K. E. Powell and G. M. Christenson, "Physical activity, exercise and physical
fitness: definitions and distinctions for health-related research," Public Health Reports, vol.
100, no. 2, p. 126–131, 1985.
13. Z. Zhang, "Microsoft Kinect sensor and its effect," IEEE MultiMedia, vol. 19, no. 2, pp. 4-
10, 2012.
  • ... In particular, [4,5,6,7,8] described an embodied conversational agent, where an animated computer character simulates face-to-face conversations using a synthesised voice which is synchronized with other non-verbal behaviours, [9] a conversational agent integrated in the smartphone, and [10,11,12] a robot. Screen-based interventions can be found as well: a touch-based digital photo [13,14], a PC-based virtual board game [15], a display that were tackled by 3D camera [16,17].Other traditional intervention media found in the reviewed papers were emails [15], phone calls [15,18] and printed material such as booklets [16] and manuals [15]. It is worth noting that despite of the diffusion of chatbots in the past years, conversational agents were always embodied with virtual avatars or robots in previous work. ...
    ... In particular, [4,5,6,7,8] described an embodied conversational agent, where an animated computer character simulates face-to-face conversations using a synthesised voice which is synchronized with other non-verbal behaviours, [9] a conversational agent integrated in the smartphone, and [10,11,12] a robot. Screen-based interventions can be found as well: a touch-based digital photo [13,14], a PC-based virtual board game [15], a display that were tackled by 3D camera [16,17].Other traditional intervention media found in the reviewed papers were emails [15], phone calls [15,18] and printed material such as booklets [16] and manuals [15]. It is worth noting that despite of the diffusion of chatbots in the past years, conversational agents were always embodied with virtual avatars or robots in previous work. ...
    Conference Paper
    The ability to engage the user in a conversation and the credibility of the system are two fundamental characteristics of virtual coaches. In this paper, we present the architecture of a conversational e-coach for promoting healthy lifestyles in older age, developed in the context of the NESTORE H2020 EU project. The proposed system allows multiple access points to a conversational agent via different interaction modalities: a tangible companion that embodies the virtual coach will leverage voice and other non-verbal cues in the domestic environment, while a mobile app will integrate a text-based chat for a ubiquitous intervention. In both cases, the conversational agent will deliver personalized interventions based on behavior change models and will promote trust by means of emotionally rich conversations.
  • ... Many automated health interventions have been introduced in recent years. Some focus on guiding patients through physical therapy and exercise [69]. Others use web interfaces to strengthen cognitive performance and promote social interaction [70]. ...
    Article
    Full-text available
    Smart cities use information and communication technologies (ICTs) to scale services include utilities and transportation to a growing population. In this paper, we discuss how smart city ICTs can also improve healthcare effectiveness and lower healthcare cost for smart city residents. We survey current literature and introduce original research to offer an overview of how smart city infrastructure supports strategic healthcare using both mobile and ambient sensors combined with machine learning. Finally, we consider challenges that will be faced as healthcare providers make use of these opportunities.
  • Article
    Most health care takes place in the home, especially for people with chronic diseases and conditions of aging. Often the most effective components of the management of these conditions have to do with improving health behaviors, such as exercise, diet and adherence to agreed-upon treatments. With escalating health care costs, it is important to focus on technologies that facilitate home-based care with less expensive personnel, trained to provide coaching for improving health behaviors. The focus of this project is to develop a general-purpose architecture for providing computer-based health interventions for elders in their homes, facilitated by a professional health coach. This architecture incorporates knowledge representations to enable use of known principles of health behavior change. For example, we include modules to assess health behavior goals, motivations, barriers and readiness to change. In addition, based on our needs assessment of stakeholders (elders, family, caregivers, clinicians, service providers, researchers, government, and industry), we have focused on a design to facilitate the participation of family members and low-skilled caregivers as part of the care team. Sophisticated user models and authentication methods are required to determine the appropriate data access and user interface to the system for each user. We use current interoperability standards to anticipate and prepare for linkage with available electronic medical records and personal health records. The primary benefit of this approach is that multiple health behavior interventions can be made available using a common architecture and format that encourages a principled approach to the delivery of care to the home.
  • Article
    Although the positive effects of exercise on the well-being and quality of independent living for older adults are well-accepted, many elderly individuals lack access to exercise facilities, or the skills and motivation to perform exercise at home. To provide a more engaging environment that promotes physical activity, various fitness applications have been proposed. Many of the available products, however, are geared toward a younger population and are not appropriate or engaging for an older population. To address these issues, we developed an automated interactive exercise coaching system using the Microsoft Kinect. The coaching system guides users through a series of video exercises, tracks and measures their movements, provides real-time feedback, and records their performance over time. Our system consists of exercises to improve balance, flexibility, strength, and endurance, with the aim of reducing fall risk and improving performance of daily activities. In this paper, we report on the development of the exercise system, discuss the results of our recent field pilot study with six independently-living elderly individuals, and highlight the lessons learned relating to the in-home system setup, user tracking, feedback, and exercise performance evaluation.
  • Conference Paper
    Full-text available
    Among older adults, falling down while doing everyday tasks is the leading cause for injuries, disabilities and can even result in death. Furthermore, even when no injury has occurred the fear of falling can result in loss of confidence and independence. The two major factors in the loss of balance is weakening of the muscles and reduced cognitive skills. While exercise programmes can reduce the risk of falling by 40%, patient compliance with these programmes is low. We present the Microsoft-Kinect based step training program system that we have developed specifically for elderly patients. The program measures physical health and cognitive abilities and incorporates an individualized adaptive program for improvements. The real-time data obtained from the program is similar to clinical evaluations typically conducted by doctors and the game-like exercises result in increased adherence to the exercise regimes
  • Article
    Full-text available
    Exergames have been suggested as an innovative approach to enhance physical activity in the elderly. The objective of this review was to determine the effectiveness of exergames on validated quantitative physical outcomes in healthy elderly individuals. We used Centre for Review and Disseminations guidance to conduct systematic reviews. Four electronic databases were searched. We included randomized controlled trials (RCTs), the study participants were healthy elderly individuals, and the intervention of interest was exergaming. The title and abstract screening of the 1861 citations identified 36 studies as potentially eligible for this review, and an additional nine were identified from reference lists. The full text screening identified seven studies with a total of 311 participants, all reporting RCTs with low-to-moderate methodological quality. Six of the seven studies found a positive effect of exergaming on the health of the elderly. However, the variation of intervention approaches and outcome data collected limited the extent to which studies could be compared. This review demonstrates how exergames have a potential to improve physical health in the elderly. However, there is a need for additional and better-designed studies that assess the effectiveness and long-term adherence of exergames designed specifically for the elderly.
  • Article
    Full-text available
    use of virtual reality and commercial gaming systems (VR/gaming) at home by older adults is receiving attention as a means of enabling physical activity. to summarise evidence for the effectiveness and feasibility of VR/gaming system utilisation by older adults at home for enabling physical activity to improve impairments, activity limitations or participation. a systematic review searching 12 electronic databases from 1 January 2000-10 July 2012 using key search terms. Two independent reviewers screened yield articles using pre-determined selection criteria, extracted data using customised forms and applied the Cochrane Collaboration Risk of Bias Tool and the Downs and Black Checklist to rate study quality. fourteen studies investigating the effects of VR/gaming system use by healthy older adults and people with neurological conditions on activity limitations, body functions and physical impairments and cognitive and emotional well-being met the selection criteria. Study quality ratings were low and, therefore, evidence was not strong enough to conclude that interventions were effective. Feasibility was inconsistently reported in studies. Where feasibility was discussed, strong retention (≥70%) and adherence (≥64%) was reported. Initial assistance to use the technologies, and the need for monitoring exertion, aggravation of musculoskeletal symptoms and falls risk were reported. existing evidence to support the feasibility and effectiveness VR/gaming systems use by older adults at home to enable physical activity to address impairments, activity limitations and participation is weak with a high risk of bias. The findings of this review may inform future, more rigorous research.
  • Article
    Full-text available
    Recent advances in 3D depth cameras such as Microsoft Kinect sensors (www.xbox.com/en-US/kinect) have created many opportunities for multimedia computing. The Kinect sensor lets the computer directly sense the third dimension (depth) of the players and the environment. It also understands when users talk, knows who they are when they walk up to it, and can interpret their movements and translate them into a format that developers can use to build new experiences. While the Kinect sensor incorporates several advanced sensing hardware, this article focuses on the vision aspect of the Kinect sensor and its impact beyond the gaming industry.
  • Article
    Full-text available
    Background Physical activity (PA) in older people is critically important in the prevention of disease, maintenance of independence and improvement of quality of life. Little is known about the physical activity of the older adults or their compliance with current physical activity guidelines. Methods A systematic literature search of the published literature was conducted. Included were published reports of original research that independently reported: the PA level of non-institutional older adults (aged 60 years and over); and the proportion of older adults in the different samples who met PA recommendations or guidelines. The review was restricted to studies published since 2000 to provide a current picture of older adults’ PA levels. Results Fifty three papers were included in the review. The percentage of older adults meeting recommended physical activity ranged from 2.4 – 83.0% across the studies. Definitions of “recommended” physical activity in older adults varied across the studies as did approaches to measurement which posed methodological challenges to data analysis. Older age groups were less likely than the reference group to be regularly active, and women were less likely than men to achieve regular physical activity, especially leisure time physical activity, when measured by both subjective and objective criteria. Conclusion The review highlights the need for studies which recruit representative random samples of community based older people and employ validated measurement methods consistently to enable comparison of PA levels over time and between countries.
  • Conference Paper
    This paper reports current progress on a project that aims to find the factors that play an important role in motivating older adults to maintain a physical exercise routine, a habit recommended by doctors but difficult to sustain. Our initial data gathering includes an interview with an expert in aging and physical therapy, and a focus group with older adults on the topics of exercise and technology. Based on these data, an early prototype game has been implemented for the Microsoft Kinect that aims to help encourage older adults to exercise. The Kinect application has been tested for basic usability and found to be promising. Next steps include play-tests with older adults, iterative development of the game to add motivational features, and evaluation of the game's success in encouraging older adults to maintain an exercise regimen.
  • Article
    Full-text available
    The Microsoft Kinect camera is becoming increasingly popular in many areas aside from entertainment, including human activity monitoring and rehabilitation. Many people, however, fail to consider the reliability and accuracy of the Kinect human pose estimation when they depend on it as a measuring system. In this paper we compare the Kinect pose estimation (skeletonization) with more established techniques for pose estimation from motion capture data, examining the accuracy of joint localization and robustness of pose estimation with respect to the orientation and occlusions. We have evaluated six physical exercises aimed at coaching of elderly population. Experimental results present pose estimation accuracy rates and corresponding error bounds for the Kinect system.
  • Article
    Full-text available
    Background The effectiveness of paraprofessional home-visitations on improving the circumstances of disadvantaged families is unclear. The purpose of this paper is to systematically review the effectiveness of paraprofessional home-visiting programs on developmental and health outcomes of young children from disadvantaged families. Methods A comprehensive search of electronic databases (e.g., CINAHL PLUS, Cochrane, EMBASE, MEDLINE) from 1990 through May 2012 was supplemented by reference lists to search for relevant studies. Through the use of reliable tools, studies were assessed in duplicate. English language studies of paraprofessional home-visiting programs assessing specific outcomes for children (0-6 years) from disadvantaged families were eligible for inclusion in the review. Data extraction included the characteristics of the participants, intervention, outcomes and quality of the studies. Results Studies that scored 13 or greater out of a total of 15 on the validity tool (n = 21) are the focus of this review. All studies are randomized controlled trials and most were conducted in the United States. Significant improvements to the development and health of young children as a result of a home-visiting program are noted for particular groups. These include: (a) prevention of child abuse in some cases, particularly when the intervention is initiated prenatally; (b) developmental benefits in relation to cognition and problem behaviours, and less consistently with language skills; and (c) reduced incidence of low birth weights and health problems in older children, and increased incidence of appropriate weight gain in early childhood. However, overall home-visiting programs are limited in improving the lives of socially high-risk children who live in disadvantaged families. Conclusions Home visitation by paraprofessionals is an intervention that holds promise for socially high-risk families with young children. Initiating the intervention prenatally and increasing the number of visits improves development and health outcomes for particular groups of children. Future studies should consider what dose of the intervention is most beneficial and address retention issues.