ArticlePDF Available

Novel Approach: Interactive Markerless Motion Capture Use for Integrated Data Reporting in Health Care and Remote Patient Monitoring Pilot Study



Content may be subject to copyright.
Novel Approach: Interactive Markerless
Motion Capture Use for Integrated Data
Reporting in Health Care and Remote
Patient Monitoring
Pilot Study
Imen Maaroufi Clark, MS, Bryan T. Arkwright, MHA, Thomas B. Foster IV DMA, MS, Rachael Schmid, MA - Editor
Pilot Study
Novel Approach: Interactive Markerless
Motion Capture Use for Integrated Data
Reporting in Health Care and Remote
Patient Monitoring
Starting from the late 20th century, the study of Human motion sensing and recognition has seen
an increased focus on new technologies, such as motion capture systems and computer vision-
based tracking. Marker-based motion tracking has found favor in the health care industry for its
accuracy and ability to have a repeatable outcome.[1] This method is high cost, requires
implementation in a laboratory environment with high data systems, and relies primarily on
algorithms (rather than on in-person providers) to analyze a patient’s movement. New
technologies, including improved facial recognition and body movement detection using a 2D
camera, have increased the viability and appeal of markerless motion capture methods.
In this paper, we present the Helping FriendsTM 1.0 toolset by Point Motion Inc., which enables
subject matter experts to integrate their targeted programs and protocols for specific patient
populations into a 2D motion capture tool that is markerless, capturing the individuals’
movements and tracking their performance over time. Through its auditory feedback system,
this technology aims to address further gaps in the current motion capture technology for health
care market, including a lack of patient-centered experiences, as well as a lack of reward-based
experiences upon completing assigned programs/exercises. The study of this technology is
valuable insofar as it provides evidence that the collected data metrics can provide caregivers
with information that better supports their clinical decisions and allows them to more effectively
monitor their clients remotely.
The ultimate vision of Helping FriendsTM 1.0 is to allow end users/patients to engage in enriching
activities through accessible technology, while eliminating the need to implement an external
device in order to provide results, thus empowering a greater number of patients and care
Pilot Study
Main body:
The purpose of this study was to test the Helping FriendsTM 1.0 toolset for its success in the
following areas: as a software that motivates patients for their physical exercise, through
auditory feedback; in providing reproducible outcome; in generating instant reports that track
performances over time.
We tested these capabilities through a pilot study at St. Mary’s Healthcare System for Children in
New York with the support of several individuals, including technology administrators, music
therapists, and physical therapists. We worked with this team to identify use cases for the study.
10 patients represented the final selection of use cases: 5 males and 5 females. The group age
varied between young children and young adults, with a mean age of 15 years old (Standard
Deviation= 8.97, ~9). The group members had differing cognitive and physical challenges and
represented four categories of patient populations—inpatients, day program attendees,
locomotor training (LT) patients, and music therapy group attendeesallowing for the
assessment of different patient populations' needs. The variation of targeted goals within those
populations allowed for further diversification of the group.
Total duration of the pilot study was 3 months, from September 16th, 2019 until December 16th,
2019. The Helping FriendsTM 1.0 motion capture technology consists of capturing 17 keypoints in
the human body: 5 points in the head and 12 others in the upper and lower body extremities.
The end user/patient is asked to match represented poses that activate and deactivate sounds
upon completion. The poses are designated by the subject matter expert and run by the patient
with the specialist in a facility, or with a guardian remotely.
We registered 870 minutes of total programs run, with 15 average sessions per month. Total
sessions run was 29 sessions, with an average of 22 minutes per session. In all sessions, we were
able to demonstrate the use of this technology run through common digital devices such as
tablets and computers using the built-in camera, and no additional device.
Limitations of Helping FriendsTM 1.0 are mainly capability for tracking only one person per screen
and tendency of cameras to pick a bulky object in a poor environment (i.e. poor lighting, busy
backgrounds, etc.). Improvements are possible with integration of additional accuracy and model
This pilot resulted in 3 main groups, based on the amount of time spent on gameplay. The first
group represents 5 participants, who spent 15 minutes or more on the session. The second
group represents 4 participants, who spent less than 15 minutes in the gameplay. The third
group was composed of a single participant who registered less than 5 minutes in gameplay.
With a registered 70% retention rate in this pilot, we were able to confirm significant
motivational experiences via Helping FriendsTM 1.0, as well as significant opportunity for use of
the toolset in remote care (presenting revenue generation opportunities for care centers).
Pilot Study
The software program/activity Helping FriendsTM 1.0 showed ability to collect data automatically,
allowing for generated reports showing performance over time and scores on specific tasks
related to regular or rehabilitative exercise. Perhaps most valuably: This study showcased the
ability of the Helping Friends™ 1.0 toolset to create a variety of engaging experiences for
different patient populations and to motivate them while acting on their general wellness.
The Quadruple Aim of health care (improved patient experience; better outcomes and improving
populations’ health; lower costs of care; and improved clinical experience) was used as the
ultimate measure of discussed findings. The abilities of Helping Friends™ 1.0 demonstrated in
this study contributed to those aims, supporting the claim that “allowing end users/patients to
engage in enriching
activities through accessible technology, while eliminating the need to
implement an external device in order to provide results, will empower a greater number of
patients and care providers.”
Key points:
The solution provides reproducible outcomes when used according to our guidelines for
the optimal tracking setting (lighting, background, etc.) (table 3 on Collected Data Types)
The presented solution motivates patients for their physical exercise through auditory
The solution generates instant reports and tracks performances over time per patient.
Markerless motion capture, motion tracking in health care, motion in therapy, auditory
feedback, music interaction, remote patient monitoring, telehealth
Starting from the late 20th century, the study of Human motion sensing and recognition has seen
increased focus on new technologies, such as motion capture systems and motion analysis
processes, across numerous industries.[1] Motion tracking for individuals in rehabilitation has
been a topic of research and discussion among professionals since the 1980s[2], leading to the
implementation of the marker-based motion tracking technologies seen most often in health
care settings today. Marker-based motion tracking has found favor in the health care industry for
its accuracy and ability to have a repeatable outcome.[1] However, new technologies, including
improved facial recognition and body movement detection using a 2D camera, have increased
the viability and appeal of markerless motion capture methods.
The goal of motion capture is to record and analyze human motion, and its power as pertains to
health care lies in the potential for widespread application. While marker-based motion tracking
technology has been the preferred solution for select patient groups—such as gate tracking and
joint rehabilitation in the treatment of athletes and artistic performers—there remains the issue
of accessibility. Marker-based motion tracking is high cost, requires implementation in a
Pilot Study
laboratory environment with high data systems, and relies primarily on algorithms (rather than
on the patient-centered care of in-person providers) to analyze a patient’s movements. Each of
these factors narrows accessibility for a wide range of individuals who could uniquely benefit
from motion capture technology.
The use of markerless technology in health care is now feasible and fits the needs of multiple
patient populations. This study tests the use of markerless 2D technology in health care settings,
a technology that meanwhile addresses further gaps in the current market: a lack of patient-
centered experiences using motion capture technology, as well as a lack of reward-based
experiences upon completing assigned programs/exercises. Our ultimate vision is that allowing
the end users/patients to engage in enriching
activities through accessible technology, while
eliminating the need to implement an external device in order to provide results, will empower a
greater number of patients and care providers.
This study provides a discussion of several use-cases of the presented Point Motion Inc.
technology, Helping Friends™ 1.0. Two-dimensional motion tracking uses simple body models
[3]; it assesses movements in the arms, neck mobility, trunk control, lower body movement, and
the different degrees of those movements. The technology allows for six degrees of freedom
(6DoF) while the patient moves in a 3D space. Point Motion Inc.’s technology uses the built-in
camera (such as in computers and tablets) for movement tracking. Motions are captured and
automatically entered into an electronic data record platform, providing the medical core with
information about patients’ performance over time. (For further analysis, like investigating risks
associated with different sports techniques, more complex 3D models with extensive laboratory
set-ups are required.) Through the toolset, specialists have the capacity to queue assigned
programs remotely, setting a mastery level and target for each patient.
From the patient perspective: A series of movements are displayed, and the patient attempts to
match the movements. Completion of programs provides data on a provided portal that
registers success or failure to match poses, i.e. right arm up, leg extension for balance, 90-degree
elbow bend, etc. Once the patient hits their goal, they move on to the next assigned task in the
program queue.
With concern for the current technology gaps in patient-centered and reward-based experiences
[3, 4], the motion capture tool of Point Motion Inc. allows for an auditory feedback system,
where movements trigger musical phrases, narration, and songs. The platform allows the loading
of programs that has defined outcomes; subject matters experts publish a ‘rehabilitation and
yoga flow program’, ‘autism and connecting in rhythms program, Parkinson and balance
measurement program, etc., where every program has specific sounds and movements libraries
and assigned targets to be mastered by the patient. The musical aspect prompts the patient and
guides them to the completion of tasks, using a cause-effect mechanism.
The presented tool is meant to capture relevant data that provides clinical insightnot treat,
cure, or diagnose a condition. It is sufficient to say that this study of the Helping Friends™ 1.0
toolset is valuable insofar as it provides evidence that the collected data metrics can provide
Pilot Study
caregivers with information that better supports their clinical decisionsand allows them to
more effectively monitor their clients remotely.
Sample Description:
The purpose of this study is to test the Helping FriendsTM 1.0 toolset for success in the following
1. as a software that motivates users/patients for their physical activity, through auditory
2. in providing reproducible outcomes
3. in generating instant reports that track performances over time.
During our pilot program, we were supported by a team that helped run the pilot post-
onboarding and training. The team was composed of the members in Table 1:
Main Contact
Manager in medical day sessions
Music Therapist
Physical Therapist
Occupational Therapist
Restorative Tech
Senior TA
Program Supervisor
Recreation Department
Tech in Locomotive Program
Tech in Locomotive Program
Tech in Locomotive Program
We have worked with this team to identify use cases and best participants to fit this study. The
resulting design was a group of 10 patients, 5 males and 5 females, from a group of 21 overall
qualified patients. The group age varied between young kids and young adults, with a mean age
of ~15 (15.3) years old (standard deviation= 8.97).
Table 1. Helping Friends
1.0 Pilot Study Team
Pilot Study
Four categories of patient populations were part of this pilot: inpatients, day program attendees,
locomotor training (LT) patients, and music therapy group attendees. We note that inpatients
were also part of the music therapy group.
The patients had different cognitive and physical challenges. Having a heterogenous group, in
gender, health needs, and age, was intended for a better understanding of different patient
populationsneeds. The variation of targeted outcomes through specific protocols allowed for
the diversification of the group. This diversification also allowed for differing sessions and
number of templates run per client. We will discuss session numbers and duration per category
in the Results section.
Technology Component Description:
Markerless Two-Dimensional Motion Capture:
The motion capture tool presented is the Helping FriendsTM 1.0 toolset by Point Motion Inc. The
toolset’s motion capture technology consists of capturing 17 keypoints in the human body: 5
points in the head and 12 others in the upper and lower body extremities (Figure 1). The
appearing skeleton is a representation of calibration using this system (Figure 2). We will review
the technology in terms of accuracy, reproducible outcomes, and metrics collected in the next
section of this paper.
Figure 1. Keypoints Tracked in the Human Body
Calibration Screen
Pilot Study
The solution piloted is a combination of technologies intended to achieve the effect of motion
tracking without handheld or wearable devices, in favor of
using the built-in camera on common electronic devices
such as computers or tablets. Helping FriendsTM 1.0 is a
designed pose matching experience in which patients are
cued to match displayed poses on the screen and, when
matched correctly, receive auditory feedback in the form
of music. By moving their bodies, patients trigger and
‘make’ music.
The used technology allows for pose estimation via
computer vision techniques that detect human figures in
images and videos. Meaning that one could determine, for
example, where a patient’s elbow shows up in an image.
To be clear, this technology does not recognize who is in
an image; there is no personal identifiable information
associated with the pose detection. The algorithm simply
estimates the location of key body joints (Table 2).
The most accurate set up method is to choose a blank wall without bulky objects, poor lighting,
or busy backgrounds. The calibration screen will appear before the start of the gameplay,
allowing the patient to see what the camera is capturing at that moment. Accordingly, patients
Left Eye
Right Eye
Left Ear
Right Ear
Left Shoulder
Right Shoulder
Left Elbow
Right Elbow
Left Wrist
Right Wrist
Left Hip
Right Hip
Left Knee
Right Knee
Left Ankle
Right Ankle
Figure 2. Calibration Screen
Table 2. Keypoints Tracked in the Human Body
Pilot Study
will move closer or further to ensure positioning within the camera’s window, so that it registers
the correct poses and/or durations during the session.
It is permissible to have the administrator help the patient with setup if needed. In that scenario,
once the administrator sees a functioning image in the calibration screen, they can exit the
calibration and start the game for the concerned patient. Only the patient should be present in
front of the camera so as not to skew the captured data.
Figure 3: Conceptual frameworks: Human body motion capture systems
Pilot Study
In this study, there were no significant differences in screen conditions and digital environments
displayed during participants’ game sessions.
Automated Data Collection:
Helping FriendsTM 1.0 by Point Motion Inc. is synched to the Electronic Data Record (EDR) by
UnitusTI, where data is hosted, and reports are generated. This system has the capability of
delivering a consistent user experience, with reproducible outcomes. It is important to note that
the motion capture and data collection processes used are for gross movements, not finer
movements. Our data collection includes metrics that reflect a patient’s range of motion,
endurance, reaction time, and patient responses to visual/verbal cues within the game. Every
protocol integrated in the motion capture tool has different targeted outcomes. In order for the
automated data collection to succeed, the motion-capture component must register those
movements. The resulting data reflects specific metrics that are collected throughout the
process of reproducing the movements shown on the patient’s screen. The patient’s movements
are meanwhile compared to the target set by the specialist.
Helping FriendsTM 1.0 allows for the collection and automatic storage of 6 metrics per patient
account, via the EDR by UnitusTI: correct poses, incorrect poses, no response, data collection
poses, reaction time, and reaction time general. This data allows doctors to assess performance
of their patients over time and provides insights into the cognitive and physical conditions of the
patient. (See Table 3, below).
Pilot Study
Types (DATs)
Target Type
Data Type
1. Correct poses
Returns the
number of poses correctly
performed by the patient.
Patient selects how many
consecutive sessions a target has to
be achieved in order to be
2. Incorrect poses
Returns the number of poses
incorrectly performed by the
Patient selects how many
consecutive sessions a target has to
be achieved in order to be
3. No response
Returns a number that represents
the number of times where the
system shows a pose, but the
patient does not take any action
(detected by the camera).!
4. Data
The system returns a collection of
articles. Each article shown
represents the state of success or
failure of the pose at a certain time
in the session.
Discrete Trial
Array of
checks and
5. Reaction Time
The time elapsed between the
moment the system shows the pose
(starts the beat) and the moment
the patient successfully performs
the same pose. The system returns a
unique value.
6. Reaction Time
This variable returns the average
reaction time using only correct
pose responses.
Table 3. Collected Data Types
Pilot Study
In this study, specialists selected their preferred metrics for automated reports and graph
Integrated Program: Sound Sage
The Helping FriendsTM 1.0 toolset is centered around ‘bringing music where people need it most’.
The cause-effect mechanism represented when musical phrases are activated upon movement
execution, , is used to enable higher patient engagement.
It would seem that most specialists realize the potential of music to facilitate enjoyment and
engagement for patients in health care contexts. However, many health care professionals may
not realize the potential for using musical experiences to address specific client needs in a
meaningful and quantifiable way. In fact, music has been used effectively as a therapeutic
medium to support engagement in physical activity,[5, 6, 7] increase endurance during physical
activity,[8, 9, 10] and support motor planning.[11,12,13] In this study, subject matter experts
leveraged the presented toolset into targeted programs with identified practical outcomes.
These programs are supported with a HIPAA and FERPA compliant portal for managing client
programs and generating session reports.
Sound Sage program was created by Dr. Thomas Foster, using the Helping FriendsTM 1.0 toolset
as an ongoing assessment tool and as protocol for intervention. The Sound Sage program
presents an enriching
and meaningful music-based platform for supporting growth in client
engagement, sustained attention, motor planning, proprioception, range of motion, strength,
and endurance. Sound Sage program was used in this pilot for its relevance to all age ranges and
abilities. The program’s compositional elements (frequencies, tempos, and textures) are
informed by the latest research in the fields of music and sound therapy and allows it to be
equally beneficial for reducing anxiety.
The program templates include a variety of structures and progressive experience types in order
to meet data collection and intervention needs for all age ranges and ability levels of the
selected group.
The following table of movements/activities represents Sound Sage program templates provided
for this study (Table 4).
Pilot Study
In this study, the music was customized per participant preference or most engaging music
Anatomical Region
Movement Form(s)
Movement Range
Both Sides
Flexion Extension Lateral
Rotation Lateral Flexion
Full Natural Range
Both Sides
Flexion/Forward Elevation
Incremental Full and Range
(45-180 degrees)
Both Sides
Thoracic Rotation
Full Natural Range
Left Side
Assisted/2 Ranges
Standing Static Balance
Both Sides
Knee Flexion
Non-assisted/Eyes Open
Standing Static Balance
Eyes Open/ Ankle Flection
Assisted/Eyes Open
Combination w/ Guided
Meditation Qi Gong
Full Natural Range
Table 4. Sound Sage Core Movements
Pilot Study
The duration of product use for this pilot study was two months, from October 17th to December
18th, 2020 (onboarding and training period not included). We registered a total of 870 minutes
use of the Helping FriendsTM 1.0 toolset with clients at St. Mary’s Healthcare System for Children.
We registered a 70% retention rate in the program, with 1 registered case of drop-out because
of the participant’s lower engagement towards the gameplay.
Participants spent an average 22-minute per session (Standard Deviation= 13.91) engaging in
music activation through movement mirroring (i.e. lifting both arms, one arm). The registered
time included administration of test and setup. Note that time of test and setup administration
decreases throughout the toolset adoption lifecycle.
This pilot resulted an observation in variance in the amount of time spent on gameplay (Table 5).
The participants were apportioned into 3 groups: Group 1: represents 5 participants, who
registered 15 minutes or more on average play per session. Group 2: composed of 4 participants,
who spent 15 or less minutes on average play per session. Group 3: one participant, who spent
less than 5min on gameplay.
The number of sessions per patient varied for reasons classified as: different needs/different
available program templates, adoption rate per client, engagement rate per client, and case
specificities. Some patients, despite their high motivation to engage in the gameplay, were
presented challenges in accurate motion capture due to the need for physical support from
others. This resulted in limited sessions with that respective patient population (LT).
The use of the Helping Friends1.0 toolset of Point Motion Inc. successfully allowed for:
execution of auditory feedback upon accurately mirrored movements, automated data
collection, and enabled customized care per patient needs via the provided Sound Sage
program/music options.
Pilot Study
Group 1
Group 2
Group 3
Moderately Engaged
Not Engaged
Number of
* number of discontinued participations
* number of discontinued participations
Duration of
in min/ session
x ≥ 15
5 x < 15
x <5
Retention rate
per group
Total retention
of total
variation of
o Different need Vs Available program templates during Pilot
o Different adoption rate per client
o Different engagement rate per client
o Variation of attention span
o Variation of physical abilities per client
o Variation of cognitive abilities per client
Not Engaged
Table 5. Study Results
Pilot Study
The Electronic Data Records platform’s reporting system generated graphs representing patient
performance over time (allowing for i.e. qualifications to outcome-based reimbursement codes),
as well as on-demand full reports of patient performance. All metrics and data were collected
automatically, without a manual data entry process. The system allowed for collection of
different metrics in different forms: Label (i.e. Discrete Trial, Task Analysis), Time (i.e. Duration),
and Numeric (i.e. Frequency).
Below are two graphs representing patient performance graph representations (Figure 3, Figure
4). The x-axis represents time (dates of the software use). The y-axis represents the percentage
of success rate per program template (per selected patient profile).
Note: While using the software, the administrator was able to zoom in and out of the graph to view the x-
axis in months, weeks, or days.
A single point representation:
represents the first attempt/game of the patient in his respective
target. He scored a 90% on the total gameplay, meaning he missed 10% of the movements or
entered some poses late. A secondary round of gameplay provides an additional score that can
be used for a two-time use graph.
Figure 4. Graph 1: A Single Point Representation
Pilot Study
Line graph representation:
visualizes the value of the patient’s performance over 1 month,
within the Yoga Flow- Upper Body Mobility Program template run within the Music Therapy
session. The software’s line graph generation enabled us to see baseline trends in the patient’s
activity. In short time frames, we identify the baseline or trend line. Extreme patterns in
performance can be identified over the long term. It is expected that pattern deviations from
this baseline would be identifiable through longer reporting (8+ months).
Unexpected Observations:
During this pilot study, we noted one individual who was able to execute a pose previously
misjudged not feasible. This patient has severely limited activity of his right arm. To his physical
therapists’ surprise, he was able to raise his right arm during gameplay.
In a second case, among the participants in the music therapy program: the music therapist
reported that a patient who, upon hearing the Sound Sage program music outside of her
gameplay session, continued to simulate the correlated pose.
Discussing Findings:
During this pilot study, we demonstrated the ability of the Helping FriendsTM 1.0 toolset to:
automatically collect data per patient during clinical sessions via the Electronic Data Record
Figure 5. Graph 2: Line Graph Representation
Pilot Study
platform UnitusTI; provide auditory feedback upon pose completion via the Sound Sage
program; and generate meaningful reports showing performance over time and scores on
specific tasks related to regular or rehabilitative exercise.
The presented solution provided reproducible outcomes when used according to Point Motion,
Inc. guidelines for the optimal tracking setting (lighting, background, etc.). The presented
solution motivated patients for their physical exercise through the auditory feedback. The instant
reports generated by this solution were able to accommodate several metrics, as specified by
the specialist, enabling targeted tracking of patients’ cognitive and motor function ability
performances over time.
Clinical Relevance:
As innovators in the current landscape of health care, we have chosen to keep the Quadruple
Aim of health care as the ultimate reference for, and measure of, the Helping Friends™ 1.0
toolset and this pilot study:
Improved Patient Experience:
The Helping FriendsTM 1.0 toolset improved patient experience by motivating patients through
auditory feedback. We witnessed the gamification of therapy, which gave patients a way of
expression through music, meanwhile reducing the burden of their challenges. Our unexpected
observations and the 70 % retention rate support this conclusion. A further example: In a music
therapy session, we observed one of the participants using the software to trigger and stimulate
drum sounds while the music therapist played guitar. This represents a shift in the patient from
passive member to active member, enriching
the patient’s therapeutic experience.
Furthermore, the use of visual display components and activated sounds were successfully
executed for all patients in a customized manner. Administrators were able to select specific
programs to fit patients’ respective needs and habits, allowing for more customized care. (The
Helping FriendsTM 2.0 version will further customize the gameplay experience by providing
different gameplay environment choices [i.e. park setting, or a gym setting].)
Better Outcomes and Improving Populations’ Health:
The Helping FriendsTM 1.0 toolset simultaneously aims to facilitate better health care outcomes.
In addition to increasing patient engagement in care and education, the toolset equipped
caregivers to more closely and consistently track patient performance over time, as illustrated by
our software-generated graphs (Figure 3, Figure 4). Although the Helping FriendsTM 1.0 toolset is
not a medical tool, it enabled reproducible monitoring of general wellness and cognitive activity
by synching automated data collection to the Electronic Data Record (EDR) for all gameplay
throughout the study. Furthermore, specialists are able to use the toolset to increase activity per
patient by assigning exercises and following performance remotely. Research has shown that
home care helps health care institutions not just by reducing costs but also by reducing the
Pilot Study
number of patient readmissions. Notably, the level of physical activity registered for those at-
home patients is higher than that of patients in hospitals. [14],[15]
Lower Costs of Care:
According to the Centers for Medicare and Medicaid Services, the U.S. spent 18% of GDP on
health care in 2020.[16] Efforts to decrease doctor visits by ensuring good health for
populations, continuous remote surveillance, and affordability of doctor visits are a central focus
in today’s health care landscape. During this pilot program, the remote setting/telehealth was
not used, as all patients chosen for the pilot visit the hospital regularly (either inpatients, or day
care). A primary goal of this study was to ensure that the setup, onboarding and training, and
program execution were viable. However, the Helping FriendsTM 1.0 toolset software has remote
capabilities embedded. Pre-loaded programs can be offered to the patient from the comfort of
their home, and reports can be accessed by their specialist who can take actions if necessary.
The 70% retention rate in this program indicates significant opportunity for use of the toolset in
remote care. In an upcoming study, we will be analyzing remote assessment use cases in more
Improved Clinical Experience:
The pressure that the medical core currently faces has raised concerns about provider burnout.
The Medscape’s 2020 Physician Lifestyle Report showed that among 15,000 respondents 42% of
physician respondents reported burnout. [17] Additionally, attention has been brought to the
correlation between decreased staff engagement/burnout and lower patient satisfaction [18].
The successful execution of the automated data collection feature in this study shows promise as
a positive player in the burnout solution. The Helping FriendsTM 1.0 toolset alleviates
administrative burdens by eliminating all manual data entry process during a session.
Meanwhile, the toolset’s program queue feature and cloud solution capabilities allow the
specialist to assign programs in advance and access data from anywhere, anytime.
Reimbursement and Telehealth Applications:
The Helping FriendsTM 1.0 toolset and the integrated UnitusTI EDR platform are HIPAA and FERPA
compliant. It is worth noting that, adoption of this toolset, allows interventions to move beyond
traditional video techniques and verbal reporting.
While Telehealth technology has been around for nearly 40 years, we see a widespread on its
implementation especially during the COVID-19 pandemic in 2020. As we continue the path to
finding the best ways for care delivery, it is important to acknowledge the exponential growth
telehealth is seeing and the improvement of Remote Patient Monitoring (RPM) tools that will
empower both the caregiver and the care recipient. Studies in the field forecast growth in
telehealth that is paralleled with a growth in remote diagnostic equipment.
Pilot Study
In terms of reimbursement: With our current results, if an institution has 100 clients, and we
register 3 sessions/month per client (2.9), 70 will be retained. The health care institution could
perform 203 sessions per month, equating to billing and collecting reimbursement at
approximately $8,120 per month in revenue ($97,440 per year). This estimate assumes a very
conservative average collection of $40 per session.
The Helping FriendsTM 1.0 software has areas of improvement that indicate opportunities to
surpass current capabilities:
Currently, the accuracy of Helping FriendsTM 1.0 is negatively impacted when incorrectly set up,
or when set up in a poor environment. Poor environments include variables which negatively
impact the accuracy of the motion capture software, i.e. very bright or very poor lighting and
background activity (such as people or bulky objects behind the person).
If there are multiple persons in the motion capture framing, keypoints from those persons will be
estimated as part of the patient’s single pose—meaning that even if the patient accurately
executes the pose, the system will capture, for example, one person’s right arm and another
person’s left knee and fail to register the accurate patient movement as success.[19]
With regards to program selection, the caregiver must decide the exercises and the programs for
the patient. Point Motion, Inc. has filed a patent for machine learning and Artificial Intelligence
(AI) to automate that process.
The Helping FriendsTM 1.0 automated data collection library has two different functions for
different use cases: tracking a single user or tracking multiple users. The fastest and most
accurate tracking comes in the form of single user tracking, which is where Point Motion, Inc. has
chosen to start for the reliability and consistency of the technology. One limitation of single user
tracking is that locomotor training (LT) program patients do not benefit highly from this version
of the toolset. During this study, there were challenges with tracking because of the multiple
individuals needed to support the patient’s stillness during exercises requiring trunk flexion. The
best option to accommodate this need was to keep the camera pointing to the body parts of the
concerned patient, without capturing parts of the supporters’ bodies. Because we used a tablet
on a tripod, it was possible to control, to a degree, the angle of the capturebut not for every
This pilot study demonstrates significant innovation in the field of markerless motion detection
technology: a way of managing and assigning specific programs for different patient populations
Pilot Study
and target specific outcomes. The technology is accessible, allows for data collection in real time
on the cloud and distributed remotely or in-person on demand.
Perhaps most valuably: This study showcased the ability of the Helping Friends™ 1.0 toolset to
create a variety of engaging experiences for different patient populations and to motivate them
while acting on their general wellness (i.e. sustained attention for cognitively challenged
individuals, pose accuracy for physically challenged individuals, and prime/free play for less
responsive individuals).
The ultimate vision of Helping FriendsTM 1.0 is to allow end users/patients to engage in
activities through accessible technology, while eliminating the need to implement
sophisticated external devices in order to monitor patient’s adherence and progress over time.
Thus, in a larger scheme, this technology aims to empower a greater number of patients and
care providers. Using the Quadruple Aim of Healthcare as a measure of, and guide to, a
personalized care method that improves both patient and clinical experience, contribute to
improve overall health outcomes, while decreasing costs.
In today’s changing world, where remote care services are rapidly becoming the new norm and
Telehealth services are no longer a luxury but a necessity to monitor clients’ performance,
Helping FriendsTM 1.0 is a tool that allows for innovation at lower costs and, above all, wider
accessibility of care despite social and economic disparities.
Pilot Study
[1] Colyer SL, Evans M, Cosker DP, Salo AIT. A Review of the Evolution of Vision-Based
Motion Analysis and the Integration of Advanced Computer Vision Methods Towards
Developing a Markerless System. Sports Med Open. 2018;4:24.
018-0139-y. Accessed July 21, 2020.
[2] Zhou H, Hu H. Human Motion Tracking for Rehabilitation—A Survey. Biomed Signal Process
Control. 2008;3(1):1-18. Accessed July 21, 2020.
[3] The Conversation. How Pleasure Affects Our Brains. NeuroscienceNews. Published April 28, 2018. Accessed July
21, 2020.
[4] Zald DH, Zatorre RJ. Music. In: Gottfried JA, ed. Neurobiology of Sensation and Reward. Boca
Raton, FL: CRC Press/Taylor & Francis; 2011.
Accessed July 21, 2020.
[5] Neugebauer S, Serghiou M, Herndon D, Suman O. Effects of a 12-Week Rehabilitation
Program With Music & Exercise Groups on Range of Motion in Young Children With Severe
Burns. J Burn Care Res. 2008;29(66):939-948.
Accessed July 21, 2020.
[6] Boldt S. The Effects of Music Therapy on Motivation, Psychological Well-Being, Physical
Comfort, and Exercise Endurance of Bone Marrow Transplant Patients. J Music Ther.
1996;33(3):164-188. Accessed July 21, 2020.
[7] Johnson G, Otto D, Clair AA. The effect of Instrumental and Vocal Music on Adherence to a
Physical Rehabilitation Exercise Program with Persons who are Elderly. J Music Ther.
2001;38(2):82-96. Accessed July 21, 2020.
[8] Karageorghis CI, Priest DL, Williams LS, Hirani RM, Lannon KM, Bates BJ. Ergogenic and
Psychological Effects of Synchronous Music During Circuit-Type Exercise. Psychol Sport Exerc.
2010;11(6):551-559. Accessed July 21, 2020.
[9] Karageorghis CI, Priest DL. Music in the Exercise Domain: A Review and Synthesis (Part II). Int
Rev Sport Exerc Psychol. 2012;5(1)67-84.
Accessed July 21, 2020.
[10] Altenmüller E, Schlaug G. Music, Brain, and Health: Exploring Biological Foundations of
Music’s Health Effects. In: MacDonald RAR, Kreutz G, Mitchell L, eds. Music, Health, and
Wellbeing. New York, NY: Oxford University Press; 2012.12-24.
Pilot Study
[11] Stephan KM, Thaut MH, Wunderlich G, Schicks W, Tian B, Tellmann L, et al. Conscious
and Subconscious Sensorimotor Synchronization—Prefrontal Cortex and the Influence of
Awareness. Neuroimage. 2002;15:345-352. Accessed
July 21, 2020.
[12] Grahn JA, Henry MJ, McAuley JD. FMRI Investigation of Cross-Modal Interactions in Beat
Perception: Audition Primes Vision, but Not Vice Versa. Neuroimage. 2011;54:1231-1243. Accessed July 21, 2020.
[13] Konoike N, Kotozaki Y, Miyachi S, Miyauchi CM, Yomogida Y, Akimoto Y, et al. Rhythm
Information Represented in the Fronto-Parieto-Cerebellar Motor System. Neuroimage.
2012;63:328-338. Accessed July 21, 2020.
[14] McIIvennan CK, Eapen ZJ, Allen LA. Hospital Readmissions Reduction Program.
Circulation. 2015;131(20):1796-1803.
Accessed July 21, 2020.
[15] Levine DM, Ouchi K, Blanchfield B, Saenz A, Burke K, Paz M, et al. Hospital-Level Care
at Home for Acutely Ill Adults: A Randomized Controlled Trial. Ann Intern Med. 2020;
172(2):77-85. Accessed July 21, 2020.
[16] Statistica. U.S. national health expenditure as percent of GDP from 1960 to
nt%20developments%20tell%20us%20that,GDP%20share%20among%20developed%20countries. Accessed
July 28. 2020.
[17] Kane L. Medscape National Physician Burnout & Suicide Report 2020: The Generational
Divide. Medscape.!
[18] McIlvennan CK, Eapen ZJ, Allen LA. Hospital Readmissions Reduction
Program. Circulation. 2015;131(20):1796-
1803. Accessed July 28, 2020.
[19] Real-time Human Pose Estimation in the Browser with TensorFlow.js. TensorFlow Blog. Published May 7,
2018. Accessed July 21, 2020.
Pilot Study
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Background: Substitutive hospital-level care in a patient's home may reduce cost, health care use, and readmissions while improving patient experience, although evidence from randomized controlled trials in the United States is lacking. Objective: To compare outcomes of home hospital versus usual hospital care for patients requiring admission. Design: Randomized controlled trial. ( NCT03203759) Setting: Academic medical center and community hospital. Patients: 91 adults (43 home and 48 control) admitted via the emergency department with selected acute conditions. Intervention: Acute care at home, including nurse and physician home visits, intravenous medications, remote monitoring, video communication, and point-of-care testing. Measurements: The primary outcome was the total direct cost of the acute care episode (sum of costs for nonphysician labor, supplies, medications, and diagnostic tests). Secondary outcomes included health care use and physical activity during the acute care episode and at 30 days. Results: The adjusted mean cost of the acute care episode was 38% (95% CI, 24% to 49%) lower for home patients than control patients. Compared with usual care patients, home patients had fewer laboratory orders (median per admission, 3 vs. 15), imaging studies (median, 14% vs. 44%), and consultations (median, 2% vs. 31%). Home patients spent a smaller proportion of the day sedentary (median, 12% vs. 23%) or lying down (median, 18% vs. 55%) and were readmitted less frequently within 30 days (7% vs. 23%). Limitation: The study involved 2 sites, a small number of home physicians, and a small sample of highly selected patients (with a 63% refusal rate among potentially eligible patients); these factors may limit generalizability. Conclusion: Substitutive home hospitalization reduced cost, health care use, and readmissions while increasing physical activity compared with usual hospital care.
Full-text available
Human motion tracking for rehabilitation has been an active research topic since the 1980s. It has been motivated by the increased number of patients who have suffered a stroke, or some other motor function disability. Rehabilitation is a dynamic process which allows patients to restore their functional capability to normal. To reach this target, a patients’ activities need to be continuously monitored, and subsequently corrected. This paper reviews recent progress in human movement detection/tracking systems in general, and existing or potential application for stroke rehabilitation in particular. Major achievements in these systems are summarised, and their merits and limitations individually presented. In addition, bottleneck problems in these tracking systems that remain open are highlighted, along with possible solutions.
Full-text available
How we measure time and integrate temporal cues from different sensory modalities are fundamental questions in neuroscience. Sensitivity to a "beat" (such as that routinely perceived in music) differs substantially between auditory and visual modalities. Here we examined beat sensitivity in each modality, and examined cross-modal influences, using functional magnetic resonance imaging (fMRI) to characterize brain activity during perception of auditory and visual rhythms. In separate fMRI sessions, participants listened to auditory sequences or watched visual sequences. The order of auditory and visual sequence presentation was counterbalanced so that cross-modal order effects could be investigated. Participants judged whether sequences were speeding up or slowing down, and the pattern of tempo judgments was used to derive a measure of sensitivity to an implied beat. As expected, participants were less sensitive to an implied beat in visual sequences than in auditory sequences. However, visual sequences produced a stronger sense of beat when preceded by auditory sequences with identical temporal structure. Moreover, increases in brain activity were observed in the bilateral putamen for visual sequences preceded by auditory sequences when compared to visual sequences without prior auditory exposure. No such order-dependent differences (behavioral or neural) were found for the auditory sequences. The results provide further evidence for the role of the basal ganglia in internal generation of the beat and suggest that an internal auditory rhythm representation may be activated during visual rhythm perception.
Hospital readmission measures have been touted not only as a quality measure but also as a means to bend the healthcare cost curve. The Affordable Care Act (ACA) established the Hospital Readmission Reduction Program (HRRP) in 2012. Under this program, hospitals are financially penalized if they have higher-than-expected risk-standardized 30-day readmission rates for acute myocardial infarction, heart failure, and pneumonia. The HRRP has garnered significant attention from the medical community, both positive and negative. Here, we describe the reasons the HRRP was implemented, the penalties levied, the impact it has had on transitional care and readmissions, the pros and cons of the policy, and its future. Hospital readmissions are associated with unfavorable patient outcomes and high financial costs.1,2 Causes of readmissions are multifactorial, and rates vary substantially by institution.3,4 Historically, nearly 20% of all Medicare discharges had a readmission within 30 days.1 The Medicare Payment Advisory Commission has estimated that 12% of readmissions are potentially avoidable. Preventing even 10% of these readmissions could save Medicare $1 billion.5 Therefore, reducing hospital readmissions has been made a national priority. In 2008, the Medicare Payment Advisory Commission recommended to Congress that the Centers for Medicare & Medicaid Services (CMS) begin confidentially reporting readmission rates and resource use to hospitals and physicians.6 In 2009, CMS began publicly reporting hospital-level readmission rates, which were added to the Hospital Compare Web site.7 Before 2012, hospitals had little direct financial incentive to reduce readmissions. For Medicare beneficiaries with inpatient stays, hospitals receive payment with the inpatient prospective payment system (IPPS). This payment, based on a diagnosis-related group (DRG), covers the inpatient stay and any outpatient diagnostic and admission-related outpatient nondiagnostic services provided by the institution on the date of the patient’s admission or within 3 days immediately …
This study assessed the effects of music with bone marrow transplant patients on exercise endurance, motivation, psychological well-being, and physical comfort. Exercise, although difficult due to treatment effects, is necessary to reduce or prevent muscle atrophy. The subjects were 3 females and 3 males ranging in age from 14 to 53 years. Two subjects received 10 sessions, while 4 others were seen between two to three sessions each. A music/nonmusic reversal design ABABAB (etc.) was used. independent variables included live and recorded music for patiicipation, relaxation and imagery, progressive relaxation exercises, range of motion exercises, and aerobic exercise. The dependent variables included an observational behavior scale, subject self-report of relaxation, comfort, pain and nausea levels, exercise endurance, and an end-of-study effectiveness questionnaire. Long term results (10 sessions) indicated that music sessions were most effective in increasing subjects' self-report relaxation and comfort levels. Endurance for both subjects increased during sessions with music. More cooperafive behavior and higher participation levels were also observed during music sessions. Short term results (two to three sessions) indicated that music sessions were more effective in increasing relaxation and comfort levels. No short term subjects showed an increase in endurance during music versus nonmusic sessions.
Objectives: Motivational music when synchronized with movement has been found to improve performance in anaerobic and aerobic endurance tasks, although gender differences pertaining to the potential benefits of such music have seldom been investigated. The present study addresses the psychological and ergogenic effects of synchronous music during circuit-type exercise. Design: A mixed-model design was employed in which there was a within-subjects factor (two experimental conditions and a control) and a between-subjects factor (gender). Methods: Participants (N =26) performed six circuit-type exercises under each of three synchronous conditions: motivational music, motivationally-neutral (oudeterous) music, and a metronome control. Dependent measures comprised anaerobic endurance, which was assessed using the number of repetitions performed prior to the failure to maintain synchronicity, and post-task affect, which was assessed using Hardy and Rejeski's (1989) Feeling Scale. Mixed-model 3 (Condition) x 2 (Gender) ANOVAs, ANCOVAs, and MANOVA were used to analyze the data. Results: Synchronous music did not elicit significant (p <.05) ergogenic or psychological effects in isolation; rather, significant (p <.05) Condition x Gender interaction effects emerged for both total repetitions and mean affect scores. Women and men showed differential affective responses to synchronous music and men responded more positively than women to metronomic regulation of their movements. Women derived the greatest overall benefit from both music conditions. Conclusions: Men may place greater emphasis on the metronomic regulation of movement than the remaining, extra-rhythmical, musical qualities. Men and women appear to exhibit differential responses in terms of affective responses to synchronous music.
Previous studies indicate that rehabilitation programs supplemented with a strength and endurance-based exercise program improve lean body mass, pulmonary function, endurance, strength, and functional outcomes in severely burned children over the age of 7-years when compared with standard of care (SOC). To date, supplemental exercise programming for severely burned children under the age of 7-years has not yet been explored. The purpose of this study was to determine if a 12-week rehabilitation program supplemented with music & exercise, was more effective in improving functional outcomes than the SOC alone. This is a descriptive study that measured elbow and knee range of motion (ROM) in 24 severely burned children between ages 2 and 6 years. Groups were compared for demographics as well as active and passive ROM to bilateral elbows and knees. A total of 15 patients completed the rehabilitation with supplemental music and exercise, and data was compared with 9 patients who received SOC. Patients receiving the 12-week program significantly improved ROM in all joints assessed except for one. Patients receiving SOC showed a significant improvement in only one of the joints assessed. Providing a structured supplemental music and exercise program in conjunction with occupational and physical therapy seems to improve both passive and active ROM to a greater extent than the SOC alone.
This study compared live, instrumental music, vocal music, and no music on the repetition frequencies for 14 prescribed physical therapy rehabilitation exercises. Male (N = 4) and female (N = 15) residents of care centers for older adults served as subjects. They ranged in age from 65 to 90 years (M= 84.3), and were either referred to physical therapy or were already involved in a physical therapy exercise pro-gram. All subjects (N = 19) participated in 6 treatment sessions under 3 conditions: Two sessions with live instrumental music, two with live vocal music, and two with no music. In all music sessions, familiar and recognizable songs were paired with specific exercises. Each exercise in all conditions had the same duration, and a metronome established consistent tempos. Each session was videotaped for later review and data collection. Analyses of variance were calculated for treatment effects and mean differences among the three conditions which yielded significant treatment effects and treatment differences among conditions for 6 of the 14 exercises. Unsolicited comments from subjects indicated preference for music over no music conditions while exercising. Further study is required to establish the relationship between music enhanced exercises and adherence to exercise regimens in populations of older adults.
One of the most compelling challenges for modern neuroscience is the influence of awareness on behavior. We studied prefrontal correlates of conscious and subconscious motor adjustments to changing auditory rhythms using regional cerebral blood flow measurements. At a subconscious level, movement adjustments were performed employing bilateral ventral mediofrontal cortex. Awareness of change without explicit knowledge of the nature of change led to additional ventral prefrontal and premotor but not dorsolateral prefrontal activations. Only fully conscious motor adaptations to a changing rhythmic pattern showed prominent involvement of anterior cingulate and dorsolateral prefrontal cortex. These results demonstrate that while ventral prefrontal areas may be engaged in motor adaptations performed subconsciously, only fully conscious motor control which includes motor planning will involve dorsolateral prefrontal cortex.