Conference PaperPDF Available

ClimbAX: Skill Assessment for Climbing Enthusiasts

Authors:

Abstract and Figures

In recent years the sport of climbing has seen consistent in- crease in popularity. Climbing requires a complex skill set for successful and safe exercising. While elite climbers re- ceive intensive expert coaching to refine this skill set, this progression approach is not viable for the amateur popu- lation. We have developed ClimbAX – a climbing perfor- mance analysis system that aims for replicating expert as- sessments and thus represents a first step towards an auto- matic coaching system for climbing enthusiasts. Through an accelerometer based wearable sensing platform, climber’s movements are captured. An automatic analysis procedure detects climbing sessions and moves, which form the ba- sis for subsequent performance assessment. The assessment parameters are derived from sports science literature and in- clude: power, control, stability, speed. ClimbAX was evalu- ated in a large case study with 47 climbers under competition settings. We report a strong correlation between predicted scores and official competition results, which demonstrate the effectiveness of our automatic skill assesment system.
Content may be subject to copyright.
ClimbAX: Skill Assessment for
Climbing Enthusiasts
Cassim Ladha, Nils Y. Hammerla, Patrick Olivier, Thomas Pl
¨
otz
Culture Lab, School of Computing Science
Newcastle University
Newcastle upon Tyne, UK
{firstname.lastname}@newcastle.ac.uk
ABSTRACT
In recent years the sport of climbing has seen consistent in-
crease in popularity. Climbing requires a complex skill set
for successful and safe exercising. While elite climbers re-
ceive intensive expert coaching to refine this skill set, this
progression approach is not viable for the amateur popula-
tion. We have developed ClimbAX a climbing performance
analysis system that aims for replicating expert assessments
and thus represents a first step towards an automatic coach-
ing system for climbing enthusiasts. Through an accelerom-
eter based wearable sensing platform, climber’s movements
are captured. An automatic analysis procedure detects climb-
ing sessions and moves, which form the basis for subsequent
performance assessment. The assessment parameters are de-
rived from sports science literature and include: power, con-
trol, stability, speed. ClimbAX was evaluated in a large case
study with 53 climbers under competition settings. We re-
port a strong correlation between predicted scores and offi-
cial competition results, which demonstrate the effectiveness
of our automatic skill assessment system.
Author Keywords
Sports analysis, Climbing, Skill Assessment, Activity
Recognition
ACM Classification Keywords
H.1.2 User/Machine Systems I.5 Pattern Recognition: J.4 So-
cial and Behavioral Sciences
INTRODUCTION
The sport of climbing has become increasingly popular and is
now widely enjoyed as a recreation activity as well as a com-
petitive sport. For example, in the UK the sport “has been
on a upward trend since 2005” with a continuous increase
in participation [10]. The Italian Alpine Club, which is the
world’s largest, reports the sport in general has had a popula-
tion growth of 10% since 2009 [9]. As a recreational activity
climbing holistically improves both physical and mental fit-
ness, provides a basis for social interactions, and is a way to
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
UbiComp’13 Adjunct, September 8–12, 2013, Zurich, Switzerland.
Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM 978-1-4503-2215-7/13/09...$15.00.
http://dx.doi.org/10.1145/2493432.2493492
enjoy the outdoors. Climbing is also being recognised as a
competitive activity, and was considered for inclusion in the
2020 Olympics [22].
Similar to other sports, professional climbing requires phys-
ical conditioning, applied sports science and training. Elite
climbers follow strict training programmes defined with the
assistance of and monitored by a coach. In a typical session,
a coach will assess the climber through observation and then
provide feedback by commenting on their technique, or sug-
gest training routes that will assist in addressing weaknesses.
At amateur level, coaching is also desirable and is a service
offered by indoor climbing centres. However, the sheer num-
ber of climbing enthusiasts render detailed and frequent feed-
back from a coach, as it is received by elite athletes, imprac-
tical for the amateur. Consequently, amateur coaching is of-
ten a group exercise with a typical 1:8 coach to student ratio.
The heterogeneity of such groups in terms of climbing skills
and experience results in only general feedback rather than
in-depth, personalised recommendations and advice.
A wealth of related work exists on self assessment of phys-
ical activities using mobile sensing platforms (e.g., [25] and
references therein). Commercially available devices, such as
Nike fuel band [27] and Fitbit [15], are effective for improv-
ing levels of activity simply through providing and visualis-
ing statistics to the user that are related to the frequency and
—to some extent— fatigue [5]. Some sports self assessment
tools are available that focus on the technical skills of the ath-
lete by providing detailed information, not just about the fre-
quency, but also about the quality of the particular activities.
Examples include the automatic analysis of golf swings [19]
or automatic assistance for swimmers [4].
In line with the aforementioned analysis tools, we have iden-
tified the assessment of climbing skill as a case for ubiqui-
tous computing. We have embarked on developing ClimbAX
a sensing and analysis system that replicates professional
climbing assessment as it is conducted by human coaches.
ClimbAX utilises wrist-worn accelerometers to capture a
climber’s movements in naturalistic settings. Climbing
episodes and individual hold transitions are detected auto-
matically, forming the basis for performance analysis. A va-
riety of performance attributes are developed in this work,
which, while being meaningful to climbers, resemble tra-
ditional, subjective assessment performed by a professional
coach. This climbing skill assessment aims to support fu-
ture automatic coaching systems that incorporate this objec-
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
235
(a) Illustration of the ClimbAX assessment system
including visualisation of analysis results produced.
(b) Examples of climbing sub-disciplines: Indoor bouldering; Sport climbing;
Deep water solo; Ice climbing; Traditional climbing; Aid climbing (i to vi)
Figure 1. Overview of the ClimbAX system for automatic climbing skill assessment and its potential application cases. See text for description.
tive performance information to devise training plans tailored
to the individual.
ClimbAX records the climber’s movements using a wrist-
worn sensing platform that logs high-resolution, tri-axial ac-
celerometer data. This platform is small and sturdy, and does
not hinder the climber in their activities. The aggregated data
is then processed using an unsupervised analysis procedure,
which automatically:
i) filters out climbing from background activities;
ii) segments climbing sessions with respect to transitions be-
tween holds, i.e., those moments where the climber re-
mains stationary (fixating themselves on the face they are
scaling); and
iii) performs climbing skill assessment based on an objective
quality scoring scheme.
We evaluated our assessment system in a large field study in
a premiere indoor climbing centre assessing the performance
of 47 participants of an open bouldering competition event
and 6 climbers practicing sport climbing.
The sensing and analysis system presented in this paper al-
lows amateur climbers to track a set of physical performance
skills, which can be used either for self-directed training
or as a basis for external coaching, and thus improve their
performance whilst maintaining health and safety. Figure
1 illustrates the developed system and its potential applica-
tion cases. Objectively measuring climbing relevant parame-
ters represents an important building block for an automatic
coaching system as we are aiming for with ClimbAX.
CLIMBING AS A SPORT
The term Climbing is used to collectively group many sub-
disciplines each having their own distinctions relating to ter-
rain type, accepted ethics regarding protection and tactics
used to ascend [18]. Figures 1(b) shows examples of the most
widely performed sub-disciplines of climbing.
Popular types of climbing are: i) Bouldering, which involves
the ascent of relatively low level routes on free standing boul-
ders with just a crash pad to protect the climber in the case
of a fall; and ii) Sport climbing, where the climber clips their
rope into bolts that are pre-placed into the rock, and in case
of a fall, a second person (“belayer”) will hold fast the rope
(with assistance of a friction device) to prevent the climber
hitting the ground. Further outdoor climbing sub-disciplines
include: iii) Deep Water Solo (also known as Psicobloc),
where the climber uses water below to break a fall; iv) Ice
climbing, where the climber uses the assistance of crampons
and ice tools to ascend; v) Traditional, a discipline that em-
ploys a strict ethic that all protection placed in the rock must
be placed by hand and be removable without damaging the
rock; vi) Aid climbing, where the climber is permitted to use
placed protection as hand and foot holds. Alpinism is an-
other discipline that combines aid- and ice climbing at high
altitudes. Bouldering and Sport climbing are also frequenty
practiced indoors on man-made walls, often constructed from
plywood, using shaped resin holds.
Dangers and Difficulties
Climbing carries risks both in the form of objective danger
(for example, a rock falling) as well as an injury through
poor judgement of the condition of the terrain, or through
poor climbing performance. Little can be done regarding the
former other than carefully assessing the general conditions
(e.g., weather, composition of the targeted face to be scaled),
whereas the main influencing factor for the latter is lack of ex-
perience and misperception of one’s own skills [36]. Unreal-
istic judgments can lead to wrong decisions regarding the in-
dividual appropriateness of particular climbing routes, which
can have fatal consequences.
The decision whether or not to embark on a particular route
is heavily influenced by knowledge of the climber’s abilities,
which is typically gained through comparison to others who
have already completed the particular route. Making objec-
tive comparisons between climbers’ abilities can lead to both
more informed and confident decisions regarding whether a
particular route is appropriate for an individual.
Climbing routes are typically ranked according to their dif-
ficulty using established grading schemes, such as the inter-
nationally recognised French grading system for sport climbs
or the Hueco “V” grading system for boulder problems [18].
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
236
Gradings typically do not transfer well between sub-disci-
plines. However, they share the underlying principle of judg-
ing how difficult climbs are technically. In the case where
there is an apparent objective danger (often judged by the
outcome of a fall) a second grade is often given that can be
used to interpret the “seriousness” of the route. In the British
Traditional System, this grading is descriptive rather than nu-
meric. For example, a route may be classified as ”Difficult”
or “Very Severe” [18].
What it takes to get high
Across its sub-disciplines climbing requires a range of physi-
cal abilities. For example, climbing large mountain routes re-
quires very good all round stamina, endurance and tolerance
to high altitudes, whereas challenges linked to bouldering are
often gymnastic in nature and require physical strength, good
general coordination, and muscular flexibility. Furthermore,
within each sub-discipline there is also scope to specialise for
a particular type of terrain. Some climbers for example prefer
scaling steep overhanging rocks, which requires very good
upper body strength. Others focus on routes that consist of
large numbers of hard individual moves, which necessitates
power endurance. Despite this diversity all climbers need to
possess a core skill set, which subsumes at least four main
physically trainable competencies: i) Power used to transition
between holds [31]; ii) Control over limb movement [34]; iii)
Speed of ascent; and iv) Stability whilst on a hold [21, 38].
Investigating the reasons for good or bad climbing perfor-
mance, some studies have gone as far as measuring plasma
cortisol (stress hormone) in climbers during and after high
stress activities [13]. Positive correlations to confidence as
well as to somatic and cognitive anxiety in climbing were
found. Other studies have measured heart rates as both fa-
tigue and stress indicators. These however, did not unveil
any insight due to muscles operating in anaerobic state during
climbing [24]. In contrast to such biochemical parameters the
climber’s experience is difficult to assess. Experience helps a
climber to identify the most efficient way to climb through
a challenging sequence of moves, and it can help identify
the most likely weather conditions that will result in a suc-
cessful ascent (climbing is highly dependant on rock friction
which increases as temperature decreases). In either case it
is difficult to reason about the mental state or experience of a
climber other than through observing how they perform phys-
ically.
It has been demonstrated that parameters relating to the phys-
ical performance of a climber can be measured at the inter-
face of the hand and the hold. These parameters vary from
core body strength to balance and contact strength [17]. Re-
lated studies have exclusively used holds instrumented with
strain gauges or vision based systems where climbers were in-
strumented with markers. While these methodologies demon-
strate the validity of the parameters, they are not suitable for
deployments in realistic settings. Only very few and rather
explorative attempts to automatically asses climbing skills in
a real-world context have been undertaken thus far. For ex-
ample, Pansiot et al. attached an accelerometer to a climber’s
head for recording their movements [28]. In a small study
with 4 participants they derived climbing skill parameters,
which although interesting, did not map to any recognised
parameters from the sports science literature.
AUTOMATIC CLIMBING PERFORMANCE ASSESSMENT
The key to performance improvement in climbing is both in-
creased frequency of exercise [11] and training specific weak-
nesses and elements of technique [21]. In the elite class these
training goals are typically managed with the assistance of
a coach. Although a direct transfer of such manual coach-
ing programs to the population of amateur climbers is desir-
able, resource limitations render expert coaching impractica-
ble. Alternatively, automatic assessments have the potential
to make coaching more widely accessible.
Structured and guided self-monitoring and self-assessment
represent a reasonable alternative to costly professional
coaching. A few technical systems have been developed that
support amateur climbers in keeping track of their exercises.
For example, smart phone applications are available that walk
climbers through sets of fixed routines and record the date
they were completed; essentially corresponding to an elec-
tronic climbing diary for retrospective (manual) analysis [6,
8]. Such technology supported climbing diaries (and vari-
ants thereof) can effectively support climbers in keeping up
regular exercising or even increasing participation frequency,
which in general has positive effects on their health [14].
Automatic coaching aids for climbing are required to not only
report the frequency and duration of exercise but also a per-
formance breakdown that is presented using terminology that
is familiar to the sport. ClimbAX has been designed to com-
ply with these requirements. Figure 2 gives an overview of
our system. Movements are captured using small, wrist-worn
sensing devices, which are configured to record tri-axial ac-
celeration data with high temporal resolution. After a ses-
sion (which can contain multiple climbs) the sensor data is
uploaded to an analysis platform where climbing orientated
data is automatically filtered out (climb segmentation) and the
moves within each climb are automatically detected (move
segmentation). Based on the extracted moves, the actual as-
sessment is then performed, which is informed by standard
climbing grading schemes. Finally, the results are visualised
both on a session summary basis and at the more fine-grained
level of detail corresponding to particular skill criteria from
the assessment.
Recording
With a view on practical deployments in realistic, i.e., non-
laboratory, climbing scenarios we adopted a body-worn sens-
ing approach for capturing climbing activities. Apart from
the advantage of universal applicability due to minimal re-
quirements on existing infrastructure (such as independence
on calibrated camera setups [33]), a wearable, and thus mo-
bile, sensing platform has the benefit of providing detailed
and high-resolution data through direct measurements of the
climber’s movements. Accelerometry in general has proven
very effective for assessments of human movements in a va-
riety of application domains [7]. In line with previous, explo-
rative studies [28, 32] we employ tri-axial accelerometers for
our automatic climbing assessment framework.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
237
Figure 2. ClimbAX: System overview (see text for description).
Transmissions of rotational and vibrational forces in the range
of 0.2 20Hz (human movement range) that are exerted
through the fingers have been shown to be measurable using
an accelerometer placed on the wrist [26]. Consequently, and
coupled with the high level of user compliance the wrist af-
fords, ClimbAX sensor system was designed around a watch
embodiment. Since climbing requires good symmetry and
balance we instrument both wrists of the climber in order to
capture the movements of the hand that is transitioning as
well as the hand supporting the body during transitioning.
Actual applicability for realistic climbing scenarios requires
the movement capturing subsystem of ClimbAX to record
for a minimum of one day, to be light-weight, scratch proof
and hypo-allergenic, and to be sturdy enough for operating
in chalky/dusty environments. Accordingly we designed a
watch-like sensing platform as shown in Figure 3. At its core
is a 16-bit, 16 MIPS PIC24 processor, and a 14-bit tri-axial
accelerometer (MMA8451Q by Freescale). Sensor readings
are sampled at a rate of 100Hz, which provides sufficiently
detailed movement information. Samples are stored onto
a 4Gb sized NAND flash memory chip along with associ-
ated timestamps (accurate to 20ppm and generated from the
PIC24). Communication with the device, e.g., for configura-
tion and data download, is based on a micro-USB connector.
The internals of the sensing platform are potted into a poly-
carbonate injection moulded case, which is housed by a sil-
icone wrist band. The band was designed to be thin enough
to see the screen through yet still provide a scratch-proof and
replaceable fixing method. The design of the band, firmware
and software tools were released as Open Hardware under the
Openmovement platform [39].
Climb Segmentation
Our vision of a climbing analysis system comprises an ac-
cessory for assessing climbing activities in a naturalistic set-
ting, i.e., not imposing any additional constraints or require-
ments that would hinder the core exercise. In line with this,
ClimbAX detects climbing activities, which alleviates its user
from the necessity of interacting with the device, e.g., click-
ing a button before, during or after each climb.
During every-day activities, arm based movements are sub-
ject to what is commonly referred to as symmetry-bias [35].
Motions by, e.g., one arm automatically initiate a counter
movement by the other arm to keep balance. This symme-
try is often used to characterise gait, particularly for neuro-
degenerative conditions [40]. During climbing this symmetry
between the upper extremities is broken as it is crucial for one
limb to stay attached to the hold, minimising its movement.
Along with tremors related to high intensity activities (vibra-
tions of the hands when on holds caused by fatigue or extreme
exertion) this gives rise to specific climbing patterns as they
are recorded on the wrists. Our automatic climb detection is
based on the analysis of these characteristic movement pat-
terns, which we found are more discriminative than simple
assessments of simultaneous upwards wrist orientation with
respect to gravity.
Detecting episodes of climbing within continuous streams of
accelerometry data corresponds to segmentation of time se-
ries data, for which two general processing paradigms exist:
i) explicit identification of start- and end-points of semanti-
cally contiguous bouts (segments); and ii) implicit segmen-
tation through extraction of analysis frames and subsequent,
isolated classification regarding the patterns of interest [23].
Ambiguity in transitions between non-climbing and climb-
ing activities effectively renders explicit segmentation tech-
niques impractical for climb detection. However, the afore-
mentioned break of symmetry-bias during climbing results
in substantially different sensor data distributions for climb-
ing and non-climbing episodes. Exploiting this, we employ
an implicit segmentation approach for climb detection us-
ing a sliding window procedure that extracts analysis frames
thereby integrating sensor data from both wrists.
Our sliding window procedure extracts frames of 5s length
with an overlap of 1s, which captures climbing activities
very effectively. For analysing symmetry-biases (and breaks
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
238
Figure 3. Wearable sensing platform for recording climbing activities
that consists of a high-resolution tri-axial accelerometer, OLED screen
(not used), on-board processing unit (PIC), battery, and flash memory.
therein) we concatenate the tri-axial sensor readings of both
wrists into a unified representation. For these frames we then
calculate feature vectors that represent the characteristics of
the performed activities in a compact way. We employ a
feature learning approach based on Restricted Bolzman Ma-
chines (RBM) [20], which has been demonstrated as being
very effective for activity recognition tasks [29]. Following
the original approach, we employ 900 hidden units to match
the input dimensionality (see below). For our climb detec-
tion procedure we down-sample the accelerometer data to
30Hz. Cross-validation experiments suggest that this has no
adverse effect on the overall effectiveness while at the same
time greatly alleviating requirements on the sample sets re-
quired for robust RBM training.
Feature vectors are then fed into a statistical classification
system that discriminates climbing from non-climbing on a
per-frame basis. We have evaluated a number of classifica-
tion approaches and found that logistic regression works best
for climb detection. Finally, the sequences of predicted activ-
ity labels undergo temporal smoothing for outlier elimination,
resulting in effective segmentation. Figure 4 summarises the
climb detection procedure.
Move Segmentation
Even the most complex climbing activities essentially consist
of sequences of atomic movement units. These moves are
defined as:
Continuous limb movements that are temporally sur-
rounded by pauses, i.e., static episodes with no signif-
icant displacement of the particular limb of interest.
Consequently, quality analysis of climbing performance is
typically based on assessments of individual moves.
ClimbAX follows the general approach of move-based anal-
ysis. After climbing sessions have been detected (cf. previ-
ous section), we segment moves on a per-limb basis, which
is important for the generation of detailed assessment infor-
mation. Although the aforementioned definition of moves
suggests a straightforward implementation through detecting
smooth sensor displacement trajectories, there are two things
to consider when assessing real-world climbing activities: i)
Typically moves between holds require hand adjustments to
reach a stable and comfortable position. Such adjustments
add “jitter” to the beginning and end of the actual reaching
Figure 4. Overview of climb detection procedure: Sliding window frame
extraction and feature learning (using Restricted Bolzman Machine
RBM) for capturing characteristics of movement patterns, which are
classified using statistical classification backend.
movements; ii) Moves can also correspond to the turning of
the hand on a single hold without any reaching movement in-
volved, e.g., for repositioning to rest more comfortably or to
prepare for the next (reaching) move.
Taking these considerations into account our move detection
focuses on segmenting hands being on holds, which is charac-
terised by low energy values of the acceleration signals, inter-
rupted by temporally short high energy episodes. The latter
involves a hand moving to a hold, it’s adjustment and other
climbing activities such as clipping the rope (e.g., for sport
climbing; see ii in Figure1(b)). We calculate short-term ener-
gies on raw acceleration data using the same sliding window
procedure that has been applied for climb detection (previous
section). Algorithm 1 summarises the move detection proce-
dure.
Assessment
Quality assessment of climbing as it is performed by profes-
sional coaches is —across its sub-disciplines (Figure 1(b))—
based on a move-specific analysis of certain key criteria that
characterise a set of commonly accepted core skills every
climber needs to possess and develop [28]:
Power the ability to transfer isometric strength into a move.
Holds that are further apart will require a climber to be
more powerful to transition between them.
Control the ability to transition smoothly between holds.
Often a climber is required to shift their centre of mass to
enable a hold transition to be made, which requires both
core body strength and balance. Poor control will result in
jerky limb movements whereas good control corresponds
to smooth transitions between stances.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
239
Algorithm 1 Automatic Move Detection in ClimbAX
Input: limb index l (left or right); energy threshold t; climb
segment s, frame length F
Output: moves M = {m
i
(s)} for given climb segment s
procedure DETECTMOVES(s, l, t)
M = Initialise moves set
for all Frames {f } do Sliding window procedure
Calculate short term energy:
E
f
=
P
F
i=1
f
x
(i)
2
+ f
y
(i)
2
+ f
z
(i)
2
1/2
if E
f
> t then Energy thresholding
M = M
S
f
else
continue
end if
end for
Perform median smoothing Outlier elimination
end procedure
Stability the ability to remain composed while holding
onto holds. Small or sloping holds are difficult to grip typ-
ically resulting in poor stability as postural or finger repo-
sitions are required to maintain a stance on a hold.
Speed defined as timing observation. In most cases, com-
pleting a climb in the shortest possible time is desirable.
We aim for replicating expert assessments by measuring the
aforementioned core skills in the climbing episodes extracted
from the sensor signals.
Intuitively, the power of a climber corresponds to the peak
(physical) work they can perform over time, which has been
used to assess a climber’s arm power in a well controlled ex-
periment in [12]. This immediate measure of the arm’s dis-
placement over time, however, fails to capture the context of
the move performed, i.e. the perceived quality of the holds
and footrests involved in a climbing sequence. This context
is nevertheless crucial to gain insights about a climber’s abil-
ities. Even a climber with very little power will be able to
perform a long reaching and quick move from good quality
holds, while the same climber will struggle with small holds
that are difficult to grab.
A low quality hold induces high intensity tremors as much
strength is required to pull or hang from it. The signal cap-
tured from this hand will therefore exhibit a higher signal
energy compared to a good quality hold. In order to assess
power P for a climb that involves i moves, we measure the re-
lationship between the signal energy E
i
m
of the moving hand
to the signal energy E
i
h
of the hand residing on a hold during
a move:
p
i
=
E
i
m
E
i
h
(1)
P =
max ({p
i
}) (2)
Coaching guides describe Control as the smoothness of hand
movements during hold transitions [18, 21]. Intuitively a
0.5s 1s
0
0.5
1
1.5
2
magn. [g]
Low control
0.5s 1s
0
0.5
1
1.5
2
magn. [g]
High control
Figure 5. Two example moves demonstrating low and high control. The
left plot shows the motion magnitude (gravity removed) from a climber
with a low score for control. The right plot shows the same move by the
climbing with the highest estimated control.
climber that shows good control has a great level of coordi-
nation, good timing and moves efficiently between holds. A
controlled hold transition corresponds to a smooth movement
of the hand, without hesitation, that precisely reaches the op-
timal hand position on the target hold. Poor control often
results in over-shooting beyond the hold, hitting the wall dur-
ing the transition, and high impact forces on the target hold
due to imprecision (see Figure 5).
Control C can thus be characterised as the ratio of energy in
short bursts (impacts) against energy in the long run (smooth
motion) captured from the moving arm.
c
i
= max
e
s
t
e
l
t
tT
i
!
(3)
C = mean (c
i
) , (4)
where c
i
is the control of move i (over time T
i
), while e
s
t
, and
e
l
t
are short-term signal energies calculated using a sliding
window with length t
s
and t
l
respectively (t
l
t
s
).
Stability in climbing is a measure for how well attached the
hands remain to the hold while not engaged in a hold transi-
tion. Poor stability, i.e., unnecessary movements of the hand
while on a hold, is most commonly caused by a combination
of poor flexibility and core body strength. These unnecessary
movements usually correspond to sharp changes in acceler-
ation when, e.g., the hand position on the hold is adjusted.
Stability S for a climb is therefore inversely proportional to
the variance of the first derivative of motion magnitude (jerk)
while the hand is not moving:
S = std
m
h
t
1
, (5)
where m
h
is the motion magnitude of each hand on hold.
Coaches use the Speed of a climber to asses both their route
reading ability as well as their fatigue. While there are many
ways to define speed (e.g., time taken to ascend a route, or
time between limb movements) we chose to measure speed
V as the number of moves per second. This methods is thus
insensitive to route length and can be directly derived from
the climb and move segmentation outputs.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
240
ClimbAX calculates estimates of these core skills for every
detected move and combines the values into a 4-dimensional
skill representation s = {P, C, S, V } R
4
. In doing so we
effectively translate continuous accelerometry data collected
by the sensing platform worn on both hands of the climber
into sequences of core skill values, which is the basis for
both individual and comparative assessment, as well as for
progress tracking all at a great level of detail, which repli-
cates and translates to best practice in existing manual assess-
ment.
EXPERIMENTAL EVALUATION
Datasets
Two different datasets were collected in order to evaluate: i)
the climb segmentation; ii) the move segmentation; and iii)
the automated skill assessment.
The first dataset (sport climbing) consists of a total of 42
climbs recorded from 6 participants at two different indoor
climbing walls (i, and iii in Figure 7). Participants were asked
to wear a set of sensors for the duration of their visit to the
climbing wall and to go about their regular climbing activi-
ties without any specified protocol. After their climbing ses-
sion participants were asked to produce a diary containing
the exact start and end times of each climb. A climb here is
defined as the moment the subject starts climbing until they
are back on the ground, i.e., it may contain resting and falls.
Crucially the data recorded is not limited to climbing activ-
ities but contains other activities such as belaying, walking
around, resting, etc.
The second dataset (competition) was collected during a lo-
cal bouldering competition, where a total of 47 subjects per-
formed a single climbing problem, which was part of the of-
ficial competition set (purple holds in Figure 6). The route
was set up with the particular needs of a performance evalu-
ation in mind. Care was taken so that it contains moves that
require both control and power, without favouring one partic-
ular skill set or side of the body. Participants were recruited
among all competitors with no particular preference, result-
ing in a representative sample of the audience for such com-
petitions. Based on video recordings the recorded data was
annotated for climbs and the exact sequence of moves per-
formed by each participant. In addition to the recordings, the
competition results for the majority of the participants were
also collected. Both datasets are summarised in Table 1.
Dataset Participants Climbs Moves Scores
Sport climbing 6 42
Competition 47 47 770 40
Total 53 89 770 40
Table 1. Summary of (annotated) data collected in 2 different studies.
Segmentation of climbing episodes
In order to evaluate the performance of the climb detection
described in this work a 10-fold cross validation was per-
formed on the combination of both the sport climbing and
the competition dataset. For each dataset, frames of 5-second
length are extracted with a shift of 1 second. The identity
of each frame is decided based on a majority vote based on
Figure 6. A subject on the route climbed by all participants in the
competition dataset (purple holds). Increasing numbers indicate the in-
tended sequence of holds (h for hands and f for feet) although some vari-
ation in solutions was observed.
Figure 7. ClimbAX: The locations used for data collection: i) Indoor
sports climbing wall. ii) Indoor bouldering wall under competition set-
tings. iii) Indoor sports climbing wall with large overhang.
the ground truth annotations. This set of frames is then split
into 10 partitions, each containing a continuous segment of
the data (with respect to time), which is retained throughout
all experiments. An RBM with Gaussian visible units and bi-
nary hidden units is trained for 250 epochs for each fold. For
each frame, the activation probabilities of the hidden units are
retained as feature representation.
Three different classifiers were trained based on the features
extracted by the RBM: i) k-nearest neighbour (k = 1); ii) de-
cision trees (c4.5); and iii) standard logistic regression. Re-
sults are reported in Table 2. After obtaining the results for
each frame independently it is straight-forward to apply tem-
poral smoothing based on a window of n samples and a ham-
ming window. Using a simple threshold to detect a climbing
episode heavily improves the recognition results. Figure 8
illustrates ROC curves for the different classifiers after tem-
poral smoothing is applied (based on a 50-sample window).
Logistic regression on the raw, 900-dimensional feature rep-
resentation clearly outperforms all other classifiers investi-
gated.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
241
Method Precision Recall Specificity
c45* 0.43 0.64 0.81
knn* 0.66 0.78 0.91
logR 0.79 0.71 0.96
PCA+logR* 0.80 0.66 0.96
Table 2. Performance of climb detection using different classifiers on
raw prediction results (no temporal smoothing).
Dataset Precision Recall Specificity
Sport climbing 0.85 0.88 0.96
Competition 0.88 0.86 0.98
Overall 0.87 0.87 0.97
Table 3. Performance of climb detection using ‘logR’ after temporal
smoothing. The Sport Climbing dataset contains approx. 17% climbing
activity along with different activities typical for a visit to a climbing
centre.
Table 3 illustrates the best segmentation results for the differ-
ent datasets using logistic regression. Overall the results im-
prove dramatically if temporal smoothing is employed with
a precision of 0.87 and a recall of 0.87. The results for the
Sport Climbing dataset are particularly interesting as they in-
clude plenty of activities unrelated to climbing. This dataset
was captured during typical visits to a climbing centre and
includes activities such was warming up, stretching, drink-
ing coffee, and walking among others. Some activities that
are similar to climbing activity are included as well, such as
rope handling and belaying. Overall climbing constitutes just
17% of this set, yet it can still be detected very reliably with
a specificity of 0.96.
Segmentation of moves
Based on the extracted climbing episodes from the competi-
tion dataset we apply the process described in this work to
extract moves, separately for each limb. Each move is treated
as an event, and is deemed detected if it overlaps with an au-
tomatically extracted move. Overall this results in a precision
of 0.79 and a recall of 0.82. The imprecision of the method
is largely due to the boundaries extracted by the climb de-
tection, which may exclude moves at the very start and end
of a climb. These boundary conditions have a significant
impact on the performance figures since the short climbing
sequences in this dataset just contain approx. 10 individual
moves per hand (see Figure 6). However, our results indicate
that the extracted moves still adequately reflect the climbers’
overall skill.
Assessment parameter evaluation
Based on the extracted climbing episodes along with their
segmented moves, one set of performance attributes (power,
stability, control and speed) is estimated for each climber
using the process described above. The competition scores
recorded in the competition dataset effectively correspond to
an objective, unbiased estimate of a participants climbing
ability. Out of the 47 participants, 40 handed in a scoring
sheet, which provide the basis for the evaluation of a simple
linear model. In this experiment, a linear regression is fitted
in a leave-one-climber-out cross-validation and used to pre-
dict competition scores based on the performance attributes.
0 0.1 0.2 0.3 0.4
0
0.2
0.4
0.6
0.8
1
false positive rate
true positive rate
ROC curves for different classifiers
knn
c45
logR
PCA−logR
Figure 8. ROC curves of different classifiers for climb detection after
temporal smoothing. Logistic Regression on the raw features (‘logR’)
clearly outperforms other classifiers. Its performance remains compa-
rable to KNN if the dimensionality of the features is reduced using PCA
to 100 dimensions.
The scatter plot in Figure 9 illustrates the results. The pre-
dicted scores show an overall positive correlation to the re-
corded competition scores of 0.74, indicating that our per-
formance attributes are suitable to capture some elements of
climbing skill. This is an extremely encouraging result, as the
performance of a climber during a competition is influenced
by many things a body-worn sensing system is incapable of
measuring (such as mood, form, etc.). Furthermore, since just
a single climb is observed from each participant, long term
characteristics such as (power) endurance and tiredness can
not be observed.
Another parameter that has strong implications on climbing
style is that of body-weight. Remaining on the wall, even
on very difficult and small holds, requires less strength for a
very lean climber. We believe that the route we set for this
experiment favoured lean climbers with a transition on a dif-
ficult hold (hold h9 in Figure 6). Inspired by this insight we
performed an additional experiment in which climbers with a
body-mass index (bmi) of less than 20 are removed from the
set. Following the same approach as for the last experiment,
the performance improves significantly with an overall corre-
lation of 0.84. A scatter plot illustrating the performance of
this reduced set is illustrated in Figure 10.
RELATED WORK
Current best practice for the assessment of climbing activi-
ties corresponds to manual observation and judgment, typi-
cally performed by an experienced coach. While such expert
assessments work well for elite climbers, practical resource
limitations prevent generalisation to the large number of ama-
teur climbers. The desire for automated climbing assessment
served as the motivation for the development of the ClimbAX
system presented in this paper.
Monitoring general sports activities using ubiquitous comput-
ing technology has become very popular in the recent past.
The proliferation of inexpensive, miniaturised sensing hard-
ware together with the availability of sufficient computational
power in mobile devices has lead to a wealth of applications
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
242
100 150 200 250 300 350
100
150
200
250
300
350
competition score
predicted score
prediction of competition scores (overall)
Figure 9. Scatter plot of climbers’ performance in the competition, il-
lustrating the correlation between predicted scores and the ground truth
(0.76). The estimated performance parameters of each climb s R
4
are
used to train a linear model in a leave-one-out cross-validation.
100 150 200 250 300 350
100
150
200
250
300
350
competition score
predicted score
prediction of competition scores (bmi > 20)
Figure 10. Prediction performance when climbers with a bmi of less
than 20 are removed from the set. The prediction shows a correlation to
ground truth of 0.84.
[2]. Apart from logging sports activities a few systems have
also focused on assessments of their qualities. To name but
a few examples, Fothergill et al. developed an automatic
coaching system for rowers [16], Ahmadi and colleagues ex-
plored the use of wearable computing for skill assessment
in tennis [1], M
¨
oller et al. described skill assessment in fit-
ness exercises using a mobile phone [25], and Grober instru-
mented a golf club with accelerometers to analyse the quality
of golf swings [19].
Hardly any approaches have so far been published that are
related to the automatic analysis of climbing activities. No-
table exceptions are the exploration of body-attached sensors
as a means for movement analysis in rock climbers [32], and
the use of ear-mounted accelerometers for climber perfor-
mance monitoring [28]. However, both studies have either
focused on the exploration of the general feasibility of wear-
able climbing assessment, or targeted very specific aspects of
climbing activities. In contrast, our work goes much further
by developing a complete framework for generic skill assess-
ment in climbing activities.
Activity recognition underlying the presented climbing as-
sessment is closely related to gesture recognition using wear-
able computing techniques, which is one of the major re-
search fields within the ubiquitous and wearable computing
community [30]. A large variety of applications has been
explored, ranging from analysing activities of daily living,
health-related aspects, or work-related activities [3, 37]. A
wealth of analysis techniques have been employed, whereas
the majority of them focus on discriminating the activities of
interest rather than assessing their quality.
DISCUSSION
Climbing has become very popular and is now being enjoyed
by a large population who value it as a sociable leisure ac-
tivity that combines physical activities with outdoor experi-
ences in a unique way. Similar to other sports, climbing re-
quires physical fitness and coordination, and progression can
only be achieved through repetitive and dedicated practicing.
Elite climbers reach (and maintain) their expertise with the
support of individualised coaching. Such coaching specifi-
cally targets the improvement of individual weaknesses that
are identified by experts who continuously analyse their per-
formance. Unfortunately, such expert coaching and perfor-
mance assessment is not available for most climbers at the
amateur level. As a consequence and especially in the light
of the complexity of climbing, many amateurs lose motiva-
tion by not making enough progress in developing their skills
or even put their health on jeopardy through inappropriate or
dangerous climbing.
We have embarked on developing an automatic assessment
system that analyses the quality of climbing ClimbAX. Ulti-
mately such a system represents an important building block
for a digital, personal climbing coach that replicates individ-
ualised expert assessment of climbing skills as it is currently
conducted by human coaches. In this paper we presented a
body-worn sensing system and explored analysis techniques
that effectively segment and quantify measures relating to
climbing ability. With the assistance of coaches and sport
science literature, four core parameters were designed that
are relevant for climbing skills: power, control, stability and
speed.
We have demonstrated that an automatic analysis approach
based on the combined evaluation of aforementioned core
climbing skills correlates to scores achieved under compe-
tition conditions. This comparison is, however, limited when
used for either very good climbers or absolute beginners. In
the case of beginners, not enough data was captured as often
the climber fell from the route in the first few moves. In the
case of very the elite climbers, the route was not significantly
hard enough to test their ability. Our results indicate that
climbers with lean body-shape were favoured by the route set
for our experiments with much improved results upon their
removal from the assessment.
While our results are encouraging, they are just based on
a single climb per participant. Crucial aspects such as en-
durance (defined as resilience to fatigue) are inaccessible to
the system and a considerable amount of work necessary until
am automatic, personal climbing coach becomes reality.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
243
FUTURE WORK
This work explores the automatic assessment of climbing
ability, with the aim to provide a basis for a (semi-) auto-
mated, personalised coaching system. However, the transi-
tion from raw performance attributes towards individualised
training recommendations is not explored. Of particular inter-
est here is to investigate if automated training recommenda-
tions are beneficial for a climber’s progression and how this
benefit compares to that of a dedicated professional coach,
which will be explored in future studies.
ACKNOLEDGEMENTS
Contributions to photography from Alpine Exposures. We
would like to acknowledge Climb Newcastle Bouldering Wall
and Gravity Climbing Centre. Parts of this work have been
funded by the RCUK Research Hub on Social Inclusion
through the Digital Economy (SiDE).
REFERENCES
1. A. Ahmadi, D. Rowlands, and D. A. James. Towards a wearable device
for skill assessment and skill acquisition of a tennis player during the
first serve. Sports Technology, 2(3-4):129–136, Feb. 2010.
2. D. Andre and D. L. Wolf. Recent advances in free-living physical
activity monitoring: a review. Journal of Diabetes Science and
Technology, 1(5):760–7, Sept. 2007.
3. L. Atallah and G.-Z. Yang. The use of pervasive sensing for behaviour
profiling a survey. Pervasive and Mobile Computing, 5:447–464, 2009.
4. M. B
¨
achlin, K. F
¨
orster, and G. Tr
¨
oster. Swimmaster: A wearable
assistant for swimmer. In Proc. Int. Conf. Ubiquitous Comp.
(UbiComp), 2009.
5. D. T. Barry, T. Hill, and D. Im. Muscle fatigue measured with evoked
muscle vibrations. Muscle & Nerve, 15(3):303–309, 1992.
6. Beastmaker. bit.ly/YvAzlR. accessed: March 11th, 2013.
7. L. Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Z. Yu. Sensor-Based
Activity Recognition. IEEE Trans on Systems, Man, and Cybernetics
Part C: Applications and Reviews, 99, 2012.
8. Climbcoach. www.climbcoach.org. accessed: March 11th, 2013.
9. Club alpino italiano. www.cai.it. accessed: March 11th, 2013.
10. W. Coldwell. Indoor climbing: the rise of bouldering-only centres. The
Guardian Online Edition, bit.ly/MWPd1m, 28th June 2012.
accessed: March 11th, 2013.
11. P. Cordier, M. M. France, J. Pailhous, and P. Bolon. Entropy as a global
variable of the learning process. Human Movement Science,
13(6):745–763, 1994.
12. N. Draper, T. Dickson, G. Blackwell, S. Priestley, S. Fryer,
H. Marshall, J. Shearman, M. Hamlin, D. Winter, and G. Ellis.
Sport-specific power assessment for rock climbing. The Journal of
Sports Medicine and Physical Fitness, 51(3):417–425, Sept. 2011.
13. N. Draper, T. Dickson, S. Fryer, G. Blackwell, D. Winter, C. Scarrott,
and G. Ellis. Plasma cortisol concentrations and perceived anxiety in
response to on-sight rock climbing. Int. Journal of Sports Medicine,
33(1):13–7, 2012.
14. P. H. Fentem. Benefits of Exercise in Health and Disease. British
Medical Journal, 308:1291–1295, 1994.
15. Fitbit. www.fitbit.com. accessed: March 11th, 2013.
16. S. Fothergill, R. Harle, and S. Holden. Modeling the model athlete:
Automatic coaching of rowing technique. In Proc. Joint IAPR Int.
Workshop on Structural, Syntactic, and Statistical Pattern Recognition
(SSPR & SPR), 2008.
17. F. K. Fuss and G. Niegl. Instrumented climbing holds and performance
analysis in sport climbing. Sports Technology, 1(6):301–313, Mar.
2009.
18. A. Fyffe and I. Peter. The Handbook of Climbing. Pelham Books, 1997.
19. R. D. Grober. An Accelerometer Based Instrumentation of the Golf
Club: Measurement and Signal Analysis. Arxiv preprint
arXiv:1001.0956, 2009.
20. G. E. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for
deep belief nets. Neural computation, 18(7):1527–1554, 2006.
21. E. J. Horst. Training for Climbing: The Definitive Guide to Improving
Your Performance. Falcon, 2nd edition, 2008.
22. IOC announces new events for Sochi 2014, shortlisted sports for 2020.
Olympic.org, bit.ly/WPhBGU, 4th July 2011. accessed: March
11th, 2013.
23. E. Keogh, S. Chu, D. Hart, and M. Pazzani. Segmenting time series: A
survey and novel approach. Data mining in time series databases,
57:1–22, 2004.
24. C. M. Mermier, R. A. Robergs, S. M. McMinn, and V. H. Heyward.
Energy expenditure and physiological responses during indoor rock
climbing. British journal of sports medicine, 31(3):224–228, 1997.
25. A. M
¨
oller, L. Roalter, S. Diewald, M. Kranz, N. Hammerla, P. Olivier,
and T. Pl
¨
otz. GymSkill: A Personal Trainer for Physical Exercises. In
Proc. Int. Conf. Pervasive Computing and Communications (PerCom),
2012.
26. A. Murgia, P. J. Kyberd, P. H. Chappell, and C. M. Light. Marker
placement to describe the wrist movements during activities of daily
living in cyclical tasks. Clinical Biomechanics, 19(3):248–254, 2004.
27. Nike fuel.
www.nike.com/us/en_us/c/nikeplus-fuelband.
accessed: March 11th, 2013.
28. J. Pansiot, R. C. King, D. G. McIlwraith, and B. P. L. Lo. ClimBSN:
Climber performance monitoring with BSN. In Proc. 5th Int. Summer
School and Symposium on Medical Devices and Biosensors, 2008.
29. T. Pl
¨
otz, N. Hammerla, and P. Olivier. Feature Learning for Activity
Recognition in Ubiquitous Computing. In Proc. Int. Joint Conf. on Art.
Intelligence (IJCAI), 2011.
30. S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, D. Howard, K. Meijer,
and R. Crompton. Activity identification using body-mounted sensors
a review of classification techniques. Physiological Measurement,
30(4):1 33, 2009.
31. F. Quaine, L. Martin, and J. Blanchi. Effect of a leg movement on the
organisation of the forces at the holds in a climbing position 3-D kinetic
analysis. Human Movement Science, 16(2-3):337–346, 1997.
32. T. Schmid, R. Shea, J. Friedman, and M. B. Srivastava. Movement
analysis in rock-climbers. In Proc. Int. Conf. Information Proc. in
Sensor Networks (IPSN), 2007.
33. F. Sibella, I. Frosio, F. Schena, and N. Borghese. 3d analysis of the
body center of mass in rock climbing. Human Movement Science,
26(6):841 852, 2007.
34. M. Testa, L. Martin, and B. Deb
ˆ
u. Effects of the type of holds and
movement amplitude on postural control associated with a climbing
task. Gait & posture, 9(1):57–64, 1999.
35. P. Treffner and M. Turvey. Symmetry, broken symmetry, and
handedness in bimanual coordination dynamics. Experimental Brain
Research, 107(3):463–478, 1996.
36. M. F. Twight and J. Martin. The Extreme Alpinism: Climbing Light,
Fast, and High. The Mountaineers Books, 1999.
37. J. A. Ward, P. Lukowicz, G. Tr
¨
oster, and T. E. Starner. Activity
recognition of assembly tasks using body-worn microphones and
accelerometers. IEEE Trans. Pattern Analysis and Machine Intelligence
(TPAMI), 28(10):1553–67, Oct. 2006.
38. D. White and P. Olsen. A time motion analysis of bouldering style
competitive rock climbing. The Journal of Strength & Conditioning,
24(5):1356–1360, 2010.
39. Openmovement sensing platform. www.openmovement.co.uk.
accessed: March 11th, 2013.
40. G. Yogev, M. Plotnik, C. Peretz, N. Giladi, and J. M. Hausdorff. Gait
asymmetry in patients with parkinson’s disease and elderly fallers:
when does the bilateral coordination of gait require attention?
Experimental brain research, 177(3):336–346, 2007.
Session: Sport and Fitness
UbiComp’13, September 8–12, 2013, Zurich, Switzerland
244
... HAR has become a mature research field and the community is moving on from developing systems that help explore What? and When? something of interest is happening, towards How (well)? it is being performed. This emerging field is referred to as quality or skill assessment and the first systems have been developed for automated assessments in various sports thereby providing insights into an athlete's skill, e.g., [26,31]. In addition, skill assessment systems have been developed for training procedures in, e.g., medical setting where students learn and master surgical skills with automatically generated feedback, e.g., [38,47]. ...
... These are either pre-defined by domain experts, such as certain movement parameters, or cover more specific signal parameters such as smoothness or energy. With pre-defined parameters, typically shallow definitions of skill are used that measure relevant parameters on fixed length temporal contexts (e.g., [26,42]). For generalised skill assessment, generic signal representations are used that are not limited to specific domains. ...
... For example, Ladha et. al. [26] presented a system that measured the four key parameters of a climber's movements: i) power; ii) control; iii) stability; and iv) speed. The actual assessment through such (or other) key parameters is then either based on regression approaches that predict concrete values-either of the key parameters or, derived from it, of quality scores as human judges would provide them-or on translating the problem to a conventional classification task where the classes are defined by, e.g., levels of expertise. ...
Article
Full-text available
Human activity recognition is progressing from automatically determining what a person is doing and when, to additionally analyzing the quality of these activities—typically referred to as skill assessment. In this chapter, we propose a new framework for skill assessment that generalizes across application domains and can be deployed for near-real-time applications. It is based on the notion of repeatability of activities defining skill. The analysis is based on two subsequent classification steps that analyze (1) movements or activities and (2) their qualities, that is, the actual skills of a human performing them. The first classifier is trained in either a supervised or unsupervised manner and provides confidence scores, which are then used for assessing skills. We evaluate the proposed method in two scenarios: gymnastics and surgical skill training of medical students. We demonstrate both the overall effectiveness and efficiency of the generalized assessment method, especially compared to previous work.
... Furthermore, there is currently a high scientific interest in bouldering. Machine learning has been applied to classify different motion modes including gripping [16] and to detect which route is taken [17]. They try to create wearables that rate climbers' performance or analyse routes popularity [16], [17]. ...
... Machine learning has been applied to classify different motion modes including gripping [16] and to detect which route is taken [17]. They try to create wearables that rate climbers' performance or analyse routes popularity [16], [17]. ...
... We correct this error growth with the event-domain knowledge that athletes grip at hold positions. We follow our methodology presented in Section II to utilize the available knowledge: 1) We detect the event of gripping based on the IMU data using the method of [16]. This detector detects low acceleration phases in a moving window approach. ...
Preprint
Full-text available
State estimation can significantly benefit from prior knowledge about a system’s dynamics and state. In this paper, we investigate a special class of prior knowledge: Events that correspond to a subset of the state space. This class of knowledge was introduced in pedestrian activity classification to improve the position estimation. We argue that the methodology can be generalized and applied to other applications in human motion tracking, in which the same class of knowledge is available. We apply this methodology to estimate the pose of climbers using inertial sensors and previously measured route maps. For our evaluation, we collected an open source dataset with 27 participants, including IMU data and ground truth positions of the hands. We detect gripping holds (as events), estimate the transition between holds in a least squares optimizer and use a particle filter to deploy the route map constraints (as state subset). In this scenario, our approach achieves a position median of 0.133m and thus demonstrates its possible effectiveness for this application class.
... We will define activity as movements of a person at some moment. On the one hand, most of the articles are dealing with solutions where sensors are positioned on the body of the climber: for example, Kosmalla et al. in [24] and Ladha et al. in [25] employed wrist-worn inertia measurement units (IMUs) to distinguish climbing from background activities. Five IMUs sensors are placed on the bodies of climbers by Seifert et al. in [39] to identify clusters of the patterns that represent the movements. ...
... We created a binary classification problem for each window, where class 1 is "rope pulling", and class 0 is "not rope pulling". As in [25], we generated 60 features for every window and compared the performance of several machine learning models to solve this classification task. We varied the number of strides between windows in order to find the best performing one, i.e., with the best accuracy in identifying the exact time interval of the rope pulling activity. ...
Conference Paper
Full-text available
Sport climbing is becoming more and more popular, even among non-specialists. While new routes are built each year, both indoor and outdoor, there is no effective tool for supporting climbers to choose the most appropriate routes, either for training or simply enjoying. Route recommendation is hard and risky because a reliable evaluation of the climber’s capabilities, status and subjective difficulty perception is necessary. This can be achieved only with the exploitation of Internet of Things (IoT) sensors for the automatic recording of climbers’ activity. In this research, we want to further extend the still young research subject of activity recognition in sport climbing and combine this with new recommender systems (RSs) techniques for route suggestion. We have developed an initial solution for unobtrusively and automatically detecting climbers’ activities in a gym, and we are now connecting this information with the manual inserted diary data of climbers by means of a mobile application. We present the design and the open research questions for a system that leverages sensor data and explicit feedback to generate a climber’s profile and recommend suitable routes.
... These sensors generate tri-axial acceleration signals when the climber moves the respective body part. The research method in [8,10] requires a climber to wear a pair of accelerometers on wrists, the approach in [5] involves additionally the placement of accelerometers on climber's legs and pelvis. A less invasive approach is proposed in [11], where a single accelerometer is placed on the climber's ear. ...
... In the case of acceleration signals, classification performance can be further tuned by removing the low-frequency component, which is mainly tied to the terrestrial gravity [3]. Previous studies utilized statistical models [5], logistic regression and Restricted Boltzmann Machines [10], or Random Forest, Support Vector Machine, and Naive Bayes classifier [6]. Some research [4,6] reported promising results of convolutional neural networks. ...
... However, most previous studies focused on survey systems for the climber alone. Among others they relied on video based [5,6] or sensor driven [7][8][9][10] systems. The latter ones can also be differentiated between wrist or ear worn systems or even systems integrated into the harness of the climber [11]. ...
Article
Full-text available
Sports climbing has grown as a competitive sport over the last decades. This has leading to an increasing interest in guaranteeing the safety of the climber. In particular, operational errors, caused by the belayer, are one of the major issues leading to severe injuries. The objective of this study is to analyze and predict the severity of a pendulum fall based on the movement information from the belayer alone. Therefore, the impact force served as a reference. It was extracted using an Inertial Measurement Unit (IMU) on the climber. Additionally, another IMU was attached to the belayer, from which several hand-crafted features were explored. As this led to a high dimensional feature space, dimension reduction techniques were required to improve the performance. We were able to predict the impact force with a median error of about 4.96%. Pre-defined windows as well as the applied feature dimension reduction techniques allowed for a meaningful interpretation of the results. The belayer was able to reduce the impact force, which is acting onto the climber, by over 30%. So, a monitoring system in a training center could improve the skills of a belayer and hence alleviate the severity of the injuries.
... Insights from machine learning lead to intelligent training protocols, which can help an athlete run faster, jump higher, and lift heavier weights than before [11,32,41]. Researchers have also utilized machine learning techniques to prevent injuries in athletes [6,8,12] or assess the efficacy of training activities [18,31,44,47,51,78]. Several elite professional sports such as basketball and baseball have already adopted machine learning in their injury management strategies [9]. ...
Article
For the past several decades, machine learning has played an important role in sports science with regard to player performance and result prediction. However, it is still challenging to quantify team-level game performance because there is no strong ground truth. Thus, a team cannot receive feedback in a standardized way. The aim of this study was twofold. First, we designed a metric called LAX-Score to quantify a collegiate lacrosse team's athletic performance. Next, we explored the relationship between our proposed metric and practice sensing features for performance enhancement. To derive the metric, we utilized feature selection and weighted regression. Then, the proposed metric was statistically validated on over 700 games from the last three seasons of NCAA Division I women's lacrosse. We also explored our biometric sensing dataset obtained from a collegiate team's athletes over the course of a season. We then identified the practice features that are most correlated with high-performance games. Our results indicate that LAX-Score provides insight into athletic performance beyond wins and losses. Moreover, though COVID-19 has stalled implementation, the collegiate team studied applied our feature outcomes to their practices, and the initial results look promising with regard to better performance.
... The device weighs 5.8 g and has an accelerometer, a gyroscope, a temperature sensor, and a pulse sensor. Lapinski et al. (5) evaluated professional baseball pitchers and batters by using wearable sensor systems, and Ladha et al. (6) proposed a climbing performance analysis system using a watch-like sensing platform that measures acceleration. Kosmalla et al. (7) also proposed a system for climbing using wrist-worn inertia measurement units. ...
Article
Wearable devices with motion sensors, such as accelerometers and gyroscopes, are expected to become popular. There is a lot of research on recognizing gestures using data obtained from motion sensors. A gesture is a one-off motion and its trajectory, i.e., the waveform of the gesture part, is considered to be important. After segmenting the data, gestures are recognized using a template matching method. However, there is no method to accurately detect when a specific action is performed during a gesture. Although it is possible for a player in a game to perform a throwing motion by the user performing a pitching action, it is difficult to reflect a particular moment, such as the user’s release point, in the player. The authors previously proposed a method using a wrist-worn sensor for determining the moment of touching a card in competitive karuta (a Japanese card game) and developed a system that judges the player who took a card first in a competitive karuta match. As reported in this paper, we improved the estimation method to apply our study to a variety of gestures other than those in competitive karuta and propose a method of detecting the timing of a specific action. Our system was evaluated for three types of release points, baseball throws, basketball free throws, and dart throws, with 11 subjects who had an accelerometer and a gyroscope attached to the wrist. The percentage of release point estimation errors of 12 ms or less was determined to be 100% for baseball, 87.6% for basketball, and 91.1% for darts.
Conference Paper
Full-text available
Hệ thống giám sát hành vi người qua wifi/internet được triển khai để dự đoán năm hành động cơ bản bao gồm: ngồi, nằm, đứng, đi bộ và chạy bộ. Bài viết này trình bày giải pháp phân loại hành động dựa trên các ngưỡng đặc trưng gia tốc điển hình theo thời gian thực. Để phát hiện các ngưỡng đặc trưng tín hiệu, nhóm nghiên cứu xây dựng mô hình phân loại hành động ứng dụng thuật toán học máy cây quyết định với đầu vào là các đặc trưng dữ liệu gia tốc ba trục. Từ đó, các hoạt động thường ngày của con người được theo dõi, quan sát từ xa trên điện thoại thông minh và máy chủ dữ liệu qua mạng wifi/internet. Nghiên cứu này hướng đến xây dựng hệ thống giám sát hành vi người trong tòa nhà có hiệu suất cao, giá thành rẻ và hoạt động theo thời gian thực. Kết quả thực nghiệm đạt được độ chính xác hơn 90% là tốt khi gắn cố định thiết bị phân loại trên eo người các tình nguyện viên tham gia thử nghiệm.
Chapter
This chapter is concerned with the use of wearable devices for disabled and extreme sports. These sporting disciplines offer unique challenges for sports scientists and engineers. Disabled athletes often rely on and utilize more specialist equipment than able-bodied athletes. Wearable devices could be particularly useful for monitoring athlete-equipment interactions in disability sport, with a view to improving comfort and performance, while increasing accessibility and reducing injury risks. Equipment also tends to be key for so called “extreme” sports, such as skiing, snowboarding, mountain biking, bicycle motocross, rock climbing, surfing, and white-water kayaking. These sports are often practiced outdoors in remote and challenging environments, with athletes placing heavy demands on themselves and their equipment. Extreme sports also encompass disability sports, like sit skiing and adaptive mountain biking, and the popularity and diversity of such activities is likely to increase with improvements in technology and training, as well as with the support of organizations like the High Fives Foundation (highfivesfoundation.org) and Disability Snowsport, United Kingdom (disabilitysnowsport.org.uk). Within this chapter in these two sporting contexts, wearable devices are broadly associated with those that can be used to monitor the kinetics and kinematics of an athlete and their equipment. This chapter will first consider image-based alternatives and then focus on wearable sensors, in three main sections covering, (1) sports wearables, (2) disability sport and the use of wearables, and (3) extreme sport and the use of wearables, as well as making recommendations for the future.
Article
Full-text available
Research on sensor-based activity recognition has, recently, made significant progress and is attracting growing attention in a number of disciplines and application domains. However, there is a lack of high-level overview on this topic that can inform related communities of the research state of the art. In this paper, we present a comprehensive survey to examine the development and current status of various aspects of sensor-based activity recognition. We first discuss the general rationale and distinctions of vision-based and sensor-based activity recognition. Then, we review the major approaches and methods associated with sensor-based activity monitoring, modeling, and recognition from which strengths and weaknesses of those approaches are highlighted. We make a primary distinction in this paper between data-driven and knowledge-driven approaches, and use this distinction to structure our survey. We also discuss some promising directions for future research.
Conference Paper
Full-text available
We present GymSkill, a personal trainer for ubiquitous monitoring and assessment of physical activity using standard fitness equipment. The system records and analyzes exercises using the sensors of a personal smartphone attached to the gym equipment. Novel fine-grained activity recognition techniques based on pyramidal Principal Component Breakdown Analysis (PCBA) provide a quantitative analysis of the quality of human movements. In addition to overall quality judgments, GymSkill identifies interesting portions of the recorded sensor data and provides suggestions for improving the individual performance, thereby extending existing work. The system was evaluated in a case study where 6 participants performed a variety of exercises on balance boards. GymSkill successfully assessed the quality of the exercises, in agreement with the professional judgment provided by a physician. User feedback suggests that GymSkill has the potential to serve as an effective tool for motivating and supporting lay people to overcome sedentary, unhealthy lifestyles. GymSkill is available in the Android Market as #x2018;VMI Fit #x2019;.
Conference Paper
Full-text available
The recent growth in popularity in sport climbing is partly due to the safe environment provided by indoor climbing walls, particularly for novice climbers. Sport climbing involves a wide range of skills and abilities. The purpose of this paper is to present a wearable sensing platform and an analysis framework for assessing general climbing performance during training. To provide the required freedom of movement, a single miniaturized ear-worn 3D accelerometer-based sensor is used. Independent features derived from the accelerometer data are then translated into climbing-specific measures, such as motion fluidity, strength, as well as endurance. Based on these indices, the overall level of the climber and the associated climbing styles can be quantified.
Article
In three different events (national climbing Championship, sport climbing world cup, and training session), one hold was instrumented with two 3‐D force transducers. Subsequently, the mechanical parameters of climbing were defined and analyzed, and the force vector diagrams visualized for quantification of performance.The more experienced a climber is, the smaller the contact force, the shorter the contact time, the smaller the impulse, the better the smoothness factor, the higher the friction coefficient, the more continuous the movement of the center of pressure (in specific holds), and the smaller the Hausdorff dimension (less chaotic force time graph). The Hausdorff dimension correlates highly with all other parameters and with the appearance of the vector diagrams, and is thus suited to serve as the most important performance parameter. Training improves the mechanical parameters. The measurement and analysis of mechanical parameters and their visualization in terms of force vector diagrams are a useful tool for quantifying the performance of a climber on a specific instrumented hold.
Article
In three different events (national climbing Championship, sport climbing world cup, and training session), one hold was instrumented with two 3-D force transducers. Subsequently, the mechanical parameters of climbing were defined and analyzed, and the force vector diagrams visualized for quantification of performance. The more experienced a climber is, the smaller the contact force, the shorter the contact time, the smaller the impulse, the better the smoothness factor, the higher the friction coefficient, the more continuous the movement of the center of pressure (in specific holds), and the smaller the Hausdorff dimension (less chaotic force time graph). The Hausdorff dimension correlates highly with all other parameters and with the appearance of the vector diagrams, and is thus suited to serve as the most important performance parameter. Training improves the mechanical parameters. The measurement and analysis of mechanical parameters and their visualization in terms of force vector diagrams are a useful tool for quantifying the performance of a climber on a specific instrumented hold. © 2008 John Wiley and Sons Asia Pte Ltd
Article
This paper describes three-dimensional force data collected during postural shifts performed by individuals simulating rock climbing skills. Starting from a quadrupedal vertical posture, six expert climbers had to release their right footholds and maintain the posture for a few seconds. The analysis of the vertical and the horizontal forces (lateral and antero-posterior forces) applied on the holds was performed before, during and after the onset of the voluntary movement. The results show that the vertical and the horizontal force changes were initiated in synchrony at the same hold. Furthermore, the changes in the forces occurred before the release of the leg. Therefore, they were not a response to, but preparatory for postural change. The force variations were characterized by loadings of the vertical forces and by loadings and unloadings of the horizontal forces. This type of force variation on the holds and their timings seemed necessary to create the dynamic conditions for the onset of the voluntary movement and to counteract the perturbations due to this movement, which balanced the climber on the wall.
Article
With the maturity of sensing and pervasive computing techniques, extensive research is being carried out in using different sensing techniques for understanding human behaviour. An introduction to key modalities of pervasive sensing is presented. Behaviour modelling is then highlighted with a focus on probabilistic models. The survey discusses discriminative approaches as well as relevant work on behaviour pattern clustering and variability. The influence of interacting with people and objects in the environment is also discussed. Finally, challenges and new research opportunities are highlighted.
Article
The work reported here is a contribution to the study of a complex motor behavior, viewed globally. The question raised is the nature of the process by which an environment-sensorimotricity coupling is organized. The material is the trajectory of a constrained free climbing task, and the main concept is entropy. The entropy of the climber's trajectory is used to measure the degree of structuring in the successive states of the subject-environment system during the learning of a complex task. It will be shown that the entropy of the trajectory decreases as learning progresses, and that the shape of the entropy curve is a function of the climber's level of expertise. A model of constraint relaxation is proposed to describe the learning process. Then, based on a theory of probabilistic inference, an attemp is made to show that this natural biological process obeys the thermodynamic laws of neural networks.
Conference Paper
In this paper we introduce the concept of a wearable assistant for swimmer, called SwimMaster. The SwimMaster consists of acceleration sensors with micro-controllers and feedback interface modules that swimmer wear while swimming. With four different evaluation studies and a total of 22 subjects we demonstrate the functionality and power of the SwimMaster system. We show how a wide range of swim parameters can be monitored and used for a continuous swim performance evaluation. These parameters include the time per lane, the swimming velocity and the number of strokes per lane. Also swim style specific factors like the body balance and the body rotation are extracted. Finally three feedback modalities are tested and evaluated. With these means we show the ability of the SwimMaster to assist a swimmer in achieving the desired exercise goals by constantly monitoring his/her swim performance and providing the necessary feedback to achieve the desired workout goals.