ArticlePDF Available

Abstract and Figures

Quality assessment in cricket is a complex task that is performed by understanding the combination of individual activities a player is able to perform and by assessing how well these activities are performed. We present a framework for inexpensive and accessible, automated recognition of cricketing shots. By means of body-worn inertial measurement units, movements of batsmen are recorded, which are then analysed using a parallelised, hierarchical recognition system that automatically classifies relevant categories of shots as required for assessing batting quality. Our system then generates meaningful visualisations of key performance parameters, including feet positions, attack/defence, and distribution of shots around the ground. These visualisations are the basis for objective skill assessment thereby focusing on specific personal improvement points as identified through our system. We evaluated our framework through a deployment study where 6 players engaged in batting exercises. Based on the recorded movement data we could automatically identify 20 classes of unique batting shot components with an average F1-score greater than 88%. This analysis is the basis for our detailed analysis of our study participants’ skills. Our system has the potential to rival expensive vision-based systems but at a fraction of the cost.
Content may be subject to copyright.
62
Activity Recognition for ality Assessment of Baing Shots in
Cricket using a Hierarchical Representation
AFTAB KHAN, Toshiba Research Europe Limited, Telecommunications Research Laboratory, UK
JAMES NICHOLSON, PaCT Lab, Department of Psychology, Northumbria University, UK
THOMAS PLÖTZ, School of Interactive Computing, Georgia Institute of Technology, USA
Quality assessment in cricket is a complex task that is performed by understanding the combination of individual activities a
player is able to perform and by assessing how well these activities are performed. We present a framework for inexpensive
and accessible, automated recognition of cricketing shots. By means of body-worn inertial measurement units, movements
of batsmen are recorded, which are then analysed using a parallelised, hierarchical recognition system that automatically
classies relevant categories of shots as required for assessing batting quality. Our system then generates meaningful
visualisations of key performance parameters, including feet positions, attack/defence, and distribution of shots around the
ground. These visualisations are the basis for objective skill assessment thereby focusing on specic personal improvement
points as identied through our system. We evaluated our framework through a deployment study where 6 players engaged
in batting exercises. Based on the recorded movement data we could automatically identify 20 classes of unique batting
shot components with an average F1-score greater than 88%. This analysis is the basis for our detailed analysis of our study
participants’ skills. Our system has the potential to rival expensive vision-based systems but at a fraction of the cost.
CCS Concepts:
Computing methodologies Supervised learning by classication
;
Human-centered computing
Ubiquitous and mobile computing systems and tools;Information visualization;
Additional Key Words and Phrases: Activity Recognition, Skill Assessment, Hierarchical models, Sports
ACMReferenceFormat:
AftabKhan,JamesNicholson,andThomasPlötz.2017.ActivityRecognitionforQualityAssessmentofBattingShotsinCricket
usingaHierarchicalRepresentation.Proc.ACMInteract.Mob.WearableUbiquitousTechnol.1,3,Article62(September2017),
31pages.
%0*http://doi.org/10.1145/3130927
1 INTRODUCTION
Wearable technology-based activity recognition has been of great interest in recent times due to the potential of
automating the quantication of daily events. For example, various techniques have been used to automatically
identify activities of daily living (ADL) – from grooming and bathing based on non-intrusive sensors [
46
] to more
complex and unscripted activities using cameras [
39
]. Wearable activity recognition has also been used to better
Authors’ address: Aftab Khan, Toshiba Telecommunications Research Laboratory, Toshiba Research Europe, 32 Queen Square, Bristol,
UK; email: aftab.khan@toshiba-trel.com; James Nicholson, PaCT Lab, Department of Psychology, Northumbria University, UK; email:
james.nicholson@northumbria.ac.uk; Thomas Plötz, School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA; email:
thomas.ploetz@gatech.edu.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
copies are not made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page.
Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy
otherwise, or republish, to post on servers or to redistribute to lists, requires prior specic permission and/or a fee. Request permissions from
permissions@acm.org.
© 2017 Copyright held by the owner/author(s). Publication rights licensed to Association for Computing Machinery.
2474-9567/2017/9-ART62 $15.00
%0*http://doi.org/10.1145/3130927
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:2 A. Khan et al.
understand and support users with specic health conditions such as Parkinson’s Disease [
1
] and Dementia [
10
],
including for disease state prediction [14].
Activity recognition technology has also been of great interest in sports with applications for both tracking
the performance of individual players [
28
] and teams [
33
,
38
]. Sports are an interesting context for activity
recognition as there are usually several sub-activities, each with their own mechanics, and which may dier
according to various circumstances. As such, activity recognition can help with understanding actions of interest
and analysing sub-mechanics of specic activities to help with the ever-challenging nature of scouting and the
development of promising athletes.
In this paper we focus on cricket, a popular sport that consists of a large number of complex activities. Batting
plays a key role in the overall game, and as such it is important to have the ability of making suitable judgements
regarding the quality of a particular player’s batting prowess. Batting is a complicated construct, however, and
can be broken down into many types of shots that are played (with over 24 shot categories), thus analysing
cricket shots at such a detailed level requires an approach capable of recognising these shots reliably. Therefore,
we begin to address this problem by presenting a hierarchical shot analysis system using wearable accelerometers
to automatically identify various types of shots.
Four sensors equipped with 3-axis accelerometers, gyroscopes and magnetometers are used for each limb in
this framework for detecting these shots. Our hierarchical framework allows us to analyse 5 dierent levels of
batting shot attributes that can directly be linked to a player’s ability. Utilising such a framework would also
allow players and coaches to analyse the batter’s performance, not only by considering the shots played but also
by studying whether the feet movement was correct. For example, a shot (like an o drive) can be played with
or without proper feet movements and this detail can be used to distinguish between a skilled batsman and a
beginner.
One of the main goals of our work is to provide young amateur cricketers with an opportunity to objectively
analyse their batting quality and, using the feedback, eventually assess their skill as compared to a) other players;
b) professional players; but mainly to c) track their performance. Currently, expensive vision-based systems such
as PitchVision can be used by players to assess their game, but these systems require a semi-professional setup
at a training centre in addition to an initial outlay of at least £2,200. While some wealthy cricket organisations
can aord such systems for scouting and developing talent, more modestly endowed cricket organisations in
poorer nations would benet from a low-cost solution. Additionally, young cricketers could greatly benet from
a low-cost automatic system to help them assess their batting sessions for raw improvements by addressing weak
points – e.g., if they missed playing specic shots. From a professional cricketing organisation’s point of view,
ranking young cricketers is a manual and laborious process for coaches and selectors that can often result in
unintentional subjectivity playing a role in the process. A reasonably-priced accelerometer-based system can
facilitate this process by reliably and objectively ranking a large number of new players based on batting quality
and then selecting a subset for further scrutiny by coaches based on the attributes associated with their shots.
In order to make a suitable assessment of the quality of players, it is very important to understand the activities
they are involved in. In an automated system, automatic activity recognition is of paramount importance, therefore
most of the skill assessment frameworks rely on accurate activity recognition systems. Cricket games in this
context pose a new set of challenges due to a complex activity structure where there are a large number of
unique shots that make activity recognition very complicated to achieve. However, without the ability to identify
these shots it is impossible to perform reliable automated batting assessment. Automated systems like the one
presented in this paper can eectively provide a decision support for coaches and young players to easily track
their performance and compare themselves against other players. The ultimate vision is to realise automated
skill ranking of players and eectively selections can be made for various teams based on the results of such a
quality assessment framework.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:3
The focus of this paper is on an exploration to what extent an inexpensive and thus accessible hardware
setup combined with automated sensor data analysis can provide meaningful and objective insights into the
performance of ambitious amateur cricketers. Here we present a framework for recognising cricketing shots
using an IMU-based setup as a precursor to a quality assessment system. We propose a hierarchical representation
of shots that groups activities based on certain aspects of a batting shot such as feet position and shot direction.
Multiple visualisation techniques –well-accepted and already used for cricket analysis, for example, on television
broadcasting– are then used to show the results of the activity recognition model. Aiming for a general proof-
of-concept we evaluate the eectiveness of the two main categories of classication backends (discriminative
modelling, and instance based learning) for assessing the quality of cricket shots. The visualisations we create
show the types of shot the player uses in a session across a possible range of over 24 unique shot categories that
have been identied in the sport of cricket [
44
]. These visualisations make it easy to understand the strong and
weak areas of a player’s overall batting session and also provide a method for players to easily compare their
session with others. Through comparison between amateur and professional players, and even between multiple
sessions belonging to the same player over time, we can therefore end up with a measure of batting quality and
performance tracking. In this paper, we present quantitative assessment results that illustrate the value of the
proposed analysis framework. The results were based on a carefully designed case study involving participants
covering the full range of skill sets – from novices to semi-professional. Our participantsâĂŹ shots were recorded
during a series of training sessions and later expert-annotated for automatically identifying the shot types and
shot quality. Our automated assessment system is able to consistently replicate the ground truth assessment to a
very high degree, which is encouraging as these results provide evidence for the targeted proof-of-concept and
essentially render the developed system ready for larger scale deployment. As such this paper, which is the rst of
its kind that successfully tackles automated quality assessment in cricket (shots), represents the foundations for
subsequent, large scale and especially longitudinal quality analysis and tracking studies in amateur cricket. The
availability of such a system has substantial potential for the enormous amateur cricket community worldwide –
many of which living in underprivileged circumstances and thus not having access to professional coaching. An
inexpensive assessment system like the one presented in this paper can help many to gain and maintain skills in
their favourite sport and as such to open doors to healthy and happy lives.
2 BACKGROUND
2.1 Activity Recognition using Wearables in Sports
Automated recognition of relevant activities plays a key role in a number of sports-related application scenarios.
Simple event detection, for example, has been explored in golf. Work by [
17
] used a Hidden Markov Model along
with a 6-D IMU for putt detection with the aim of improving golfers’ training. The method resulted in a detection
rate of 96%, with a sensitivity of 88.8%.
More complicated activity recognition has also been explored by researchers in the eld. For example, activity
recognition has been used in rugby, where a GPS receiver and an accelerometer in a wearable sensor were used
to automatically identify collisions and tackles [
20
]. The device was located between the shoulder blades of
the players, and a combination of Support Vector Machine and Hidden Conditional Random Field models were
selected to learn the relationship between the source and the target data to automatically detect collision events.
The recall and precision of the system was 93
.
3% and 95
.
89% respectively. Other work has looked at recognising
aspects of the game, such as scrums and tackles, using Bayes Network, Random Forest, Multilayer Perceptron
and a Naïve Bayes classiers [
19
]. The Naïve Bayes classier achieved the highest accuracy with 97
.
2%, and the
overall sensitivity was 100% with a precision of 11
.
6%. The hardware used for this work consisted of a location
positioning system and IMUs for the players.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:4 A. Khan et al.
In swimming, SwimMaster [
5
] was able to extract swim parameters – including time per lane, swimming
velocity, and number of strokes per lane – as well as performing a real-time assessment of the performance
using acceleration sensors. The swimmer is then able to use the data to constantly monitor their swimming
performance and provide the necessary feedback to achieve their workout goals.
Various work has looked at activity recognition in snowboarding due to the increasing popularity of the
sport. A simple setup requiring a gyroscope taped to the board and a mobile GPS receiver allowed the system to
recognise turns, placement of the rider, and direction of travel as well as nine other activities [
16
]. They used
a combination of two bespoke algorithms based on the continuous activity recognition approach by [
31
] and
achieved an average accuracy of 90
.
5%. On the other hand, [
11
] used an IMU with a Naïve Bayes classier to
recognise a sequence of events that culminated in one of two tricks with an average accuracy of 91
.
5%. The
same hardware was used to detect skateboarding tricks [
12
], although four dierent classiers were evaluated in
this work: Naïve Bayes, Partial Decision Tree, Support Vector Machine, and
k
-nearest neighbour. NB and SVM
performed best, with an accuracy of 97.8%.
Another system designed to collect sporting data using wearable sensors is ClimbAX [
28
]–awearable sensing
platform that records a climber’s movements and is able to output assessment parameters including stability,
control, power and speed, comparable in standard to ocial expert assessments. [
38
] evaluated a wearable sensing
system to monitor basketball players using multiple inertial measurement units. The system was able to recognise
human movements and classify them into walking, jogging, running or sprinting, as well as identifying shooting
events.
Activity recognition has also been used for improving personal tness. RecoFit was designed to monitor
repetitive exercise activities such as weight training by using wearable inertial sensors [
34
]. In the rst analysis
stage the system discriminates between exercise and non-exercise movements using an L2 Support Vector
Machine. Based on this it then recognises and counts the repetition of activities with an overall accuracy of
96%. Similarly, GymSkill used the sensors from smartphones attached to gym equipment to monitor the user’s
exercises and analysed using a pyramidal Principal Component Breakdown Analysis. Based on the data, the
system then presents suggestions for improving the performance [32].
2.2 The Sport of Cricket
In this paper, we explore activity recognition with regards to batting in cricket, which consists of a large number
of complex activities. Figure 1 illustrates the main aspects of how cricket is played at various levels ranging from
grass-roots to international cricket at Test level. The sport of cricket is arguably the second most popular sport
in the world after soccer based on the viewing population [
45
]; there are an estimated 2-3 billion fans. As an
example, the cricket world cup alone in 2015 was watched by 2
.
2 billion people on television. Some high-prole
leagues such as in India have been reportedly valued at $3
.
6 billion [
3
]. Several such leagues are also played in
other countries including Australia, South Africa, and Pakistan every year.
Cricket is a bat-and-ball game played between two teams of eleven players on a cricket eld, at the centre of
which is a rectangular 22-yard-long pitch. One team, designated as the batting team, attempts to score as many
runs as possible, whilst their opponents eld –that is one of the players throws the ball towards the batsman
and others catch the ball– during which they bowl with an aim to get the batting team out –that is to force
the opposing team into a batting error such as missing the ball and hitting the stumps. A run corresponds to
a batsman (the player holding the bat and trying to hit the ball that was thrown at them by the elding team)
successfully running from their batting position to the opposite end of the pitch (22 yards as mentioned above).
After the batting team is out, the two teams then swap roles. The winning team is the one that scores the most
runs during their batting period. With this in mind, it is obvious that batting performance is a very important
aspect of the game and as such one that players choose to work on improving continuously.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:5
ĚͿ dϮϬ/ͬK/ĐͿ ůƵď
ďͿ :ƵŶŝŽƌĂͿ'ƌĂƐƐƌŽŽƚƐ
ĞͿ dĞƐƚ
Fig. 1. Images of cricket played at various levels including (a) Grass-roots, (b) Junior club cricket, (c) Club cricket, (d)
Twenty20 International/One-day International cricket and (e) Test cricket. All photos public domain.
Cricket matches are typically structured into “innings”, that is each team take turns for batting in which
multiple “overs” (a set of 6 legal bowling deliveries) are involved. The overall goal of the team batting rst is to
score as many runs as possible (whilst the other team tries to get each batsman out, which triggers the end of the
innings or restricts the batting team’s score). In the second innings, the batting team chases the target of the rst
innings’ score, which was set by the team batting rst, whilst the bowling team tries to restrict scoring or get all
of the players out before they reach the target. If the team batting second reaches the target, they win the match.
A summarised list of cricket specic terms used in this paper are shown in Table 1
1
. International cricket has
three formal formats as summarised below.
T20I (Twenty-20 International)
With 20 overs (each comprising 6 legal bowling deliveries) per innings,
this is the shortest format lasting up to 3 hours (there are 2 innings in which each team bats and scores
runs).
ODI (One Day International)
In this format, there are a total of 50 overs per innings, which can last up to
7 hours. Both of the T20I and ODI formats are played in colored clothing.
Test
This is the longest format with a maximum of 4 innings played over 5 days and a minimum of 450 overs
across all days (weather permitting). There are 3 sessions per day with two breaks for lunch and tea. This
is the traditional form of cricket and is played in white clothing. In total there are 10 test playing nations
determined by the International Cricket Council.
All of the above formats require dierent playing strategies in general for example in the shortest format
a more aggressive form of the game is required whilst at the Test level, defensive game is necessary. In some
1http://www.espncricinfo.com/ci/content/story/239756.html
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:6 A. Khan et al.
Table 1. A subset of cricket terminologies as used in this paper
Term Description
Back foot Batsman’s foot that is nearer to the stump
Back foot shot A shot played when the batsman’s weight is on the back foot
Bails A small piece of wood that is placed on top of the stumps
Delivery Bowling act
Dismiss State at which the batsman must cease batting
Fielding (team) the team which is not batting
Front foot Batsman’s foot that is nearer to the bowler
Front foot shot A shot played when the batsman’s weight is on the front foot
Innings A player’s or team’s batting or bowling turn
LBW (leg before wicket) A form of dismissal in which the ball hits the leg of the batsman in front of the stumps
Leg-side For a right-handed batsman, the left side of the pitch
O-side For a right-handed batsman, the right side of the pitch
Out State of the batsman when dismissed
Pitch A rectangular surface in the centre of the ground
Pitch (of the ball) To bounce before reaching the batsman
Runs (1) A single run is scored if a batsman, after hitting the ball, runs the length of the pitch once (2 for running twice)
Stumps One of the three wooden posts
Wicket Interchangeably used for a) stumps (ball hitting the wickets), b) out (bowler takes a wicket), c) pitch (the wicket is good for batting)
cases, mainly depending on the situation, a defensive game can be the best strategy in the shortest formats if
the team loses early wickets. This is to ensure that a batting team plays all of the available overs. Similarly in
test cricket, for example, in the 3rd innings of the match and on the 4th day a team might want to quickly score
runs to set a target for the bowling team in the 4th innings prior to nal session of the day. This will require
an attacking game (with the risk of losing wickets which can result in the end of their innings), however if the
team succeeds in getting a higher score earlier, it provides sucient time to get the batting team out in the 4th
innings and win the match on the 5th day. If the target is too high and impossible to chase in the 4th innings, the
batting team usually employs a defensive strategy and plays for a draw. This strategy, although it seems simple,
requires a sound defensive technique since on the 5th day, the cricket pitch has usually deteriorated with cracks.
The ball can spin o these cracks extraordinarily and/or have uneven bounce o the surface leading to batsmen
getting out (getting caught by a elder after the ball hits the edge of the bat or getting LBW – see below). On rare
occasions, the ball does not even bounce enough for the batsman to judge properly.
There are many factors that should be considered when selecting a shot, such as the feet position relative to
the ball, the angle the ball approaches, the spin and speed of the ball, as well as its current trajectory. Judging all
these factors in a split second and choosing the best kind of shot to respond with contributes to the diculties
associated with batting in cricket. Figure 2 illustrates the most relevant shot categories around the ground with
numbers indicating the indices of these shots colored according to the feet position. Shots in these directions can
be played on the front-foot (in which the batsman moves towards the ball to hit it) or the back-foot (in which the
batsman moves back).
More complex shots (also known as unorthodox shots) can also be played which are very rewarding in terms of
the number of runs but have higher risks associated with them in which the batsman can get out. For example, a
batsman can play a switch-hit in which a right-handed batsman switches the stance to the one normally associated
with left-handed players (and vice-versa) and does this very quickly right before the ball is delivered so that the
bowler is unable to adjust the line/length of the ball. In this case, the eld which is originally set for a right-handed
batsman provides opportunity for the batsman to play scoring shots in the vacant areas of the ground. However,
since the batsman is not in his natural stance, the risk of getting out is higher.
Other examples of risky feet positioning involve a batsmen taking multiple steps to get close to the pitch of
the ball and hitting the ball hard. Getting close to where the ball pitches, allows the batsman to hit a shot for a
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:7
^ƚƌĂŝŐŚƚƌŝǀĞ
KŶƌŝǀĞ
KĨĨƌŝǀĞ
ŽǀĞƌ
ƌŝǀĞ
^ůŽŐ
WƵůů
Ƶƚ
>ĂƚĞƵƚ
>ĞŐ'ůĂŶĐĞ
KĨĨ
^Ƌ͘
Ƶƚ
,ŽŽŬ
ϭ͕Ϯ
ϯ͕ϰ
ϱ͕ϲϳ͕ϴ
ϵ͕ϭϬ
ϭϭ͕ϭϮ
ϭϯ͕ϭϰ
ϭϱ͕ϭϲ
ϭϳ͕ϭϴ
ϭϵ͕ϮϬ
Ϯϭ͕ϮϮ
^Ƌ͘
ƌŝǀĞ
Ϯϯ͕Ϯϰ
΀&ƌŽŶƚ&ŽŽƚ͕ĂĐŬ&ŽŽƚ΁
Fig. 2. Shot types around the ground for a right-handed batsman with front-foot and back-foot shots separately labelled
(numbers indicating the shot indices). Baing direction is le to right.
maximum number of runs (4 along the ground or 6 runs if the ball is hit so hard, it bounces outside the ground
boundary). If the batsman misses the ball, he can get out stumped, in which the wicket-keeper collects the ball in
his gloves and dislodges the bails that are placed on top of the three stumps before the batsman can get back
inside the crease (bounded area close to the stumps in which the batsman is safe). Alternatively, whilst playing
slow bowling, a batsman can initially come slightly forward towards the ball and when he sees that the ball might
not bounce close to him, he moves back (very close to the stumps) allowing the ball to have bounced and spun
enabling the batsman to properly see it and hit it for a run-scoring shot. In this scenario both front and back foot
movements happen resulting in a very complex shot. There can be multiple risks associated with such a shot for
example if the batsman moves back in line with the stumps and the ball hits his legs if he misses then he could
be adjudicated as out which corresponds to one of the mode of dismissals called LBW (leg before wicket). The
batsman can also misjudge the distance whilst moving back and forth and hit the wicket with his feet dislodging
the bails (which is another mode of dismissal) leading to the batsman getting out. Compiling a list of all shot
categories is therefore very dicult. However, a list of the most important shots is shown in Figure 2. This list
serves as the basis for the hierarchical representation and thus the automated assessment as it is developed in
this paper.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:8 A. Khan et al.
2.3 Activity Recognition for Cricket
Activity recognition in cricket is a challenging task due to the complexity of the activities involved. Batting can
be broken down into the many types of shots that are played, with over 24 shot categories encapsulating all the
possibilities. These categories combine the type of swing and the foot positioning that the player uses based
on the direction of the incoming ball. Depending on where the bowler places the ball on its way to the wicket,
batsmen employ dierent tactics to maximise chance of success (scoring runs).
There has been limited previous work looking at activity recognition in cricket. [
6
] proposed a platform for
providing objective feedback to batsmen on their performance using bat-mounted tri-axial accelerometers. The
feedback was based on the velocity and twist of the bat, along with an estimated impact time and therefore
presented the batsman with a measure of strike quality, rather than any detailed analysis of the shot. In this work,
a single shot category of ‘defensive drives’ was used to analyse the ball’s impact when hitting the bat.
On the other hand, [
48
] have looked at determining the dierences between novice and expert batsmen. This
work has focused on the batters performing a batting task in a highly-controlled environment where they were
required to hit various targets using dierent bats from a machine-projected ball. This work did not utilise sensors
but rather relied on expert observation, and as such this method was time-consuming for both participants
and evaluators due to its articial environment which required setting up. Recent work has approached shot
classication in cricket by using motion vectors in video frames with 4 possible shots being recognisable [
18
].
While the proposed method is convenient as it can be done using standard videos from training sessions rather
than with a specialistic setup, the average accuracy of the system was at the lower end (60%).
Somewhat related to cricket is the sport of baseball which is also a bat-and-ball game played between two
teams similarly taking turns for batting. However, there are many dierences in the way runs are scored and
other rules of the game. For batting, the fundamental dierence is in the primary goal of the batsman that is
mainly dictated by gameplay. In baseball, there are 9 innings played within a few hours whilst in test cricket
4 innings are played over 5 days (30 hours). This means, that in cricket, batsmen are trained to develop their
defensive game to be able to survive longer. There are other dierences for example in awarding penalty points
to batsmen in baseball for swinging and missing the ball (three strikes result in out); in cricket a batsman can
bat a lot longer with no penalty for missing the ball. Due to the length of batting durations, a greater range of
the types of bowling deliveries, and dynamics of the eld, cricket batting requires a very dierent technique to
be able to manoeuvre the ball and score runs. The total number of shots in cricket are therefore higher than in
baseball. Coaching systems in baseball such as [
35
] focus mainly on the path of the swing enabling assessment of
batting. In this vision based system, dierences between consecutive frames are used for generating the path of
the bat and arms. In cricket, such an assessment system can be utilised to assess the swing type, however other
relevant factors such as feet movements, and direction of the shot would be cumbersome to infer.
In this paper we focus on the automated recognition of batting shots in cricket using a hierarchical framework
utilising low-cost hardware and a simple setup. A similar form of hierarchical analysis framework was deployed
in [
25
], in which occupancy estimation was performed at multiple levels of granularity using environmental
sensors. Based on the automated recognition results we generate visualisations that are oriented on the current
best standard as it is typically used for illustration in, e.g., television broadcasting of cricket. These automatically
generated visualisations are informative through representing each batting session and thus serving, for example,
as a decision support for coaches during initial scouting sessions where a bottleneck for amateur players can
make the process very time-consuming. Individual players can use the visualisations to improve their range of
shots by focusing on less-successful shots, thus improving their overall technical game without the need for
expensive professional setups.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:9
3 HIERARCHICAL RECOGNITION OF BATTING SHOTS
3.1 System Overview
The ultimate vision of our work is the development of an analysis system for the fully automated assessment of
cricket skills, which is the basis for autonomous, aordable coaching systems. This is in line with related work
where wearable sensing systems and automated sensor data analysis methods have been used for (semi-)automated
quality assessments, for example, in other sports as outlined in the previous section.
In this paper we present the fundamental building block of such a skill assessment system for the sport of
cricket consisting of an automated recognition system for the analysis of batting shots –arguably the most
important technical skill a cricketer has to master for successfully playing the game – and, based on the results
of this automated classication step, visualisation and analysis schemes that facilitate objective skill assessments
as they are required for devising tailored, that is personalised training programs as professional coaches would
develop.
Figure 3 gives an overview of the developed system. A cricket player attaches four sensors – standard, o-
the-shelf nine-axis inertial measurement units (IMUs) – to his/her limbs using wrist bands or tape, which is a
straightforward process to which most cricket players are used to anyway (given that during batting sessions
players are supposed to wear protective equipment). For our deployment we used Axivity WAX9 IMUs [
4
]. The
sensors are wirelessly paired via Bluetooth with an application running on a smartphone where the data is
collected and stored. A batting session can then take place as they are typically conducted (either indoors in
specic batting arenas or outdoors on any kind of pitch that is large enough for playing cricket) without any
disruptions or discomfort to the player. Upon conclusion of the session our analysis system processes the sensor
readings from all four devices, recognises the various shot types, and generates visualisations of key performance
indicators (“skills”) as they are relevant for the analysis of a cricketer’s performance. These results form the basis
for informed, objective, and personalised coaching programs that tackle individual weaknesses of a player. The
advantage of our system is its aordability and ubiquitous applicability, which enables cricketers not only at
(semi-)professional level but rather at all levels of expertise down to grass-root movements in remote villages, to
enjoy their sport and receive helpful and objective feedback that will form the basis for excelling in the technical
skills that are so important for mastering the sport.
3.2 Methodology Background
Batting in cricket can result in both intended and unintended shots, with both intended and unintended results.
Due to the complexity of batting shots involving several sub-components, our batting shot analysis framework is
based on a hierarchical approach. The prerequisite for the overall assessment stage is a system that captures
dierent aspects of the batting shot. According to the background of the game of cricket (as explained in the
previous section) and the general best practice in assessing the quality of cricket matches, we formulate these
aspects as follows:
Has the batsman played a valid shot?
This allows recognition of shots among all the activities performed by a player. Whilst batting, a player runs
between the two ends of the pitch to score, walks to change ends or engage in stretching exercises between shots.
Recognition of shots among all the activities performed by players is very useful.
Has the bat successfully hit the ball or not?
This enables delineation between hits (shots in which the bat touches the ball) and misses (shots in which the bat
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:10 A. Khan et al.
,ŝĞƌĂƌĐŚŝĐĂů
ĐƚŝǀŝƚLJ
ZĞĐŽŐŶŝƚŝŽŶ
^LJƐƚĞŵ
ϭ͘^ŚŽƚ
Ϯ͘EŽ^ŚŽƚ
ϯ͘hŶŬŶŽǁŶ
ϭ͘,ŝƚ
Ϯ͘DŝƐƐ
ϯ͘hŶŬŶŽǁŶ
ϭ͘KĨĨ
Ϯ͘KŶ
ϯ͘^ƚƌĂŝŐŚƚ
ϰ͘hŶŬŶŽǁŶ
ϭ͘&ƌŽŶƚ&ŽŽƚ
Ϯ͘ĂĐŬ&ŽŽƚ
ϯ͘ŽǁŶͲƚŚĞͲŐƌŽƵŶĚ
ϰ͘hŶŬŶŽǁŶ
ϭ͘ĞĨĞŶĚ
Ϯ͘ƌŝǀĞ
ϯ͘ŽǀĞƌ
ϰ͘WƵůů
ϱ͘Ƶƚ
ϲ͘hŶŬŶŽǁŶ
ǁϭ
ǁϮ
KĨĨͮ^ƚƌĂŝŐŚƚͮKŶ
^ŚŽƚʹ,ŝƚʹKŶʹ&ƌŽŶƚ&ŽŽƚʹWƵůů
^ŚŽƚʹDŝƐƐʹhŶŬ ʹhŶŬ ͲhŶŬ
^ŚŽƚʹ,ŝƚʹKĨĨʹĂĐŬ&ŽŽƚʹĞĨĞŶĚ
^ŚŽƚʹ,ŝƚʹKĨĨʹ&ƌŽŶƚ&ŽŽƚʹƌŝǀĞ
>ϭ >Ϯ >ϰ >ϱ
(a) Low-level baing shot analysis
,ŝĞƌĂƌĐŚŝĐĂůĐƚŝǀŝƚLJZĞĐŽŐŶŝƚŝŽŶ^LJƐƚĞŵ
>ĞǀĞůͲϭůĂƐƐŝĨŝĐĂƚŝŽŶ
Ğ͘Ő͕͘ƐŚŽƚͬŶŽͲƐŚŽƚ
>ĞǀĞůͲϮůĂƐƐŝĨŝĐĂƚŝŽŶ
Ğ͘Ő͕͘,ŝƚͬDŝƐƐ
>ĞǀĞůͲŶůĂƐƐŝĨŝĐĂƚŝŽŶ
Ğ͘Ő͕͘ƌŝǀĞͬWƵůů
sŝƐƵĂůŝnjĂƚŝŽŶƐĨŽƌ^ŬŝůůŶĂůLJƐŝƐ
^ĞŐŵĞŶƚĂƚŝŽŶ
ĂŶĚ
&ĞĂƚƵƌĞ
džƚƌĂĐƚŝŽŶ
ZĂǁ
ĂƚĂ
͙
ி
(b) Hierarchical baing shot classification linked to automated visualisation of baing skills.
Fig. 3. Overview of our system for automated baing shot recognition for skill assessment in cricket (see text for description).
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:11
misses the ball). Inexperienced players face diculties in hitting the ball initially which can be improved over
time. Experienced players can naturally hit the ball and also hit it in a certain direction of the ground.
What was the direction of the shot?
With this, we can observe the ground coverage of a player and identify weak zones around the ground. There are
three main sides that a batsman can target including O,On or Straight (as shown in Figure 2).
What were the positions of the batsman’s feet?
In order to execute perfect shots, feet positions are very important as such correct and quicker assumption of
batting stance enables better shots. For batsmen, this is one of the most important skills and which also takes a
long time to master. The goal is to get close to the pitch of the ball in order to nullify the eect of spin (in the
case of slow bowling) or swing (in the case of fast bowling). The closer a batsman gets to the pitch of the ball, the
better it is. However, in cases where the bowler shortens the length, the batsman then has to move back to allow
some more time for him to judge the ball’s trajectory and hit it.
What type of swing was used by the batsman?
As opposed to the direction of the shot, this denes the type of swing executed to play a shot in a certain direction.
There are various types of shots such as drives in which a batsman swings along the line of the ball (this shot
mainly covers the straighter regions of the ground). In the case of cuts and pulls, a batsman swings the bat across
the line of the ball; such shots mainly cover the o and on sides of the ground.
Each of these aspects are modelled separately representing multiple layers of the overall system’s granularity.
Comparable methods have already been used for assessment in sports such as tennis or badminton [
26
]. In tennis,
for example, shots are analysed at multiple levels of representation by looking at rallies, and within each rally
representing hits on the near and far side of the tennis court. Such a representation helps in: a) understanding
various components of an activity; and b) modelling them in a machine learning framework to automatically
recognise/ predict these activities.
For cricket batting, we dene
L=
5 levels for detecting relevant aspects of a cricket shot. These levels are
based on the structure of a cricket shot and can be separately detected:
L1 – Shot detection
a) Shot,
b) No-shot,
c) Unknown.
L2: – Hit detection
a) Hit,
b) Miss,
c) Unknown.
L3 –Direction detection
a) O,
b) On,
c) Straight,
d) Unknown.
L4 – Feet position detection
a) Front foot,
b) Back foot,
c) Down-the-ground,
d) Unknown.
L5 – Type of shot detection
a) Defend,
b) Drive,
c) Cover,
d) Pull,
e) Cut,
f) Unknown.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:12 A. Khan et al.
In order to eectively assess a player’s batting quality, these 5 levels need to be analysed separately. These
levels represent components of a batting shot that collectively, when mastered, result in a successful execution of
a shot. For example, a Drive on the O side (cf. Figure 2) can be played with both front and back foot positioning
of the feet. However, if the ball is pitched close to the feet, it becomes a risky shot to play on the back-foot. As
such, if the ball misses the bat, it may hit the stumps or one of the legs in front of the stumps, both leading to the
batsman getting out, i.e., a failed attempt. In fact, players spend a long time training in nets to get into correct
positions to play shots. Analysing these separately can greatly help in dening focus points for improving aspects
of the shot that are not (yet) correctly executed by an individual player.
In the context of cricket training, after batting shot recognition has taken place the aforementioned ve levels
can be analysed separately for assessing batting quality. For example, at each level we can draw a comparison
between professional and amateur cricketers, thereby providing feedback which can assist in the coaching of
young players. Using this feedback, players can focus their practice on particular areas of their game, and as
improvements are made over time they will be automatically observed by our system, allowing players to track
their performance. For example, for a beginner, it is important to successfully hit the ball with as many of
their shots as possible and therefore level-1 and level-2 would be the most important levels to focus on initially.
Extending this for professional players – where every movement is critical to execute the perfect cricketing shot,
the type of shot and the technique utilised also become more important – and therefore analysing all 5 levels is
relevant.
The main benets of using a hierarchical approach are two-fold: a) it provides activity outputs in a manner
that coaches or trainers can easily understand for assessing batting abilities of players; and b) it makes the
classication approach for activity recognition easier to train, as the number of classes per level are relatively few.
3.3 Segmentation and Feature Extraction
We build separate predictive models for each level within our hierarchy, where each level employs a similar,
supervised classication approach. Raw data is initially segmented using a sliding window approach, with
a window length of
w=
1
.
6 seconds (derived from a systematic evaluation performed for a range of values
originating from the length of batting shots in our dataset). A range of features are then calculated to abstract
from the raw data to our feature space. These features are then utilised in parallel for hierarchical classication. As
explained in the previous section, in the context of our cricket shot classication framework we have
L=
5 levels
and therefore ve classiers. Classiers are trained hierarchically with the same feature vectors but dierent
labels each that are used specically for each individual level (described in Figure 3), resulting in ve recognition
models, each of which focussing on a dierent aspect of the batting shot.
These features must be able to capture the unique characteristics important to the ve levels in our hierarchy.
Our 4-sensor setup is ideal in this context as most of the body movements can be captured to a sucient quality
for accurate recognition. For the accelerometer features
fa
, we use statistical features such as the mean
μ
, median
x
, standard deviation
σ
, and range
x1,xn
of each axes to capture the underlying distribution of the sensor data.
For each axis of our four 3-axes sensors we therefore produce (μ,σ,x,x1,xn).
Additionally, we also include the ECDF representation
fi
(Equation 1), which samples from the empirical
cumulative distribution function to provide
d=
15 coecients per axis which includes information about the
data distribution within a frame [
13
]. The ECDF representation of raw sensor data is provenly benecial as it
captures the real distribution of sensor data in a compact and meaningful way [13, 27, 40].
ECDF ={x,j:Pj
c(x)=pj}(1)
C={pi}∈Rd
[0,1],pi<pi+1(2)
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:13
We also use additional representations that are important for understanding more complex shots: Signal
energy (Equation 3), signal entropy (Equation 4), total squared jerk (Equation 5), and fast Fourier transform (FFT)
coecients (d=16)calculated on each axes.
E=1
N
N
i=1
x2(3)
I=
N
i=1
xlog (|x|)(4)
J=
N
i=1
Dxx(5)
Other features are also calculated for each sensor utilising multiple axes including RMS of acceleration (Equation
6), RMS velocity (Equation 7), and the maximal pairwise cross-correlation between the acceleration axes (Equation
8) [9].
Ra=1
N
N
i=1(x2+y2+z2)(6)
Rv=1
N
N
i=1(xdx)2+(yd x )2+(zdx )2(7)
CC(x,y)=n1
max
d=1(1
N
N
i=1
xi·yid)(8)
By combining all of these features we end up with feature vector
faRd=106 for each accelerometer:
fa=μ,σ,x,x1,xn,C,E,I,J,FFT
c,Ra,Rv,CC(x,y)T(9)
For the gyroscope and magnetometer we focus on a smaller subset of the total features in order to reduce the
overall dimensionality of the feature vector while including additional information. This subset includes mean,
median, standard deviation, signal energy, and entropy. For each of these sensors we calculate these features on
the magnitude of the sensor rather than individual axes, resulting in
fдRd=5
and
fmRd=5
, each containing
the following (μ,σ,x,E,I).
Finally, we then combine the features via concatenation so that the features for each sensor combine to form a
sub-feature vector representing the overall sensor
fs
(Equation 10). Features from the overall representation for
all 4 of our sensors are then concatenated to form the overall feature vector
FRd=464 (Equation 11).
fs={
fa,
fд,
fm}(10)
F={
f1,
f2,
f3,
f4}(11)
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:14 A. Khan et al.
Fig. 4. (L) Wagon Wheel representing each shot with a line, used in TV broadcasting. (C) Wagon Wheel showing the
proportion of runs scored in key areas around the pitch. (R) Wagon Wheel with more granular proportion of runs around the
pitch.
3.4 Hierarchical Classification
Five separate models are built for recognising the various aspects of a batting shot. These aspects are represented
by each level in our hierarchy. Given the exploratory character of this paper we have evaluated the eectiveness
of the predominant approaches as classication backends in our system, namely: discriminative models (Decision
Trees, and Support Vector Machines – SVM), and instance based learning model (k-Nearest Neighbour classier).
Ground truth annotation was collected from video footage and used for training and validating our models.
Labels are assigned for every frame extracted using the aforementioned sliding window procedure thereby
employing majority voting (necessary due to overlapping subsequent frames). Overall, ground truth annotation
is based on the hierarchical labelling scheme as detailed in Section 3.2 for which an example batting shot can
look like; Shot-Hit-FF-O-Cut, representing all 5 levels of the representation hierarchy.
Utilising our feature matrix
F
we train ve distinct models. For the purpose of evaluating more than one
classier to determine the best possible set of 5 models, we repeat this modelling process in separate experiments
for the three classication paradigms previously mentioned. Each of our ve models use a dierent label which
corresponds to the ve dierent levels in our representation hierarchy, allowing us to capture dierent aspects of
the batting shot. By modelling the dierent aspects separately (the ve levels) we signicantly reduce the number
of classes that we intend to recognise with a single model. For
k
-NN we optimise the value of
k
, and for Support
Vector Machines we perform a grid search procedure for optimising the cost and gamma parameters of the RBF
kernel [43].
3.5 Visualisations
The visualisations produced by our framework are oriented on the standard “Wagon Wheel chart” that is widely
used by international cricket teams for training as well as by cricket broadcasters to summarise a batter’s
performance. Versions of this chart have been used in the game for over one hundred years [
30
], thus indicating
the value of the depicted information and formatting. The traditional Wagon Wheel presents the cricket pitch
from a bird’s eye view, and draws straight lines from the wicket to the nal destination of each ball, resulting in a
summary of a batter’s innings (see Figure 4(L)). Other versions have simply reported the percentage or number
of runs scored in key areas (see Figure 4(CR)), again resulting in a concise summary of the batter’s performance.
These two Wagon Wheel styles can both be used to assess weaknesses and dominance of batsmen, for example
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:15
Fig. 5. Front-on view of a baing net session where this dataset was collected. O and on sides are towards the le and
right of the image respectively.
by showing Michael Hussey’s preference for on side pulls when scoring (see Figure 4(C); note Michael Husssey is
a left-handed batsman) or Younis Khan’s cover-drives (see Figure 4(R)).
With this in mind we decided to follow the de facto visualisation techniques in cricket, thus assuring that
all players and coaches would have an instant understanding of the results. The bar charts employed in other
visualisations, i.e. Front/Back foot preference (see Figure 8) and Attacking vs. Defensive Shots (see Figure 9) bar
charts were designed to be as intuitive as possible for players and coaches. While these graphs were not grounded
on established visualisations within the sport, we used a simple two-way bar chart to indicate the proportion of
attacking and defensive shots (or front vs. back foot) – thus eectively relaying the binary options. While no
formal evaluation was carried out for these bar graphs, feedback from non-cricketing colleagues veried the
intuitiveness of the visuals. These bar graphs, in conjunction with the Wagon Wheel visualizations, provide a
complete representation of batting shots.
4 EXPERIMENTS
4.1 Dataset
To evaluate our system and to validate its general eectiveness, we conducted a study where we invited participants
to a local (indoor) cricket centre that features dedicated practice areas –so-called batting nets– where cricketers
can practice batting to improve their abilities for match situations. A bowling machine shoots balls towards the
batter thereby varying speed and direction such that the player can practice the various skills relevant for good
batting. Figure 5 gives an impression of the setting in the cricket centre where we ran our data recording session.
Our study was conducted in these dedicated batting areas and participants wore the sensing system as described
before while engaging in batting sessions. Although we collected our dataset in batting nets, the system has
been designed for deployments during match situations where identical shots are played. The only dierence
is in the running activity between shots which can be treated as another activity type and can be added to the
classication system. Activity recognition for walking and running is widely studied in the literature and very
good accuracies are achieved in recognising these activities [42].
For a total of 6 participants (5 males and 1 female; average age: 27
.
50 (
±
4
.
46)) we collected data over 11 sessions,
which translates into almost 3 hours of raw sensor data. Arguably, this dataset is relatively small and as such
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:16 A. Khan et al.
zŽƌŬĞƌ &Ƶůů
WŝƚĐŚĞĚ
'ŽŽĚ
>ĞŶŐƚŚ
^ŚŽƌƚ
WŝƚĐŚĞĚ
ŽƵŶĐĞƌ
^ĞŶƐŽƌ>ŽĐĂƚŝŽŶ
ƌŝǀĞ ƌŝǀĞ͕
^ǁĞĞƉ
Ƶƚ͕
WƵůů
,ŽŽŬ
>ĞĂǀĞ
ĞĨĞŶĐĞ
>ĞŶŐƚŚƐ
^ŚŽƚƐ
KĨĨ
Fig. 6. Illustration of the bowling lengths during our baing sessions. Some preferred shot types for those lengths are also
illustrated. Decision regarding o and on side is usually made based on the line of the ball.
does not qualify as a large-scale user study. However, the focus of this paper is on the exploration of general
feasibility of an automated cricket skill assessment system based on inexpensive sensing infrastructure and
machine learning based sensor data analysis. As such our focus was more on a detailed evaluation for a cohort of
participants that span the full range of experience and existing cricket skills as one would expect for the intended
target user groups – amateurs who would benet substantially from an automated and especially consistent
feedback procedure. Typically this population does not have any access at all to qualied feedback not to mention
coaching. The aim of the work presented in this paper is to develop a prototypical system that can subsequently
be used for large scale deployments thereby specically targeting longitudinal studies that, for example, track the
performance of whole teams over the course of, e.g., a complete cricket season. The evaluation conducted for this
paper will lead to important insights that will allow us to realistically judge the capabilities of our system.
Our participants’ cricket experience and expertise ranged from beginners (rst time batting) to players with
semi-professional batting experience. We have categorised each player into one of four categories, beginner,
intermediate,experienced, and semi-professional players, detailed below:
P1: Experienced (>10 years of batting experience):
Player 1 had played cricket at both school and college levels playing as a lead batsman. With several years
of experience in unprofessional cricket, player 1 was capable of playing most of the shots comfortably.
P2: Intermediate (>5 and <10 years of batting experience):
Player 2 had only played cricket at the school level and was able to play some of the shots comfortably.
Player 2 had a particular diculty with feet movements for example, most of the shots were played with
incorrect feet positions.
P3: Beginner (<5 years of batting experience):
Player 3 had only played cricket at the school level for a short time. However, player 3 was able to play
shots with both front foot and back foot successfully.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:17
Table 2. Dataset summary
P Gender Sessions Total Shots Total shots per activity within the hierarchical shot representation
L1 L2 L3 L4 L5
Shot No-shot Hit Miss Leave Prac. O On Str. FF BF DG Defend Drive Cover Pull Cut
1 Male 4 364 273 91 208 57 8 83 80 89 34 102 147 16 60 71 12 33 41
2 Male 1 56 52 4 32 16 4 0 22 19 3 44 4 0 4 8 3 2 10
3 Male 1 95 72 23 61 9 2 21 41 18 6 35 35 0 5 11 21 16 14
4 Female 1 161 98 63 59 35 4 59 20 41 28 55 38 1 5 35 13 24 7
5 Male 2 74 66 8 56 10 0 8 46 19 0 44 22 0 1 52 6 4 1
6 Male 2 222 143 79 90 47 6 73 25 97 5 96 40 1 10 40 10 25 5
947 704 243 506 174 24 244 234 283 76 376 286 18 85 217 65 104 78
P4: Semi-professional (>10 years of batting experience including club level experience):
Player 4 was an experienced player who was able to play most of the shots with correct feet positions;
depending on the bowling length, player 4 was able to very quickly change stance and play shots. Player 4
actively plays as a batsman at the county club level. Shots played by player 4 were also timed perfectly in
most cases (high power shots).
P5: Beginner (<5 years of batting experience):
Player 5 had no exposure to playing cricket prior to data collection and played cricket for the rst time. In
most shots, player 5 preferred driving the ball but had diculty in playing other kinds of shots.
P6: Beginner (<5 years of batting experience):
Player 6 had a good knowledge of cricket but very little experience of playing cricket. In particular, player
6 had diculties with feet movements and getting into the correct position for a shot.
A detailed breakdown of the collected dataset is given in Table 2. Each session was on average 15
.
63
±
5
.
44
minutes long, where sensors (each equipped with an accelerometer, a gyroscope and a magnetometer) were
placed on all four limbs of the players (as illustrated in Figure 6; note that sensors on the lower limbs were placed
behind the protective pads). Due to the high speed motion of batting shots, sampling rate of 100Hz was used
(higher than required for standard HAR models [
23
]). The bowling machine we used served balls with speeds
varying between 40 to 70mph, representing a range of bowling styles from slow spin (where a bowler bowls
slow and spins using the wrist and/or ngers) to fast bowling. Line (direction of the ball in line with the pitch)
was varied slightly between straight and o-side depending on the spin and swing generated by the machine.
Also, length (point in front of the batsman where the ball is pitched) was varied between ‘short pitched’ and ‘full
pitched’ (cf. Figure 6).
Players were given complete freedom to execute any desired shots but were encouraged to play as many
varieties of valid shots as possible. Videos for each session were recorded for annotation purposes and an initial
synchronisation ritual was executed for temporal alignment between the sensors and the camera to facilitate
the annotation: a vigorous shake of the sensors in plain view of the camera that enabled subsequent manual
alignment of the separate timelines of both the sensors and the video footage (see [41] for details).
4.2 Annotation Protocol
We annotated each batting shot to give us information on each of the levels in our hierarchy. By default any
segment of the data not annotated as a batting shot was classied as the ’unknown’ class described in our rst
level. For every shot that was picked out as a batting shot (Level 1: Shot/No shot), we annotated whether the
shot successfully hit the ball or not (Level 2: Hit/Miss), the direction of the shot (Level 3: O/On/Straight), what
kind of footing was employed (Level 4: Front foot/Back foot/Down-the-ground) and nally what type of shot
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:18 A. Khan et al.
sŝĚĞŽ ŶŶŽƚĂƚŝŽŶ'ƌŝĚ
^LJŶĐŚƌŽŶŝƐĞĚĐĐĞůĞƌŽŵĞƚĞƌdƌĂĐŬƐ
ŶŶŽƚĂƚŝŽŶWĂŶĞů
Fig. 7. Screen-shot of the annotation soware indicating various panels including the video synchronised accelerometer
data (note, gyroscope and magnetometer data is not indicated in this but is utilised in our recognition framework). Annotated
shots with their start/end times and duration are also indicated.
it was (Level 5: Defend/Drive/Cover/Pull/Cut). The nal label therefore contained information for all 5 levels.
Annotations were performed using the ELAN annotation software
2
. A screen-shot of the software in operation is
shown in Figure 7. Batting shots were annotated by two annotators with knowledge of cricket shots and rules of
cricket. After annotation of a session by one annotator, the other annotator veried the annotated shots. In the
case of disagreements, shots were reviewed multiple times to reach consensus. Most of the disagreements arose
from confusion caused by the direction of the batting shot and the direction in which the ball went (this was
mainly caused by the ball hitting the edge of the bat and going in unintended directions). This was resolved by
taking the ball direction as the true clue for shot category annotation (which is also consistent with the batting
visualizations that professional cricketers have access to in which direction of the ball is mainly considered
irrespective of the type of bat swing).
4.3 Evaluation Methodology
As discussed in Section 3, for the batting type classication we performed recognition using three standard
classication techniques: Decision Trees; k-Nearest Neighbours; and Support Vector Machines. We employed a
leave-one-participant-out approach for classication in order to demonstrate realistic general applicability of the
proposed system [
15
]. For each level, separate classiers were trained to output activities per level (see Figure 3).
An average class-weighted F1-score was used to compare the results of these techniques as below:
2https://tla.mpi.nl/tools/tla-tools/elan/
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:19
Table 3. Leave-one-player-out results for baing classification; each row represents the test results for an individual player
by training the models using all sessions of the other players.
P Hierarchical activity recognition results for batting shots
L1 L2 L3 L4 L5
DT k-NN SVM DT k-NN SVM DT k-NN SVM DT k-NN SVM DT k-NN SVM
1 86.30 90.60 90.72 83.57 87.42 82.22 87.96 89.09 88.91 86.70 87.75 90.02 86.24 86.75 88.36
2 88.94 93.54 84.92 86.56 88.97 85.14 88.58 87.85 89.04 90.57 90.88 90.13 87.36 87.36 87.88
3 89.93 91.23 82.91 87.27 89.44 83.33 90.13 90.41 87.60 89.88 90.00 87.10 87.70 88.93 87.46
4 74.81 76.68 65.98 68.95 71.72 66.10 82.76 83.35 82.56 84.39 86.20 90.18 82.79 84.54 82.75
5 90.84 92.28 85.02 86.15 88.84 86.30 91.95 91.71 89.06 90.12 91.45 90.91 91.24 92.31 92.93
6 88.62 90.37 83.44 86.59 88.66 83.70 87.75 86.96 85.52 88.11 88.57 90.80 87.69 89.55 91.62
μ86.57 89.11 82.16 83.18 85.84 81.13 88.18 88.22 87.11 88.29 89.14 89.85 87.17 88.24 88.50
σ5.96 6.20 8.40 7.08 6.95 7.50 3.09 2.93 2.61 2.40 2.00 1.40 2.72 2.66 3.57
F1c=2
|C|
iCi
|Cipi×ri
pi+ri
(12)
where
C
is the set of all classes whilst
pi
and
ri
are the precision and recall values that can be calculated as
follows:
pi=tpi
tpi+fp
i
,ri=tpi
tpi+fn
i
(13)
where
tpi
,
fp
i
and
fn
i
are the true positives, false positives and false negatives for a given class
iC
respectively.
Results of the automated batting type classication are given in what follows.
4.4 Results: Classification of Baing Types
Results for these individual levels using all three classiers (individually) are shown in Table 3. On average 88
.
30%
class-weighted F1 score was achieved using the best performing classiers per level. Results across all the players
and all levels for all classication methods are shown including their means with the standard deviations across
all levels. Average results are computed across all participants which also indicate the expected performance of
these classiers on all levels if data from a new participant is tested using the trained models.
Our results indicate a very high reliability at all levels and for all classication methods with no signicant
dierence in performance. This means that any of the evaluated classication techniques can be used to train
models at all levels of our representation hierarchy to classify batting shots. A batting shot can thus be reliably
classied using such sensors and our method by splitting it into various categories. This approach can be readily
applied in domains where activities can similarly be represented such as in baseball or tennis [26].
Our proposed system is modelled in a manner to reect generalization i.e., the leave-one-participant-out
cross validation technique enables us to have models that will increasingly get more general as data from new
participants is used. Performance of a personalized approach i.e., a model specic for each user might yield better
performance but it has a signicant limitation in the number of models that are required to be trained.
Reliable results from these levels are of signicant importance as it is based on these results that coaches or
players can infer about the performance in the overall context of the game. Based on the output of the automated
shot recognition systems, players can retrospectively analyse their shot statistics not only individually but also
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:20 A. Khan et al.
Table 4. Aggregated results using the hierarchical representations using the leave-one-player-out cross-validation scheme.
P Attack-Defence Feet Positions Ground Coverage1Ground Coverage2
L1-L2-L5 L1-L2-L4 L1-L2-L3 L1-L2-L3-L4-L5
DT k-NN SVM DT k-NN SVM DT k-NN SVM DT k-NN SVM
1 77.29 85.14 86.62 78.72 85.79 87.30 78.62 86.33 88.72 75.89 84.89 86.35
2 86.19 87.13 88.45 86.23 88.14 89.91 85.93 86.73 87.94 84.62 86.31 87.58
3 85.64 85.27 85.91 86.51 87.05 88.48 85.40 86.58 87.11 83.85 85.68 85.94
4 67.76 71.86 72.55 66.85 70.39 70.75 67.19 71.38 73.39 64.97 69.85 70.46
5 85.60 88.64 89.25 85.33 88.70 89.45 85.55 88.67 89.68 83.53 87.35 88.11
6 83.50 85.27 86.62 83.70 87.10 86.87 85.14 86.06 88.31 82.60 85.49 86.41
μ81.00 83.89 84.90 81.23 84.53 85.46 81.31 84.29 85.86 79.24 83.26 84.14
σ7.28 6.05 6.18 7.60 7.00 7.30 7.45 6.39 6.17 7.68 6.62 6.75
1O-On-St, 2Octants
their combined eect resulting in generating focus points that players or coaches can utilize to improve game
performance. In this paper, we demonstrate not only how to automatically recognize activities but also utilizing
the results of the automated system in a manner that can be used for quality analysis and trend tracking in
batting.
In order to perform such analysis of batting in cricket, various shot categories within the hierarchical represen-
tation need to be aggregated. For example for feet position analysis, levels 1, 2 and 4 need to be analysed. Table 4
shows the aggregated results for all scenarios discussed in this paper. Predictions from each level are separately
produced and then collectively considered for calculating the class-weighted F1-scores. These aggregated results,
although showing a slight drop in performance, are very promising as also indicated by the visualizations below.
4.5 ality Analysis of Baing Sessions
With our approach, key statistics and visualisations related to a player’s batting style can be generated, which
are the basis for assessing the quality of a player. In what follows we provide an exemplary exploration of some
of the key analyses that can be performed using our automated system, which illustrates the practical utility of
our system in general and the low-level batting short classication in particular.
4.5.1 Feet Position Analysis. When batting in cricket, a player’s feet are positioned to ultimately guide the ball
in the correct direction. Even executing the upper-body mechanics to perfection can result in an unexpected shot
if the player’s feet are not placed appropriately. Using our system, feet positions can be analysed using levels-1,
2, and 4 of the representation hierarchy where all the hits are considered (ignoring miss and deliberate leave
type shots). In our dataset, we found that players 1 and 2 have a balanced playing style with an almost equal
distribution of “front foot" and “back foot" shot types (see Figure 8). Players 5 and 6 have opposite tendencies in
playing back foot and front foot shots respectively.
With such information, players can practice shots to improve on both kinds of feet positions. Figure 8 also
shows our model’s results with regards to feet position (using the same representation hierarchy) showing a
positive correlation with the actual distribution of feet positions.
4.5.2 Aacking and Defensive Shot Analysis. A player can play aggressively to try and score runs (attacking)
or they can be more cautious and play defensive shots to prevent getting out. Such an analysis of attack against
defense can also be performed for individual players using our system (see Figure 9). This is achieved using
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:21
90% 70% 50% 30% 10% 10% 30% 50% 70% 90%
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
)URQW)RRW
%DFN)RRW
Ground Truth Prediction
Fig. 8. True and predicted percentages of shots played on either back foot or front foot for individual players across all of their
baing sessions. This was generated using the results of our automated analysis framework at level-4of the representation
hierarchy.
50% 40% 30% 20% 10% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Player 1
Player 2
Player 3
Player 4
Player 5
Player 6
$WWDFN'HIHQFH
Ground Truth Prediction
Fig. 9. True and predicted percentages of shots to compare aacking against the defensive shots for individual players
across all of their baing sessions. This was generated using the results of our automated analysis framework at level-5of
the representation hierarchy by comparing the shot ‘Defend’ against all other shots.
level-1, level-2, and level-5 of the representation hierarchy where all the valid shots are compared against the
defensive shot. In most cases, our participants played dierent kinds of attacking shots with player 1 playing the
most number of defensive shots. This is of particular importance in cricket as there are various formats in which
cricket is played, some of which require a more attacking game. For example, in the T20 format, which lasts for a
few hours, most of the players generally play an aggressive game whilst in the test format (which can last for up
to 5 days), players generally engage in a more defensive game.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:22 A. Khan et al.
On:42%
39%
Off:41%
44%
Straight:18%
17%
Player 1
On:42%
44%
Off:51%
40%
Straight:7%
16%
Player 2
On:31%
37%
Off:60%
51%
Straight:9%
12%
Player 3
On:49%
48%
Off:23%
23%
Straight:28%
29%
Player 4
On:79%
71%
Off:17%
20%
Straight:3%
9%
Player 5
On:30%
36%
Off:70%
64%
Straight:0%
0%
Player 6
0
10
20
30
40
50
60
70
80
90
100
Fig. 10. Analysis of shots illustrating the percentages of shots played in a certain direction utilising the representation
hierarchy. Prediction results also showing the percentages of shots in a certain direction using the proposed system and are
shown in red. Black arrow indicates the baing direction.
Depending on the format a player is interested in, our approach can automatically help in tracking a particular
form of batting style and practice shots that are of more interest. For example, in an aggressive game scenario,
players would focus more on hitting shots that would yield higher number of runs which has an associated risk
of getting out whilst in defensive game scenarios, players would prefer hitting shots along the ground, reducing
the risk of getting out caught (a batsman gets out when the ball is caught by a member of the opposition team
before it hits the ground).
4.5.3 Ground Coverage Inference. Ground coverage is an important metric that reveals the direction of shots
played by a batsman. With our approach, we can do this either using: a) shot direction (focusing on
L
3 for
example O or On side shots); and/ or b) shot types (focusing on L5 for example drives and pulls). For players
who are new to the game and at a stage where it would be dicult to recognise all specialised shot categories,
ground coverage can be analysed using only the batting shot direction (i.e., using information upto
L
3) which is
associated with the three main areas around a batsman.
Figure 10 shows the distribution of shots around the ground divided into 3 main areas named as o,on and
straight (see also Figure 2, which highlights these regions, too). This distribution is derived using level-1, level-2,
and level-3 of the representation hierarchy considering all shots in level-1 and all hits in level-2.
Most of the shots are usually played according to the type of the incoming ball (see Figure 6 for various types
of bowling lengths for example). Shot analysis, therefore, can be performed in order to see how batsmen execute
their shots in response to the kind of bowling they are exposed to. In our results, we can see that most of the
shots played by the players are either on the o or the on side of the pitch. Player 5 has largely played on-side
shots (79% of the time and hierarchically classied using our approach as 71%), whilst for player 6 o-side shots
are favoured more. Player 4 has a good shot distribution all around the ground including straight shots played
28% of the time.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:23
36%
0%
20%
4%0%
3%
0%
37%
40%
0%
21%
0%0%
6%
0%
34%
Player 1
0%
0%
0%
0%0%
41%
0%
59%
0%
0%
10%
0%0%
20%
0%
70%
Player 2
25%
0%
28%
2%0%
7%
0%
38%
24%
0%
10%
0%0%
17%
0%
48%
Player 3
18%
0%
21%
3%0%
6%
0%
52%
16%
0%
9%
0%0%
9%
0%
66%
Player 4
20%
0%
26%
0%0%
1%
0%
52%
25%
0%
25%
0%0%
0%
0%
50%
Player 5
34%
0%
1%
0%0%
0%
0%
64%
43%
0%
5%
5%0%
0%
0%
48%
Player 6
0
10
20
30
40
50
60
70
80
90
100
Fig. 11. Shot analysis showing percentages of shots played around the ground with prediction results shown in red. There
are 8octants and is a standard representation used in professional cricket to analyse the number of runs scored in various
directions. Baing direction is indicated by the black arrow.
Table 5. Label schemes for constructing the octant visualisations based on the proposed hierarchical representation approach.
Octant Angle Range Labels scheme per octant
1 0°– 45° L1 (SHOT) & L2 (HIT) & L3 (ON) & L4 (FRONT | BACK) & L5 (PULL)
2 45°– 90° L1 (SHOT) & L2 (HIT) & L3 (ON) & L4 (FRONT | BACK) & L5 (LEG GLANCE)
3 90°– 135° L1 (SHOT) & L2 (HIT) & L3 (OFF) & L4 (FRONT | BACK) & L5 (LATE CUT)
4 135°– 180° L1 (SHOT) & L2 (HIT) & L3 (OFF) & L4 (FRONT | BACK) & L5 (CUT)
5 180°– 225° L1 (SHOT) & L2 (HIT) & L3 (OFF) & L4 (FRONT | BACK) & L5 (COVER DRIVE)
6 225°– 270° L1 (SHOT) & L2 (HIT) & L3 (OFF | STRAIGHT) & L4 (FRONT | BACK) & L5 (DRIVE)
7 270°– 315° L1 (SHOT) & L2 (HIT) & L3 (ON | STRAIGHT) & L4 (FRONT | BACK) & L5 (DRIVE)
8 315°– 360° L1 (SHOT) & L2 (HIT) & L3 (ON) & L4 (FRONT | BACK) & L5 (SLOG)
For more skilled players, greater granularity of shot direction analysis is very important. Our system is capable
of providing shot distribution around the ground in this manner as illustrated in Figure 11, and constructed using
a label scheme shown in Table 5. This label scheme is based on the presence of certain types of shots around the
ground. For example, for Octant 6 (which is located south west of the ground between 180
°
-225
°
) there are shots
(L1) that are hits (L2) on either the O -side or Straight (L3) and are either front or back foot (L4) drives (L5).
These octant-specic shots are then shown as percentages of all shots around various parts of the ground.
There are very few shots played towards the north side of the ground behind the batting direction (illustrated
using the black arrow) that usually indicate either on-side leg-glances (a delicate touch shot in which the ball is
angled by the batsman behind them) and o-side edges (usually a miss-hit) that goes towards north-west of the
ground). This shows a very good representation of the types of shots a batsman is good at or can improve upon.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:24 A. Khan et al.
29%
0%
8%
8%0%
10%
0%
45%
44%
0%
6%
0%0%
0%
0%
50%
Player 1
Session 1
56%
0%
6%
3%0%
0%
0%
36%
56%
0%
18%
0%0%
3%
0%
23%
Session 2
47%
0%
35%
6%0%
0%
0%
12%
46%
0%
25%
0%0%
11%
0%
18%
Session 3
6%
0%
42%
0%0%
4%
0%
48%
14%
0%
28%
0%0%
8%
0%
50%
Session 4





Fig. 12. Session-wise illustration of shots around the ground for player 1. Prediction results using the proposed method are
shown in red. Black arrow indicates the baing direction
For example, player 1 has a good distribution of straighter shots and also o and on drives, but very little cut
shots towards the o-side. As a coach, this would be one area to focus on by reconguring the bowling machines
to bowl at such lines and lengths where the batsman is encouraged to play this type of shot.
In the case of player 2, there is an opposite picture suggesting that player 2 is largely a straight and o-side
player and improvement on the on-side game is required. Note the discrepancy between Figures 10 and 11 is due
to the defensive shots which are not included in Figure 11; the main focus for this visualization is to determine
the potential scoring shots. This is important as a team within which players collectively score more wins the
game, however a defensive game is equally important to ensure staying not-out. Also note that the cover-drive
and slog regions are empty as no such shots in these areas were executed by the players which illustrates another
additional focus point for the players to practice on. For player 2, it suggests that the majority of shots played
towards the on side are defensive shots and would greatly benet from practising shots such as the pull or
on-drive.
4.5.4 Baing Trend Analysis and Focus Point Generation. Players and coaches need to be able to digest a
session’s-worth of shots and forge an action plan to improve weaker shots. In order to track batting trends in a
player, the proposed framework can also be utilised to observe session-wise changes in shots. Figure 12 shows
such a session-wise distribution of shots for Player 1. Not only does this reect the change in the conguration
of the bowling machine during these sessions but also the change in strategy of shots by the player. In sessions 1
and 2, the bowling machine was kept at a full to good length (see Figure 6) and therefore straighter drives south
of the ground are the most favoured and can clearly be observed.
In session 3, bowling speeds were slowed down to spin (particularly o-spin where the ball after bouncing on
the pitch turns west to east). This encourages the player to play shots on the on side mainly, either straighter
on-drives or shots towards the east. In session 4, faster bowling speeds were congured initially on a full/yorker
length and later on a short-pitched length (see Figure 2). This resulted in initial straighter shots, mainly the
o-drive as the bowling line was also on the o side. However, most of the short pitched balls were played
employing the pull shot towards the east of the ground.
Utilizing the proposed system, such tactical changes in gameplay, which are of great importance for a player
or a coach, can be analysed. Players can observe the change in their gameplay in response to various conditions
(e.g., bowling speeds, spin, line and length) and focus their training by practising shots that they feel need further
improvement.
Table 6 provides an overview of all of our players with their preferred feet positioning and shot tendencies.
Players 1 and 2 have a balanced gameplay in terms of their feet positioning (as also illustrated in Figure 8).
However there is a dierence in their preferred shot tendencies; player 1 preferring Straight and On sides whilst
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:25
Table 6. Focus points for all players summarised based on the analysis of the output of our proposed system including
Hit/Miss ratio; higher value indicating majority of shots hiing the bat.
Player Feet preference Shot Tendencies Hit/Miss Ratio Focus points
1 Balanced Straight & On 3.51 O side
2 Balanced Straight & O 1.81 On side & Improve hit percentage
3 FF Straight & On 7.47 O side (BF drives or BF Pulls)
4 BF Straight & O 2.15 On side (FF drives) & Improve hit percentage
5 BF Straight & On 1.86 O side (FF drives) & Improve hit percentage
6 FF Straight 5.29 O and On side (using BF)
player 2 prefers Straight and O sides. Player 1 also has a higher hit to miss ratio of 3
.
51 indicating higher
percentages of shots played in a given session compared with Player 2’s 1.81.
Based on the automated analysis our system provided, not only players can be ranked but also dierent focus
points can be generated. Based on feet positions, shot tendencies and strike rates, we have dened focus points
specically tailored for individual players. For example, Player 5, who has a poor strike rate (hit/miss ratio) not
only needs to improve this by practising more to hit the ball but also to play on the O side more using the
front-foot in particular.
In the absence of our system, achieving such a detailed level of analysis can be very time consuming for
coaches and players. Considering the complexity of shots, it is dicult to gain a comprehensive understanding of
a player’s batting prowess by a simple direct observation where shots would have to be manually noted either
real-time or using a video of the session. A variety of vision-based systems are available to some coaches to
support them in the data collection and analysis. However, these alternative systems aimed at (semi-)professional
players are either very expensive to deploy, require an expert manual setup, or are otherwise not accurate enough
to generate meaningful measures of low-level activities. With our approach, we can achieve the rudimentary
measures of shot classication and present it as a visual package at a fraction of the cost for both amateur
cricketers (who would otherwise have no way of assessing their batting) and professional cricketers (as a low-cost
alternative).
We note that the proposed system is suitable for both indoor and outdoor batting sessions, as the types of
batting shots played indoors or outdoors are exactly the same. We evaluated the performance of our system in an
indoor net session solely for ground truth annotation purposes.
4.6 Influence of Bowling Type
The bowling line and length play a major role in a batsman’s shot choice of batting, as well as in the method the
shot is played (see Figure 6). For example, a short length ball is more likely to be pulled, i.e., shot towards the on side
of the ground played using a back-foot stance, whilst a full length ball is more likely to be driven (usually played
straight down the ground). Thus, knowing the type of the delivery is important for understanding the resulting
shot. Our proposed system does not automatically record the length of the deliveries, but a well-organised and
systematic batting session will portray this information. Then, separate visualisations will be created for each
type of delivery (i.e. short length, good length, and full length). While this approach would perhaps be most
eective when using a bowling machine due to consistency in deliveries, it is possible to collect the data using a
bowler. The ideal method for structuring the batting session would be to systematically cycle through all delivery
types, thus splitting the session into three sub-sessions based on bowling types. The appropriate instructions
would then be relayed to either the bowler or the coach responsible for overseeing the session.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:26 A. Khan et al.
4.7 Run-time Analysis
In order to assess the computational performance of the proposed framework (run-time analysis), we consider a
deployment scenario that enables quality analysis of batting shots. In order to meaningfully assess cricket batting,
session-wise analysis must be performed (i.e., not at the sample level). This level of assessment granularity allows
capturing trends in cricket batting including strengths and weaknesses of a batsman. For example, for scouting
purposes, where a large number of players are required to be assessed, a nal statistical representation of batting
quality is of greater importance (instead of manually analysing hundreds of shots that a player plays). This has to
be performed, not only session wise, but also longitudinally (across multiple sessions) to assess batting quality
including trends in improvements for example.
In our dataset, the average length of a batting session is 15
.
63
m±
5
.
44. All the shots played during these batting
sessions (see also Table 2) can be recognised in less than 2 seconds (per player and per level) on average using a
pre-trained model (see Section 3). Although the proposed system is capable of producing recognition results in
under 62ms per shot, session-wise assessment performance is more valuable. This performance was recorded
during the testing phase of our experiments for all folds (leave-one-player-out cross validation) and all levels
of the representation hierarchy. We performed our analysis on a desktop workstation with an i7 950 processor
clocked at 3
.
07GHz and 12GB DDR3 RAM. The system utilises multiple cores to provide greater performance and
is suitable for any stationary deployment. For mobile deployments state of the art processors can achieve similar
performance, enabling nearly-immediate feedback. For mobile deployments a pre-trained model can be deployed
directly on a mobile device.
In professional cricket, batting duration can vary a lot from lasting a single delivery (if a batsman gets out with
the rst ball bowled) to days in test cricket (the longest batting recorded in cricket history lasted for 970 minutes
over 4 days
3
). With our approach, statistical analysis of shots can be provided as soon as a batting session is over.
Considering the longest ever batting session of 970 minutes, our approach can produce results in 129.3 seconds
per level if all the shots are recognised at the end of the session. However, predicting shots as soon as the data is
recorded can also be performed allowing an even quicker feedback at the end of a batting session.
5 DISCUSSION
The overarching goal of our work is focused on developing technology that enables participation and positive
impact for everyone with specic attention to under-represented groups. This paper contributes to the wider
context of facilitating for the amateur and hobby sports community to foster wider uptake and sustained
engagement that promotes healthier living and societal engagement. Cricket is a tremendously popular sport
with millions and millions of supporters and hobby players all over the world. As with so many amateur sports,
professional assessment and coaching in cricket is typically not widely and easily accessible. It is the intention
behind our work to eventually overcome this shortage of professional support by developing and deploying
inexpensive thus widely accessible yet accurate and consistent automated assessment means that enable cricket
enthusiasts to objectively reect upon their skills and as such to develop as players. As an additional benet the
automated, objective assessment using an analysis system like the one we are developing will allow for entirely
new ways of engagement, such as through remote scouting as described below.
In this paper we presented the rst exploration regarding the general eectiveness of an automated cricket
assessment system that consists of inexpensive, wearable movement sensors and an automated, machine-learning
based sensor data analysis framework. We proposed a hierarchical shot recognition mechanism for accurately
identifying cricket shots with the aim of providing an objective, automated assessment system for coaches and
players. The proposed system is important for cricket due to the popularity and wide-reach of the sport [
45
], but
also because of the nancial inequalities that are common across the participating countries. The proposed system
3http://stats.espncricinfo.com/wi/content/records/284006.html
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:27
and hardware are inexpensive and relatively simple to set up, and thus can be used in developing countries and
by individuals who cannot aord expensive setups. Teams like the West Indies rely on players from a network of
islands, but each individual island is relatively nancially unstable compared to other cricketing countries like
India where domestic leagues generate billions of dollars over the course of two months [
29
]. This is also true
for developing cricket countries as well such as Zimbabwe, Afghanistan, and Kenya, where an overall lack of
nances makes it dicult for scouts to reach geographically-isolated talent. However, there is a wealth of talent
that rises from these countries, and a scalable and aordable system like the one proposed in this paper would
help reach youth players who otherwise would go unnoticed. Scouting can also be an issue in wealthier countries
like Australia and India, where large numbers of amateur players require the limited attention of scouts and
coaches. There is also a case to be made about the blind evaluation of cricketers, as there are concerns about the
representativeness of specic backgrounds at an elite level [
36
]. The presented system enables fair and objective
skill assessment.
Our system can provide a measure of batting quality through an analysis of the results of our activity recognition
system. By providing a detailed breakdown of the dierent aspects and types of shot categories used in a session,
we open up for a comparison between players. Using each aspect of the shot: bat-and-ball contact, foot positioning,
direction and shot type we can provide information to the players, which can be used to draw a comparison
between skilled and unskilled batsman. By highlighting the areas that are most profoundly dierent between
the amateur player and the professionals we provide feedback which can assist the coaching of amateur players.
Using this feedback players can focus their practice and track their improvements over time. For a beginner-level
player, it is most likely they will need to focus on the most important aspect for that level: hitting the ball.
Extending this for professional players – where each type of shot can be important to use in the appropriate
scenario – they will be interested to know if their shot success suers for certain types of shot, meaning the full
breadth of the information our system oers becomes important.
We developed informative visualisations analysing feet positions, attack/defence and distribution of shots
around the ground. These visualisations are automatically produced after a batting session and players/coaches
can observe weak points to improve upon. These visualisations can then be utilised by coaches to lter through
large numbers of players without the need to invest large amounts of time attending to batting sessions, or
to evaluate players from far away geographical locations without having to commit to lengthy travel. This is
especially important at early grass-roots levels where there is an inux of players and a limited number of scouts.
Using these visualisations, a coach can get an immediate picture of each aspiring cricketers’ technical prowess
before deciding on whether to invest any further time with observations and/or development.
The low-cost hardware and resulting visualisations can also be used by the individual players themselves
to improve their game. It is well known that a person’s feelings can aect the recollection of memories (e.g.
[
21
]), and this no dierent during lengthy cricket batting sessions where valuable details can be forgotten or
misremembered – for example shots played well or shots missed. When players can conveniently access detailed
graphics depicting their performance they can then tailor future training sessions to improving their weaker
shots.
5.1 The System in Action: A Vision for Cricket Kenya
To illustrate the system’s potential in facilitating remote scouting of players, we present the following deploy-
ment scenario. While every club employs their own distinct practices, we base this scenario on the reported
infrastructure information and likely practices.
Kenya is an up-and-coming cricketing nation with hopes of becoming a full member of the elite cricket group
(the International Cricket Council). However, travelling throughout the country can be challenging due to limited
railway routes and the poor condition of roads [
37
]. Thus, when a player displays some potential in a remote
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:28 A. Khan et al.
area of the country, a coach from Cricket Kenya (the central organisation in the country) may be unwilling
or even unable to travel a long distance to evaluate the potential of one player. Cricket Kenya are based in
the capital, Nairobi, which is generally well-connected to most areas of the country, and is also served by two
airports. However, accessing smaller towns is not always straightforward. Based on Cricket Kenya’s published
Development Program, schools and universities are important sources for talent [
7
]. Just focusing on universities,
a visit to Egerton University would take three and a half hours, while a visit to Meru University would take
nearly ve hours. Scouting universities further aeld would take ve and a half hours (University of Kabianga)
and just shy of seven hours for Maseno University. With regards to schools and colleges, there are hundreds in
Kenya spread out all over the country.
Therefore, instead of subjecting a scout to travel for long durations, the institution (e.g. school, university,
local cricket club) could simply request the sensors from Cricket Kenya who would then mail these small items.
Coaches at the institution could then download our app (which was not within scope of this paper) onto their
smartphones (internet access in Kenya is the highest per-person in Africa [
47
]) in preparation for the batting
sessions. Once the sensors arrive, the sessions can take place like any other batting sessions and all the data will
be recorded and analysed on the smartphone. Coaches then have the option of uploading the data to Cricket
Kenya’s servers (approximately 10 megabytes for a standard 15 minute session) or simply sending it on a memory
card alongside the returned sensors. Coaches at Cricket Kenya can then evaluate the player’s performance before
deciding whether to travel for a formal appraisal.
Of course, the process would be even simpler if the institution owned the sensors, which is not unrealistic
given the low cost of the hardware. In this case, the institution would simply need to upload the collected data
onto Cricket Kenya’s servers for analysis. In the case of poor internet connectivity, the data could be mailed to
Cricket Kenya on an inexpensive memory card.
5.2 Limitations & Future Work
Although our system provides a thorough representation of batting shots, there are certain aspects of batting
that also need to be assessed in order to fully judge a player’s batting calibre. These include, batting stance which
is the position a batsman assumes before playing a shot. Correct stance results in good shots played around
the ground. Similarly, batting shots can be modelled as a function of the bowling type. Our system could be
implemented alongside a similar recognition system for bowling, which would provide even more context to
the batting metrics and allow an even better automatic understanding of batting decisions. In essence, such
a combination would help coaches automatically evaluate the mental responsiveness of the player – i.e. their
response to dierent bowling types – which is another key performance indicator for batting skills.
As a future work, other aspects of cricket, including bowling and elding, can also be analysed using the
proposed system by representing these activities hierarchically. For example, understanding various types of
bowling and sub-activities within it can be of great value. For batting skill assessment, i.e., understanding the
quality level of a player, other aspects – in addition to the range of shots utilised – are also important to consider,
such as the mental aspect of the game. Other complex interactions can also be analysed such as understanding
the types of shots played by a batsman as a response to the type of bowling used by the bowler. Related to this is
the response time with respect to the ball leaving a bowler’s hand and the location of the ball hitting the bat in a
shot can also be considered. A generalised approach such as [
22
,
24
] can also be utilised in this context to assess
the quality of gameplay and rank players according to their skill levels. The method proposed in this paper can
also be used in other sports (such as baseball) by employing transfer learning techniques such as [
2
,
8
] (originally
proposed for singles and doubles tennis with extension to badminton).
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation • 62:29
6 CONCLUSION
The game of cricket is very popular worldwide, and while some countries are wealthy enough to sustain the
scouting and development of elite players, there are many that cannot aord the state-of-the-art technology
required to compete. We explored the suitability of an inexpensive sensor-based system and machine learning
based sensor data analysis techniques that require minimal setup to aid the development of cricket batters by
automatically recognising batting activities and visualising the results in standard diagrams summarising batting
sessions with the aim of improving weaknesses in their game. The generated visualisations are based on those
used by international coaches and broadcasts, and can help both players and coaches to pinpoint weaknesses in
the batting game by observing the distribution of shots and the foot movement associated with the shots. These
visualisations allow hobbyist players who don’t have access to a coach to hone their skill at cricket. Additionally,
the diagrams could allow scouts to obtain a overall sense of a player’s ability without being physically present,
facilitating their ability to perform preliminary scouting on a variety of players who would otherwise not be
considered due to geographical constraints. The system presented in this paper is the rst of its kind that enables
automated and objective assessment of batting skills in cricket using an inexpensive sensing infrastructure and
robust, machine learning based analysis techniques.
ACKNOWLEDGEMENTS
The authors would like to thank the administration of the South Northumberland Cricket Club, Newcastle
upon Tyne, for their support. The authors would also like to acknowledge the support of Open Lab, Newcastle
University where the majority of this work was conducted.
REFERENCES
[1]
Mark Albert, Santiago Toledo, Mark Shapiro, and Konrad Koerding. 2012. Using Mobile P hones for Activity Recognition in ParkinsonâĂŹs
Patients. Frontiers in Neurology 3 (2012), 158. https://doi.org/10.3389/fneur.2012.00158
[2]
I. Almajai, F. Yan, T. de Campos, A. Khan, W. Christmas, D. Windridge, and J. Kittler. 2012. Anomaly Detection and Knowledge Transfer in
Automatic Sports Video Annotation. Springer Berlin Heidelberg, Berlin, Heidelberg, 109–117.
[3]
American Appraisal. 2015. ON A STICKY WICKET; A Concise Report on brand values in the Indian Premier League. (2015). http:
//american-appraisal.in/AA-Files/Images_IN/PDF/BrandValuesIPL_April2015.pdf
[4] Axivity. 2015. WAX9 - 9-Axis Bluetooth Streaming IMU. (2015). http://axivity.com/les/resources/WAX9_Data_Sheet.pdf
[5] Marc Bächlin, Kilian Förster, and Gerhard Tröster. 2009. SwimMaster: a wearable assistant for swimmer. In Proc. UbiComp. ACM.
[6] A Busch and D James. 2007. Analysis of cricket shots using inertial sensors. The impact of technology on sport II (2007), 317–322.
[7] Cricket Kenya. 2015. (2015). http://cricketkenya.co.ke/structure.php
[8]
T E deCampos, A Khan, F Yan, N FarajiDavar, D Windridge, J Kittler, and W Christmas. 2013. A framework for automatic sports video
annotation with anomaly detection and transfer learning. In Machine Learning and Cognitive Science, collocated with EUCOGIII.
[9]
Davide Figo, Pedro C. Diniz, Diogo R. Ferreira, and João M. P. Cardoso. 2010. Preprocessing techniques for context recognition from
accelerometer data. Personal and Ubiquitous Computing 14, 7 (mar 2010), 645–662.
[10]
K. S. Gayathri, Susan Elias, and Balaraman Ravindran. 2015. Hierarchical activity recognition for dementia care using Markov Logic
Network. Personal and Ubiquitous Computing 19, 2 (2015), 271–285.
[11]
Benjamin H Groh, Martin Fleckenstein, and Bjoern M Eskoer. 2016. Wearable trick classication in freestyle snowboarding. In Wearable
and Implantable Body Sensor Networks (BSN), 2016 IEEE 13th International Conference on. IEEE, 89–93.
[12]
Benjamin H Groh, Thomas Kautz, Dominik Schuldhaus, and Bjoern M Eskoer. 2015. IMU-based trick classication in skateboarding. In
KDD Workshop on Large-Scale Sports Analytics.
[13]
N. Hammerla, R. Kirkham, P. Andras, and T. Plötz. 2013. On Preserving Statistical Characteristics of Accelerometry Data using their
Empirical Cumulative Distribution. In Proc. ISWC.
[14]
Nils Y. Hammerla, James M. Fisher, Peter Andras, Lynn Rochester, Richard Walker, and Thomas Plotz. 2015. PD Disease State Assessment
in Naturalistic Environments Using Deep Learning. In Proc. AAAI. AAAI Press, 1742–1748. http://dl.acm.org/citation.cfm?id=2886521.
2886562
[15]
Nils Y. Hammerla and Thomas Plötz. 2015. Let’s (Not) Stick Together: Pairwise Similarity Biases Cross-validation in Activity Recognition.
In Proc. Ubicomp.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
62:30 A. Khan et al.
[16]
Thomas Holleczek, Jona Schoch, Bert Arnrich, and Gerhard Tröster. 2010. Recognizing turns and other snowboarding activities with a
gyroscope. In Wearable Computers (ISWC), 2010 International Symposium on. IEEE, 1–8.
[17]
Ulf Jensen, Frank A. Dassler, Marcus Schmidt, Markus Hennig, Thomas Jaitner, and Björn Eskoer. 2014. A Mobile System to Investigate
Putting Kinematics in Motor Learning. In Book of Abstracts of the 19th Annual Congress of the European College of Sport Science,A.De
Haan, C. J. De Ruiter, and E. Tsolakidis (Eds.). 207–208. https://www5.informatik.uni-erlangen.de/Forschung/Publikationen/2014/
Jensen14-AMS.pdf
[18]
D Karmaker, AZME Chowdhury, MSU Miah, MA Imran, and MH Rahman. 2015. Cricket shot classication using motion vector. In
Computing Technology and Information Management (ICCTIM), 2015 Second International Conference on. IEEE, 125–129.
[19]
Thomas Kautz, Benjamin Groh, and Bjoern M Eskoer. 2015. Sensor fusion for multi-player activity recognition in game-sports. In
Workshop on Large-Scale Sports Analytics, 21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
[20]
Daniel Kelly, Garrett F Coughlan, Brian S Green, and Brian Cauleld. 2012. Automatic detection of collisions in elite level rugby union
using a wearable sensing device. Sports Engineering 15, 2 (2012), 81–92.
[21]
Elisabeth A Kensinger. 2011. What we remember (and forget) about positive and negative experiences. Psychological Science Agenda 1
(2011), 1–5.
[22]
Aftab Khan, Eugen Berlin, Sebastian Mellor, Robin Thompson, Nils Hammerla, Roisin McNaney, Patrick Olivier, and Thomas Plötz.
2016. How Did I Do?: Automatic Skill Assessment from Activity Data. GetMobile: Mobile Comp. and Comm. 19, 4 (March 2016), 18–22.
https://doi.org/10.1145/2904337.2904344
[23]
Aftab Khan, Nils Hammerla, Sebastian Mellor, and Thomas PlÃűtz. 2016. Optimising sampling rates for accelerometer-based human
activity recognition. Pattern Recognition Letters 73 (2016), 33 – 40. https://doi.org/10.1016/j.patrec.2016.01.001
[24]
Aftab Khan, Sebastian Mellor, Eugen Berlin, Robin Thompson, Roisin McNaney, Patrick Olivier, and Thomas Plötz. 2015. Beyond
Activity Recognition: Skill Assessment from Accelerometer Data. In Proc. UbiComp. ACM. https://doi.org/10.1145/2750858.2807534
[25]
Aftab Khan, James Nicholson, Sebastian Mellor, Daniel Jackson, Karim Ladha, Cassim Ladha, Jon Hand, Joseph Clarke, Patrick Olivier,
and Thomas Plötz. 2014. Occupancy Monitoring Using Environmental &#38; Context Sensors and a Hierarchical Analysis Framework.
In Proc. BuildSys. ACM. https://doi.org/10.1145/2674061.2674080
[26]
A. Khan, D. Windridge, and J. Kittler. 2014. Multilevel Chinese Takeaway Process and Label-Based Processes for Rule Induction in the
Context of Automated Sports Video Annotation. IEEE Transactions on Cybernetics 44, 10 (Oct 2014), 1910–1923. https://doi.org/10.1109/
TCYB.2014.2299955
[27]
Cassim Ladha, Nils Hammerla, Emma Hughs, Patrick Olivier, and Thomas Ploetz. 2013. Dog’s Life: Wearable Activity Recognition for
Dogs. In Proc. Int. Conf. Ubiquitous Comp. (UbiComp).
[28]
Cassim Ladha, Nils Y Hammerla, Patrick Olivier, and Thomas Plötz. 2013. ClimbAX: skill assessment for climbing enthusiasts. In Proc.
UbiComp. ACM.
[29]
Laghate, GauravSharma, Ravi Teja. 2016. (2016). http://m.economictimes.com/industry/services/advertising/
ipl-brand-valuation-soars- to-4- 16-billion-du-phelps/articleshow/52930766.cms
[30] Mallett, Ashley. 2012. (2012). http://www.espncricinfo.com/magazine/content/story/597862.html
[31]
David Minnen, Tracy Westeyn, Thad Starner, J Ward, and Paul Lukowicz. 2006. Performance metrics and evaluation issues for continuous
activity recognition. Performance Metrics for Intelligent Systems 4 (2006).
[32]
Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr, Matthias Kranz, Nils Hammerla, Patrick Olivier, and Thomas Plötz. 2012.
Gymskill: A personal trainer for physical exercises. In Pervasive Computing and Communications (PerCom), 2012 IEEE International
Conference on. IEEE, 213–220.
[33] Raúl Montoliu, Raúl Martín-Félez, Joaquín Torres-Sospedra, and Adolfo Martínez-Usó. 2015. Team activity recognition in Association
Football using a Bag-of-Words-based method. Human Movement Science 41 (2015), 165 – 178. https://doi.org/10.1016/j.humov.2015.03.007
[34]
Dan Morris, T Scott Saponas, Andrew Guillory, and Ilya Kelner. 2014. RecoFit: using a wearable sensor to nd, recognize, and count
repetitive exercises. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 3225–3234.
[35] MotionView. 2017. (2017). http://www.baseballcoachsystems.com/video-analysis-software-intro.php
[36] Muir, Hugh. 2015. (2015). https://www.theguardian.com/uk-news/2015/apr/13/england-cricket-problem-non-white- asian
[37]
Nations Encyclopedia. 2017. (2017). http://www.nationsencyclopedia.com/economies/Africa/
Kenya-INFRASTRUCTURE- POWER-AND-COMMUNICATIONS.html
[38]
Le Nguyen Ngu Nguyen, Daniel Rodríguez-Martín, Andreu Català, Carlos Pérez-López, Albert Samà, and Andrea Cavallaro. 2015.
Basketball Activity Recognition Using Wearable Inertial Measurement Units. In Proceedings of the XVI International Conference on
Human Computer Interaction (Interacción). ACM, 60:1–60:6. https://doi.org/10.1145/2829875.2829930
[39]
Hamed Pirsiavash and Deva Ramanan. 2012. Detecting activities of daily living in rst-person camera views. In Proc. CVPR. IEEE,
2847–2854.
[40]
Thomas Ploetz, Nils Hammerla, and Patrick Olivier. 2011. Feature Learning for Activity Recognition in Ubiquitous Computing. In Proc.
Int. Joint Conf. on Art. Intelligence (IJCAI).
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
Activity Recognition for Quality Assessment of Batting Shots in Cricket using a Hierarchical Representation
62:31
[41]
Thomas Plötz, Chen Chen, Nils Y. Hammerla, and Gregory D. Abowd. 2012. Automatic Synchronization of Wearable Sensors and
Video-Cameras for Ground Truth Annotation – A Practical Approach. In Proceedings of the 2012 16th Annual International Symposium on
Wearable Computers (ISWC) (ISWC ’12). IEEE Computer Society, Washington, DC, USA, 100–103. https://doi.org/10.1109/ISWC.2012.15
[42]
Attila Reiss and Didier Stricker. 2012. Introducing a New Benchmarked Dataset for Activity Monitoring. In Proc. ISWC. https:
//doi.org/10.1109/ISWC.2012.13
[43]
Bernhard Schölkopf and Alexander J. Smola. 2001. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and
Beyond. MIT Press, Cambridge, MA, USA.
[44]
Priya Singh. 2014. MYCA – Batting Skills. Missouri Youth Cricket Association. http://www.mocricket.org/materials/myca_batting_skills.
pdf
[45] Sporteology. 2016. Top 10 Most Popular Sports in The World. (2016). http://sporteology.com/top-10-popular-sports-world/7/
[46]
Emmanuel Munguia Tapia, Stephen S Intille, and Kent Larson. 2004. Activity recognition in the home using simple and ubiquitous
sensors. In International Conference on Pervasive Computing. Springer, 158–175.
[47] Wangalwa, Elyane. 2014. (2014). http://www.cnbcafrica.com/news/east-africa/2014/09/09/kenya- leads-internet/
[48]
Juanita R Weissensteiner, Bruce Abernethy, and Damian Farrow. 2011. Hitting a cricket ball: what components of the interceptive action
are most linked to expertise? Sports Biomechanics 10, 4 (2011), 324–338.
Received February 2017; revised May 2017; accepted September 2017
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, No. 3, Article 62. Publication date:
September 2017.
... In tennis, multiclass classification was used to detect and classify stroke type (Brzostowski & Szwach, 2018;Whiteside, Cant, Connolly, & Reid, 2017), and shot type (Anand et al., 2017;Ganser, Hollaus, & Stabinger 2021). A CNN architecture was used to differentiate types of shooting postures in basketball (Fan, Bi, Wang, Zhang, & Sun, 2021), an SVM model was used to classify throw type and approach in handball (van den Tillaar, Bhandurge, & Stewart, 2021) and to analyze shots in cricket (Khan, Nicholson, & Plötz, 2017). ...
... Grade level was specified in cricket (McGrath, Neville, Stewart, & Cronin, 2019), Australian football (Cust, Sweeting, Ball, & Robertson, 2021), tennis (Whiteside et al., 2017) and soccer . Other studies reported the amateur professionalism in basketball (Hollaus et al., 2020), badminton (Steels et al., 2020), cricket (Jowitt, Durussel, Brandon, & King, 2020;Khan et al., 2017;McGrath, Neville, Stewart, Clinning, & Cronin, J, 2021), golf (Kim & Park, 2020), handball (van den Tillaar et al., 2021), netball (Smith & Bedford, 2020), tennis (Brzostowski & Szwach 2018;Ganser et al., 2021), volleyball (Kautz et al., 2017;Wang et al., 2018), and years of experience was reported in basketball (Fan et al., 2021). Wang, Guo, and Zhao (2016) specified that the badminton players were either members of the university club or team. ...
... Detection methods to segment the motion states were reported in 22 studies. Window segmentation which involves splitting the sensor signals into distinct windows of time (measured in seconds), was the most common method of activity detection (Bo, 2022;Brzostowski & Szwach, 2018;Cust et al., 2021;Jowitt et al., 2020;Khan et al., 2017;Kim and Park, 2020;McGrath et al., 2019;McGrath et al. 2021;Reilly et al., 2021;Salim et al., 2020;Salman et al., 2017;Shahar, Ghazali, As✁ari, & Swee, 2020). Smith & Bedford, 2020;Steels et al., 2020;Tan & Xie, 2021;van den Tillaar et al., 2021;Whiteside et al., 2017;Zhang et al., 2021). ...
Article
Full-text available
There is an ever-present need to objectively measure and analyze sports motion for the determination of correct patterns of motion for skill execution. Developments in performance analysis technologies such as inertial measuring units (IMUs) have resulted in enormous data generation. However, these advances present challenges in analysis, interpretation, and transformation of data into useful information. Artificial intelligence (AI) systems can process and analyze large amounts of data quickly and efficiently through classification techniques. This study aimed to systematically review the literature on Machine Learning (ML) and Deep Learning (DL) methods applied to IMU data inputs for evaluating techniques or skills in individual swing and team sports. Electronic database searches (IEEE Xplore, PubMed, Scopus, and Google Scholar) were conducted and aligned with the PRISMA statement and guidelines. A total of 26 articles were included in the review. The Support Vector Machine (SVM) was identified as the most utilized model, as per 7 studies. A deep learning approach was reported in 6 studies, in the form of a Convolutional Neural Network (CNN) architecture. The in-depth analysis highlighted varying methodologies across all sports inclusive of device specifications, data preprocessing techniques and model performance evaluation. This review highlights that each step of the ML modeling process is iterative and should be based on the specific characteristics of the movement being analyzed.
... HCI research broadly explored sports technology for nondisabled persons. For example, systems were developed for endurance disciplines like running [100], cycling [144], or swimming [62,80] but also team-sports like handball [55,73], basketball [17], or cricket [59], and adventure sports or sports with smaller audiences, such as climbing [56], skiing [46], or martial arts [18]. There is a broad ACM Trans. ...
Article
Full-text available
Equitable access to sport for disabled people remains challenging, and technology is often viewed as a way of addressing barriers. However, little is known about how disability is approached in such research and the purpose of sport that is afforded to disabled people. We address this issue in a review of 60 publications in the field of Human-Computer Interaction. We leverage Template Analysis in combination with Mueller and Young’s lenses on virtues of sport to also explore the experiential side of sports technology for disabled people. Our results are threefold: (1) We show that disability shifts the intended purpose of sports technology away from leisure to health, and that technologies such as exergames are viewed as an opportunity to replace real-world sport to address barriers and increase motivation. (2) We highlight that in(ter)dependence plays a strong role in technology development, but that disabled people are not extensively involved in research. (3) We show that virtues beyond health as per Mueller and Young do apply to existing work, but that value frameworks need to be re-worked in the context of disability, placing a stronger emphasis on sport as leisure, and the enriching role that technology can play.
... Wearable devices were widely used to measure running activity and provide feedback to users [4], [5], [15]. Moreover, Khan et al. [11] utilized hierarchical representations by leveraging wrist-wearable devices to evaluate athletics' performance in cricket, and further developed a system for gymnastics and medical training [10]. LAX-score [9] is a score, calculated through physiological and motion signals, proposed by Jung et al. to quantify the team performance in lacrosse. ...
... However, similar studies published demonstrated successful applications with relatively small participant samples. Khan et al. [60] explored activity recognition in cricket using only six participants, mostly amateurs, highlighting the feasibility of such systems even with limited data. Hölzemann and Van Laerhoven [61] achieved promising results in recognizing basketball activities with IMUs using only three participants. ...
Article
Full-text available
1) Background: Tennis has changed toward power-driven gameplay, demanding a nu-anced understanding of performance factors. This review explores the role of machine learning in enhancing tennis performance. (2) Methods: A systematic search identified articles utilizing machine learning in tennis performance analysis. (2) Results: Machine learning applications show promise in psychological state monitoring, talent identification, match outcome prediction, spatial and tactical analysis, and injury prevention. Coaches can leverage wearable technologies for per-sonalized psychological state monitoring, data-driven talent identification, and tactical insights for informed decision-making. (4) Conclusions: Machine learning offers coaches insights to refine coaching methodologies and optimize player performance in tennis. By integrating these insights, coaches can adapt to the demands of the sport by improving the players' outcomes. As technology progresses, continued exploration of machine learning's potential in tennis is warranted for further advancements in performance optimization.
... Nicholson et al. [15] have proposed an Activity recognition technology using Hierarchical Representation to assess the quality of cricket shots. Automated recognition of cricket shots is done using low-cost hardware and a simple setup that allows players to analyze the statistics of a shot played. ...
Chapter
This book series aims to provide a forum for researchers from both academia and industry to share their latest research contributions in the area of computing technologies and Data Sciences and thus to exchange knowledge with the common goal of shaping the future. The best way to create memories is to gather and share ideas, creativity and innovations. The content of the book is as follows
... While other fields focus on reducing the number and size of sensors used in research, the sports industry is adopting multimodal sensors to gain a comprehensive understanding of player movements and physical states. These sensors facilitate various analyses, from simple posture classification to performance analysis, and include inertial measurement units (IMUs) [24][25][26][27][28][29] , eye trackers [30][31][32] , pressure sensors 33,34 , skeleton tracking sensors [35][36][37][38][39][40] , electromyography (EMG) sensors 41,42 , and capacitive sensors 43 . ...
Article
Full-text available
The sports industry is witnessing an increasing trend of utilizing multiple synchronized sensors for player data collection, enabling personalized training systems with multi-perspective real-time feedback. Badminton could benefit from these various sensors, but there is a scarcity of comprehensive badminton action datasets for analysis and training feedback. Addressing this gap, this paper introduces a multi-sensor badminton dataset for forehand clear and backhand drive strokes, based on interviews with coaches for optimal usability. The dataset covers various skill levels, including beginners, intermediates, and experts, providing resources for understanding biomechanics across skill levels. It encompasses 7,763 badminton swing data from 25 players, featuring sensor data on eye tracking, body tracking, muscle signals, and foot pressure. The dataset also includes video recordings, detailed annotations on stroke type, skill level, sound, ball landing, and hitting location, as well as survey and interview data. We validated our dataset by applying a proof-of-concept machine learning model to all annotation data, demonstrating its comprehensive applicability in advanced badminton training and research.
... Regardless of the number of informational cues, or the quality of these, they must all be used by batters to produce a singular batting stroke. While there are over 25 different batting strokes (based on human annotations of stroke types commonly used in performance analysis), most of them are classified as front-foot or back-foot strokes based on the batter's gross body movement -whether the batter moves forward or backward from their original stance (Khan et al., 2017). These front-foot or back-foot strokes have been studied in relation to the biomechanical and kinematic determinants of individual strokes such as the front foot drive (Peploe et al., 2014;Stretch et al., 1998), but have not been studied together in an interactive, competitive environment where they serve as decision-driven actions. ...
Article
Full-text available
This study aims to investigate the alternative model structure based on a feature selection algorithm on multiple features-framework of human activity recognition (HAR) via wearable sensor-based modality. Neighborhood component analysis (NCA) is a linear transformation that maximizes the accuracy of specified classification events used as the benchmark for the proposed algorithm. Also, the effect of different combinations of sensor configurations of two, three, and all four sensors on the performance of the developed model was studied. The effectiveness and shortcomings of best sensor configuration were highlighted. Results were compared between different sensor configurations and benchmark HAR dataset. To maximize the regularization of NCA, fine-tuning the algorithm to maximize relevance and minimize redundancy (MRMR) was proposed. Results demonstrated that RNCA-MRMR could establish an efficient algorithm that can satisfy the model validation tests with significant advantages over feature number and predictive accuracy at 93.5%, 93.7%, and 94.5% for two, three, and all four sensors respectively. Furthermore, the adaptability of RNCA-MRMR to different data characteristics has ensured an optimal and task-specific representation of the data. In essence, the combined strength of RNCA and MRMR provides a versatile and effective approach for extracting meaningful features and enhancing the overall performance of machine learning models.
Research
Full-text available
This research paper presents a comprehensive approach for automated classification of individuals on the cricket field as umpires or non-umpires, along with gesture and shot recognition. The proposed system leverages CNN, LSTM, and MediaPipe frameworks to enhance decision-making processes and provide real-time analysis in cricket matches. To achieve this, a custom dataset was created for cricket shot detection, while a pre-existing dataset called SNOW was utilized for umpire/non-umpire and umpire gesture detection. The results demonstrate the effectiveness of the approach, with an accuracy of 86.93% in umpire and non-umpire classification, 80.51% in umpire gesture detection, and 83.34% in cricket shot detection. The recognized gestures include No Action, Wide, Six, No ball, and Out, while the classified shots encompass Pre Stance, Stance, Pull Shot, and Straight Drive. The paper includes comparative analysis, discussions on the advantages and limitations of the proposed approach, and insights into future research directions.
Article
Full-text available
Human activity recognition (HAR), i.e., the automated detection and classification of specific activities that a person pursues, is one of the core concerns of mobile and ubiquitous computing. Multimodal sensing facilities of modern mobile devices allow for detailed capture of contextual information, most importantly movement data recorded with inertial measurement units that are now standard in most mobile devices. The majority of HAR applications aim at automatically documenting when something of interest has happened and what that was. For example, the popular moves app on iOS and Android devices "automatically records any walking, cycling, and running [a user does]" [7] and as such automatically generates a life log for those interested in their daily movement patterns. Beyond the mere recognition of certain activities of interest, few applications currently go a step further and analyze the quality of a person's activities, i.e., how (well) their activities were performed, which directly corresponds to a person's abilities or skills.
Conference Paper
Full-text available
The use of wearable sensors for automatic recognition of human activities has pervaded both professional and recreational sports. While many activities involving only a single athlete can be classified robustly, the automatic classification of complex activities involving several athletes is still in its infancy. In this paper, we present a novel approach for the recognition of such multi-player activities in the context of game sports. Our method is based on the fusion of position measurements with inertial measurements in a set of interaction features. We demonstrate the efficacy of our method in the recognition of tackles and scrums in Rugby Sevens. The results of our current work suggest that the proposed features can be leveraged to achieve classification accuracies of more than 97%.
Conference Paper
Full-text available
The popularity of skateboarding continuously grows for athletes performing the sport and for spectators following competitions. The presentation and the assessment of the ath-letes' performance can be supported by state-of-the-art motion analysis and pattern recognition methods. In this paper, we present a trick classification analysis based on motion data of inertial measurement units. Six tricks were performed by seven skateboarders. A trick event detection algorithm and four different classification methods were applied to the collected data. A sensitivity of the event detection of 94.2 % was achieved. The classification of correctly detected trick events provides an accuracy of 97.8 % for the best performing classifiers. The proposed algorithm holds the potential to be extended to a real-time application that could be used to make competitions fairer, to better present the assessment to spectators and to support the training of athletes.
Conference Paper
Full-text available
Cricket shots cannot be detected yet from single video sample without multiple view camera and other tools like sonar, speedometer. Extracting salient feature and optical flow from videos of cricket shots is still a challenge. In cricket, body parts movement created several different directional optical flows. So we propose two approaches related to classifying the shots. Our methodology defines 8 classes of angle ranges to detect cricket shots. Our method is grounded on Motion vectors that help to measure the angle of any precise cricket shot. An adequate accuracy level for the shots is established for this particular approach.
Conference Paper
Full-text available
The next generation of human activity recognition applications in ubiquitous computing scenarios focuses on assessing the quality of activities, which goes beyond mere identification of activities of interest. Objective quality assessments are often difficult to achieve, hard to quantify, and typically require domain specific background information that bias the overall judgement and limit generalisation. In this paper we propose a framework for skill assessment in activity recognition that enables automatic quality analysis of human activities. Our approach is based on a hierarchical rule induction technique that effectively abstracts from noise-prone activity data and assesses activity data at different temporal contexts. Our approach requires minimal domain specific knowledge about the activities of interest, which makes it largely generalisable. By means of an extensive case study we demonstrate the effectiveness of the proposed framework in the context of dexterity training of 15 medical students engaging in 50 attempts of surgical activities.
Conference Paper
Full-text available
Analysis of human movement is a growing research area within the field of sport monitoring which aims to enhance the performance of athletes, predicting injuries or optimizing training programs. Camera-based techniques are the most spread method to evaluate although frequently this method can be cumbersome and, furthermore, specific movements where performance is analyzed are not possible to distinguish. Wearable inertial systems however, are capable to ameliorate this matter. This paper presents a new wearable sensing system with the aim to record human movements in the field of sport. A new paradigm is presented with the purpose of monitoring basketball players with multiple inertial measurement units. A data plan collection has been designed and experimental results show the potential ability of the system in basketball activity recognition.
Book
A comprehensive introduction to Support Vector Machines and related kernel methods. In the 1990s, a new type of learning algorithm was developed, based on results from statistical learning theory: the Support Vector Machine (SVM). This gave rise to a new class of theoretically elegant learning machines that use a central concept of SVMs—-kernels—for a number of learning tasks. Kernel machines provide a modular framework that can be adapted to different tasks and domains by the choice of the kernel function and the base algorithm. They are replacing neural networks in a variety of fields, including engineering, information retrieval, and bioinformatics. Learning with Kernels provides an introduction to SVMs and related kernel methods. Although the book begins with the basics, it also includes the latest research. It provides all of the concepts necessary to enable a reader equipped with some basic mathematical knowledge to enter the world of machine learning using theoretically well-founded yet easy-to-use kernel algorithms and to understand and apply the powerful algorithms that have been developed over the last few years.
Conference Paper
The ability to generalise towards either new users or unforeseen behaviours is a key requirement for activity recognition systems in ubiquitous computing. Differences in recognition performance for the two application cases can be significant, and user-dependent performance is typically assumed to be an upper bound on performance. We demonstrate that this assumption does not hold for the widely used cross-validation evaluation scheme that is typically employed both during system bootstrapping and for reporting results. We describe how the characteristics of segmented time-series data render random cross-validation a poor fit, as adjacent segments are not statistically independent. We develop an alternative approach -- meta-segmented cross validation -- that explicitly circumvents this issue and evaluate it on two data-sets. Results indicate a significant drop in performance across a variety of feature extraction and classification methods if this bias is removed, and that prolonged, repetitive activities are particularly affected.
Conference Paper
Digital motion analysis in freestyle snowboarding requires a stable trick detection and accurate classification. Freestyle snowboarding contains several trick categories that all have to be recognized for an application in training sessions or competitions. While previous work already addressed the classification of specific tricks or turns, there is no known method that contains a full pipeline for detection and classification of tricks from multiple categories. In this paper, we suggest a classification pipeline containing the detection, categorization and classification of tricks of two major freestyle trick categories. We evaluated our algorithm based on data from two different acquisitions with a total number of eleven athletes and 275 trick events. Tricks of both categories were categorized with recall results of 96.6% and 97.4%. The classification of the tricks was evaluated to an accuracy of 90.3% for the first and 93.3% for the second category.