Conference PaperPDF Available

Laboratory versus Ecological Running: A Comparison of Foot Strike Angle and Pattern Estimation

Authors:

Abstract

The purpose of the current study was to evaluate the ecological validity of two previously developed laboratory-based random forest machine learning models and train two new ecologically valid models for the 1) prediction of foot strike angle (FSA) and 2) classification of foot strike pattern (FSP) from wearable insoles during running. The original models performed worse with track-surface running data inputs than in their original validation (prediction RMSE = 6.84° vs. 3.65°, classification accuracy = 79.5% vs. 94.1%). The new models, trained using track-surface data, improved the estimation of FSA (RMSE = 4.10°) and FSP (accuracy = 84.8%). To ensure estimation accuracy, future models should be trained with respect to the environment/conditions in which they will be implemented. INTRODUCTION: The use of wearable sensors has enabled the collection of immense biomechanical information of sporting activities in their natural environments. Further, the employment of robust machine learning prediction and classification techniques on wearable sensor data can diversify the information collected from these sensors. Machine learning models work by capitalizing on distinctive features of large-scale data sets, thus the model training inputs should be accurate and reliable (Alpaydin, 2014). Training such models using data collected in the laboratory setting is beneficial because the environment can be controlled and gold-standard equipment employed. However, these benefits come with some challenges, in that biomechanically equipped laboratories are often limited in space, which may affect the nature of movement. This is especially relevant for movements like over ground running. In these cases, although lab-based models can be trained with high quality data inputs, the resulting model may not be fully applicable to more ecologically valid implementations. Alternatively, models trained with high ecological validity may lack gold-standard references for accurate model training due to equipment and logistical limitations. Ultimately, the accuracy of modelling techniques should be investigated specific to practical, in-field applications. Thus, the purpose of the current study was to evaluate the accuracy and precision of two previously trained and validated laboratory-based random forest machine learning models (Moore et al., 2020) for the 1) prediction of foot strike angle (FSA) and 2) classification of foot strike pattern (FSP) when data from running on a track-surface was used in the model execution. An additional purpose of the current study was to train two new models using the independent and dependent variables from track-surface running as data inputs for comparison to the original lab-based models.
LABORATORY VERSUS ECOLOGICAL RUNNING: A COMPARISON OF FOOT
STRIKE ANGLE AND PATTERN ESTIMATION
Stephanie R. Moore1, Christina Kranzinger2, Gerda Strutzenberger3,
Magdalena Taudes4, Aaron Martinez1, Hermann Schwameder1, Josef Kröll1
Department of Sport and Exercise Science, University of Salzburg, Austria1
Salzburg Research Forschungsgesellschaft m.b.H., Salzburg, Austria2
Motion Analysis Zurich, Department of Orthopaedics, Balgrist University
Hospital, Children's Hospital, University of Zurich, Switzerland3
University of Applied Science Technikum Wien, Vienna, Austria4
The purpose of the current study was to evaluate the ecological validity of two previously
developed laboratory-based random forest machine learning models and train two new
ecologically valid models for the 1) prediction of foot strike angle (FSA) and 2) classification
of foot strike pattern (FSP) from wearable insoles during running. The original models
performed worse with track-surface running data inputs than in their original validation
(prediction RMSE = 6.84° vs. 3.65°, classification accuracy = 79.5% vs. 94.1%). The new
models, trained using track-surface data, improved the estimation of FSA (RMSE = 4.10°)
and FSP (accuracy = 84.8%). To ensure estimation accuracy, future models should be
trained with respect to the environment/conditions in which they will be implemented.
KEYWORDS: track running, random forest, pressure insoles
INTRODUCTION: The use of wearable sensors has enabled the collection of immense
biomechanical information of sporting activities in their natural environments. Further, the
employment of robust machine learning prediction and classification techniques on wearable
sensor data can diversify the information collected from these sensors. Machine learning
models work by capitalizing on distinctive features of large-scale data sets, thus the model
training inputs should be accurate and reliable (Alpaydin, 2014). Training such models using
data collected in the laboratory setting is beneficial because the environment can be controlled
and gold-standard equipment employed. However, these benefits come with some challenges,
in that biomechanically equipped laboratories are often limited in space, which may affect the
nature of movement. This is especially relevant for movements like over ground running. In
these cases, although lab-based models can be trained with high quality data inputs, the
resulting model may not be fully applicable to more ecologically valid implementations.
Alternatively, models trained with high ecological validity may lack gold-standard references
for accurate model training due to equipment and logistical limitations. Ultimately, the accuracy
of modelling techniques should be investigated specific to practical, in-field applications.
Thus, the purpose of the current study was to evaluate the accuracy and precision of two
previously trained and validated laboratory-based random forest machine learning models
(Moore et al., 2020) for the 1) prediction of foot strike angle (FSA) and 2) classification of foot
strike pattern (FSP) when data from running on a track-surface was used in the model
execution. An additional purpose of the current study was to train two new models using the
independent and dependent variables from track-surface running as data inputs for
comparison to the original lab-based models.
METHODS: The original models were trained and tested using independent variables collected
from two-part wearable pressure insoles (separated by anterior/posterior sections) and a
dependent, criterion measure of FSA or FSP from 3D marker-based kinematics (Moore et al.,
2020). The current data collection closely resembled that of the original data collection in order
to create comparable independent variable inputs for use in the model (Moore et al., 2020). To
explore the whole spectrum foot strike patterns, researchers in the previous study investigated
125
Moore et al.: LABORATORY VERSUS ECOLOGICAL RUNNING: A COMPARISON OF FOOT STRIKE
Published by NMU Commons, 2021
six types of FSPs: participants’ natural pattern (NA), rear foot (RF), mid foot (MF), fore foot
(FF), extreme-RF, and extreme-FF patterns. However, it was reported that participants found
it difficult to execute MF steps (ultimately limiting the amount of MF steps in model training)
and that “extreme” conditions felt very unnatural to them. Therefore, the current study collected
an additional MF trial and removed the “extreme” conditions to account for these issues. As a
result, in the current study, 13 recreational male and female runners (8 men, 5 women; Mean
± SD; 1.75 ± 0.09 m, 70.9 ± 11.7 kg, 31.3 ± 7.5 yr) were asked to complete five trials with the
NA condition performed first, followed by trials of the RF, MF1, MF2, and FF strike patterns in
a randomized order. Each trial consisted of 3-4 100 m loops on an indoor track surface (to
ensure 20 steps were collected per condition per participant) at a self-selected, comfortable
speed (3.34 ± 0.63 m∙s-1). The capture volume was a 15 m straight phase, allowing anywhere
between 4 and 14 steps to be collected per lap (dependent on the use of one or two working
insoles). In total, after data loss and processing, 1,244 steps with the following pattern
distribution were included in the current analyses: RF = 671, MF = 436, FF = 137.
Participants were equipped with LoadsolTM wearable pressure insoles (Novel GmbH; Munich,
Germany) over the insole of their personal running shoes. The insole system recorded at its
maximum sampling rate (100 Hz). Three-dimensional (3D) motion capture of both feet was
collected using a 10-camera Qualysis motion capture system (2020.2, Göteborg, Sweden).
Retroreflective markers were attached at the medial and lateral malleoli, the head of the 2nd
metatarsal, the heel (placed at the same height as the 2nd metatarsal) and the lateral side of
the 1st and 5th metatarsals. Marker trajectories were captured at 200 Hz and filtered using a 15
Hz low-pass filter (> 99% of signal power retention; Moore et al., 2020). The foot-segments
were modelled in Visual 3D X64 Professional (v6.03.06; Germantown, MD, USA) and virtual
foot-segments were modelled ensuring that the shoe-elicited angulation was negated (C-
Motion, Inc., 2017). The resulting FSA was extracted relative to the ground surface.
The subsequent processing steps were performed using a custom code in MATLAB (R2020a;
The MathWorks, Inc., Natick, MA, USA). First, the kinematic and insole data were synchronized
using a stomp event performed before and after each trial. The instant of ground contact of the
stomps was found via the force threshold defined by Seiberl and colleagues (2018) for the
insoles, and the peak positive acceleration of the foot segment for the kinematic data. Due to
the difference in sample rate between the systems, the kinematic data was down-sampled
(using linear interpolation) to match the LoadsolTM data length (at 100 Hz).
Initial contact and toe off events for each running step were determined using the same
thresholds used for stomp detection (Seiberl et al., 2018). The ten time and force related
variables (i.e. independent variables; including impulse ratio, peak force, and peak rate of force
development during the first 33% or 100% of the stance phase) in the original model were
computed from the LoadsolTM sensors between the initial contact and toe off events (Moore et
al., 2020). Finally, the angle of the foot segment at initial contact (i.e. FSA) was extracted for
each step performed within the capture volume as a criterion measure.
Following the methods of the original study, the LoadsolTM independent variables of the 1,244
track-surface steps collected were fed into the original random forest models in the manner of
a validation set (Moore et al., 2020). One random forest model was used to predict the FSA
(FRSTPRED), while the second served to classify the FSP (FRSTCLASS). Therefore, the models
estimated the FSA and FSP using the insole data. To assess the accuracy and precision of
the models use on the ecological (track-surface) running data, the estimates were compared
to the kinematic ground truth measure. The accuracy of the FRSTPRED was reported as the root
mean squared error (RMSE) and mean absolute error (MAE) of the estimated vs. criterion
FSA. The precision was quantified as the range of the Bland-Altman 95% limits of agreement
(Bland & Altman, 2010). The criterion measure of the FSP was pre-classified as RF (FSA >
8.0°), MF (−1.6° ≤ FSA ≤ 8.0°), or FF (FSA < 1.6°) using the recommendations of Altman and
Davis (2012). Subsequently, the FRSTCLASS was assessed using confusion matrices and model
accuracy, classifier recall, and classifier precision were reported in the same manner as the
original assessment of the models (for a detailed explanation see Moore et al., 2020).
Finally, two new random forest models were trained and assessed using the same methods
as the previous model-development (Breiman, 2001), however the track-surface running data
126
ISBS Proceedings Archive, Vol. 39 [2021], Iss. 1, Art. 33
https://commons.nmu.edu/isbs/vol39/iss1/33
was used as the input for the model training. As a result, 70% of the track-surface steps (n =
870) were used for model training, while 30% were used for model validation (n = 374). The
aims of the models were the same: FSA prediction and FSP classification (henceforth referred
to as ECO_FRSTPRED and ECO_FRSTCLASS, respectively). The most important independent
variables for the new models were determined by ranking those with the highest “mean
decrease Gini” (Calle & Urrea, 2011).
RESULTS: The accuracy and precision of the FRSTPRED and FRSTCLASS models with track-
surface running inputs are presented in Table 1. When fed into the original FRSTPRED model,
the track-surface data had worse performance when compared to the original validation set
(greater RMSE, MAE, bias and precision; Table 1). Similarly, the accuracy of the FRSTCLASS
for predicting FSP during track-surface running was 14.6 percentage points worse than the
accuracy of the original validation set (Table 1). The performance of the new models
(ECO_FRSTPRED and ECO_FRSTCLASS) are also presented in Table 1. Track-surface running
FSA was predicted with less error and improved Bland-Altman bias and precision when using
the new model. All classification performance metrics were greater when the track-surface
running was assessed with the ECO_FRSTCLASS (as opposed to the original model). The most
important variable (highest mean decrease Gini) was the peak rate of force development ratio
between the aft insole sensor region and the total foot (Peak RFD_Aft) for both new models.
Table 1. The performance metrics of two previously published random forest models (FRSTPRED and
FRSTCLASS) are presented with track-surface running inputs. For ease of comparison, the accuracy
and precision of the models using the original laboratory data (validation set) is also included. Further,
the performance of the validation set of two new random forest models (ECO_FRSTPRED and
ECO_FRSTCLASS) using the track-surface data set are also presented.
Model
Original Lab Models*
New Ecological
Models
Data set used for
accuracy/precision
assessment
Original Lab
Validation Set
(steps = 1,047)
Track-Surface
Validation Set
(steps = 1,244)
30% Track-Surface
Validation Set**
(steps = 374)
FSA Prediction model
FRSTPRED
FRSTPRED
ECO_FRSTPRED
RMSE
3.65
6.84
4.10
MAE
2.69
4.78
2.98
B-A Bias
-0.11°
-1.42°
-0.36°
B-A Precision
14.30°
26.23°
16.02°
FSP Classification model
FRSTCLASS
FRSTCLASS
ECO_FRSTCLASS
Model Accuracy
(%)
ALL
94.1
79.5
84.8
Classifier Recall
(%)
RF
96.4
87.6
90.4
MF
76.7
78.1
80.7
FF
96.3
44.5
70.3
Classifier Precision
(%)
RF
97.0
86.3
87.7
MF
74.8
69.6
79.6
FF
95.9
83.6
89.7
*the original models used were consistent with those published by Moore et al., 2020; ** 70% of the
data were used for model development; FSA = foot strike angle; FSP = foot strike pattern; B-A =
Bland-Altman; RF = rear foot, MF = mid foot, FF = fore foot
DISCUSSION: All accuracy and precision metrics for the prediction of FSA and classification
of FSP indicated inferior performance of laboratory-trained random forest models when
implemented with track-surface running data inputs (Table 1). Further, when new models with
the same purpose were trained using the track-surface running data, the models performed
better to predict and classify foot strike from track-surface running than the models developed
in the lab. However, the original lab-based models and lab-based validation data set (Moore
et al., 2020) boasted better performance than the new models (i.e. when track-surface running
127
Moore et al.: LABORATORY VERSUS ECOLOGICAL RUNNING: A COMPARISON OF FOOT STRIKE
Published by NMU Commons, 2021
was used in model training). Importantly, the original models (FRSTPRED and FRSTCLASS) were
trained using a larger and more diverse data set (participants = 30, cases used for model
training = 2,442 steps) than the newly trained models (ECO_FRSTPRED and ECO_FRSTCLASS).
However, the original data set had a low percentage of MF steps in the validation set, which
also had the lowest recall and precision of the FRSTCLASS. Therefore, the increased number of
MF steps included in the track-surface running likely influences the overall model accuracy (as
they are most likely to be mis-classified/predicted). Further, the average speed of the
participants had a greater variability when track running was assessed (standard deviation of
the average running velocity = 0.63 m∙s-1 vs. 0.40 m∙s-1 during the original laboratory trials).
Therefore, the better accuracy and precision of the original models may be due to the greater
number of data inputs in model training, the lower percentage of MF strikes included in the
original validation, and the more uniform speed of the participants.
The most important variable (Peak RFD_Aft) of the ECO_FRSTPRED and ECO_FRSTCLASS were
consistent with each other and that of the original FRSTPRED/FRSTCLASS models (Moore et al.,
2020). This indicates that the foot strike of both the laboratory and track-surface running can
be distinguished from similar variables. However, because there was more predictive error of
FRSTPRED and lower classification accuracy of FRSTCLASS when implemented with track-
surface data, there is likely differences in the features of the independent variables. Therefore,
future models should be trained with respect to the conditions (i.e. track-surface or laboratory)
where the models will be implemented. Importantly, the insole-based independent variables
may also be altered with running velocity differences, shoe midsole design, or running style
(Seiberl et al., 2018) however further investigations are needed to determine their influence on
the efficacy of random forest models to predict and classify foot strike during running.
Although the ECO_FRSTCLASS had a lower overall accuracy than FRSTCLASS (84.8% vs. 94.1%,
respectively), the new model was able to classify MF strike patterns with greater recall (+ 4.0
percentage points) and precision (+ 4.8 percentage points). This suggests that the larger
number of MF strikes in the training data set may have improved the MF model performance.
CONCLUSION: The current ecological (track-surface) validation of two previously-developed
random forest machine learning models proves the robustness of the modelling technique in
that the main variables for estimation of both types of running are similar. However, to obtain
the highest accuracy and precision during implementation, future models should be trained
with respect to the environment in which they will be implemented.
REFERENCES
Alpaydin, E. (2014). Introduction to Machine Learning (3rd ed.). Massachusetts Institute of Technology.
Altman, A. R., & Davis, I. S. (2012). A kinematic method for footstrike pattern detection in barefoot and
shod runners. Gait & Posture, 35(2), 298300.
Bland, J. M., & Altman, D. G. (2010). Statistical methods for assessing agreement between two methods
of clinical measurement. International Journal of Nursing Studies, 47(8), 931936.
https://doi.org/10.1016/j.ijnurstu.2009.10.001
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5.
Calle, M. L., & Urrea, V. (2011). Letter to the Editor: Stability of Random Forest importance measures.
Briefings in Bioinformatics, 12(1), 8689. https://doi.org/10.1093/bib/bbq011
C-Motion, Inc. (2017). Visual3D Wiki Documentation. https://c-
motion.com/v3dwiki/index.php?title=Main_Page
Moore, S., Kranzinger, C., Fritz, J., Stöggl, T., Kröll, J., & Schwameder, H. (2020). Foot strike angle
prediction and pattern classification using LoadsolTM wearable sensors: A comparison of machine
learning techniques. Sensors, 20, 6737. https://doi.org/10.3390/s20236737
Seiberl, W., Jensen, E., Merker, J., Leitel, M., & Schwirtz, A. (2018). Accuracy and precision of loadsol®
insole force-sensors for the quantification of ground reaction force-based biomechanical running
parameters. European Journal of Sport Science, 18(8), 11001109.
https://doi.org/10.1080/17461391.2018.1477993
ACKNOWLEDGEMENTS: This work was partly funded by the Austrian Federal Ministry for
Climate Action, Environment, Energy, Mobility, Innovation and Technology, the Austrian
Federal Ministry for Digital and Economic Affairs, and the federal state of Salzburg.
128
ISBS Proceedings Archive, Vol. 39 [2021], Iss. 1, Art. 33
https://commons.nmu.edu/isbs/vol39/iss1/33
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The foot strike pattern performed during running is an important variable for runners, performance practitioners, and industry specialists. Versatile, wearable sensors may provide foot strike information while encouraging the collection of diverse information during ecological running. The purpose of the current study was to predict foot strike angle and classify foot strike pattern from Loadsol TM wearable pressure insoles using three machine learning techniques (multiple linear regression-MR, conditional inference tree-TREE, and random forest-FRST). Model performance was assessed using three-dimensional kinematics as a ground-truth measure. The prediction-model accuracy was similar for the regression, inference tree, and random forest models (RMSE: MR = 5.16 • , TREE = 4.85 • , FRST = 3.65 • ; MAPE: MR = 0.32 • , TREE = 0.45 • , FRST = 0.33 •), though the regression and random forest models boasted lower maximum precision (13.75 • and 14.3 • , respectively) than the inference tree (19.02 •). The classification performance was above 90% for all models (MR = 90.4%, TREE = 93.9%, and FRST = 94.1%). There was an increased tendency to misclassify mid foot strike patterns in all models, which may be improved with the inclusion of more mid foot steps during model training. Ultimately, wearable pressure insoles in combination with simple machine learning techniques can be used to predict and classify a runner's foot strike with sufficient accuracy.
Article
Force plates represent the “gold standard” in measuring running kinetics to predict performance or to identify the sources of running-related injuries. As these measurements are generally limited to laboratory analyses, wireless high-quality sensors for measuring in the field are needed. This work analysed the accuracy and precision of a new wireless insole forcesensor for quantifying running-related kinetic parameters. Vertical ground reaction force (GRF) was simultaneously measured with pit-mounted force plates (1 kHz) and loadsol® sensors (100 Hz) under unshod forefoot and rearfoot running-step conditions. GRF data collections were repeated four times, each separated by 30 min treadmill running, to test influence of extended use. A repeated-measures ANOVA was used to identify differences between measurement devices. Additionally, mean bias and Bland–Altman limits of agreement (LoA) were calculated. We found a significant difference (p < .05) in ground contact time, peak force, and force rate, while there was no difference in parameters impulse, time to peak, and negative force rate. There was no influence of time point of measurement. The mean bias of ground contact time, impulse, peak force, and time to peak ranged between 0.6% and 3.4%, demonstrating high accuracy of loadsol® devices for these parameters. For these same parameters, the LoA analysis showed that 95% of all measurement differences between insole and force plate measurements were less than 12%, demonstrating high precision of the sensors. However, highly dynamic behaviour of GRF, such as force rate, is not yet sufficiently resolved by the insole devices, which is likely explained by the low sampling rate.
Article
Footstrike patterns during running can be classified discretely into a rearfoot strike, midfoot strike and forefoot strike by visual observation. However, the footstrike pattern can also be classified on a continuum, ranging from 0% to 100% (extreme rearfoot to extreme forefoot) using the strike index, a measure requiring force plate data. When force data are not available, an alternative method to quantify the strike pattern must be used. The purpose of this paper was to quantify the continuum of foot strike patterns using an easily attainable kinematic measure, and compare it to the strike index measure. Force and kinematic data from twenty subjects were collected as they ran across an embedded force plate. Strike index and the footstrike angle were identified for the four running conditions of rearfoot strike, midfoot strike and forefoot strike, as well as barefoot. The footstrike angle was calculated as the angle of the foot with respect to the ground in the sagittal plane. Results indicated that the footstrike angle was significantly correlated with strike index. The linear regression model suggested that strike index can be accurately estimated, in both barefoot and shod conditions, in the absence of force data.
Article
In clinical measurement comparison of a new measurement technique with an established one is often needed to see whether they agree sufficiently for the new to replace the old. Such investigations are often analysed inappropriately, notably by using correlation coefficients. The use of correlation is misleading. An alternative approach, based on graphical techniques and simple calculations, is described, together with the relation between this analysis and the assessment of repeatability.
Article
The goal of this article (letter to the editor) is to emphasize the value of exploring ranking stability when using the importance measures, mean decrease accuracy (MDA) and mean decrease Gini (MDG), provided by Random Forest. We illustrate with a real and a simulated example that ranks based on the MDA are unstable to small perturbations of the dataset and ranks based on the MDG provide more robust results.