Pedestrian Activity Classification to Improve Human Tracking and Localization

Abstract

The advent of miniaturized sensing technology
(MEMS: Micro-Electro-Mechanical Systems) gave the possibility
to collect data on different aspects of human activities. E.g. it is
possible to recognize activities using hip-mounted sensing.
Furthermore a dynamic changing filter model could be developed
for the use in existing pedestrian localization systems (fusion
algorithm) depending on the human activity to improve the
estimated position. Nine degrees of freedom (DOF) sensors
(acceleration, angular rate and magnetic field), such as those
integrated in almost every smartphone, have been taken as the
basis. With the presented algorithm it is possible to distinguish
seven activities: standing, walking, running, lying, cycling,
throwing and entering or leaving a car. Each activity is identified
by using a linear classifier and a decision tree approach. The
linear classifier parameters are set by using the k-means
algorithm of the clustered features. The features for classification
process include quantities like variance, frequency component,
mean and the absolute value. Finally, when no periodical
movement pattern is detected, the system successfully
discriminates between standing and lying by estimating the
projection of the gravitational field vector.Measurement data was
collected with twenty subjects performing the seven different
activities. Using the algorithm, activities were classified correctly
with an accuracy of 91 %.An advantage of the presented method
is the simple implementation, because of the low complexity of
the algorithm. Furthermore, the algorithm works with low-cost
sensors and is independent of any infrastructure. Possible
applications are a indoor navigation scenarios in combination
with Wi-Fi localization or protection of vulnerable road users in
traffic situations by communicating the data to the vehicle.

Full text

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
Pedestrian Activity Classification to Improve Human
Tracking and Localization
Marcus Bocksch, Jochen Seitz, Jasper Jahn
Fraunhofer Institute for Integrated Circuits IIS
Nordostpark 93
Erlangen-Nuremberg, Germany
Email: marcus.bocksch@iis.fraunhofer.de
Abstract—The advent of miniaturized sensing technology
(MEMS: Micro-Electro-Mechanical Systems) gave the possibility
to collect data on different aspects of human activities. E.g. it is
possible to recognize activities using hip-mounted sensing.
Furthermore a dynamic changing filter model could be developed
for the use in existing pedestrian localization systems (fusion
algorithm) depending on the human activity to improve the
estimated position. Nine degrees of freedom (DOF) sensors
(acceleration, angular rate and magnetic field), such as those
integrated in almost every smartphone, have been taken as the
basis. With the presented algorithm it is possible to distinguish
seven activities: standing, walking, running, lying, cycling,
throwing and entering or leaving a car. Each activity is identified
by using a linear classifier and a decision tree approach. The
linear classifier parameters are set by using the k-means
algorithm of the clustered features. The features for classification
process include quantities like variance, frequency component,
mean and the absolute value. Finally, when no periodical
movement pattern is detected, the system successfully
discriminates between standing and lying by estimating the
projection of the gravitational field vector.Measurement data was
collected with twenty subjects performing the seven different
activities. Using the algorithm, activities were classified correctly
with an accuracy of 91 %.An advantage of the presented method
is the simple implementation, because of the low complexity of
the algorithm. Furthermore, the algorithm works with low-cost
sensors and is independent of any infrastructure. Possible
applications are a indoor navigation scenarios in combination
with Wi-Fi localization or protection of vulnerable road users in
traffic situations by communicating the data to the vehicle.
Keywords: motion classification, pedestrian activity recognition,
motion sensors, low-cost sensor, inertial sensor
I. I
NTRODUCTION
This The Fraunhofer Institute for Integrated Circuits (IIS)
develops different pedestrian localization systems based on
sensor fusion for different environments. We are working on a
seamless citywide indoor and outdoor Wi-Fi positioning
system (awiloc
®
) based on a database of so called fingerprints.
Reasons for choosing this approach are that fingerprinting is
reported to achieve higher precision than base station based
methods and in general positions of publically available access
points are unknown and subject to privacy protection [1]. Also,
we have a high precision Pseudolite System (PLIIS) for
localization in GPS-denied environments and GPS-Inertial
Navigation System (INS) positioning systems as well.
The heart of those systems are Bayesian estimators like
Kalman or particle filter. In each time step, the filter estimates
a new position based on the assumed motion model. For this
reason a correct motion model is very important for the
precision of the position. But what if the pedestrian changes
his motion state? An algorithm is needed to estimate the
pedestrian activity and adapt the motion model of the
localization filter dynamically.
Furthermore, an INS Localization system provides a
suitable position and orientation estimate only for a very
limited period of time. The first problem is initialization of an
inertial navigation system (INS) - an absolute position is
needed. Second, the systematic errors that are present in small
low-cost inertial sensors quickly accumulate to unacceptable
position errors. Such characteristics are not appropriate to
compute position by double integration of acceleration.
An alternative is to use the sensor signal pattern to classify
the motion. Depending on the motion, each step can be
detected and the length estimated [13] to compute the distance
travelled from the last known or estimated position. When the
information on the distance travelled is combined with the
azimuth information, the current position can be calculated
(Dead Reckoning) [1,7].
In this paper we present a pedestrian activity classification
algorithm based on IMU and magnetic field sensors. It can be
easily adapted to fit different sensor platforms.

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
II. M
ATERIALS AND METHODS
A. Measurement Systems
An IMU uses a combination of a three-axial accelerometer
and a three-axial angular rate sensor. With these sensors
accelerations and rotations in six degrees of freedom can be
measured [6].
In this work micro-electro-mechanical system (MEMS)
sensors are used. The reasons are their low cost, low power
consumption, small size and light weight. Such MEMS sensors
are available as three-axial accelerometer, three-axial angular
rate and three-axial magnetometers. Nowadays, they can be
found in various consumer products such as smart phones,
gaming devices, etc. For the mentioned reasons, these sensors
are a suitable base for pedestrian activity classification.
B. Hardware Systems
Figure 1 shows a self-designed hardware prototype for
demonstration purposes.
Figure 1. Demonstrator hardware for Pedestrian activity classification
The prototype consists of two parts. The first part is a belt-
worn sensor platform and the second is an Android based
tablet. Both modules communicate via Bluetooth to each other.
The battery-operated sensor platform (icos-X) works full
autonomous and can be hence worn near the center of mass of
the body. It is also a self-designed highly integrated electronic
system for various applications [4]. The icos-X with hardware
architecture is shown in Figure 2. For the demonstrator
hardware we use only the processor board, IMU board and the
powersupply board. The small package size makes it attractive
for use in pedestrian applications.
The data are measured with an accelerometer, gyroscope
and magnetometer. The accelerations and rotation rates are
measured in the range between ±2g and ±2000 °/s,
respectively. The range of the magnetometer is between ±0.7
Gauss. This sensitivity is sufficient for the mentioned
activities. The sample rate (fs) is set to 50 Hz.
Figure 2. Modular HW/SW platform (icos-X)
The data are measured with an accelerometer, a gyroscope
and a magnetometer. The accelerations and rotation rates are
measured in the range between ±2g and ±2000 °/s,
respectively. The range of the magnetometer is between ±0.7
Gauss. This sensitivity is sufficient for the mentioned
activities. The sample rate (fs) is set to 50 Hz.
These sensors are connected to the microprocessor, which
performs the signal processing. The algorithm results are
transmitted wireless to the Android device. At the end, the
classified activities are presented to the user as animations. In
addition the visualization-app shows the counted steps and the
estimated velocity [5, 14].
C. Feature Extraction
The classification algorithm distinguishes between
standing, walking, running, lying (on the ground), cycling,
throwing and entering- or leaving-a-car. Some features from
acceleration, angular rate and magnetic field describe similar
relations and are not complementary. To reduce not required
features the Stepwise-Selection-Method [3] was used.
The algorithm is based on the parameters which are
extracted from the measured data using a time window
∆
t. The
key sensor input for most of the activities is the accelerometer
data. Very important features are the simple moving average
value (SMA) and the variance (
s
2
) of the acceleration signal.
The SMA-value is computed from the acceleration vector
acc= (acc
x
acc
y
acc
z
) and the average in respect to the time
window
∆
t:

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
∑
∆+
=
++
∆
=
tt
tt
zyx
n
accaccacc
t
SMA
0
0
222
1
(1)
The variance value is the sum of each axis computed
variance of the acceleration signal:
∑
∆+
=
∆
=
tt
tt
mean
n
acc
t
acc
0
0
1
(2)
∑
∆+
=
−
∆
=
tt
tt
mean
n
accacc
t
0
0
||
1
²
σ
(3)
The advantage compared to some other features is the
independence from the orientation of the sensor module.
D. Data Collection Phase
The algorithm is calibrated and validated with data from 14
male and 6 female persons. The age of the persons was
between 20 and 30 years. All measurements for standing,
walking, running, cycling and entering or leaving a car were
performed on level asphalt ground. The data-collection phase
was split in four parts.
The first phase consisted of a 150 meter long way for the
activities walking and running. In second phase we collected
data for the activity bicycle riding. A standard city bicycle was
used without mechanical suspension. Each person had to ride
two different bicycles for one kilometer. In third phase we were
walking to the car, entering the car, sitting in the car, leaving
the car and go back to the start position. Two different cars
with 5 doors in normal size were used. Measurements like
open-the-door without entering the car, or pass-the-car were
also part of the phase. In phase four we collected data for lying
on the ground and the special state throwing. The state
throwing was measured by a two meter free fall of the sensor-
system.
E. Classification
Only one half of these data sets (training data) were used
for estimating the thresholds of the decision rules. The
computed features are drawn in scatter plot [8].
At first, the centers of gravity of each cluster were
determined. The most well-known and commonly used
partitioning method for that is the k-means algorithm.
2
1
),(
∑ ∑
= ∈
=
k
i Cim
ji
j
mpdistE
(4)
The k-means algorithm takes the input parameter, and
partitions a set of j features into k clusters [9]. dist is the
distance measure between a data point m
j
(mj,x|mj,y) and the
cluster center pi(pi,x|pi,y) with 1<= k <= N. N describes the
sum of all features. Both pi and m
j
are multidimensional. The
computed distance is based on the most widely used Euclidean
distance:
||),(
jiji
mpmpdist −=
(5)
E is the sum of the squared Euclidean distance for all
features in the data set. The aim is to minimize E for each
cluster. The derivation of E is the cluster center pi, it is
computed by the labeled trainings data.
0
)( =
∂
∂
i
i
p
pE
;
∑
∑
∈
∈
=
ij
ij
Cm
Cm
j
i
m
p1
(6)
At the end, each cluster is represented by its centroid. Two
centroids describe a straight line Pi,l with i != l. The
orthogonal line through the midpoint Mi,l of two centroids is
the result for thresholds Ti,l (straight line) of the classifications
[15].
xi,xl,
yl,xi,yi,xl,
xi,xl,
yi,yl,
,,,
pp
pppp
pp
pp
−
−
−
−
+=+= xnxmP
lilili
(7)








=
+
+
2
pp
;
2
pp
);(
yl,yi,
xl,xi,
,
yx
MMM
li
(8)







++−=
x
li
y
li
li
M
m
Mx
m
xT
,,
,
11
)(
(9)
F. Activity Classification Algorithm
At each step the algorithm will provide a new computed
feature. The decision rules are modeled with a decision tree:
1) Classifier state throwing:
If the acceleration sensor unit is in free-fall motion, the
SMA features (∆t = 1/fs) is close to zero. Therefore, it will
check periodically, if it is close to zero. If it is the case, the
state throwing is classified.
2) Classifier state lying (on the ground):
Measurements from the accelerometer have two major
contributions: local gravity and acceleration due to motion. The
gravity is taken as 1g (equal 9.81 m/s
2
). For that reasons, lying
is classified by the orientation of the earth-gravitational-field
vector. In this case, the wearing position must be known.
3) Classifier state running:
Typical characteristics for walking/running are found in
frequency domain (peak frequency and amplitude) and

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
variance [8]. The linking of two features like SMA and variance
provides non-overlapping clusters (see Figure 3).
Figure 3. Scatter plot to classificate running
4) Classifier state entering-a-car:
A vehicle consists mainly of ferromagnetic materials. If a
ferromagnetic material (e.g. hollow cylinder) is placed in a
homogeneous magnetic field, the field lines are distorted. The
magnetic field inside of the cylinder is also reduced [10]. This
effect occurs in similar fashion by the activity entering-a-car
(see Figure 4).
Figure 4. Change / Distortion of the earth magnetic field close to
ferromagnetic material (entering-a-car)
An indicator for the state is a minimum detection of the
filtered SMA value. After that it has to be checked, if the
current signal is reduced and if the amplitude of the variance is
low.
This characteristic is not sufficient enough, because passing
a metal door would yield similar results. A second
characteristic is the signal pattern of the angular rate sensor.
Each axis provides different information. The rotation of the
body is measured be the vertical component (upward). In this
way it is easy to recognize e.g. the door-open-phase. The
algorithm uses the data of the horizontal axis (toward the left
side of the body). This axis has the advantage that the
measured signals are independent of the entry side of the
vehicle. The movement into the vehicle produces a forward
movement with a subsequent backward movement of the body
into the seat. In the following door-close-phase the movements
have a similar pattern. Figure 5 shows the forward and
backward rotations of all persons. The characteristic can be
filtered by a simple band-pass filter with the cutoff frequencies
at 0.5 and 1.5 Hz . If the both conditions occur in a time span of
three seconds, the state is classified.
Figure 5. Signal pattern of the angular rate sensor (horizontal axis, toward to
the left side of the body): (a) time domain, (b) frequency domain

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
5) Classifier state standing:
If the variance of the acceleration sensor is below a certain
threshold, the state standing is classified.
6) Classifier state cycling or walking:
The difference between cycling and walking can be found
in the frequency domain. The middle step frequency for
walking is around 1.5 Hz [11]. In contrast with cycling, the
frequencies are nearly evenly distributed [2]. To distinguish
between these two states a bandpass (BP) and highpass (HP)
filter is used to get the characteristics. Movements with and
without ground contact is classified by a simple threshold value
of the BP/HP value. To make the classification more reliable,
the feature for cycling must occur multiple times. If the
verification for cycling is true, cycling is classified. If not,
walking is detected.
Figure 6 displays the state transition model of all pedestrian
activities. It is optimized to reduce false detections. At first, it
is initialized with the state standing. From class standing it can
be changed to all other classes and vice versa. It is also possible
to stay in each class. Not all transitions are allowed e.g. from
the class lying it is not possible to change to falling or walking.
Figure 6. State transition model of all motions
III. R
ESULTS
The results of the classification were obtained from the
other half of the measured data. Table I shows the classification
results in percent.
TABLE I. C
LASSIFICATION
R
ESULTS
labeled data
values / percent
standing
walking
running
entering
a car
throwing
cycling
lying
standing 92 0 0 24 0 7 0
walking 8 100 0 0 0 21 0
running 0 0 100 0 0 0 0
entering a
car 0 0 0 76 0 0 0
throw 0 0 0 0 100 0 0
cycling 0 0 0 0 0 72 0
classification
lying 0 0 0 0 0 0 100
The sum of each column is 100 percent. Column one e.g.,
standing is classified correctly with 92 % and with 8 %
wrongly as walking. Using the algorithm, all activities were
classified correctly with an accuracy of 91 %.
IV. C
ONCLUSIONS
In this paper, an algorithm for pedestrian activity
classification is presented. With the algorithm it is possible to
distinguish seven activities: standing, walking, running, lying,
cycling, throwing and entering or leaving a car. Nine degrees
of freedom (DOF) sensors (acceleration, angular rate and
magnetic field), such as those integrated in almost every
smartphone, have been taken as the basis. The activities are
classified by using a linear classifier and a decision tree
approach. The mean accuracy of all activities is 91%.
To show the algorithm results, a demonstrator for
Pedestrian activity classification was developed and
successfully presented on many exhibitions.
In the next version, the algorithm will provide qualities
criterion for all activities.
A
CKNOWLEDGMENT
Most of the work was performed as part of the project Ko-
TAG, which is part of the project initiative Ko-FAS, and has
been partially funded by the German Bundesministerium für
Wirtschaft und Technologie (Federal Department of
Commerce and Technology) under contract number 19S9011.
Ko-TAG is a research project in the field of safety
improvements for vulnerable road users. The project analyzes
the potential of cooperative sensor technologies to create a
consistent environmental model. This environmental model
supports the driver in assessing the current traffic situation by
detecting and highlighting vulnerable road users like kids or
cyclists.

2013 International Conference on Indoor Positioning and Indoor Navigation, 28
th
-31
th
October 2013
R
EFERENCES
[1] Seong Yun Cho, “MEMS Based Pedestrian Navigation System”, The
journal of navigation, 2006
[2] M. Rulsch, M. Benz, C. Arzt, C. Podolak, J. Zhong, R. Couronné, „A
lightweight approach for activity classification on microcontrollers“
[3] L. Fahrmeir, T. Kneib, S. Lang, „Regression – Modelle, Methoden und
Anwendung“, Spring, 2. Auflage, New York, 2009
[4] progenoX GmbH, "icos-X: Integrierte Kommunikations-, Ortungs- und
Sensorplattform", 2013
[5] BUSINESS GEOMATICS, “Lokalisierung auf Hüfthöhe”
[6] M. S. Grewal, L. R. Weill, A. P. Andrews, „Global Positioning
Systems, Inertial Navigation and Integration“, Second Edition, New
Jersey, 2007
[7] J. Yang, “Toward Physical Activity Diary: Motion Recognition Using
Simple Acceleration Features with Mobile Phones”, Proc. 1st Int.
Workshop on Interactive Multimedia for Consumer Electronics (IMCE)
at ACM Multimedia 2009, ACM Press, Oct. 2009, pp. 1-10.
[8] J. Kantola, M. Perttuner, T. Leppänen, J. Collin, J. Riekki, “Context
Awareness for GPS-Enabled Phones”
[9] Junjie Wu, “Advances in K-means Clustering: A Data Mining
Thinking”, Springer, Berlin, Heidelberg, 2012
[10] Bergmann Schaefer, “Elektromagnetismus” Band 2, 8 Auflage, 1999
[11] Q. Ladetto, V. Gabaglio, B. Merminod, P. Terrier, Y. Schutz, “Human
Walking Analysis Assisted by DGPS, GNSS, Ediburgh, 2000
[12] 1J. Seitz, T. Vaupel, S. Haimerl, J. Guti´errez Boronat, and J. Thielecke,
“Wi-fi azimuth and position tracking using directional received signal
strength measurements” in Workshop on Sensor Data Fusion (SDF
2012): Trends, Solutions, Applications, Sept. 2012, pp. 72 –77.
[13] J. Jahn,U. Batzer,J. Seitz, L. Patino-Studencka,J. G. Boronat,
“Comparison and Evaluation of Acceleration Based Step Length
Estimators for Handheld Devices”, In Proceedings of IEEE 2nd
Conference on Indoor Positioning and Indoor Navigation, Zurich,
Switzerland, 15–17 September 2010; pp. 1–6.
[14] Fraunhofer IIS, „Algorithms for motion classification”,
http://www.iis.fraunhofer.de, 2013
[15] M. Koecher, “Lineare Algebra und analytische Geometrie”, 4 Auflage,
Springer, 2003