ArticlePDF Available

Knowledge Exploitation for Human Micro-Doppler Classification

Authors:

Abstract and Figures

Micro-Doppler radar signatures have great potential for classifying pedestrians and animals, as well as their motion pattern, in a variety of surveillance applications. Due to the many degrees of freedom involved, real data need to be complemented with accurate simulated radar data to be able to successfully design and test radar signal processing algorithms. In many cases, the ability to collect real data is limited by monetary and practical considerations, whereas in a simulated environment, any desired scenario may be generated. Motion capture (MOCAP) has been used in several works to simulate the human micro-Doppler signature measured by radar; however, validation of the approach has only been done based on visual comparisons of micro-Doppler signatures. This work validates and, more importantly, extends the exploitation of MOCAP data not just to simulate micro-Doppler signatures but also to use the simulated signatures as a source of a priori knowledge to improve the classification performance of real radar data, particularly in the case when the total amount of data is small.
Content may be subject to copyright.
Knowledge Exploitation for Human Micro-
Doppler Classification
Cesur Karabacak, Sevgi Z. Gurbuz, Ali C. Gurbuz, Mehmet B. Guldogan, Gustaf Hendeby
and Fredrik Gustafsson
Linköping University Post Print
N.B.: When citing this work, cite the original article.
Cesur Karabacak, Sevgi Z. Gurbuz, Ali C. Gurbuz, Mehmet B. Guldogan, Gustaf Hendeby and
Fredrik Gustafsson, Knowledge Exploitation for Human Micro-Doppler Classification, 2015,
IEEE Geoscience and Remote Sensing Letters, (12), 10.
http://dx.doi.org/10.1109/LGRS.2015.2452311
©2015 IEEE. Personal use of this material is permitted. However, permission to
reprint/republish this material for advertising or promotional purposes or for creating new
collective works for resale or redistribution to servers or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the IEEE.
http://ieeexplore.ieee.org/
Postprint available at: Linköping University Electronic Press
http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-120433
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, REVISED MAY 2015 1
Knowledge Exploitation for Human Micro-Doppler
Classification
Cesur Karabacak, Sevgi Z. Gurbuz, Member, IEEE, Ali C. Gurbuz, Member, IEEE, Mehmet B.
Guldogan, Member, IEEE, Gustaf Hendeby, Member, IEEE, and Fredrik Gustafsson, Fellow, IEEE
Abstract—Micro-Doppler radar signatures have a great po-
tential for classifying pedestrians and animals, as well as their
motion pattern, in a variety of surveillance applications. Due
to the many degrees of freedom involved, real data needs to be
complemented with accurate simulated radar data to successfully
be able to design and test radar signal processing algorithms.
In many cases, the ability to collect real data is limited by
monetary and practical considerations, whereas in a simulated
environment any desired scenario may be generated. Motion
capture has been used in several works to simulate the human
micro-Doppler signature measured by radar; however, validation
of the approach has only been done based on visual comparisons
of micro-Doppler signatures. This work validates and, more
importantly, extends the exploitation of motion capture data not
just to simulate micro-Doppler signatures but also to use the
simulated signatures as a source of a priori knowledge to improve
the classification performance of real radar data, especially in the
case when the total amount of data is small.
Index Terms—Motion Capture, Human Micro-Doppler, Clas-
sification, Knowledge-Based Signal Processing.
I. INTRODUCTION
THE detection, recognition and classification of human
targets and their activities is a topic of great interest
that has many critical applications, such as search and rescue,
intelligent environments, border control, and security, to name
just a few. Human recognition and classification are typically
accomplished based on discerning differences in the radar
micro-Doppler signature for different targets and activities.
Micro-Doppler refers to the additional frequency modulations
observed about the central Doppler frequency that are caused
not by the gross translational motion of the target, but by vi-
brations or rotations of parts of the target [1]. For example, the
rotation of a helicopter blade, vehicle wheels, and tank tracks,
Manuscript received January 15, 2015.
C. Karabacak is with the Department of Electrical and Electronics Engi-
neering, TOBB University of Economics and Technology, Ankara, Turkey,
as well as the Meteksan Defense Industries, Inc., Ankara, Turkey (e-mail:
ckarabacak@meteksan.com).
S. Z. Gurbuz is with the Department of Electrical and Electronics Engi-
neering, TOBB University of Economics and Technology, Ankara, Turkey, as
well as the TUBITAK Space Technologies Research Institute, Ankara, Turkey
(e-mail: szgurbuz@etu.edu.tr; sevgigurbuz@tubitak.gov.tr).
A. C. Gurbuz is with the Department of Electrical and Electronics En-
gineering, TOBB University of Economics and Technology, Ankara, Turkey
(e-mail: acgurbuz@etu.edu.tr).
M. B. Guldogan is with the Department of Electrical and Electron-
ics Engineering, Turgut Ozal University, Ankara, Turkey (e-mail: bguldo-
gan@turgutozal.edu.tr).
G. Hendeby is with the Department of Electrical Engineering, Linkping
University, Sweden (e-mail: hendeby@isy.liu.se).
F. Gustafsson is with the Department of Electrical Engineering, Linkping
University, Sweden (e-mail: fredrik.gustafsson@liu.se).
as well as the periodic motion of the limbs of animals and
humans all result in visually distinguishable micro-Doppler
signatures. These can then be exploited for automatic target
recognition (ATR) and activity classification.
However, the development and testing of classification algo-
rithms requires data reflecting the wide variety of potential tar-
get characteristics. This includes animals whose signatures are
easily confused with those of humans, such as dogs, donkeys,
cows, and sheep, as well as humans of varying size, build,
and gait that are engaging in different activities at varying
angles, ranges, and velocities. Experimentally generating a
database of this size is both costly and impractical, therefore,
much research, as well as development of commercial systems
involving classification of human micro-Doppler, has utilized
methods for simulating micro-Doppler signatures.
There are two main approaches to simulating human micro-
Doppler signatures: kinematic modeling and motion capture.
The radar return from a human target is generally possible
to represent as the superposition of the returns from point
targets located at varying locations on the human body [2],
[3]. Simulating these returns requires knowledge of the time-
varying positions of each of these point targets, which can
be derived either from kinematic models or from motion
capture data. Some of the kinematic models that have been
utilized in micro-Doppler research studies are quite simplistic,
consisting of just a sinusoidal model of torso motion [4], or
animation of just three parts, the torso and the legs, with a
basic sinusoidal function [5]. The most comprehensive and
ubiquitous kinematic model is the Boulic walking model [6],
which uses a combination of equations and charts to describe
the time-varying motion of 17 points on the human body. It has
been successfully used in a number of human micro-Doppler
studies, e.g. [7]–[9].
The primary disadvantage of kinematic modeling, however,
is that there are no models covering the entire range of
human or animal activity, thus, only special cases can be
simulated. Motion capture data, on the other hand, is derived
from observations of alternative sensors, such as video or
infrared sensors. Typically, a finite number of points on the
human body are marked and their time-varying positions to
the sensor are computed using skeleton-tracking software.
These measurements are then used to simulate the distance
measurements that would have been made had the subject been
observed by radar. Motion capture (MOCAP) based data thus
enables the simulation of any desired sequence of motion for
any subject, including animals, while also incorporating the
individual variations of a subject. Since first being proposed
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, REVISED MAY 2015 2
by Ram, et al. [10] in 2008, motion capture data has been
used for a variety of radar micro-Doppler studies, includ-
ing analysis of the effect of walls [11], [12], the ground
[13], [14], polarization, and operating frequency [14] on the
human micro-Doppler signature. Motion capture techniques
have also been successfully applied to generate radar micro-
Doppler signatures for animals [15]. Indeed, several works
have subsequently utilized motion capture-based simulated
micro-Doppler as a basis for studying the extraction of features
relevant to gait analysis [16] and classification [17]–[20].
However, validation of the approach has primarily been done
by simple visual comparison with measured micro-Doppler
signatures [21]–[24].
In fact, most classification algorithms depend upon accurate
extraction of a set of features from the available data to achieve
high performance. For features derived from simulated data to
yield representative classification results on measured data, the
statistical properties of the features extracted from simulated
data should be comparable to those extracted from real data.
Otherwise, the classification results obtained from simulated
data could be misleading and unrepresentative of the actual
classifier performance.
Beyond validation of simulated data, however, the main goal
of this paper is to show that simulations derived from motion
capture data are indeed accurate enough representations to
be exploited as source of a priori knowledge suitable for
improving micro-Doppler classification performance. More
specifically, this work proposes a novel application of sim-
ulated MOCAP data as a source of training data for real radar
systems, especially in the case when the amount of measured
data is small.
With this aim, in Sec. II, first the radar system used
to collect measured micro-Doppler signatures is described.
Then, in Sec. III, different methods to simulate human micro-
Doppler signatures are discussed, while in Sec. IV the features
extracted from MOCAP data are compared with those of
measured data. Finally, in Sec. V, a novel approach to exploit
simulated micro-Doppler signatures as a priori knowledge for
first training classifiers before testing on measured data is
presented along with classification results.
II. EX PERIM EN TAL MEASUREMENTS OF HUMAN
MIC RO-DOP PL ER SIGNATURES
The experimental radar measurements used in this arti-
cle were captured using a SIRS 77 TD radar, developed by
SAAB AB. It is a frequency modulated continuous wave
(FMCW) radar with linear FM sweeps using the carrier
frequency 77 GHz. The received signal is mixed with the
transmitted signal and only the low bandwidth difference
signal needs to be digitized. This simplifies the hardware
design. The pulse compression (converting to ranges) is done
using a fast Fourier transform (FFT). The range resolution is
approximately 1m. The Doppler filtering utilizes the phase
shift between different sweeps, similar to what is done in
pulsed radars. In the used radar modes, the unambiguous radial
velocity was ensured to be enough for a moving human.
The measurements were conducted on moving persons in
Sweden in the summer 2009, on a day with favorable dry
weather. The experiments were performed on fairly flat ground
covered with short grass, at a place where there were no
buildings to disturb the measurements. The radar was pointing
approximately horizontally. Three different adults (one woman
and two men, of average build and weight) performed the
activities moving either radially towards or away from the
radar (aspect angle 0/180). Five different activities were
performed: running, jogging, walking, creeping and crawling.
After the acquisition the data was pulse compressed, fol-
lowed by an ideal high-pass filter to remove clutter, and
finally converted to a micro-Doppler signature by using a short
time Fourier transform (STFT) with a Hamming window. The
micro-Doppler spectra from the measurements have previously
been studied and analyzed in [25].
III. HUMAN MIC RO-DOP PL ER SIGNATURES
The micro-Doppler signature for any human activity is the
result of a complex combination of the time-varying motion
of each point on the human body. Suppose the entire human
body is divided into Kparts, which are in turn modeled by
point targets. Then the entire human return, sh, can be written
as the sum of returns from each individual point target [26]:
sh=
K
X
i=1
at,irectˆ
ttd,i
τej2πfc(ttd,i )+πγ (ˆ
ttd,i)2
.(1)
Here, τis the pulse width; cis the speed of light; γis the
chirp slope; and fcis the transmitted center frequency. The
total time elapsed, t, can be written in terms of the pulse
repetition interval (PRI), T; the fast-time ˆ
t, which denotes the
time as measured from the start of each PRI; and transmitted
pulse number (slow-time), n, as t=T(n1) + ˆ
t. The time
delay td,i is measured in terms of the total elapsed time tand
is defined as the round-trip travel time between the antenna
and the ith body part. Thus, the time delay is related to the
target range, R, as td= 2R/c. In this way, the received return
may be viewed as a two-dimensional signal that is a function
of fast-time and slow-time. The amplitude, at,i, is defined for
each point target from the range equation as
at,i =Ptσi
(4π)3
2R2
iLsLa
,(2)
where Gis the antenna gain, λis wavelength, Ptis the
transmitted power, σiis the radar cross-section (RCS) for
the ith body part, Lsrepresents system losses, and Lais
the atmospheric loss. Although this expression includes range
or geometry dependent factors, such as the antenna gain and
atmospheric losses, this work assumes that such factors are
constant for each body part. The RCS is modeled according to
the approximate shape of the body parts, i.e., a sphere for the
head, and ellipsoids for the remaining parts. Thus, grouping
all factors except for the range, Ri, and RCS, σi, of the ith
body part into a constant, A, the amplitude may be expressed
as at,i =i/R2
i.
A. Kinematic Model-Based Signature Simulation
In the literature several different kinematic models have
been used to simulate micro-Doppler. The simplest kinematic
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, REVISED MAY 2015 3
(a) (b) (c) (d) (e)
Fig. 1. (a) The SIRS 77 TD radar used to collect micro-Doppler measurements, and sample walking spectrograms as obtained from (b) measured SIRS77 TD
radar data, (c) Boulic kinematic model, (d) Kinect data, and (e) MOCAP database.
model [27] considers just the oscillation of the torso, which is
represented by a sinusoidal function. Slightly expanding upon
the torso model, the lower-body model [28] comprise three
points, the torso and two legs, where each leg is animated
similarly to a swinging pendulum. The spectrograms of both
these models have some similarity to real data, but important
components of the micro-Doppler signature are absent.
The most widely used model is the Boulic walking model,
which is based on the results of a detailed experimental study
of gait analysis, and uses a combination of equations and
charts to animate 17 different points and joints on the human
body. Thus, the Boulic model is not a closed-form model. All
equations and charts, however, basically depend on just two
physical parameters: the speed at which the person is walking,
and the height of the thigh from the ground (HT).
The most significant limitation of the Boulic model is that
it is only valid for walking, and for speeds in which the
v/HT ratio is less than 2.6 Hz. Moreover, because it is based
on a model, individual gait variations cannot be incorporated
into the simulations. Nevertheless, the resulting micro-Doppler
signatures bear a close similarity to measured data in general
shape, as shown in Fig. 1(a)-(c), with the key exception that it
is much more crisp and clean. The strongest return is due
to the torso, which in both signatures appears as a strong
sinusoid around the central Doppler shift that corresponds to
the average radial velocity of the person. Limb motion appears
as larger amplitude oscillations, the largest being that of the
legs.
B. Motion Capture-Based Signature Simulation
More realistic human spectrograms may be obtained by
exploiting video human motion capture data, from which the
time-varying ranges of points on the human body can be
computed. High quality motion capture is typically accom-
plished by placing infrared markers upon the human body and
observing these positions with a camera. Commercial motion
capture sensor suites are available, but these are typically
quite expensive ($30,000–$60,000). A low-cost system using
the Kinect sensor has recently been developed [29], but the
accuracy is less than that of commercial systems.
Databases storing motion capture data are available [30]–
[32] and have been used for micro-Doppler signature simu-
lation. For example, the Carnegie Mellon University (CMU)
Motion Research Laboratory has developed a freely distributed
library of human motion capture data [32]. It includes a
wide range of activities, such as walking, running, crawling,
jumping, and boxing, to name just a few. The data was
collected with the aid of 41 markers placed on the human
body and was recorded by 12 infrared cameras at a frequency
of 120 Hz. The database contains a total of 2,605 different
motion records belonging to 112 different subjects.
The primary disadvantage of using MOCAP databases is
that the test subjects used and test scenarios recorded are not
in the control of the user. This deficiency can be remedied
by using Kinect, however, the signature quality is poorer than
that of the CMU Motion Capture Library, as can be seen from
Fig. 1(d) and (e).
IV. EVALUATIO N OF SI MULATED SIGNATURES
The performance of classification algorithms is highly de-
pendent upon the features extracted from the micro-Doppler
signatures. Thus, in this section, CMU MOCAP spectrograms
are compared with measured SIRS 77 TD radar data based
upon the distributions of features used for classification.
A. Feature Extraction
Feature extraction, one of the most critical steps in a
classification system, is the process of computing numerical
indicators (features) that will enable the discrimination of
different classes of data. Moreover, feature extraction enables
the classification process to progress using a small set of
features rather than make computations on the much larger
set of raw data. In this way, the computation time of the
classification process is also reduced.
Proper feature selection is important to optimize classifica-
tion performance. In this study, three features (the mean of
the torso velocity, the variance of the upper envelope, and
the variance of the lower envelope) [33] are extracted from
micro-Doppler spectrograms. The mean of the torso velocity
is a representative measurement of the translational motion of
the target. The variances of the upper and the lower envelopes
reflect movements of arms and legs of the subject. These
three features are together an effective set to define different
human activities and can be computed from the spectrogram,
as illustrated in Fig. 2.
The envelopes of the spectrograms are extracted using a
percentile technique proposed by Van Dorp and Groen [34]
in 2008. First, the element of the envelope in each column of
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, REVISED MAY 2015 4
(a) (b)
Fig. 2. Envelope extraction from spectrograms: (a) original spectrogram, and
(b) extracted envelopes.
the spectrogram data is computed. To do this, the cumulative
amplitude distribution for each column is computed:
P(v, t) =
v
X
v=vmin
s(v, t)vmax
X
v=vmin
s(v, t)(3)
where vis the radial velocity, tis time, and sthe matrix storing
the values for amplitude (power) of the spectrogram. For each
column, the value P(U(t), t) = 0.97 is used to calculate the
upper envelope; P(C(t), t) = 0.5is used to calculate the
central envelope; and P(L(t), t)=0.03 is used to calculate
the lower envelope. Once these three envelopes are extracted,
their average value is computed, representing three different
features in a three dimensional feature space.
B. Statistical Comparison of the Features
The three features introduced in the previous subsection
are extracted from both measured and MOCAP data. In
both databases, there are four different human motion types:
walking, running, crawling, and creeping. The duration of each
data file is 1.5s and the subjects move towards radar during
the data acquisition process. Distribution of data types in these
two databases is given in Table I.
To compare the distribution of features obtained from MO-
CAP data with that of real data, scatter graphs of features
from each data set are presented in Fig. 3 and 4, respectively.
In both data sets, running is clearly separable from the other
classes of motion, while features appear to have similar
values for creeping and crawling, limiting their discriminating
power. Walking appears to be more easily distinguishable
from MOCAP-based simulated spectrograms than would be
possible from real measured data.
Classification performance attained using a nearest neighbor
(NN) classifier also validates the comparability of MOCAP-
derived signatures with measured signatures. Using 84 MO-
CAP signatures for training, 25 of the remaining 28 MOCAP
signatures were correctly classified, with just three crawling
signatures confused with creeping. In comparison, 15 of 16
measured SIRS 77 TD radar signatures were correctly classi-
fied when 40 measured signatures were used as training.
V. SI MU LATED SI GNATUR ES AS PRIO R KNOWL ED GE
To date, simulated micro-Doppler signatures have primarily
been used just as a means to approximate real data in per-
formance testing. In fact, simulated signatures represent prior
knowledge that can be exploited to improve the classification
TABLE I
DISTRIBUTION OF DATA TYPES
CLASSES
Walking Running Crawing Creeping
MOCAP 28 28 28 28
Measured 18 15 11 12
Fig. 3. Scatter graph of the features for the MOCAP database.
Fig. 4. Scatter graph of the features for the SIRS 77 TD radar.
performance of measured data. An important factor affecting
performance is the amount of data used to train the classifier.
Increasing the size of the training set tends to increase the
rate of correct classification. However, in real operational
scenarios, the total data set size may not be that large, limiting
the ability to train the classifier. In this work, this constraint
is overcome by using simulated data to train measured data.
To assess the potential performance attainable by training
with MOCAP data, classification of the SIRS 77 TD radar data
is accomplished in two ways: 1) a subset of measured data
is used to train a classifier to test the remaining measured
data; and 2) MOCAP-derived signatures are used to train all
measured data. With the first approach, it was found that
a correct classification rate of 94% was achieved when at
least 70% of the data was used for training. When MOCAP
data was used to train the classifier, the highest classification
performance attained was 93% when all 112 MOCAP-derived
signatures were used for training (Fig. 5). Confusion matrices
corresponding to the best classification results are shown in
Table II, as well as classification metrics in Table III.
VI. CONCLUSION
This work validates the use of MOCAP-derived simulated
data through the examination of the statistical distributions
of features, as well as through comparison of classification
performance. Moreover, a novel application of using simulated
MOCAP data to train classifiers for testing real data is
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, REVISED MAY 2015 5
Fig. 5. Dependency of classification performance on number of MOCAP-
signatures used for training. Vertical bars denote confidence interval (standard
deviation) of results.
TABLE II
CLASSIFICATION PERFORMANCE OF N N ON MOCAP-BAS ED A ND
SIRS 77 TD MEASURED RA DAR DATA
Walking Running Crawing Creeping
Training with SIRS77 TD Measured Radar Data
Walking 5 0 0 0
Running 0 4 0 0
Crawling 0 0 3 0
Creeping 0 0 1 3
Training with MOCAP-Based Simulated Data
Walking 18 0 0 0
Running 0 15 0 0
Crawling 0 0 11 0
Creeping 0 0 4 8
TABLE III
COMPARISON OF CLASSIFICATION METRICS
Walking Running Crawing Creeping
Training with SIRS77 TD Measured Radar Data
Sensitivity 1 1 1 0.75
Specificity 1 1 0.92 1
Precision 1 1 0.75 1
Training with MOCAP-Based Simulated Data
Sensitivity 1 1 1 0.67
Specificity 1 1 0.91 1
Precision 1 1 0.73 1
proposed. This method can be applied using different MOCAP
data sets, as well as different features and classifiers. Generally
including as many different individuals as possible in the
training set mitigates any potential losses due to different styles
of gait. In summary, results show that simulated data can yield
comparable results to that as achieved with training on real
data. This result is especially significant for the classification
of small data sets, when the amount of training data is
insufficient.
ACKNOWLEDGMENT
The authors gratefully acknowledge support from SAAB
and funding from Security Link, the EU FP7 Project No.
PIRG-GA-2010-268276 and TUBITAK Career No. 113E105.
REFERENCES
[1] V. Chen, The micro-Doppler effect in radar, Artech House, 2011.
[2] J.L. Geishemer, W.S. Marshall, E. Greneker, A continuous-wave (CW)
radar for gait analysis, in Proc. 25th Asilomar Conf. on Signals, Systems
and Computers, pp. 834-838, 2001.
[3] P. Van Dorp, and F.C.A. Groen, Human walking estimation with radar,
IEE Radar Sonar Navig.,Vol. 150, No. 5, pp 356365, Oct. 2003.
[4] G.E. Smith, K. Woodbridge, and C.J. Baker, Multistatic micro-doppler
signature of personnel, in Proc. IEEE Radar Conference, 2008.
[5] D. Tahmoush, and J. Silvious, Simplified model of dismount micro-
Doppler and RCS, in Proc. IEEE Radar Conference, 2010.
[6] R. Boulic, N.M. Thalmann, and D. Thalmann, D. A global human walking
model with real-time kinematic personification, Vis. Comput., Vol. 6, No.
6, pp 344358, Nov. 1990.
[7] T. Damarla, M. Bradley, A. Mehmood, J.M. Sabatier, J.M., Classification
of Animals and People Ultrasonic Signatures, IEEE Sensors Journal.
[8] S.R. Groot, A.G. Yarovoy, R.I.A. Harmanny, J.N. Driessen, Model-based
classification of human motion: Particle filtering applied to the Micro-
Doppler spectrum, in Proc. EuRAD, 2012.
[9] I. Bilik, and J. Tabrikian, ”Radar target classification using Doppler
signatures of human locomotion models,” IEEE Trans. AES, Vol.43, No.4,
pp.1510-1522, October 2007.
[10] S.S. Ram, H. Ling, ”Simulation of human microDopplers using com-
puter animation data,” IEEE Radar Conference, 2008.
[11] S.S. Ram, C. Christianson, H. Ling, ”Simulation of high range-resolution
profiles of humans behind walls,” IEEERadar Conf., 2009.
[12] S.S. Ram, C. Christianson, Y. Kim, H. Ling, ”Simulation and Analysis
of Human Micro-Dopplers in Through-Wall Environments,” IEEE Trans.
Geoscience and Remote Sensing, vol.48, no.4, pp.2015-2023, April 2010.
[13] S.S. Ram, R. Bhalla, H. Ling, ”Simulation of human radar signatures
in the presence of ground,” IEEE Antennas and Propagation Society
International Symposium, 1-5 June 2009.
[14] J. Park, J.T. Johnson, N. Majurec, M. Frankford, E. Culpepper, J.
Reynolds, J. Tenbarge, L. Westbrook, ”Software defined radar studies
of human motion signatures,” IEEE Radar Conference, 2012.
[15] S.S. Ram, and H. Ling, ”Microdoppler signature simulation of computer
animated human and animal motions,” IEEE Antennas and Propagation
Society International Symposium, 2008.
[16] A. Ghaleb, L. Vignaud, J-M. Nicolas, ”Micro-Doppler analysis of
pedestrians in ISAR imaging,” IEEE Radar Conference, 2008.
[17] L. Fei, H. Binke, Z. Hang, D. Hao, ”Human gait recognition using micro-
Doppler features,” 12th Global Symp. on Millimeter Waves (GSMM),
pp.326-329, 27-30 May 2012.
[18] S.Z. Gurbuz, B. Tekeli, M. Yuksel, et. al., ”Importance ranking of
features for human micro-Doppler classification with a radar network,”
IEEE Int. Conf. on Information Fusion, 9-12 July 2013.
[19] R.G. Raj, V.C. Chen, R. Lipps, ”Analysis of radar human gait signa-
tures,” Signal Processing, IET , vol.4, no.3, pp.234,244, June 2010.
[20] Y. Yang, W. Zhang, and C. Lu, ”Classify human motions using micro-
Doppler radar.” SPIE Defense and Security Symp., 2008.
[21] V.C. Chen, W.J. Miceli, B. Himed, ”Micro-Doppler analysis in ISAR -
review and perspectives,” Int.Radar Conference, 12-16 Oct. 2009.
[22] V.C. Chen, ”Doppler signatures of radar backscattering from objects
with micro-motions,” IET Signal Processing, 2, (3), pp.291-300, 2008.
[23] C. Karabacak, S.Z. Gurbuz, A.C. Gurbuz, ”Radar simulation of human
micro-Doppler signature from video motion capture data,” IEEE Signal
Processing and Communications Applications Conference, 2013.
[24] L. Vignaud, A. Ghaleb, J. Le Kernec, J-M. Nicolas, ”Radar high
resolution range and micro-Doppler analysis of human motions,” IEEE
Int.Radar Conference, 12-16 Oct. 2009.
[25] S. Bjorklund, H. Petersson, A. Nezirovic, M.B. Guldogan, F. Gustafsson,
Millimeter-Wave Radar Micro-Doppler Signatures of Human Motion,
International Radar Symposium, Sept. 2011.
[26] W.G. Carrara, R.S. Goodman, R.M. Majewski, Spotlight Synthetic
Aperture Radar, Artech House, Boston, 1995.
[27] D. Tahmoush, and J. Silvious, Simplified model of dismount mi-
croDoppler and RCS, in Proc. IEEE Radar Conference, 2010.
[28] G.E. Smith, K. Woodbridge, and C.J. Baker, Multistatic micro-Doppler
signature of personnel, in Proc. IEEE Radar Conference, 2008.
[29] B. Erol, C. Karabacak, S.Z. Gurbuz, A Kinect-based human micro-
Doppler simulator, Aerospace and Electronic Systems Mag., accepted.
[30] SNU Movement Research Lab [Online]. Available: http://mrl.snu.ac.kr/
[31] ACCAD Motion Capture Lab. [Online]. Available: http://accad.osu.edu/
research/mocap/mocap data.htm
[32] CMU Graphics Lab Motion Capture Database [Online]. Available: http:
//mocap.cs.cmu.edu/
[33] Y. Kim and H. Ling, ”Human activity classification based on micro-
Doppler signatures using a support vector machine,” IEEE Trans. Geo-
science and Remote Sensing, Vol. 47, Iss. 5, pp. 1328 - 1337, May 2009.
[34] P. Van Dorp, F.C.A. Groen, ”Feature-based human motion parameter
estimation with radar,” IET Radar, Sonar and Navigation, Vol.2, No.2,
pp.135-145, April 2008.
... The envelope extraction method employed in this study is the percentile method, originally proposed by Van Dorp and Groen [40] for Doppler Spectrogram analysis. This method has been extensively used in literature for the extraction of Doppler envelopes of lower-limbs during walking [41], due to its low noise-sensitivity and low computational cost. The percentile method is based on the normalised cumulative amplitude distribution of each RDT frame which is computed by ...
Article
Full-text available
Objective: In recent years, radar technology has been extensively utilized in contactless human behavior monitoring systems. The unique capabilities of ultra-wideband (UWB) radars compared to conventional radar technologies, due to time-of-flight measurements, present new untapped opportunities for in-depth monitoring of human movement during overground locomotion. This study aims to investigate the deployability of UWB radars in accurately capturing the gait patterns of healthy individuals with no known walking impairments. Methods: A novel algorithm was developed that can extract ten clinical spatiotemporal gait features using the Doppler information captured from three monostatic UWB radar sensors during a 6-meter walking task. Key gait events are detected from lower-extremity movements based on the joint range-Doppler-time representation of recorded radar data. The estimated gait parameters were validated against a gold-standard optical motion tracking system using 12 healthy volunteers. Results: On average, nine gait parameters can be consistently estimated with 90-98% accuracy, while capturing 94.5% of participants' gait variability and 90.8% of inter-limb symmetry. Correlation and Bland-Altman analysis revealed a strong correlation between radar-based parameters and the ground-truth values, with average discrepancies consistently close to 0. Conclusion: Results prove that radar sensing can provide accurate biomarkers to supplement clinical human gait assessment, with quality similar to gold standard assessment. Significance: Radars can potentially allow a transition from expensive and cumbersome lab-based gait analysis tools toward a completely unobtrusive and affordable solution for in-home deployment, enabling continuous long-term monitoring of individuals for research and healthcare applications.
... For this aim, the classification of human movements by micro-Doppler radar processing has been studied by many researchers. [3][4][5] The time-varying human micro-Doppler characteristics originating from complex movements of the human body (i.e., arms, legs, hands, and | 1 feet) to simple ones such as head and torso provide unique signatures that can be visually separated from other target types such as animals. 6 This signature is closely related to the movement that man has made. ...
Article
Full-text available
In this study, we introduce a radar sensor that we have developed for the micro‐Doppler signature measurements together with three consecutive signal preprocessing routines that are to be used for primary detection and classification of human gait and movements. The developed C‐band radar sensor has been used in a series of human walking and running experiments that were conducted at Mersin University facilities. First, the raw data collected in time domain are transformed to two‐dimensional (2D) time‐frequency plane to investigate the Doppler characteristics of human movements. To detect and extract the main body signature of the human, a Kalman filter‐based torso signature detection methodology is introduced as the first preprocessing tool. Then, to be able to better detect the other body parts’ signatures, the torso's response is thoroughly extracted from the 2D time‐Doppler image as the second preprocessing routine by applying a circular shift procedure along each time‐bin. As the last procedure, envelopes of the micro‐Doppler signatures on the 2D time‐Doppler image plane are determined by another routine based on human Doppler threshold method. To assess the success and the validity of the proposed pre‐signal processing tools, they are applied to the measured data collected via different experiments of human movements. After applying all these steps, the torso signature and envelopes of all other body parts’ signatures are marked and determined with good fidelity.
... 2) Radar-based gait recognition methods In recent years, some researchers have investigated radarbased gait recognition by extracting time-Doppler gait features. Traditional methods generally extract handcraft features [35][36][37], such as time, frequency, and Doppler bandwidth, to classify human identity. These methods require professional knowledge and have a poor performance. ...
Article
In recent years, gait recognition has emerged as an important and promising solution for human identification. Generally, gait recognition is based on a single type of sensor, such as a camera or a radar. However, data of a single modality may only capture inadequate gait features of a person, such as camera data lacking the intuitive micro-motion pattern information and radar data lacking the information about gait appearance, making gait-based human identification system vulnerable to complex covariate conditions, e.g. cross-view and cross-walking-condition. To build a robust and reliable gait-based human identification system, in this study, we propose a multi-sensor gait recognition framework with deep convolutional neural networks (CNNs) by fusing camera gait energy images (GEIs) and radar time-Doppler spectrograms. To learn the fine-grained gait appearance features, we propose a body-part spatial attention (BPSA) module to obtain more discriminative body part representations of GEIs. To learn the gait micro-motion pattern, we propose a long-short temporal relation modeling (LSTRM) module to obtain the local and global micro-motion representation of time-Doppler spectrograms. Finally, we fuse the discriminative body part representation and the micro-motion pattern at the multi-scale feature space to obtain richer and more robust gait features for human identification. We provide an extensive empirical evaluation in terms of various complex covariate conditions, namely, cross-view and cross-walking-condition. Experiments on 121 subjects with eight views and three walking conditions of camera and radar data show our proposed method is more robust and accurate.
Article
Full-text available
The unprecedented non-contact, non-invasive, and privacy-preserving nature of radar sensors has enabled various healthcare applications, including vital sign monitoring, fall detection, gait analysis, activity recognition, fitness evaluation, and sleep monitoring. Machine learning (ML) is revolutionizing every domain, with radar-based healthcare being no exception. Progress in the field of healthcare radars and ML is complementing the existing radar-based healthcare industry. This article provides an overview of ML usage for two major healthcare applications: vital sign monitoring and activity recognition. Vital sign monitoring is the most promising healthcare application of radar, as it can predict several chronic cardiac and respiratory diseases. Activity recognition is also a prominent application since the inability to perform activities may result in critical suffering. The article presents an overview of commercial radars, radar hardware, and historical progress of healthcare radars, followed by the usage of ML for healthcare radars. Subsequently, the paper discusses how ML can overcome the limitations of conventional radar data processing chains for healthcare radars. The article also touches upon recent generative ML concepts used in healthcare radars. Among several interesting findings, it was discovered that ML does not completely replace existing vital sign monitoring algorithms; rather, ML is deployed to overcome the limitations of traditional algorithms. On the other hand, activity recognition always relies on ML approaches. The most widely used algorithms for both applications are Convolutional Neural Network (CNN) followed by Support Vector Machine (SVM). Generative AI has the capability to augment data and is expected to have a significant impact soon. Recent trends, lessons learned from these trends, and future directions for both healthcare applications are presented in detail. Finally, the future work section discusses a wide range of healthcare topics for humans, ranging from neonates to elderly individuals.
Article
The moving direction affects the radar micro-Doppler signatures induced by human movement, with the least being attained when the human moves tangentially with regard to the line-of-sight. Tangential spectrograms contain minimal data on human movements and exhibit an extensive level of visual similarity, providing a significant obstacle in the classification of human motion. At the same time, the high costs associated with obtaining measured radar spectrograms result in a restricted quantity of data, making it more challenging to identify human motion spectrograms in one-shot situations. Considering the aforementioned difficulties, we propose a one-shot human motion recognition method with tangential spectrograms based on metric learning which creatively uses simulation data as the support set and measured data as the query set. In our model, Local Region Enhancement (LRE) component is introduced to concentrate solely on echo signal regions near the central frequency by employing global information to improve the efficacy of valuable feature channels and diminish the impact of extraneous ones. Spatial Group-wise Refinement (SGR) component has been proposed to extract discriminative features from central regions by assigning weights to distinct spatial positions within each semantic group in local feature maps. Compared to other methods, the experimental results indicate that the proposed methodology enhances the classifier’s efficacy, resulting in a rise in the classification accuracy of 3.4% at least.
Article
Gesture and sign language recognition technology enables machines to understand the meaning of human hand movements. In human-computer interaction, it is expected that gesture/sign language recognition technologies will overcome equipment size and application environment constraints; in information communication, gesture/sign language recognition technology will assist healthy people in more easily entering the world of deaf people and better understanding and meeting their inner emotional needs; and in patient monitoring, gesture/sign language recognition technologies will detect abnormal behavior in the elderly or patients and reduce possible safety issues. As a result, it has significant implications for both research and broad application. Because the radar sensor can work normally in a wide range of illumination and weather conditions, as well as penetrate the shelter to receive the moving object echo signal and preserve individual privacy, it is becoming increasingly popular in a variety of recognition tasks. A primary step in using radar sensors for gesture/sign language recognition is to suppress clutter to highlight useful motion information. To evaluate the clutter suppression effect, however, an objective metric is required. We present an objective assessment metric based on the pseudo-reference image and an automatic threshold selection method based on Otsu for clutter suppression, as well as subjective and objective experiments demonstrating their effectiveness and universality in gesture/sign language recognition. Notably, our proposed metric can be used for any recognition task that uses micro-Doppler spectrograms as the dataset.
Chapter
Deep convolution neural networks (DCNN) have been an excellent classifier in various fields such as radar images classification. However, the number of training samples deeply influences the performance of networks and radar samples are not easy to be obtained. In this paper, we utilize micro-Doppler simulated data from Kinect sensor and online dataset to train networks and figure out the differences. In order to enhance the reliability of Kinect simulation, a filter for Kinect data is proposed. Later, transfer learning is used to improve the performance. Moreover, bistatic deep convolutional neural networks (bistatic-DCNN) are proposed. After transfer learning, experimental results show that the performances of both 7-class and 13-class motion classification reach over 98%.KeywordsTransfer learningMicro-Doppler simulationBistatic deep convolutional neural networksHuman motion classification
Chapter
Micro-Doppler effects in narrowband radar are introduced in this chapter. We mainly analyze the micro-Doppler effects induced by rotations, vibrations, and precessions of targets. Rotations, vibrations, and precessions are typical micromotion dynamics of targets in the real world. Typical rotations include rotations of helicopter rotors, mechanical scanning radar antennas, turbine blades, etc. Typical vibrations include engine-induced car surface vibrations, mechanical oscillations of a bridge, etc. Precession is a major movement form of spatial targets, for example, precessions usually go with a ballistic missile in its midcourse flight. Analyzing the micro-Doppler effects of rotations, vibrations, and precessions provides a basic foundation for further discussions on micro-Doppler effects of complicated motions. The influence of radar platform vibration on micro-Doppler effect analysis is also investigated. Finally, related MATLAB codes are provided on the companion Website: http://booksite.elsevier.com/9780128098615/ .
Article
Until recently, human surveillance has primarily been accomplished using video cameras. However, radar offers unique advantages over optical sensors, such as being able to operate at far distances, under adverse weather conditions, and at nighttime, when optical devices are unable to acquire meaningful data. Radar is capable of recognizing human activities by classifying the micro-Doppler signature of a subject. Micro-Doppler is caused by any rotating or vibrating parts of a target, and results in frequency modulations centered about the main Doppler shift caused by the translational motion of the target [1]. Thus, the rotation of a helicopter blade, wheels of a vehicle, or treads of a tank all result in micro-Doppler. In the case of humans, the complex motion of the limbs that occur in the course of any activity all result in a micro-Doppler signature visually distinguishable from other targets, even animals [2]---[3], which can then be exploited for human detection [4]---[5], automatic target recognition (ATR) [6]---[7], and activity classification [8].
Conference Paper
In this article, a novel motion model-based particle filter implementation is proposed to classify human motion and to estimate key state variables, such as the motion type, i.e. running or walking, and the subject's height. Micro-Doppler spectrum is used as the observable information. The system and measurement models of the human movements are built using three parameters (relative torso velocity, height of the body, gait phase). The algorithm developed has been verified on simulated and experimental data.
Conference Paper
The motion of the human limbs results in the unique micro Doppler features which can be used for classification and recognition of different gaits. In this paper, a general human model, together with motion capture data of human motion, is displayed to calculate the expected radar echo and spectrogram of the target. Then the echoes of human torso and limbs are separated based on Chirplet signal representation. The cadence of human motion is extracted from the echo of the limbs to classify the different human gaits. Finally the validity of this feature is verified by computer simulation.
Conference Paper
Over the past decade, the human micro-Doppler signature has been a subject of intense research. In particular, much work has been done in relation to computing features for use in a variety of classification problems, such as arm swing detection, activity classification, and target identification. Although dozens of features have been proposed for these purposes, little work has examined the issue of which features are more important - i.e., have a greater impact on classification performance - than others. In this work, an information theoretic approach is applied to compute the importance ranking of features prior to classification for the specific problem of discriminating human walking from running. Results show that the ranking of features according to mutual information directly relates to classification performance using support vector machines.
Conference Paper
The availability of data sets on which signal processing techniques may be tested is critical to the development of human detection, identification, and classification algorithms. However, in many cases real radar data of the desired characteristics may be expensive or difficult to obtain. In this case, synthetic or simulated data is desired. Much of the simulated data used in publications is derived from the Boulic kinematic model. But, the Boulic model is only valid for walking and is not applicable to compute the micro-Doppler signatures of other human motions. The Carnegie Mellon University motion capture library includes data from a wide range of human activities and provides the time-varying position of body parts. In this work, this video motion capture data is used to generate the radar micro-Doppler signature for many human activities. Observations about the micro-Doppler signatures computed are also shared.
Article
Personnel detection using inexpensive nonimaging sensors is becoming increasingly important for several applications, namely, border surveillance, perimeter protection, and urban operations. In this paper, we explore the utility of ultrasonic sensors to distinguish between people and animals walking. We explore the phenomenology associated with human and animal walking and identify model-based features in the spectrogram. In particular, we study the properties of micro-Doppler returns from various body parts (limbs) of the people and animals to identify the features. Finally, we develop two algorithms for classifying people and animals using the micro-Doppler signatures: one algorithm for the case when the signal-to-noise ratio (SNR) is high and another for low SNR. A support vector machine and a Bayesian classifier were used to classify the targets when the SNR is low. We present the results of the algorithms applied to actual data collected at a horse farm.
Article
The detection and monitoring of human motion with radar has numerous applications in surveillance, urban military operations, search-and-rescue, and other areas. Recent studies have shown that movements of humans generate unique micro-Doppler signatures that can be exploited to classify human motions. This motivates an improved understanding of human Doppler signatures. Numerous simulations and measurements of human “dismount” signatures has been performed in the past, but most have been focused on a single radar center-frequency and have not taken polarization effects into consideration. In this paper, human modeling and motion measurements using multiple radar frequencies are proposed to explore the impact of the radar frequency on human range/Doppler signatures. Furthermore, ground effects on human targets are investigated using a four path model. The OSU Software defined radar (SDR) system, which can be tuned from 2GHz to 18 GHz with 500MHz bandwidth, was used for the measurements. This radar can operate at two frequencies simultaneously, allowing for dual frequency human measurements. Also, different polarizations are considered to understand human Doppler signatures. Modeling efforts are based on a finite dielectric cylinder approximation, so that the human body is modeled as a collection of dielectric cylinders. Scattering signatures are computed neglecting scattering interactions among these cylinders.
Article
In this paper, human motion model and RCS (radar cross section) simulation of radar returns from human are investigated. Micro-Doppler signatures [1-6] induced by human motions are studied. It shows that the time-frequency representation of micro-Doppler signature provides distinctive time-varying features for human motions. Motion of different body part has different micro-Doppler signature. Thus, micro-Doppler can be a promising method for classifying human activities. Measurement data using an experimental X-band micro-Doppler radar were collected and the results are compared with the corresponding simulation results. The classification of human motions based on micro-Doppler signatures is also discussed.