ArticlePDF Available

A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer

Authors:

Abstract and Figures

Physical-activity recognition via wearable sensors can provide valuable information regarding an individual's degree of functional ability and lifestyle. In this paper, we present an accelerometer sensor-based approach for human-activity recognition. Our proposed recognition method uses a hierarchical scheme. At the lower level, the state to which an activity belongs, i.e., static, transition, or dynamic, is recognized by means of statistical signal features and artificial-neural nets (ANNs). The upper level recognition uses the autoregressive (AR) modeling of the acceleration signals, thus, incorporating the derived AR-coefficients along with the signal-magnitude area and tilt angle to form an augmented-feature vector. The resulting feature vector is further processed by the linear-discriminant analysis and ANNs to recognize a particular human activity. Our proposed activity-recognition method recognizes three states and 15 activities with an average accuracy of 97.9% using only a single triaxial accelerometer attached to the subject's chest.
Content may be subject to copyright.
1166 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 5, SEPTEMBER 2010
A Triaxial Accelerometer-Based Physical-Activity
Recognition via Augmented-Signal Features
and a Hierarchical Recognizer
Adil Mehmood Khan, Young-Koo Lee, Sungyoung Y. Lee, and Tae-Seong Kim, Member, IEEE
Abstract—Physical-activity recognition via wearable sensors can
provide valuable information regarding an individual’s degree of
functional ability and lifestyle. In this paper, we present an ac-
celerometer sensor-based approach for human-activity recogni-
tion. Our proposed recognition method uses a hierarchical scheme.
At the lower level, the state to which an activity belongs, i.e., static,
transition, or dynamic, is recognized by means of statistical signal
features and artificial-neural nets (ANNs). The upper level recog-
nition uses the autoregressive (AR) modeling of the acceleration
signals, thus, incorporating the derived AR-coefficients along with
the signal-magnitude area and tilt angle to form an augmented-
feature vector. The resulting feature vector is further processed
by the linear-discriminant analysis and ANNs to recognize a par-
ticular human activity. Our proposed activity-recognition method
recognizes three states and 15 activities with an average accuracy
of 97.9% using only a single triaxial accelerometer attached to the
subject’s chest.
Index Terms—Accelerometer, artificial-neural nets (ANNs), au-
toregressive (AR) modeling, human-activity recognition.
I. INTRODUCTION
AUTOMATIC recognition of human activities is one of the
important and challenging research areas in proactive and
ubiquitous computing: first, due to its potential in providing
personalized support for many different applications such as
smart environments and surveillance and second, due to its con-
nection to many different fields of studies such as lifecare and
healthcare.
Human-activity recognition requires an objective and reli-
able technique that can be used under the conditions of daily
living. Complex sensors such as cameras in computer vision
have been used to recognize activities. In general, the computer
vision-based techniques for tracking and activity recognition
often work well in a laboratory or well-controlled environment.
Manuscript received July 10, 2009; revised December 23, 2009 and
March 17, 2010; accepted May 20, 2010. Date of publication June 7,
2010; date of current version September 3, 2010. This research was sup-
ported by the Ministry of Knowledge Economy (MKE), Korea, under the
Information Technology Research Center (ITRC) support program super-
vised by the National IT Industry Promotion Agency (NIPA) (NIPA-2010-
(C1090-1021-0003 )).
A. M. Khan, Y.-K. Lee, and S. Y. Lee are with the Department of Com-
puter Engineering, Kyung Hee University, Yongin-si 446-701, Korea (e-mail:
kadil@oslab.khu.ac.kr; yklee@khu.ac.kr; sylee@oslab.khu.ac.kr).
T.-S. Kim is with the Department of Biomedical Engineering, Kyung Hee
University, Yongin-si 446-701, Korea (e-mail: tskim@khu.ac.kr).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TITB.2010.2051955
However, they fail in achieving the same level of accuracy un-
der a real-home setting due to the clutter, variable lighting, and
highly varied activities that take place in the natural environ-
ments [1]. Motion capture with the body-fixed accelerometers
offers an appropriate alternative for the assessment of daily
physical activities [2].
Many human-activity recognition systems have been devel-
oped in the past which incorporate the use of accelerometers.
Some of these investigated the use of acceleration signals to an-
alyze and classify different types of the same physical activity,
e.g., walking along the corridor, upstairs, and downstairs [3],
[4]. Others employed them for recognizing a wide set of daily
physical activities such as lying, sitting, standing, walking, and
running [2], [5]–[19]. Minnen et al. [20] have explored the dis-
covery of recurring activities such as hammering, sawing, fill-
ing, drilling, sanding, and grinding from the accelerometer data,
while [21]–[23] focused on the elderly persons’ fall detection
and prevention.
Most of these systems investigated the use of multiple ac-
celerometers attached to different sites on a subject’s body [2],
[3], [5], [8], [10], [11], [15], [17]–[21]. However, this approach
is not feasible for the long-term activity monitoring because
of two or more sensor-attachment sites and cable connections.
Comparatively, a small number of studies have investigated the
use of a single accelerometer mounted at waist or sternum [4],
[6], [7], [9], [12]–[14], [16], [22], [23]. Such systems achieved
good recognition results for some basic activities including ly-
ing, walking, and running. However, they could not achieve the
same level of accuracy for some static activities such as standing
and sitting, transitions such as lie-stand, sit-stand, and stand-sit,
and dynamic activities such as walking downstairs and walking
upstairs.
As for the features, most studies have used the frequency-
derived features employing fast Fourier transform [17], [20] and
wavelet transform [2]–[4]. Others used the signal-magnitude
area (SMA) [6], [7], [16], [24], tilt angle (TA) [6]–[8], [16] or
parameters such as averages, energy, entropy, standard devia-
tion, or correlations [10], [13], [18]. However, these features
are calculated over long time-windows which reduce their abil-
ity to detect the short-duration movements, e.g., the transitions
between sitting and standing or taking a couple of steps.
As for the recognition techniques, a large number of classifi-
cation methods have been investigated. Some studies incorpo-
rated the idea of simple heuristic classifiers [2], [5]–[9], [11],
[16], whereas others employed more generic and automatic
methods from the machine learning literature including the
1089-7771/$26.00 © 2010 IEEE
KHAN et al.: TRIAXIAL ACCELEROMETER-BASED PHYSICAL-ACTIVITY RECOGNITION 1167
decision trees, nearest neighbor and Bayesian networks [10],
[13], [17], [18], support vector machines [13], neural networks
[3], [15], [17], [21], Gaussian mixture models (GMM) [14], and
Markov chains [12], [19], [20].
Thus, the existing literature on physical-activity recognition
using accelerometers varies in approach, intention, and out-
come. Individual researchers have employed their own device(s)
to collect the data for a particular set of movement(s) and have
investigated a wide variety of algorithms and methods. The most
significant breakthrough is presented in [24], where a single tri-
axial accelerometer is developed and evaluated for accessing
daily physical activities. It was later used in [6], [7], and [16]
with varying success rates. However, the primary drawback of
these systems is their rule-based heuristic nature as finding such
a set of rules is a time-consuming process. Allen et al. [14] pro-
posed an improvement by employing a GMM-based approach as
a more general and sophisticated approach to physical-activity
recognition using a single triaxial accelerometer. However, this
scheme introduces more complexity because it requires training
a separate GMM for each physical activity. It also requires an
adaptation method to adapt the system to a particular subject
when faced with the limited training data. Moreover, all previ-
ous studies presented some difficulty in distinguishing between
the sitting and standing postures. Accuracy of 78% has been
achieved so far by relying on the improved knowledge of the
transitional movements between sitting and standing to distin-
guish these postures.
In our previous study on human-activity recognition using
a triaxial accelerometer [25], we proposed the autoregressive
(AR) modeling [26] of the triaxial acceleration signals and used
the AR-coefficients augmented with the SMA and TA to form
an augmented-feature vector. The proposed feature vector was
then used to recognize lying, standing, walking, and running.
It outperformed the features used in the previous studies by
achieving a recognition rate of 99%. However, the accuracy
decreased significantly with the addition of the new activities.
In this paper, we aim to overcome the limitations of our previ-
ous physical-activity recognition system and intend to develop a
system that is capable of recognizing a broad set of daily physi-
cal activities using only a single triaxial accelerometer. Our pro-
posed system uses a hierarchical-recognition scheme, i.e., the
state recognition at the lower level using statistical features and
the activity recognition at the upper level using the augmented-
feature vector [25] followed by the linear-discriminant analysis
(LDA) to extract only the meaningful features. Our proposed
system is capable of recognizing 15 physical activities of daily
living with a much improved recognition rate compared to the
previous studies.
II. METHOD
A. Sensor Device and Data Collection
We used an accelerometer sensor called Witilt v2.5, a
2.4-GHz wireless triaxial tilt sensor from Sparkfun. It employs
a FreeScale MMA7260Q triple-axis accelerometer and a Class
1 Bluetooth link from BlueRadios. The sampling frequency was
20 Hz and the range of the sensor output was ±6g. The blue-
Fig. 1. Sensor device being worn by a subject.
TAB LE I
CLASSIFIED STATES AND ACTIVITIES RECOGNIZED IN THIS STUDY
tooth module communicated with a universal serial bus dongle
attached to a computer working as a transceiver to receive and
store the sensor data. In general, the output of any body-worn
accelerometer depends on the position at which it is placed [27].
Accelerometers are normally attached to the part of the body
whose movement is being analyzed such as arm, wrist, thigh,
etc. However, since we wished to study the whole body move-
ments, we decided to place the sensor at a position closer to the
center of mass, i.e., the subject’s chest as shown in Fig. 1.
The dataset for our experiment was collected in an unsuper-
vised study. Six healthy subjects, i.e., three females and three
males with the mean age of 27, wore the device each day for a
period of one month to collect the 15 activities which are listed in
Table I. Annotations were performed using a bluetooth headset
combined with a speech-recognition software [28]. A sample se-
quence of the activities performed at home is: sitting (2 min)
sit-stand standing (2 min) stand-lie lying (2 min)
lie-stand standing (40 s) walking (2 min) standing
(40 s) walking-upstairs standing (40 s) walking-
downstairs standing (40 s) stand-sit sitting (40 s)
sit-lie lying (40 s) lie-sit.
The subjects were provided with approximate time duration
for each activity, as shown in the activity sequence, except for
the walking upstairs and downstairs. The time duration of these
activities depended on the length of stairs at each subject’s home
and, thus, varied among the subjects. Although the dataset col-
lected under this protocol was structured, it was still acquired
1168 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 5, SEPTEMBER 2010
Fig. 2. Set of the sample acceleration signals of the human activities for each axis of the triaxial accelerometer.
under the less controlled conditions than in the most prior
studies.
The subjects were trained on the use of data collection and
annotation applications. Each subject then collected the activity
data at home without the researchers’ supervision. They made
the annotations themselves throughout the data collection. We
collected approximately 21 h of the activity data, i.e., 3.5 h per
subject. The activity dataset for each subject was then divided
randomly into the training and test sets in a roughly 40%–60%
split. A representative set of the activity signals is shown in
Fig. 2.
B. Proposed System Architecture
The architecture of our proposed activity-recognition system
is illustrated in Fig. 3. A component-based description of the
system is given in the following.
1) Noise Reduction: The real-time output of an accelerome-
ter contains some noise that needs to be filtered out before using
it for the activity recognition. The noise reduction unit, shown in
Fig. 3, incorporates a three-point moving average filter to filter
out the signal outliers.
2) State Recognition: The purpose of the state recognition
is to determine the state to which an activity belongs. Features,
including the mean, standard deviation, spectral entropy, and
correlation, as the state features, are extracted from the noise-
reduced acceleration signal and processed by a classifier. These
parameters have been used for physical-activity recognition in
Fig. 3. Block diagram for our proposed recognition technique.
several existing studies with varying success rates [10], [13],
[18]. However, we proposed their usage for the state recognition
only. The mean, range of the possible acceleration values, and
periodicity in the acceleration data differ slightly between the
physical activities but greatly between the states. Hence, they
are more suitable for the state recognition.
3) Activity Recognition: Once the state of a given activ-
ity is recognized, the noise-reduced acceleration signal is fed
to the activity-recognition module which uses our proposed
augmented-feature vector [25], i.e., the AR-coefficients, SMA,
and TA. Though the TA has been used previously to differentiate
certain postures [6]–[8], [16], the decisions made on its values
were purely heuristic and rule based. We proposed using the TA
KHAN et al.: TRIAXIAL ACCELEROMETER-BASED PHYSICAL-ACTIVITY RECOGNITION 1169
Fig. 4. Block diagram for our activity-recognition method, showing the com-
ponents of the augmented-feature vector.
as a part of an augmented-feature vector. The block diagram
of this module is shown in Fig. 4. A brief description of each
feature is given as follows.
1) Autoregressive coefficients: An AR model can be repre-
sented as
y(t)=
p
i=1
α(i)y(ti)+ε(t)(1)
where α(i)is the AR-coefficients, y(t)the time series
under investigation which in our case is the acceleration
signal, pthe order of the filter, and ε(t)the noise term.
2) Signal-magnitude area (SMA): It is calculated according
to
SMA =
N
i=1
(|x(i)|)+(|y(i)|)+(|z(i)|)(2)
where x(i),y(i), and z(i)indicate the acceleration signal
along the x-axis, y-axis, and z-axis, respectively.
3) Tilt angle (TA): It refers to the relative tilt of the body in
space. It is defined as the angle between the positive z-axis
and the gravitational vector gand is calculated according
to
ϑ=arcos(z).(3)
C. Linear-Discriminant Analysis (LDA)
The LDA module takes the augmented-feature vector as input
and utilizes the class specific information to maximize the ratio
of the between and within-class scatter information. It seeks the
vectors in the underlying space to create the best discrimination
among different classes. It is well known for the feature ex-
traction and dimension reduction. The optimal discrimination
projection matrix Dopt is chosen from the maximization of the
ratio of the determinant of the between and within-class scatter
matrices as
Dopt = arg max
D
DTSBD
|DTSWD|=[d1,d
2,...,d
t]T(4)
where SBand SWare the between and within-class scatter
matrices, respectively. Further details on the LDA are available
in [29]. The 3-D feature plots for the four transitions before and
after applying the LDA are shown in Figs. 5 and 6, respectively.
Fig. 5. 3-D feature plot for the four transitions before the LDA.
Fig. 6. 3-D feature plot for the four transitions after the LDA.
D. Classifier
It is the fundamental element of the system which should be
adaptive and capable of providing the correct classification. In
other words, it must correctly understand the features even if
those features are considerably different from the ones it was
fed before. For this reason, we decided to use artificial-neural
nets (ANNs) based on the feed-forward backpropagation algo-
rithm. Perceptron neural nets with different number of layers
and neurons were tested in order to optimize the performance.
The maximal value of the weights in the neuron connections was
normalized to the modulus of 1. Different steps of the increment
for the weights were also investigated. The training of ANNs
was also repeated several times by changing the input order in
a random fashion.
For the state recognition, only one network (SNN) was
trained. The inputs to SNN were the state features. For the activ-
ity recognition, three networks were trained for three states, i.e.,
an ANN to recognize the static activities, an ANN to recognize
the transitions, and an ANN to recognize the dynamic activities.
1170 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 5, SEPTEMBER 2010
The inputs to each of these ANNs were the output of the LDA
module as shown in Fig. 4.
III. EXPERIMENTAL RESULTS
A. Determination of the AR-Model Order
In order to determine the optimal AR-model order, root mean
square error (rmse) values were calculated for each activity and
the behavior of the rmse curve against different model orders
was analyzed. We observed a decreasing trend against the AR-
model order. The curve tended to even out near the model order
of 10 which suggested that the optimum model order lied in the
neighboring values of 10.
B. Classification Results
In order to get the meaningful coefficients, the AR-modeling
of any time series generally requires the length of the series to
be significantly larger than the model order. We used a sam-
pling frequency of 20 Hz. It resulted in the failure of seizing the
meaningful AR-coefficients on the second-by-second basis, i.e.,
a window size of 20 samples. After several trials, we concluded
that the window size of 64, i.e., 3.2 s, with no overlapping be-
tween consecutive windows, is the most appropriate in our case.
First, this time interval was not too long to result in a delayed
response. Second, it provided enough raw data for extracting the
meaningful AR-coefficients. Thus, for the classification, accel-
eration data was fed every 3.2 s to the recognition system and
the output was compared to the true activity. The performance
of the proposed-recognition system was then validated in the
following four studies.
1) Test on State Recognition: The state features were ex-
tracted and the SNN was trained to recognize the three states.
The average-recognition accuracy was almost 99% which indi-
cates the feasibility of employing the proposed features for the
state recognition.
2) Test on Feature Augmentation: This study was performed
in order to try a number of configurations of the front-end
features for the activity recognition before landing on the
augmented-feature vector which is shown in Fig. 4. We per-
formed three different trials. Each trial involved a differ-
ent combination of the features for classifying the test ac-
tivities, i.e., {AR-coefficients},{AR-coefficients, SMA}, and
{AR-coefficients, SMA, and TA}. The recognition system used
in this study was single-level (SL-HAR), i.e., the features were
calculated and fed directly to the activity-recognition module
without the state recognition and a single ANN was trained to
recognize only the four basic activities, including lying, stand-
ing, walking, and running. The recognition results are summa-
rized in Table II.
3) Single-Level (SL) Versus Hierarchical (H)-HAR System:
The purpose of this study was to evaluate the performance of
the SL-HAR versus H-HAR system to recognize the 15 activ-
ities which are listed in Table I. This study was performed in
two steps. First, the state features and augmented-feature set
were calculated and fed to the SL-HAR system to recognize
the test activities. Second, the proposed H-HAR system was
TAB LE I I
AVERAGE-RECOGNITION RESULTS(IN PERCENT)FOR THE FOUR ACTIVITIES FOR
THE FEATURE-AUGMENTATION TESTS
TABLE III
AVERAGE RECOGNITION RESULTS(IN PERCENT)FOR ALL ACTIVITIES FOR THE
SL-HAR VERSUS H-HAR TEST
used to recognize the test activities. The recognition results are
summarized in Table III. They clearly indicate that the H-HAR
system outperformed the SL-HAR system by achieving an av-
erage recognition accuracy of 97.9%. The choice of 15 physical
activities, belonging to three states, generated complex deci-
sion boundaries in the feature space. The two-level recognition
broke down the overall recognition problem into two subprob-
lems and, thus, enabled the H-HAR to incorporate a set of two
ANNs to solve these complex decision boundaries.
4) Performance Evaluation With Limited Training Data:
Accelerometer-based human-activity recognition systems usu-
ally suffer a loss in accuracy during the practical deployment,
which is caused by either training the system with the data from
other people or with the limited training data from the person
for whom it is intended. To address this problem, we estimated
the subject independent classifier performance of the H-HAR
system using the sixfold cross validation, where six represents
the number of subjects participated in our study. Of these six
subjects, the activity dataset from a single subject was retained
as the validation dataset for testing the model and the activ-
ity datasets from the remaining five subjects were used as the
training datasets. This process was repeated six times, i.e., the
number of folds. The results from these folds are summarized
in Table IV. The average recognition accuracy of 97.65% indi-
cates that our proposed human-activity recognition scheme can
achieve high recognition rates for a specific subject even if an
adequate amount of training data from the intended subject is
unavailable.
KHAN et al.: TRIAXIAL ACCELEROMETER-BASED PHYSICAL-ACTIVITY RECOGNITION 1171
TAB LE I V
AVERAGE RECOGNITION RESULTS FOR ALL ACTIVITIES WHEN THE TRAINING
DATA FROM THE INTENDED SUBJECT IS NOT AVAI L A B L E
IV. DISCUSSION AND CONCLUSION
The aim of this paper is to provide an accurate and robust
human-activity recognition system for the u-lifecare environ-
ments. The proposed system incorporates the use of a single
triaxial accelerometer. It is feasible to be used by the free-living
subjects as it relies only on a single point of sensor’s attach-
ment to their bodies. It is effective in a sense that it is capable
of recognizing a broad set of daily physical activities with an
average accuracy of 97.9%. It is able to distinguish between the
sitting and standing postures, sit-stand and stand-sit transitions,
and walking-upstairs and walking-downstairs movements using
only a single triaxial accelerometer.
Although several systems have been proposed in the past
to monitor daily physical activities using accelerometers, this
system appears promising in several regards. First, its perfor-
mance compares favorably with the previously proposed sys-
tems. Mathie et al. proposed a rule-based heuristic system for
the classification of daily physical activities. However, the pri-
mary drawback of this system is its rule-based heuristic nature
as finding such a set of rules is a time-consuming process. It
also showed difficulty in distinguishing between the sit-stand
and stand-sit transitions and between the sitting and standing
postures [6], [16]. Allen et al. proposed an improvement by em-
ploying a GMM-based approach but it also exhibited difficulty
in distinguishing between sitting and standing postures [14].
Bao et al. used an ambulatory system based on the decision
trees to classify 20 activities using a seminaturalistic dataset
collected outside the laboratory [10]. However, the recognition
accuracy was only 84.26%. It also required subjects to wear five
biaxial accelerometers simultaneously on different parts of their
bodies and, thus, may not be feasible for the long-term activity
recognition in the free-living subjects. Ermes et al. proposed a
system based on a hybrid model classifier to assess the feasibil-
ity of the activity recognition in out-of-lab settings using both
the supervised and unsupervised activity data [17]. However,
the overall accuracy was 89% and the system showed inability
in differentiating between the sitting and standing postures.
Second, the dataset used in this study was collected by the
subjects at home without the researchers’ supervision and the
annotations were made on the spot. The use of a bluetooth
headset together with a speech-recognition software for the an-
notations resulted in very little interference while performing
the activities. Third, our proposed system is based on a novel
hierarchical recognition scheme. Our results show that it is su-
perior to a single-level recognition system as a large number of
activities result in complex decision boundaries in the feature
space which are difficult for a single classifier to solve.
Fourth, a novel augmented-feature vector was employed
to achieve the activity recognition within each state. It was
composed of the AR-coefficients, SMA, and TA. The AR-
coefficients were obtained by modeling the triaxial accelera-
tion signals using the AR-models. The recognition accuracy
of 97.9% illustrates the success of employing the proposed
augmented-feature vector for the activity recognition which in-
directly signifies the feasibility of using the AR-analysis. Fifth,
our results illustrate the success of our proposed recognition
system in achieving an acceptable recognition rate even in the
absence of an adequate amount of the training data from the
intended subject.
Thus, the experimental results of our proposed hierarchical
recognition scheme have shown significant potential in its ability
to accurately and robustly model the ambulatory data using a
single triaxial accelerometer. The next stage of evaluation will
be the assessment of our classification technique for the elderly
in a free-living context.
REFERENCES
[1] E. M. Tapia, S. S. Intille, and K. Larson, “Activity recognition in the
home using simple and ubiquitous sensors,” in Pervasive Computing,
Berlin/Heidelberg, Germany: Springer-Verlag, 2004, pp. 158–175.
[2] B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, and C. J. B¨
ula,
P. Robert, “Ambulatory system for human motion analysis using a kine-
matic sensor: Monitoring of daily physical activity in the elderly,” IEEE
Trans. Biomed. Eng., vol. 50, no. 6, pp. 711–723, Jun. 2003.
[3] J. Mantyjarvi, J. Himberg, and T. Seppanen, “Recognizing human motion
with multiple acceleration sensors,” in Proc. IEEE Int. Conf. Syst., Man,
Cybern., vol. 2, pp. 747–752, 2001.
[4] M. Sekine, T. Tamura, M. Akay, T. Fujimoto, T. Togawa, and Y. Fukui,
“Discrimination of walking patterns using wavelet-based fractal analysis,”
IEEE Trans. Neural Syst. Rehabil. Eng., vol. 10, no. 3, pp. 188–196, Sep.
2002.
[5] F. Foerster, M. Smeja, and J. Fahrenberg, “Detection of posture and motion
by accelerometry: A validation study in ambulatory monitoring,” Comput.
Hum. Behav., vol. 15, pp. 571–583, 1999.
[6] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “A pilot
study of long term monitoring of human movements in the home using
accelerometry,” J. Telemed. Telecare, vol. 10, pp. 144–151, 2004.
[7] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and
B. G. Celler, “Implementation of a real-time human movement classi-
fier using a tri-axial accelerometer for ambulatory monitoring,” IEEE
Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 156–67, Jan. 2006.
[8] P. H. Veltink, H. B. J. Bussmann, W. de Vries, W. L. J. Martens, and
R. C. van Lummel, “Detection of static and dynamic activities using uniax-
ial accelerometers,” IEEE Trans. Rehabil. Eng., vol. 4, no. 4, pp. 375–385,
Dec. 1996.
[9] S. H. Lee, H. D. Park, S. Hong, K. J. Lee, and Y. H. Kim, “A study on
the activity classification using a tri-axial accelerometer,” in Proc. 25th
Annu. IEEE Int. Conf. Eng. Med. Biol. Soc., 2003, vol. 3, pp. 2941–
2943.
[10] L. Bao and S. S. Intille, “Activity recognition from user-annotated ac-
celeration data,” in Pervasive Computing, Berlin/Heidelberg, Germany:
Springer-Verlag, 2004, pp. 158–175.
[11] J. B. J. Bussmann, W. L. Martens, J. H. M. Tulen, F. C. Schasfoort, H. J.
van den Berg-Emons, and H. J. Stam, “Measuring daily behaviour using
1172 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 5, SEPTEMBER 2010
ambulatory accelerometry: the activity monitor,” Behav. Res. Methods,
Instrum., Comput., vol. 33, pp. 349–356, 2001.
[12] M. Sung, C. Marci, and A. Pentland, “Wearable feedback systems for
rehabilitation,” J. Neuroeng. Rehabil, vol. 2, pp. 2–17, 2005.
[13] N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman, “Activityrecognition
from accelerometer data,” in Proc. 20th Nat. Conf. Artif. Intell., 2005,
pp. 1541–1546.
[14] F. Allen, E. Ambikairajah, N. Lovell, and B. Celler, “Classification of a
known sequence of motions and postures from accelerometry data using
adapted gaussian mixture models,” Physiol. Meas., vol. 27, pp. 935–951,
2006.
[15] K. Kiani, C. J. Snijders, and E. S. Gelsema, “Computerized analysis of
daily life motor activity for ambulatory monitoring,” Technol. HealthCare,
vol. 5, pp. 307–318, 1997.
[16] M. Mathie, B. Celler, N. Lovell, and A. Coster, “Classification of ba-
sic daily movements using a triaxial accelerometer,” Med. Biol. Eng.
Comput., vol. 42, pp. 679–687, 2004.
[17] M. Ermes, J. Parkka, J. Mantyjarvi, and I. Korhonen, “Detection of daily
activities and sports with wearable sensors in controlled and uncontrolled
conditions,” IEEE Trans. Inf. Technol. Biomed., vol. 12, no. 1, pp. 20–26,
Jan. 2008.
[18] U. Maurer, A. Smailagic, D. Siewiorek, and M. Deisher, “Activity recog-
nition and monitoring using multiple sensors on different body positions,”
in Proc. Int. Workshop Wearable Implantable Body Sens. Netw., 2006,
pp. 113–116.
[19] N. Kern, B. Schiele, H. Junker, P. Lukowicz, and G. Troster, “Wearable
sensing to annotate meeting recordings,” Pers.Ubiquitous Comput.,vol.7,
pp. 263–274, 2003.
[20] D. Minnen, T. Starner, J. Ward, P. Lukowicz, and G. Troester, “Recogniz-
ing and discovering human actions from on-body sensor data,” in Proc.
IEEE Int. Conf. Multimedia Expo, 2005, pp. 1545–1548.
[21] D. Giansanti, V. Macellari, and G. Maccioni, “New neural network classi-
fier of fall-risk based on the Mahalanobis distance and kinematic parame-
ters assessed by a wearable device,” Physiol. Meas., vol. 29, pp. N11–N19,
2008.
[22] M. R. Narayanan, M. E. Scalzi, S. J. Redmond, S. R. Lord, B. G. Celler, and
N. H. Lovell, “A wearable triaxial accelerometry system for longitudinal
assessment of falls risk,” in Proc. 30th Annu. IEEE Int. Conf. Eng. Med.
Biol. Soc., 2008, pp. 2840–2843.
[23] M. Marschollek, K. Wolf, M. Gietzelt, G. Nemitz, H. M. Z. Schwabe-
dissen, and R. Haux, “Assessing elderly persons’ fall risk using spectral
analysis on accelerometric data—A clinical evaluation study,” in Proc.
30th Annu. IEEE Int. Conf. Eng. Med. Biol. Soc., 2008, pp. 3682–3685.
[24] C. V. Bouten, K. T. Koekkoek, M. Verduin, R. Kodde, and J. D. Janssen, “A
triaxial accelerometer and portable data processing unit for the assessment
of daily physical activity,” IEEE Trans. Biomed. Eng., vol. 44, no. 3,
pp. 136–147, Mar. 1997.
[25] A. M. Khan, Y.K. Lee, and T.-S. Kim, “Accelerometer signal-based human
activity recognition using augmented autoregressive model coefficients
and artificial neural nets,” in Proc. 30th Annu. IEEE Int. Conf. Eng. Med.
Biol. Soc., 2008, pp. 5172–5175.
[26] K. Roth, I. Kauppinen, P. Esquef, and V. Valimaki, “Frequency warped
burg’s method,” in Proc. IEEE WASPAA, Mohonk, pp. 5–8, 2003.
[27] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “Accelerom-
etry: Providing an integrated, practical method for long-term, ambulatory
monitoring of human movement,” Physiol. Meas., vol. 25, pp. R1–R20,
2004.
[28] T. van Kasteren, A. Noulas, G. Englebienne, and B. Kr¨
ose, “Accurate
activity recognition in a home setting,” in Proc. UbiComp : Proc. 10th Int.
Conf. Ubiquitous Comput., New York: ACM, 2008, pp. 1–9.
[29] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs.
fisherfaces: Recognition using class specific linear projection,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711–720, Jul. 1997.
Adil Mehmood Khan received the B.S. degree in
information technology from the National University
of Sciences and Technology (NUST), Rawalpindi,
Pakistan, in 2005. He is currently working toward the
Ph.D. degree in the Department of Computer Engi-
neering, Kyung Hee University, Yongin-si, Korea.
His current research interests include ubiquitous
computing, context-aware computing for handheld
devices, wearable sensing, and pattern recognition.
Young-Koo Lee, photograph and biography not available at the time of
publication.
Sungyoung Y. Lee, photograph and biography not available at the time of
publication.
Tae-Seong Kim, photograph and biography not available at the time of
publication.
... Accelerometers, gyroscope, and magnetometer are standard wearable sensors that can be comfortably worn by humans [13], and these sensors can be incorporated into portable devices like smart bands, helmets, smartwatches, and smartphones [14]. In the past, most HAR systems relied on accelerometers to recognize various daily actions, including standing, sitting, walking, jogging, and lying [15][16][17][18]. Most of these systems employ several accelerometers fixed on body parts of the human. ...
... Andrey Ignatov [13] proposed the real-time HAR system using accelerometer data. Researchers of [15] recognized 15 physical activities using a triaxial accelerometer which was mounted on human chest. Smoke activity detection using single IMU is accomplished by the authors of [25]. ...
... Results obtained by k-NN, ANN, and NB algorithms were poor compared to the ML algorithms considered in the experimentation. Authors of [15] proposed the hierarchical method for accelerometer based HAR. At a lower level, statistical signal characteristics and ANN were used to classify the activity to which it belongs, such as static, transitional, or dynamic. ...
Article
Full-text available
Human activity recognition (HAR) has attracted considerable research attention due to its essential role in various domains, ranging from healthcare to security, safety, and entertainment. HAR has undergone a paradigm shift from simple single-task detection to the more complex task of identifying multiple simultaneous activities as technology advances. A wide range of methods, including sensing modalities, identification algorithms, a specified list of recognized activities, and end application goals, have been used in the literature to investigate activities carried out by single individuals. However, there appears to be a research gap when it comes to scenarios in which several people engage in individual or concurrent activities. Although numerous reviews and surveys have previously addressed HAR, with the continual expansion of literature, there is a necessity for an updated assessment of the status of HAR literature. The system encompasses various operational modules, including data acquisition, noise elimination, and distortion reduction through preprocessing, followed by feature extraction, feature selection, and classification. Recent advancements have introduced state-of-the-art techniques for feature extraction and selection, which are categorized using traditional machine learning classifiers. However, a notable limitation is observed, as many of these techniques rely on basic feature extraction processes, hindering their capability to recognize complex activities. This article reviewed 190 articles with respect to data collection, segmentation, feature extraction, energy efficiency, personalized models, and machine learning (ML) and deep learning (DL) approaches for sensor-based HAR. Open challenges and future enhancements of HAR are also discussed in this article.
... Popular features for such tasks include mean, standard deviation, maximum, peak-to-peak, root-mean-square, and correlation 978-1-4799-7560-0/15/$31 c 2015 IEEE between values of modality axes [4]. Another suggested option is using autoregressive modeling to form augmented-feature vectors [5]. ...
... We benchmark our procedure against standard feature extraction methods. We computed 568 features using mean, standard deviation, correlation, RMSE from [4], Energy [17], average absolute difference [18], largest 5 FFT magnitudes, autocorrelation, kurtosis, skew from [7], autoregression coefficients [5], and number of zeros for the original time series, and first difference time series. We also computed the mean, standard deviation, kurtosis, and skew for the magnitudes of the FFT [16]. ...
Preprint
Accurate and computationally efficient means for classifying human activities have been the subject of extensive research efforts. Most current research focuses on extracting complex features to achieve high classification accuracy. We propose a template selection approach based on Dynamic Time Warping, such that complex feature extraction and domain knowledge is avoided. We demonstrate the predictive capability of the algorithm on both simulated and real smartphone data.
... In [129], this accuracy has been computed for experimental and natural setting, 95.6% and 66% respectively. Moreover, recognition accuracy was equal to 92.25% in [13], 95% in [130], 97% in [131] and as high as 98% in [132]. Performing activities in a natural environment is unsupervised and less controlled. ...
Article
Full-text available
Human activity recognition systems using wearable sensors is an important issue in pervasive computing, which applies to various domains related to healthcare, context aware and pervasive computing, sports, surveillance and monitoring, and the military. Three approaches can be considered for activity recognition: video sensor-based, physical sensor-based, and environmental sensor-based. This paper investigates the related work regarding the physical sensor-based approaches to motion processing. In this paper, a wide range of artificial intelligence models, from single classifications to methods based on deep learning, have been reviewed. The human activity detection accuracy of different methods, under natural and experimental conditions poses several challenges. These challenges cause problems regarding the accuracy and applicability of the proposed methods. This paper analyzes the methods, challenges, approaches, and future work. The goal of this paper is to establish a clear distinction in the field of motion detection using inertial sensors.
... Their work's limitation was the data collection involved only 7 participants (between the ages of 22 and 28). In [35], the authors used a single accelerometer (triaxis) for HAR. In their research, the accelerometer had to be attached to the chest of an individual in a specific orientation. ...
Article
Full-text available
Analyzing physical activities through wearable devices is a promising research area for improving health assessment. This research focuses on the development of an affordable and real-time Human Activity Recognition (HAR) system designed to operate on low-performance microcontrollers. The system utilizes data from a body-worn accelerometer to recognize and classify human activities, providing a cost-effective, easy-to-use, and highly accurate solution. A key challenge addressed in this study is the execution of efficient motion recognition within a resource-constrained environment. The system employs a Random Forest (RF) classifier, which outperforms Gradient Boosting Decision Trees (GBDT), Support Vector Machines (SVM), and K-Nearest Neighbors (KNN) in terms of accuracy and computational efficiency. The proposed features Average absolute deviation (AAD), Standard deviation (STD), Interquartile range (IQR), Range, and Root mean square (RMS). The research has conducted numerous experiments and comparisons to establish optimal parameters for ensuring system effectiveness, including setting a sampling frequency of 50 Hz and selecting an 8-s window size with a 40% overlap between windows. Validation was conducted on both the WISDM public dataset and a self-collected dataset, focusing on five fundamental daily activities: Standing, Sitting, Jogging, Walking, and Walking the stairs. The results demonstrated high recognition accuracy, with the system achieving 96.7% on the WISDM dataset and 97.13% on the collected dataset. This research confirms the feasibility of deploying HAR systems on low-performance microcontrollers and highlights the system’s potential applications in patient support, rehabilitation, and elderly care.
... Furthermore, multiple-stage training through long short-term memory (LSTM)-convolutional neural network (CNN)-LSTM, artificial neural network (ANN), CNN-LSTM [2], and several deep learning frameworks [3], [4] are also suggested by some recent literatures [5], [6], [7]. However, augmented-signal features and a hierarchical recognizer are combined to get a highly functional trained structure reported in [8]. Reviewing existing literature it can be concluded that achieving a high level of accuracy in classifying human activities with raw 1-D time domain data from accelerometers and gyroscope sensors is challenging due to the inability of shallow networks to extract meaningful patterns, especially for similar activities, such as walking (WA), lying (LA), and sitting (SI). ...
Article
Full-text available
A person's movement or relative positioning can be effectively captured by different types of sensors and corresponding sensor output can be utilized in various manipulative techniques for the classification of different human activities. This letter proposes an effective scheme for human activity recognition, which introduces two unique approaches within a multi-structural architecture, named FusionActNet. The first approach aims to capture the static and dynamic behavior of a particular action by using two dedicated residual networks and the second approach facilitates the final decision-making process by introducing a guidance module. A two-stage training process is designed where at the first stage, residual networks are pre-trained separately by using static (where the human body is immobile) and dynamic (involving movement of the human body) data. In the next stage, the guidance module along with the pre-trained static/dynamic models are used to train the given sensor data. Here the guidance module learns to emphasize the most relevant prediction vector obtained from the static/dynamic models, which helps to effectively classify different human activities. The proposed scheme is evaluated using two benchmark datasets and compared with state-of-the-art methods. The results clearly demonstrate that our method outperforms existing approaches in terms of accuracy, precision, recall, and F1 score, achieving 97.35% and 95.35% accuracy on the UCI HAR and Motion-Sense datasets, respectively which highlights both the effectiveness and stability of the proposed scheme.
Article
Full-text available
We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space-if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed “Fisherface” method has error rates that are lower than those of the eigenface technique for tests on the Harvard and Yale face databases
Article
Full-text available
We describe our initial efforts to learn high level human behaviors from low level gestures observed using on-body sensors. Such an activity discovery system could be used to index captured journals of a person's life automatically. In a medical context, an annotated journal could assist therapists in helping to describe and treat symptoms characteristic to behavioral syndromes such as autism. We review our current work on user-independent activity recognition from continuous data where we identify “interesting” user gestures through a combination of acceleration and audio sensors placed on the user's wrists and elbows. We examine an algorithm that can take advantage of such a sensor framework to automatically discover and label recurring behaviors, and we suggest future work where correlations of these low level gestures may indicate higher level activities.
Chapter
Full-text available
We develop a face recognition algorithm which is insensitive to gross variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face under varying illumination direction lie in a 3-D linear subspace of the high dimensional feature space — if the face is a Lambertian surface without self-shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's Linear Discriminant and produces well separated classes in a low-dimensional subspace even under severe variation in lighting and facial expressions. The Eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed Fisherface method has error rates that are significantly lower than those of the Eigenface technique when tested on the same database.
Conference Paper
Full-text available
This paper describes a design of an algorithm for analyzing human activity using a body-fixed triaxial accelerometer on the back. In the first step, we distinguish static and dynamic activity period using AC signal analysis. Then five static activities were classified by applying the threshold in DC signal corresponding to the static activity period. Also, after taking AC signal and negative peak signal in the dynamic activity period, the four dynamic activities were classified by adaptive threshold method. To evaluate the performance of the proposed algorithm, the measured signals obtained from 12 subjects were applied to the proposed algorithm and the results were compared with the simultaneously measured video data. As a result, the activity classification rate of 95.1% on average was obtained. Overall results show that the proposed classification algorithm has a possibility to be used to analyze the static and dynamic physical activity.
Conference Paper
Full-text available
Activity recognition fits within the bigger framework of context awareness. In this paper, we report on our efforts to recognize user activity from accelerometer data. Activity recognition is formulated as a classifica- tion problem. Performance of base-level classifiers and meta-level classifiers is compared. Plurality Voting is found to perform consistently well across different set- tings.
Article
During the past several years, researchers have demonstrated that when new wireless sensors are placed in the home environment, data collected from them can be used by software to automatically infer context, such as the activities of daily living. This context-inference can then be exploited in novel applications for healthcare, communication, education, and entertainment. Prior work on automatic context-inference has cleared the way to a reduction in costs associated with manufacturing the sensor technologies and computing resources required by these systems. However, this prior work does not specifically address another major expense of wide-scale deployment of the proposed systems: the expense of expert installation of the sensor devices. To date, most of the context-detection algorithms proposed assume that an expert carefully installs the home sensors and that an expert is involved in acquiring the necessary training examples. End-user sensor installation may offer several advantages over professional sensor installations: 1.) It may greatly reduces the high cost of time required for an expert installation, especially if large numbers of sensors are required for an application, 2.) The process of installing the sensors may give the users a greater sense of control over the technology in their homes, and 3.) End-User Installations also may improve algorithmic performance by leveraging the end-user's domain expertise. An end-user installation method is proposed using "stick on" wireless object usage sensors. The method is then evaluated employing two in-situ, exploratory user studies, where volunteers live in a home fitted with an audio-visual monitoring system. Each participant was given a phone-based tool to help him or her self-install the object usage sensors. They each lived with the sensors for over a week. They were also asked to provide some training data on their everyday activities using multiple methods. Data collected from the two studies is used to qualitatively compare the end-user installation with two professional installation methods. Based on the two exploratory experiments, design guidelines for user self-installation of home sensors are proposed.
Conference Paper
We propose to use wearable computers and sensor systemsto generate personal contextual annotations in audiovisual recordings of meetings. In this paper we arguethat such annotations are essential and effective to allowretrieval of relevant information from large audio-visualdatabases. The paper proposes several useful annotationsthat can be derived from cheap and unobtrusive sensors. Italso describes a hardware platform designed to implementthis concept and presents first experimental results.
Article
The suitable placement of a small number of calibrated piezoresistive accelerometer devices may suffice to assess postures and motions reliably. This finding, which was obtained in a previous investigation, led to the further development of this methodology and to an extension from the laboratory to conditions of daily life. The intention was to validate the accelerometric assessment against behavior observation and to examine the retest reliability. Twenty-four participants were recorded, according to a standard protocol consisting of nine postures/motions (repeated once) which served as reference patterns. The recordings were continued outside the laboratory. A participant observer classified the postures and motions. Four sensor placements (sternum, wrist, thigh, and lower leg) were used. The findings indicated that the detection of posture and motion based on accelerometry is highly reliable. The correlation between behavior observation and kinematic analysis was satisfactory, although some participants showed discrepancies regarding specific motions.
Conference Paper
In this work, a system for recognizing activities in the home setting using a set of small and simple state-change sensors is introduced. The sensors are designed to be “tape on and forget” devices that can be quickly and ubiquitously installed in home environments. The proposed sensing system presents an alternative to sensors that are sometimes perceived as invasive, such as cameras and microphones. Unlike prior work, the system has been deployed in multiple residential environments with non-researcher occupants. Preliminary results on a small dataset show that it is possible to recognize activities of interest to medical professionals such as toileting, bathing, and grooming with detection accuracies ranging from 25% to 89% depending on the evaluation criteria used.