Activity Recognition on an Accelerometer
Embedded Mobile Phone with Varying Positions
Lin.Sun1,Daqing Zhang1, Bin.Li1, Bin.Guo1, Shijian Li2
1Handicom Lab, TELECOM SudParis,
9, Rue Charles Fourier, 91011, France
2Department of Computer Science, Zhejiang University, Hangzhou, 310027, China
Abstract. This paper uses accelerometer-embedded mobile phones to
monitor one’s daily physical activities for sake of changing people’s seden-
tary lifestyle. In contrast to the previous work of recognizing user’s phys-
ical activities by using a single accelerometer-embedded device and plac-
ing it in a known position or fixed orientation, this paper intends to
recognize the physical activities in the natural setting where the mobile
phone’s position and orientation are varying, depending on the position,
material and size of the hosting pocket. By specifying 6 pocket positions,
this paper develops a SVM based classifier to recognize 7 common phys-
ical activities. Based on 10-folder cross validation result on a 48.2 hour
data set collected from 7 subjects, our solution outperforms Yang’s solu-
tion and SHPF solution by 5∼6%. By introducing an orientation insen-
sitive sensor reading dimension, we boost the overall F-score from 91.5%
to 93.1%. With known pocket position, the overall F-score increases to
Keywords: activity recognition, SVM, mobile phone, accelerometer
The prevailing sedentary lifestyle in modern society has lead to various physical
and mental diseases, such as obesity, coronary heart diseases, type II diabetes
and depression, which request enormous medical cost. According to World Health
Organization, there are at least 1.9 million people die as a result of physical in-
activity annually . In U.S. alone, it leads to about 300, 000 preventable deaths
and more than 90$ billion direct health cost annually . Even though people are
aware of the benefits of exercises, there is a lack of external intervention which
can properly bring the busy people out of the sedentary routine, thus an auto-
matic and personal reminder will be very helpful if it can monitor one’s physical
activities and persuade people to participate in physical activities regularly at
the right time and place.
2L. Sun et al.
Activity recognition technology is a key enabling technology to tackle this
problem as it’s able to monitor individual’s physical daily activities and the
lasting duration so as to estimate the calories consumed each day. Based on the
consumed calorie, the system can provide recommendation and advices when
they fail to complete enough exercise and also build systems to encourage people
to conduct more activities [4,5,3]. There are several ways to recognize people’s
daily activities. One way is using cameras to visually detect people’s motion
[8,7]. The drawback of this solution is that to monitor a moving person, large
number of cameras need be deployed with high cost. And also the system should
be designed to aggregate the information from each camera and deal with the
influential factors such as lighting condition, mounting distance and angel, which
make the system very complicate and impractical. Another way is using personal
companion devices such as mobile phones or watches with sensing and comput-
ing power embedded to detect physical activities. The merit of this solution is
that we don’t need to deploy additional devices and the system is simple and
easy to use. Since people carry their personal companion devices all the time
and have the full control of their own devices, thus those devices won’t make
the users feel intrusive or cause extra money burden. Out of the two companion
devices, the watches are normally placed on the wrist. Since the casual moving of
arms doesn’t have a direct and obvious relationship with ongoing activities, also
modern watches are still not powerful enough to do data processing, therefore
personal watches have a lot of constraints in detecting one’s physical activities.
On the contrary, mobile phones are becoming increasingly intelligent and pow-
erful. When they are carried by people in pockets or bags, they are moving with
the pace of the human body, thus they appear to be the ideal platforms for
detecting people’s physical activities such as sitting, walking, running and etc.
Modern mobile phones like iPhone or Nokia N97 are embedded with various sen-
sors such as the accelerometer, approximity sensor, magnetometer, GPS and etc.
Of all these embedded sensors, the accelerometer is commonly used for activity
recognition. Although GPS could detect one’s movement in terms of location
and speed, it cannot tell the user moves in an accurate manner. In particular,
GPS doesn’t work inside buildings where people spend most of their time in.
Therefore, using the accelerometer-embedded mobile phones to recognize peo-
ple’s physical activities becomes the primary choice among all the solutions.
With the accelerometer-embedded mobile phone, there are two possibilities
to monitor people’s physical activities. One is turning the mobile phone as a
pedometer, measuring the step counts and calorie consumption  for each user.
The other is recognizing precise physical activities such as walking, running,
bicycling, driving and etc. Apparently the pedometer solution is quite simple, it
provides good indication for each user’s calorie consumed. While it works well
for the cases of walking, running, taking staircases, etc., it fails to estimate the
calorie consumption correctly in the case of bicycling (helpful to the health but
cannot be measured by pedometer). On the contrary, recognizing one’s physical
activities and the lasting duration can infer more accurate and comprehensive
information about people’s life style. Besides informing the calorie consumed
Activity Recognition with Varying Positions and Orientations3
more accurately, the activity patterns can inform users’ preferences and habits,
which can serve as the basis for further exercise recommendation.
 shows that 60% of men put their mobile phones in their pockets. With
different clothes dressed each day, people are used to putting the mobile phone
in different pocket (often the most convenient one). Depending on the position,
material and size of the pocket, the mobile phones often have varying orienta-
tion, especially when the very pocket swings with human body. Till now, the
prior work on activity recognition with accelerometer-equipped mobile devices
assumes a fixed mobile phone position or certain orientation [11–17], this as-
sumption usually doesn’t hold for the usual case of carrying the phone in the
pocket. In this work we choose to recognize seven most representative daily ac-
tivities that are strongly linked to physical exercises, and we intend to investigate
the activity recognition issue assuming that the mobile phone is freely placed in
one of the pockets. Under this assumption, the accelerometer sensor inside the
phone will take the position and orientation associated with the moving pocket.
With the varying orientation of the mobile phone, the experienced force will
cause varying effect on the three components of the acceleration signal . This
paper attempts to propose an orientation independent sensor reading dimension
which can relieve the effect of the varying orientation on the performance of
the activity recognition. For the position variation of the mobile phone, besides
training a single optimal SVM classifier for all seven physical activities in all
the pocket positions, we would like to train an optimal classifier for each pocket
location and hopefully can select the right classifier according to the mobile
phone position detected in the future.
The rest of the paper is organized as follows: in Section 2, the related work
about activity recognition using mobile or wearable devices is summarized. Then
in Section 3, our design hypothesis is elaborated to set-up the stage for the re-
search work. Section 4 presents the detailed design process for feature extraction
and classification, aiming at developing an orientation insensitive algorithm. Sec-
tion 5 describes the experimentation strategy to select the optimal size of the
window as well as the optimal set of SVM parameters corresponding to different
pocket position. In Section 6, the experimental results and analysis are provided
to demonstrate the effectiveness of the proposed approaches for tackling the
varying orientation and position issue. Finally, Section 7 gives the conclusions
about the paper.
Activity recognition with wearable sensors has been a hot research field in the
last decade. Much research work has been done to recognize physical activities
such as sitting, standing, running and so on for wellbeing management. In order
to differentiate diverse activities or gestures, sensors are best placed at locations
where the intrinsic characteristics of the target activities can be well captured.
For example, an accelerometer placed in the ankle can measure the leg motion
properly, and a barometers fixed on human body can detect the altitude change
14L. Sun et al.
Table 6. Precision, recall and F-score comparisons.
Overall Precision Overall Recall Overall F-score
Generic SVM without Magnitude
Generic SVM with Magnitude
Individual SVM with Magnitude
The authors would like to thank Chanaphan Prasomwong and Wei Wang for
building the data collection program with python in Nokia N97 and collecting
2. Manson, J.E., Skerrett, P.J, Greenland, P. and VanItallie, T.B.: The Escalating
Pandemics of Obesity and Sedentary Lifestyle: A Call to Action for Clinicians. In:
Arch Intern Med, 164, 3 2004, pp. 249–258.
3. Consolvo S. et al.: Activity Sensing in the Wild: A Field Trial of UbiFit Garden.
In: CHI’08 (2008).
4. Lin J., Mamykina L., Lindtner S., Delajoux G., Strub H.: Fish’n’Steps: Encourag-
ing Activitiy with an Interactive Computer Game. In: Ubicomp 2006 pp.261-278,
5. Anderson I., Maitlan J. Sherwood S., Barkhuus L., Chalmers M. Hall M., Brown B.,
Muller H.: Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented
Mobile Phones. In: Mobile Networks and Applications. pp. 185–199 (2007).
6. Maitland J., Sherwood S., Barkhuus L., Anderson I., Hall M., Brown B., Chalmers
M. Muller H.: Increasing the Awareness of Daily Activity Levels with Pervasive
Computing. In: Proc. of Pervasive Health’06 (2006).
7. Pavan T., Chellappa R., Subrahmanian V.S., Udrea O.: Machine Recognition of
Human Activities: A survey. In: IEEE Transactions on Circuits and Systems for
Video Technology, Vol. 18, No.11 (2008).
8. Hu W., Tan T. Wang L., Maybank S.: A survey on Visual Surveillance of Object
Motion and Behaviors. In: IEEE Transactions on Systems, Man, and Cybernetics.
Part C: Applications and Reviews. Vol.34, No.3. (2004).
9. Fujiki Y.: iPhone as a Physical Activity Measurement Platform. In: CHI’10 USA
10. Ichikawa F., Chipchase J., Grignani R.: Where is the Phone? A Study of Mobile
Phone Location in Public Spaces. In: The Second International Conference on
Mobile Technology, Application and Systems. pp. 797–804 (2005).
11. Bao L., Intille, S.: Activity Recognition from User-Annotated Acceleration Data.
In: Proc. Pervasive pp. 1-17 (2004).
12. Parkka J., Ermes M., Korpipaa P., Mantyjarvi J. Peltolla J. Korhonen I.: Activity
Classification Using Realistic Data From Wearable Sensors. IEEE Transactions on
Information Technology in Biomedicine. pp. 119–128 (2006).
HealthOrganization: Movefor Health.
Activity Recognition with Varying Positions and Orientations15
13. Lester J., Choudhury T., Kern, N., Borriello G. Hannaford B.: A hybrid discrim-
inative /generative approach for modeling human activities. In: Pro. of the Inter-
national Joint Conference on Artificial Intelligence (IJCAI), pp. 776–772 (2005).
14. Ravi N., Dandekar N., Mysore P., Littman M.L.: Activity Recognition from Ac-
celerometer Data. In: AAAI, pages 1541–1546 (2005).
15. Maurer U., Smailagic A., Siewiorek D.P., Deisher M.: Activity Recognition and
Monitoring Using Multiple Sensors on Different Body Positions. In: Proc. Of
the International Workshop on Wearable and Implantable Body Sensor Netowrks
(BSN’06) pp. 113–116 (2006).
16. Lester J., Choudhury T., Kern, N., Borriello G.: A Practical Approach to Recognize
Physical Activities. In: Pro. Pervasive pp. 1–16 (2006).
17. Yang J.: Toward Physical Activity Diary: Motion Recognition Using Simple Ac-
celeration Features with Mobile Phones. In:IMCE’09 Beijing, China (2009).
18. Mizell D.: Using gravity to estimate accelerometer orientation. In: ISWC’03, Proc.
Of the 7th IEEE International Symposium on Wearable Computers. pp. 252 USA
19. Wu J., Pan G., Zhang D. Qi G. Li S.: Gesture Recognition with a 3-D Accelerom-
eter. In:UIC’09 pp.25–38 (2009).
20. Chih-Chung Chang and Chih-Jen Lin, LIBSVM: a library for support vector ma-
chines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
21. Baek, J., Kim, S., Kim, H., Cho, J., Yun, B.: Recognition of User Activity for
User Interface on a Mobile Device. In: Procs. of the 24th South East Asia Regional
Computer Conference. Thailand (2007).