No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Recognizing multi-user activities using wearable sensors in a smart home
Abstract
The advances of wearable sensors and wireless networks offer many opportunities to recognize human activities from sensor readings in pervasive computing. Existing work so far focuses mainly on recognizing activities of a single user in a home environment. However, there are typically multiple inhabitants in a real home and they often perform activities together. In this paper, we investigate the problem of recognizing multi-user activities using wearable sensors in a home setting. We develop a multi-modal, wearable sensor platform to collect sensor data for multiple users, and study two temporal probabilistic models—Coupled Hidden Markov Model (CHMM) and Factorial Conditional Random Field (FCRF)—to model interacting processes in a sensor-based, multi-user scenario. We conduct a real-world trace collection done by two subjects over two weeks, and evaluate these two models through our experimental studies. Our experimental results show that we achieve an accuracy of 96.41% with CHMM and an accuracy of 87.93% with FCRF, respectively, for recognizing multi-user activities.
... More sensors have been combined with wearables, such as audio, temperature, humidity, and light sensors in [40] to recognize multi-user activities. Ince et al. [41] have combined a wrist worn accelerometer and environmental sensors to recognize brushing teeth, washing face, and shaving activities. ...
... When wearable sensors are used, data is collected continuously at a pre-defined frequency depending on the sensors capabilities, the resolution needed, and the storage capacity or transfer speed of the device. Commonly used accelerometer frequencies range from 10 Hz [37] for simple activities such as standing, sitting, or lying down to 50 Hz for most smartphone based systems [42,43], to 90 Hz [33], 125 Hz [31], 128 Hz [40], or 150 Hz [36] for higher resolution systems. ...
... High-pass filters are commonly used to remove the unwanted DC component in accelerometer data, which corresponds to the contribution of gravity [33,36,37,41]. Low-pass filters have been used on accelerometer data in [40,43]. Nguyen et al. [54] have used a threshold based approach to eliminate noise from sound data. ...
In this paper, we review the evolution of the field of sensor-based activity recognition from early offline implementations to more recent, real-time distributed solutions. This review aims to give a wide overview of the state of the field, and it is aimed at anyone interested in exploring IoT oriented solutions for activity recognition, and wanting to know more about the various challenges, existing solutions, and axes of optimizations to carry out wireless sensor based real-time distributed activity recognition for healthcare applications.
... Analyzing the papers selected and summarized in Table S13, it can be observed that 78% of them exclusively analyze smart homes, 16% take into consideration smart buildings in general, 3% analyze both smart homes and buildings, while the remaining 3% of the selected papers refer to smart workplace environments. The authors of these scientific articles make use of different types of sensors in their analyses, including wireless sensor networks [70,115,[119][120][121][122][123][124]; passive infrared motion sensors [82,97,117,118,122,125,126]; motion sensors [25,70,81,118,120,127,128]; environmental sensors [10,25,81,82,[116][117][118]123,[127][128][129][130][131][132]; temperature sensors [116,118,120,123,125,[131][132][133]; humidity sensors [123,[131][132][133]; pressure sensors [128,130,131,133]; light sensors [3,123,132]; unobtrusive sensing infrastructures [116]; real and virtual sensors [134]; radar sensors [135]; accelerometers [127,136]; light-emitting diodes (LED) [3]; electricity and electrical sensors [81,131,132]; smartphone sensors [127,131]; microphones [125,129]; distributed sensor networks [137]; simple non-intrusive sensors [82]; infrared sensors [124,[129][130][131]; actuators and home automation equipment [125]; shelf binary sensors [128]; biosensors [33]; smart meters [138]; acoustics and CO2 sensors [133]; nonwearable ambient sensors [131]. ...
... Analyzing the papers selected and summarized in Table S13, it can be observed that 78% of them exclusively analyze smart homes, 16% take into consideration smart buildings in general, 3% analyze both smart homes and buildings, while the remaining 3% of the selected papers refer to smart workplace environments. The authors of these scientific articles make use of different types of sensors in their analyses, including wireless sensor networks [70,115,[119][120][121][122][123][124]; passive infrared motion sensors [82,97,117,118,122,125,126]; motion sensors [25,70,81,118,120,127,128]; environmental sensors [10,25,81,82,[116][117][118]123,[127][128][129][130][131][132]; temperature sensors [116,118,120,123,125,[131][132][133]; humidity sensors [123,[131][132][133]; pressure sensors [128,130,131,133]; light sensors [3,123,132]; unobtrusive sensing infrastructures [116]; real and virtual sensors [134]; radar sensors [135]; accelerometers [127,136]; light-emitting diodes (LED) [3]; electricity and electrical sensors [81,131,132]; smartphone sensors [127,131]; microphones [125,129]; distributed sensor networks [137]; simple non-intrusive sensors [82]; infrared sensors [124,[129][130][131]; actuators and home automation equipment [125]; shelf binary sensors [128]; biosensors [33]; smart meters [138]; acoustics and CO2 sensors [133]; nonwearable ambient sensors [131]. ...
... Analyzing the papers selected and summarized in Table S13, it can be observed that 78% of them exclusively analyze smart homes, 16% take into consideration smart buildings in general, 3% analyze both smart homes and buildings, while the remaining 3% of the selected papers refer to smart workplace environments. The authors of these scientific articles make use of different types of sensors in their analyses, including wireless sensor networks [70,115,[119][120][121][122][123][124]; passive infrared motion sensors [82,97,117,118,122,125,126]; motion sensors [25,70,81,118,120,127,128]; environmental sensors [10,25,81,82,[116][117][118]123,[127][128][129][130][131][132]; temperature sensors [116,118,120,123,125,[131][132][133]; humidity sensors [123,[131][132][133]; pressure sensors [128,130,131,133]; light sensors [3,123,132]; unobtrusive sensing infrastructures [116]; real and virtual sensors [134]; radar sensors [135]; accelerometers [127,136]; light-emitting diodes (LED) [3]; electricity and electrical sensors [81,131,132]; smartphone sensors [127,131]; microphones [125,129]; distributed sensor networks [137]; simple non-intrusive sensors [82]; infrared sensors [124,[129][130][131]; actuators and home automation equipment [125]; shelf binary sensors [128]; biosensors [33]; smart meters [138]; acoustics and CO2 sensors [133]; nonwearable ambient sensors [131]. ...
Lately, many scientists have focused their research on subjects like smart buildings, sensor devices, virtual sensing, buildings management, Internet of Things (IoT), artificial intelligence in the smart buildings sector, improving life quality within smart homes, assessing the occupancy status information, detecting human behavior with a view to assisted living, maintaining environmental health, and preserving natural resources. The main purpose of our review consists of surveying the current state of the art regarding the recent developments in integrating supervised and unsupervised machine learning models with sensor devices in the smart building sector with a view to attaining enhanced sensing, energy efficiency and optimal building management. We have devised the research methodology with a view to identifying, filtering, categorizing, and analyzing the most important and relevant scientific articles regarding the targeted topic. To this end, we have used reliable sources of scientific information, namely the Elsevier Scopus and the Clarivate Analytics Web of Science international databases, in order to assess the interest regarding the above-mentioned topic within the scientific literature. After processing the obtained papers, we finally obtained, on the basis of our devised methodology, a reliable, eloquent and representative pool of 146 papers scientific works that would be useful for developing our survey. Our approach provides a useful up-to-date overview for researchers from different fields, which can be helpful when submitting project proposals or when studying complex topics such those reviewed in this paper. Meanwhile, the current study offers scientists the possibility of identifying future research directions that have not yet been addressed in the scientific literature or improving the existing approaches based on the body of knowledge. Moreover, the conducted review creates the premises for identifying in the scientific literature the main purposes for integrating Machine Learning techniques with sensing devices in smart environments, as well as purposes that have not been investigated yet.
... As a remark, the authors of [64] mentioned the usage of time-and frequency-domain features, but they did not specify which ones exactly. [45,46,57,69,72,76,82,90,91] Maximun Largest value in the window [45,46,57,69,72,76,82,90,91] Energy Average sum of squares [49,50,71,75,76,78,82,91] Signal Magnitude Area [47,50,53,78,80,82,91] IQR Interquartile Range [55,76,78,82,89,91] Root Mean Square Square root of the arithmetic mean [50,55,56,61,75,90] Kurtosis [45,46,60,68,80] Skewness [45,46,50,75,80] MinMax Difference between the Maximum and the Minimum in the window [50,61,63,75] Application-based features refer to features that were created for a certain application or dataset. These features are based on geometric, structure and kinematic relations. ...
... The authors of [52] trained a HMM model per axis (x, y, and z) of pre-processed acceleration measurements, fusing them with a weighted sum. The authors of [29,62,71] also deployed HMMs, and the authors of [49] used couple HMMs. The authors of [78] used a hierarchical conditional HMMs. ...
This contribution provides a systematic literature review of Human Activity Recognition for Production and Logistics. An initial list of 1243 publications that complies with predefined Inclusion Criteria was surveyed by three reviewers. Fifty-two publications that comply with the Content Criteria were analysed regarding the observed activities, sensor attachment, utilised datasets, sensor technology and the applied methods of HAR. This review is focused on applications that use marker-based Motion Capturing or Inertial Measurement Units. The analysed methods can be deployed in industrial application of Production and Logistics or transferred from related domains into this field. The findings provide an overview of the specifications of state-of-the-art HAR approaches, statistical pattern recognition and deep architectures and they outline a future road map for further research from a practitioner’s perspective.
... The greatest weakness of ambient sensors is they are very sensitive to noise that causes considerable difficulty for processing data and for learning a recognition model. This problem, fortunately, can be ameliorated by statistical modelling methods as what have been shown in previous works ( Tran et al. 2017, Singla et al. 2020, Cook 2012, Chen and Tong 2014, Alemdar et al. 2013, Wang et al. 2011). ...
... For activity recognition, studies in both ARAS and CASAS projects show that HMMs and CRFs have been the most successful techniques (Tran et. al 2017, Chen and Tong 2014, Alemdar et al. 2013, Wang et al. 2011). Recent work on combining dependencies can improve the recognition accuracy with a large margin (Tran et. ...
This chapter demonstrates a system that can turn a normal house to a smart house for daily activity monitoring with the use of ambient sensors. We first introduce our smarter safer home platform and its applications in supporting independent livings of seniors in their own home. Then we show a proof of concept which includes a novel activity annotation method through voice recording and deep learning techniques for automatic activity recognition. Our multi-resident activity recognition system (MRAR) is designed to support multiple occupants in a house with minimum impact on their living styles. We evaluate the system in a real house lived by a family of three. The experimental results show that it is promising to develop a smart home system for multiple residents which is portable and easy to deploy.
... During the years, various smart environment systems have been proposed. They have been used in many scenarios, such as family houses [14], offices [15], shopping malls [16], and museums [17], and have been applied for a variety of purposes, including tracking people in buildings [18], counting people numbers [19], recognizing human behavior [20], etc. In addition, energy consumption can be monitored, and the indoor environment can be controlled automatically by using appropriate sensors and controllers [21]. ...
... Vera et al. [49] proposed a system to count people using depth cameras mounted in the zenithal position: people are detected in each camera and the tracklets that belong to the same person are determined. Even though vision-based methods are efficient and reliable [14,50,51], they are unsuitable for smart homes due to privacy reasons. Algorithms based on wearable devices are infeasible for detecting visitors who do not wear such devices. ...
Population aging requires innovative solutions to increase the quality of life and preserve autonomous and independent living at home. A need of particular significance is the identification of behavioral drifts. A relevant behavioral drift concerns sociality: older people tend to isolate themselves. There is therefore the need to find methodologies to identify if, when, and how long the person is in the company of other people (possibly, also considering the number). The challenge is to address this task in poorly sensorized apartments, with non-intrusive sensors that are typically wireless and can only provide local and simple information. The proposed method addresses technological issues, such as PIR (Passive InfraRed) blind times, topological issues, such as sensor interference due to the inability to separate detection areas, and algorithmic issues. The house is modeled as a graph to constrain transitions between adjacent rooms. Each room is associated with a set of values, for each identified person. These values decay over time and represent the probability that each person is still in the room. Because the used sensors cannot determine the number of people, the approach is based on a multi-branch inference that, over time, differentiates the movements in the apartment and estimates the number of people. The proposed algorithm has been validated with real data obtaining an accuracy of 86.8%.
... In multi-resident smart homes, HMMs have been studied intensively, as being showed in previous works (Alemdar et al. 2013;Chen and Tong 2014;Singla et al. 2010;Cook 2012). For UDB approach, Conditional Random Field (CRF) and factorial CRF are the most popular for smart home datasets (Crandall and Cook 2008;Hsu et al. 2010;Benmansour et al. 2015;Wang et al. 2011). Also multiple different settings of RNN models including different activation functions (e.g., tanh), different recurrent units (e.g., gated RNN) and so on, are tested in this work. ...
... Viterbi algorithm (Rabiner 1990). For the separate labels, different HMMs have been used such as parallel HMMs, coupled HMMs (Wang et al. 2011;Son et al. 2017). In this paper we use factorial HMM with cross dependency shown in Fig. 1d, as this variant achieves better performance than the other HMMs (Son et al. 2017). ...
Increasing attention to the research on activity monitoring in smart homes has motivated the employment of ambient intelligence to reduce the deployment cost and solve the privacy issue. Several approaches have been proposed for multi-resident activity recognition, however, there still lacks a comprehensive benchmark for future research and practical selection of models. In this paper, we study different methods for multi-resident activity recognition and evaluate them on the same sets of data. In particular, we explore the effectiveness and efficiency of temporal learning algorithms using sequential data and non-temporal learning algorithms using temporally-manipulated features. In the experiments we compare and analyse the results of the studied methods using datasets from three smart homes.
... To deal with the complexities of the multiple residents in the same smart home, researchers have addressed the issue by algorithmic modification on graphical nodes with a probabilistic model. Many of the previous works have been studied on datasets obtained from multi types of sensor technologies i.e. mixed data from ambient sensors and wearable sensors [15] [16]. Meanwhile, using only single type of sensor data i.e. ambient sensors through wireless sensor networks, many works involved modelling activity recognition of multi-resident with probabilistic models. ...
... However, the main disadvantages of ambient sensors are that they need to be well placed in the environment and within an adequate setting. Wearable sensors, on the other hand, can be worn by users in parts of their body or in their clothes [3], [7], [15]. While wearable sensors have the advantages of monitoring and collecting data regardless of the location of the users, some of the main disadvantages are that the users are responsible for their correct use and for charging the battery. ...
... This approach requires keeping a focus on the end users' needs and expectations to maximize the acceptance of the SH and its services [6,7] and aims at providing a universally-usable smart solution -potentially, a SH delivering customized services to residents belonging to different segments of population. The developments in the fields of pervasive computing and residents' activities monitoring allow to recognize different users within the same domestic environments, thus enabling the personalization of services within the same SH [8]. However, this capability comes with a cost: a potential loss of residents' privacy within their domestic environments, as they are continuously monitored by sensor networks, during their interactions with appliances and in the ways they consume energy. ...
... Many solutions exist for HAR in a controlled environment. These solutions mostly involve the deployment of numerous wearable and pervasive sensors [12], which can lead to increased cost, privacy concerns and more often inconvenience. To alleviate these challenges, attention of the research community has directed to low-cost unobtrusive sensors [13]. ...
Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.
... While these systems are efficient and reliable, the use of cameras makes them unsuitable for smart some environments for obvious privacy reasons. Finally, wearables such as RFID readers can be used to identify and track occupants of a Smart Home (e.g. in [16], but they require the user to carry the device with them and are therefore not usable to detect persons who are only visiting and do not carry such a device. ...
Smart home environments equipped with distributed sensor networks are capable of helping people by providing services related to health, emergency detection or daily routine management. A backbone to these systems relies often on the system's ability to track and detect activities performed by the users in their home. Despite the continuous progress in the area of activity recognition in smart homes, many systems make a strong underlying assumption that the number of occupants in the home at any given moment of time is always known. Estimating the number of persons in a Smart Home at each time step remains a challenge nowadays. Indeed, unlike most (crowd) counting solution which are based on computer vision techniques, the sensors considered in a Smart Home are often very simple and do not offer individually a good overview of the situation. The data gathered needs therefore to be fused in order to infer useful information. This paper aims at addressing this challenge and presents a probabilistic approach able to estimate the number of persons in the environment at each time step. This approach works in two steps: first, an estimate of the number of persons present in the environment is done using a Constraint Satisfaction Problem solver, based on the topology of the sensor network and the sensor activation pattern at this time point. Then, a Hidden Markov Model refines this estimate by considering the uncertainty related to the sensors. Using both simulated and real data, our method has been tested and validated on two smart homes of different sizes and configuration and demonstrates the ability to accurately estimate the number of inhabitants.
... Such data bridge the communication gap between the elderly individual and examiner to help identify any interventions required. In this method, data are collected through PIR sensors and surveillance systems which are fully integrated with the living environment of the user as shown in Figure 4. Other sensors for CO 2 , smoke, light, and temperature were installed to control the variables in the ambient living environment (Wang et al., 2011). Table 1 lists the various sensor functionalities used to classify activities of elderly people who live alone at different times (Skubic et al., 2009). ...
... Activity recognition using sensors is typically classified in terms of wearable sensors versus dense sensors. Wearable sensors can be worn by users in parts of their body or in their clothes [3], [5], [15]. Dense sensors are attached to objects in the environment with which the user interacts (e.g. ...
This paper presents a probabilistic approach for the identification of abnormal behaviour in Activities of Daily Living (ADLs) from sensor data collected from 30 participants. The ADLs considered are: (i) preparing and drinking tea, and (ii) preparing and drinking coffee. Abnormal behaviour identified in the context of these activities can be an indicator of a progressive health problem or the occurrence of a hazardous incident. The approach presented considers the temporal aspect of the sequences of actions that are part of each ADL and that vary between participants. The average and standard deviation for the durations of each action were calculated to define an average time and a range in which a behaviour could be considered as normal for each stage and activity. The Cumulative Distribution Function (CDF) was used to obtain the probabilities of abnormal behaviours related to the early and late completion of activities and stages within an activity. The data analysis show that CDF can provide accurate and reliable results regarding the presence of abnormal behaviour in stages and activities that last over a minute. Finally, this approach could be used to train machine learning algorithms for the abnormal behaviour detection.
... The results obtained from the study show that the proposed method can achieve high accuracy, but that forgetting to wear the tag is the main issue with this method that greatly reduces the accuracy. Likewise, Wang et al. (2011), investigated the challenge of recognising multi-occupancy activities utilising wearable sensors in a home. The idea of this research was to study two probabilistic temporal models; the Coupled Hidden Markov Model (CHMM) and factorial conditional random Field (FCRF), to model and classify multi-occupancy activities. ...
Human activity recognition (HAR) is used to support older adults to live independently in their own homes. Once activities of daily living (ADL) are recognised, gathered information will be used to identify abnormalities in comparison with the routine activities. Ambient sensors, including occupancy sensors and door entry sensors, are often used to monitor and identify different activities. Most of the current research in HAR focuses on a single-occupant environment when only one person is monitored, and their activities are categorised. The assumption that home environments are occupied by one person all the time is often not true. It is common for a resident to receive visits from family members or health care workers, representing a multi-occupancy environment. Entropy analysis is an established method for irregularity detection in many applications; however, it has been rarely applied in the context of ADL and HAR. In this paper, a novel method based on different entropy measures, including Shannon Entropy, Permutation Entropy, and Multiscale-Permutation Entropy, is employed to investigate the effectiveness of these entropy measures in identifying visitors in a home environment. This research aims to investigate whether entropy measures can be utilised to identify a visitor in a home environment, solely based on the information collected from motion detectors [e.g., passive infra-red] and door entry sensors. The entropy measures are tested and evaluated based on a dataset gathered from a real home environment. Experimental results are presented to show the effectiveness of entropy measures to identify visitors and the time of their visits without the need for employing extra wearable sensors to tag the visitors. The results obtained from the experiments show that the proposed entropy measures could be used to detect and identify a visitor in a home environment with a high degree of accuracy.
... Such data bridge the communication gap between the elderly individual and examiner to help identify any interventions required. In this method, data are collected through PIR sensors and surveillance systems which are fully integrated with the living environment of the user as shown in Figure 4. Other sensors for CO 2 , smoke, light, and temperature were installed to control the variables in the ambient living environment (Wang et al., 2011). Table 1 lists the various sensor functionalities used to classify activities of elderly people who live alone at different times (Skubic et al., 2009). ...
... Dans la partie suivante, certaines techniques statistiques sont examinées. [170] ont été utilisés dans plusieurs fonctionnalités, y compris la reconnaissance des activités quotidiennes [166,171,172], la détection des anomalies [84,169,170] et la prédiction du ...
Les progrès de la technologie des capteurs et leur disponibilité ont permis de mesurerdiverses propriétés et activités des habitants dans une maison intelligente.Cependant, l’obtention de connaissances significatives à partir d’une grande quantitéd’informations collectées à partir d’un réseau de capteurs n’est pas une tâchesimple. En raison de la complexité du comportement des habitants, l’extraction d’informationssignificatives et la prédiction précise des valeurs représentant les activitésfutures d’un occupant sont des défis de recherche [5].L’objectif principal de notre travail de thèse est d’assurer une analyse efficace desdonnées recueillies à partir des capteurs d’occupation dans une maison intelligente.Cette recherche tente de trouver une solution efficace pour surveiller les personnesâgées vivant d’une façon autonome dans leur propre maison. Par conséquent, cetravail se base sur la reconnaissance et l’évaluation des activités quotidiennes d’unepersonne âgée dans le but d’observer, de prédire et de suivre l’évolution de son étatde dépendance, de santé et de détecter par la même occasion, la présence d’une perteou d’une perturbation de l’autonomie en temps réel.Afin d’atteindre l’objectif principal de cette recherche, les objectifs suivants sontidentifiés :— Étudier différentes méthodes pour présenter et extraire l’énorme ensembledes données hétérogènes (bas niveau) détectées par les capteurs pour lesadapter dans un format approprié, lisible (haut niveau) pour reconnaitre etprédire le comportement de la personne.— Suivre l’état de santé de l’habitant via son comportement quotidien et selonsa routine.— Étudier les moyens appropriés d’exploration et de prédiction des comportementsdans la maison intelligente pour extraire le modèle comportementalde la personne.— Proposer un modèle de reconnaissance et de prédiction des activités quotidiennesadaptables à la personne, performant de point de vue de la précisionet de la rapidité.— Comparer les performances des différentes techniques de prédiction (les modèlesproposés) pour évaluer la technique la plus appropriée pour les donnéescollectées à partir d’un habitat intelligent.— Examiner les différentes techniques de détection pour évaluer les informationsimportantes concernant les valeurs aberrantes et tout comportementanormal.— Évaluer l’état de la santé de la personne à partir de son comportementquotidien, son profil et ses habitudes.
... The main motivations for measuring KPIs are to cut down maintenance costs, repair costs and to reduce failures and unscheduled maintenance [13], [14]. Activity recognition using wearable sensors [15], [16] is a popular research domain in areas such as healthcare [17], defence [18], smart home [19], and sports [20]. However, such solutions are still uncommon in manufacturing industries. ...
... Activity recognition using sensors is typically classified in terms of wearable sensors versus dense sensors. Wearable sensors can be worn by users in parts of their body or in their clothes [3], [7], [18]. Dense sensors are attached to objects in the environment with which the user interacts (e.g. ...
This paper presents a probabilistic approach for the identification of abnormal behaviour in Activities of Daily Living (ADLs) from dense sensor data collected from 30 participants. The ADLs considered are related to preparing and drinking (i) tea, and (ii) coffee. Abnormal behaviour identified in the context of these activities can be an indicator of a progressive health problem or the occurrence of a hazardous incident. The approach presented considers the temporal and sequential aspects of the actions that are part of each ADL and that vary between participants. The average and standard deviation for the duration and number of steps of each activity are calculated to define the average time and steps and a range within which a behaviour could be considered as normal for each stage and activity. The Cumulative Distribution Function (CDF) is used to obtain the probabilities of abnormal behaviours related to the early and late completion of activities and stages within an activity in terms of time and steps. Analysis shows that CDF can provide precise and reliable results regarding the presence of abnormal behaviour in stages and activities that last over a minute or consist of many steps. Finally, this approach could be used to train machine learning algorithms for abnormal behaviour detection.
... The autoregressive coefficient is the coefficient found by the Burg method, which conforms to the autoregressive model of inputs. This operation is applied to the signals in the time domain, and produces outputs corresponding to the four features of the algorithm sequence [31]. Considering that each point is proportional to its amplitude, the weighted average of the signals gives the average frequency of the signals. ...
The purpose of activity recognition is to identify activities through a series of observations of the experimenter’s behavior and the environmental conditions. In this study, through feature selection algorithms, we researched the effects of a large number of features on human activity recognition (HAR) assisted by an inertial measurement unit (IMU), and applied them to smartphones of the future. In the research process, we considered 585 features (calculated from tri-axial accelerometer and tri-axial gyroscope data). We comprehensively analyzed the features of signals and classification methods. Three feature selection algorithms were considered, and the combination effect between the features was used to select a feature set with a significant effect on the classification of the activity, which reduced the complexity of the classifier and improved the classification accuracy. We used five classification methods (support vector machine [SVM], decision tree, linear regression, Gaussian process, and threshold selection) to verify the classification accuracy. The activity recognition method we proposed could recognize six basic activities (BAs) (standing, going upstairs, going downstairs, walking, lying, and sitting) and postural transitions (PTs) (stand-to-sit, sit-to-stand, stand-to-lie, lie-to-stand, sit-to-lie, and lie-to-sit), with an average accuracy of 96.4%.
... For example, there are many systems in which radio waves are used. These include activity and gesture recognition systems using Wi-Fi [26][27][28] or radio frequency identification (RFID) systems [29][30][31][32] and a healthcare reporting generation support system that is designed for nursing homes that uses Bluetooth low energy (BLE) components [33,34]. However, because these systems are directly affected by radio wave reflections, they need to be reconfigured every time the environment or device location changes. ...
As aging populations continue to grow, primarily in developed countries, there are increasing demands for the system that monitors the activities of elderly people while continuing to allow them to pursue their individual, healthy, and independent lifestyles. Therefore, it is required to develop the activity of daily living (ADL) sensing systems that are based on high-performance sensors and information technologies. However, most of the systems that have been proposed to date have only been investigated and/or evaluated in experimental environments. When considering the spread of such systems to typical homes inhabited by elderly people, it is clear that such sensing systems will need to meet the following five requirements: (1) be inexpensive; (2) provide robustness; (3) protect privacy; (4) be maintenance-free; and, (5) work with a simple user interface. In this paper, we propose a novel senior-friendly ADL sensing system that can fulfill these requirements. More specifically, we achieve an easy collection of ADL data from elderly people while using a proposed system that consists of a small number of inexpensive energy harvesting sensors and simple annotation buttons, without the need for privacy-invasive cameras or microphones. In order to evaluate the practicality of our proposed system, we installed it in ten typical homes with elderly residents and collected the ADL data over a two-month period. We then visualized the collected data and performed activity recognition using a long short-term memory (LSTM) model. From the collected results, we confirmed that our proposed system, which is inexpensive and non-invasive, can correctly collect resident ADL data and could recognize activities from the collected data with a high recall rate of 72.3% on average. This result shows a high potential of our proposed system for application to services for elderly people.
... Over the years, many researchers have tried to estimate daily life human activities, such as walking and sitting up and down from acceleration and/or gyro sensor data obtained from wearable devices and/or smartphones. In this paper, we refer to the research done respectively by Zhan et al. and Wang et al. [12] [13]. Only a few motions were given in this research; distinctions among activities were not recognized. ...
A system is needed for quantitatively evaluating the activity recovery level of functional disable people. Although functional recovery is administered to hemiplegic patients during rehabilitation, some patients who have recovered function in a rehabilitation facility are still unable to perform daily activities at home. Therefore, recovering activities of daily living (ADL) has become more important than functional recovery. Since existing ADL recovery level indices are based on responses to questionnaires, judgment of recovery level is easily affected by an evaluator's subject. We have developed a system for collecting and storing motion data on daily life activities for use in quantitatively evaluating ADL recovery levels. Evaluation of the system using data measured for a healthy participant with restricted movement and two actual hemiplegic patients demonstrated that slight differences in disability levels can be detected. This system is thus well suited for quantitative ADL assessment for patients with a disability.
... A wide variety of sensors have been utilized to capture relevant contextual factors in smart homes, including (but not limited to) those that measure ambient temperature, record sound, motion, wifi signal characteristics, visual information (through, for example, cameras), and proximity information [2,7,61]. Furthermore, body worn sensors are also widely used for direct recording of an occupant's movements [60]. In the context of the smart home domain, activity recognition specifically serves to identify and log the occupant's activities of daily living (ADL) from temporal-ordered sensor data. ...
Smart home environments are designed to provide services that help improve the quality of life for the occupant via a variety of sensors and actuators installed throughout the space. Many automated actions taken by a smart home are governed by the output of an underlying activity recognition system. However, activity recognition systems may not be perfectly accurate and therefore inconsistencies in smart home operations can lead a user to wonder "why did the smart home do that?" In this work, we build on insights from Explainable Artificial Intelligence (XAI) techniques to contribute computational methods for explainable activity recognition. Specifically, we generate explanations for smart home activity recognition systems that explain what about an activity led to the given classification. To do so, we introduce four computational techniques for generating natural language explanations of smart home data and compare their effectiveness at generating meaningful explanations. Through a study with everyday users, we evaluate user preferences towards the four explanation types. Our results show that the leading approach, SHAP, has a 92% success rate in generating accurate explanations. Moreover, 84% of sampled scenarios users preferred natural language explanations over a simple activity label, underscoring the need for explainable activity recognition systems. Finally, we show that explanations generated by some XAI methods can lead users to lose confidence in the accuracy of the underlying activity recognition model, while others lead users to gain confidence. Taking all studied factors into consideration, we make a recommendation regarding which existing XAI method leads to the best performance in the domain of smart home automation, and discuss a range of topics for future work in this area.
... MICAR falls in this category. In [49], the residents wear a specifically designed RFID wristband reader in charge of detecting the interaction with tagged objects. However, the use of sophisticated wearable devices in realistic deployments is questionable, especially considering elderly subjects that should constantly wear them. ...
The sensor-based recognition of Activities of Daily Living (ADLs) in smart-home environments enables several important applications, including the continuous monitoring of fragile subjects in their homes for healthcare systems. The majority of the approaches in the literature assume that only one resident is living in the home. Multi-inhabitant ADLs recognition is significantly more challenging, and only a limited effort has been devoted to address this setting by the research community. One of the major open problems is called data association , which is correctly associating each environmental sensor event (e.g., the opening of a fridge door) with the inhabitant that actually triggered it. Moreover, existing multi-inhabitant approaches rely on supervised learning, assuming a high availability of labeled data. However, collecting a comprehensive training set of ADLs (especially in multiple-residents settings) is prohibitive. In this work, we propose MICAR: a novel multi-inhabitant ADLs recognition approach that combines semi-supervised learning and knowledge-based reasoning. Data association is performed by semantic reasoning, combining high-level context information (e.g., residents’ postures and semantic locations) with triggered sensor events. The personalized stream of sensor events is processed by an incremental classifier, that is initialized with a limited amount of labeled ADLs. A novel cache-based active learning strategy is adopted to continuously improve the classifier. Our results on a dataset where up to 4 subjects perform ADLs at the same time show that MICAR reliably recognizes individual and joint activities while triggering a significantly low number of active learning queries.
... The proposed method minimizes the amount of features required to detect and recognize movement [232]. In addition, a multiuser movement detection platform was proposed by Wang et al. [233]. The proposed platform used Coupled HMM and Factorial Conditional Random Field to model sensors' data transformation and communication. ...
Applications of machine learning (ML) methods have been used extensively to solve various complex challenges in recent years in various application areas, such as medical, financial, environmental, marketing, security, and industrial applications. ML methods are characterized by their ability to examine many data and discover exciting relationships, provide interpretation, and identify patterns. ML can help enhance the reliability, performance, predictability, and accuracy of diagnostic systems for many diseases. This survey provides a comprehensive review of the use of ML in the medical field highlighting standard technologies and how they affect medical diagnosis. Five major medical applications are deeply discussed, focusing on adapting the ML models to solve the problems in cancer, medical chemistry, brain, medical imaging, and wearable sensors. Finally, this survey provides valuable references and guidance for researchers, practitioners, and decision-makers framing future research and development directions.
... Indeed, SVM training requires solving a quadratic programming (QP) problem in a number of coefficients equal to the number of training examples, which, in turn, makes standard numerical techniques for QP infeasible for a such large dataset. To overcome this difficulty, some practical techniques decompose the problem into manageable sub-problems over part of the data Lester et al. (2005) or, perform component-wise optimization (Kim et al. 2013), or, to some extent, iterative pairwise comparison (Wang et al. 2011). Other researchers suggested to transform the batch SVMs to the incremental ones by adapting an incremental or online learning techniques (Cauwenberghs et al. 2001;Syed et al. 1999). ...
Smart homes are equipped with several sensor networks to keep an eye on both residents and their environment, to interpret the current situation and to react immediately. Handling large scale dataset of sensory events on real time to enable efficient interventions is challenging and very difficult. To deal with these data flows and challenges, traditional streaming data classification approaches can be boosted by use of incremental learning. In this paper, we presented two new Incremental SVM methods to improve the performance of SVM classification in the context of human activity recognition tasks. Two feature extraction methods elaborated by refining dependency sensor extraction feature and focusing on the last sensor event only have been suggested. On the other hand, a clustering based approach and a similarity based approach have been suggested to boost learning performance of the incremental SVM algorithms capitalizing on the relationship between data chunk and support vectors of previous chunk. We demonstrate through several simulations on two major publicly available data sets (Aruba and Tulum), the feasibility and improvements in learning and classification performances in real time achieved by our proposed methods over the state-of-the-art. For instance, we have shown that the introduced similarity-based incremental learning is 5 to 9 times faster than other methods in terms of training performances. Similarly, the introduced Last-state sensor feature method induces at least 5% improvement in terms of F1-score when using baseline SVM classifier.
... These methods may not be applicable for detecting multi-person activity in real-world scenarios. Previous research has shown that the difficulty of identifying human-human interactions (HHIs), which involve multiple interacting people (e.g., high five and pushing interactions), is more challenging than identifying single-human activities (e.g., running and sitting activities) [16]. A three-layer CNN [17] is proposed, which employs publicly available CSI data [18], converting it into a 2D grayscale image to recognize the HHIs. ...
In recent years, Wi-Fi infrastructures have become ubiquitous, providing device-free passive-sensing features. Wi-Fi signals can be affected by their reflection, refraction, and absorption by moving objects in their path. The channel state information (CSI), a signal property indicator, of the Wi-Fi signal can be analyzed for human activity recognition (HAR). Deep learning-based HAR models can enhance performance and accuracy without sacrificing computational efficiency. However, to save computational power, an inception network, which uses a variety of techniques to boost speed and accuracy, can be adopted. In contrast, the concept of spatial attention can be applied to obtain refined features. In this paper, we propose a human–human interaction (HHI) classifier, CSI-IANet, which uses a modified inception CNN with a spatial-attention mechanism. The CSI-IANet consists of three steps: i) data processing, ii) feature extraction, and iii) recognition. The data processing layer first uses the second-order Butterworth low-pass filter to denoise the CSI signal and then segment it before feeding it to the model. The feature extraction layer uses a multilayer modified inception CNN with an attention mechanism that uses spatial attention in an intense structure to extract features from captured CSI signals. Finally, the refined features are exploited by the recognition section to determine HHIs correctly. To validate the performance of the proposed CSI-IANet, a publicly available HHI CSI dataset with a total of 4800 trials of 12 interactions was used. The performance of the proposed model was compared to those of existing state-of-the-art methods. The experimental results show that CSI-IANet achieved an average accuracy of 91.30%, which is better than that of the existing best method by 5%.
... (307) dos(as) alunos(as) avaliaram as aulas como "muito bom"; 21% (83), como "bom"; 2% (10), como "regular"; e apenas 1% (4), como "ruim" ou "muito ruim". (BELLE et al., 2015;BISSONNETTE;BERGERON, 2017;GATZOULIS;IAKOVIDIS, 2007;GROSSGLAUSER;SANER, 2014;HANDEL, 2011;MARTIN, 2012;NICHOLAS et al., 2015;SOH et al., 2013;SRIVASTAVA et al., 2015;WEST et al., 2012). ...
Massive Open Online Courses (MOOCs) são cursos abertos e massivos ofertados em ambientes virtuais de aprendizagem, que permitem o compartilhamento de experiências, conhecimentos e informações. Este capítulo objetiva realizar um mapeamento da produção de MOOCs na saúde no Brasil, nas dissertações e teses e das iniciativas governamentais na área da saúde. Trata-se de um estudo qualitativo, com base na análise documental dos sites dos Ministérios da Saúde e da Educação brasileiros, do MOOC-List.com e da Biblioteca Digital Brasileira de Teses e Dissertações (BDTD). Embora não use a nomenclatura MOOC, o Brasil investe em iniciativas pedagógicas massivas, on-line e abertas. Os resultados indicam que a produção acadêmica sobre MOOCs na saúde é incipiente. Há um potencial a ser explorado, que inclui a educação permanente e continuada dos profissionais de saúde e a troca de experiências e conhecimentos entre pesquisadores, profissionais e usuários do sistema de saúde brasileiro.
Wearable systems have unlocked new sensing paradigms in various applications such as human activity recognition, which can enhance effectiveness of mobile health applications. Current systems using wearables are not capable of understanding their surroundings, which limits their sensing capabilities. For instance, distinguishing certain activities such as attending a meeting or class, which have similar motion patterns but happen in different contexts, is challenging by merely using wearable motion sensors. This article focuses on understanding user's surroundings, i.e., environmental context, to enhance capability of wearables, with focus on detecting complex activities of daily living (ADL). We develop a methodology to automatically detect the context using passively observable information broadcasted by devices in users’ locale. This system does not require specific infrastructure or additional hardware. We develop a pattern extraction algorithm and probabilistic mapping between the context and activities to reduce the set of probable outcomes. The proposed system contains a general ADL classifier working with motion sensors, learns personalized context, and uses that to reduce the search space of activities to those that occur within a certain context. We collected real-world data of complex ADLs and by narrowing the search space with context, we improve average F1-score from 0.72 to 0.80.
During the past years, there has been rapid development in Ambient Assisted Living technologies and Smart Home solutions for the growing aging population. A smart home system consists of many different components that interact with each other and can provide services that are context-aware and personalized. In order to develop a flexible, scalable, and acceptable smart home solution, certain improvements are required in different areas such as meeting end-user requirements, activity recognition, privacy aspect, and interaction with the system. In this thesis, we explored different ways to further improve the performances of the major components of a smart home system that include activity recognition, privacy-preserving, and dialogue systems. The research in this thesis has been conducted in five main areas. First, we proposed a privacy-enabled smart home framework in an open-source software platform. Next, we perform different user studies and interviews with the older people and end-users to understand their needs better, their perceptions and attitudes towards smart home applications such as activity monitoring, data sharing, privacy notions, and views regarding voice assistants. Third, we investigated deep learning algorithms on smart home sensor datasets for both single and multiple resident activity recognition. Using deep learning models, we achieved promising results on raw sensor datasets, in comparison to traditional machine learning methods. Furthermore, we investigated various class imbalance techniques on smart home datasets to address the class imbalance problem. Fourth, we proposed and implemented a privacy-preserving mechanism for data transformation and anonymization that allows sharing of encoded data instead of original data with third parties and thus, protects users’ sensitive information. In the last part of this thesis, we provided an overview of the dialogue system and presented a new human-annotated dialogue dataset with benchmark metrics using deep learning models to evaluate the quality of dialogue replies for the given context. Overall, the thesis addresses the challenges in existing research and components of smart home systems and the results obtained will potentially help developers and service providers in designing practical and adaptable smart home solutions.
We develop CACE (Constraints And Correlations mining Engine), a framework that significantly improves the recognition accuracy of complex daily activities in multi-inhabitant smarthomes. CACE views the implicit relationships between the activities of multiple people as an asset, and exploits such constraints and correlations in a hierarchical fashion, taking advantage of both person-specific sensor data (generated by wearable devices) and person-independent ambient sensor data (generated by ambient sensors). To effectively utilize such couplings, CACE first uses a multi-target particle filtering approach over ambient sensors captured movement data, to identify the number of distinct users and infer individual-specific movement trajectories. We then utilize a Hierarchical Dynamic Bayesian Network (HDBN)-based model for activity recognition. This model utilizes the inter-and-intra individual correlations and constraints, at both micro-activity and macro-activity levels, to recognize individual activities accurately. These constraints are learnt automatically using data-mining techniques, and help to dramatically reduce the computational complexity of HDBN-based inferencing. Empirical studies using a real-world testbed of 5 multi-inhabitant smarthomes shows that CACE is able to achieve an activity recognition accuracy of ≍95%, with a 16-fold reduction in computational overhead compared to traditional hybrid classification approaches.
The rapid development of flexible and stretchable materials has portrayed a bright future of flexible electronics in pursuing the concept of “Internet of Things.” Transistors, as one of the fundamental units in electronic devices, are being widely studied in exploring the possibility of increasing their flexibility and/or stretchability to fulfill the requirement in flexible electronics, from basic logic circuits to intelligent devices. However, the mechanical stimuli induced by bending and stretching activities can lead to unexpected performance degradations. Here, an in‐depth analysis of approaches to modulating the sensitivity of flexible and stretchable transistors, from a mechanically sensitive type to an insensitive type is demonstrated. Starting with the structural design, the mechanism of sensitivity modulation is explicitly analyzed, such as effects based on the modulation of dielectric capacitance, conductance of active channels, or contact resistance at interfaces in sensitive transistors and strain patterning technology applied to an insensitive type. The sensitivity modulation for transistors is further analyzed regarding material utilization based on different dimensional forms. Finally, the potential applications where flexible and stretchable transistors can be employed as an outlook are discussed. Deformation‐induced strain has become an inevitable problem for flexible and stretchable transistors and hinders the stable performance of flexible electronics in applications. A series of methods to optimize transistors’ performance under different strains have been proposed. This review on the state‐of‐the‐art approaches is presented to manipulate the strains inside transistors, from delicate structural designs to rational material utilizations.
Activity recognition is a fundamental way to support context-aware services for users in smart spaces. Data sources such as video or wearable devices are used in many recognition approaches, but there are challenges in utilizing them in the real world. Recent approaches propose deep learning-based methods on IoT sensor data streams to overcome the issues. Since they only describe single user-based spaces, they are vulnerable to complex sequences of events triggered by multiple users. When multiple users exist in a space, various overlapping events occur with longer correlations than a single user situation. Additionally, ambient sensor-based events appear far more than actuator-based events, making it difficult to extract actuator-based events as important features. We propose a transformer-based approach to derive long-term event correlations and important events as elements of activity patterns. We also develop a duration incorporated embedding method to differentiate between the same type but different duration events and add a sequential manner to the transformer approach. In the experiments section, we prove that our approach outperforms the existing approaches based on real datasets.
The smart home is one application of intelligent environments, where sensors are equipped to detect the status inside the domestic home. With the development of sensing technologies, more signals can be obtained with heterogenous statistical properties with faster processing speed. To make good use of the technical advantages, data-driven methods are becoming popular in intelligent environments. On the other hand, to recognize human activity is one essential target to understand the status inside a smart home. In this chapter, the authors focus on the human activity recognition (HAR) problem, which is the recognition of lower levels of activities, using data-driven models.
This work investigates the origins of electrical performance degradation under uniaxial stretching of a silver filled polyurethane ink (DuPont PE 874) screen printed onto a thermoplastic polyurethane substrate. The ink develops surface ruptures at strains of only a few percent yet remains conductive through continued elongation. We identify increasing sensitivity to surface damage beyond 10% applied strain, ɛapp, as the trace width, w, is reduced from 2 to 0.1 mm. This lowers the threshold strain for open circuit failure, from approximately 180% for w = 2 mm down to 25% for w = 0.1 mm. The damage progression remains largely consistent across trace widths: surface cracks coalesce to form longer channels, which grow perpendicular to the direction of elongation. These channels both deepen and widen with increasing ɛapp and some become laterally linked. The evolution of the network of interlinked channels is not width dependent, but a width effect manifests as a result of the channels constituting a larger fraction of specimen width for narrower traces. In addition, the narrower traces exhibit reduced cross sections due to an edge taper—an artifact of the screen printing process—which attenuates ink thickness by as much as 50% for w = 0.1 mm.
Smart toilet provides a feasible platform for the long-term analysis of person’s health. Common solutions for identification are based on camera or radio-frequency identification (RFID) technologies, but it is doubted for privacy issues. Here, we demonstrate an artificial intelligence of toilet (AI-toilet) based on a triboelectric pressure sensor array offering a more private approach with low cost and easily deployable software. The pressure sensor array attached on the toilet seat is composed of 10 textile-based triboelectric sensors, which can leverage the different pressure distribution of individual users' seating manner to get the biometric information. 6 users can be correctly identified with more than 90% accuracy using deep learning. The signals from pressure sensors also can be used for recording the seating time on the toilet. The system integrates a camera sensor to analyze the simulated urine by comparing with urine chart and classify the types and quantities of objects using deep learning. All information including two-factor user identification and entire seating time using pressure sensor array, and data from the urinalysis and stool analysis were automatically transferred to a cloud system and were further shown in user's mobile devices for better tracking their health status.
Emerging flexible artificial sensory systems using neuromorphic electronics have been considered as a promising solution for processing massive data with low power consumption. The construction of artificial sensory systems with synaptic devices and sensing elements to mimic complicated sensing and processing in biological systems is a prerequisite for the realization. To realize high-efficiency neuromorphic sensory systems, the development of artificial flexible synapses with low power consumption and high-density integration is essential. Furthermore, the realization of efficient coupling between the sensing element and the synaptic device is crucial. This Review presents recent progress in the area of neuromorphic electronics for flexible artificial sensory systems. We focus on both the recent advances of artificial synapses, including device structures, mechanisms, and functions, and the design of intelligent, flexible perception systems based on synaptic devices. Additionally, key challenges and opportunities related to flexible artificial perception systems are examined, and potential solutions and suggestions are provided.
Telecommunication had witnessed a fulfilling
and successful growth in terms of design and technology in
the last few years. The use of Copper wire in cable
communication systems is more or less inept and
undependable. The introduction of fibre optics makes
copper wire out of date. Fibre optics is a cabling technology
used in communicating information in the form of signal
through pulse light of an optic from one place to another.
It is shown that undersea cabling system has increased
rather than diminished in importance as radio
communication has grown up in the last 25 years or so.
Great and progressive changes can be predicted for the next
decade following the extended use of synthetic insulating
materials and the submerged repeater. These modern
developments are described in some detail, and their present
effect and possibilities for the future on communication are
considered.
Furthermore, undersea cable system uses the latest
technology Wavelength Division Multiplexing (WDM) which
multiplexes the number of signals in one single optical fibre.
WDM permits a bi-directional links in one single fibre
optics. This paper will be discussing on the technology and
design of fibre optics, challenges and importance of an
undersea cabling system, also with recommendation.
As the adoption rate of commercial smart home solutions increases, it drives the development of novel system features, needed to support advanced user scenarios. Being able to remotely control the household is not enough already, and the efforts are made to provide context-aware and intelligent homes, which detect user activities, learn about user habits, adapt to particular users, create intelligent alarms, seamlessly integrate with remote services, etc. On the other hand, significant efforts have already been made by the research community to identify data mining scenarios applicable to smart home solutions, and to propose and improve the algorithms suitable for this purpose. This article summarizes state-of-the-art research in the field of machine learning for smart home solutions.
The rise of the smart building has promoted various pervasive computing technologies to be used in the intelligent building system. People tend to be in groups inside a building, working together, taking classes, having meetings, etc. Being able to recognize such group activities will be key to making a functionalized building an activity-aware smart system. In that way, the system can adjust the surrounding atmosphere automatically according to the detected group activity and adapt to the needs of individuals or groups. Exiting works on group activity recognition (GAR) mainly focus on computer vision through surveillance hardware, which suffers from privacy and illumination problems. We decided to use the smartphone to identify GAR, considering it’s pervasive and ubiquitous properties as well as various built-in sensors such as an accelerometer, gyroscope, microphone, etc. In this paper, we first conduct an extensive literature review relating to GAR in smart buildings. Our goal is to recognize fine-grained group activity by utilizing coarse-grained smartphone sensors.
Recent growing interest in ambient intelligent environments has driven a desire for effective models to reason about activities of multiple residents. Such models are the keystone for the future of smart homes where occupants can be assisted with non-intrusive technologies. Much attention has been put on this research, however current works tend to focus on developing statistical algorithms for prediction, whilst there still lacks a study to fully understand the relations of residents’ behaviours and how they are reflected through the sensors’ states. In this paper we investigate the dependencies of the activities from residents and their interaction with the environments. We represent such dependencies in Bayesian networks that leads to construction of six variants of Hidden Markov Models (HMMs). Furthermore, we argue that a complete model should embody more than one type of dependency. Therefore, we propose an ensemble of HMMs, and then generalize it to a novel mixed-dependency model. In the experiments we perform intensive evaluation of our study on multi-resident activity recognition task. The results show that the proposed models outperform other models in three smart home environments, thus asserting our hypothesis.
Human activity plays a significant role in various fields, such as manufacturing, healthcare, and public safety; therefore, recognizing human activity is crucial to enable smart innovative services. The development of ubiquitous sensing and pervasive computing allows studying what humans perform in real time and mobility. Single- and multi-user activity recognition (AR) differ by the number of involved users. With recent developments of multi-sensor and multi-information fusion, multi-user activity recognition is gradually becoming an emerging and relevant research frontier. In this paper, we propose a software architecture which combines cloud and edge computing with collaborative body sensor networks (CBSNs) to support the development of CBSNs-enabled services and in particular we provide its case-study in the context of multi-user AR.
We address the problem of recognizing sequences of human interaction patterns in meetings, with the goal of structuring them in semantic terms. The investigated patterns are inherently group-based (defined by the individual activities of meeting participants, and their interplay), and multimodal (as captured by cameras and microphones). By defining a proper set of individual actions, group actions can be modeled as a two-layer process, one that models basic individual activities from low-level audio-visual features, and another one that models the interactions. We propose a two-layer Hidden Markov Model (HMM) framework that implements such concept in a principled manner, and that has advantages over previous works. First, by decomposing the problem hierarchically, learning is performed on low-dimensional observation spaces, which results in simpler models. Second, our framework is easier to interpret, as both individual and group actions have a clear meaning, and thus easier to improve. Third, different HMM models can be used in each layer, to better reflect the nature of each subproblem. Our framework is general and extensible, and we illustrate it with a set of eight group actions, using a public five-hour meeting corpus. Experiments and comparison with a single-layer HMM baseline system show its validity.
Recognizing patterns of human activities is an important enabling technology for building intelligent home environ- ments. Existing approaches to activity recognition often fo- cus on mutually exclusive activities only. In reality, peo- ple routinely carry out multiple concurrent activities. It is therefore necessary to model the co-temporal relationships among activities. In this paper, we propose using Factorial Conditional Random Fields (FCRFs) for recognition of mul- tiple concurrent activities. We designed experiments to com- pare our FCRFs model with Linear Chain Condition Random Fields (LCRFs) in learning and performing inference with the MIT House n data set, which contains annotated data col- lected from multiple sensors in a real living environment. The experimental results show that FCRFs can effectively im- prove the F-score in activity recognition for up to 8% in the presence of multiple concurrent activities.
Understanding and recognizing human activities from sensor readings is an important task in pervasive computing. Existing work on activity recognition mainly focuses on recognizing activities for a single user in a smart home environment. However, in real life, there are often multiple inhabitants live in such an environment. Recognizing activities of not only a single user, but also multiple users is essential to the development of practical context-aware applications in pervasive computing. In this paper, we investigate the fundamental problem of recognizing activities for multiple users from sensor readings in a home environment, and propose a novel pattern mining approach to recognize both single-user and multi-user activities in a unified solution. We exploit emerging pattern -a type of knowledge pattern that describes significant changes between classes of data - for constructing our activity models, and propose an emerging pattern based multi-user activity recognizer (epMAR) to recognize both single-user and multiuser activities. We conduct our empirical studies by collecting real-world activity traces done by two volunteers over a period of two weeks in a smart home environment, and analyze the performance in detail with respect to various activity cases in a multi-user scenario. Our experimental results demonstrate that our epMAR recognizer achieves an average accuracy of 89.72% for all the activity cases.
In this paper we consider a class of human activities—atomic activities—which can be represented as a set of measurements over a finite temporal window (e.g., the motion of human body parts during a walking cycle) and which has a relatively small space of variations in performance. A new approach for modeling and recognition of atomic activities that employs principal component analysis and analytical global transformations is proposed. The modeling of sets of exemplar instances of activities that are similar in duration and involve similar body part motions is achieved by parameterizing their representation using principal component analysis. The recognition of variants of modeled activities is achieved by searching the space of admissible parameterized transformations that these activities can undergo. This formulation iteratively refines the recognition of the class to which the observed activity belongs and the transformation parameters that relate it to the model in its class. We provide several experiments on recognition of articulated and deformable human motions from image motion parameters.
Spatial scaffolding is a naturally occurring human teaching behavior, in which teachers use their bodies to spatially structure the learning environment to direct the attention of the learner. Robotic systems can take advantage of simple, highly reliable ...
A sensor system capable of automatically recognizing activities would allow many potential ubiquitous applications. In this paper, we present an easy to install sensor network and an accurate but inexpensive annotation method. A recorded dataset consisting of 28 days of sensor data and its annotation is described and made available to the community. Through a number of experiments we show how the hidden Markov model and conditional random fields perform in recognizing activities. We achieve a timeslice accuracy of 95.6% and a class accuracy of 79.4%.
We study activity recognition using 104 hours of annotated data collected from a person living in an instrumented home. The home contained over 900 sensor inputs, including wired reed switches, current and water flow inputs, object and person motion detectors, and RFID tags. Our aim was to compare dierent sensor modalities on data that approached "real world" conditions, where the subject and annotator were unaliated with the authors. We found that 10 infra-red motion detectors outperformed the other sensors on many of the activities stud- ied, especially those that were typically performed in the same location. However, several activities, in particular "eating" and "reading" were dicult to detect, and we lacked data to study many fine-grained activi- ties. We characterize a number of issues important for designing activity detection systems that may not have been as evident in prior work when data was collected under more controlled conditions.
We describe a real-time computer vision and machine learning system for modeling and recognizing human behaviors in a visual surveillance task. The system is particularly concerned with detecting when interactions between people occur, and classifying the type of interaction. Examples of interesting interaction behaviors include following another person, altering one's path to meet another, and so forth. Our system combines top-down with bottom-up information in a closed feedback loop, with both components employing a statistical Bayesian approach. We propose and compare two different state-based learning architectures, namely HMMs and CHMMs, for modeling behaviors and interactions. The CHMM model is shown to work much more efficiently and accurately.
Finally, to deal with the problem of limited training data, a synthetic ‘Alife-style’ training system is used to develop flexible prior models for recognizing human interactions. We demonstrate the ability to use these a priori models to accurately classify real human behaviors and interactions with no additional tuning or training.
With the convergence of technologies in artificial intelligence, human- computer interfaces, and pervasive computing, the idea of a "smart environment" is becoming a reality. While we all would like the benefits of an environment that automates many of our daily tasks, a smart environment that makes the wrong decisions can quickly becoming annoying. In this paper, we describe a simulation tool that can be used to visualize activity data in a smart home, play through proposed automation schemes, and ultimately provide guidance to automating the smart environment. We describe how automation policies can adapt to resident feedback, and demonstrate the ideas in the context of the MavHome smart home.
Accurate recognition and tracking of human activities is an important goal of ubiquitous computing. Recent advances in the development of multi-modal wearable sensors enable us to gather rich datasets of human activities. However, the problem of automatically identifying the most useful features for modeling such activities remains largely unsolved. In this paper we present a hybrid approach to recognizing activities, which combines boosting to discriminatively select useful features and learn an ensemble of static classifiers to recognize different activities, with hidden Markov models (HMMs) to capture the temporal regularities and smoothness of activities. We tested the activity recognition system using over 12 hours of wearable-sensor data collected by volunteers in natural unconstrained environments. The models succeeded in identifying a small set of maximally informative features, and were able identify ten different human activities with an accuracy of 95%.
The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. A primary challenge that needs to be tackled to meet this need is the ability to recognize and track functional activities that people perform in their own homes and everyday settings. In this paper, we look at approaches to perform real-time recognition of Activities of Daily Living. We enhance other related research efforts to develop approaches that are effective when activities are interrupted and interleaved. To evaluate the accuracy of our recognition algorithms we assess them using real data collected from participants performing activities in our on-campus smart apartment testbed.
In order to provide relevant information to mobile users, such as workers engaging in the manual tasks of maintenance and assembly, a wearable computer requires information about the user's specific activities. This work focuses on the recognition of activities that are characterized by a hand motion and an accompanying sound. Suitable activities can be found in assembly and maintenance work. Here, we provide an initial exploration into the problem domain of continuous activity recognition using on-body sensing. We use a mock "wood workshop" assembly task to ground our investigation. We describe a method for the continuous recognition of activities (sawing, hammering, filing, drilling, grinding, sanding, opening a drawer, tightening a vise, and turning a screwdriver) using microphones and three-axis accelerometers mounted at two positions on the user's arms. Potentially "interesting" activities are segmented from continuous streams of data using an analysis of the sound intensity detected at the two different locations. Activity classification is then performed on these detected segments using linear discriminant analysis (LDA) on the sound channel and hidden Markov models (HMMs) on the acceleration data. Four different methods at classifier fusion are compared for improving these classifications. Using user-dependent training, we obtain continuous average recall and precision rates (for positive activities) of 78 percent and 74 percent, respectively. Using user-independent training (leave-one-out across five users), we obtain recall rates of 66 percent and precision rates of 63 percent. In isolation, these activities were recognized with accuracies of 98 percent, 87 percent, and 95 percent for the user-dependent, user-independent, and user-adapted cases, respectively.
An important issue to be addressed in a smart home environment is how to provide appropriate services according to the preference of inhabitants. In this paper, we aim at developing a system to learn a multiple users' preference model that represents relationships among users as well as dependency between services and sensor observations. Thus, the service can be inferred based on the learnt model. To achieve this, we propose a three-layer model in our work. At the first layer, raw data from sensors are interpreted as context information after noise removal. The second layer is dynamic Bayesian networks which model the observation sequences including inhabitants' location and electrical appliance (EA) information. At the highest layer, we integrate second layer's, environment information and the relations between inhabitants to provide the service to inhabitants. Therefore, the system can infer appropriate services to inhabitants at right time and right place, and let them feel comfortable. In experiments, we show our model can provide reliable and appreciate services to inhabitants in a smart home environment.
In this paper we present results related to achieving finegrained activity recognition for context-aware computing applications. We examine the advantages and challenges of reasoning with globally unique object instances detected by an RFID glove. We present a sequence of increasingly powerful probabilistic graphical models for activity recognition. We show the advantages of adding additional complexity and conclude with a model that can reason tractably about aggregated object instances and gracefully generalizes from object instances to their classes by using abstraction smoothing. We apply these models to data collected from a morning household routine.
We address the problem of recognizing sequences of human interaction patterns in meetings, with the goal of structuring them in semantic terms. The investigated patterns are inherently group-based (defined by the individual activities of meeting participants, and their interplay), and multimodal (as captured by cameras and microphones). By defining a proper set of individual actions, group actions can be modeled as a two-layer process, one that models basic individual activities from low-level audio-visual features, and another one that models the interactions. We propose a two-layer Hidden Markov Model (HMM) framework that implements such concept in a principled manner, and that has advantages over previous works. First, by decomposing the problem hierarchically, learning is performed on low-dimensional observation spaces, which results in simpler models. Second, our framework is easier to interpret, as both individual and group actions have a clear meaning, and thus easier to improve. Third, different HMM models can be used in each layer, to better reflect the nature of each subproblem. Our framework is general and extensible, and we illustrate it with a set of eight group actions, using a public five-hour meeting corpus. Experiments and comparison with a single-layer HMM baseline system show its validity.
Dynamic Probabilistic Networks (DPNs) are exploited for modeling the temporal relationships among a set of different object temporal events in the scene for a coherent and robust scene-level behaviour interpretation. In particular, we develop a Dynamically Multi-Linked Hidden Markov Model (DML-HMM) to interpret group activities involving multiple objects captured in an outdoor scene. The model is based on the discovery of salient dynamic interlinks among multiple temporal events using DPNs. Object temporal events are detected and labeled using Gaussian Mixture Models with automatic model order selection. A DML-HMM is built using Schwarz's Bayesian Information Criterion based factorisation resulting in its topology being intrinsically determined by the underlying causality and temporal order among different object events. Our experiments demonstrate that its performance on modelling group activities in a noisy outdoor scene is superior compared to that of a Multi-Observation Hidden Markov Model (MOHMM), a Parallel Hidden Markov Model (PaHMM) and a Coupled Hidden Markov Model (CHMM).
Our goal is to exploit human motion and object context to perform action recognition and object classification. Towards this end, we introduce a framework for recognizing actions and objects by measuring image-, object- and action-based information from video. Hidden Markov models are combined with object context to classify hand actions, which are aggregated by a Bayesian classifier to summarize activities. We also use Bayesian methods to differentiate the class of unknown objects by evaluating detected actions along with low-level, extracted object features. Our approach is appropriate for locating and classifying objects under a variety of conditions including full occlusion. We show experiments where both familiar and previously unseen objects are recognized using action and context information
A key aspect of pervasive computing is using computers and sensor networks to effectively and unobtrusively infer users' behavior in their environment. This includes inferring which activity users are performing, how they're performing it, and its current stage. Recognizing and recording activities of daily living is a significant problem in elder care. A new paradigm for ADL inferencing leverages radio-frequency-identification technology, data mining, and a probabilistic inference engine to recognize ADLs, based on the objects people use. We propose an approach that addresses these challenges and shows promise in automating some types of ADL monitoring. Our key observation is that the sequence of objects a person uses while performing an ADL robustly characterizes both the ADL's identity and the quality of its execution. So, we have developed Proactive Activity Toolkit (PROACT).
We compare discriminative and generative learning as typified by logistic regression and naive Bayes. We show, contrary to a widely held belief that discriminative classifiers are almost always to be preferred, that there can often be two distinct regimes of performance as the training set size is increased, one in which each algorithm does better. This stems from the observation -- which is borne out in repeated experiments -- that while discriminative learning has lower asymptotic error, a generative classifier may also approach its (higher) asymptotic error much faster.
Our goal is to exploit human motion and object context to perform action recognition and object classification. Towards this end, we introduce a framework for recognizing actions and objects by measuring image-, object- and action-based information from video. Hidden Markov models are combined with object context to classify hand actions, which are aggregated by a Bayesian classifier to summarize activities. We also use Bayesian methods to differentiate the class of unknown objects by evaluating detected actions along with lowlevel, extracted object features. Our approach is appropriate for locating and classifying objects under a variety of conditions including full occlusion. We show experiments where both familiar and previously unseen objects are recognized using action and context information. 1. Introduction This paper proposes a novel approach to human activity recognition that uses context information of particular objects in the scene. We define classes that contain object-s...
In sequence modeling, we often wish to represent complex interaction between labels, such as when performing multiple, cascaded labeling tasks on the same sequence, or when long-range dependencies exist. We present dynamic conditional random fields (DCRFs), a generalization of linear-chain conditional random fields (CRFs) in which each time slice contains a set of state variables and edges---a distributed state representation as in dynamic Bayesian networks (DBNs)---and parameters are tied across slices. Since exact inference can be intractable in such models, we perform approximate inference using several schedules for belief propagation, including tree-based reparameterization (TRP). On a natural-language chunking task, we show that a DCRF performs better than a series of linear-chain CRFs, achieving comparable performance using only half the training data.
The Index of ADL was developed to study results of treatment and prognosis in the elderly and chronically ill. Grades of the Index summarize over-all performance in bathing, dressing, going to toilet, transferring, continence, and feeding. More than 2,000 evaluations of 1,001 individuals demonstrated use of the Index as a survey instrument, as an objective guide to the course of chronic illness, as a tool for studying the aging process, and as an aid in rehabilitation teaching. Of theoretical interest is the observation that the order of recovery of Index functions in disabled patients is remarkably similar to the order of development of primary functions in children. This parallelism, and similarity to the behavior of primitive peoples, suggests that the Index is based on primary biological and psychosocial function, reflecting the adequacy of organized neurological and locomotor response.
Comparison of generative and discriminative classifiers is an ever-lasting topic. As an important contribution to this topic,
based on their theoretical and empirical comparisons between the naïve Bayes classifier and linear logistic regression, Ng
and Jordan (NIPS 841–848, 2001) claimed that there exist two distinct regimes of performance between the generative and discriminative
classifiers with regard to the training-set size. In this paper, our empirical and simulation studies, as a complement of
their work, however, suggest that the existence of the two distinct regimes may not be so reliable. In addition, for real
world datasets, so far there is no theoretically correct, general criterion for choosing between the discriminative and the
generative approaches to classification of an observation x into a class y; the choice depends on the relative confidence we have in the correctness of the specification of either p(y|x) or p(x, y) for the data. This can be to some extent a demonstration of why Efron (J Am Stat Assoc 70(352):892–898, 1975) and O’Neill
(J Am Stat Assoc 75(369):154–160, 1980) prefer normal-based linear discriminant analysis (LDA) when no model mis-specification
occurs but other empirical studies may prefer linear logistic regression instead. Furthermore, we suggest that pairing of
either LDA assuming a common diagonal covariance matrix (LDA-Λ) or the naïve Bayes classifier and linear logistic regression
may not be perfect, and hence it may not be reliable for any claim that was derived from the comparison between LDA-Λ or the
naïve Bayes classifier and linear logistic regression to be generalised to all generative and discriminative classifiers.
Understanding and recognizing human activities from sensor readings is an important task in pervasive computing. In this paper, we investigate the fundamental problem of recognizing activities for multiple users from sensor readings in a home environment, and propose a novel pattern mining approach to recognize both single-user and multi-user activities in a unified solution. We exploit Emerging Pattern - a type of knowledge pattern that describes significant changes between classes of data - for constructing our activity models, and propose an Emerging Pattern based Multi-user Activity Recognizer (epMAR) to recognize both single-user and multi-user activities.
The world’s population is aging at a phenomenal rate. Certain types of cognitive decline, in particular some forms of memory impairment, occur much more frequently in the elderly. This paper describes Autominder, a cognitive orthotic system intended to help older adults adapt to cognitive decline and continue the satisfactory performance of routine activities, thereby potentially enabling them to remain in their own homes longer. Autominder achieves this goal by providing adaptive, personalized reminders of (basic, instrumental, and extended) activities of daily living. Cognitive orthotic systems on the market today mainly provide alarms for prescribed activities at fixed times that are specified in advance. In contrast, Autominder uses a range of AI techniques to model an individual’s daily plans, observe and reason about the execution of those plans, and make decisions about whether and when it is most appropriate to issue reminders. Autominder is currently deployed on a mobile robot, and is being developed as part of the Initiative on Personal Robotic Assistants for the Elderly (the Nursebot project).
Formalizing computational models for everyday human activities remains an open challenge. Many previous approaches towards this end assume prior knowledge about the structure of activities, using which explicitly defined models are learned in a completely supervised manner. For a majority of everyday environments however, the structure of the in situ activities is generally not known a priori. In this paper we investigate knowledge representations and manipulation techniques that facilitate learning of human activities in a minimally supervised manner. The key contribution of this work is the idea that global structural information of human activities can be encoded using a subset of their local event subsequences, and that this encoding is sufficient for activity-class discovery and classification.In particular, we investigate modeling activity sequences in terms of their constituent subsequences that we call event n-grams. Exploiting this representation, we propose a computational framework to automatically discover the various activity-classes taking place in an environment. We model these activity-classes as maximally similar activity-cliques in a completely connected graph of activities, and describe how to discover them efficiently. Moreover, we propose methods for finding characterizations of these discovered classes from a holistic as well as a by-parts perspective. Using such characterizations, we present a method to classify a new activity to one of the discovered activity-classes, and to automatically detect whether it is anomalous with respect to the general characteristics of its membership class. Our results show the efficacy of our approach in a variety of everyday environments.
We present the use of layered probabilistic representations for modeling human activities, and describe how we use the representation to do sensing, learning, and inference at multiple levels of temporal granularity and abstraction and from heterogeneous data sources. The approach centers on the use of a cascade of Hidden Markov Models named Layered Hidden Markov Models (LHMMs) to diagnose states of a user’s activity based on real-time streams of evidence from video, audio, and computer (keyboard and mouse) interactions. We couple these LHMMs with an expected utility analysis that considers the cost of misclassification. We describe the representation, present an implementation, and report on experiments with our layered architecture in a real-time office-awareness setting.
Activity recognition is a key component for creating intel- ligent, multi-agent systems. Intrinsically, activity recogni- tion is a temporal classification problem. In this paper, we compare two models for temporal classification: hidden Markov models (HMMs), which have long been applied to the activity recognition problem, and conditional random fields (CRFs). CRFs are discriminative models for label- ing sequences. They condition on the entire observation sequence, which avoids the need for independence assump- tions between observations. Conditioning on the observa- tions vastly expands the set of features that can be incorpo- rated into the model without violating its assumptions. Us- ing data from a simulated robot tag domain, chosen because it is multi-agent and produces complex interactions between observations, we explore the dierences in performance be- tween the discriminatively trained CRF and the generative HMM. Additionally, we examine the eect of incorporating features which violate independence assumptions between observations; such features are typically necessary for high classification accuracy. We find that the discriminatively trained CRF performs as well as or better than an HMM even when the model features do not violate the indepen- dence assumptions of the HMM. In cases where features de- pend on observations from many time steps, we confirm that CRFs are robust against any degradation in performance.
Recognising behaviours of multiple people, especially high-level behaviours, is an important task in surveillance systems. When the reliable assignment of people to the set of observations is unavailable, this task becomes compli- cated. To solve this task, we present an approach, in which the hierarchical hidden Markov model (HHMM) is used for modeling the behaviour of each person and the joint probabilistic data association filters (JPDAF) is applied for data association. The main contributions of this paper lie in the integration of multiple HHMMs for recognising high-level behaviours of multiple people and the construction of the Rao-Blackwellised particle filters (RBPF) for ap- proximate inference. Preliminary experimental results in a real environment show the robustness of our integrated method in behaviour recognition and its advantage over the use of Kalman filter in tracking people.
In this work, algorithms are developed and evaluated to de- tect physical activities from data acquired using five small biaxial ac- celerometers worn simultaneously on different parts of the body. Ac- celeration data was collected from 20 subjects without researcher su- pervision or observation. Subjects were asked to perform a sequence of everyday tasks but not told specifically where or how to do them. Mean, energy, frequency-domain entropy, and correlation of acceleration data was calculated and several classifiers using these features were tested. De- cision tree classifiers showed the best performance recognizing everyday activities with an overall accuracy rate of 84%. The results show that although some activities are recognized well with subject-independent training data, others appear to require subject-specific training data. The results suggest that multiple accelerometers aid in recognition because conjunctions in acceleration feature values can effectively discriminate many activities. With just two biaxial accelerometers - thigh and wrist - the recognition performance dropped only slightly. This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves.
Activity recognition is significant in intelligent surveillance. In this paper, we present a novel approach to the recognition of interacting activities based on dynamic Bayesian network (DBN). In this approach the features representing the object motion are divided into two classes: global features and local features, which are at two different spatial scales. Global features describe object motion at a large spatial scale and relations between objects or between the object and environment, and local ones represent the motion details of objects of interest. We propose a new DBN model structure with state duration to model human interacting activities. This DBN model structure combines the global features with local ones harmoniously. The effectiveness of this novel approach is demonstrated by experiment
In this paper we introduce a new dynamic Bayesian network that separates the speakers and their speak- ing turns in a multi-person conversation. We pro- tect the speakers' privacy by using only features from which intelligible speech cannot be recon- structed. The model we present combines data from multiple audio streams, segments the streams into speech and silence, separates the different speakers, and detects when other nearby individu- als who are not wearing microphones are speaking. No pre-trained speaker specific models are used, so the system can be easily applied in new and dif- ferent environments. We show promising results in two very different datasets that vary in background noise, microphone placement and quality, and con- versational dynamics.
This paper presents a synergistic track- and body-level analysis framework for multi-person inter- action and activity analysis in the context of video sur- veillance. The proposed two-level analysis framework covers human activities both in wide and narrow fields of view with distributed camera sensors. The track-level analysis deals with the gross-level activity patterns of multiple tracks in various wide-area surveillance situa- tions. The body-level analysis focuses on detailed-level activity patterns of individuals in isolation or in groups. 'Spatio-temporal personal space' is introduced to model various patterns of grouping behavior between persons. 'Adaptive context switching' is proposed to mediate the track-level and body-level analysis depending on the interpersonal configuration and imaging fidelity. Our approach is based on the hierarchy of action concepts: static pose, dynamic gesture, body-part action, single- person activity, and group interaction. Event ontology with human activity hierarchy combines the multi-level analysis results to form a semantically meaningful event description. Experimental results with real-world data show the effectiveness of the proposed framework.
Recognizing human activities from sensor readings has recently attracted much research interest in pervasive computing due to its potential in many applications, such as assistive living and healthcare. This task is particularly challenging because human activities are often performed in not only a simple (i.e., sequential), but also a complex (i.e., interleaved or concurrent) manner in real life. Little work has been done in addressing complex issues in such a situation. The existing models of interleaved and concurrent activities are typically learning-based. Such models lack of flexibility in real life because activities can be interleaved and performed concurrently in many different ways. In this paper, we propose a novel pattern mining approach to recognize sequential, interleaved, and concurrent activities in a unified framework. We exploit Emerging Pattern—a discriminative pattern that describes significant changes between classes of data—to identify sensor features for classifying activities. Different from existing learning-based approaches which require different training data sets for building activity models, our activity models are built upon the sequential activity trace only and can be applied to recognize both simple and complex activities. We conduct our empirical studies by collecting real-world traces, evaluating the performance of our algorithm, and comparing our algorithm with static and temporal models. Our results demonstrate that, with a time slice of 15 seconds, we achieve an accuracy of 90.96 percent for sequential activity, 88.1 percent for interleaved activity, and 82.53 percent for concurrent activity. Index Terms—Human activity recognition, pattern analysis, emerging pattern, classifier design and evaluation.
We present methods for coupling hidden Markov models (hmms) to model systems of multiple interacting processes. The resulting models have multiple state variables that are temporally coupled via matrices of conditional probabilities. We introduce a deterministic O(T (CN) 2 ) approximation for maximum a posterior (MAP) state estimation which enables fast classification and parameter estimation via expectation maximization. An "N-heads" dynamic programming algorithm samples from the highest probability paths through a compact state trellis, minimizing an upper bound on the cross entropy with the full (combinatoric) dynamic programming problem. The complexity is O(T (CN) 2 ) for C chains of N states apiece observing T data points, compared with O(TN 2C ) for naive (Cartesian product), exact (state clustering), and stochastic (Monte Carlo) methods applied to the same inference problem. In several experiments examining training time, model likelihoods, classification accuracy, and ro...