Conference Paper

Monitoring Activities of Daily Living Using Audio Analysis and a RaspberryPI: A Use Case on Bathroom Activity Monitoring

If you want to read the PDF, try requesting it from the authors.

Abstract

A framework that utilizes audio information for recognition of activities of daily living (ADLs) in the context of a health monitoring environment is presented in this chapter. We propose integrating a Raspberry PI single-board PC that is used both as an audio acquisition and analysis unit. So Raspberry PI captures audio samples from the attached microphone device and executes a set of real-time feature extraction and classification procedures, in order to provide continuous and online audio event recognition to the end user. Furthermore, a practical workflow is presented, that helps the technicians that setup the device to perform a fast, user-friendly and robust tuning and calibration procedure. As a result, the technician is capable of “training” the device without any need for prior knowledge of machine learning techniques. The proposed system has been evaluated against a particular scenario that is rather important in the context of any healthcare monitoring system for the elder: In particular, we have focused on the “bathroom scenario” according to which, a Raspberry PI device equipped with a single microphone is used to monitor bathroom activity on a 24/7 basis in a privacy-aware manner, since no audio data is stored or transmitted. The presented experimental results prove that the proposed framework can be successfully used for audio event recognition tasks.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Cheng et al. [14] presented a framework for activity monitoring by sEMG and accelerometer signals, and obtained a recognition accuracy of > 98%. However, considerable challenges continue to exist in searching for the best feature set from original sEMG datasets [15]. The sEMG can be influenced by many disturbing factors, such as electrode displacement, postural changes, and individualdependent features, such as condition of muscles, subcutaneous fat, and skin surface [16]. ...
... Wavelet transformation is widely used in sEMG processing [17]. It decomposes sEMG into many sub-bands including accurate information [15]. Coherence analyzes the relation between two signals in time-frequency space, which is popular and useful in digital signal analysis, especially in wavelet transformation [18]. ...
Article
Full-text available
Daily activity monitoring plays an important role among frail or elderly people and has caught attention. Surface electromyography (sEMG) can extract the feature of activity, but it is not stable because of electrode displacement, postural changes, and individual-dependent features, such as the condition of muscles, subcutaneous fat, and skin surface. To effectively extract the feature of sEMG signal, we proposed a new method of feature extraction based on coherence analysis. The sEMG signals were recorded from gastrocnemius, tibialis anterior, rectus femoris, and semitendinosus. After de-noising, sEMG signals were decomposed into 32-scale by wavelet transformation, and their wavelet coefficients were employed to calculate wavelet coherence coefficients (WCC). We employed T test to find out if the coherence between sEMG signals was statistically different among six activities. The 32nd scale WCC of RF–ST and ST–TA as eigenvector was entered into the support vector machine (SVM). The six activities, namely, standing, walking, running, stair-ascending, stair-descending, and falling, were successfully identified by the WCC feature with the SVM classifier. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.
... The inventory of machine perception methods developed in order to instantiate the architecture into the RADIO prototype includes acoustic event detection [7,8,9], machine vision [10,11], pattern recognition in range data [12], and localization [13]. ...
Conference Paper
Full-text available
Demographic and epidemiologic transitions have brought a new health care paradigm where life expectancy is increasing as well as the need for long-term care. To meet the resulting challenge, healthcare systems need to take full advantage of new opportunities offered by technical advancements in ICT. The RADIO project explores a novel approach to user acceptance and unobtrusiveness: an integrated smart home/assistant robot system where health monitoring equipment is an obvious and accepted part of the user’s daily life. By using the smart home/assistant robot as sensing equipment for health monitoring, we mask the functionality of the sensors rather than the sensors themselves. In this manner, sensors do not need to be discrete and cumbersome to install; they do however need to be perceived as a natural component of the smart home/assistant robot functionalities.
Conference Paper
Full-text available
In this paper, we present an architecture for recognizing events related to activities of daily living in the context of a health monitoring environment. The proposed approach explores the integration of a Raspberry PI singleboard PC both as an audio acquisition and analysis unit. A set of real-time feature extraction and classification procedures has been implemented and integrated on the Raspberry PI device, in order to provide continuous and online audio event recognition. In addition, a tuning and calibration workflow is presented, according to which the technicians installing the device in a fast ans user-friendly manner, without any requirements for machine learning expertise. The proposed approach has been evaluated against a particular scenario that is rather important in the context of any healthcare monitoring system for the elder, namely the ”bathroom scenario” according to which a single microphone installed on a Raspberry PI device is used to monitor bathroom activity in a 24/7 basis. Experimental results indicate a satisfactory performance rate on the classification process (around 70% for five bathroom-related audio classes) even when less than two minutes of annotated data are used for training in the installation procedure. This makes the whole procedure non demanding in terms of time and effort needed to be calibrated by the technician.
Article
Full-text available
One of the greatest challenges in Ambient Assisted Living is to design health smart homes that anticipate the needs of its inhabitant while maintaining their safety and comfort. It is thus essential to ease the interaction with the smart home through systems that naturally react to voice command using microphones rather than tactile interfaces. However, efficient audio analysis in such noisy environment is a challenging task. In this paper, a real-time audio analysis system, the AuditHIS system, devoted to audio analysis in smart home environment is presented. AuditHIS has been tested thought three experiments carried out in a smart home that are detailed. The results show the difficulty of the task and serve as basis to discuss the stakes and the challenges of this promising technology in the domain of AAL.
Article
Full-text available
In this work, the process followed for the development of a specific Ambient Assisted Living system is presented. The proposed systems has been designed to monitor elders which live alone and want to keep living independently. The process covers all the phases in intelligent system development: requirement analysis, conceptual model specification, architectural design and evaluation. One of the main contributions of the proposed work is an exhaustive evaluation methodology that is integrated in the development process. A relevant characteristic of the evaluation process is that, from initial phases, commercial presentations of the products functionalities is possible. Another important contribution is related with the capability of the system to adapt its behavior to that of the monitored elder. The presented system is called Necesity. It has become a commercial product which is already working and giving service to elders in the South-East of Spain.
Conference Paper
Full-text available
One of the greatest challenges in Ambient Assisted Living is to design health smart homes that could be able to anticipate the needs of its inhabitant while maintaining their comfort and their safety with an adaptation of the house environment and a facilitation of the connections to the outside world. The most likely to benefit from these smart homes are people in loss of autonomy such as the disabled people or the elderly with cognitive deficiencies. But it becomes essential to ease the interactions with the smart home through dedicated interfaces, in particular, thanks to systems reactive to vocal orders. Audio recognition is also a promising way to ensure more safety by contributing to detection of distress situations. This paper presents the stakes and the challenges of this domain based on some experiments carried out concerning distress call recognition and sound classification at home.
Article
Full-text available
Gait velocity has been shown to quantitatively estimate risk of future hospitalization, a predictor of disability, and has been shown to slow prior to cognitive decline. In this paper, we describe a system for continuous and unobtrusive in-home assessment of gait velocity, a critical metric of function. This system is based on estimating walking speed from noisy time and location data collected by a ??sensor line?? of restricted view passive infrared motion detectors. We demonstrate the validity of our system by comparing with measurements from the commercially available GAITRite walkway system gait mat. We present the data from 882 walks from 27 subjects walking at three different subject-paced speeds (encouraged to walk slowly, normal speed, or fast) in two directions through a sensor line. The experimental results show that the uncalibrated system accuracy (average error) of estimated velocity was 7.1 cm/s (SD = 11.3 cm/s), which improved to 1.1 cm/s (SD = 9.1 cm/s) after a simple calibration procedure. Based on the average measured walking speed of 102 cm/s, our system had an average error of less than 7% without calibration and 1.1% with calibration.
Conference Paper
Full-text available
Human collective set of experiences makes us who we are and help us delineate a path for our ongoing life. Ageing, however, progressively limits our ability to save, in our internal memory, these same experiences, or, at least, limits our capability to remember them. The capability to remember, intrinsic to our memory, is a very important one to us, as a human been, being this what deferent’s us from several other species. In this paper we present a memory assistant sub module of a bigger project, the VirtualECare, which ability will be to remember us, not our past experiences, but our routine day-to-day tasks and activities, in a somehow proactive manner, thus, allowing us to have some relaxation about them, and focus the remaining of our ability in most important facts.
Article
Full-text available
One of the greatest challenges in Ambient Assisted Living is to design health smart homes that anticipate the needs of its inhabitant while maintaining their safety and comfort. It is thus essential to ease the interaction with the smart home through systems that naturally react to voice command using microphones rather than tactile interfaces. However, efficient audio analysis in such noisy environment is a challenging task. In this paper, a real-time audio analysis system, the AuditHIS system, devoted to audio analysis in smart home environment is presented. AuditHIS has been tested thought three experiments carried out in a smart home that are detailed. The results show the difficulty of the task and serve as basis to discuss the stakes and the challenges of this promising technology in the domain of AAL. © 2016 Rae Lesser Blumberg and Samuel Cohn. All rights reserved.
Article
Full-text available
Traditional classification approaches generalize poorly on image classification tasks, because of the high dimensionality of the feature space. This paper shows that support vector machines (SVM's) can generalize well on difficult image classification problems where the only features are high dimensional histograms. Heavy-tailed RBF kernels of the form K(x, y) = e(-rho)Sigma(i)/xia-yia/b with a < or = 1 and b < or = 2 are evaluated on the classification of images extracted from the Corel stock photo collection and shown to far outperform traditional polynomial or Gaussian radial basis function (RBF) kernels. Moreover, we observed that a simple remapping of the input x(i)-->x(i)(a) improves the performance of linear SVM's to such an extend that it makes them, for this problem, a valid alternative to RBF kernels.
Article
Full-text available
With the rapid growth of the elderly population, there is a need to support the ability of elders to maintain an independent and healthy lifestyle in their homes rather than through more expensive and isolated care facilities. One approach to accomplish these objectives employs the concepts of ambient intelligence to remotely monitor an elder's activities and condition. The SmartHouse project uses a system of basic sensors to monitor a person's in-home activity; a prototype of the system is being tested within a subject's home. We examined whether the system could be used to detect behavioral patterns and report the results in this paper. Mixture models were used to develop a probabilistic model of behavioral patterns. The results of the mixture-model analysis were then evaluated by using a log of events kept by the occupant.
Article
Full-text available
The output of a classifier should be a calibrated posterior probability to enable post-processing. Standard SVMs do not provide such probabilities. One method to create probabilities is to directly train a kernel classifier with a logit link function and a regularized maximum likelihood score. However, training with a maximum likelihood score will produce non-sparse kernel machines. Instead, we train an SVM, then train the parameters of an additional sigmoid function to map the SVM outputs into probabilities. This chapter compares classification error rate and likelihood scores for an SVM plus sigmoid versus a kernel method trained with a regularized likelihood error function. These methods are tested on three data-mining-style data sets. The SVM+sigmoid yields probabilities of comparable quality to the regularized maximum likelihood kernel method, while still retaining the sparseness of the SVM.
Conference Paper
Microphones are remarkably powerful sensors of human behavior and context. However, audio sensing is highly susceptible to wild fluctuations in accuracy when used in diverse acoustic environments (such as, bedrooms, vehicles, or cafes), that users encounter on a daily basis. Towards addressing this challenge, we turn to the field of deep learning; an area of machine learning that has radically changed related audio modeling domains like speech recognition. In this paper, we present DeepEar - the first mobile audio sensing framework built from coupled Deep Neural Networks (DNNs) that simultaneously perform common audio sensing tasks. We train DeepEar with a large-scale dataset including unlabeled data from 168 place visits. The resulting learned model, involving 2.3M parameters, enables DeepEar to significantly increase inference robustness to background noise beyond conventional approaches present in mobile devices. Finally, we show DeepEar is feasible for smartphones by building a cloud-free DSP-based prototype that runs continuously, using only 6% of the smartphone's battery daily.
Chapter
This chapter has an introductory purpose. A chapter outline is provided, along with general notes on the book’s exercises and the companion software. Before we proceed, it is important to note that, although in this book the term audio does not exclude the speech signal, we are not focusing on traditional speech-related problems that have been studied by the research community for decades, e.g., speech recognition and coding.
Book
Introduction to Audio Analysis serves as a standalone introduction to audio analysis, providing theoretical background to many state-of-the-art techniques. It covers the essential theory necessary to develop audio engineering applications, but also uses programming techniques, notably MATLAB®, to take a more applied approach to the topic. Basic theory and reproducible experiments are combined to demonstrate theoretical concepts from a practical point of view and provide a solid foundation in the field of audio analysis. Audio feature extraction, audio classification, audio segmentation, and music information retrieval are all addressed in detail, along with material on basic audio processing and frequency domain representations and filtering. Throughout the text, reproducible MATLAB® examples are accompanied by theoretical descriptions, illustrating how concepts and equations can be applied to the development of audio analysis systems and components. A blend of reproducible MATLAB® code and essential theory provides enable the reader to delve into the world of audio signals and develop real-world audio applications in various domains.
Conference Paper
In this paper an automated bathroom activity monitoring system based on acoustics is described. The system is designed to recognize and classify major activities occurring within a bathroom based on sound. Carefully designed HMM parameters using MFCC features are used for accurate and robust bathroom sound event classification. Experiments to validate the utility of the system were performed firstly in a constrained setting as a proof-of-concept and later in an actual trial involving real people using their bathroom in the normal course of their daily lives. Preliminary results are encouraging with the accuracy rate for most sound categories being above 84%. We sincerely believe that the system contributes towards increased understanding of personal hygiene behavioral problems that significantly affect both informal care-giving and clinical care of dementia patients.
This study examined frail elders' acceptance of the concept of home monitoring devices. With the potential of such devices to ultimately assist many older persons, acceptance of the device in the individual's home is a critical component. Elders who view devices negatively--as unnecessary, unattractive, or intrusive--may be less likely to use the device if installed or less likely to allow them to be installed. For the participants in the current study, the results suggest strong acceptance of the concept of home health monitoring and the devices to make the system work. When questioned on device appearance a majority of the subjects felt that the devices would be acceptable in their homes, and initial reactions to the devices were primarily favorable. Equipment characteristics have been identified as one of the determining factors of perceived intrusiveness of home monitoring devices (Fisk, 1997). Study participants made several suggestions pertaining to device features and appearance. A common criticism related to device size, especially concerning the blood pressure cuff which was referred to as "gaudy" by one study participant. Participants offered suggestions such as making devices smaller, providing control for volume adjustment, and providing voice activation. At least one participant expressed a concern over device functioning in the event of distance traveling. Subjective comments such as "I think it would help many people," "It's very reasonable and important in several ways," and "...people would be more independent and safe," provided anecdotal support of device acceptance. Although a majority of the study participants had favorable responses to the devices and monitoring systems, many of their subjective comments reflected positive views regarding use by others as opposed to personal use. This finding may suggest that the participants did not personally identify with the need to use such devices but rather viewed the devices as relevant and acceptable for "the person who absolutely needs it." However, a majority of the participants identified "relieving personal worry" as a possible benefit of the home monitoring system which suggests personal identification with the potential benefits. The findings of study participants' willingness to pay, and a desire to maintain communications on a consistent basis with monitoring services, may demonstrate overall acceptance of the idea of home monitoring devices/services and establishes a need for continued research in product development. During the interviews, many of the subjects expressed enthusiasm and interest over the prospect of the home monitoring devices and systems with which they were relatively unfamiliar. This suggests a need for further consumer education regarding use of home monitoring devices and systems. Aside from the relatively small sample size, one limitation of this study relates to the study participants' understanding of the devices in relation to their current needs. The questionnaire results provided hypothetical acceptance of devices from a usefulness and aesthetic point of view. However, the findings may not totally reflect the study participants' actual willingness or desire to utilize the applicable devices in their homes. Further research is suggested in the area of assessment of potential consumer groups' perceptions of their current health status, functional limitations and needs, and more extensive research regarding perceptions of the potential benefits of using home monitoring devices. Further research in product development and clinical trials of existing home monitoring devices is also recommended.
-). pyAudioAnalysis: Python audio analysis library: Feature extraction, classification, segmentation and applications
  • Giannakopoulos
  • T Giannakopoulos
Giannakopoulos, 15. Giannakopoulos, T. (2015-). pyAudioAnalysis: Python audio analysis library: Feature extraction, classification, segmentation and applications. [Online; accessed 2015-04-27].
MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval
  • K Hyoung-Gook
  • M Nicolas
  • T Sikora
Automatic monitoring of activities of daily living based on real-life acoustic sensor data: a preliminary study
  • Vuegen
Vuegen et al., 2013. Vuegen, L., Van Den Broeck, B., Karsmakers, P., Vanrumste, B., et al. (2013). Automatic monitoring of activities of daily living based on real-life acoustic sensor data: a preliminary study. In Fourth workshop on speech and language processing for assistive technologies (SLPAT): Proceedings, pages 113-118. Association for Computational Linguistics (ACL).
D2.2: Early detection methods and relevant system requirements Available at http://radio-project Pattern Recognition, Fourth Edition
  • S Theodoridis
  • K Koutroumbas
RADIO Project, 2015. RADIO Project (2015). D2.2: Early detection methods and relevant system requirements. Available at http://radio-project.eu/deliverables. Theodoridis and Koutroumbas, 2008. Theodoridis, S. and Koutroumbas, K. (2008). Pattern Recognition, Fourth Edition. Academic Press, Inc.
Automatic monitoring of activities of daily living based on real-life acoustic sensor data: a preliminary study
  • L Vuegen
  • B Van Den Broeck
  • P Karsmakers
  • B Vanrumste