No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Patch-based Learning for Radar-based Fall Event Detection using Gramian Angular Field
ResearchGate has not been able to resolve any citations for this publication.
With an increase in the population of older adults in developed nations globally, research on radar-based human activity recognition for reliable and accurate fall event detection has accelerated exponentially. Radars are safe, contactless, and privacy-preserving sensors that facilitate ‘aging in place’. A plethora of research papers have been published in this field in the last 5 years. The primary goal of all research works is to recognize the human activities from the backscattered radar returns. Despite being a well-researched field, technology-transfer from lab to market is implausible due to several underlying issues that are yet to be addressed. These issues will serve as a potential barrier when implementing the developed technologies in real-life. This article aims to reveal some of these issues that are important for successful technology-transfer of radar-based human activity recognition systems, and potential solutions to mitigate these issues are proposed.
Vision-based fall detection systems have experienced fast development over the last years. To determine the course of its evolution and help new researchers, the main audience of this paper, a comprehensive revision of all published articles in the main scientific databases regarding this area during the last five years has been made. After a selection process, detailed in the Materials and Methods Section, eighty-one systems were thoroughly reviewed. Their characterization and classification techniques were analyzed and categorized. Their performance data were also studied, and comparisons were made to determine which classifying methods best work in this field. The evolution of artificial vision technology, very positively influenced by the incorporation of artificial neural networks, has allowed fall characterization to become more resistant to noise resultant from illumination phenomena or occlusion. The classification has also taken advantage of these networks, and the field starts using robots to make these systems mobile. However, datasets used to train them lack real-world data, raising doubts about their performances facing real elderly falls. In addition, there is no evidence of strong connections between the elderly and the communities of researchers.
Accidental falls are a major source of loss of autonomy, deaths, and injuries among the elderly. Accidental falls also have a remarkable impact on the costs of national health systems. Thus, extensive research and development of fall detection and rescue systems are a necessity. Technologies related to fall detection should be reliable and effective to ensure a proper response. This paper provides a comprehensive review on state-of-the-art fall detection technologies considering the most powerful deep learning methodologies. We reviewed the most recent and effective deep learning methods for fall detection and categorized them into three categories: Convolutional Neural Network (CNN) based systems, Long Short-Term Memory (LSTM) based systems, and Auto-encoder based systems. Among the reviewed systems, three dimensional (3D) CNN, CNN with 10-fold cross-validation, LSTM with CNN based systems performed the best in terms of accuracy, sensitivity, specificity, etc. The reviewed systems were compared based on their working principles, used deep learning methods, used datasets, performance metrics, etc. This review is aimed at presenting a summary and comparison of existing state-of-the-art deep learning based fall detection systems to facilitate future development in this field.
Background:
Falls in older adults are a reasonably common occurrence and about 10% of these experience multiple falls annually. These falls may be serious and may cause significant morbidity and mortality. These can also threaten the independence of older people and may be responsible for an individual's loss of independence and socioeconomic consequences. These falls may add extra burden to the health care and to direct and indirect costs.
Methodology:
An extensive search of literature was done on the important data bases of PubMed, SCOPUS, and Google Scholar on this topic and all the useful information was derived from the relevant articles for this review.
Results:
We found that the falls in older individuals are often multi factorial and hence a multidisciplinary approach is required to prevent and manage these falls. The risk factors leading to the falls could be divided into extrinsic, intrinsic and situational factors. The commonest and serious injuries are to the head and fractures, due to fragility of bones.
Discussion:
The falls in elderly are on rise and taking the shape of an epidemic. Prevention of these falls is far better than the management. Safe living environment of the elderly people helps in prevention of these falls. The management of the falls should focus on the causative factors, apart from treating the injuries caused by the falls.
Inspired by recent successes of deep learning in computer vision, we propose
a novel framework for encoding time series as different types of images,
namely, Gramian Angular Summation/Difference Fields (GASF/GADF) and Markov
Transition Fields (MTF). This enables the use of techniques from computer
vision for time series classification and imputation. We used Tiled
Convolutional Neural Networks (tiled CNNs) on 20 standard datasets to learn
high-level features from the individual and compound GASF-GADF-MTF images. Our
approaches achieve highly competitive results when compared to nine of the
current best time series classification approaches. Inspired by the bijection
property of GASF on 0/1 rescaled data, we train Denoised Auto-encoders (DA) on
the GASF images of four standard and one synthesized compound dataset. The
imputation MSE on test data is reduced by 12.18%-48.02% when compared to using
the raw data. An analysis of the features and weights learned via tiled CNNs
and DAs explains why the approaches work.
With the increasing morbidity and mortality rate in older adults above 65 years of age due to accidental fall, privacy-preserving radar-based fall event detection is becoming crucial. Deep learning algorithm like vision transformers (ViT) for human fall-event detection using different radar domain representation have shown excellent fall-detection accuracy. However, such techniques are computationally very expensive and unsuitable when training datasets are small. Patch-based learning models such as Multi-Layer Perceptron-Mixer (MLP-Mixer) and Convolutional-Mixer (ConvMixer) models have been developed as alternatives to ViT. In this work, the decision outputs of light-weight ConvMixer models with different domain representations of radar returns as inputs are fused for classifying the events as fall or non-fall. This proposed approach of event classification utilizes supplementary information present in different domains for enhancing the classification accuracy. Evaluation done on publicly available dataset shows an improved performance of the multi-domain ConvMixer model over ViT and MLP-Mixer. This further justifies the choice of light weight ConvMixer as a preferred learnable model when only limited training dataset is available.
An image is worth 16×16 words: Transformers for image recognition at scale
- Dosovitskiy
MLP-Mixer: An all-MLP architecture for vision
- Tolstikhin
A battle of network structures: An empirical study of CNN, transformer, and MLP
- Zhao
Do you even need attention? A stack of feed-forward layers does surprisingly well on imagenet
- L Melas-Kyriazif
L. Melas-Kyriazif, "Do you even need attention? A stack of feed-forward
layers does surprisingly well on imagenet," arXiv:2105.02723, 2021.
Patches are all you need?
- Trockman
A. Trockman and J. Z. Kolter, "Patches are all you need?,"
arXiv:2201.09792, 2022.
Radar signatures of human activities
- Fioranelli
Vision transformer for small-size datasets
- Lee
Radar signatures of human activities
- F Fioranelli
- S A Shah
- H Li
- A Shrestha
- S Yang
- J L Kernec
F. Fioranelli, S. A. Shah, H. Li, A. Shrestha, S. Yang, and J. L. Kernec,
"Radar signatures of human activities," Distributed by University of
Glasgow, 2019.
Vision transformer for small-size datasets
- S H Lee
- S Lee
- B C Song
S. H. Lee, S. Lee, and B. C. Song, "Vision transformer for small-size
datasets," arXiv:2112.13492, 2021.
A battle of network structures: An empirical study of CNN, transformer, and MLP
- Y Zhao
- G Wang
- C Tang
- C Luo
- W Zeng
- Z.-J Zha
Y. Zhao, G. Wang, C. Tang, C. Luo, W. Zeng, and Z.-J. Zha, "A battle of
network structures: An empirical study of CNN, transformer, and MLP,"
arXiv:2108.13002v2, 2021.
An image is worth 16×16 words: Transformers for image recognition at scale
- A Dosovitskiy
- L Beyer
- A Kolesnikov
- D Weissenborn
- X Zhai
- T Unterthiner
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai,
T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly,
J. Uszkoreit, and N. Houlsby, "An image is worth 16x16 words:
Transformers for image recognition at scale," arXiv:2010.11929, 2020.
MLP-Mixer: An all-MLP architecture for vision
- I Tolstikhin
- N Houlsby
- A Kolesnikov
- L Beyer
- X Zhai
- T Un-Terthiner
I. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit, M. Lucic,
and A. Dosovitskiy, "MLP-Mixer: An all-MLP architecture for vision,"
arXiv:2105.01601, 2021.