Figure 1 - uploaded by Mostafa Mirshekari
Content may be subject to copyright.
System overview. (a) shows the conceptual intuition behind our localization approach. Waves resulted from the footstep reach different sensors at different times. Such difference is estimated for localizing the source. (b) shows three steps of the algorithm.
Source publication
We introduce a sensing system which leverages footstep-induced structural vibration for occupant localization. Such localization is important for many smart building applications, such as efficient building management, senior/health care, and security. Compared to other sensing approaches, footstep-induced vibration provides a sensing system which...
Context in source publication
Similar publications
Marble is a material that is commonly used for building components such as furniture, flooring, countertops, bathrooms, windows in homes etc. Due to the many uses of marble in various aspects, marble surface detection is important for this industry to improve quality and avoid financial problems that may occur. In previous research, many methods su...
A low false alarm and narrow-wide band compatible signal detection algorithm is proposed by combining the multiscale wavelet transform extremum detection with the spectrum energy detection. The signals with different bandwidths are detected at respective wavelet scales. Signal detection at a single scale adopts the detection strategy of combining t...
The interaction between a dense forest canopy and atmosphere is a complex fluid-dynamical problem with a wide range of practical applications, spanning from the aspects of carbon sequestration to the spread of wildfires through a forest. To delineate the eddy processes specific to canopy flows, we develop an $\mathcal{L}$-moment based event framewo...
Significant attention has been recently paid to the determination of floor acceleration demands used for the design of acceleration-sensitive non-structural components in buildings. This study makes a contribution through the use of experimental shake table data from one 6-storey and two 3-storey reinforced concrete buildings tested at the E-Defens...
Rapid and accurate structural damage assessment after an earthquake is important for efficient emergency management. The widespread application of surveillance cameras provides a new possibility for improving the efficiency of assessment. However, it is still challenging to directly assess the structural seismic damage based on videos captured by i...
Citations
... Although vibration-based in-home gait health monitoring has succeeded in temporal gait parameter estimation and gait symptom/balance characterization [5,8], it has limited performance in footstep localization, leading to inaccurate spatial parameter estimation. Previous studies developed localization methods mainly based on the time difference of arrival (TDoA) at multiple vibration sensors [6,7,[16][17][18][19][20]24]. The average localization error of such methods is about 0.4 m, which is not accurate enough for gait analysis because the average human step length is ∼0.5 m. ...
... After that, we localize the footsteps through multi-lateration based on the velocity profile model and the time difference of arrival (TDoA) over multiple sensors. We choose the TDoA-based approach because it is less sensitive to the change of floor types and the shoe types [18,19], so it is more scalable to different people's homes. Finally, we compute the spatial gait parameters based on the estimated footstep location for in-home gait analysis. ...
In-home gait analysis is important for providing early diagnosis and adaptive treatments for individuals with gait disorders. Existing systems include wearables and pressure mats, but they have limited scalability due to dense deployment and device carrying/charging requirements. Recently, vision-based systems have been developed to enable scalable, accurate in-home gait analysis, but it faces privacy concerns due to the exposure of people's appearances and daily activities. To overcome these limitations, our prior work developed footstep-induced structural vibration sensing for in-home gait monitoring, which is device-free, wide-ranged, and perceived as more privacy-friendly. Although it has succeeded in temporal parameter estimation, it shows limited performance for spatial gait parameter estimation due to the low accuracy in footstep localiza-tion. In particular, the localization error mainly comes from the estimation error of the wave arrival time at the vibration sensors and its error propagation to wave velocity estimations. To this end, we present GaitVibe+, a vibration-based footstep localization method fused with temporarily installed cameras for in-home gait analysis. Our method has two stages: fusion and operating stages. In the fusion stage, both cameras and vibration sensors are installed to record only a few trials of the subject's footstep data, through which we characterize the uncertainty in wave arrival time and model the wave velocity profiles for the given structure. In the operating stage, we remove the camera to preserve privacy at home. The footstep localization is conducted by estimating the time difference of arrival (TDoA) over multiple vibration sensors, whose accuracy is improved through the reduced uncertainty and velocity modeling during the fusion stage. We evaluate GaitVibe+ through a real-world experiment with 50 walking trials. With only 3 trials of multi-modal fusion, our approach has an average localization error of 0.22 meters, which reduces the spatial gait parameter error by 4.1x (from 111.4% to 27.1%) compared to the existing work.
... [eess.SP] 7 Dec 2022 footstep localization, leading to inaccurate spatial parameter estimation. Previous studies developed localization methods mainly based on the time difference of arrival (TDoA) at multiple vibration sensors [6,7,[16][17][18][19][20]24]. The average localization error of such methods is about 0.4 m, which is not accurate enough for gait analysis because the average human step length is ∼0.5 m. ...
... After that, we localize the footsteps through multi-lateration based on the velocity profile model and the time difference of arrival (TDoA) over multiple sensors. We choose the TDoA-based approach because it is less sensitive to the change of floor types and the shoe types [18,19], so it is more scalable to different people's homes. Finally, we compute the spatial gait parameters based on the estimated footstep location for in-home gait analysis. ...
In-home gait analysis is important for providing early diagnosis and adaptive treatments for individuals with gait disorders. Existing systems include wearables and pressure mats, but they have limited scalability. Recent studies have developed vision-based systems to enable scalable, accurate in-home gait analysis, but it faces privacy concerns due to the exposure of people's appearances. Our prior work developed footstep-induced structural vibration sensing for gait monitoring, which is device-free, wide-ranged, and perceived as more privacy-friendly. Although it has succeeded in temporal gait event extraction, it shows limited performance for spatial gait parameter estimation due to imprecise footstep localization. In particular, the localization error mainly comes from the estimation error of the wave arrival time at the vibration sensors and its error propagation to wave velocity estimations. Therefore, we present GaitVibe+, a vibration-based footstep localization method fused with temporarily installed cameras for in-home gait analysis. Our method has two stages: fusion and operating. In the fusion stage, both cameras and vibration sensors are installed to record only a few trials of the subject's footstep data, through which we characterize the uncertainty in wave arrival time and model the wave velocity profiles for the given structure. In the operating stage, we remove the camera to preserve privacy at home. The footstep localization is conducted by estimating the time difference of arrival (TDoA) over multiple vibration sensors, whose accuracy is improved through the reduced uncertainty and velocity modeling during the fusion stage. We evaluate GaitVibe+ through a real-world experiment with 50 walking trials. With only 3 trials of multi-modal fusion, our approach has an average localization error of 0.22 meters, which reduces the spatial gait parameter error from 111% to 27%.
... To characterize the dispersive (i.e., frequency-dependent) wave propagation velocities, we have decomposed the signal using the wavelet transform which is suitable for analyzing and decomposing non-stationary signals (e.g., impulsive signals such as footsteps) [19,48]. The wavelet decomposition can be described as [49], ...
In this paper, we characterize the effects of obstructions on footstep-induced floor vibrations to enable obstruction-invariant indoor occupant localization. Occupant localization is important in smart building applications such as smart healthcare and energy management. Maintenance and installment requirements limit the application of current sensing approaches (e.g., mobile-based, RF-based, and pressure-based sensing) in real-life applications. To overcome these limitations, prior work has utilized footstep-induced structural vibrations for occupant localization. The main intuition behind these approaches is that the footstep-induced floor vibration waves take different amounts of time to arrive at different sensors. These Time-Differences-of-Arrival (TDoA) can then be leveraged to locate the footstep by assuming similar velocities between the footstep and various sensor locations. This assumption makes these approaches suitable for open areas; however, real buildings have various types of obstructions (e.g., walls, furniture, etc.) which affect wave propagation velocities and hence significantly reduce localization accuracy. Therefore, the prior work requires unobstructed paths between footsteps and sensors for accurate occupant localization, which increases the sensing density requirement and thus, instrumentation and maintenance costs. We have observed that the obstruction mass is one of the key factors in affecting the wave propagation velocity and reducing the localization accuracy. Therefore, to overcome the obstruction challenge, we localize footsteps by considering different velocities between the footsteps and sensors depending on the existence and mass of obstruction on the wave path. Specifically, we (1) detect and estimate the mass of the obstruction by characterizing the wave attenuation rate, (2) use this estimated mass to find the propagation velocities for localization by modeling the velocity-mass relationship through the lamb wave characteristics, and (3) introduce a non-isotropic multilateration approach which robustly leverages these propagation velocities to locate the footsteps (and the occupants). In field experiments, we achieved average localization error of 0.61 meters, which is (1) the same as the average localization error when there is no obstruction and (2) 1.6X improvement compared to the baseline approach.
... However, vibration-based localization in buildings and homes is a challenging task due to vibration signal effects like dispersion, reflections, heterogeneity, and material discontinuity. Multiple applica-tions have been developed using footstep vibration and time difference of arrival (TDoA) to locate indoor pedestrians as shown in the works of Mirshekari et al. [210,211] (Fig. 8(1) and 8(3) respectively). However, TDoA presumed wave propagation in ideal scenarios and environments. ...
... In the same year 2016, Mirshekari et al. [211] introduce the use of the concept of multilateration in footsteps localization. The approach consisted of three steps; first, the footstep is detected using a thresholding method base impulse-like-excitation [199]; then, a TDoA estimation using time-frequency representations of the signal to extract the high energy peaks used as peak-based TDoA; finally, the footstep localization is estimated using multilateration because the foot strike is unknown and it can be leveraged the TDoAs of the signal to localize the source [219]. ...
... Pedestrian localization methods based on induced foot-step vibration. (1) Occupant localization approach proposed in[210]; (2) Framework for seismic sensor-based event detection and localization proposed in[212]; (3) Footstep localization method proposed in[211]. ...
Current sensor technologies enable the passive and continuous monitoring of human behaviors as well as infrastructures to ensure personal safety and assess individual health state. One passive technology that has the potential of gathering personal data is the vibration sensor. In this paper, we carry out an extensive survey of the current vibration-based sensing technologies for human and infrastructure safety as well as health monitoring. These technologies utilize structural and body vibration as a source of data, and they can be incorporated into wearable or non-wearable devices. Furthermore, the vibration sensing technology utilizes low-cost and low-power sensors, which make it attractive for indoor and outdoor monitoring. We have classified the technologies into five categories: vibration-based sensing for assessing human health, recognizing personal behavior, inferring occupancy information, evaluating personal safety, and monitoring infrastructure health. In each category, we also classify the approaches that utilize single and multiple sensors. Moreover, we discuss the different types of signal processing and machine learning techniques that are applied to each approach.
... To overcome these limitations, vibration-based sensing is introduced for occupant localization [21][22][23][24]. This approach does not need the occupants to carry a device (i.e., is non-intrusive) and allows sparse sensor deployment compared to other sensing approaches such as pressure-based sensing (e.g., smart carpets) which need a sensor for every footstep location and RF-based sensing approaches which require either extensive training for fingerprinting or need every person to carry a device to provide a good resolution [25] However, current localization approaches for vibration-based sensing either perform only coarse-grained localization [21] or are based on detailed prior information which limits their use to only specific locations and buildings where the localization models are calibrated [22,23,26]. ...
... Among the TFR approaches, wavelet decomposition has been successfully utilized for impulsive excitations (such as footstep-induced vibration) because it is suitable for representing non-stationary signals [35,24,26,[32][33][34]37]. The main wavelet-based approaches for source localization either utilize: (1) the ridge [32,33,37,35] (i.e., peak amplitude in scaletime plane) which is difficult to robustly estimate in real floors with environmental noise or (2) highest energy scale components [26,24] which potentially have large noise (ambient vibration) because this scale might correspond to the fundamental frequency of the floor. ...
... Among the TFR approaches, wavelet decomposition has been successfully utilized for impulsive excitations (such as footstep-induced vibration) because it is suitable for representing non-stationary signals [35,24,26,[32][33][34]37]. The main wavelet-based approaches for source localization either utilize: (1) the ridge [32,33,37,35] (i.e., peak amplitude in scaletime plane) which is difficult to robustly estimate in real floors with environmental noise or (2) highest energy scale components [26,24] which potentially have large noise (ambient vibration) because this scale might correspond to the fundamental frequency of the floor. Therefore, these approaches are not well-suited for source localization and result in large errors. ...
In this paper, we present an occupant localization approach through sensing footstep-induced floor vibrations. Occupant location information is an important part of many smart building applications, such as energy and space management in a personal home or patient tracking in a hospital room. Adoption of current occupant location sensing approaches in smart buildings (e.g., camera, radio frequency (RF), mobile devices, etc.) is often limited due to the maintenance, installment, and calibration requirements of these sensing systems. To overcome these limitations, we introduce a new approach to use footstep-induced structural vibration for step-level occupant localization. The intuition behind this approach is that footsteps induce floor vibrations which are received in different vibration sensor locations at different times. This paper focuses on localizing a single occupant within each sensing range. To localize the footsteps, we utilize the time differences of arrival (TDoA) of the footstep-induced vibrations. However, this approach involves two main challenges: (1) the vibration wave propagation in the floor is of dispersive nature (i.e., different frequency components travel at different velocities) and (2) due to floor heterogeneity, these wave propagation velocities vary in different structures as well as in different locations in a structure. These issues lead to large localization inaccuracies or calibration requirements. To address dispersion challenge, we present a decomposition-based dispersion mitigation technique which extracts specific components (which have similar propagation characteristics) for localization. To address velocity variations, we introduce an adaptive multilateration approach that employs heuristics to constrain the search space and robustly locate the footsteps when the propagation velocity is unknown. Constraining the search space overcomes the additional complexity which is resulted from adding an unknown variable (propagation velocity). We evaluated our approach using the field experiments in 3 different types of buildings (both commercial and residential) with human participants. The results show an average localization error of 0.34 m, which corresponds to a 6X reduction in error compared to a baseline method. Furthermore, our approach resulted in standard deviation of as low as 0.18 m, which corresponds to a 8.7X improvement in precision compared to the baseline approach.
Current sensor technologies facilitate device-free and non-invasive monitoring of target activities and infrastructures to ensure a safe and inhabitable environment. Device-free techniques for sensing the surrounding environment are an emerging area of research where a target does not need to carry or attach any device to provide information about its motion or the surrounding environment. Consequently, there has been an increasing interest in device-free sensing. Seismic sensors are extremely effective tools for gathering target motion information. In this paper, we provide a comprehensive overview of the seismic sensor-based device-free sensing process and highlight the key techniques within the research field. We classify the existing literature into three categories, viz., (i) target detection, (ii) target localization, and (iii) target identification, and activity recognition. The techniques in each category are divided into multiple subcategories in a structured manner to comprehensively discuss the details. We also discuss the challenges associated with contemporary cutting-edge research and suggest potential solutions.
Current sensor technologies enable the passive and continuous monitoring of human behaviors as well as infrastructures to ensure personal safety and assess individual health state. One passive technology that has the potential of gathering personal data is the vibration sensor. In this paper, we carry out an extensive survey of the current vibration-based sensing technologies for human and infrastructure safety as well as health monitoring. These technologies utilize structural and bodies vibration as a source of data, and they can be incorporated in wearable or non-wearable devices. Furthermore, the vibration sensing technology utilizes low-cost and low-power sensors, which make it attractive for indoor and outdoor monitoring. We have classified the technologies into five categories: vibration-based sensing for assessing human health, recognizing personal behavior, inferring occupancy information, evaluating personal safety, and monitoring infrastructure health. In each category, we also classify the approaches that utilize single and multiple sensors. Moreover, we discuss the different types of signal processing and machine learning techniques that are applied to each approach.
Smart homes of the future are envisioned to have the ability to recognize many types of home activities such as running a washing machine, flushing the toilet, and using a microwave. In this paper, we present a new sensing technology, VibroSense, which is able to recognize 18 different types of activities throughout a house by observing structural vibration patterns on a wall or ceiling using a laser Doppler vibrometer. The received vibration data is processed and sent to a deep neural network which is trained to distinguish between 18 activities. We conducted a system evaluation, where we collected data of 18 home activities in 5 different houses for 2 days in each house. The results demonstrated that our system can recognize 18 home activities with an average accuracy of up to 96.6%. After re-setup of the device on the second day, the average recognition accuracy decreased to 89.4%. We also conducted follow-up experiments, where we evaluated VibroSense under various scenarios to simulate real-world conditions. These included simulating online recognition, differentiating between specific stages of a device's activity, and testing the effects of shifting the laser's position during re-setup. Based on these results, we discuss the opportunities and challenges of applying VibroSense in real-world applications.
This paper presents an indoor area occupancy counting system utilizing the ambient structural vibration induced by pedestrian footsteps. Our system achieves 99.55% accuracy in pedestrian footsteps detection, 0.2 people mean estimation error in pedestrian traffic estimation, and 0.2 area occupant activity estimation error in real-world uncontrolled experiments.
This paper introduces an indirect train traffic monitoring method to detect and infer real-time train events based on the vibration response of a nearby building. Monitoring and characterizing traffic events are important for cities to improve the efficiency of transportation systems (e.g., train passing, heavy trucks, and traffic). Most prior work falls into two categories: (1) methods that require intensive labor to manually record events or (2) systems that require deployment of dedicated sensors. These approaches are difficult and costly to execute and maintain. In addition, most prior work uses dedicated sensors designed for a single purpose, resulting in deployment of multiple sensor systems. This further increases costs. Meanwhile, with the increasing demands of structural health monitoring, many vibration sensors are being deployed in commercial buildings. Traffic events create ground vibration that propagates to nearby building structures inducing noisy vibration responses. We present an information-theoretic method for train event monitoring using commonly existing vibration sensors deployed for building health monitoring. The key idea is to represent the wave propagation in a building induced by train traffic as information conveyed in noisy measurement signals. Our technique first uses wavelet analysis to detect train events. Then, by analyzing information exchange patterns of building vibration signals, we infer the category of the events (i.e., southbound or northbound train). Our algorithm is evaluated with an 11-story building where trains pass by frequently. The results show that the method can robustly achieve a train event detection accuracy of up to a 93% true positive rate and an 80% true negative rate. For direction categorization, compared with the traditional signal processing method, our information-theoretic approach reduces categorization error from 32.1 to 12.1%, which is a 2.5× improvement.
