No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
FreeCount: Device-Free Crowd Counting with Commodity WiFi
... The existing WiFi-based ICC methods can be mainly divided into two categories, depending on the need of manual feature extraction [6]- [13]. One relies on explicit manual features engineered from raw data, such as mean value, variance, median, and range, and then uses threshold-based or learningbased classifiers like support vector machine (SVM) to identify the crowd number. ...
... The performance of the former method is critically related to the data feature selection. For example, to select the best-performing features, Zou et al. [6] proposed a "Transfer Kernel Learning (TKL)" method that selects data features based on a mutual information criterion from a feature pool including several statistical, transformation-based, and shape-based features. The performance of the latter method highly depends on iteratively training with a large number of labeled samples. ...
... Although human activity causes fluctuation of both amplitude and phase, many studies only use the amplitude information for ICC (like in [6], [11], [12]), mainly because the phase information often suffers from more severe hardware measurement noise such as carrier frequency offset and sampling time offset [21]- [23]. Instead of using raw phase measurements, Zong et al. [13] computed the phase difference between adjacent antennas as the input to an SVM-based ICC classifier. ...
Accurate indoor crowd counting (ICC) is a key enabler to many smart home/office applications. In this paper, we propose a Domain-Agnostic and Sample-Efficient wireless indoor crowd Counting (DASECount) framework that suffices to attain robust cross-domain detection accuracy given very limited data samples in new domains. DASECount leverages the wisdom of few-shot learning (FSL) paradigm consisting of two major stages: source domain meta training and target domain meta testing. Specifically, in the meta-training stage, we design and train two separate convolutional neural network (CNN) modules on the source domain dataset to fully capture the implicit amplitude and phase features of CSI measurements related to human activities. A subsequent knowledge distillation procedure is designed to iteratively update the CNN parameters for better generalization performance. In the meta-testing stage, we use the partial CNN modules to extract low-dimension features out of the high-dimension input target domain CSI data. With the obtained low-dimension CSI features, we can even use very few shots of target domain data samples (e.g., 5-shot samples) to train a lightweight logistic regression (LR) classifier, and attain very high cross-domain ICC accuracy. Experiment results show that the proposed DASECount method achieves over 92.68\%, and on average 96.37\% detection accuracy in a 0-8 people counting task under various domain setups, which significantly outperforms the other representative benchmark methods considered.
... However, these approaches require deploying specific sensors in the interest region that cannot be blocked. General WiFi sensors-based [22][23][24][25] methods can overcome the above drawbacks, but they need people to keep moving. Therefore, they cannot detect static crowds, for instance, passengers in the elevator. ...
... In radio communications, the OFDM systems are widely used to divide the wireless network spectrum into orthogonal sub-carriers. However, the emitted WiFi signals often do not reach the receive antenna directly due to obstacle blocking and multi-path effects of signal propagation [23]. A portion of the signals passes through the medium, while other signals are reflected or absorbed by the medium. ...
... For the M1 [25] and M2 [22] methods, about 50% of the results are accurate due to the RSSI-based schemes performing poorly in multi-path complex environments and scene migration. FreeCount [23] (M3) can achieve more than 60% accuracy because CSI is more sensitive to the diversity of transmission channels than RSSI. Due to M4 [24] not only estimating the number of people but also providing human dynamics monitoring through participant number estimation, its F1-score reaches more than 80%. ...
Elevators have become a kind of indispensable facility for everyday life, which bring people both convenience and safety hazards. Specifically in the household environment, an elevator’s lifespan is expected to be more than 20 years. An appropriate and regularly maintained counterweight is conducive to extending elevator life. This paper proposes a passenger counting approach in the elevator for regular counterweight adjustment based on commodity WiFi called ECC. Since the running time of the elevator between two adjacent floors is short, the major challenge of ECC is how to count passengers from the limited captured data. This paper first theoretically analyzes the relationship between the number of passengers and the variation of channel state information (CSI). Then ECC constructs a multi-dimensional feature by extracting the average of amplitude (AOA), time-varying spectrum (TVS), and percentage of non-zero elements (PEM) features from the limited data. Finally, the random forest (RF) classifier is used for passenger counting and the local optimization problem is solved by expanding the feature dataset through data segmentation. ECC is implemented by using off-the-shelf IEEE 802.11n devices, and its performance is evaluated via extensive experiments in typical real-world scenes. The estimated precision of ECC can reach more than 95%, and more than 97% of estimation errors are less than 2 persons, which demonstrates the superior effectiveness and generalizability of ECC.
... Wi-Fi derived occupant counts have been shown to have strong correlation with actual ground truth counts where R 2 values ranging from 0.85-0.96 have been reported in the literature [25,[28][29][30] as well as Wi-Fi device to occupant ratio of 1.27 [31] and 1.2 [32]. Wi-Fi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [16]. ...
... In solving the occupancy estimation problem, [23] achieved 98.85% occupancy detection accuracy, 0.096 Normalized Root Mean Square Derivative occupancy count accuracy and 1.385 m occupancy tracking accuracy using their framework that utilized Received Signal Strength (RSS) levels and device Media Access Control (MAC) addresses. [30] achieved 90% accuracy using Channel State Information (CSI) to infer occupant counts, where the sensitivity of transmitted radio signals to occupant movements was captured. Their approach eliminated the need for occupant terminal devices such as smartphones. ...
... There is the possibility of overestimated occupant counts, which is critical when determining ( ) and ( ) based on, as overestimated occupant counts will result in much earlier ( ) or later ( ) and diminished energy savings. In addition to the preprocessing steps we take in Section 4.2.1, another remedy that has been used in literature is to calibrate device counts against ground truth occupancy [28][29][30] however, this approach assumes static ratio of device counts to actual occupancy counts. Park et al. investigated the use of Capture-Recapture, a methodology adopted from the field of zoology, to infer the spatial and temporal variation in Wi-Fi device count to actual occupancy count ratio as an alternative to a statically determined ratio [57]. ...
HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an Energy Conservation Measure (ECM) where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building's real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from ECMs that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived Heating, Ventilation and Air Conditioning (HVAC) schedules and energy savings that leverages the ubiquity of energy smart meters and Wi-Fi infrastructure in commercial buildings. We estimate the schedules by clustering Wi-Fi derived occupancy profiles and, estimate energy savings by shifting ramp-up and setback times observed in typical operational/static load profiles that are obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%-10.8% (summer) and 0.2%-5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from Building Energy Performance Simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%-2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%-5%.
... WiFi derived occupant counts have been shown to have strong correlation with actual ground truth counts where R 2 values ranging from 0.85 -0.96 have been reported in the literature [23,24,25,21] as well as a strong occupant to device ratio of 1.27 [26]. WiFi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [13]. ...
... In solving the occupancy estimation problem, [19] achieved 98.85% occupancy detection accuracy, 0.096 Normalized Root Mean Square Derivative occupancy count accuracy and 1.385 m occupancy tracking accuracy using their framework that utilized Received Signal Strength (RSS) levels and device MAC addresses. [25] achieved 90% accuracy using Channel State Information (CSI) to infer occupant counts where the sensitivity of transmitted radio signals to occupant movements was captured. Their approach eliminated the need for occupant terminal devices such as smartphones. ...
... There is the possibility of overestimated occupant counts which is critical when determining ( ) and ( ) based on as overestimated occupant counts will result in much earlier ( ) or later ( ) and diminished energy savings. A remedy that has been used in literature is to calibrate device counts against ground truth occupancy [23,24,25] however, this approach assumes static ratio of device counts to actual occupancy counts. Park et al. investigated the use of Capture-Recapture, a methodology adopted from the field of zoology, to infer the spatial and temporal variation in WiFi device count to actual occupancy count ratio as an alternative to a statically determined ratio. ...
HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an energy conservation measure where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building's real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from energy conservation measures that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived HVAC schedules and energy savings that leverages the ubiquity of energy smart meters and WiFi infrastructure in commercial buildings. We estimate the schedules by clustering WiFi-derived occupancy profiles and, energy savings by shifting ramp-up and setback times observed in typical/measured load profiles obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%-10.8% (summer) and 0.2%-5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from building energy performance simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%-2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%-5%.
... By analysing the patterns of its wireless signal, today's AP has evolved beyond a pure WiFi router, but is also widely used as a type of 'sensor device' to enable new services for human sensing. Particularly, recent studies have found that WiFi signals in the form of Channel State Information (CSI) [1], [2] are extremely promising for a variety of devicefree human sensing tasks, such as occupancy detection [3], activity recognition [4], [5], [6], [7], fall detection [8], gesture recognition [9], [10], human identification [11], [12], and people counting [13], [14]. Unlike the coarse-grained received signal strengths, WiFi CSI records more fine-grained information about how a signal propagates between WiFi J. Yang, X. Chen, D. Wang, H. Zou devices and how a signal is reflected from the surrounding environment in which humans move around. ...
... In contrast, by feeding a massive amount of data into machine learning [22] or deep learning networks, [9], [5], learning based achieve remarkable performances in complicated sensing tasks. Various deep neural networks are designed to enable many applications including activity recognition [23], gesture recognition [9], human identification [11], [12], [24], and people counting [13], [14]. Though deep learning models have a strong ability of function approximation, they require tremendous labeled data that is expensive to collect and suffer from the negative effect of distribution shift caused by environmental dynamics [25]. ...
WiFi sensing has been evolving rapidly in recent years. Empowered by propagation models and deep learning methods, many challenging applications are realized such as WiFi-based human activity recognition and gesture recognition. However, in contrast to deep learning for visual recognition and natural language processing, no sufficiently comprehensive public benchmark exists. In this paper, we review the recent progress on deep learning enabled WiFi sensing, and then propose a benchmark, SenseFi, to study the effectiveness of various deep learning models for WiFi sensing. These advanced models are compared in terms of distinct sensing tasks, WiFi platforms, recognition accuracy, model size, computational complexity, feature transferability, and adaptability of unsupervised learning. It is also regarded as a tutorial for deep learning based WiFi sensing, starting from CSI hardware platform to sensing algorithms. The extensive experiments provide us with experiences in deep model design, learning strategy skills and training techniques for real-world applications. To the best of our knowledge, this is the first benchmark with an open-source library for deep learning in WiFi sensing research. The benchmark codes are available at https://github.com/xyanchen/WiFi-CSI-Sensing-Benchmark.
... This sensing methodology is advantageous because it has the tendency to be non-intrusive and low-cost by leveraging existing wireless network infrastructure such as WiFi routers, Bluetooth beacons and mobile devices to passively infer occupant information. WiFi derived occupant counts have been shown to have a strong correlation with actual ground truth counts, where R 2 values ranging from 0.85 to 0.96 have been reported in the literature [14,[18][19][20]. WiFi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [8]. ...
... While these methods tackle overestimated counts problem, under counting remains an issue and are associated with privacy risks. To remedy the problems originating from the need of a device-carrying occupants, researchers have utilized Channel State Information (CSI), where the effect of occupant movement on transmitted WiFi signals is used to derive occupancy information [20,27,28]. Demrozi et al. proposed using Bluetooth Low Energy (BLE) as a cheaper but comparably accurate alternative to CSI technology for a device-free occupant detection and counting system [29]. ...
The occupancy information in buildings is fundamental for smart buildings (e.g., occupant-centric controls). Opportunistic occupancy detection (OOD) uses connection data of mobile devices. While OOD has been developed and applied, one critical drawback is that it requires the ground truth of occupants to calibrate, which is limited to gather. Here, we introduce CROOD: a capture and recapture (CRc) inspired OOD. In ecology, CRc has been established for the estimation of animal populations, when the manual count is impossible. We adopt this unique approach to estimate the number of mobile devices in a building. Then, using a simple estimate on the total population, CROOD determines the relationship between the numbers of occupants and mobile devices. We evaluate CROOD numerically on the synthetic building populations and demonstrate its application in a university library using WiFi connection data. We find that CROOD can estimate the number of mobile devices and subsequently the number of occupants with 1–2 weeks to converge a reasonable accuracy. A long term experiment shows that CROOD can adapt to varying population characteristics (e.g., occupants bring more mobile devices), outperforming the reference sample mean estimator. The real building implementation demonstrates that while in the first 1–2 weeks, the sample mean estimator is superior, eventually CROOD adapts and provides better estimates without ground-truth calibration. Although CROOD has a limitation of building types and systems, our results envision that CROOD could be a viable addition to other OOD methods to better utilize existing mobile device connection data to estimate occupancy in buildings.
... To assess the effectiveness of the proposed method, we compare it with various crowd counting methods and fusion methods. The baselines include WiFi-based methods (FreeCount [24], WiCount [25]) and multimodal fusion methods, including Early Fusion LSTM (EF-LSTM) [26], Late Fusion LSTM (LF-LSTM) [26], Low-rank Multimodal Fusion method (LMF) [27], and Transformer Routing (TRAR) [28]. Table I compares the performance of our proposed Trans-Fusion model with various other models, including WiFibased models (FreeCount and WiCount) and multimodal fusion models (EF-LSTM, LF-LSTM, LMF, and TRAR). ...
Current crowd-counting models often rely on single-modal inputs, such as visual images or wireless signal data, which can result in significant information loss and suboptimal recognition performance. To address these shortcomings, we propose TransFusion, a novel multimodal fusion-based crowd- counting model that integrates Channel State Information (CSI) with image data. By leveraging the powerful capabilities of Transformer networks, TransFusion effectively combines these two distinct data modalities, enabling the capture of comprehen- sive global contextual information that is critical for accurate crowd estimation. However, while transformers are well capable of capturing global features, they potentially fail to identify finer- grained, local details essential for precise crowd counting. To mitigate this, we incorporate Convolutional Neural Networks (CNNs) into the model architecture, enhancing its ability to extract detailed local features that complement the global context provided by the Transformer. Extensive experimental evaluations demonstrate that TransFusion achieves high accuracy with minimal counting errors while maintaining superior efficiency.
... This level of precision satisfies the requirements of most crowd-aware applications, accommodating varying crowd sizes within WiFi-covered areas. The paper [180] introduces FreeCount, a device-free crowd-counting method using WiFi routers. A novel OpenWrt firmware is used to capture router CSI data and enable accurate estimation of occupants using two routers. ...
The rapid advancements in wireless technologies have led to numerous research studies that provide evidence of the successful utilization of wireless signals, particularly, WiFi signals for human activity sensing in the indoor environment. As a promising technology, WiFi-based human sensing can be utilized for a variety of applications such as smart healthcare, smart homes, security, industry, office indoor environments etc., due to the availability of rich infrastructure. Furthermore, compared to other radio frequency (RF) based systems such as radio detection and ranging (RADAR) and radio frequency identification (RFID), WiFi is non-invasive, has low-cost, and provides ubiquitous coverage in the indoor setup. However, due to the limited accuracy and high complexity of the model-based approaches for human sensing, the systems empowered by the deep learning (DL) techniques have achieved remarkable performance gains and showed more robustness in dealing with complicated human sensing tasks. The article explores the physical layer parameters used in WiFi sensing such as received signal strength indicator (RSSI) and channel state information (CSI), the estimated parameters such as angle-of-arrival (AoA) and Doppler shift (DS) along with frequency modulated continuous wave (FMCW) RADAR technology. Moreover, the preliminary signal processing stages that are applied to the received WiFi signals in the DL assisted systems are discussed. This article provides a comprehensive literature survey on the recent advances in DL-empowered WiFi sensing focusing on human activity recognition and movement tracking followed by fall detection, single task-multi task classification, crowd monitoring and sensing, indoor localization, gaits recognition, and pose estimation. Furthermore, the paper highlight the challenges in the existing literature and discusses the possible future research directions in WiFi-based human sensing assisted by DL techniques.
... This task has garnered significant attention in recent years, with extensive research and implementation across various real-world scenarios, including smart buildings (Zou et al. 2018), traffic monitoring (Marsden et al. 2018;Zhang et al. 2017), and public spaces in Saudi Arabia (Alotibi et al. 2019). By harnessing real-time image data, crowd counting facilitates applications such as video surveillance (Wang, Hou, and Chau 2019), enhanced security (Chan, John Liang, and Vasconcelos 2008), and efficient bandwidth allocation (Zou et al. 2017). ...
Crowd counting, a crucial computer vision task, aims at estimating the number of individuals in various environments. Each person in crowd counting datasets is typically annotated by a point at the center of the head. However, challenges like dense crowds, diverse scenarios, significant obscuration, and low resolution lead to inevitable label noise, adversely impacting model performance. Driven by the need to enhance model robustness in noisy environments and improve accuracy, we propose the Loss Filtering Factor (LFF) and the corresponding Label Noise Robust Crowd Counting (LNRCC) training scheme. LFF innovatively filters out losses caused by label noise during training, enabling models to focus on accurate data, thereby increasing reliability. Our extensive experiments demonstrate the effectiveness of LNRCC, which consistently improves performance across all models and datasets, with an average enhancement of 3.68% in Mean Absolute Error (MAE), 6.7% in Mean Squared Error (MSE) and 4.68% in Grid Average Mean Absolute Error (GAME). The universal applicability of this approach, coupled with its ease of integration into any neural network model architecture, marks a significant advancement in the field of computer vision, particularly in addressing the pivotal issue of accuracy in crowd counting under challenging conditions.
... However, carrying out multi-object tracking using CSI is challenging due to the complexity of overlapped signals in temporal and spectral domains. Additionally, environmental monitoring techniques such as crowd counting [29] and intrusion detection [30] are popular with CSI-based methods, but they require a room to be equipped with both a transmitter (TX) and a receiver (RX), which is costly and impractical. ...
Device-free human presence detection is a crucial technology for various applications, including home automation, security, and healthcare. While camera-based systems have traditionally been used for this purpose, they raise privacy concerns. To address this issue, recent research has explored the use of wireless channel state information (CSI) extracted from commercial WiFi access points (APs) to provide detailed channel characteristics. In this paper, we propose a device-free human presence detection system for multi-room scenarios using a time-selective conditional dual feature extract recurrent network (TCD-FERN). Our system is designed to capture significant time features on current human features using a dynamic and static data preprocessing technique.We extract both moving and spatial features of people and differentiate between line-of-sight (LoS) and non-line-of-sight (NLoS) cases. Subcarrier fusion is carried out in order to provide more objective variation of each sample while reducing the computational complexity. A voting scheme is further adopted to mitigate the feature attenuation problem caused by room partitions, with around 3% improvement of human presence detection accuracy. Experimental results have revealed the significant improvement of leveraging subcarrier fusion, dual-feature recurrent network, time selection and condition mechanisms. Compared to the existing works in open literature, our proposed TCD-FERN system can achieve above 97% of human presence detection accuracy for multi-room scenarios with the adoption of fewer WiFi APs.
... This information can be used to allocate resources and ensure public safety. • Shopping Malls Management: Shopping malls and retail stores can use crowd counting to analyze foot traffic and customer behavior [25], [26]. This can help store managers optimize store layout, staffing, and marketing strategies. ...
p>In machine vision, the tasks of both the crowd counting and the crowd density estimation attracted a number of researchers from different research institutes throughout the world. These tasks are of significance importance while analyzing the surveillance videos and images both at the real and offline modes. A number of different conventional and modern algorithms are being applied to estimate the accurate amount of people from the crowd from the target image or video file but failed to provide the required results due to heterogeneity of the data instances. To resolve the issue, machine learning models like deep networks like Convolution Neural Network (CNN) and its variants were introduced. In this paper, a detailed and comprehensive review of the work related to both of the crowd counting and density estimation using deep network is provided in quite a very unique way to make it useful for the researchers working in the field of computer vision. Following are the salient features of our review that play key role in making our contribution worthwhile and unique among the surveys and reviews of similar kind: (i) our survey intends to classify the noteworthy literature with respect to the application and the tasks related to crowd counting and density estimation, (ii) the survey also tried to cover in-depth contributions in crowd counting existing works at different level like images or videos (iii) this state-of-the-art review covers each article in the following dimensions: the designated task performed, source level, results obtained, features used, and (iv) lastly it concludes the summary of the related articles according to the publishing years, related tasks (or subtasks), and types of classifiers used. In the end, major challenges and tasks related to crowd counting and density estimation in computer vision are also discussed.</p
... This information can be used to allocate resources and ensure public safety. • Shopping Malls Management: Shopping malls and retail stores can use crowd counting to analyze foot traffic and customer behavior [25], [26]. This can help store managers optimize store layout, staffing, and marketing strategies. ...
In machine vision, the tasks of both the crowd counting and the crowd density estimation attracted a number of researchers from different research institutes throughout the world. These tasks are of significance importance while analyzing the surveillance videos and images both at the real and offline modes. A number of different conventional and modern algorithms are being applied to estimate the accurate amount of people from the crowd from the target image or video file but failed to provide the required results due to heterogeneity of the data instances. To resolve the issue, machine learning models like deep networks like Convolution Neural Network (CNN) and its variants were introduced. In this paper, a detailed and comprehensive review of the work related to both of the crowd counting and density estimation using deep network is provided in quite a very unique way to make it useful for the researchers working in the field of computer vision. Following are the salient features of our review that play key role in making our contribution worthwhile and unique among the surveys and reviews of similar kind: (i) our survey intends to classify the noteworthy literature with respect to the application and the tasks related to crowd counting and density estimation, (ii) the survey also tried to cover in-depth contributions in crowd counting existing works at different level like images or videos (iii) this state-of-the-art review covers each article in the following dimensions: the designated task performed, source level, results obtained, features used, and (iv) lastly it concludes the summary of the related articles according to the publishing years, related tasks (or subtasks), and types of classifiers used. In the end, major challenges and tasks related to crowd counting and density estimation in computer vision are also discussed.
... However, carrying out multi-object tracking using CSI is challenging due to the complexity of overlapped signals in the time and frequency domains. Additionally, environmental monitoring techniques such as crowd counting [20] and intrusion detection [21] are popular with CSI-based methods, but they require a room to be equipped with both a transmitter (TX) and a receiver (RX), which is costly and impractical. Presence detection, which infers the presence of people in indoor environments based on the variation of received signals [22], [23], faces the challenge of false detection caused by a range of events that result in time-varying signals. ...
Human presence detection is a crucial technology for various applications, including home automation, security, and healthcare. While camera-based systems have traditionally been used for this purpose, they raise privacy concerns. To address this issue, recent research has explored the use of channel state information (CSI) approaches that can be extracted from commercial WiFi access points (APs) and provide detailed channel characteristics. In this thesis, we propose a device-free human presence detection system for multi-room scenarios using a time-selective conditional dual feature extract recurrent Network (TCD-FERN). Our system is designed to capture significant time features with the condition on current human features using a dynamic and static (DaS) data preprocessing technique to extract moving and spatial features of people and differentiate between line-of-sight (LoS) path blocking and non-blocking cases. To mitigate the feature attenuation problem caused by room partitions, we employ a voting scheme. We conduct evaluation and real-time experiments to demonstrate that our proposed TCD-FERN system can achieve human presence detection for multi-room scenarios using fewer commodity WiFi APs.
... This subsection shows applied AI-based optimization algorithms for density estimation in crowd detection and localization approaches. FreeCount is a device-free method of crowd counting proposed by Zou et al. [61] that relies solely on common WiFi routers to calculate the precise population density of an area. In areas of average size, FreeCount can operate with just two routers. ...
The Internet of Things (IoT) provides a collaborative infrastructure to communicate smart devices with cloud-edge healthcare applications, medical devices, wearable biosensors, etc. On the other hand, crowd counting as one of computer vision approaches is an emerging topic to detect any objects with static or dynamic mobility in the IoT environments. Smart crowd counting enables pattern recognition for many intelligent applications such as microbiology, surveillance, healthcare systems, crowdedness estimation, and other environmental case studies. According to complicated capturing systems in the IoT environments, crowd counting methods can influence on performance of object detection in the critical case studies using Artificial Intelligence (AI)-based approaches such as machine learning, deep learning, collaborative learning, fuzzy logic and meta-heuristic algorithms. This paper provides a new comprehensive technical analysis for existing AI-based crowd counting approaches in healthcare and medical systems, biotechnology and IoT environments. Meanwhile, it presents a discussion on the existing case studies with respect to analyzing technical aspects and applied algorithms to enhance pattern prediction factors. Finally, some new innovative efforts and challenges are presented for new research upcoming and open issues.
... By analyzing the patterns of its wireless signals, current-day APs have evolved beyond being pure WiFi routers and also serve as a type of ''sensor device'' to enable new services for human sensing. In particular, recent studies have pointed out that WiFi signals in the form of channel state information (CSI) 1,2 are extremely promising for a variety of device-free human-sensing tasks, such as occupancy detection, 3 activity recognition, [4][5][6][7] fall detection, 8 gesture recognition, 9,10 human identification, [11][12][13] people counting, 14,15 and pose estimation. 16 Unlike coarsegrained received signal strengths, WiFi CSI records more finegrained information in terms of the propagation of a signal between WiFi devices and their reflection with respect to the environment of human beings. ...
Over the recent years, WiFi sensing has been rapidly developed for privacy-preserving, ubiquitous human-sensing applications, enabled by signal processing and deep-learning methods. However, a comprehensive public benchmark for deep learning in WiFi sensing, similar to that available for visual recognition, does not yet exist. In this article, we review recent progress in topics ranging from WiFi hardware platforms to sensing algorithms and propose a new library with a comprehensive benchmark, SenseFi. On this basis, we evaluate various deep-learning models in terms of distinct sensing tasks, WiFi platforms, recognition accuracy, model size, computational complexity, and feature transferability. Extensive experiments are performed whose results provide valuable insights into model design, learning strategy, and training techniques for real-world applications. In summary, SenseFi is a comprehensive benchmark with an open-source library for deep learning in WiFi sensing research that offers researchers a convenient tool to validate learning-based WiFi-sensing methods on multiple datasets and platforms.
... The WiFi-based gait recognition method uses RF signals from WiFi-enabled devices to determine human identity. The transmitter emits WiFi signals, which are reflected by different body parts of the walking subject and then recorded by CSI data at the receiver [19], which has empowered many applications including occupancy detection [20], crowd counting [21], [22], human activity recognition [23], [24], [25], [26], [27], person identification [8], [28], vital sign detection [29], pose estimation [30] and gesture recognition [31], [32], [33]. To use WiFi sensing in the real world, current research aims at efficient communication [16], model security [34] and dataefficient training [35]. ...
As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.
... The authors show the percentage of non-zero elements (PEM) metric and the cumulative density function (CDF) of error estimation for scenarios with different speeds. FreeCount is a crowd counting scheme using WiFi routers presented in [26]. Their model presented high precision for indoors in rooms with moderate size. ...
Currently, there is a requirement in many countries to keep public and work spaces safe due to COVID-19. In fact, indoor spaces must be monitored to control the allowed capacity, which can vary depending on the alert level of a city at a given time. This has motivated some researchers to investigate several technologies to implement methods and strategies to enable the reopening of these spaces in a safe manner. In this paper, we propose a crowd counting detection system that addresses the problem of controlling the indoor capacity of offices inside buildings. The proposed solution uses an existing communication technology such as WiFi in order to determine the crowd counting for the indoor environment. In particular, the existing infrastructure consists of two Wireless LAN Controllers (WLC) and several APs deployed in a building, which allows us to estimate the number of people based on the access to Wireless Access Points (APs). Thus, the proposed system takes into account when a mobile device connects/disconnects to the AP to increase or decrease the number of people in a particular office and sends the respective alert to the system administrator when this capacity is about to be exceeded or already surpassed.
... Previous researches have shown that human pose can affect the propagation of WiFi signals and these signal variations are reflected by Channel State Information (CSI). Enabled by advanced deep learning methods, many studies have been conducted to enable various applications, such as human activity recognition [5]- [7], gesture recognition [8], [9], human identification [10], [11], and people counting [12], [13]. Many advanced learning methods further promote the automation and performance of these tasks, such as transfer learning [14], [15], unsupervised learning [16], and modelrobust learning [17]. ...
Avatar refers to a representative of a physical user in the virtual world that can engage in different activities and interact with other objects in metaverse. Simulating the avatar requires accurate human pose estimation. Though camera-based solutions yield remarkable performance, they encounter the privacy issue and degraded performance caused by varying illumination, especially in smart home. In this paper, we propose a WiFi-based IoT-enabled human pose estimation scheme for metaverse avatar simulation, namely MetaFi. Specifically, a deep neural network is designed with customized convolutional layers and residual blocks to map the channel state information to human pose landmarks. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. WiFi is ubiquitous and robust to illumination, making it a feasible solution for avatar applications in smart home. The experiments are conducted in the real world, and the results show that the MetaFi achieves very high performance with a PCK@50 of 95.23%.
... Counting the number of occupants is essential for building monitoring and management. The use of WPS for counting the number of occupants can help control people in specific places and monitor their entry and exit [30,[50][51][52][53]. In addition, WPS can be a good alternative for counting people inside the building, such as shopping centers, airports, and hospitals [54]. ...
Contact tracing is one of the critical tools for fighting against pandemic disease outbreaks, such as the fast-growing SARS-CoV-2 virus and its different variants. At present, automated contact tracing systems face two main challenges: (1) requiring application installation on smart devices and (2) protecting the users’ privacy. This study introduces a conceptual passive contact tracing system using indoor WiFi positioning to address these challenges and investigate the role of such a system in commercial buildings. In this regard, this study uses a simulated small-office layout in a case study to demonstrate the applicability of the proposed system. The special use of the proposed contact tracing system could be academic facilities and office buildings, where (1) the WiFi infrastructure already exists and therefore implementing such a system could be cost-effective, and (2) the same users use the facility regularly, enabling the system to notify the users upon a confirmed case once they are back in the building and connected to the WiFi system. Such technology can not only enhance the current automated contact tracing system in commercial buildings by illuminating the need to use smartphone applications while protecting users’ privacy, but could also reduce the risk of infection in indoor environments. The developed system can benefit facility managers, business owners, policy makers, and authorities in assisting to find occupants’ high-risk contacts and control the spread of SARS-CoV-2 or similar infectious diseases in commercial buildings, particularly university campuses and office buildings.
... Alternatively, at the physical layer, Channel State Information (CSI) describes how WiFi signal propagates from a transmitter (TX) to a receiver (RX) through multiple paths at the granularity of Frequency Division Multiplexing (OFDM) subcarriers, which is more sensitive to the presence and movements of an object and is more robust to background noise. Recent literature has witnessed many successful employments of CSI measurements for various applications, such as crowd counting (Zou et al. 2017b) and human ac-The Thirty-Second AAAI Conference on Artificial Intelligence tivity recognition (Wang et al. 2015). Being an off-the-shelf and fine-grained sensing measurement without the introduction of any extra infrastructure or user involvement, CSI data is the ideal sensing recourse for device-free and low-cost human identification. ...
We propose AutoID, a human identification system that leverages the measurements from existing WiFi-enabled Internet of Things (IoT) devices and produces the identity estimation via a novel sparse representation learning technique. The key idea is to use the unique fine-grained gait patterns of each person revealed from the WiFi Channel State Information (CSI) measurements, technically referred to as shapelet signatures, as the "fingerprint" for human identification. For this purpose, a novel OpenWrt-based IoT platform is designed to collect CSI data from commercial IoT devices. More importantly, we propose a new optimization-based shapelet learning framework for tensors, namely Convex Clustered Concurrent Shapelet Learning (C3SL), which formulates the learning problem as a convex optimization. The global solution of C3SL can be obtained efficiently with a generalized gradient-based algorithm, and the three concurrent regularization terms reveal the inter-dependence and the clustering effect of the CSI tensor data. Extensive experiments are conducted in multiple real-world indoor environments, showing that AutoID achieves an average human identification accuracy of 91% from a group of 20 people. As a combination of novel sensing and learning platform, AutoID attains substantial progress towards a more accurate, cost-effective and sustainable human identification system for pervasive implementations.
... Then the E-eyes system is developed to achieve better performance by dividing human activities into in-place and dynamic ones [23]. The FreeCount system leverages a feature selection scheme based on information theory to conduct people counting [24]. These early-stage works show good performance on normal activities such as walking and sitting, but they cannot identify fine-grained subtle gestures. ...
WiFi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by Channel State Information (CSI) extracted from WiFi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this paper, we firstly explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an automatic WiFi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in WiFi sensing. The AutoFi is implemented on a pair of Atheros WiFi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public datasets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar datasets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that the AutoFi takes a huge step toward automatic WiFi sensing without any developer engagement while overcoming the cross-site issue.
... Zou et al. [25] proposed FreeCount, which is a device-free crowd counting scheme using a modified CSI tool running on commercial WiFi devices. They adopted the transfer kernel learning (TKL) model to take account of temporal variation of CSI measurements, and trained the model with 20 features based on de-noised CSI data by wavelet filter, which are categorized in common statistics, transformationbased, and shape-based features. ...
Wireless sensing represented by WiFi channel state information (CSI) is now enabling various fields of applications such as person identification, human activity recognition, occupancy detection, localization, and crowd estimation these days. So far, those fields are mostly considered as separate topics in WiFi CSI-based methods, on the contrary, some camera and vision-based crowd estimation systems intuitively estimate both crowd size and location at the same time. Our work is inspired by the idea that WiFi CSI also may be able to perform the same as the camera does. In this paper, we construct
Wi-CaL
, a simultaneous crowd counting and localization system by using ESP32 modules for WiFi links. We extract several features that contribute to dynamic state (moving crowd) and static state (location of the crowd) from the CSI bundles, then assess our system by both conventional machine learning (ML) and deep learning (DL). As a result of ML-based evaluation, we achieved 0.35 median absolute error (MAE) of counting and 91.4% of localization accuracy with five people in a small-sized room, and 0.41 MAE of counting and 98.1% of localization accuracy with 10 people in a medium-sized room, by leave-one-session-out cross-validation. We compared our result with percentage of non-zero elements metric (PEM), which is a state-of-the-art metric for crowd counting, and confirmed that our system shows higher performance (0.41 MAE, 81.8% of within-1-person error) than PEM (0.62 MAE, 66.5% of within-1-person error).
WiFi-based pose estimation is a technology with great potential for the development of smart homes and metaverse avatar generation. However, current WiFi-based pose estimation methods are predominantly evaluated under controlled laboratory conditions with sophisticated vision models to acquire accurately labeled data. Furthermore, WiFi CSI is highly sensitive to environmental variables, and direct application of a pre-trained model to a new environment may yield suboptimal results due to domain shift. In this paper, we propose a domain adaptation algorithm, AdaPose, designed specifically for WiFi-based pose estimation. The proposed method aims to identify consistent human poses that are highly resistant to environmental dynamics and WiFi signal noises. To achieve this goal, we introduce ICAL that aligns domain shifts considering instance-wise pose distribution variance, and CECE module that enhances WiFi CSI feature representation by emphasizing channel-wise similarity between source and target domains. We conduct extensive experiments on both our self-collected pose estimation dataset and a large public MM-Fi dataset. The results demonstrate the effectiveness and robustness of AdaPose in eliminating domain shift, thereby facilitating the widespread application of WiFi-based pose estimation in smart cities.
In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high demand on effective pose estimation. Though camera-based solutions yield remarkable performance, they encounter privacy issues and degraded performance caused by varying illumination, especially in the smart home. In this article, we propose a WiFi-based Internet of Things-enabled human pose estimation scheme for metaverse avatar simulation, namely, MetaFi++. Specifically, WPFormer is designed with a shared convolutional module and a Transformer block to map the channel state information of WiFi signals to human pose landmarks, effectively exploring spatial information of human pose through self-attention. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. Due to the ubiquitous existence of WiFi and robustness to various illumination conditions, WiFi-based human poses are suitable to instruct the movement of digital avatars in the metaverse, promoting avatar applications in smart homes. The experiments are conducted in the real world, and the results show that the MetaFi++ achieves very high performance with a PCK@50 of 97.30%. Our codes are available in
https://github.com/pridy999/metafi_pose_estimation
.
Purpose: The main purpose of this study was to examine the application of sensors and sensor networks for detection of people crowds in developing cities. This paper discusses unique challenges associated with people crowd detection especially in the urban towns of developing countries and gives a comparative review and analysis of popular human sensing approaches in the detection of people crowds. Methodology: This study provides a survey and categorization of popular human sensing approaches using literature especially published within the past two decades. The paper then analyzes current human sensing technologies vis-à-vis people crowd detection in developing cities. The respective strengths and shortfalls of various approaches are highlighted. Finally, by means of examples, a comparative analysis of different human sensing categories is carried out. Findings: The spontaneous, dynamic and chaotic nature of people crowds, together with the poor infrastructural development characteristic of developing economies pose unique challenges to the effectiveness of people crowd detection systems. Although there are advances in crowd detection, most of these are in the area of non-people crowds, while most of the research done on people crowd detection have been on indoor crowd settings. In addition, challenges unique to people crowd detection in developing countries include: scalability and cost of crowd detection systems, security of the detection system infrastructure, confidentiality of subjects being monitored, requirements for incentives and the ability to support passive and real time people crowd detection. Unique contribution to theory, practice and policy: This study emphasizes the need for both indoor and outdoor people crowd detection systems appropriate for the needs of developing cities. The study contributes to the body of knowledge since people crowds unlike other types of crowds present a unique set of challenges that call for special attention.
Accurate indoor crowd counting (ICC) is a key enabler to many smart home/office applications. Recent development of WiFi-based ICC technology relies on detecting the variation of wireless channel state information (CSI) caused by human motions and has gained increasing popularity due to its low hardware cost, reliability under all lighting conditions, and privacy preservation in sensing data processing. To attain high estimation accuracy, existing WiFi-based ICC methods often require a large amount of labeled CSI training data samples for each application domain, i.e., a particular WiFi transceiver or background deployment. This makes large-scale deployment of WiFi-based ICC technology across dissimilar domains extremely difficult and costly. In this paper, we propose a Domain-Agnostic and Sample-Efficient wireless indoor crowd Counting (DASECount) framework that suffices to attain robust cross-domain detection accuracy given very limited data samples in new domains. DASECount leverages the wisdom of few-shot learning (FSL) paradigm consisting of two major stages: source domain meta training and target domain meta testing. Specifically, in the meta-training stage, we design and train two separate convolutional neural network (CNN) modules on the source domain dataset to fully capture the implicit amplitude and phase features of CSI measurements related to human activities. A subsequent knowledge distillation procedure is designed to iteratively update the CNN parameters for better generalization performance. In the meta-testing stage, we use the partial CNN modules to extract low-dimension features out of the high-dimension input target domain CSI data. With the obtained low-dimension CSI features, we can even use very few shots of target domain data samples (e.g., 5-shot samples) to train a lightweight logistic regression (LR) classifier, and attain very high cross-domain ICC accuracy. Experiment results show that the proposed DASECount method achieves over 92.68%, and on average 96.37% detection accuracy in a 0-8 people counting task under various domain setups, which significantly outperforms the other representative benchmark methods considered.
Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included in
https://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark
.
Device-free activity recognition plays a crucial role in smart building, security, and human–computer interaction, which shows its strength in its convenience and cost-efficiency. Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models, but it suffers from the limitation of manual feature design. Deep learning overcomes such issues by automatic high-level feature extraction, but its performance degrades due to the requirement of massive annotated data and cross-site issues. To deal with these problems, transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics. This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition. We begin with the motivation of deep learning and transfer learning, and then introduce the major sensor modalities. Then the deep and transfer learning techniques for device-free human activity recognition are introduced. Eventually, insights on existing works and grand challenges are summarized and presented to promote future research.
As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.
Recently, wireless sensing is gaining immense attention in the Internet of things (IoT) for crowd counting and occupancy detection. As wireless signals propagate, they tend to scatter and reflect in various directions depending on the number of people in the indoor environment. The combined effect of these variations on wireless signals is characterized by the channel state information (CSI), which can be further exploited to identify the presence of people. State-of-the-art CSI-based supervised crowd counting systems are vulnerable to temporal and environmental dynamics in practical scenarios as their performance degrades with fluctuations in the indoor environments due to multipath fading. Inspired by the breakthroughs of transfer learning and advancement in edge computing, we have leveraged in this work the concept of transfer learning to minimize this problem via exploiting the trained model from source environment for other indoor environments to perform device-free crowd counting (CrossCount) at the target rooms. Our results show that this technique can combat the dynamics of the environment and achieves 4.7% better accuracy with 40% reduction in training time as compared to conventional convolutional neural networks. In essence, our results imply the future possibility of harnessing crowdsourced CSI data collected at different indoor environments to boost the accuracy and efficiency of local crowd counting systems.
Existing channel-state information (CSI)-based human authentication systems in the literature require a large amount of CSI data to train deep neural network (DNN) models and are ineffective for unknown intruder detection. To address this issue, we propose a CSI-based human authentication system (CAUTION) which is able to learn distinctive gait features of different users through CSI data to perform human authentication in this article. By taking advantage of few-shot learning, CAUTION is able to construct an accurate user identification model with a very limited number of CSI training data. By converting the CSI samples into low-dimensional representations on the feature plane, it computes central points for different users as their CSI profiles and introduces an intruder threshold to measure whether the CSI data matches one of the user classes by a margin. The intruder threshold is able to be optimized without any intruders’ data. CAUTION does not require a large number of training data and provides an effective way to train the system for unknown intruder detection. We have tested CAUTION at different places and compared it with state-of-the-art CSI-based authentication systems. The experimental results demonstrate that CAUTION is able to perform accurate human authentication with a limited amount of CSI training data (one-fifth of data needed by compared systems) and outperforms the compared human authentication systems.
With the widespread of commercial communication equipment, WiFi signals are ubiquitous in human life. Therefore, utilizing WiFi signals to implement intelligent sensing applications is an inevitable trend. In WiFi sensing applications, through-the-wall crowd counting is a challenging problem. In the through-the-wall scenario, the wireless signal transmitted through the wall will carry a lot of noises and is severely attenuated. Therefore, the influence of human activities on the wireless signal is difficult to extract. To solve this problem, we propose TWCC, a through-the-wall crowd counting system using ambient WiFi signals. TWCC utilizes commercial WiFi equipments to extract the phase difference data of the channel state information (CSI) and transform it to sense the environment. First, TWCC preprocesses the data to remove uncorrelated noise, and then combines the subcarrier correlation to achieve through-the-wall human detection. When people exist, TWCC extracts features from four domains as feature groups, namely time domain, subcarrier domain, frequency domain, and time-frequency domain. Then TWCC uses different backpropagation (BP) neural networks for the features of the four domains and combines with weighting and threshold judgment to realize the through-the-wall crowd counting detection. Extensive real-world experiments show that TWCC achieves an average recognition accuracy of about 90% and maintains strong robustness to different speeds and environments.
Visible light sensing technique has shown great potential in locating human in the building. For indoor positioning using fingerprints, how to reduce the offline labor cost while maintaining accurate location estimation is of vital importance. In this paper, a novel coarse fingerprint-assisted device-free localization (DFL) approach is proposed for locating multiple targets via visible light sensing, which does not need too much calibration effort to build high accurate fingerprint database. Furthermore, a successive cancellation algorithm based on the coarse single target fingerprint database is designed to address the problem of pseudo target positioning in the multi-target DFL. Since the intersection points of shadowed links could not be guaranteed to be symmetrically located around the center of mass of the human body, the estimated location would be biased from the true location of the target. To solve this problem, a convex hull-based localization algorithm is introduced to improve positioning accuracy. Extensive simulation results show that our proposed approach exhibits superior performance in terms of target counting and localization accuracy.
Experiments show that operation efficiency and reliability of buildings can greatly benefit from rich and relevant datasets. More specifically, data can be analyzed to detect and diagnose system and component failures that undermine energy efficiency. Among the huge quantity of information, some features are more correlated with the failures than others. However there has been little research to date focusing on determining the types of data that can optimally support Fault Detection and Diagnosis (FDD). This paper presents a novel optimal feature selection method, named Information Greedy Feature Filter (IGFF), to select essential features that benefit building FDD. On one hand, the selection results can serve as reference for configuring sensors in the data collection stage, especially when the measurement resource is limited. On the other hand, with the most informative features selected by IGFF, the performance of building FDD could be improved and theoretically justified. A case study on Air Handling Unit (AHU) is conducted based on the dataset of the ASHRAE Research Project 1312. Numerical results show that, compared with several baselines, the FDD performances of conventional classification methods are greatly enhanced by IGFF.
Device-free localization without any device attached to the target is playing a critical role in many emerging applications. This paper presents an accurate model-based device-free localization system LiFS, implemented on cheap commercial off-the-shelf Wi-Fi devices. Unlike previous work, LiFS is able to localize a target accurately without offline training. The basic idea is simple: channel state information (CSI) is sensitive to a target's location and by modelling the CSI measurements of multiple wireless links as a set of power fading based equations, the target location can be determined. However, due to rich multipaths indoors, the received signal strength (RSS) or even the fine-grained CSI can not be easily modelled. We observe that even in a rich multipath environment, not all subcarriers are affected equally by multipaths. Our novel pre-processing scheme tries to identify the subcarriers not affected by multipath. Thus, CSIs on the identified ``clean'' subcarriers can be input into the proposed model for localization.
We design, implement and evaluate LiFS with extensive experiments in three different scenes. Without knowing the majority transceivers' locations, LiFS achieves a median accuracy of 0.5~m and 1.1~m in Line-of-Sight and Non-Line-of-Sight scenarios respectively, outperforming the state-of-the-art systems. Besides single target localization, LiFS is able to differentiate two sparsely-located targets and localize each of them at a high accuracy.
We propose Veto-Consensus Multiple Kernel Learning (VCMKL), a novel way of combining multiple kernels such that one class of samples is described by the logical intersection (consensus) of base kernelized decision rules, whereas the other classes by the union (veto) of their complements. The proposed configuration is a natural fit for domain description and learning with hidden subgroups. We first provide generalization risk bound in terms of the Rademacher complexity of the classifier, and then a large margin multi-ν learning objective with tunable training error bound is formulated. Seeing that the corresponding optimization is non-convex and existing methods severely suffer from local minima, we establish a new algorithm, namely Parametric Dual Descent Procedure (PDDP) that can approach global optimum with guarantees. The bases of PDDP are two theorems that reveal the global convexity and local explicitness of the parameterized dual optimum, for which a series of new techniques for parametric program have been developed. The proposed method is evaluated on extensive set of experiments, and the results show significant improvement over the state-of-the-art approaches.
Estimating an occupant’s location is arguably the most fundamental sensing task in smart buildings. The applications for fine-grained, responsive building operations require the location sensing systems to provide location estimates in real time, also known as indoor tracking. Existing indoor tracking systems require occupants to carry specialized devices or install programs on their smartphone to collect inertial sensing data. In this paper, we propose MapSentinel, which performs non-intrusive location sensing based on WiFi access points and ultrasonic sensors. MapSentinel combines the noisy sensor readings with the floormap information to estimate locations. One key observation supporting our work is that occupants exhibit distinctive motion characteristics at different locations on the floormap, e.g., constrained motion along the corridor or in the cubicle zones, and free movement in the open space. While extensive research has been performed on using a floormap as a tool to obtain correct walking trajectories without wall-crossings, there have been few attempts to incorporate the knowledge of space use available from the floormap into the location estimation. This paper argues that the knowledge of space use as an additional information source presents new opportunities for indoor tracking. The fusion of heterogeneous information is theoretically formulated within the Factor Graph framework, and the Context-Augmented Particle Filtering algorithm is developed to efficiently solve real-time walking trajectories. Our evaluation in a large office space shows that the MapSentinel can achieve accuracy improvement of 31 . 3 % compared with the purely WiFi-based tracking system.
The location and contextual status (indoor or outdoor) is fundamental and critical information for upper-layer applications, such as activity recognition and location-based services (LBS) for individuals. In addition, optimizations of building management systems (BMS), such as the pre-cooling or heating process of the air-conditioning system according to the human traffic entering or exiting a building, can utilize the information, as well. The emerging mobile devices, which are equipped with various sensors, become a feasible and flexible platform to perform indoor-outdoor (IO) detection. However, power-hungry sensors, such as GPS and WiFi, should be used with caution due to the constrained battery storage on mobile device. We propose BlueDetect: an accurate, fast response and energy-efficient scheme for IO detection and seamless LBS running on the mobile device based on the emerging low-power iBeacon technology. By leveraging the on-broad Bluetooth module and our proposed algorithms, BlueDetect provides a precise IO detection service that can turn on/off on-board power-hungry sensors smartly and automatically, optimize their performances and reduce the power consumption of mobile devices simultaneously. Moreover, seamless positioning and navigation services can be realized by it, especially in a semi-outdoor environment, which cannot be achieved by GPS or an indoor positioning system (IPS) easily. We prototype BlueDetect on Android mobile devices and evaluate its performance comprehensively. The experimental results have validated the superiority of BlueDetect in terms of IO detection accuracy, localization accuracy and energy consumption.
WiFi based Indoor Positioning System (IPS) has become the most popular and practical system to provide Location Based Service (LBS) in indoor environments due to the availability of massive existing WiFi network infrastructures in buildings. WiFi based IPSs leverage received signal strengths (RSSs) from large numbers of WiFi access points (APs) and estimate the location of clients. Proper AP selection methods are required to select a subset of APs that provide useful information for localization in order to reduce the computational load and preserve or even enhance the localization accuracy of the entire IPS. Existing approaches select and measure the discriminative ability of APs individually during the offline phase without the consideration of the dependence among them. In this paper, based on mutual information of APs, we present online mutual information (OnlineMI), a novel AP selection strategy that measures the collective discriminative ability among APs. Furthermore, the proposed AP selection process is conducted online to adapt various environmental dynamics. Weighted k nearest neighbor (WKNN) approach is further employed as the localization algorithm after the OnlineMI AP selection strategy. Extensive experiments are carried out, and performance evaluation and comparison with existing methods demonstrates the superiority of the proposed OnlineMI-WKNN approach.
Indoor Positioning System (IPS) has become one of the most attractive research fields due to the increasing demands on Location Based Services (LBSs) in indoor environments. Various IPSs have been developed under different circumstances, and most of them adopt the fingerprinting technique to mitigate pervasive indoor multipath effects. However, the performance of the fingerprinting technique severely suffers from device heterogeneity existing across commercial off-the-shelf mobile devices (e.g. smart phones, tablet computers, etc.) and indoor environmental changes (e.g. the number, distribution and activities of people, the placement of furniture, etc.). In this paper, we transform the Received Signal Strength (RSS) to a standardized location fingerprint based on the Procrustes analysis, and introduce a similarity metric, termed Signal Tendency Index (STI), for matching standardized fingerprints. An analysis on the capability of the proposed STI in handling device heterogeneity and environmental changes is presented. We further develop a robust and precise IPS by integrating the merits of both the STI and Weighted Extreme Learning Machine (WELM). Finally, extensive experiments are carried out and a performance comparison with existing solutions verifies the superiority of the proposed IPS in terms of robustness to device heterogeneity.
In this paper, we are interested in counting the total number of people walking in an area based on only WiFi received signal strength indicator (RSSI) measurements between a pair of stationary transmitter/receiver antennas. We propose a framework based on understanding two important ways that people leave their signature on the transmitted signal: blocking the line of sight (LOS) and scattering effects. By developing a simple motion model, we first mathematically characterize the impact of the crowd on blocking the LOS. We next probabilistically characterize the impact of the total number of people on the scattering effects and the resulting multipath fading component. By putting the two components together, we then develop a mathematical expression for the probability distribution of the received signal amplitude as a function of the total number of occupants, which will be the base for our estimation using Kullback-Leibler divergence. To confirm our framework, we run extensive indoor and outdoor experiments with up to and including nine people and show that the proposed framework can estimate the total number of people with a good accuracy with only a pair of WiFi cards and the corresponding RSSI measurements.
The increasing demands of location-based services have spurred the rapid development of indoor positioning system and indoor localization system interchangeably (IPSs). However, the performance of IPSs suffers from noisy measurements. In this paper, two kinds of robust extreme learning machines (RELMs), corresponding to the close-to-mean constraint, and the small-residual constraint, have been proposed to address the issue of noisy measurements in IPSs. Based on whether the feature mapping in extreme learning machine is explicit, we respectively provide random-hidden-nodes and kernelized formulations of RELMs by second order cone programming. Furthermore, the computation of the covariance in feature space is discussed. Simulations and real-world indoor localization experiments are extensively carried out and the results demonstrate that the proposed algorithms can not only improve the accuracy and repeatability, but also reduce the deviation and worst case error of IPSs compared with other baseline algorithms.
We present the results, experiences and lessons learned from comparing a diverse set of technical approaches to indoor localization during the 2014 Microsoft Indoor Localization Competition. 22 different solutions to indoor localization from different teams around the world were put to test in the same unfamiliar space over the course of 2 days, allowing us to directly compare the accuracy and overhead of various technologies. In this paper, we provide a detailed analysis of the evaluation study's results, discuss the current state-of-the-art in indoor localization, and highlight the areas that, based on our experience from organizing this event, need to be improved to enable the adoption of indoor location services.
Location-based services (LBS) have attracted a great deal of attention recently. Outdoor localization can be solved by the GPS technique, but how to accurately and efficiently localize pedestrians in indoor environments is still a challenging problem. Recent techniques based on WiFi or pedestrian dead reckoning (PDR) have several limiting problems, such as the variation of WiFi signals and the drift of PDR. An auxiliary tool for indoor localization is landmarks, which can be easily identified based on specific sensor patterns in the environment, and this will be exploited in our proposed approach. In this work, we propose a sensor fusion framework for combining WiFi, PDR and landmarks. Since the whole system is running on a smartphone, which is resource limited, we formulate the sensor fusion problem in a linear perspective, then a Kalman filter is applied instead of a particle filter, which is widely used in the literature. Furthermore, novel techniques to enhance the accuracy of individual approaches are adopted. In the experiments, an Android app is developed for real-time indoor localization and navigation. A comparison has been made between our proposed approach and individual approaches. The results show significant improvement using our proposed framework. Our proposed system can provide an average localization accuracy of 1 m.
In recent years, developing Indoor Positioning System (IPS) has become an attractive research topic due to the increasing demands on Location-Based Service (LBS) in indoor environment. Several advantages of Radio Frequency Identification (RFID) Technology, such as anti-interference, small, light and portable size of RFID tags, and its unique identification of different objects, make it superior to other wireless communication technologies for indoor positioning. However, certain drawbacks of existing RFID-based IPSs, such as high cost of RFID readers and active tags, as well as heavy dependence on the density of reference tags to provide the LBS, largely limit the application of RFID-based IPS. In order to overcome these drawbacks, we develop a cost-efficient RFID-based IPS by using cheaper active RFID tags and sensors. Furthermore, we also proposed three localization algorithms: Weighted Path Loss (WPL), Extreme Learning Machine (ELM) and integrated WPL-ELM. WPL is a centralized model-based approach which does not require any reference tags and provides accurate location estimation of the target effectively. ELM is a machine learning fingerprinting-based localization algorithm which can provide higher localization accuracy than other existing fingerprinting-based approaches. The integrated WPL-ELM approach combines the fast estimation of WPL and the high localization accuracy of ELM. Based on the experimental results, this integrated approach provides a higher localization efficiency and accuracy than existing approaches, e.g., the LANDMARC approach and the support vector machine for regression (SVR) approach.
This article describes a method for indoor positioning of human-carried active Radio Frequency Identification (RFID) tags based on the Sampling Importance Resampling (SIR) particle filtering algorithm. To use particle filtering methods, it is necessary to furnish statistical state transition and observation distributions. The state transition distribution is obstacle-aware and sampled from a precomputed accessibility map. The observation distribution is empirically determined by ground truth RSS measurements while moving the RFID tags along a known trajectory. From this data, we generate estimates of the sensor measurement distributions, grouped by distance, between the tag and sensor. A grid of 24 sensors is deployed in an office environment, measuring Received Signal Strength (RSS) from the tags, and a multithreaded program is written to implement the method. We discuss the accuracy of the method using a verification data set collected during a field-operational test.
Advances in low cost sensor and networking technologies in smart buildings have given researchers access to a multitude of time series data, including temperature, humidity, real and reactive power consumption of specific nodes or devices, occupant presence and activities, etc. Time series generated by sensor networks reflect various phenomena in buildings and are naturally related to each other. Hence quantitative techniques are required to exploit dependence among different types of sequences in order to allow smart applications such as non-intrusive activity detection, energy usage prediction, demand side management and control. Past research of relational analysis has focused on symmetric correlative statistics. On the other hand, asymmetric causal relations can capture more dynamic and complex relationships and is able to reveal directed influence among series. However, most traditional causal analysis relies on stationarity, while the statistics of real sensor measurement in smart buildings is rarely time invariant. In this paper, a statistical time series analysis framework is proposed to examine causal relationships among time series that are highly non-stationary. The Granger causality identification is extended to sensor data in buildings and the issue of non-stationarity is initially addressed by using modified Hodrick-Prescott (HP) filter which is able to extract simpler trend components. Subsequently, Autoregressive Integrated Moving Average model with exogenous variables (ARIMAX) model is trained for different components of two series. Finally, Granger causality is tested for both directions by F-statistics. The above procedure is performed on actual energy-consumption time series to exploit potential causal relations.
Buildings are known to be the largest consumers of electricity in the United States, and often times the dominant energy consumer is the HVAC system. Despite this fact, in most buildings the HVAC system is run using primitive static control algorithms based on fixed work schedules causing wasted energy during periods of low occupancy. In this paper we present a novel control architecture that uses occupancy sensing to guide the operation of a building HVAC system. We show how we can enable aggressive duty-cycling of building HVAC systems - that is, turn them ON or OFF - to save energy while meeting building performance requirements using inexpensive sensing and control methods. We have deployed our occupancy sensor network across an entire floor of a university building and our data shows several periods of low occupancy with significant opportunities to save energy over normal HVAC schedules. Furthermore, by interfacing with the building Energy Management System (EMS) directly and using real-time occupancy data collected by our occupancy nodes, we measure electrical energy savings of 9.54% to 15.73% and thermal energy savings of 7.59% to 12.85% for the HVAC system by controlling just one floor of our four floor building.
We are pleased to announce the release of a tool that records detailed measurements of the wireless channel along with received 802.11 packet traces. It runs on a commodity 802.11n NIC, and records Channel State Information (CSI) based on the 802.11 standard. Unlike Receive Signal Strength Indicator (RSSI) values, which merely capture the total power received at the listener, the CSI contains information about the channel between sender and receiver at the level of individual data subcarriers, for each pair of transmit and receive antennas.
Our toolkit uses the Intel WiFi Link 5300 wireless NIC with 3 antennas. It works on up-to-date Linux operating systems: in our testbed we use Ubuntu 10.04 LTS with the 2.6.36 kernel. The measurement setup comprises our customized versions of Intel's close-source firmware and open-source iwlwifi wireless driver, userspace tools to enable these measurements, access point functionality for controlling both ends of the link, and Matlab (or Octave) scripts for data analysis. We are releasing the binary of the modified firmware, and the source code to all the other components.
We propose Veto-Consensus Multiple Kernel Learning (VCMKL), a novel way of combining multiple kernels such that one class of samples is described by the logical intersection (consensus) of base kernelized decision rules, whereas the other classes by the union (veto) of their complements. The proposed configuration is a natural fit for domain description and learning with hidden subgroups. We first provide generalization risk bound in terms of the Rademacher complexity of the classifier, and then a large margin multi-ν learning objective with tunable training error bound is formulated. Seeing that the corresponding optimization is non-convex and existing methods severely suffer from local minima, we establish a new algorithm, namely Parametric Dual Descent Procedure (PDDP) that can approach global optimum with guarantees. The bases of PDDP are two theorems that reveal the global convexity and local explicitness of the parameterized dual optimum, for which a series of new techniques for parametric program have been developed. The proposed method is evaluated on extensive set of experiments, and the results show significant improvement over the state-of-the-art approaches.
We consider a class of non-convex learning problems that can be formulated as jointly optimizing regularized hinge loss and a set of auxiliary variables. Such problems encompass but are not limited to various versions of semi-supervised learning,learning with hidden structures, robust learning, etc. Existing methods either suffer from local minima or have to invoke anon-scalable combinatorial search. In this paper, we propose a novel learning procedure, namely Parametric Dual Maximization(PDM), that can approach global optimality efficiently with user specified approximation levels. The building blocks of PDM are two new results: (1) The equivalent convex maximization reformulation derived by parametric analysis.(2) The improvement of local solutions based on a necessary and sufficient condition for global optimality. Experimental results on two representative applications demonstrate the effectiveness of PDM compared to other approaches.
This paper presents a WiFi-based device-free crowd counting and occupancy estimation system that can be used in rooms/environments different from the ones in which the training process has been performed. Therefore, crowd counting is achieved without requiring another training phase in each new environment. The proposed approach analyzes the shape of the Doppler spectrum of the received signal which is correlated to the number of people moving in the monitored environment. Unlike a radar-like approach, the use of a reference signal is not required. Experimental results are presented for two different rooms/environments without any constraint on the movements of the volunteers.
The Global Positioning System (GPS) can be readily used for outdoor localization, but GPS signals are degraded in indoor environments. How to develop a robust and accurate indoor localization system is an emergent task. In this paper, we propose a smartphone inertial sensor-based indoor localization and tracking system with occasional iBeacon corrections. Some important issues in a smartphone-based pedestrian dead reckoning (PDR) approach, i.e., step detection, walking direction estimation, and initial point estimation, are studied. One problem of the PDR approach is the drift with walking distance. We apply a recent technology, iBeacon, to occasionally calibrate the drift of the PDR approach. By analyzing iBeacon measurements, we define an efficient calibration range where an extended Kalman filter is utilized. The proposed localization and tracking system can be implemented in resource-limited smartphones. To evaluate the performance of the proposed approach, real experiments under two different environments have been conducted. The experimental results demonstrated the effectiveness of the proposed approach. We also tested the localization accuracy with respect to the number of iBeacons.
Power delay profiles characterize multipath channel features, which are widely used in motion- or localization-based applications. Recent studies show that the power delay profile may be derived from the CSI traces collected from commodity WiFi devices, but the performance is limited by two dominating factors. The resolution of the derived power delay profile is determined by the channel bandwidth, which is however limited on commodity WiFi. The collected CSI reflects the signal distortions due to both the channel attenuation and the hardware imperfection. A direct derivation of power delay profiles using raw CSI measures, as has been done in the literature, results in significant inaccuracy. In this paper, we present Splicer, a software-based system that derives high-resolution power delay profiles by splicing the CSI measurements from multiple WiFi frequency bands. We propose a set of key techniques to separate the mixed hardware errors from the collected CSI measurements. Splicer adapts its computations within stringent channel coherence time and thus can perform well in presence of mobility. Our experiments with commodity WiFi NICs show that Splicer substantially improves the accuracy in profiling multipath characteristics, reducing the errors of multipath distance estimation to be less than 2m. Splicer can immediately benefit upper-layer applications. Our case study with recent single-AP localization achieves a median localization error of 0.95m.
Despite of several years of innovative research, indoor localization is still not mainstream. Existing techniques either employ cumbersome fingerprinting, or rely upon the deployment of additional infrastructure. Towards a solution that is easier to adopt, we propose CUPID, which is free from these restrictions, yet is comparable in accuracy. While existing WiFi based solutions are highly susceptible to indoor multipath, CUPID utilizes physical layer (PHY) information to extract the signal strength and the angle of only the direct path, successfully avoiding the effect of multipath reflections. Our main observation is that natural human mobility, when combined with PHY layer information, can help in accurately estimating the angle and distance of a mobile device from an wireless access point (AP). Real-world indoor experiments using off-the-shelf wireless chipsets confirm the feasibility of CUPID. In addition, while previous approaches rely on multiple APs, CUPID is able to localize a device when only a single AP is present. When a few more APs are available, CUPID can improve the median localization error to 2.7m, which is comparable to schemes that rely on expensive fingerprinting or additional infrastructure.
Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.
From rssi to csi: Indoor localization via channel response
- Z Yang
- Z Zhou
- Y Liu
Z. Yang, Z. Zhou, and Y. Liu, "From rssi to csi: Indoor localization
via channel response," ACM Computing Surveys (CSUR), vol. 46, no. 2,
p. 25, 2013.