No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
FreeCount: Device-Free Crowd Counting with Commodity WiFi
... The existing WiFi-based ICC methods can be mainly divided into two categories, depending on the need of manual feature extraction [6]- [13]. One relies on explicit manual features engineered from raw data, such as mean value, variance, median, and range, and then uses threshold-based or learningbased classifiers like support vector machine (SVM) to identify the crowd number. ...
... The performance of the former method is critically related to the data feature selection. For example, to select the best-performing features, Zou et al. [6] proposed a "Transfer Kernel Learning (TKL)" method that selects data features based on a mutual information criterion from a feature pool including several statistical, transformation-based, and shape-based features. The performance of the latter method highly depends on iteratively training with a large number of labeled samples. ...
... Although human activity causes fluctuation of both amplitude and phase, many studies only use the amplitude information for ICC (like in [6], [11], [12]), mainly because the phase information often suffers from more severe hardware measurement noise such as carrier frequency offset and sampling time offset [21]- [23]. Instead of using raw phase measurements, Zong et al. [13] computed the phase difference between adjacent antennas as the input to an SVM-based ICC classifier. ...
Accurate indoor crowd counting (ICC) is a key enabler to many smart home/office applications. In this paper, we propose a Domain-Agnostic and Sample-Efficient wireless indoor crowd Counting (DASECount) framework that suffices to attain robust cross-domain detection accuracy given very limited data samples in new domains. DASECount leverages the wisdom of few-shot learning (FSL) paradigm consisting of two major stages: source domain meta training and target domain meta testing. Specifically, in the meta-training stage, we design and train two separate convolutional neural network (CNN) modules on the source domain dataset to fully capture the implicit amplitude and phase features of CSI measurements related to human activities. A subsequent knowledge distillation procedure is designed to iteratively update the CNN parameters for better generalization performance. In the meta-testing stage, we use the partial CNN modules to extract low-dimension features out of the high-dimension input target domain CSI data. With the obtained low-dimension CSI features, we can even use very few shots of target domain data samples (e.g., 5-shot samples) to train a lightweight logistic regression (LR) classifier, and attain very high cross-domain ICC accuracy. Experiment results show that the proposed DASECount method achieves over 92.68\%, and on average 96.37\% detection accuracy in a 0-8 people counting task under various domain setups, which significantly outperforms the other representative benchmark methods considered.
... However, these approaches require deploying specific sensors in the interest region that cannot be blocked. General WiFi sensors-based [22][23][24][25] methods can overcome the above drawbacks, but they need people to keep moving. Therefore, they cannot detect static crowds, for instance, passengers in the elevator. ...
... In radio communications, the OFDM systems are widely used to divide the wireless network spectrum into orthogonal sub-carriers. However, the emitted WiFi signals often do not reach the receive antenna directly due to obstacle blocking and multi-path effects of signal propagation [23]. A portion of the signals passes through the medium, while other signals are reflected or absorbed by the medium. ...
... For the M1 [25] and M2 [22] methods, about 50% of the results are accurate due to the RSSI-based schemes performing poorly in multi-path complex environments and scene migration. FreeCount [23] (M3) can achieve more than 60% accuracy because CSI is more sensitive to the diversity of transmission channels than RSSI. Due to M4 [24] not only estimating the number of people but also providing human dynamics monitoring through participant number estimation, its F1-score reaches more than 80%. ...
Elevators have become a kind of indispensable facility for everyday life, which bring people both convenience and safety hazards. Specifically in the household environment, an elevator’s lifespan is expected to be more than 20 years. An appropriate and regularly maintained counterweight is conducive to extending elevator life. This paper proposes a passenger counting approach in the elevator for regular counterweight adjustment based on commodity WiFi called ECC. Since the running time of the elevator between two adjacent floors is short, the major challenge of ECC is how to count passengers from the limited captured data. This paper first theoretically analyzes the relationship between the number of passengers and the variation of channel state information (CSI). Then ECC constructs a multi-dimensional feature by extracting the average of amplitude (AOA), time-varying spectrum (TVS), and percentage of non-zero elements (PEM) features from the limited data. Finally, the random forest (RF) classifier is used for passenger counting and the local optimization problem is solved by expanding the feature dataset through data segmentation. ECC is implemented by using off-the-shelf IEEE 802.11n devices, and its performance is evaluated via extensive experiments in typical real-world scenes. The estimated precision of ECC can reach more than 95%, and more than 97% of estimation errors are less than 2 persons, which demonstrates the superior effectiveness and generalizability of ECC.
... Wi-Fi derived occupant counts have been shown to have strong correlation with actual ground truth counts where R 2 values ranging from 0.85-0.96 have been reported in the literature [25,[28][29][30] as well as Wi-Fi device to occupant ratio of 1.27 [31] and 1.2 [32]. Wi-Fi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [16]. ...
... In solving the occupancy estimation problem, [23] achieved 98.85% occupancy detection accuracy, 0.096 Normalized Root Mean Square Derivative occupancy count accuracy and 1.385 m occupancy tracking accuracy using their framework that utilized Received Signal Strength (RSS) levels and device Media Access Control (MAC) addresses. [30] achieved 90% accuracy using Channel State Information (CSI) to infer occupant counts, where the sensitivity of transmitted radio signals to occupant movements was captured. Their approach eliminated the need for occupant terminal devices such as smartphones. ...
... There is the possibility of overestimated occupant counts, which is critical when determining ( ) and ( ) based on, as overestimated occupant counts will result in much earlier ( ) or later ( ) and diminished energy savings. In addition to the preprocessing steps we take in Section 4.2.1, another remedy that has been used in literature is to calibrate device counts against ground truth occupancy [28][29][30] however, this approach assumes static ratio of device counts to actual occupancy counts. Park et al. investigated the use of Capture-Recapture, a methodology adopted from the field of zoology, to infer the spatial and temporal variation in Wi-Fi device count to actual occupancy count ratio as an alternative to a statically determined ratio [57]. ...
HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an Energy Conservation Measure (ECM) where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building's real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from ECMs that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived Heating, Ventilation and Air Conditioning (HVAC) schedules and energy savings that leverages the ubiquity of energy smart meters and Wi-Fi infrastructure in commercial buildings. We estimate the schedules by clustering Wi-Fi derived occupancy profiles and, estimate energy savings by shifting ramp-up and setback times observed in typical operational/static load profiles that are obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%-10.8% (summer) and 0.2%-5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from Building Energy Performance Simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%-2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%-5%.
... WiFi derived occupant counts have been shown to have strong correlation with actual ground truth counts where R 2 values ranging from 0.85 -0.96 have been reported in the literature [23,24,25,21] as well as a strong occupant to device ratio of 1.27 [26]. WiFi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [13]. ...
... In solving the occupancy estimation problem, [19] achieved 98.85% occupancy detection accuracy, 0.096 Normalized Root Mean Square Derivative occupancy count accuracy and 1.385 m occupancy tracking accuracy using their framework that utilized Received Signal Strength (RSS) levels and device MAC addresses. [25] achieved 90% accuracy using Channel State Information (CSI) to infer occupant counts where the sensitivity of transmitted radio signals to occupant movements was captured. Their approach eliminated the need for occupant terminal devices such as smartphones. ...
... There is the possibility of overestimated occupant counts which is critical when determining ( ) and ( ) based on as overestimated occupant counts will result in much earlier ( ) or later ( ) and diminished energy savings. A remedy that has been used in literature is to calibrate device counts against ground truth occupancy [23,24,25] however, this approach assumes static ratio of device counts to actual occupancy counts. Park et al. investigated the use of Capture-Recapture, a methodology adopted from the field of zoology, to infer the spatial and temporal variation in WiFi device count to actual occupancy count ratio as an alternative to a statically determined ratio. ...
HVAC systems account for a significant portion of building energy use. Nighttime setback scheduling is an energy conservation measure where cooling and heating setpoints are increased and decreased respectively during unoccupied periods with the goal of obtaining energy savings. However, knowledge of a building's real occupancy is required to maximize the success of this measure. In addition, there is the need for a scalable way to estimate energy savings potential from energy conservation measures that is not limited by building specific parameters and experimental or simulation modeling investments. Here, we propose MARTINI, a sMARt meTer drIveN estImation of occupant-derived HVAC schedules and energy savings that leverages the ubiquity of energy smart meters and WiFi infrastructure in commercial buildings. We estimate the schedules by clustering WiFi-derived occupancy profiles and, energy savings by shifting ramp-up and setback times observed in typical/measured load profiles obtained by clustering smart meter energy profiles. Our case-study results with five buildings over seven months show an average of 8.1%-10.8% (summer) and 0.2%-5.9% (fall) chilled water energy savings when HVAC system operation is aligned with occupancy. We validate our method with results from building energy performance simulation (BEPS) and find that estimated average savings of MARTINI are within 0.9%-2.4% of the BEPS predictions. In the absence of occupancy information, we can still estimate potential savings from increasing ramp-up time and decreasing setback start time. In 51 academic buildings, we find savings potentials between 1%-5%.
... By analysing the patterns of its wireless signal, today's AP has evolved beyond a pure WiFi router, but is also widely used as a type of 'sensor device' to enable new services for human sensing. Particularly, recent studies have found that WiFi signals in the form of Channel State Information (CSI) [1], [2] are extremely promising for a variety of devicefree human sensing tasks, such as occupancy detection [3], activity recognition [4], [5], [6], [7], fall detection [8], gesture recognition [9], [10], human identification [11], [12], and people counting [13], [14]. Unlike the coarse-grained received signal strengths, WiFi CSI records more fine-grained information about how a signal propagates between WiFi J. Yang, X. Chen, D. Wang, H. Zou devices and how a signal is reflected from the surrounding environment in which humans move around. ...
... In contrast, by feeding a massive amount of data into machine learning [22] or deep learning networks, [9], [5], learning based achieve remarkable performances in complicated sensing tasks. Various deep neural networks are designed to enable many applications including activity recognition [23], gesture recognition [9], human identification [11], [12], [24], and people counting [13], [14]. Though deep learning models have a strong ability of function approximation, they require tremendous labeled data that is expensive to collect and suffer from the negative effect of distribution shift caused by environmental dynamics [25]. ...
We are excited to share our work “Deep Learning and Its Applications to WiFi Human Sensing” which is the first open-sourced benchmark including well-processed public datasets collected by multiple WiFi sensing platforms.
IEEE 802.11bf will come into existence in 2024, and WiFi sensing will enable ubiquitous human perception in smart homes and buildings.
Paper: https://arxiv.org/abs/2207.07859
Code: https://github.com/CHENXINYAN-sg/WiFi-CSI-Sensing-Benchmark
... This sensing methodology is advantageous because it has the tendency to be non-intrusive and low-cost by leveraging existing wireless network infrastructure such as WiFi routers, Bluetooth beacons and mobile devices to passively infer occupant information. WiFi derived occupant counts have been shown to have a strong correlation with actual ground truth counts, where R 2 values ranging from 0.85 to 0.96 have been reported in the literature [14,[18][19][20]. WiFi-based occupancy can be utilized in solving occupancy estimation, future occupancy prediction and occupancy pattern clustering problems [8]. ...
... While these methods tackle overestimated counts problem, under counting remains an issue and are associated with privacy risks. To remedy the problems originating from the need of a device-carrying occupants, researchers have utilized Channel State Information (CSI), where the effect of occupant movement on transmitted WiFi signals is used to derive occupancy information [20,27,28]. Demrozi et al. proposed using Bluetooth Low Energy (BLE) as a cheaper but comparably accurate alternative to CSI technology for a device-free occupant detection and counting system [29]. ...
The occupancy information in buildings is fundamental for smart buildings (e.g., occupant-centric controls). Opportunistic occupancy detection (OOD) uses connection data of mobile devices. While OOD has been developed and applied, one critical drawback is that it requires the ground truth of occupants to calibrate, which is limited to gather. Here, we introduce CROOD: a capture and recapture (CRc) inspired OOD. In ecology, CRc has been established for the estimation of animal populations, when the manual count is impossible. We adopt this unique approach to estimate the number of mobile devices in a building. Then, using a simple estimate on the total population, CROOD determines the relationship between the numbers of occupants and mobile devices. We evaluate CROOD numerically on the synthetic building populations and demonstrate its application in a university library using WiFi connection data. We find that CROOD can estimate the number of mobile devices and subsequently the number of occupants with 1–2 weeks to converge a reasonable accuracy. A long term experiment shows that CROOD can adapt to varying population characteristics (e.g., occupants bring more mobile devices), outperforming the reference sample mean estimator. The real building implementation demonstrates that while in the first 1–2 weeks, the sample mean estimator is superior, eventually CROOD adapts and provides better estimates without ground-truth calibration. Although CROOD has a limitation of building types and systems, our results envision that CROOD could be a viable addition to other OOD methods to better utilize existing mobile device connection data to estimate occupancy in buildings.
... The works closest to our research are those that focus on counting people in spaces and that leave aside the analysis of the behavior of the people counted. These solutions, in general, are classified into two groups, those that analyze the Wi-Fi signal [29][30][31][32] and those that focus on the analysis of the information packets sent by the Wi-Fi devices [33][34][35]. Solutions based on Wi-Fi signal analysis (the received signal strength indicator or RSSI and the channel state or CSI, above all) show promising results but for in indoor setups. ...
... From the point of view of the accuracy of the results, regardless of the approach used, the necessary infrastructure, or the type of experimentation, the analyzed articles that were used as a basis for our research show very different values, between 80% and 96% [29,30,35]. These values can only be considered for direct comparison with special caution, as they come from developments based on very different experimentation approaches. ...
Estimating the number of people present in a given venue in real-time is extremely useful
from security, management, and resource optimization perspective. This article presents the architecture of a system based on the use ofWi-Fi sensor devices that allows estimating, almost in real-time, the number of people attending an event that is taking place in a venue. The estimate is based on the analysis of the “probe request” messages periodically transmitted by smartphones to determine the existence of Wi-Fi access points in the vicinity. The method considers the MAC address randomization mechanisms introduced in recent years in smartphones, which prevents the estimation of the number of devices by simply counting different MAC addresses. To solve this difficulty, our Wi-Fi sensors analyze other fields present in the header of the IEEE 802.11 frames, the information elements, to extract a unique fingerprint from each smartphone. The designed system was tested in a set of real scenarios, obtaining an estimate of attendance at different public events with an accuracy close to 95%.
... 1 ...
... The number of persons counting within an indoor environment is the demand for many applications to accommodate their services and management [1]. The IoT products are the consecration in this regard to count the number of visitors in such areas as shops, airports, supermarkets, cinemas, stadiums, etc. ...
... 2.7.6. Freecount: Device-Free Crowd Counting with Commodity Wi-Fi FreeCount [67] is a high-accuracy system for crowd counting that uses a set of features of three kinds: statistics, frequency domain and shape. In their publication, Zou et al. focus in the problem of temporal variation of the channel conditions and the unpractical need of re-training the classifier every period of time. ...
... From this foundation, different authors take different approaches for crowd counting. For instance, Zou et al. [67], use a set of features coming from statistics of magnitude and phase, Fourier transformation and shaped metrics, all combined to achieve predictors of human motion. Di Domenico et al. [25], extract Average Spectral Kurtosis as the unique feature they use for their model, while Yang et al. [68], use only the first link of the MIMO grid for feature extraction. ...
Measuring the quantity of people in a given space has many applications, ranging from marketing to safety. A family of novel approaches to measuring crowd size relies on inexpensive Wi-Fi equipment, taking advantage of the fact that Wi-Fi signals get distorted by people’s presence, so by identifying these distortion patterns, we can estimate the number of people in such a given space. In this work, we refine methods that leverage Channel State Information (CSI), which is used to train a classifier that estimates the number of people placed between a Wi-Fi transmitter and a receiver, and we show that the available multi-link information allows us to achieve substantially better results than state-of-the-art single link or averaging approaches, that is, those that take the average of the information of all channels instead of taking them individually. We show experimentally how the addition of each of the multiple links information helps to improve the accuracy of the prediction from 44% with one link to 99% with 6 links.
... This information can be used to allocate resources and ensure public safety. • Shopping Malls Management: Shopping malls and retail stores can use crowd counting to analyze foot traffic and customer behavior [25], [26]. This can help store managers optimize store layout, staffing, and marketing strategies. ...
p>In machine vision, the tasks of both the crowd counting and the crowd density estimation attracted a number of researchers from different research institutes throughout the world. These tasks are of significance importance while analyzing the surveillance videos and images both at the real and offline modes. A number of different conventional and modern algorithms are being applied to estimate the accurate amount of people from the crowd from the target image or video file but failed to provide the required results due to heterogeneity of the data instances. To resolve the issue, machine learning models like deep networks like Convolution Neural Network (CNN) and its variants were introduced. In this paper, a detailed and comprehensive review of the work related to both of the crowd counting and density estimation using deep network is provided in quite a very unique way to make it useful for the researchers working in the field of computer vision. Following are the salient features of our review that play key role in making our contribution worthwhile and unique among the surveys and reviews of similar kind: (i) our survey intends to classify the noteworthy literature with respect to the application and the tasks related to crowd counting and density estimation, (ii) the survey also tried to cover in-depth contributions in crowd counting existing works at different level like images or videos (iii) this state-of-the-art review covers each article in the following dimensions: the designated task performed, source level, results obtained, features used, and (iv) lastly it concludes the summary of the related articles according to the publishing years, related tasks (or subtasks), and types of classifiers used. In the end, major challenges and tasks related to crowd counting and density estimation in computer vision are also discussed.</p
... This information can be used to allocate resources and ensure public safety. • Shopping Malls Management: Shopping malls and retail stores can use crowd counting to analyze foot traffic and customer behavior [25], [26]. This can help store managers optimize store layout, staffing, and marketing strategies. ...
In machine vision, the tasks of both the crowd counting and the crowd density estimation attracted a number of researchers from different research institutes throughout the world. These tasks are of significance importance while analyzing the surveillance videos and images both at the real and offline modes. A number of different conventional and modern algorithms are being applied to estimate the accurate amount of people from the crowd from the target image or video file but failed to provide the required results due to heterogeneity of the data instances. To resolve the issue, machine learning models like deep networks like Convolution Neural Network (CNN) and its variants were introduced. In this paper, a detailed and comprehensive review of the work related to both of the crowd counting and density estimation using deep network is provided in quite a very unique way to make it useful for the researchers working in the field of computer vision. Following are the salient features of our review that play key role in making our contribution worthwhile and unique among the surveys and reviews of similar kind: (i) our survey intends to classify the noteworthy literature with respect to the application and the tasks related to crowd counting and density estimation, (ii) the survey also tried to cover in-depth contributions in crowd counting existing works at different level like images or videos (iii) this state-of-the-art review covers each article in the following dimensions: the designated task performed, source level, results obtained, features used, and (iv) lastly it concludes the summary of the related articles according to the publishing years, related tasks (or subtasks), and types of classifiers used. In the end, major challenges and tasks related to crowd counting and density estimation in computer vision are also discussed.
... However, carrying out multi-object tracking using CSI is challenging due to the complexity of overlapped signals in the time and frequency domains. Additionally, environmental monitoring techniques such as crowd counting [20] and intrusion detection [21] are popular with CSI-based methods, but they require a room to be equipped with both a transmitter (TX) and a receiver (RX), which is costly and impractical. Presence detection, which infers the presence of people in indoor environments based on the variation of received signals [22], [23], faces the challenge of false detection caused by a range of events that result in time-varying signals. ...
Human presence detection is a crucial technology for various applications, including home automation, security, and healthcare. While camera-based systems have traditionally been used for this purpose, they raise privacy concerns. To address this issue, recent research has explored the use of channel state information (CSI) approaches that can be extracted from commercial WiFi access points (APs) and provide detailed channel characteristics. In this thesis, we propose a device-free human presence detection system for multi-room scenarios using a time-selective conditional dual feature extract recurrent Network (TCD-FERN). Our system is designed to capture significant time features with the condition on current human features using a dynamic and static (DaS) data preprocessing technique to extract moving and spatial features of people and differentiate between line-of-sight (LoS) path blocking and non-blocking cases. To mitigate the feature attenuation problem caused by room partitions, we employ a voting scheme. We conduct evaluation and real-time experiments to demonstrate that our proposed TCD-FERN system can achieve human presence detection for multi-room scenarios using fewer commodity WiFi APs.
... This subsection shows applied AI-based optimization algorithms for density estimation in crowd detection and localization approaches. FreeCount is a device-free method of crowd counting proposed by Zou et al. [61] that relies solely on common WiFi routers to calculate the precise population density of an area. In areas of average size, FreeCount can operate with just two routers. ...
The Internet of Things (IoT) provides a collaborative infrastructure to communicate smart devices with cloud-edge healthcare applications, medical devices, wearable biosensors, etc. On the other hand, crowd counting as one of computer vision approaches is an emerging topic to detect any objects with static or dynamic mobility in the IoT environments. Smart crowd counting enables pattern recognition for many intelligent applications such as microbiology, surveillance, healthcare systems, crowdedness estimation, and other environmental case studies. According to complicated capturing systems in the IoT environments, crowd counting methods can influence on performance of object detection in the critical case studies using Artificial Intelligence (AI)-based approaches such as machine learning, deep learning, collaborative learning, fuzzy logic and meta-heuristic algorithms. This paper provides a new comprehensive technical analysis for existing AI-based crowd counting approaches in healthcare and medical systems, biotechnology and IoT environments. Meanwhile, it presents a discussion on the existing case studies with respect to analyzing technical aspects and applied algorithms to enhance pattern prediction factors. Finally, some new innovative efforts and challenges are presented for new research upcoming and open issues.
... By analyzing the patterns of its wireless signals, current-day APs have evolved beyond being pure WiFi routers and also serve as a type of ''sensor device'' to enable new services for human sensing. In particular, recent studies have pointed out that WiFi signals in the form of channel state information (CSI) 1,2 are extremely promising for a variety of device-free human-sensing tasks, such as occupancy detection, 3 activity recognition, [4][5][6][7] fall detection, 8 gesture recognition, 9,10 human identification, [11][12][13] people counting, 14,15 and pose estimation. 16 Unlike coarsegrained received signal strengths, WiFi CSI records more finegrained information in terms of the propagation of a signal between WiFi devices and their reflection with respect to the environment of human beings. ...
Over the recent years, WiFi sensing has been rapidly developed for privacy-preserving, ubiquitous human-sensing applications, enabled by signal processing and deep-learning methods. However, a comprehensive public benchmark for deep learning in WiFi sensing, similar to that available for visual recognition, does not yet exist. In this article, we review recent progress in topics ranging from WiFi hardware platforms to sensing algorithms and propose a new library with a comprehensive benchmark, SenseFi. On this basis, we evaluate various deep-learning models in terms of distinct sensing tasks, WiFi platforms, recognition accuracy, model size, computational complexity, and feature transferability. Extensive experiments are performed whose results provide valuable insights into model design, learning strategy, and training techniques for real-world applications. In summary, SenseFi is a comprehensive benchmark with an open-source library for deep learning in WiFi sensing research that offers researchers a convenient tool to validate learning-based WiFi-sensing methods on multiple datasets and platforms.
... The WiFi-based gait recognition method uses RF signals from WiFi-enabled devices to determine human identity. The transmitter emits WiFi signals, which are reflected by different body parts of the walking subject and then recorded by CSI data at the receiver [19], which has empowered many applications including occupancy detection [20], crowd counting [21], [22], human activity recognition [23], [24], [25], [26], [27], person identification [8], [28], vital sign detection [29], pose estimation [30] and gesture recognition [31], [32], [33]. To use WiFi sensing in the real world, current research aims at efficient communication [16], model security [34] and dataefficient training [35]. ...
As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.
... The authors show the percentage of non-zero elements (PEM) metric and the cumulative density function (CDF) of error estimation for scenarios with different speeds. FreeCount is a crowd counting scheme using WiFi routers presented in [26]. Their model presented high precision for indoors in rooms with moderate size. ...
Currently, there is a requirement in many countries to keep public and work spaces safe due to COVID-19. In fact, indoor spaces must be monitored to control the allowed capacity, which can vary depending on the alert level of a city at a given time. This has motivated some researchers to investigate several technologies to implement methods and strategies to enable the reopening of these spaces in a safe manner. In this paper, we propose a crowd counting detection system that addresses the problem of controlling the indoor capacity of offices inside buildings. The proposed solution uses an existing communication technology such as WiFi in order to determine the crowd counting for the indoor environment. In particular, the existing infrastructure consists of two Wireless LAN Controllers (WLC) and several APs deployed in a building, which allows us to estimate the number of people based on the access to Wireless Access Points (APs). Thus, the proposed system takes into account when a mobile device connects/disconnects to the AP to increase or decrease the number of people in a particular office and sends the respective alert to the system administrator when this capacity is about to be exceeded or already surpassed.
... Previous researches have shown that human pose can affect the propagation of WiFi signals and these signal variations are reflected by Channel State Information (CSI). Enabled by advanced deep learning methods, many studies have been conducted to enable various applications, such as human activity recognition [5]- [7], gesture recognition [8], [9], human identification [10], [11], and people counting [12], [13]. Many advanced learning methods further promote the automation and performance of these tasks, such as transfer learning [14], [15], unsupervised learning [16], and modelrobust learning [17]. ...
Avatar refers to a representative of a physical user in the virtual world that can engage in different activities and interact with other objects in metaverse. Simulating the avatar requires accurate human pose estimation. Though camera-based solutions yield remarkable performance, they encounter the privacy issue and degraded performance caused by varying illumination, especially in smart home. In this paper, we propose a WiFi-based IoT-enabled human pose estimation scheme for metaverse avatar simulation, namely MetaFi. Specifically, a deep neural network is designed with customized convolutional layers and residual blocks to map the channel state information to human pose landmarks. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. WiFi is ubiquitous and robust to illumination, making it a feasible solution for avatar applications in smart home. The experiments are conducted in the real world, and the results show that the MetaFi achieves very high performance with a PCK@50 of 95.23%.
... Counting the number of occupants is essential for building monitoring and management. The use of WPS for counting the number of occupants can help control people in specific places and monitor their entry and exit [30,[50][51][52][53]. In addition, WPS can be a good alternative for counting people inside the building, such as shopping centers, airports, and hospitals [54]. ...
Contact tracing is one of the critical tools for fighting against pandemic disease outbreaks, such as the fast-growing SARS-CoV-2 virus and its different variants. At present, automated contact tracing systems face two main challenges: (1) requiring application installation on smart devices and (2) protecting the users’ privacy. This study introduces a conceptual passive contact tracing system using indoor WiFi positioning to address these challenges and investigate the role of such a system in commercial buildings. In this regard, this study uses a simulated small-office layout in a case study to demonstrate the applicability of the proposed system. The special use of the proposed contact tracing system could be academic facilities and office buildings, where (1) the WiFi infrastructure already exists and therefore implementing such a system could be cost-effective, and (2) the same users use the facility regularly, enabling the system to notify the users upon a confirmed case once they are back in the building and connected to the WiFi system. Such technology can not only enhance the current automated contact tracing system in commercial buildings by illuminating the need to use smartphone applications while protecting users’ privacy, but could also reduce the risk of infection in indoor environments. The developed system can benefit facility managers, business owners, policy makers, and authorities in assisting to find occupants’ high-risk contacts and control the spread of SARS-CoV-2 or similar infectious diseases in commercial buildings, particularly university campuses and office buildings.
... Alternatively, at the physical layer, Channel State Information (CSI) describes how WiFi signal propagates from a transmitter (TX) to a receiver (RX) through multiple paths at the granularity of Frequency Division Multiplexing (OFDM) subcarriers, which is more sensitive to the presence and movements of an object and is more robust to background noise. Recent literature has witnessed many successful employments of CSI measurements for various applications, such as crowd counting (Zou et al. 2017b) and human ac-The Thirty-Second AAAI Conference on Artificial Intelligence tivity recognition (Wang et al. 2015). Being an off-the-shelf and fine-grained sensing measurement without the introduction of any extra infrastructure or user involvement, CSI data is the ideal sensing recourse for device-free and low-cost human identification. ...
We propose AutoID, a human identification system that leverages the measurements from existing WiFi-enabled Internet of Things (IoT) devices and produces the identity estimation via a novel sparse representation learning technique. The key idea is to use the unique fine-grained gait patterns of each person revealed from the WiFi Channel State Information (CSI) measurements, technically referred to as shapelet signatures, as the "fingerprint" for human identification. For this purpose, a novel OpenWrt-based IoT platform is designed to collect CSI data from commercial IoT devices. More importantly, we propose a new optimization-based shapelet learning framework for tensors, namely Convex Clustered Concurrent Shapelet Learning (C3SL), which formulates the learning problem as a convex optimization. The global solution of C3SL can be obtained efficiently with a generalized gradient-based algorithm, and the three concurrent regularization terms reveal the inter-dependence and the clustering effect of the CSI tensor data. Extensive experiments are conducted in multiple real-world indoor environments, showing that AutoID achieves an average human identification accuracy of 91% from a group of 20 people. As a combination of novel sensing and learning platform, AutoID attains substantial progress towards a more accurate, cost-effective and sustainable human identification system for pervasive implementations.
... Then the E-eyes system is developed to achieve better performance by dividing human activities into in-place and dynamic ones [23]. The FreeCount system leverages a feature selection scheme based on information theory to conduct people counting [24]. These early-stage works show good performance on normal activities such as walking and sitting, but they cannot identify fine-grained subtle gestures. ...
WiFi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by Channel State Information (CSI) extracted from WiFi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this paper, we firstly explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an automatic WiFi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in WiFi sensing. The AutoFi is implemented on a pair of Atheros WiFi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public datasets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar datasets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that the AutoFi takes a huge step toward automatic WiFi sensing without any developer engagement while overcoming the cross-site issue.
... Zou et al. [25] proposed FreeCount, which is a device-free crowd counting scheme using a modified CSI tool running on commercial WiFi devices. They adopted the transfer kernel learning (TKL) model to take account of temporal variation of CSI measurements, and trained the model with 20 features based on de-noised CSI data by wavelet filter, which are categorized in common statistics, transformationbased, and shape-based features. ...
Wireless sensing represented by WiFi channel state information (CSI) is now enabling various fields of applications such as person identification, human activity recognition, occupancy detection, localization, and crowd estimation these days. So far, those fields are mostly considered as separate topics in WiFi CSI-based methods, on the contrary, some camera and vision-based crowd estimation systems intuitively estimate both crowd size and location at the same time. Our work is inspired by the idea that WiFi CSI also may be able to perform the same as the camera does. In this paper, we construct
Wi-CaL
, a simultaneous crowd counting and localization system by using ESP32 modules for WiFi links. We extract several features that contribute to dynamic state (moving crowd) and static state (location of the crowd) from the CSI bundles, then assess our system by both conventional machine learning (ML) and deep learning (DL). As a result of ML-based evaluation, we achieved 0.35 median absolute error (MAE) of counting and 91.4% of localization accuracy with five people in a small-sized room, and 0.41 MAE of counting and 98.1% of localization accuracy with 10 people in a medium-sized room, by leave-one-session-out cross-validation. We compared our result with percentage of non-zero elements metric (PEM), which is a state-of-the-art metric for crowd counting, and confirmed that our system shows higher performance (0.41 MAE, 81.8% of within-1-person error) than PEM (0.62 MAE, 66.5% of within-1-person error).
... The ability to estimate the total number of people is important to improve the convenience of when utilizing facilities [1]. For example, by counting people in queues, airports, restaurants and shopping malls attempt to improve their services, and also prepare for an effective evacuation guidance during times of emergency [2]. Most of the existing works have realized such estimation by leveraging either image recognition techniques [3] or Wi-Fi signals [4], [5]. ...
As smartphones have become widespread in the past decade, Wi-Fi signal-based crowd estimation schemes are receiving increased attention. These estimation schemes count the number of unique MAC addresses in Wi-Fi signals, hereafter called probe requests (PRs), instead of counting the number of people. However, these estimation schemes have low accuracy of crowd estimation under MAC address randomization that replaces a unique MAC address with various dummy MAC addresses. To solve this problem, in this paper, we propose an indoor crowd estimation scheme using the number of PRs under MAC address randomization. The main idea of the proposed scheme is to leverage the fact that the number of PRs per a unit of time changes in proportion to the number of smartphones. Since a smartphone tends to send a constant number of PRs per a unit of time, the proposed scheme can estimate the accurate number of smartphones. Various experiment results show that the proposed scheme reduces estimation error by at most 75% compared to the conventional Wi-Fi signal-based crowd estimation scheme in an indoor environment.
... Therefore, the system based on the sensor is generally expensive to use or is likely to cause inconvenience to users. Compared with the above two types of technologies, the sensing system based on Wi-Fi signals [10][11][12] can overcome the above shortcomings and has the advantages of convenience, low cost, non-contact, and non-infringement of privacy. Wang et al. [13] proposed the CARM system, which uses the Wi-Fi signal phase to extract the characteristics of motion duration and motion speed, and builds a hidden Markov model to classify single-person activity. ...
Abstract Indoor human wireless sensing is crucial for many applications such as smart homes and security monitoring. It needs to understand both the number of people and their activities, which is often difficult, especially in large spaces. This paper proposes a combined people number counting and action recognition method by using the channel state information (CSI) of Wi‐Fi signal, which aims to simultaneously estimate the number of people and their actions in wireless sensing applications. First, a jump region removal algorithm is developed to calibrate the phase difference. Second, a cumulative sliding variance algorithm is designed for the detection of moving targets in the environment. Then a multi‐dimensional feature is formed by the amplitude, frequency domain, and phase difference of CSI, and based on it, a two‐level classifier based on the SVM algorithm is used to estimate the number of people and their actions. Experimental results show that the average accuracy of the system for the people counting is 93.5%, and the activity recognition rate of one‐person and two‐person is 99.6% and 94.6% respectively.
... Additional hardware adds additional cost to the estimation process. There exist another procedures that depends on using WiFi to estimate the number of people as our case in this paper [10]- [20], all of them lack accuracy, and used a small number of people in testing. The authors in [21] proposed a framework to estimate the number of people in a specific area depending on the concept that the transmitted signal carries the signature of people, in other words, blocking the Line of Sight and Propagation effects. ...
WiFi access points are widely spread everywhere in all our daily life routines. Using these devices to provide services other than the Internet is becoming familiar nowadays.This paper conducts an experimental study to estimate the number of people in an indoor environment through two system setups, line of sight, and non-line of sight. Relationship modeling between WiFi received signal and the number of people uses polynomial regression. The experiment comprised of two stages: first is the data collection from a controlled number of people. Then, the collected data used to train the system through polynomial regression. The second is testing the system’s effectiveness by applying it to an uncontrolled environment. Testing results revealed efficiency in using WiFi received signal strength to do the people counting (up to 60) because of the accuracy achievements of 93.17% in the line of sight system. The non-line of sight system disclosed randomness in the received signal strength indicator regardless of the change in the number of people. The randomness is mainly caused by the fading effect of the concrete wall. Therefore it is inefficient to use the non-line of sight system in concrete buildings.
In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high demand on effective pose estimation. Though camera-based solutions yield remarkable performance, they encounter privacy issues and degraded performance caused by varying illumination, especially in the smart home. In this article, we propose a WiFi-based Internet of Things-enabled human pose estimation scheme for metaverse avatar simulation, namely, MetaFi++. Specifically, WPFormer is designed with a shared convolutional module and a Transformer block to map the channel state information of WiFi signals to human pose landmarks, effectively exploring spatial information of human pose through self-attention. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. Due to the ubiquitous existence of WiFi and robustness to various illumination conditions, WiFi-based human poses are suitable to instruct the movement of digital avatars in the metaverse, promoting avatar applications in smart homes. The experiments are conducted in the real world, and the results show that the MetaFi++ achieves very high performance with a PCK@50 of 97.30%. Our codes are available in
https://github.com/pridy999/metafi_pose_estimation
.
Purpose: The main purpose of this study was to examine the application of sensors and sensor networks for detection of people crowds in developing cities. This paper discusses unique challenges associated with people crowd detection especially in the urban towns of developing countries and gives a comparative review and analysis of popular human sensing approaches in the detection of people crowds. Methodology: This study provides a survey and categorization of popular human sensing approaches using literature especially published within the past two decades. The paper then analyzes current human sensing technologies vis-à-vis people crowd detection in developing cities. The respective strengths and shortfalls of various approaches are highlighted. Finally, by means of examples, a comparative analysis of different human sensing categories is carried out. Findings: The spontaneous, dynamic and chaotic nature of people crowds, together with the poor infrastructural development characteristic of developing economies pose unique challenges to the effectiveness of people crowd detection systems. Although there are advances in crowd detection, most of these are in the area of non-people crowds, while most of the research done on people crowd detection have been on indoor crowd settings. In addition, challenges unique to people crowd detection in developing countries include: scalability and cost of crowd detection systems, security of the detection system infrastructure, confidentiality of subjects being monitored, requirements for incentives and the ability to support passive and real time people crowd detection. Unique contribution to theory, practice and policy: This study emphasizes the need for both indoor and outdoor people crowd detection systems appropriate for the needs of developing cities. The study contributes to the body of knowledge since people crowds unlike other types of crowds present a unique set of challenges that call for special attention.
Accurate indoor crowd counting (ICC) is a key enabler to many smart home/office applications. Recent development of WiFi-based ICC technology relies on detecting the variation of wireless channel state information (CSI) caused by human motions and has gained increasing popularity due to its low hardware cost, reliability under all lighting conditions, and privacy preservation in sensing data processing. To attain high estimation accuracy, existing WiFi-based ICC methods often require a large amount of labeled CSI training data samples for each application domain, i.e., a particular WiFi transceiver or background deployment. This makes large-scale deployment of WiFi-based ICC technology across dissimilar domains extremely difficult and costly. In this paper, we propose a Domain-Agnostic and Sample-Efficient wireless indoor crowd Counting (DASECount) framework that suffices to attain robust cross-domain detection accuracy given very limited data samples in new domains. DASECount leverages the wisdom of few-shot learning (FSL) paradigm consisting of two major stages: source domain meta training and target domain meta testing. Specifically, in the meta-training stage, we design and train two separate convolutional neural network (CNN) modules on the source domain dataset to fully capture the implicit amplitude and phase features of CSI measurements related to human activities. A subsequent knowledge distillation procedure is designed to iteratively update the CNN parameters for better generalization performance. In the meta-testing stage, we use the partial CNN modules to extract low-dimension features out of the high-dimension input target domain CSI data. With the obtained low-dimension CSI features, we can even use very few shots of target domain data samples (e.g., 5-shot samples) to train a lightweight logistic regression (LR) classifier, and attain very high cross-domain ICC accuracy. Experiment results show that the proposed DASECount method achieves over 92.68%, and on average 96.37% detection accuracy in a 0-8 people counting task under various domain setups, which significantly outperforms the other representative benchmark methods considered.
Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included in
https://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark
.
Device-free activity recognition plays a crucial role in smart building, security, and human–computer interaction, which shows its strength in its convenience and cost-efficiency. Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models, but it suffers from the limitation of manual feature design. Deep learning overcomes such issues by automatic high-level feature extraction, but its performance degrades due to the requirement of massive annotated data and cross-site issues. To deal with these problems, transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics. This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition. We begin with the motivation of deep learning and transfer learning, and then introduce the major sensor modalities. Then the deep and transfer learning techniques for device-free human activity recognition are introduced. Eventually, insights on existing works and grand challenges are summarized and presented to promote future research.
As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.
Recently, wireless sensing is gaining immense attention in the Internet of things (IoT) for crowd counting and occupancy detection. As wireless signals propagate, they tend to scatter and reflect in various directions depending on the number of people in the indoor environment. The combined effect of these variations on wireless signals is characterized by the channel state information (CSI), which can be further exploited to identify the presence of people. State-of-the-art CSI-based supervised crowd counting systems are vulnerable to temporal and environmental dynamics in practical scenarios as their performance degrades with fluctuations in the indoor environments due to multipath fading. Inspired by the breakthroughs of transfer learning and advancement in edge computing, we have leveraged in this work the concept of transfer learning to minimize this problem via exploiting the trained model from source environment for other indoor environments to perform device-free crowd counting (CrossCount) at the target rooms. Our results show that this technique can combat the dynamics of the environment and achieves 4.7% better accuracy with 40% reduction in training time as compared to conventional convolutional neural networks. In essence, our results imply the future possibility of harnessing crowdsourced CSI data collected at different indoor environments to boost the accuracy and efficiency of local crowd counting systems.
Existing channel-state information (CSI)-based human authentication systems in the literature require a large amount of CSI data to train deep neural network (DNN) models and are ineffective for unknown intruder detection. To address this issue, we propose a CSI-based human authentication system (CAUTION) which is able to learn distinctive gait features of different users through CSI data to perform human authentication in this article. By taking advantage of few-shot learning, CAUTION is able to construct an accurate user identification model with a very limited number of CSI training data. By converting the CSI samples into low-dimensional representations on the feature plane, it computes central points for different users as their CSI profiles and introduces an intruder threshold to measure whether the CSI data matches one of the user classes by a margin. The intruder threshold is able to be optimized without any intruders’ data. CAUTION does not require a large number of training data and provides an effective way to train the system for unknown intruder detection. We have tested CAUTION at different places and compared it with state-of-the-art CSI-based authentication systems. The experimental results demonstrate that CAUTION is able to perform accurate human authentication with a limited amount of CSI training data (one-fifth of data needed by compared systems) and outperforms the compared human authentication systems.
With the widespread of commercial communication equipment, WiFi signals are ubiquitous in human life. Therefore, utilizing WiFi signals to implement intelligent sensing applications is an inevitable trend. In WiFi sensing applications, through-the-wall crowd counting is a challenging problem. In the through-the-wall scenario, the wireless signal transmitted through the wall will carry a lot of noises and is severely attenuated. Therefore, the influence of human activities on the wireless signal is difficult to extract. To solve this problem, we propose TWCC, a through-the-wall crowd counting system using ambient WiFi signals. TWCC utilizes commercial WiFi equipments to extract the phase difference data of the channel state information (CSI) and transform it to sense the environment. First, TWCC preprocesses the data to remove uncorrelated noise, and then combines the subcarrier correlation to achieve through-the-wall human detection. When people exist, TWCC extracts features from four domains as feature groups, namely time domain, subcarrier domain, frequency domain, and time-frequency domain. Then TWCC uses different backpropagation (BP) neural networks for the features of the four domains and combines with weighting and threshold judgment to realize the through-the-wall crowd counting detection. Extensive real-world experiments show that TWCC achieves an average recognition accuracy of about 90% and maintains strong robustness to different speeds and environments.
Visible light sensing technique has shown great potential in locating human in the building. For indoor positioning using fingerprints, how to reduce the offline labor cost while maintaining accurate location estimation is of vital importance. In this paper, a novel coarse fingerprint-assisted device-free localization (DFL) approach is proposed for locating multiple targets via visible light sensing, which does not need too much calibration effort to build high accurate fingerprint database. Furthermore, a successive cancellation algorithm based on the coarse single target fingerprint database is designed to address the problem of pseudo target positioning in the multi-target DFL. Since the intersection points of shadowed links could not be guaranteed to be symmetrically located around the center of mass of the human body, the estimated location would be biased from the true location of the target. To solve this problem, a convex hull-based localization algorithm is introduced to improve positioning accuracy. Extensive simulation results show that our proposed approach exhibits superior performance in terms of target counting and localization accuracy.
Driven by the Internet of Things (IoT), many device-free crowd density estimation techniques can roughly estimate the crowd density based on the relationship between the dynamic crowd and the variation of wireless signals. However, they cannot distinguish the path information of different persons in a fine-grained manner. In this paper, we propose Wisual, a Channel State Information (CSI) based device-free crowd density estimation framework and can visualize the distribution of people. The major challenge of Wisual is how to extract proper quantifiable indexes to distinguish the path information of multiple targets and maximize the resolution of crowd density estimation. To address this challenge, Wisual first presents a method for estimating the frequency of the CSI propagation path (FoC) for the moving persons and constructs a joint multi-feature parameter (JMFP) spectrum matrix with the other two parameters. Then the multi-target spectrum matrix is put into a proposed deep-learning model called CSI stream 3D convolutional neural networks (CS-3DCNN) for implementing crowd density estimation and the target path information differentiation. Finally, Wi-Fi imaging is implemented based on the 2D-MUSIC algorithm, which shows the approximate distribution situation of indoor persons through the spectrograms. The experimental results in typical real-world scenes demonstrate that Wisual can forecast the crowd density with 98% precision and accurately display the frequency spectra of moving persons. Besides, the results also prove the superior effectiveness, scalability, and generalizability of the proposed framework.
Channel State Information (CSI) based human activity recognition has received great attention in recent years due to its advantages in privacy protection, insensitivity to illumination, and no requirement for wearable devices. In this paper, we propose a Multimodal Channel State Information Based Activity Recognition (MCBAR) system that leverages existing WiFi infrastructures and monitors human activities from CSI measurements. MCBAR aims to address the performances degradation of WiFi-based human recognition systems due to environmental dynamics. Specifically, we address the issue of non-uniformly distributed unlabelled data with rarely-performed activities by taking advantages of the generative adversarial network (GAN) and semi-supervised learning. We apply a multimodal generator to approximate the CSI data distribution in different environment settings with limited measured CSI data. The generated CSI data using the multimodal generator can provide better diversity for knowledge transfer. This multimodal generator improves the ability of MCBAR to recognize specific activities with various CSI patterns caused by environmental dynamics. Compared to state-of-the-art CSI-based recognition systems, MCBAR is more robust as it is able to handle the non-uniformly distributed CSI data collected from a new environment setting. In addition, diverse generated data from the multimodal generator improves the stability of the system. We have tested MCBAR under multiple experimental settings at different places. The experimental results demonstrate that our algorithm overcomes environmental dynamics and outperforms existing human activity recognition systems.
Experiments show that operation efficiency and reliability of buildings can greatly benefit from rich and relevant datasets. More specifically, data can be analyzed to detect and diagnose system and component failures that undermine energy efficiency. Among the huge quantity of information, some features are more correlated with the failures than others. However there has been little research to date focusing on determining the types of data that can optimally support Fault Detection and Diagnosis (FDD). This paper presents a novel optimal feature selection method, named Information Greedy Feature Filter (IGFF), to select essential features that benefit building FDD. On one hand, the selection results can serve as reference for configuring sensors in the data collection stage, especially when the measurement resource is limited. On the other hand, with the most informative features selected by IGFF, the performance of building FDD could be improved and theoretically justified. A case study on Air Handling Unit (AHU) is conducted based on the dataset of the ASHRAE Research Project 1312. Numerical results show that, compared with several baselines, the FDD performances of conventional classification methods are greatly enhanced by IGFF.
Device-free localization without any device attached to the target is playing a critical role in many emerging applications. This paper presents an accurate model-based device-free localization system LiFS, implemented on cheap commercial off-the-shelf Wi-Fi devices. Unlike previous work, LiFS is able to localize a target accurately without offline training. The basic idea is simple: channel state information (CSI) is sensitive to a target's location and by modelling the CSI measurements of multiple wireless links as a set of power fading based equations, the target location can be determined. However, due to rich multipaths indoors, the received signal strength (RSS) or even the fine-grained CSI can not be easily modelled. We observe that even in a rich multipath environment, not all subcarriers are affected equally by multipaths. Our novel pre-processing scheme tries to identify the subcarriers not affected by multipath. Thus, CSIs on the identified ``clean'' subcarriers can be input into the proposed model for localization.
We design, implement and evaluate LiFS with extensive experiments in three different scenes. Without knowing the majority transceivers' locations, LiFS achieves a median accuracy of 0.5~m and 1.1~m in Line-of-Sight and Non-Line-of-Sight scenarios respectively, outperforming the state-of-the-art systems. Besides single target localization, LiFS is able to differentiate two sparsely-located targets and localize each of them at a high accuracy.
We propose Veto-Consensus Multiple Kernel Learning (VCMKL), a novel way of combining multiple kernels such that one class of samples is described by the logical intersection (consensus) of base kernelized decision rules, whereas the other classes by the union (veto) of their complements. The proposed configuration is a natural fit for domain description and learning with hidden subgroups. We first provide generalization risk bound in terms of the Rademacher complexity of the classifier, and then a large margin multi-ν learning objective with tunable training error bound is formulated. Seeing that the corresponding optimization is non-convex and existing methods severely suffer from local minima, we establish a new algorithm, namely Parametric Dual Descent Procedure (PDDP) that can approach global optimum with guarantees. The bases of PDDP are two theorems that reveal the global convexity and local explicitness of the parameterized dual optimum, for which a series of new techniques for parametric program have been developed. The proposed method is evaluated on extensive set of experiments, and the results show significant improvement over the state-of-the-art approaches.
Estimating an occupant's location is arguably the most fundamental sensing task in smart buildings. The applications for fine-grained, responsive building operations require the location sensing systems to provide location estimates in real time, also known as indoor tracking. Existing indoor tracking systems require occupants to carry specialized devices or install programs on their smartphone to collect inertial sensing data. In this paper, we propose MapSentinel, which performs non-intrusive location sensing based on WiFi access points and ultrasonic sensors. MapSentinel combines the noisy sensor readings with the floormap information to estimate locations. One key observation supporting our work is that occupants exhibit distinctive motion characteristics at different locations on the floormap, e.g., constrained motion along the corridor or in the cubicle zones, and free movement in the open space. While extensive research has been performed on using a floormap as a tool to obtain correct walking trajectories without wall-crossings, there have been few attempts to incorporate the knowledge of space use available from the floormap into the location estimation. This paper argues that the knowledge of space use as an additional information source presents new opportunities for indoor tracking. The fusion of heterogeneous information is theoretically formulated within the Factor Graph framework, and the Context-Augmented Particle Filtering algorithm is developed to efficiently solve real-time walking trajectories. Our evaluation in a large office space shows that the MapSentinel can achieve accuracy improvement of 31 . 3 % compared with the purely WiFi-based tracking system.
The location and contextual status (indoor or outdoor) is fundamental and critical information for upper-layer applications, such as activity recognition and location-based services (LBS) for individuals. In addition, optimizations of building management systems (BMS), such as the pre-cooling or heating process of the air-conditioning system according to the human traffic entering or exiting a building, can utilize the information, as well. The emerging mobile devices, which are equipped with various sensors, become a feasible and flexible platform to perform indoor-outdoor (IO) detection. However, power-hungry sensors, such as GPS and WiFi, should be used with caution due to the constrained battery storage on mobile device. We propose BlueDetect: an accurate, fast response and energy-efficient scheme for IO detection and seamless LBS running on the mobile device based on the emerging low-power iBeacon technology. By~leveraging the on-broad Bluetooth module and our proposed algorithms, BlueDetect provides a precise IO detection service that can turn on/off on-board power-hungry sensors smartly and automatically, optimize their performances and reduce the power consumption of mobile devices simultaneously. Moreover, seamless positioning and navigation services can be realized by it, especially in a semi-outdoor environment, which cannot be achieved by GPS or an indoor positioning system (IPS) easily. We~prototype BlueDetect on Android mobile devices and evaluate its performance comprehensively. The~experimental results have validated the superiority of BlueDetect in terms of IO detection accuracy, localization accuracy and energy consumption.
WiFi based Indoor Positioning System (IPS) has become the most popular and practical system to provide Location Based Service (LBS) in indoor environments due to the availability of massive existing WiFi network infrastructures in buildings. WiFi based IPSs leverage received signal strengths (RSSs) from large numbers of WiFi access points (APs) and estimate the location of clients. Proper AP selection methods are required to select a subset of APs that provide useful information for localization in order to reduce the computational load and preserve or even enhance the localization accuracy of the entire IPS. Existing approaches select and measure the discriminative ability of APs individually during the offline phase without the consideration of the dependence among them. In this paper, based on mutual information of APs, we present online mutual information (OnlineMI), a novel AP selection strategy that measures the collective discriminative ability among APs. Furthermore, the proposed AP selection process is conducted online to adapt various environmental dynamics. Weighted k nearest neighbor (WKNN) approach is further employed as the localization algorithm after the OnlineMI AP selection strategy. Extensive experiments are carried out, and performance evaluation and comparison with existing methods demonstrates the superiority of the proposed OnlineMI-WKNN approach.
Indoor Positioning System (IPS) has become one of the most attractive research fields due to the increasing demands on Location Based Services (LBSs) in indoor environments. Various IPSs have been developed under different circumstances, and most of them adopt the fingerprinting technique to mitigate pervasive indoor multipath effects. However, the performance of the fingerprinting technique severely suffers from device heterogeneity existing across commercial off-the-shelf mobile devices (e.g. smart phones, tablet computers, etc.) and indoor environmental changes (e.g. the number, distribution and activities of people, the placement of furniture, etc.). In this paper, we transform the Received Signal Strength (RSS) to a standardized location fingerprint based on the Procrustes analysis, and introduce a similarity metric, termed Signal Tendency Index (STI), for matching standardized fingerprints. An analysis on the capability of the proposed STI in handling device heterogeneity and environmental changes is presented. We further develop a robust and precise IPS by integrating the merits of both the STI and Weighted Extreme Learning Machine (WELM). Finally, extensive experiments are carried out and a performance comparison with existing solutions verifies the superiority of the proposed IPS in terms of robustness to device heterogeneity.
In this paper, we are interested in counting the total number of people walking in an area based on only WiFi received signal strength indicator (RSSI) measurements between a pair of stationary transmitter/receiver antennas. We propose a framework based on understanding two important ways that people leave their signature on the transmitted signal: blocking the line of sight (LOS) and scattering effects. By developing a simple motion model, we first mathematically characterize the impact of the crowd on blocking the LOS. We next probabilistically characterize the impact of the total number of people on the scattering effects and the resulting multipath fading component. By putting the two components together, we then develop a mathematical expression for the probability distribution of the received signal amplitude as a function of the total number of occupants, which will be the base for our estimation using Kullback-Leibler divergence. To confirm our framework, we run extensive indoor and outdoor experiments with up to and including nine people and show that the proposed framework can estimate the total number of people with a good accuracy with only a pair of WiFi cards and the corresponding RSSI measurements.
The increasing demands of location-based services have spurred the rapid development of indoor positioning system and indoor localization system interchangeably (IPSs). However, the performance of IPSs suffers from noisy measurements. In this paper, two kinds of robust extreme learning machines (RELMs), corresponding to the close-to-mean constraint, and the small-residual constraint, have been proposed to address the issue of noisy measurements in IPSs. Based on whether the feature mapping in extreme learning machine is explicit, we respectively provide random-hidden-nodes and kernelized formulations of RELMs by second order cone programming. Furthermore, the computation of the covariance in feature space is discussed. Simulations and real-world indoor localization experiments are extensively carried out and the results demonstrate that the proposed algorithms can not only improve the accuracy and repeatability, but also reduce the deviation and worst case error of IPSs compared with other baseline algorithms.
We present the results, experiences and lessons learned from comparing a diverse set of technical approaches to indoor localization during the 2014 Microsoft Indoor Localization Competition. 22 different solutions to indoor localization from different teams around the world were put to test in the same unfamiliar space over the course of 2 days, allowing us to directly compare the accuracy and overhead of various technologies. In this paper, we provide a detailed analysis of the evaluation study's results, discuss the current state-of-the-art in indoor localization, and highlight the areas that, based on our experience from organizing this event, need to be improved to enable the adoption of indoor location services.
Location-based services (LBS) have attracted a great deal of attention recently. Outdoor localization can be solved by the GPS technique, but how to accurately and efficiently localize pedestrians in indoor environments is still a challenging problem. Recent techniques based on WiFi or pedestrian dead reckoning (PDR) have several limiting problems, such as the variation of WiFi signals and the drift of PDR. An auxiliary tool for indoor localization is landmarks, which can be easily identified based on specific sensor patterns in the environment, and this will be exploited in our proposed approach. In this work, we propose a sensor fusion framework for combining WiFi, PDR and landmarks. Since the whole system is running on a smartphone, which is resource limited, we formulate the sensor fusion problem in a linear perspective, then a Kalman filter is applied instead of a particle filter, which is widely used in the literature. Furthermore, novel techniques to enhance the accuracy of individual approaches are adopted. In the experiments, an Android app is developed for real-time indoor localization and navigation. A comparison has been made between our proposed approach and individual approaches. The results show significant improvement using our proposed framework. Our proposed system can provide an average localization accuracy of 1 m.
In recent years, developing Indoor Positioning System (IPS) has become an attractive research topic due to the increasing demands on Location-Based Service (LBS) in indoor environment. Several advantages of Radio Frequency Identification (RFID) Technology, such as anti-interference, small, light and portable size of RFID tags, and its unique identification of different objects, make it superior to other wireless communication technologies for indoor positioning. However, certain drawbacks of existing RFID-based IPSs, such as high cost of RFID readers and active tags, as well as heavy dependence on the density of reference tags to provide the LBS, largely limit the application of RFID-based IPS. In order to overcome these drawbacks, we develop a cost-efficient RFID-based IPS by using cheaper active RFID tags and sensors. Furthermore, we also proposed three localization algorithms: Weighted Path Loss (WPL), Extreme Learning Machine (ELM) and integrated WPL-ELM. WPL is a centralized model-based approach which does not require any reference tags and provides accurate location estimation of the target effectively. ELM is a machine learning fingerprinting-based localization algorithm which can provide higher localization accuracy than other existing fingerprinting-based approaches. The integrated WPL-ELM approach combines the fast estimation of WPL and the high localization accuracy of ELM. Based on the experimental results, this integrated approach provides a higher localization efficiency and accuracy than existing approaches, e.g., the LANDMARC approach and the support vector machine for regression (SVR) approach.
This article describes a method for indoor positioning of human-carried active Radio Frequency Identification (RFID) tags based on the Sampling Importance Resampling (SIR) particle filtering algorithm. To use particle filtering methods, it is necessary to furnish statistical state transition and observation distributions. The state transition distribution is obstacle-aware and sampled from a precomputed accessibility map. The observation distribution is empirically determined by ground truth RSS measurements while moving the RFID tags along a known trajectory. From this data, we generate estimates of the sensor measurement distributions, grouped by distance, between the tag and sensor. A grid of 24 sensors is deployed in an office environment, measuring Received Signal Strength (RSS) from the tags, and a multithreaded program is written to implement the method. We discuss the accuracy of the method using a verification data set collected during a field-operational test.
Advances in low cost sensor and networking technologies in smart buildings have given researchers access to a multitude of time series data, including temperature, humidity, real and reactive power consumption of specific nodes or devices, occupant presence and activities, etc. Time series generated by sensor networks reflect various phenomena in buildings and are naturally related to each other. Hence quantitative techniques are required to exploit dependence among different types of sequences in order to allow smart applications such as non-intrusive activity detection, energy usage prediction, demand side management and control. Past research of relational analysis has focused on symmetric correlative statistics. On the other hand, asymmetric causal relations can capture more dynamic and complex relationships and is able to reveal directed influence among series. However, most traditional causal analysis relies on stationarity, while the statistics of real sensor measurement in smart buildings is rarely time invariant. In this paper, a statistical time series analysis framework is proposed to examine causal relationships among time series that are highly non-stationary. The Granger causality identification is extended to sensor data in buildings and the issue of non-stationarity is initially addressed by using modified Hodrick-Prescott (HP) filter which is able to extract simpler trend components. Subsequently, Autoregressive Integrated Moving Average model with exogenous variables (ARIMAX) model is trained for different components of two series. Finally, Granger causality is tested for both directions by F-statistics. The above procedure is performed on actual energy-consumption time series to exploit potential causal relations.
Buildings are known to be the largest consumers of electricity in the United States, and often times the dominant energy consumer is the HVAC system. Despite this fact, in most buildings the HVAC system is run using primitive static control algorithms based on fixed work schedules causing wasted energy during periods of low occupancy. In this paper we present a novel control architecture that uses occupancy sensing to guide the operation of a building HVAC system. We show how we can enable aggressive duty-cycling of building HVAC systems - that is, turn them ON or OFF - to save energy while meeting building performance requirements using inexpensive sensing and control methods. We have deployed our occupancy sensor network across an entire floor of a university building and our data shows several periods of low occupancy with significant opportunities to save energy over normal HVAC schedules. Furthermore, by interfacing with the building Energy Management System (EMS) directly and using real-time occupancy data collected by our occupancy nodes, we measure electrical energy savings of 9.54% to 15.73% and thermal energy savings of 7.59% to 12.85% for the HVAC system by controlling just one floor of our four floor building.
We are pleased to announce the release of a tool that records detailed measurements of the wireless channel along with received 802.11 packet traces. It runs on a commodity 802.11n NIC, and records Channel State Information (CSI) based on the 802.11 standard. Unlike Receive Signal Strength Indicator (RSSI) values, which merely capture the total power received at the listener, the CSI contains information about the channel between sender and receiver at the level of individual data subcarriers, for each pair of transmit and receive antennas.
Our toolkit uses the Intel WiFi Link 5300 wireless NIC with 3 antennas. It works on up-to-date Linux operating systems: in our testbed we use Ubuntu 10.04 LTS with the 2.6.36 kernel. The measurement setup comprises our customized versions of Intel's close-source firmware and open-source iwlwifi wireless driver, userspace tools to enable these measurements, access point functionality for controlling both ends of the link, and Matlab (or Octave) scripts for data analysis. We are releasing the binary of the modified firmware, and the source code to all the other components.
We propose Veto-Consensus Multiple Kernel Learning (VCMKL), a novel way of combining multiple kernels such that one class of samples is described by the logical intersection (consensus) of base kernelized decision rules, whereas the other classes by the union (veto) of their complements. The proposed configuration is a natural fit for domain description and learning with hidden subgroups. We first provide generalization risk bound in terms of the Rademacher complexity of the classifier, and then a large margin multi-ν learning objective with tunable training error bound is formulated. Seeing that the corresponding optimization is non-convex and existing methods severely suffer from local minima, we establish a new algorithm, namely Parametric Dual Descent Procedure (PDDP) that can approach global optimum with guarantees. The bases of PDDP are two theorems that reveal the global convexity and local explicitness of the parameterized dual optimum, for which a series of new techniques for parametric program have been developed. The proposed method is evaluated on extensive set of experiments, and the results show significant improvement over the state-of-the-art approaches.
We consider a class of non-convex learning problems that can be formulated as jointly optimizing regularized hinge loss and a set of auxiliary variables. Such problems encompass but are not limited to various versions of semi-supervised learning,learning with hidden structures, robust learning, etc. Existing methods either suffer from local minima or have to invoke anon-scalable combinatorial search. In this paper, we propose a novel learning procedure, namely Parametric Dual Maximization(PDM), that can approach global optimality efficiently with user specified approximation levels. The building blocks of PDM are two new results: (1) The equivalent convex maximization reformulation derived by parametric analysis.(2) The improvement of local solutions based on a necessary and sufficient condition for global optimality. Experimental results on two representative applications demonstrate the effectiveness of PDM compared to other approaches.
This paper presents a WiFi-based device-free crowd counting and occupancy estimation system that can be used in rooms/environments different from the ones in which the training process has been performed. Therefore, crowd counting is achieved without requiring another training phase in each new environment. The proposed approach analyzes the shape of the Doppler spectrum of the received signal which is correlated to the number of people moving in the monitored environment. Unlike a radar-like approach, the use of a reference signal is not required. Experimental results are presented for two different rooms/environments without any constraint on the movements of the volunteers.
The Global Positioning System (GPS) can be readily used for outdoor localization, but GPS signals are degraded in indoor environments. How to develop a robust and accurate indoor localization system is an emergent task. In this paper, we propose a smartphone inertial sensor-based indoor localization and tracking system with occasional iBeacon corrections. Some important issues in a smartphone-based pedestrian dead reckoning (PDR) approach, i.e., step detection, walking direction estimation, and initial point estimation, are studied. One problem of the PDR approach is the drift with walking distance. We apply a recent technology, iBeacon, to occasionally calibrate the drift of the PDR approach. By analyzing iBeacon measurements, we define an efficient calibration range where an extended Kalman filter is utilized. The proposed localization and tracking system can be implemented in resource-limited smartphones. To evaluate the performance of the proposed approach, real experiments under two different environments have been conducted. The experimental results demonstrated the effectiveness of the proposed approach. We also tested the localization accuracy with respect to the number of iBeacons.
Power delay profiles characterize multipath channel features, which are widely used in motion- or localization-based applications. Recent studies show that the power delay profile may be derived from the CSI traces collected from commodity WiFi devices, but the performance is limited by two dominating factors. The resolution of the derived power delay profile is determined by the channel bandwidth, which is however limited on commodity WiFi. The collected CSI reflects the signal distortions due to both the channel attenuation and the hardware imperfection. A direct derivation of power delay profiles using raw CSI measures, as has been done in the literature, results in significant inaccuracy. In this paper, we present Splicer, a software-based system that derives high-resolution power delay profiles by splicing the CSI measurements from multiple WiFi frequency bands. We propose a set of key techniques to separate the mixed hardware errors from the collected CSI measurements. Splicer adapts its computations within stringent channel coherence time and thus can perform well in presence of mobility. Our experiments with commodity WiFi NICs show that Splicer substantially improves the accuracy in profiling multipath characteristics, reducing the errors of multipath distance estimation to be less than $2m$. Splicer can immediately benefit upper-layer applications. Our case study with recent single-AP localization achieves a median localization error of $0.95m$.
Despite of several years of innovative research, indoor localization is still not mainstream. Existing techniques either employ cumbersome fingerprinting, or rely upon the deployment of additional infrastructure. Towards a solution that is easier to adopt, we propose CUPID, which is free from these restrictions, yet is comparable in accuracy. While existing WiFi based solutions are highly susceptible to indoor multipath, CUPID utilizes physical layer (PHY) information to extract the signal strength and the angle of only the direct path, successfully avoiding the effect of multipath reflections. Our main observation is that natural human mobility, when combined with PHY layer information, can help in accurately estimating the angle and distance of a mobile device from an wireless access point (AP). Real-world indoor experiments using off-the-shelf wireless chipsets confirm the feasibility of CUPID. In addition, while previous approaches rely on multiple APs, CUPID is able to localize a device when only a single AP is present. When a few more APs are available, CUPID can improve the median localization error to 2.7m, which is comparable to schemes that rely on expensive fingerprinting or additional infrastructure.
Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.
From rssi to csi: Indoor localization via channel response
- Z Yang
- Z Zhou
- Y Liu
Z. Yang, Z. Zhou, and Y. Liu, "From rssi to csi: Indoor localization
via channel response," ACM Computing Surveys (CSUR), vol. 46, no. 2,
p. 25, 2013.