Shijia Pan

University of California, Merced | UCM · Department of Computer Science and Engineering

IoT (Internet of Things) devices, such as network-enabled wearables, are carried by increasingly more people throughout daily life. Information from multiple devices can be aggregated to gain insights into a person’s behavior or status. For example, an elderly care facility could monitor patients for falls by combining fitness bracelet data with vi...

In this paper, we present MuteIt, an ear-worn system for recognizing unvoiced human commands. MuteIt presents an intuitive alternative to voice-based interactions that can be unreliable in noisy environments, disruptive to those around us, and compromise our privacy. We propose a twin-IMU set up to track the user's jaw motion and cancel motion arti...

Buildings account for more than 40% of the energy US primary energy consumption. Of all the building services, heating, ventilation, and air-conditioning (HVAC) account for almost 50% of that energy use. Despite all the resources used, many users are not satisfied with the comfort conditions in buildings. The main problems for this lack of balance...

Internet-of-Things (IoT) systems have become pervasive for smart homes. In recent years, many of these IoT sensing systems are developed to enable in-home long-term monitoring applications, such as personalized services in smart homes, elderly/patient monitoring, etc. However, these systems often require complicated and expensive installation proce...

In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect s...

In this paper, we characterize the effects of obstructions on footstep-induced floor vibrations to enable obstruction-invariant indoor occupant localization. Occupant localization is important in smart building applications such as smart healthcare and energy management. Maintenance and installment requirements limit the application of current sens...

In this paper, we characterize the effects of obstructions on footstep-induced floor vibrations to enable obstruction-invariant indoor occupant localization. Occupant localization is important in smart building applications such as healthcare and energy management. In prior works, footstep-based vibration sensing has been introduced for non-intrusi...

Non-intrusive monitoring of fine-grained activities of daily living (ADL) enables various smart healthcare applications. For example, ADL pattern analysis for older adults at risk can be used to assess their loss of safety or independence. Prior work in the area of ADL recognition has focused on coarse-grained ADL recognition at the context-level (...

A common pain point for physical retail stores is live inventory monitoring, i.e., knowing how many items of each product are left on the shelves. About 4% of sales are lost due to an average 5–10% out-of-shelf stockout rate, while additional supplies existed in the warehouse. Traditional techniques rely on manual inspection, per-item tagging using...

Fine-grained human activities recognition focuses on recognizing event-or action-level activities, which enables a new set of Internet-of-Things (IoT) applications such as behavior analysis. Prior work on fine-grained human activities recognition relies on supervised sensing, which makes the fine-grained labeling labor-intensive and difficult to sc...

This paper introduces a window-based sequence-to-one approach with dynamic voting for nurse care activity recognition using acceleration-based wearable sensors. Nurse care activity recognition is an essential part of ensuring high quality patient care and providing constructive and concrete feedback to the care team. Some of the current sensing app...

The resolution of GPS measurements, especially in urban areas, is insufficient for identifying a vehicle's lane. In this work, we develop a deep LSTM neural network model LaNet that determines the lane vehicles are on by periodically classifying accelerometer samples collected by vehicles as they drive in real time. Our key finding is that even adj...

The signal quality of real-world infrastructure sensing systems varies significantly over the deployment conditions and hardware implementation [2]. Quantifying signal quality allows further optimization of sensing configuration and deployment to enhance the application performance. In our previous work, we choose a standard excitation and apply it...

This paper presents a floor-vibration-based step-level occupant-detection approach that enables detection across different structures through model transfer. Detecting the occupants through detecting their footsteps (i.e., step-level occupant detection) is useful in various smart building applications such as senior/healthcare and energy management...

Fine-grained non-intrusive monitoring of activities of daily living (ADL) enables various smart building applications, including ADL pattern assessments for older adults at risk for loss of safety or independence. Prior work in this area has focused on coarsegrained ADL recognition at the activity level (e.g., cooking, cleaning, sleeping), and/or c...

Auto-checkout technology could revolutionize physical retail by bringing down operating costs and enabling automated stores. The first step towards fully autonomous stores is automated live inventory monitoring. Existing automated approaches tend to focus on vision only and are cost prohibitive, slow, or inaccurate for practical real-world applicat...

A common pain point for physical retail stores is live inventory monitoring, i.e. knowing how many items of each product are left on the shelves. About 4% of sales are lost due to an average 5-10% out-of-shelf stockout rate, while additional supplies existed in the warehouse. Traditional techniques rely on manual inspection, per-item tagging using...

Sensing signal quality affects signal processing efficiency, feature extraction, and learning accuracy. An efficient and accurate assessment of sensing system signal quality is essential for 1) large-scale cyber-physical system deployment and 2) datasets sharing and comparison. In this paper, we present a signal quality assessment -- S-score -- for...

Structural vibration-based human sensing provides an alternative approach for device-free human monitoring, which is used for healthcare, space and energy usage management, etc. Prior work on this approach mainly focused on one person walking scenarios, which limits their widespread application. The challenge with multiple walkers is that the obser...

Sleep disorder impairs people's health. To better analyze user's sleep quality, it is important to monitor people's sleep stages. Prior methods, such as polysomnography (PSG) and Fitbit, are often intrusive and having potential to change user's daily sleep routine. We present a non-intrusive approach to identify sleep stages through bed-frame vibra...

A simple hardware array for obtaining highresolution structural vibration data will enable future civil infrastructures to become general sensing platforms that are easy and practical to deploy and maintain over the lifetime of a building.

Internet of Things (IoT) provides streaming, large-amount, and multimodal sensing data over time. The statistical properties of these data are always characterized very differently by time and sensing modalities, which are hardly captured by conventional learning methods. Continual and multimodal learning allows integrating, adapting, and generaliz...

In the real-world ubiquitous computing systems, it is difficult to require a significant amount of data to obtain accurate information through pure data-driven methods. Performance of data-driven methods relies on the quantity and 'quality' of data. They perform well when sufficient amount of data is available, which is regarded as ideal conditions...

With the growth of wearable devices, numerous health and smart building applications are enabled. As a result, many people wear multiple devices for different applications, such as fitness tracking. Being able to match devices' physical identity (e.g., smartwatch on Bob's left forearm) to their virtual identity (e.g., IP 192.168.0.22) is often a re...

Secure pairing is an important problem especially due to large number of IoT devices. In this paper, we propose PosePair++, to enable a camera to securely pair with IoT devices which are equipped with IMU sensors. Existing context-based pairing approaches do not adequately address this problem due to differing sensing modalities. To address this ch...

Gait health monitoring is critical for condition diagnosis and fall prediction in elderly populations. Existing methods for gait health monitoring (e.g. direct observation and sensing) are not suitable for non-clinical environments due to qualitative assessments or operational limitations. Our method utilizes footstep-induced floor vibration sensin...

We present Deskbuddy, a vibration-based system that can track a user's activities through their desk. Tracking sitting and other office related activities let us remind the user to have healthier working habits, as well as giving information about how office spaces are used. Many solutions have been proposed for office activity tracking, but they e...

Context-aware mobile application (App) usage prediction benefits a variety of applications such as precise bandwidth allocation, App launch acceleration, etc. Prior works have explored this topic through individual data profiles and contextual information. However, it is still a challenging problem because of the following three aspects: i. App usa...

This paper presents an indoor area occupancy counting system utilizing the ambient structural vibration induced by pedestrian footsteps. Our system achieves 99.55% accuracy in pedestrian footsteps detection, 0.2 people mean estimation error in pedestrian traffic estimation, and 0.2 area occupant activity estimation error in real-world uncontrolled...

Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute, a novel occupant identification solution that makes use of existing and prevalent on-object senso...

Individual perspiration level indicates a person’s physical status as well as their comfort level. Therefore, continuous perspiration level measurement enables people to monitor these conditions for applications including fitness assessment, athlete physical status monitoring, and patient/elderly care. Prior work on perspiration (sweat) sensing req...

Automated monitoring of animal behavior can detect changes in animal welfare and health problems. Different animal behaviors can point to disease, unrest or inadequate management, and detecting such behaviors in real time allows automated monitoring to be a valuable tool in livestock production [2]. When monitoring pigs, important sow events such a...

Continuous heart rate monitoring in cars can allow ambient health monitoring and help track driver stress and fatigue. We present a data set from accelerometers embedded in a car seat which includes ten people sitting in the passenger seat of a moving car, and surface ECG data of each user to provide ground truth of the heartbeat. This data can be...

Occupant activity level plays an important part in many smart building applications, such as power management [5] and elderly care [1]. There are several approaches to obtain occupant states, including visual-, acoustic-, and radio frequency based techniques. However, visual-based approaches require light-of-sight, while acoustic-based approaches a...

Occupant activity level information is essential in many smart home applications, such as energy management and elderly care. Various methods have been proposed for detecting occupant activities through vision-, acoustic-, or radio frequency-based methods. However, the visual-based methods function only when occupants are in the visual field, the a...

Perspiration level monitoring enables numerous applications such as physical condition estimation, personal comfort monitoring, health/exercise monitoring, and inference of environmental conditions of the user. Prior works on perspiration (sweat) sensing require users to manually hold a device or attach adhesive sensors directly onto their skin, li...

Large-scale fine-grained air pollution information has both financial for city managers and health benefits for all city residents. Sensors installed on fleet of vehicles (like taxis) to collect air pollution data provides a low cost, low-maintenance and a potentially high coverage approach. The challenges here are: 1) Sensing coverage is sparse in...

Real-world ubiquitous computing systems face the challenge of requiring a significant amount of data to obtain accurate information through pure data-driven approaches. The performance of these data-driven systems greatly depends on the quantity and 'quality' of data. In ideal conditions, pure data-driven methods perform well due to the abundance o...

Swarms of Unmanned Aerial Vehicles (drones) could provide great benefit when used for disaster response and indoor search and rescue scenarios. In these harsh environments where GPS availability cannot be ensured, prior work often relies on cameras for control and localization. This creates the challenge of identifying each drone, i.e., finding the...

In this paper, we present an occupant localization approach through sensing footstep-induced floor vibrations. Occupant location information is an important part of many smart building applications, such as energy and space management in a personal home or patient tracking in a hospital room. Adoption of current occupant location sensing approaches...

Easily establishing pairing between Internet-of-Things (IoT) devices is important for fast deployment in many smart home scenarios. Traditional pairing methods, including passkey, QR code, and RFID, often require specific user interfaces, surface's shape/material, or additional tags/readers. The growing number of low-resource IoT devices without an...

Continuous heart rate variability (HRV) monitoring in cars can allow ambient health monitoring and help track driver stress and fatigue. Current approaches that involve wearable or externally mounted sensors are accurate but inconvenient for the user. In particular, prior approaches often fail when noise from human motion or car noise are present....

Many human activities induce excitations on ambient structures with various objects, causing the structures to vibrate. Accurate vibration excitation source detection and characterization enable human activity information inference, hence allowing human activity monitoring for various smart building applications. By utilizing structural vibrations,...

Indoor emergency response situations, such as urban fire, are characterized by dangerous constantly changing operating environments with little access to situational information for first responders. In situ information about the conditions, such as the extent and evolution of an indoor fire, can augment rescue efforts and reduce risk to emergency...

Occupant identification proves crucial in many smart home applications such as automated home control and activity recognition. Previous solutions are limited in terms of deployment costs, identification accuracy, or usability. We propose SenseTribute, a novel occupant identification solution that makes use of existing and prevalent on-object senso...

We present FootprintID, an indoor pedestrian identification system that utilizes footstep-induced structural vibration to infer pedestrian identities for enabling various smart building applications. Previous studies have explored other sensing methods, including vision-, RF-, mobile-, and acoustic-based methods. They often require specific sensing...

This paper presents a collaboratively adaptive vibration monitoring system that captures high-fidelity structural vibration signals induced by pedestrians. These signals can be used for various human activities' monitoring by inferring information about the impact sources, such as pedestrian footsteps, door opening and closing, and dragging objects...

Touch surfaces are intuitive interfaces for computing devices. Most of the traditional touch interfaces (vision, IR, capacitive, etc.) have mounting requirements, resulting in specialized touch surfaces limited by their size, cost, and mobility. More recent work has shown that vibration-based touch sensing techniques can localize taps/knocks, which...