Fangting Xie’s research while affiliated with University of Science and Technology of China and other places

The measurement of bio-mechanical loads is important for various applications, such as healthcare, sports, and human-computer interaction. To obtain a detailed view of the bio-mechanical loads, it is necessary to monitor the strain distribution on the body surface which is perpendicular to the pressure. The current strain sensing methods lack scalability and comfort for body-scaled usage. We design a fully-textile strain sensor matrix that can be realized based solely on off-the-shelf materials and textile industry machines. The novel matrix design reduces the wiring scale to the square root of the sensor unit scale, enabling a practical, portable, and wearable deployment of strain distribution sensing. We created a matrix with 256 sensing points and explored its applications in both environmental and wearable scenarios. The results prove the necessity of measuring strain distribution, that the three-dimensional bio-mechanical loads shall be obtained in detail. The data from the on-body applications also show good performance in human posture classification, motion monitoring, and environmental condition recognition.

Long-term in-bed monitoring benefits automatic and real-time health management within healthcare, and the advancement of human shape reconstruction technologies further enhances the representation and visualization of users' activity patterns. However, existing technologies are primarily based on visual cues, facing serious challenges in non-light-of-sight and privacy-sensitive in-bed scenes. Pressure-sensing bedsheets offer a promising solution for real-time motion reconstruction. Yet, limited exploration in model designs and data have hindered its further development. To tackle these issues, we propose a general framework that bridges gaps in data annotation and model design. Firstly, we introduce SMPLify-IB, an optimization method that overcomes the depth ambiguity issue in top-view scenarios through gravity constraints, enabling generating high-quality 3D human shape annotations for in-bed datasets. Then we present PI-HMR, a temporal-based human shape estimator to regress meshes from pressure sequences. By integrating multi-scale feature fusion with high-pressure distribution and spatial position priors, PI-HMR outperforms SOTA methods with 17.01mm Mean-Per-Joint-Error decrease. This work provides a whole

Poses are the fundamentals of human activities and there are growing applications in healthcare, fitness and virtual reality. Despite massive advances in estimating human poses using cameras, these approaches are not suitable in open areas where people could move freely. Recent advances in wearable pressure sensing systems bring the possibility to estimate human poses in open areas in a more comfortable way compared with existing IMU approaches. In this study, using a textile-based full-body pressure sensing garment, we collected synchronized pressure and visual data pairs of various human poses. Using a camera-based pose estimation model to generate pose labels, we designed and implemented a deep learning pipeline to infer 3D human poses using only the full-body pressure data. The pipeline is evaluated using leave-one-out cross validation and it has 98.71mm joint position error under unseen-participant scenarios. We demonstrate the feasibility of full-body pressure sensing system in estimating human poses and showed that the smart garment could be a possible alternative in estimating human poses in open areas.

Pressure-sensitive smart textiles are widely applied in the fields of healthcare, sports monitoring, and intelligent homes. The integration of devices embedded with pressure sensing arrays is expected to enable comprehensive scene coverage and multi-device integration. However, the implementation of identity recognition, a fundamental function in this context, relies on extensive device-specific datasets due to variations in pressure distribution across different devices. To address this challenge, we propose a novel user identification method based on contrastive learning. We design two parallel branches to facilitate user identification on both new and existing devices respectively, employing supervised contrastive learning in the feature space to promote domain unification. When encountering new devices, extensive data collection efforts are not required; instead, user identification can be achieved using limited data consisting of only a few simple postures. Through experimentation with two 8-subject pressure datasets (BedPressure and ChrPressure), our proposed method demonstrates the capability to achieve user identification across 12 sitting scenarios using only a dataset containing 2 postures. Our average recognition accuracy reaches 79.05%, representing an improvement of 2.62% over the best baseline model.

Wearable sensing fabrics have great potential for applications such as human-computer interaction, motion monitoring, and human shape reconstruction. However, existing fabric sensors tend to sacrifice wearing comfort for sensing functionality, which restricts their application scenarios and hampers long-term usability due to poor wearability. To address this challenge, a novel fabric structure was designed to work with a flat knitting machine to realize a single-ply and knit-only textile pressure-sensing matrix. The sensing fabric has a sensing range of 0.255 to 35.65 kPa, with a maximum sensitivity of 0.72 kPa ⁻¹ . Although its sensing performance fluctuates under fatigue testing, washing and drying, folding, and stretching operations, it still supports use. It also has thermal comfort performance comparable to a regular T-shirt. We produced a pair of sensing shorts containing 256 sensing units and collected a total of 224,483 frames of data containing 18 postures from 6 participants. Posture classification using ResNet-18 achieved 88.2% accuracy.

Humans spend about one-third of their lives resting. Reconstructing human dynamics in in-bed scenarios is of considerable significance in sleep studies, bedsore monitoring, and biomedical factor extractions. However, the mainstream human pose and shape estimation methods mainly focus on visual cues, facing serious issues in non-line-of-sight environments. Since in-bed scenarios contain complicated human-environment contact, pressure-sensing bedsheets provide a non-invasive and privacy-preserving approach to capture the pressure distribution on the contact surface, and have shown prospects in many downstream tasks. However, few studies focus on in-bed human mesh recovery. To explore the potential of reconstructing human meshes from the sensed pressure distribution, we first build a high-quality temporal human in-bed pose dataset, TIP, with 152K multi-modality synchronized images. We then propose a label generation pipeline for in-bed scenarios to generate reliable 3D mesh labels with a SMPLify-based optimizer. Finally, we present PIMesh, a simple yet effective temporal human shape estimator to directly generate human meshes from pressure image sequences. We conduct various experiments to evaluate PIMesh's performance, showing that PIMesh archives 79.17mm joint position errors on our TIP dataset. The results demonstrate that the pressure-sensing bedsheet could be a promising alternative for long-term in-bed human shape estimation.