Feng Jiang’s research while affiliated with Harbin Institute of Technology and other places

Referring expression comprehension (REC) aims at achieving object localization based on natural language descriptions. However, existing REC approaches are constrained by object category descriptions and single-attribute intention descriptions, hindering their application in real-world scenarios. In natural human-robot interactions, users often express their desires through individual states and intentions, accompanied by guiding gestures, rather than detailed object descriptions. To address this challenge, we propose Multi-ref EC, a novel task framework that integrates state descriptions, derived intentions, and embodied gestures to locate target objects. We introduce the State-Intention-Gesture Attributes Reference (SIGAR) dataset, which combines state and intention expressions with embodied references. Through extensive experiments with various baseline models on SIGAR, we demonstrate that properly ordered multi-attribute references contribute to improved localization performance, revealing that single-attribute reference is insufficient for natural human-robot interaction scenarios. Our findings underscore the importance of multi-attribute reference expressions in advancing visual-language understanding.

The excellent “mirror” effect of medium and high-strength aluminum alloy profiles from the 6-series, achieved through anodizing, is highly valued by customers. Metallographic analysis is a key method for predicting the anodizing effect. However, traditional metallographic analysis methods suffer from unstable accuracy and low efficiency. To address these issues, this paper successfully develops a metallographic grading system by constructing a dataset and integrating computer vision with machine-learning techniques. Based on grain classification, the system automatically determines the metallographic grade by analyzing the proportion of good grain areas. After applying SMOTE sampling and 10-fold cross-validation to the machine-learning algorithm, we conducted a comparative analysis of the model’s performance from the perspectives of accuracy, good grain recall rate, bad grain recall rate, and AUC. The XGBoost model, selected as the final predictive model from 18 machine-learning models due to its superior performance, achieved a grain classification accuracy of 96.21% and a good grain recall rate of 98.07%. Both the accuracy and good grain recall standard deviations were less than 0.02. These results indicate that the model can effectively distinguish between good and bad grains with high robustness. Additionally, the average time for metallographic grading is less than 9 s. In comparison to the instability of traditional manual grading, this method significantly enhances both the accuracy and efficiency of metallographic analysis while also reducing grading costs.

Graph Neural Network (GNN) has been widely applied in multivariate time series forecasting due to its excellent relationship modeling capabilities. However, current methods still face limitations in computational efficiency or time series expression capabilities. To address these issues, we propose an Efficient Graph-Enhanced Neural Network (EGENN), which consists of an adjacency matrix generator, GNN, and projection module. Firstly, EGENN designs a spectral similarity-based graph construction method and further enhances the expressive power of temporal features. Secondly, we introduce an inter-layer attention graph convolutional network, which adaptively aggregates information from different network depths to better capture complex patterns. Finally, a predictive projection strategy fusing wavelet convolutions and patch-wise transformation is proposed to produce compact parameterization and extended receptive fields. Experiments on five datasets from different domains show that our model achieves state-of-the-art prediction performance while maintaining low computational resource consumption.

Accurate and reliable human pose estimation (HPE) is essential in interactive systems, particularly for applications requiring personalized adaptation, such as controlling cooperative robots and wearable exoskeletons, especially for healthcare monitoring equipment. However, continuously maintaining diverse datasets and frequently updating models for individual adaptation are both resource intensive and time-consuming. To address these challenges, we propose a meta-transfer learning framework that integrates multimodal inputs, including high-frequency surface electromyography (sEMG), visual-inertial odometry (VIO), and high-precision image data. This framework improves both accuracy and stability through a knowledge fusion strategy, resolving the data alignment issue, ensuring seamless integration of different modalities. To further enhance adaptability, we introduce a training and adaptation framework with few-shot learning, facilitating efficient updating of encoders and decoders for dynamic feature adjustment in real-time applications. Experimental results demonstrate that our framework provides accurate, high-frequency pose estimations, particularly for intra-subject adaptation. Our approach enables efficient adaptation to new individuals with only a few new samples, providing an effective solution for personalized motion analysis with minimal data.

Recently, a large number of image compressive sensing (CS) methods with deep unfolding networks (DUNs) have been proposed. However, existing methods either use fixed-scale blocks for sampling that leads to limited insights into the image content or employ a plain convolutional neural network (CNN) in each iteration that weakens the perception of broader contextual prior. In this paper, we propose a novel DUN (dubbed SVASNet) for image compressive sensing, which achieves scale-variable adaptive sampling and hybrid-attention Transformer reconstruction with a single model. Specifically, for scale-variable sampling, a sampling matrix-based calculator is first employed to evaluate the reconstruction distortion, which only requires measurements without access to the ground truth image. Then, a Block Scale Aggregation (BSA) strategy is presented to compute the reconstruction distortion under block divisions at different scales and select the optimal division scale for sampling. To realize hybrid-attention reconstruction, a dual Cross Attention (CA) submodule in the gradient descent step and a Spatial Attention (SA) submodule in the proximal mapping step are developed. The CA submodule introduces inter-phase inertial forces in the gradient descent, which improves the memory effect between adjacent iterations. The SA submodule integrates local and global prior representations of CNN and Transformer, and explores local and global affinities between dense feature representations. Extensive experimental results show that the proposed SVASNet achieves significant improvements over the state-of-the-art methods.

Human trust plays a crucial role in the effectiveness of human-robot collaboration. Despite its significance, the development and maintenance of an optimal trust level are obstructed by the complex nature of influencing factors and their mechanisms. This study investigates the effects of cognitive load on human trust within the context of a hybrid human-robot collaboration task. An experiment is conducted where the humans and the robot, acting as team members, collaboratively construct pyramids with differentiated levels of task complexity. Our findings reveal that cognitive load exerts diverse impacts on human trust in the robot. Notably, there is an increase in human trust under conditions of high cognitive load. Furthermore, the rewards for performance are substantially higher in tasks with high cognitive load compared to those with low cognitive load, and a significant correlation exists between human trust and the failure risk of performance in tasks with low and medium cognitive load. By integrating interdependent task steps, this research emphasizes the unique dynamics of hybrid human-robot collaboration scenarios. The insights gained not only contribute to understanding how cognitive load influences trust but also assist developers in optimizing collaborative target selection and designing more effective human-robot interfaces in such environments.