Xin Guo’s research while affiliated with Zhengzhou University and other places

Human skeleton-based action recognition is an important task in the field of computer vision. In recent years, masked autoencoder (MAE) has been used in various fields due to its powerful self-supervised learning ability and has achieved good results in masked data reconstruction tasks. However, in visual classification tasks such as action recognition, the limited ability of the encoder to learn features in the autoencoder structure results in poor classification performance. We propose to enhance the encoder's feature extraction ability in classification tasks by leveraging the latent space of variational autoencoder (VAE) and further replace it with the latent space of vector quantized variational autoencoder (VQVAE). The constructed models are called SkeletonMVAE and SkeletonMVQVAE, respectively. In SkeletonMVAE, we constrain the latent variables to represent features in the form of distributions. In SkeletonMVQVAE, we discretize the latent variables. These help the encoder learn deeper data structures and more discriminative and generalized feature representations. The experiment results on the NTU-60 and NTU-120 datasets demonstrate that our proposed method can effectively improve the classification accuracy of the encoder in classification tasks and its generalization ability in the case of few labeled data. SkeletonMVAE exhibits stronger classification ability, while SkeletonMVQVAE exhibits stronger generalization in situations with fewer labeled data.

Human skeleton-based action recognition is an important task in the field of computer vision. In recent years, masked autoencoder (MAE) has been used in various fields due to its powerful self-supervised learning ability and has achieved good results in masked data reconstruction tasks. However, in visual classification tasks such as action recognition, the limited ability of the encoder to learn features in the autoencoder structure results in poor classification performance. We propose to enhance the encoder's feature extraction ability in classification tasks by leveraging the latent space of variational autoencoder (VAE) and further replace it with the latent space of vector quantized variational autoencoder (VQVAE). The constructed models are called SkeletonMVAE and SkeletonMVQVAE, respectively. In SkeletonMVAE, we constrain the latent variables to represent features in the form of distributions. In SkeletonMVQVAE, we discretize the latent variables. These help the encoder learn deeper data structures and more discriminative and generalized feature representations. The experiment results on the NTU-60 and NTU-120 datasets demonstrate that our proposed method can effectively improve the classification accuracy of the encoder in classification tasks and its generalization ability in the case of few labeled data. SkeletonMVAE exhibits stronger classification ability, while SkeletonMVQVAE exhibits stronger generalization in situations with fewer labeled data.

Industrial fault diagnosis faces unique challenges with high-dimensional data, long time-series, and complex couplings, which are characterized by significant information entropy and intricate information dependencies inherent in datasets. Traditional image processing methods are effective for local feature extraction but often miss global temporal patterns, crucial for accurate diagnosis. While deep learning models like Vision Transformer (ViT) capture broader temporal features, they struggle with varying fault causes and time dependencies inherent in industrial data, where adding encoder layers may even hinder performance. This paper proposes a novel global and local feature fusion sequence-aware ViT (GLF-ViT), modifying feature embedding to retain sampling point correlations and preserve more local information. By fusing global features from the classification token with local features from the encoder, the algorithm significantly enhances complex fault diagnosis. Experimental analyses on data segment length, network depth, feature fusion and attention head receptive field validate the approach, demonstrating that a shallower encoder network is better suited for high-dimensional time-series fault diagnosis in complex industrial processes compared to deeper networks. The proposed method outperforms state-of-the-art algorithms on the Tennessee Eastman (TE) dataset and demonstrates excellent performance when further validated on a power transmission fault dataset.

In recent years, few-shot detection has become a popular research direction in the field of industrial defect detection, which aims to perform defect detection tasks accurately using a limited number of labeled samples. The dual-branch architecture, which utilizes class center features to aggregate query features is one of the most commonly used methods to solve the few-shot detection problem due to its simplicity and effectiveness. Previous few-shot detection algorithms based on dual-branch architecture typically aggregate features by directly using the features of labeled samples or by performing simple averaging operations on those features to compute the class center features. However, the acquisition of class center features is related to the sample distribution of novel classes, resulting in poor robustness of such methods. Additionally, the regression of bounding boxes and the classification of objects are two coupled problems that are difficult to optimize simultaneously. Therefore, this paper proposes a decoupled few-shot detection algorithm to address the challenges that exist in the current few-shot detection. More specifically, we propose a Vector Quantization Feature Aggregation (VQFA) method, which utilizes vector quantization to map the features of support images into discrete vector sets and each vector comes from a learnable codebook. By employing this operation, we can acquire a category representation that is both robust and accurate, thereby enabling enhanced interactions between support and query features. Besides, we propose a Decoupled Few-Shot (DeFS) module to decouple classification and localization tasks so that the two tasks can obtain different visual regions to achieve better detection results. Experimental results on the public dataset NEU-DET and GC10-DET demonstrate that our method significantly outperforms other methods on various few-shot scenes, which proves the effectiveness of our proposed method.

Semantic segmentation is a fundamental task in multimedia processing, which can be used for analyzing, understanding, editing contents of images and videos, among others. To accelerate the analysis of multimedia data, existing segmentation researches tend to extract semantic information by progressively reducing the spatial resolutions of feature maps. However, this approach introduces a misalignment problem when restoring the resolution of high-level feature maps. In this paper, we design a Semantic Refinement Module (SRM) to address this issue within the segmentation network. Specifically, SRM is designed to learn a transformation offset for each pixel in the upsampled feature maps, guided by high-resolution feature maps and neighboring offsets. By applying these offsets to the upsampled feature maps, SRM enhances the semantic representation of the segmentation network, particularly for pixels around object boundaries. Furthermore, a Contextual Refinement Module (CRM) is presented to capture global context information across both spatial and channel dimensions. To balance dimensions between channel and space, we aggregate the semantic maps from all four stages of the backbone to enrich channel context information. The efficacy of these proposed modules is validated on three widely used datasets-Cityscapes, Bdd100K, and ADE20K-demonstrating superior performance compared to state-of-the-art methods. Additionally, this paper extends these modules to a lightweight segmentation network, achieving an mIoU of 82.5% on the Cityscapes validation set with only 137.9 GFLOPs.

Low computational complexity and high segmentation accuracy are both essential to the real-world semantic segmentation tasks. However, to speed up the model inference, most existing approaches tend to design light-weight networks with a very limited number of parameters, leading to a considerable degradation in accuracy due to the decrease of the representation ability of the networks. To solve the problem, this paper proposes a novel semantic segmentation method to improve the capacity of obtaining semantic information for the light-weight network. Specifically, a feature refinement module (FRM) is proposed to extract semantics from multi-stage feature maps generated by the backbone and capture non-local contextual information by utilizing a transformer block. On Cityscapes and Bdd100K datasets, the experimental results demonstrate that the proposed method achieves a promising trade-off between accuracy and computational cost, especially for Cityscapes test set where 80.4% mIoU is achieved and only 214.82 GFLOPs are required.

Semantic segmentation is a fundamental task in multimedia processing, which can be used for analyzing, understanding, editing contents of images and videos, among others. To accelerate the analysis of multimedia data, existing segmentation researches tend to extract semantic information by progressively reducing the spatial resolutions of feature maps. However, this approach introduces a misalignment problem when restoring the resolution of high-level feature maps. In this paper, we design a Semantic Refinement Module (SRM) to address this issue within the segmentation network. Specifically, SRM is designed to learn a transformation offset for each pixel in the upsampled feature maps, guided by high-resolution feature maps and neighboring offsets. By applying these offsets to the upsampled feature maps, SRM enhances the semantic representation of the segmentation network, particularly for pixels around object boundaries. Furthermore, a Contextual Refinement Module (CRM) is presented to capture global context information across both spatial and channel dimensions. To balance dimensions between channel and space, we aggregate the semantic maps from all four stages of the backbone to enrich channel context information. The efficacy of these proposed modules is validated on three widely used datasets-Cityscapes, Bdd100K, and ADE20K-demonstrating superior performance compared to state-of-the-art methods. Additionally, this paper extends these modules to a lightweight segmentation network, achieving an mIoU of 82.5% on the Cityscapes validation set with only 137.9 GFLOPs.

Graph convolutional networks (GCNs) are an effective skeleton-based human action recognition (HAR) technique. GCNs enable the specification of CNNs to a non-Euclidean frame that is more flexible. The previous GCN-based models still have a lot of issues: (I) The graph structure is the same for all model layers and input data.

Accurate channel state information is crucial for directional beamforming in reconfigurable intelligent surface (RIS) assisted millimeter wave (mmWave) multiple-input multiple-output (MIMO) systems. In realistic scenarios, the components of RIS are susceptible to coverage by small particles, leading to blockage of certain elements, which increases the difficulty of channel estimation. In such cases, we propose a two-stage channel training scheme based on Kronecker decomposition to achieve joint RIS blockage and channel estimation. Specifically, we utilize the regularization parameter to recover the sparse blockage coefficients effectively. Additionally, we introduce a denoising algorithm to accelerate the convergence rate of sparse recovery and enhance estimation accuracy. Simulation results show that our proposed method outperforms the existing algorithms regarding RIS blockage and channel estimation accuracy.