Xianrui Liu’s research while affiliated with Shanghai University of Engineering Science and other places

Person Re-Identification (Re-ID) technology is being developed rapidly due to the successful application of deep convolutional neural networks. However, the prevailing Re-ID models are usually built upon manually design backbones. In this paper, we propose using the TGAS-ReID which is automatically designed convolutional network backbones for Re-ID to substitute the backbones originally designed for classification such as ResNet and VGG. In the Re-ID tasks to search for a cell structure, greedy decisions are made instead of deriving the architecture after comprehensive training. In other words, at each decision epoch, according to the topological order, we first decide the candidates’ pool of the edges to progressively reduce the coupling of the internal nodes of the DAG. An edge is then selected based on edge importance, edge certainty, and selection stability. We then make a greedy optimal choice for the selected edge and prune the relevant parameters. To further improve the backbone’s representation capability of the features, we further introduce the triplet loss with batch hard mining as the retrieval loss. Extensive experiments demonstrate that the searched structure of the backbones reaches a performance level close to the previous work with a 20.8% shorter searching time. The proposed method also prevents the final CNNs network from suffering the well-known performance collapse by avoiding aggregation of the skip-connections.

Image Captioning is the task of providing a natural language description for an image. It has caught significant amounts of attention from both computer vision and natural language processing communities. Most image captioning models adopt deep encoder-decoder architectures to achieve state-of-the-art performances. However, it is difficult to model knowledge on relationships between input image region pairs in the encoder. Furthermore, the word in the decoder hardly knows the correlation to specific image regions. In this paper, a novel deep encoder-decoder model is proposed for image captioning which is developed on sparse Transformer framework. The encoder adopts a multi-level representation of image features based on self-attention to exploit low-level and high-level features, naturally the correlations between image region pairs are adequately modeled as self-attention operation can be seen as a way of encoding pairwise relationships. The decoder improves the concentration of multi-head self-attention on the global context by explicitly selecting the most relevant segments at each row of the attention matrix. It can help the model focus on the more contributing image regions and generate more accurate words in the context. Experiments demonstrate that our model outperforms previous methods and achieves higher performance on MSCOCO and Flickr30k datasets. Our code is available at https://github.com/2014gaokao/ImageCaptioning.

The increased number of video cameras makes an explosive growth in the amount of captured video, especially the increase of millions of surveillance cameras that operate 24 hours a day. Since video browsing and retrieval is time consuming, while video synopsis is one of the most effective ways for browsing and indexing such video that enables the review of hours of video in just minutes. How to generate the video synopsis and preserve the essential activities in the original video is still a costly and labor-intensive and time-intensive work. This paper proposes an approach to generating video synopsis with complete foreground and clearer trajectory of moving objects. Firstly, the one-stage CNN-based object detecting has been employed in object extraction and classification. Then, combining integrating the attention-RetinaNet with Local Transparency-Handling Collision (LTHC) algorithm is given out which results in the trajectory combination optimization and makes the trajectory of the moving object more clearly. Finally, the experiments show that the useful video information is fully retained in the result video, the detection accuracy is improved by 4.87% and the compression ratio reaches 4.94, but the reduction of detection time is not obvious.