Lulu Liu’s research while affiliated with Xi’an Jiaotong-Liverpool University and other places

With the development of Unmanned Surface Vehicles (USVs), the perception of inland waterways has become significant to autonomous navigation. RGB cameras can capture images with rich semantic features, but they would fail in adverse weather and at night. As a perception sensor that has initially emerged in recent years, 4D millimeter-wave radar (4D mmWave radar) can work in all weather and has more abundant point-cloud features than ordinary radar, but it also suffers from water-surface clutter seriously. Furthermore, the shape and outline of dense point cloud captured by 4D mmWave radar are irregular. CNN-based neural networks treat features as 2D rectangle grids, which excessively favor image modality and are unfriendly to radar modality. Therefore, we transform both features of image and radar into non-Euclidean space as graph structures. In this paper, we focus on robust panoptic perception in inland waterways. Firstly, we propose the first Clutter-Point-Removal (CPR) algorithm for 4D mmWave radar, removing water-surface clutter and improving the recall of radar targets. Secondly, we propose a high-performance panoptic perception model based on the graph neural network called Mask-VRDet, fusing features of vision and radar to simultaneously perform object detection and semantic segmentation. To the best of our knowledge, Mask-VRDet is the first riverway panoptic perception model based on vision-radar graphical fusion. It outperforms other single-modal and fusion models, and achieves state-of-the-art performance on our collected dataset. We release our code at https://github.com/GuanRunwei/Mask-VRDet-Official.

With its small size, low cost and all-weather operation, millimeter-wave radar can accurately measure the distance, azimuth and radial velocity of a target compared to other traffic sensors. However, in practice, millimeter-wave radars are plagued by various interferences, leading to a drop in target detection accuracy or even failure to detect targets. This is undesirable in autonomous vehicles and traffic surveillance, as it is likely to threaten human life and cause property damage. Therefore, interference mitigation is of great significance for millimeter-wave radar-based target detection. Currently, the development of deep learning is rapid, but existing deep learning-based interference mitigation models still have great limitations in terms of model size and inference speed. For these reasons, we propose Radar-STDA, a Radar-Spatial Temporal Denoising Autoencoder. Radar-STDA is an efficient nano-level denoising autoencoder that takes into account both spatial and temporal information of range-Doppler maps. Among other methods, it achieves a maximum SINR of 17.08 dB with only 140,000 parameters. It obtains 207.6 FPS on an RTX A4000 GPU and 56.8 FPS on an NVIDIA Jetson AGXXavier respectively when denoising range-Doppler maps for three consecutive frames. Moreover, we release a synthetic data set called Ra-inf for the task, which involves 384,769 range-Doppler maps with various clutters from objects of no interest and receiver noise in realistic scenarios. To the best of our knowledge, Ra-inf is the first synthetic dataset of radar interference. To support the community, our research is open-source via the link \url{https://github.com/GuanRunwei/rd_map_temporal_spatial_denoising_autoencoder}.

With recent developments, the performance of automotive radar has improved significantly. The next generation of 4D radar can achieve imaging capability in the form of high-resolution point clouds. In this context, we believe that the era of deep learning for radar perception has arrived. However, studies on radar deep learning are spread across different tasks, and a holistic overview is lacking. This review paper attempts to provide a big picture of the deep radar perception stack, including signal processing, datasets, labelling, data augmentation, and downstream tasks such as depth and velocity estimation, object detection, and sensor fusion. For these tasks, we focus on explaining how the network structure is adapted to radar domain knowledge. In particular, we summarise three overlooked challenges in deep radar perception, including multi-path effects, uncertainty problems, and adverse weather effects, and present some attempts to solve them.