Suping Wu’s research while affiliated with Ningxia University and other places

Monocular image-based 3D fine face reconstruction techniques aim to reconstruct 3D faces with rich face details from a single image. Existing methods have achieved remarkable results, but they cannot accurately extract light and perspective information, resulting in reconstructed faces with poor details and more noise. To this end, we propose a method for 3D face reconstruction using multi-collaborative learning network. Specifically, we design an illumination and view feature extraction network, which combines the ideas of FCN-style [8] point-by-point addition and UNet-style [7] channel dimension splicing and fusion. In this way, features at different scales can be better filtered and integrated, and key semantic information can be extracted, we can make full use of the effective features at different scales to obtain accurate light and view information. In addition, in order to be able to obtain a more comprehensive and realistic albedo characterisation, we propose a multi-resolution co-optimization module. Extensive experimental results on several evaluation datasets show that our method achieves significant improvements and excellent performance compared to state-of-the-art methods.

Recent studies have witnessed that many self-supervised methods obtain clear progress on the multi-view stereo (MVS). However, existing methods ignore the edge structure information of the reconstructed target, which includes the outer silhouette and the edge information of the internal structure. This may lead to less satisfactory edges and completeness of the reconstruction result. To solve this problem, we propose an extractor for extracting edge structure maps, and we innovatively design an edge structure Loss to constrain the network to pay more attention to edge structure features of the reference view to improve the texture details of the reconstruction results. Specially, we utilize the idea of constructing cost volume in multi-view stereo and warp the edge structure map of the source view to the reference view to provide reliable self-supervision. In addition, we design a masking mechanism that combines local and global properties, which ensures robustness and improves the reconstruction completeness of the model for complex samples. Furthermore, we adopt an effective parallel acceleration approach to improve the training speed and reconstruction efficiency. Extensive experiments on the DTU and Tanks &Temples benchmarks demonstrate that our method improves both accuracy and completeness in comparison with other unsupervised work. In addition, our parallel method improves efficiency while ensuring accuracy. The code will be published.

3D detailed face reconstruction based on monocular images aims to reconstruct a 3D face from a single image with rich face detail. The existing methods have achieved significant results, but still suffer from inaccurate face geometry reconstruction and artifacts caused by mistaking hair for wrinkle information. To address these problems, we propose a bi-directional optimization network for de-obscured 3D high-fidelity surface reconstruction. Specifically, our network is divided into two stages: face geometry fitting and face detail optimization. In the first stage, we design a global and local bi-directional optimized feature extraction network that uses both local and global information to jointly constrain the face geometry and ultimately achieve an accurate 3D face geometry reconstruction. In the second stage, we decouple the hair and the face using a segmentation network and use the distribution of depth values in the facial region as a prior for the hair part, after which the FPU-net detail extraction network we designed is able to reconstruct finer 3D face details while removing the hair occlusion problem. With only a small number of training samples, extensive experimental results on multiple evaluation datasets show that our method achieves competitive performance and significant improvements over state-of-the-art methods.

In this paper, we propose a Detail Geometry Learning Network (DGLN) approach to investigate the problem of self-supervised high-fidelity face reconstruction from monocular images. Unlike existing methods that rely on detail generators to generate “pseudo-details” where most of the reconstructed detail geometries are inconsistent with real faces. Our DGLN can ensure face personalization and also correctly learn more local face details. Specifically, our method includes two stages: the personalization stage and the detailization stage. In the personalization stage, we design a multi-perception interaction module (MPIM) to adaptively calibrate the weighted responses by interacting with information from different receptive fields to extract distinguishable and reliable features. To further enhance the geometric detail information, in the detailization stage, we develop a multi-resolution refinement network module (MrNet) to estimate the refined displacement map with features from different layers and different domains (i.e. coarse displacement images and RGB images). Finally, we design a novel normal smoothing loss that improves the reconstructed details and realisticity. Extensive experiments demonstrate the superiority of our method over previous work.