SHE Wei1,2,3, KONG Xiangji1,3, GUO Shuming2,4, TIAN Zhao1,3, LI Yinghao1,2,3
Abstract:
Based on deep learning MVS methods, neural networks suffered from a large number of parameters and high GPU memory consumption. To address this issue, a lightweight deep convolutional recurrent network recurrent network-based MVS method was proposed. Firstly, the original images passed through a lightweight multi-scale fea ture extraction network to obtain high-level semantic feature maps. Then, a sparse cost volume to reduce the com putational workload was constructed. Next, GPU memory consumption was reduced by using a simple plane sweep ing technique that utilized by a convolutional recurrent network for cost volume regularization. Finally, sparse depth maps were extended to dense depth maps using an extension module. With a refinement algorithm, the proposed approach achieved a certain level of accuracy. The proposed approach was compared to state-of-the-art methods on the DTU dataset including traditonal MVS methods Camp, Furu, Tola, and Gipuma, and also including deep learn ing-based MVS methods SurfaceNet, PU-Net, MVSNet, R-MVSNet, Point-MVSNet, Fast-MVSNet, GBI-Net, and TransMVSNet. The results demonstrated that the proposed approach reduced GPU consumption to approximately 3.1 GB during the prediction stage, and the differences in precision compared to other methods were relatively small.