MVS Method Based on Lightweight Deep Convolutional Recurrent Network

NAVIGATE

Table of Contents

STATISTICS

Viewed287

Downloads237

MVS Method Based on Lightweight Deep Convolutional Recurrent Network

[HTML] PDF下载 (237)

[1]SHE Wei,KONG Xiangji,GUO Shuming,et al.MVS Method Based on Lightweight Deep Convolutional Recurrent Network[J].Journal of Zhengzhou University (Engineering Science),2024,45(04):11-18.[doi:10.13705/ j.issn.1671-6833.2024.04.003]

Copy

Journal of Zhengzhou University (Engineering Science)[ISSN 1671-6833/CN 41-1339/T] Volume: 45 Number of periods: 2024 04 Page number: 11-18 Column: Public date: 2024-06-16

Title:: MVS Method Based on Lightweight Deep Convolutional Recurrent Network

Author(s):: SHE Wei¹; 2; 3; KONG Xiangji¹; 3; GUO Shuming²; 4; TIAN Zhao¹; 3; LI Yinghao¹; 2; 3; School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China; 2.Songshan Laboratory, Zhengzhou 450046, China; 3.Zhengzhou Key Laboratory of Blockchain and Data Intelligence, Zhengzhou 450002, China; 4.China National Dig ital Switching System Engineering & Technological R&D Center, Zhengzhou 450002, China

Keywords:: lightweight; deep convolutional recurrent network; MVS method; regularization; DTU dataset

CLC:: TP39TP751.1

DOI:: 10.13705/ j.issn.1671-6833.2024.04.003

Abstract:: Based on deep learning MVS methods, neural networks suffered from a large number of parameters and high GPU memory consumption. To address this issue, a lightweight deep convolutional recurrent network recurrent network-based MVS method was proposed. Firstly, the original images passed through a lightweight multi-scale fea ture extraction network to obtain high-level semantic feature maps. Then, a sparse cost volume to reduce the com putational workload was constructed. Next, GPU memory consumption was reduced by using a simple plane sweep ing technique that utilized by a convolutional recurrent network for cost volume regularization. Finally, sparse depth maps were extended to dense depth maps using an extension module. With a refinement algorithm, the proposed approach achieved a certain level of accuracy. The proposed approach was compared to state-of-the-art methods on the DTU dataset including traditonal MVS methods Camp, Furu, Tola, and Gipuma, and also including deep learn ing-based MVS methods SurfaceNet, PU-Net, MVSNet, R-MVSNet, Point-MVSNet, Fast-MVSNet, GBI-Net, and TransMVSNet. The results demonstrated that the proposed approach reduced GPU consumption to approximately 3.1 GB during the prediction stage, and the differences in precision compared to other methods were relatively small.

References:: [1] YAN X C, YANG J M, YUMER E, et al. Perspective transformer nets: learning single-view 3D object recon struction without 3D supervision[C]∥Proceedings of the 30th International Conference on Neural Information Pro cessing Systems. New York: ACM, 2016: 1704-1712.
[2] SUN X Y, WU J J, ZHANG X M, et al. Pix3D: dataset and methods for single-image 3D shape modeling[C]∥ 2018 IEEE/CVF Conference on Computer Vision and Pat tern Recognition. Piscataway: IEEE, 2018: 2974-2983.

[3] FURUKAWA Y, HERNÁNDEZ C. Multi-view stereo: a tutorial[J]. Foundations and Trends in Computer Graph ics and Vision, 2015, 9(1/2): 1-148.

[4] 纪勇, 刘丹丹, 罗勇, 等. 基于霍夫投票的变电站设备三维点云识别算法[J]. 郑州大学学报(工学版), 2019, 40(3): 1-6, 12.

JI Y, LIU D D, LUO Y, et al. Recognition of three-di mensional substation equipment based on Hough transform [J]. Journal of Zhengzhou University (Engineering Sci ence), 2019, 40(3): 1-6, 12.

[5] KUTULAKOS K N, SEITZ S M. A theory of shape by space carving[J]. International Journal of Computer Vi sion, 2000, 38(3): 199-218.

[6] HUANG P H, MATZEN K, KOPF J, et al. DeepMVS: learning multi-view stereopsis[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2821-2830.

[7] JI M Q, GALL J, ZHENG H T, et al. SurfaceNet: an end to-end 3D neural network for multiview stereopsis[C]∥2017 IEEE International Conference on Computer Vision (IC CV). Piscataway: IEEE, 2017: 2326-2334.

[8] YAO Y, LUO Z X, LI S W, et al. MVSNet: depth infer ence for unstructured multi-view stereo[C]∥European Conference on Computer Vision. Cham: Springer, 2018: 785-801.

[9] YAO Y, LUO Z X, LI S W, et al. Recurrent MVSNet for high-resolution multi-view stereo depth inference[C]∥ 2019 IEEE/CVF Conference on Computer Vision and Pat tern Recognition (CVPR). Piscataway: IEEE, 2019: 5520-5529.

[10]杜弘志, 张腾, 孙岩标, 等. 基于门控循环单元的立体匹配方法研究[J]. 激光与光电子学进展, 2021, 58 (14): 387-394.

DU H Z, ZHANG T, SUN Y B, et al. Stereo matching method based on gated recurrent unit networks[J]. Laser & Optoelectronics Progress, 2021, 58(14): 387-394.

[11] CHEN R, HAN S F, XU J, et al. Point-based multi view stereo network[C]∥2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 1538-1547.

[12] YU Z H, GAO S H. Fast-MVSNet: sparse-to-dense multi-view stereo with learned propagation and Gauss Newton refinement[C]∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 1946-1955.

[13] MA L B, LI N, YU G, et al. Pareto-wise ranking classi fier for multi-objective evolutionary neural architecture search[J]. IEEE Transactions on Evolutionary Computa tion, 2023: 1-12.

[14] LI N, MA L B, YU G, et al. Survey on evolutionary deep learning: principles, algorithms, applications, and open issues[J]. ACM Computing Surveys, 2024, 56 (2): 1-34.

[15] COLLINS R T. A space-sweep approach to true multi-im age matching[C]∥Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Rec ognition. Piscataway: IEEE, 2002: 358-363.

[16] CAMPBELL N D, VOGIATZIS G, HERNÁNDEZ C, et al. Using multiple hypotheses to improve depth-maps for multi-view stereo[C]∥ 10th European Conference on Computer Vision. New York: ACM, 2008: 766-779.

[17] FURUKAWA Y, PONCE J. Accurate, dense, and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1362-1376.

[18] TOLA E, STRECHA C, FUA P. Efficient large-scale multi-view stereo for ultra high-resolution image sets[J]. Machine Vision and Applications, 2012, 23(5): 903-920.

[19] GALLIANI S, LASINGER K, SCHINDLER K. Massively parallel multiview stereopsis by surface normal diffusion [C]∥2015 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2015: 873-881.

[20] YU L Q, LI X Z, FU C W, et al. PU-net: point cloud upsampling network[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2790-2799.

[21] MI Z X, DI C, XU D. Generalized binary search network for highly-efficient multi-view stereo[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 12981-12990.

[22] DING Y K, YUAN W T, ZHU Q T, et al. TransMVS Net: global context-aware multi-view stereo network with transformers[C]∥2022 IEEE/CVF Conference on Com puter Vision and Pattern Recognition (CVPR). Piscata way: IEEE, 2022: 8575-8584.

Similar References:

Memo

Last Update: 2024-06-14