[1]佘 维,孔祥基,郭淑明,等.基于轻量化深度卷积循环网络的MVS方法[J].郑州大学学报(工学版),2024,45(04):11-18.[doi:10.13705/ j.issn.1671-6833.2024.04.003]
　SHE Wei,KONG Xiangji,GUO Shuming,et al.MVS Method Based on Lightweight Deep Convolutional Recurrent Network[J].Journal of Zhengzhou University (Engineering Science),2024,45(04):11-18.[doi:10.13705/ j.issn.1671-6833.2024.04.003]
点击复制

基于轻量化深度卷积循环网络的MVS方法()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
45
期数:
2024年04期
页码:
11-18
栏目:
出版日期:
2024-06-16

文章信息/Info

Title:
MVS Method Based on Lightweight Deep Convolutional Recurrent Network
文章编号:
1671-6833(2024)04-0011-08
作者:
佘 维1; 2; 3;  孔祥基1; 3;  郭淑明2; 4;  田 钊1; 3;  李英豪1; 2; 3
1.郑州大学 网络空间安全学院,河南 郑州 450002;2.嵩山实验室,河南 郑州 450046;3.郑州市区块链与数据智能重点实验室,河南 郑州 450002;4.国家数字交换系统工程技术研究中心,河南 郑州 450002
Author(s):
SHE Wei1; 2; 3;  KONG Xiangji1; 3;  GUO Shuming2; 4;  TIAN Zhao1; 3;  LI Yinghao1; 2; 3
School of Cyber Science and Engineering, Zhengzhou University, Zhengzhou 450002, China; 2.Songshan Laboratory, Zhengzhou 450046, China; 3.Zhengzhou Key Laboratory of Blockchain and Data Intelligence, Zhengzhou 450002, China; 4.China National Dig ital Switching System Engineering & Technological R&D Center, Zhengzhou 450002, China
关键词:
轻量化;  深度卷积循环网络;  MVS方法;  正则化;  DTU数据集
Keywords:
lightweight;  deep convolutional recurrent network;  MVS method;  regularization;  DTU dataset
分类号:
TP39TP751.1
DOI:
10.13705/ j.issn.1671-6833.2024.04.003
文献标志码:
A
摘要:
针对基于深度学习的MVS方法存在网络参数量大、显存占用较高的问题,提出一种基于轻量化深度卷积 循环网络的MVS方法。首先,采用轻量化多尺度特征提取网络提取图像的高层语义特征图,构建稀疏代价体减小 计算体积;其次,使用卷积循环网络对代价体进行正则化,一次平面扫描完成正则化过程,减少显存占用;最后,通 过深度图扩展模块扩展稀疏深度图为稠密深度图,并结合优化算法保证重建精度。在DTU数据集上与最近的方 法进行对比,包括传统MVS方法Camp、Furu、Tola、Gipuma,基于深度学习的MVS方法SurfaceNet、PU-Net、MVSNet、 R-MVSNet、Point-MVSNet、Fast-MVSNet、GBI-Net、TransMVSNet。实验结果表明:所提方法在精度上与其他方法保持 较小差距的前提下,能够将预测时显存开销降低至3.1 GB。
Abstract:
Based on deep learning MVS methods, neural networks suffered from a large number of parameters and high GPU memory consumption. To address this issue, a lightweight deep convolutional recurrent network recurrent network-based MVS method was proposed. Firstly, the original images passed through a lightweight multi-scale fea ture extraction network to obtain high-level semantic feature maps. Then, a sparse cost volume to reduce the com putational workload was constructed. Next, GPU memory consumption was reduced by using a simple plane sweep ing technique that utilized by a convolutional recurrent network for cost volume regularization. Finally, sparse depth maps were extended to dense depth maps using an extension module. With a refinement algorithm, the proposed approach achieved a certain level of accuracy. The proposed approach was compared to state-of-the-art methods on the DTU dataset including traditonal MVS methods Camp, Furu, Tola, and Gipuma, and also including deep learn ing-based MVS methods SurfaceNet, PU-Net, MVSNet, R-MVSNet, Point-MVSNet, Fast-MVSNet, GBI-Net, and TransMVSNet. The results demonstrated that the proposed approach reduced GPU consumption to approximately 3.1 GB during the prediction stage, and the differences in precision compared to other methods were relatively small.

参考文献/References:

[1] YAN X C, YANG J M, YUMER E, et al. Perspective transformer nets: learning single-view 3D object recon struction without 3D supervision[C]∥Proceedings of the 30th International Conference on Neural Information Pro cessing Systems. New York: ACM, 2016: 1704-1712. 

[2] SUN X Y, WU J J, ZHANG X M, et al. Pix3D: dataset and methods for single-image 3D shape modeling[C]∥ 2018 IEEE/CVF Conference on Computer Vision and Pat tern Recognition. Piscataway: IEEE, 2018: 2974-2983. 
[3] FURUKAWA Y, HERNÁNDEZ C. Multi-view stereo: a tutorial[J]. Foundations and Trends in Computer Graph ics and Vision, 2015, 9(1/2): 1-148. 
[4] 纪勇, 刘丹丹, 罗勇, 等. 基于霍夫投票的变电站设 备三维点云识别算法[J]. 郑州大学学报(工学版), 2019, 40(3): 1-6, 12. 
JI Y, LIU D D, LUO Y, et al. Recognition of three-di mensional substation equipment based on Hough transform [J]. Journal of Zhengzhou University (Engineering Sci ence), 2019, 40(3): 1-6, 12. 
[5] KUTULAKOS K N, SEITZ S M. A theory of shape by space carving[J]. International Journal of Computer Vi sion, 2000, 38(3): 199-218. 
[6] HUANG P H, MATZEN K, KOPF J, et al. DeepMVS: learning multi-view stereopsis[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2821-2830. 
[7] JI M Q, GALL J, ZHENG H T, et al. SurfaceNet: an end to-end 3D neural network for multiview stereopsis[C]∥2017 IEEE International Conference on Computer Vision (IC CV). Piscataway: IEEE, 2017: 2326-2334. 
[8] YAO Y, LUO Z X, LI S W, et al. MVSNet: depth infer ence for unstructured multi-view stereo[C]∥European Conference on Computer Vision. Cham: Springer, 2018: 785-801. 
[9] YAO Y, LUO Z X, LI S W, et al. Recurrent MVSNet for high-resolution multi-view stereo depth inference[C]∥ 2019 IEEE/CVF Conference on Computer Vision and Pat tern Recognition (CVPR). Piscataway: IEEE, 2019: 5520-5529. 
[10]杜弘志, 张腾, 孙岩标, 等. 基于门控循环单元的立 体匹配方法研究[J]. 激光与光电子学进展, 2021, 58 (14): 387-394. 
DU H Z, ZHANG T, SUN Y B, et al. Stereo matching method based on gated recurrent unit networks[J]. Laser & Optoelectronics Progress, 2021, 58(14): 387-394. 
[11] CHEN R, HAN S F, XU J, et al. Point-based multi view stereo network[C]∥2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 1538-1547. 
[12] YU Z H, GAO S H. Fast-MVSNet: sparse-to-dense multi-view stereo with learned propagation and Gauss Newton refinement[C]∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 1946-1955. 
[13] MA L B, LI N, YU G, et al. Pareto-wise ranking classi fier for multi-objective evolutionary neural architecture search[J]. IEEE Transactions on Evolutionary Computa tion, 2023: 1-12. 
[14] LI N, MA L B, YU G, et al. Survey on evolutionary deep learning: principles, algorithms, applications, and open issues[J]. ACM Computing Surveys, 2024, 56 (2): 1-34. 
[15] COLLINS R T. A space-sweep approach to true multi-im age matching[C]∥Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Rec ognition. Piscataway: IEEE, 2002: 358-363. 
[16] CAMPBELL N D, VOGIATZIS G, HERNÁNDEZ C, et al. Using multiple hypotheses to improve depth-maps for multi-view stereo[C]∥ 10th European Conference on Computer Vision. New York: ACM, 2008: 766-779. 
[17] FURUKAWA Y, PONCE J. Accurate, dense, and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1362-1376. 
[18] TOLA E, STRECHA C, FUA P. Efficient large-scale multi-view stereo for ultra high-resolution image sets[J]. Machine Vision and Applications, 2012, 23(5): 903-920. 
[19] GALLIANI S, LASINGER K, SCHINDLER K. Massively parallel multiview stereopsis by surface normal diffusion [C]∥2015 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2015: 873-881. 
[20] YU L Q, LI X Z, FU C W, et al. PU-net: point cloud upsampling network[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2790-2799. 
[21] MI Z X, DI C, XU D. Generalized binary search network for highly-efficient multi-view stereo[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 12981-12990. 
[22] DING Y K, YUAN W T, ZHU Q T, et al. TransMVS Net: global context-aware multi-view stereo network with transformers[C]∥2022 IEEE/CVF Conference on Com puter Vision and Pattern Recognition (CVPR). Piscata way: IEEE, 2022: 8575-8584.

相似文献/References:

[1]卢晓明,贾建军,周成林,等.1m口径空间相机主望远镜组件设计[J].郑州大学学报(工学版),2018,39(03):72.[doi:10.13705/j.issn.1671-6833.2018.03.008]
　Lu Xiaoming,Jia Jianjun,Zhou Chenglin,et al.Component Design on Telescope with 1m-caliber Space Camera[J].Journal of Zhengzhou University (Engineering Science),2018,39(04):72.[doi:10.13705/j.issn.1671-6833.2018.03.008]
[2]袁守利,林家辉.FSAE方程式赛车车架的设计与轻量化[J].郑州大学学报(工学版),2018,39(04):18.[doi:1013705/j.issn.1671-6833.2018.01.005]
　Yuan Shouli,Lam Ka Fai.The Design of  FSAE Formula Frame Light weight[J].Journal of Zhengzhou University (Engineering Science),2018,39(04):18.[doi:1013705/j.issn.1671-6833.2018.01.005]
[3]贾云飞,郑红木,刘闪亮.基于YOLOv5s 的金属制品表面缺陷的轻量化算法研究[J].郑州大学学报(工学版),2022,43(05):31.[doi:10.13705/j.issn.1671-6833.2022.05.001]
　JIA Yunfei,ZHENG Hongmu,LIU Shanliang.Lightweight Surface Defect Detection Method of Metal Products Based on YOLOv5s[J].Journal of Zhengzhou University (Engineering Science),2022,43(04):31.[doi:10.13705/j.issn.1671-6833.2022.05.001]

更新日期/Last Update: 2024-06-14