[1]刘宇翔,张茂军,颜深,等.基于多任务学习的初始图像对选取方法[J].郑州大学学报(工学版),2021,42(1):56-62.[doi:10.13705/j.issn.1671-6833.2021.01.009]
　LIU Yuxiang,ZHANG Maojun,YAN Shen,et al.Selecting Initial Image Pairs Based on Multi-task Learning[J].Journal of Zhengzhou University (Engineering Science),2021,42(1):56-62.[doi:10.13705/j.issn.1671-6833.2021.01.009]
点击复制

基于多任务学习的初始图像对选取方法()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
42
期数:
2021年1期
页码:
56-62
栏目:
出版日期:
2021-03-14

文章信息/Info

Title:
Selecting Initial Image Pairs Based on Multi-task Learning
作者:
刘宇翔,张茂军,颜深,李京蓓,彭杨
国防科技大学 系统工程学院，湖南 长沙 410073

Author(s):
LIU Yuxiang, ZHANG Maojun, YAN Shen, LI Jingbei, PENG Yang
School of Systems Engineering, National University of Defense Technology, Changsha 410073, China
关键词:
Keywords:
incremental SfM;  initial image pair selection;  multi-task learning;  scene graph
分类号:
TP391.4
DOI:
10.13705/j.issn.1671-6833.2021.01.009
文献标志码:
A
摘要:

初始图像对选取是增量式从运动中恢复结构的一个关键环节 ， 但传统方法中存在计算效率低 、 对特殊场景不鲁棒的问题 。 因此 ， 提出基于多任务学习的初始图像对选取网络以提高该过程的效率 ， 并针对某些特殊场景容易出现初始图像对位于场景边缘的问题 ， 提出结合场景连接图的初始对选取策略 。 该策略首先构建图像间的拓扑结构 ， 通过图像间连接的疏密程度判断初始图像对是否处于场景中心 ， 从而避免初始图像对处于场景边缘导致重建不完整的问题 。 对比传统 SfM （ structure from motion ） 中的初始图像对选取方法 ， 结果表明 ： 所提出的方法在多种不同场景中的选取速度提 升 5 倍以上 ； 同时 ， 提出的结合场景图的选取策略可使得特殊场景中重建的空间点数量增加 10 倍 ， 且重投影误差下降 0.05px ， 显著提升了在特殊场景中初始图像对选取的鲁棒性 ， 证明了所提方法的有效性 ， 在提高了效率的同时 ， 能够很好地保证特殊场景重建的完整性和稳定性 。


Abstract:
The selection of the initial image pair was the key to the incremental structure from motion (SfM). However, traditional selection methods had some problems such as low computational efficiency and poor robustness in some special scenes. In this paper, an initial image pair selection network based on multi-task learning was proposed to improve the efficiency of selection, and a selection strategy combined with the scene connection graphs was proposed. The strategy firstly constructed the topological structure between the images, and then judged whether the initial image pair was in the center area of the scene based on the density of the connections between the images, so as to avoid the incomplete reconstruction in some special scenes due to the selected initial image pair being in the edge of the whole scene. Compared with traditional SfM (structure from motion) methods, the selecting speed of the proposed method in a variety of different scenes was increased by more than 5 times. At the same time, the proposed selection strategy combined with scene graphs could increase the number of reconstructed spatial points in special scenes by 10 times, and reduce the reprojection error by 0.05 px, which significantly improved the robustness of the initial image pair selection in special scenes. This proved the effectiveness of the proposed method. While improving the efficiency, it could ensure the completeness and stability of the reconstruction of special scenes.

参考文献/References:

[1] BEDER C, STEFFEN R. Determining an initial image pair for fixing the scale of a 3d reconstruction from an image sequence[C]//Joint Pattern Recognition Symposium. Berlin: Springer, 2006: 657-666.

[2] HANER S, HEYDEN A. Covariance propagation and next best view planning for 3D reconstruction[C]//European Conference on Computer Vision. Berlin: Springer, 2012:545-556.
[3] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 4104-4113.
[4] MOULON P, MONASSE P, MARLET R. Adaptive structure from motion with a contrario model estimation[C]//Asian Conference on Computer Vision. Berlin: Springer, 2012:257-270.
[5] 张艺琨,唐雁,陈强. 基于多特征融合的三维模型检索[J].郑州大学学报(工学版), 2019, 40(1):1-6.
[6] RUDER S. An overview of multi-task learning in deep neural networks[EB/OL]. (2017-06-15)[2020-07-25].https://arxiv.org/abs/1706.05098.
[7] SHEN T W, LUO Z W, ZHOU L, et al. Matchable image retrieval by learning from surface reconstruction[C]//Asian Conference on Computer Vision. Berlin: Springer, 2018:415-431.
[8] KENDALL A, GRIMES M, CIPOLLA R. Posenet: a convolutional network for real-time 6-DOF camera relocalization[C]//Proceedings of the IEEE International Conference on Computer Vision. New York: IEEE, 2015:2938-2946.
[9] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York:IEEE, 2015:1-9.
[10] AKENINE-MOLLER T, HAINES E, HOFFMAN N. Real-time rendering[M]. 3rd ed. New York: A K Peters/CRC Press, 2019.
[11] 姜三. 无人机倾斜影像高效SfM重建关键技术研究[D]. 武汉:武汉大学, 2018.
[12] FUKUDA S, YOSHIHASHI R, KAWAKAMI R, et al. Cross-connected networks for multi-task learning of detection and segmentation[EB/OL]. (2018-05-15)[2020-07-25]. https://arxiv.org/abs/1805.05569.
[13] ZHANG Y, YANG Q. An overview of multi-task learning[J]. National science review, 2018, 5(1): 30-43.
[14] KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018:7482-7491.
[15] KENDALL A, CIPOLLA R. Modelling uncertainty in deep learning for camera relocalization[C]//2016 IEEE International Conference on Robotics and Automation (ICRA). New York: IEEE, 2016:4762-4769.
[16] IVRTC dataset[EB/OL].(2019-10-31)[2020-07-25].http://www.ivrtc.org/?page_id=660.

更新日期/Last Update: 2021-03-15