[1]余松森,龙嘉濠,周 诺,等.基于相机运动轨迹的鲁棒无人机航拍稳像算法[J].郑州大学学报(工学版),2024,45(05):77-85.[doi:10.13705/j.issn.1671-6833.2024.02.004]
　YU Songsen,LONG Jiahao,ZHOU Nuo,et al.Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory[J].Journal of Zhengzhou University (Engineering Science),2024,45(05):77-85.[doi:10.13705/j.issn.1671-6833.2024.02.004]
点击复制

基于相机运动轨迹的鲁棒无人机航拍稳像算法()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
45
期数:
2024年05期
页码:
77-85
栏目:
出版日期:
2024-08-08

文章信息/Info

Title:
Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory
文章编号:
1671-6833(2024)05-0077-09
作者:
余松森;  龙嘉濠;  周 诺;  梁 军
华南师范大学 软件学院,广东 佛山 528225
Author(s):
YU Songsen;  LONG Jiahao;  ZHOU Nuo;  LIANG Jun
School of Software, South China Normal University, Foshan 528225, China
关键词:
无人机;  视频稳定;  特征提取;  运动估计;  峰值信噪比
Keywords:
UAV;  video stabilization;  feature extraction;  motion estimation;  peak signal-to-noise ratio
分类号:
TP391
DOI:
10.13705/j.issn.1671-6833.2024.02.004
文献标志码:
A
摘要:
针对高空颠簸环境影响无人机采集延时稳定图像问题,提出了一种适用于悬停拍摄及移动拍摄的航拍视频防抖算法。 通过无人机摄像头获取延时摄影视频,提取全局范围内的部分视频帧进行直方图分布比较,进而确定视频是否是在无人机移动状态下拍摄的,并将视频划分为包含与不包含摄像机主动运动两类。 对于包含摄像机主动运动的一类,首先,采用 FAST 角点检测加光流法进行特征点提取配对;其次,利用 RANSAC 算法剔除误匹配特征点并进行摄像机运动轨迹估计;最后,利用高斯滤波对运动估计参数进行平滑以得到稳定的摄像机运动轨迹。对于不包含摄像机主动运动的一类,首先,对首帧进行网格划分并基于 Harris 矩阵提取各网格的特征点;其次,在后续帧对这些特征点进行光流追踪;再次,通过反向光流及 Harris 矩阵计算,增加特征点约束,完成特征点提取及匹配;最后,利用保留的特征点估计后续帧至首帧的稳定变换。 利用该算法对视频进行场景分类及画面稳定,实验结果表明:视频分类模块能正确区分两类视频;对比其他两种方法,稳定后的视频图像平均峰值信噪比提升更大;对于不含摄像机主动运动的视频,可实现画面绝对稳定,图像平均峰值信噪比提升超过 39%,而其他两种方法仅提升 10% ~ 12%。
Abstract:
Aiming at the problem of high altitude turbulence environment to the time-delay stable image acquisitionof UAV, an anti-shaking algorithm for aerial video was proposed for hovering shooting and moving shooting. Firstlyfrom the time-delay photography video captured the UAV camera, some video frames were extracted globally tocompare their histogram distributions. This comparison could identify whether the video contained active cameramotion or not, and help categorize the video accordingly. For videos with active camera motion, FAST cornerdetection and optical flow methods were used to extract and match feature points. The RANSAC algorithm couldremove all mismatched feature points, and estimate the camera′s motion trajectory. The resulting motion estimationparameters were then smoothed using Gaussian filtering, producing a stable camera motion trajectory. For videoswithout active camera motion, the first frame was divided into grids and feature points were extracted based onHarris matrix. Optical flow tracking was carried out on these feature points in subsequent frames. Reverse opticalflow and Harris matrix calculation were used to extract and match feature points, to increase the constraint of featurepoints. Finally, the retained feature points were used to estimate the stable transformation from subsequent framesto the first frame. Experimental results showed that the video classification module could correctly distinguishbetween the two types of videos. The algorithm was used to classify the video scene and stabilize the picture.Compared to other methods, this algorithm could improve the average peak signal-to-noise ratio of stabilized videoimages the most. For videos without active camera motion, the image could be absolutely stable, and the averagepeak signal-to-noise ratio of the image was increased by more than 39%, while the other two methods only by 10%to 12%.

参考文献/References:

[1] 苑玉彬,吴一全,赵朗月,等. 基于深度学习的无人机航拍视频多目标检测与跟踪研究进展[ J] . 航空学报,2023,44(18) :028334.

YUAN Y B, WU Y Q, ZHAO L Y, et al. Research progress of UAV aerial video multi-object detection andtracking based on deep learning[ J] . Acta Aeronautica etAstronautica Sinica,2023,44(18) :028334.
[2] GRUNDMANN M, KWATRA V, ESSA I. Auto-directedvideo stabilization with robust L1 optimal camera paths[C]∥Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway: IEEE, 2011: 225-232.
[3] LEE K Y, CHUANG Y Y, CHEN B Y, et al. Video stabilization using robust feature trajectories [ C ] ∥ 2009IEEE 12th International Conference on Computer Vision.Piscataway: IEEE, 2010: 1397-1404.
[4] 唐佳林, 郑杰锋, 李熙莹, 等. 基于特征匹配与运动补偿的视频稳像算法[ J] . 计算机应用研究, 2018, 35(2) : 608-610, 614.
TANG J L, ZHENG J F, LI X Y, et al. Video stabilization algorithm based on feature matching and motion compensation [ J ] . Application Research of Computers,2018, 35(2) : 608-610, 614.
[5] 谢亚晋, 徐之海, 冯华君, 等. 基于最小生成树与改进卡尔曼滤波器的实时电子稳像方法[ J] . 光子学报,2018, 47(1) : 0110002.
XIE Y J, XU Z H, FENG H J, et al. Real-time videostabilization based on minimal spanning tree and modifiedKalman filter[ J] . Acta Photonica Sinica, 2018, 47(1) :0110002.
[6] 王杰, 经俊森, 陈正伟, 等. 基于 Harris 和卡尔曼滤波的农业机器人田间稳像算法[ J] . 农业机械学报,2023, 54(1)30-36, 53.
WANG J, JING J S, CHEN Z W, et al. Field image stabilization algorithm for agricultural robot based on Harrisand Kalman filter[ J] . Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(1)30-36, 53.
[7] YU J Y, RAMAMOORTHI R. Learning video stabilization using optical flow[ C]∥2020 IEEE / CVF Conferenceon Computer Vision and Pattern Recognition ( CVPR) .Piscataway: IEEE, 2020: 8156-8164.
[8] ALI M K, YU S, KIM T H. Learning deep video stabilization without optical flow [ EB / OL ] . ( 2020 - 11 - 19 )[ 2023 - 06 - 16 ] . https: ∥ doi. org / 10. 48550 / arXiv.2011. 09697.
[9] XU Y F, ZHANG J, MAYBANK S J, et al. DUT: learning video stabilization by simply watching unstable videos[ J] . IEEE Transactions on Image Processing, 2022, 31:4306-4320.
[10] BRADLEY A, KLIVINGTON J, TRISCARI J, et al.Cinematic-L1 video stabilization with a log-homographymodel[C]∥Conference on Applications of Computer Vision (WACV) . Piscataway: IEEE, 2021: 1040-1048.
[11] LEE Y C, TSENG K W, CHEN Y T, et al. 3D videostabilization with depth estimation by CNN-based optimization[C]∥Conference on Computer Vision and PatternRecognition (CVPR) . Piscataway: IEEE, 2021: 10616-10625.
[12] LIU F, GLEICHER M, JIN H, et al. Content-preservingwarps for 3D video stabilization [ J] . ACM Transactionson Graphics,2009,28(3) :44.
[13] LIU S, WANG Y, YUAN L, et al. Video stabilizationwith a depth camera[C]∥Conference on Computer Visionand Pattern Recognition ( CVPR ) . Piscataway: IEEE,2012: 89-95.
[14] LIU Y L, LAI W S, YANG M H, et al. Hybrid neural fusion for full-frame video stabilization [ C ] ∥InternationalConference on Computer Vision ( ICCV ). Piscataway:IEEE, 2022: 2279-2288.
[15] SHI Z M, SHI F H, LAI W S, et al. Deep online fusedvideo stabilization [ C ] ∥Conference on Applications ofComputer Vision ( WACV) . Piscataway: IEEE, 2022:865-873.
[16] WANG M, YANG G Y, LIN J K, et al. Deep online video stabilization with multi-grid warping transformationlearning [ J ] . IEEE Transactions on Image Processing,2018, 28(5) : 2283-2292.
[17] XU S Z, HU J, WANG M, et al. Deep video stabilization using adversarial networks [ J ] . Computer GraphicsForum, 2018, 37(7) : 267-276.
[18] ZHAO M, LING Q. PWStableNet: learning pixel-wisewarping maps for video stabilization [ J] . IEEE Transactions on Image Processing, 2020, 29: 3582-3595.
[19] 吴国楠, 周超超, 尹文波. 基于 Harris 角点与改进 Hu矩的电子稳像算法[ J] . 计算机工程, 2013, 39( 3) :300-305, 310.
WU G N, ZHOU C C, YIN W. Electronic image stabili-zation algorithm based on Harris corner and modified Humoments[ J] . Computer Engineering, 2013, 39(3) : 300-305, 310.
[20] ROSTEN E, DRUMMOND T. Machine learning for highspeed corner detection[C]∥ Proceedings of the 9th European conference on Computer Vision. Berlin: Springer,2006: 430-443.
[21] STANKOVIC’L, BRAJOVIC’M, STANKOVIC’I, et al.RANSAC-based signal denoising using compressive sensing[ J] . Circuits, Systems, and Signal Processing, 2021,40(8) : 3907-3928.
[22] 孙辉. Bayer 彩色序列图像的电子稳像[ J] . 光学精密工程, 2014, 22(2) :481-488.
SUN H. Electronic image stabilization for Bayer color video sequence image [ J] . Optics and Precision Engineering, 2014, 22(2) :481-488.
[23] 王金鑫, 秦子龙, 曹泽宁, 等. 基于八叉树的修正克里金 空 间 插 值 算 法 [ J] . 郑 州 大 学 学 报 ( 工 学 版) ,2021, 42(6) : 21-27.
WANG J X, QIN Z L, CAO Z N, et al. Modified Krigingspatial interpolation algorithm based on octree mechanism[ J] . Journal of Zhengzhou University ( Engineering Science) , 2021, 42(6) : 21-27.
[24] 华蓓, 陈前, 黄汝维. 基于特征匹配的图像真伪检测方法的研究[ J] . 郑州大学学报( 工学版) , 2022, 43(2) : 22-27.
HUA B, CHEN Q, HUANG R W. Research of image authenticity detection method based on feature matching[ J] . Journal of Zhengzhou University ( Engineering Science) , 2022, 43(2) : 22-27.
[25] AHN B, HAN Y, KWEON I S. Real-time facial landmarks tracking using active shape model and LK opticalflow[ C]∥2012 9th International Conference on Ubiquitous Robots and Ambient Intelligence ( URAI) . Piscataway: IEEE, 2013: 541-543.

相似文献/References:

[1]刘广瑞,王庆海,姚冬艳.基于改进人工蜂群算法的多无人机协同任务规划[J].郑州大学学报(工学版),2018,39(03):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
　Liu Guangrui,Wang Qinghai,Yao Dongyan.Multi-UAV Cooperative Mission Planning Based on Improved Artificial Bee Colony Algorithm[J].Journal of Zhengzhou University (Engineering Science),2018,39(05):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
[2]闫怡汝,王寅.基于鸽群优化的复杂环境下无人机侦查航迹优化 [J].郑州大学学报(工学版),2019,40(04):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]
　Yan Yiru,Wang Yin. undefined Pigeon-inspired Optimization Based Trajectory Planning Methodfor UAVs in a Complex Urban Environment [J].Journal of Zhengzhou University (Engineering Science),2019,40(05):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]

更新日期/Last Update: 2024-09-02