Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory

NAVIGATE

Table of Contents

STATISTICS

Viewed412

Downloads346

Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory

[HTML] PDF下载 (346)

[1]YU Songsen,LONG Jiahao,ZHOU Nuo,et al.Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory[J].Journal of Zhengzhou University (Engineering Science),2024,45(05):77-85.[doi:10.13705/j.issn.1671-6833.2024.02.004]

Copy

Journal of Zhengzhou University (Engineering Science)[ISSN 1671-6833/CN 41-1339/T] Volume: 45 Number of periods: 2024 05 Page number: 77-85 Column: Public date: 2024-08-08

Title:: Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory

Author(s):: YU Songsen; LONG Jiahao; ZHOU Nuo; LIANG Jun; School of Software, South China Normal University, Foshan 528225, China

Keywords:: UAV; video stabilization; feature extraction; motion estimation; peak signal-to-noise ratio

CLC:: TP391

DOI:: 10.13705/j.issn.1671-6833.2024.02.004

Abstract:: Aiming at the problem of high altitude turbulence environment to the time-delay stable image acquisitionof UAV, an anti-shaking algorithm for aerial video was proposed for hovering shooting and moving shooting. Firstlyfrom the time-delay photography video captured the UAV camera, some video frames were extracted globally tocompare their histogram distributions. This comparison could identify whether the video contained active cameramotion or not, and help categorize the video accordingly. For videos with active camera motion, FAST cornerdetection and optical flow methods were used to extract and match feature points. The RANSAC algorithm couldremove all mismatched feature points, and estimate the camera′s motion trajectory. The resulting motion estimationparameters were then smoothed using Gaussian filtering, producing a stable camera motion trajectory. For videoswithout active camera motion, the first frame was divided into grids and feature points were extracted based onHarris matrix. Optical flow tracking was carried out on these feature points in subsequent frames. Reverse opticalflow and Harris matrix calculation were used to extract and match feature points, to increase the constraint of featurepoints. Finally, the retained feature points were used to estimate the stable transformation from subsequent framesto the first frame. Experimental results showed that the video classification module could correctly distinguishbetween the two types of videos. The algorithm was used to classify the video scene and stabilize the picture.Compared to other methods, this algorithm could improve the average peak signal-to-noise ratio of stabilized videoimages the most. For videos without active camera motion, the image could be absolutely stable, and the averagepeak signal-to-noise ratio of the image was increased by more than 39%, while the other two methods only by 10%to 12%.

References:: [1] 苑玉彬,吴一全,赵朗月,等. 基于深度学习的无人机航拍视频多目标检测与跟踪研究进展[ J] . 航空学报,2023,44(18) :028334.
YUAN Y B, WU Y Q, ZHAO L Y, et al. Research progress of UAV aerial video multi-object detection andtracking based on deep learning[ J] . Acta Aeronautica etAstronautica Sinica,2023,44(18) :028334.
[2] GRUNDMANN M, KWATRA V, ESSA I. Auto-directedvideo stabilization with robust L1 optimal camera paths[C]∥Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway: IEEE, 2011: 225-232.
[3] LEE K Y, CHUANG Y Y, CHEN B Y, et al. Video stabilization using robust feature trajectories [ C ] ∥ 2009IEEE 12th International Conference on Computer Vision.Piscataway: IEEE, 2010: 1397-1404.
[4] 唐佳林, 郑杰锋, 李熙莹, 等. 基于特征匹配与运动补偿的视频稳像算法[ J] . 计算机应用研究, 2018, 35(2) : 608-610, 614.
TANG J L, ZHENG J F, LI X Y, et al. Video stabilization algorithm based on feature matching and motion compensation [ J ] . Application Research of Computers,2018, 35(2) : 608-610, 614.
[5] 谢亚晋, 徐之海, 冯华君, 等. 基于最小生成树与改进卡尔曼滤波器的实时电子稳像方法[ J] . 光子学报,2018, 47(1) : 0110002.
XIE Y J, XU Z H, FENG H J, et al. Real-time videostabilization based on minimal spanning tree and modifiedKalman filter[ J] . Acta Photonica Sinica, 2018, 47(1) :0110002.
[6] 王杰, 经俊森, 陈正伟, 等. 基于 Harris 和卡尔曼滤波的农业机器人田间稳像算法[ J] . 农业机械学报,2023, 54(1)30-36, 53.
WANG J, JING J S, CHEN Z W, et al. Field image stabilization algorithm for agricultural robot based on Harrisand Kalman filter[ J] . Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(1)30-36, 53.
[7] YU J Y, RAMAMOORTHI R. Learning video stabilization using optical flow[ C]∥2020 IEEE / CVF Conferenceon Computer Vision and Pattern Recognition ( CVPR) .Piscataway: IEEE, 2020: 8156-8164.
[8] ALI M K, YU S, KIM T H. Learning deep video stabilization without optical flow [ EB / OL ] . ( 2020 - 11 - 19 )[ 2023 - 06 - 16 ] . https: ∥ doi. org / 10. 48550 / arXiv.2011. 09697.
[9] XU Y F, ZHANG J, MAYBANK S J, et al. DUT: learning video stabilization by simply watching unstable videos[ J] . IEEE Transactions on Image Processing, 2022, 31:4306-4320.
[10] BRADLEY A, KLIVINGTON J, TRISCARI J, et al.Cinematic-L1 video stabilization with a log-homographymodel[C]∥Conference on Applications of Computer Vision (WACV) . Piscataway: IEEE, 2021: 1040-1048.
[11] LEE Y C, TSENG K W, CHEN Y T, et al. 3D videostabilization with depth estimation by CNN-based optimization[C]∥Conference on Computer Vision and PatternRecognition (CVPR) . Piscataway: IEEE, 2021: 10616-10625.
[12] LIU F, GLEICHER M, JIN H, et al. Content-preservingwarps for 3D video stabilization [ J] . ACM Transactionson Graphics,2009,28(3) :44.
[13] LIU S, WANG Y, YUAN L, et al. Video stabilizationwith a depth camera[C]∥Conference on Computer Visionand Pattern Recognition ( CVPR ) . Piscataway: IEEE,2012: 89-95.
[14] LIU Y L, LAI W S, YANG M H, et al. Hybrid neural fusion for full-frame video stabilization [ C ] ∥InternationalConference on Computer Vision ( ICCV ). Piscataway:IEEE, 2022: 2279-2288.
[15] SHI Z M, SHI F H, LAI W S, et al. Deep online fusedvideo stabilization [ C ] ∥Conference on Applications ofComputer Vision ( WACV) . Piscataway: IEEE, 2022:865-873.
[16] WANG M, YANG G Y, LIN J K, et al. Deep online video stabilization with multi-grid warping transformationlearning [ J ] . IEEE Transactions on Image Processing,2018, 28(5) : 2283-2292.
[17] XU S Z, HU J, WANG M, et al. Deep video stabilization using adversarial networks [ J ] . Computer GraphicsForum, 2018, 37(7) : 267-276.
[18] ZHAO M, LING Q. PWStableNet: learning pixel-wisewarping maps for video stabilization [ J] . IEEE Transactions on Image Processing, 2020, 29: 3582-3595.
[19] 吴国楠, 周超超, 尹文波. 基于 Harris 角点与改进 Hu矩的电子稳像算法[ J] . 计算机工程, 2013, 39( 3) :300-305, 310.
WU G N, ZHOU C C, YIN W. Electronic image stabili-zation algorithm based on Harris corner and modified Humoments[ J] . Computer Engineering, 2013, 39(3) : 300-305, 310.
[20] ROSTEN E, DRUMMOND T. Machine learning for highspeed corner detection[C]∥ Proceedings of the 9th European conference on Computer Vision. Berlin: Springer,2006: 430-443.
[21] STANKOVIC’L, BRAJOVIC’M, STANKOVIC’I, et al.RANSAC-based signal denoising using compressive sensing[ J] . Circuits, Systems, and Signal Processing, 2021,40(8) : 3907-3928.
[22] 孙辉. Bayer 彩色序列图像的电子稳像[ J] . 光学精密工程, 2014, 22(2) :481-488.
SUN H. Electronic image stabilization for Bayer color video sequence image [ J] . Optics and Precision Engineering, 2014, 22(2) :481-488.
[23] 王金鑫, 秦子龙, 曹泽宁, 等. 基于八叉树的修正克里金空间插值算法 [ J] . 郑州大学学报 ( 工学版) ,2021, 42(6) : 21-27.
WANG J X, QIN Z L, CAO Z N, et al. Modified Krigingspatial interpolation algorithm based on octree mechanism[ J] . Journal of Zhengzhou University ( Engineering Science) , 2021, 42(6) : 21-27.
[24] 华蓓, 陈前, 黄汝维. 基于特征匹配的图像真伪检测方法的研究[ J] . 郑州大学学报( 工学版) , 2022, 43(2) : 22-27.
HUA B, CHEN Q, HUANG R W. Research of image authenticity detection method based on feature matching[ J] . Journal of Zhengzhou University ( Engineering Science) , 2022, 43(2) : 22-27.
[25] AHN B, HAN Y, KWEON I S. Real-time facial landmarks tracking using active shape model and LK opticalflow[ C]∥2012 9th International Conference on Ubiquitous Robots and Ambient Intelligence ( URAI) . Piscataway: IEEE, 2013: 541-543.

Similar References:

Memo

Last Update: 2024-09-02