[1]王 峰,马星宇,孟鹏帅,等.基于深度强化学习的无人机边缘计算任务卸载策略[J].郑州大学学报(工学版),2025,46(04):16-23.[doi:10.13705/j.issn.1671-6833.2025.01.018]
　WANG Feng,MA Xingyu,MENG Pengshuai,et al.Task Offloading Strategy of UAV Edge Computing Based on Deep Reinforcement Learning[J].Journal of Zhengzhou University (Engineering Science),2025,46(04):16-23.[doi:10.13705/j.issn.1671-6833.2025.01.018]
点击复制

基于深度强化学习的无人机边缘计算任务卸载策略()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
46
期数:
2025年04期
页码:
16-23
栏目:
出版日期:
2025-07-10

文章信息/Info

Title:
Task Offloading Strategy of UAV Edge Computing Based on Deep Reinforcement Learning
文章编号:
1671-6833(2025)04-0016-08
作者:
王 峰1;  马星宇2;  孟鹏帅2;  赵 薇2;  翟伟光2
1. 太原理工大学 电 气 与 动 力 工 程 学 院,山 西 太 原 030024;2. 太 原 理 工 大 学 电 子 信 息 工 程 学 院,山 西 太 原 030600
Author(s):
WANG Feng1;  MA Xingyu2;  MENG Pengshuai2;  ZHAO Wei2;  ZHAI Weiguang2
1. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China; 2. College of Electronic Information Engineering, Taiyuan University of Technology, Taiyuan 030600, China
关键词:
无人机;  边缘计算;  任务卸载;  深度强化学习;  资源分配
Keywords:
UAV;  edge computing;  task offloading;  deep reinforcement learning;  resource allocation
分类号:
TN929. 5TP391. 9V279
DOI:
10.13705/j.issn.1671-6833.2025.01.018
文献标志码:
A
摘要:
针对地理条件较为复杂的环境中存在的缺乏基础设施、任务延时高和带宽需求量大等问题,提出一种联合任务卸载和功率分配的多级移动边缘计算(MEC)系统模型。 所提模型考虑将配备 MEC 的服务器部署在无人机附近提供计算服务,综合分析无人机的任务卸载、功耗和计算资源分配等问题并给出度量方法,同时考虑无人机可执行的任务类型以及任务对无人机的 CPU 和 GPU 要求,将该问题表述为混合整数非线性问题。 针对该问题提出一种基于深度强化学习的计算任务卸载算法,该算法基于改进双深度 Q 学习算法,在深度强化学习中利用深度神经网络找到无人机之间的映射,从状态空间中找到潜在的模式并估计最优动作,并使用无模型的 DRL 方法,使每个无人机根据局 部 观 察 快 速 作 出 卸 载 决 策。 仿 真 结 果 表 明:所 提 算 法 相 比 LCGP 算 法,平 均 卸 载 成 本 降 低 了42. 8%;相比 DDPG 算法,能耗减少了 16%;相比 DDQN 算法,任务执行延迟减少了 12. 9%。
Abstract:
Aiming at the problems such as lack of infrastructure, high task delay and high bandwidth demand in complex geographical conditions, a multi-stage mobile edge computing system model which combined computing offloading and power distribution was proposed. In this model, a server equipped with MEC was deployed near the UAV to provide computing services, and the problems such as task offloading, power consumption and computing resource allocation of the UAV were comprehensively analyzed and the measurement methods were given. At the same time, the types of tasks that the UAV could perform and the requirements of the CPU and GPU on the UAV were considered. The problem was expressed as a mixed integer nonlinear problem. A task computing offloading algorithm based on deep reinforcement learning was proposed to solve this problem. Based on the improved double deep Q learning algorithm, the algorithm used deep neural network to find the mapping between UAVs in deep reinforcement learning, finding potential patterns from the state space and estimating the optimal action, and used model-free DRL method to enable each UAV to make quick offloading decisions based on local observations. Simulation results showed that the proposed algorithm reduced the average offloading cost by 42. 8% compared with LCGP algorithm. Compared with DDPG algorithm, the energy consumption was reduced by 16%. Compared with DDQN algorithm, the task execution delay was reduced by 12. 9%.

参考文献/References:

[1] PANWAR P, SHABAZ M, NAZIR S, et al. Generic edge computing system for optimization and computation offloading of unmanned aerial vehicle[ J] . Computers and Electrical Engineering, 2023, 109: 108779.

[2] 陈新颖, 盛敏, 李博, 等. 面向 6G 的无人机通信综述 [ J] . 电子与信息学报, 2022, 44(3) : 781-789. 
CHEN X Y, SHENG M, LI B, et al. Survey on unmanned aerial vehicle communications for 6G[ J] . Journal of Electronics & Information Technology, 2022, 44 (3) : 781-789. 
[3] NGUYEN V, KHANH T T, VAN NAM P, et al. Towards flying mobile edge computing [ C] ∥2020 International Conference on Information Networking ( ICOIN) . Piscataway: IEEE, 2020: 723-725. 
[4] MA M L, GONG C Y, WU L T, et al. FLIRRAS: fast learning with integrated reward and reduced action space for online multitask offloading [ J ] . IEEE Internet of Things Journal, 2023, 10(6) : 5406-5417. 
[5] AHANI G, YUAN D. BS-assisted task offloading for D2D networks with presence of user mobility[ C]∥2019 IEEE 89th Vehicular Technology Conference ( VTC2019- Spring) . Piscataway: IEEE, 2019: 1-5. 
[6] LI N N, HAO W M, ZHOU F H, et al. Smart grid enabled computation offloading and resource allocation for SWIPT-based MEC system [ J ] . IEEE Transactions on Circuits and Systems Ⅱ: Express Briefs, 2022, 69( 8) : 3610-3614. 
[7] WANG L, HUANG P Q, WANG K Z, et al. RL-based user association and resource allocation for multi-UAV enabled MEC[ C]∥2019 15th International Wireless Communications & Mobile Computing Conference ( IWCMC) . Piscatawy: IEEE, 2019: 741-746. 
[8] 王丙琛, 司怀伟, 谭国真. 基于深度强化学习的自动驾 驶车 控 制 算 法 研 究 [ J] . 郑 州 大 学 学 报 ( 工 学 版 ) , 2020, 41(4) : 41-45, 80. 
WANG B C, SI H W, TAN G Z. Research on autopilot control algorithm based on deep reinforcement learning [ J] . Journal of Zhengzhou University ( Engineering Science) , 2020, 41(4) : 41-45, 80. 
[9] YE Y T, WEI W S, GENG D Q, et al. Dynamic coordination in UAV swarm assisted MEC via decentralized deep reinforcement learning[C]∥2020 International Conference on Wireless Communications and Signal Processing ( WCSP) . Piscataway: IEEE, 2020: 1064-1069. 
[10] WANG L, WANG K Z, PAN C H, et al. Deep reinforcement learning based dynamic trajectory control for UAV-assisted mobile edge computing[ J] . IEEE Transactions on Mobile Computing, 2022, 21(10) : 3536-3550. 
[11] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[ EB / OL] . (2019- 07 - 05 ) [ 2024 - 07 - 01 ] . https:∥arxiv. org / abs/ 1509. 02971. 
[12] MAO S, HE S F, WU J S. Joint UAV position optimization and resource scheduling in space-air-ground integrated networks with mixed cloud-edge computing[ J] . IEEE Systems Journal, 2021, 15(3) : 3992-4002. 
[13] LIU Q, SHI L, SUN L L, et al. Path planning for UAVmounted mobile edge computing with deep reinforcement learning[ J] . IEEE Transactions on Vehicular Technology, 2020, 69(5) : 5723-5728. 
[14] ZHANG L, ZHANG Z Y, MIN L, et al. Task offloading and trajectory control for UAV-assisted mobile edge computing using deep reinforcement learning [ J] . IEEE Access, 2021, 9: 53708-53719. 
[15] WANG Y T, CHEN W W, LUAN T H, et al. Task offloading for post-disaster rescue in unmanned aerial vehicles networks [ J] . IEEE / ACM Transactions on Networking, 2022, 30(4) : 1525-1539. 
[16] 李斌, 刘文帅, 费泽松. 面向空天地异构网络的边缘 计算部分任务卸载策略[ J] . 电子与信息学报, 2022, 44(9) : 3091-3098. 
LI B, LIU W S, FEI Z S. Partial computation offloading for mobile edge computing in space-air-ground integrated network[ J] . Journal of Electronics & Information Technology, 2022, 44(9) : 3091-3098. 
[17] HUO Y, LIU Q Y, GAO Q H, et al. Joint task offloading and resource allocation for secure OFDMA-based mobile edge computing systems [ J] . Ad Hoc Networks, 2024, 153: 103342. 
[18] XU Y C, CHEN L L, LU Z H, et al. An adaptive mechanism for dynamically collaborative computing power and task scheduling in edge environment [ J] . IEEE Internet of Things Journal, 2023, 10(4) : 3118-3129. 
[19] 李斌. 基于多智能体强化学习的多无人机边缘计算任 务卸载[ J] . 无线电工程, 2023, 53(12) : 2731-2740. 
LI B. Multi-agent reinforcement learning-based task offloading for multi-UAV edge computing [ J] . Radio Engineering, 2023, 53(12) : 2731-2740. 
[20] LI W, ZHOU F, CHOWDHURY K R, et al. QTCP: adaptive congestion control with reinforcement learning [ J] . IEEE Transactions on Network Science and Engineering, 2019, 6(3) : 445-458. 
[21] WANG P, NI W L. An enhanced dueling double deep Qnetwork with convolutional block attention module for traffic signal optimization in deep reinforcement learning[ J] . IEEE Access, 2024, 12: 44224-44232.

相似文献/References:

[1]刘广瑞,王庆海,姚冬艳.基于改进人工蜂群算法的多无人机协同任务规划[J].郑州大学学报(工学版),2018,39(03):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
　Liu Guangrui,Wang Qinghai,Yao Dongyan.Multi-UAV Cooperative Mission Planning Based on Improved Artificial Bee Colony Algorithm[J].Journal of Zhengzhou University (Engineering Science),2018,39(04):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
[2]闫怡汝,王寅.基于鸽群优化的复杂环境下无人机侦查航迹优化 [J].郑州大学学报(工学版),2019,40(04):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]
　Yan Yiru,Wang Yin. undefined Pigeon-inspired Optimization Based Trajectory Planning Methodfor UAVs in a Complex Urban Environment [J].Journal of Zhengzhou University (Engineering Science),2019,40(04):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]
[3]余松森,龙嘉濠,周 诺,等.基于相机运动轨迹的鲁棒无人机航拍稳像算法[J].郑州大学学报(工学版),2024,45(05):77.[doi:10.13705/j.issn.1671-6833.2024.02.004]
　YU Songsen,LONG Jiahao,ZHOU Nuo,et al.Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory[J].Journal of Zhengzhou University (Engineering Science),2024,45(04):77.[doi:10.13705/j.issn.1671-6833.2024.02.004]
[4]郭歆莹,等.RIS辅助无人机通信系统波束赋形双重循环优化算法[J].郑州大学学报(工学版),2025,46(02):67.[doi:10.13705/j.issn.1671-6833.2025.02.005]
　GUO Xinying,,et al.Beamforming Dual-loop Optimization Algorithm for RIS-assisted UAV Communication Systems[J].Journal of Zhengzhou University (Engineering Science),2025,46(04):67.[doi:10.13705/j.issn.1671-6833.2025.02.005]
[5]杨火根,王 艳,骆 伟.改进粒子群算法的无人机 B 样条曲线路径规划[J].郑州大学学报(工学版),2025,46(04):8.[doi:10.13705/j.issn.1671-6833.2025.04.014]
　YANG Huogen,WANG Yan,et al.Improved Particle Swarm Algorithm for B-spline Curve Path Planning of Unmanned Aerial Vehicles[J].Journal of Zhengzhou University (Engineering Science),2025,46(04):8.[doi:10.13705/j.issn.1671-6833.2025.04.014]

更新日期/Last Update: 2025-07-12