- Title:
- Task Offloading Strategy of UAV Edge Computing Based on Deep Reinforcement Learning
- CLC:
- TN929. 5TP391. 9
- DOI:
- 10.13705/j.issn.1671-6833.2025.01.018
- Abstract:
- Aiming at the problems such as lack of infrastructure, high task delay and high bandwidth demand in complex geographical conditions, this paper proposes a multi-stage mobile edge computing system model which combines computing offload and power distribution. In this model, a server equipped with MEC is deployed near the UAV to provide computing services, and the problems such as task offloading, power consumption and computing resource allocation of the UAV are comprehensively analyzed and the measurement methods are given. At the same time, the types of tasks that the UAV can perform and the requirements of the CPU and GPU on the UAV are considered. The problem is expressed as a mixed integer nonlinear problem. Secondly, a task computing offloading algorithm based on deep reinforcement learning is proposed to solve this problem. Based on the improved double deep Q learning algorithm, the algorithm uses deep neural network to find the mapping between drones in deep reinforcement learning, find potential patterns from the state space and estimate the optimal action, and uses model-free DRL method. Enable each drone to make quick unloading decisions based on local observations. In order to verify the effectiveness of the proposed scheme, detailed simulation is carried out in this paper. Simulation results show that the proposed algorithm reduces the average unloading cost by 42.8% compared with LCGP algorithm. Compared with DDPG algorithm, the energy consumption is reduced by 16%. Compared with DDQN algorithm, the task execution delay is reduced by 12.9%
- References:
-
[ 1 ] PANWAR P, SHABAZ M, NAZIR S, et al. Generic edge computing system for optimization and computation offloading of unmanned aerial vehicle [ J ] . Computers and Electrical Engineering, 2023, 109: 108779.
[ 2 ]陈新颖, 盛敏, 李博, 等. 面向 6G 的无人机通信综述[ J] . 电子与信息学报, 2022, 44(3) : 781-789.
CHEN X Y, SHENG M, LI B, et al. Survey on unmanned aerial vehicle communications for 6G[ J] . Journal of Electronics & Information Technology, 2022, 44 (3) : 781-789.
[ 3 ] NGUYEN V, KHANH T T, VAN NAM P, et al. Towards flying mobile edge computing[ C]∥2020 International Conference on Information Networking ( ICOIN) . Piscataway: IEEE, 2020: 723-725.
[ 4 ] MA M L, GONG C Y, WU L T, et al. FLIRRAS: fast learning with integrated reward and reduced action space for online multitask offloading [ J ] . IEEE Internet of Things Journal, 2023, 10(6) : 5406-5417.
[ 5 ] AHANI G, YUAN D. BS-assisted task offloading for D2D networks with presence of user mobility[ C]∥2019 IEEE 89th Vehicular Technology Conference ( VTC2019-Spring) . Piscataway: IEEE, 2019: 1-5.
[ 6 ] LI N N, HAO W M, ZHOU F H, et al. Smart grid enabled computation offloading and resource allocation for SWIPT-based MEC system [ J] . IEEE Transactions on Circuits and Systems II: Express Briefs, 2022, 69(8) : 3610-3614.
[ 7 ] WANG L, HUANG P Q, WANG K Z, et al. RL-based user association and resource allocation for multi-UAV enabled MEC [ C ] ∥2019 15th International Wireless Communications & Mobile Computing Conference ( IWCMC) . Piscatawy: IEEE, 2019: 741-746.
[ 8 ] 王丙琛, 司怀伟, 谭国真. 基于深度强化学习的自动驾驶车控制算法研究[ J] . 郑州大学学报( 工学版) , 2020, 41(4) : 41-45, 80.
WANG B C, SI H W, TAN G Z. Research on autopilot control algorithm based on deep reinforcement learning [ J] . Journal of Zhengzhou University ( Engineering Science) , 2020, 41(4) : 41-45, 80.
[ 9 ] YE Y T, WEI W S, GENG D Q, et al. Dynamic coordination in UAV swarm assisted MEC via decentralized
deep reinforcement learning[ C]∥2020 International Conference on Wireless Communications and Signal Processing ( WCSP) . Piscataway: IEEE, 2020: 1064-1069.
[10] WANG L, WANG K Z, PAN C H, et al. Deep reinforcement learning based dynamic trajectory control for
UAV-assisted mobile edge computing[ J] . IEEE Transactions on Mobile Computing, 2022, 21(10) : 3536-3550.
[11] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[ EB / OL] . (2019- 07 - 05 ) [ 2024 - 06 - 01 ] . https:∥arxiv. org /abs / 1509. 02971.
[12] MAO S, HE S F, WU J S. Joint UAV position optimization and resource scheduling in space-air-ground integrated networks with mixed cloud-edge computing[ J] . IEEE Systems Journal, 2021, 15(3) : 3992-4002.
[13] LIU Q, SHI L, SUN L L, et al. Path planning for UAVmounted mobile edge computing with deep reinforcement learning[ J] . IEEE Transactions on Vehicular Technology, 2020, 69(5) : 5723-5728.
[14] ZHANG L, ZHANG Z Y, MIN L, et al. Task offloading and trajectory control for UAV-assisted mobile edge computing using deep reinforcement learning [ J] . IEEE Access, 2021, 9: 53708-53719.
[15] WANG Y T, CHEN W W, LUAN T H, et al. Task offloading for post-disaster rescue in unmanned aerial vehicles networks [ J] . IEEE / ACM Transactions on Networking, 2022, 30(4) : 1525-1539.
[16] 李斌, 刘文帅, 费泽松. 面向空天地异构网络的边缘计算部分任务卸载策略[ J] . 电子与信息学报, 2022, 44(9) : 3091-3098.
LI B, LIU W S, FEI Z S. Partial computation offloading for mobile edge computing in space-air-ground integrated
network[ J] . Journal of Electronics & Information Technology, 2022, 44(9) : 3091-3098.
[17] HUO Y, LIU Q Y, GAO Q H, et al. Joint task offloading and resource allocation for secure OFDMA-based mobile edge computing systems [ J] . Ad Hoc Networks, 2024, 153: 103342.
[18] XU Y C, CHEN L L, LU Z H, et al. An adaptive mechanism for dynamically collaborative computing power and task scheduling in edge environment [ J] . IEEE Internet of Things Journal, 2023, 10(4) : 3118-3129.
[19] 李斌. 基于多智能体强化学习的多无人机边缘计算任务卸载[ J] . 无线电工程, 2023, 53(12) : 2731-2740. LI B. Multi-agent reinforcement learning-based task offloading for multi-UAV edge computing [ J] . Radio Engi
neering, 2023, 53(12) : 2731-2740.
[20] LI W, ZHOU F, CHOWDHURY K R, et al. QTCP: adaptive congestion control with reinforcement learning
[ J] . IEEE Transactions on Network Science and Engineering, 2019, 6(3) : 445-458.
[21] WANG P, NI W L. An enhanced dueling double deep Qnetwork with convolutional block attention module for traffic signal optimization in deep reinforcement learning[ J] . IEEE Access, 2024, 12: 44224-44232.
- Similar References:
-