Research on Intelligent Routing Technology Based on Deep Reinforcement Learning

NAVIGATE

Table of Contents

STATISTICS

Viewed1385

Downloads892

Research on Intelligent Routing Technology Based on Deep Reinforcement Learning

[HTML] PDF下载 (892)

[1]HUANG Wanwei,ZHENG Xiangyu,ZHANG Chaoqin,et al.Research on Intelligent Routing Technology Based on Deep Reinforcement Learning[J].Journal of Zhengzhou University (Engineering Science),2023,44(01):44-51.[doi:10.13705/j.issn.1671-6833.2022.04.018]

Copy

Journal of Zhengzhou University (Engineering Science)[ISSN 1671-6833/CN 41-1339/T] Volume: 44卷 Number of periods: 2023 01 Page number: 44-51 Column: Public date: 2022-12-06

Title:: Research on Intelligent Routing Technology Based on Deep Reinforcement Learning

Author(s):: HUANG Wanwei¹; ZHENG Xiangyu¹; ZHANG Chaoqin²; WANG Sunan³; ZHANG Xiaohui⁴; 1.College of Software Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, China; 2.College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, China; 3.School of Electronic and Communication Engineering, Shenzhen Polytechnic, Shenzhen 518055, China; 4.Henan Xin′an Communication Technology Co., Ltd., Zhengzhou 450001, China

Keywords:: quality of experience; software defined network; deep reinforcement learning; routing algorithms; recurrent deterministic policy gradient

CLC:: TP393

DOI:: 10.13705/j.issn.1671-6833.2022.04.018

Abstract:: To solve the problems of slow convergence speed, high average delay, and low bandwidth utilization of existing intelligent routing algorithms, in this study, a multi-path intelligent routing algorithm RDPG-Route based on deep reinforcement learning (DRL) was proposed. In the algorithm, the recurrent determi-nistic policy gradient (RDPG) was used as the training framework, the long short-term memory (LSTM) was introduced as the neural network. The algorithm advantages of RDPG were used to handle high-latitude problems and the storage capacity of the memory in the LSTM loop core, the dynamically changing network state could be input to the neural network for training. After the algorithm training converged, the action value output by the neural network was used as the network link weight, and the traffic was divided based on the multi-path routing strategy to realize the intelligent dynamic adjustment of the network routing. Finally, RDPG-Route routing algorithm was compared with ECMP, DRL-TE, and DRL-R-DDPG routing algorithms respectively. The results indicated that RDPG-Route had better convergence and effectiveness. Compared with other optimal intelligent routing algorithm, RDPG-Route could reduce the average end-to-end delay by at least 7.2%, improve the throughput by 6.5%, and reduce the packet loss rate by 8.9% and the maximum link utilization rate by 6.3%.

References:: [1] 刘振鹏, 王鑫鹏, 李明, 等. 基于时延和负载均衡的多控制器部署策略 [ J] . 郑州大学学报 ( 工学版) , 2021, 42(3) : 19-25, 32.
LIU Z P, WANG X P, LI M, et al. Multi-controller deployment strategy based on delay and load balancing[ J] . Journal of Zhengzhou university ( engineering science) , 2021, 42(3) : 19-25, 32.
[2] SCHWARZMANN S, MARQUEZAN C C, TRIVISONNO R, et al. Accuracy vs. cost trade-off for machine learning based QoE estimation in 5G networks [ C] / / IEEE International Conference on Communications ( ICC) . Piscataway: IEEE, 2020:1-6
[3] LIU Y F, ZHAO B, ZHAO P Y, et al. A survey: typical security issues of software-defined networking[ J] . China communications, 2019, 16(7) : 13-31.
[4] REZA M, JAVAD M, RAOUF S, et al. Network traffic classification using machine learning techniques over software defined networks[ J] . International journal of advanced computer science and applications, 2017, 8( 7) : 220-225.
[5] TANG F X, MAO B M, FADLULLAH Z M, et al. On removing routing protocol from future wireless networks: a real-time deep learning approach for intelligent traffic control [ J ] . IEEE wireless communications, 2018, 25 (1) : 154-160.
[6] RAO Z H, XU Y Y, PAN S M. A deep learning-based constrained intelligent routing method [ J] . Peer-to-peer networking and applications, 2021, 14(4) : 2224-2235.
[7] LIU W X, CAI J, CHEN Q C, et al. DRL-R: deep reinforcement learning approach for intelligent routing in software-defined data-center networks[ J] . Journal of network and computer applications, 2021, 177: 102865.
[8] CHEN B, SUN P H, ZHANG P, et al. Traffic engineering based on deep reinforcement learning in hybrid IP / SR network [ J ] . China communications, 2021, 18 ( 10 ) : 204-213.
[9] 王丙琛, 司怀伟, 谭国真. 基于深度强化学习的自动驾驶车控制算法研究[ J] . 郑州大学学报( 工学版) , 2020, 41(4) : 41-45, 80.
WANG B C, SI H W, TAN G Z. Research on autopilot control algorithm based on deep reinforcement learning [ J] . Journal of Zhengzhou university ( engineering science) , 2020, 41(4) : 41-45, 80.
[10] HEESS N, HUNT J J, LILLICRAP T P, et al. Me-morybased control with recurrent neural networks [ EB / OL] . (2015- 12 - 14 ) [ 2021 - 10 - 20 ] . https: / / arxiv. org / abs/ 1512. 04455v1.
[11] XI L, WU J N, XU Y C, et al. Automatic generation control based on multiple neural networks with actor-critic strategy[J]. IEEE transactions on neural networks and learning systems, 2021, 32(6): 2483-2493.
[12] FANG L L, LI X Y, WU Y R, et al. Deep recurrent Qlearning method for single intersection signal control [ C] / / 13th Asia Pacific Transportation Development Conference. Reston, USA: ASCE, 2020: 148-156.
[13] YAO Z, WANG Y, MENG L M, et al. DDPG-based energy-efficient flow scheduling algorithm in software-defined data centers[ J] . Wireless communications and mobile computing, 2021, 2021: 6629852.
[14] 李琳, 李玉泽, 张钰嘉, 等. 基于多估计器平均值的深度确定性策略梯度算法[ J] . 郑州大学学报( 工学版) , 2022, 43(2) : 15-21.
LI L, LI Y Z, ZHANG Y J, et al. Deep deterministic policy gradient algorithm based on mean of multiple estimators[ J] . Journal of Zhengzhou university ( engineering science) , 2022, 43(2) : 15-21.
[15] LI S, LI W Q, COOK C, et al. Independently recurrent neural network ( IndRNN) : building a longer and deeper RNN[C] / / 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition. Psicataway: IEEE, 2018: 5457-5466.
[16] SHERSTINSKY A. Fundamentals of recurrent neural network (RNN) and long short-term memory ( LSTM) network[ J] . Physica D: nonlinear phenomena, 2020, 404: 132306.
[17] WEHRLE K, GÜNEŞ M, GROSS J. Modeling and tools for network simulation[M] . Berlin: Springer-Verlag Berlin Heidelberg, 2010.
[18] PATHAK S, MANI A, SHARMA M, et al. A novel salp swarm algorithm for controller placement problem [ J ] . Trends in computational intelligence, security and Internet of Things, 2020, 1358:24-36.
[19] BULL P, MURPHY S, BRUNO JUNIOR N, et al. A flow analysis and preemption fra

Similar References:

Memo

Last Update: 2022-12-07