Large-scale Agile Software Project Scheduling Based on Deep Reinforcement Learning

NAVIGATE

Table of Contents

STATISTICS

Viewed975

Downloads993

Large-scale Agile Software Project Scheduling Based on Deep Reinforcement Learning

[HTML] PDF下载 (993)

[1]SHEN Xiaoning,MAO Mingjian,SHEN Ruyi,et al.Large-scale Agile Software Project Scheduling Based on Deep Reinforcement Learning[J].Journal of Zhengzhou University (Engineering Science),2023,44(05):17-23.[doi:10.13705/j.issn.1671-6833.2023.05.003]

Copy

Journal of Zhengzhou University (Engineering Science)[ISSN 1671-6833/CN 41-1339/T] Volume: 44卷 Number of periods: 2023 05 Page number: 17-23 Column: Public date: 2023-08-20

Title:: Large-scale Agile Software Project Scheduling Based on Deep Reinforcement Learning

Author(s):: SHEN Xiaoning¹; 2; 3; 4 ; MAO Mingjian¹ ; SHEN Ruyi ¹ ; SONG Liyan ⁵; 1. School of Automation, Nanjing University of Information Science and Technology, Nanjing 210044, China; 2. Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing 210044, China; 3. Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, Nanjing 210044, China; 4. Jiangsu Engineering Research Center on Meteorological Energy Using and Control ( C-MEIC) , Nanjing 210044, China; 5. Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Southern University of Science and Technology, Shenzhen 518055, China

Keywords:: reinforcement learning; large-scale; agile software project scheduling; deep Q network; composite scheduling rules; priority experience replay; strong coupling

CLC:: TP311. 5；TP301. 6

DOI:: 10.13705/j.issn.1671-6833.2023.05.003

Abstract:: This study aimed to solve the scheduling problem of large-scale agile software project. It was decomposed into three strong-coupled subproblems: story selection, story allocation and task allocation. Dynamic events such as the addition and deletion of user stories, the change of employee′s working hours in each sprint, and other constraints such as team development speed, task duration and skills were introduced. To maximize the total value of user stories completed by the project, a large-scale agile software project scheduling mathematical model was established. According to the characteristics of the problem, the Markov decision process was designed. Ten state features were used to describe the agile scheduling environment at the beginning of each sprint; 12 composite scheduling rules were designed as candidate actions of the agent; and rewards were defined according to the objective function of the scheduling model. A priority experience replay double deep Q network algorithm based on composite scheduling rules was proposed to solve the built model. The double Q network strategy and priority experience replay strategy were introduced to avoid the over-estimation problem of deep Q network and improve the utilization efficiency of trajectory information in the experience replay pool. In order to verify the effectiveness of the proposed algorithm, experiments were carried out in six large-scale agile software project scheduling numerical examples, and the convergence of the proposed algorithm was analyzed. According to the performance measurement of the algorithm, it was compared with the existing representative algorithm DQN, double deep Q network and 12 single composite scheduling rules. The results showed that it had the highest average cumulative reward value in 6 different numerical examples.

References:: [1] 王映红. 企业大规模敏捷转型探索与实践[ J] . 金融科技时代, 2017, 25(11) : 84-85.
WANG Y H. Exploration and practice of large-scale agile transformation of enterprises [ J ] . Financial Technology Time, 2017, 25(11) : 84-85.
[2] BIESIALSKA K, FRANCH X, MUNTÉS-MULERO V. Mining dependencies in large-scale agile software development projects: a quantitative industry study[C]∥Evaluation and Assessment in Software Engineering. New York:ACM, 2021: 20-29.
[3] COHN M. Agile estimating and planning [ M ] . Upper Saddle River: Prentice Hall,2005.
[4] ZAPOTECAS-MARTÍNEZ S, GARCÍA-NÁJERA A, CERVANTES H. Multi-objective optimization in the agile software project scheduling using decomposition [ C ] ∥ Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. New York: ACM, 2020: 1495-1502.
[5] ROQUE L, ARAÚJO A A, DANTAS A, et al. Human Resource Allocation in Agile Software Projects Based on Task Similarities[C]∥International Symposium on Search Based Software Engineering. Cham: Springer, 2016: 291 -297.
[6] XIAO J, AO X T, TANG Y. Solving software project scheduling problems with ant colony optimization[J]. Computers & Operations Research, 2013, 40(1): 33-46.
[7] SHEN X N, GUO Y N, LI A M. Cooperative coevolution with an improved resource allocation for large-scale multiobjective software project scheduling [ J ] . Applied Soft Computing, 2020, 88: 106059.
[8] PADBERG F, WEISS D. Optimal scheduling of software projects using reinforcement learning[ C]∥2011 18th Asia-Pacific Software Engineering Conference. Piscataway: IEEE, 2012: 9-16.
[9] LIU W T, SU S, TANG T, et al. A DQN-based intelligent control method for heavy haul trains on long steep downhill section[ J] . Transportation Research Part C: Emerging Technologies, 2021, 129: 103249.
[10] HUYNH T N, DO D T T, LEE J. Q-Learning-based parameter control in differential evolution for structural optimization[J]. Applied Soft Computing, 2021, 107: 107464.
[11] LUO S. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning [ J ] . Applied Soft Computing, 2020, 91: 106208.
[12] LI Y X, GU W B, YUAN M H, et al. Real-time datadriven dynamic scheduling for flexible job shop with insufficient transportation resources using hybrid deep Q network[ J] . Robotics and Computer-Integrated Manufacturing, 2022, 74: 102283.
[13] Van HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-Learning [ C]∥Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. New York: ACM, 2016: 2094-2100.
[14] SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay[EB / OL]. ( 2016- 02- 25) [ 2023- 02-09]. https:∥arxiv. org / abs/ 1511. 05952.
[15] 王培崇, 尹欣洁, 李丽荣. 一种具有学习机制的海鸥优化算法 [ J] . 郑州大学学报 ( 工学版) , 2022, 43 (6) : 8-14.
WANG P C, YIN X J, LI L R. An improved seagull optimization algorithm with learning[J]. Journal of Zhengzhou University (Engineering Science), 2022, 43(6): 8-14.

Similar References:

Memo

Last Update: 2023-09-03