[1]张建华,陶莹,赵思.基于TD3算法的光伏电站参与电力系统频率控制策略[J].郑州大学学报(工学版),2024,45(pre1):9.[doi:10.13705/j.issn.1671-6833.2024.06.023]
　ZHANG Jianhua,TAO Ying,ZHAO Si.Frequency Control Strategy of Photovoltaic Participation in Power System Based on TD3 Algorithm[J].Journal of Zhengzhou University (Engineering Science),2024,45(pre1):9.[doi:10.13705/j.issn.1671-6833.2024.06.023]
点击复制

基于TD3算法的光伏电站参与电力系统频率控制策略()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
45
期数:
2024年pre1
页码:
9
栏目:
出版日期:
2025-01-30

文章信息/Info

Title:
Frequency Control Strategy of Photovoltaic Participation in Power System Based on TD3 Algorithm
作者:
张建华; 陶莹; 赵思
(华北电力大学 控制与计算机工程学院,北京102206)
Author(s):
ZHANG Jianhua;  TAO Ying;  ZHAO Si
(School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China)
关键词:
光伏并网系统; 一次调频; 深度强化学习; 双延迟深度确定性策略梯度算法; 控制性能
Keywords:
photovoltaic grid-connected system;  primary frequency regulation ;  deep reinforcement learning ;  twin delayed deep deterministic policy gradient algorithm ;  control performance
分类号:
TM615
DOI:
10.13705/j.issn.1671-6833.2024.06.023
摘要:
针对光伏电力输出具有间歇性和随机性对维持系统频率稳定构成的挑战，本文提出了一种基于深度强化学习的快速频率调节方法，该方法无需依赖特定的机理模型，适用于解决与光伏发电相关的强不确定性问题。首先，本文构建了一个简化的光伏发电系统模型。随后，基于双延迟深度确定性策略梯度算法设计了一种新型频率控制器。未验证所提控制策略的有效性，将其与传统下垂控制、滑模控制及基于深度确定性策略梯度算法的控制策略进行了比较。仿真结果表明，在施加两种不同的负荷扰动后，基于所提控制策略的性能指标表现优异，如最大频率偏差低于其他控制算法，充分验证了所提控制策略的有效性和优越性。
Abstract:
Addressing the challenges posed by the intermittency and randomness of photovoltaic power output to maintaining system frequency stability, this paper proposes a rapid frequency regulation method based on deep reinforcement learning. This method does not require a specific mechanistic model and is suitable for solving strong uncertainty problems related to photovoltaic power generation. Firstly, a simplified photovoltaic power generation system model is constructed in this paper. Subsequently, a novel frequency controller is designed based on the twin delayed deep deterministic policy gradient algorithm. To verify the effectiveness of the proposed control strategy, it is compared with traditional droop control, sliding mode control, and a control strategy based on the deep deterministic policy gradient algorithm. The simulation results show that the performance indicators of the proposed control strategy are excellent, such as the maximum frequency deviation is lower than that of other control algorithms, which fully verifies the effectiveness and superiority of the proposed control strategy after applying two different load disturbances

参考文献/References:

[1].罗朋, 陈光浩, 杨东红, 等. 用于新能源发电的新型升降压转换器及其控制策略[J]. 郑州大学学报(工学版), 2024, 45(2): 97-105.
LUO P, CHEN G H, YANG D H, et al. A novel buck-boost converter and its control strate-gy for new energy power generation[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45(2): 97-105.
[2].王燕, 杨秀媛, 徐剑锋, 等. 民用可控负荷参与需求响应的控制策略[J]. 发电技术, 2020, 41(6): 638-649.
WANG Y, YANG X Y, XU J F, et al. Control strategy of civil controllable load participating in demand response[J]. Power Generation Technology, 2020, 41(6): 638-649.
[3].王明东, 杨岙迪, 李龙好, 等. 基于 VSG 下垂优化控制的新能源电力系统惯性提升[J]. 郑州大学学报(工学版), 2024, 45(3): 127-133.
WANG M D, YANG A D, LI L H, et al. Inertia lifting of new energy power system-based VSG droop optimal control[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45(3): 127-133.
[4].SU Y, LI H Y , CUI Y, et al. An adaptive PV frequency control strategy based on real-time inertia estimation[J]. IEEE Transactions on Smart Grid, 2020, 12(3): 2355-2364.
[5].ZHANG J H , ZHANG B, LI Q, et al. Fast frequency regulation method for power system with two-stage photovoltaic plants[J]. IEEE Transactions on Sustainable Energy, 2022, 13(3): 1779-1789.
[6].RAJESH T, GUNAPRIYA B, SABARIMUTHU M, et al. Frequency control of PV-connected micro grid system using fuzzy logic controller[J]. Materials Today: Proceedings, 2021, 45(2): 2260-2264.
[7].VANTI S, BANA P R, D’ARCO S, et al. Single-stage grid-connected PV system with finite control set model predictive control and an improved maximum power point tracking[J]. IEEE Transactions on Sustainable Energy, 2021, 13(2): 791-802.
[8].LI Z W , CHENG Z P , SI J K , et al. Adaptive power point tracking control of PV system for primary frequency regulation of AC microgrid with high PV integration[J]. IEEE Transactions on Power Systems, 2021, 36(4): 3129-3141.
[9].CUI W Q , JIANG Y, ZHANG B S . Reinforcement learning for optimal primary frequency control: A Lyapunov approach[J]. IEEE Transactions on Power Systems, 2022, 38(2): 1676-1688.
[10].ZHANG Y, SHI X H , ZHANG H X , et al. Review on deep learning applications in frequency analysis and control of modern power system[J]. International Journal of Electrical Power & Energy Systems, 2022, 136: 107744.
[11].ZHANG Z D , ZHANG D X , QIU R C. Deep reinforcement learning for power system applications: An overview[J]. CSEE Journal of Power and Energy Systems, 2019, 6(1): 213-225.
[12].Yan Z M , Xu Y. Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with continuous action search[J]. IEEE Transactions on Power Systems, 2018, 34(2): 1653-1656.
[13].Sun J, Zhu Z Q, Li H Q, et al. An integrated critic-actor neural network for reinforcement learning with application of DERs control in grid frequency regulation[J]. International Journal of Electrical Power & Energy Systems, 2019, 111: 286-299.
[14].杨悦, 王丹, 胡博, 等. 基于改进多智能体 Q 学习的多源最优联合调频控制策略研究[J]. 电力系统保护与控制, 2022, 50(7): 135-144.
YANG Y, WANG D, HU B, et al. Multi-source optimal joint frequency modulation control strategy based on improved multi-agent Q-learning[J]. Power System Protection and Control, 2022, 50(7): 135-144.
[15].Mu C X, Tang Y F, He H B. Improved sliding mode design for load frequency control of power system integrated an adaptive learning strategy[J]. IEEE Transactions on Industrial Electronics, 2017, 64(8): 6742-6751.
[16].CHEN X D , ZHANG M, WU Z G , et al. Model-Free load frequency control of nonlinear power systems based on deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2024 , 20(4): 6825-6833.
[17].秦智慧, 李宁, 刘晓彤, 等. 无模型强化学习研究综述[J]. 计算机科学, 2021, 48(3): 180-187.
QIN Z H, LI N, LIU X T, et al. Overview of research on model-free reinforcement learning[J]. Computer Science, 2021, 48(3): 180-187.
[18].FUJIMOTO S, HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[C]//International conference on machine learning. PMLR, 2018: 1587-1596.
[19].赵大伟, 马进, 钱敏慧, 等. 光伏电站参与大电网一次调频的控制增益研究[J]. 电网技术, 2019, 43(2): 425-433.
ZHAO D W, MA J, QIAN M H, et al. Research on control gain for photovoltaic power plants participating in primary frequency regulation of large power grid[J]. Power System Technology, 2019, 43(2): 425-433.
[20].KHALID J, RAMLI M A M, KHAN M S, et al. Efficient load frequency control of renewable integrated power system: A twin delayed DDPG-based deep reinforcement learning approach[J]. IEEE Access, 2022, 10: 51561-51574.
[21].Tang Y, Hu W, Cao D, et al. Deep reinforcement learning aided variable-frequency triple-phase-shift control for dual-active-bridge converter[J]. IEEE Transactions on Industrial Electronics, 2022, 70(10): 10506-10515.
[22].KINGMA D P, BA J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980, 2014.
[23].LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. arXiv preprint arXiv: 1509.02971, 2015.

备注/Memo

备注/Memo:
基金项目：国家自然科学基金（61973116）、国家重点研发计划（2019YFB1505400）、中央高校基本科研业务费专项资金（2023JC001）资助项目作者简介：张建华（1969—），女，北京海淀人，华北电力大学教授，博士，博士生导师，研究方向为多调频资源耦合系统快速调频的协同优化及协调控制等，E-mail：zjh@ncepu.edu.cn。陶莹（1999—），女，浙江丽水人，硕士研究生，
更新日期/Last Update: 2024-11-12