[1]张建华,陶　莹,赵　思.基于TD3算法的光伏电站参与电力系统频率控制策略[J].郑州大学学报(工学版),2025,46(03):42-49.[doi:10.13705/j.issn.1671-6833.2024.06.023]
　ZHANG Jianhua,TAO Ying,ZHAO Si.Frequency Control Strategy of Photovoltaic Participation in Power System Based on TD3 Algorithm[J].Journal of Zhengzhou University (Engineering Science),2025,46(03):42-49.[doi:10.13705/j.issn.1671-6833.2024.06.023]
点击复制

基于TD3算法的光伏电站参与电力系统频率控制策略()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
46
期数:
2025年03期
页码:
42-49
栏目:
出版日期:
2025-05-13

文章信息/Info

Title:
Frequency Control Strategy of Photovoltaic Participation in Power System Based on TD3 Algorithm
文章编号:
1671-6833(2025)03-0042-08
作者:
张建华;  陶　莹;  赵　思
华北电力大学 控制与计算机工程学院,北京 102206
Author(s):
ZHANG Jianhua;  TAO Ying;  ZHAO Si
School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
关键词:
光伏并网系统;  一次调频;  深度强化学习;  双延迟深度确定性策略梯度算法;  控制性能
Keywords:
photovoltaic grid-connected system;  primary frequency regulation;  deep reinforcement learning;  twin delayed deep deterministic policy gradient algorithm;  control performance
分类号:
TM615
DOI:
10.13705/j.issn.1671-6833.2024.06.023
文献标志码:
A
摘要:
针对光伏电力输出具有间歇性和随机性对维持电力系统频率稳定构成的挑战,提出了一种基于双延迟深度确定性策略梯度算法的快速频率调节方法,该方法无须依赖特定的机理模型,适用于解决与光伏发电相关的强不确定性问题。首先,构建了一个简化的光伏发电系统模型;其次,基于双延迟深度确定性策略梯度算法设计了一种新型频率控制器;最后,将所提控制策略与传统下垂控制、滑模控制及基于深度确定性策略梯度算法的控制策略进行了比较。结果表明:在分别施加负荷单次阶跃扰动和负荷连续阶跃扰动的两种场景中,基于所提控制策略的频率偏差均明显低于其他3种控制算法,时间乘绝对误差积分准则比性能最差的下垂控制分别减小了41.7%和31.8%,充分验证了所提控制策略在调频过程动态性能和稳态性能方面的优越性。
Abstract:
To address the challenges posed by the intermittency and randomness of photovoltaic (PV) power output to maintaining stable power system frequency, a rapid frequency regulation method based on the twin delayed deep deterministic policy gradient (TD3) algorithm was proposed. No need to rely on specific mechanistic models, this method could tackle the strong uncertainties associated with PV power generation. Firstly, a simplified model of the PV power generation system was constructed. Secondly, a novel frequency controller was designed leveraging the TD3 algorithm. Lastly, the proposed control strategy was compared with traditional droop control, sliding mode control, and a control strategy based on the deep deterministic policy gradient (DDPG) algorithm. The results demonstrated that, in two scenarios, single-step and continuous-step load disturbances respectively, the frequency deviations based on the proposed control strategy were significantly lower than those of the other three control algorithms. Specifically, the integral of time-weighted absolute error (ITAE) criterion showed a reduction of 41.7% and 31.8% compared to the worst-performing droop control, thoroughly validating the superiority of the proposed control strategy in terms of both dynamic and steady-state performance during frequency regulation.

参考文献/References:

[1]罗朋, 陈光浩, 杨东红, 等. 用于新能源发电的新型升降压转换器及其控制策略[J]. 郑州大学学报(工学版), 2024, 45(2): 97-105. 

LUO P, CHEN G H, YANG D H, et al. A novel buckboost converter and its control strategy for new energy power generation[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45(2): 97-105. 
[2]王燕, 杨秀媛, 徐剑锋, 等. 民用可控负荷参与需求响应的控制策略[J]. 发电技术, 2020, 41(6): 638-649. 
WANG Y, YANG X Y, XU J F, et al. Control strategy of civil controllable load participating in demand response [J]. Power Generation Technology, 2020, 41(6): 638649. 
[3]王明东, 杨岙迪, 李龙好, 等. 基于VSG下垂优化控制的新能源电力系统惯性提升[J]. 郑州大学学报(工学版), 2024, 45(3): 127-133. 
WANG M D, YANG A D, LI L H, et al. Inertia lifting of new energy power system based on VSG droop optimal control[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45(3): 127-133. 
[4]SU Y, LI H Y, CUI Y, et al. An adaptive PV frequency control strategy based on real-time inertia estimation[J]. IEEE Transactions on Smart Grid, 2021, 12(3): 2355-2364. 
[5]ZHANG J H, ZHANG B, LI Q, et al. Fast frequency regulation method for power system with two-stage photovoltaic plants[J]. IEEE Transactions on Sustainable Energy, 2022, 13(3): 1779-1789. 
[6]RAJESH T, GUNAPRIYA B, SABARIMUTHU M, et al. Frequency control of PV-connected micro grid system using fuzzy logic controller[J]. Materials Today: Proceedings, 2021, 45: 2260-2264. 
[7]VANTI S, BANA P R, D′ARCO S, et al. Single-stage grid-connected PV system with finite control set model predictive control and an improved maximum power point tracking[J]. IEEE Transactions on Sustainable Energy, 2022, 13(2): 791-802. 
[8]LI Z W, CHENG Z P, SI J K, et al. Adaptive power point tracking control of PV system for primary frequency regulation of AC microgrid with high PV integration[J]. IEEE Transactions on Power Systems, 2021, 36(4): 3129-3141. 
[9]CUI W Q, JIANG Y, ZHANG B S. Reinforcement learning for optimal primary frequency control: a Lyapunov approach[J]. IEEE Transactions on Power Systems, 2023, 38(2): 1676-1688. 
[10] ZHANG Y, SHI X H, ZHANG H X, et al. Review on deep learning applications in frequency analysis and control of modern power system[J]. International Journal of Electrical Power & Energy Systems, 2022, 136: 107744. 
[11] ZHANG Z D, ZHANG D X, QIU R C. Deep reinforcement learning for power system applications: an overview [J]. CSEE Journal of Power and Energy Systems, 2020, 6(1): 213-225. 
[12] YAN Z M, XU Y. Data-driven load frequency control for stochastic power systems: a deep reinforcement learning method with continuous action search[J]. IEEE Transactions on Power Systems, 2019, 34(2): 1653-1656. 
[13] SUN J, ZHU Z Q, LI H Q, et al. An integrated criticactor neural network for reinforcement learning with application of DERs control in grid frequency regulation[J]. International Journal of Electrical Power & Energy Systems, 2019, 111: 286-299. 
[14]杨悦, 王丹, 胡博, 等. 基于改进多智能体Q学习的多源最优联合调频控制策略研究[J]. 电力系统保护与控制, 2022, 50(7): 135-144. 
YANG Y, WANG D, HU B, et al. Multi-source optimal joint frequency modulation control strategy based on improved multi-agent Q-learning[J]. Power System Protection and Control, 2022, 50(7): 135-144. 
[15] MU C X, TANG Y F, HE H B. Improved sliding mode design for load frequency control of power system integrated an adaptive learning strategy[J]. IEEE Transactions on Industrial Electronics, 2017, 64(8): 6742-6751. 
[16] CHEN X D, ZHANG M, WU Z G, et al. Model-free load frequency control of nonlinear power systems based on deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics, 2024, 20(4): 6825-6833. 
[17]秦智慧, 李宁, 刘晓彤, 等. 无模型强化学习研究综述[J]. 计算机科学, 2021, 48(3): 180-187. 
QIN Z H, LI N, LIU X T, et al. Overview of research on model-free reinforcement learning[J]. Computer Science, 2021, 48(3): 180-187. 
[18] FUJIMOTO S, VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[EB/ OL]. (2018-10-22)[2024-06-09]. http:∥arxiv.org/ abs/1802.09477. 
[19]赵大伟, 马进, 钱敏慧, 等. 光伏电站参与大电网一次调频的控制增益研究[J]. 电网技术, 2019, 43 (2): 425-433. 
ZHAO D W, MA J, QIAN M H, et al. Research on control gain for photovoltaic power plants participating in primary frequency regulation of large power grid[J]. Power System Technology, 2019, 43(2): 425-433. 
[20] KHALID J, RAMLI M A M, KHAN M S, et al. Efficient load frequency control of renewable integrated power system: a twin delayed DDPG-based deep reinforcement learning approach[J]. IEEE Access, 2022, 10: 5156151574. 
[21] TANG Y H, HU W H, CAO D, et al. Deep reinforcement learning aided variable-frequency triple-phase-shift control for dual-active-bridge converter[J]. IEEE Transactions on Industrial Electronics, 2023, 70(10): 10506-10515. 
[22] KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. (2017-01-30)[2024-06-09]. https:∥arxiv.org/abs/1412.6980. 
[23] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB/OL]. (2019-07-05) [2024-06-09]. https:∥arxiv. org/ abs/1509.02971.

更新日期/Last Update: 2025-05-22