Performance Optimization of GCR in GRAPES Based on CPU+GPU Heterogeneous Parallel
[1]HUANG Dongqiang,HUANG Jianqiang,JIA Jinfang,et al.Performance Optimization of GCR in GRAPES Based on CPU+GPU Heterogeneous Parallel[J].Journal of Zhengzhou University (Engineering Science),2022,43(06):15-21.[doi:10.13705/j.issn.1671-6833.2022.03.018]
Copy
Journal of Zhengzhou University (Engineering Science)[ISSN
1671-6833/CN
41-1339/T] Volume:
43
Number of periods:
2022 06
Page number:
15-21
Column:
Public date:
2022-09-02
- Title:
- Performance Optimization of GCR in GRAPES Based on CPU+GPU Heterogeneous Parallel
- CLC:
- TP311. 1；O244
- DOI:
- 10.13705/j.issn.1671-6833.2022.03.018
- Abstract:
- In order to improve the computational efficiency of the GRAPES(global/regional assimilation and prediction system) numerical weather prediction model, and to improve the performance of the dynamic framework, In order to solve the problem that the GCR algorithm was time-consuming in GRAPES mode, a CPU+GPU heterogeneous parallel preprocessing GCR algorithm was implemented. Firstly, incomplete LU decomposition was used to preprocess the coefficient matrix to reduce the number of iterations. On this basis, fine-grained parallelism of OpenMP and coarse-grained parallelism of MPI were implemented. OpenMP parallelism was mainly used to improve the performance of the program by using compiler guidance to the loop body without data dependence in the way of loop unrolling. MPI parallelism was used to divide the data into various processes and improve the scalability of parallel programs by non-blocking communication and optimizing the amount of communication data. MPI was responsible for process communication and iterative control between nodes, while CUDA was responsible for processing computation-intensive tasks. The time-consuming matrix calculation part of GCR was transferred to GPU for processing, and memory optimization and data transmission optimization were adopted to reduce the data transmission overhead between CPU and GPU. The experimental results showed that the parallel acceleration ratio of OpenMP was 2.24 times that of the serial program, the parallel acceleration ratio of MPI was 3.32 times that of the serial program, and the parallel acceleration ratio of MPI+CUDA was 4.69 times that of the serial program. The performance optimization of the generalized conjugate redundancy algorithm on the heterogeneous platform was realized, and the computational efficiency of the program was improved.
- References:
-
[1] 沈旭军， 王建军， 李志强， 等.中国数值天气预报研究与业务发展[J] .气象研究， 2020， 34（4） ： 675-698.
[2] 张志远，周宇峰，刘利，等.MASNUM 海浪模式的性 能特点分析与并行优化[ J] .计算机研究与发展， 2015， 52（4） ： 851-860.[3] 肖洒，魏敏，邓帅，等.基于 GPU-OpenACC 的气候模 式加 速 优 化 研 究 [ J ] . 气 象, 2019, 45 ( 7 ) : 1001-1008.XIAO S, WEI M, DENG S, et al. Research on accelerated optimization of climate models based on GPUOpenACC [ J ] . Meteorological monthly, 2019, 45 (7) : 1001-1008.[4] 魏敏,王彬,何香,等. BCCAGCM 模式在神威·太湖 之光系 统 的 优 化 [ J ] . 应 用 气 象 学 报, 2019, 30 (4) : 502-512.WEI M, WANG B, HE X, et al. Optimizing BCCAGCM on sunway TaihuLight[ J] . Journal of applied meteorological science, 2019, 30(4) : 502-512.[5] DIWAN G C, MOHAMED M S. Iterative solution of Helmholtz problem with high-order isogeometric analysis and finite element method at mid-range frequencies [ J] . Computer methods in applied mechanics and engineering, 2020, 363: 112855.[6] 金之雁,杨磊,林隽民,等. 广义共轭余差法的通信 避免算法[ J] . 计算机工程与应用, 2020, 56( 3) : 74-79.JIN Z Y, YANG L, LIN J M, et al. Communication avoiding algorithm of generalized conjugate residual method[ J] . Computer engineering and applications, 2020, 56(3) : 74-79.[7] LI L F, XUE W, RANJAN R, et al. A scalable Helmholtz solver in GRAPES over large-scale multicore cluster [ J ] . Concurrency and computation: practice and experience, 2013, 25(12) : 1722-1737.[8] 刘钊. 基于国产高性能计算机的 GRAPES 性能优化 研究[D] . 上海: 上海交通大学, 2014.LIU Z. Study of GRAPES numerical weather prediction system optimization on domestic high performance computers[D]. Shanghai: Shanghai Jiao Tong University, 2014.[9] 王卓薇,许先斌,赵武清,等. 基于 GPU 的 GRAPES 模型并行 加 速 及 性 能 优 化 [ J] . 计 算 机 研 究 与 发 展, 2013, 50(2) : 401-411.WANG Z W, XU X B, ZHAO W Q, et al. Parallel acceleration and performance optimization for GRAPES model based on GPU [ J ] . Journal of computer research and development, 2013, 50(2) : 401-411.[10] 王克文,冶梦雨,刘艳红. 建立电力系统状态空间方 程的并行方法[ J] . 郑州大学学报(工学版) , 2021, 42(1) : 15-20.WANG K W, YE M Y, LIU Y H. Parallel method for establishing state space equation of power system[ J] . Journal of Zhengzhou university ( engineering science) , 2021, 42(1) : 15-20.[11] 薛纪善,陈德辉. 数值预报系统 GRAPES 的科学设 计与应用[M] . 北京: 科学出版社, 2008.XUE J S, CHEN D H. Scientific design and application of numerical prediction system GRAPES [ M ] . Beijing: Science Press, 2008.[12] 李建斌,王鹏程,傅侃,等. 基于预处理共轭梯度迭 代法的电力系统状态估计算法[ J] . 电力系统自动 化, 2021, 45(14) : 90-96.LI J B, WANG P C, FU K, et al. State estimation algorithm of power system based on preconditioned conjugate gradient iteration [ J ] . Automation of electric power systems, 2021, 45(14) : 90-96.[13] 刘宇,曹建文. 适用于 GRAPES 数值天气预报软件 的 ILU 预条件 子 [ J] . 计 算 机 工 程 与 设 计, 2008, 29(3) : 731-734.LIU Y, CAO J W. ILU preconditioner for NWP system: GRAPES[ J] . Computer engineering and design, 2008, 29(3) : 731-734.[14] LI Y, XIE H H, XU R, et al. A parallel generalized conjugate gradient method for large scale eigenvalue problems[ J] . CCF transactions on high performance computing, 2020, 2(2) : 111-122.[15] GANDER M J, GRAHAM I G, SPENCE E A. Applying GMRES to the Helmholtz equation with shifted Laplacian preconditioning: what is the largest shift for which wavenumber-independent convergence is guaranteed[ J ] . Numerische mathematik, 2015, 131 ( 3 ) : 567-614.[16] HUANG J Q, XUE W, BIAN H D, et al. Helmholtz solving and performance optimization in global / regional assimilation and prediction system [ J] . Tsinghua science and technology, 2020, 26(3) : 335-346.
- Similar References:
Memo
-
Last Update:
2022-10-03