[1]李学相,曹淇,刘成明.基于无配对生成对抗网络的图像超分辨率重建[J].郑州大学学报(工学版),2021,42(5):1-6.[doi:10.13705/j.issn.1671-6833.2021.05.018]
　LI Xuexiang,CAO Qi,LIU Chengming.Image Super-resolution Based on No Match Generative Adversarial Network[J].Journal of Zhengzhou University (Engineering Science),2021,42(5):1-6.[doi:10.13705/j.issn.1671-6833.2021.05.018]
点击复制

基于无配对生成对抗网络的图像超分辨率重建()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
42
期数:
2021年5期
页码:
1-6
栏目:
出版日期:
2021-09-10

文章信息/Info

Title:
Image Super-resolution Based on No Match Generative Adversarial Network
作者:
李学相,曹淇,刘成明
郑州大学 软件学院，河南 郑州 450002
Author(s):
LI Xuexiang, CAO Qi, LIU Chengming
School of Software, Zhengzhou University, Zhengzhou 450002, China

关键词:
Keywords:
super-resolution;  deep learning;  generative adversarial network;  no matching;  second-order statistic
DOI:
10.13705/j.issn.1671-6833.2021.05.018
文献标志码:
A
摘要:
针对基于生成对抗网络的图像超分辨率重建方法依赖配对数据集训练且结果不稳定的问题，提出了一个新的基于无配对图像的模型 NM-SRGAN。首先，通过使用循环生成对抗网络作预处理模块，使模型可以不依赖配对数据集进行训练且获得更好的输入图像，同时该模型取消了 BN 层的使用，解决了结果不稳定的问题。然后，使用了协方差矩阵捕捉图像的二阶信息，增加了二阶损失函数，更加注重于捕捉图像细节区域部分的变化。最后，通过使用新的 VGG 损失函数提升了图像的边缘纹理细节。对提出的 NM-SRGAN 模型在 4 个标准数据集上进行测试评估，使用客观评价标准对结果图进行评价，NM-SRGAN模型较目前若干先进模型中的最佳峰值信噪比分别提升了 0. 19、0. 03、0. 13、0. 02 dB，在 4 个数据集上的评价值均达到最高。实验结果表明，该模型在稳定性、图像质量和细节方面较经典算法均有较好的提升。
Abstract:
Image super-resolution reconstruction based on generative adversarial networks (GAN) is subject to the dataset training with an unstable result. To solve this problem, a new NM-SRGAN model is established. The cycle-gan is firstly used as the preprocess module to make the model free from the dataset for training with better input of the image, and the model cancels BN layer to solve the unstable results. Besides, covariance matrix is adopted to capture the second-order information of the image, and second-order loss function is added with a focus on the changes of the image details. The new VGG loss function is used to improve the marginal texture of the image. The proposed NM-SRGAN model is verified by four standard datasets, and the resulting images are assessed by the objective evaluation indices. Compared with the existing models, NM-SRGAN model has an improved evaluation value of 0.19, 0.03, 0.13, and 0.02 dB, respectively, reaching up to the maximum among the four datasets. Results show that the proposed method, compared with traditional algorithms, has achieved better improvements in stability and image quality with better details.

参考文献/References:

[1] NASROLLAHI K,MOESLUND T B.Super-resolution:a comprehensive survey[J].Machine vision and applications,2014,25(6):1423-1468.

[2] YUAN Y,LIU S Y,ZHANG J W,et al.Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Piscataway:IEEE,2018:814-823.
[3] DONG C,LOY C C,HE K M,et al.Learning a deep convolutional network for image super-resolution[J].Lecture notes in computer science,2014,8692:184-199.
[4] YANG C Y,MA C,YANG M H.Single-image super-resolution:a benchmark[J].Lecture notes in computer science, 2014,8692: 372-386.
[5]LEDIG C，THEIS L，HUSZR F，et al． Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference onComputer Vision and Pattern Recognition (CVPR) .Pisc away: IEEE，2017: 105－114．
[6]WANG X T，YU K，WU S X， et al． ESRGAN: enhanced super-resolution generative adversarial networks［C]//15th European Conference on Computer Vision，ECCV 2018． Cham: Springer，2018: 1－16．
[7]IOFFE S， SZEGEDY C． Batch normalization: accelerating deep network training by reducing internal covariate shift［EB/OL］．( 2015－02－ 11) ［2020－12－21］．https: / /arxiv．org /abs/1502. 03167．
[8]ACHARYA D，HUANG Z W，PAUDEL D P，et al．Covariance pooling for facial expression recognition［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops ( CVPRW) ．Piscataway: IEEE，2018: 480－486．
[9]王坤峰，苟超，段艳杰，等．生成式对抗网络 GAN 的研究进展与展望［J］．自动化学报，2017，43(3) : 321－332．
[10]ZHU J Y，PARK T，ISOLA P，et al．Unpaired image-toimage translation using cycle-consistent adversarial networks［C］/ /2017 IEEE International Conference on Computer Vision ( ICCV) ． Piscataway: IEEE，2017: 2242－2251．
[11]苏咪咪，叶中行．协方差矩阵奇异情况下的最优投资组合［J］．应用概率统计，2005(3) : 244－248．
[12]李方彪，何昕，魏仲慧，等．生成式对抗神经网络的多帧红外图像超分辨率重建［J］．红外与激光工程，2018，47( 2) : 26－33．
[13]HUANG Z W，GOOL L V．A Riemannian network for SPD matrix learning［EB/OL］．( 2016－12－22) ［2020－12－21］．https: / /arxiv．org /abs/1608. 04233．
[14]JOHNSON J，ALAHI A，LI F F． Perceptual losses for real-time style transfer and super-resolution［EB/OL］．( 2016－08－29) ［2020－12－21］． https: //arxiv．org /abs/1608. 08155．
[15]毛晓波，张志超．基于二维经验模态分解的单幅图像超分辨率重建［J］． 郑州大学学报 (工学版) ，2014，35( 5) : 15－18．

相似文献/References:

[1]袁航,钟发海,聂上上,等.基于卷积神经网络的道路拥堵识别研究[J].郑州大学学报(工学版),2019,40(2):21.[doi:10.13705/j.issn.1671-6833.2019.02.008]
　LUO Ronghui,YUAN Hang,ZHONG Fahai,et al.The Research of Traffic Jam Detection Based on Convolutional Neural Network[J].Journal of Zhengzhou University (Engineering Science),2019,40(5):21.[doi:10.13705/j.issn.1671-6833.2019.02.008]
[2]朱俊丞,杨之乐,郭媛君,等.深度学习在电力负荷预测中的应用综述[J].郑州大学学报(工学版),2019,40(5):12.[doi:10.13705/j.issn.1671-6833.2019.05.005]
　Zhu Juncheng,Young Joy,Guo Yuanjun,et al.A review of the application of deep learning in power load forecasting[J].Journal of Zhengzhou University (Engineering Science),2019,40(5):12.[doi:10.13705/j.issn.1671-6833.2019.05.005]
[3]黄文锋,徐珊珊,孙燚,等.基于多分辨率卷积神经网络的火焰检测[J].郑州大学学报(工学版),2019,40(5):79.[doi:10.13705/j.issn.1671-6833.2019.05.022]
　Huang Wenfeng,Susan Hsu,Sun Yi,et al.Fire Detection Based on Multi-resolution Convolution Neural Network in Various Scenes[J].Journal of Zhengzhou University (Engineering Science),2019,40(5):79.[doi:10.13705/j.issn.1671-6833.2019.05.022]
[4]陈义飞,郭胜,潘文安,等.基于多源传感器数据融合的三维场景重建[J].郑州大学学报(工学版),2021,42(2):81.[doi:10.13705/j.issn.1671-6833.2021.02.008]
　Chen Yifei,Guo Sheng,Pun Wan-On,et al.3D Scene Reconstruction Based on Multi-source Sensor Data Fusion[J].Journal of Zhengzhou University (Engineering Science),2021,42(5):81.[doi:10.13705/j.issn.1671-6833.2021.02.008]
[5]王希鹏,李永,李智,等.融合图像深度的抗遮挡目标跟踪算法[J].郑州大学学报(工学版),2021,42(5):19.[doi:10.13705/j.issn.1671-6833.2021.05.011]
　Wang Xipeng,Li Yong,Li Zhi,et al.Anti-occlusion Target Tracking Algorithm Based on Image Depth[J].Journal of Zhengzhou University (Engineering Science),2021,42(5):19.[doi:10.13705/j.issn.1671-6833.2021.05.011]
[6]卢晨辉,冯硕,易爱华,等.基于深度学习的加油站销量预测与营销策略应用研究[J].郑州大学学报(工学版),2022,43(1):1.[doi:10.13705/j.issn.1671-6833.2022.01.014]
　LU Chenhui,FENG Shuo,YI Aihua,et al.Gasoline Station Sales Prediction Method Based on Deep Learning and Its Application of Promotion Strategy[J].Journal of Zhengzhou University (Engineering Science),2022,43(5):1.[doi:10.13705/j.issn.1671-6833.2022.01.014]
[7]陈浩杰,黄锦,左兴权,等.基于宽度&深度学习的基站网络流量预测方法[J].郑州大学学报(工学版),2022,43(1):7.[doi:10.13705/j.issn.1671-6833.2022.01.011]
　CHEN Haojie,HUANG Jin,ZUO Xingquan,et al.Base Station Network Traffic Prediction Method Based on Wide & Deep Learning[J].Journal of Zhengzhou University (Engineering Science),2022,43(5):7.[doi:10.13705/j.issn.1671-6833.2022.01.011]
[8]成科扬,荣兰,蒋森林,等.基于深度学习的遥感图像超分辨率重建技术综述[J].郑州大学学报(工学版),2022,43(5):8.[doi:10.13705/j.issn.1671-6833.2022.05.013]
　CHENG Keyang,RONG Lan,JIANG Senlin,et al.Overview of Methods for Remote Sensing Image Super-resolution Reconstruction Based on Deep Learning[J].Journal of Zhengzhou University (Engineering Science),2022,43(5):8.[doi:10.13705/j.issn.1671-6833.2022.05.013]
[9]院老虎,常玉坤,刘家夫.基于改进YOLOv5s的雾天场景车辆检测方法[J].郑州大学学报(工学版),2023,44(3):37.[doi:10.13705/j.issn.1671-6833.2023.03.005]
　YUAN Laohu,CHANG Yukun,LIU Jiafu.Vehicle Detection Method Based on Improved YOLOv5s in Foggy Scene[J].Journal of Zhengzhou University (Engineering Science),2023,44(5):37.[doi:10.13705/j.issn.1671-6833.2023.03.005]
[10]高宇飞,马自行,徐 静,等.基于卷积和可变形注意力的脑胶质瘤图像分割[J].郑州大学学报(工学版),2024,45(2):27.[doi:10.13705/j.issn.1671-6833.2023.05.007]
　GAO Yufei,MA Zixing,XU Jing,et al.Brain Glioma Image Segmentation Based on Convolution and Deformable Attention[J].Journal of Zhengzhou University (Engineering Science),2024,45(5):27.[doi:10.13705/j.issn.1671-6833.2023.05.007]

更新日期/Last Update: 2021-10-11