[1]逯 鹏,李科研,张宏坡,等.可微分神经网络架构搜索综述[J].郑州大学学报(工学版),2027,48(XX):1-10.[doi:10. 13705 / j. issn. 1671-6833. 2026. 02. 007]
　LU Peng,LI Keyan,ZHANG Hongpo,et al.Overview of Differentiable Neural Network Architecture Search[J].Journal of Zhengzhou University (Engineering Science),2027,48(XX):1-10.[doi:10. 13705 / j. issn. 1671-6833. 2026. 02. 007]
点击复制

可微分神经网络架构搜索综述()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
48
期数:
2027年XX
页码:
1-10
栏目:
出版日期:
2027-12-10

文章信息/Info

Title:
Overview of Differentiable Neural Network Architecture Search
作者:
逯 鹏1,2,3, 李科研1,2, 张宏坡3,4, 陈立伟1, 武家辉1,2, 刘帅兵1,2
1. 郑州大学 电气与信息工程学院,河南 郑州 450001;2. 机器人感知与控制河南省工程实验室,河南 郑州 450001;3. 互联网医疗与健康服务河南省协同创新中心,河南 郑州 450052;4. 郑州大学 网络管理中心,河南 郑州 450001
Author(s):
LU Peng1,2,3, LI Keyan1,2, ZHANG Hongpo3,4, CHEN Liwei1, WU Jiahui1,2, LIU Shuaibing1,2
1. School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China; 2. Robot Perception and Control Henan Engineering Laboratory, Zhengzhou 450001, China; 3. Henan Collaborative Innovation Center for Internet based Medicaland Health Services, Zhengzhou 450052, China; 4. Network Management Center, Zhengzhou University, Zhengzhou 450001, China
关键词:
神经网络架构搜索;  深度学习;  可微分神经网络架构搜索;  连续优化;  性能估计
Keywords:
neural architecture search;  deep learning;  differentiable neural architecture search;  Continuous optimization;  performance estimation
分类号:
TP181
DOI:
10. 13705 / j. issn. 1671-6833. 2026. 02. 007
文献标志码:
A
摘要:
神经网络架构搜索( neural architecture search,NAS) 是深度学习领域的交叉研究,旨在自动化地设计神经网络结构。 NAS 需要反复训练和评估大量候选网络,计算代价高。 可微分神经网络架构搜索( differentiable neuralarchitecture search,DNAS)将离散架构搜索问题转化为可微的连续优化问题,降低了计算代价。 首先从搜索空间、搜索策略和性能评估策略 3 个方面构建了可微分网络架构搜索算法框架;其次,分析、对比和总结了其参数化操作的性能估计偏差、架构过拟合与搜索稳定性问题以及优化搜索空间与提升效率问题的改进策略;然后,比较分析了典型 DNAS 算法在图像分类数据集上的错误率、参数量以及搜索消耗时间与实验硬件条件;最后指出 DNAS 在边缘设备部署、医学信号分析和跨模态匹配等复杂场景中的应用潜力,并提出面向多目标优化、任务驱动搜索空间设计及跨任务迁移复用的未来研究方向。
Abstract:
Neural Architecture Search (NAS) is an interdisciplinary study in the field of deep learning, which aims to automate the design of neural network structures. NAS requires repeated training and evaluation of a large number of candidate networks, which is computationally expensive. Differentiable Neural Architecture Search (DNAS) transforms the discrete architecture search problem into a differentiable continuous optimization problem, which reduces the computational cost. Firstly, a search algorithm framework for differentiable network architecture was constructed from three aspects: search space, search strategy and performance evaluation strategy. Secondly, the performance estimation bias, architecture overfitting and search stability problems of parameterization operation, as well as the improvement strategies of optimizing search space and improving efficiency are analyzed, compared and summarized. Then, the error rate, parameter quantity, search time and experimental hardware conditions of typical DNAS algorithms on image classification datasets were compared and analyzed. Finally, it points out the application potential of DNAS in complex scenarios such as edge device deployment, medical signal analysis, and cross-modal matching, and proposes future research directions toward multi-objective optimization, task-driven search space design, and cross-task transfer and reuse.

参考文献/References:

[1] Dos Santos C F G, Papa J P. Avoiding overfitting: a survey on regularization methods for convolutional neural networks[ J] . ACM Computing Surveys, 2022, 54(10s) : 1-25.

[2] Hanin B. Which neural net architectures give rise to exploding and vanishing gradients? [C]∥Proceedings of the32nd International Conference on Neural Information Pro￾cessing Systems. New York: ACM, 2018: 580-589.
[3] Sun Renke, Huangfu Zhiyu, Chen Hu, et al. Survey ofneural architecture search[ J] . Journal of Computer Applications, 2024, 44(10) : 2983-2994. [孙仁科, 皇甫志宇, 陈虎, 等. 神经架构搜索综述 [ J] . 计算机应用, 2024, 44(10) : 2983-2994. ]
[4] Liu Hanxiao, Simonyan K, Yang Yiming. DARTS: differentiable architecture search [ PP / OL ] . V2. arXiv(2019 - 04 - 23 ) [ 2025 - 10 - 09 ] . https: ∥doi. org /10. 48550 / arXiv. 1806. 09055.
[5] Heuillet A, Nasser A, Arioui H, et al. Efficient automation of neural network design: a survey on differentiableneural architecture search[ J] . ACM Computing Surveys,2024, 56(11) : 1-36.
[6] Zoph B, Vasudevan V, Shlens J, et al. Learning transferable architectures for scalable image recognition[ C]∥Proceedings of the 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE,2018: 8697-8710.
[7] He Chaoyang, Ye Haishan, Shen Li, et al. MiLeNAS:efficient neural architecture search via mixed-level reformulation[C]∥Proceedings of the 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway: IEEE, 2020: 11990-11999.
[8] Chu Xiangxiang, Zhou Tianbao, Zhang Bo, et al. FairDARTS: eliminating unfair advantages in differentiablearchitecture search[C]∥Computer vision – ECCV 2020.Cham: Springer International Publishing, 2020: 465-480.
[9] Shin J, Park K, Kang D K. TA-DARTS: temperature annealing of discrete operator distribution for effective differential architecture search [ J] . Applied Sciences, 2023,13(18) : 10138.郑 州 大 学 学 报 (工 学 版)
[10] Mun J, Ha S, Lee J. DE-DARTS: neural architecturesearch with dynamic exploration[ J] . ICT Express, 2023,9(3) : 379-384.
[11] Ma Benteng, Zhang Yanning, Xia Yong. Momentum recursive DARTS [ J ] . Pattern Recognition, 2024,156: 110710.
[12] Liu Wenbo, Deng Tao, An Rui, et al. DARTS-CGW:research on differentiable neural architecture search algorithm based on coarse gradient weighting [ C] ∥Patternrecognition and computer vision. Singapore: Springer Nature Singapore, 2025: 31-44.
[13] Wang Ruochen, Cheng Minhao, Chen Xiangning, et al.Rethinking architecture selection in differentiable NAS[ PP / OL] . V1. arXiv (2021-08-10) [2025-10-09] .https:∥doi. org / 10. 48550 / arXiv. 2108. 04392.
[14] Xie Weisheng, Li Hui, Fang Xuwei, et al. DARTS-PTCORE: collaborative and regularized perturbation-basedarchitecture selection for differentiable NAS [ J] . Neurocomputing, 2024, 580: 127522.
[15] Fang Xuwei, Xie Weisheng, Li Hui, et al. DARTSEAST: an edge-adaptive selection with topology first differentiable architecture selection method[ J] . Applied Intelligence, 2025, 55(7) : 526.
[16] He Hongyi, Liu Longjun, Zhang Haonan, et al. IS-DARTS: stabilizing DARTS through precise measurement oncandidate importance[ J] . Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38( 11) : 12367-12375.
[17] Xiao Han, Wang Ziwei, Zhu Zheng, et al. ShapleyNAS: discovering operation contribution for neural architecture search[ C]∥Proceedings of the 2022 IEEE / CVFConference on Computer Vision and Pattern Recognition(CVPR) . Piscataway: IEEE, 2022: 11882-11891.
[18] Yang Le, Zheng Ziwei, Han Yizeng, et al. OStr-DARTS: differentiable neural architecture search based on operation strength[ J] . IEEE Transactions on Cybernetics,2024, 54(11) : 6559-6572.
[19] Zhou Yuan, Xie Xukai, Kung S Y. Exploiting operationimportance for differentiable neural architecture search[ J] . IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(11) : 6235-6248.
[20] Sun Zihao, Hu Yu, Lu Shun, et al. Agnas: attentionguided micro and macro-architecture search[C]∥International Conference on Machine Learning. New York:ACM, 2022: 20777-20789.
[21] Xue Yu, Han Xiaolong, Wang Zehong. Self-adaptiveweight based on dual-attention for differentiable neural architecture search [ J ] . IEEE Transactions on IndustrialInformatics, 2024, 20(4) : 6394-6403.
[22] Chen Xin, Xie Lingxi, Wu Jun, et al. Progressive DARTS: bridging the optimization gap for NAS in the wild[ J ] . International Journal of Computer Vision, 2021,129(3) : 638-655.
[23] Hao Debei, Pei Songwei. MIG-DARTS: towards effectivedifferentiable architecture search by gradually mitigatingthe initial-channel gap between search and evaluation[ J] . Neural Computing and Applications, 2025, 37(8) :6085-6096.
[24] Zhou Qinqin, Zheng Xiawu, Cao Liujuan, et al. ECDARTS: inducing equalized and consistent optimizationinto DARTS[C]∥Proceedings of the 2021 IEEE / CVF International Conference on Computer Vision ( ICCV) . Piscataway: IEEE, 2021: 11966-11975.
[25] Yu Hongyuan, Peng Houwen, Huang Yan, et al. Cyclicdifferentiable architecture search[ J] . IEEE Transactionson Pattern Analysis and Machine Intelligence, 2023, 45(1) : 211-228.
[26] Ye Peng, Li Baopu, Li Yikang, et al. β-DARTS: betadecay regularization for differentiable architecture search[C]∥Proceedings of the 2022 IEEE / CVF Conference onComputer Vision and Pattern Recognition ( CVPR) . Piscataway: IEEE, 2022: 10864-10873.
[27] Zeng Xianhua, Wu Jie, Xia Yaoguang, et al. A differentiable neural architecture search algorithm with architecture norm regularization[ J] . Scientia Sinica ( Informationis) , 2024, 54( 9) : 2181- 2199. [ 曾宪华, 吴杰, 夏耀光, 等. 一种结构范数正则化的可微神经结构搜索算法[ J] . 中国科学: 信息科学, 2024, 54( 9) : 2181-2199. ]
[28] Chen Xiangning , Hsieh C J. Stabilizing differentiable architecture search via perturbation-based regularization[C]∥ Proceedings of the 37th International Conferenceon Machine Learning. New York: ACM, 2020: 1554 -1565 .
[29] Jing Kun, Chen Luoyu, Xu Jungang. An architecture entropy regularizer for differentiable neural architecturesearch[ J] . Neural Networks, 2023, 158: 111-120.
[30] Hu Liwei, Wang Zidong, Li Han, et al. ℓ - DARTS:light-weight differentiable architecture search with robustness enhancement strategy [ J ] . Knowledge-Based Systems, 2024, 288: 111466.
[31] Movahedi S, Adabinejad M, Imani A, et al. $ Łambda$ -DARTS: mitigating performance collapse by harmonizing operation selection among cells[ PP / OL] . V2. arXiv(2023 - 03 - 01 ) [ 2025 - 10 - 26 ] . https: ∥doi. org /10. 48550 / arXiv. 2210. 07998.
[32] Chu Xiangxiang, Zhang Bo. Noisy differentiable architecture search[ C]∥Proceedings of the British Machine Vi-sion Conference 2021. British Machine Vision Association, 2021: 434.
[33] Zela A, Elsken T, Saikia T, et al. Understanding and robustifying differentiable architecture search [ PP / OL ] .V2. arXiv (2020-01-28) [2025-08-11] . https:∥doi.org / 10. 48550 / arXiv. 1909. 09656.
[34] Huang Yihua, Ji Zipeng, Jiang Shen, et al. Operationlevel early stopping for robustifying differentiable NAS[C]∥Proceedings of the Advances in Neural InformationProcessing Systems 36. Neural Information ProcessingSystems Foundation, Inc. ( NeurIPS ) , 2023: 70983-71007.
[35] Chen Xiangning, Wang Ruochen, Cheng Minhao, et al.DrNAS: dirichlet neural architecture search [ PP / OL ] .V4. arXiv (2021-03-16) [2025-10-26] . https:∥doi.org / 10. 48550 / arXiv. 2006. 10355.
[36] Li Guohao, Qian Guocheng, Delgadillo I C, et al.SGAS: sequential greedy architecture search [ C] ∥Proceedings of the 2020 IEEE / CVF Conference on ComputerVision and Pattern Recognition ( CVPR ) . Piscataway:IEEE, 2020: 1617-1627.
[37] Cai Zicheng, Chen Lei, Liu Peng, et al. EG-NAS: neural architecture search with fast evolutionary exploration[ J] . Proceedings of the AAAI Conference on ArtificialIntelligence, 2024, 38(10) : 11159-11167.
[38] Heuillet A, Tabia H, Arioui H, et al. D-DARTS: distributed differentiable architecture search [ J ] . PatternRecognition Letters, 2023, 176: 42-48.
[39] Huang Lan, Sun Shiqi, Zeng Jia, et al. U-DARTS: uniform-space differentiable architecture search [ J] . Information Sciences, 2023, 628: 339-349.
[40] Gu Yuchao, Wang Lijuan, Liu Yun, et al. DOTS: decoupling operation and topology in differentiable architecture search [ C ] ∥Proceedings of the 2021 IEEE / CVFConference on Computer Vision and Pattern Recognition(CVPR) . Piscataway: IEEE, 2021: 12306-12315.
[41] Liu Qinglin, Lin Jingbo, Lv Xiaoqian, et al. ATL-DAS:automatic topology learning for differentiable architecturesearch[ J] . Displays, 2023, 80: 102541.
[42] Xu Yuhui, Xie Lingxi, Zhang Xiaopeng , et al. PCDARTS: partial channel connections for memory-efficientarchitecture search [ C ] ∥ International Conference onLearning Representations. Addis Ababa: ICLR,, 2019.
[43] Xue Yu, Qin Jiafeng. Partial connection based on channel attention for differentiable neural architecture search[ J] . IEEE Transactions on Industrial Informatics, 2023,19(5) : 6804-6813.
[44] Cai Zicheng, Chen Lei, Liu Hailin. EPC-DARTS: efficient partial channel connection for differentiable architecture search[ J] . Neural Networks, 2023, 166: 344-353.
[45] Pan Jie, Zheng Xuechi, Zou Xiaoyu. Differentiable neural architecture search with channel performance measurement[ J] . Control and Decision, 2024, 39 ( 7) : 2151 -2160. [潘杰, 郑学驰, 邹筱瑜. 基于通道性能度量的神经网络结构搜索算法 [ J] . 控制与决策, 2024, 39(7) : 2151-2160. ]
[46] Li Zhongnian, Xu Yixin, Ying Peng, et al. LMD-DARTS: low-memory, densely connected, differentiable architecture search[ J] . Electronics, 2024, 13(14) : 2743.
[47] Wang Wenna, Zhang Xiuwei, Cui Hengfei, et al. FPDARTS: fast parallel differentiable neural architecturesearch for image classification[ J] . Pattern Recognition,2023, 136: 109193.
[48] Li Shu, Mao Yuxu, Zhang Fuchang, et al. DLW-NAS:differentiable light-weight neural architecture search[ J] .Cognitive Computation, 2023, 15(2) : 429-439.
[49] Peng Cheng, Li Yangyang, Shang Ronghua, et al. ReCNAS: resource-constrained neural architecture searchbased on differentiable annealing and dynamic pruning[ J] . IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2) : 2805-2819.
[50] Yin Shantong, Niu Ben, Wang Rui, et al. Spatial andchannel level feature redundancy reduction for differentiable neural architecture search [ J ] . Neurocomputing,2025, 630: 129713.
[51] Biju G M, Pillai G N. Sequential node search for fasterneural architecture search [ J ] . Knowledge-Based Systems, 2024, 300: 112145.
[52] Lu Peng, Wang Hanzhang, Mao Xiaobo, et al. Pulsewave classification model based on convolutional autoencoder[ J]. Journal of Zhengzhou University ( EngineeringScience), 2021, 42(5): 56-61. [逯鹏, 王汉章, 毛晓波, 等. 基于 卷 积 自 编 码 器 网 络 的 脉 搏 波 分 类 模 型[J]. 郑州大学学报(工学版), 2021, 42(5): 56-61. ]

相似文献/References:

[1]郝旺身,陈耀,孙浩,等.基于全矢-CNN的轴承故障诊断研究[J].郑州大学学报(工学版),2020,41(5):92.[doi:10.13705/j.issn.1671-6833.2020.03.004]
　HAO Wangshen,CHEN Yao,SUN Hao,et al.Bearing Fault Diagnosis Based on Full Vector-CNN[J].Journal of Zhengzhou University (Engineering Science),2020,41(XX):92.[doi:10.13705/j.issn.1671-6833.2020.03.004]
[2]韩刚涛,马瑞鹏,吴 迪.基于时频图切割的宽带信号智能检测与识别[J].郑州大学学报(工学版),2023,44(3):44.[doi:10.13705/j.issn.1671-6833.2023.03.008]
　HAN Gangtao,MA uipeng,WU Di.Intelligent Detection and Identification of Broadband Signals Based on Time-frequency Map Cutting[J].Journal of Zhengzhou University (Engineering Science),2023,44(XX):44.[doi:10.13705/j.issn.1671-6833.2023.03.008]
[3]汪 烨,周思源,翁知远,等.一种面向用户反馈的智能分析与服务设计方法[J].郑州大学学报(工学版),2023,44(3):58.[doi:10.13705/j.issn.1671-6833.2022.06.004]
　Wang Ye,Zhou Siyuan,Weng Zhiyuan,et al.An Intelligent Analysis and Service Design Method for User Feedback[J].Journal of Zhengzhou University (Engineering Science),2023,44(XX):58.[doi:10.13705/j.issn.1671-6833.2022.06.004]
[4]张涛,葛育伟,韩旭,等.基于对抗机制的彩色图像隐写分析算法[J].郑州大学学报(工学版),2023,44(4):10.[doi:10.13705/j.issn.1671-6833.2023.04.013]
　ZHANG Tao,GE Yuwei,HAN Xu,et al.Color Image Steganalysis Algorithm Based on Adversarial Mechanisms[J].Journal of Zhengzhou University (Engineering Science),2023,44(XX):10.[doi:10.13705/j.issn.1671-6833.2023.04.013]
[5]魏明军,王镆涵,刘亚志,等.基于特征融合和混合注意力的小目标检测[J].郑州大学学报(工学版),2024,45(3):72.[doi:10. 13705/ j. issn. 1671-6833. 2024. 03. 001]
　WEI Mingjun,WANG Mohan,LIU Yazhi,et al.Small Object Detection Based on Feature Fusion and Mixed Attention[J].Journal of Zhengzhou University (Engineering Science),2024,45(XX):72.[doi:10. 13705/ j. issn. 1671-6833. 2024. 03. 001]
[6]陈晓鹏,徐　鹏,王占涛,等.基于单目视觉前馈的机械臂目标快速三维瞄准方法[J].郑州大学学报(工学版),2025,46(2):11.[doi:10.13705/j.issn.1671-6833.2025.02.001]
　CHEN Xiaopeng,XU Peng,WANG Zhantao,et al.Fast 3D Aiming Method of Manipulators Based on Monocular Visual Feedforward[J].Journal of Zhengzhou University (Engineering Science),2025,46(XX):11.[doi:10.13705/j.issn.1671-6833.2025.02.001]
[7]刘　凯,汪佳琴,李汉涛.基于深度学习的车辆轨迹预测研究综述[J].郑州大学学报(工学版),2025,46(5):77.[doi:10.13705/j.issn.1671-6833.2025.02.006]
　LIU Kai,WANG Jiaqin,LI Hantao.A Review of Vehicle Trajectory Prediction Based on Deep Learning[J].Journal of Zhengzhou University (Engineering Science),2025,46(XX):77.[doi:10.13705/j.issn.1671-6833.2025.02.006]
[8]张　蓓,徐　硕,钟燕辉,等.基于改进YOLOv8算法的半刚性基层松散病害识别方法[J].郑州大学学报(工学版),2025,46(5):122.[doi:10.13705/j.issn.1671-6833.2025.05.006]
　ZHANG Bei,XU Shuo,ZHONG Yanhui,et al.Detection Method of Loose Defects in Semi-rigid Base Based on Improved YOLOv8 Algorithm[J].Journal of Zhengzhou University (Engineering Science),2025,46(XX):122.[doi:10.13705/j.issn.1671-6833.2025.05.006]
[9]陈　燕,韦紫君,廖宇翔,等.基于RoBERTa和指针网络的中文实体与关系联合抽取方法[J].郑州大学学报(工学版),2026,47(2):41.[doi:10.13705/j.issn.1671-6833.2025.05.007]
　CHEN Yan,WEI Zijun,LIAO Yuxiang,et al.Joint Extraction Method of Chinese Entities and Relations Based on RoBERTa and Pointer Network[J].Journal of Zhengzhou University (Engineering Science),2026,47(XX):41.[doi:10.13705/j.issn.1671-6833.2025.05.007]
[10]韩　涛,丁乐言,马　明,等.基于参数优化和注意力机制的电动汽车充电负荷预测[J].郑州大学学报(工学版),2026,47(2):85.[doi:10.13705/j.issn.1671-6833.2026.02.001]
　HAN Tao,DING Leyan,MA Ming,et al.Charging Load Prediction for Electric Vehicle Based on Parameter Optimization and Attention Mechanism[J].Journal of Zhengzhou University (Engineering Science),2026,47(XX):85.[doi:10.13705/j.issn.1671-6833.2026.02.001]

备注/Memo

备注/Memo:
收稿日期：2025-03-03基金项目：作者简介：(1974—)，男，河南郑州人，大学教授，博士，主要从事 。 式中：为操作混合权重；代表操作类型；为第i个节点到第j个节点之间操作的权重，代表之后需要搜索的架构参数。将搜索空间连续化后，DARTS将架构搜索转化为优化架构参数与网络权重的双层优化问题：在训练集上优化权重，在验证集上优化架构参数，以提升模型泛化能力。该过程可形式化为一个双层优化问题，用以下公式表示
更新日期/Last Update: 2026-05-26