Overview of Differentiable Neural Network Architecture Search
[1]LU Peng,LI Keyan,ZHANG Hongpo,et al.Overview of Differentiable Neural Network Architecture Search[J].Journal of Zhengzhou University (Engineering Science),2026,47(5):58-67.[doi:10.13705/j.issn.1671-6833.2026.02.007]
Copy
Journal of Zhengzhou University (Engineering Science)[ISSN
1671-6833/CN
41-1339/T] Volume:
47
Number of periods:
2026 Issue 5
Page number:
58-67
Column:
Public date:
2026-09-09
- Title:
- Overview of Differentiable Neural Network Architecture Search
- Keywords:
- neural architecture search; deep learning; differentiable neural architecture search; continuous optimi; zation; performance estimation
- CLC:
- TP181
- Abstract:
- Neural Architecture Search (NAS) was an interdisciplinary study in the field of deep learning, which aimed to automate the design of neural network structures. NAS required repeated training and evaluation of a large number of candidate networks, which was computationally expensive. Differentiable Neural Network Architecture Search (DNAS) transformed the discrete architecture search problem into a differentiable continuous optimization problem, which reduced the computational cost. Firstly, a search algorithm framework for differentiable network architecture was constructed from three aspects: search space, search strategy and performance evaluation strategy. Secondly, the performance estimation bias, architecture overfitting and search stability problems of parameterization operation, as well as the improvement strategies of optimizing search space and improving efficiency were analyzed, compared and summarized. Then, the error rate, parameter quantity, search time and experimental hardware conditions of typical DNAS algorithms on image classification datasets were compared and analyzed. Finally, it pointed out the application potential of DNAS in complex scenarios such as edge device deployment, medical signal analysis, and cross‑modal matching, and proposed future research directions toward multi‑objective optimization, task‑driven search space design, and cross‑task transfer and reuse.
- References:
-
[1] Dos Santos C F G, Papa J P. Avoiding overfitting: a survey on regularization methods for convolutional neural networks[J]. ACM Computing Surveys, 2022, 54(10s):1‑25.
[2] Hanin B. Which neural net architectures give rise to exploding and vanishing gradients?[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM, 2018: 580‑589.
[3] Sun Renke, Huangfu Zhiyu, Chen Hu, et al. Survey of neural architecture search[J]. Journal of Computer Applications, 2024, 44(10):2983‑2994.［孙仁科,皇甫志宇,陈虎,等.神经网络架构搜索综述[J].计算机应用,2024,44(10):2983‑2994.］
[4] Liu Hanxiao, Simonyan K, Yang Yiming. DARTS: differentiable architecture search[PP/OL]. V2. arXiv (2019‑04‑23)[2025‑12‑09]. https://doi.org/10.48550/arXiv.1806.09055.
[5] Heuillet A, Nasser A, Arioui H, et al. Efficient automation of neural network design: a survey on differentiable neural architecture search[J]. ACM Computing Surveys, 2024, 56(11):1‑36.
[6] Zoph B, Vasudevan V, Shlens J, et al. Learning transferable architectures for scalable image recognition[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8697‑8710.
[7] He Chaoyang, Ye Haishan, Shen Li, et al. MiLeNAS: efficient neural architecture search via mixed‑level reformulation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 11990‑11999.
[8] Chu Xiangxiang, Zhou Tianbao, Zhang Bo, et al. Fair DARTS: eliminating unfair advantages in differentiable architecture search[C]//Computer vision‑ECCV 2020. Cham: Springer International Publishing, 2020: 465‑480.
[9] Shin J, Park K, Kang D K. TA‑DARTS: temperature annealing of discrete operator distribution for effective differentiable architecture search[J]. Applied Sciences, 2023, 13(18):10138.
[10] Mun J, Ha S, Lee J. DE‑DARTS: neural architecture search with dynamic exploration[J]. ICT Express, 2023, 9(3):379‑384.
[11] Ma Benteng, Zhang Yanning, Xia Yong. Momentum recursive DARTS[J]. Pattern Recognition, 2024, 156:110710.
[12] Liu Wenbo, Deng Tao, An Rui, et al. DARTS‑CGW: research on differentiable neural architecture search algorithm based on coarse gradient weighting[C]//Pattern Recognition and Computer Vision. Singapore: Springer Nature Singapore, 2025: 31‑44.
[13] Wang Ruochen, Cheng Minhao, Chen Xiangning, et al. Rethinking architecture selection in differentiable NAS[PP/OL]. V1. arXiv (2021‑08‑10)[2025‑10‑09]. https://doi.org/10.48550/arXiv.2108.04392.
[14] Xie Weisheng, Li Hui, Fang Xuwei, et al. DARTS‑PT‑CORE: collaborative and regularized perturbation‑based architecture selection for differentiable NAS[J]. Neurocomputing, 2024, 580:127522.
[15] Fang Xuwei, Xie Weisheng, Li Hui, et al. DARTS‑EAST: an edge‑adaptive selection with topology first differentiable architecture selection method[J]. Applied Intelligence, 2025, 55(7):526.
[16] He Hongyi, Liu Longjun, Zhang Haonan, et al. IS‑DARTS: stabilizing DARTS through precise measurement on candidate importance[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(11):12367‑12375.
[17] Xiao Han, Wang Ziwei, Zhu Zheng, et al. Shapley‑NAS: discovering operation contribution for neural architecture search[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 11882‑11891.
[18] Yang Le, Zheng Ziwei, Han Yizeng, et al. OStr‑DARTS: differentiable neural architecture search based on operation strength[J]. IEEE Transactions on Cybernetics, 2024, 54(11):6559‑6572.
[19] Zhou Yuan, Xie Xukai, Kung S Y. Exploiting operation importance for differentiable neural architecture search[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(11):6235‑6248.
[20] Sun Zihao, Hu Yu, Lu Shun, et al. AGNAS: attention‑guided micro and macro‑architecture search[C]//International Conference on Machine Learning. New York: ACM, 2022: 20777‑20789.
[21] Xue Yu, Han Xiaolong, Wang Zehong. Self‑adaptive weight based on dual‑attention for differentiable neural architecture search[J]. IEEE Transactions on Industrial Informatics, 2024, 20(4):6394‑6403.
[22] Chen Xin, Xie Lingxi, Wu Jun, et al. Progressive DARTS: bridging the optimization gap for NAS in the wild[J]. International Journal of Computer Vision, 2021, 129(3):638‑655.
[23] Hao Debei, Pei Songwei. MIG‑DARTS: towards effective differentiable architecture search by gradually mitigating the initial‑channel gap between search and evaluation[J]. Neural Computing and Applications, 2025, 37(8):6085‑6096.
[24] Zhou Qinqin, Zheng Xiawu, Cao Liujuan, et al. EC‑DARTS: inducing equalized and consistent optimization into DARTS[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 11966‑11975.
[25] Yu Hongyuan, Peng Houwen, Huang Yan, et al. Cyclic differentiable architecture search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1):211‑228.
[26] Ye Peng, Li Baopu, Li Yikang, et al. β‑DARTS: beta‑decay regularization for differentiable architecture search[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 10864‑10873.
[27] Zeng Xianhua, Wu Jie, Xia Yaoguang, et al. A differentiable neural architecture search algorithm with architecture norm regularization[J]. Scientia Sinica Informationis, 2024, 54(9):2181‑2199.［曾宪华,吴杰,夏耀光,等.一种结构范数正则化的可微神经结构搜索算法[J].中国科学:信息科学,2024,54(9):2181‑2199.］
[28] Chen Xiangning, Hsieh C J. Stabilizing differentiable architecture search via perturbation‑based regularization[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM, 2020:1554‑1565.
[29] Jing Kun, Chen Luoyu, Xu Jungang. An architecture entropy regularizer for differentiable neural architecture search[J]. Neural Networks, 2023, 158:111‑120.
[30] Hu Liwei, Wang Zidong, Li Han, et al. ℓ‑DARTS: light‑weight differentiable architecture search with robustness enhancement strategy[J]. Knowledge‑Based Systems, 2024, 288:111466.
[31] Movahedi S, Adabinejad M, Imani A, et al. DARTS: mitigating performance collapse by harmonizing operation selection among cells[PP/OL]. V2. arXiv (2023‑03‑01)[2025‑12‑09]. https://doi.org/10.48550/arXiv.2210.07998.
[32] Chu Xiangxiang, Zhang Bo. Noisy differentiable architecture search[C]//Proceedings of the British Machine Vision Conference 2021. Durham: British Machine Vision Association, 2021:434.
[33] Zela A, Elsken T, Saikia T, et al. Understanding and robustifying differentiable architecture search[PP/OL]. V2. arXiv (2020‑01‑28)[2025‑12‑09]. https://doi.org/10.48550/arXiv.1909.09656.
[34] Jiang Shen, Li Zifeng, Zhu Guanghui, et al. Operation‑level early stopping for robustifying differentiable NAS[C]//37th Conference on Neural Information Processing Systems. Vancouver: NeurIPS, 2023:70983‑71007.
[35] Chen Xiangning, Wang Ruochen, Cheng Minhao, et al. DrNAS: dirichlet neural architecture search[PP/OL]. V4. arXiv (2021‑03‑16)[2025‑12‑09]. https://doi.org/10.48550/arXiv.2006.10355.
[36] Li Guohao, Qian Guocheng, Delgadillo I C, et al. SGAS: sequential greedy architecture search[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020:1617‑1627.
[37] Cai Zicheng, Chen Lei, Liu Peng, et al. EG‑NAS: neural architecture search with fast evolutionary exploration[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(10):11159‑11167.
[38] Heuillet A, Tabia H, Arioui H, et al. D‑DARTS: distributed differentiable architecture search[J]. Pattern Recognition Letters, 2023, 176:42‑48.
[39] Huang Lan, Sun Shiqi, Zeng Jia, et al. U‑DARTS: uniform‑space differentiable architecture search[J]. Information Sciences, 2023, 628:339‑349.
[40] Gu Yuchao, Wang Lijuan, Liu Yun, et al. DOTS: decoupling operation and topology in differentiable architecture search[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021:12306‑12315.
[41] Liu Qinglin, Lin Jingbo, Lv Xiaoqian, et al. ATL‑DAS: automatic topology learning for differentiable architecture search[J]. Displays, 2023, 80:102541.
[42] Xu Yuhui, Xie Lingxi, Zhang Xiaopeng, et al. PC‑DARTS: partial channel connections for memory‑efficient architecture search[C]//International Conference on Learning Representations. Addis Ababa: ICLR, 2019.
[43] Xue Yu, Qin Jiafeng. Partial connection based on channel attention for differentiable neural architecture search[J]. IEEE Transactions on Industrial Informatics, 2023, 19(5):6804‑6813.
[44] Cai Zicheng, Chen Lei, Liu Hailin. EPC‑DARTS: efficient partial channel connection for differentiable architecture search[J]. Neural Networks, 2023, 166:344‑353.
[45] Pan Jie, Zheng Xuechi, Zou Xiaoyu. Differentiable neural architecture search with channel performance measurement[J]. Control and Decision, 2024, 39(7):2151‑2160.［潘杰,郑学驰,邹筱瑜.基于通道性能度量的神经网络结构搜索算法[J].控制与决策,2024,39(7):2151‑2160.］
[46] Li Zhongnian, Yu Xixin, Ying Peng, et al. LMD‑DARTS: low‑memory, densely connected, differentiable architecture search[J]. Electronics, 2024, 13(14):2743.
[47] Wang Wenna, Zhang Xiuwei, Cui Hengfei, et al. FP‑DARTS: fast parallel differentiable neural architecture search for image classification[J]. Pattern Recognition, 2023, 136:109193.
[48] Li Shu, Mao Yuxu, Zhang Fuchang, et al. DLW‑NAS: differentiable light‑weight neural architecture search[J]. Cognitive Computation, 2023, 15(2):429‑439.
[49] Peng Cheng, Li Yangyang, Shang Ronghua, et al. ReC‑NAS: resource‑constrained neural architecture search based on differentiable annealing and dynamic pruning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2):2805‑2819.
[50] Yin Shantong, Niu Ben, Wang Rui, et al. Spatial and channel level feature redundancy reduction for differentiable neural architecture search[J]. Neurocomputing, 2025, 630:129713.
[51] Biju G M, Pillai G N. Sequential node search for faster neural architecture search[J]. Knowledge‑Based Systems, 2024, 300:112145.
- Similar References:
Memo
-
Last Update:
2026-09-07
