文献类型：期刊文献

英文题名：Autorep: Automatic network search with structured reparameterized based linear operation expansion and gradient proxy guided reduction

作者：Qiu, Guhao[1];Chen, Ruoxin[1];Chen, Zhihua[1];Dai, Lei[1];Li, Ping[2,3];Sheng, Bin[4]

机构：[1]East China Univ Sci & Technol, Dept Comp Sci & Engn, Shanghai 200237, Peoples R China;[2]Hong Kong Polytech Univ, Dept Comp, Hong Kong, Peoples R China;[3]Hong Kong Polytech Univ, Sch Design, Hong Kong, Peoples R China;[4]Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China

年份：2026

卷号：199

外文期刊名：NEURAL NETWORKS

收录：;EI(收录号：20261420413106);WOS:【SCI-EXPANDED(收录号:WOS:001692686900001)】；

基金：This work was supported by the National Natural Science Foundation of China, (Grant Nos. 62272164, 62572188, and 62306113) .

语种：英文

外文关键词：Neural architecture search; Structured reparameterization; Lightweight neural network

摘要：Convolution neural network and Vision Transformer have achieved large success in various computer vision tasks. However, the huge computation cost hinders its application and it is hard to design efficient methods to obtain lightweight architectures with both mannually designed strategies and automatically searching methods. In this paper, we focus on introducing the specific structural reparameterization strategy in SuperNet training to improve the performance of one-shot based neural architecture search algorithm. During the SuperNet training process, each candidate operation is expanded by a series of equivalent operation branches to fully utilize the representation potential. To alleviate the training difficulty and avoid bringing too much computation costs, the operation reduction strategy and prior sampling strategy are used after validating the sampled subnetworks. The operation reduction strategy is to remove the low-effect extended linear layer. The reduction step needs to firstly select the candidate operation based on SynFlow proxy and then select the extended linear layer from the selected operation based on the accuracy difference before and after removal.