文献类型：期刊文献

中文题名：基于强化学习和3σ准则的组合剪枝方法

英文题名：Combination pruning method based on reinforcement learning and 3σcriterion

作者：徐少铭[1];李钰[1];袁晴龙[1]

机构：[1]华东理工大学信息科学与工程学院,上海200237

年份：2023

卷号：57

期号：3

起止页码：486

中文期刊名：浙江大学学报（工学版）

外文期刊名：Journal of Zhejiang University：Engineering Science

收录：CSTPCD;;EI(收录号：20232114135442);Scopus;北大核心:【北大核心2020】；CSCD:【CSCD2023_2024】；

语种：中文

中文关键词：深度神经网络;模型压缩;稀疏率;强化学习;组合剪枝

外文关键词：deep neural network;model compression;sparsity rate;reinforcement learning;combination pruning

摘要：针对结构复杂、参数冗余的深度神经网络无法部署到资源受限的嵌入式系统的问题,受稀疏率对性能影响的启示,提出基于强化学习和3σ准则的组合剪枝方法.根据稀疏率对准确率的影响,确定最佳全局稀疏率,使稀疏率和精度达到较好平衡.在最佳全局稀疏率的指导下,利用强化学习方法自动搜索每层卷积层的最佳剪枝率,根据剪枝率剪去不重要的权重.通过3σ准则确定全连接层每层的权重剪枝阈值,对全连接层进行权重剪枝.通过再训练来恢复模型识别的精度.实验结果表明,所提剪枝方法可以将网络VGG16、ResNet56和ResNet50的参数,分别压缩83.33%、70.1%和80.9%,模型的识别准确率分别降低1.55%、1.98%和1.86%.
In order to resolve the problem that deep neural network with complex structure and redundant parameters could not be deployed to the resource constrained embedded system,an efficient combination pruning method based on reinforcement learning and 3σcriterion was proposed,which was inspired by the effect of sparsity rate on performance.Firstly,an optimal global sparsity rate was determined according to the influence of sparsity rate on accuracy,which could achieve a good balance between sparsity rate and accuracy.Secondly,under the guidance of optimal global sparsity rate,the reinforcement learning method was used to search the optimal pruning rate of each convolutional layer automatically,and the unimportant weights were cut off on the basis of the pruning rate.Then,the weight pruning threshold of each fully connected layer was determined by 3σcriterion,and for each fully connected layer,the weight which below the threshold would be pruned.Finally,the accuracy of model recognition was restored by retraining.Experimental results showed that the proposed pruning method could compress the parameters of VGG16,ResNet56 and ResNet50 network by 83.33%,70.1%and 80.9%respectively,and the model’s recognition accuracy could be reduced by 1.55%,1.98%and 1.86%respectively.