文献类型：期刊文献

中文题名：基于剪枝与量化的卷积神经网络压缩方法

英文题名：Convolutional Neural Networks Compression Based on Pruning and Quantization

作者：孙彦丽[1];叶炯耀[1]

机构：[1]华东理工大学信息科学与工程学院,上海200237

年份：2020

卷号：47

期号：8

起止页码：261

中文期刊名：计算机科学

外文期刊名：Computer Science

收录：CSTPCD;;北大核心:【北大核心2017】；CSCD:【CSCD_E2019_2020】；

语种：中文

中文关键词：卷积神经网络;权重剪枝;量化感知;参数量化;网络压缩

外文关键词：Convolutional neural networks;Weight pruning;Quantization-aware-training;Parameters quantization;Network compression

摘要：随着深度学习的发展,卷积神经网络作为其重要算法之一,被广泛应用到计算机视觉、自然语言处理及语音处理等各个领域,并取得了比传统算法更为优秀的成绩。但是,卷积神经网络结构复杂,参数量和计算量巨大,使得很多算法必须在GPU上实现,导致卷积神经网络难以应用在资源不足且实时性要求很高的移动端。为了解决上述问题,文中提出通过同时优化卷积神经网络的结构和参数来对卷积神经网络进行压缩,以使网络模型尺寸变小。首先,根据权重对网络模型结果的影响程度来对权重进行剪枝,保证在去除网络结构冗余信息的同时保留模型的重要连接;然后通过量化感知(quantization-aware-trai-ning)对卷积神经网络的浮点型权重和激活值进行完全量化,将浮点运算转换成定点运算,在降低网络模型计算量的同时减少网络模型的尺寸。文中选用tensorflow深度学习框架,在Ubuntu16.04操作系统中使用Spyder编译器对所提算法进行验证。实验结果表明,该算法使结构简单的LeNet模型从1.64M压缩至0.36M,压缩比达到78%,准确率只下降了了0.016;使轻量级网络Mobilenet模型从16.9M压缩至3.1M,压缩比达到81%,准确率下降0.03。实验数据说明,在对卷积神经网络权重剪枝与参数量化之后,该算法可以做到在准确率损失较小的情况下,对模型进行有效压缩,解决了卷积神经网络模型难以部署到移动端的问题。
With the development of deep learning,Convolutional Neural Networks(CNN),as one of its important algorithms,is widely applied in a variety of fields such as target detection,natural language processing,speech recognition and image identification,and achieves better results than the traditional algorithm.The number of parameters and calculation are increasing with the depth of network structure,resulting in many algorithms must be implemented on GPU.So it is difficult to apply the CNN model to mobile terminals which has limited resources and high real-time request.To solve this problem,this paper presents a method of optimizing network structure and parameters simultaneously.Firstly,the algorithms prunes weight according to its influence on the results of network,and ensure that the redundant information is removed while retaining the important connection of the mo-del.Then,this paper quantizes the float-point weight and activation of CNN.This changes float-point operation to fixed-point ope-ration.It not only reduces the computational complexity of the network model,but also reduces the size of the network model.To verify the algorithm,deep learning framework tensorflow is selected,and Spyder compiler is used in Ubuntu 16.04 operating system.The experimental results show that this method reduces size of LeNet model with simple structure from 1.64M to 0.36M.Compression ratio reaches 78%,but the accuracy drops only 0.016.And it also reduces the size of MobileNet with Lightweight network from 16.9M to 3.1M.Compression ratio reaches 81%with the accuracy dropping only 0.03.The data show that combining the weight pruning and parameter quantification of convolution neural network can effectively compress the convolution neural network within the acceptable range of accuracy loss.So,this method solve the difficulty of deploying the convolution neural network to the mobile terminal.