CN113065640B - Image classification network compression method based on convolution kernel shape automatic learning - Google Patents

Image classification network compression method based on convolution kernel shape automatic learning Download PDF

Info

Publication number
CN113065640B
CN113065640B CN202110283921.3A CN202110283921A CN113065640B CN 113065640 B CN113065640 B CN 113065640B CN 202110283921 A CN202110283921 A CN 202110283921A CN 113065640 B CN113065640 B CN 113065640B
Authority
CN
China
Prior art keywords
convolution kernel
network
image classification
convolution
term
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110283921.3A
Other languages
Chinese (zh)
Other versions
CN113065640A (en
Inventor
张科
刘广哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN202110283921.3A priority Critical patent/CN113065640B/en
Publication of CN113065640A publication Critical patent/CN113065640A/en
Application granted granted Critical
Publication of CN113065640B publication Critical patent/CN113065640B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to an image classification network compression method based on convolution kernel shape automatic learning, and belongs to the technical field of image processing and recognition. By applying a plurality of sparse regularization constraints to the parameters of each position in the conventional convolution kernel, the parameters in the convolution kernel are thinned in the network training process, and the automatically learned convolution kernel shape can be obtained by setting a clipping threshold according to the compression rate, so that the redundant parameters in the convolution kernel can be effectively eliminated. The method is applied to the image classification task, the compression rate of the network model can be further improved while the classification accuracy is ensured, the parameter quantity and the calculation amount of the model are reduced, and the method is convenient to deploy and apply in the mobile equipment with limited resources.

Description

Image classification network compression method based on convolution kernel shape automatic learning
Technical Field
The invention belongs to the technical field of image processing and recognition, and particularly relates to an image classification network compression method based on convolution kernel shape automatic learning.
Background
Image classification and recognition are an important subject in the field of machine vision, and early image recognition methods mainly rely on manually extracted features, which have low accuracy and limited applicability to different scenes. With the appearance of a deep learning method, a convolutional neural network achieves great achievement in the machine vision fields of image recognition, target detection and the like, and the deep neural network can effectively extract advanced semantic features in images and can reach recognition capability exceeding that of human beings.
But, while network performance is improved, network structure is more and more complex, and requirements on storage capacity and operation capacity of the computing device are higher and higher, so that application and development of the computing device in mobile devices with limited resources are limited. The large-scale neural network model has larger redundancy, not all parameters play an effective role in network performance, and excessive parameters can cause problems of slow network convergence, over-fitting of parameters and the like. In order to facilitate the deployment and application of the neural network, the neural network compression method is getting more and more attention.
Parameter pruning is an effective neural network compression method, and the effect of reducing the complexity of a model is achieved by cutting redundant or unimportant parameters in a network. Wei Yue, chen Shichao, zhu Fenghua (model pruning method based on sparse convolutional neural network; computer engineering. DOI. Https:// doi. Org/10.19678/j. Issn. 1000-3428.0059375) provides a model pruning algorithm based on sparse convolutional neural network, sparse regularization constraint is applied to convolutional layers and Batch Normalization (BN) layers in the training process, so that the network weight becomes sparse, a pruning threshold is set, filter channels with lower importance in the network are pruned, and the accuracy of a training recovery model is fine-tuned, so that the purpose of convolutional neural network compression is achieved. This method belongs to the structured pruning method and prunes with the convolution channel as the smallest unit, but cannot remove redundant parameters inside the convolution kernel. Smaller pruning units are required if higher compression rates are to be achieved.
Disclosure of Invention
Technical problem to be solved
The existing sparse pruning method of the convolutional neural network performs sparse training on the whole convolutional channel, and cannot eliminate redundant parameters in the convolutional kernel, so that the network model compression rate is low, and the final image classification accuracy is affected. The invention provides an image classification network compression method based on convolution kernel shape automatic learning.
Technical proposal
An image classification network compression method based on convolution kernel shape automatic learning is characterized by comprising the following steps:
step 1: building a convolutional neural network for image classification;
step 2: introducing coefficient matrix F into the traditional convolution process, and adding weight sparse regularization term into the loss functionDistribution equilibrium regular term->Inter-group equalization regularization term/>
Wherein lambda is 1 、λ 2 、λ 3 Is a coefficient for balancing the terms;
solving the three loss functions in the network training process for each term f in the coefficient matrix ij For back-propagating the update coefficient matrix F; obtaining a sparse coefficient matrix F after training is completed;
step 3: setting a threshold according to the model compression rate which is expected to be reached, and f which is lower than the threshold ij Removing convolution kernel parameters of the corresponding positions to obtain the convolution kernel shape of each convolution layer;
step 4: and replacing the original conventional convolution kernel with the sparse shape obtained by automatic learning, and carrying out network training again to obtain a final image classification neural network model.
Preferably: the convolutional neural network in step 1 is VGG.
Preferably: the convolutional neural network in step 1 is ResNet.
Advantageous effects
According to the image classification network compression method based on the automatic learning of the convolution kernel shape, provided by the invention, the parameters of each position in the conventional convolution kernel are subjected to multiple sparse regularization constraints, the parameters in the convolution kernel are thinned in the network training process, and the automatic learning of the convolution kernel shape can be obtained by setting a clipping threshold according to the compression rate, so that the redundant parameters in the convolution kernel can be effectively eliminated. The method is applied to the image classification task, the compression rate of the network model can be further improved while the classification accuracy is ensured, the parameter quantity and the calculation amount of the model are reduced, and the method is convenient to deploy and apply in the mobile equipment with limited resources.
The image classification network compression method based on the convolution kernel shape automatic learning, which is designed by the invention, can automatically learn the convolution kernel shape of each convolution layer in the network training process, so that the feeling of the convolution kernel is also adapted to the network depth, and simultaneously, the redundant parameters in the convolution kernel are eliminated, thereby achieving a good network compression effect.
The convolution kernel shape automatic learning method provides a new thought for the design of the high-efficiency network structure, and the convolution kernel shape is added into the search space of the neural structure search (NAS), so that a larger search space can be obtained, and further, the richer target characteristics are extracted, and the network performance is improved.
The image classification network compression method based on convolution kernel shape automatic learning, which is designed by the invention, can effectively compress the parameter quantity in the neural network convolution layer, for example, the parameter quantity of 59.07% and the calculated quantity of 51.91% of the VGG-16 network can be reduced on the premise of not reducing the accuracy, and the image classification network can be conveniently deployed in the terminal mobile equipment.
The image classification network compression method based on convolution kernel shape automatic learning can reduce redundant parameters in a convolution layer, so that the overfitting risk of the image classification network is reduced, the classification accuracy of the network is improved, and the classification accuracy of 0.72% can be improved when the VGG-16 network is compressed.
Drawings
Fig. 1 is a diagram of a convolution calculation process incorporating a matrix of convolution kernel coefficients.
Fig. 2 is a 3×3 convolution kernel parameter number and packet division diagram.
Fig. 3 is a convolution kernel shape auto-learning flow chart.
FIG. 4 is a convolution kernel shape of each convolution layer automatically learned over a CIFAR-10 dataset using a VGG-16 network.
Detailed Description
The invention will now be further described with reference to examples, figures:
the invention provides an image classification network compression method based on convolution kernel shape automatic learning, wherein the convolution kernel shape automatic learning flow is shown in figure 3. The following describes embodiments of the present invention in connection with image classification examples, but the technical content of the present invention is not limited to the described scope, and the embodiments include the following steps:
(1) A convolutional neural network for image classification is built, and an image dataset with a large number of training samples and labels is constructed.
(2) For a convolution layer in a neural network, the convolution calculation process of a conventional convolution kernel is as follows:
Y＝X*w
in the method, in the process of the invention,is the input feature map tensor, < >>Is the output feature map tensor, < >>Is a convolution weight parameter, c and n are the number of input and output channels, h and w are the height and width of the input feature map, h 'and w' are the height and width of the output feature map, k×k is the size of the convolution kernel, and is an image convolution operation.
The convolution kernels of the n output channels are equally divided into d groups, and each group contains n/d convolution channels. To sparse the parameters inside the convolution kernel, a coefficient matrix is introducedThe convolution weights w of each group are multiplied point by point, and then the convolution operation is carried out on the input X, namely:
Y＝X*(F⊙w)
in the formula, the term ". As used herein, is a point-wise multiplication operation. The entire calculation process is referred to in fig. 1.
The loss function in the conventional convolutional neural network training process is as follows:
in the method, in the process of the invention,is a class loss term, related to the input image and predictive label during the network training process, +.>Is a regularization term for weight decay that can reduce network overfitting.
Introducing coefficient matrix F into the traditional convolution process, and adding weight sparse regularization term into the loss functionDistribution equilibrium regular term->Inter-group equalization canonical term->Solving the three loss functions in the network training process for each term f in the coefficient matrix ij For back-propagating the update coefficient matrix F. And obtaining a sparse coefficient matrix F after training is completed.
In order to achieve the purpose of automatically learning the shape of the convolution kernel, sparse regularization constraint is applied to the coefficient matrix F, and the loss function is as follows:
in the method, in the process of the invention,is a regularization term that tends to sparse the convolution kernel weights, +.>Is a regularization term that equalizes the convolution kernel parameter distribution,/->Is a regularized term that equalizes parameters between groups, lambda 1 、λ 2 、λ 3 Is a coefficient for balancing the terms.
Each regular term is constructed separately below. Taking a 3×3 convolution kernel as an example, 9 parameters in the convolution kernel are numbered and divided into angles (G corner ) Edge (G) edge ) Heart (G) center ) Three groups, with specific reference to fig. 2, each group is numbered separately:
1)the method is a regularization term for enabling weights to tend to be sparse, and comprises the following steps:
wherein k is j For coefficients of different positions, which constitute a vector k e R 1×9 For applying different canonical constraints to parameters at different positions of diagonal, edge, heart, e.g. taking k= [4,2,4,2,1,2,4,2,4 ]]Represents the pair G corner Position application 4 times G center For G edge Applying 2 times G center And thus more focus on preserving parameters close to the center of the convolution kernel.
g (·) is the regularized norm, e.g. using L 1 Norms, then:
during the training process, due toIndependent of the training sample, it can be obtained in advanceWith respect to each coefficient f ij For back-propagating the update coefficient matrix F. The partial derivatives are:
where sgn (·) is a sign function.
2)The method is a regular term for balancing the distribution of the convolution kernel parameters, and is used for considering the convolution parameters in all directions, so as to avoid the situation of characteristic diagram deviation. For all d groups of coefficients f at the same position j ij Summing the absolute values to obtain:
for G corner And G edge F of each of (F) j And (3) carrying out difference between every two pairs, and obtaining the square sum:
according to the rule of the chain-type derivation,with respect to each coefficient f ij The partial derivatives of (2) are:
in the method, in the process of the invention,
3)the method is a regular term for equalizing parameters among groups, and the excessive difference of the number of d parameters among groups is avoided. Calculation of G in d groups respectively corner 、G edge 、G center F at each position ij Sum of absolute values, yield:
wherein F is i corner Representing the position G in the ith group of convolution kernels corner Coefficient f of position ij Sum of absolute values, F i edge Representing the position G in the ith group of convolution kernels edge Coefficient f of position ij Sum of absolute values, F i center Representing the position G in the ith group of convolution kernels center Coefficient f of position ij Sum of absolute values.
For each F in d groups i corner 、F i edge 、F i center And (3) carrying out difference between every two pairs, and obtaining the square sum:
in the method, in the process of the invention,represents G corner Position-generated inter-group equalization loss, +.>Represents G edge Position-generated inter-group equalization loss, +.>Represents G center Position-generated groupInter-equalization loss. The total inter-group equalization loss is:
according to the rule of the chain-type derivation,with respect to each coefficient f ij The partial derivatives of (2) are:
in the method, in the process of the invention,
(3) Performing sparse training on the coefficient matrix F in (1) by using the loss function in (2), obtaining sparse F after training is completed, setting a clipping threshold value, and setting F lower than the threshold value ij And removing the convolution kernel parameters of the corresponding positions to obtain the automatically learned convolution kernel shape.
(4) And replacing the original conventional convolution kernel with the sparse shape obtained by automatic learning, and carrying out network training again to obtain a final neural network model. The network parameters and the calculated amount of the model are lower than those of the original model, and the effect of network compression can be achieved while the correct classification result is ensured.
Based on the method for automatically learning the shape of the convolution kernel, the shape of the convolution kernel adapting to each convolution layer can be obtained, and redundant parameters in the convolution kernel can be effectively removed, so that the purpose of model compression is achieved.
FIG. 4 is various convolutional layers of a VGG-16 network automatically learned over a CIFAR-10 datasetD=2 is selected in the learning process, namely the convolution kernels of n output channels of each convolution layer are divided into 2 groups averagely, and 60% of parameters in the network are removed during clipping. The leftmost column (scheme one) is added with sparse regularization term onlyAnd the positions of the angle, the edge and the center are the same constraint coefficient k j The results obtained. The second column (scheme II) of left number is to take different constraint coefficients k from the positions of diagonal, side and center based on the first column j The result is that more parameters near the center of the convolution kernel remain than in the first column. The third column (scheme III) of left number is to add distribution equilibrium regular term based on the second column>As a result, the convolution kernel obtained gives consideration to the convolution parameters in all directions, in particular, layers 1 and 10, compared with the second column, and thus the problem that the extracted feature map is shifted in a certain direction can be avoided. The fourth column (scheme four) of the left number is to add the inter-group equalization regularization term on the basis of the third column>The results obtained balance the number of parameters of the two sets of convolution kernels better than the third column, especially layer 5.
Table 1 shows the compression results of the present invention, the original VGG-16 model contains 15.0M parameters and 314M calculation, and the accuracy on CIFAR-10 is 93.45%; model parameters obtained by adopting a traditional structured pruning method are 5.4M, calculated amount is 206M, and accuracy rate is reduced to 93.40%; by adopting the convolution kernel shape automatic learning method, various regular constraints are added to improve the accuracy of the model while compressing the network, so that various constraints can play a beneficial role on the compression result, the parameter of the finally obtained model (scheme IV) is 6.14M, the calculated amount is 151M, and the accuracy is improved by 94.17%.
Table 1 the network compression results table of the present invention

Claims (3)

1. An image classification network compression method based on convolution kernel shape automatic learning is characterized by comprising the following steps:
step 1: building a convolutional neural network for image classification;
step 2: introducing coefficient matrix F into the traditional convolution process, and adding weight sparse regularization term into the loss functionDistribution equilibrium regular term->Inter-group equalization canonical term->
Wherein lambda is 1 、λ 2 、λ 3 Is a coefficient used to balance the terms,is a loss-of-category term,/->Is a regularization term for weight decay;
the weight sparse regularization termIs calculated according to the formula:
wherein k is j For coefficients of different positions, f ij Representing the coefficients; "i" represents the ith group, "j" the jth position;
the distribution equilibrium regular termIs calculated according to the formula:
wherein G is corner Is the angle G edge Is a side; f (F) i Representing the coefficient f in the ith set of convolution kernels ij Sum of absolute values, F j Representing the coefficient f in the convolution kernel at the j-th position ij Sum of absolute values;
the inter-group equalization regular termIs calculated according to the formula:
wherein,represents G corner Position-generated inter-group equalization loss, +.>Representative ofG edge Position-generated inter-group equalization loss, +.>Represents G center The inter-group equalization loss generated by the position;
solving the three loss functions in the network training process for each term f in the coefficient matrix ij For back-propagating the update coefficient matrix F; obtaining a sparse coefficient matrix F after training is completed;
step 3: setting a threshold according to the model compression rate which is expected to be reached, and f which is lower than the threshold ij Removing convolution kernel parameters of the corresponding positions to obtain the convolution kernel shape of each convolution layer;
step 4: and replacing the original conventional convolution kernel with the sparse shape obtained by automatic learning, and carrying out network training again to obtain a final image classification neural network model.
2. The method for compressing the image classification network based on the convolutional kernel shape automatic learning according to claim 1, wherein the convolutional neural network in the step 1 is VGG.
3. The method for compressing the image classification network based on the convolutional kernel shape automatic learning according to claim 1, wherein the convolutional neural network in the step 1 is ResNet.
CN202110283921.3A 2021-03-17 2021-03-17 Image classification network compression method based on convolution kernel shape automatic learning Active CN113065640B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110283921.3A CN113065640B (en) 2021-03-17 2021-03-17 Image classification network compression method based on convolution kernel shape automatic learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110283921.3A CN113065640B (en) 2021-03-17 2021-03-17 Image classification network compression method based on convolution kernel shape automatic learning

Publications (2)

Publication Number Publication Date
CN113065640A CN113065640A (en) 2021-07-02
CN113065640B true CN113065640B (en) 2024-01-09

Family

ID=76560834

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110283921.3A Active CN113065640B (en) 2021-03-17 2021-03-17 Image classification network compression method based on convolution kernel shape automatic learning

Country Status (1)

Country Link
CN (1) CN113065640B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200224A (en) * 2014-08-28 2014-12-10 西北工业大学 Valueless image removing method based on deep convolutional neural networks
CN107609525A (en) * 2017-09-19 2018-01-19 吉林大学 Remote Sensing Target detection method based on Pruning strategy structure convolutional neural networks
WO2018028255A1 (en) * 2016-08-11 2018-02-15 深圳市未来媒体技术研究院 Image saliency detection method based on adversarial network
KR20180052063A (en) * 2016-11-07 2018-05-17 한국전자통신연구원 Convolution neural network system and operation method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200224A (en) * 2014-08-28 2014-12-10 西北工业大学 Valueless image removing method based on deep convolutional neural networks
WO2018028255A1 (en) * 2016-08-11 2018-02-15 深圳市未来媒体技术研究院 Image saliency detection method based on adversarial network
KR20180052063A (en) * 2016-11-07 2018-05-17 한국전자통신연구원 Convolution neural network system and operation method thereof
CN107609525A (en) * 2017-09-19 2018-01-19 吉林大学 Remote Sensing Target detection method based on Pruning strategy structure convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Accelerating convolutional networks via global & dynamic filter pruning";S.Lin et al.;《Joint Conf. Artif. Intell》;全文 *
基于融合特征以及卷积神经网络的环境声音分类系统研究;张科;苏雨;王靖宇;王霰宇;张彦华;;西北工业大学学报(第01期);全文 *

Also Published As

Publication number Publication date
CN113065640A (en) 2021-07-02

Similar Documents

Publication Publication Date Title
CN108985317B (en) Image classification method based on separable convolution and attention mechanism
CN108596258B (en) Image classification method based on convolutional neural network random pooling
CN111079781B (en) Lightweight convolutional neural network image recognition method based on low rank and sparse decomposition
CN107145889B (en) Target identification method based on double CNN network with RoI pooling
CN112036512B (en) Image classification neural network architecture searching method and device based on network clipping
CN111696101A (en) Light-weight solanaceae disease identification method based on SE-Inception
CN109214353B (en) Training method and device for rapid detection of face image based on pruning model
CN111507319A (en) Crop disease identification method based on deep fusion convolution network model
CN108205703B (en) Multi-input multi-output matrix average value pooling vectorization implementation method
CN109376787B (en) Manifold learning network and computer vision image set classification method based on manifold learning network
CN112381764A (en) Crop disease and insect pest detection method
CN111275165A (en) Network intrusion detection method based on improved convolutional neural network
CN113420651B (en) Light weight method, system and target detection method for deep convolutional neural network
CN115049814B (en) Intelligent eye protection lamp adjusting method adopting neural network model
CN113837314A (en) Hyperspectral image classification method based on hybrid convolutional neural network
QingJie et al. Research on image retrieval using deep convolutional neural network combining L1 regularization and PRelu activation function
CN116740119A (en) Tobacco leaf image active contour segmentation method based on deep learning
CN109740481A (en) Atrial fibrillation Modulation recognition method of the CNN based on jump connection in conjunction with LSTM
CN113505856B (en) Non-supervision self-adaptive classification method for hyperspectral images
CN111461978A (en) Attention mechanism-based resolution-by-resolution enhanced image super-resolution restoration method
CN113065640B (en) Image classification network compression method based on convolution kernel shape automatic learning
CN110399887B (en) Representative color extraction method based on visual saliency and histogram statistical technology
Rui et al. Smart network maintenance in an edge cloud computing environment: An adaptive model compression algorithm based on model pruning and model clustering
CN113837263B (en) Gesture image classification method based on feature fusion attention module and feature selection
CN112785663B (en) Image classification network compression method based on convolution kernel of arbitrary shape

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant