CN109472352A - A deep neural network model tailoring method based on feature map statistical features - Google Patents

Abstract

The invention discloses a deep neural network model cutting method based on statistical features of feature maps. The implementation steps are as follows: Step 1. For the feature layers in the deep neural network model, calculate the statistical features of the feature maps corresponding to each output channel; wherein The feature layer consists of two parts: the convolution layer and the activation layer, or consists of three parts: the convolution layer, the normalization layer and the activation layer; step 2, according to the statistical features of the feature maps corresponding to each output channel in the feature layer, calculate The evaluation index of each output channel in the feature layer; Step 3, according to the evaluation index, determine the importance of each output channel in the feature layer, and remove the unimportant output channels and their corresponding parameters. The invention can effectively reduce the dimension of the feature layer of the neural network, improve the operation efficiency of the network model, and at the same time, the network scale can be reduced, and the impact on the accuracy is small.

Description

A deep neural network model tailoring method based on feature map statistical features

technical field

The invention belongs to the fields of artificial intelligence and pattern recognition, and in particular relates to deep neural network model compression.

Background technique

Deep learning has achieved remarkable results in solving high-level abstract cognitive problems, bringing artificial intelligence to a new level and providing a technical foundation for high-precision, multi-type target detection, recognition and tracking. However, complex computing and huge resource requirements make neural networks only deployed on high-performance computing platforms, limiting their application on mobile devices. In 2015, Deep Compression published by Han applied network model clipping, weight sharing, quantization, and encoding to model compression, which achieved good results in model storage and led researchers to study network compression methods. At present, the research on deep learning model compression methods can be mainly divided into the following directions:

(1) The design of finer models, using more detailed and efficient model design, can greatly reduce the size of the model, and also has good performance.

(2) Model tailoring. Networks with complex structures have very good performance, and their parameters are also redundant. Therefore, for a trained model network, an effective evaluation method is usually found to judge the importance of parameters. Unimportant connections or convolution kernels are cropped to reduce model redundancy.

(3) The sparseness of the kernel. During the training process, the update of the weight is induced to make it more sparse. For the sparse matrix, a more compact storage method can be used, but the operation efficiency of the sparse matrix operation on the hardware platform is not high. High, vulnerable to bandwidth, so the speedup is not noticeable.

The method of trimming the pre-trained network model is currently the most used method in model compression. It is usually to find an effective evaluation index to judge the importance of neurons or feature maps, and connect unimportant or The convolution kernel is pruned to reduce the redundancy of the model. Li proposed a clipping method based on magnitude, using the sum of the absolute values of the weights to judge its importance, and using the sum of the absolute values of all the weights in the convolution kernel as the evaluation index of the convolution kernel. Hu defines a variable APoZ (Average Percentage of Zeros) to measure the number of values activated to 0 in each convolution kernel, as a criterion for evaluating whether a convolution kernel is important. Luo proposed a cropping method based on entropy value, which uses entropy value to determine the importance of convolution kernels. Anwar adopts a random clipping method, and then counts the performance of the model for each random method to determine the locally optimal clipping method. Tian's LDA analysis found that for each category, the activations between many convolution kernels are highly irrelevant, so this can be used to eliminate a large number of filters with only a small amount of information without affecting the performance of the model.

To sum up, the limitations of existing solutions are as follows:

a. The sparse method of the kernel only considers the compressed storage of the network, the compression effect during operation is not obvious, and the speed is not significantly improved;

b. Using the size of the weight as the evaluation index only considers the numerical characteristics of the weight itself, and does not consider the data characteristics of the network layer, and the compression effect is not high;

c. The calculation of the evaluation index is more complicated and consumes a lot of computing power;

d. The random cropping method is too random, and it is easy to destroy the parameter characteristics of the network itself;

Therefore, it is necessary to provide a method that is simple in calculation method, can fully consider the redundancy in the network, has strong applicability, and does not rely on a dedicated acceleration library to achieve deep neural network compression and acceleration.

SUMMARY OF THE INVENTION

In order to solve the defects of the prior art, the present invention discloses a deep neural network model trimming method based on feature map statistics. Compared with other compression methods, the present invention cuts the parameter layer of the network by using various statistical features of the feature layer of the neural network as the evaluation criteria, fully considers the numerical and statistical characteristics of the network, and achieves good compression efficiency. , while improving the running speed.

The technical scheme adopted in the present invention is:

A deep neural network model clipping method based on statistical features of feature maps, comprising the following steps:

Step 1. For the feature layer in the deep neural network model, calculate the statistical features of the feature map corresponding to each output channel; the feature layer is composed of two parts: the convolution layer and the activation layer, or the convolution layer, normalization layer The layer (BatchNorm layer) and the activation layer are composed of three parts; the present invention only calculates the feature layer of the layer (feature layer/full connection layer) that stores parameters later;

Step 2. Calculate the evaluation index of each output channel in the feature layer according to the statistical features of the feature map corresponding to each output channel in the feature layer;

Step 3: Determine the importance of each output channel in the feature layer according to the evaluation index, and remove the unimportant output channels and their corresponding parameters.

In an iterative step (epoch) of the deep neural network, the samples are sent to the neural network in different batches for calculation. In the step 2, the feature maps corresponding to each output channel of each feature layer in the network model are subjected to feature statistics in batches; for the i-th feature layer, the statistical features of the feature maps corresponding to all output channels include a mean vector. and a standard deviation vector S_v _i , the calculation steps are as follows:

S11: Initialization; set N _sum to count the number of samples that have been processed, initialized to 0; N _batch is the number of batches, N _batch = ceil (total number of samples/N), N is the number of samples in a batch, ceil ( ) is the round-up function; n _batch is the count of the current batch, initialized to 1; initialized mean vector The sum standard deviation vector S_v _i is a 1×C _i zero vector, where C _i is the number of output channels of the ith feature layer;

S12: Represent the output (features maps) X _i of the i-th feature layer corresponding to the n-th _batch of samples as a four-dimensional tensor of size N×C _i ×H _i ×W _i , where H _i and Wi are the height and width of the feature map (feature map) corresponding to the output channel of the _ith feature layer; the feature map (feature map) X of the jth output channel of the feature layer corresponding to the kth sample in the batch _ikj is a two-dimensional matrix of size H _i ×W _i , k=1, 2, . . . , N, j=1, 2, . . . , C _i ;

S13: Perform dimension transformation (view or reshape) on X _i to obtain a three-dimensional tensor X ^* _i of size N×C _i ×(H _i ×W _i ), that is, transform each feature map X _ikj in X _i by two The dimensional matrix is stretched to a one-dimensional vector X ^* _ikj ;

S14: Compute statistical features of X ^* _ikj , including mean and standard deviation S _ikj (both scalars):

in, represents the mth element in X ^* _kj ;

S15: by k=1, 2, ..., N, j=1, 2, ..., C _i forms a matrix of size N × C _i is the mean value A size N×C _i is a standard deviation matrix S_m _i formed by S _ikj , k=1, 2,..., N, j=1, 2,..., C _i ;

S16: For the mean matrix And the standard deviation matrix S_m _i is averaged by channel (average filter), and the processing formula is as follows:

N _sum = N _sum + N (5)

in, Yes The row vector corresponding to the kth row in ; is the row vector corresponding to the kth row in S_m _i ;

S17: Determine whether the current batch is the last batch, that is, whether there are n _batch = N _batch , if so, terminate the batch cycle, and the current mean vector The sum standard deviation vector S_v _i is the statistical feature of the feature map corresponding to all output channels of the feature layer; otherwise, update the count of the current batch: n _batch = n _batch +1, and jump to S12 to continue execution.

Further, in the step 2, the evaluation index of the jth output channel in the ith feature layer The calculation formula is as follows:

in, and S_v _ij is the mean vector corresponding to the feature layer and the jth element in the standard deviation vector S_v _i , representing the mean and standard deviation of the feature map corresponding to the jth output channel in the feature layer, α and β are two scale factors (hyperparameters), α is the mean threshold, when the mean along with The value becomes smaller and moves in the direction of negative infinity; otherwise, it is the mean value but along with The value becomes larger and moves to the zero direction; β is the threshold of the standard deviation S_v _ij , when the standard deviation S_v _ij <β, As the value of S_v _ij becomes smaller, it moves to the zero direction; otherwise, the Move toward positive infinity as the value of S_v _ij increases. when the mean When the value of and standard deviation S_v _ij is small, the α-subterm plays a leading role; when the mean When the value of the standard deviation S_v _ij is large, the β-subterm plays a leading role. There are two ways to determine α and β. One is to specify according to the empirical value. The range of hyperparameter values is set from low to high, and the evaluation index is calculated according to the value set each time, so as to tailor the neural network model, and Retrain the cropped model to restore the accuracy, and gradually achieve an optimal effect (that is, the maximum number of channels cropped when the network accuracy does not drop beyond the set threshold); the second is scaling, that is and β=η∑S_v _ij /C _i , μ and η are scale factors, the value range is (0, 0.4), the method can be dynamically adjusted according to the parameters of the network. ε is a minimal value that prevents division by zero.

Further, in the step 3, for the _i -th feature layer Li, if Then it is determined that the jth output channel in the feature layer is not important, and the channel and its corresponding parameters are removed.

Further, for the _ith feature layer Li, the steps of removing unimportant channels and their corresponding parameters are as follows:

S31: Record the corresponding evaluation criteria The set R _i of the channels of , the number of elements in the set R _i is recorded as length(R _i );

S32: Represent the convolution kernel Wi of the feature layer _{Li as a four-dimensional tensor of size C i-1 ×C i ×K hi ×K wi , and the bias B i corresponding} to the convolution kernel Wi is a size is a 1×C _i vector, where C _i-1 represents the number of output channels of the previous feature layer _Li-1 , if Li is the first feature layer, then C _{i -1} represents the number of sample input channels, K _hi and _Kwi are the height and width of the convolution kernel respectively; remove the elements corresponding to the channels belonging to the set _Ri in the _convolution kernel Wi to form a new convolution kernel Its size is with a new convolution kernel to replace the convolution kernel _Wi ; remove the elements corresponding to the channels belonging to the set _Ri in the bias _Bi to form a new bias Its size is

S33: If the next layer _Li+1 of the feature layer Li is still a feature layer, express the convolution kernel Wi ₊₁ of the next layer Li ₊₁ as a size C _i ×C _{i +1} ×K A four-dimensional tensor of _h(i+1) ×K _w(i+1) , where C _i+1 represents the number of output channels of the next layer L _i+1 , K _h(i+1) and K _{w(i+ 1)} are the height and width of the convolution kernel Wi ₊₁ respectively; ; Remove the elements corresponding to the channels belonging to the set R _i in the convolution kernel Wi ₊₁ to form a new convolution kernel Its size is with a new convolution kernel to replace the convolution kernel Wi ₊₁ ;

S34: If the next layer _Li+1 of the feature layer Li is a fully connected layer, express the parameter Vi ₊₁ of the next layer _{Li+1 as a size (C i × K hi} ×K _wi )× The matrix of C _i+1 ; remove the elements corresponding to the channels belonging to the set R _i in the parameter V _i+1 to form a new parameter Its size is with new parameters to replace the parameter V _i+1 .

After completing the model pruning, the deep neural network needs to undergo several iterations to retrain to restore the accuracy of the network. The number of iterations is related to the pruned feature layer and the evaluation criteria. The pruned feature layer is close to the input layer, and the number of iterations is less; the pruned feature layer is close to the output layer, and the number of iterations is more. In the evaluation criteria, the higher the values of α and β, the more iterations are required to restore the accuracy of the network.

Beneficial effects:

Compared with the prior art, the present invention makes full use of the statistical features of the deep neural network, and constructs an evaluation index based on the mean and standard deviation, which can effectively reduce the dimension of the feature layer of the neural network, improve the training speed of the deep neural network, and reduce the The frame size and number of weights of the deep neural network are improved, and the running speed/efficiency of the deep neural network is improved, and the impact on the accuracy is small. It has the following features and effects:

First, the present invention considers the statistical characteristics of the neural network when constructing the evaluation index for convolution kernel clipping, and uses the mean and standard deviation of the feature layer to take into account the features of the neural network value and the features in the feature layer. The data features of the feature layer reflect the effect of the parameters of the convolution kernel, and then the feature maps and corresponding convolution kernels with poor performance are cut out, which realizes the reduction of the network model framework and the compression of the parameter amount.

Second, in the criterion formula, hyperparameters α and β can be flexibly set to change the number of removed channels; when the statistical features fall close to 0, the α-sub-item will play a leading role; on the contrary, when Statistical features fall in the region far from 0, and the β-subterm will play a dominant role.

Third, the present invention proposes a new evaluation criterion on the basis of fully considering the statistical characteristics of the neural network. The algorithm has lower complexity and better performance, and can be deployed on real-time networks and embedded devices.

Description of drawings

Fig. 1 is the flow chart of the present invention;

FIG. 2 is a schematic structural diagram of a feature layer;

3 is a schematic diagram of the internal structure of the feature layer;

Fig. 4 is an example of the existence form of the feature layer in the neural network; Fig. 4(a) is a plurality of continuous feature layers, and Fig. 4(b) is a single feature layer;

Fig. 5 is the overall block diagram designed by the present invention;

Fig. 6 is the schematic diagram of model selection and cutting of the present invention;

Detailed ways

The present invention will be described in detail below with reference to specific embodiments, and the following embodiments will help those skilled in the art to further understand the present invention. The examples described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention, but should not be construed as limiting the present invention.

Figure 1 is a schematic flowchart of a deep neural network model clipping method based on feature map statistics in this example. By removing specific feature maps and corresponding convolution kernels, the network model framework is reduced and the parameter amount is compressed. , for the model cutting method, the specific implementation steps are:

(1) For the feature layer in the deep neural network, calculate the statistical features of each feature map in the feature layer in turn;

(2) Construct judging criteria according to statistical characteristics;

(3) The feature maps that do not meet the evaluation criteria and their corresponding convolution kernels are removed.

It should be noted that the operation object of the example of the present invention is the feature layer of the deep neural network that has been trained and converged, wherein the feature layer is a combination of the convolution layer and the activation layer (or can be described as an activation function, a nonlinear layer) two parts. It is composed of three parts: convolution layer, BatchNorm layer (normalization layer) and activation layer, as shown in Figure 2. For network types and modules, including but not limited to convolutional layers, batch normalization layers, activation layers, fully connected layers, and Resnet modules. In the deep neural network framework, the schematic diagram of the internal structure of the feature layer is shown in Figure 3. The _ith feature layer is Li, the convolution kernel of the _ith feature layer is Wi _, and the bias corresponding to the convolution kernel Wi is B _i . The present invention only calculates the feature layer of the layer (feature layer/fully connected layer) that has stored parameters in the back, such as other feature layers except the last feature layer in Fig. 4(a), but does not store parameters later. The feature layer of the layer is calculated, such as the feature layer in Figure 4(b), that is, if there is only a pooling layer, a normalization layer, an activation layer or a softmax layer behind the feature layer, this feature layer will not be operated.

In an iterative step (epoch) of the deep neural network, the samples are sent to the neural network in different batches for calculation. For the statistical features of the feature map in the network model, feature statistics are performed in batches. The following takes the _i -th feature layer Li as an example to illustrate, and the implementation steps are as follows:

S31: Initialize intermediate variables. N _sum is used to count the number of samples that have been processed, initialized to 0; N _batch is the number of batches, N _batch = ceil (total number of samples/N), N is the number of samples in a batch, ceil( ) is upward Rounding function; n _batch is the count of the current batch, initialized to 0; mean and standard deviation S _ikj are scalars, k = 1, 2, ..., N, j = 1, 2, ..., C _i , initialized to 0; mean matrix and the standard deviation matrix S_m _i is initialized as a zero matrix of size (N, C _i ), while the mean vector The sum standard deviation vector S_v _i is initialized as a zero vector of size (1, C _i ), where C _i is the number of output channels of the feature layer _Li .

S32: Perform view or reshape (dimension transformation) on the Li output of the _i -th feature layer corresponding to the n-th _batch of samples to obtain The size is changed from (N, C _i , H _i , Wi ) to (N, C _i , H _i *W _{i )} , which is equivalent to stretching the two-dimensional feature map X _ikj into a one-dimensional representation X ^* _ikj , k∈[1,N] is the index of the sample in the ith feature layer Li, j∈[1,C _i ] is the index of the feature map of the jth channel in the _ith feature layer _Li , it needs to be emphasized is the feature map X _ikj represents the element set of size (H _i , Wi ) of the j-th channel in the _i - _th feature layer Li, and the feature map X ^* _ikj represents the feature layer The set of elements of size (H _i *W _i ) of the jth channel in ;

S33: Calculate the feature layer Statistical features of the jth channel feature map X ^* _ikj in: mean and standard deviation S _ikj :

For feature layer For any feature map X ^* _ikj , the mean value is used and standard deviation S _ikj as statistical features of feature map X ^* _ikj . at the feature level In a batch of , a mean matrix of size (N, C _i ) can be generated and standard deviation matrix S_m _i ;

S34: For the mean matrix and the standard deviation matrix S_m _i are averaged by channel, which is implemented as follows:

N _sum = N _sum + N (5)

Among them, N _sum is the superposition of the number of samples in the first n _{batches, which is used to count the number of samples that have been processed; N is the number of samples in the nth batch ;} is the mean matrix The result after mean filtering, is the mean of all channels corresponding to the kth sample; S_v _i is the result of mean filtering by the standard deviation matrix S_m _i , is the standard deviation of all channels corresponding to the kth sample.

S35: Update the current batch: n _batch = n _batch +1, if n _batch = N _batch , then terminate the batch cycle; otherwise, the mean and standard deviation S _ikj set to 0; mean matrix and standard deviation matrix S_m _i reset to zero matrix, while mean vector The sum standard deviation vector S_v _i is also reset to the zero vector, and jumps to S32 to continue the execution. The mean vector obtained by iterating over the above batches and the standard deviation vector S_v _i , the evaluation index for the _jth channel of the feature layer Li The calculation process is as follows:

in, and S_v _ij are the mean and standard deviation of the _jth channel in the feature layer Li, α and β are two hyperparameters, these two scale factors are used to define the mean And the working range of standard deviation S_v _ij , ε is a minimum value, preventing the existence of division by zero. In the case of small hyperparameters, scaling is used, i.e. and β=η∑S_v _ij /C _i , μ and η are scale factors, the value range is (0, 0.4), when the hyperparameter becomes larger, the step-by-step approximation method is adopted, and the range of the hyperparameter value is from low to high , and gradually achieve an optimal effect.

Remove the feature maps that do not meet the evaluation criteria and their associated parameters, and calculate the evaluation index for the feature layer _Li of the C _i output channels Record the corresponding evaluation indicators The set of channels R _i of , the steps of removing the corresponding channel are as follows:

S71: The size of the convolution kernel Wi of the feature layer Li is (C _{i -1} , C _i , K _hi , K _wi ), and C _{i -1} represents the number of output channels or samples of the previous feature layer _Li-1 The number of input channels (if the feature layer _{Li is the first feature layer), C i is the number of output channels of the current feature layer Li, K hi , K wi are the size of} the convolution kernel. Build a new convolution kernel Its size is Indicates the number of channels (C _i -length(R _i )) after subtracting the number of elements contained in the set R _i from C _i . Cut out the elements corresponding to the channels that do not _belong to the set _Ri from the first dimension of the convolution kernel Wi, and copy them to the new convolution kernel , and then use the new convolution kernel to replace the convolution kernel _Wi ; construct a new bias Its size is Cut out the elements corresponding to the channels that do not belong to the set _Ri from the offset B _i and copy them to the new offset then use the new bias instead of bias _Bi ;

S72: If there is a feature layer after the feature layer _{Li, the size of the convolution kernel Wi+1 for the next feature layer Li+1 is (C i , C i +1} , K _h(i+1) ×K _w(i+1) ), C _i+1 represents the number of output channels of the next feature layer _Li+1 . Build a new convolution kernel Its size is Cut out the elements corresponding to the channels that do not belong to the set R _i from the first dimension of the convolution kernel Wi ₊₁ , and copy them to the new convolution kernel , and then use the new convolution kernel to replace the convolution kernel Wi ₊₁ ;

S73: If the feature layer Li is followed by a fully connected layer, the number of output channels of the connection layer is C _{i +1} , and the corresponding parameter size is (C _i ×K _hi ×K _wi , C _i+1 ). build a new parameter Its size is Cut out the elements corresponding to the channels that do not belong to the set R _i from the first dimension of the parameter V _i+1 , and copy them to the new parameter , then use the new parameters to replace the parameter V _i+1 ;

Further, after completing the model pruning, the deep neural network needs to undergo several iterations to retrain to restore the accuracy of the network. The number of iterations is related to the pruned feature layer and the evaluation criteria. The pruned feature layer is close to the input layer, and the number of iterations is less; the pruned feature layer is close to the output layer, and the number of iterations is more. In the evaluation criteria, the higher the values of α and β, the more iterations are required to restore the accuracy of the network.

Those skilled in the art can understand that all or part of the steps carried by the above implementation method can be implemented by program instructions, and the program can be stored in a computer-readable storage medium. The specific implementation of the present invention has been described above. It should be understood that the present invention compresses the feature layer and the fully connected layer. Therefore, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection of the present invention. within the range.

Claims (5)

1. a deep neural network model clipping method based on feature map statistical features, is characterized in that, in order to improve the compression efficiency and acceleration performance of the network, for the optimization carried out by deep neural network, the following steps are implemented: Step 1. For the feature layer in the deep neural network model, calculate the statistical features of the feature map corresponding to each output channel; the feature layer is composed of two parts: the convolution layer and the activation layer, or the convolution layer, normalization layer The layer and the activation layer are composed of three parts; Step 2. Calculate the evaluation index of each output channel in the feature layer according to the statistical features of the feature map corresponding to each output channel in the feature layer; Step 3: Determine the importance of each output channel in the feature layer according to the evaluation index, and remove the unimportant output channels and their corresponding parameters. 2. the deep neural network model clipping method based on feature map statistical feature according to claim 1, is characterized in that, in described step 2, to the feature map corresponding to each output channel of each feature layer in the deep neural network model, divide Perform feature statistics in batches; for the i-th feature layer, the statistical features of the feature maps corresponding to all output channels include a mean vector and a standard deviation vector S_v _i , the calculation steps are as follows: S11: Initialization; set N _sum to count the number of samples that have been processed, initialized to 0; N _batch is the number of batches, N _batch = ceil (total number of samples/N), N is the number of samples in a batch, ceil ( ) is the round-up function; n _batch is the count of the current batch, initialized to 1; initialized mean vector The sum standard deviation vector S_v _i is a 1×C _i zero vector, where C _i is the number of output channels of the ith feature layer; S12: Represent the output X _i of the i-th feature layer corresponding to the n-th _batch of samples as a four-dimensional tensor of size N×C _i ×H _i ×W _{i ,} where H _i and Wi are respectively The height and width of the feature map corresponding to the output channel of the i-th feature layer; the feature map X _ikj of the j-th output channel of the feature layer corresponding to the k-th sample in the batch is a two-dimensional H _i ×W _i dimensional matrix, k=1,2,...,N,j=1,2,...,C _i ; S13: Stretch each feature map X _ikj in _Xi from a two-dimensional matrix into a one-dimensional vector X ^* _ikj ; S14: Compute statistical features of X ^* _ikj , including mean and standard deviation S _ikj : in, represents the mth element in X ^* _ikj ; S15: by Form a matrix of size N × C _i as the mean value A size N×C _i is a standard deviation matrix S_m _i formed by S _ikj , k=1, 2,..., N, j=1, 2,..., C _i ; S16: For the mean matrix And the standard deviation matrix S_m _i is averaged by channel, and the processing formula is as follows: N _sum = N _sum + N (5) in, Yes The row vector corresponding to the kth row in ; is the row vector corresponding to the kth row in S_m _i ; S17: Determine whether the current batch is the last batch, that is, whether there is n _batch =N _atch , if so, terminate the batch cycle, and the current mean vector The sum standard deviation vector S_v _i is the statistical feature of the feature map corresponding to all output channels of the feature layer; otherwise, update the count of the current batch: n _batch = n _batch +1, and jump to S12 to continue execution. 3. the deep neural network model clipping method based on feature map statistical feature according to claim 2, is characterized in that, in described step 2, the evaluation index of the jth output channel in the ith feature layer The calculation formula is as follows: in, and S_v _ij is the mean vector corresponding to the feature layer The jth element in the sum standard deviation vector S_v _i represents the mean and standard deviation of the feature map corresponding to the jth output channel in the feature layer, α and β are two scale factors, and ε is a minimum value. 4. the deep neural network model clipping method based on feature map statistical feature according to claim 3, is characterized in that, in described step 3, for the _i -th feature layer Li, if Then it is determined that the jth output channel in the feature layer is not important, and the channel and its corresponding parameters are removed. 5. the deep neural network model clipping method based on feature map statistical feature according to claim 4, is characterized in that, for the _i -th feature layer Li, the step of removing unimportant channel and its corresponding parameter is as follows : S31: Record the corresponding evaluation criteria The set R _i of the channels of , the number of elements in the set R _i is recorded as length(R _i ); S32: Represent the convolution kernel Wi of the feature layer _{Li as a four-dimensional tensor of size C i-1 ×C i ×K hi ×K wi , and the bias B i corresponding} to the convolution kernel Wi is a size is a 1×C _i vector, where C _i-1 represents the number of output channels of the previous feature layer _Li-1 , if Li is the first feature layer, then C _{i -1} represents the number of sample input channels, K _hi and _Kwi are the height and width of the convolution kernel _Wi , respectively; remove the elements corresponding to the channels belonging to the set _Ri in the _convolution kernel Wi to form a new convolution kernel _Wi ^* , the size of which is Replace the convolution kernel Wi with a new _convolution kernel Wi ^* ; remove the element corresponding to the channel belonging to the set _{Ri in the bias B i to form} a new bias Its size is S33: If the next layer _Li+1 of the feature layer Li is still a feature layer, express the convolution kernel Wi ₊₁ of the next layer Li ₊₁ as a size C _i ×C _{i +1} ×K A four-dimensional tensor of _h(i+1) ×K _w(i+1) , where C _i+1 represents the number of output channels of the next layer L _i+1 , K _h(i+1) and K _{w(i+ 1)} are the height and width of the convolution kernel Wi ₊₁ respectively; remove the elements corresponding to the channels belonging to the set R _i in the convolution kernel Wi ₊₁ to form a new convolution kernel Its size is with a new convolution kernel to replace the convolution kernel Wi ₊₁ ; S34: If the next layer _Li+1 of the feature layer Li is a fully connected layer, express the parameter Vi ₊₁ of the next layer _{Li+1 as a size (C i × K hi} ×K _wi )× The matrix of C _i+1 ; remove the elements corresponding to the channels belonging to the set R _i in the parameter V _i+1 to form a new parameter Its size is with new parameters to replace the parameter V _i+1 .