CN114372565A - Target detection network compression method for edge device - Google Patents

Abstract

The invention belongs to the technical field of edge device target detection, and discloses a target detection network compression method for edge devices. The target detection network compression method for edge devices includes: optimizing the network structure of SkyNet, and compressing feature maps and weight parameters. Quantization; reconstructs the forward inference structure, incorporating part of the computational processing in depthwise separable convolutions. From the perspective of algorithm optimization, the present invention takes SkyNet as an example, and proposes a compression processing technology for the target detection network, which reduces the difficulty of deploying the target detection network on edge devices. The invention cuts out the network and is more suitable for edge devices. The present invention performs quantization processing and greatly reduces the size of the network model. The present invention performs merging processing, which greatly reduces the computational load of the network.

Description

A target detection network compression method for edge devices

technical field

The invention belongs to the technical field of edge device target detection, and in particular relates to a target detection network compression method for edge devices.

Background technique

Currently, the task of object detection is to find and identify objects of interest in images, which are widely used in scenarios such as autonomous driving, face detection, and video surveillance. In recent years, target detection algorithms based on convolutional neural networks have achieved better performance than traditional methods, but due to their huge amount of computation and parameters, most convolutional neural networks are deployed on general-purpose CPUs or general-purpose GPUs. Due to its large size and volume, it is difficult to achieve real-time detection of edge devices. There is an urgent need for a lightweight network or a way to compress the network, so that convolutional neural networks can directly perform reasoning work on edge devices.

In order to solve this problem, prior art 1 proposes a lightweight network of Mobilenet, and prior art 2 proposes a lightweight network of Xception, which uses Depthwise Separable Convolution (DSC, Depthwise Separable Convolution) instead of standard convolution, In order to reduce the amount of calculation and parameters, the operation efficiency of the target detection network on the edge device is improved to a certain extent. Existing technology 3 designs a hardware-friendly network Skynet for edge devices on the basis of depthwise separable convolution. Compared with MobileNet and Xception, Skynet has a more regular structure and a higher module reuse rate, but it is still relatively difficult for deployment platforms. High computing power requirements.

In order to meet the edge application scenarios, many neural network optimization techniques have been proposed, among which network compression is the main one, which is roughly divided into network quantization and network pruning. The quantization part uses low-precision numbers instead of high-precision numbers to participate in the convolution calculation, and uses partial precision loss to avoid floating-point operations, or uses clustering to replace part of the weights with a central value; pruning is to use Network sparsity, because the smaller the weight in the neural network, the smaller the impact on the final prediction result, so the network can be skipped by judging whether the network weight is zero.

Through the above analysis, the existing problems and defects of the existing technology are: the existing target detection network is inefficient in operation on edge devices, has high requirements on the devices, consumes a lot of power, and is bulky, and cannot perform real-time detection of edge devices.

The difficulty of solving the above problems and defects is: the use of convolutional neural networks can achieve high-precision target detection tasks, but its characteristics of large amount of parameters and large amount of calculation make it difficult to achieve on edge devices.

The significance of solving the above problems and defects is: through this set of compression methods, the amount of network parameters and calculations can be reduced, and network deployment can be carried out on edge devices with low power consumption and low cost, and the accuracy loss is within a reasonable range.

SUMMARY OF THE INVENTION

In view of the problems existing in the prior art, the present invention provides a target detection network compression method for edge devices.

The present invention is implemented in this way, a target detection network compression method for edge devices, comprising:

Optimize the network structure of SkyNet, quantify the feature map and weight parameters; reconstruct the forward inference structure, and combine some computing processing in the depthwise separable convolution.

Further, the target detection network compression method for edge devices includes the following steps:

Step 1, by removing the bypass branch structure in the SkyNet network, deleting the partial channel of the first layer output to carry out the optimization and cutting of the SkyNet network, to obtain the optimized SkyNet network;

Step 2: Compress the SkyNet network after retraining by quantizing the weights to 7 bits, quantizing the feature maps to 8 bits, fusing the bias parameters and scaling coefficients, and quantizing them to 32 bits;

Step 3: Merge the SkyNet network structure by merging the convolution layer and the normalization layer, merging activation processing, quantization, inverse quantization and saturation truncation processing into FETCH processing.

Further, the described optimization and cropping of the SkyNet network comprises the following steps:

First, prune the Skynet branch; take each depthwise separable convolution as the minimum unit layer, and prune the output of the first layer into 32 channels;

Second, pooling is added after the sixth layer, and the last layer is modified to be depthwise separable convolution, and the overall network structure is optimized to be straight.

Further, the optimized SkyNet network includes:

3-channel input layer CHL3, intermediate layer CHL32, CHL96, CHL192, CHL384, CHL512, CHL96 and regression layer CHL30;

The convolution between each layer of the optimized SkyNet network uses depthwise separable convolution.

Further, the described compression of the SkyNet network after retraining comprises the following steps:

(1) Select the maximum value in the convolution kernel corresponding to each output channel as the maximum value of quantization, and use the maximum value quantization method to quantify the weight:

q_w=w×scale_w;

Among them, w represents the vector of the original weight corresponding to each channel; q_w represents the vector of the quantized weight; scale_w represents that the scaling coefficient is a scalar;

(2) Use the KL relative entropy to select the threshold value, and use saturated quantization to quantify the feature map: select the threshold value T, and scale the value of the original distribution within the range of ±T to -127 to +127, and the part beyond the range is scaled. Perform saturation processing, and directly take the saturation value to represent the value of the part outside the range.

Further, using the KL relative entropy to select a threshold, and adopting saturated quantization to perform feature map quantization includes:

1) Use the KL relative entropy to select the threshold:

Among them, p represents the original distribution before quantization, q represents the distribution after quantization using the threshold T, H(p) represents the information entropy of the original distribution, H(p, q) represents the cross entropy between the original distribution and the quantized distribution, DKL(p|| q) represents the KL relative entropy;

2) Calculate the scaling factor scale_fm:

3) The bias and scaling coefficients are fixed-point, and the floating-point units appearing in the forward inference are combined, enlarged and rounded, and the final coefficients are saved by 32-bit integers;

4) Calculate the inverse quantization coefficient after combining and enlarging:

Bias_merge=int(bias×scale_next_fm×shift_coe);

Among them, scale_w represents the weight quantization coefficient; scale_fm currently represents the quantization coefficient of the feature map of the previous layer; bias represents the bias; scale_next_fm represents the quantization coefficient of the next layer; Scale_merge represents the inverse quantization coefficient after merging and enlarging; shift_coe represents the magnification.

Further, the described merging of SkyNet network structure includes:

(1) Combine the convolutional layer and the normalization layer, and the combined output y ₃ is:

in:

where y ₁ represents the convolution output; x represents the input; w represents the weight; b represents the bias; x, w and b are all vectors; μ represents the mean; σ represents the standard deviation; γ represents the scaling factor; β represents the scaling offset; ε=1e-6; W represents the weight after fusion; B represents the bias after fusion;

(2) Combine activation processing, quantization, inverse quantization, and saturation truncation into FETCH processing; the FETCH processing converts 32 bits into 8 bits; and performs ReLu activation and saturation truncation.

Further, the described performing ReLu activation and saturation cutoff includes:

Judging the positive and negative of the input data, if it is positive, it will perform saturation truncation; if it is negative, it will output the activated value of 0.

Another object of the present invention is to provide a target detection network compression system for edge devices, including:

The network pruning module is used to optimize and trim the SkyNet network by removing the bypass branch structure in the SkyNet network and deleting part of the output channels of the first layer to obtain the optimized SkyNet network;

The network compression module compresses the SkyNet network after retraining by quantizing the weights to 7 bits, quantizing the feature maps to 8 bits, fusing the bias parameters and scaling coefficients, and fixing them to 32 bits;

The network structure merging module is used to merge the SkyNet network structure by merging the convolution layer and the normalization layer, merging activation processing, quantization, inverse quantization and saturation truncation processing into FETCH processing.

Combined with all the above-mentioned technical solutions, the advantages and positive effects possessed by the present invention are:

From the perspective of algorithm optimization, the present invention takes SkyNet as an example, and proposes a compression processing technology for the target detection network, which reduces the difficulty of deploying the target detection network on edge devices. The invention cuts out the network and is more suitable for edge devices. The present invention performs quantization processing and greatly reduces the size of the network model. The present invention performs merging processing, which greatly reduces the computational load of the network.

Description of drawings

FIG. 1 is a flowchart of a target detection network compression method for an edge device provided by an embodiment of the present invention.

FIG. 2 is a structural diagram of an optimized Skynet network provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of a saturated intercept scaling and quantization provided by an embodiment of the present invention.

FIG. 4 is a schematic diagram of a combined convolution layer and a normalization layer provided by an embodiment of the present invention.

FIG. 5 is a schematic diagram of a FETCH operation provided by an embodiment of the present invention.

FIG. 6 is a schematic diagram of a calculation process of each layer before fixed-pointization provided by an embodiment of the present invention.

FIG. 7 is a schematic diagram of a calculation process of each layer after fixed pointization provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In view of the problems existing in the prior art, the present invention provides a target detection network compression method for edge devices. The present invention will be described in detail below with reference to the accompanying drawings.

The target detection network compression method for edge devices provided by the embodiment of the present invention includes:

Optimize the network structure of SkyNet, quantify the feature map and weight parameters; reconstruct the forward inference structure, and combine some computing processing in the depthwise separable convolution.

As shown in FIG. 1 , the target detection network compression method for edge devices provided by an embodiment of the present invention includes the following steps:

S101, optimize and cut the SkyNet network by removing the bypass branch structure in the SkyNet network and deleting part of the output channels of the first layer to obtain the optimized SkyNet network;

S102, compress the SkyNet network after retraining by quantizing the weights to 7 bits, quantizing the feature maps to 8 bits, fusing the bias parameters and scaling coefficients, and quantizing them to 32 bits;

S103 , merge the SkyNet network structure by merging the convolution layer and the normalization layer, and merging activation processing, quantization, inverse quantization, and saturation truncation processing into FETCH processing.

The optimization and tailoring of the SkyNet network provided by the embodiment of the present invention includes the following steps:

First, prune the Skynet branch; take each depthwise separable convolution as the minimum unit layer, and prune the output of the first layer into 32 channels;

Second, pooling is added after the sixth layer, and the last layer is modified to be depthwise separable convolution, and the overall network structure is optimized to be straight.

The optimized SkyNet network provided by the embodiment of the present invention includes:

3-channel input layer CHL3, intermediate layer CHL32, CHL96, CHL192, CHL384, CHL512, CHL96 and regression layer CHL30;

The convolution between each layer of the optimized SkyNet network uses depthwise separable convolution.

The compression of the SkyNet network after retraining provided by the embodiment of the present invention includes the following steps:

(1) Select the maximum value in the convolution kernel corresponding to each output channel as the maximum value of quantization, and use the maximum value quantization method to quantify the weight:

q_w=w×scale_w;

Among them, w represents the vector of the original weight corresponding to each channel; q_w represents the vector of the quantized weight; scale_w represents that the scaling coefficient is a scalar;

(2) Use the KL relative entropy to select the threshold value, and use saturated quantization to quantify the feature map: select the threshold value T, and scale the value of the original distribution within the range of ±T to -127 to +127, and the part beyond the range is scaled. Perform saturation processing, and directly take the saturation value to represent the value of the part outside the range.

Using the KL relative entropy to select a threshold provided by the embodiment of the present invention, and adopting saturated quantization to perform feature map quantization includes:

1) Use the KL relative entropy to select the threshold:

Among them, p represents the original distribution before quantization, q represents the distribution after quantization using the threshold T, H(p) represents the information entropy of the original distribution, H(p, q) represents the cross entropy between the original distribution and the quantized distribution, DKL(p|| q) represents the KL relative entropy;

2) Calculate the scaling factor scale_fm:

3) The bias and scaling coefficients are fixed-point, and the floating-point units appearing in the forward inference are combined, enlarged and rounded, and the final coefficients are saved by 32-bit integers;

4) Calculate the inverse quantization coefficient after combining and enlarging:

Bias_merge=int(bias×scale_next_fm×shift_coe);

Among them, scale_w represents the weight quantization coefficient; scale_fm currently represents the quantization coefficient of the feature map of the previous layer; bias represents the bias; scale_next_fm represents the quantization coefficient of the next layer; Scale_merge represents the inverse quantization coefficient after merging and enlarging; shift_coe represents the magnification.

The merging of the SkyNet network structure provided by the embodiment of the present invention includes:

(1) Combine the convolutional layer and the normalization layer, and the combined output y ₃ is:

in:

where y ₁ represents the convolution output; x represents the input; w represents the weight; b represents the bias; x, w and b are all vectors; μ represents the mean; σ represents the standard deviation; γ represents the scaling factor; β represents the scaling offset; ε=1e-6; W represents the weight after fusion; B represents the bias after fusion;

(2) Combine activation processing, quantization, inverse quantization, and saturation truncation into FETCH processing; the FETCH processing converts 32 bits into 8 bits; and performs ReLu activation and saturation truncation.

The ReLu activation and saturation truncation provided by the embodiment of the present invention includes:

Judging the positive and negative of the input data, if it is positive, it will perform saturation truncation; if it is negative, it will output the activated value of 0.

The technical solutions of the present invention will be further described below with reference to specific embodiments.

Example 1:

The target detection network compression method for edge devices provided by the embodiment of the present invention includes:

(1) Optimize cutting

In the present invention, the Skynet branch is first pruned, secondly, each depth separable convolution is used as the minimum unit layer, the output of the first layer is pruned into 32 channels, and then a pooling operation is added after the sixth layer, and finally, The last layer is modified to be depthwise separable convolution, and the overall network structure is optimized to be straight.

The optimized Skynet is shown in Figure 2. The overall network contains a total of 8 layers, namely the 3-channel input layer (CHL3), the middle layer (CHL32, CHL96, CHL192, CHL384, CHL512, CHL96), and the regression layer (CHL30). The convolution between layers is implemented using depthwise separable convolution (DSC), and the network structure is regular, which facilitates module reuse and forms an efficient computing structure.

The size of the input image used in the present invention is 160×160, and the size of each layer of image is shown in Table 1:

Table 1 The specific dimensions of each layer

(2) Compress the model

The quantization part is the quantization of weights, the quantization of feature maps, and the quantization of bias and scaling coefficients, respectively.

The weight adopts the maximum value quantization method, and the maximum value is selected from the maximum value in the convolution kernel corresponding to each output channel.

Let the original weight corresponding to each channel be a vector w, the quantized weight is a vector q_w, and the scaling coefficient is a scalar scale_w, then the corresponding relationship is shown in formula (1) (2). Evenly distributed between -63 and 63.

q_w=w×scale_w (2)

The feature map quantization adopts saturated quantization, and uses the KL relative entropy to select the threshold value, which significantly reduces the loss of accuracy.

Saturated quantization is shown in Figure 3. A threshold T is selected, and the original distribution is scaled to -127 to +127 in the range of ±T. The red value table in the figure is out of range after scaling, and saturation processing is performed. , directly take the saturation value to represent this part of the value.

Let the original distribution before quantization be p, the distribution after quantization with the threshold T is q, the information entropy of the original distribution is H(p), the cross entropy between the original distribution and the quantized distribution is H(p,q), and the KL relative entropy is DKL(p ||q), then there is the following formula:

The available scaling factor is scale_fm, where:

The bias and scaling coefficients are fixed-point, and the floating-point units appearing in the forward inference are combined, enlarged and rounded, and the final coefficients are stored in 32-bit integers. Let the weight quantization coefficient be scale_w, the current layer feature map quantization coefficient is scale_fm, the bias is bias, and the next layer quantization coefficient is scale_next_fm.

Let the inverse quantization coefficient after merging and magnification be Scale_merge, the bias coefficient be Bias_merge, and the magnification factor be shift_coe, there are:

Bias_merge=int(bias×scale_next_fm×shift_coe) (7)

(3) Merge network structure

The convolution layer and the normalization layer are combined. The depthwise separable convolution includes the normalization layer. When training the network model, the normalization layer can accelerate the network convergence, control over-fitting, and solve the problems of gradient disappearance and gradient explosion. After the model training is completed, all parameters have been fixed. At this time, the convolution layer parameters and normalization layer parameters in the network are combined, as shown in Figure 4, which can effectively simplify the network structure, reduce the amount of calculation, and improve the calculation efficiency.

Let the convolution output be y1, the input be x, the weight be w, the bias be b, x, w and b are all vectors, the output of the normalization layer is y2, the mean is μ, the standard deviation is σ, and the scaling factor is γ, The scaling offset is β, and ε=1e-6, then the convolution calculation formula is:

y ₁ =wx+b (9)

The normalization layer calculation formula is:

in:

Substitute Equation (10) into Equation (11), let W be the weight after fusion, B be the offset after fusion, and output y3 after merging:

in:

After the fusion, there is no normalization layer calculation, which reduces the size of the model, saves computing resources, and improves the performance of forward reasoning.

The activation operation, quantization, inverse quantization, and saturation truncation are combined into a FETCH operation, as shown in Figure 5.

The fetch operation performs the conversion from 32bit to 8bit, which essentially completes the two processes of ReLu activation and saturation truncation. After calculating with the fixed-point data, the result needs to be reduced to the previous 1/shift_coe. For edge devices, since shift_coe is a power of 2, it can be directly fetched, and the positive and negative input data needs to be checked in the middle. Judgment, if it is positive, the saturation truncation is performed; if it is negative, the value 0 after activation is output.

Figure 6 shows the inference steps before fixed-pointization and network structure merging, and Figure 7 shows the optimized inference steps.

Example 2:

(1) Optimize cropping and retraining

Retraining is carried out on the model after the Skynet network structure is optimized to be straight. Table 2 shows the accuracy comparison of Skynet before and after pruning. Taking the average precision (AP, as shown in Equation 23) as the evaluation index, the accuracy dropped by less than 0.03. In the case of meeting the needs of practical applications, it greatly improves the performance of Skynet. Efficiency of real-time computing on edge devices.

Table 2 Accuracy comparison before and after Skynet pruning

model type average precision full model 0.797 model after pruning 0.770

(2) Compress the model and merge the network structure

Quantizing the pruned and retrained model and combining activation operations, quantization, inverse quantization, and saturation truncation operations, compared with the uncompressed model, the accuracy is only reduced by 2.34%, while the network model size is reduced by 74.5% .

Before and after model compression optimization, the comparison table about model size, average accuracy and accuracy loss is as follows

shown in Table 3.

Before compression after compression type of data 32bit floating point 7bit/8bit/32bit fixed point Model size (MB) 1.41 0.359 Average Precision (AP) 0.770 0.752 loss of precision --- 2.34% compression ratio --- 74.5%

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this, any person skilled in the art is within the technical scope disclosed by the present invention, and all within the spirit and principle of the present invention Any modifications, equivalent replacements and improvements made within the scope of the present invention should be included within the protection scope of the present invention.

Claims (8)

1. An object detection network compression method for an edge device, the object detection network compression method for an edge device comprising:

optimizing a network structure for SkyNet, and quantizing the characteristic diagram and the weight parameter; and reconstructing a forward reasoning structure and combining part of calculation processing in the depth separable convolution.

2. The object detection network compression method for edge devices of claim 1, wherein the object detection network compression method for edge devices comprises the steps of:

removing a bypass branch structure in the SkyNet network and deleting a partial channel output by a first layer to perform optimized cutting of the SkyNet network to obtain an optimized SkyNet network;

step two, the SkyNet network is compressed after the weight is quantized to 7 bits, the characteristic diagram is quantized to 8 bits, the bias parameters and the scaling coefficients are fused, and the point is fixed to 32 bits for retraining;

and step three, merging the convolution layer and the normalization layer, merging the activation processing, the quantization, the inverse quantization and the saturation truncation processing into FETCH processing, and merging the SkyNet network structure.

3. The target detection network compression method for an edge device of claim 2, wherein the performing optimized pruning of the SkyNet network comprises the steps of:

first, the Skynet branches are pruned; pruning the output of the first layer into 32 channels by taking each depth separable convolution as a minimum unit layer;

secondly, adding pooling treatment after the sixth layer, modifying the last layer into depth separable convolution, and optimizing the whole network structure into a straight cylinder type.

4. The target detection network compression method for an edge device of claim 2, wherein the optimized SkyNet network comprises:

a 3-channel input layer CHL3, intermediate layers CHL32, CHL96, CHL192, CHL384, CHL512, CHL96, and a regression layer CHL 30;

the convolution between each layer of the optimized SkyNet network uses a deep separable convolution.

5. The method of object detection network compression for an edge device of claim 2, wherein the compression of the retrained SkyNet network comprises the steps of:

(1) selecting the maximum value in the convolution kernel corresponding to each output channel as the maximum value of quantization, and performing weight quantization by adopting a maximum value quantization mode:

q_w＝w×scale_w；

wherein w represents a vector of the original weight corresponding to each channel; q _ w represents a vector of quantized weights; scale _ w represents that the scaling factor is a scalar;

(2) selecting a threshold value by utilizing the KL relative entropy, and quantizing the feature diagram by adopting a saturation type quantization method: selecting a threshold value T, scaling the value of the original distribution within the range of +/-T to-127 to +127 in equal proportion, carrying out saturation treatment on the part out of the range, and directly taking the value of the part out of the range represented by the saturation value.

6. The method according to claim 5, wherein the selecting the threshold value using the KL relative entropy and performing the feature map quantization using saturation quantization comprises:

1) selecting a threshold value by utilizing KL relative entropy:

wherein p represents original distribution before quantization is set, q represents distribution after quantization using a threshold value T, H (p) represents original distribution information entropy, H (p, q) represents cross entropy of original distribution and distribution after quantization, and DKL (p | | q) represents KL relative entropy;

2) calculating a scaling factor scale _ fm:

3) performing fixed-point processing on the bias and scaling coefficients, merging floating point number units appearing in forward reasoning, amplifying and rounding, and storing the final coefficient by adopting a 32-bit integer number;

4) calculating the inverse quantization coefficient after merging, amplification and rounding:

Bias_merge＝int(bias×scale_next_fm×shift_coe)；

wherein scale _ w represents a weight quantization coefficient; scale _ fm represents the quantization coefficient of the front layer feature map; bias represents bias; scale _ next _ fm represents the next layer quantized coefficient; scale _ merge represents the inverse quantization coefficient after merging, amplification and rounding; bias _ merge represents a Bias coefficient; shift _ coe represents magnification.

7. The method of object detection network compression for an edge device of claim 2, wherein said merging the SkyNet network structure comprises:

(1) merging the convolution layer and the normalization layer to obtain a merged output y₃Comprises the following steps:

wherein:

wherein, y₁Represents the convolution output; x represents an input; w represents a weight; b represents a deviation; x, w and b are vectors; μ represents a mean value; σ represents the standard deviation; γ represents a scaling coefficient; β represents a scaling offset; 1 e-6; w represents a post-fusion weight; b represents post-fusion bias;

(2) merging activation processing, quantization, inverse quantization and saturation truncation into FETCH processing; the FETCH processing converts 32 bits into 8 bits; performing ReLu activation and saturation truncation; the performing ReLu activation and saturation truncation comprises:

judging the positive and negative of the input data, and if the positive is true, performing saturation truncation; if the output is negative, the activated value is 0.

8. An object detection network compression system for an edge device implementing the object detection network compression method for the edge device according to any one of claims 7, wherein the object detection network compression system for the edge device comprises:

the network pruning module is used for carrying out optimized cutting on the SkyNet network by removing a bypass branch structure in the SkyNet network and deleting a part of channels output by the first layer to obtain the optimized SkyNet network;

the network compression module is used for compressing the SkyNet network after the weight is quantized to 7 bits, the characteristic diagram is quantized to 8 bits, the bias parameters and the scaling coefficients are fused, and the point is fixed to 32 bits for retraining;

and the network structure merging module is used for merging the convolution layer and the normalization layer, merging the activation processing, the quantization, the inverse quantization and the saturation truncation processing into FETCH processing and merging the SkyNet network structure.

Images