CN117291250A - Neural network pruning method for image segmentation - Google Patents

Abstract

The invention relates to the technical field of image segmentation, in particular to a neural network pruning method for image segmentation, which comprises the following steps: training a U-Net network using a Kvasir polyp segmentation dataset; calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0; calculating gradient information in the back propagation process; obtaining a measure of the importance of the neuron to the loss or function of the given data; model parameter pruning; the beneficial effects are as follows: combining the forward signal and the reverse signal results in a measure of importance to the given data set, and neurons having a score less than a threshold are clipped according to the measure of importance. By pruning redundant parameters, the volume of the model can be reduced, and the reasoning efficiency can be improved, so that the requirements of actual deployment can be better met.

Description

A neural network pruning method for image segmentation

Technical field

The present invention relates to the technical field of image segmentation, specifically a neural network pruning method for image segmentation.

Background technique

Image segmentation is an important task in the field of computer vision, which aims to divide pixels in an image into different regions with semantic meaning. Image segmentation plays a key role in many application areas, including medical image analysis, autonomous driving, image editing, and augmented reality.

In the existing technology, in the early stages of computer vision, image segmentation was mainly based on low-level features, such as edges, textures, and colors. Classic algorithms include edge detection-based methods, such as the Canny edge detection algorithm, and region growing-based methods, such as threshold-based segmentation algorithms. These methods are mainly based on heuristic rules and hand-designed features. With the rapid development of deep learning, convolutional neural networks have made significant breakthroughs in the field of image segmentation. Deep learning methods can automatically learn feature representations through end-to-end training and achieve better performance on large-scale data. Well-known deep learning methods include fully convolutional networks, U-Net and Mask R-CNN, etc. These methods have achieved high accuracy in image segmentation tasks and can handle the segmentation of complex scenes and multiple objects; in recent years, deep learning has developed rapidly, and more and more scholars are applying it to image segmentation. field, among which the segmentation model represented by the Un-Net network is the most widely used.

However, the latency of large segmentation models has been a major bottleneck for successful deployment. Although using smaller split models is an option, this often sacrifices model performance.

Contents of the invention

The purpose of the present invention is to provide a neural network pruning method for image segmentation to solve the problems raised in the above background technology.

To achieve the above objectives, the present invention provides the following technical solution: a neural network pruning method for image segmentation, which method includes the following steps:

S1. Use the Kvasir polyp segmentation data set to train the U-Net network;

S2. Calculate the activation degree of neurons during the forward propagation process and find the neurons with an activation value of 0;

S3. Calculate the gradient information in the back propagation process;

S4. Combine steps S2 and S3 to obtain the importance measure of the loss or function of the neuron to the given data;

S5. Model parameter pruning.

Preferably, in step S1, the Kvasir polyp segmentation data set is used for the segmentation task of intestinal polyp images. The Kvasir data set includes intestinal polyp images from endoscopy, including images of different categories including polyps and normal tissue, The resolution of the image is 1080p and the size is 576x576 pixels.

Preferably, in step S2, the forward signal is usually represented by a pre-activation value. When the pre-activation of a neuron is zero, it means that the input signal of the neuron is equal to zero after the weighted sum of weights and biases. In the neural network , pre-activation is usually obtained by linear transformation of the input signal, that is, a state where the activation function has not been applied for non-linear transformation;

If the preactivation is zero, then the neuron has no effect on the output of the function because it does not contribute to the final output;

If the input connection of the neuron has zero weight, the neuron is removed, that is, the neuron has no importance;

A neuron is significant if the input connections are non-zero, and the forward signal takes into account the impact of the data on a specific neuron.

Preferably, in step S3, the reverse signal is usually obtained through backpropagation loss. When the output connection of the neuron is zero, the activation degree of the neuron is positive, and the impact on the function is also insignificant. The gradient provides provides information about how the function or loss changes when the neuron is removed.

Preferably, in step S3, during the back propagation process of the neural network, first calculate the gradient of the loss function on the network output, and then back propagate the gradient along the connections of the network to calculate the gradient of each neuron; The process of backpropagation gives the gradient information of the neuron, that is, the gradient indicates the change of the loss function when the neuron is removed;

If the output connection weight of a neuron is zero, it means that it has no effect on the output of the neuron in the subsequent layer; by calculating the gradient of the neuron, we can understand how the function or loss function changes when the neuron is removed.

Preferably, in step S4, combining the forward and backward signals to obtain the impact of the neuron on the loss or function of the given data includes the following parts:

For a data set with a number of M, the importance measure I _i of each neuron in the U-Net network can be obtained by the following formula (1)

Where _xi is the preactivation and _δxi is its gradient, the above formula satisfies the rule that if the incoming or outgoing connections are zero, the importance should be low, otherwise the importance should be higher.

Preferably, in step S4, combining the forward and backward signals to obtain the impact of the neuron on the loss or function of the given data also includes the following parts:

The resulting importance measure I _i is similar to Taylor's first-order expansion. The metric index gives lower importance when the gradient is negative. Square the importance measure I _i and increase the neurons with negative gradients in backpropagation. The improved importance measure I _i is shown in formula (2):

in represents the square of the importance measure I _i .

Preferably, in step S4, combining the forward and backward signals to obtain the impact of the neuron on the loss or function of the given data also includes the following parts:

In the process of calculating the gradient, the gradient is normalized, usually normalized to 1, to ensure that each data point has an equal contribution in calculating the importance. By normalizing the gradient, the difference in gradient amplitude is eliminated. , ensuring that each input sample has the same weight when calculating importance, ensuring that different input samples have a fair contribution to the calculation of the overall importance score.

Preferably, in step S5, the model parameters are pruned on a data set D∈[x ₀ , x ₁ ,..., x _n ] of size N. In the neural network, the input and output are simply expressed as: y _n = f(x _n ), the target t _n is the label that you want the model to predict or classify. For each target t _n , calculate the gradient with respect to each input x _n , revealing the expected output for the given target, and for each input The degree of contribution of the output, the formula is shown in (3):

Then normalize the gradient Δy _n :

Afterwards, the gradient Backpropagation is performed to calculate the square of the importance measure I _i for each convolution kernel/>

Compared with the prior art, the beneficial effects of the present invention are:

The neural network pruning method for image segmentation proposed by the present invention uses the Kvasir polyp segmentation data set to train the U-Net network; calculates the activation degree of neurons during the forward propagation process, and finds neurons with an activation value of 0; calculate Gradient information in the back propagation process; combine the forward signal and the reverse signal to obtain the importance measure for the given data set, and crop out neurons with scores less than the threshold based on the importance measure. By pruning redundant parameters, the size of the model can be reduced and the inference efficiency improved, thereby better meeting the needs of actual deployment.

Description of drawings

Figure 1 is a flow chart of the method of the present invention;

Figure 2 is a comparison chart of parameter changes of the U-Net model before and after pruning in the present invention.

Detailed ways

In order to clearly and completely describe the purpose and technical solution of the present invention, and make the advantages more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described here are part of the embodiments of the present invention, rather than all embodiments. They are only used to explain the embodiments of the present invention and are not used to limit the embodiments of the present invention. Those of ordinary skill in the art will not All other embodiments obtained without creative efforts belong to the scope of protection of the present invention.

Referring to Figures 1 to 2, the present invention provides a technical solution: a neural network pruning method for image segmentation. The method includes the following steps:

S1. Use the Kvasir polyp segmentation data set to train the U-Net network;

S2. Calculate the activation degree of neurons during the forward propagation process and find the neurons with an activation value of 0;

S3. Calculate the gradient information in the back propagation process;

S4. Combine S2 and S3 to obtain the importance measure of the loss or function of the neuron to the given data;

S5. Model parameter pruning.

In step S1, the Kvasir polyp segmentation data set is a data set widely used in the field of medical image segmentation and is specifically used for the segmentation task of intestinal polyp images. The dataset was developed by a research team at the University of Trondheim in Norway to facilitate the development of intestinal polyp-related research and algorithms. The Kvasir dataset contains images of intestinal polyps from endoscopy. The dataset contains images of different categories including polyps and normal tissue. The resolution of the image is 1080p and the size is 576x576 pixels. This dataset has been widely used to train and evaluate deep learning and computer vision algorithms to improve intestinal polyp detection.

In step S2, the forward signal is typically represented by a preactivation value. When the pre-activation of a neuron is zero, it means that the input signal of the neuron is equal to zero after the weighted sum of weights and biases. In neural networks, pre-activation is usually obtained by linear transformation of the input signal, that is, a state where the activation function has not yet been applied for non-linear transformation. If the preactivation is zero, then the neuron has no effect on the output of the function because it does not contribute to the final output. Furthermore, a neuron can be removed if its input connection has zero weight, i.e. it has no importance. A neuron has significance if its input connections are non-zero. The forward signal takes into account the impact of the data on a specific neuron.

In step S3, the reverse signal is usually obtained by backpropagation loss. When a neuron's output connection is zero, its effect on the function is insignificant, even if the neuron's activation level is positive. The gradient provides information about how the function or loss changes when neurons are removed.

In the back propagation process of the neural network, the gradient of the loss function on the network output is first calculated, and then this gradient is back propagated along the connections of the network to calculate the gradient of each neuron. This backpropagation process can give the gradient information of the neuron, that is, the gradient indicates the change of the loss function when the neuron is removed. If the output connection weight of a neuron is zero, it means that it has no effect on the output of subsequent layers of neurons. Even if the neuron has positive activation, its impact on the result or loss of the function will still be negligible. Therefore, the contribution of such neurons to the overall network is considered unimportant when computing gradients. By calculating the gradient of a neuron, you can learn how the function or loss function changes when the neuron is removed.

In step S4, combining the forward and backward signals to obtain the impact of the neuron on the loss or function of the given data includes the following parts:

1) For a data set with a number of M, the importance measure I _i of each neuron in the U-Net network can be obtained by the following formula (1)

where _xi is the preactivation and _δxi is its gradient. The above formula satisfies the rule that if there are zero incoming or outgoing connections, the importance should be low, otherwise the importance should be higher.

2) The importance measure I _i obtained in 1) above is similar to Taylor's first-order expansion. This metric gives less importance when the gradient is negative, however this is a problem because even if it has low importance for the gradient in backpropagation, removing it will cause a significant change in the loss function. Therefore, the importance measure I _i in 1) is squared, increasing the importance of neurons with negative gradients in backpropagation. The improved importance measure I _i is as formula (2)

shown.

in Represents the square of the importance measure I _i .

3) During the gradient calculation process, different input samples may produce gradients of different magnitudes.

Since the magnitude of the gradient is crucial for calculating importance, different input samples contribute differently to the overall importance score. To solve this problem, the gradients need to be normalized so that they have the same magnitude, usually normalized to 1.

This ensures that each data point contributes equally when calculating importance. By normalizing the gradients, gradient magnitude differences are eliminated, ensuring that each input sample has the same weight when calculating importance. This ensures that different input samples contribute fairly to the calculation of the overall importance score. Whether they are samples that produce larger gradient magnitudes or samples that produce smaller gradient magnitudes, their contributions are considered equally.

In step S5, the model parameter pruning is performed on a data set D∈[x ₀ , x ₁ ,..., x _n ] of size N. In a neural network, its input and output can be simply expressed as: y _n =f(x _n ), and the target t _n is the label that the model is expected to predict or classify. For each target t _n , calculate its gradient with respect to each input x _n , which can reveal the expected output for a given target and how much each input contributes to the output. The formula is shown in (3).

Then normalize the gradient Δy _n :

Afterwards, this gradient Backpropagation is performed to calculate the square of the importance measure I _i for each convolution kernel/>

For the above steps, this patent uses the Kvasir polyp segmentation data set for verification on the U-Net model. First, the importance score of each neuron on the given data set is calculated. Secondly, prune out the P least important neurons/channels out of the total N neurons/channels according to the importance measure (Is). By sorting the importance scores from high to low, and selecting the lowest P scores for pruning.

Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, and substitutions can be made to these embodiments without departing from the principles and spirit of the invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (9)

1. A neural network pruning method for image segmentation, characterized by: the method comprises the following steps:

s1, training a U-Net network by using a Kvasir polyp segmentation data set;

s2, calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0;

s3, calculating gradient information in the back propagation process;

s4, combining the steps S2 and S3 to obtain the importance measurement of the loss or function of the neuron on the given data;

s5, model parameter pruning.

2. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S1, a Kvasir polyp segmentation dataset is used for the segmentation task of the intestinal polyp image, the Kvasir dataset comprising intestinal polyp images from endoscopy, images of different categories including polyps and normal tissue, the resolution of the images being 1080p, size 576x576 pixels.

3. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S2, the forward signal is typically represented by a pre-activation value, when the pre-activation of the neuron is zero, which means that the input signal of the neuron is equal to zero after weighted summation of the weight and the bias, and in the neural network, the pre-activation is typically obtained by performing linear transformation on the input signal, that is, a state in which the activation function has not been applied to perform nonlinear transformation;

if the pre-activation is zero, the neuron has no effect on the output of the function, as it does not contribute to the final output;

if the input connections of the neurons are zero weight, the neurons are removed, i.e. the neurons are not significant;

if the input connection is non-zero, the neurons are of importance and the forward signal takes into account the effect of the data on a particular neuron.

4. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S3, the inverse signal is typically obtained by means of an inverse propagation loss, the degree of activation of the neurons being positive when the output connections of the neurons are zero, the effect on the function being insignificant, the gradient providing information on how the function or loss changes when the neurons are removed.

5. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S3, during the back propagation of the neural network, firstly calculating the gradient of the loss function output to the network, and then back propagating the gradient along the connection of the network, so as to calculate the gradient of each neuron; the back-propagation process gives the gradient information of the neuron, i.e. the gradient indicates the change in the loss function when the neuron is removed;

if the output connection weight of the neuron is zero, it means that it has no effect on the output of the neuron of the subsequent layer; by calculating the gradient of the neuron, the change in function or loss function when the neuron is removed is known.

6. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S4, the effect of combining the forward and backward signals to obtain the loss or function of the neuron on the given data comprises the following parts:

for a number M of data sets, an importance metric I for each neuron in the U-Net network _i Can be obtained from the following formula (1)

Wherein x is _i Is pre-activated, delta x _i Is its gradient, the above formula satisfies the rule that if the incoming or outgoing connection is zero, the importance should be low, otherwise the importance is higher.

7. A neural network pruning method for image segmentation according to claim 6, wherein: in step S4, the effect of the resulting neuron on the loss or function of the given data by combining the forward and backward signals further comprises the following:

the obtained importance measure I _i Similar to the Taylor first-order expansion, the metric gives a lower importance when the gradient is negative, the importance metric I _i Squaring to increase the importance of neurons with negative gradients in back propagation, and measuring the importance I after improvement _i As shown in formula (2):

wherein the method comprises the steps ofRepresentative importance metric I _i Square of (d).

8. A neural network pruning method for image segmentation according to claim 6, wherein: in step S4, the effect of the resulting neuron on the loss or function of the given data by combining the forward and backward signals further comprises the following:

in the process of calculating the gradient, the gradient is normalized to be 1, so that equal contribution of each data point in the process of calculating the importance is ensured, gradient amplitude difference is eliminated through the normalization of the gradient, equal weight of each input sample in the process of calculating the importance is ensured, and fair contribution of different input samples to the calculation of the overall importance score is ensured.

9. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S5, model parameters are pruned in a data set D ε [ x ] of size N ₀ ,x ₁ ,…,x _n ]Pruning is performed, and in the neural network, input and output are simply expressed as: y is _n ＝f(x _n ) Target t _n Is a label that hopes model prediction or classification, for each target t _n Calculate x with respect to each input _n Revealing the desired output for a given target, the extent of contribution of each input to the output, as shown in (3):

then the gradient deltay _n Normalization is carried out:

thereafter, gradient is carried outCounter-propagating to compute the importance metric I for each convolution kernel _i Square +.>