CN117291250A - Neural network pruning method for image segmentation - Google Patents

Neural network pruning method for image segmentation Download PDF

Info

Publication number
CN117291250A
CN117291250A CN202311156130.XA CN202311156130A CN117291250A CN 117291250 A CN117291250 A CN 117291250A CN 202311156130 A CN202311156130 A CN 202311156130A CN 117291250 A CN117291250 A CN 117291250A
Authority
CN
China
Prior art keywords
neuron
importance
gradient
neurons
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311156130.XA
Other languages
Chinese (zh)
Inventor
刘玉国
段强
张连超
姜凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Inspur Science Research Institute Co Ltd
Original Assignee
Shandong Inspur Science Research Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Inspur Science Research Institute Co Ltd filed Critical Shandong Inspur Science Research Institute Co Ltd
Priority to CN202311156130.XA priority Critical patent/CN117291250A/en
Publication of CN117291250A publication Critical patent/CN117291250A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/082Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of image segmentation, in particular to a neural network pruning method for image segmentation, which comprises the following steps: training a U-Net network using a Kvasir polyp segmentation dataset; calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0; calculating gradient information in the back propagation process; obtaining a measure of the importance of the neuron to the loss or function of the given data; model parameter pruning; the beneficial effects are as follows: combining the forward signal and the reverse signal results in a measure of importance to the given data set, and neurons having a score less than a threshold are clipped according to the measure of importance. By pruning redundant parameters, the volume of the model can be reduced, and the reasoning efficiency can be improved, so that the requirements of actual deployment can be better met.

Description

Neural network pruning method for image segmentation
Technical Field
The invention relates to the technical field of image segmentation, in particular to a neural network pruning method for image segmentation.
Background
Image segmentation is an important task in the field of computer vision, aimed at dividing pixels in an image into different regions with semantic meaning. Image segmentation plays a key role in many application fields including medical image analysis, autopilot, image editing, and augmented reality.
In the prior art, in the early stages of computer vision, image segmentation was based mainly on low-level features such as edges, textures, colors, etc. Classical algorithms include edge detection-based methods, such as the Canny edge detection algorithm, and region growth-based methods, such as the threshold-based segmentation algorithm. These methods are based primarily on heuristic rules and manually designed features. With the rapid development of deep learning, convolutional neural networks have made a significant breakthrough in the field of image segmentation. The deep learning method can automatically learn characteristic representation through end-to-end training, and obtain better performance on large-scale data. Well known deep learning methods include full convolutional networks, U-Net, mask R-CNN, and the like. The methods achieve high accuracy in image segmentation tasks and can process complex scenes and segmentation of multiple objects; in recent years, deep learning has been rapidly developed, and more scholars apply the deep learning to the field of image segmentation, wherein a segmentation model represented by a Un-Net network is most widely applied.
However, the delay of large segmentation models has been the major bottleneck in successful deployment. Although the use of smaller segmentation models is an option, this tends to sacrifice the performance of the model.
Disclosure of Invention
The invention aims to provide a neural network pruning method for image segmentation, so as to solve the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions: a neural network pruning method for image segmentation, the method comprising the steps of:
s1, training a U-Net network by using a Kvasir polyp segmentation data set;
s2, calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0;
s3, calculating gradient information in the back propagation process;
s4, combining the steps S2 and S3 to obtain the importance measurement of the loss or function of the neuron on the given data;
s5, model parameter pruning.
Preferably, in step S1, a Kvasir polyp segmentation dataset is used for the segmentation task of the intestinal polyp image, the Kvasir dataset comprising images of different classes including polyps and normal tissue from an endoscopic examination, the images having a resolution of 1080p and a size of 576x576 pixels.
Preferably, in step S2, the forward signal is generally represented by a pre-activation value, when the pre-activation of the neuron is zero, which means that the input signal of the neuron is equal to zero after weighted summation of the weight and the bias, and in the neural network, the pre-activation is generally obtained by performing linear transformation on the input signal, that is, a state in which the activation function has not been applied to perform nonlinear transformation;
if the pre-activation is zero, the neuron has no effect on the output of the function, as it does not contribute to the final output;
if the input connections of the neurons are zero weight, the neurons are removed, i.e. the neurons are not significant;
if the input connection is non-zero, the neurons are of importance and the forward signal takes into account the effect of the data on a particular neuron.
Preferably, in step S3, the inverse signal is typically obtained by means of a back propagation loss, the degree of activation of the neuron being positive when the output connection of the neuron is zero, the effect on the function being insignificant, the gradient providing information on how the function or loss changes when the neuron is removed.
Preferably, in step S3, during the back propagation of the neural network, the gradient of the loss function output to the network is calculated first, and then the gradient is propagated back along the connection of the network, so as to calculate the gradient of each neuron; the back-propagation process gives the gradient information of the neuron, i.e. the gradient indicates the change in the loss function when the neuron is removed;
if the output connection weight of the neuron is zero, it means that it has no effect on the output of the neuron of the subsequent layer; by calculating the gradient of the neuron, the change in function or loss function when the neuron is removed is known.
Preferably, in step S4, the effect of the resulting neuron on the loss or function of a given data, by combining the forward and backward signals, comprises the following parts:
for a number M of data sets, an importance metric I for each neuron in the U-Net network i Can be obtained from the following formula (1)
Wherein x is i Is pre-activated, delta x i Is its gradient, the above formula satisfies the rule that if the incoming or outgoing connection is zero, the importance should be low, otherwise the importance is higher.
Preferably, in step S4, the effect of the resulting neuron on the loss or function of a given data, in combination with the forward and backward signals, further comprises the following:
the obtained importance measure I i Similar to the Taylor first-order expansion, the metric gives a lower importance when the gradient is negative, the importance metric I i Squaring to increase the importance of neurons with negative gradients in back propagation, and measuring the importance I after improvement i As shown in formula (2):
wherein the method comprises the steps ofRepresentative importance metric I i Square of (d).
Preferably, in step S4, the effect of the resulting neuron on the loss or function of a given data, in combination with the forward and backward signals, further comprises the following:
in the process of calculating the gradient, the gradient is normalized to be 1, so that equal contribution of each data point in the process of calculating the importance is ensured, gradient amplitude difference is eliminated through the normalization of the gradient, equal weight of each input sample in the process of calculating the importance is ensured, and fair contribution of different input samples to the calculation of the overall importance score is ensured.
Preferably, in step S5, the model parameters are pruned in a data set D ε [ x ] of size N 0 ,x 1 ,…,x n ]Pruning is performed, and in the neural network, input and output are simply expressed as: y is n =f(x n ) Target t n Is a label that hopes model prediction or classification, for each target t n Calculate x with respect to each input n Revealing the desired output for a given target, the extent of contribution of each input to the output, as shown in (3):
then the gradient deltay n Normalization is carried out:
thereafter, gradient is carried outCounter-propagating to compute the importance metric I for each convolution kernel i Square +.>
Compared with the prior art, the invention has the beneficial effects that:
the neural network pruning method for image segmentation provided by the invention uses a Kvasir polyp segmentation data set to train a U-Net network; calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0; calculating gradient information in the back propagation process; combining the forward signal and the reverse signal results in a measure of importance to the given data set, and neurons having a score less than a threshold are clipped according to the measure of importance. By pruning redundant parameters, the volume of the model can be reduced, and the reasoning efficiency can be improved, so that the requirements of actual deployment can be better met.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a comparison chart of the parameter changes of the U-Net model before and after pruning according to the invention.
Detailed Description
In order to make the objects, technical solutions, and advantages of the present invention more apparent, the embodiments of the present invention will be further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are some, but not all, embodiments of the present invention, are intended to be illustrative only and not limiting of the embodiments of the present invention, and that all other embodiments obtained by persons of ordinary skill in the art without making any inventive effort are within the scope of the present invention.
Referring to fig. 1 to 2, the present invention provides a technical solution: a neural network pruning method for image segmentation, the method comprising the steps of:
s1, training a U-Net network by using a Kvasir polyp segmentation data set;
s2, calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0;
s3, calculating gradient information in the back propagation process;
s4, combining the S2 and the S3 to obtain the importance measurement of the loss or the function of the neuron on given data;
s5, model parameter pruning.
In step S1, the Kvasir polyp segmentation dataset is a dataset widely used in the field of medical image segmentation, dedicated to the segmentation task of intestinal polyp images. The dataset was developed by the research team at university of norwegian helman, aimed at facilitating the development of intestinal polyp-related studies and algorithms. The Kvasir dataset contains intestinal polyp images from endoscopy, which contains images of different categories including polyps and normal tissue. The resolution of the image is 1080p, with a size of 576x576 pixels. The data set has been widely used for training and evaluation of deep learning and computer vision algorithms to enhance intestinal polyp detection
In step S2, the forward signal is typically represented by a pre-activation value. When the pre-activation of a neuron is zero, this means that the input signal of the neuron is zero after weighted summation of weights and offsets. In neural networks, pre-activation is typically obtained by linear transformation of the input signal, i.e. the state in which the activation function has not been applied for non-linear transformation. If the pre-activation is zero, then the neuron has no effect on the output of the function, as it does not contribute to the final output. Furthermore, if the input connection of a neuron is zero weight, that neuron can be removed, i.e., the neuron is not of importance. Neurons are of importance if the input connection is non-zero. The forward signal takes into account the effect of the data on a particular neuron.
In step S3, the reverse signal is typically obtained by reverse propagation loss. When the output connection of a neuron is zero, even if the degree of activation of the neuron is positive, its effect on the function is insignificant. The gradient provides information on how the function or loss changes when the neuron is removed.
In the back propagation of the neural network, the gradient of the loss function output to the network is first calculated, and then this gradient is back propagated along the network connections, thereby calculating the gradient of each neuron. This back-propagation process may give the gradient information of the neuron, i.e. the gradient indicates the change in the loss function when the neuron is removed. If the output connection weight of a neuron is zero, it means that it has no effect on the output of the subsequent layer neurons. Even though the neuron has a positive degree of activation, its impact on the outcome or loss of function is still insignificant. Thus, the contribution of such neurons to the overall network is considered unimportant when calculating the gradient. By calculating the gradient of the neuron, the change in function or loss function when the neuron is removed can be understood.
In step S4, the combination of the forward and backward signals yields the effect of the neuron on the loss or function of the given data comprising the following parts:
1) For a number M of data sets, an importance metric I for each neuron in the U-Net network i Can be obtained from the following formula (1)
Wherein x is i Is pre-activated, delta x i Is its gradient. The above formula satisfies the rule that if the incoming or outgoing connection is zero, the importance should be low, otherwise the importance is higher.
2) Importance measure I obtained in 1) above i Similar to taylor first order expansion. This metric gives less importance when the gradient is negative, however this is a problem because even if it is of less importance for the gradient in back propagation, its removal results in a significant change in the loss function. Thus, for importance measure I in 1) i Squaring was performed to increase the importance of neurons with negative gradients in the back propagation. Improved importance measure I i Such as formula (2)
As shown.
Wherein the method comprises the steps ofRepresentative importance metric I i Square (square).
3) In calculating the gradient, different input samples may produce gradients of different magnitudes.
Since the magnitude of the gradient is critical to computing importance, the contribution of different input samples to the overall importance score is different. To solve this problem, the gradient needs to be normalized to have the same magnitude, usually normalized to 1.
This ensures that each data point has equal contribution in calculating importance. By normalizing the gradient, the gradient magnitude difference can be eliminated, ensuring that each input sample has the same weight when calculating importance. This ensures that different input samples have a fair contribution to the calculation of the overall importance score. Whether samples with larger or smaller gradient magnitudes are produced, their contributions are equally considered.
In step S5, the model parameter pruning is performed on a data set D E [ x ] with the size of N 0 ,x 1 ,…,x n ]Trimming. In a neural network, its input and output can be simply expressed as: y is n =f(x n ) Target t n Is a label for which model prediction or classification is desired. For each target t n Calculate it relative to each input x n Which reveals the desired output for a given target, the extent to which each input contributes to the output. The formula is shown as (3).
Then the gradient deltay n Normalization is carried out:
thereafter, the gradient isCounter-propagating to compute the importance metric I for each convolution kernel i Square +.>
For the above steps, the present patent verifies on the U-Net model using the Kvasir polyp segmentation dataset, first, calculating the importance score for each neuron on a given dataset. Second, P least significant neurons/channels out of a total of N neurons/channels are pruned according to the importance metric (Is). Pruning is performed by ordering the importance scores from high to low and selecting the lowest P scores.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (9)

1. A neural network pruning method for image segmentation, characterized by: the method comprises the following steps:
s1, training a U-Net network by using a Kvasir polyp segmentation data set;
s2, calculating the activation degree of the neurons in the forward propagation process, and finding out the neurons with the activation value of 0;
s3, calculating gradient information in the back propagation process;
s4, combining the steps S2 and S3 to obtain the importance measurement of the loss or function of the neuron on the given data;
s5, model parameter pruning.
2. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S1, a Kvasir polyp segmentation dataset is used for the segmentation task of the intestinal polyp image, the Kvasir dataset comprising intestinal polyp images from endoscopy, images of different categories including polyps and normal tissue, the resolution of the images being 1080p, size 576x576 pixels.
3. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S2, the forward signal is typically represented by a pre-activation value, when the pre-activation of the neuron is zero, which means that the input signal of the neuron is equal to zero after weighted summation of the weight and the bias, and in the neural network, the pre-activation is typically obtained by performing linear transformation on the input signal, that is, a state in which the activation function has not been applied to perform nonlinear transformation;
if the pre-activation is zero, the neuron has no effect on the output of the function, as it does not contribute to the final output;
if the input connections of the neurons are zero weight, the neurons are removed, i.e. the neurons are not significant;
if the input connection is non-zero, the neurons are of importance and the forward signal takes into account the effect of the data on a particular neuron.
4. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S3, the inverse signal is typically obtained by means of an inverse propagation loss, the degree of activation of the neurons being positive when the output connections of the neurons are zero, the effect on the function being insignificant, the gradient providing information on how the function or loss changes when the neurons are removed.
5. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S3, during the back propagation of the neural network, firstly calculating the gradient of the loss function output to the network, and then back propagating the gradient along the connection of the network, so as to calculate the gradient of each neuron; the back-propagation process gives the gradient information of the neuron, i.e. the gradient indicates the change in the loss function when the neuron is removed;
if the output connection weight of the neuron is zero, it means that it has no effect on the output of the neuron of the subsequent layer; by calculating the gradient of the neuron, the change in function or loss function when the neuron is removed is known.
6. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S4, the effect of combining the forward and backward signals to obtain the loss or function of the neuron on the given data comprises the following parts:
for a number M of data sets, an importance metric I for each neuron in the U-Net network i Can be obtained from the following formula (1)
Wherein x is i Is pre-activated, delta x i Is its gradient, the above formula satisfies the rule that if the incoming or outgoing connection is zero, the importance should be low, otherwise the importance is higher.
7. A neural network pruning method for image segmentation according to claim 6, wherein: in step S4, the effect of the resulting neuron on the loss or function of the given data by combining the forward and backward signals further comprises the following:
the obtained importance measure I i Similar to the Taylor first-order expansion, the metric gives a lower importance when the gradient is negative, the importance metric I i Squaring to increase the importance of neurons with negative gradients in back propagation, and measuring the importance I after improvement i As shown in formula (2):
wherein the method comprises the steps ofRepresentative importance metric I i Square of (d).
8. A neural network pruning method for image segmentation according to claim 6, wherein: in step S4, the effect of the resulting neuron on the loss or function of the given data by combining the forward and backward signals further comprises the following:
in the process of calculating the gradient, the gradient is normalized to be 1, so that equal contribution of each data point in the process of calculating the importance is ensured, gradient amplitude difference is eliminated through the normalization of the gradient, equal weight of each input sample in the process of calculating the importance is ensured, and fair contribution of different input samples to the calculation of the overall importance score is ensured.
9. A neural network pruning method for image segmentation according to claim 1, characterized in that: in step S5, model parameters are pruned in a data set D ε [ x ] of size N 0 ,x 1 ,…,x n ]Pruning is performed, and in the neural network, input and output are simply expressed as: y is n =f(x n ) Target t n Is a label that hopes model prediction or classification, for each target t n Calculate x with respect to each input n Revealing the desired output for a given target, the extent of contribution of each input to the output, as shown in (3):
then the gradient deltay n Normalization is carried out:
thereafter, gradient is carried outCounter-propagating to compute the importance metric I for each convolution kernel i Square +.>
CN202311156130.XA 2023-09-08 2023-09-08 Neural network pruning method for image segmentation Pending CN117291250A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311156130.XA CN117291250A (en) 2023-09-08 2023-09-08 Neural network pruning method for image segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311156130.XA CN117291250A (en) 2023-09-08 2023-09-08 Neural network pruning method for image segmentation

Publications (1)

Publication Number Publication Date
CN117291250A true CN117291250A (en) 2023-12-26

Family

ID=89247227

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311156130.XA Pending CN117291250A (en) 2023-09-08 2023-09-08 Neural network pruning method for image segmentation

Country Status (1)

Country Link
CN (1) CN117291250A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117808072A (en) * 2024-02-28 2024-04-02 苏州元脑智能科技有限公司 Model pruning method, image processing method, device, equipment and medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117808072A (en) * 2024-02-28 2024-04-02 苏州元脑智能科技有限公司 Model pruning method, image processing method, device, equipment and medium
CN117808072B (en) * 2024-02-28 2024-05-14 苏州元脑智能科技有限公司 Model pruning method, image processing method, device, equipment and medium

Similar Documents

Publication Publication Date Title
CN109977918B (en) Target detection positioning optimization method based on unsupervised domain adaptation
WO2021184817A1 (en) Method for segmenting liver and focus thereof in medical image
CN109583501B (en) Method, device, equipment and medium for generating image classification and classification recognition model
CN109784283B (en) Remote sensing image target extraction method based on scene recognition task
CN112184752A (en) Video target tracking method based on pyramid convolution
CN110599537A (en) Mask R-CNN-based unmanned aerial vehicle image building area calculation method and system
CN107481188A (en) A kind of image super-resolution reconstructing method
CN110349185B (en) RGBT target tracking model training method and device
CN112330684B (en) Object segmentation method and device, computer equipment and storage medium
CN107945210B (en) Target tracking method based on deep learning and environment self-adaption
CN112541532B (en) Target detection method based on dense connection structure
CN113706581B (en) Target tracking method based on residual channel attention and multi-level classification regression
CN112446891A (en) Medical image segmentation method based on U-Net network brain glioma
CN111145145B (en) Image surface defect detection method based on MobileNet
CN117291250A (en) Neural network pruning method for image segmentation
CN113158789B (en) Target detection method, system, device and medium for remote sensing image
CN116503399B (en) Insulator pollution flashover detection method based on YOLO-AFPS
CN117038055B (en) Pain assessment method, system, device and medium based on multi-expert model
CN113450313A (en) Image significance visualization method based on regional contrast learning
CN111739037A (en) Semantic segmentation method for indoor scene RGB-D image
CN117611599B (en) Blood vessel segmentation method and system integrating centre line diagram and contrast enhancement network
Kiratiratanapruk et al. Automatic detection of rice disease in images of various leaf sizes
CN114913337A (en) Camouflage target frame detection method based on ternary cascade perception
CN112801092B (en) Method for detecting character elements in natural scene image
CN114202473A (en) Image restoration method and device based on multi-scale features and attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination