CN115953652B - Method, device, equipment and medium for pruning target detection network batch normalization layer - Google Patents
Method, device, equipment and medium for pruning target detection network batch normalization layer Download PDFInfo
- Publication number
- CN115953652B CN115953652B CN202310244347.XA CN202310244347A CN115953652B CN 115953652 B CN115953652 B CN 115953652B CN 202310244347 A CN202310244347 A CN 202310244347A CN 115953652 B CN115953652 B CN 115953652B
- Authority
- CN
- China
- Prior art keywords
- target detection
- channel
- pruning
- normalization layer
- batch normalization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 183
- 238000010606 normalization Methods 0.000 title claims abstract description 119
- 238000013138 pruning Methods 0.000 title claims abstract description 107
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 104
- 238000004140 cleaning Methods 0.000 claims abstract description 53
- 230000006870 function Effects 0.000 claims description 22
- 230000005540 biological transmission Effects 0.000 claims description 18
- 230000007547 defect Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 7
- 238000007689 inspection Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002950 deficient Effects 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 238000009966 trimming Methods 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 8
- 238000002372 labelling Methods 0.000 description 8
- 238000012360 testing method Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a target detection network batch normalization layer pruning method, device, equipment and medium, which are used for solving the technical problem that the existing target detection network pruning algorithm has larger precision loss. The invention comprises the following steps: collecting image data; cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data; training a preset target detection model by adopting the training set to obtain a pre-trained target detection model; sparsifying the pre-training target detection model to obtain a sparsified target detection network; and determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model.
Description
Technical Field
The present invention relates to the field of pruning technologies, and in particular, to a method, an apparatus, a device, and a medium for pruning a target detection network batch normalization layer.
Background
With the development of the deep learning network for several years, the detection precision of the target detection algorithm based on the deep learning is far superior to that of the traditional algorithm, but the deep learning network has higher requirements on the computing and storage capacities of hardware equipment. Object detection is an important branch in the current computer vision field, has extremely wide application scenes, and has more parameters and more complex model structure compared with a classification network. The model compression and pruning technology reduces the parameter quantity and floating point calculation amount of the model while maintaining the original network precision as much as possible, so that the network can run efficiently with low delay on various hardware platforms, thereby being more close to the requirements of actual scenes.
The residual structure well solves the problem of deep network gradient disappearance, and is widely used in backbone networks for various tasks nowadays. However, due to its special network structure, there is a certain link between the channels of different layers, which makes pruning of the network difficult. The pruning algorithm based on the batch normalization layer is a very convenient channel pruning method, and the importance of each channel is measured by using the channel scaling factor of the batch normalization layer, and when the scaling factor is smaller, the importance of the channel can be described as smaller, so that the channel can be pruned. However, the pruning method ignores the bias term of the batch normalization layer, and when the bias is large, the bias is directly subtracted from the channel, so that large precision loss is caused.
Disclosure of Invention
The invention provides a target detection network batch normalization layer pruning method, device, equipment and medium, which are used for solving the technical problem that the existing target detection network pruning algorithm has larger precision loss.
The invention provides a target detection network batch normalization layer pruning method, which comprises the following steps:
collecting image data;
cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data;
training a preset target detection model by adopting the training set to obtain a pre-trained target detection model;
sparsifying the pre-training target detection model to obtain a sparsified target detection network;
and determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model.
Optionally, the step of acquiring image data includes:
acquiring image information acquired by a preset power transmission line for inspection of the unmanned aerial vehicle;
acquiring an image to be detected of a target from the image information;
acquiring a defect detection data set of a power transmission line;
and marking the image to be detected by adopting the transmission line defect detection data set to obtain image data.
Optionally, the step of cleaning the image data to obtain cleaning data and generating a training set by using the cleaning data includes:
cleaning abnormal data in the image data to obtain cleaning data; the cleaning data comprises normal equipment data and defective equipment data;
the proportion of the normal equipment data and the defect equipment data is adjusted to obtain sample data;
and extracting training data from the sample data according to a preset proportion to generate a training set.
Optionally, the step of thinning the pre-training target detection model to obtain a thinned target detection network includes:
obtaining a scaling factor of a batch normalization layer of the pre-training target detection model;
regularizing the scaling factor to obtain a regularized scaling factor;
and generating a sparse target detection network by adopting the regularized scaling factor.
Optionally, the step of determining the to-be-pruned channel of the batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain the target detection model includes:
determining a to-be-pruned channel of the sparse target detection network batch normalization layer, and generating a pseudo pruning model of the sparse target detection network according to the to-be-pruned channel;
fine tuning the pseudo pruning model to generate an adjustment model;
and merging the bias of the channels to be pruned in the batch normalization layer of the adjustment model into the next batch normalization layer of the batch normalization layer, and pruning the channels to be pruned to obtain the target detection model.
Optionally, the step of determining a to-be-pruned channel of the sparse target detection network and generating a pseudo pruning model of the sparse target detection network according to the to-be-pruned channel includes:
determining a channel corresponding to a regularized scaling factor with a value larger than a preset parameter threshold as a reserved channel, and calculating the channel number of the reserved channel;
adjusting the channel number of the reserved channels to be a multiple of 8 to obtain an adjusted channel number;
all regularized scaling is arranged according to the order from big to small, and an arrangement order is obtained;
determining a target parameter threshold by adopting the arrangement sequence and the adjustment channel number;
determining a channel corresponding to the regularized scaling factor with the value not larger than the target parameter threshold as a channel to be pruned;
and setting the mask of the channel to be pruned to 0 to obtain a pseudo pruning model.
Optionally, the step of fine tuning the pseudo pruning model to generate an adjustment model includes:
acquiring a loss function of the pre-training target detection model;
and fine tuning the pseudo pruning model by adopting the loss function to generate an adjustment model.
The invention also provides a target detection network batch normalization layer pruning device, which comprises:
the image data acquisition module is used for acquiring image data;
the cleaning module is used for cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data;
the training module is used for training a preset target detection model by adopting the training set to obtain a pre-training target detection model;
the sparsification module is used for sparsifying the pre-training target detection model to obtain a sparsified target detection network;
and the pruning module is used for determining the to-be-pruned channels of the batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channels in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channels to obtain a target detection model.
The invention also provides an electronic device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the target detection network batch normalization layer pruning method according to any one of the above claims according to instructions in the program code.
The invention also provides a computer readable storage medium for storing program code for performing the object detection network batch normalization layer pruning method according to any one of the above.
From the above technical scheme, the invention has the following advantages: the invention provides a target detection network batch normalization layer pruning method, which comprises the following steps: collecting image data; cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data; training a preset target detection model by adopting a training set to obtain a pre-trained target detection model; sparsifying the pre-training target detection model to obtain a sparsified target detection network; and determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in batch normalization into a next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model. According to the method, the pre-training target detection model is thinned, the channel to be pruned is extracted from the thinned target detection network, and the bias of the channel to be pruned of the batch normalization layer is combined into the next batch normalization layer, so that the influence of the bias is reduced in the pruning process, and the precision loss in the pruning process is reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from these drawings without inventive faculty for a person skilled in the art.
Fig. 1 is a flowchart of steps of a target detection network batch normalization layer pruning method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of a method for pruning a target detection network batch normalization layer according to another embodiment of the present invention;
fig. 3 is a block diagram of a target detection network batch normalization layer pruning device according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a target detection network batch normalization layer pruning method, device, equipment and medium, which are used for solving the technical problem that the existing target detection network pruning algorithm has larger precision loss.
In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions in the embodiments of the present invention are described in detail below with reference to the accompanying drawings, and it is apparent that the embodiments described below are only some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a target detection network batch normalization layer pruning method according to an embodiment of the present invention.
The invention provides a target detection network batch normalization layer pruning method, which specifically comprises the following steps:
in the embodiment of the invention, the image information can be acquired by the unmanned aerial vehicle inspection transmission line, and the required image to be detected containing the target to be detected is selected in a manual sorting mode; and then acquiring a transmission line defect detection data set, and marking normal equipment and defect equipment in the image to be detected by using a marking tool, wherein the marked information can comprise category, name and position information. The location information may include Xmin (X coordinate of upper left corner of the labeling frame), ymin (Y coordinate of upper left corner of the labeling frame), xmax (X coordinate of lower right corner of the labeling frame), and Ymax (Y coordinate of lower right corner of the labeling frame), and be stored as xml tag file in VOC data format.
102, cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data;
after the image data is obtained, the image data can be cleaned to obtain cleaning data, and a training set, a verification set and a test set are generated by adopting the cleaning data. Wherein the ratio of the training set, the validation set and the test set may be 3:1:1.
after the training set is obtained, the training set may be used to train the preset target detection model to obtain a pre-trained target detection model.
after the pre-training target detection model is obtained, the pre-training target detection model can be subjected to sparsification processing, and a sparsified target detection network is obtained.
In the embodiment of the invention, after the sparsification of the pre-training target detection network is completed, the to-be-pruned channels of the batch normalization layer of the sparsified target detection network can be determined, the bias of the to-be-pruned channels in the batch normalization layer is combined into the next batch normalization layer of the batch normalization layer, and the to-be-pruned channels are pruned to obtain the target detection model.
According to the method, the pre-training target detection model is thinned, the channel to be pruned is extracted from the thinned target detection network, and the bias of the channel to be pruned of the batch normalization layer is combined into the next batch normalization layer, so that the influence of the bias is reduced in the pruning process, and the precision loss in the pruning process is reduced.
Referring to fig. 2, fig. 2 is a flowchart illustrating steps of a target detection network batch normalization layer pruning method according to another embodiment of the present invention. The method specifically comprises the following steps:
in the embodiment of the invention, the image information can be acquired by the unmanned aerial vehicle inspection transmission line, and the required image to be detected containing the target to be detected is selected in a manual sorting mode; and then acquiring a transmission line defect detection data set, and marking normal equipment and defect equipment in the image to be detected by using a marking tool, wherein the marked information can comprise category, name and position information. The location information may include Xmin (X coordinate of upper left corner of the labeling frame), ymin (Y coordinate of upper left corner of the labeling frame), xmax (X coordinate of lower right corner of the labeling frame), and Ymax (Y coordinate of lower right corner of the labeling frame), and be stored as xml tag file in VOC data format.
after the image data is obtained, the image data can be cleaned to obtain cleaning data, and a training set, a verification set and a test set are generated by adopting the cleaning data. Wherein the ratio of the training set, the validation set and the test set may be 3:1:1.
In the embodiment of the present invention, the step of cleaning the image data to obtain cleaning data and generating the training set by using the cleaning data may include the following sub-steps:
s51, cleaning abnormal data in the image data to obtain cleaning data; the cleaning data comprises normal equipment data and defective equipment data;
s52, adjusting the proportion of normal equipment data and defect equipment data to obtain sample data;
and S53, extracting training data from the sample data according to a preset proportion, and generating a training set.
In particular implementations, the anomaly data may include data that is abnormal in brightness, noisy, and blurred in images. After the abnormal data is cleaned, the normal equipment data and the abnormal equipment data can be adjusted to be in line with the ratio of 1:1 as much as possible, and then the training set, the verification set and the test set are divided from the image data according to the number ratio of 3:1:1.
after the training set is obtained, the training set may be used to train the preset target detection model to obtain a pre-trained target detection model.
In a specific implementation, training of the RetinaNet detection network can be performed by using the mobilenet v2 and the ResNet50 pre-trained on ImageNet as backbone networks, and the loss function of the training network is as follows:
wherein ,
for the Focal loss, from the viewpoint of sample difficulty classification, the model training problem caused by sample unbalance is solved, and the corresponding formula of the Focal loss is as follows:
wherein ,
，/>
for a predetermined constant, ++>
The confidence of the network to the prediction of the training samples in the training process is obtained.
wherein ,
and (3) for pre-training the predicted value of the target detection model, wherein y is the true value of the sample.
The Loss function combines the advantages of both L1 Loss and L2 Loss, i.e., a smooth L2 Loss is used when x (representing the characteristics of any image in the training set in the input network) is small, and a stable L1 Loss is used when x is large.
after the pre-training target detection model is obtained, the pre-training target detection model can be subjected to sparsification processing, and a sparsified target detection network is obtained.
In one example, the step of sparsifying the pre-trained target detection model to obtain a sparse target detection network may comprise the sub-steps of:
s71, obtaining scaling factors of a batch normalization layer of a pre-training target detection model;
s72, regularizing the scaling factors to obtain regularized scaling factors;
s73, generating a sparse target detection network by adopting a regularized scaling factor.
In a specific implementation, for a pre-trained target detection model, scaling factors of batch normalization of a backbone network and a detection head can be regularized to trend to 0, and regularization losses are calculated as follows:
wherein ,
representing a total regularization of the sparse training; />
A canonical loss of batch normalization scaling factor L1 representing the non-output layer of the residual module; />
A Group Lasso regularization loss of the residual module output layer is represented; />
Scaling factor L1 regularization loss representing a detection head portion BN layer; />
、/>
、/>
Each representing the coefficients of three regularization losses.
And then generating an adjustment pre-training target detection model by adopting the regularized scaling factor to generate a sparse target detection network.
And step 208, determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain the target detection model.
In the embodiment of the invention, after the sparsification of the pre-training target detection network is completed, the to-be-pruned channels of the batch normalization layer of the sparsified target detection network can be determined, the bias of the to-be-pruned channels in the batch normalization layer is combined into the next batch normalization layer of the batch normalization layer, and the to-be-pruned channels are pruned to obtain the target detection model.
In one example, the step of determining a to-be-pruned channel of a batch normalization layer of a sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into a next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model may include the sub-steps of:
s81, determining a to-be-pruned channel of a sparse target detection network batch normalization layer, and generating a pseudo pruning model of the sparse target detection network according to the to-be-pruned channel;
in an embodiment of the present invention, step S81 may include the following sub-steps:
s811, determining a channel corresponding to a regularized scaling factor with a value larger than a preset parameter threshold as a reserved channel, and calculating the number of channels of the reserved channel;
s812, adjusting the channel number of the reserved channels to be a multiple of 8 to obtain an adjusted channel number;
s813, arranging all regularized scaling according to the sequence from large to small to obtain an arrangement sequence;
s814, determining a target parameter threshold by adopting the arrangement sequence and the number of the adjustment channels;
s815, determining a channel corresponding to the regularized scaling factor with the value not larger than the target parameter threshold as a channel to be pruned;
s816, setting the mask of the channel to be pruned to 0 to obtain a pseudo pruning model.
In the embodiment of the invention, after the sparse target detection network is acquired, pseudo pruning operation can be performed on the sparse target detection network so as to determine a channel needing pruning.
In a specific implementation, the pseudo pruning operation is to change the scaling factor towards 0 to 0 directly, keep the training from updating, and save the configuration for fine tuning, in which case the bias in the fine tuning process will also be trained as a parameter following the network. The embodiment of the invention can realize pseudo pruning operation by adding the mask to the channel, and the batch normalization layer PBN formula after pseudo pruning is as follows:
wherein ,meanrepresenting the mean value of the features of the whole training set image;biasan offset term representing the image feature that passes into the PBN layer.
When the regularization scaling factor scale tends to 0, the corresponding channel mask is changed to 0, so that the characteristics of the channel do not have any influence on output in the forward propagation process, which is equivalent to scale being 0, and in the backward propagation process, the gradient cannot be transferred to the regularization scaling factor due to mask being 0, namely the regularization scaling factor is kept unchanged all the time in the training process.
It is noted that the pseudo pruning cannot achieve the purpose of model compression, and the number of parameters and calculation amount of the network are not reduced. When generating the mask, the same threshold value is used for multiple layers of the network, so that the number of channels of each layer after pruning is uncertain, and the number of the reserved channels after pruning is ensured to be a multiple of 8 in order to really achieve the aim of acceleration in consideration of the characteristics of the current computer architecture. Therefore, for a scale parameter of a certain to-be-pruned batch normalization layer, a preset parameter threshold is set as t, and then:
mask=1（scale>t）
at this time, the number of channels of the element (reserved channel) of 1 in the mask is not necessarily a multiple of 8, so the number of channels needs to be adjusted to obtain a new number of channels (adjusted number of channels) channel_num8 as:
wherein ,
representing a summation function; />
Representing a rounding function. The new threshold (target parameter threshold) is:
wherein ,
scale is ordered from large to small for the ordering function.
After the target parameter threshold is obtained, a channel corresponding to the regularized scaling factor with the value not larger than the target parameter threshold can be used as a channel to be pruned, and then the mask of the channel to be pruned is set to 0 to obtain the pseudo pruning model.
S82, fine tuning is carried out on the pseudo pruning model to generate an adjustment model;
in an embodiment of the present invention, step S82 may include the following sub-steps:
s821, obtaining a loss function of the pre-training target detection model;
s822, fine tuning is performed on the pseudo pruning model by adopting the loss function, and an adjustment model is generated.
In the embodiment of the invention, the pseudo pruning model can be finely adjusted through the loss function of the pre-training target detection model so as to compensate the precision loss caused by sparse pruning. Wherein fine tuning of the pseudo pruning model may be achieved by reducing the loss function lfinetene:
s83, merging the bias of the channels to be pruned in the batch normalization layer of the adjustment model into the next batch normalization layer of the batch normalization layer, and pruning the channels to be pruned to obtain the target detection model.
In the embodiment of the invention, after fine adjustment of the pseudo pruning model is completed, the offset corresponding to the channel with the mask of 0 of the adjustment model can be combined into the next batch normalization layer, then the channel with the mask of 0 is directly pruned, and the parameter quantity of the obtained target detection model is smaller than that of the pre-training target detection network.
Further, in the embodiment of the invention, mAP can be used as a main evaluation index of the target detection network, meanwhile, the parameter quantity and the floating point calculation quantity of the network are considered, the target detection network subjected to pruning is tested by using a test set, the performance of an algorithm is evaluated by using mAP performance indexes, and the target detection network with better comprehensive performance of the detection effect of the test set in the defect detection data set of the power transmission line is obtained through multiple tests.
According to the method, the pre-training target detection model is thinned, the channel to be pruned is extracted from the thinned target detection network, and the bias of the channel to be pruned of the batch normalization layer is combined into the next batch normalization layer, so that the influence of the bias is reduced in the pruning process, and the precision loss in the pruning process is reduced.
Referring to fig. 3, fig. 3 is a block diagram of a target detection network batch normalization layer pruning device according to an embodiment of the present invention.
The embodiment of the invention provides a target detection network batch normalization layer pruning device, which comprises the following steps:
an image data acquisition module 301, configured to acquire image data;
the training set generating module 302 is configured to clean the image data to obtain cleaning data, and generate a training set by using the cleaning data;
the training module 303 is configured to train the preset target detection model by using a training set to obtain a pre-trained target detection model;
the sparsification module 304 is configured to sparsify the pre-training target detection model to obtain a sparsified target detection network;
the pruning module 305 is configured to determine a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merge the bias of the to-be-pruned channel in the batch normalization layer into a next batch normalization layer of the batch normalization layer, and prune the to-be-pruned channel to obtain the target detection model.
In an embodiment of the present invention, the image data acquisition module 301 includes:
the image information acquisition sub-module is used for acquiring image information acquired by the unmanned aerial vehicle inspection preset power transmission line;
the image to be detected acquires a sub-module for acquiring an image to be detected from the image information;
the power transmission line defect detection data set acquisition sub-module is used for acquiring a power transmission line defect detection data set;
and the marking sub-module is used for marking the image to be detected by adopting the transmission line defect detection data set to obtain image data.
In an embodiment of the present invention, the training set generating module 302 includes:
the cleaning submodule is used for cleaning abnormal data in the image data to obtain cleaning data; the cleaning data comprises normal equipment data and defective equipment data;
the proportion adjustment sub-module is used for adjusting the proportion of normal equipment data and defect equipment data to obtain sample data;
the training set generation sub-module is used for extracting training data from the sample data according to a preset proportion to generate a training set.
In an embodiment of the present invention, the sparsification module 304 includes:
the scaling factor obtaining sub-module is used for obtaining scaling factors of a batch normalization layer of the pre-training target detection model;
the regularization submodule is used for regularizing the scaling factors to obtain regularized scaling factors;
and the sparse target detection network generation sub-module is used for generating a sparse target detection network by adopting the regularized scaling factor.
In an embodiment of the present invention, pruning module 305 includes:
the pseudo pruning sub-module is used for determining a to-be-pruned channel of the thinned target detection network batch normalization layer and generating a pseudo pruning model of the thinned target detection network according to the to-be-pruned channel;
the fine adjustment sub-module is used for fine adjustment of the pseudo pruning model to generate an adjustment model;
and the pruning sub-module is used for merging the offset of the channels to be pruned in the batch normalization layer of the adjustment model into the next batch normalization layer of the batch normalization layer, pruning the channels to be pruned, and obtaining the target detection model.
In an embodiment of the present invention, the pseudo pruning submodule includes:
the channel number calculation unit is used for determining a channel corresponding to a regularized scaling factor with a value larger than a preset parameter threshold as a reserved channel and calculating the channel number of the reserved channel;
the adjusting channel number calculating unit is used for adjusting the channel number of the reserved channels to be a multiple of 8 to obtain an adjusting channel number;
the ordering unit is used for ordering all regularized scaling according to the order from big to small to obtain an ordering order;
a target parameter threshold determining unit, configured to determine a target parameter threshold by using the arrangement order and the adjustment channel number;
the to-be-pruned channel determining unit is used for determining a channel corresponding to a regularized scaling factor with the value not larger than a target parameter threshold value as the to-be-pruned channel;
and the pseudo pruning unit is used for setting the mask of the pruning channel to be pruned to 0 to obtain a pseudo pruning model.
In an embodiment of the present invention, a trimming sub-module includes:
the loss function acquisition unit is used for acquiring a loss function of the pre-training target detection model;
and the adjustment model generation unit is used for fine-tuning the pseudo pruning model by adopting the loss function to generate an adjustment model.
The embodiment of the invention also provides electronic equipment, which comprises a processor and a memory:
the memory is used for storing the program codes and transmitting the program codes to the processor;
the processor is used for executing the target detection network batch normalization layer pruning method according to the instructions in the program codes.
The invention also provides a computer readable storage medium, which is used for storing program codes, and the program codes are used for executing the target detection network batch normalization layer pruning method.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. The target detection network batch normalization layer pruning method is characterized by comprising the following steps of:
collecting image data;
cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data;
training a preset target detection model by adopting the training set to obtain a pre-trained target detection model;
sparsifying the pre-training target detection model to obtain a sparsified target detection network;
determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into a next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model;
the step of determining the to-be-pruned channels of the batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channels in the batch normalization layer into the next batch normalization layer of the batch normalization layer, pruning the to-be-pruned channels to obtain a target detection model, and the method comprises the following steps:
determining a to-be-pruned channel of the sparse target detection network batch normalization layer, and generating a pseudo pruning model of the sparse target detection network according to the to-be-pruned channel;
fine tuning the pseudo pruning model to generate an adjustment model;
merging the bias of the channel to be pruned in the batch normalization layer of the adjustment model into the next batch normalization layer of the batch normalization layer, pruning the channel to be pruned, and obtaining a target detection model;
the step of determining a to-be-pruned channel of the sparse target detection network and generating a pseudo pruning model of the sparse target detection network according to the to-be-pruned channel comprises the following steps:
determining a channel corresponding to a regularized scaling factor with a value larger than a preset parameter threshold as a reserved channel, and calculating the channel number of the reserved channel;
adjusting the channel number of the reserved channels to be a multiple of 8 to obtain an adjusted channel number;
arranging all regularized scaling factors according to the sequence from big to small to obtain an arrangement sequence;
determining a target parameter threshold by adopting the arrangement sequence and the adjustment channel number;
determining a channel corresponding to the regularized scaling factor with the value not larger than the target parameter threshold as a channel to be pruned;
setting 0 to the mask of the channel to be pruned to obtain a pseudo pruning model;
the loss function of the training preset target detection model is as follows:
wherein ,
for Focal loss, the corresponding formula for Focal loss is as follows:
wherein ,
gamma is a preset constant, and p is the prediction confidence of the network to the training sample in the training process;
wherein ,
and (3) for pre-training the predicted value of the target detection model, wherein y is the true value of the sample.
2. The method of claim 1, wherein the step of acquiring image data comprises:
acquiring image information acquired by a preset power transmission line for inspection of the unmanned aerial vehicle;
acquiring an image to be detected of a target from the image information;
acquiring a defect detection data set of a power transmission line;
and marking the image to be detected by adopting the transmission line defect detection data set to obtain image data.
3. The method of claim 1, wherein the step of cleaning the image data to obtain cleaning data and generating a training set using the cleaning data comprises:
cleaning abnormal data in the image data to obtain cleaning data; the cleaning data comprises normal equipment data and defective equipment data;
the proportion of the normal equipment data and the defect equipment data is adjusted to obtain sample data;
and extracting training data from the sample data according to a preset proportion to generate a training set.
4. The method of claim 1, wherein the step of sparsifying the pre-trained target detection model results in a sparsified target detection network comprising:
obtaining a scaling factor of a batch normalization layer of the pre-training target detection model;
regularizing the scaling factor to obtain a regularized scaling factor;
and generating a sparse target detection network by adopting the regularized scaling factor.
5. The method of claim 1, wherein the step of trimming the pseudo pruning model to generate an adjusted model comprises:
acquiring a loss function of the pre-training target detection model;
and fine tuning the pseudo pruning model by adopting the loss function to generate an adjustment model.
6. The utility model provides a target detection network batched normalization layer pruning device which characterized in that includes:
the image data acquisition module is used for acquiring image data;
the training set generation module is used for cleaning the image data to obtain cleaning data, and generating a training set by adopting the cleaning data;
the training module is used for training a preset target detection model by adopting the training set to obtain a pre-training target detection model;
the sparsification module is used for sparsifying the pre-training target detection model to obtain a sparsified target detection network;
the pruning module is used for determining a to-be-pruned channel of a batch normalization layer of the sparse target detection network, merging the bias of the to-be-pruned channel in the batch normalization layer into the next batch normalization layer of the batch normalization layer, and pruning the to-be-pruned channel to obtain a target detection model;
wherein, pruning module includes:
the pseudo pruning sub-module is used for determining a to-be-pruned channel of the thinned target detection network batch normalization layer and generating a pseudo pruning model of the thinned target detection network according to the to-be-pruned channel;
the fine adjustment sub-module is used for fine adjustment of the pseudo pruning model to generate an adjustment model;
the pruning sub-module is used for merging the offset of the channels to be pruned in the batch normalization layer of the adjustment model into the next batch normalization layer of the batch normalization layer, pruning the channels to be pruned, and obtaining a target detection model;
wherein, pseudo-branch trimming submodule includes:
the channel number calculation unit is used for determining a channel corresponding to a regularized scaling factor with a value larger than a preset parameter threshold as a reserved channel and calculating the channel number of the reserved channel;
the adjusting channel number calculating unit is used for adjusting the channel number of the reserved channels to be a multiple of 8 to obtain an adjusting channel number;
the ordering unit is used for arranging all regularized scaling factors according to the sequence from big to small to obtain an arrangement sequence;
a target parameter threshold determining unit, configured to determine a target parameter threshold by using the arrangement order and the adjustment channel number;
the to-be-pruned channel determining unit is used for determining a channel corresponding to a regularized scaling factor with the value not larger than a target parameter threshold value as the to-be-pruned channel;
the pseudo pruning unit is used for setting the mask of the pruning channel to be pruned to 0 to obtain a pseudo pruning model;
the loss function of the training preset target detection model is as follows:
wherein ,
for Focal loss, the corresponding formula for Focal loss is as follows:
wherein ,
gamma is a preset constant, and p is the prediction confidence of the network to the training sample in the training process;
wherein ,
and (3) for pre-training the predicted value of the target detection model, wherein y is the true value of the sample.
7. An electronic device, the device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the target detection network batch normalization layer pruning method of any one of claims 1-5 according to instructions in the program code.
8. A computer readable storage medium storing program code for performing the object detection network batch normalization layer pruning method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244347.XA CN115953652B (en) | 2023-03-15 | 2023-03-15 | Method, device, equipment and medium for pruning target detection network batch normalization layer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310244347.XA CN115953652B (en) | 2023-03-15 | 2023-03-15 | Method, device, equipment and medium for pruning target detection network batch normalization layer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115953652A CN115953652A (en) | 2023-04-11 |
| CN115953652B true CN115953652B (en) | 2023-06-02 |
Family
ID=85891466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310244347.XA Active CN115953652B (en) | 2023-03-15 | 2023-03-15 | Method, device, equipment and medium for pruning target detection network batch normalization layer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115953652B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116128879B (en) * | 2023-04-17 | 2023-08-01 | 广东电网有限责任公司肇庆供电局 | Lightweight transmission line defect detection method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723944A (en) * | 2022-03-28 | 2022-07-08 | 长春长光博翔无人机有限公司 | Image analysis method, storage medium, and electronic device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112784885B (en) * | 2021-01-11 | 2022-05-24 | 腾讯科技（深圳）有限公司 | Automatic driving method, device, equipment, medium and vehicle based on artificial intelligence |
| AU2022201841A1 (en) * | 2021-03-16 | 2022-10-06 | Monsters Aliens Robots Zombies Inc. | Method and system for image processing |
| CN113128355A (en) * | 2021-03-29 | 2021-07-16 | 南京航空航天大学 | Unmanned aerial vehicle image real-time target detection method based on channel pruning |
| CN115082813A (en) * | 2022-06-27 | 2022-09-20 | 北京智芯半导体科技有限公司 | Detection method, unmanned aerial vehicle, detection system and medium |
-
2023
- 2023-03-15 CN CN202310244347.XA patent/CN115953652B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723944A (en) * | 2022-03-28 | 2022-07-08 | 长春长光博翔无人机有限公司 | Image analysis method, storage medium, and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115953652A (en) | 2023-04-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115953652B (en) | Method, device, equipment and medium for pruning target detection network batch normalization layer | |
| US20200402221A1 (en) | Inspection system, image discrimination system, discrimination system, discriminator generation system, and learning data generation device | |
| CN112464718B (en) | Target detection method based on YOLO-Terse network and storage medium | |
| CN112419268A (en) | Method, device, equipment and medium for detecting image defects of power transmission line | |
| CN111476307A (en) | Lithium battery surface defect detection method based on depth field adaptation | |
| CN116310850A (en) | Remote sensing image target detection method based on improved RetinaNet | |
| CN116805387B (en) | Model training method, quality inspection method and related equipment based on knowledge distillation | |
| CN113466839B (en) | Side-scan sonar sea bottom line detection method and device | |
| CN112101313B (en) | Machine room robot inspection method and system | |
| CN113743455A (en) | Target retrieval method, device, electronic equipment and storage medium | |
| CN108803658A (en) | Cruising inspection system based on unmanned plane | |
| CN116543295A (en) | Lightweight underwater target detection method and system based on degradation image enhancement | |
| CN115546689A (en) | Video time sequence abnormal frame detection method based on unsupervised frame correlation | |
| CN115018787A (en) | Anomaly detection method and system based on gradient enhancement | |
| CN115601293A (en) | Object detection method and device, electronic equipment and readable storage medium | |
| CN114550129A (en) | Machine learning model processing method and system based on data set | |
| CN114359891A (en) | Three-dimensional vehicle detection method, system, device and medium | |
| CN113269678A (en) | Fault point positioning method for contact network transmission line | |
| CN113723536A (en) | Power inspection target identification method and system | |
| CN111931920A (en) | Target detection method, device and storage medium based on cascade neural network | |
| CN113989267B (en) | Battery defect detection method based on lightweight neural network | |
| CN115880486B (en) | Target detection network distillation method and device, electronic equipment and storage medium | |
| CN115331082B (en) | Path generation method of tracking sound source, training method of model and electronic equipment | |
| CN117437455A (en) | Method, device, equipment and readable medium for determining wafer defect mode | |
| CN113762287A (en) | Intelligent detection system and method for underwater target and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |