CN113920107A - Insulator damage detection method based on improved yolov5 algorithm - Google Patents

Abstract

The invention discloses an insulator damage detection method based on an improved yolov5 algorithm, which comprises the steps of firstly acquiring a patrol video to obtain insulator pictures, expanding the pictures through rotation, noise adding and mirror image operation, and taking the expanded insulator pictures as a real insulator sample library; preprocessing pictures in a real insulator sample library by using a generating countermeasure network GAN, fusing a defective insulator picture with various complex backgrounds, expanding a defective insulator data set to obtain an insulator sample expansion library, dividing the data into a training set and a test set, then labeling the selected training set by using a labeling tool Labellmg, and storing information of the labeled insulator picture to obtain sample data; the existing target detection network yolov5 is improved, and finally, an insulator defect detection result is obtained. The invention solves the problem of low insulator detection precision in the prior art under the heterocyclic environment.

Description

Insulator damage detection method based on improved yolov5 algorithm

Technical Field

The invention belongs to the technical field of image processing, and particularly relates to an insulator breakage detection method based on an improved yolov5 algorithm.

Background

With the continuous enlargement of the scale of high-voltage and ultrahigh-voltage transmission lines in China, the inconvenience of transmission line inspection is caused. In the years, the computer vision technology is rapidly developed, so that people have new understanding in the vision field, the computer vision technology is profoundly influencing the life of people, and technical support is provided for the safety detection of a power system. Before the computer technology is not mature, the insulators can be identified and screened only by manpower, so that visual fatigue is easy to generate, and the inspection efficiency is reduced. At present, the computer technology is mature, the computer technology can replace manpower to identify high-definition pictures of insulators, so that the power inspection is developed to the direction of full automation, and a real intelligent power grid is realized.

Unmanned aerial vehicle imaging platform is built by the attached imaging device of unmanned aerial vehicle and ripe computer vision technique and forms, and transmission line's the mode of patrolling and examining turns to more efficient unmanned aerial vehicle by the manual work of tradition and patrols and examines. The high-definition camera borne by the unmanned aerial vehicle can shoot high-definition pictures of the insulators on the power transmission line, and the pictures contain GPS data on each tower and label data on each insulator. If the insulator information in the picture can be accurately positioned and screened, a basis can be provided for subsequent insulator defect inspection and maintenance. The picture that unmanned aerial vehicle shot is transmitted to the customer end by 5G network, utilizes image recognition technology to carry out the defect detection of insulator under the complex environment at last.

Disclosure of Invention

The invention aims to provide an insulator damage detection method based on an improved yolov5 algorithm, which solves the problem of low insulator detection precision in a complex environment in the prior art.

The technical scheme adopted by the invention is that the insulator breakage detection method based on the improved yolov5 algorithm is implemented according to the following steps:

step 1, collecting inspection videos, obtaining insulator pictures through the inspection videos, expanding the pictures through rotation, noise adding and mirror image operations, and taking the expanded insulator pictures as a real insulator sample library;

step 2, preprocessing the pictures in the real insulator sample library obtained in the step 1 by using a generative countermeasure network GAN, fusing the defective insulator pictures with various complex backgrounds, expanding the defective insulator data set to obtain an insulator sample expansion library, dividing the data into a training set and a test set, and selecting 20% of the training set and 80% of the training set as the training set;

step 3, labeling the training set selected in the step 2 by using a labeling tool Labellmg, and storing information of the labeled insulator picture to obtain sample data;

step 4, improving the existing target detection network yolov5, and performing iterative training on the improved target detection network yolov5 by using the sample data obtained in the step 3 to obtain the optimal target detection network weight data and the reference network of the test set;

and 5, processing the test set obtained in the step 2 by using the reference network of the test set obtained in the step 4 to obtain an insulator defect detection result.

The present invention is also characterized in that,

the step 1 is as follows:

step 1.1, shooting a large number of inspection videos through a high-definition camera carried by an unmanned aerial vehicle, and obtaining a large number of insulator pictures from the inspection videos;

step 1.2, the insulator picture obtained in the step 1.1 is rotated, noised and mirrored to expand a data set;

and step 1.3, putting the insulator picture obtained in the step 1.1 and the data set expanded in the step 1.2 together to be used as a real insulator sample library.

The step 2 is as follows:

step 2.1, inputting the real insulator sample library obtained in the step 1 into a generation countermeasure network model GAN, thereby generating simulated insulator pictures and carrying out picture quality grading and sorting, screening the simulated insulator pictures according to the picture quality grading and sorting result, generating a simulated sample library, training the existing target detection network yolov5 according to the real insulator sample library and the simulated sample library, obtaining the optimal expansion ratio of the real insulator pictures and the simulated insulator pictures according to the training result of the existing target detection network yolov5 model, calculating the number of expansion samples of the simulated insulator pictures, adding the expansion samples into the real insulator sample library after obtaining the expansion samples from the simulated sample library, and generating an insulator sample expansion library;

and 2.2, selecting 20% of the obtained insulator sample expansion library as a test set and 80% of the obtained insulator sample expansion library as a training set.

The step 3 is as follows:

step 3.1, determining an insulator picture to be marked, finding an area where the insulator is located in the picture, and then marking to obtain a marking frame of the area where the insulator is located;

step 3.2, marking the marking frame of the region where the insulator is located, which is obtained in the step 3.1, by using two types of state labels, namely a normal insulator or a defective insulator, according to the state of the insulator in the picture, and obtaining the marking information of the marking frame with the state label and the region where the insulator is located;

step 3.3, generating corresponding xml documents by the marking information obtained in the step 3.2, wherein the marking information in each document comprises 4 position attributesAnd 1 category attribute, wherein the 4 position attributes are respectively the abscissa x of the central point position of the marking frame of the region where the insulator is positioned_minWith ordinate y_maxAnd the width w and the height h of the marking frame of the region where the insulator is located, and the category attribute represents the state information of the normal insulator or the defective insulator divided in the step 3.2;

and 3.1, 3.2 and 3.3, obtaining the marked insulator picture and the xml document with the marked information as sample data.

The step 4 is as follows:

step 4.1, the yolov5 target detection network performs feature detection on the pictures in the insulator sample expansion library under the large scale 19 × 19, the medium scale 38 × 38 and the small scale 76 × 76, and performs up-sampling and feature fusion among the 3 scales, wherein the fusion process specifically comprises the following steps: the yolov5 target detection network respectively extracts the characteristics of large-scale 19 x 19, medium-scale 38 x 38 and small-scale 76 x 76 insulator samples through a bottleneck cross-stage local network, and then the characteristic information of the large-scale insulator picture sample is subjected to element-by-element addition through upsampling and the characteristics of the medium-scale insulator picture sample extracted through the bottleneck cross-stage local network to obtain a final output characteristic diagram of the medium-scale 38 x 38; similarly, the characteristic information of the medium-scale insulator picture is up-sampled and the characteristics of the small-scale insulator picture sample are added element by element to obtain a final output characteristic diagram of small scale 76 x 76, so that output characteristic diagrams under large scale, medium scale and small scale are respectively obtained;

step 4.2, extracting insulator defect features through convolution operation on the basis of the output feature diagram obtained in the step 4.1, then performing convolution and up-sampling operation, performing 3 x 3 convolution and 2 times up-sampling twice after extracting the defect features on the 19 x 19 large-scale output feature diagram, performing 3 x 3 convolution and 2 times up-sampling once after extracting the defect features on the 38 x 38 medium-scale output feature diagram, and performing 3 x 3 convolution once after extracting the defect features on the 76 x 76 small-scale output feature diagram, thereby obtaining the feature diagram with uniform size;

step 4.3, after embedding the CBAM module into the feature extraction operation of the step 4.1, the CBAM module is formed by mixing channel attention and space attention, and the CBAM module is specifically shown in formulas (1) to (4); the CBAM module infers an attention feature map along a channel of the input intermediate feature map and 2 dimensional directions of a space, so that important contents and important positions in the picture are concerned, and the attention map is multiplied by the input intermediate feature map to perform self-adaptive feature refinement, namely learned features are updated; the CBAM module is represented by the following formula:

M_C(F)＝σ(MLP(AvgPool(F))+MLP(Max-Pool(F)))； (3)

M_S(F)＝σ(f^7×7(AvgPool(F₁)；MaxPool(F₁)))； (4)

wherein F represents the input characteristic diagram, the output characteristic diagram of the channel attention module, and F₂A graph of the output characteristics of the spatial attention module is shown,

representing element-wise multiplication operations, i.e. corresponding multiplication of co-located elements, M_C(F) For operations to be performed with channel attention, M_S(F) For operations performed on spatial attention, AvgPool denotes global mean pooling, MaxPool denotes global maximum pooling, MLP is multi-layered perceptron, σ is sigmoid activation operation, f^7×7Represents performing a convolution of 7 × 7;

step 4.4, CBAM module processing includes two stages: 1. performing global average pooling and global maximum pooling on input F, respectively adding obtained results through a multi-layer perceptron, and activating by sigmoid to obtain M_C(F) (ii) a Let M_C(F) Performing dot multiplication on the sum F to generate a final channel attention feature map F₁(ii) a 2. Will pay attention based on the channelForce derived F₁Performing global average pooling and global maximum pooling, and performing concat fusion operation; reducing the convolution dimension into one channel, and obtaining M by sigmoid activation_S(F) Then M is added_S(F) And F₁Performing dot product operation to generate final space attention feature diagram F₂I.e. CBAM module output; the attention feature extraction module is formed by a CBAM module mixing space attention and channel attention;

step 4.5, splicing the feature maps with uniform sizes obtained in the step 4.2, inputting the feature maps into the attention feature extraction module obtained in the step 4.4 to refine the features, and then performing 3 × 3 convolution twice, wherein the operation is a semantic segmentation module;

step 4.6, adopting a CIoU loss function as a prediction box regression loss function L of the improved yolov5 algorithm_CIoUDefined as:

L_CIoU＝1-IoU+R_CIoU+αv (5)

wherein IoU is the cross-over ratio, R_CIoUIs a penalty item; α v is an influence factor, where α is a parameter used for weighing, and v is a parameter used for measuring the uniformity of the aspect ratio; b^gtA prediction box indicating that the category is a defect; b represents a prediction box of which the category is insulator, b and b^gtRespectively represent b and b^gtA center point of (a); rho is the Euclidean distance; c represents a diagonal distance of the target minimum bounding rectangle;

the expression for the α and v parameters is as follows:

wherein w and h are independently substitutedTable predicts the width and height of the box; w is a^gtAnd h^gtThe width and height of the real frame respectively;

obtaining an improved yolov5 target detection module by improving a loss function of a yolov5 network;

step 4.7, combining the improved yolov5 target detection module obtained in step 4.6 with the semantic segmentation module obtained in step 4.5 to obtain an improved target detection network yolov 5;

step 4.8, training the improved target detection network yolov5 built in the step 4.7, defining the current iteration number as mu, and initializing mu to 1; maximum number of iterations is mu_max(ii) a Carrying out random initialization on each layer of parameters in the depth feature fused yolov5 network for the mu time so as to obtain a depth feature fused yolov5 network for the mu time iteration;

and 4.9, inputting the training set into the yolov5 network with the depth feature fused after the mu-th iteration obtained in the step 4.8 for training, obtaining the optimum improved target detection network yolov5 weight parameters through training, and finally obtaining the reference network of the test set.

Step 4.9 is specifically as follows:

step 4.9.1, firstly training an improved yolov5 target detection module, and carrying out contour detection on all pictures in a training set to obtain a labeled contour point (x)_i,y_i) And i is 1,2,3,4.. n, then the contour maps are scratched, and the minimum bounding box of all the contour maps is calculated, wherein the coordinate of the upper left corner of the minimum bounding box is (x)_min,y_max) The coordinate of the lower right corner is (x)_max,y_min) And performing loss calculation and precision calculation with the output result obtained after iterative training of the improved yolov5 target detection module obtained in the step 4.6, wherein the loss calculation is shown in formulas (5) to (8), and the precision calculation is shown in formulas (11) to (14); carrying out convolution operation on the training set by a front 52 convolution layer of a trunk CSPDarkNet-53 of an improved yolov5 target detection module, wherein each layer of convolution operation is followed by a batch regularization layer and a Leaky ReLU activation layer;

step 4.9.2, training semantic segmentation module, training result in step 4.9.1On the basis, the confidence coefficient IoU is used for screening, the confidence coefficient calculation is shown as a formula (13), and the block diagram with the retained confidence coefficient larger than 0.5 is made into N₁Training data set due to N₁The sizes of the pictures in the training data set are different, and N is clustered by using a k-means clustering method₁Clustering all pictures in the training data set to obtain the optimal mask size, and then carrying out N on the optimal mask size₁Training the picture size resize in the dataset to the optimal mask size, forming N₂Training a set; will N₂Inputting the training set into the semantic segmentation module obtained in the step 4.5 for training;

the k-means clustering method comprises the following steps:

(1) selecting k characteristic graphs in a sample set, and taking the central points of the k characteristic graph objects as clustering centers to obtain k clustering centers;

(2) calculating Euclidean distances d between the residual sample objects in the step (1) and k clustering centers respectively, dividing the shortest distance calculated by each sample object into clusters corresponding to the mean value, so that each sample point has k distances, merging the points into the cluster with the smallest distance,

b (x) in formula (9)_i)、b(y_i) Is the abscissa and ordinate of the ith sample, c (x)_i)、c(y_i) The horizontal and vertical coordinates of the ith central point;

(3) recalculating mean values for regenerated clusters

And continuing the operation of the step (2);

(4) judging whether the current mean vector changes or not, and if not, exiting;

and 4.9.3, carrying out mu times of iterative training, and storing the trained network parameters to obtain the reference network of the test set.

Step 5, constructing a reference network of the test set, selecting the network parameters with the best effect in the mu iteration results as the reference network of the test set, inputting the test set into the reference network for testing to obtain an insulator defect test result, and verifying the network performance, wherein the method specifically comprises the following steps:

step 5.1, inputting the test set selected in the step 2.2 into the reference network of the test set obtained in the step 4.9 for testing;

step 5.2, verifying the performance of the network: the network performance evaluation standard is mainly evaluated by adopting Precision, Recall rate and average Precision rate average mAP, and the calculation formula is as follows:

wherein:

TP represents a detected positive case; FP represents a detected negative case; TN indicates an undetected negative case; FN indicates an undetected positive case; t _ bbox represents the marked real frame of the insulator; p _ bbox represents the insulating subframe calculated by the algorithm;

the mAP is the area of a region 0 surrounded by a Precision-call curve and a coordinate axis, n is the number of groups of the calculated pictures, and the approximate formula of the difference value is as follows:

step 5, inputting the test set into the improved target detection network yolov5 for testing to obtain an insulator defect test result, which is as follows:

(1) the loss function initial value of the insulator damage detection network of the improved yolov5 algorithm is lower, and the final loss function value is reduced to about 0.25, which indicates that the algorithm performance is better;

(2) from the accuracy and recall rate curve of the improved algorithm, it can be seen that the accuracy and recall rate are always kept in a stable state during the training process, and after the training is completed, the accuracy can be maintained above 97%, the recall rate is always around 100%, and the value of mAP reaches 99.85%, which indicates that the stability and accuracy of the improved algorithm meet the requirements.

The insulator damage detection method based on the improved yolov5 algorithm has the advantages that aiming at the problem of insufficient sample data sets, a GAN generation type countermeasure network is adopted to expand the data sets; the yolov5 network is improved, and the detection precision and efficiency are improved by improving the loss function of the yolov5 network; a semantic segmentation module is fused in the network, and an attention mechanism is added, so that the detection precision is greatly improved.

Drawings

FIG. 1 is a flow chart of an insulator breakage detection method based on an improved yolov5 algorithm;

FIG. 2 is a schematic diagram of a CBAM attention module;

FIG. 3 is a flow chart of a semantic segmentation section;

FIG. 4 is a diagram showing the effect of detecting defects in insulators;

FIG. 5 improved yolov5 mAP values;

FIG. 6 is a graph of the change in loss function for the improved algorithm;

FIG. 7 is a graph of accuracy change for the improved algorithm;

FIG. 8 is a graph of the change in recall of the improved algorithm.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The invention discloses an insulator breakage detection method based on an improved yolov5 algorithm, which is implemented by the following steps as shown in a flow chart shown in figure 1:

step 1, collecting inspection videos, obtaining insulator pictures through the inspection videos, expanding the pictures through rotation, noise adding and mirror image operations, and taking the expanded insulator pictures as a real insulator sample library;

the step 1 is as follows:

step 1.1, shooting a large number of inspection videos through a high-definition camera carried by an unmanned aerial vehicle, and obtaining a large number of insulator pictures from the inspection videos;

step 1.2, the insulator picture obtained in the step 1.1 is rotated, noised and mirrored to expand a data set;

and step 1.3, putting the insulator picture obtained in the step 1.1 and the data set expanded in the step 1.2 together to be used as a real insulator sample library.

Step 2, preprocessing the pictures in the real insulator sample library obtained in the step 1 by using a generative countermeasure network GAN, fusing the defective insulator pictures with various complex backgrounds, expanding the defective insulator data set to obtain an insulator sample expansion library, dividing the data into a training set and a test set, and selecting 20% of the training set and 80% of the training set as the training set;

with reference to fig. 2 to 5, step 2 is as follows:

step 2.1, inputting the real insulator sample library obtained in the step 1 into a generation countermeasure network model GAN, thereby generating simulated insulator pictures and carrying out picture quality grading and sorting, screening the simulated insulator pictures according to the picture quality grading and sorting result, generating a simulated sample library, training the existing target detection network yolov5 according to the real insulator sample library and the simulated sample library, obtaining the optimal expansion ratio of the real insulator pictures and the simulated insulator pictures according to the training result of the existing target detection network yolov5 model, calculating the number of expansion samples of the simulated insulator pictures, adding the expansion samples into the real insulator sample library after obtaining the expansion samples from the simulated sample library, and generating an insulator sample expansion library;

the generated countermeasure network model consists of a generated model G and a discriminant model D, under the ideal condition, the generated network can generate a sample which is fake and genuine and cheats the discriminant model D, the discriminant model D can distinguish the fake sample generated by the generated model, the generated model and the discriminant model form a dynamic game process, and after the network is trained, the generated network G with excellent performance can be obtained and used for expanding a data set and solving the problem of insufficient data.

And 2.2, selecting 20% of the obtained insulator sample expansion library as a test set and 80% of the obtained insulator sample expansion library as a training set.

Step 3, labeling the training set selected in the step 2 by using a labeling tool Labellmg, and storing information of the labeled insulator picture to obtain sample data;

the step 3 is as follows:

step 3.1, determining an insulator picture to be marked, finding an area where the insulator is located in the picture, and then marking to obtain a marking frame of the area where the insulator is located;

step 3.2, marking the marking frame of the region where the insulator is located, which is obtained in the step 3.1, by using two types of state labels, namely a normal insulator or a defective insulator, according to the state of the insulator in the picture, and obtaining the marking information of the marking frame with the state label and the region where the insulator is located;

step 3.3, generating corresponding xml documents by the marking information obtained in the step 3.2, wherein the marking information in each document comprises 4 position attributes and 1 category attribute, and the 4 position attributes are respectively an abscissa x of the central point position of a marking frame of the region where the insulator is located_minWith ordinate y_maxAnd the width w and the height h of the marking frame of the region where the insulator is located, and the category attribute represents the state information of the normal insulator or the defective insulator divided in the step 3.2;

and 3.1, 3.2 and 3.3, obtaining the marked insulator picture and the xml document with the marked information as sample data.

Step 4, improving the existing target detection network yolov5, and performing iterative training on the improved target detection network yolov5 by using the sample data obtained in the step 3 to obtain the optimal target detection network weight data and the reference network of the test set;

the step 4 is as follows:

the powerful feature extraction capability of yolov5 is mainly derived from Backbone network Backbone, and different features of pictures are extracted by using a convolution method. yolov5 uses a bottleneck cross-stage local network as a backbone network to extract rich information features from the input pictures. The Bottleneck cross-stage local network is formed by combining a standard Bottleneck layer Bottleneck and a cross-stage local network. The cross-stage local network CSPNet solves the problem of gradient repetition in the backhaul of other large convolutional neural network frameworks, integrates gradient change into a characteristic diagram, and reduces the operation amount.

The Bottleneck layer Bottleneck in the Bottleneck cross-stage local network Bottleneck CSP uses a residual error neural network structure, so that the network depth can be effectively increased, the gradient loss is reduced, and the feature extraction capability of the network is improved.

Step 4.1, the yolov5 target detection network adopts a method similar to a characteristic pyramid, the images in the insulator sample expansion library can be subjected to characteristic detection under the large scale 19 × 19, the medium scale 38 × 38 and the small scale 76 × 76, and the up-sampling and characteristic fusion are performed among the 3 scales, so that the accuracy of the algorithm for detecting the small target is enhanced. The fusion process is specifically as follows: the yolov5 target detection network respectively extracts the characteristics of large-scale 19 x 19, medium-scale 38 x 38 and small-scale 76 x 76 insulator samples through a bottleneck cross-stage local network, and then the characteristic information of the large-scale insulator picture sample is subjected to element-by-element addition through upsampling and the characteristics of the medium-scale insulator picture sample extracted through the bottleneck cross-stage local network to obtain a final output characteristic diagram of the medium-scale 38 x 38; similarly, the characteristic information of the medium-scale insulator picture is up-sampled and the characteristics of the small-scale insulator picture sample are added element by element to obtain a final output characteristic diagram of small scale 76 x 76, so that output characteristic diagrams under large scale, medium scale and small scale are respectively obtained, and the fusion of deep information and shallow information is realized; processing features in this manner allows the network to detect using deeper features.

Step 4.2, extracting insulator defect features through convolution operation on the basis of the output feature diagram obtained in the step 4.1, then performing convolution and up-sampling operation, performing 3 x 3 convolution and 2 times up-sampling twice after extracting the defect features on the 19 x 19 large-scale output feature diagram, performing 3 x 3 convolution and 2 times up-sampling once after extracting the defect features on the 38 x 38 medium-scale output feature diagram, and performing 3 x 3 convolution once after extracting the defect features on the 76 x 76 small-scale output feature diagram, thereby obtaining the feature diagram with uniform size; the above operation is for unifying feature size.

Step 4.3, after embedding the CBAM module into the feature extraction operation of the step 4.1, the CBAM module is formed by mixing channel attention and space attention, and the CBAM module is specifically shown in formulas (1) to (4); the CBAM module infers an attention feature map along a channel of the input intermediate feature map and 2 dimensional directions of a space, so that important contents and important positions in the picture are concerned, and the attention map is multiplied by the input intermediate feature map to perform self-adaptive feature refinement, namely learned features are updated; the CBAM module introducing the mixed channel domain and the space domain is used for acquiring rich context information in the insulator picture, learning the importance degree of each characteristic channel and each characteristic space, representing the defect characteristics in the insulator picture in a fine granularity manner, and reducing the weight of irrelevant information, so that the accuracy of defect identification is improved. The CBAM module is represented by the following formula:

M_C(F)＝σ(MLP(AvgPool(F))+MLP(Max-Pool(F)))； (3)

M_S(F)＝σ(f^7×7(AvgPool(F₁)；MaxPool(F₁)))； (4)

wherein F represents the input characteristic diagram, the output characteristic diagram of the channel attention module, and F₂A graph of the output characteristics of the spatial attention module is shown,

representing element-wise multiplication operations, i.e. corresponding multiplication of co-located elements, M_C(F) For operations to be performed with channel attention, M_S(F) For operations performed on spatial attention, AvgPool denotes global mean pooling, MaxPool denotes global maximum pooling, MLP is multi-layered perceptron, σ is sigmoid activation operation, f^7×7Represents performing a convolution of 7 × 7;

step 4.4, CBAM module processing includes two stages: 1. performing global average pooling and global maximum pooling on input F, respectively adding obtained results through a multi-layer perceptron, and activating by sigmoid to obtain M_C(F) (ii) a Let M_C(F) Performing dot multiplication on the sum F to generate a final channel attention feature map F₁(ii) a 2. F will be based on channel attention₁Performing global average pooling and global maximum pooling, and performing concat fusion operation; reducing the convolution dimension into one channel, and obtaining M by sigmoid activation_S(F) Then M is added_S(F) And F₁Performing dot product operation to generate final space attention feature diagram F₂I.e. CBAM module output; the CBAM module, which is a mixture of spatial attention and channel attention, may constitute an attention feature extraction module.

Step 4.5, splicing the feature maps with uniform sizes obtained in the step 4.2, inputting the feature maps into the attention feature extraction module obtained in the step 4.4 to refine the features, and then performing 3 × 3 convolution twice, wherein the operation is a semantic segmentation module;

and 4.6, optimizing a loss function, wherein the IoU algorithm is the most widely used algorithm, the yolov5 selects and adopts the GIoU to calculate the regression loss, and the GIoU inherits the advantage of IoU and makes up the defect that the distance between non-overlapped frames cannot be measured by IoU. However, when both prediction boxes are contained in the real box and their area sizes are identical, the GIoU loss function has the effect of the IoU loss functionLikewise, the relative positional relationship cannot be distinguished. Based on the problem, after three important geometric factors of the overlapping area, the center point distance and the aspect ratio are fully considered, the CIoU loss function is adopted as a prediction box regression loss function L of the improved yolov5 algorithm_CIoUDefined as:

L_CIoU＝1-IoU+R_CIoU+αv (5)

wherein IoU is the cross-over ratio, R_CIoUIs a penalty item; α v is an influence factor, where α is a parameter used for weighing, and v is a parameter used for measuring the uniformity of the aspect ratio; b^gtA prediction box indicating that the category is a defect; b represents a prediction box of which the category is insulator, b and b^gtRespectively represent b and b^gtA center point of (a); rho is the Euclidean distance; c represents a diagonal distance of the target minimum bounding rectangle;

the expression for the α and v parameters is as follows:

wherein w and h represent the width and height of the prediction box, respectively; w is a^gtAnd h^gtThe width and height of the real frame respectively;

obtaining an improved yolov5 target detection module by improving a loss function of a yolov5 network;

step 4.7, combining the improved yolov5 target detection module obtained in step 4.6 with the semantic segmentation module obtained in step 4.5 to obtain an improved target detection network yolov 5;

step 4.8, training the improved target detection network yolov5 built in the step 4.7, and determiningDefining the current iteration number as mu, and initializing mu as 1; maximum number of iterations is mu_max(ii) a Carrying out random initialization on each layer of parameters in the depth feature fused yolov5 network for the mu time so as to obtain a depth feature fused yolov5 network for the mu time iteration;

and 4.9, inputting the training set into the yolov5 network with the depth feature fused after the mu-th iteration obtained in the step 4.8 for training, obtaining the optimum improved target detection network yolov5 weight parameters through training, and finally obtaining the reference network of the test set.

Step 4.9 is specifically as follows:

step 4.9.1, firstly training an improved yolov5 target detection module, and carrying out contour detection on all pictures in a training set to obtain a labeled contour point (x)_i,y_i) And i is 1,2,3,4.. n, then the contour maps are scratched, and the minimum bounding box of all the contour maps is calculated, wherein the coordinate of the upper left corner of the minimum bounding box is (x)_min,y_max) The coordinate of the lower right corner is (x)_max,y_min) And performing loss calculation and precision calculation with the output result obtained after iterative training of the improved yolov5 target detection module obtained in the step 4.6, wherein the loss calculation is shown in formulas (5) to (8), and the precision calculation is shown in formulas (11) to (14); carrying out convolution operation on the training set by a front 52 convolution layer of a trunk CSPDarkNet-53 of an improved yolov5 target detection module, wherein each layer of convolution operation is followed by a batch regularization layer and a Leaky ReLU activation layer;

step 4.9.2, training a semantic segmentation module, screening by using the confidence IoU on the basis of the training result of the step 4.9.1, calculating the confidence as shown in a formula (13), and making a block diagram with the confidence greater than 0.5 into N₁Training data set due to N₁The sizes of the pictures in the training data set are different, and N is clustered by using a k-means clustering method₁Clustering all pictures in the training data set to obtain the optimal mask size, and then carrying out N on the optimal mask size₁Training the picture size resize in the dataset to the optimal mask size, forming N₂Training a set; therefore, regression work can be reduced, the calculation efficiency is improved, and the network training time is reduced, so that a better region can be obtainedDomain proposal improves detection accuracy by adding N₂Inputting the training set into the semantic segmentation module obtained in the step 4.5 for training;

the k-means clustering method comprises the following steps:

(1) selecting k characteristic graphs in a sample set, and taking the central points of the k characteristic graph objects as clustering centers to obtain k clustering centers;

(2) calculating Euclidean distances d between the residual sample objects in the step (1) and k clustering centers respectively, dividing the shortest distance calculated by each sample object into clusters corresponding to the mean value, so that each sample point has k distances, merging the points into the cluster with the smallest distance,

b (x) in formula (9)_i)、b(y_i) Is the abscissa and ordinate of the ith sample, c (x)_i)、c(y_i) The horizontal and vertical coordinates of the ith central point;

(3) recalculating mean values for regenerated clusters

And continuing the operation of the step (2);

(4) judging whether the current mean vector changes or not, and if not, exiting;

and 4.9.3, carrying out mu times of iterative training, and storing the trained network parameters to obtain the reference network of the test set.

And 5, processing the test set obtained in the step 2 by using the reference network of the test set obtained in the step 4 to obtain an insulator defect detection result.

Step 5, constructing a reference network of the test set, selecting the network parameters with the best effect in the mu iteration results as the reference network of the test set, inputting the test set into the reference network for testing to obtain an insulator defect test result, and verifying the network performance, wherein the method specifically comprises the following steps:

step 5.1, inputting the test set selected in the step 2.2 into the reference network of the test set obtained in the step 4.9 for testing;

step 5.2, verifying the performance of the network: the evaluation criteria of network performance are mainly evaluated by Precision, Recall and mean Precision average value mAP (mean average Precision), and the calculation formula is as follows:

wherein:

TP represents a detected positive case; FP represents a detected negative case; TN indicates an undetected negative case; FN indicates an undetected positive case; t _ bbox represents the marked real frame of the insulator; p _ bbox represents the insulating subframe calculated by the algorithm;

it is obviously not suitable if precision or recall is used as a measure for the detection accuracy of a model. Therefore, an mAP which is one of the most important indexes in the target detection is also needed, and is an average AP value of a plurality of test sets, the mAP is an area of a region 0 surrounded by a Precision-call curve and a coordinate axis, n is the number of groups of calculation pictures, and then the difference value is approximated by the formula:

step 5, inputting the test set into the improved target detection network yolov5 for testing to obtain an insulator defect test result, wherein the test result of the experiment is as follows:

(1) the initial value of the loss function of the insulator damage detection network of the modified yolov5 algorithm is low, and the final loss function value is reduced to about 0.25, which shows that the performance of the algorithm is good, and the specific change of the loss function value is shown in fig. 6;

(2) it can be seen from the accuracy and recall rate curves of the improved algorithm that the accuracy and recall rate are always kept in a stable state during the training process, and after the training is completed, the accuracy can be maintained above 97%, the recall rate is always around 100%, the value of mAP reaches 99.85%, which indicates that the stability and accuracy of the improved algorithm meet the requirements, and the specific changes of the accuracy and recall rate are shown in FIGS. 7 and 8.

Claims (8)

1. An insulator breakage detection method based on an improved yolov5 algorithm is characterized by comprising the following steps:

step 1, collecting inspection videos, obtaining insulator pictures through the inspection videos, expanding the pictures through rotation, noise adding and mirror image operations, and taking the expanded insulator pictures as a real insulator sample library;

step 2, preprocessing the pictures in the real insulator sample library obtained in the step 1 by using a generative countermeasure network GAN, fusing the defective insulator pictures with various complex backgrounds, expanding the defective insulator data set to obtain an insulator sample expansion library, dividing the data into a training set and a test set, and selecting 20% of the training set and 80% of the training set as the training set;

step 3, labeling the training set selected in the step 2 by using a labeling tool Labellmg, and storing information of the labeled insulator picture to obtain sample data;

step 4, improving the existing target detection network yolov5, and performing iterative training on the improved target detection network yolov5 by using the sample data obtained in the step 3 to obtain the optimal target detection network weight data and the reference network of the test set;

and 5, processing the test set obtained in the step 2 by using the reference network of the test set obtained in the step 4 to obtain an insulator defect detection result.

2. The method for detecting the damage of the insulator based on the modified yolov5 algorithm according to claim 1, wherein the step 1 is as follows:

step 1.1, shooting a large number of inspection videos through a high-definition camera carried by an unmanned aerial vehicle, and obtaining a large number of insulator pictures from the inspection videos;

step 1.2, the insulator picture obtained in the step 1.1 is rotated, noised and mirrored to expand a data set;

and step 1.3, putting the insulator picture obtained in the step 1.1 and the data set expanded in the step 1.2 together to be used as a real insulator sample library.

3. The method for detecting the damage of the insulator based on the modified yolov5 algorithm according to claim 2, wherein the step 2 is as follows:

step 2.1, inputting the real insulator sample library obtained in the step 1 into a generation countermeasure network model GAN, thereby generating simulated insulator pictures and carrying out picture quality grading and sorting, screening the simulated insulator pictures according to the picture quality grading and sorting result, generating a simulated sample library, training the existing target detection network yolov5 according to the real insulator sample library and the simulated sample library, obtaining the optimal expansion ratio of the real insulator pictures and the simulated insulator pictures according to the training result of the existing target detection network yolov5 model, calculating the number of expansion samples of the simulated insulator pictures, adding the expansion samples into the real insulator sample library after obtaining the expansion samples from the simulated sample library, and generating an insulator sample expansion library;

and 2.2, selecting 20% of the obtained insulator sample expansion library as a test set and 80% of the obtained insulator sample expansion library as a training set.

4. The method for detecting the damage of the insulator based on the modified yolov5 algorithm according to claim 3, wherein the step 3 is as follows:

step 3.1, determining an insulator picture to be marked, finding an area where the insulator is located in the picture, and then marking to obtain a marking frame of the area where the insulator is located;

step 3.2, marking the marking frame of the region where the insulator is located, which is obtained in the step 3.1, by using two types of state labels, namely a normal insulator or a defective insulator, according to the state of the insulator in the picture, and obtaining the marking information of the marking frame with the state label and the region where the insulator is located;

step 3.3, generating corresponding xml documents by the marking information obtained in the step 3.2, wherein the marking information in each document comprises 4 position attributes and 1 category attribute, and the 4 position attributes are respectively an abscissa x of the central point position of a marking frame of the region where the insulator is located_minWith ordinate y_maxAnd the width w and the height h of the marking frame of the region where the insulator is located, and the category attribute represents the state information of the normal insulator or the defective insulator divided in the step 3.2;

and 3.1, 3.2 and 3.3, obtaining the marked insulator picture and the xml document with the marked information as sample data.

5. The method for detecting insulator breakage based on the modified yolov5 algorithm according to claim 4, wherein the step 4 is as follows:

step 4.1, the yolov5 target detection network performs feature detection on the pictures in the insulator sample expansion library under the large scale 19 × 19, the medium scale 38 × 38 and the small scale 76 × 76, and performs up-sampling and feature fusion among the 3 scales, wherein the fusion process specifically comprises the following steps: the yolov5 target detection network respectively extracts the characteristics of large-scale 19 x 19, medium-scale 38 x 38 and small-scale 76 x 76 insulator samples through a bottleneck cross-stage local network, and then the characteristic information of the large-scale insulator picture sample is subjected to element-by-element addition through upsampling and the characteristics of the medium-scale insulator picture sample extracted through the bottleneck cross-stage local network to obtain a final output characteristic diagram of the medium-scale 38 x 38; similarly, the characteristic information of the medium-scale insulator picture is up-sampled and the characteristics of the small-scale insulator picture sample are added element by element to obtain a final output characteristic diagram of small scale 76 x 76, so that output characteristic diagrams under large scale, medium scale and small scale are respectively obtained;

step 4.2, extracting insulator defect features through convolution operation on the basis of the output feature diagram obtained in the step 4.1, then performing convolution and up-sampling operation, performing 3 x 3 convolution and 2 times up-sampling twice after extracting the defect features on the 19 x 19 large-scale output feature diagram, performing 3 x 3 convolution and 2 times up-sampling once after extracting the defect features on the 38 x 38 medium-scale output feature diagram, and performing 3 x 3 convolution once after extracting the defect features on the 76 x 76 small-scale output feature diagram, thereby obtaining the feature diagram with uniform size;

step 4.3, after embedding the CBAM module into the feature extraction operation of the step 4.1, the CBAM module is formed by mixing channel attention and space attention, and the CBAM module is specifically shown in formulas (1) to (4); the CBAM module infers an attention feature map along a channel of the input intermediate feature map and 2 dimensional directions of a space, so that important contents and important positions in the picture are concerned, and the attention map is multiplied by the input intermediate feature map to perform self-adaptive feature refinement, namely learned features are updated; the CBAM module is represented by the following formula:

M_C(F)＝σ(MLP(AvgPool(F))+MLP(Max-Pool(F)))； (3)

M_S(F)＝σ(f^7×7(AvgPool(F₁)；MaxPool(F₁)))； (4)

wherein F represents the input characteristic diagram, the output characteristic diagram of the channel attention module, and F₂Representing spatial attention modulesOutputting a characteristic diagram of the image to be displayed,

representing element-wise multiplication operations, i.e. corresponding multiplication of co-located elements, M_C(F) For operations to be performed with channel attention, M_S(F) For operations performed on spatial attention, AvgPool denotes global mean pooling, MaxPool denotes global maximum pooling, MLP is multi-layered perceptron, σ is sigmoid activation operation, f^7×7Represents performing a convolution of 7 × 7;

step 4.4, CBAM module processing includes two stages: 1. performing global average pooling and global maximum pooling on input F, respectively adding obtained results through a multi-layer perceptron, and activating by sigmoid to obtain M_C(F) (ii) a Let M_C(F) Performing dot multiplication on the sum F to generate a final channel attention feature map F₁(ii) a 2. F will be based on channel attention₁Performing global average pooling and global maximum pooling, and performing concat fusion operation; reducing the convolution dimension into one channel, and obtaining M by sigmoid activation_S(F) Then M is added_S(F) And F₁Performing dot product operation to generate final space attention feature diagram F₂I.e. CBAM module output; the attention feature extraction module is formed by a CBAM module mixing space attention and channel attention;

step 4.5, splicing the feature maps with uniform sizes obtained in the step 4.2, inputting the feature maps into the attention feature extraction module obtained in the step 4.4 to refine the features, and then performing 3 × 3 convolution twice, wherein the operation is a semantic segmentation module;

step 4.6, adopting a CIoU loss function as a prediction box regression loss function L of the improved yolov5 algorithm_CIoUDefined as:

L_CIoU＝1-IoU+R_CIoU+αv (5)

wherein IoU is the cross-over ratio，R_CIoUIs a penalty item; α v is an influence factor, where α is a parameter used for weighing, and v is a parameter used for measuring the uniformity of the aspect ratio; b^gtA prediction box indicating that the category is a defect; b represents a prediction box of which the category is insulator, b and b^gtRespectively represent b and b^gtA center point of (a); rho is the Euclidean distance; c represents a diagonal distance of the target minimum bounding rectangle;

the expression for the α and v parameters is as follows:

wherein w and h represent the width and height of the prediction box, respectively; w is a^gtAnd h^gtThe width and height of the real frame respectively;

obtaining an improved yolov5 target detection module by improving a loss function of a yolov5 network;

step 4.7, combining the improved yolov5 target detection module obtained in step 4.6 with the semantic segmentation module obtained in step 4.5 to obtain an improved target detection network yolov 5;

step 4.8, training the improved target detection network yolov5 built in the step 4.7, defining the current iteration number as mu, and initializing mu to 1; maximum number of iterations is mu_max(ii) a Carrying out random initialization on each layer of parameters in the depth feature fused yolov5 network for the mu time so as to obtain a depth feature fused yolov5 network for the mu time iteration;

and 4.9, inputting the training set into the yolov5 network with the depth feature fused after the mu-th iteration obtained in the step 4.8 for training, obtaining the optimum improved target detection network yolov5 weight parameters through training, and finally obtaining the reference network of the test set.

6. The method for detecting insulator breakage based on the modified yolov5 algorithm of claim 5, wherein the step 4.9 is as follows:

step 4.9.1, firstly training an improved yolov5 target detection module, and carrying out contour detection on all pictures in a training set to obtain a labeled contour point (x)_i,y_i) And i is 1,2,3,4.. n, then the contour maps are scratched, and the minimum bounding box of all the contour maps is calculated, wherein the coordinate of the upper left corner of the minimum bounding box is (x)_min,y_max) The coordinate of the lower right corner is (x)_max,y_min) And performing loss calculation and precision calculation with the output result obtained after iterative training of the improved yolov5 target detection module obtained in the step 4.6, wherein the loss calculation is shown in formulas (5) to (8), and the precision calculation is shown in formulas (11) to (14); carrying out convolution operation on the training set by a front 52 convolution layer of a trunk CSPDarkNet-53 of an improved yolov5 target detection module, wherein each layer of convolution operation is followed by a batch regularization layer and a Leaky ReLU activation layer;

step 4.9.2, training a semantic segmentation module, screening by using the confidence IoU on the basis of the training result of the step 4.9.1, calculating the confidence as shown in a formula (13), and making a block diagram with the confidence greater than 0.5 into N₁Training data set due to N₁The sizes of the pictures in the training data set are different, and N is clustered by using a k-means clustering method₁Clustering all pictures in the training data set to obtain the optimal mask size, and then carrying out N on the optimal mask size₁Training the picture size resize in the dataset to the optimal mask size, forming N₂Training a set; will N₂Inputting the training set into the semantic segmentation module obtained in the step 4.5 for training;

the k-means clustering method comprises the following steps:

(1) selecting k characteristic graphs in a sample set, and taking the central points of the k characteristic graph objects as clustering centers to obtain k clustering centers;

(2) calculating Euclidean distances d between the residual sample objects in the step (1) and k clustering centers respectively, dividing the shortest distance calculated by each sample object into clusters corresponding to the mean value, so that each sample point has k distances, merging the points into the cluster with the smallest distance,

b (x) in formula (9)_i)、b(y_i) Is the abscissa and ordinate of the ith sample, c (x)_i)、c(y_i) The horizontal and vertical coordinates of the ith central point;

(3) recalculating mean values for regenerated clusters

And continuing the operation of the step (2);

(4) judging whether the current mean vector changes or not, and if not, exiting;

and 4.9.3, carrying out mu times of iterative training, and storing the trained network parameters to obtain the reference network of the test set.

7. The method for detecting insulator damage based on the modified yolov5 algorithm of claim 6, wherein the step 5 is to construct a reference network of the test set, select the most effective network parameters in the results of the mu iterations as the reference network of the test set, input the test set into the reference network for testing to obtain the test result of the insulator defect, and verify the network performance, and the method comprises the following steps:

step 5.1, inputting the test set selected in the step 2.2 into the reference network of the test set obtained in the step 4.9 for testing;

step 5.2, verifying the performance of the network: the network performance evaluation standard is mainly evaluated by adopting Precision, Recall rate and average Precision rate average mAP, and the calculation formula is as follows:

wherein:

TP represents a detected positive case; FP represents a detected negative case; TN indicates an undetected negative case; FN indicates an undetected positive case; t _ bbox represents the marked real frame of the insulator; p _ bbox represents the insulating subframe calculated by the algorithm;

the mAP is the area of a region 0 surrounded by a Precision-call curve and a coordinate axis, n is the number of groups of the calculated pictures, and the approximate formula of the difference value is as follows:

8. the method for detecting insulator breakage based on modified yolov5 algorithm of claim 7, wherein the step 5 inputs the test set into the modified target detection network yolov5 for testing to obtain the insulator defect test result, which is as follows:

(1) the loss function initial value of the insulator damage detection network of the improved yolov5 algorithm is lower, and the final loss function value is reduced to about 0.25, which indicates that the algorithm performance is better;

(2) from the accuracy and recall rate curve of the improved algorithm, it can be seen that the accuracy and recall rate are always kept in a stable state during the training process, and after the training is completed, the accuracy can be maintained above 97%, the recall rate is always around 100%, and the value of mAP reaches 99.85%, which indicates that the stability and accuracy of the improved algorithm meet the requirements.

Images