CN114022432A - Improved yolov 5-based insulator defect detection method - Google Patents

Improved yolov 5-based insulator defect detection method Download PDF

Info

Publication number
CN114022432A
CN114022432A CN202111261977.5A CN202111261977A CN114022432A CN 114022432 A CN114022432 A CN 114022432A CN 202111261977 A CN202111261977 A CN 202111261977A CN 114022432 A CN114022432 A CN 114022432A
Authority
CN
China
Prior art keywords
network
insulator
prediction
training
improved
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111261977.5A
Other languages
Chinese (zh)
Other versions
CN114022432B (en
Inventor
唐靓
余明慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei University of Technology
Original Assignee
Hubei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei University of Technology filed Critical Hubei University of Technology
Priority to CN202111261977.5A priority Critical patent/CN114022432B/en
Publication of CN114022432A publication Critical patent/CN114022432A/en
Application granted granted Critical
Publication of CN114022432B publication Critical patent/CN114022432B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Quality & Reliability (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to an insulator defect detection technology, in particular to an insulator defect detection method based on improved yolov5, which comprises the steps of collecting an insulator image to form a data set; labeling the data set by using a LabelImg labeling tool; the collected images are processedData enhancement processing; introducing a triplet attention module into a backbone network of YOLOv 5; optimizing a loss function; CIoU is taken as a bounding-box Loss function Loss of the improved YOLOv5 algorithmCIoU(ii) a Training the improved network; and inputting the insulator image data set into a trained YOLOv5 network to obtain whether a defective insulator exists in the input image and the position of the defect. The method reduces missing detection, eliminates indirect correspondence between the channel and the weight, achieves the effect of improving the accuracy rate with smaller calculation cost, and improves the accurate positioning capability of the model prediction framework.

Description

Improved yolov 5-based insulator defect detection method
Technical Field
The invention belongs to the technical field of insulator defect detection, and particularly relates to an improved yolov 5-based insulator defect detection method.
Background
Electricity is one of the most basic elements that make the world rotate, and the transmission of high and low voltage electricity is very important for its practical application. In the transmission of high voltage electricity, insulators are used to support and separate electrical conductors from the passage of current. Low voltage distribution lines are a way of distributing power from a distribution grid to end users. An important aspect of primary power distribution systems is the continuous supply of power and the efficient performance of its equipment. The insulator string serves to insulate and provide mechanical strength in the primary overhead distribution line. In summary, insulators are indispensable devices in power systems. Generally, insulators are exposed to the harsh environment of high electric fields, as well as to various harsh weather conditions, such as burning days, typhoons or hurricanes, thunderstorms, sleet, snowstorms, and the like. The harsh environment can make the insulator easily damaged, and then threaten the safety of electric network system and the use of electric power. Such important components, once damaged, pose serious problems for both the power supply and public safety. For example: in the rainy season of each year, people get electric shock and get lost due to exposure under the telegraph pole. Therefore, it is necessary to research an effective insulator defect detection method to ensure the safety and reliability of power transmission.
The current defect detection methods can be divided into three types, namely physical methods, traditional vision-based methods and deep learning-based methods. The physical methods mainly include an ultrasonic method and an ultraviolet pulse method based on manual operation. Among vision-based insulator defect detection methods, HOG + SVM and Haar + AdaBoost are most commonly used. The traditional detection algorithm mainly uses a sliding window to select an interested area, extracts the characteristics of each window, and then classifies characteristic samples to obtain a detection result. In addition, a method for matching based on contour features and gray level similarity is used for classifying the perfect insulators and the defective insulators. Among vision-based insulator defect detection methods, HOG + SVM and Haar + AdaBoost are most commonly used. These methods generally extract image features based on accumulated experience, and are inefficient, limited in accuracy, and time consuming. With the continuous improvement of computer performance, the detection method based on the deep learning framework is widely applied. The method can effectively compensate the loss of the characteristic information in the process of extracting the characteristics of the artificial image, and improve the efficiency of fault detection. Many effective target detection algorithms have been proposed, such as fast R-CNN (fast region-based connected logical network), yolo (you only look) once, ssd (single shot multi-detector), dcnn (dynamic connected logical network), etc. Due to the fact that the thin and long shape characteristics of the insulator and different defect changes cause various and complex expression forms of defects in the image and the acquired insulator image contains a large amount of irrelevant background information, the accuracy and the speed of the existing insulator detection method are still to be improved.
Disclosure of Invention
Aiming at the problems of accuracy and speed of the insulator detection method, the invention provides the insulator defect detection method based on the improved YOLOv5 algorithm.
In order to solve the technical problems, the invention adopts the following technical scheme: the insulator defect detection method based on the improved yolov5 comprises the following steps:
step 1, collecting an insulator image forming data set;
step 2, labeling the data set by using a LabelImg labeling tool, wherein the labeled types are insulators and defects;
step 3, performing data enhancement processing on the acquired image to expand a data set;
step 4, introducing a triplet attention module into a backbone network of YOLOv 5;
step 5, optimizing a loss function; CIoU is taken as a bounding-box Loss function Loss of the improved YOLOv5 algorithmCIoU；
CIoU is defined as:
Figure BDA0003326095140000021
wherein the content of the first and second substances,
Figure BDA0003326095140000022
for the weighting function, v is a parameter for measuring the uniformity of the aspect ratio, c represents the diagonal distance of the smallest closed region that can contain the prediction box and the real box, and ρ2(b，bgt) Representing the Euclidean distance, h, between the center points of the prediction and real framesgt、ωgtRespectively representing the height and width of the real box, hp、ωpRespectively representing the height and width of the prediction box;
step 6, optimizing the non-maximum value inhibition NMS method for processing the final result; Soft-NMS was used as a method to process the final results:
the score function expression of Soft-NMS is:
Figure BDA0003326095140000023
wherein S isiAs a prediction block BiScore of (A), BMFor the highest scoring prediction box, NtIn order to be an overlap threshold value, the threshold value is,
Figure BDA0003326095140000031
a Gaussian penalty function, sigma is a hyperparameter selected according to experience;
step 7, training the improved network; setting a learning rate, a learning rate momentum, a batch size, a small total training round, weight attenuation and a maximum iteration number as training parameters, and training the improved YOLOv5 network;
and 8, inputting the collected insulator image data set into a trained YOLOv5 network to obtain whether a defect insulator exists in the input picture and the position of the defect.
In the insulator defect detection method based on the improved yolov5, the triplet attention module adopts three parallel branch structures, wherein two of the three branch structures are used for extracting the interdependence relationship between two spatial dimensions and the channel dimension C, and the other branch structure is used for extracting the spatial feature dependency relationship; in the first two branches, the triplet entry rotates the original input rotation tensor χ 90 ° counterclockwise along the H and W axes, respectively, and converts the shape of the tensor from cxhxc W to wxhxc C and hxc W; in the third branch, the tensor is input in the original shape C multiplied by H multiplied by W, the tensor of the C dimension is reduced to 2 dimensions through the Z-pool layer, and the average convergence feature and the maximum convergence feature in the dimension are connected;
z-pool is defined as: Z-Pool (x) [ MaxPool ]0d(χ),AvgPpool0d(χ)]
Wherein 0d is the 0 th dimension where the maximum pooling operation and the average pooling operation occur;
the simplified tensor passes through a standard convolution layer with the kernel size of K and a batch normalization layer, and attention weight of corresponding dimensionality generated by a Sigmoid activation function is added to the rotation tensor; at the final output, the output of the first branch is rotated 90 ° clockwise along the H axis, and the output of the second branch is rotated 90 ° clockwise along the W axis to ensure the same shape as the input; finally, the outputs of the three branches are evenly aggregated into an output;
the output tensor is defined as:
Figure BDA0003326095140000032
wherein σ is Sigmoid activation function, ψ1、ψ2、ψ3The standard two-dimensional convolution layers defined by the kernel size K in the three branches representing the triplet attentions,
Figure BDA0003326095140000033
respectively representing the tensors after rotation in the first two branches of the triplet entry,
Figure BDA0003326095140000034
respectively representing the tensors of the three branches of the triplet attention after passing through the Z-pool layer.
In the insulator defect detection method based on the improved yolov5, the flow of processing the final detection result by using Soft-NMS instead of NMS is as follows;
1) the prediction frames are sorted first, and then the prediction frame B with the highest score is usedMMove to the Final test ListIn D, all the rest prediction frames are allocated with the same mark Bi；
2) When a certain prediction box BiAnd BMIs larger than a certain threshold value NtWhen the Soft-NMS will recalculate the score S for that prediction boxiAnd compares it with a certain confidence threshold OtComparing, when the score of the prediction box is SiGreater than a certain confidence threshold OtIf so, moving the prediction box into a final detection list D, otherwise, deleting the prediction box;
3) for the rest frames BiThe above process is repeated until the initial list is empty.
In the insulator defect detection method based on the improved YOLOv5, the training of the improved YOLOv5 network comprises the following steps:
a. during network training, the data set is uniformly scaled to 640 × 640 size, and training is performed on an improved YOLOv5s network model with depth _ multiplex being 0.33 and width _ multiplex being 0.50;
b. the parameter updating mode is a random gradient descent SGD method, the initial learning rate is 0.01, the momentum term is 0.937, the weight attenuation coefficient is 0.0005, the batch size of model training is set to be 16, and the weight of the model is updated by regularization of a BN layer each time;
c. enhancement coefficients of hue H, saturation S, and brightness V are set to 0.015, 0.7, and 0.4, respectively; the total number of training rounds is set to 300;
d. after training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using a test set;
e. the final output of the network identifies the location frame of the insulator and its defects and the probability of belonging to a particular category.
Compared with the prior art, the method adds the triplet attention module in the backbone network to extract the semantic dependence between different dimensions and eliminate the indirect correspondence between the channel and the weight, thereby achieving the effect of improving the accuracy rate with smaller calculation cost. And modifying the DIoU loss function into the CIoU loss function so as to improve the accurate positioning capability of the model prediction frame and enhance the convergence effect of the model. And (3) adopting Soft-NMS to replace Non-Maximum Suppression (NMS) to process the detection result, solving the problem that the shielded detection object is easy to overlook and reducing the missing detection.
Drawings
Fig. 1 is a flow chart of a modified yolov 5-based insulator defect detection method according to an embodiment of the invention;
FIG. 2 is a diagram of a modified yolov5 network architecture according to one embodiment of the present invention;
FIG. 3 is a network structure diagram of a triple event module according to an embodiment of the present invention;
FIG. 4 is a flow chart of the Soft-NMS algorithm according to one embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the following embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The present invention is further illustrated by the following examples, which are not to be construed as limiting the invention.
In order to solve the problem that the existing insulator detection method is unbalanced in accuracy and speed, the embodiment improves the existing YOLOv5 algorithm, so that a better effect can be obtained in the aspect of insulator defect detection. The specific process of this embodiment is shown in fig. 1, and the original network is trained on the collected data set to obtain the experimental effect of the original network. Then, the data set is expanded, and the network is improved, so that the loss function, the backbone network and the result processing method of the network are mainly modified. Secondly, the improved network is trained by the expanded data set. And finally, adjusting network parameters according to the experimental result.
The embodiment is realized by the following technical scheme, and the insulator defect detection method based on the improved yolov5 comprises the following steps:
1. an insulator image dataset (containing defective insulator images and complete insulator images) is acquired.
2. And labeling the data set by using a LabelImg labeling tool, wherein the labeled types are insulators and defects.
3. Data enhancement processing is performed on the acquired images to expand the data set.
4. A triplet attention module is introduced into a backbone network of YOLOv 5.
Attention mechanisms have been widely used in recent years for various computer vision tasks, as they are able to establish interdependencies between channels or spatial locations. The attention mechanism can improve the standard convolutional layer generated feature representation by directly constructing a weighted spatial mask of the dependency or spatial attention between the channels. The purpose of learning attention weights is to allow the network the ability to learn where to focus and further on the target object. Unlike some attention approaches that require a large number of additional learning parameters, triplet attentions are proposed based on how to build a computationally small but effective attention, but while maintaining similar or providing better performance. the triple attribute establishes the dependency relationship between different dimensions through rotation operation and residual transformation, eliminates the indirect correspondence between the channel and the weight, and achieves the effect of improving the accuracy rate with smaller calculation overhead. The triplet attention module is introduced into the yolov5 backbone network.
5. The loss function is optimized.
CIoU is taken as a bounding-box Loss function Loss of the improved YOLOv5 algorithmCIoU. The CIOU takes the distance, the overlapping rate, the scale and the punishment items between the real frame and the prediction frame into consideration, so that the regression of the prediction frame becomes more stable, the problems of divergence and the like in the training process can not occur like other loss functions, and the punishment factors take the length-width ratio of the prediction frame to the length-width ratio of the target frame into consideration, so that the model can be converged more quickly and better.
CIoU is defined as:
Figure BDA0003326095140000061
wherein the content of the first and second substances,
Figure BDA0003326095140000062
for the weighting function, v is a parameter for measuring the uniformity of the aspect ratio, c represents the diagonal distance of the smallest closed region that can contain the prediction box and the real box, and ρ2(b，bgt) Representing the Euclidean distance, h, between the center points of the prediction and real framesgt、ωgtRespectively representing the height and width of the real box, hp、ωpRepresenting the height and width of the prediction box, respectively.
6. And optimizing a non-maximum suppression method for processing the final result.
Non-maximum suppression (NMS) is an integral part of the target detection process. First, it orders all prediction boxes according to score. Selecting the prediction box B with the largest scoreMAnd inhibit and BMAll other prediction blocks that have significant overlap (using a predefined threshold). This process is recursively applied to the remaining blocks. Depending on the design of the algorithm, if an object is within a predefined overlap threshold, it will result in missed detection. To this end, the present invention uses Soft-NMS as the method of processing the end result. The core idea of the Soft-NMS is to attenuate the detection scores of all other targets to be their sum with BMOverlapping continuous functions. Therefore, no object is eliminated during this process, thereby reducing missed detections.
The score function expression of Soft-NMS is:
Figure BDA0003326095140000063
wherein S isiAs a prediction block BiScore of (A), BMFor the highest scoring prediction box, NtIn order to be an overlap threshold value, the threshold value is,
Figure BDA0003326095140000071
gaussian penaltyThe function, σ, is an empirically chosen hyperparameter.
7. The improved network is trained. And setting training parameters such as learning rate, learning rate momentum, batch size, small total training round, weight attenuation, maximum iteration number and the like, and training the improved YOLOv5 network.
8. And inputting the collected insulator image data set into a trained YOLOv5 network to obtain whether a defective insulator exists in the input picture and the position of the defect.
The specific implementation of the insulator defect detection method based on the improved yolov5 comprises the following steps:
firstly, acquiring an insulator image to form a data set, and labeling the data set before the data set enters network model training to obtain an xml file conforming to a Pascal VOC data format, wherein the content of the xml file comprises the image name, the image path, the height/width of the image, the position of the center point of a real frame and the width/height information. The data set is then augmented by adaptive contrast, rotation, random gray scale variation, translation, cropping, color channel normalization, Mosaic, and the like. The Mosaic is to turn over, zoom, change color gamut and the like for four pictures respectively, and place the pictures according to four directions, and then form one picture, so that the capability of the model for detecting small targets is enhanced. The data set constructed by the invention contains 1268 images in total for deep learning, model training and detection requirement meeting.
Second, as shown in fig. 2, the improved YOLOv5 network consists of a backbone network, a neck network and a detection network. The backbone Network consists of Focus, convention with Batch simulation and LeakyRelu (CBL), Mix constraint (MixConv), Cross-Stage Partial Network (CSP), Spatial gradient Pooling (SPP), and triple attribute. The neck consists of CBL, CSP, Upesampling and AFF. In order to detect the position and the category of a target, characteristics need to be extracted from an image, and positioning and classification are carried out by using a backbone network to capture the characteristics; the neck network integrates the characteristics through the initial output characteristics of the backbone network, and adapts to the size, so that the overall performance of the system structure is improved; the detection network receives three outputs of the neck network, outputs a prediction of bounding box position, object confidence, and object class for each feature mapping output layer.
The Focus module is used for carrying out slicing operation on the feature map, and can reserve more complete image down-sampling data for subsequent feature extraction; the CBL module is used for extracting feature information in the feature map after slicing; the main idea of CSP networks is to generate two paths for the input, the main path having CBL or result, and the other path performing a convolution function to merge the results of the two paths. The CSP module reduces the calculation cost while ensuring the precision; the SPP module is used for realizing the fusion of the local features and the global features, thereby improving the representation capability of the feature map. Because the shape of the insulator defect is an object in different background images, the accuracy of the insulator defect detection is improved. The visual attention mechanism is a brain signal processing mechanism specific to human vision. Human vision obtains a target area needing important attention, namely a focus of attention in general, by rapidly scanning a global image, and then puts more attention resources into the area to obtain more detailed information of the target needing attention, and suppresses other useless information. Therefore, a triplet module is added into a backbone network of the YOLOv5 network to extract semantic dependence between different dimensions, so that indirect correspondence between channels and weights is eliminated, and the effect of improving accuracy is achieved with low calculation overhead.
The AFF module of the neck network is used to fuse attention-based features from the same or cross-over layer, including short-hop connections and long-hop connections, even within itself for initial fusion.
And thirdly, taking a YOLOv5 network as a basic framework, adding a triplet attention module into the YOLOv5 network, modifying a loss function into a CIoU loss function, and adopting Soft-NMS to replace NMS as an improvement basis to build an improved insulator defect detection network model.
(1) the basic structure of the triplet entry is shown in FIG. 3. the triplet entry adopts three parallel branch structures, two of which are used for extracting the interdependence between two spatial dimensions and the channel dimension C, and the other extracting the spatial feature dependency. In the first two branches, the triplet attention rotates the original input rotated tensor χ 90 ° counterclockwise along the H and W axes, respectively, and converts the shape of the tensor from cxhxw to wxhxc and hxcxw in the third branch, the tensor is input in its original shape cxhxw, then the C dimension tensor is reduced to 2 dimensions by the Z-pool layer and the average and maximum aggregate features in that dimension are connected, which enables the layer to reduce its depth to make further calculations lighter while preserving rich representations of the actual tensor.
Z-pool is defined as: Z-Pool (x) [ MaxPool ]0d(χ),AvgPpool0d(χ)]
Where 0d is the 0 th dimension where the maximum pooling operation and the average pooling operation occur. For example, Z-Pool with tensor shape C H W results in 2H W.
The simplified tensor is then passed through a standard convolution layer and a batch normalization layer with a kernel size of K, and finally the attention weight of the corresponding dimension generated by the Sigmoid activation function is added to the rotation tensor. At the final output, the output of the first branch is rotated 90 ° clockwise along the H axis and the output of the second branch is rotated 90 ° clockwise along the W axis to ensure the same shape as the input. Finally, the outputs of the three branches are aggregated evenly into an output.
The output tensor is defined as:
Figure BDA0003326095140000081
wherein σ is Sigmoid activation function, ψ1、ψ2、ψ3The standard two-dimensional convolution layers defined by the kernel size K in the three branches representing the triplet attentions,
Figure BDA0003326095140000091
respectively representing the tensors after rotation in the first two branches of the triplet entry,
Figure BDA0003326095140000092
respectively representing the tensors of the three branches of the triplet attention after passing through the Z-pool layer.
A triplet attention module is added into a backbone network of the YOLOv5 network to extract semantic dependence among different dimensions, and indirect correspondence between channels and weights is eliminated, so that the effect of improving accuracy is achieved with low calculation cost.
(2) The loss function is optimized. IoU is cross-over ratio, which is a common index in target detection, and its main function is to determine positive and negative samples and calculate the distance between the prediction box and the real box. The definition of IoU presents a problem in itself. If the two prediction and real boxes are disjoint, IoU is 0. Meanwhile, due to 0 loss, there is no slope retreat; therefore, learning and training exercises cannot be performed. To solve these problems, the GIoU idea is proposed.
The principle of GIoU is:
Figure BDA0003326095140000093
however, the algorithm still has the problems of unstable regression of the target frame, easy divergence and the like in the training process. Some of the blocks of target detection do not overlap resulting in a possible regression strategy for GIoU to regress to the IoU regression strategy. Therefore, in order to directly minimize the normalized distance between the prediction frame and the real frame to achieve faster convergence speed, and make the regression more accurate and fast when overlapping with or even containing the real frame, DIoU is proposed.
The principle of DIoU is:
Figure BDA0003326095140000094
wherein, bgtThe center points of the prediction box and the real box are respectively represented. However, the DIoU calculation does not consider the aspect ratio, but only considers the overlapping area of the bounding box and bgtDistance of the center point of (a). However, the consistency of the w and h ratios between the prediction and real boxes is also highly significant. On this basis, CIoU loss is proposed. The penalty term of the CIoU is based on the penalty term of the DIoU plus influence factors alpha, v which take into account the aspect ratio of the prediction box to fit the real box.
The principle of the CIoU is as follows:
Figure BDA0003326095140000095
wherein the content of the first and second substances,
Figure BDA0003326095140000096
for the weighting function, v is a parameter for measuring the uniformity of the aspect ratio, c represents the diagonal distance of the smallest closed region that can contain the prediction box and the real box, and ρ2(b，bgt) Representing the Euclidean distance, h, between the center points of the prediction and real framesgt、ωgtRespectively representing the height and width of the real box, hp、ωpRepresenting the height and width of the prediction box, respectively.
Therefore, in the embodiment, the DIoU loss function is modified into the CIoU loss function, so that the accurate positioning capability of the model prediction framework is improved, and the convergence effect of the model is enhanced.
(3) To further improve the detection of occluded targets by the algorithm, Non-Maximum Suppression (NMS) will also be optimized. NMS is applied to most of the most advanced detectors to obtain the final result, since it greatly reduces the number of false positives. The NMS algorithm flow is described generally as follows
Firstly, sorting all prediction frames in a list according to confidence; secondly, the prediction box B with the highest score is usedMMoving to a final detection list D, and allocating unique identifiers B to the rest of prediction boxesi(ii) a Third, remove and BMThe overlapping area is larger than a certain threshold value NtAny prediction block B ofi(ii) a For the rest frames BiThis process is repeated until the initial list is empty. Since in the NMS a certain threshold is set to decide BMWhich boxes in the field should be retained and which should be deleted. However, if an object does exist, but with BMHas an overlap ratio of more than NtIts detection will be ignored.
The scoring function expression of NMS is:
Figure BDA0003326095140000101
the core idea of Soft-NMS is to use penalty function to attenuate and BMInstead of setting these scores to zero, the scores of the overlapping prediction boxes. The present invention therefore uses the Soft-NMS instead of the NMS to process the final test results.
The flow chart of the Soft-NMS algorithm is shown in FIG. 4, and the prediction boxes are sorted first, and then the prediction box B with the highest score is usedMMoving to a final detection list D, and distributing the same identification B to all the rest prediction boxesi(ii) a But when a certain prediction box BiAnd BMIs larger than a certain threshold value NtInstead of deleting the prediction box, the Soft-NMS will recalculate the score S for the prediction boxiAnd compares it with a certain confidence threshold OtComparing, when the score of the prediction box is SiGreater than a certain confidence threshold OtIf so, moving the prediction box into a final detection list D, otherwise, deleting the prediction box; for the rest frames BiThe above process is repeated until the initial list is empty. It follows that the sum of B and the sum of B can be calculated using the Soft-NMS algorithmMThe prediction box score with large overlap will be greatly reduced, while the distance BMThe farther prediction blocks are not affected. Therefore, the missed detection rate of the network can be greatly reduced.
The score function expression of Soft-NMS is:
Figure BDA0003326095140000111
wherein S isiAs a prediction block BiScore of (A), BMFor the highest scoring prediction box, NtIn order to be an overlap threshold value, the threshold value is,
Figure BDA0003326095140000112
the gaussian penalty function, σ, is an empirically selected hyperparameter.
(4) The improved YOLOv5 network was trained. During network training, the data set is uniformly scaled to 640 × 640 size, and training is performed on an improved YOLOv5s network model with depth _ multiplex being 0.33 and width _ multiplex being 0.50. The parameter updating method is a random gradient descent (SGD) method, the initial learning rate is 0.01, the momentum term is 0.937, the weight attenuation coefficient is 0.0005, the batch size of model training is set to be 16, and the weight of the model is updated by regularization of a BN layer each time. Enhancement coefficients of hue (H), saturation (S), and brightness (V) were set to 0.015, 0.7, and 0.4, respectively. The total number of training rounds was set to 300. And after the training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using the test set. The final output of the network is the location box identifying the insulator and its defects and the probability of belonging to a particular category.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.

Claims (4)

1. Insulator defect detection method based on modified yolov5, its characterized in that: the method comprises the following steps:
step 1, collecting an insulator image forming data set;
step 2, labeling the data set by using a LabelImg labeling tool, wherein the labeled types are insulators and defects;
step 3, performing data enhancement processing on the acquired image to expand a data set;
step 4, introducing a triplet attention module into a backbone network of YOLOv 5;
step 5, optimizing a loss function; CIoU is taken as a bounding-box Loss function Loss of the improved YOLOv5 algorithmCIoU；
CIoU is defined as:
Figure FDA0003326095130000011
wherein the content of the first and second substances,
Figure FDA0003326095130000012
alpha is a weight function, v is a parameter for measuring the uniformity of the aspect ratioNumber, c represents the diagonal distance, ρ, of the smallest enclosed area that can contain the predicted box and the real box2(b，bgt) Representing the Euclidean distance, h, between the center points of the prediction and real framesgt、ωgtRespectively representing the height and width of the real box, hp、ωpRespectively representing the height and width of the prediction box;
step 6, optimizing the non-maximum value inhibition NMS method for processing the final result; Soft-NMS was used as a method to process the final results:
the score function expression of Soft-NMS is:
Figure FDA0003326095130000013
wherein S isiAs a prediction block BiScore of (A), BMFor the highest scoring prediction box, NtIn order to be an overlap threshold value, the threshold value is,
Figure FDA0003326095130000014
a Gaussian penalty function, sigma is a hyperparameter selected according to experience;
step 7, training the improved network; setting a learning rate, a learning rate momentum, a batch size, a small total training round, weight attenuation and a maximum iteration number as training parameters, and training the improved YOLOv5 network;
and 8, inputting the collected insulator image data set into a trained YOLOv5 network to obtain whether a defect insulator exists in the input picture and the position of the defect.
2. The improved yolov 5-based insulator defect detection method according to claim 1, wherein the method comprises the following steps: the triplet attention module adopts three parallel branch structures, wherein two of the three parallel branch structures are used for extracting the interdependence relation between two space dimensions and the channel dimension C, and the other branch structure is used for extracting the space characteristic interdependence relation; in the first two branches, the triplet entry rotates the original input rotation tensor χ 90 ° counterclockwise along the H and W axes, respectively, and converts the shape of the tensor from cxhxc W to wxhxc C and hxc W; in the third branch, the tensor is input in the original shape C multiplied by H multiplied by W, the tensor of the C dimension is reduced to 2 dimensions through the Z-pool layer, and the average convergence feature and the maximum convergence feature in the dimension are connected;
z-pool is defined as: Z-Pool (x) [ MaxPool ]0d(χ),AvgPpool0d(χ)]
Wherein 0d is the 0 th dimension where the maximum pooling operation and the average pooling operation occur;
the simplified tensor passes through a standard convolution layer with the kernel size of K and a batch normalization layer, and attention weight of corresponding dimensionality generated by a Sigmoid activation function is added to the rotation tensor; at the final output, the output of the first branch is rotated 90 ° clockwise along the H axis, and the output of the second branch is rotated 90 ° clockwise along the W axis to ensure the same shape as the input; finally, the outputs of the three branches are evenly aggregated into an output;
the output tensor is defined as:
Figure FDA0003326095130000021
wherein σ is Sigmoid activation function, ψ1、ψ2、ψ3The standard two-dimensional convolution layers defined by the kernel size K in the three branches representing the triplet attentions,
Figure FDA0003326095130000022
respectively representing the tensors after rotation in the first two branches of the triplet entry,
Figure FDA0003326095130000023
respectively representing the tensors of the three branches of the triplet attention after passing through the Z-pool layer.
3. The improved yolov 5-based insulator defect detection method according to claim 1, wherein the method comprises the following steps: the flow of processing the final detection result by adopting Soft-NMS to replace NMS is as follows;
1) the prediction boxes are sorted first, and then the prediction box with the highest score is sortedBMMoving to a final detection list D, and distributing the same identification B to all the rest prediction boxesi；
2) When a certain prediction box BiAnd BMIs larger than a certain threshold value NtWhen the Soft-NMS will recalculate the score S for that prediction boxiAnd compares it with a certain confidence threshold OtComparing, when the score of the prediction box is SiGreater than a certain confidence threshold OtIf so, moving the prediction box into a final detection list D, otherwise, deleting the prediction box;
3) for the rest frames BiThe above process is repeated until the initial list is empty.
4. The improved yolov 5-based insulator defect detection method according to claim 1, wherein the method comprises the following steps: training the improved YOLOv5 network includes the steps of:
a. during network training, the data set is uniformly scaled to 640 × 640 size, and training is performed on an improved YOLOv5s network model with depth _ multiplex being 0.33 and width _ multiplex being 0.50;
b. the parameter updating mode is a random gradient descent SGD method, the initial learning rate is 0.01, the momentum term is 0.937, the weight attenuation coefficient is 0.0005, the batch size of model training is set to be 16, and the weight of the model is updated by regularization of a BN layer each time;
c. enhancement coefficients of hue H, saturation S, and brightness V are set to 0.015, 0.7, and 0.4, respectively; the total number of training rounds is set to 300;
d. after training is finished, storing the obtained weight file of the recognition model, and evaluating the performance of the model by using a test set;
e. the final output of the network identifies the location frame of the insulator and its defects and the probability of belonging to a particular category.
CN202111261977.5A 2021-10-28 2021-10-28 Insulator defect detection method based on improved yolov5 Active CN114022432B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111261977.5A CN114022432B (en) 2021-10-28 2021-10-28 Insulator defect detection method based on improved yolov5

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111261977.5A CN114022432B (en) 2021-10-28 2021-10-28 Insulator defect detection method based on improved yolov5

Publications (2)

Publication Number Publication Date
CN114022432A true CN114022432A (en) 2022-02-08
CN114022432B CN114022432B (en) 2024-04-30

Family

ID=80058216

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111261977.5A Active CN114022432B (en) 2021-10-28 2021-10-28 Insulator defect detection method based on improved yolov5

Country Status (1)

Country Link
CN (1) CN114022432B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术（天津）有限公司 Roof panel defect detection method and system
CN114677362A (en) * 2022-04-08 2022-06-28 四川大学 Surface defect detection method based on improved YOLOv5
CN114882410A (en) * 2022-05-11 2022-08-09 华东交通大学 Tunnel ceiling lamp fault detection method and system based on improved positioning loss function
CN114972261A (en) * 2022-05-27 2022-08-30 东北大学 Method for identifying surface quality defects of plate strip steel
CN115311542A (en) * 2022-08-25 2022-11-08 杭州恒胜电子科技有限公司 Target detection method, device, equipment and medium
CN115410060A (en) * 2022-11-01 2022-11-29 山东省人工智能研究院 Public safety video-oriented global perception small target intelligent detection method
CN115619778A (en) * 2022-12-06 2023-01-17 南京迈能能源科技有限公司 Power equipment defect identification method and system, readable storage medium and equipment
CN116311077A (en) * 2023-04-10 2023-06-23 东北大学 Pedestrian detection method and device based on multispectral fusion of saliency map
CN116468730A (en) * 2023-06-20 2023-07-21 齐鲁工业大学(山东省科学院) Aerial insulator image defect detection method based on YOLOv5 algorithm
CN117036363A (en) * 2023-10-10 2023-11-10 国网四川省电力公司信息通信公司 Shielding insulator detection method based on multi-feature fusion

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018165753A1 (en) * 2017-03-14 2018-09-20 University Of Manitoba Structure defect detection using machine learning algorithms
CN108961235A (en) * 2018-06-29 2018-12-07 山东大学 A kind of disordered insulator recognition methods based on YOLOv3 network and particle filter algorithm
CN111311597A (en) * 2020-03-27 2020-06-19 国网福建省电力有限公司龙岩供电公司 Unmanned aerial vehicle inspection method and system for defective insulator
CN111583198A (en) * 2020-04-23 2020-08-25 浙江大学 Insulator picture defect detection method combining FasterR-CNN + ResNet101+ FPN
CN112184654A (en) * 2020-09-24 2021-01-05 上海电力大学 High-voltage line insulator defect detection method based on generation countermeasure network
CN112819804A (en) * 2021-02-23 2021-05-18 西北工业大学 Insulator defect detection method based on improved YOLOv5 convolutional neural network
WO2021139069A1 (en) * 2020-01-09 2021-07-15 南京信息工程大学 General target detection method for adaptive attention guidance mechanism
CN113205505A (en) * 2021-05-14 2021-08-03 湖北工业大学 Insulator defect detection method based on improved ResNeSt-RPN
CN113297996A (en) * 2021-05-31 2021-08-24 贵州电网有限责任公司 Unmanned aerial vehicle aerial photographing insulator target detection method based on YoloV3
CN113379699A (en) * 2021-06-08 2021-09-10 上海电机学院 Transmission line insulator defect detection method based on deep learning
CN113506290A (en) * 2021-07-29 2021-10-15 广东电网有限责任公司 Method and device for detecting defects of line insulator

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018165753A1 (en) * 2017-03-14 2018-09-20 University Of Manitoba Structure defect detection using machine learning algorithms
CN108961235A (en) * 2018-06-29 2018-12-07 山东大学 A kind of disordered insulator recognition methods based on YOLOv3 network and particle filter algorithm
WO2021139069A1 (en) * 2020-01-09 2021-07-15 南京信息工程大学 General target detection method for adaptive attention guidance mechanism
CN111311597A (en) * 2020-03-27 2020-06-19 国网福建省电力有限公司龙岩供电公司 Unmanned aerial vehicle inspection method and system for defective insulator
CN111583198A (en) * 2020-04-23 2020-08-25 浙江大学 Insulator picture defect detection method combining FasterR-CNN + ResNet101+ FPN
CN112184654A (en) * 2020-09-24 2021-01-05 上海电力大学 High-voltage line insulator defect detection method based on generation countermeasure network
CN112819804A (en) * 2021-02-23 2021-05-18 西北工业大学 Insulator defect detection method based on improved YOLOv5 convolutional neural network
CN113205505A (en) * 2021-05-14 2021-08-03 湖北工业大学 Insulator defect detection method based on improved ResNeSt-RPN
CN113297996A (en) * 2021-05-31 2021-08-24 贵州电网有限责任公司 Unmanned aerial vehicle aerial photographing insulator target detection method based on YoloV3
CN113379699A (en) * 2021-06-08 2021-09-10 上海电机学院 Transmission line insulator defect detection method based on deep learning
CN113506290A (en) * 2021-07-29 2021-10-15 广东电网有限责任公司 Method and device for detecting defects of line insulator

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
丘灵华;朱铮涛;: "基于深度学习的输电线路绝缘子缺陷检测研究", 计算机应用研究, no. 1, 30 June 2020 (2020-06-30) *
张晓春等: "基于红外图像匹配的零值绝缘子检测", 《电测与仪表》, vol. 43, no. 03, 31 December 2019 (2019-12-31) *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114677362A (en) * 2022-04-08 2022-06-28 四川大学 Surface defect detection method based on improved YOLOv5
CN114677362B (en) * 2022-04-08 2023-09-12 四川大学 Surface defect detection method based on improved YOLOv5
CN114529546A (en) * 2022-04-24 2022-05-24 科大天工智能装备技术（天津）有限公司 Roof panel defect detection method and system
CN114882410B (en) * 2022-05-11 2023-09-12 华东交通大学 Tunnel dome lamp fault detection method and system based on improved positioning loss function
CN114882410A (en) * 2022-05-11 2022-08-09 华东交通大学 Tunnel ceiling lamp fault detection method and system based on improved positioning loss function
CN114972261A (en) * 2022-05-27 2022-08-30 东北大学 Method for identifying surface quality defects of plate strip steel
CN115311542A (en) * 2022-08-25 2022-11-08 杭州恒胜电子科技有限公司 Target detection method, device, equipment and medium
CN115410060A (en) * 2022-11-01 2022-11-29 山东省人工智能研究院 Public safety video-oriented global perception small target intelligent detection method
CN115619778A (en) * 2022-12-06 2023-01-17 南京迈能能源科技有限公司 Power equipment defect identification method and system, readable storage medium and equipment
CN116311077A (en) * 2023-04-10 2023-06-23 东北大学 Pedestrian detection method and device based on multispectral fusion of saliency map
CN116311077B (en) * 2023-04-10 2023-11-07 东北大学 Pedestrian detection method and device based on multispectral fusion of saliency map
CN116468730B (en) * 2023-06-20 2023-09-05 齐鲁工业大学(山东省科学院) Aerial Insulator Image Defect Detection Method Based on YOLOv5 Algorithm
CN116468730A (en) * 2023-06-20 2023-07-21 齐鲁工业大学(山东省科学院) Aerial insulator image defect detection method based on YOLOv5 algorithm
CN117036363A (en) * 2023-10-10 2023-11-10 国网四川省电力公司信息通信公司 Shielding insulator detection method based on multi-feature fusion
CN117036363B (en) * 2023-10-10 2024-01-30 国网四川省电力公司信息通信公司 Shielding insulator detection method based on multi-feature fusion

Also Published As

Publication number Publication date
CN114022432B (en) 2024-04-30

Similar Documents

Publication Publication Date Title
CN114022432A (en) Improved yolov 5-based insulator defect detection method
CN110414377B (en) Remote sensing image scene classification method based on scale attention network
CN111723654B (en) High-altitude parabolic detection method and device based on background modeling, YOLOv3 and self-optimization
CN109359559B (en) Pedestrian re-identification method based on dynamic shielding sample
CN108647655B (en) Low-altitude aerial image power line foreign matter detection method based on light convolutional neural network
CN108229550B (en) Cloud picture classification method based on multi-granularity cascade forest network
CN110032925B (en) Gesture image segmentation and recognition method based on improved capsule network and algorithm
CN112348036A (en) Self-adaptive target detection method based on lightweight residual learning and deconvolution cascade
CN112132197A (en) Model training method, image processing method, device, computer equipment and storage medium
CN112464911A (en) Improved YOLOv 3-tiny-based traffic sign detection and identification method
CN111160407A (en) Deep learning target detection method and system
CN109670555B (en) Instance-level pedestrian detection and pedestrian re-recognition system based on deep learning
CN110222636B (en) Pedestrian attribute identification method based on background suppression
CN115294473A (en) Insulator fault identification method and system based on target detection and instance segmentation
CN114627269A (en) Virtual reality security protection monitoring platform based on degree of depth learning target detection
CN111126155B (en) Pedestrian re-identification method for generating countermeasure network based on semantic constraint
Wang et al. Fault detection for power line based on convolution neural network
CN113536944A (en) Distribution line inspection data identification and analysis method based on image identification
CN111291785A (en) Target detection method, device, equipment and storage medium
CN112396126B (en) Target detection method and system based on detection trunk and local feature optimization
CN114694019A (en) Remote sensing image building migration extraction method based on anomaly detection
CN114359167A (en) Insulator defect detection method based on lightweight YOLOv4 in complex scene
CN112380985A (en) Real-time detection method for intrusion foreign matters in transformer substation
CN117152746B (en) Method for acquiring cervical cell classification parameters based on YOLOV5 network
Song et al. Detection of Insulator Defects Based on Improved YOLOv3

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant