CN117132531A - Lightweight-based YOLOv5 insulator defect detection method - Google Patents

Lightweight-based YOLOv5 insulator defect detection method Download PDF

Info

Publication number
CN117132531A
CN117132531A CN202310596152.1A CN202310596152A CN117132531A CN 117132531 A CN117132531 A CN 117132531A CN 202310596152 A CN202310596152 A CN 202310596152A CN 117132531 A CN117132531 A CN 117132531A
Authority
CN
China
Prior art keywords
convolution
feature
algorithm
yolov5
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310596152.1A
Other languages
Chinese (zh)
Inventor
房国志
于铭旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin University of Science and Technology
Original Assignee
Harbin University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin University of Science and Technology filed Critical Harbin University of Science and Technology
Priority to CN202310596152.1A priority Critical patent/CN117132531A/en
Publication of CN117132531A publication Critical patent/CN117132531A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Quality & Reliability (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

A lightweight YOLOv 5-based insulator defect detection method belongs to the field of image recognition algorithms. The insulator damage detection is performed by adopting aerial images through traditional image processing, so that the problem of low detection precision exists. A method for detecting defects of an insulator based on light-weight YOLOv5 comprises the steps of firstly improving a YOLOv5s algorithm by using an attention mechanism module ECA-Net network, and optimizing a model structure; and then, the modified YOLOv5 algorithm is subjected to light weight treatment by using a MobileNetv2 model. The method can reduce the calculation complexity, improve the detection speed and meet the requirement of real-time detection.

Description

Lightweight-based YOLOv5 insulator defect detection method
Technical Field
The invention relates to a light-weight-based YOLOv5 insulator defect detection method.
Background
With the development of society, the demands of various industries, particularly industry and commerce, on electric power are greatly improved, the investment and construction of a power system in China are increasingly accelerated, and in the power system, a transmission line is used as a carrier for electric energy transmission and is the most critical one.
The transmission line mainly comprises equipment such as an insulator, a ground wire and the like, wherein the insulator is one of the most important and indispensable components in the transmission network. Because the insulators are distributed more, the problems of string falling, self-explosion, dirt and the like often occur due to the influence of natural environment and the factors of service life, so that the insulators work abnormally and cause faults, and unnecessary losses are caused. It is counted that the accidents caused by the faults of the insulators are more than half. In recent years, the situation that the line is shut down and disconnected on a large scale is caused by the fact that excessive insulator pollution and flashover occur in a power grid, and great loss and influence are brought to enterprises. Therefore, regular inspection of insulators is a primary task.
In order to ensure the normal operation of the power grid, a large amount of resources are consumed each year to maintain the power line so as to eliminate hidden danger in time, prevent further deterioration and ensure the normal, stable and safe operation of the line. The power transmission line inspection method comprises manual inspection, helicopter inspection and unmanned aerial vehicle inspection. Traditional manual inspection is low in efficiency and safe and difficult to guarantee. Although helicopter inspection can improve inspection efficiency, the defects of high cost and low flexibility of the helicopter inspection can not be applied on a large scale. With the rapid development of artificial intelligence and the mature application of unmanned aerial vehicle technology, unmanned aerial vehicle inspection is widely used. Unmanned aerial vehicle patrols and examines has safe, efficient advantage, utilizes the airborne camera equipment to shoot the power equipment in the transmission line, and the rethread is artifical to patrol and examine the image discrimination and judge whether there is the trouble. However, the inspection image has large data characteristics of large volume and low value density, and misjudgment and missed judgment caused by visual fatigue are easy to occur in manual interpretation. With the improvement of the processing capacity of hardware equipment and the gradual rise of a deep learning technology and a target recognition technology, the acquisition and transmission of aerial images become very convenient, the requirements for accurately positioning and recognizing the images are gradually improved, and related technologies enter a high-speed development channel.
The unmanned aerial vehicle is high in resolution, complex in background and numerous in color of images acquired in the inspection process, the acquired insulators are various in types, the insulator positions are judged by naked eyes, whether the insulator positions are normal or not is judged, and the requirements of automation and intellectualization are difficult to meet. At present, intelligent power line equipment identification technology based on inspection image data is being updated and iterated at a high speed. The traditional target recognition method mainly comprises the steps of firstly extracting features, and then adopting a classifier to perform feature recognition, such as an edge detection algorithm, a burn algorithm, an LSD algorithm, a HOG algorithm, a SIFT algorithm, an SHT algorithm and the like, and has the problems of low recognition precision, poor robustness to various environments, no pertinence, time consumption and the like. The depth convolution model can be used for accurate identification and judgment of target detection and is applied to real-time detection. For example, a target detection algorithm that extracts target features through CNN (ConvolutionalNeuralNetworks) may perform accurate recognition. The current target detection algorithms can be mainly divided into two types: one is a two-order algorithm based on a detection frame and a classifier, such as FasterR-CNN, masker-CNN and the like, and the detection accuracy is high, but the speed is slower due to the deeper network structure; the other is a regression-based first-order algorithm, such as YOLO (YouOnlyLookOnce) and SSD (SingleShotMultiBoxDetector), and the like, and is characterized in that end-to-end detection is adopted, the detection speed is high, and the requirement of real-time detection is met.
The application of the novel technology enables the speed of unmanned aerial vehicle images detected by a computer to be greatly improved, thereby greatly reducing the workload of field personnel and improving the inspection effect. Therefore, the method is designed based on the YOLOv5 algorithm, is suitable for an efficient, intelligent and real-time insulator defect detection method under a complex background, and has wide practical application value and practical significance for on-line monitoring and intelligent inspection of a power grid system.
In summary, in the conventional insulator image detection method, if hough transform straight line detection, edge detection, watershed algorithm, local gradient detection, and through space and color information, manual intervention is usually required to obtain a better detection effect when extracting features, and the adaptability is weak, the generalization capability is not strong enough, and the method is only suitable for simple background or large target images. In addition, the accuracy of the primary detection in the traditional method is difficult to further improve in the post-processing of the image, and the requirement of high-precision and effective detection of the target in the complex image background can not be met.
The images acquired by the unmanned aerial vehicle often have the problems of complex background, large interference to detection targets, uneven illumination, complex image feature distribution, different target sizes, target overlapping and the like, so that the high-precision detection of the insulator is difficult to achieve the expected effect. These all make high-precision and effective detection of insulator damage by using aerial images through traditional image processing difficult.
Therefore, it is necessary to find a deep learning target detection algorithm aiming at the characteristics of the aerial image, and the rapidity and the effectiveness are both considered. According to actual tests and researches, the YOLO series algorithm has higher detection speed, and based on the detection speed, the invention provides an improved algorithm for detecting the defects of the insulators aiming at YOLOv5s, so that the detection accuracy and the detection speed are higher.
Disclosure of Invention
The invention aims to solve the problems that an image acquired by an unmanned aerial vehicle is often complicated in background, a detection target is greatly interfered, illumination is uneven, image characteristics are complex in distribution, targets are different in size, targets are overlapped and the like, so that the high-precision detection of an insulator is difficult to achieve an expected effect, and the problem that the detection precision of detecting the damage of the insulator is low by adopting an aerial image through traditional image processing is solved, and provides a lightweight YOLOv 5-based insulator defect detection method.
The above object is achieved by the following technical scheme:
according to the insulator defect detection method based on lightweight YOLOv5, firstly, an attention mechanism module ECA-Net network is utilized to improve a YOLOv5s algorithm, and a model structure is optimized; and then, the modified YOLOv5 algorithm is subjected to light weight treatment by using a MobileNetv2 model.
Further, the operation of improving the YOLOv5s algorithm by using the attention mechanism module ECA-Net network comprises the steps of adding an attention mechanism, designing a multi-scale feature fusion structure and designing a candidate box optimization algorithm.
Further, the process of adding the attention mechanism specifically comprises the following steps:
the ECA-Net network is based on an SE-Net improved channel attention model, and one-dimensional convolution with a convolution kernel of k is used for replacing a full-connection layer to carry out channel weighting of k adjacent ranges, so that local intersection and channel interaction are realized, the attention of the network to local characteristic information is enhanced, and the distinguishing capability of a background and a target is enhanced; SE-Net refers to the Squeeze-and-specifiationNetworks, chinese meaning compression and excitation network;
the k-value is calculated as follows, where channels is the number of channels of the input feature:
further, the operation of designing the multi-scale feature fusion structure is to use the three-scale feature graphs for detecting targets with different sizes after the model outputs three features through the backbone feature extraction layer; distinguishing simple targets by shallow features, distinguishing complex targets by deep features, and simultaneously carrying out feature fusion on a multi-scale feature map, wherein the method comprises the following steps:
(1) Combining a Bi-directional feature pyramid network Bi-FPN at a reinforced feature extraction layer part of YOLOv5, and fusing bidirectional cross-scale connection and weighted feature graphs, fusing features of different scales, combining features of a shallower layer under the same scale, and repeatedly stacking by taking one Bi-FPN as a circulating unit to obtain more high-level feature fusion;
(2) Designing a feature fusion layer structure based on Bi-FPN;
a circulation unit is added in the Bi-FPN structure, namely, the circulation unit comprises two Bi-FPN sub-blocks, the sub-blocks are beneficial to further enhancing the fusion of the features and compensating the information loss of different feature layers; meanwhile, a new residual error path is added in the characteristics of the middle layer size, so that information loss is prevented.
Further, the candidate box optimization algorithm specifically comprises the following operations:
taking the obtained IoU to take a Gaussian index, multiplying the Gaussian index by an original score in a weight form, and then reordering; the calculation process is as follows: b= { b_1, …, b_n } N candidate boxes are input, the scores of the candidate boxes are s= { s_1, …, s_n }, N is a threshold value for NMS algorithm suppression, and the maximum probability M is selected; wherein IoU is the ratio of the intersection to the union of two regions;
the calculation formula of the Soft-NMS algorithm is shown as follows:
n represents the threshold value of the contact ratio of the rest target and the target with the largest probability, if the contact ratio is larger than the threshold value, the contact ratio of the rest target and the target is higher, and in the condition that the contact ratio information is the same, the candidate region with the largest probability is selected as the candidate region of the category, so that the other region is eliminated; if the overlap ratio is small, the new candidate region is different from the information contained in the region with the maximum probability, so that more different candidate regions are obtained.
Further, the operation of performing the light-weight processing on the improved YOLOv5 algorithm by using the MobileNetv2 model specifically includes:
(1) The design is based on a MobileNet v2 lightweight model:
the MobileNet v2 network structure is formed by stacking a series of Bottleneck, the backbone network of the YOLOv5 network is replaced by Bottleneck in MobileNet v2, the Bottleneck is divided into two structures according to different steps, when the Bottleneck step is 1, the input characteristic diagram firstly expands the channel number through convolution of 1X 1, and the expansion coefficient is 6; the middle layer is a 3 multiplied by 3 depth convolution, the last layer is a 1 multiplied by 1 point convolution and a linear activation function, an output characteristic diagram is obtained, element-by-element addition operation is carried out on the output characteristic diagram and an input characteristic diagram, and the obtained characteristic diagram is input into the next layer; when the Bottleneck stride is 2, the middle layer is a depth convolution with the size of 3×3, and no jump connection and element-by-element addition operation exist in the structure, and the rest of the structure is the same as the Bottleneck module with the stride of 1;
(2) Replacing the normal convolution with a depth separable convolution at the YOLOv5 feature fusion portion; the depth separable convolution comprises two processes, namely a channel-by-channel convolution and a point-by-point convolution, wherein the channel-by-channel convolution is only convolved by a convolution kernel, and the number of channels of a feature map generated by the process is the same as the number of channels of an input; the size of a convolution kernel of point-by-point convolution is 1 multiplied by M, M is the number of channels of the previous layer, the convolution operation carries out weighted combination on the feature images of the previous step in the depth direction to generate new feature images, and the number of the output feature images is the same as that of the convolution kernels;
the input of the network is D F ×D F X M, output is D F ×D F X N, convolution kernel size D k ×D k The standard convolution parameter is D k ×D k X M N, calculated as D F ×D F ×D k ×D k X M x N; applying input and output of the same size of the depth separable convolution, the obtained parameter quantity is D k ×D k ×M×D F ×D F +M×N×D F ×D F The method comprises the steps of carrying out a first treatment on the surface of the The convolution is represented as a depth separable convolution, reducing the computational effort by:
the beneficial effects of the invention are as follows:
in order to detect the insulator defect in the complex background in the high-resolution aerial image, a high-precision insulator defect detection algorithm based on YOLOv5 is provided, and the precision of insulator defect detection is improved; aiming at the problems of high model complexity and large calculation resources of the improved YOLOv5 algorithm model, the light-weight YOLOv5 insulator defect detection algorithm is provided, the complexity is reduced, the detection speed is improved, and the requirement of real-time detection is met.
According to the invention, an aerial insulator is taken as a research object, and for the problem of insulator defect detection in a power transmission line, the method is suitable for insulator defect detection by designing an improved algorithm based on a YOLOv5 model. The method aims at the application scene of insulator detection on the actual transmission line, improves the original target detection model, enables the whole network to be more attached to the insulator for detection, and improves detection accuracy. The improved YOLOv5 algorithm model has higher precision, but has the problem that the computing resource and the complexity of the model exceed the capacities of a mobile terminal and embedded equipment. A YOLOv5 insulator defect detection algorithm based on light weight is provided from the light weight perspective, so that the complexity of a model is reduced, and the detection speed is improved. The method comprises the following steps:
(1) Aiming at the problem of lower accuracy of insulator defect detection under a complex background in an aerial image, an ECA-Net channel attention mechanism is fused at a skeleton feature extraction layer of YOLOv5, and effective distinction between a background and a target insulator is realized by increasing the weight of an important channel; adding a Bi-FPN bidirectional feature pyramid network into the feature fusion layer to effectively reserve the defect features of the target insulator; and a detection frame algorithm Soft-NMS is integrated into the prediction layer to screen the target frame again, so that the false deletion probability of the overlapped insulator is reduced, and the defect detection accuracy of the insulator is improved.
(2) Aiming at the problems of overlarge improved YOLOv5 model and complex calculation. The method has the advantages that the lightweight model MobileNet v2 is utilized to optimize the YOLOv5 feature extraction network, the main network of the YOLOv5 network is replaced by the Bottleneck in the MobileNet v2, and the common convolution of the feature fusion part is replaced by the deep separable convolution, so that the complexity of the model is reduced, the detection speed is improved, and the requirement of real-time detection is met.
Drawings
FIG. 1 is a diagram of an ECA-Net network architecture in accordance with the present invention;
FIG. 2 is a diagram of a Bi-FPN network architecture according to the present invention;
FIG. 3 is a block diagram of a Bi-FPN based feature fusion layer according to the present invention;
FIG. 4 is a flow chart of a Soft-NMS calculation according to the present invention;
FIG. 5 is a residual block diagram in accordance with the present invention;
FIG. 6 is an inverted residual block diagram in accordance with the present invention;
FIG. 7 is a diagram of the Bottleneck structure of the present invention;
fig. 8 is a depth separable convolution process in accordance with the present invention.
Detailed Description
Referring to fig. 1-8, the present invention provides a technical solution:
a lightweight YOLOv 5-based insulator defect detection method comprises the following steps:
(1) Designing a high-precision insulator defect detection algorithm based on YOLOv 5;
under the condition of high similarity between an insulator and a background, false detection and omission detection are easy to occur, and an attention mechanism algorithm is fused at a backbone feature extraction layer of a YOLOv5 model.
The problem of smaller target areas of defective insulators in the data results in a model with poor effect in final detection. By analyzing the defect of the YOLOv5 model in the feature fusion processing layer, a new multi-scale feature fusion algorithm is combined with the YOLOv5 model, and the Bi-FPN (Bi-directionFeaturyRamiddnworks) multi-scale feature fusion algorithm is combined with the YOLOv5 model, so that the insulator defect detection precision is improved.
In order to solve the problem that the accuracy of the model is reduced due to the fact that the insulator target portion is blocked in the data set, by analyzing the defects of the existing candidate frame algorithm, a YOLOv5 model is combined with a new candidate frame optimization algorithm Soft-NMS (SoftNon-Maximum support) to improve the detection efficiency of the insulator overlapping targets, and therefore the accuracy of the model is improved.
(2) Designing a YOLOv5 insulator defect detection algorithm based on light weight;
the high-precision insulator defect detection algorithm based on YOLOv5 has higher accuracy, but has the problems of high model complexity and large parameter quantity and calculation amount. In order to meet the real-time performance of insulator defect detection, a YOLOv5 insulator defect detection algorithm based on light weight is proposed from the viewpoint of light weight. And the method utilizes the MobileNet v2 and the depth separable convolution to carry out light-weight processing on the YOLOv5, replaces a backbone network of the YOLOv5 network with the Bottleneck in the MobileNet v2, and replaces the common convolution of the feature fusion part with the depth separable convolution, thereby reducing the model parameter quantity and the calculation resource, reducing the model complexity and enabling the model complexity to meet the requirements of being deployed at a mobile terminal and embedded equipment.
Firstly, improving a YOLOv5s algorithm by using an attention mechanism module ECA-Net network, and optimizing a model structure; although the improved YOLOv5 algorithm has high precision, the model structure is complex and the calculated amount is large, so in order to meet the real-time property of insulator detection, the improved YOLOv5 algorithm meets the requirements of being deployed at a mobile terminal and embedded equipment, and then the improved YOLOv5 algorithm is subjected to light-weight treatment by using a mobilenet 2 model.
The operation of improving the YOLOv5s algorithm by using the attention mechanism module ECA-Net network comprises the steps of adding an attention mechanism, designing a multi-scale feature fusion structure and designing a candidate frame optimization algorithm.
The process of adding the attention mechanism specifically comprises the following steps:
the ECA-Net network is an improved channel attention model based on SE-Net (squeize-and-experientationNet), and one-dimensional convolution with a convolution kernel of k is used for replacing a full-connection layer to carry out channel weighting of k adjacent ranges, so that local cross and channel interaction are realized, the attention of the network to local characteristic information is enhanced, and the distinguishing capability of a background and a target is enhanced. The ECA-Net network architecture is shown in fig. 1.
The k value is calculated as shown in the following formula, wherein channels is the channel number of the input feature; every 5 channels in fig. 1 are output as one channel, and padding is performed by padding to keep the number of channels unchanged:
the operation of designing the multi-scale feature fusion structure is that after the model outputs three features through the backbone feature extraction layer, the three-scale feature images are used for detecting targets with different sizes; because the image has multi-scale targets with different characteristics, simple targets are distinguished by using shallow features, complex targets are distinguished by using deep features, and meanwhile, the multi-scale feature images are subjected to feature fusion, so that information loss among the scale features can be effectively compensated, and the detection capability of the model on the multi-scale targets is improved; comprising the following steps:
(1) Aiming at the characteristic of small insulator defect targets, the method combines a Bi-directional feature pyramid network Bi-FPN at a reinforced feature extraction layer part of YOLOv5, has the core ideas of efficient Bi-directional cross-scale connection and weighted feature map fusion, combines features of different scales and features of shallower layers under the same scale, and repeatedly stacks by taking a Bi-FPN as a circulating unit to obtain more high-layer feature fusion, so that the loss of targets in small feature maps and the loss of target position information in large feature maps are effectively compensated, and the detection capability of the small targets is improved. The Bi-FPN network structure is shown in FIG. 2.
(2) Designing a feature fusion layer structure based on Bi-FPN;
as shown in fig. 3, which shows a YOLOv5 model structure based on a Bi-FPN structure, a cyclic unit is added in the Bi-FPN structure, that is, the Bi-FPN structure includes two Bi-FPN sub-blocks, and the red dashed frame in fig. 3 is a sub-block thereof, which is beneficial to further enhancing feature fusion and compensating for information loss of different feature layers; meanwhile, a new residual error path is added in the characteristics of the middle layer size, so that information loss is prevented.
The candidate frame optimization algorithm specifically comprises the following operations:
in the aerial photographing process, the front insulator strings can partially shield the rear insulator strings due to the angle, and all insulator strings cannot be guaranteed to be single visible, so that the overlapped area is ignored in the detection process of the actual model, the detection efficiency of the model is reduced under certain conditions, and the candidate frame problem is mainly selected by the NMS algorithm of the model. The invention improves the NMS algorithm of YOLOv5, which reduces the number of candidate frames in the process of coincidence frame screening. Non-maximum suppression is performed using the positions and scores of the boxes.
In the original NMS algorithm, non-maximum suppression is achieved by using the location and score of the box. First, a frame having the largest score belonging to the same category in a certain region is selected, and non-maximum suppression is performed for each category by category cycling. Then, sorting the categories according to the scores from large to small, obtaining the frame with the largest score each time, and calculating the coincidence degree with all other prediction frames. If the overlap is too large, it will be eliminated. The process of eliminating the candidate frames only through the coincidence degree can lead to eliminating the overlapped targets. The overlap is high and if a normal NMS algorithm is used at this time, the insulator that is later overlapped will be removed due to the low score.
While Soft-NMS algorithm takes score and coincidence degree into consideration at the same time when non-maximum value inhibition is carried out, and takes the Gaussian index of IoU to multiply the original score through the form of weight, and then reorders; the calculation process is as follows: b= { b_1, …, b_n } N candidate boxes are input, the scores of the candidate boxes are s= { s_1, …, s_n }, N is a threshold value for NMS algorithm suppression, and the maximum probability M is selected; wherein IoU is the ratio of the intersection to the union of two regions;
the calculation flow chart of the Soft-NMS algorithm is shown in FIG. 4, and the calculation formula is shown as follows:
n represents the threshold value of the contact ratio of the rest target and the target with the largest probability, if the contact ratio is larger than the threshold value, the contact ratio of the rest target and the target is higher, and in the condition that the contact ratio information is the same, the candidate region with the largest probability is selected as the candidate region of the category, so that the other region is eliminated; if the overlap ratio is small, the new candidate region is different from the information contained in the region with the maximum probability, so that more different candidate regions are obtained.
The operation of performing light weight processing on the improved YOLOv5 algorithm by using the MobileNetv2 model specifically comprises the following steps:
(1) The design is based on a MobileNet v2 lightweight model:
MobileNet uses depth separable convolution instead of normal convolution, and proposes a lightweight network architecture design to compress and accelerate models, thereby meeting the demands of deployment on low-power mobile devices. The mobilenet v2 mainly provides two innovations based on the mobilenet v 1: linear bottleneck layers (linerbotlenecks) and inverted residual blocks (invertedresidual blocks).
The conventional residual block is shown in fig. 5, the number of channels of the input feature map is compressed by using a convolution kernel of 1×1, then feature information is extracted by using a convolution kernel of 3×3, finally the number of channels of the feature map is expanded back by using a convolution kernel of 1×1, the change of the number of channels in the whole process is that the channels are compressed first and then expanded, the activation functions of all convolution layers are ReLU, an output feature map is obtained, the output feature map and the input feature map are subjected to element-by-element addition operation, and the addition result is input to the next layer.
The inverted residual block is shown in fig. 6, the number of channels of the input feature map is expanded by using a convolution kernel with the size of 1×1, then information on the space dimension of the feature map is extracted by using a depth convolution with the size of 3×3, after the channel is activated by using a ReLU6 function, the number of channels of the feature map is finally compressed back by using a point convolution with the size of 1×1, the change of the number of channels in the whole process is that the channel is expanded first and then compressed, the feature map output by the layer is obtained by activating the function, the element-by-element addition operation is carried out on the output feature map and the input feature map, and the addition result is input into the next layer. The ReLU6 activation function differs from ReLU in that ReLU6 specifies a maximum output value of 6. The authors of MobileNetv2 have shown that when the number of channels in the feature map is small, much useful feature information is lost using the ReLU activation function, while more feature information can be retained using the linear activation function, and the last point convolution will lose some of the feature information while compressing the number of channels, and the number of channels in the feature map is small, thus using the linear activation function, known as the linear bottleneck layer.
The Bottleneck is a basic module for forming the MobileNet v2, the MobileNet v2 network structure is formed by stacking a series of Bottleneck, and the Bottleneck in the MobileNet v2 is used for replacing the backbone network of the YOLOv5 network, so that the complexity of a model is reduced, and the detection speed is improved. The Bottleneck is divided into two structures according to different stride, as shown in fig. 7, when the Bottleneck stride is 1, the input feature map firstly expands the channel number by convolution of 1×1, and the expansion coefficient is 6; the middle layer is a 3×3 deep convolution, the last layer is a 1×1 point convolution and linear activation function, the layer is the linear bottleneck layer, an output feature map is obtained, the output feature map and an input feature map are subjected to element-by-element addition operation, and the obtained feature map is input into the next layer; when the Bottleneck stride is 2, the middle layer is a depth convolution with the size of 3×3, and no jump connection (shortcut) and element-by-element addition operation exist in the structure, and the rest of the structure is the same as the Bottleneck module with the stride of 1; conv in FIG. 7 represents a normal convolution, DWConv represents a deep convolution, PWConv represents a point convolution, add represents an element-by-element addition operation, stride represents a Stride, and Linear represents a Linear activation function.
(2) The common convolution is replaced by the depth separable convolution in the YOLOv5 feature fusion part, the number of the depth separable convolution is less than that of the common convolution parameters, the same effect is achieved, and the operation cost is low; the depth separable convolution considers the region firstly and then considers the channel, so that the separation of the channel and the region is realized; the depth separable convolution comprises two processes, namely a channel-by-channel convolution (DepthwiseConvolution, DW) and a point-by-point convolution (PointConvolution, PW), wherein the channel-by-channel convolution is only convolved by one convolution kernel, and the number of channels of the feature map generated by the process is the same as the number of channels of the input; the size of a convolution kernel of point-by-point convolution is 1 multiplied by M, M is the number of channels of the previous layer, the convolution operation carries out weighted combination on the feature images of the previous step in the depth direction to generate new feature images, and the number of the output feature images is the same as that of the convolution kernels; the depth separable convolution process is shown in fig. 8.
The input of the network is D F ×D F X M, output is D F ×D F X N, convolution kernel size D k ×D k The standard convolution parameter is D k ×D k X M N, calculated as D F ×D F ×D k ×D k X M x N; applying input and output of the same size of the depth separable convolution, the obtained parameter quantity is D k ×D k ×M×D F ×D F +M×N×D F ×D F The method comprises the steps of carrying out a first treatment on the surface of the The convolution is represented as a depth separable convolution, reducing the computational effort by:
the embodiments of the present invention are disclosed as preferred embodiments, but not limited thereto, and those skilled in the art will readily appreciate from the foregoing description that various extensions and modifications can be made without departing from the spirit of the present invention.

Claims (6)

1. A lightweight YOLOv 5-based insulator defect detection method is characterized by comprising the following steps of: firstly, improving a YOLOv5s algorithm by using an attention mechanism module ECA-Net network, and optimizing a model structure; and then, the modified YOLOv5 algorithm is subjected to light weight treatment by using a MobileNetv2 model.
2. The lightweight YOLOv 5-based insulator defect detection method of claim 1, wherein: the operation of improving the YOLOv5s algorithm by using the attention mechanism module ECA-Net network comprises the steps of adding an attention mechanism, designing a multi-scale feature fusion structure and designing a candidate frame optimization algorithm.
3. The lightweight YOLOv 5-based insulator defect detection method of claim 2, wherein: the process of adding the attention mechanism specifically comprises the following steps:
the ECA-Net network is based on an SE-Net improved channel attention model, and one-dimensional convolution with a convolution kernel of k is used for replacing a full-connection layer to carry out channel weighting of k adjacent ranges, so that local intersection and channel interaction are realized, the attention of the network to local characteristic information is enhanced, and the distinguishing capability of a background and a target is enhanced; SE-Net refers to the Squeeze-and-specifiationNetworks, chinese meaning compression and excitation network;
the k-value is calculated as follows, where channels is the number of channels of the input feature:
4. the light-weight YOLOv 5-based insulator defect detection method according to claim 3, wherein the method comprises the steps of: the operation of designing the multi-scale feature fusion structure is that after the model outputs three features through the backbone feature extraction layer, the three-scale feature images are used for detecting targets with different sizes; distinguishing simple targets by shallow features, distinguishing complex targets by deep features, and simultaneously carrying out feature fusion on a multi-scale feature map, wherein the method comprises the following steps:
(1) Combining a Bi-directional feature pyramid network Bi-FPN at a reinforced feature extraction layer part of YOLOv5, and fusing bidirectional cross-scale connection and weighted feature graphs, fusing features of different scales, combining features of a shallower layer under the same scale, and repeatedly stacking by taking one Bi-FPN as a circulating unit to obtain more high-level feature fusion;
(2) Designing a feature fusion layer structure based on Bi-FPN;
a circulation unit is added in the Bi-FPN structure, namely, the circulation unit comprises two Bi-FPN sub-blocks, the sub-blocks are beneficial to further enhancing the fusion of the features and compensating the information loss of different feature layers; meanwhile, a new residual error path is added in the characteristics of the middle layer size, so that information loss is prevented.
5. The lightweight YOLOv 5-based insulator defect detection method of claim 4, wherein: the candidate frame optimization algorithm specifically comprises the following operations:
taking the obtained IoU to take a Gaussian index, multiplying the Gaussian index by an original score in a weight form, and then reordering; the calculation process is as follows: b= { b_1, …, b_n } N candidate boxes are input, the scores of the candidate boxes are s= { s_1, …, s_n }, N is a threshold value for NMS algorithm suppression, and the maximum probability M is selected; wherein IoU is the ratio of the intersection to the union of two regions;
the calculation formula of the Soft-NMS algorithm is shown as follows:
n represents the threshold value of the contact ratio of the rest target and the target with the largest probability, if the contact ratio is larger than the threshold value, the contact ratio of the rest target and the target is higher, and in the condition that the contact ratio information is the same, the candidate region with the largest probability is selected as the candidate region of the category, so that the other region is eliminated; if the overlap ratio is small, the new candidate region is different from the information contained in the region with the maximum probability, so that more different candidate regions are obtained.
6. The lightweight YOLOv 5-based insulator defect detection method of claim 5, wherein: the operation of performing light weight processing on the improved YOLOv5 algorithm by using the MobileNetv2 model specifically comprises the following steps:
(1) The design is based on a MobileNet v2 lightweight model:
the MobileNet v2 network structure is formed by stacking a series of Bottleneck, the backbone network of the YOLOv5 network is replaced by Bottleneck in MobileNet v2, the Bottleneck is divided into two structures according to different steps, when the Bottleneck step is 1, the input characteristic diagram firstly expands the channel number through convolution of 1X 1, and the expansion coefficient is 6; the middle layer is a 3 multiplied by 3 depth convolution, the last layer is a 1 multiplied by 1 point convolution and a linear activation function, an output characteristic diagram is obtained, element-by-element addition operation is carried out on the output characteristic diagram and an input characteristic diagram, and the obtained characteristic diagram is input into the next layer; when the Bottleneck stride is 2, the middle layer is a depth convolution with the size of 3×3, and no jump connection and element-by-element addition operation exist in the structure, and the rest of the structure is the same as the Bottleneck module with the stride of 1;
(2) Replacing the normal convolution with a depth separable convolution at the YOLOv5 feature fusion portion; the depth separable convolution comprises two processes, namely a channel-by-channel convolution and a point-by-point convolution, wherein the channel-by-channel convolution is only convolved by a convolution kernel, and the number of channels of a feature map generated by the process is the same as the number of channels of an input; the size of a convolution kernel of point-by-point convolution is 1 multiplied by M, M is the number of channels of the previous layer, the convolution operation carries out weighted combination on the feature images of the previous step in the depth direction to generate new feature images, and the number of the output feature images is the same as that of the convolution kernels;
the input of the network is D F ×D F X M, output is D F ×D F X N, convolution kernel size D k ×D k The standard convolution parameter is D k ×D k X M N, calculated as D F ×D F ×D k ×D k X M x N; applying input and output of the same size of the depth separable convolution, the obtained parameter quantity is D k ×D k ×M×D F ×D F +M×N×D F ×D F The method comprises the steps of carrying out a first treatment on the surface of the The convolution is represented as a depth separable convolution, reducing the computational effort by:
CN202310596152.1A 2023-05-24 2023-05-24 Lightweight-based YOLOv5 insulator defect detection method Pending CN117132531A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310596152.1A CN117132531A (en) 2023-05-24 2023-05-24 Lightweight-based YOLOv5 insulator defect detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310596152.1A CN117132531A (en) 2023-05-24 2023-05-24 Lightweight-based YOLOv5 insulator defect detection method

Publications (1)

Publication Number Publication Date
CN117132531A true CN117132531A (en) 2023-11-28

Family

ID=88861742

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310596152.1A Pending CN117132531A (en) 2023-05-24 2023-05-24 Lightweight-based YOLOv5 insulator defect detection method

Country Status (1)

Country Link
CN (1) CN117132531A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117468083A (en) * 2023-12-27 2024-01-30 浙江晶盛机电股份有限公司 Control method and device for seed crystal lowering process, crystal growth furnace system and computer equipment

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117468083A (en) * 2023-12-27 2024-01-30 浙江晶盛机电股份有限公司 Control method and device for seed crystal lowering process, crystal growth furnace system and computer equipment
CN117468083B (en) * 2023-12-27 2024-05-28 浙江晶盛机电股份有限公司 Control method and device for seed crystal lowering process, crystal growth furnace system and computer equipment

Similar Documents

Publication Publication Date Title
Zhang et al. Deep convolutional neural networks for forest fire detection
CN109241982B (en) Target detection method based on deep and shallow layer convolutional neural network
CN111428748B (en) HOG feature and SVM-based infrared image insulator identification detection method
CN111696075A (en) Intelligent fan blade defect detection method based on double-spectrum image
CN111368754B (en) Airport runway foreign matter detection method based on global context information
CN116310785B (en) Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN115205264A (en) High-resolution remote sensing ship detection method based on improved YOLOv4
Ni et al. An improved Faster R-CNN for defect recognition of key components of transmission line
CN117132531A (en) Lightweight-based YOLOv5 insulator defect detection method
CN112367400A (en) Intelligent inspection method and system for power internet of things with edge cloud coordination
CN114283137A (en) Photovoltaic module hot spot defect detection method based on multi-scale characteristic diagram inference network
CN115984543A (en) Target detection algorithm based on infrared and visible light images
CN109657682B (en) Electric energy representation number identification method based on deep neural network and multi-threshold soft segmentation
CN111767826A (en) Timing fixed-point scene abnormity detection method
Zu et al. Detection of common foreign objects on power grid lines based on Faster R-CNN algorithm and data augmentation method
Zhang et al. Defect detection of anti-vibration hammer based on improved faster R-CNN
CN116580324A (en) Yolov 5-based unmanned aerial vehicle ground target detection method
CN117351409A (en) Intelligent concrete dam face operation risk identification method
Wang et al. Forest fire detection method based on deep learning
Chen et al. Real-time detection of UAV detection image of power line insulator bursting based on YOLOV3
CN116721288A (en) Helmet detection method and system based on YOLOv5
CN111178275A (en) Fire detection method based on convolutional neural network
CN111783891B (en) Customized object detection method
Yan et al. YOLOv5-CSF: an improved deep convolutional neural network for flame detection
CN115100546A (en) Mobile-based small target defect identification method and system for power equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination