CN116612124B - Transmission line defect detection method based on double-branch serial mixed attention - Google Patents
Transmission line defect detection method based on double-branch serial mixed attention Download PDFInfo
- Publication number
- CN116612124B CN116612124B CN202310897591.6A CN202310897591A CN116612124B CN 116612124 B CN116612124 B CN 116612124B CN 202310897591 A CN202310897591 A CN 202310897591A CN 116612124 B CN116612124 B CN 116612124B
- Authority
- CN
- China
- Prior art keywords
- transmission line
- representing
- defect detection
- attention
- serial mixed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 167
- 230000007547 defect Effects 0.000 title claims abstract description 81
- 238000001514 detection method Methods 0.000 title claims abstract description 72
- 238000003062 neural network model Methods 0.000 claims abstract description 33
- 238000010586 diagram Methods 0.000 claims description 42
- 230000004927 fusion Effects 0.000 claims description 17
- 238000000034 method Methods 0.000 claims description 14
- 238000011176 pooling Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 238000012935 Averaging Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 239000010410 layer Substances 0.000 description 29
- 238000007689 inspection Methods 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 6
- 230000002776 aggregation Effects 0.000 description 5
- 238000004220 aggregation Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 235000019879 cocoa butter substitute Nutrition 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 239000012212 insulator Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000004568 cement Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 239000011229 interlayer Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a transmission line defect detection method based on double-branch serial mixed attention, which belongs to the technical field of transmission line defect detection, and comprises the steps of constructing a transmission line defect detection deep neural network model based on double-branch serial mixed attention, wherein the transmission line defect detection deep neural network model based on double-branch serial mixed attention comprises a double-branch serial mixed attention DBSA and a characteristic pyramid WCFPN; and performing defect detection on the power transmission line by using a power transmission line defect detection deep neural network model based on the double-branch serial mixed attention. The invention solves the problem that the existing transmission line defect detection method can not well overcome two difficulties of a small target and a complex background.
Description
Technical Field
The invention belongs to the technical field of defect detection of power transmission lines, and particularly relates to a power transmission line defect detection method based on double-branch serial mixed attention.
Background
Along with the continuous increase of the power demand of China, the total length of the power transmission line of China is also rapidly increased. As an important component of the power system, the condition of the power transmission line is related to the safety and stability of the power system, so that the method has great research significance for periodically checking the power transmission line and timely finding out the defects of the power transmission line to prevent large-area power failure.
In the defect detection of the power system, unmanned aerial vehicle inspection is widely used because of the convenient advantage, but at present, the picture acquired by the unmanned aerial vehicle still depends on manual naked eye inspection, and the mode is low in efficiency and easy to cause the problems of missing inspection and false inspection. In recent years, the rapid development of deep learning and computer vision makes algorithms such as target detection and semantic segmentation based on deep learning widely applied in the medical and traffic fields, and the application of the target detection algorithms to the defect detection of the power transmission line can make the inspection of the power grid more efficient and intelligent.
The target detection algorithm based on deep learning is mainly divided into two types, one is a regression-based single-stage algorithm, and the other is a two-stage algorithm based on a regional recommendation network (RPN). However, the algorithm has low detection speed and cannot meet the real-time requirement in power grid inspection. Therefore, the invention mainly researches a single-stage target detection algorithm, and applies the single-stage target detection algorithm to the field of power transmission lines to accurately and efficiently detect defects in the power transmission lines.
Compared with a general target detection scene, the defect detection of the power transmission line has the following difficulties: (1) The defect size is small, the definition of the aerial photo of the unmanned aerial vehicle is high, the resolution ratio is high, the sizes of foreign matters, insulators, pins and the like are small, the defect size is small, the whole picture occupies a small proportion, and some defects are difficult to distinguish by naked eyes. (2) The background is complex, the transmission line is in various places throughout the country, the background of the aerial inspection images is different, mountains, rivers, forests, snowlands and the like are included, and a large amount of irrelevant noise brings great interference to detection.
Therefore, the existing target detection algorithm can not well overcome two difficulties of a small target and a complex background, and has the problem of low scene detection precision in the aspects of transmission line foreign matter, insulator self-explosion, pin missing, cement pole breakage, damper breakage and the like.
Disclosure of Invention
Aiming at the defects in the prior art, the power transmission line defect detection method based on the double-branch serial mixed attention solves the problem that the existing power transmission line defect detection method cannot well overcome two difficulties of a small target and a complex background, so that the detection precision of defects in the complex background in a high-resolution inspection picture is improved.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the scheme provides a transmission line defect detection method based on double-branch serial mixed attention, which comprises the following steps:
s1, constructing a transmission line defect detection deep neural network model based on double-branch serial mixed attention, wherein the transmission line defect detection deep neural network model based on double-branch serial mixed attention comprises a double-branch serial mixed attention DBSA and a feature pyramid WCFPN;
s2, performing defect detection on the power transmission line by using a power transmission line defect detection deep neural network model based on double-branch serial mixed attention.
The beneficial effects of the invention are as follows: the invention provides a transmission line defect detection depth neural network model based on double-branch serial mixed attention, a DBSA attention mechanism and a WCFPN feature pyramid, and the DBSA serial double-branch mixed attention is designed, so that the model can replay more weights to defects, and the interference of irrelevant background information on detection is reduced. Meanwhile, by utilizing the characteristic pyramid WCFPN, the transmission line characteristics inside the pyramid are fully interacted by adopting cross-scale fusion and layer jump connection, the capability of detecting small targets by a model is improved, and the problem that the existing transmission line defect detection method cannot well overcome two difficulties of the small targets and complex backgrounds is solved.
Further, in the step S1, a deep neural network model for detecting defects of the transmission line based on the dual-branch serial mixed attention is constructed, which specifically includes:
embedding the dual-branch serial mixed attention DBSA into a Backbone network backbond of the YOLOv7, and constructing the dual-branch serial mixed attention-based power transmission line defect detection deep neural network model by using a feature pyramid WCFPN as a Neck structure Neck of the dual-branch serial mixed attention-based power transmission line defect detection deep neural network model.
The beneficial effects of the above-mentioned further scheme are: according to the invention, the dual-branch serial mixed attention DBSA is added into the backbone network, and the characteristic pyramid WCFPN is utilized to provide more effective information transmission from the early node to the later node, so that experiments are carried out on five data sets, and higher defect detection precision is obtained.
Still further, the step S2 includes the steps of:
s201, acquiring transmission line data and performing data enhancement processing on the transmission line data;
s202, inputting the transmission line data subjected to the enhancement processing into a Backbone network backhaul;
s203, extracting three power transmission line characteristic diagrams with different sizes by using a Backbone network backhaul and a dual-branch serial mixed attention DBSA;
s204, carrying out information exchange on three output wire feature diagrams with different sizes by utilizing cross-scale fusion and jump connection of a feature pyramid WCFPN;
and S205, detecting by using a detection Head of the transmission line defect detection depth neural network model based on the double-branch serial mixed attention according to the information exchange result to obtain the position and the type of the output line defect, and finishing the defect detection of the transmission line.
The beneficial effects of the above-mentioned further scheme are: the invention utilizes the dual-branch serial mixed attention DBSA to enhance the feature extraction capability of the model, ensures that the deep neural network model fully focuses on the defect information, suppresses irrelevant information such as background and the like, simultaneously ensures that the high-level semantic information and the bottom space information of the feature pyramid WCFPN are more fully interacted, and improves the capability of the model for detecting small targets, thereby better detecting the defects in the power transmission line.
Still further, the step S203 includes the steps of:
s2031, extracting input reinforced transmission line data through a convolution network to obtain three original feature graphs with different sizes, and respectively carrying out global average pooling compression processing on the feature graphs with different sizes along two branches of height and width to generate transmission line width featuresf 1 And transmission line height characteristicsf 2 ；
S2032, characteristic of width of transmission linef 1 And transmission line height characteristicsf 2 Averaging along the channel dimension to generate one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 ；
S2033, according to one-dimensional power transmissionLine width informationg 1 And one-dimensional power transmission line height informationg 2 Obtaining a one-dimensional power transmission line width attention factor and a one-dimensional height power transmission line height attention factor by utilizing one-dimensional convolution operation;
s2034, calculating power transmission line space attention of three feature graphs with different sizes according to the one-dimensional power transmission line width attention factor and the one-dimensional height power transmission line height attention factor;
s2035, carrying out recalibration calculation on the three original characteristic diagrams with different sizes by utilizing the power transmission line space attentiveness of the three characteristic diagrams with different sizes, and obtaining three power transmission line characteristic diagrams with different sizes extracted by the double-branch serial mixed attentiveness DBSA.
The beneficial effects of the above-mentioned further scheme are: the invention utilizes the dual-branch serial mixed attention DBSA to enhance the feature extraction capability of the model, so that the model fully focuses on the defect information and suppresses irrelevant information such as background.
Still further, the transmission line width featuref 1 And transmission line height characteristicsf 2 The expression of (2) is as follows:
wherein ,representing the width characteristics of the transmission line, < >>First step of representing characteristic diagram of transmission linejColumn (S)/(S)>First step of representing characteristic diagram of transmission linezMultiple channels (I)>Representing the first on the transmission line characteristic diagramiLine 1jCharacteristic value of column and z-th channel, respectively>First step of representing characteristic diagram of transmission lineiGo (go)/(go)>Representing the height characteristics of the transmission line, +.>、/> and />Respectively show the heightshWidth of the containerwChannel and method for manufacturing the samecSubscript of->、/> and />Respectively indicate->、/> and />Is a maximum value of (a).
The beneficial effects of the above-mentioned further scheme are: the invention compresses the characteristic information along the height and the width respectively, so that the model extracts the width characteristic and the height characteristic of the input information.
Still further, the one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 Is of (2)The expression is as follows:
wherein ,representing the characteristic diagram edge of the transmission linejWidth information after column compression, ++>Representing the characteristic diagram edge of the transmission lineiAnd (5) carrying out compressed height information.
The beneficial effects of the above-mentioned further scheme are: the invention compresses the width characteristic and the height characteristic along the channel dimension to generate the width information and the height information.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a structure of a dual-branch serial mixed attention DBSA in the present embodiment.
Fig. 3 is a structural diagram of the feature pyramid WCFPN in the present embodiment.
Fig. 4 is a schematic diagram of a cross-scale fusion and hierarchical aggregation structure L-ELAN of the feature pyramid WCFPN in this embodiment.
Fig. 5 is a schematic structural diagram of a model of a deep neural network model for detecting defects of a power transmission line based on dual-branch serial mixed attention in this embodiment.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
Examples
The invention utilizes the attention mechanism to enhance the feature extraction capability of the deep neural network model, so that the model fully focuses on defect information, suppresses irrelevant information such as background and the like, and simultaneously enables the high-level semantic information and the bottom space information of the feature pyramid to be more fully interacted, thereby improving the capability of the model to detect small targets, and further better detecting defects in the power transmission line. As shown in fig. 1, the invention provides a transmission line defect detection method based on double-branch serial mixed attention, which comprises the following steps:
s1, constructing a transmission line defect detection depth neural network model based on double-branch serial mixed attention, wherein the transmission line defect detection depth neural network model based on double-branch serial mixed attention comprises a double-branch serial mixed attention DBSA and a characteristic pyramid WCFPN, and the transmission line defect detection depth neural network model specifically comprises:
as shown in fig. 5, embedding the dual-branch serial mixed attention DBSA into a Backbone network backbond of YOLOv7, and using a feature pyramid WCFPN as a Neck structure neg of a transmission line defect detection depth neural network model based on the dual-branch serial mixed attention to complete construction of the transmission line defect detection depth neural network model based on the dual-branch serial mixed attention, wherein in fig. 5, CBS represents a convolutional layer operation, including Conv (convolution) +bn (batch normalization process) +sulu (activation function); DBSA means a dual-branch serial mixed attention DBSA proposed by the present invention; ELAN represents a plurality of convolution operations, consisting of a plurality of CBSs; L-ELAN represents a hierarchical aggregation structure proposed by the invention, and the specific structure is shown in figure 4; c represents that the feature graphs are spliced in the channel dimension; MP represents Maxpool (max pooling) +CBS; repconv represents a structural reparameterization convolution operation.
In this embodiment, the loss function of the transmission line defect detection deep neural network modelThe method comprises the following steps:
wherein ,、/> and />All represent loss weights, +.>Representing a loss of classification,/->Indicating loss of position->Representing a confidence loss, S representing the feature map size, B representing the number of bounding boxes predicted per grid, 1 representing an indication function, +.>Indicate->The->Predicted class probability of the individual bounding box, +.>Indicate->The->True class of the individual bounding box,/>Indicate->The->Whether or not the individual bounding box contains an object +.>Log function with a representation base of 2, +.>Weight representing position loss, +.>Indicate->The->True position information of the individual bounding boxes, +.>Representing the predicted location information of the object,nrepresentation->、y、w、hOne of the four is->Representing the height coordinates of the prediction box,yrepresenting the width coordinates of the prediction frame,wthe width size of the prediction frame is represented,hrepresenting the height size of the prediction box, +.>Weights representing confidence loss, +.>Indicate->The->Whether or not the bounding box contains no objects.
S2, performing defect detection on the power transmission line by using a power transmission line defect detection deep neural network model based on double-branch serial mixed attention, wherein the implementation method is as follows:
s201, acquiring transmission line data and performing data enhancement processing on the transmission line data;
s202, inputting the transmission line data subjected to the enhancement processing into a Backbone network backhaul;
s203, extracting three power transmission line characteristic diagrams with different sizes by using a Backbone network backhaul and a dual-branch serial mixed attention DBSA, wherein the implementation method is as follows:
s2031, extracting input reinforced transmission line data through a convolution network to obtain three original feature graphs with different sizes, and respectively carrying out global average pooling compression processing on the feature graphs with different sizes along two branches of height and width to generate transmission line width featuresf 1 And transmission line height characteristicsf 2 ；
S2032, characteristic of width of transmission linef 1 And transmission line height characteristicsf 2 Averaging along the channel dimension to generate one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 ；
S2033, according to the one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 Obtaining a one-dimensional power transmission line width attention factor and a one-dimensional height power transmission line height attention factor by utilizing one-dimensional convolution operation;
s2034, calculating power transmission line space attention of three feature graphs with different sizes according to the one-dimensional power transmission line width attention factor and the one-dimensional height power transmission line height attention factor;
s2035, carrying out recalibration calculation on the three original characteristic diagrams with different sizes by utilizing the power transmission line space attentiveness of the three characteristic diagrams with different sizes, and obtaining three power transmission line characteristic diagrams with different sizes extracted by the double-branch serial mixed attentiveness DBSA.
In this embodiment, the dual-branch serial mixed attention DBSA allows the model to place more weight on the defect of the input picture rather than the background, thereby suppressing the interference of irrelevant backgrounds and overcoming the difficulty of complex power transmission line backgrounds.
S204, carrying out information exchange on three output wire feature diagrams with different sizes by utilizing cross-scale fusion and jump connection of a feature pyramid WCFPN;
in the embodiment, the full fusion of the high-level semantic information and the bottom space information enables the model to better recognize the association between the whole and the part, so that the position of the small target defect is accurately positioned, and the difficulty of detecting the small target in the defect detection of the power transmission line is further overcome.
And S205, detecting by using a detection Head of the transmission line defect detection depth neural network model based on the double-branch serial mixed attention according to the information exchange result to obtain the position and the type of the output line defect, and finishing the defect detection of the transmission line.
In the embodiment, the invention designs a transmission line defect detection deep neural network model based on the dual-branch serial mixed attention, designs the dual-branch serial mixed attention DBSA and the characteristic pyramid WCFPN, and constructs a model more suitable for transmission line defect detection. The method comprises the steps of detecting defects of a power transmission line through a constructed deep neural network model, entering a main network of the model after data enhancement, extracting three power transmission line feature diagrams with different sizes through a series of rolling and pooling operations such as double-branch serial mixed attention, and the like, exchanging information of potential semantics of different spatial scales and different layers through cross-scale fusion and layer jump connection of a feature pyramid WCFPN, entering a detection head for detection, and outputting positions and types of the defects.
In this embodiment, the dual-branch serial mixed attention structure proposed by the present invention is shown in fig. 2, GAP represents global Average pooling, channel Average represents averaging along the Channel direction, conv1d represents one-dimensional convolution,a feature map representing the input is presented,Hthe height of the input is indicated and,Wthe width of the input is indicated and,Crepresenting the number of channels entered>A feature map obtained by DBSA attention correction calculation is shown. Input->The height attention factor and the width attention factor of the characteristic diagram of the power transmission line are respectively extracted through double branch paths, the two are multiplied to obtain spatial attention, and the characteristic diagram obtained through spatial attention calculation is ∈ ->And carrying out global average pooling and one-dimensional convolution to obtain the channel attention. Finally, the feature map obtained by the calculation of the spatial attention +.>Then the output +.>, wherein ,Rrepresenting the dimension of the input.
Space characteristic diagram of power transmission lineThe spatial distribution information of the characteristics of the transmission line is included, the extracted spatial information can be more accurately positioned to the position of the defect, and the information is inputGlobal average pooling compression is performed along the two branches of height and width respectively to generate +.>Width characteristics of-> and />Height characteristics of->。
wherein ,representing the width characteristics of the transmission line, < >>First step of representing characteristic diagram of transmission linejColumn (S)/(S)>First step of representing characteristic diagram of transmission linezMultiple channels (I)>Representing the first on the transmission line characteristic diagramiLine 1jCharacteristic value of column and z-th channel, respectively>First step of representing characteristic diagram of transmission lineiGo (go)/(go)>Representing the height characteristics of the transmission line, +.>、/> and />Respectively show the heightshWidth of the containerwChannel and method for manufacturing the samecSubscript of->、/> and />Respectively indicate->、/> and />Is a maximum value of (a).
Width characteristicsAnd height characteristics->Averaging along the channel dimension respectively to generate and />One-dimensional width information of (a)g 1 And one-dimensional height informationg 2 Then one-dimensional convolution operation is used to obtain one-dimensional width attention factor and one-dimensional height attention factor, which are multiplied to obtain +.>Is a spatial concentration of (c).
wherein ,representing the characteristic diagram edge of the transmission linejWidth information after column compression, ++>Representing the characteristic diagram edge of the transmission lineiAnd (5) carrying out compressed height information.
Feature map with improved spatial attentionThere is->The number of channels is different, but the importance degree of each channel is different, and different weights are distributed to each channel through the attention of the channel, so that the model can better identify the content of the input characteristic, and further better identify the type of the defect in the inspection image. First of all to input->Global averaging pooling is performed to pool +.>Adding and averaging the characteristic information to obtain the corresponding characteristic of each channel>And then, a one-dimensional convolution kernel is adopted to replace a full-connection layer of a compression and excitation network SENET to carry out cross-channel interaction, so that more accurate channel attention is obtained while the complexity of a model is remarkably reduced:
wherein ,represent the firstzTransmission line characteristic information on individual channels, +.>Representing the height of the input feature +.>Representing the width of the input feature +.>Channel value representing input feature, +.>Representing the characteristic value, < > at a certain coordinate on the characteristic map>Represent the firstiGo (go)/(go)>Represent the firstjColumn (S)/(S)>Represent the firstzAnd a plurality of channels.
In this embodiment, the feature pyramid structure WCFPN provided by the present invention is shown in fig. 3 below, fig. 3 is input to a backbone network, C3, C4, and C5 represent three feature graphs with different sizes, M, N, L, P represent different pyramid feature layers, four feature layers are all M, N, L, P in the pyramid (feature layer M includes three feature layers of M3, M4, and M5, feature layer N includes three feature layers of N3, N4, and N5, feature layer L includes three feature layers of L3, L4, and L5, feature layer P includes three feature layers of P3, P4, and P5), and features of P3, P4, and P5 extracted by the backbone network are generated through trans-scale feature fusion and skip connection in the pyramid.
In this embodiment, the cross-scale feature fusion of the feature pyramid WCFPN strengthens the connection between features in the pyramid, and specific fusion details are described by taking the N4 node as an example. As shown in fig. 4, 1X1 in fig. 4 represents a convolution operation with a convolution kernel size of 1X 1; 3X3 represents a convolution operation with a convolution kernel size of 3X 3. The feature sources of the current layer N4 fusion are three feature graphs with different sizes of the previous layer M3, the previous layer M4 and the current layer N5, and in the fusion process, the upsampling and downsampling operations are respectively carried out on N5 and M3 to obtain the feature graphs with the same size as M4And splicing the three feature maps into C in the channel dimension, and then completing fusion through the hierarchical aggregation structure L-ELAN. The hierarchical aggregation structure L-ELAN of the invention enables the deep network to learn and converge more effectively by controlling the gradient path, the spliced characteristic diagrams are firstly grouped in the channel dimension, and the lower branch passes throughAnd continuously extracting the features by the convolution kernel, and finally splicing four feature graphs with the same size in the channel dimension to generate a next layer of features N4. The cross-scale fusion of the feature pyramid WCFPN connects the hierarchical features of the previous layer and the current layer, so that the internal connection between the features is more focused, and the hierarchical aggregation structure L-ELAN enables the high-level semantic features and the low-level spatial features to be fully interacted. In order to enable information interaction to occur between features of different layers of the pyramid, the feature gold word WCFPN also introduces layer jump connection, as shown in fig. 3, on the dimension of the same feature map with the same resolution, the feature information of the first layer M participates in the feature fusion of the third layer L, the feature information N of the second layer participates in the feature fusion of the fourth layer P, and the layer jump connection enables the feature fusion operation to consider interlayer characteristics on the same resolutionThe method provides more effective information transmission from the early node to the later node, so that the feature fusion obtains deeper generalized information.
Claims (4)
1. The transmission line defect detection method based on the double-branch serial mixed attention is characterized by comprising the following steps of:
s1, constructing a transmission line defect detection deep neural network model based on double-branch serial mixed attention, wherein the transmission line defect detection deep neural network model based on double-branch serial mixed attention comprises a double-branch serial mixed attention DBSA and a feature pyramid WCFPN;
in the step S1, a transmission line defect detection deep neural network model based on double-branch serial mixed attention is constructed, and the transmission line defect detection deep neural network model specifically comprises the following components:
embedding the dual-branch serial mixed attention DBSA into a Backbone network backbond of the YOLOv7, and using a characteristic pyramid WCFPN as a Neck structure Neck of the transmission line defect detection depth neural network model based on the dual-branch serial mixed attention to finish the construction of the transmission line defect detection depth neural network model based on the dual-branch serial mixed attention;
loss function of transmission line defect detection deep neural network modelThe method comprises the following steps:
wherein ,、/> and />All represent loss weights, +.>Representing a loss of classification,/->Indicating loss of position->Representing a confidence loss, S representing the feature map size, B representing the number of bounding boxes predicted per grid, 1 representing an indication function, +.>Represent the firstThe->Predicted class probability of the individual bounding box, +.>Indicate->The->True class of the individual bounding box,/>Indicate->The->Whether or not the individual bounding box contains an object +.>The log function with a base of 2 is indicated,weight representing position loss, +.>Indicate->The->True position information of the individual bounding boxes, +.>Representing the predicted location information of the object,nrepresentation->、y、w、hOne of the four is->Representing the height coordinates of the prediction box,yrepresenting the width coordinates of the prediction frame,wthe width size of the prediction frame is represented,hrepresenting the height size of the prediction box, +.>Weights representing confidence loss, +.>Indicate->The->Whether the bounding box does not contain an object;
s2, performing defect detection on the power transmission line by using a power transmission line defect detection deep neural network model based on double-branch serial mixed attention;
the step S2 includes the steps of:
s201, acquiring transmission line data and performing data enhancement processing on the transmission line data;
s202, inputting the transmission line data subjected to the enhancement processing into a Backbone network backhaul;
s203, extracting three power transmission line characteristic diagrams with different sizes by using a Backbone network backhaul and a dual-branch serial mixed attention DBSA;
s204, carrying out information exchange on three output wire feature diagrams with different sizes by utilizing cross-scale fusion and jump connection of a feature pyramid WCFPN;
and S205, detecting by using a detection Head of the transmission line defect detection depth neural network model based on the double-branch serial mixed attention according to the information exchange result to obtain the position and the type of the output line defect, and finishing the defect detection of the transmission line.
2. The method for detecting defects of a transmission line based on dual-branch serial mixed attention according to claim 1, wherein said step S203 comprises the steps of:
s2031, extracting three original features with different sizes from input transmission line data subjected to enhancement processing through a convolution networkThe map, the characteristic map of different sizes is processed by global average pooling compression along the two branches of height and width, and the transmission line width characteristic is generatedf 1 And transmission line height characteristicsf 2 ；
S2032, characteristic of width of transmission linef 1 And transmission line height characteristicsf 2 Averaging along the channel dimension to generate one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 ；
S2033, according to the one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 Obtaining a one-dimensional power transmission line width attention factor and a one-dimensional height power transmission line height attention factor by utilizing one-dimensional convolution operation;
s2034, calculating power transmission line space attention of three feature graphs with different sizes according to the one-dimensional power transmission line width attention factor and the one-dimensional height power transmission line height attention factor;
s2035, carrying out recalibration calculation on the three original characteristic diagrams with different sizes by utilizing the power transmission line space attentiveness of the three characteristic diagrams with different sizes, and obtaining three power transmission line characteristic diagrams with different sizes extracted by the double-branch serial mixed attentiveness DBSA.
3. The method for detecting defects of a power transmission line based on double-branch serial mixed attention according to claim 2, wherein the width characteristic of the power transmission line is as followsf 1 And transmission line height characteristicsf 2 The expression of (2) is as follows:
wherein ,representing the width characteristics of the transmission line, < >>First step of representing characteristic diagram of transmission linejColumn (S)/(S)>First step of representing characteristic diagram of transmission linezMultiple channels (I)>Representing the first on the transmission line characteristic diagramiLine 1jCharacteristic value of column and z-th channel, respectively>First step of representing characteristic diagram of transmission lineiGo (go)/(go)>Representing the height characteristics of the transmission line, +.>、/> and />Respectively show the heightshWidth of the containerwChannel and method for manufacturing the samecSubscript of->、/> and />Respectively indicate->、/> and />Is a maximum value of (a).
4. The transmission line defect detection method based on dual-branch serial mixed attention according to claim 3, wherein the one-dimensional transmission line width informationg 1 And one-dimensional power transmission line height informationg 2 The expression of (2) is as follows:
wherein ,representing the characteristic diagram edge of the transmission linejWidth information after column compression, ++>Representing the characteristic diagram edge of the transmission lineiAnd (5) carrying out compressed height information. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897591.6A CN116612124B (en) | 2023-07-21 | 2023-07-21 | Transmission line defect detection method based on double-branch serial mixed attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310897591.6A CN116612124B (en) | 2023-07-21 | 2023-07-21 | Transmission line defect detection method based on double-branch serial mixed attention |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116612124A CN116612124A (en) | 2023-08-18 |
| CN116612124B true CN116612124B (en) | 2023-10-20 |
Family
ID=87682289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310897591.6A Active CN116612124B (en) | 2023-07-21 | 2023-07-21 | Transmission line defect detection method based on double-branch serial mixed attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116612124B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663346A (en) * | 2022-01-30 | 2022-06-24 | 河北工业大学 | Strip steel surface defect detection method based on improved YOLOv5 network |
| CN115187583A (en) * | 2022-08-19 | 2022-10-14 | 南京信息工程大学 | Lightweight road defect detection method based on improved YOLOv5 |
| CN115239710A (en) * | 2022-09-21 | 2022-10-25 | 南京信息工程大学 | Insulator defect detection method based on attention feedback and double-space pyramid |
| WO2022227913A1 (en) * | 2021-04-25 | 2022-11-03 | 浙江师范大学 | Double-feature fusion semantic segmentation system and method based on internet of things perception |
| CN115600666A (en) * | 2022-09-15 | 2023-01-13 | 云南电网有限责任公司电力科学研究院（Cn） | Self-learning method and device for power transmission and distribution line defect detection model |
| CN115731164A (en) * | 2022-09-14 | 2023-03-03 | 常州大学 | Insulator defect detection method based on improved YOLOv7 |
| CN116071294A (en) * | 2022-11-08 | 2023-05-05 | 中国计量大学 | Optical fiber surface defect detection method and device |
| CN116309270A (en) * | 2022-12-06 | 2023-06-23 | 西安交通大学 | Binocular image-based transmission line typical defect identification method |
| CN116343064A (en) * | 2022-09-06 | 2023-06-27 | 国网浙江省电力有限公司湖州供电公司 | Lightweight improved insulator defect automatic detection method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4184387B1 (en) * | 2021-11-19 | 2024-04-17 | Tata Consultancy Services Limited | Method and system for personalized substitute product recommendation |
-
2023
- 2023-07-21 CN CN202310897591.6A patent/CN116612124B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022227913A1 (en) * | 2021-04-25 | 2022-11-03 | 浙江师范大学 | Double-feature fusion semantic segmentation system and method based on internet of things perception |
| CN114663346A (en) * | 2022-01-30 | 2022-06-24 | 河北工业大学 | Strip steel surface defect detection method based on improved YOLOv5 network |
| CN115187583A (en) * | 2022-08-19 | 2022-10-14 | 南京信息工程大学 | Lightweight road defect detection method based on improved YOLOv5 |
| CN116343064A (en) * | 2022-09-06 | 2023-06-27 | 国网浙江省电力有限公司湖州供电公司 | Lightweight improved insulator defect automatic detection method |
| CN115731164A (en) * | 2022-09-14 | 2023-03-03 | 常州大学 | Insulator defect detection method based on improved YOLOv7 |
| CN115600666A (en) * | 2022-09-15 | 2023-01-13 | 云南电网有限责任公司电力科学研究院（Cn） | Self-learning method and device for power transmission and distribution line defect detection model |
| CN115239710A (en) * | 2022-09-21 | 2022-10-25 | 南京信息工程大学 | Insulator defect detection method based on attention feedback and double-space pyramid |
| CN116071294A (en) * | 2022-11-08 | 2023-05-05 | 中国计量大学 | Optical fiber surface defect detection method and device |
| CN116309270A (en) * | 2022-12-06 | 2023-06-23 | 西安交通大学 | Binocular image-based transmission line typical defect identification method |
Non-Patent Citations (6)
| Title |
|---|
| Chien-Yao Wang等.Yolov7: Trainable bag-of-freebies sets new state-of-art for real-time object detectors.《arXiv:2207.02696》.2022,1-15. * |
| DAMO-YOLO : A Report on Real-Time Object Detection Design;Xianzhe Xu;《arXiv:2211.15444v4》;1-10 * |
| EfficientDet: Scalable and Efficient Object Detection;Mingxing Tan等;《CVPR 2020》;10781-10790, 图3 * |
| 许德刚等.改进YOLOv6 的遥感图像目标检测算法.《计算机工程与应用》.2023,1-12. * |
| 轻量化高精度双通道注意力机制模块;陈晓雷等;《计算机科学与探索》;第17卷(第04期);857-867 * |
| 逯长虹等.基于改进Yolov5算法的农村公路路面裂缝检测研究.《中原工学院学报》.2023,第34卷(第03期),53-61. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116612124A (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Surface defect detection methods for industrial products: A review | |
| Saberironaghi et al. | Defect detection methods for industrial products using deep learning techniques: A review | |
| Zheng et al. | Insulator-defect detection algorithm based on improved YOLOv7 | |
| Luo et al. | Deep learning-based building extraction from remote sensing images: A comprehensive review | |
| Qiu et al. | Detection of transmission line insulator defects based on an improved lightweight YOLOv4 model | |
| Cao et al. | Front vehicle detection algorithm for smart car based on improved SSD model | |
| Qiu et al. | IDOD-YOLOV7: Image-dehazing YOLOV7 for object detection in low-light foggy traffic environments | |
| Song et al. | Tsr-yolo: A chinese traffic sign recognition algorithm for intelligent vehicles in complex scenes | |
| CN114519819B (en) | Remote sensing image target detection method based on global context awareness | |
| Fang et al. | Improved mask R-CNN multi-target detection and segmentation for autonomous driving in complex scenes | |
| Zhou et al. | Object detection in autonomous driving scenarios based on an improved faster-RCNN | |
| CN116168246A (en) | Method, device, equipment and medium for identifying waste slag field for railway engineering | |
| CN114332473A (en) | Object detection method, object detection device, computer equipment, storage medium and program product | |
| Wu et al. | YOLO-LWNet: A lightweight road damage object detection network for mobile terminal devices | |
| Jiang et al. | Tabcellnet: Deep learning-based tabular cell structure detection | |
| Li et al. | YOLO-RFF: An industrial defect detection method based on expanded field of feeling and feature fusion | |
| Zhang et al. | A lightweight vehicle-pedestrian detection algorithm based on attention mechanism in traffic scenarios | |
| Siriborvornratanakul | Downstream Semantic Segmentation Model for Low‐Level Surface Crack Detection | |
| Zhao et al. | Exploration of Vehicle Target Detection Method Based on Lightweight YOLOv5 Fusion Background Modeling | |
| Zhang et al. | A fitting recognition approach combining depth-attention YOLOv5 and prior synthetic dataset | |
| Jia et al. | Multi‐stream densely connected network for semantic segmentation | |
| Qiu et al. | Techniques for the automatic detection and hiding of sensitive targets in emergency mapping based on remote sensing data | |
| Yang et al. | An Improved Mask R-CNN Micro-Crack Detection Model for the Surface of Metal Structural Parts | |
| Jiang et al. | Rdd-yolov5: road defect detection algorithm with self-attention based on unmanned aerial vehicle inspection | |
| CN117975218A (en) | Small target detection method based on mixed attention and feature centralized multi-scale fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |