CN116523858A - Attention mechanism-based oil leakage detection method for power equipment and storage medium - Google Patents
Attention mechanism-based oil leakage detection method for power equipment and storage medium Download PDFInfo
- Publication number
- CN116523858A CN116523858A CN202310438083.1A CN202310438083A CN116523858A CN 116523858 A CN116523858 A CN 116523858A CN 202310438083 A CN202310438083 A CN 202310438083A CN 116523858 A CN116523858 A CN 116523858A
- Authority
- CN
- China
- Prior art keywords
- network
- feature
- attention
- input
- oil leakage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 230000007246 mechanism Effects 0.000 title claims abstract description 19
- 238000003860 storage Methods 0.000 title claims abstract description 8
- 238000011176 pooling Methods 0.000 claims abstract description 87
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 8
- 238000000605 extraction Methods 0.000 claims description 36
- 238000000034 method Methods 0.000 claims description 24
- 238000010586 diagram Methods 0.000 claims description 17
- 238000012360 testing method Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 238000013434 data augmentation Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 239000003921 oil Substances 0.000 claims 16
- 239000010773 plant oil Substances 0.000 claims 3
- 230000004927 fusion Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 101100441251 Arabidopsis thaliana CSP2 gene Proteins 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an attention mechanism-based oil leakage detection method for electric equipment and a storage medium, wherein an image of the electric equipment to be detected is acquired and is input into an oil leakage detection model, the model comprises a back bone network, a Neck network and a Head network which are adjacent in sequence, an attention pooling capture module is arranged in a jumper layer of the Neck network, and a hierarchical channel attention module is added in the back bone network and at the tail end of the Neck network; extracting features of images of the power equipment to be detected through a backhaul network to obtain feature images with different depths; inputting the feature images with different depths into a Neck network through an attention pooling capture module to obtain feature images with different sizes; the feature images with different sizes pass through a hierarchical channel attention module arranged at the tail end of a Neck network to obtain different target information; different target information is input to a Head network, and an oil leakage detection result of the power equipment is obtained.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an oil leakage detection method and a storage medium for power equipment based on an attention mechanism.
Background
Electric power is a very important part in various industrial production, social and economic development, national infrastructure and other aspects. Oil leakage detection is an important part of maintenance of power equipment, and damage to the power equipment can be effectively prevented by timely, accurately and comprehensively finding out oil leakage conditions in the power equipment, so that related hazards are effectively reduced.
The original oil leakage condition of the power equipment generally adopts a manual inspection mode, which is time-consuming and labor-consuming, and faces dangerous, complex and changeable field environments. Along with development of science and technology, the manual inspection mode is gradually converted into an unmanned aerial vehicle image collection mode, and then a professional electric power maintenance engineer judges an oil leakage area according to working experience.
The image recognition technology is applied to the direction of oil leakage detection of the power equipment, and automatically detects whether the power equipment leaks oil or not, so that the detection of the power equipment leaks oil gradually becomes one of the main flow directions in the direction, but the problems of complex background, small target, darker oil stain, difficulty in detection, and the like exist in the oil leakage detection of the power equipment, and the related image recognition method is easy to cause the problems of low efficiency and misrecognition. For example, in the related technology, a method and a system for identifying oil leakage of a transformer are provided in the patent application document with the application number of CN114757938A, a self-attention mechanism is introduced to promote global feature fusion, semantic association among pixel features is enhanced, loss of detail information is reduced, and classification accuracy is improved; however, the scheme only enhances the characteristics on the channel when the attention operation is carried out, but ignores the spatial characteristics, and does not consider the influence of objects with different dimensions when the attention operation is carried out on the channel.
Disclosure of Invention
The invention aims to solve the technical problem of obtaining better oil leakage detection effect.
The invention solves the technical problems by the following technical means:
in one aspect, an attention mechanism-based oil leakage detection method for an electric device is provided, and the method comprises the following steps:
acquiring an image of power equipment to be detected, and taking the image of the power equipment to be detected as input of an oil leakage detection model, wherein the oil leakage detection model adopts a YOLOv5 neural network, the YOLOv5 neural network comprises a back bone network, a Neck network and a Head network which are adjacent in sequence, an attention pooling capture module is arranged in a jumper layer of the Neck network, and a hierarchical channel attention module is added in the back bone network and at the tail end of the Neck network;
extracting features of the images of the power equipment to be detected through the backhaul network, and sequentially obtaining feature images with different depths;
inputting the feature images with different depths into the Neck network through the attention pooling capturing module to obtain feature images with different sizes preliminarily output by the Neck network;
the feature images with different sizes pass through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information;
and inputting different target information into the Head network to build a feature pyramid, so as to obtain an oil leakage detection result of the power equipment.
Further, the backhaul network comprises a first feature extraction branch network, a second feature extraction branch network and a third feature extraction branch network which are sequentially connected, wherein the attention module is arranged in the third feature extraction branch network;
the image of the power equipment to be tested is used as the input of the first characteristic extraction branch network, and the first characteristic extraction branch network outputs a characteristic diagram phi 1 (x) The second feature extraction branch network outputs a feature map phi 2 (x) The third feature extraction branch network outputs a feature map phi 3 (x)。
Further, the feature maps of different depths are phi respectively 1 (x)、φ 2 (x) And phi 3 (x) A plurality of attention pooling capturing modules are arranged in a jumper layer of the negk network, the feature images with different depths are input to the negk network through the attention pooling capturing modules, and feature images with different sizes preliminarily output by the negk network are obtained, and the method comprises the following steps:
the characteristic diagram phi 1 (x) And phi 3 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 4 (x)；
The characteristic diagram phi 3 (x) And phi 4 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 5 (x)；
The characteristic diagram phi 2 (x) And phi 5 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 6 (x)。
Further, do notThe characteristic diagrams with the same size are phi respectively 4 (x)、φ 5 (x) And phi 6 (x) And the feature images with different sizes pass through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information:
O 4 (x)＝GCA(Conv(φ 4 (x)))
O 5 (x)＝GCA(Conv(φ 5 (x)))
O 6 (x)＝GCA(Conv(φ 6 (x)))
wherein O is 4 (x)、O 5 (x) And O 6 (x) GCA () represents the convolution of the attention module on the channel and Conv () represents the convolution operation, respectively, for different target information.
Further, the hierarchical channel attention module includes convolution layers conv_1, conv_2, conv_3, average pooling layers avgpool_1, avgpool_2, avgpool_3, maximum pooling layers maxpool_1, maxpool_2, maxpool_3, and multi-layer perceptrons mlp_1, mlp_2, mlp_3, mlp_4;
the input features are respectively processed through the convolution layers Conv_1, conv_2 and Conv_3 to obtain the graded feature output C on the corresponding channel 1(x) 、C 3(x) 、C 5(x) Feature C 1(x) Characteristic C as input to average pooling layer Avgpool_1 and maximum pooling layer Maxpool_1, respectively 3(x) Characteristic C as input to average pooling layer Avgpool_2 and maximum pooling layer Maxpool_2, respectively 5(x) As inputs to the average pooling layer avgpool_3 and the maximum pooling layer maxpool_3, respectively;
the output of the average pooling layer Avgpool_1 and the output of the maximum pooling layer Maxpool_1 are input to the multi-layer perceptron MLP_1 after splicing operation, the output of the average pooling layer Avgpool_2 and the output of the maximum pooling layer Maxpool_2 are input to the multi-layer perceptron MLP_2 after splicing operation, the output of the average pooling layer Avgpool_3 and the output of the maximum pooling layer Maxpool_3 are input to the multi-layer perceptron MLP_3 after splicing operation, and the output of the multi-layer perceptrons MLP_1, MLP_2 and MLP_3 are input to the multi-layer perceptron MLP_4 after splicing operation, so that a hierarchical channel characteristic sequence is obtained;
and the result of multiplying the hierarchical channel feature sequence and the input feature is spliced and reconstructed with the input feature to form the shape of the input feature.
Further, the attention pooling capturing module comprises three first convolution layers, three multi-head attention networks and three second convolution layers, any one of the feature maps with different sizes is respectively input into each first convolution layer, the output of each first convolution layer is connected with one multi-head attention network through a Reshape operation, and the output of the multi-head attention network is connected with one second convolution layer through the Reshape operation;
and the outputs of the three second convolution layers are connected through splicing operation and then output to a third convolution layer.
Further, the step of the attention pooling capture module processing the feature maps of different sizes includes:
outputting feature graphs of different channels to an input feature graph through different first convolution layers;
the feature images of different channels are flattened to two-dimensional feature images and then pass through a multi-head attention network corresponding to each channel to obtain feature sequences corresponding to different channels;
after the feature sequences corresponding to different channels are transformed to the sizes of the feature graphs corresponding to the channels, the feature sequences are spliced and output after passing through different second convolution layers respectively.
Further, before the acquiring the image of the electrical equipment to be tested and taking the image of the electrical equipment to be tested as the input of the oil leakage detection model, the method further includes:
collecting an oil leakage data set of the power equipment;
carrying out oil stain and oil stain range marking on sample data in the data set by using Labelme, and dividing the data set into a test set, a training set and a verification data set;
and training, testing and verifying the oil leakage detection model by using the training set, the testing set and the verification data set respectively to obtain the trained oil leakage detection model.
Further, after the collecting the electrical equipment oil leakage data set, the method further comprises:
and screening, size cutting and data augmentation are carried out on the sample data in the data set, so that a preprocessed data set is obtained.
In a second aspect, the invention also proposes a computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements a method as described above.
The invention has the advantages that:
(1) In consideration of the problems that the background is complex, the oil leakage area is small, the oil leakage detection system is difficult to achieve the balance of precision and speed and the like in the maintenance and detection of the power equipment, the attention pooling capture module is arranged in the jumper layer of the Neck network by improving the YOLOv5 network structure, the fusion capacity of the network between deep and shallow characteristics is improved, the remote characteristic extraction capacity of the network is improved, the classifying channel attention module is added in the backstone network and at the tail end of the Neck network, and the characteristic extraction and expression capacity of the network on a target object is improved; compared with the traditional YOLOv5 network structure, the feature on the channel is enhanced through the hierarchical channel attention module, and the feature characterization capability is enhanced in the space dimension through the pooling space attention module, so that the feature extraction and feature fusion capability of the network can be improved while the original algorithm low parameter quantity and low calculation amount are maintained, the feature expression capability is improved better, and the accuracy of the oil leakage detection of the system can be improved.
(2) And a transducer method is applied to the attention pooling capturing module, and a feature pyramid module consisting of Tatransducers connected with different channels is constructed in parallel, so that the characterization capability of shallow features is improved, and the feature fusion effect between deep features and shallow features is enhanced.
(3) The hierarchical channel attention module adopts a structure combining coarse and fine, and analyzes feature layers with different depths layer by layer, and when channel attention operation is carried out, objects with different scales are perceived through convolution with different kernel sizes, so that the expression capability of the features is improved; by adopting the mode of maximum pooling and average pooling on the channel, the aim of obtaining remote dependence of each pixel point by using a transducer can be fulfilled under the condition of keeping shallow characteristic space information as much as possible, and meanwhile, the calculated amount is remarkably reduced.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
Fig. 1 is a flowchart of an oil leakage detection method of an electric power device based on an attention mechanism according to an embodiment of the present invention;
FIG. 2 is a network architecture diagram of an oil leak detection model in an embodiment of the invention;
FIG. 3 is a network block diagram of an attention pooling capture module in an embodiment of the invention;
fig. 4 is a network configuration diagram of a hierarchical channel attention module in an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described in the following in conjunction with the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1 to 2, a first embodiment of the present invention proposes an attention mechanism-based oil leakage detection method for an electrical device, the method comprising the steps of:
s10, acquiring an image of power equipment to be detected, and taking the image of the power equipment to be detected as input of an oil leakage detection model, wherein the oil leakage detection model adopts a YOLOv5 neural network, the YOLOv5 neural network comprises a back bone network, a Neck network and a Head network which are adjacent in sequence, an attention pooling capture module is arranged in a jumper connection layer of the Neck network, and a hierarchical channel attention module is added in the back bone network and at the tail end of the Neck network;
s20, extracting features of the images of the power equipment to be detected through the backhaul network, and sequentially obtaining feature images with different depths;
s30, inputting the feature images with different depths into the Neck network through the attention pooling capturing module to obtain feature images with different sizes which are preliminarily output by the Neck network;
s40, passing the feature images with different sizes through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information;
s50, inputting different target information into the Head network to build a feature pyramid, and obtaining an oil leakage detection result of the power equipment.
According to the embodiment, by improving the YOLOv5 network structure, an attention pooling capturing module is arranged in a jumper layer of a Neck network, the fusion capability of the network between deep and shallow features is improved, the capability of the network for extracting remote features is improved, and a hierarchical channel attention module is added in a backhaul network and at the tail end of the Neck network, so that the feature extraction and expression capability of the network on a target object is improved; meanwhile, the system has global capturing and local strengthening capabilities, so that the characteristic expression capability is better improved, and the accuracy of oil leakage detection of the system is improved.
In an embodiment, the backhaul network includes a first feature extraction branch network, a second feature extraction branch network, and a third feature extraction branch network that are sequentially connected, where the third feature extraction branch network is provided with the hierarchical channel attention module;
the image of the power equipment to be tested is used as the input of the first characteristic extraction branch network, and the first characteristic extraction branch network outputs a characteristic diagram phi 1 (x) The second feature extraction branch network outputs a feature map phi 2 (x) The third feature extraction branch network outputs a feature map phi 3 (x)。
It should be noted that, feature maps phi of different depths extracted via a backhaul network 1 (x)、φ 2 (x) And phi 3 (x) Typically with different information, shallow features are smaller in receptive field and can retain more detailed appearance information, while deepThe network receptive field is larger, and more semantic information can be generally extracted. The fusion of the two features can increase semantic information of shallow features, so that enough context information exists when shallow segmentation is performed, and meanwhile, the deep features can have target detail information.
Further, the traditional neural network does not have an attention mechanism, the obtained target object features are not outstanding, and in the embodiment, by adding the hierarchical channel attention module in the third feature extraction branch network, the responses of objects with different dimensions on different channels can be obtained, and the reinforced response capability of the feature map on the channels is improved, so that the feature representation capability of the network on the target object is improved.
In an embodiment, the negk network includes a plurality of feature extraction networks, each feature extraction network includes a CSP2 module, a convolutional layer Conv, and a hierarchical channel attention module GCA, and a plurality of attention pooling capture modules TPC are disposed in a jumper layer of the negk network, and each of the attention pooling capture modules TPC is connected to the CSP2 module in one of the feature extraction networks.
The step S30: the feature images with different depths are input into the Neck network through the attention pooling capture module to obtain feature images with different sizes which are preliminarily output by the Neck network, and the method specifically comprises the following steps:
the characteristic diagram phi 1 (x) And phi 3 (x) After Concat operation and splicing, the processed images are input into a feature extraction network in the Neck network through the corresponding attention pooling capture module, and a feature graph phi preliminarily output by the Neck network is obtained 4 (x) The formula is:
φ 4 (x)＝CSP 2 (TPC(Concat(φ 1 (x),φ 3 (x))))
the characteristic diagram phi 3 (x) And phi 4 (x) After Concat operation and splicing, the processed images are input into a feature extraction network in the Neck network through the corresponding attention pooling capture module, and a feature graph phi preliminarily output by the Neck network is obtained 5 (x) The formula is:
φ 5 (x)＝CSP 2 (TPC(Concat(φ 4 (x),φ 3 (x))))
the characteristic diagram phi 2 (x) And phi 5 (x) After Concat operation and splicing, the processed result is input into the Neck network through the corresponding attention pooling capture module, and a feature map phi preliminarily output by the Neck network is obtained 6 (x) The formula is:
φ 6 (x)＝CSP 2 (TPC(Concat(φ 2 (x),φ 5 (x))))
in the GPS 2 () Representing CPS module operation, CSP2 represents CSP module (Cross Stage Partial, CSPNet) in YOLOv5 original network, TPC () represents attention pooling capture operation, and Concat () represents splice operation.
Incidentally, phi 4 (x)，φ 5 (x) And phi 6 (x) For the preliminary output of the negk network, feature maps of three different feature sizes are represented.
In one embodiment, the step S40: and passing the feature images with different sizes through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information:
O 4 (x)＝GCA(Conv(φ 4 (x)))
O 5 (x)＝GCA(Conv(φ 5 (x)))
O 6 (x)＝GCA(Conv(φ 6 (x)))
wherein O is 4 (x)、O 5 (x) And O 6 (x) GCA () represents the convolution of the hierarchical channel attention module on the channel and Conv () represents the convolution operation, respectively, for different target information.
It should be noted that, through the attention network, the target response position can be better mined. Thus, three feature maps phi are taken 4 (x)，φ 5 (x) And phi 6 (x) The required target information can be further extracted through the corresponding GCA module.
Further, target information O 4 (x)，O 5 (x) And O 6 (x) A feature pyramid (Feature Pyramid Networks, FPN) is built and input to the Head network to obtain the final predictions. Thus combineIn operation, the FPN layer conveys strong semantic features from top to bottom, the feature pyramid conveys strong positioning features from bottom to top, and parameter aggregation is performed on different detection layers from different backbone layers.
In an embodiment, the step S50 of inputting different target information to the Head network to build a feature pyramid to obtain an oil leakage detection result of the power device includes:
the target information O 4 (x)，O 5 (x) And O 6 (x) Downsampling to the same size by convolution, such as 19 x 19;
detecting the target information by using an Anchor-free head network respectively, wherein a classification head is used for extracting category information of the target, and a regression head is used for regressing the distance from each point on the feature map to the boundary of the target;
and (3) regressing the distance of the target object through the point with the highest response value on the classification map, and carrying out weighted summation on the results of the three feature maps to obtain the oil leakage detection result.
In the actual detection process, three detection heads in the Head network respectively respond to oil stains with different sizes and oil stain targets
In one embodiment, as shown in fig. 3, the hierarchical channel attention module GCA includes convolution layers conv_1, conv_2, conv_3, average pooling layers avgpool_1, avgpool_2, avgpool_3, maximum pooling layers maxpool_1, maxpool_2, maxpool_3, and multi-layer perceptrons mlp_1, mlp_2, mlp_3, mlp_4;
the input features are respectively processed through the convolution layers Conv_1, conv_2 and Conv_3 to obtain the graded feature output C on the corresponding channel 1(x) 、C 3(x) 、C 5(x) Feature C 1(x) Characteristic C as input to average pooling layer Avgpool_1 and maximum pooling layer Maxpool_1, respectively 3(x) Characteristic C as input to average pooling layer Avgpool_2 and maximum pooling layer Maxpool_2, respectively 5(x) As inputs to the average pooling layer avgpool_3 and the maximum pooling layer maxpool_3, respectively;
the output of the average pooling layer Avgpool_1 and the output of the maximum pooling layer Maxpool_1 are input to the multi-layer perceptron MLP_1 after splicing operation, the output of the average pooling layer Avgpool_2 and the output of the maximum pooling layer Maxpool_2 are input to the multi-layer perceptron MLP_2 after splicing operation, the output of the average pooling layer Avgpool_3 and the output of the maximum pooling layer Maxpool_3 are input to the multi-layer perceptron MLP_3 after splicing operation, and the output of the multi-layer perceptrons MLP_1, MLP_2 and MLP_3 are input to the multi-layer perceptron MLP_4 after splicing operation, so that a hierarchical channel characteristic sequence is obtained;
and the result of multiplying the hierarchical channel feature sequence and the input feature is spliced and reconstructed with the input feature to form the shape of the input feature.
Specifically, the hierarchical channel attention module GCA captures the spatial feature similarity between the same or adjacent channels by convolving on the channels. First, the input features are rebuilt to feature maps in the spatial direction, with length and width H, W. Subsequently, the feature map obtains feature outputs classified on the channels through convolution layers of kernel sizes 1×1,3×3,5×5, i.e., conv_1, conv_2, conv_3, respectively:
C 1(x) ＝Conv 1×1 (x)
C 3(x) ＝Conv 3×3 (x)
C 5(x) ＝Conv 5×5 (x)
these three characteristic outputs C 1(x) 、C 3(x) 、C 5(x) And obtaining channel characteristic representations along the space dimension through the average pooling layer and the maximum pooling layer respectively. These three feature representations are then passed through corresponding multi-layer perceptrons (Multilayer Perceptron, MLP), namely mlp_1, mlp_2, mlp_3, respectively, to obtain enhanced feature representations:
C 1(x) '＝MLP(Avgpool(C 1(x) )+Maxpool(C 1(x) ))
C 3(x) '＝MLP(Avgpool(C 3(x) )+Maxpool(C 3(x) ))
C 5(x) '＝MLP(Avgpool(C 5(x) )+Maxpool(C 5(x) ))
the resulting enhanced feature representation is then linearly weighted to a feature sequence as an output of the hierarchical channel feature sequence. Finally, the sequence is multiplied by the original feature, and then reconstructed to the original feature shape, the gradient efficiency is ensured by a residual error module as shown in fig. 4, the original feature is reserved in the actual process, the feature map obtained by GCA is multiplied by a learnable parameter, and the two parameters are added to obtain the final result.
It should be noted that, the GCA uses convolution with different convolution kernel sizes to convolve the original image to obtain the attention output result of the feature map, and then combines the output results of the feature map to obtain the final feature output, that is, a structure combining coarse and fine is adopted to analyze the feature layers with different depths layer by layer, so as to improve the expression capability of the features; the edge information and the average information of the target are respectively extracted in a mode of maximum pooling and average pooling on the channel, so that the aim of obtaining remote dependence of each pixel point by using a transducer can be fulfilled under the condition that shallow characteristic space information is kept as much as possible, and meanwhile, the calculated amount is remarkably reduced; and the output size is consistent with the input size, so that the applicability of the module in a network is ensured, the GCA module is added in the characteristic extraction process of the network, and the characteristic characterization capability of the model is improved.
In an embodiment, as shown in fig. 4, the attention pooling capturing module TPC includes three first convolution layers Conv1, three multi-head attention networks MHA, and three second convolution layers Conv2, wherein for any one of the feature maps of different sizes, the output of each of the first convolution layers Conv1 is connected to one of the multi-head attention networks MHA through Reshape operation, and the output of the multi-head attention network MHA is connected to one of the second convolution layers Conv2 through Reshape operation;
and the outputs of the three second convolution layers Conv2 are connected through splicing operation and then output to a third convolution layer Conv3.
It should be noted that, the attention pooling capturing module TPC constructs in parallel a feature pyramid module composed of tansformers connected to different channels, so as to improve the capability of characterizing shallow features, so as to enhance the feature fusion effect between deep features and shallow features.
Further, the attention pooling capturing module TPC is formed by stacking feature maps of different channels through a multi-head attention network, and the step of processing the feature maps of different sizes includes:
(1) Outputting characteristic diagrams x of different channels to an input characteristic diagram through different first convolution layers Conv1 1 ,x 2 And x 3 The different channel numbers are (C/2), (C/4) and (C/8);
(2) Feature images of different channels are flattened to a two-dimensional feature image through Reshape operation, and then feature sequences corresponding to the different channels are obtained through a multi-head attention network corresponding to each channel;
in particular, the feature maps are flattened into two-dimensional feature maps and passed through a multi-headed attention network, which can learn different channel features based on the same attention mechanism, and then recombine the different features. Given the query q, key k, and value v, each attention header h i (i=1.,. The calculation method of h) is:
h i ＝f(W i (q) q,W i (k) k,W i (v) v)
wherein h is i Is the output of the modified feature map and q, k, v is the input. In this formula, all three are derived from the current feature, W i The function f that is a learnable parameter, and represents attention pooling, may be additive attention and scaled "dot-product" attention. The output of the multi-head attention needs to undergo another linear transformation, which corresponds to the result after h head splices. Based on this design, each head may be concerned with a different portion of the input channel. A more complex function than a simple weighted average can be represented.
(3) And after the feature sequences corresponding to different channels are transformed to the feature graphs corresponding to the channels through Reshape operation, splicing and outputting the feature sequences after passing through different second convolution layers Conv 2.
Specifically, the signature sequences of all three different channels are reshaped to x 1 ,x 2 And x 3 After last passing through the second convolution layer, the three feature mapsAnd are Concat together to obtain the final output.
Further, in the pair of feature graphs x 3 When convolving, the feature map is up-sampled as a (C/4) channel to obtain the final identical channel representation.
It should be noted that, the attention pooling capturing module TPC uses features of different scales to capture remote information of the transducer through a specially constructed feature pyramid module, and adds TPC modules between deep and shallow features of the network, so as to improve the fusion capability of the features of the deep and shallow layers of the model, and the output size and the input size of the modules are consistent, thereby ensuring the applicability of the modules in the network.
In one embodiment, in the step S10: before acquiring the image of the electric equipment to be detected and taking the image of the electric equipment to be detected as the input of the oil leakage detection model, the method further comprises the following steps:
collecting an oil leakage data set of the power equipment;
carrying out oil stain and oil stain range marking on sample data in the data set by using Labelme, and dividing the data set into a test set, a training set and a verification data set according to the proportion of 1:7:2;
and training, testing and verifying the oil leakage detection model by using the training set, the testing set and the verification data set respectively, and reserving optimal parameters to obtain the trained oil leakage detection model.
In an embodiment, after the collecting the electrical equipment oil leakage dataset, the method further comprises:
and screening, size cutting and data augmentation are carried out on the sample data in the data set, so that a preprocessed data set is obtained.
Specifically, the size clipping refers to clipping to a uniform size, and the data augmentation can specifically adopt a flipping operation and the like.
Further, in the training process, epoch is set to 250, batch size is set to 32, and the size of the input image is set to 640×640.
It should be noted that, by improving YOLOv5 algorithm, GCA module and TPC module are added in the network, so as to improve the feature extraction and feature fusion capability of the network while maintaining the original algorithm with low parameter number and low calculation amount, aiming at the conditions of fire, breakdown, short circuit, burning loss, etc. caused by oil leakage. In practical application, mAP is utilized to detect the system, accuracy and Recall, and the result proves that the algorithm has excellent detection result and greatly improves the performance on the basis of the original network.
Furthermore, a second embodiment of the present invention provides a computer readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for detecting oil leakage of an electrical device based on an attention mechanism as set forth in the first embodiment.
It should be noted that, in other embodiments of the computer readable storage medium or the implementation method of the present invention, reference may be made to the above method embodiments, which are not repeated here.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (10)
1. An attention mechanism-based oil leakage detection method for electric equipment, which is characterized by comprising the following steps:
acquiring an image of power equipment to be detected, and taking the image of the power equipment to be detected as input of an oil leakage detection model, wherein the oil leakage detection model adopts a YOLOv5 neural network, the YOLOv5 neural network comprises a back bone network, a Neck network and a Head network which are adjacent in sequence, an attention pooling capture module is arranged in a jumper layer of the Neck network, and a hierarchical channel attention module is added in the back bone network and at the tail end of the Neck network;
extracting features of the images of the power equipment to be detected through the backhaul network, and sequentially obtaining feature images with different depths;
inputting the feature images with different depths into the Neck network through the attention pooling capturing module to obtain feature images with different sizes preliminarily output by the Neck network;
the feature images with different sizes pass through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information;
and inputting different target information into the Head network to build a feature pyramid, so as to obtain an oil leakage detection result of the power equipment.
2. The attention mechanism-based power equipment oil leakage detection method as set forth in claim 1, wherein the backhaul network includes a first feature extraction branch network, a second feature extraction branch network, and a third feature extraction branch network connected in sequence, the third feature extraction branch network having the attention module disposed therein;
the image of the power equipment to be tested is used as the input of the first characteristic extraction branch network, and the first characteristic extraction branch network is used for inputtingGo out the characteristic map phi 1 (x) The second feature extraction branch network outputs a feature map phi 2 (x) The third feature extraction branch network outputs a feature map phi 3 (x)。
3. The method for detecting oil leakage of power equipment based on attention mechanism as recited in claim 1, wherein said feature maps of different depths are phi respectively 1 (x)、φ 2 (x) And phi 3 (x) A plurality of attention pooling capturing modules are arranged in a jumper layer of the negk network, the feature images with different depths are input to the negk network through the attention pooling capturing modules, and feature images with different sizes preliminarily output by the negk network are obtained, and the method comprises the following steps:
the characteristic diagram phi 1 (x) And phi 3 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 4 (x)；
The characteristic diagram phi 3 (x) And phi 4 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 5 (x)；
The characteristic diagram phi 2 (x) And phi 5 (x) After being spliced, the processed images are input into the Neck network through the corresponding attention pooling capture module to obtain a feature map phi preliminarily output by the Neck network 6 (x)。
4. The method for detecting oil leakage of power equipment based on attention mechanism as claimed in claim 1, wherein the feature patterns of different sizes are phi respectively 4 (x)、φ 5 (x) And phi 6 (x) And the feature images with different sizes pass through a hierarchical channel attention module arranged at the tail end of the Neck network to obtain different target information:
O 4 (x)＝GCA(Conv(φ 4 (x)))
O 5 (x)＝GCA(Conv(φ 5 (x)))
O 6 (x)＝GCA(Conv(φ 6 (x)))
wherein O is 4 (x)、O 5 (x) And O 6 (x) GCA () represents the convolution of the attention module on the channel and Conv () represents the convolution operation, respectively, for different target information.
5. The attention mechanism based power equipment oil leakage detection method of claim 1 or 4, wherein the hierarchical channel attention module comprises convolution layers conv_1, conv_2, conv_3, average pooling layers avgpool_1, avgpool_2, avgpool_3, maximum pooling layers maxpool_1, maxpool_2, maxpool_3, and multi-layer perceptrons mlp_1, mlp_2, mlp_3, mlp_4;
the input features are respectively processed through the convolution layers Conv_1, conv_2 and Conv_3 to obtain the graded feature output C on the corresponding channel 1(x) 、C 3(x) 、C 5(x) Feature C 1(x) Characteristic C as input to average pooling layer Avgpool_1 and maximum pooling layer Maxpool_1, respectively 3(x) Characteristic C as input to average pooling layer Avgpool_2 and maximum pooling layer Maxpool_2, respectively 5(x) As inputs to the average pooling layer avgpool_3 and the maximum pooling layer maxpool_3, respectively;
the output of the average pooling layer Avgpool_1 and the output of the maximum pooling layer Maxpool_1 are input to the multi-layer perceptron MLP_1 after splicing operation, the output of the average pooling layer Avgpool_2 and the output of the maximum pooling layer Maxpool_2 are input to the multi-layer perceptron MLP_2 after splicing operation, the output of the average pooling layer Avgpool_3 and the output of the maximum pooling layer Maxpool_3 are input to the multi-layer perceptron MLP_3 after splicing operation, and the output of the multi-layer perceptrons MLP_1, MLP_2 and MLP_3 are input to the multi-layer perceptron MLP_4 after splicing operation, so that a hierarchical channel characteristic sequence is obtained;
and the result of multiplying the hierarchical channel feature sequence and the input feature is spliced and reconstructed with the input feature to form the shape of the input feature.
6. The attention mechanism-based power equipment oil leakage detection method as set forth in claim 1 wherein the attention pooling capture module comprises three first convolution layers, three multi-headed attention networks and three second convolution layers, wherein any one of the feature maps of different sizes is input to each of the first convolution layers, the output of each of the first convolution layers is connected to one of the multi-headed attention networks via Reshape operation, and the output of the multi-headed attention network is connected to one of the second convolution layers via Reshape operation;
and the outputs of the three second convolution layers are connected through splicing operation and then output to a third convolution layer.
7. The attention mechanism based power plant oil leak detection method as recited in claim 1 or 6, wherein the step of the attention pooling capture module processing feature maps of different sizes includes:
outputting feature graphs of different channels to an input feature graph through different first convolution layers;
the feature images of different channels are flattened to two-dimensional feature images and then pass through a multi-head attention network corresponding to each channel to obtain feature sequences corresponding to different channels;
after the feature sequences corresponding to different channels are transformed to the sizes of the feature graphs corresponding to the channels, the feature sequences are spliced and output after passing through different second convolution layers respectively.
8. The attention mechanism based power equipment oil leakage detection method as set forth in claim 1 wherein prior to the acquiring an image of the power equipment under test and taking the image of the power equipment under test as input to an oil leakage detection model, the method further comprises:
collecting an oil leakage data set of the power equipment;
carrying out oil stain and oil stain range marking on sample data in the data set by using Labelme, and dividing the data set into a test set, a training set and a verification data set;
and training, testing and verifying the oil leakage detection model by using the training set, the testing set and the verification data set respectively to obtain the trained oil leakage detection model.
9. The attention-based power plant oil leakage detection method as set forth in claim 7 wherein after the collecting the power plant oil leakage dataset, the method further comprises:
and screening, size cutting and data augmentation are carried out on the sample data in the data set, so that a preprocessed data set is obtained.
10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310438083.1A CN116523858A (en) | 2023-04-21 | 2023-04-21 | Attention mechanism-based oil leakage detection method for power equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310438083.1A CN116523858A (en) | 2023-04-21 | 2023-04-21 | Attention mechanism-based oil leakage detection method for power equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116523858A true CN116523858A (en) | 2023-08-01 |
Family
ID=87398819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310438083.1A Pending CN116523858A (en) | 2023-04-21 | 2023-04-21 | Attention mechanism-based oil leakage detection method for power equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116523858A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557493A (en) * | 2023-08-30 | 2024-02-13 | 四川轻化工大学 | Transformer oil leakage detection method, system, electronic equipment and storage medium |
-
2023
- 2023-04-21 CN CN202310438083.1A patent/CN116523858A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557493A (en) * | 2023-08-30 | 2024-02-13 | 四川轻化工大学 | Transformer oil leakage detection method, system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339858B (en) | Oil and gas pipeline marker identification method based on neural network | |
| CN108961235B (en) | Defective insulator identification method based on YOLOv3 network and particle filter algorithm | |
| CN114202672A (en) | Small target detection method based on attention mechanism | |
| CN111368690B (en) | Deep learning-based video image ship detection method and system under influence of sea waves | |
| Komorowski et al. | Minkloc++: lidar and monocular image fusion for place recognition | |
| CN110619638A (en) | Multi-mode fusion significance detection method based on convolution block attention module | |
| Alonso et al. | Coral-segmentation: Training dense labeling models with sparse ground truth | |
| CN110929736A (en) | Multi-feature cascade RGB-D significance target detection method | |
| CN110705566B (en) | Multi-mode fusion significance detection method based on spatial pyramid pool | |
| CN110826411B (en) | Vehicle target rapid identification method based on unmanned aerial vehicle image | |
| CN112115879B (en) | Self-supervision pedestrian re-identification method and system with shielding sensitivity | |
| CN115439694A (en) | High-precision point cloud completion method and device based on deep learning | |
| CN111881743B (en) | Facial feature point positioning method based on semantic segmentation | |
| CN115690542A (en) | Improved yolov 5-based aerial insulator directional identification method | |
| CN113269224A (en) | Scene image classification method, system and storage medium | |
| CN116523858A (en) | Attention mechanism-based oil leakage detection method for power equipment and storage medium | |
| CN117541534A (en) | Power transmission line inspection method based on unmanned plane and CNN-BiLSTM model | |
| CN113537119B (en) | Transmission line connecting part detection method based on improved Yolov4-tiny | |
| CN116681885B (en) | Infrared image target identification method and system for power transmission and transformation equipment | |
| CN117853417A (en) | Weld defect identification method and device, electronic equipment and storage medium | |
| CN113516652A (en) | Battery surface defect and adhesive detection method, device, medium and electronic equipment | |
| CN117523394A (en) | SAR vessel detection method based on aggregation characteristic enhancement network | |
| CN116934696A (en) | Industrial PCB defect detection method and device based on YOLOv7-Tiny model improvement | |
| KR20210037199A (en) | Apparatus for dividing, tagging an image and for detecting defect of facilities using the same | |
| CN114998726A (en) | Buried pipeline detection method based on three-component magnetic field curve image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |