CN113361520B

CN113361520B - Transmission line equipment defect detection method based on sample offset network

Info

Publication number: CN113361520B
Application number: CN202110611302.2A
Authority: CN
Inventors: 毛进伟; 罗旺; 陈海鹏
Original assignee: Nari Information and Communication Technology Co
Current assignee: Nari Information and Communication Technology Co
Priority date: 2021-06-01
Filing date: 2021-06-01
Publication date: 2022-08-09
Anticipated expiration: 2041-06-01
Also published as: CN113361520A

Abstract

The invention discloses a method for detecting the defect of power transmission line equipment based on a sample offset network, which comprises the steps of sending a data set into a designed convolutional neural network model for training, and perfecting the parameters of the convolutional neural network model; and deploying the trained convolutional neural network model to detection equipment, and detecting the defects of the transmission line equipment. The invention utilizes the mosaic data enhancement method to process data, can supplement images lacking in samples, enriches background information of defect types and reduces the condition of network overfitting. By the aid of the feature extraction and feature fusion module, the diversity of input images is enriched, the network can accurately judge the region of interest, and the detection capability of the network is enhanced. And the classification task and the regression task obtain different candidate regions by correcting the positions of the candidate frames. The number of the actions identified by the identification method has expandability, and the expansion operation is simple and easy for developers to operate.

Description

Transmission line equipment defect detection method based on sample offset network

Technical Field

The invention relates to a power transmission line equipment defect detection method based on a sample offset network, and belongs to the technical field of image data processing and neural networks in the field of artificial intelligence.

Background

Because the scale of the power grid in China is increasing day by day, the inspection quantity and quality of the power field have higher requirements on power personnel, the environment of the power transmission field is complex, various devices are distributed, the devices are connected through various overhead lines and difficult to accurately observe, and all the devices are required to normally cooperate to ensure that the power transmission line can stably run. The early fault inspection mode mainly depends on field workers to perform manual analysis and diagnosis through manual detection experience, the mode has high professional requirement level on the workers, the shutdown detection is often accompanied, a large amount of manpower, material resources and financial resources are consumed, the time is long, the danger is high, and the influence of personal experience is easily caused. Therefore, advanced vision technology is urgently needed to be introduced into the industry to improve the working efficiency of workers. Object detection is a problem of current computer vision field, and is also a current research focus. For example, in the electric power job site, under the environment that is difficult to observe such as construction environment, adopt unmanned aerial vehicle or the robot of patrolling and examining of carrying on target detection technique to survey the environment more effectively.

In recent years, with the development of big data technology and technological innovation in the field of computer vision, more and more researchers turn the research direction to Deep Learning (Deep Learning). The deep learning technology simulates a human neural structure to establish a neural network, learns the analysis and processing method of human brain on various data in reality, and automatically adjusts and updates according to the actual situation, so that experts of most industries begin to explore and utilize the method to solve the actual problem faced by fault diagnosis of an industrial system.

Because the convolutional neural network can extract the multilayer characteristics of the detected image by increasing the network depth, the characteristic diversity is improved, and meanwhile, the convolutional neural network integrates the characteristic extraction, selection and classification through an end-to-end architecture, so that the network parameters are optimized on the whole, and the characteristic expression capability is enhanced. Compared with the classical artificial parameter adjusting method, the deep neural network makes a great breakthrough in aspects such as detection speed precision and the like, so that a great number of scholars apply the deep neural network to the field of computer vision. Under the verification of a multi-category image data set, with the optimization of a network structure, deep learning makes great progress in the field of target detection, and the deep learning method is popularized and applied due to the advantages of high detection speed, high detection accuracy, strong self-learning capability and the like.

Therefore, how to detect the defect of the power transmission line equipment by using the convolutional neural network is a technical problem which needs to be solved urgently by the technical personnel in the field.

Disclosure of Invention

The purpose is as follows: in order to overcome the defects in the prior art, the invention provides the power transmission line equipment defect detection method based on the sample offset network, and the method is high in identification precision and capable of identifying various types of defects.

The technical scheme is as follows: in order to solve the technical problems, the technical scheme adopted by the invention is as follows:

a method for detecting defects of power transmission line equipment based on a sample offset network comprises the following steps:

sending the data set into a designed convolutional neural network model for training, and perfecting the parameters of the convolutional neural network model;

and deploying the trained convolutional neural network model on detection equipment, and detecting the defects of the transmission line equipment.

Preferably, the data set acquisition step is as follows:

collecting related images of the defects of the power transmission line equipment, marking the collected related images, enhancing the data of the mosaic images to manufacture a data set, and dividing the data set into a test set and a training set according to a certain proportion.

Preferably, the collected correlation images should meet the following requirements: (1) the picture is clear and recognizable, and the focal length is correct; (2) the defect target is complete in the image, and the main features are not shielded; (3) the picture adopts an RGB three-channel color mode; (4) image preprocessing is performed on extraneous background that affects target detection.

Preferably, the method for labeling the collected related images includes the following steps:

and 4.1 marking the defect target by adopting a minimum positive circumscribed rectangle frame.

4.2 labeling all defect types in the defect image of the device, unless the following occurs:

a) the defect target visible region pixels are less than 30px by 30 px.

b) The defect target visible region cannot characterize the defect features.

4.3 mark the visible area of the defect target, and the detection frame contains all the pixels of the defect target.

4.4 the labeling label naming mode is ' Pinyin initial letter of defect description plus ' _ ' plus ' Pinyin initial letter of defect classification '.

4.5 the equipment patrols the image annotation file name and keeps consistent with the original image file name.

4.6 the markup document is an xml document meeting VOC standards.

As a preferred scheme, the designed convolutional neural network model includes: feature extraction network, candidate frame extraction network and positioning classification network.

The feature extraction network includes: a residual network and a feature pyramid network; the residual network uses Resnet101, which inputs image sizes (1666, 1000) and output channels 2048. The feature pyramid network uses FPN, the input of which is the C2, C3, C4, C5 features of the residual network, and the output channel is 256.

The candidate frame extraction network uses an RPN network, the RPN network adopts three anchor frames with the proportion of 1:1, 1:2 and 2:1, the characteristic channel is 256, and the loss calculation adopts cross entropy loss and L1 norm loss.

The positioning classification network adopts a parallel structure of a sample offset module and a shared detection head, and the sample offset module comprises: the first positioning branch and the first classification branch are formed by adopting one convolution layer and two fully-connected layers together, the convolution kernel size of the convolution layer is (3 x 3), the output of each layer is {128, 128, 2}, and the output is an offset matrix with the size of 1 x 2; the first classification branch uses an offset matrix with output of {128, 128, 2} and output of 1 x 2 for each of the three fully connected layers. The shared detection head comprises two layers of full connection layers, a second positioning branch and a second classification branch, the two layers of full connection layers are full connection layers with shared channels being 1024, the second classification branch is a full connection layer with the channels being 1024 and is classified by softmax, the second positioning branch is a full connection layer with the channels being 4, and cross entropy loss and L1 norm loss are adopted for network loss.

Preferably, the mosaic image data enhancement method includes the following steps:

taking data of a batch from the collected related images;

randomly extracting 4 pictures, cutting and splicing random positions, and synthesizing a new picture;

repeating the batch size times to finally obtain a plurality of pictures subjected to mosaic data enhancement;

the new data for each batch is then added to the training set as supplemental data.

Preferably, the method for processing candidate frames by the positioning classification network includes the following steps:

after superimposing the localization offset Δ L for each candidate frame P, the localization loss and the classification loss are calculated separately.

After superimposing the classification offset Δ C for each candidate frame P, the localization loss and the classification loss are calculated separately.

For each candidate box P, which passes directly through two fully-connected layers, the localization loss and classification loss are calculated separately.

Has the advantages that: compared with the prior art, the method for detecting the defects of the power transmission line equipment based on the sample offset network has the following remarkable progress:

1. the data are processed by using a mosaic data enhancement method, images lacking in samples can be supplemented, background information of defect types is enriched, and the condition of network overfitting is reduced.

2. By the aid of the feature extraction and feature fusion module, the diversity of input images is enriched, the network can accurately judge the region of interest, and the detection capability of the network is enhanced.

3. And detecting the defect image of the power transmission line by a sample offset module. Aiming at the characteristics of multiple defect types and complex background information of power equipment of the power transmission line, the positions of the candidate frames are corrected, so that different candidate regions are obtained by a classification task and a regression task, wherein the classification task obtains a region with rich semantic information, and the regression task obtains a region with rich boundary information.

4. The number of the actions identified by the identification method has expandability, and the expansion operation is simple and easy for developers to operate.

Drawings

FIG. 1 is a diagram of a target processing architecture of the present invention.

FIG. 2 is a schematic diagram of data labeling.

Fig. 3 is a method of data enhancement.

Fig. 4 is an overall configuration diagram of the network herein.

FIG. 5 is a diagram illustrating a process for training loss.

Fig. 6 is a schematic diagram of detection of a correlation image.

Detailed Description

The present invention will be further described with reference to the accompanying drawings.

step 1, collecting images related to defects of power transmission line equipment, carrying out marking and mosaic image data enhancement preprocessing on the collected data, making the data into a data set, and dividing the data set into a test set and a training set according to a certain proportion;

step 2, designing a convolutional neural network model, mainly dividing the convolutional neural network model into three parts of networks of feature extraction, candidate frame extraction and defect positioning classification, and designing different neural network structures aiming at different parts of networks;

step 3, sending the data set into a designed convolutional neural network model for training, and enabling the convolutional neural network model to spontaneously improve self parameters;

and 4, deploying the trained convolutional neural network model on detection equipment for practical use.

In step 1, the collected data should meet the following requirements: (1) the picture is clear and recognizable, and the focal length is correct; (2) the defect target is complete in the image, and the main features are not shielded; (3) the picture is preferably in an RGB three-channel color mode; (4) it is desirable to perform appropriate image preprocessing, such as blacking or cropping, on extraneous background that affects target detection.

In the step 1, the data needs to be subjected to image annotation, the annotation needs to be stored according to the voc format, and the image should be a minimum circumscribed rectangular frame with defects during annotation.

In the step 2, the convolutional neural network model comprises three parts, namely a feature extraction network, a candidate box extraction network and a positioning classification network.

Further, the feature extraction network, the residual error network and the feature pyramid network; the residual network uses Resnet101, which inputs image sizes (1666, 1000) and output channels 2048. The feature pyramid network uses FPN, the input of which is the C2, C3, C4, C5 features of the residual network, and the output channel is 256.

The candidate frame extraction network uses an RPN network, the network adopts three anchor frames with the proportion of 1:1, 1:2 and 2:1, the characteristic channel is 256, and the loss calculation adopts cross entropy loss and L1 norm loss.

In the step 3, the training set and the test set are divided according to the proportion of 4:1, and the training set which is input into the convolutional neural network model and used for adjusting parameters is the training set.

After the scheme is adopted, due to the advantages of the convolutional neural network, as long as the number of samples is enough, the defect types can be expanded to more by adjusting the parameters. The method has important practical application significance in the aspects of defect detection, target detection and the like.

Example (b):

the method obtains the defect detection model of the power transmission line equipment based on the preset training set and the deep learning structure training, and can detect six defect types, namely bird nest, self-explosion of an insulator, damage of a vibration damper, burying of a tower foundation, water immersion of the tower foundation and damage of a bird baffle plate.

Fig. 1 is a diagram of an overall architecture for defect detection, in which collected image data is subjected to image preprocessing and input to a convolutional neural network model for model training, the network is divided into three parts, namely feature extraction, candidate frame extraction and positioning classification, and network parameters are optimized through multiple iterations to finally obtain a trained convolutional neural network model. And inputting the test set image into the trained convolutional neural network model for testing, and judging the overall performance of the convolutional neural network model through the detection indexes.

Fig. 2 and fig. 3 illustrate the image preprocessing process, wherein fig. 2 illustrates data labeling, and since network training and testing both require training images and corresponding label information, labeling of collected image data is performed in advance. In order to meet the input requirement of the convolutional neural network model, the method adopts labellimg for labeling, and the labeling tool can process the image and generate voc format data for network training.

The data annotation is stored as follows:

4.2 it is desirable to label all defect types in the defect image of the device unless the following occurs:

a) the pixels of the visible region of the defect target are less than 30px by 30 px;

b) the defect target visible area cannot characterize the defect features.

4.3 mark the visible area of the defect object and the inspection box needs to contain all pixels of the defect object.

And 4.5, the name of the image annotation file in the equipment inspection is preferably consistent with the name of the original image file.

4.6 Annotation file is an xml file that meets VOC standards.

Fig. 3 is a process of enhancing mosaic data, in which data of one batch is first taken out from a total data set, then 4 pictures are randomly extracted, clipping and splicing at random positions are performed, new pictures are synthesized, the batch size is repeated for several times, finally, several pictures with the batch size after being enhanced by the mosaic data are obtained, and new data of each batch is added to a training set as supplementary data. Compared with other data enhancement methods, the mosaic needs four pictures for synthesis at one time, background diversity of the images is enriched, and a single GPU can achieve a good effect by calculating pixels of multiple images at one time in the normalization process. And because the original data image is randomly cut and scaled to a new position area, more small target objects appear in the data set after the mosaic data is enhanced, the overfitting situation is reduced, and the detection capability of the target detection network on the small objects is enhanced.

Fig. 4 shows the overall structure of the convolutional neural network model, and the essence of the sample shift module is to make the network autonomously obtain more effective candidate regions according to tasks, rather than using a uniform candidate frame, where the Backbone network (backhaul) of the feature extraction network is ResNet and the feature fusion network is FPN. After feature extraction, the input image enters a candidate frame extraction network RPN to obtain a series of candidate frames. And each candidate box enters the positioning classification network independently and is subjected to classification and regression calculation loss respectively. The positioning classification network is divided into three parts, and for each candidate frame P, after a position offset delta L and a position offset delta C are respectively superposed on the P in the first part and the second part, the positioning loss and the classification loss are respectively calculated; in the third part, P directly passes through two fully connected layers to calculate the classification and localization loss. The delta L is a positioning offset, because the convolutional neural network can capture spatial information more easily, the positioning offset is obtained by training a convolutional layer and a full-link layer, and the edge characteristics of the target can be better obtained by the candidate frame after the offset is superposed; and the delta C is a classification offset, is trained by using a full connection layer, and is superposed with the original candidate frame to enable the offset candidate frame to obtain more semantic information of the target.

Fig. 5 is a process of loss variation in training, and it can be seen that the loss is a relatively normal trend, and the loss continuously decreases in the training process and tends to converge after a certain number of times.

Fig. 6 is a detection example diagram of a related network, and it can be seen that the design method can still accurately detect a small defect in a power transmission line.

When the trained network structure meets the design requirements, the model can be transplanted to other equipment terminals for use. If the trained convolutional neural network does not meet the design requirements, the number of the neurons of each hidden layer needs to be modified. It is appropriate to modify the number of neurons to which value, and repeated testing is required. If the method for modifying the number of the neurons of each hidden layer has little influence on the identification accuracy, the number of the hidden layers or the number of training samples is recommended to be added.

Therefore, according to the principle that the demand characteristics of the classification task and the regression task are different, the candidate frame offset module capable of spontaneously adjusting the positions of the candidate frames is designed, and the module can calculate two offsets through the simple convolution layer and the full connection layer, so that the classification task can obtain the candidate frames with more semantic information, the regression task can obtain the candidate frames with more boundary information, the classification and positioning capabilities of the model are effectively improved, and the detection of the defects of the power transmission line equipment is realized.

The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.

Claims

1. A method for detecting defects of power transmission line equipment based on a sample offset network is characterized by comprising the following steps: the method comprises the following steps:

deploying the trained convolutional neural network model on detection equipment, and detecting the defects of the transmission line equipment;

the designed convolutional neural network model comprises the following components: a feature extraction network, a candidate frame extraction network and a positioning classification network;

the feature extraction network includes: a residual network and a feature pyramid network; the residual network uses Resnet101, the network inputs image size (1666, 1000), the output channel is 2048; the feature pyramid network uses FPN, the input of the network is the C2, C3, C4 and C5 features of the residual error network, and the output channel is 256;

the candidate frame extraction network uses an RPN network, the RPN network adopts three anchor frames with the proportion of 1:1, 1:2 and 2:1, the characteristic channel is 256, and the loss calculation adopts cross entropy loss and L1 norm loss;

the positioning classification network adopts a parallel structure of a sample offset module and a shared detection head, and the sample offset module comprises: the first positioning branch and the first classification branch are formed by adopting one convolution layer and two fully-connected layers together, the convolution kernel size of the convolution layer is (3 x 3), the output of each layer is {128, 128, 2}, and the output is an offset matrix with the size of 1 x 2; the first classification branch adopts an offset matrix with the output of each layer of three fully-connected layers being {128, 128, 2} and the output being 1 × 2; the shared detection head comprises two layers of full connection layers, a second positioning branch and a second classification branch, the two layers of full connection layers are full connection layers with shared channels being 1024, the second classification branch is a full connection layer with the channels being 1024 and is classified by softmax, the second positioning branch is a full connection layer with the channels being 4, and cross entropy loss and L1 norm loss are adopted for network loss.

2. The method for detecting the defect of the power transmission line equipment based on the sample offset network according to claim 1, wherein the method comprises the following steps: the data set acquisition steps are as follows:

3. The method for detecting the defect of the power transmission line equipment based on the sample offset network as claimed in claim 2, wherein: the relevant images collected should meet the following requirements: (1) the picture is clear and recognizable, and the focal length is correct; (2) the defect target is complete in the image, and the main features are not shielded; (3) the picture adopts an RGB three-channel color mode; (4) image preprocessing is performed on extraneous background that affects target detection.

4. The method for detecting the defect of the power transmission line equipment based on the sample offset network as claimed in claim 2, wherein: the method for labeling the collected related images comprises the following steps:

4.1 marking a defect target by adopting a minimum positive external rectangular frame;

b) the defect target visible area cannot represent defect characteristics;

4.3 marking the visible area of the defect target, wherein the detection frame comprises all pixels of the defect target;

4.4 the labeling label naming mode is ' Pinyin initial letter of defect description plus ' _ ' plus ' Pinyin initial letter of defect classification ';

4.5 the equipment patrols the image annotation file name and keeps consistent with the original image file name;

4.6 the markup document is an xml document meeting VOC standards.

5. The method for detecting the defect of the power transmission line equipment based on the sample offset network as claimed in claim 2, wherein: the mosaic image data enhancement comprises the following steps:

taking data of a batch from the collected related images;

6. The method for detecting the defect of the power transmission line equipment based on the sample offset network according to claim 1, wherein the method comprises the following steps: the method for processing the candidate frame by the positioning classification network comprises the following steps:

after the positioning offset delta L is superposed on each candidate frame P, respectively calculating the positioning loss and the classification loss;

after the classification offset delta C is superposed on each candidate frame P, respectively calculating the positioning loss and the classification loss;