CN107680090A - Based on the electric transmission line isolator state identification method for improving full convolutional neural networks - Google Patents

Abstract

The invention discloses a kind of based on the electric transmission line isolator state identification method for improving full convolutional neural networks, comprise the following steps：S1, pass through unmanned plane collection electric transmission line isolator picture；S2, classification recurrence and position recurrence are carried out to picture by target detection network Faster R CNN, intercept out individually insulation sub-pictures；S3, the sub-pictures that will insulate do semantic segmentation by full convolutional neural networks；S4, the segmentation that becomes more meticulous carried out by full condition of contact random field；S5, filter out with morphological operation method noise in image；S6, by deep learning sorter network to insulate subclassification, judge insulator state.The present invention to the insulation sub-pictures marked by being trained and arameter optimization, effectively electric transmission line isolator state can be identified, avoid the randomness of the subjective impact that threshold value is manually set in conventional insulators state recognition and artificial extraction feature, line walking efficiency can be greatly improved, reduces line walking difficulty.

Description

Power transmission line insulator state identification method based on improved full convolution neural network

Technical Field

The invention belongs to the field of deep learning image processing and the field of electric power defect identification, and particularly relates to a transmission line insulator state identification method based on an improved full convolution neural network.

Background

The insulator in the high-voltage transmission line is a very important part in the transmission line, but the insulator is exposed in the field for a long time, is damaged by wind, snow, rain and fog and human factors, is a device with multiple faults, and if the insulator has a fault hidden trouble, the safety of the high-voltage power grid is greatly threatened, and loss which is difficult to estimate is possibly caused. However, most of transmission lines are in traffic dead zones and unmanned zones, which results in high inspection difficulty and long inspection cycle. With the increasing scale of high-voltage transmission lines, manual line patrol is influenced by factors such as weather and topography, and the task of patrolling insulator faults is increasingly heavy.

The traditional mode identification method is adopted in the traditional transmission line insulator state identification, firstly, the hue, the color saturation, the brightness and the color space and the like are used as characteristics, the maximum inter-class variance method is used for segmenting the image, and an insulator foreground communicating region is obtained; then, the state of the insulator is determined by combining statistical characteristics such as a histogram. The disadvantages of this approach are many, firstly, the color space is susceptible to illumination; secondly, the maximum inter-class variance method needs to manually set a threshold, but the background of the power transmission line is complex, and the threshold segmentation method cannot smoothly segment the insulators from the background.

In 2012, deep learning attracts people's attention and has achieved good recognition effect in image recognition and detection. The invention researches the application of deep learning in the state recognition of the insulator of the power transmission line.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide the power transmission line insulator state identification method based on the improved full convolution neural network, which can effectively identify the state of the insulator of the power transmission line by training and optimizing parameters of the marked insulator picture, judge whether the insulator bursts or not, greatly improve the line inspection efficiency and reduce the line inspection difficulty.

The purpose of the invention is realized by the following technical scheme: the method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network comprises the following steps:

s1, collecting the insulator pictures of the power transmission line by an unmanned aerial vehicle, preprocessing the collected pictures, and adjusting the pictures to be the same size;

s2, performing classification regression and position regression on the picture through a target detection network Faster R-CNN to obtain the pixel position of the insulator on the picture and the confidence coefficient of the pixel position; then, cutting the obtained insulator pixel position, and intercepting an independent insulator picture;

s3, semantic segmentation is carried out on the insulator picture through a full convolution neural network, pixel-level classification is carried out on the picture, and insulators are extracted from the background;

s4, performing refined segmentation on the semantic segmentation result obtained in the S3 through a full-connection conditional random field, and enhancing edge constraint;

s5, filtering noise points in the image by using a morphological operation method;

and S6, classifying the insulators through a deep learning classification network, and judging the state of the insulators.

Further, the resolution of the picture obtained after the preprocessing in the step S1 is 3936 × 2624.

Further, the step S2 includes the following sub-steps:

s21, normalizing the picture to 224 × 224 size;

s22, roll base layer feature extraction: obtaining 512 feature maps with the size of 14 x 14 through 16 convolution layers and 5 times of downsampling; then, each feature map is processed as follows: sliding convolution kernels with the size of 3 x 3 on the characteristic diagram, and setting an anchoring (anchor) mechanism, namely taking the center of each convolution kernel as a datum point respectively, and then selecting 3 different area sizes and 3 different size proportions around the datum point to generate 9 candidate regions;

s23, removing the candidate frames which are mapped to the original image and exceed the boundary of the original image from the candidate areas;

s24, mapping the obtained candidate region onto a feature map, sampling the region of interest to obtain a 512-dimensional feature vector, performing classification regression and position regression on the 512-dimensional feature vector respectively, and calculating by combining the size of the original image generated in the step S1 to obtain the pixel position of the insulator on the image and the confidence degree of the insulator;

and S25, cutting the obtained insulator pixel positions, and intercepting an individual insulator string.

Further, the step S3 includes the following sub-steps:

s31, obtaining the original pictures by 15 convolution layers and 5 times of down samplingAnda feature map of size;

s32, comparing the original pictureThe feature map of the size is up-sampled and amplified by 2 times and then is compared with the original imageAccumulating the size characteristic graphs;

s33, the characteristic diagram obtained by accumulating S32 is up sampled and amplified by 2 times, and then the characteristic diagram and the original drawing are addedAccumulating the size characteristic graphs;

s34, up-sampling and amplifying the feature map obtained by accumulating in the S33 by 8 times, and finally restoring the obtained feature map to the size of the input image;

and S35, performing pixel prediction on each pixel in the picture obtained in the S34 through a classification layer, outputting a prediction result, and obtaining a label value of each pixel.

Further, the step S4 includes the following sub-steps:

s41, calculating a unitary energy item according to the prediction result output in the step S35:

wherein,representing a unary energy term, x_iRepresents the label value, P (x), at pixel i_i) Represents the probability of the label at pixel i;

s42, calculating a binary energy term:

wherein,representing a binary energy term; when i ≠ j, μ (x)_i,x_j) 1, otherwise μ (x)_i,x_j) 0; that is, when the labels are different, there is a penalty; sigma_α、σ_β、σ_γThe width parameter of the Gaussian kernel function is used for controlling the radial action range of the function; p is a radical of_i、p_jRespectively representing the position coordinates of pixels i and j; i is_i、I_jRespectively representing the RGB values of pixels i and j; omega₁、ω₂Is a linear weight;

s43, in the fully-connected conditional random field model, the energy of the label x is expressed as:

and S44, finding the most possible segmentation by minimizing energy to obtain a semantic segmentation refinement result.

Further, the step S5 includes the following sub-steps:

s51, graying and binarizing the image;

s52, carrying out edge detection on the image;

and S53, detecting the insulator in the image by adopting a Hough transform ellipse recognition method, and filtering noise points in the image.

Further, the step S6 includes the following sub-steps:

s61, inputting the de-noised pictures obtained in the step S5 into a classification network, and normalizing the pictures into a size of 224 × 224;

s62, outputting a characteristic diagram of 1 x 1 through the convolution layer, the activation function and the down-sampling layer;

and S63, classifying the feature maps through the classification layer, outputting classification probability and judging the state of the insulator.

The invention has the beneficial effects that: the invention provides an insulator state recognition method by using an improved fully-connected neural network and a classification network, and designs an insulator burst detection system based on a deep neural network. Firstly, collecting an insulator picture of a power transmission line by using an unmanned aerial vehicle; then, extracting a target of the insulator by adopting a full convolution neural network, and refining an extraction result by using a full connection condition random field to refine edges; then filtering noise points in the picture by adopting a morphological operation method; and finally, designing a classification network to judge whether the insulator is in good condition or burst. The method adopts a mode of combining an improved full convolution neural network, a morphological denoising network and a classification network, and can effectively identify the state of the insulator of the power transmission line and judge whether the insulator bursts or not by training and optimizing parameters of the marked insulator picture. The subjective influence of artificially setting a threshold value and the randomness of artificially extracting characteristics in the traditional insulator state identification are avoided, the line inspection efficiency can be greatly improved, and the line inspection difficulty is reduced.

Drawings

Fig. 1 is a flowchart of a transmission line insulator state identification method of the present invention;

FIG. 2 is an insulator picture acquired by an unmanned aerial vehicle according to an embodiment of the present invention;

FIG. 3 is an insulator image obtained after denoising according to the embodiment of the present invention;

FIG. 4 is a graph of test accuracy for an embodiment of the present invention.

Detailed Description

The technical scheme of the invention is further explained by combining the drawings and the specific embodiment.

As shown in fig. 1, the method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network comprises the following steps:

s1, collecting the insulator pictures of the power transmission line by an unmanned aerial vehicle, preprocessing the collected pictures, and adjusting the pictures to be the same size, wherein the resolution of the preprocessed pictures is 3936 x 2624;

s2, performing classification regression and position regression on the picture through a target detection network Faster R-CNN to obtain the pixel position of the insulator on the picture and the confidence coefficient of the pixel position; then, cutting the obtained insulator pixel position, and intercepting an independent insulator picture; the method comprises the following substeps:

s21, normalizing the picture to 224 × 224 size, and calling GPU to accelerate calculation;

s22, roll base layer feature extraction: obtaining 512 feature maps with the size of 14 x 14 through 16 convolution layers and 5 times of downsampling; then, each feature map is processed as follows: sliding convolution kernels with the size of 3 x 3 on the feature map, setting an anchoring (anchor) mechanism, namely taking the center of each convolution kernel as a datum point, and then selecting 3 different area sizes (128, 256 and 512 corresponding to 3, 6 and 12 respectively of the feature map) and 3 different size proportions (1:1, 1:2 and 2:1) around the datum point to generate 9 candidate regions;

s23, removing the candidate frames which are mapped to the original image and exceed the boundary of the original image from the candidate areas;

s24, mapping the obtained candidate region onto a feature map, sampling the region of interest to obtain a 512-dimensional feature vector, performing classification regression and position regression on the 512-dimensional feature vector respectively, and calculating by combining the size of the original image generated in the step S1 to obtain the pixel position of the insulator on the image and the confidence degree of the insulator;

and S25, cutting the obtained insulator pixel positions, and intercepting an individual insulator string.

S3, semantic segmentation is carried out on the insulator picture through a full convolution neural network, pixel-level classification is carried out on the picture, and insulators are extracted from the background; the method comprises the following substeps:

s31, obtaining the original pictures by 15 convolution layers and 5 times of down samplingAnda feature map of size;

s32, comparing the original pictureThe feature map of the size is up-sampled and amplified by 2 times and then is compared with the original imageAccumulating the size characteristic graphs;

s33, the characteristic diagram obtained by accumulating S32 is up sampled and amplified by 2 times, and then the characteristic diagram and the original drawing are addedAccumulating the size characteristic graphs;

s34, up-sampling and amplifying the feature map obtained by accumulating in the S33 by 8 times, and finally restoring the obtained feature map to the size of the input image;

and S35, performing pixel prediction on each pixel in the picture obtained in the S34 through a classification layer, outputting a prediction result, and obtaining a label value of each pixel.

S4, performing refined segmentation on the semantic segmentation result obtained in the S3 through a full-connection conditional random field, and enhancing edge constraint; the method comprises the following substeps:

s41, calculating a unitary energy item according to the prediction result output in the step S35:

wherein,representing a unary energy term, x_iRepresents the label value, P (x), at pixel i_i) Represents the probability of the label at pixel i;

s42, calculating a binary energy term:

wherein,representing a binary energy term; when i ≠ j, μ (x)_i,x_j) 1, otherwise μ (x)_i,x_j) 0; that is, when the labels are different, there is a penalty; the remaining expressions are two Gaussian kernel functions in different feature spaces, the first one is based on a bilateral Gaussian function and based on a pixel position p and an RGB value I, pixels with similar RGB and positions are forced to be divided into similar labels, and the second one only considers the pixel position and is equal to applying a smoothing term; sigma_α、σ_β、σ_γThe width parameter of the Gaussian kernel function is used for controlling the radial action range of the function; p is a radical of_i、p_jRespectively representing pixelsi. The position coordinates of j; i is_i、I_jRespectively representing the RGB values of pixels i and j; omega₁、ω₂Is a linear weight;

s43, in the fully-connected conditional random field model, the energy of the label x is expressed as:

and S44, finding the most possible segmentation by minimizing energy to obtain a semantic segmentation refinement result.

S5, filtering noise points in the image by using a morphological operation method; the method comprises the following substeps:

s51, graying and binarizing the image;

s52, carrying out edge detection on the image;

and S53, detecting the insulator in the image by adopting a Hough transform ellipse recognition method, and filtering noise points in the image.

S6, classifying the insulators through a deep learning classification network, and judging the state of the insulators; the method comprises the following substeps:

s61, inputting the de-noised pictures obtained in the step S5 into a classification network, and normalizing the pictures into a size of 224 × 224;

s62, outputting a characteristic diagram of 1 x 1 through the convolution layer, the activation function and the down-sampling layer;

and S63, classifying the feature maps through the classification layer, outputting classification probability and judging the state of the insulator.

The technical scheme of the invention is further illustrated by the following specific examples.

The total number of the insulator pictures of the power transmission line collected in the embodiment is 3980, and the pixel size of the pictures is 3936 × 2624. Selecting 2900 target detection training samples as Faster R-CNN at random; 450 insulator pictures with the pixel size of 500 × 500 are intercepted, and semantically segmented samples are manufactured; designing a target detection model FasterR-CNN based on a deep convolutional network VGG16-Net, using 2900 selected pictures as training samples to train the model, and iterating 60000 times in total; designing a semantic segmentation model FCN-8s based on a full convolution neural network, and iterating 40000 times in total to obtain the semantic segmentation model; and designing a classification model GoogLeNet based on the deep convolutional network, and iterating for 20000 times in total to obtain the classification model for judging the state of the insulator.

And (3) a testing stage:

1) and (3) taking 500 pictures in total of the test samples, firstly, taking in a target detection model to obtain the positions of the sub-targets of the insulator: outputting the target frame with the confidence coefficient larger than 0.9, as shown in fig. 2, obtaining two boxes with insulators in the diagram, wherein the confidence coefficients of the two insulator sub target frames are 0.996 and 0.999 respectively, and then cutting out the insulator string according to the position of the boxes.

2) Inputting the insulator picture into a semantic segmentation model to obtain an insulator segmented picture;

3) then, performing edge finishing on the segmentation result through a full-connection conditional random field;

4) denoising the picture by using an ellipse recognition method of Hough transform, wherein the obtained result is shown in FIG. 3, and FIGS. 3(a) and 3(b) are images obtained by performing noise filtering on two insulator strings in FIG. 2 respectively;

5) the total number of classified samples is 735, wherein 397 defective samples and 338 non-defective samples are input into the classification network for classification, and the final classification accuracy is 100%, as shown in fig. 4. In the figure, insulator represents an insulator, mask represents a frame on which the present embodiment depends, and wj represents a user name of a computer used in the present embodiment.

It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Those skilled in the art can make various other specific changes and combinations based on the teachings of the present invention without departing from the spirit of the invention, and these changes and combinations are within the scope of the invention.

Claims (7)

1. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network is characterized by comprising the following steps of:

s1, collecting the insulator pictures of the power transmission line by an unmanned aerial vehicle, preprocessing the collected pictures, and adjusting the pictures to be the same size;

s2, performing classification regression and position regression on the picture through a target detection network Faster R-CNN to obtain the pixel position of the insulator on the picture and the confidence coefficient of the pixel position; then, cutting the obtained insulator pixel position, and intercepting an independent insulator picture;

s3, performing semantic segmentation on the insulator picture through a full convolution neural network;

s4, performing refined segmentation on the semantic segmentation result obtained in the S3 through a full-connection conditional random field, and enhancing edge constraint;

s5, filtering noise points in the image by using a morphological operation method;

and S6, classifying the insulators through a deep learning classification network, and judging the state of the insulators.

2. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network as claimed in claim 1, wherein the resolution of the picture obtained after preprocessing in the step S1 is 3936 × 2624.

3. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network of claim 1, wherein the step S2 includes the following substeps:

s21, normalizing the picture to 224 × 224 size;

s22, roll base layer feature extraction: obtaining 512 feature maps with the size of 14 x 14 through 16 convolution layers and 5 times of downsampling; then, each feature map is processed as follows: sliding convolution kernels with the sizes of 3 x 3 on the feature map, respectively taking the center of each convolution kernel as a datum point, and then selecting 3 different area sizes and 3 different size proportions around the datum point to generate 9 candidate regions;

s23, removing the candidate frames which are mapped to the original image and exceed the boundary of the original image from the candidate areas;

s24, mapping the obtained candidate region onto a feature map, sampling the region of interest to obtain a 512-dimensional feature vector, performing classification regression and position regression on the 512-dimensional feature vector respectively, and calculating by combining the size of the original image generated in the step S1 to obtain the pixel position of the insulator on the image and the confidence degree of the insulator;

and S25, cutting the obtained insulator pixel positions, and intercepting an individual insulator string.

4. The method for identifying the state of the insulator of the power transmission line based on the improved fully convolutional neural network as claimed in claim 3, wherein the step S3 comprises the following substeps:

s31, obtaining the original pictures by 15 convolution layers and 5 times of down samplingAnda feature map of size;

s32, comparing the original pictureThe feature map of the size is up-sampled and amplified by 2 times and then is compared with the original imageAccumulating the size characteristic graphs;

s33, the characteristic diagram obtained by accumulating S32 is up sampled and amplified by 2 times, and then the characteristic diagram and the original drawing are addedAccumulating the size characteristic graphs;

s34, up-sampling and amplifying the feature map obtained by accumulating in the S33 by 8 times, and finally restoring the obtained feature map to the size of the input image;

and S35, performing pixel prediction on each pixel in the picture obtained in the S34 through a classification layer, outputting a prediction result, and obtaining a label value of each pixel.

5. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network as claimed in claim 4, wherein the step S4 includes the following sub-steps:

s41, calculating a unitary energy item according to the prediction result output in the step S35:

<mrow> <msub> <mi>&amp;theta;</mi> <mrow> <mi>i</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mn>1</mn> <mo>-</mo> <mi>log</mi> <mi>P</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> </mrow>

wherein,representing a unary energy term, x_iRepresents the label value, P (x), at pixel i_i) Represents the probability of the label at pixel i;

s42, calculating a binary energy term:

<mrow> <msub> <mi>&amp;theta;</mi> <mrow> <mi>i</mi> <mi>j</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> </mrow> </msub> <mo>=</mo> <mi>&amp;mu;</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>&amp;lsqb;</mo> <msub> <mi>&amp;omega;</mi> <mn>1</mn> </msub> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <mrow> <mo>|</mo> <mo>|</mo> <msub> <mi>p</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>p</mi> <mi>j</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow> <mrow> <mn>2</mn> <msubsup> <mi>&amp;sigma;</mi> <mi>&amp;alpha;</mi> <mn>2</mn> </msubsup> </mrow> </mfrac> <mo>-</mo> <mfrac> <mrow> <mo>|</mo> <mo>|</mo> <msub> <mi>I</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>I</mi> <mi>j</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow> <mrow> <mn>2</mn> <msubsup> <mi>&amp;sigma;</mi> <mi>&amp;beta;</mi> <mn>2</mn> </msubsup> </mrow> </mfrac> <mo>)</mo> </mrow> <mo>+</mo> <msub> <mi>&amp;omega;</mi> <mn>2</mn> </msub> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <mrow> <mo>|</mo> <mo>|</mo> <msub> <mi>p</mi> <mi>i</mi> </msub> <mo>-</mo> <msub> <mi>p</mi> <mi>j</mi> </msub> <mo>|</mo> <msup> <mo>|</mo> <mn>2</mn> </msup> </mrow> <mrow> <mn>2</mn> <msubsup> <mi>&amp;sigma;</mi> <mi>&amp;gamma;</mi> <mn>2</mn> </msubsup> </mrow> </mfrac> <mo>)</mo> </mrow> <mo>&amp;rsqb;</mo> </mrow>

wherein,representing a binary energy term; when i ≠ j, μ (x)_i,x_j) 1, otherwise μ (x)_i,x_j)＝0；σ_α、σ_β、σ_γIs the width parameter of the Gaussian kernel function; p is a radical of_i、p_jRespectively representing the position coordinates of pixels i and j; i is_i、I_jRespectively representing the RGB values of pixels i and j; omega₁、ω₂Is a linear weight;

s43, in the fully-connected conditional random field model, the energy of the label x is expressed as:

<mrow> <mi>E</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mo>&amp;Sigma;</mo> <mi>i</mi> </munder> <mi>&amp;theta;</mi> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>)</mo> </mrow> <mo>+</mo> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </munder> <msub> <mi>&amp;theta;</mi> <mrow> <mi>i</mi> <mi>j</mi> </mrow> </msub> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>x</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>;</mo> </mrow>

and S44, obtaining a semantic segmentation refinement result by minimizing energy.

6. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network of claim 5, wherein the step S5 includes the following sub-steps:

s51, graying and binarizing the image;

s52, carrying out edge detection on the image;

and S53, detecting the insulator in the image by adopting a Hough transform ellipse recognition method, and filtering noise points in the image.

7. The method for identifying the state of the insulator of the power transmission line based on the improved full convolution neural network as claimed in claim 6, wherein the step S6 includes the following sub-steps:

s61, inputting the de-noised pictures obtained in the step S5 into a classification network, and normalizing the pictures into a size of 224 × 224;

s62, outputting a characteristic diagram of 1 x 1 through the convolution layer, the activation function and the down-sampling layer;

and S63, classifying the feature maps through the classification layer, outputting classification probability and judging the state of the insulator.