CN111784633A - Insulator defect automatic detection algorithm for power inspection video - Google Patents

Abstract

The invention discloses an automatic insulator defect detection algorithm for power inspection videos, which comprises the following steps: step 1, classifying and enhancing sample data of an insulator of an original electric power tower; step 2, building and generating a convolutional neural network model, and training the network model through the data enhanced in the step 1; step 3, processing the image to be detected, extracting HOG characteristics from the picture passing through the network model through the established network model, and determining the approximate position of the insulator in the picture; step 4, after the approximate position of the insulator in the picture is determined in the step 3, detecting whether the insulator is defective or not by a CNN/LSTM deep learning method; the method disclosed by the invention overcomes the defects of the existing method for detecting the insulator defect of the power tower, uses a deep learning method and avoids the influence of other environmental factors in a sample. And the image is further processed and analyzed by combining the image characteristics, so that the detection process is more accurate and efficient.

Description

Insulator defect automatic detection algorithm for power inspection video

Technical Field

The invention belongs to the technical field of image processing, machine vision and artificial intelligence, and relates to an insulator defect automatic detection algorithm for power inspection videos.

Background

On the power transmission line, the safety of the power tower and the line is ensured to be correct, the insulator is important for timely finding and processing problems in the overhead power transmission line, due to the influence of environment and power change, the insulator can cause pollution and defect of different degrees, the service life of the power transmission line can be seriously damaged, even equipment can be burnt to cause power failure, serious accidents of power production are caused, in order to avoid insulator faults, the number of times of maintenance can only be increased in the power production process, the manual inspection efficiency is low, the reliability is poor, the cost is high, the automatic inspection means for the insulator faults is not available at present, and related algorithms and systems still have the defects in the aspects of precision and efficiency.

At present, the detection of insulators in the power industry in related fields at home and abroad is roughly divided into an electric quantity measurement method and a non-electric quantity measurement method; the electric quantity measuring method analyzes the running state of the insulator in real time by acquiring current and voltage information; the method adopts a three-layer BP neural network to identify a target, and adopts a scanning analysis insulator image window longitudinal sectioning mode to judge the state of the insulator;

in recent years, the increase of power transmission lines leads to increasingly complex line operation and maintenance environments, the application of the unmanned aerial vehicle in power inspection makes up the deficiency of a manual inspection mode, and an automatic and high-precision detection means of a power tower insulator aiming at data inspection of the unmanned aerial vehicle is an important direction for technical development in the field.

Disclosure of Invention

The invention aims to provide an insulator defect automatic detection algorithm for power inspection videos, which can automatically judge whether insulator defects exist or not from power inspection video data.

The invention adopts an automatic insulator defect detection algorithm for power inspection video, which is implemented by the following steps:

step 1, classifying and enhancing sample data of an insulator of an original electric power tower;

step 2, building and generating a convolutional neural network model, training the network model through the data enhanced in the step 1, and classifying through the convolutional neural network model;

step 3, processing the data classified in the step 2, extracting HOG characteristics from the picture passing through the network model through the established network model, and determining the approximate position of the insulator in the picture;

and 4, after the approximate position of the insulator in the picture is determined in the step 3, detecting whether the insulator is defective or not by a CNN/LSTM deep learning method.

The invention is also characterized in that:

the specific content of the step 1 comprises:

firstly, extracting sample data of an insulator of an existing power tower to obtain an original sample image, dividing the data into two types of data including the insulator and data without the insulator, and dividing the two types of data into test set data and training set data;

then calling an image processing library in a python environment, defining enhancement factors, and performing data enhancement on the original image;

the step 2 specifically includes screening and classifying data through a convolutional neural network, and eliminating invalid pictures without insulators, and the specific steps are as follows:

step 2.1, building a convolutional neural network model by using Tensorflow, and taking the 3-channel image data obtained in the step 1, namely a training data set, as input for network model training;

the method comprises the steps that the first layer of convolution layer is provided, input data pass through 16 convolution kernels, the size of each convolution kernel is 3 x 3, the step size is 1, and padding is set as same;

and the second layer of pooling layer max pool, the pooling window is 2 x 2, the step size is 2, padding is set as same, and the data is sampled to one half of the original data.

Extracting image features through multiple convolutions, increasing deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;

the definition loss function of the network model is shown in equation (1):

loss＝-[plog(t)+(1-p)log(1-t)](1)

dividing the output result into two types, wherein p is the expected classification probability of the image, and t is the actual classification probability of the image; the full connection layer converts the obtained data features through feature weighting to realize data one-dimension, then softmax operation is carried out on the result to carry out normalization processing, and the judgment result of the image is output by combining a cross entropy formula;

2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, indicating that the model gradually converges, using the test set data as the input for generating the network, entering the trained network model, and checking the accuracy of output;

the specific content in the step 3 is as follows:

step 3.1, processing the test set data and extracting HOG characteristics:

carrying out Gaussian filtering processing on the test set data image, then carrying out down-sampling decomposition on the image subjected to smoothing processing, respectively carrying out 1/4 down-sampling on the size of the image to obtain a new image, carrying out continuous down-sampling to obtain thumbnails of different sizes, finally forming a three-layer pyramid, and gradually reducing the size of the image from the bottom layer to the top layer, and gradually reducing the resolution of the image to obtain a Gaussian pyramid model; then, respectively extracting HOG features from each layer of image of the obtained Gaussian pyramid model to form a final HOG feature vector of the image;

step 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1;

the specific content of the HOG feature extraction in the step 3 comprises the following steps:

step 3.1.1, carrying out color space standardization on the Gaussian pyramid model by adopting a Gamma correction method;

step 3.1.2, calculating the gradient of each pixel of the image:

the horizontal direction and vertical direction gradients of pixel points (x, y) in the image are respectively shown as formula (2) and formula (3):

G_X(x,y)＝H(x+1,y)-H(x-1,y) (2)

G_y(x,y)＝H(x,y+1)-H(x,y-1) (3)；

after the horizontal gradient and the vertical gradient are obtained, the gradient amplitude and the gradient direction corresponding to the pixel point are obtained, and are respectively shown as a formula (4) and a formula (5):

step 3.1.3, dividing the gray level image into small cells which cannot slide;

step 3.1.4, dividing the pixel gradient direction into regions by using angles, and counting the gradient histogram of each cell to form a descriptor of each cell;

step 3.1.5, forming each plurality of cells into a block, and connecting the feature descriptors of all the cells in the block in series to obtain the HOG feature descriptors of the block;

step 3.1.6, connecting HOG feature descriptors of all blocks in the image in series to obtain the HOG feature of the image;

the specific operation contents for determining the approximate position of the insulator in the picture by using the extracted HOG features in the step 3 are as follows:

when the block is in the sliding calculation process in the image, the current gradient amplitude value and the gradient histogram are similar to the value of the adjacent block, and the region is judged to contain the insulator; judging the characteristic gradient histogram of the whole picture to obtain ROI regional distribution of the picture;

the specific content of the step 4 comprises:

establishing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate;

there are 8 sets of parameters to be learned by LSTM, which are: weight matrix W of forgetting gate_fAnd bias term b_fWeight matrix W of input gates_iAnd bias term b_iWeight matrix W of output gates_oAnd bias term b_oAnd computing a weight matrix W of cell states_cAnd bias term b_c；

The forgetting gate and the input gate are shown in formula (6) and formula (7), respectively:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (6)

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (7)

in the formula W_fIs the weight matrix of the forgetting gate, [ h ]_t-1,x_t]Representing the concatenation of two vectors into a longer vector, b_fIs the bias term of the forgetting gate, sigma is sigmoid function;

the output gate is shown in equation (8):

o_t＝σ(W_o·[h_t-1,x_t]+b_o) (8)；

selecting a result of forgetting the previous moment through a forgetting gate, obtaining the output of the cell at the current moment by combining the input of the moment, judging the results through an output gate, taking the judgment result of the output gate as the input of a convolutional neural network, screening and extracting the image characteristics of the ROI region once again through an LSTM network, judging the defect condition of the insulator, and finally classifying the obtained judgment result in a classification convolutional neural network;

the training data of the classified convolutional neural network is divided into two types, pictures of intact insulators and defective insulators are stored respectively, the two types of data are divided into a training set and a verification set, and then the training set is input into the convolutional neural network for training;

the process of inputting the convolutional neural network for training is as follows:

the first layer of convolution layer, input data passes through 5 convolution kernels, the size of the convolution kernels is 3 x 5, and the step size is 1; a second pooling layer max pool with a pooling window of 2 x 2, step size of 2, down-sampling the data; the third layer of convolution layers passes through 5 convolution kernels, the size of each convolution kernel is 3 x 5, and the step length is 1; the max pool of the fourth pooling layer, the pooling window is 2 x 2, and the step length is 2;

increasing deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy as a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained from the fourth layer are subjected to one-dimensional conversion, the data obtained from the fourth layer are subjected to feature transformation through feature weighting, and then softmax normalization is performed to obtain a final judgment result:

wherein the loss function is defined as shown in equation (9):

loss＝loss_MES+loss_CNN(9)

where loss is the loss of content, including the MSE loss of the LSTM network_MESLoss of decision for convolutional networks_CNNAs shown in equation (10) and equation (11), respectively:

in the formula (10), H and W represent the width and height of an image, I_x,yRepresenting the x, y position, I, of the real image_x,y' represents the x, y position of the output image to be passed through the network module; in equation (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network.

And according to the output result, obtaining the corresponding position in the video frame sequence, thereby outputting the final defect detection result.

The invention has the beneficial effects that:

aiming at the limitation and the deficiency of the existing electric power tower insulator defect detection method, the invention uses the deep learning method to improve the precision of insulator defect detection and effectively solves the problem of the non-robust detection algorithm caused by the insufficient number of training samples of the deep learning. Due to the fact that the deep learning network model with the memory capacity is adopted, the algorithm provided by the invention can extract insulator defect information recorded by multi-frame image data in a video, and therefore the detection accuracy is higher than that of an existing detection algorithm based on single-frame image data. Due to the fact that the pre-screening process of the power inspection video data is introduced, the method has low calculation complexity and high detection efficiency, and is more suitable for application scenes of processing a large amount of inspection video data in the power inspection process.

Drawings

FIG. 1 is a CNN network structure table diagram in an insulator defect automatic detection algorithm for power inspection video according to the invention;

FIG. 2 is a diagram of an overall network model structure in an insulator defect automatic detection algorithm for power inspection video according to the present invention;

FIG. 3 is a basic structure diagram of an LSTM network in the algorithm for automatically detecting the defect of the insulator facing to the power inspection video;

FIG. 4 is a flowchart of the HOG feature positioning insulator in the insulator defect automatic detection algorithm for power inspection video according to the present invention;

fig. 5 is a flow chart of defect detection in the automatic insulator defect detection algorithm for the power inspection video.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The invention provides an automatic insulator defect detection algorithm for power inspection videos, which is implemented by the following steps as shown in fig. 2 and 5:

step 1, classifying and enhancing original electric power tower insulator sample data:

extracting the sample data of the existing electric power tower insulator to obtain an original sample image, and dividing the data into two categories: the method comprises the following steps of (1) containing insulators and not containing insulators, and dividing the insulators into test set data and training set data;

calling an image processing library in a python environment, defining enhancement factors, and performing data enhancement on an original image: data inversion, data rotation, data amplification, data clipping, data translation, noise disturbance and the like, and a plurality of samples different from original data can be obtained through data enhancement, so that the data diversity is improved. The increase of the number of data also enhances the accuracy of the training model, and overcomes the limitation of a small data set on a certain condition;

step 2, building and generating a convolutional neural network model, and training the network model through the data enhanced in the step 1:

step 2, screening the data through a convolutional neural network, and eliminating invalid pictures without insulators, wherein the specific process is as follows:

step 2.1, building a convolutional neural network model by using Tensorflow, wherein a basic structure of a feature extraction network of the convolutional neural network model is composed of a series of convolutional units and pooling units, 3-channel image data obtained in the step 1, namely a training data set, is used as input for network model training, a convolutional structure table is shown in FIG. 1, H, W and C represent the size and the number of channels of an input image, and finally S is an output judgment result;

the method comprises the steps that the first layer of convolution layer is provided, input data pass through 16 convolution kernels, the size of each convolution kernel is 3 x 3, the step size is 1, and padding is set as same;

and the second layer of pooling layer max pool, the pooling window is 2 x 2, the step size is 2, padding is set as same, and the data is sampled to one half of the original data.

Extracting image features through multiple convolutions, increasing deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;

the definition loss function of the network model is shown in equation (1):

loss＝-[plog(t)+(1-p)log(1-t)](1)

because the network is used for eliminating irrelevant background pictures without insulators, the output result only needs to be divided into two types, p is the expected classification probability of the image, and t is the actual classification probability of the image; the full connection layer converts the obtained data features through feature weighting to realize data one-dimension, then softmax operation is carried out on the result to carry out normalization processing, and the judgment result of the image is output by combining a cross entropy formula;

2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, indicating that the model gradually converges, using the test set data as the input for generating the network, entering the trained network model, and checking the accuracy of output;

step 3, processing the test set data obtained by classification in the step 1, extracting HOG characteristics from the picture passing through the network model through the established network model, and determining the approximate position of the insulator in the picture:

step 3.1, as shown in fig. 4, processing the picture to be detected and extracting HOG features:

carrying out Gaussian filtering processing on an image to be detected, then carrying out down-sampling decomposition on the image after smoothing processing, respectively carrying out 1/4 down-sampling on the size of the image to obtain a new image, continuously carrying out down-sampling to obtain thumbnails with different sizes, finally forming a three-layer pyramid, and gradually reducing the size of the image from the bottom layer to the top layer, and gradually reducing the resolution of the image to obtain a Gaussian pyramid model; then, respectively extracting HOG features from each layer of image of the obtained Gaussian pyramid model to form a final HOG feature vector of the image;

the specific process for extracting the HOG features comprises the following steps:

step 3.1.1, the color space of the Gaussian pyramid model image is standardized by adopting a Gamma correction method, the contrast of the image can be adjusted, the influence caused by local shadow and illumination change of the image can be effectively reduced, and meanwhile, the interference of noise can be inhibited;

step 3.1.2, calculating the gradient (including the size and the direction) of each pixel of the image, capturing contour information through gradient information, and further weakening the interference of illumination:

the horizontal direction and vertical direction gradients of pixel points (x, y) in the image are respectively shown as formula (2) and formula (3):

G_X(x,y)＝H(x+1,y)-H(x-1,y) (2)

G_y(x,y)＝H(x,y+1)-H(x,y-1) (3)；

after the horizontal gradient and the vertical gradient are obtained, the gradient amplitude and the gradient direction corresponding to the pixel point are obtained, and are respectively shown as a formula (4) and a formula (5):

step 3.1.3, dividing the gray level image into small cells which cannot slide;

step 3.1.4, dividing the gradient direction of the pixel into regions by using angles, and counting a gradient histogram (the number of different gradients in a certain region) of each cell to form a descriptor of each cell;

step 3.1.5, forming each plurality of cells into a block, and connecting the feature descriptors of all the cells in the block in series to obtain the HOG feature descriptors of the block;

step 3.1.6, connecting HOG characteristic descriptors of all blocks in the image in series to obtain the information of the HOG characteristic, the gradient image, the gradient amplitude and the like of the image;

step 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1: after the relevant information of the picture to be detected is obtained, the gradient histogram of the insulator at the edge part with larger change shows a special distribution in a certain gradient direction. Considering that the difference between the gradient histogram of the insulator and the background part in the picture is large, and because the insulators are basically arranged periodically, the difference between the gradient amplitude values of the insulators is small, and therefore when the block calculates the sliding in the image, the current gradient amplitude value is similar to the value of the gradient histogram and the value of the adjacent block, the region is judged to contain the insulators; judging the characteristic gradient histogram of the whole picture to obtain ROI regional distribution of the picture;

calculating gradient positioning, performing frequency domain transformation on the positioned area, and rotating according to the detected main direction and the main direction, so that insulators in all pictures are adjusted to be in one direction, and the accuracy of processing and judging by a neural network is improved;

step 4, after the approximate position of the insulator in the picture is determined in the step 3, detecting whether the insulator is defective or not by a CNN/LSTM deep learning method:

establishing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate; as shown in fig. 3, the forgetting gate is responsible for deciding how many unit states from the previous time to the current time are reserved; the input gate is responsible for determining how many unit states input to the current time are reserved at the current time; the output gate is responsible for determining the output of the unit state at the current moment;

there are 8 sets of parameters to be learned by LSTM, which are: weight matrix W of forgetting gate_fAnd bias term b_fWeight matrix W of input gates_iAnd bias term b_iWeight matrix W of output gates_oAnd bias term b_oAnd computing a weight matrix W of cell states_cAnd bias term b_c；

The forgetting gate and the input gate are shown in formula (6) and formula (7), respectively:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (6)

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (7)

in the formula W_fIs the weight matrix of the forgetting gate, [ h ]_t-1,x_t]Representing the concatenation of two vectors into a longer vector, b_fIs the bias term of the forgetting gate, sigma is sigmoid function;

the output gate is shown in equation (8):

o_t＝σ(W_o·[h_t-1,x_t]+b_o)(8)；

selecting a result of forgetting the previous moment through a forgetting gate, obtaining the output of the cell at the current moment by combining the input of the moment, judging the results through an output gate, and determining the final output of the LSTM network by the output gate and the state of the cell; taking the result of the output gate as the input of a convolutional neural network, screening the image with the determined ROI area through an LSTM network again to extract picture characteristics once, and then transmitting the obtained result into the convolutional neural network to judge the defect condition of the insulator;

the training data of the convolutional neural network is divided into two types, pictures of intact insulators and defective insulators are stored respectively, the two types of data are divided into a training set and a verification set, and then the training set is input into the convolutional neural network for training;

the process of inputting the convolutional neural network for training is as follows:

the first layer of convolution layer, input data passes through 5 convolution kernels, the size of the convolution kernels is 3 x 5, and the step size is 1; a second pooling layer max pool with a pooling window of 2 x 2, step size of 2, down-sampling the data; the third layer of convolution layers passes through 5 convolution kernels, the size of each convolution kernel is 3 x 5, and the step length is 1; the max pool of the fourth pooling layer, the pooling window is 2 x 2, and the step length is 2;

increasing deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy as a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained from the fourth layer are subjected to one-dimensional conversion, the data obtained from the fourth layer are subjected to feature transformation through feature weighting, and then softmax normalization is performed to obtain a final judgment result:

wherein the loss function is defined as shown in equation (9):

loss＝loss_MES+loss_CNN(9)

where loss is the loss of content, including the MSE loss of the LSTM network_MESLoss of decision for convolutional networks_CNNAs shown in equation (10) and equation (11), respectively:

in the formula (10), H and W represent the width and height of an image, I_x,yRepresenting the x, y position, I, of the real image_x,y' represents the x, y position of the output image to be passed through the network module; in equation (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network.

And according to the output result, obtaining the corresponding position in the video frame sequence, thereby outputting the final defect detection result.

The invention relates to an automatic insulator defect detection algorithm for power inspection videos, which comprises the following specific processes:

firstly, classifying through CNN, and analyzing whether the image has a classification network of insulators or not;

then, extracting the features of the Gaussian pyramid HOG, namely, carrying out ROI segmentation on the insulator part of the picture;

then, an LSTM detection neural network is adopted to detect and judge whether the insulator is damaged (whether the insulator is damaged is judged by a plurality of continuous characteristic images), and the LSTM is input into a plurality of frames of insulator characteristic images (which can be understood as a small segment of short characteristic video) processed by HOG, so that the insulator damage detection is a multi-frame characteristic detection method utilizing time domain and space domain correlation, and is particularly suitable for processing inspection videos;

the ordered arrangement mode of the three steps ensures that the method is suitable for inspecting videos instead of image data, thereby improving the efficiency and increasing the precision; because the LSTM detects the probability after the neural network judges, so the invention uses the first classification convolution neural network to classify finally, and obtains the exact defect position, because the CNN and the LSTM input differently, the two networks firstly train independently, and then cascade the two networks to train, thus improving the convergence and precision of each network and the final network.

Claims (8)

1. The utility model provides a video insulator defect automatic detection algorithm patrols and examines to electric power which characterized in that specifically implements according to following step:

step 1, classifying and enhancing sample data of an insulator of an original electric power tower;

step 2, building and generating a convolutional neural network model, training the network model through the data enhanced in the step 1, and classifying through the convolutional neural network model;

step 3, processing the data classified in the step 2, extracting HOG characteristics from the picture passing through the network model through the established network model, and determining the approximate position of the insulator in the picture;

and 4, after the approximate position of the insulator in the picture is determined in the step 3, detecting whether the insulator is defective or not by a CNN/LSTM deep learning method.

2. The algorithm for automatically detecting insulator defects for power inspection videos according to claim 1, wherein the specific content of the step 1 includes:

firstly, extracting sample data of an insulator of an existing power tower to obtain an original sample image, dividing the data into two types of data including the insulator and data without the insulator, and dividing the two types of data into test set data and training set data;

and then calling an image processing library in a python environment, defining an enhancement factor, and performing data enhancement on the original image.

3. The algorithm for automatically detecting insulator defects for the power inspection video according to claim 1, wherein the step 2 specifically comprises the steps of screening and classifying data through a convolutional neural network, and removing invalid pictures without insulators, and the method specifically comprises the following steps:

step 2.1, building a convolutional neural network model by using Tensorflow, and taking the 3-channel image data obtained in the step 1, namely a training data set, as input for network model training;

the method comprises the steps that the first layer of convolution layer is provided, input data pass through 16 convolution kernels, the size of each convolution kernel is 3 x 3, the step size is 1, and padding is set as same;

and the second layer of pooling layer max pool, the pooling window is 2 x 2, the step size is 2, padding is set as same, and the data is sampled to one half of the original data.

Extracting image features through multiple convolutions, increasing deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;

the definition loss function of the network model is shown in equation (1):

loss＝-[plog(t)+(1-p)log(1-t)](1)

dividing the output result into two types, wherein p is the expected classification probability of the image, and t is the actual classification probability of the image; the full connection layer converts the obtained data features through feature weighting to realize data one-dimension, then softmax operation is carried out on the result to carry out normalization processing, and the judgment result of the image is output by combining a cross entropy formula;

and 2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, indicating that the model gradually converges, using the test set data as the input for generating the network, entering the trained network model, and checking the accuracy of output.

4. The automatic detection algorithm for insulator defect for power inspection video according to claim 1 or 2, characterized in that the specific content in the step 3 is as follows:

step 3.1, processing the test set data and extracting HOG characteristics:

carrying out Gaussian filtering processing on the test set data image, then carrying out down-sampling decomposition on the image subjected to smoothing processing, respectively carrying out 1/4 down-sampling on the size of the image to obtain a new image, carrying out continuous down-sampling to obtain thumbnails of different sizes, finally forming a three-layer pyramid, and gradually reducing the size of the image from the bottom layer to the top layer, and gradually reducing the resolution of the image to obtain a Gaussian pyramid model; then, respectively extracting HOG features from each layer of image of the obtained Gaussian pyramid model to form a final HOG feature vector of the image;

and 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1.

5. The algorithm for automatically detecting insulator defect oriented to the power inspection video according to claim 1, wherein the step 3 of extracting specific content of the HOG feature comprises the following steps:

step 3.1.1, carrying out color space standardization on the Gaussian pyramid model by adopting a Gamma correction method;

step 3.1.2, calculating the gradient of each pixel of the image:

the horizontal direction and vertical direction gradients of pixel points (x, y) in the image are respectively shown as formula (2) and formula (3):

G_X(x,y)＝H(x+1,y)-H(x-1,y) (2)

G_y(x,y)＝H(x,y+1)-H(x,y-1) (3)；

after the horizontal gradient and the vertical gradient are obtained, the gradient amplitude and the gradient direction corresponding to the pixel point are obtained, and are respectively shown as a formula (4) and a formula (5):

step 3.1.3, dividing the gray level image into small cells which cannot slide;

step 3.1.4, dividing the pixel gradient direction into regions by using angles, and counting the gradient histogram of each cell to form a descriptor of each cell;

step 3.1.5, forming each plurality of cells into a block, and connecting the feature descriptors of all the cells in the block in series to obtain the HOG feature descriptors of the block;

and 3.1.6, connecting HOG characteristic descriptors of all blocks in the image in series to obtain the HOG characteristic of the image.

6. The automatic insulator defect detection algorithm for the power inspection video according to claim 1 or 5, wherein the specific operation contents for determining the approximate position of the insulator in the picture by using the extracted HOG features in the step 3 are as follows:

when the block is in the sliding calculation process in the image, the current gradient amplitude value and the gradient histogram are similar to the value of the adjacent block, and the region is judged to contain the insulator; and judging the characteristic gradient histogram of the whole picture to obtain ROI area distribution of the picture.

7. The algorithm for automatically detecting insulator defects for power inspection videos according to claim 1, wherein the specific content of the step 4 includes:

establishing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate;

there are 8 sets of parameters to be learned by LSTM, which are: weight matrix W of forgetting gate_fAnd bias term b_fWeight matrix W of input gates_iAnd bias term b_iWeight matrix W of output gates_oAnd bias term b_oAnd computing a weight matrix W of cell states_cAnd bias term b_c；

The forgetting gate and the input gate are shown in formula (6) and formula (7), respectively:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (6)

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (7)

in the formula W_fIs the weight matrix of the forgetting gate, [ h ]_t-1,x_t]Representing the concatenation of two vectors into a longer vector, b_fIs the bias term of the forgetting gate, sigma is sigmoid function;

the output gate is shown in equation (8):

o_t＝σ(W_o·[h_t-1,x_t]+b_o) (8)；

selecting a result of forgetting the previous moment through a forgetting gate, obtaining the output of the cell at the current moment by combining the input of the moment, judging the results through an output gate, taking the judgment result of the output gate as the input of a convolutional neural network, screening and extracting the image characteristics of the ROI region once again through an LSTM network, judging the defect condition of the insulator, and finally classifying the obtained judgment result in a classification convolutional neural network.

8. The algorithm for automatically detecting insulator defects in power inspection videos according to claim 7, wherein the training data of the classified convolutional neural network are divided into two types, pictures of intact insulators and defective insulators are stored respectively, the two types of data are divided into a training set and a verification set, and then the training set is input into the convolutional neural network for training;

the process of inputting the convolutional neural network for training is as follows:

the first layer of convolution layer, input data passes through 5 convolution kernels, the size of the convolution kernels is 3 x 5, and the step size is 1; a second pooling layer max pool with a pooling window of 2 x 2, step size of 2, down-sampling the data; the third layer of convolution layers passes through 5 convolution kernels, the size of each convolution kernel is 3 x 5, and the step length is 1; the max pool of the fourth pooling layer, the pooling window is 2 x 2, and the step length is 2;

increasing deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy as a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained from the fourth layer are subjected to one-dimensional conversion, the data obtained from the fourth layer are subjected to feature transformation through feature weighting, and then softmax normalization is performed to obtain a final judgment result:

wherein the loss function is defined as shown in equation (9):

loss＝loss_MES+loss_CNN(9)

where loss is the loss of content, including the MSE loss of the LSTM network_MESLoss of decision for convolutional networks_CNNAs shown in equation (10) and equation (11), respectively:

in the formula (10), H and W represent the width and height of an image, I_x,yRepresenting the x, y position, I, of the real image_x,y' represents the x, y position of the output image to be passed through the network module; in equation (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network.

And according to the output result, obtaining the corresponding position in the video frame sequence, thereby outputting the final defect detection result.

Images