CN111784633B - Insulator defect automatic detection algorithm for electric power inspection video - Google Patents
Insulator defect automatic detection algorithm for electric power inspection video Download PDFInfo
- Publication number
- CN111784633B CN111784633B CN202010452724.5A CN202010452724A CN111784633B CN 111784633 B CN111784633 B CN 111784633B CN 202010452724 A CN202010452724 A CN 202010452724A CN 111784633 B CN111784633 B CN 111784633B
- Authority
- CN
- China
- Prior art keywords
- image
- data
- insulator
- layer
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000012212 insulator Substances 0.000 title claims abstract description 89
- 238000001514 detection method Methods 0.000 title claims abstract description 38
- 230000007547 defect Effects 0.000 title claims abstract description 35
- 238000007689 inspection Methods 0.000 title claims abstract description 28
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 47
- 238000012549 training Methods 0.000 claims abstract description 38
- 238000000034 method Methods 0.000 claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 15
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000013135 deep learning Methods 0.000 claims abstract description 8
- 230000002950 deficient Effects 0.000 claims abstract description 8
- 230000002708 enhancing effect Effects 0.000 claims abstract description 4
- 238000011176 pooling Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 12
- 238000012216 screening Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005304 joining Methods 0.000 claims description 3
- 238000003062 neural network model Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 abstract 1
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 229910052573 porcelain Inorganic materials 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Analytical Chemistry (AREA)
- Signal Processing (AREA)
- Quality & Reliability (AREA)
- Chemical & Material Sciences (AREA)
- Biochemistry (AREA)
- Multimedia (AREA)
- Pathology (AREA)
- Immunology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an insulator defect automatic detection algorithm for an electric power inspection video, which comprises the following steps: step 1, classifying and enhancing original electric power tower insulator sample data; step 2, constructing and generating a convolutional neural network model, and training the network model through the data enhanced in the step 1; step 3, processing the image to be detected, extracting HOG characteristics from the picture passing through the network model through the established network model, and determining the approximate position of the insulator in the picture; step 4, after determining the approximate position of the insulator in the picture through the step 3, detecting whether the insulator is defective or not through a CNN/LSTM deep learning method; the method disclosed by the invention solves the defects of the existing power tower insulator defect detection method, adopts a deep learning method, and avoids the influence of other environmental factors in a sample. And the image is further processed and analyzed by combining with the image characteristics, so that the detection process is more accurate and efficient.
Description
Technical Field
The invention belongs to the technical fields of image processing, machine vision and artificial intelligence, and relates to an insulator defect automatic detection algorithm for an electric power inspection video.
Background
On the power transmission line, the safety of a power tower and the line is guaranteed, the timely discovery and treatment of the problems of an insulator in an overhead power transmission line are vital, the insulator can cause dirt and defects of different degrees due to the influence of environment and power change, the service life of the power transmission line can be seriously damaged, power failure can be caused even by burning equipment, serious accidents of power production are caused, in order to avoid the faults of the insulator, the power production process can only increase the times of maintenance, the manual inspection efficiency is low, the reliability is poor, the cost is high, the automatic inspection means for the faults of the insulator are not available at present, and the related algorithm and system still have the defects in precision and efficiency.
At present, the detection of insulators in the power industry in the related fields at home and abroad is roughly divided into an electric quantity measurement method and a non-electric quantity measurement method; the electric quantity measuring method is used for analyzing the running state of the insulator in real time by collecting current and voltage information; the non-electric quantity measuring method comprises an ultrasonic detection method, a laser Doppler vibration method, non-contact detection of infrared thermal imaging and the like, and an insulator damage condition visual detection method based on local area characteristics of a porcelain insulator is proposed in 2005, and the method adopts a three-layer BP neural network to identify a target, and adopts a scanning analysis insulator sub-image window longitudinal line cutting mode to judge the state of the insulator;
in recent years, the increase of transmission lines causes the line operation and maintenance environment to be more and more complex, the application of unmanned aerial vehicles in electric power inspection makes up the defect of a manual inspection mode, and an automatic and high-precision detection means for an electric power tower insulator of unmanned aerial vehicle inspection data is an important direction of technical development in the field.
Disclosure of Invention
The invention aims to provide an automatic insulator defect detection algorithm for an electric power inspection video, which can realize automatic judging whether insulator defects exist in electric power inspection video data.
The invention adopts an insulator defect automatic detection algorithm for an electric power inspection video, which is implemented by the following steps:
step 1, classifying and enhancing original electric power tower insulator sample data;
step 2, constructing and generating a convolutional neural network model, training the network model through the data enhanced in the step 1, and classifying through the convolutional neural network model;
step 3, processing the data classified in the step 2, extracting HOG characteristics from the pictures passing through the network model through the established network model, and determining the approximate position of the insulator in the pictures;
and 4, determining the approximate position of the insulator in the picture through the step 3, and detecting whether the insulator is defective or not through a CNN/LSTM deep learning method.
The invention is also characterized in that:
the specific content of the step 1 comprises the following steps:
firstly, extracting sample data of an existing electric power tower insulator to obtain an original sample image, dividing the data into two types of data including the insulator and the insulator-free data, and dividing the two types of data into test set data and training set data;
then invoking an image processing library in a python environment, defining enhancement factors, and carrying out data enhancement on the original image;
the step 2 specifically includes screening and classifying data through a convolutional neural network, and removing invalid pictures without insulators, and specifically includes the following steps:
step 2.1, constructing a convolutional neural network model by using Tensorflow, and taking the 3-channel image data obtained in the step 1, namely a training data set, as input of training of the network model;
the first layer of convolution layer, the input data passes through 16 convolution kernels, the convolution kernels are 3 by 3, the step length is 1, and the padding is set as the same;
the second pooling layer max pool, the pooling window is 2 x 2, the step size is 2, the padding is set as the same, and the data is downsampled to one half of the original data.
Extracting image features through multiple convolutions, adding deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;
the defined loss function of the network model is shown in formula (1):
loss＝-[plog(t)+(1-p)log(1-t)] (1)
dividing the output result into two classes, wherein p is the expected classification probability of the image, and t is the actual classification probability of the image; the full-connection layer converts the obtained data features through feature weighting to realize data unidimensionation, and then the softmax operation normalizes the result and combines the cross entropy formula to output the judgment result of the image;
step 2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, describing that the model gradually converges, using the test set data as the input for generating a network, entering the trained network model, and checking the output accuracy;
the specific content in the step 3 is as follows:
step 3.1, processing the test set data and extracting HOG characteristics:
carrying out Gaussian filtering treatment on the data image of the test set, then carrying out downsampling decomposition on the smoothed image, respectively carrying out 1/4 downsampling on the size of the smoothed image to obtain a new image, continuously downsampling to obtain thumbnails with different sizes, finally forming a three-layer pyramid, gradually reducing the image size from the bottommost layer to the topmost layer, and gradually reducing the image resolution to obtain a Gaussian pyramid model; then, extracting HOG features from each layer of image of the obtained Gaussian pyramid model respectively to form a final HOG feature vector of the image;
step 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1;
the step 3 of extracting the HOG characteristic specific content comprises the following steps:
step 3.1.1, adopting a Gamma correction method to normalize a color space of the Gaussian pyramid model;
step 3.1.2, calculating the gradient of each pixel of the image:
the gradients of the pixel points (x, y) in the image in the horizontal direction and the vertical direction are respectively shown as the formula (2) and the formula (3):
G X (x,y)＝H(x+1,y)-H(x-1,y) (2)
G y (x,y)＝H(x,y+1)-H(x,y-1) (3)；
after obtaining the horizontal gradient and the vertical gradient, obtaining the gradient amplitude and the gradient direction corresponding to the pixel point, wherein the gradient amplitude and the gradient direction are respectively shown as the formula (4) and the formula (5):
step 3.1.3, dividing the gray level image into small cells which cannot slide;
step 3.1.4, dividing intervals by angles in the direction of the pixel gradient, and counting the gradient histogram of each cell to form a descriptor of each cell;
step 3.1.5, forming a block by each cell, and connecting feature descriptors of all cells in the block in series to obtain HOG feature descriptors of the block;
step 3.1.6, connecting HOG feature descriptors of all blocks in the image in series to obtain HOG features of the picture;
the specific operation content of determining the approximate position of the insulator in the picture by using the extracted HOG characteristics in the step 3 is as follows:
when the block calculates the sliding process in the image, judging that the region contains an insulator if the current gradient amplitude value is similar to the gradient histogram and the value of the adjacent block; judging the characteristic gradient histogram of the whole picture to obtain the ROI region distribution of the image;
the specific content of the step 4 comprises the following steps:
constructing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate;
LSTM needs 8 groups of parameters to learn, respectively: weight matrix W of forgetting gate f And bias term b f Weight matrix W of input gate i And bias term b i Weight matrix W of output gate o And bias term b o And a weight matrix W for calculating cell states c And bias term b c ；
The forget gate and the input gate are shown in formula (6) and formula (7), respectively:
f t ＝σ(W f ·[h t-1 ,x t ]+b f ) (6)
i t ＝σ(W i ·[h t-1 ,x t ]+b i ) (7)
w in the formula f Is the weight matrix of forgetting gate, [ h ] t-1 ,x t ]Representing the joining of two vectors into one longer vector, b f Is a bias term for forgetting gates, σ is a sigmoid function;
the output gate is shown in equation (8):
o t ＝σ(W o ·[h t-1 ,x t ]+b o ) (8)；
selecting a result of forgetting the last moment through a forgetting gate, combining the input of the moment to obtain the output of a cell at the current moment, judging the results through an output gate, taking the judging result of the output gate as the input of a convolutional neural network, screening and extracting the image of the determined ROI area through an LSTM network once again, judging the defect condition of an insulator, and finally classifying the obtained judging result in the input classification convolutional neural network;
the training data of the classified convolutional neural network are divided into two types, pictures of an intact insulator and a defective insulator are respectively stored, the two types of data are divided into a training set and a verification set, and then the training data are input into the convolutional neural network for training;
the training process of the input convolutional neural network comprises the following steps:
the first layer of convolution layer, the input data passes through 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a second pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2, and the data is downsampled; the third layer of convolution layer is subjected to 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a fourth pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2;
adding deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy by using a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained by the fourth layer are unidimensionalized, the data obtained by the fourth layer are converted through feature weighting, and then softmax is normalized to obtain a final judging result:
wherein the defined loss function is as shown in formula (9):
loss＝loss MES +loss CNN (9)
where loss is the loss of content, including MSE loss for LSTM networks MES And loss of judgment loss of convolution network CNN As shown in the formula (10) and the formula (11), respectively:
in the formula (10), H and W represent the width and height of the image, I x,y Representing the x, y position, I of a real image x,y ' represents the x, y position of the output image to be passed through the network module; in the formula (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network.
And obtaining the corresponding position in the video frame sequence according to the output result, thereby outputting the final defect detection result.
The beneficial effects of the invention are as follows:
aiming at the limitation and the deficiency of the existing power tower insulator defect detection method, the invention improves the insulator defect detection precision by using a deep learning method, and effectively solves the problem of the non-robust detection algorithm caused by the deficiency of the number of training samples of the deep learning. Because the deep learning network model with memory capability is adopted, the algorithm provided by the invention can extract insulator defect information recorded by multi-frame image data in video, so that the detection accuracy is higher than that of the existing detection algorithm based on single-frame image data. The pre-screening process of the power inspection video data is introduced, so that the method has lower computational complexity and higher detection efficiency, and is more suitable for application scenes of processing a large amount of inspection video data in the power inspection process.
Drawings
FIG. 1 is a diagram showing a CNN network structure in an automatic insulator defect detection algorithm for an electric power inspection;
FIG. 2 is a diagram of the overall network model in the automatic insulator defect detection algorithm for the power inspection of the invention;
FIG. 3 is a basic block diagram of an LSTM network in an automatic detection algorithm for insulator defects for an electric power inspection;
FIG. 4 is a flowchart of a HOG feature locating insulator in an automatic detection algorithm for insulator defects for an electric power inspection;
fig. 5 is a flowchart of defect detection in an automatic insulator defect detection algorithm for an electric power inspection video.
Detailed Description
The invention will be described in detail below with reference to the drawings and the detailed description.
The invention provides an insulator defect automatic detection algorithm for an electric power inspection video, which is implemented according to the following steps as shown in fig. 2 and 5:
step 1, classifying and enhancing original electric power tower insulator sample data:
extracting existing electric power tower insulator sample data to obtain an original sample image, and dividing the data into two main types: the test set data and the training set data are divided into the test set data and the training set data;
calling an image processing library in a python environment, defining enhancement factors, and carrying out data enhancement on an original image: data overturning, data rotation, data amplification, data clipping, data translation, noise disturbance and the like, and a plurality of samples different from original data can be obtained through data enhancement, so that data diversity is improved. The increase of the data number also enhances the accuracy of the training model, and overcomes the limitation of a small data set on a certain condition;
step 2, constructing and generating a convolutional neural network model, and training the network model through the data enhanced in the step 1:
step 2, screening data through a convolutional neural network, and removing invalid pictures without insulators, wherein the specific process is as follows:
step 2.1, constructing a convolutional neural network model by using Tensorflow, wherein the basic structure of a characteristic extraction network of the convolutional neural network model consists of a series of convolutional units and pooling units, the 3-channel image data obtained in the step 1, namely a training data set, is used as the input of network model training, a convolutional structure table is shown in fig. 1, H, W and C represent the size and the number of channels of an input image, and finally S is the output judgment result;
the first layer of convolution layer, the input data passes through 16 convolution kernels, the convolution kernels are 3 by 3, the step length is 1, and the padding is set as the same;
the second pooling layer max pool, the pooling window is 2 x 2, the step size is 2, the padding is set as the same, and the data is downsampled to one half of the original data.
Extracting image features through multiple convolutions, adding deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;
the defined loss function of the network model is shown in formula (1):
loss＝-[plog(t)+(1-p)log(1-t)] (1)
because the network is used for eliminating irrelevant background pictures without insulators, the output result is only required to be divided into two types, p is the expected classification probability of the image, and t is the actual classification probability of the image; the full-connection layer converts the obtained data features through feature weighting to realize data unidimensionation, and then the softmax operation normalizes the result and combines the cross entropy formula to output the judgment result of the image;
step 2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, describing that the model gradually converges, using the test set data as the input for generating a network, entering the trained network model, and checking the output accuracy;
step 3, processing the test set data obtained by classification in the step 1, extracting HOG characteristics from pictures passing through the network model through the established network model, and determining the approximate position of the insulator in the pictures:
step 3.1, as shown in fig. 4, processing the picture to be detected and extracting HOG features:
carrying out Gaussian filtering treatment on an image to be detected, then carrying out downsampling decomposition on the image subjected to the smoothing treatment, respectively carrying out 1/4 downsampling on the size of the image to obtain a new image, continuously downsampling to obtain thumbnails with different sizes, finally forming a three-layer pyramid, gradually reducing the image size from the bottommost layer to the topmost layer, and gradually reducing the image resolution to obtain a Gaussian pyramid model; then, extracting HOG features from each layer of image of the obtained Gaussian pyramid model respectively to form a final HOG feature vector of the image;
the specific process for extracting HOG features is as follows:
step 3.1.1, adopting a Gamma correction method to normalize a color space of the Gaussian pyramid model image, adjusting the contrast of the image, effectively reducing the influence caused by local shadow and illumination change of the image, and simultaneously inhibiting noise interference;
step 3.1.2, calculating the gradient (including the size and the direction) of each pixel of the image, capturing contour information through gradient information, and further weakening the interference of illumination at the same time:
the gradients of the pixel points (x, y) in the image in the horizontal direction and the vertical direction are respectively shown as the formula (2) and the formula (3):
G X (x,y)＝H(x+1,y)-H(x-1,y) (2)
G y (x,y)＝H(x,y+1)-H(x,y-1) (3)；
after obtaining the horizontal gradient and the vertical gradient, obtaining the gradient amplitude and the gradient direction corresponding to the pixel point, wherein the gradient amplitude and the gradient direction are respectively shown as the formula (4) and the formula (5):
step 3.1.3, dividing the gray level image into small cells which cannot slide;
step 3.1.4, the direction of the pixel gradient can be divided into intervals by angles, and the gradient histogram (the number of different gradients in a certain interval) of each cell is counted, so that the descriptor of each cell can be formed;
step 3.1.5, forming a block by each cell, and connecting feature descriptors of all cells in the block in series to obtain HOG feature descriptors of the block;
step 3.1.6, connecting HOG feature descriptors of all blocks in the image in series to obtain HOG feature, gradient image and gradient amplitude information of the picture;
step 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1: after the related information of the picture to be detected is obtained, the gradient histogram can be distributed in a specific gradient direction at the edge part with larger change of the insulator. Considering that the gradient histogram of the insulators and the background part in the picture are large in difference, and the insulators are basically arranged periodically, the difference of the gradient amplitude values among the insulators is small, so that when the block is similar to the gradient histogram and the adjacent block in the sliding process in the image calculation, the region is judged to contain the insulators; judging the characteristic gradient histogram of the whole picture to obtain the ROI region distribution of the image;
calculating gradient positioning, carrying out frequency domain transformation on the positioned area, and rotating according to the detected main direction, so that insulators in all pictures are adjusted to be in one direction, and the accuracy of neural network processing judgment is improved;
step 4, after determining the approximate position of the insulator in the picture through the step 3, detecting whether the insulator is defective or not through a CNN/LSTM deep learning method:
constructing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate; as shown in fig. 3, the forget gate is responsible for deciding how much of the last time cell state remains to the current time cell state; the input gate is responsible for deciding how much of the cell state at the current time to keep input to the current time; the output gate is responsible for determining how much output exists in the unit state at the current moment;
LSTM needs 8 groups of parameters to learn, respectively: weight matrix W of forgetting gate f And bias term b f Weight matrix W of input gate i And bias term b i Weight matrix W of output gate o And bias term b o And a weight matrix W for calculating cell states c And bias term b c ；
The forget gate and the input gate are shown in formula (6) and formula (7), respectively:
f t ＝σ(W f ·[h t-1 ,x t ]+b f ) (6)
i t ＝σ(W i ·[h t-1 ,x t ]+b i ) (7)
w in the formula f Is the weight matrix of forgetting gate, [ h ] t-1 ,x t ]Representing the joining of two vectors into one longer vector, b f Is a bias term for forgetting gates, σ is a sigmoid function;
the output gate is shown in equation (8):
o t ＝σ(W o ·[h t-1 ,x t ]+b o ) (8)；
the result of the last moment is selected through a forgetting gate, the output of the cell at the current moment is obtained by combining the input of the moment, the results are judged through an output gate, and the final output of the LSTM network is determined by the output gate and the state of the cell together; taking the output gate result as the input of the convolutional neural network, screening and extracting the image of the determined ROI region through the LSTM network for one time, and then transmitting the obtained result into the convolutional neural network to judge the defect condition of the insulator;
the training data of the convolutional neural network are divided into two types, pictures of a perfect insulator and a defective insulator are respectively stored, the two types of data are divided into a training set and a verification set, and then the training data are input into the convolutional neural network for training;
the training process of the input convolutional neural network comprises the following steps:
the first layer of convolution layer, the input data passes through 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a second pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2, and the data is downsampled; the third layer of convolution layer is subjected to 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a fourth pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2;
adding deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy by using a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained by the fourth layer are unidimensionalized, the data obtained by the fourth layer are converted through feature weighting, and then softmax is normalized to obtain a final judging result:
wherein the defined loss function is as shown in formula (9):
loss＝loss MES +loss CNN (9)
where loss is the loss of content, including MSE loss for LSTM networks MES And loss of judgment loss of convolution network CNN As shown in the formula (10) and the formula (11), respectively:
in the formula (10), H and W represent the width and height of the image, I x,y Representing the x, y position, I of a real image x,y ' represents the x, y position of the output image to be passed through the network module; in the formula (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network.
And obtaining the corresponding position in the video frame sequence according to the output result, thereby outputting the final defect detection result.
The invention discloses an insulator defect automatic detection algorithm for an electric power inspection video, which comprises the following specific processes:
firstly, classifying by CNN, and analyzing whether a picture has a classification network of insulators or not;
then extracting the HOG characteristics of the Gaussian pyramid, namely, performing ROI segmentation on the insulator-existing part of the picture;
then, an LSTM detection neural network is adopted to detect and judge whether the insulator is damaged (whether the insulator is damaged or not is judged by a plurality of continuous feature images), and the LSTM input is a plurality of frame insulator feature images (which can be understood as a short feature video) after HOG processing, so that the detection of the insulator damage is a multi-frame feature detection method which utilizes the correlation of a time domain and a space domain, and is particularly suitable for processing inspection video;
the ordered arrangement mode of the three steps enables the method to be suitable for inspection video data instead of image data, thereby improving efficiency and precision; because the LSTM detection neural network is judged to be a probability, the invention classifies by adopting the one-time classification convolutional neural network to obtain an exact defect position, and the CNN and the LSTM are firstly trained independently of each other and then are trained by cascading the two networks due to different inputs, so that the convergence and the precision of each network and the final network are improved.
Claims (4)
1. An insulator defect automatic detection algorithm for an electric power inspection video is characterized by comprising the following steps:
step 1, classifying and enhancing original electric power tower insulator sample data:
firstly, extracting sample data of an existing electric power tower insulator to obtain an original sample image, dividing the data into two types of data including the insulator and the insulator-free data, and dividing the two types of data into test set data and training set data;
then invoking an image processing library in a python environment, defining enhancement factors, and carrying out data enhancement on the original image;
step 2, constructing and generating a convolutional neural network model, training the network model through the data enhanced in the step 1, and classifying through the convolutional neural network model, specifically, screening and classifying the data through the convolutional neural network, and eliminating invalid pictures without insulators;
step 3, processing the data classified in the step 2, extracting HOG characteristics from the pictures passing through the network model through the established network model, and determining the approximate position of the insulator in the pictures, wherein the method comprises the following steps of:
step 3.1, processing the test set data and extracting HOG characteristics:
carrying out Gaussian filtering treatment on the data image of the test set, then carrying out downsampling decomposition on the smoothed image, respectively carrying out 1/4 downsampling on the size of the smoothed image to obtain a new image, continuously downsampling to obtain thumbnails with different sizes, finally forming a three-layer pyramid, gradually reducing the image size from the bottommost layer to the topmost layer, and gradually reducing the image resolution to obtain a Gaussian pyramid model; then, extracting HOG features from each layer of image of the obtained Gaussian pyramid model respectively to form a final HOG feature vector of the image;
the method for extracting the HOG characteristic specific content comprises the following steps of:
step 3.1.1, adopting a Gamma correction method to normalize a color space of the Gaussian pyramid model;
step 3.1.2, calculating the gradient of each pixel of the image:
the gradients of the pixel points (x, y) in the image in the horizontal direction and the vertical direction are respectively shown as the formula (2) and the formula (3):
G X (x,y)＝H(x+1,y)-H(x-1,y) (2)
G y (x,y)＝H(x,y+1)-H(x,y-1) (3)；
after obtaining the horizontal gradient and the vertical gradient, obtaining the gradient amplitude and the gradient direction corresponding to the pixel point, wherein the gradient amplitude and the gradient direction are respectively shown as the formula (4) and the formula (5):
step 3.1.3, dividing the gray level image into small cells which cannot slide;
step 3.1.4, dividing intervals by angles in the direction of the pixel gradient, and counting the gradient histogram of each cell to form a descriptor of each cell;
step 3.1.5, forming a block by each cell, and connecting feature descriptors of all cells in the block in series to obtain HOG feature descriptors of the block;
step 3.1.6, connecting HOG feature descriptors of all blocks in the image in series to obtain HOG features of the picture;
step 3.2, determining the approximate position of the insulator in the picture according to the HOG characteristics extracted in the step 3.1, wherein the specific operation content is as follows:
when the block calculates the sliding process in the image, judging that the region contains an insulator if the current gradient amplitude value is similar to the gradient histogram and the value of the adjacent block; judging the characteristic gradient histogram of the whole picture to obtain the ROI region distribution of the image;
step 4, after determining the approximate position of the insulator in the picture through the step 3, detecting whether the insulator is defective or not through a CNN/LSTM deep learning method, specifically: taking the output of the LSTM as the input of a convolutional neural network, screening and extracting the image with the determined ROI area through the LSTM to obtain a picture characteristic, judging the defect condition of the insulator, and finally transmitting the obtained judging result into a classified convolutional neural network for classification; the training data of the classified convolutional neural network are divided into two types, pictures of an intact insulator and a defective insulator are respectively stored, the two types of data are divided into a training set and a verification set, and then the training data are input into the convolutional neural network for training.
2. The automatic detection algorithm for insulator defect for electric power inspection according to claim 1, wherein the step 2 specifically comprises the steps of screening and classifying data through a convolutional neural network, and excluding invalid pictures without insulators, and the specific steps are as follows:
step 2.1, constructing a convolutional neural network model by using Tensorflow, and taking the 3-channel image data obtained in the step 1, namely a training data set, as input of training of the network model;
the first layer of convolution layer, the input data passes through 16 convolution kernels, the convolution kernels are 3 by 3, the step length is 1, and the padding is set as the same;
a second pooling layer max pool, wherein the pooling window is 2 x 2, the step length is 2, the padding is set as the same, and the data is downsampled to one half of the original data;
extracting image features through multiple convolutions, adding deviation on a result obtained after the convolutions, and activating an output result by using an activation function ReLU;
the defined loss function of the network model is shown in formula (1):
loss＝-[plog(t)+(1-p)log(1-t)] (1)
dividing the output result into two classes, wherein p is the expected classification probability of the image, and t is the actual classification probability of the image; the full-connection layer converts the obtained data features through feature weighting to realize data unidimensionation, and then the softmax operation normalizes the result and combines the cross entropy formula to output the judgment result of the image;
and 2.2, inputting the data enhanced in the step 1 into the network model established in the step 2.1 to start training, when the loss value tends to be stable, gradually converging the model, using the test set data as the input for generating a network, entering the trained network model, and checking the output accuracy.
3. The automatic insulator defect detection algorithm for the power inspection according to claim 1, wherein the specific content of the step 4 comprises:
constructing an LSTM neural network model by using Tensorflow, wherein the basic structure of the LSTM model realizes three gate operations, namely a forgetting gate, an input gate and an output gate;
LSTM needs 8 groups of parameters to learn, respectively: weight matrix W of forgetting gate f And bias term b f Weight matrix W of input gate i And bias term b i Weight matrix W of output gate o And bias term b o And a weight matrix W for calculating cell states c And bias term b c ；
The forget gate and the input gate are shown in formula (6) and formula (7), respectively:
f t ＝σ(W f ·[h t-1 ,x t ]+b f ) (6)
i t ＝σ(W i ·[h t-1 ,x t ]+b i ) (7)
w in the formula f Is the weight matrix of forgetting gate, [ h ] t-1 ,x t ]Representing the joining of two vectors into one longer vector, b f Is a bias term for forgetting gates, σ is a sigmoid function;
the output gate is shown in equation (8):
o t ＝σ(W o ·[h t-1 ,x t ]+b o ) (8)；
and selecting a result of the last moment to be forgotten through a forgetting gate, combining the input of the moment to obtain the output of the cell at the current moment, judging the results through an output gate, taking the judging result of the output gate as the input of a convolutional neural network, screening and extracting the image of the determined ROI area once again through an LSTM network, judging the defect condition of the insulator, and finally classifying the obtained judging result in the input classification convolutional neural network.
4. The automatic detection algorithm for insulator defect for electric power inspection according to claim 3, wherein the training data of the classified convolutional neural network are divided into two types, pictures of an intact insulator and a defective insulator are respectively stored, the two types of data are divided into a training set and a verification set, and then the training data are input into the convolutional neural network for training;
the training process of the input convolutional neural network comprises the following steps:
the first layer of convolution layer, the input data passes through 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a second pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2, and the data is downsampled; the third layer of convolution layer is subjected to 5 convolution kernels, the convolution kernels are 3 x 5, and the step length is 1; a fourth pooling layer max pool, wherein the pooling window is 2 x 2, and the step length is 2;
adding deviation on the result obtained after convolution, activating the output result by using an activation function ReLU, and using cross entropy by using a loss function; the fifth layer and the sixth layer are all connected layers, the data obtained by the fourth layer are unidimensionalized, the data obtained by the fourth layer are converted through feature weighting, and then softmax is normalized to obtain a final judging result:
wherein the defined loss function is as shown in formula (9):
loss＝loss MES +loss CNN (9)
where loss is the loss of content, including MSE loss for LSTM networks MES And loss of judgment loss of convolution network CNN As shown in the formula (10) and the formula (11), respectively:
in the formula (10), H and W represent the width and height of the image, I x,y Representing the x, y position, I of a real image x,y ' indicate will be passed throughPassing through the x and y positions of the output image of the network module; in the formula (11), P is the expected classification probability of the image, and T is the actual classification probability of the image passing through the network;
and obtaining the corresponding position in the video frame sequence according to the output result, thereby outputting the final defect detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010452724.5A CN111784633B (en) | 2020-05-26 | 2020-05-26 | Insulator defect automatic detection algorithm for electric power inspection video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010452724.5A CN111784633B (en) | 2020-05-26 | 2020-05-26 | Insulator defect automatic detection algorithm for electric power inspection video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111784633A CN111784633A (en) | 2020-10-16 |
| CN111784633B true CN111784633B (en) | 2024-02-06 |
Family
ID=72754214
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010452724.5A Active CN111784633B (en) | 2020-05-26 | 2020-05-26 | Insulator defect automatic detection algorithm for electric power inspection video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111784633B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348744B (en) * | 2020-11-24 | 2022-07-01 | 电子科技大学 | Data enhancement method based on thumbnail |
| CN113077525A (en) * | 2021-02-06 | 2021-07-06 | 西南交通大学 | Image classification method based on frequency domain contrast learning |
| CN112837315B (en) * | 2021-03-05 | 2023-11-21 | 云南电网有限责任公司电力科学研究院 | Deep learning-based transmission line insulator defect detection method |
| CN113009416B (en) * | 2021-04-08 | 2024-03-12 | 国网江苏省电力有限公司检修分公司 | Insulator detection positioning method based on laser sensor array |
| CN113361631A (en) * | 2021-06-25 | 2021-09-07 | 海南电网有限责任公司电力科学研究院 | Insulator aging spectrum classification method based on transfer learning |
| CN113536989B (en) * | 2021-06-29 | 2024-06-18 | 广州博通信息技术有限公司 | Refrigerator frosting monitoring method and system based on frame-by-frame analysis of camera video |
| CN113538382B (en) * | 2021-07-19 | 2023-11-14 | 安徽炬视科技有限公司 | Insulator detection algorithm based on non-deep network semantic segmentation |
| CN113538412B (en) * | 2021-08-06 | 2024-08-02 | 广东电网有限责任公司 | Insulator defect detection method and device for aerial image |
| CN113538411B (en) * | 2021-08-06 | 2024-10-18 | 广东电网有限责任公司 | Insulator defect detection method and device |
| CN113807194B (en) * | 2021-08-24 | 2023-10-10 | 哈尔滨工程大学 | Enhanced power transmission line fault image recognition method |
| CN114708267B (en) * | 2022-06-07 | 2022-09-13 | 浙江大学 | Image detection processing method for corrosion defect of tower stay wire on power transmission line |
| CN117853923B (en) * | 2024-01-17 | 2024-10-18 | 国网经济技术研究院有限公司 | Power grid power infrastructure safety evaluation analysis method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106504233A (en) * | 2016-10-18 | 2017-03-15 | 国网山东省电力公司电力科学研究院 | Image electric power widget recognition methodss and system are patrolled and examined based on the unmanned plane of Faster R CNN |
| WO2018171109A1 (en) * | 2017-03-23 | 2018-09-27 | 北京大学深圳研究生院 | Video action detection method based on convolutional neural network |
| CN109166094A (en) * | 2018-07-11 | 2019-01-08 | 华南理工大学 | A kind of insulator breakdown positioning identifying method based on deep learning |
| CN109255776A (en) * | 2018-07-23 | 2019-01-22 | 中国电力科学研究院有限公司 | A kind of transmission line of electricity split pin defect automatic identifying method |
-
2020
- 2020-05-26 CN CN202010452724.5A patent/CN111784633B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106504233A (en) * | 2016-10-18 | 2017-03-15 | 国网山东省电力公司电力科学研究院 | Image electric power widget recognition methodss and system are patrolled and examined based on the unmanned plane of Faster R CNN |
| WO2018171109A1 (en) * | 2017-03-23 | 2018-09-27 | 北京大学深圳研究生院 | Video action detection method based on convolutional neural network |
| CN109166094A (en) * | 2018-07-11 | 2019-01-08 | 华南理工大学 | A kind of insulator breakdown positioning identifying method based on deep learning |
| CN109255776A (en) * | 2018-07-23 | 2019-01-22 | 中国电力科学研究院有限公司 | A kind of transmission line of electricity split pin defect automatic identifying method |
Non-Patent Citations (2)
| Title |
|---|
| 杨晓旭 ; 温招洋 ; .深度学习在输电线路绝缘子故障检测中的研究与应用.中国新通信.2018,(第10期),全文. * |
| 陈俊杰 ; 叶东华 ; 产焰萍 ; 陈凌睿 ; .基于Faster R-CNN模型的绝缘子故障检测.电工电气.2020,(第04期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111784633A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784633B (en) | Insulator defect automatic detection algorithm for electric power inspection video | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN107944396B (en) | Knife switch state identification method based on improved deep learning | |
| CN108961235B (en) | Defective insulator identification method based on YOLOv3 network and particle filter algorithm | |
| CN112199993B (en) | Method for identifying transformer substation insulator infrared image detection model in any direction based on artificial intelligence | |
| CN109753929B (en) | High-speed rail insulator inspection image recognition method | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN111797890A (en) | Method and system for detecting defects of power transmission line equipment | |
| CN111611874B (en) | Face mask wearing detection method based on ResNet and Canny | |
| CN107133943A (en) | A kind of visible detection method of stockbridge damper defects detection | |
| CN113449727A (en) | Camouflage target detection and identification method based on deep neural network | |
| CN109191421B (en) | Visual detection method for pits on circumferential surface of cylindrical lithium battery | |
| CN106610969A (en) | Multimodal information-based video content auditing system and method | |
| CN109034184B (en) | Grading ring detection and identification method based on deep learning | |
| CN113920097B (en) | Power equipment state detection method and system based on multi-source image | |
| CN113344475B (en) | Transformer bushing defect identification method and system based on sequence modal decomposition | |
| CN114298948B (en) | PSPNet-RCNN-based abnormal monitoring detection method for ball machine | |
| CN106557740B (en) | The recognition methods of oil depot target in a kind of remote sensing images | |
| CN109448009A (en) | Infrared Image Processing Method and device for transmission line faultlocating | |
| CN110415260A (en) | Smog image segmentation and recognition methods based on dictionary and BP neural network | |
| Su et al. | A new local-main-gradient-orientation HOG and contour differences based algorithm for object classification | |
| CN112419243B (en) | Power distribution room equipment fault identification method based on infrared image analysis | |
| CN116485802B (en) | Insulator flashover defect detection method, device, equipment and storage medium | |
| CN110618129A (en) | Automatic power grid wire clamp detection and defect identification method and device | |
| CN114219763A (en) | Infrared picture detection method for abnormal heating point of power distribution equipment based on fast RCNN algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |