CN114494186A - Fault detection method for high-voltage power transmission and transformation line electrical equipment - Google Patents
Fault detection method for high-voltage power transmission and transformation line electrical equipment Download PDFInfo
- Publication number
- CN114494186A CN114494186A CN202210086112.8A CN202210086112A CN114494186A CN 114494186 A CN114494186 A CN 114494186A CN 202210086112 A CN202210086112 A CN 202210086112A CN 114494186 A CN114494186 A CN 114494186A
- Authority
- CN
- China
- Prior art keywords
- image
- infrared thermal
- visible light
- electrical equipment
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000009466 transformation Effects 0.000 title claims abstract description 69
- 230000005540 biological transmission Effects 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 30
- 239000011159 matrix material Substances 0.000 claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims abstract description 19
- 238000013021 overheating Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims description 37
- 238000000034 method Methods 0.000 claims description 35
- 238000000605 extraction Methods 0.000 claims description 14
- 238000011176 pooling Methods 0.000 claims description 14
- 230000004927 fusion Effects 0.000 claims description 10
- 238000007689 inspection Methods 0.000 claims description 9
- 238000012512 characterization method Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000009413 insulation Methods 0.000 description 2
- 230000035943 smell Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012212 insulator Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
- G01R31/08—Locating faults in cables, transmission lines, or networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
- Y04S10/52—Outage or fault management, e.g. fault detection or location
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a fault detection method for high-voltage power transmission and transformation line electrical equipment, which comprises the following steps: acquiring a visible light image, an infrared thermal image and an infrared thermal image temperature value of the high-voltage power transmission and transformation line electrical equipment; constructing a target segmentation unet model, segmenting electrical equipment from the visible light image through the target segmentation unet model, and extracting the contour of a target area of the segmented electrical equipment; calculating an affine transformation matrix of the image pair; registering the visible light image and the infrared thermal image according to the affine transformation matrix, so that the outline of the electrical equipment in the visible light image is transformed into the infrared thermal image, and the target area of the electrical equipment in the infrared thermal image is positioned; whether the electrical equipment has an overheating phenomenon is detected according to the temperature value in the target area of the electrical equipment in the infrared thermal image and the overheating detection standards of different electrical equipment, so that the problem that the target area of the infrared thermal image is not accurately segmented in a complex environment is solved.
Description
Technical Field
The invention relates to the field of machine vision defect detection, in particular to an infrared automatic detection method for electrical equipment faults of a high-voltage power transmission and transformation circuit in a complex environment.
Background
Due to long-term use and the influence of factors such as external conditions, the power transmission and distribution line is easy to age, loose in wiring, further causes the disconnection and short-circuit faults of components, finally causes the problems of failure of the whole insulation and the like, if the problems are not found and solved in time, the whole electric system cannot work, and the life safety of people is possibly harmed in serious cases.
At present, the detection of electrical equipment mainly adopts a method of regular maintenance, a maintenance worker firstly observes whether insulation of the electrical equipment is burnt or not and whether a connection line is broken or not by eyes, smells whether peculiar smell exists in the air or not by a nose, and listens whether abnormal sound exists in the electrical equipment or not by ears to preliminarily find the obvious abnormal condition of the electrical equipment, and then measures whether current, voltage, resistance or other parameters of key points of the electrical element meet the requirements or not by other instruments such as a multimeter and the like to carry out detailed detection on the actual condition of the electrical equipment. The detection method has low efficiency, and the normal work of the system is seriously influenced because the system needs to be stopped during maintenance. When the electrical equipment works, due to the current effect, the electrical equipment generates heat, particularly when the electrical equipment is abnormal, due to heat loss, local temperature rise can occur at the abnormal part, the surface temperature distribution condition of the electrical equipment can be effectively obtained through the thermal infrared imager, and then the fault condition of the electrical equipment can be diagnosed. The method for diagnosing faults in the power system by using an infrared detection technology mainly comprises a temperature judgment method, a relative temperature difference method, a similar comparison method and an archive analysis method. The defects of the methods are that fault judgment is completed manually, but manual detection efficiency cannot meet application requirements.
In order to improve the detection efficiency, many scholars have studied the infrared thermal image processing, mainly classified as: one is that the infrared thermal image target area is detected directly based on the infrared thermal image, and the judgment is carried out by utilizing different standards according to different detected objects; the other type is that a target area is detected in a high-resolution visible light image based on visible light image processing, and then the target area in the visible light image is registered to an infrared thermal image by utilizing a registration algorithm, so that the target area is detected. However, these detection methods have two problems: (1) the infrared thermal image maps temperature by using RGB values, and the infrared thermal image has no obvious edge and texture, so that when the electrical equipment is in a complex environment, a target area is directly segmented based on the infrared image, the target segmentation is easy to cause inaccuracy, and the fault and the omission are caused; (2) the resolution ratio of the infrared thermal image is low, and the overheating area of the corresponding thin part is very small in the image, so that the overheating area is missed to be detected.
Disclosure of Invention
In view of the above, it is necessary to provide a method for detecting a fault of an electrical device of a high-voltage power transmission and transformation line.
The embodiment of the invention provides a fault detection method for high-voltage power transmission and transformation line electrical equipment, which comprises the following steps:
acquiring a visible light image of the high-voltage power transmission and transformation line electrical equipment, an infrared thermal image corresponding to the visible light image and an infrared thermal image temperature value;
training a unet network through a visible light image to construct a target segmentation unet model; electrical equipment is segmented from the visible light image through a target segmentation unet model, and target area outline extraction is carried out on the segmented electrical equipment;
establishing K groups of image pairs of the visible light images and the corresponding infrared thermal images, selecting L characteristic pairs in each group of image pairs, and calculating an affine transformation matrix of the image pairs according to the characteristic pairs; wherein L < K and L > -4;
registering the visible light image and the infrared thermal image according to the affine transformation matrix, so that the outline of the electrical equipment in the visible light image is transformed into the infrared thermal image, and the target area of the electrical equipment in the infrared thermal image is positioned;
and detecting whether the electrical equipment has an overheating phenomenon or not according to the temperature value in the target area of the electrical equipment in the infrared thermal image and overheating detection standards of different electrical equipment.
The embodiment of the invention provides a fault detection method for high-voltage power transmission and transformation line electrical equipment, which further comprises the following steps:
and performing multi-scale super-resolution reconstruction on the low-resolution infrared thermal image by taking the high-resolution visible light image as guidance, so that the low-resolution infrared thermal image and the high-resolution visible light image have the same resolution and size.
Further, the acquiring of the visible light image of the high voltage transmission and transformation line electrical equipment and the infrared thermal image corresponding to the visible light image specifically include:
installing an infrared thermal imager provided with a visible light camera and an infrared thermal camera on the inspection robot;
and aligning the lens of the thermal infrared imager to each part of the aerial high-voltage power transmission and transformation circuit, and simultaneously shooting visible light images of each electrical device on the high-voltage power transmission and transformation circuit and infrared thermal images corresponding to the visible light images.
Further, the training of the unet network through the visible light image to construct the target segmentation unet model specifically includes:
dividing visible light RGB image training sample set into labeled sample set DL＝{XL,YLAnd unlabeled sample set DU＝{XU}; wherein, XLFor visible RGB images, YLIs XLA true value label image of the medium electric device;
with labelled sample sets
Training a unet network;
randomly dividing the unlabeled sample set into k sub-data sets, and dividing the sub-data sets
Sequentially input into unet network, and continuously predicting by dividing network
Generating corresponding segmentation results
Form a
Wherein
I is the training sample of the ith input, i is 1,2, …, k;
will be provided with
Adding to the current round of training
In the training set, a new training set is formed
For the next round of training;
and (3) using the updated training set to train the unet network again, inputting a next batch of unlabeled samples after the training is finished, predicting to generate pseudo labels until all the unlabeled sample sets generate the pseudo labels, and stopping to obtain a trained unet model.
Further, the calculating an affine transformation matrix of the image pair according to the feature pairs specifically includes:
assuming (u ', v') infrared image coordinates and (u, v) visible image coordinates, affine transformation is performed on the image using the following equation:
where H is a 3 × 3 homography transform matrix.
Preferably, the method for detecting a fault of an electrical device of a high-voltage power transmission and transformation line according to an embodiment of the present invention further includes:
computing a transformation matrix H for K sets of imagesjJ is 1,2, …, K, and HjThe sum of the values of the elements in the matrix is averaged by HjThe average value of (c) is used as the final affine transformation matrix H.
Further, registering the visible light image and the infrared thermal image according to the affine transformation matrix specifically includes:
let a coordinate of a certain point of the visible light image be (x)1,y1) Using affine transformation of the coordinates (x)1,y1) Transforming into infrared thermal image to obtain transformed coordinate (x)2,y2) (ii) a The affine transformation is as follows:
further, the multi-scale super-resolution reconstruction of the low-resolution infrared thermal image with the high-resolution visible light image as guidance specifically includes:
inputting a high-resolution visible light RGB image with the size of 4m multiplied by 4n into a multi-scale feature extraction model, and obtaining Fc through two convolution layers conv1_ RGB and conv2_ RGB14m x 4n, comprising 64 profiles, Fc1Is a first scale feature;
outputting an image Fc to conv2_ rgb1Pooling is carried out, Fc is obtained through a pooling layer with the pooling kernel size of 2 multiplied by 2 and the step length of 2, and the specific process is as follows: fc ═ Maxpooling (Fc)1) (ii) a Wherein Maxpooling denotes maximum pooling, the size of Fc is 2 m.times.2n;
fc was obtained by passing Fc through 2 convolutional layers, conv3_ rgb, conv4_ rgb2，Fc2Size 2m × 2n, containing 64 characteristic maps, Fc2Is a second scale feature;
carrying out bilinear interpolation on the RGB infrared thermal image with the size of mxn once to obtain an image after the first interpolation with the size of 2 mx 2n, and carrying out convolution twice on the interpolated image to obtain Ft1The convolution result Ft1And Fc2Connecting to realize multi-scale one-time fusion to obtain Ft2；
Ft2Firstly, after convolution conv3_ t, the convolution kernel size is 3 multiplied by 3, the step size is 1, convolution layers with the number of output feature maps being 64 are output, and then convolution conv4_ t, the convolution kernel size is 3 multiplied by 3, the step size is 1, convolution layers with the number of output feature maps being 256 are output, so that the feature Ft3 after fusion of Ft1 and Fc2 is obtained, the size of Ft3 is 2m multiplied by 2n, and the feature Fc of visible light image branch extraction is realized2Characterization of the infrared thermal image at a scale of 2m × 2n1The guidance of (2);
to Ft3Carrying out second bilinear interpolation to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n4；
Will Ft4Convolution conv5_ t, conv6_ t twice to obtain Ft5The size is 4m multiplied by 4n, and 64 characteristic graphs are included;
will Ft5And Fc1The connection, the specific operation is represented as: ft6＝[Fc1,Ft5](ii) a Wherein, [ Fc1,Ft5]Represents p-Fc1,Ft5Connection operation of Ft6Comprises 128 characteristic graphs;
Ft6after passing through one convolutional layer conv7_ t, the convolutional kernel size is 3 multiplied by 3, the step size is 1, convolutional layers with the number of output feature maps of 64 are output, and after passing through one convolutional layer conv8_ t, the convolutional kernel size is 3 multiplied by 3, the step size is 1, and the number of output feature maps is 1, so that the finally fused feature map Ft is obtained7The feature Fc1 for visible light image branch extraction is realized on the infrared thermal image feature Ft on the scale of 4m multiplied by 4n5The guidance of (2);
to Ft7Performing secondary convolution to obtain Ft8The size is 4m multiplied by 4n, and 64 characteristic graphs are included;
carrying out bilinear interpolation on the original infrared thermal image with the size of m multiplied by n to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n0Ft is0And Ft8Adding the three times of the infrared thermal images pixel by pixel to obtain a result after three times of fusion, and setting the reconstructed high-resolution infrared thermal image as HTIs shown as HT＝Ft0+Ft8，HTThe size is 4m × 4n for high resolution reconstruction results.
Further, the Ft is obtained by realizing multi-scale one-time fusion2The method specifically comprises the following steps:
carrying out first interpolation on the low-resolution infrared thermal image with the size of mxn by adopting a bilinear interpolation method to obtain an image with the size of 2 mx 2 n;
carrying out two-layer convolution on the interpolated image conv1_ t, conv2_ t to obtain Ft1The size is 2m multiplied by 2n, and 64 characteristic graphs are included;
will Ft1And Fc2And connecting operation, specifically expressing as follows: ft2＝[Fc2，Ft1](ii) a In the formula: [ Fc ]2，Ft1]Represents p-Fc2，Ft1Connection operation of, Fc2Comprising 128 feature maps, by first fusing the interpolated image with a high resolution feature image.
Compared with the prior art, the fault detection method for the high-voltage power transmission and transformation line electrical equipment provided by the embodiment of the invention has the following beneficial effects:
the infrared thermal imager capable of shooting visible light and infrared thermal images simultaneously is used for collecting data of electrical equipment of a high-voltage power transmission and transformation circuit, a unet network is trained to accurately segment the electrical equipment in a high-resolution visible light image, the low-resolution infrared thermal image is guided and reconstructed by the characteristics of the high-resolution visible light image, the reconstructed infrared thermal image and the high-resolution visible light image have the same size and resolution, and missing detection caused by tiny electrical equipment or tiny damage, electricity leakage and the like is avoided; and mapping the contour of the target area which can be segmented in the optical image into the infrared thermal image through affine transformation, segmenting electrical equipment in the infrared thermal image, and solving the problem of inaccurate segmentation of the target area of the infrared thermal image in a complex environment.
Drawings
Fig. 1 is a flowchart of a fault detection method for electrical equipment of a high-voltage power transmission and transformation line provided in one embodiment;
FIG. 2 is a semi-supervised based iterative training unet visible light target region segmentation model provided in one embodiment;
FIG. 3 is a visible light image and a corresponding infrared thermal image taken by a thermal infrared imager as provided in one embodiment;
FIG. 4 is a graph illustrating a target segmentation result of the visible light image unit model in FIG. 3 according to an embodiment;
FIG. 5 is a multi-scale super-resolution reconstruction model for infrared thermal images provided in an embodiment;
FIG. 6 is a registration feature point of a visible light image and an infrared thermal image provided in one embodiment, with dots representing manually selected feature points;
FIG. 7 is a graph illustrating the segmentation of the target region of the affine transformation infrared thermal image in accordance with one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, in one embodiment, a fault detection method for electrical equipment of a high-voltage power transmission and transformation line is provided, the method comprising:
and S1, the thermal infrared imager capable of simultaneously shooting the visible light and the infrared thermal image is arranged on the inspection robot to shoot the visible light, the infrared thermal image and the temperature value of the high-voltage power transmission and distribution line electrical equipment.
And S2, constructing a semi-supervised unet network, iteratively training the unet network, segmenting the electrical equipment in the visible light image, and extracting the outline of the segmented electrical equipment target area.
S3, randomly selecting K groups of visible light and infrared thermal images at the beginning of detection to form K groups of image pairs, selecting L feature pairs in each group of images, calculating affine transformation matrixes according to the feature pairs, and taking the average value of the K transformation matrixes as the affine transformation matrixes.
And S4, registering the visible light image and the infrared thermal image according to the affine transformation matrix, transforming the outline of the electrical equipment in the visible light image into the infrared thermal image, and positioning the target area of the electrical equipment in the infrared thermal image according to the outline of the electrical equipment in the infrared thermal image.
And S5, detecting whether the insulator has overheating phenomenon according to the temperature value of the infrared thermal image and overheating detection standards of different electrical equipment.
Compared with other electrical equipment fault detection methods based on infrared thermal image, the steps S1-S5 have the advantages that: in a high-voltage power transmission and transformation line with a complex background, electrical equipment needing to be detected in the infrared thermal image can be accurately segmented.
And S6, carrying out multi-scale super-resolution reconstruction on the low-resolution infrared thermal image by taking the high-resolution visible light image as guidance, so that the low-resolution infrared thermal image and the high-resolution visible light image have the same resolution and size.
Compared with other electrical equipment fault detection methods based on infrared thermal image, the step S6 has the following advantages: through the super-resolution reconstruction of the low-resolution infrared thermal image, the electric leakage and overheating conditions of the small electrical equipment or the small part of the small electrical equipment caused by the small damage in the electrical equipment can be detected.
The execution sequence of the above steps S1 to S6 is determined according to actual requirements.
The implementation method of the step S1 includes the following steps:
and S11, mounting the thermal infrared imager on the inspection robot, aligning the lens of the thermal infrared imager with each component of the aerial high-voltage power transmission and distribution line, and shooting the electrical equipment on the line.
And S12, the thermal infrared imager is provided with a visible light camera and a thermal infrared camera, and the visible light image and the thermal infrared image of the scene can be shot simultaneously.
And S13, connecting the data shot by the thermal infrared imager with a computer on the inspection robot, and uploading the acquired data to the computer for real-time processing.
With reference to fig. 2 to 4, the implementation method of the step S2 includes the following steps:
the visible light RGB image is taken as a training sample, the training sample set is divided into two parts, one part is a labeled sample set DL＝{XL,YLThe other is unlabeled sample DU＝{XU}，XLFor visible RGB images, YLIs XLTrue value label image of medium electric device. Using label sample D due to insufficient number of samples for electrical equipmentLPre-training the unet segmentation model, pre-segmenting X according to the pre-training modelLTo obtain XLPseudo label Y ofUExtended to untagged data DUTo obtain D'U＝{XU,YUSet, detailed steps are as follows:
s21, using data set with labeled image
The unet network is trained.
S22, randomly dividing the data set of the unlabeled image into k sub-data sets, and dividing the sub-data sets
i-1, 2, …, k, wherein
The training samples input for the ith time are sequentially input into a unet network, and the segmentation network continuously predicts
Generating corresponding segmentation results
Form a
S23, mixing
Adding to the current round of training
In the method, a new training set is formed
For the next round of training.
And S24, using the updated training set to train the unet network again, inputting a next batch of unlabeled samples after the training is finished, predicting to generate pseudo labels, repeating the steps until all the unlabeled samples generate the pseudo labels, and stopping the circulation to obtain a trained unet model.
And S25, inputting the test image into the trained model to obtain the segmentation result of each target area of the electrical equipment.
S26, extracting the contour of each target area, and recording the contour coordinates of each target area.
In this embodiment, the number of training samples of the visible light image is 3200, wherein the number of label samples of the visible light image is 800, the visible light image and the label samples thereof form 800 pairs of training samples, the number of unlabeled samples is 2400, the unlabeled samples are averagely divided into 3 subsets, and each subset is 800 images, and the training needs to be completed by iterating 3 times. For the unet network in fig. 2, the unet network comprises three parts, the first part is a feature extraction layer, and five preliminary effective feature layers are obtained through VGG16 convolution pooling; the second part is an enhanced feature extraction part, and the five preliminary effective feature layers of the first part are used for carrying out up-sampling and feature fusion to obtain the effective feature layer finally fused with all the features. The third part is a prediction part, each point is predicted by using the finally obtained feature layer, and the purpose of effectively segmenting the front background and the target can be achieved relative to the classification of the pixel points.
With reference to fig. 6, the implementation method of the step S3 includes the following steps:
and S31, for any group of visible light and infrared thermal images, manually selecting L (L is greater than or equal to 4) points with obvious corner feature in the two images as matching points. Assuming (u ', v') infrared image coordinates and (u, v) visible image coordinates, affine transformation is performed on the image by using the formula (1):
where H is a 3 × 3 homography transform matrix.
S32, calculating transformation matrix H of K groups of images to obtain more accurate transformation matrix HjJ is 1,2, …, K, and HjThe sum of the values of the elements in the matrix is averaged by HjThe average value of (c) is used as the final affine transformation matrix H.
In the embodiment of the method, 10 groups of visible light images and infrared thermal images which are shot by the inspection robot after the inspection robot starts to perform inspection form a registration image pair, the feature pairs in each group of images are manually selected, 6 pairs of feature pairs are selected from each group of images, an affine transformation matrix of each group of images is calculated according to the feature pairs of each group of images, the finally obtained transformation matrix is an average value H of the 10 groups of image transformation matrices, the matrix is stored and used as the transformation matrix of the inspection, and the registration matrices of the acquired data are subjected to affine transformation by using H.
Referring to fig. 7, the implementation method of the step S4 includes the following steps:
s41, registering the visible light image and the infrared thermal image according to the affine transformation matrix H, transforming the outline of the target area in the visible light image to the infrared thermal image, and setting a certain coordinate of the visible light image as (x)1,y1) The coordinate (x) is transformed by affine transformation as formula (2)1,y1) Transforming into infrared thermal image to obtain transformed coordinate (x)2，y2)
And S42, transforming the contour of the target area in the visible light into the infrared thermal image through affine transformation, thereby obtaining the target area in the infrared thermal image.
The implementation method of the step S5 includes the following steps:
s51, traversing the RGB value of each infrared thermal image in each target area, and retrieving the corresponding infrared thermal image temperature value according to the RGB value;
and S52, automatically detecting the overheating condition of the electric component according to the temperature value in the target area of the electric equipment in the infrared thermal image and the overheating judgment standard of the infrared thermal image.
With reference to fig. 5, the implementation method of step S6 includes the following steps:
setting the size of the high-resolution visible light RGB image as 4m × 4n, the size of the low-resolution infrared thermal image as m × n, and reconstructing the low-resolution infrared thermal image as 4m × 4n, specifically comprising the following steps:
s61, constructing a visible light RGB image multi-scale feature extraction model, wherein the visible light RGB image feature extraction model comprises 4 convolution layers and 1 pooling layer, and obtaining Fc through two convolution layers conv1_ RGB and conv2_ RGB by taking a visible light RGB image as input1Size 4m x 4n, containing 64 profiles, Fc1Is a first scale feature.
S62, output image Fc to conv2_ rgb1Pooling is carried out, Fc is obtained through a pooling layer with the pooling kernel size of 2 multiplied by 2 and the step length of 2, and the specific process is as follows:
Fc＝Maxpooling(Fc1)
where Maxpooling indicates maximum pooling, the size of Fc is 2 m.times.2n.
S63, Fc is obtained after 2 convolutional layers, conv3_ rgb and conv4_ rgb2，Fc2Size 2m × 2n, containing 64 characteristic maps, Fc2Is a second scale feature.
S64, carrying out one-time bilinear interpolation on the RGB infrared thermal image with the size of m multiplied by n to obtain an image after the first interpolation with the size of 2m multiplied by 2n, and carrying out two-time convolution on the interpolated image to obtain Ft1The convolution result Ft1And Fc2Connecting to realize multi-scale one-time fusion to obtain Ft2And the reconstructed image contains more detail information. Specifically, the step S64 is implemented as follows:
s64-1, carrying out first interpolation on the low-resolution infrared thermal image with the size of m multiplied by n by a bilinear interpolation method to obtain an image with the size of 2m multiplied by 2 n.
S64-2, carrying out two-layer convolution on the interpolated image conv1_ t, conv2_ t to obtain Ft1The size is 2m × 2n, and 64 feature maps are included.
S64-3, converting Ft1And Fc2The connection operation, the specific operation can be expressed as:
Ft2＝[Fc2，Ft1]
in the formula: [ Fc ]2，Ft1]Represents the pair Fc2，Ft1Connection operation of, Fc2The method comprises 128 characteristic images, and the reconstructed image comprises more detail information by fusing the interpolated image and the characteristic image with high resolution for the first time.
S65、Ft2Firstly, after convolution conv3_ t, the convolution kernel size is 3 × 3, the step size is 1, and the number of convolution layers with feature map number of 64 is output, and then after convolution conv4_ t, the convolution kernel size is 3 × 3, the step size is 1, and the number of convolution layers with feature map number of 256 is output, so that the feature Ft3 after fusion of Ft1 and Fc2 is obtained. Ft3 is 2m multiplied by 2n in size, thereby realizing the characteristic Fc of visible light image branch extraction2Characterization of the infrared thermal image at a scale of 2m × 2n1The guidance of (2).
S66, pair Ft3Carrying out second bilinear interpolation to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n4。
S67, converting Ft4Convolution conv5_ t, conv6_ t twice to obtain Ft5The size is 4m × 4n, and 64 feature maps are included.
S68, converting Ft5And Fc1The specific operation can be expressed as:
Ft6＝[Fc1,Ft5]
wherein, [ Fc1,Ft5]Represents p-Fc1,Ft5Connection operation of Ft6Contains 128 characteristic maps.
S69、Ft6After one convolution layer conv7_ t, the convolution kernel size is 3 multiplied by 3, the step size is 1, and the feature map is outputThe number of convolutional layers is 64, and then the convolutional layers with the convolutional kernel size of 3 multiplied by 3, the step length of 1 and the number of output feature maps of 1 are passed through one convolutional layer conv8_ t, so as to obtain the final fused feature map Ft7The feature Fc1 for visible light image branch extraction is realized on the infrared thermal image feature Ft on the scale of 4m multiplied by 4n5The guidance of (2).
S610, pair Ft7Performing secondary convolution to obtain Ft8The size is 4m × 4n, and 64 feature maps are included.
S511, carrying out bilinear interpolation on the original infrared thermal image with the size of m multiplied by n to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n0Ft is0And Ft8Adding the three times of the infrared thermal images pixel by pixel to obtain a result after three times of fusion, and setting the reconstructed high-resolution infrared thermal image as HTCan be represented as HT＝Ft0+Ft8，HTThe size is 4m × 4n for high resolution reconstruction results.
In this example, the size of the low-resolution infrared thermal image is 320 × 240, the size of the visible light image is 1280 × 720, the reconstruction multiple is 4, and the size of the reconstructed infrared thermal image is 1280 × 720; the convolution kernel size of each convolution layer of the reconstructed model in fig. 5 is 3 × 3, the step size is 1, and the feature map size is not changed by the convolution process.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. A fault detection method for high-voltage power transmission and transformation line electrical equipment is characterized by comprising the following steps:
acquiring a visible light image of the high-voltage power transmission and transformation line electrical equipment, an infrared thermal image corresponding to the visible light image and an infrared thermal image temperature value;
training a unet network through a visible light image to construct a target segmentation unet model; electrical equipment is segmented from the visible light image through a target segmentation unet model, and target area outline extraction is carried out on the segmented electrical equipment;
establishing K groups of image pairs of the visible light images and the corresponding infrared thermal images, selecting L characteristic pairs in each group of image pairs, and calculating an affine transformation matrix of the image pairs according to the characteristic pairs; wherein L < K and L > -4;
registering the visible light image and the infrared thermal image according to the affine transformation matrix, so that the outline of the electrical equipment in the visible light image is transformed into the infrared thermal image, and the target area of the electrical equipment in the infrared thermal image is positioned;
and detecting whether the electrical equipment has an overheating phenomenon or not according to the temperature value in the target area of the electrical equipment in the infrared thermal image and overheating detection standards of different electrical equipment.
2. The method of detecting a fault in high voltage power transmission and transformation line electrical equipment according to claim 1, further comprising:
and performing multi-scale super-resolution reconstruction on the low-resolution infrared thermal image by taking the high-resolution visible light image as guidance, so that the low-resolution infrared thermal image and the high-resolution visible light image have the same resolution and size.
3. The method for detecting faults of electrical equipment of a high-voltage power transmission and transformation line according to claim 1, wherein the acquiring of the visible light image and the infrared thermal image corresponding to the visible light image of the electrical equipment of the high-voltage power transmission and transformation line specifically comprises:
installing an infrared thermal imager provided with a visible light camera and an infrared thermal camera on the inspection robot;
and aligning the lens of the thermal infrared imager to each part of the aerial high-voltage power transmission and transformation circuit, and simultaneously shooting visible light images of each electrical device on the high-voltage power transmission and transformation circuit and infrared thermal images corresponding to the visible light images.
4. The method for detecting the fault of the electrical equipment of the high-voltage power transmission and transformation line according to claim 1, wherein the constructing of the target segmentation unet model by training the unet network through the visible light image specifically comprises:
dividing visible light RGB image training sample set into labeled sample set DL＝{XL,YLAnd unlabeled sample set DU＝{XU}; wherein, XLFor visible RGB images, YLIs XLA true value label image of the medium electric device;
with labelled sample sets
Training a unet network;
randomly dividing the unlabeled sample set into k sub-data sets, and dividing the sub-data sets
Sequentially input into unet network, and continuously predicting by dividing network
Generating corresponding segmentation results
Form a
Wherein
I is the training sample of the ith input, i is 1,2, …, k;
will be provided with
Adding to the current round of training
In the training set, a new training set is formed
For the next round of training;
and (3) using the updated training set to train the unet network again, inputting a next batch of unlabeled samples after the training is finished, predicting to generate pseudo labels until all the unlabeled sample sets generate the pseudo labels, and stopping to obtain a trained unet model.
5. The method for detecting a fault in an electrical device of a high-voltage power transmission and transformation line according to claim 1, wherein the calculating an affine transformation matrix of the pair of images from the pair of features specifically comprises:
assuming (u ', v') infrared image coordinates and (u, v) visible image coordinates, affine transformation is performed on the image using the following equation:
where H is a 3 × 3 homography transform matrix.
6. The method of detecting a fault in high voltage power transmission and transformation line electrical equipment according to claim 5, further comprising:
computing a transformation matrix H for K sets of imagesjJ is 1,2, …, K, and HjThe sum of the values of the elements in the matrix is averaged by HjThe average value of (c) is used as the final affine transformation matrix H.
7. The method for detecting faults of electrical equipment of a high-voltage power transmission and transformation line according to claim 5 or 6, wherein the registering of the visible light image and the infrared thermal image according to the affine transformation matrix specifically comprises:
let a certain point coordinate of the visible light image be expressed as (x)1,y1) Using affine transformation of the coordinates (x)1,y1) Transformation ofInto infrared thermal images, resulting in transformed coordinates (x)2,y2) (ii) a The affine transformation is as follows:
8. the method for fault detection of electrical equipment of high voltage transmission and transformation line according to claim 2, wherein the multi-scale super-resolution reconstruction of the low resolution infrared thermal image using the high resolution visible light image as a guide specifically comprises:
inputting a high-resolution visible light RGB image with the size of 4m multiplied by 4n into a multi-scale feature extraction model, and obtaining Fc through two convolution layers conv1_ RGB and conv2_ RGB14m x 4n, comprising 64 profiles, Fc1Is a first scale feature;
outputting an image Fc to conv2_ rgb1Pooling is carried out, Fc is obtained through a pooling layer with the pooling kernel size of 2 multiplied by 2 and the step length of 2, and the specific process is as follows: fc ═ Maxpooling (Fc)1) (ii) a Wherein Maxpooling denotes maximum pooling, the size of Fc is 2 m.times.2n;
fc was obtained by passing Fc through 2 convolutional layers, conv3_ rgb, conv4_ rgb2，Fc2Size 2m × 2n, containing 64 characteristic maps, Fc2Is a second scale feature;
carrying out bilinear interpolation on the RGB infrared thermal image with the size of mxn once to obtain an image after the first interpolation with the size of 2 mx 2n, and carrying out convolution twice on the interpolated image to obtain Ft1The convolution result Ft1And Fc2Connecting to realize multi-scale one-time fusion to obtain Ft2；
Ft2Firstly, after convolution conv3_ t, the convolution kernel size is 3 multiplied by 3, the step size is 1, the number of convolution layers of which the output feature map number is 64 is obtained, and then after convolution conv4_ t, the convolution kernel size is 3 multiplied by 3, the step size is 1, the number of convolution layers of which the output feature map number is 256 is obtained, the feature Ft3 after fusion of Ft1 and Fc2 is obtained, and the size of Ft3 is 2m multiplied by 2n, so that the visible light image branch extraction is realizedTaken characteristic Fc2Characterization of the infrared thermal image at a scale of 2m × 2n1The guidance of (2);
for Ft3Carrying out second bilinear interpolation to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n4；
Will Ft4Convolution conv5_ t, conv6_ t twice to obtain Ft5The size is 4m multiplied by 4n, and 64 characteristic graphs are included;
will Ft5And Fc1The connection, the specific operation is represented as: ft6＝[Fc1,Ft5](ii) a Wherein, [ Fc1,Ft5]Represents p-Fc1,Ft5Connection operation of Ft6Comprises 128 characteristic graphs;
Ft6after passing through one convolutional layer conv7_ t, the convolutional kernel size is 3 multiplied by 3, the step size is 1, convolutional layers with the number of output feature maps of 64 are output, and after passing through one convolutional layer conv8_ t, the convolutional kernel size is 3 multiplied by 3, the step size is 1, and the number of output feature maps is 1, so that the finally fused feature map Ft is obtained7The feature Fc1 for visible light image branch extraction is realized on the infrared thermal image feature Ft on the scale of 4m multiplied by 4n5The guidance of (2);
to Ft7Performing secondary convolution to obtain Ft8The size is 4m multiplied by 4n, and 64 characteristic graphs are included;
carrying out bilinear interpolation on the original infrared thermal image with the size of m multiplied by n to obtain a pseudo high-resolution image Ft with the size of 4m multiplied by 4n0Ft is0And Ft8Adding the three times of the infrared thermal images pixel by pixel to obtain a result after three times of fusion, and setting the reconstructed high-resolution infrared thermal image as HTIs shown as HT＝Ft0+Ft8，HTThe size is 4m × 4n for high resolution reconstruction results.
9. The method according to claim 8, wherein the Ft is obtained by performing multi-scale one-time fusion2The method specifically comprises the following steps:
carrying out first interpolation on the low-resolution infrared thermal image with the size of mxn by adopting a bilinear interpolation method to obtain an image with the size of 2 mx 2 n;
carrying out two-layer convolution on the interpolated image conv1_ t, conv2_ t to obtain Ft1The size is 2m multiplied by 2n, and 64 characteristic graphs are included;
will Ft1And Fc2And connecting operation, specifically expressing as follows: ft2＝[Fc2，Ft1](ii) a In the formula: [ Fc ]2，Ft1]Represents p-Fc2，Ft1Connection operation of, Fc2Comprising 128 feature maps, by first fusing the interpolated image with a high resolution feature image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210086112.8A CN114494186B (en) | 2022-01-25 | 2022-01-25 | Fault detection method for high-voltage power transmission and transformation line electrical equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210086112.8A CN114494186B (en) | 2022-01-25 | 2022-01-25 | Fault detection method for high-voltage power transmission and transformation line electrical equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114494186A true CN114494186A (en) | 2022-05-13 |
| CN114494186B CN114494186B (en) | 2022-11-08 |
Family
ID=81474607
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210086112.8A Active CN114494186B (en) | 2022-01-25 | 2022-01-25 | Fault detection method for high-voltage power transmission and transformation line electrical equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494186B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197197A (en) * | 2023-10-27 | 2023-12-08 | 安徽容知日新科技股份有限公司 | Target temperature detection method and device, electronic equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170010543A (en) * | 2015-07-20 | 2017-02-01 | 서울대학교산학협력단 | Object recognition method and device using visible light sensor and thermal imagery sensor |
| CN109285141A (en) * | 2018-07-27 | 2019-01-29 | 仲恺农业工程学院 | A kind of real-time detection method of plant protection drone spraying effect |
| CN111696075A (en) * | 2020-04-30 | 2020-09-22 | 航天图景（北京）科技有限公司 | Intelligent fan blade defect detection method based on double-spectrum image |
| CN112132789A (en) * | 2020-08-30 | 2020-12-25 | 南京理工大学 | Pantograph online detection device and method based on cascade neural network |
| CN113160058A (en) * | 2021-04-28 | 2021-07-23 | 成都东方天呈智能科技有限公司 | Thermal imaging super-resolution reconstruction method combining segmentation map and storage medium |
| CN113222825A (en) * | 2021-06-03 | 2021-08-06 | 北京理工大学 | Infrared image super-resolution reconstruction method based on visible light image training and application |
| CN113920097A (en) * | 2021-10-14 | 2022-01-11 | 国网河南省电力公司电力科学研究院 | Power equipment state detection method and system based on multi-source image |
-
2022
- 2022-01-25 CN CN202210086112.8A patent/CN114494186B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20170010543A (en) * | 2015-07-20 | 2017-02-01 | 서울대학교산학협력단 | Object recognition method and device using visible light sensor and thermal imagery sensor |
| CN109285141A (en) * | 2018-07-27 | 2019-01-29 | 仲恺农业工程学院 | A kind of real-time detection method of plant protection drone spraying effect |
| CN111696075A (en) * | 2020-04-30 | 2020-09-22 | 航天图景（北京）科技有限公司 | Intelligent fan blade defect detection method based on double-spectrum image |
| CN112132789A (en) * | 2020-08-30 | 2020-12-25 | 南京理工大学 | Pantograph online detection device and method based on cascade neural network |
| CN113160058A (en) * | 2021-04-28 | 2021-07-23 | 成都东方天呈智能科技有限公司 | Thermal imaging super-resolution reconstruction method combining segmentation map and storage medium |
| CN113222825A (en) * | 2021-06-03 | 2021-08-06 | 北京理工大学 | Infrared image super-resolution reconstruction method based on visible light image training and application |
| CN113920097A (en) * | 2021-10-14 | 2022-01-11 | 国网河南省电力公司电力科学研究院 | Power equipment state detection method and system based on multi-source image |
Non-Patent Citations (4)
| Title |
|---|
| HONEY GUPTA ET.AL: "Toward Unaligned Guided Thermal Super-Resolution", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| 于淑侠 等: "彩色图像多尺度引导的深度图像超分辨率重建", 《光电工程》 * |
| 先永利: "基于眼底相机的双波长视网膜血氧仪若干问题研究及应用", 《中国博士学位论文全文数据库 工程科技II辑》 * |
| 刘清清 等: "基于3D scSE-UNet 的肝脏CT 图像半监督学习分割方法", 《浙江大学学报（工学版）》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117197197A (en) * | 2023-10-27 | 2023-12-08 | 安徽容知日新科技股份有限公司 | Target temperature detection method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114494186B (en) | 2022-11-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108846418B (en) | Cable equipment temperature abnormity positioning and identifying method | |
| CN112200178B (en) | Transformer substation insulator infrared image detection method based on artificial intelligence | |
| Han et al. | A smart thermography camera and application in the diagnosis of electrical equipment | |
| CN114494185B (en) | Electrical equipment fault detection method based on RGB-T multi-scale feature fusion | |
| CN114494186B (en) | Fault detection method for high-voltage power transmission and transformation line electrical equipment | |
| Huang et al. | Research on real-time disconnector state evaluation method based on multi-source images | |
| CN116485802B (en) | Insulator flashover defect detection method, device, equipment and storage medium | |
| CN113269744A (en) | Infrared intelligent identification method and system applied to defects of power transmission line equipment | |
| CN112419243A (en) | Power distribution room equipment fault identification method based on infrared image analysis | |
| CN112465797A (en) | Intelligent diagnosis method and system for thermal state of cable terminal | |
| CN114325274B (en) | Converter transformer valve side sleeve partial discharge detection system and method | |
| CN114882203B (en) | Image super-resolution reconstruction method for power grid inspection robot | |
| Tang et al. | Fault diagnosis of the external insulation infrared images based on Mask Region convolutional neural network and perceptual hash joint algorithm | |
| CN115410154A (en) | Method for identifying thermal fault of electrical equipment of wind power engine room | |
| CN111429411B (en) | X-ray defect image sample generation method for carbon fiber composite core wire | |
| Xu et al. | Autonomous diagnosis of overheating defects in cable accessories based on Faster RCNN and Mean-Shift algorithm | |
| CN113486873A (en) | Transformer substation equipment inspection method and system based on big data and artificial intelligence | |
| Wang et al. | Substation Equipment Defect Detection based on Temporal-spatial Similarity Calculation | |
| Yan et al. | Bimodal-based object detection and instance segmentation models for substation equipments | |
| CN117994253B (en) | High-voltage distribution line ground fault identification method | |
| Luo et al. | RETRACTED CHAPTER: Power Equipment Defect Detection Algorithm Based on Deep Learning | |
| Zhang et al. | Inspect Defect of Power Equipment via Deep Learning Method. | |
| CN117496133B (en) | Closed bus R-CNN temperature fault monitoring method based on multi-mode data | |
| Yilin et al. | Substation Equipment Fault Identification Based on Infrared Image Analysis | |
| Li et al. | Fault diagnosis method for arrester in infrared images based on improved U-Net |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |