CN116403057A - Power transmission line inspection method and system based on multi-source image fusion - Google Patents
Power transmission line inspection method and system based on multi-source image fusion Download PDFInfo
- Publication number
- CN116403057A CN116403057A CN202310677312.5A CN202310677312A CN116403057A CN 116403057 A CN116403057 A CN 116403057A CN 202310677312 A CN202310677312 A CN 202310677312A CN 116403057 A CN116403057 A CN 116403057A
- Authority
- CN
- China
- Prior art keywords
- frequency component
- image
- low
- fusion
- obtaining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 103
- 230000005540 biological transmission Effects 0.000 title claims abstract description 50
- 238000007689 inspection Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 27
- 239000012212 insulator Substances 0.000 claims abstract description 48
- 230000000007 visual effect Effects 0.000 claims abstract description 26
- 238000013507 mapping Methods 0.000 claims abstract description 23
- 230000002159 abnormal effect Effects 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 13
- 238000007499 fusion processing Methods 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 8
- 238000013136 deep learning model Methods 0.000 claims description 7
- 238000003062 neural network model Methods 0.000 description 4
- 210000002569 neuron Anatomy 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000001431 Psychomotor Agitation Diseases 0.000 description 1
- 206010038743 Restlessness Diseases 0.000 description 1
- 206010003549 asthenia Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 239000000725 suspension Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Economics (AREA)
- Biophysics (AREA)
- Water Supply & Treatment (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Public Health (AREA)
- Computational Linguistics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a transmission line inspection method and system based on multi-source image fusion, and belongs to the technical field of power inspection. The method comprises the following steps: respectively decomposing the infrared image and the visible light image to respectively obtain a high-frequency image and a low-frequency component; obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and obtaining fused low-frequency components; according to the high-frequency components of the infrared image and the high-frequency components of the visible light image, obtaining corresponding visual mapping of each high-frequency component, and further obtaining corresponding weight of each high-frequency component to obtain the fused high-frequency component; merging the fused high-frequency component and low-frequency component to obtain a fusion result image; carrying out abnormal identification results of the insulator string according to the fusion result image, and obtaining abnormal discharge identification results of the insulators of the power transmission line according to the ultraviolet images of the identified polluted insulators; the invention improves the recognition efficiency and the recognition precision.
Description
Technical Field
The invention relates to the technical field of power inspection, in particular to a power transmission line inspection method and system based on multi-source image fusion.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
An insulator string refers to an assembly of two or more insulator elements combined together, with flexible suspension wires. The insulator string is provided with a protection device which is required by fixing and running and is used for hanging wires and insulating the wires from the tower and the ground, and the insulator string is easy to generate pollution, cracks, breakage or foreign matters in a long-time passing process, so that the performance of the insulator string is reduced.
At present, an unmanned aerial vehicle inspection mode is generally adopted to shoot an insulator string to determine the running condition of the insulator string, and fusion recognition is carried out by collecting visible light images and infrared light images of an inspection site, so that the inventor finds that the following problems exist in the current fusion method:
(1) The existing fusion algorithm tends to pursue better visual effect and higher evaluation index, network hierarchy is deeper and network hierarchy is more complex, fusion method is more complex, fusion speed is slower and slower, and fewer consideration is given to the practicability and timeliness of the method;
(2) Because of the different imaging modes of the infrared image and the visible light image, the twin network is adopted to respectively extract the characteristics of the infrared image and the visible light image at present, the characteristics of the images are extracted in a biased way, and the problem of energy characteristic loss of the infrared image exists;
(3) In the identification process of carrying out pollution flashover discharge, the insulator string is generally and directly identified based on ultraviolet images, so that external interference factors are large, and whether the flashover discharge caused by the pollution of the insulator is difficult to identify.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides the transmission line inspection method and the transmission line inspection system based on multi-source image fusion, which effectively avoid the problem of energy characteristic loss in the fusion process, reduce the application of a large-scale neural network model and improve the recognition efficiency and the recognition precision.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a transmission line inspection method based on multi-source image fusion.
A transmission line inspection method based on multi-source image fusion comprises the following steps:
respectively decomposing the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and fusing the low-frequency components of the infrared image and the low-frequency components of the visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components;
obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
and merging the fused high-frequency component and low-frequency component to obtain a fusion result image, obtaining a pollution recognition result of the insulator string of the power transmission line according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the recognized pollution insulator, and obtaining an abnormal discharge recognition result of the insulator of the power transmission line according to the ultraviolet image of the recognized pollution insulator.
As a further limitation of the first aspect of the present invention, obtaining the low frequency component fusion weight according to the pixel intensity information of the infrared image includes:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
As a further limitation of the first aspect of the present invention, fusing the low frequency component of the infrared image with the low frequency component of the visible light image according to the low frequency component fusion weight comprises:
the weight of the low frequency component of the infrared image is a, and the weight of the low frequency component of the visible light image is 1-a.
As a further limitation of the first aspect of the present invention, obtaining a corresponding visual map of each high frequency component, and further obtaining a weight corresponding to each high frequency component, including:
if the visual mapping of the high frequency component of the infrared image is X1 and the visual mapping of the high frequency component of the visible light image is X2, the weight corresponding to the high frequency component of the infrared image is: the weight corresponding to the high-frequency component of the visible light image is X1/(X1 + X2): x2/(x1+x2).
As a further limitation of the first aspect of the invention, the loss function L of the image fusion process is a similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Is a fusion of (2);
wherein,,
，/>
and->
Is a super parameter.
The invention provides a transmission line inspection system based on multi-source image fusion.
A transmission line inspection system based on multi-source image fusion comprises:
an image decomposition module configured to: respectively decomposing the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
a low frequency fusion module configured to: obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and fusing the low-frequency components of the infrared image and the low-frequency components of the visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components;
a high frequency fusion module configured to: obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
a fusion identification module configured to: and merging the fused high-frequency component and low-frequency component to obtain a fusion result image, obtaining a pollution recognition result of the insulator string of the power transmission line according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the recognized pollution insulator, and obtaining an abnormal discharge recognition result of the insulator of the power transmission line according to the ultraviolet image of the recognized pollution insulator.
As a further limitation of the second aspect of the present invention, the low-frequency component fusion weight is obtained according to the pixel intensity information of the infrared image in the low-frequency fusion module, and the method includes:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
As a further limitation of the second aspect of the present invention, fusing the low frequency component of the infrared image with the low frequency component of the visible light image according to the low frequency component fusion weight comprises:
the weight of the low frequency component of the infrared image is a, and the weight of the low frequency component of the visible light image is 1-a.
As a further limitation of the second aspect of the present invention, the obtaining, in the high-frequency fusion module, a corresponding visual map of each high-frequency component, and further obtaining a weight corresponding to each high-frequency component, includes:
if the visual mapping of the high frequency component of the infrared image is X1 and the visual mapping of the high frequency component of the visible light image is X2, the weight corresponding to the high frequency component of the infrared image is: the weight corresponding to the high-frequency component of the visible light image is X1/(X1 + X2): x2/(x1+x2).
As a further limitation of the second aspect of the invention, the loss function L of the image fusion process is a similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Is a fusion of (2);
wherein,,
，/>
and->
Is a super parameter.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention creatively provides a transmission line inspection method and a transmission line inspection system based on multi-source image fusion, which are used for obtaining low-frequency component fusion weights according to pixel intensity information of an infrared image, and fusing the low-frequency component of the infrared image with the low-frequency component of a visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components, so that the problem of energy characteristic loss in the fusion process is effectively avoided, the application of a large-scale neural network model is reduced, and the recognition efficiency and recognition precision are improved.
2. The invention creatively provides a transmission line inspection method and a transmission line inspection system based on multi-source image fusion, wherein a loss function L in an image fusion process is similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Wherein l=l 1 +αL 2 + βL 3 Alpha and beta are super parameters, and heat radiation information of the infrared image and texture information of the visible light image are reserved to the maximum extent.
3. The invention creatively provides a transmission line inspection method and a transmission line inspection system based on multi-source image fusion, which are used for acquiring an ultraviolet image of an identified polluted insulator, obtaining an abnormal discharge identification result of the transmission line insulator according to the ultraviolet image of the identified polluted insulator, avoiding the influence of external interference factors on abnormal discharge identification, and being capable of more rapidly identifying flashover discharge caused by the pollution of the insulator.
Additional aspects of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is a schematic flow chart of a transmission line inspection method based on multi-source image fusion provided in embodiment 1 of the present invention;
fig. 2 is a schematic flow chart of a transmission line inspection system based on multi-source image fusion provided in embodiment 2 of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1:
as shown in fig. 1, embodiment 1 of the present invention provides a transmission line inspection method based on multi-source image fusion, which includes the following steps:
s1: respectively carrying out Laplacian pyramid decomposition on the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
s2: obtaining a low-frequency component fusion weight according to the pixel intensity information of the infrared image, and fusing the low-frequency component of the infrared image with the low-frequency component of the visible light image according to the low-frequency component fusion weight to obtain a fused low-frequency component;
s3: obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
s4: merging the fused high-frequency component and low-frequency component, and then executing inverse Laplacian transformation to obtain a fusion result image;
s5: obtaining a power transmission line insulator string pollution identification result according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the identified pollution insulator, and obtaining a power transmission line insulator abnormal discharge identification result according to the ultraviolet image of the identified pollution insulator.
Specifically, the laplacian pyramid decomposition includes: firstly, carrying out low-pass wave on an original image, then, carrying out downsampling to obtain a low-frequency component, namely an approximate component of the original image, carrying out upsampling on the low-frequency component, carrying out high-pass filtering on the component obtained by upsampling, carrying out difference between the component obtained after high-pass and the original image, and finally, obtaining a high-restlessness component obtained after Laplace decomposition.
Specifically, obtaining a low-frequency component fusion weight according to pixel intensity information of an infrared image includes:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
More specifically, it includes:
the absolute value taken from each pixel of the decomposed low frequency part in the infrared image is as follows:
wherein T reflects significant infrared characteristic distribution, T1 represents the highest level of decomposition information, i.e. low frequency information, and for a point x in the image, a larger value of T (x) results in a larger pixel intensity value and significant infrared characteristic information, and normalizing T to obtain normalized variable M.
A nonlinear transformation function is introduced to further adjust to control the relative effective information of the infrared image and the visible light image in the fused image:
in the method, in the process of the invention,
the range of (0, 1) represents the argument of the function, the function parameter +.>
Greater than 0, when->
In the event of an increase in the number of the cells,
the shape of the curve becomes steeper and the corresponding nonlinear transformation gradually increases. Thus, by adjusting->
To control the amount of infrared information in the combined result, the final low frequency information fusion weight can be expressed as:
the weight of the low-frequency component of the infrared image is a, the weight of the low-frequency component of the visible light image is 1-a, the low-frequency component of the infrared image is multiplied by a, the low-frequency component of the visible light image is multiplied by 1-a, and the two components are added to obtain a fusion result of the low-frequency component;
obtaining a corresponding visual mapping of each high-frequency component, and further obtaining a weight corresponding to each high-frequency component, including:
the visual mapping of the high-frequency component of the infrared image is X1, and the visual mapping of the high-frequency component of the visible light image is X2, the weight corresponding to the high-frequency component of the infrared image is: b=x1/(x1+x2), the weight corresponding to the high-frequency component of the visible light image is: c=x2/(x1+x2).
More specifically, b is multiplied by the high-frequency component of the infrared image, c is multiplied by the high-frequency component of the visible light image, and the two components are added to obtain a fusion result of the high-frequency component.
Optionally, the loss function L of the image fusion process is a similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Continuously carrying out weight and decomposition parameter correction to achieve the optimal fusion effect;
wherein,,
，/>
and->
Is a super parameter.
The structural similarity measure Y simulates distortion according to the similarity of brightness, contrast and structural information, and is selected to constrain the structural similarity between the input source image pairs Q1 and Q2 and the fused image F, with a structural similarity loss L in this embodiment 1 The calculation formula of (2) is as follows:
L 1 =（（1-Y（F，Q1））+（1-Y（F，Q1）））*0.5（4）
the intensity loss constraint fusion image maintains an intensity distribution similar to the source image, intensity loss L 2 The calculation formula of (2) is as follows:
wherein H and W represent the height and width of the image, respectively,
representing pixel error between images x and y, < >>
Is a positive parameter that controls the trade-off between the two terms.
Gradient loss forces the fusion image to be richTexture detail information, gradient penaltyL 3 The calculation formula of (2) is as follows:
where H and W represent the height and width of the image, respectively,
representing the Sobel gradient operator, ++>
Represents the loss of edge information between images x and y, of->
The method is used for adjusting the proportion of different modal gradient information in the fusion image.
Specifically, the deep learning model adopts a pre-trained BP neural network model, the BP neural network model comprises an input layer, an implied layer and an output layer, the implied layer uses a log sig function, the output layer uses a purelin function, and the number of neurons of the implied layer is as follows:
where n is the number of neurons in the hidden layer,n i for the number of neurons of the input layer,n 0 for the number of neurons of the output layer, a belongs to the set [1,10]。
Taking the pollution level of the identified insulator as an example, after shooting of the insulator image (including the visible light image and the infrared light image) is completed each time, counting all the pollution levels (manual labeling) of the insulator according to the standard, acquiring and recording the infrared thermal image under each pollution level according to the mode, collecting 500 groups of insulator fusion images of five pollution levels according to the description, wherein each pollution level corresponds to 100 groups, and 80 groups are selected as training groups and 20 groups are selected as test groups to train the BP neural network.
Alternatively, the loss function of the BP neural network may be:
wherein, T (i, j) and Q (i, j) are pixel values of the preprocessed image and the standard image at the (i, j) position, A is the maximum value of i, and B is the maximum value of j.
It can be appreciated that in other implementations, corresponding levels may be set for cracks, breakage, foreign objects, or the like, and the fused artificial annotation images are used for identification (each identification uses a different training model), and those skilled in the art may select according to specific working conditions, which will not be described herein.
After the identification result of the pollution grade of the insulator string is obtained, acquiring ultraviolet images of the insulator string with larger pollution grade or all the insulator strings with the pollution grade, and identifying the pollution discharge condition according to the ultraviolet images, wherein the identification method can adopt the existing scheme;
the ultraviolet image is adopted to identify the discharge condition, so that whether the insulator string with the larger pollution level has the larger discharge condition can be judged, and the corresponding relation between the pollution level and the discharge condition is generated according to the matching of the discharge condition and the pollution level, so that more accurate identification is realized.
Example 2:
as shown in fig. 2, embodiment 2 of the present invention provides a transmission line inspection system based on multi-source image fusion, including:
an image decomposition module configured to: respectively decomposing the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
a low frequency fusion module configured to: obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and fusing the low-frequency components of the infrared image and the low-frequency components of the visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components;
a high frequency fusion module configured to: obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
a fusion identification module configured to: and merging the fused high-frequency component and low-frequency component to obtain a fusion result image, obtaining a pollution recognition result of the insulator string of the power transmission line according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the recognized pollution insulator, and obtaining an abnormal discharge recognition result of the insulator of the power transmission line according to the ultraviolet image of the recognized pollution insulator.
In this embodiment, in the low-frequency fusion module, low-frequency component fusion weights are obtained according to pixel intensity information of an infrared image, including:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
In this embodiment, fusing the low-frequency component of the infrared image with the low-frequency component of the visible light image according to the low-frequency component fusion weight includes:
the weight of the low frequency component of the infrared image is a, and the weight of the low frequency component of the visible light image is 1-a.
In this embodiment, the high-frequency fusion module obtains a corresponding visual mapping of each high-frequency component, and further obtains a weight corresponding to each high-frequency component, including:
if the visual mapping of the high frequency component of the infrared image is X1 and the visual mapping of the high frequency component of the visible light image is X2, the weight corresponding to the high frequency component of the infrared image is: the weight corresponding to the high-frequency component of the visible light image is X1/(X1 + X2): x2/(x1+x2).
Optionally, the loss function L of the image fusion process is a similarity loss L 1 Loss of strengthLoss of L 2 And gradient loss L 3 Is a fusion of (2);
wherein l=l 1 +αL 2 + βL 3 Alpha and beta are hyper-parameters.
The specific working method of each module is the same as that provided in embodiment 1, and will not be described here again.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The transmission line inspection method based on multi-source image fusion is characterized by comprising the following steps of:
respectively decomposing the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and fusing the low-frequency components of the infrared image and the low-frequency components of the visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components;
obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
and merging the fused high-frequency component and low-frequency component to obtain a fusion result image, obtaining a pollution recognition result of the insulator string of the power transmission line according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the recognized pollution insulator, and obtaining an abnormal discharge recognition result of the insulator of the power transmission line according to the ultraviolet image of the recognized pollution insulator.
2. The transmission line inspection method based on multi-source image fusion according to claim 1, wherein,
obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image comprises the following steps:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
3. The transmission line inspection method based on multi-source image fusion according to claim 1 or 2, characterized in that,
fusing the low frequency component of the infrared image with the low frequency component of the visible light image according to the low frequency component fusion weight, comprising:
the weight of the low frequency component of the infrared image is a, and the weight of the low frequency component of the visible light image is 1-a.
4. The transmission line inspection method based on multi-source image fusion according to claim 1, wherein,
obtaining a corresponding visual mapping of each high-frequency component, and further obtaining a weight corresponding to each high-frequency component, including:
if the visual mapping of the high frequency component of the infrared image is X1 and the visual mapping of the high frequency component of the visible light image is X2, the weight corresponding to the high frequency component of the infrared image is: the weight corresponding to the high-frequency component of the visible light image is X1/(X1 + X2): x2/(x1+x2).
5. The transmission line inspection method based on multi-source image fusion according to claim 1, wherein,
the loss function L of the image fusion process is the similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Is a fusion of (2);
where l=l1+αl2+βl3, α and β are hyper-parameters.
6. A transmission line inspection system based on multi-source image fusion is characterized by comprising:
an image decomposition module configured to: respectively decomposing the obtained infrared image and visible light image of the power transmission line to obtain a high-frequency component and a low-frequency component of the infrared image and a high-frequency image and a low-frequency component of the visible light image;
a low frequency fusion module configured to: obtaining low-frequency component fusion weights according to pixel intensity information of the infrared image, and fusing the low-frequency components of the infrared image and the low-frequency components of the visible light image according to the low-frequency component fusion weights to obtain fused low-frequency components;
a high frequency fusion module configured to: obtaining corresponding visual mapping of each high-frequency component according to the high-frequency component of the infrared image and the high-frequency component of the visible light image, further obtaining weight corresponding to each high-frequency component, and fusing the high-frequency components according to the weight corresponding to each high-frequency component to obtain fused high-frequency components;
a fusion identification module configured to: and merging the fused high-frequency component and low-frequency component to obtain a fusion result image, obtaining a pollution recognition result of the insulator string of the power transmission line according to the fusion result image and the deep learning model, obtaining an ultraviolet image of the recognized pollution insulator, and obtaining an abnormal discharge recognition result of the insulator of the power transmission line according to the ultraviolet image of the recognized pollution insulator.
7. The transmission line inspection system based on multi-source image fusion according to claim 6, wherein,
in the low-frequency fusion module, obtaining low-frequency component fusion weight according to pixel intensity information of the infrared image, including:
and calculating the absolute value of each pixel of the low-frequency component of the infrared image, normalizing, and obtaining the low-frequency component fusion weight according to the normalized result.
8. The transmission line inspection system based on multi-source image fusion according to claim 6 or 7, wherein,
fusing the low frequency component of the infrared image with the low frequency component of the visible light image according to the low frequency component fusion weight, comprising:
the weight of the low frequency component of the infrared image is a, and the weight of the low frequency component of the visible light image is 1-a.
9. The transmission line inspection system based on multi-source image fusion according to claim 6, wherein,
in the high-frequency fusion module, obtaining a corresponding visual mapping of each high-frequency component, and further obtaining a weight corresponding to each high-frequency component, including:
if the visual mapping of the high frequency component of the infrared image is X1 and the visual mapping of the high frequency component of the visible light image is X2, the weight corresponding to the high frequency component of the infrared image is: the weight corresponding to the high-frequency component of the visible light image is X1/(X1 + X2): x2/(x1+x2).
10. The transmission line inspection system based on multi-source image fusion according to claim 6, wherein,
the loss function L of the image fusion process is the similarity loss L 1 Loss of strength L 2 And gradient loss L 3 Is a fusion of (2);
where l=l1+αl2+βl3, α and β are hyper-parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310677312.5A CN116403057B (en) | 2023-06-09 | 2023-06-09 | Power transmission line inspection method and system based on multi-source image fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310677312.5A CN116403057B (en) | 2023-06-09 | 2023-06-09 | Power transmission line inspection method and system based on multi-source image fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116403057A true CN116403057A (en) | 2023-07-07 |
| CN116403057B CN116403057B (en) | 2023-08-18 |
Family
ID=87010933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310677312.5A Active CN116403057B (en) | 2023-06-09 | 2023-06-09 | Power transmission line inspection method and system based on multi-source image fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116403057B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110182495A1 (en) * | 2010-01-26 | 2011-07-28 | General Electric Company | System and method for automatic defect recognition of an inspection image |
| JP2016032289A (en) * | 2014-07-25 | 2016-03-07 | 日本電気株式会社 | Image synthesis system, image synthesis method, image synthesis program |
| CN107451984A (en) * | 2017-07-27 | 2017-12-08 | 桂林电子科技大学 | A kind of infrared and visual image fusion algorithm based on mixing multiscale analysis |
| CN107491781A (en) * | 2017-07-21 | 2017-12-19 | 国家电网公司 | A kind of crusing robot visible ray and infrared sensor data fusion method |
| CN107507172A (en) * | 2017-08-08 | 2017-12-22 | 国网上海市电力公司 | Merge the extra high voltage line insulator chain deep learning recognition methods of infrared visible ray |
| US20180227509A1 (en) * | 2015-08-05 | 2018-08-09 | Wuhan Guide Infrared Co., Ltd. | Visible light image and infrared image fusion processing system and fusion method |
| CN110197231A (en) * | 2019-06-04 | 2019-09-03 | 南京华格信息技术有限公司 | The bird feelings detecting devices merged based on visible light and infrared light image and recognition methods |
| US20190318463A1 (en) * | 2016-12-27 | 2019-10-17 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for fusing infrared image and visible light image |
| US20190331301A1 (en) * | 2016-12-30 | 2019-10-31 | Du Yuchuan | Method for leakage detection of underground pipeline corridor based on dynamic infrared thermal image processing |
| CN110443776A (en) * | 2019-08-07 | 2019-11-12 | 中国南方电网有限责任公司超高压输电公司天生桥局 | A kind of Registration of Measuring Data fusion method based on unmanned plane gondola |
| CN112017139A (en) * | 2020-09-14 | 2020-12-01 | 南昌航空大学 | Infrared and visible light image perception fusion method |
| CN112233074A (en) * | 2020-09-30 | 2021-01-15 | 国网山西省电力公司大同供电公司 | Power failure detection method based on visible light and infrared fusion image |
| US20220044375A1 (en) * | 2019-12-17 | 2022-02-10 | Dalian University Of Technology | Saliency Map Enhancement-Based Infrared and Visible Light Fusion Method |
| CN114881905A (en) * | 2022-06-21 | 2022-08-09 | 西北工业大学 | Processing method for fusing infrared image and visible light image based on wavelet transformation |
| US20220292658A1 (en) * | 2019-10-21 | 2022-09-15 | Zhejiang Uniview Technologies Co., Ltd. | Image fusion method and apparatus, storage medium, and electronic device |
| CN115471723A (en) * | 2022-09-23 | 2022-12-13 | 安徽优航遥感信息技术有限公司 | Substation unmanned aerial vehicle inspection method based on infrared and visible light image fusion |
| CN115880221A (en) * | 2022-10-31 | 2023-03-31 | 国网福建省电力有限公司 | Power grid line defect analysis method integrating visible light and infrared image processing |
-
2023
- 2023-06-09 CN CN202310677312.5A patent/CN116403057B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110182495A1 (en) * | 2010-01-26 | 2011-07-28 | General Electric Company | System and method for automatic defect recognition of an inspection image |
| JP2016032289A (en) * | 2014-07-25 | 2016-03-07 | 日本電気株式会社 | Image synthesis system, image synthesis method, image synthesis program |
| US20180227509A1 (en) * | 2015-08-05 | 2018-08-09 | Wuhan Guide Infrared Co., Ltd. | Visible light image and infrared image fusion processing system and fusion method |
| US20190318463A1 (en) * | 2016-12-27 | 2019-10-17 | Zhejiang Dahua Technology Co., Ltd. | Systems and methods for fusing infrared image and visible light image |
| US20190331301A1 (en) * | 2016-12-30 | 2019-10-31 | Du Yuchuan | Method for leakage detection of underground pipeline corridor based on dynamic infrared thermal image processing |
| CN107491781A (en) * | 2017-07-21 | 2017-12-19 | 国家电网公司 | A kind of crusing robot visible ray and infrared sensor data fusion method |
| CN107451984A (en) * | 2017-07-27 | 2017-12-08 | 桂林电子科技大学 | A kind of infrared and visual image fusion algorithm based on mixing multiscale analysis |
| CN107507172A (en) * | 2017-08-08 | 2017-12-22 | 国网上海市电力公司 | Merge the extra high voltage line insulator chain deep learning recognition methods of infrared visible ray |
| CN110197231A (en) * | 2019-06-04 | 2019-09-03 | 南京华格信息技术有限公司 | The bird feelings detecting devices merged based on visible light and infrared light image and recognition methods |
| CN110443776A (en) * | 2019-08-07 | 2019-11-12 | 中国南方电网有限责任公司超高压输电公司天生桥局 | A kind of Registration of Measuring Data fusion method based on unmanned plane gondola |
| US20220292658A1 (en) * | 2019-10-21 | 2022-09-15 | Zhejiang Uniview Technologies Co., Ltd. | Image fusion method and apparatus, storage medium, and electronic device |
| US20220044375A1 (en) * | 2019-12-17 | 2022-02-10 | Dalian University Of Technology | Saliency Map Enhancement-Based Infrared and Visible Light Fusion Method |
| CN112017139A (en) * | 2020-09-14 | 2020-12-01 | 南昌航空大学 | Infrared and visible light image perception fusion method |
| CN112233074A (en) * | 2020-09-30 | 2021-01-15 | 国网山西省电力公司大同供电公司 | Power failure detection method based on visible light and infrared fusion image |
| CN114881905A (en) * | 2022-06-21 | 2022-08-09 | 西北工业大学 | Processing method for fusing infrared image and visible light image based on wavelet transformation |
| CN115471723A (en) * | 2022-09-23 | 2022-12-13 | 安徽优航遥感信息技术有限公司 | Substation unmanned aerial vehicle inspection method based on infrared and visible light image fusion |
| CN115880221A (en) * | 2022-10-31 | 2023-03-31 | 国网福建省电力有限公司 | Power grid line defect analysis method integrating visible light and infrared image processing |
Non-Patent Citations (4)
| Title |
|---|
| 刘坤;郭雷;陈敬松;: "一种有效的红外图像融合算法", 火力与指挥控制, no. 06 * |
| 李伟;陈红斌;: "结合NSST和LC显著性的红外与可见光图像融合", 电子技术与软件工程, no. 08 * |
| 王兴龙;朱芳;: "基于双通道PCNN的NSST域红外与可见光图像融合", 平顶山学院学报, no. 02 * |
| 金立军;田治仁;高凯;肖嵘;: "基于红外与可见光图像信息融合的绝缘子污秽等级识别", 中国电机工程学报, no. 13 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116403057B (en) | 2023-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105023014B (en) | A kind of shaft tower target extraction method in unmanned plane inspection transmission line of electricity image | |
| CN110264448B (en) | Insulator fault detection method based on machine vision | |
| CN103345733B (en) | Based on the quick enhancement method of low-illumination image improving dark channel prior | |
| CN107784661A (en) | Substation equipment infrared image classifying identification method based on region-growing method | |
| CN104483326A (en) | High-voltage wire insulator defect detection method and high-voltage wire insulator defect detection system based on deep belief network | |
| CN110148093B (en) | Image defogging improvement method based on dark channel prior | |
| CN103295010B (en) | A kind of unitary of illumination method processing facial image | |
| CN110363727B (en) | Image defogging method based on multi-scale dark channel prior cascade deep neural network | |
| CN110910440B (en) | Power transmission line length determination method and system based on power image data | |
| CN112365468A (en) | AA-gate-Unet-based offshore wind power tower coating defect detection method | |
| CN110276747B (en) | Insulator fault detection and fault rating method based on image analysis | |
| CN107977942A (en) | A kind of restored method of the single image based on multi-focus image fusion | |
| CN103955907A (en) | Method for telemetering pointer type SF6 gas density meter | |
| CN103852018A (en) | Electric transmission line icing thickness measuring algorithm based on image processing | |
| Qian et al. | CIASM-Net: a novel convolutional neural network for dehazing image | |
| CN116403057B (en) | Power transmission line inspection method and system based on multi-source image fusion | |
| CN114187210B (en) | Multi-mode dense fog removing method based on visible light-far infrared image | |
| CN113625104B (en) | Line length correction method for traveling wave fault location | |
| Liu et al. | Research on image segmentation algorithm and performance of power insulator based on adaptive region growing | |
| CN118521955A (en) | Intelligent inspection method and system for power transmission line based on machine vision | |
| CN115661597A (en) | Visible light and infrared fusion target detection method based on dynamic weight positioning distillation | |
| CN112396572B (en) | Composite insulator double-light fusion method based on feature enhancement and Gaussian pyramid | |
| CN112444522B (en) | Method for detecting defects of insulator string of power system | |
| Yan et al. | Image Defogging Method for Transmission Channel Inspection by UAV Based on Deep Multi-patch Layered Network | |
| CN114359255A (en) | Improved Yolov5 s-based road pavement repair detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: F5-1-105, Sinovation Ventures, No. 2016, Feiyue Avenue, High tech Zone, Jinan, Shandong 250000 Patentee after: Shandong Ruiying Intelligent Technology Co.,Ltd. Patentee after: Jiangsu Ruiying Zhituo Electric Power Technology Development Co.,Ltd. Address before: F5-1-105, Sinovation Ventures, No. 2016, Feiyue Avenue, High tech Zone, Jinan, Shandong 250000 Patentee before: Shandong Ruiying Intelligent Technology Co.,Ltd. Patentee before: BEIJING RAYIEE ZHITUO TECHNOLOGY DEVELOPMENT Co.,Ltd. |