CN115082318B - Super-resolution reconstruction method for infrared image of electrical equipment - Google Patents
Super-resolution reconstruction method for infrared image of electrical equipment Download PDFInfo
- Publication number
- CN115082318B CN115082318B CN202210817691.9A CN202210817691A CN115082318B CN 115082318 B CN115082318 B CN 115082318B CN 202210817691 A CN202210817691 A CN 202210817691A CN 115082318 B CN115082318 B CN 115082318B
- Authority
- CN
- China
- Prior art keywords
- module
- network
- electrical equipment
- infrared image
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000000605 extraction Methods 0.000 claims abstract description 86
- 230000006870 function Effects 0.000 claims description 51
- 238000012549 training Methods 0.000 claims description 15
- 238000010586 diagram Methods 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 10
- 238000006731 degradation reaction Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 8
- 230000015556 catabolic process Effects 0.000 claims description 6
- 238000012886 linear function Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000003331 infrared imaging Methods 0.000 claims description 3
- 238000013441 quality evaluation Methods 0.000 claims description 3
- 230000007547 defect Effects 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Image Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a super-resolution reconstruction method of an infrared image of an electrical device, which consists of a countermeasure network and a generation network, wherein the generation network is used for generating the super-resolution reconstruction image of the infrared image of the electrical device, and the countermeasure network is used for judging whether the infrared image of the electrical device with high resolution is a generated image or an original image with high resolution. The invention improves ESRGAN defects, constructs an improved batch standardization module and a new feature extraction sub-module, constructs a feature extraction network for generating an countermeasure network by using the two modules, improves the feature extraction capability of the generated network, and further improves the quality of the reconstructed image; in addition, an improved batch standardization module is introduced into the ESRGAN countermeasure network, so that the discrimination capability of the countermeasure network is improved, the infrared image reconstruction capability of the power equipment of the generation network is improved, and the generation network can reconstruct high-quality high-resolution infrared images of the power equipment from low-resolution infrared images of the power equipment.
Description
Technical Field
The invention belongs to the technical field of infrared image enhancement, and relates to a super-resolution reconstruction method of an infrared image of electrical equipment.
Background
The faults of the electrical equipment are often reflected by the temperature change of the fault part of the equipment, so that an infrared thermal imager is used for collecting infrared images of the electrical equipment and analyzing the infrared images, and the fault detection of the electrical equipment without contact can be realized. Although thermal imagers have been widely used in monitoring the status of electrical equipment, due to the high cost of high-resolution thermal imagers, power companies mostly use common thermal imagers, and the resolution of the acquired infrared images of electrical equipment is low, which directly affects the status detection and fault diagnosis accuracy of equipment based on the infrared images of electrical equipment. There are two methods for improving the resolution of infrared images of electrical equipment: one is to improve the hardware level of the infrared thermal imager, and then improve the resolution of the infrared image, the method has higher difficulty and higher cost; the other is based on the existing infrared thermal imager, and based on an infrared image super-resolution reconstruction method, the low-resolution image is reconstructed into a high-resolution image, and the method is low in cost.
The existing single-frame electrical equipment infrared image super-resolution reconstruction can be divided into three main categories: the method comprises a difference value-based electrical equipment infrared image super-resolution reconstruction method, a reconstruction model-based electrical equipment infrared image super-resolution reconstruction method and a learning-based electrical equipment infrared image super-resolution reconstruction method. The super-resolution reconstruction method of the infrared image of the electrical equipment based on interpolation is simple and easy to implement, and can obtain a smooth reconstructed image, but partial detail information is seriously lost, the visual quality of the reconstructed image is poor, and texture information is not obvious. The super-resolution reconstruction method of the infrared image of the electrical equipment based on the reconstruction model utilizes prior information to reconstruct the infrared image, and the prior information in different environments is different, so that the quality of the reconstructed image is affected. According to the super-resolution reconstruction method of the infrared image of the electrical equipment based on learning, a large amount of training data is utilized, so that a model can learn a certain corresponding relation between the infrared image of the electrical equipment with high resolution and the corresponding infrared image of the electrical equipment with low resolution, and then the trained model is utilized to reconstruct the infrared image with high resolution from the infrared image of the electrical equipment with low resolution, so that super-resolution reconstruction of the infrared image of the electrical equipment is realized.
The super-resolution reconstruction method for the infrared image of the electrical equipment based on the generation countermeasure network in the prior art belongs to a super-resolution image reconstruction method based on learning, and although the infrared image of the electrical equipment can be better reconstructed compared with other methods, the reconstructed image contour edge is not clear enough, and the state monitoring and fault diagnosis of the electrical equipment based on the infrared image of the electrical equipment are affected. Therefore, it is required to improve the performance of the super-resolution reconstruction method of the infrared image of the electrical equipment based on the generated countermeasure network, and further improve the quality of the reconstructed infrared image of the electrical equipment.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: the defects of the prior art are overcome, and the super-resolution reconstruction method of the infrared image of the electrical equipment is provided, so that the quality of the reconstructed infrared image of the electrical equipment is improved.
The technical scheme for solving the technical problems is as follows: the super-resolution reconstruction method for the infrared image of the electrical equipment is characterized by comprising an countermeasure network and a generation network, wherein the generation network is used for generating the super-resolution reconstruction image of the infrared image of the electrical equipment, the countermeasure network is used for judging whether the infrared image of the electrical equipment with high resolution is a generated image or an original high-resolution image, and the generation network and the countermeasure network are used for game to improve the image generation capability of the generation network, and the method comprises the following specific steps:
Step 1, constructing a data set:
① Acquiring an infrared image of the electrical equipment with high resolution by using a high resolution infrared imager;
② Fitting a degradation process of an infrared image of real electrical equipment by using degradation functions comprising isotropic Gaussian blur, anisotropic Gaussian blur, downsampling, 3D Gaussian noise, imager noise and JPEG noise;
③ Performing degradation treatment on the acquired high-resolution infrared imaging image to obtain a corresponding low-resolution infrared image of the electrical equipment; the obtained degraded low-resolution infrared image of the electrical equipment and the corresponding high-resolution infrared image of the electrical equipment form image pairs, a plurality of image pairs form a data set, the data set is divided into two parts, one part is used as a training data set, and the other part is used as a test data set;
Step 2, constructing an improved batch standardization module:
On the basis of a conventional batch standardization module, a characteristic pixel standard deviation adjusting module is designed to reduce the influence of the conventional standardization module on the characteristic pixel standard deviation and improve the quality of a reconstructed image;
Step 3, constructing a final feature extraction sub-module:
constructing a final feature extraction sub-module for extracting infrared image features of the electrical equipment;
step 4, constructing a generation network in an infrared image super-resolution reconstruction network of the electrical equipment:
Utilizing the improved batch standardization module constructed in the step 2 and the final feature extraction submodule constructed in the step 3 to combine the ideas of dense connection network and residual error network, and constructing a generation network of an improved ESRGAN network on the basis of ESRGAN networks;
Step 5, constructing an countermeasure network in the infrared image super-resolution reconstruction network of the electrical equipment:
Introducing the modules constructed in the step 2 into a countermeasure network of ESRGAN networks to construct a countermeasure network of an improved ESRGAN network;
step 6: training an infrared image super-resolution reconstruction network of electrical equipment:
Training the generation network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 4 and the countermeasure network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 5 by adopting the training data set constructed in the step 1;
Step 7: network model test and evaluation:
inputting the low-resolution infrared images of the electrical equipment in the test data set constructed in the step 1 into the trained generation network in the step 6, and outputting corresponding reconstructed infrared super-resolution images of the electrical equipment; calculating the peak signal-to-noise ratio of the reconstructed infrared super-resolution image of the electrical equipment, evaluating the quality of the natural image, and executing the step 9 if the peak signal-to-noise ratio and the natural image quality evaluation meet the actual application requirements, otherwise executing the step 8;
Step 8: model parameter adjustment:
Adjusting the model parameters of the infrared image super-resolution reconstruction network of the electrical equipment constructed in the step 4 and the step 5, returning to the step 6, and retraining;
step 9: model application:
And (3) applying the generating network which meets the practical application requirements and is obtained in the step (7) to super-resolution reconstruction of the infrared image of the electrical equipment, and reconstructing the infrared image of the high-resolution electrical equipment from the acquired infrared image of the low-resolution electrical equipment.
Further, 80% of the data set in the step 1 is used as a training data set, and 20% of the data set is used as a test data set.
Further, the build-up of the modified batch normalization module in step 2 is expressed as:
The improved batch standardization module consists of two branches, wherein the first branch is a conventional batch standardization module, and the second branch consists of a standard deviation function (std ()), a logarithmic function (log), a linear function (f) and an exponential function (exp ()) in sequence; the first branch needs to perform batch standardization pretreatment on the input of the module and then is used as the output of a conventional batch standardization module, the second branch needs to sequentially perform standard deviation function, logarithmic function, linear function and exponential function treatment on the input of the module and then output the module, and the output of the first branch is multiplied with the output of the second branch to obtain the final output of the improved batch standardization module.
Further, the constructing a final feature extraction submodule in the step 3 is expressed as:
The final feature extraction sub-module consists of a front-stage feature extraction sub-module and an improved batch standardization module in the step 2;
The early feature extraction submodule consists of two branches, wherein the first branch consists of a convolution of 1 multiplied by 1, a ReLU6 function, a convolution of 3 multiplied by 3, a ReLU6 function and a convolution of 1 multiplied by 1 in sequence; the second branch consists of a convolution of 1×1, a ReLU6 function and a residual network in sequence;
The residual network consists of a 3X 3 convolution and jump connection, and the output of the 3X 3 convolution and the output of the jump connection are spliced in the channel dimension, so that the number of channels of the input characteristic diagram and the output characteristic diagram of the second branch is kept unchanged; superposing the output of the first branch and the output of the second branch to obtain the output characteristics of the early-stage characteristic extraction submodule;
The early feature extraction submodule is incorporated as a network module into the modified batch normalization module of step 2 to form a final feature extraction submodule.
Further, the generating network in the step 4 of constructing the infrared image super-resolution reconstruction network of the electrical device is expressed as:
The several modules are defined as follows:
IBN module: an improved batch standardization module constructed in the step 2;
GM module: a final feature extraction sub-module constructed in the step 3;
IRDB module: a residual error dense module;
IRRDB module: a feature extraction module having a residual structure;
Gm_db module: a residual dense module based on the final feature extraction sub-module;
Gm_rdb module: the feature extraction module is based on a residual error dense module of the final feature extraction sub-module;
the generating network sequentially comprises a 3X 3 convolution module, a characteristic extracting network of a multi-level residual error structure, an up-sampling module and two 3X 3 convolution modules which are connected in series;
The characteristic extraction network of the multi-level residual error structure consists of a first characteristic extraction sub-network, a second characteristic extraction sub-network, 1 GM module and two different jump connections, wherein the first characteristic extraction sub-network and the second characteristic extraction sub-network are connected in series;
The first feature extraction sub-network consists of 8 IRRDB modules;
Each IRRDB module consists of 3 first residual modules with the same structure, 1 IBN module and a characteristic coefficient;
Each first residual error module forming IRRDB modules consists of an IRDB module, 1 characteristic coefficient and 1 jump connection;
Each IRDB module consists of 53 multiplied by 3 convolution modules, 4 LReLU functions, 1 characteristic coefficient, 1 jump connection and an IBN module;
the second feature extraction sub-network consists of 8 GM_DB modules and 8 GM_RDB modules;
Each gm_db module consists of 4 GM modules, 4 LReLU functions, one 3 x 3 convolution module, 1 feature factor, 1 hopping connection, and IBN modules;
Each GM_RDB module consists of 3 second residual modules with the same structure, 1 IBN module, 1 characteristic coefficient and 1 jump connection;
Each second residual module forming the gm_rdb is formed by 1 gm_db module, 1 characteristic coefficient and 1 jump connection; the output of the second characteristic extraction sub-network and the input of the first characteristic extraction sub-network are fused through jump connection, the fused characteristics are input into the GM module, the output of the GM module and the input of the first characteristic extraction sub-network are fused through jump connection, and the fused input characteristics sequentially pass through an up-sampling layer and 2 3X 3 convolution layers to obtain an infrared image reconstruction image of the electrical equipment.
Further, the construction of the countermeasure network in the electrical equipment infrared image super-resolution reconstruction network in the step 5 is expressed as:
On the basis of the countermeasure network in ESRGN, replacing a conventional batch standardization module in the countermeasure network with the IBN module constructed in the step 3 to form an improved countermeasure network in the electric equipment infrared image super-resolution reconstruction network; the countermeasure network in the improved infrared image super-resolution reconstruction network of the electrical equipment sequentially comprises 13×3 convolution module, 1 LReLU function, 4 submodules, 14×4 convolution module, 1 IBN module, 1 LReLU function, 1 stretching layer, 1 full connection layer, 1 LReLU function and 1 full connection layer;
The number of channels of the four sub-modules is 128, 256, 512 and 512 respectively;
the four sub-modules have the same structure and sequentially consist of 14×4 convolution, one IBN module, one LReLU function, one 3×3 convolution layer, one IBN module and one LReLU function.
The beneficial effects of the invention are as follows:
(1) The invention constructs an improved batch standardization module (IBN module), reduces the influence of the conventional batch standardization on the variance of the image characteristic pixels, introduces the improved batch standardization module into a generation network and an countermeasure network, avoids gradient disappearance, improves the network training speed and generalization capability, and further improves the infrared image reconstruction quality of electrical equipment based on the generation countermeasure network;
(2) The invention constructs a final feature extraction sub-module, constructs a residual intensive module (GM_DB module) by utilizing the final feature extraction sub-module and an improved standardization module, constructs a GM_RDB module by utilizing the constructed GM_DB module and the improved batch standardization module, and takes 8 GM_DB modules and 8 GM_RDB modules as a part of a feature extraction network in a generation network;
(3) The invention introduces an improved batch standardization module (IBN) into an RDB module in ESRGAN to obtain an improved RDB module (IRDB module), utilizes the IRDB module and the IBN module to construct a IRRDB module, and takes 8 IRRDB modules as a part of a feature extraction network in a generation network;
(4) The invention further utilizes the GM network to further extract and fuse the output characteristics of the network consisting of 8 GM_DB modules, 8 GM_RDB modules and 8 IRRDB modules, and utilizes jump connection to fuse the extracted characteristics with the input characteristics of the network, thereby extracting more infrared image characteristics of the power equipment and improving the reconstruction capability of the generating network on the infrared image of the power equipment;
(5) The invention also utilizes the improved batch standardization module (IBN module) to improve the countermeasure network, thereby improving the discrimination capability of the countermeasure network, further improving the reconstruction capability of the generated network to the infrared image of the power equipment and improving the quality of the reconstructed infrared image of the power equipment.
ESRGAN described above is the paper published 2019 by Wang X.T et al ESRGAN:Enhanced Super-Resolution Generative Adversarial Networks[C].European Conference on Computer vision,2019:63–79
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a network architecture diagram of a batch normalization module in which the network modules are modules that need to be batch normalized;
FIG. 3 is a network architecture diagram of a feature extraction submodule constructed in accordance with the present invention;
FIG. 4 is a diagram of an IRDB module network structure constructed by the present invention and a IRRDB module network structure formed by IRDB modules, IRRDB modules are used to construct a feature extraction network in a generation network;
FIG. 5 is a diagram of a network structure of a GM_DB module and a diagram of a network structure of a GM_RDB module constructed by the invention, wherein both modules are used for constructing a feature extraction network in a generation network;
FIG. 6 is a diagram of a generated network based on the construction of IBN, GM, IRRDB, GM_DB and GM_RDB modules according to the present invention;
fig. 7 is a block diagram of an countermeasure network constructed in accordance with the present invention.
Detailed Description
The invention will be further described with reference to the drawings and the detailed description.
Referring to fig. 1-7, the super-resolution reconstruction method for infrared images of electrical equipment according to the present embodiment is used for improving the quality of the reconstructed infrared images of the electrical equipment, and comprises a generating network and an countermeasure network, wherein the generating network is used for generating super-resolution reconstructed images of infrared images of the electrical equipment, the countermeasure network is used for judging whether the generated images or original high-resolution images of the infrared images of the electrical equipment with high resolution, and the image generating capability of the generating network is improved through game of the generating network and the countermeasure network; the method comprises the following specific steps:
Step 1, constructing a data set:
① Acquiring an infrared image of the electrical equipment with high resolution by using a high resolution infrared imager;
② Fitting a degradation process of an infrared image of real electrical equipment by using degradation functions comprising isotropic Gaussian blur, anisotropic Gaussian blur, downsampling, 3D Gaussian noise, imager noise and JPEG noise;
③ Performing degradation treatment on the acquired high-resolution infrared imaging image to obtain a corresponding low-resolution infrared image of the electrical equipment; the obtained degraded low-resolution infrared image of the electrical equipment and the corresponding high-resolution infrared image of the electrical equipment form image pairs, a plurality of image pairs form a data set, the data set is divided into two parts, 80% of the data set is used as a training sample set, and 20% of the data set is used as a test set;
Step 2, constructing an improved batch standardization module:
On the basis of a conventional batch standardization module, a characteristic pixel standard deviation adjusting module is designed, so that the influence of the conventional batch standardization module on the characteristic pixel standard deviation is reduced, and the quality of a reconstructed image is improved;
Step 3, constructing a final feature extraction sub-module:
constructing a final feature extraction sub-module for extracting infrared image features of the electrical equipment, and extracting more effective infrared image features of the electrical equipment;
step 4, constructing a generation network in an infrared image super-resolution reconstruction network of the electrical equipment:
Utilizing the improved batch standardization module constructed in the step 2 and the final feature extraction submodule constructed in the step 3 to combine the ideas of dense connection network and residual error network, and constructing a generation network of an improved ESRGAN network on the basis of ESRGAN networks;
Step 5, constructing an countermeasure network in the infrared image super-resolution reconstruction network of the electrical equipment:
Introducing the modules constructed in the step 2 into a countermeasure network of ESRGAN networks to construct a countermeasure network of an improved ESRGAN network;
step 6: training an infrared image super-resolution reconstruction network of electrical equipment:
Training the generation network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 4 and the countermeasure network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 5 by adopting the training data set constructed in the step 1;
Step 7: network model test and evaluation:
inputting the low-resolution infrared images of the electrical equipment in the test data set constructed in the step 1 into the trained generation network in the step 6, and outputting corresponding reconstructed infrared super-resolution images of the electrical equipment; calculating the peak signal-to-noise ratio of the reconstructed infrared super-resolution image of the electrical equipment, evaluating the quality of the natural image, and executing the step 9 if the peak signal-to-noise ratio and the natural image quality evaluation meet the actual application requirements, otherwise executing the step 8;
Step 8: model parameter adjustment:
Adjusting the model parameters of the infrared image super-resolution reconstruction network of the electrical equipment constructed in the step 4 and the step 5, returning to the step 6, and retraining;
step 9: model application:
And (3) applying the generating network which meets the practical application requirements and is obtained in the step (7) to super-resolution reconstruction of the infrared image of the electrical equipment, and reconstructing the infrared image of the high-resolution electrical equipment from the acquired infrared image of the low-resolution electrical equipment.
The build-up of the improved batch normalization module in this step is expressed as:
The improved batch normalization module is shown in fig. 2, and consists of two branches, wherein the first branch is a conventional batch normalization module, and the second branch consists of a standard deviation function (std ()), a log function (log), a linear function (f) and an exponential function (exp ()) in sequence; the first branch needs to perform batch standardization pretreatment on the input of the module and then is used as the output of a conventional batch standardization module, the second branch needs to sequentially perform standard deviation function, logarithmic function, linear function and exponential function treatment on the input of the module and then output the module, and the output of the first branch is multiplied with the output of the second branch to obtain the final output of the improved batch standardization module.
The final feature extraction submodule constructed in the step 3 is expressed as:
Constructing a final feature extraction sub-module as shown in fig. 3, wherein the final feature extraction sub-module consists of a previous feature extraction sub-module and a modified batch standardization module in the step 2;
The early feature extraction submodule consists of two branches, wherein the first branch consists of a convolution of 1 multiplied by 1, a ReLU6 function, a convolution of 3 multiplied by 3, a ReLU6 function and a convolution of 1 multiplied by 1 in sequence; the second branch consists of a convolution of 1×1, a ReLU6 function and a residual network in sequence;
The residual network consists of a 3X 3 convolution and jump connection, and the output of the 3X 3 convolution and the output of the jump connection are spliced in the channel dimension, so that the number of channels of the input characteristic diagram and the output characteristic diagram of the second branch is kept unchanged; superposing the output of the first branch and the output of the second branch to obtain the output characteristics of the early-stage characteristic extraction submodule;
The early feature extraction submodule is incorporated as a network module into the modified batch normalization module of step 2 to form a final feature extraction submodule.
The generating network in the electrical equipment infrared image super-resolution reconstruction network constructed in the step 4 is expressed as follows:
The submodules constituting the generating network are shown in fig. 4 and 5, the generating network is shown in fig. 6, and the following are defined for several modules:
IBN module: an improved batch standardization module constructed in the step 2;
GM module: a final feature extraction sub-module constructed in the step 3;
IRDB module: a residual error dense module;
IRRDB module: a feature extraction module having a residual structure;
Gm_db module: a residual dense module based on the final feature extraction sub-module;
Gm_rdb module: the feature extraction module is based on a residual error dense module of the final feature extraction sub-module;
the generating network sequentially comprises a 3X 3 convolution module, a characteristic extracting network of a multi-level residual error structure, an up-sampling module and two 3X 3 convolution modules which are connected in series;
The characteristic extraction network of the multi-level residual error structure consists of a first characteristic extraction sub-network, a second characteristic extraction sub-network, 1 GM module and two different jump connections, wherein the first characteristic extraction sub-network and the second characteristic extraction sub-network are connected in series;
The first feature extraction sub-network consists of 8 IRRDB modules;
Each IRRDB module consists of 3 first residual modules with the same structure, 1 IBN module and a characteristic coefficient;
Each first residual error module forming IRRDB modules consists of an IRDB module, 1 characteristic coefficient and 1 jump connection;
Each IRDB module consists of 53 multiplied by 3 convolution modules, 4 LReLU functions, 1 characteristic coefficient, 1 jump connection and an IBN module;
the second feature extraction sub-network consists of 8 GM_DB modules and 8 GM_RDB modules;
Each gm_db module consists of 4 GM modules, 4 LReLU functions, one 3 x 3 convolution module, 1 feature factor, 1 hopping connection, and IBN modules;
Each GM_RDB module consists of 3 second residual modules with the same structure, 1 IBN module, 1 characteristic coefficient and 1 jump connection;
Each second residual module forming the gm_rdb is formed by 1 gm_db module, 1 characteristic coefficient and 1 jump connection; the output of the second characteristic extraction sub-network and the input of the first characteristic extraction sub-network are fused through jump connection, the fused characteristics are input into the GM module, the output of the GM module and the input of the first characteristic extraction sub-network are fused through jump connection, and the fused input characteristics sequentially pass through an up-sampling layer and 2 3X 3 convolution layers to obtain an infrared image reconstruction image of the electrical equipment.
The construction of the countermeasure network in the electrical equipment infrared image super-resolution reconstruction network in the step 5 is represented as follows:
The improved countermeasure network is shown in fig. 7, and on the basis of the countermeasure network in ESRGN, a conventional batch standardization module in the countermeasure network is replaced by an IBN module constructed in the step 3, so that the countermeasure network in the improved infrared image super-resolution reconstruction network of the electrical equipment is formed; the countermeasure network in the improved infrared image super-resolution reconstruction network of the electrical equipment sequentially comprises 13×3 convolution module, 1 LReLU function, 4 submodules, 14×4 convolution module, 1 IBN module, 1 LReLU function, 1 stretching layer, 1 full connection layer, 1 LReLU function and 1 full connection layer;
The number of channels of the four sub-modules is 128, 256, 512 and 512 respectively;
the four sub-modules have the same structure and sequentially consist of 14×4 convolution, one IBN module, one LReLU function, one 3×3 convolution layer, one IBN module and one LReLU function.
The above description is only of the preferred embodiment of the present invention, and is not intended to limit the present invention in any other way, but is intended to cover any modifications or equivalent variations according to the technical spirit of the present invention, which fall within the scope of the present invention as defined by the appended claims.
Claims (1)
1. The super-resolution reconstruction method for the infrared image of the electrical equipment is characterized by comprising the following specific steps of:
Step 1, constructing a data set:
① Acquiring an infrared image of the electrical equipment with high resolution by using a high resolution infrared imager;
② Fitting a degradation process of an infrared image of real electrical equipment by using degradation functions comprising isotropic Gaussian blur, anisotropic Gaussian blur, downsampling, 3D Gaussian noise, imager noise and JPEG noise;
③ Performing degradation treatment on the acquired high-resolution infrared imaging image to obtain a corresponding low-resolution infrared image of the electrical equipment; the obtained degraded low-resolution infrared image of the electrical equipment and the corresponding high-resolution infrared image of the electrical equipment form image pairs, a plurality of image pairs form a data set, the data set is divided into two parts, one part is used as a training data set, and the other part is used as a test data set;
Step 2, constructing an improved batch standardization module:
Designing a characteristic pixel standard deviation adjusting module on the basis of a conventional batch standardization module;
Step 3, constructing a final feature extraction sub-module:
constructing a final feature extraction sub-module for extracting infrared image features of the electrical equipment;
step 4, constructing a generation network in an infrared image super-resolution reconstruction network of the electrical equipment:
Utilizing the improved batch standardization module constructed in the step 2 and the final feature extraction submodule constructed in the step 3 to construct a generation network in the improved infrared image super-resolution reconstruction network of the electrical equipment on the basis of the infrared image super-resolution reconstruction network of the electrical equipment;
Step 5, constructing an countermeasure network in the infrared image super-resolution reconstruction network of the electrical equipment:
Introducing the improved batch standardization module constructed in the step 2 into an countermeasure network of an infrared image super-resolution reconstruction network of the electrical equipment, and constructing the countermeasure network in the improved infrared image super-resolution reconstruction network of the electrical equipment;
step 6: training an infrared image super-resolution reconstruction network of electrical equipment:
Training the generation network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 4 and the countermeasure network in the electric equipment infrared image super-resolution reconstruction network constructed in the step 5 by adopting the training data set constructed in the step 1;
Step 7: network model test and evaluation:
Inputting the low-resolution infrared images of the electrical equipment in the test data set constructed in the step 1 into the trained generation network in the step 6, and outputting corresponding reconstructed infrared super-resolution images of the electrical equipment; calculating the peak signal-to-noise ratio of the reconstructed infrared super-resolution image of the electrical equipment, and evaluating the quality of the natural image; if the peak signal-to-noise ratio and the natural image quality evaluation meet the actual application requirements, executing the step 9, otherwise executing the step 8;
Step 8: model parameter adjustment:
Adjusting the model parameters of the infrared image super-resolution reconstruction network of the electrical equipment constructed in the step 4 and the step 5, returning to the step 6, and retraining;
step 9: model application:
The generating network meeting the practical application requirements, which is obtained in the step 7, is applied to super-resolution reconstruction of the infrared image of the electrical equipment, and the high-resolution infrared image of the electrical equipment is reconstructed from the acquired low-resolution infrared image of the electrical equipment;
The build-up of the modified batch normalization module in step 2 is expressed as:
The improved batch standardization module consists of two branches, wherein the first branch is a conventional batch standardization module, and the second branch consists of a standard deviation function, a logarithmic function, a linear function and an exponential function in sequence; the first branch needs to perform batch standardization pretreatment on the input of the module and then is used as the output of a conventional batch standardization module, the second branch needs to sequentially perform standard deviation function, logarithmic function, linear function and exponential function treatment on the input of the module and then outputs the module, and the output of the first branch is multiplied with the output of the second branch to obtain the final output of the improved batch standardization module;
the final feature extraction submodule constructed in the step 3 is expressed as:
The final feature extraction sub-module consists of a front-stage feature extraction sub-module and an improved batch standardization module in the step 2;
The early feature extraction submodule consists of two branches, wherein the first branch consists of a convolution of 1 multiplied by 1, a ReLU6 function, a convolution of 3 multiplied by 3, a ReLU6 function and a convolution of 1 multiplied by 1 in sequence; the second branch consists of a convolution of 1×1, a ReLU6 function and a residual network in sequence;
The residual network consists of a 3X 3 convolution and jump connection, and the output of the 3X 3 convolution and the output of the jump connection are spliced in the channel dimension, so that the number of channels of the input characteristic diagram and the output characteristic diagram of the second branch is kept unchanged; superposing the output of the first branch and the output of the second branch to obtain the output characteristics of the early-stage characteristic extraction submodule;
Incorporating the early feature extraction submodule into the improved batch standardization module of the step 2 as a network module to form a final feature extraction submodule;
The generating network in the electrical equipment infrared image super-resolution reconstruction network constructed in the step 4 is expressed as follows:
The several modules are defined as follows:
IBN module: an improved batch standardization module constructed in the step 2;
GM module: a final feature extraction sub-module constructed in the step 3;
IRDB module: a residual error dense module;
IRRDB module: a feature extraction module having a residual structure;
Gm_db module: a residual dense module based on the final feature extraction sub-module;
Gm_rdb module: the feature extraction module is based on a residual error dense module of the final feature extraction sub-module;
the generating network sequentially comprises a 3X 3 convolution module, a characteristic extracting network of a multi-level residual error structure, an up-sampling module and two 3X 3 convolution modules which are connected in series;
The characteristic extraction network of the multi-level residual error structure consists of a first characteristic extraction sub-network, a second characteristic extraction sub-network, 1 GM module and two different jump connections, wherein the first characteristic extraction sub-network and the second characteristic extraction sub-network are connected in series;
The first feature extraction sub-network consists of 8 IRRDB modules;
Each IRRDB module consists of 3 first residual modules with the same structure, 1 IBN module and a characteristic coefficient;
Each first residual error module forming IRRDB modules consists of an IRDB module, 1 characteristic coefficient and 1 jump connection;
Each IRDB module consists of 53 multiplied by 3 convolution modules, 4 LReLU functions, 1 characteristic coefficient, 1 jump connection and an IBN module;
the second feature extraction sub-network consists of 8 GM_DB modules and 8 GM_RDB modules;
Each gm_db module consists of 4 GM modules, 4 LReLU functions, one 3 x 3 convolution module, 1 feature factor, 1 hopping connection, and IBN modules;
Each GM_RDB module consists of 3 second residual modules with the same structure, 1 IBN module, 1 characteristic coefficient and 1 jump connection;
Each second residual module forming the gm_rdb is formed by 1 gm_db module, 1 characteristic coefficient and 1 jump connection; the output of the second characteristic extraction sub-network and the input of the first characteristic extraction sub-network are fused through jump connection, the fused characteristics are input into the GM module, the output of the GM module and the input of the first characteristic extraction sub-network are fused through jump connection, and the fused input characteristics sequentially pass through an up-sampling layer and 2 3X 3 convolution layers to obtain an infrared image reconstruction image of the electrical equipment;
the construction of the countermeasure network in the electrical equipment infrared image super-resolution reconstruction network in the step 5 is represented as follows:
On the basis of an countermeasure network in an electrical equipment infrared image super-resolution reconstruction network, replacing a conventional batch standardization module in the countermeasure network with an IBN module constructed in the step 3 to form an improved countermeasure network in the electrical equipment infrared image super-resolution reconstruction network; the countermeasure network in the improved infrared image super-resolution reconstruction network of the electrical equipment sequentially comprises 13×3 convolution module, 1 LReLU function, 4 submodules, 14×4 convolution module, 1 IBN module, 1 LReLU function, 1 stretching layer, 1 full connection layer, 1 LReLU function and 1 full connection layer;
the number of channels of the 4 sub-modules is 128, 256, 512 and 512 respectively;
the 4 sub-modules have the same structure and sequentially consist of 14×4 convolution, one IBN module, one LReLU function, one 3×3 convolution layer, one IBN module and one LReLU function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210817691.9A CN115082318B (en) | 2022-07-13 | 2022-07-13 | Super-resolution reconstruction method for infrared image of electrical equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210817691.9A CN115082318B (en) | 2022-07-13 | 2022-07-13 | Super-resolution reconstruction method for infrared image of electrical equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115082318A CN115082318A (en) | 2022-09-20 |
| CN115082318B true CN115082318B (en) | 2024-07-02 |
Family
ID=83259888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210817691.9A Active CN115082318B (en) | 2022-07-13 | 2022-07-13 | Super-resolution reconstruction method for infrared image of electrical equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115082318B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102160514A (en) * | 2011-02-21 | 2011-08-24 | 中国热带农业科学院农产品加工研究所 | Method for promoting rubber tapping output of rubber tree using ultrasonic wave |
| CN108982460A (en) * | 2018-08-22 | 2018-12-11 | 深圳大学 | A kind of super-resolution imaging method, device and terminal device |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114641790A (en) * | 2019-09-11 | 2022-06-17 | 以色列国家农业部、农村发展农业研究组织·沃尔卡尼中心 | Super-resolution processing method and system for infrared image |
| CN114731455B (en) * | 2019-11-20 | 2024-07-09 | 三星电子株式会社 | Apparatus and method using AI metadata related to image quality |
| CN111062872B (en) * | 2019-12-17 | 2021-02-05 | 暨南大学 | Image super-resolution reconstruction method and system based on edge detection |
| CN111583109B (en) * | 2020-04-23 | 2024-02-13 | 华南理工大学 | Image super-resolution method based on generation of countermeasure network |
| CN112561799A (en) * | 2020-12-21 | 2021-03-26 | 江西师范大学 | Infrared image super-resolution reconstruction method |
| CN112907614A (en) * | 2021-03-26 | 2021-06-04 | 东北电力大学 | Yoov 5-segnet insulator string contour extraction method based on depth feature fusion |
| CN113591645B (en) * | 2021-07-21 | 2024-09-13 | 东北电力大学 | Regional convolutional neural network-based infrared image recognition method for power equipment |
| CN114092330B (en) * | 2021-11-19 | 2024-04-30 | 长春理工大学 | Light-weight multi-scale infrared image super-resolution reconstruction method |
| CN114463176B (en) * | 2022-01-25 | 2024-03-01 | 河南大学 | Image super-resolution reconstruction method based on improved ESRGAN |
-
2022
- 2022-07-13 CN CN202210817691.9A patent/CN115082318B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102160514A (en) * | 2011-02-21 | 2011-08-24 | 中国热带农业科学院农产品加工研究所 | Method for promoting rubber tapping output of rubber tree using ultrasonic wave |
| CN108982460A (en) * | 2018-08-22 | 2018-12-11 | 深圳大学 | A kind of super-resolution imaging method, device and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115082318A (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110310227B (en) | Image super-resolution reconstruction method based on high-low frequency information decomposition | |
| CN106910161B (en) | Single image super-resolution reconstruction method based on deep convolutional neural network | |
| CN113077527B (en) | Rapid magnetic resonance image reconstruction method based on undersampling | |
| Luo et al. | Lattice network for lightweight image restoration | |
| CN111260560B (en) | Multi-frame video super-resolution method fused with attention mechanism | |
| CN111784570A (en) | Video image super-resolution reconstruction method and device | |
| CN107464222B (en) | Based on tensor space without reference high dynamic range images method for evaluating objective quality | |
| Niu et al. | Low cost edge sensing for high quality demosaicking | |
| CN113379601A (en) | Real world image super-resolution method and system based on degradation variational self-encoder | |
| CN115880158B (en) | Blind image super-resolution reconstruction method and system based on variation self-coding | |
| CN112561799A (en) | Infrared image super-resolution reconstruction method | |
| CN117635428A (en) | Super-resolution reconstruction method for lung CT image | |
| CN117114984A (en) | Remote sensing image super-resolution reconstruction method based on generation countermeasure network | |
| CN116309062A (en) | Remote sensing image super-resolution reconstruction method | |
| CN114926337A (en) | Single image super-resolution reconstruction method and system based on CNN and Transformer hybrid network | |
| CN114092329B (en) | Super-resolution fluorescence microscopic imaging method based on sub-pixel neural network | |
| CN115082318B (en) | Super-resolution reconstruction method for infrared image of electrical equipment | |
| CN117455770A (en) | Lightweight image super-resolution method based on layer-by-layer context information aggregation network | |
| CN115496819B (en) | Rapid coding spectral imaging method based on energy concentration characteristic | |
| CN117115011A (en) | De-artifact method and system based on deep unsupervised learning | |
| CN117196940A (en) | Super-resolution reconstruction method suitable for real scene image based on convolutional neural network | |
| CN116664401A (en) | Method and system for improving infrared image reconstruction capability of power equipment | |
| CN116188273A (en) | Uncertainty-oriented bimodal separable image super-resolution method | |
| CN116385284A (en) | Image enhancement system of three-dimensional magnetic resonance image | |
| CN115965527A (en) | Mobile terminal lightweight image super-resolution reconstruction method based on convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |