CN113724134A - Aerial image blind super-resolution reconstruction method based on residual distillation network - Google Patents
Aerial image blind super-resolution reconstruction method based on residual distillation network Download PDFInfo
- Publication number
- CN113724134A CN113724134A CN202110963671.8A CN202110963671A CN113724134A CN 113724134 A CN113724134 A CN 113724134A CN 202110963671 A CN202110963671 A CN 202110963671A CN 113724134 A CN113724134 A CN 113724134A
- Authority
- CN
- China
- Prior art keywords
- residual
- image
- resolution
- network
- distillation network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004821 distillation Methods 0.000 title claims abstract description 86
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000006731 degradation reaction Methods 0.000 claims abstract description 24
- 230000015556 catabolic process Effects 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 5
- 230000000694 effects Effects 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 10
- 238000005516 engineering process Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000000513 principal component analysis Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- NEVVHGHXJACRQZ-OCZHUQCSSA-N (7r,8s,9s,10r,13r,14r,17s)-17-acetyl-7,14-dihydroxy-10,13-dimethyl-2,6,7,8,9,11,12,15,16,17-decahydro-1h-cyclopenta[a]phenanthren-3-one Chemical compound C([C@H]1O)C2=CC(=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@]1(O)CC[C@H](C(=O)C)[C@@]1(C)CC2 NEVVHGHXJACRQZ-OCZHUQCSSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013140 knowledge distillation Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
- G06T3/4076—Super resolution, i.e. output image resolution higher than sensor resolution by iteratively correcting the provisional high resolution image using the original low-resolution image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention provides an aerial image blind super-resolution reconstruction method based on a residual distillation network, which comprises the following steps of: s1: acquiring a degraded unmanned aerial vehicle aerial image data set and a high-resolution aerial image data set; s2: constructing a fuzzy kernel pool according to the degraded unmanned aerial vehicle aerial image data set; s3: constructing an image degradation model according to the fuzzy kernel pool; s4: carrying out degradation processing on the high-resolution aerial image data set through an image degradation model to generate a high-low resolution image data set; s5: constructing a residual distillation network, and training the residual distillation network by using the fuzzy kernel pool and the high-low resolution image to the data set to obtain a trained residual distillation network; s6: and reconstructing an ultra-resolution image through the trained residual distillation network. The invention provides an aerial image blind super-resolution reconstruction method based on a residual distillation network, which solves the problem that the existing image super-resolution reconstruction model has very high requirements on equipment performance.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an aerial image blind super-resolution reconstruction method based on a residual distillation network.
Background
With the rapid development of 5G transmission technology and sensor technology, the unmanned aerial vehicle can be widely applied to the fields of agriculture, military, natural disaster prevention, meteorological monitoring and the like. Meanwhile, the unmanned aerial vehicle also realizes commercialization, and creates unprecedented value in the field of personal consumer goods. However, the photos taken by the drone are often accompanied by various disturbances, subject to the effects of the shooting equipment, weather and high-altitude environment. Among them, motion noise and rain fog are important causes affecting quality degradation of a photographed picture. In addition, the high-resolution image contains more picture information compared with the low-resolution image, and the information not only can improve the overall quality of the picture, but also provides more identification information for the high-level visual identification task, so that the accuracy of the identification task is improved. Due to the limitation of the size of a CMOS (complementary metal oxide semiconductor) in shooting equipment, the resolution of a shot picture is small, and the application range of aerial images is limited due to the interference of noise and a fuzzy core.
Due to the development of the convolutional neural network, the image super-resolution reconstruction model based on deep learning has achieved excellent effects. The single-image super-resolution reconstruction technology aims to obtain a high-resolution image through reconstruction of a single low-resolution image, and since the 2014 convolutional neural network is applied to single-image super-resolution reconstruction for the first time, image super-resolution reconstruction models based on the method are deeper and deeper. The deeper model enables the image super-resolution reconstruction effect to be better, but the requirement of the huge model on the equipment performance is very high, which greatly limits the development of the image super-resolution reconstruction technology.
In patent publication No. CN113096013A disclosed in 2021-07-09, a method and a system for super-resolution reconstruction of blind images based on imaging modeling and knowledge distillation are provided, in which an image correction network is provided to correct a knowledge image with intrinsic features in depth, so as to remove artifacts from the knowledge image with intrinsic features in depth, and further improve the resolution of an image with a second resolution, but the model parameters are many and the requirements on the performance of the device are high.
Disclosure of Invention
The invention provides an aerial image blind super-resolution reconstruction method based on a residual distillation network, aiming at overcoming the technical defect that the requirement of the existing image super-resolution reconstruction model on the performance of equipment is very high.
In order to solve the technical problems, the technical scheme of the invention is as follows:
an aerial image blind super-resolution reconstruction method based on a residual distillation network comprises the following steps:
s1: acquiring a degraded unmanned aerial vehicle aerial image data set and a high-resolution aerial image data set;
s2: constructing a fuzzy kernel pool according to the degraded unmanned aerial vehicle aerial image data set;
s3: constructing an image degradation model according to the fuzzy kernel pool;
s4: carrying out degradation processing on the high-resolution aerial image data set through the image degradation model to generate a high-low resolution image data set;
s5: constructing a residual distillation network, and training the residual distillation network by using the fuzzy kernel pool and the high-low resolution image to a data set to obtain a trained residual distillation network;
s6: and reconstructing an ultra-resolution image through the trained residual distillation network.
In the scheme, the super-resolution image is reconstructed by using the residual distillation network, so that the parameters of the whole network model can be reduced, and the requirement of the network model on the performance of equipment is reduced; meanwhile, the residual distillation network is trained by combining the fuzzy kernel pool and the high-low resolution image to the data set, so that the reconstruction effect of the aerial image is effectively improved.
Preferably, in step S2, extracting each image in the degraded unmanned aerial vehicle aerial image data set by using a generation countermeasure network to obtain a corresponding blur kernel, so as to construct a blur kernel pool suitable for the unmanned aerial vehicle aerial environment.
Preferably, the image degradation model is:
xdataset＝(y*kpool)↓s+n
wherein x isdatasetRepresenting degraded data, y representing data of a degradation model of the input image, kpoolRepresent the fuzzy core pool, ↓sRepresenting a down-sampling operation with a scale factor s and n representing noise.
Preferably, in the residual distillation network, the features of the input image are extracted through residual structures, and distillation residual branches are respectively led out from each residual structure, and the distillation residual branches are realized through a 1 × 1 convolutional layer, and finally, the feature information of each distillation residual branch is spliced to complete the information fusion operation, specifically as follows:
Fdistilled_1，F1＝fconv1×1(xin)，fconv3×3(xin)+xin
Fdistilled_2，F2＝fconv1×1(F1)，fconv3×3(F1)+F1
Fdistilled_3，F3＝fconv1×1(F2)，fconv3×3(F2)+F2
Fdistilled_4＝fconv1×1(F3)
Fout＝fconcat(Fdistilled_1，Fdistilled_2，Fdistilled_3，Fdistilled_4)
wherein, Fdistilled_1、Fdistilled_2、Fdistilled_3、Fdistilled_4Each represents a characteristic graph of the residual after distillation by 1X 1 convolution, F1、F2、F3Respectively representing the feature maps output by the first, second and third residual structureconv1×1Denotes a 1 × 1 convolution operation, fconv3×3Denotes a 3 × 3 convolution operation, xinRepresenting an input image, FoutRepresenting the output of the residual distillation network, fconcatThe splicing operation of the characteristic diagrams is shown.
In the above-described scheme, the number of parameters and the computational complexity can be significantly reduced by such a network structure.
Preferably, in the training process of the residual distillation network, the method further comprises: and obtaining fuzzy kernel characteristics by mapping transformation operation on the fuzzy kernels input into the residual distillation network, and splicing the fuzzy kernel characteristics with each residual characteristic graph.
In the scheme, the final reconstruction effect can be improved to the maximum extent by splicing the fuzzy core characteristics and each residual characteristic map.
Preferably, it is assumed that the information input to the residual distillation network includes a blur kernel with a size p × p and a noise level δ, and a low resolution image with a size W × H × C, where p represents the size of the blur kernel, and W, H, C represents the width, height, and number of channels of the low resolution image respectively;
firstly, vectorizing a fuzzy core to obtain a normalized size p2And the vector of x 1 is mapped into a linear space with the dimension of t by a principal component analysis technology, and finally the vector of the dimension of t is spliced with the low-latitude characteristic and the noise level characteristic of the low-resolution image to form a degradation characteristic diagram with the size of W x H x (C + 1).
Preferably, in the residual distillation network, two times of up-sampling reconstruction are included.
Preferably, the first upsampling reconstruction is:
wherein, Oupsample_1Representing the output of the first up-sampled reconstruction, FiRepresenting the output of the ith distillation residual branch in the residual distillation network, F0Represents the output of the first convolution block of the input low resolution image in the residual distillation network, and R (-) represents a 3 × 3 convolution and a ReLU activation function operation.
Preferably, the second upsampling is reconstructed into;
Oupsample_2＝R(F0)
wherein, Oupsample_2Representing the output of the second upsampled reconstruction, F0Represents the output of the first convolution block of the input low resolution image in the residual distillation network, and R (-) represents a 3 × 3 convolution and a ReLU activation function operation.
Preferably, the method further comprises the following steps: and evaluating the reconstruction effect through two evaluation indexes of the peak signal-to-noise ratio and the structural similarity of the reconstructed super-resolution image and the original high-resolution aerial image.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention provides an aerial image blind super-resolution reconstruction method based on a residual distillation network, which reconstructs a super-resolution image by using the residual distillation network, can reduce the parameters of an integral network model, thereby reducing the requirements of the network model on the performance of equipment; meanwhile, the residual distillation network is trained by combining the fuzzy kernel pool and the high-low resolution image to the data set, so that the reconstruction effect of the aerial image is effectively improved.
Drawings
FIG. 1 is a flow chart of the steps for implementing the technical solution of the present invention;
FIG. 2 is a flow chart of the present invention for performing a degradation process on a high resolution aerial image dataset;
FIG. 3 is a flow chart of super-resolution reconstruction of an aerial image according to the present invention;
fig. 4 is a flowchart for stitching the blur kernel feature with each residual feature map in the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 1, a blind super-resolution reconstruction method for aerial images based on a residual distillation network includes the following steps:
s1: acquiring a degraded unmanned aerial vehicle aerial image data set and a high-resolution aerial image data set;
s2: constructing a fuzzy kernel pool according to the degraded unmanned aerial vehicle aerial image data set;
s3: constructing an image degradation model according to the fuzzy kernel pool;
s4: carrying out degradation processing on the high-resolution aerial image data set through the image degradation model to generate a high-low resolution image data set;
s5: constructing a residual distillation network, and training the residual distillation network by using the fuzzy kernel pool and the high-low resolution image to a data set to obtain a trained residual distillation network;
s6: and reconstructing an ultra-resolution image through the trained residual distillation network.
In the specific implementation process, the super-resolution image is reconstructed by using the residual distillation network, so that the parameters of the whole network model can be reduced, and the requirement of the network model on the performance of equipment is reduced; meanwhile, the residual distillation network is trained by combining the fuzzy kernel pool and the high-low resolution image to the data set, so that the reconstruction effect of the aerial image is effectively improved.
Example 2
More specifically, as shown in fig. 2 to 4, in step S2, extracting each image in the degraded unmanned aerial vehicle aerial image data set by generating a countermeasure network to obtain a corresponding blur kernel, so as to construct a blur kernel pool suitable for the unmanned aerial vehicle aerial environment.
More specifically, the image degradation model is:
xdataset＝(y*kpool)↓s+n
wherein x isdatasetRepresenting degraded data, y representing data of a degradation model of the input image, kpoolRepresent the fuzzy core pool, ↓sRepresenting a down-sampling operation with a scale factor s and n representing noise.
In a specific implementation process, the influence of a fuzzy kernel k and noise n is not considered in the traditional single-image super-resolution reconstruction, and a low-resolution (LR) image is obtained by using a bicubic interpolation method for down-sampling. Therefore, the traditional super-resolution method cannot truly reflect the image degradation process, and the algorithm can seriously degrade the performance when applied to practical application. Therefore, in the embodiment, a real picture (such as a degraded unmanned aerial vehicle aerial image data set DPED) shot from the real world is selected as the degraded unmanned aerial vehicle aerial image data set, and the data is directly obtained from terminal equipment such as a mobile phone, so that the defect of the image in practical application can be better reflected. Extracting a corresponding fuzzy kernel pool from the low-resolution images through a simple generation countermeasure network, processing a high-resolution (HR) image in a high-resolution aerial image dataset (an unmanned aerial vehicle aerial image dataset manufactured by Tianjin university is selected in the embodiment) by the fuzzy kernel pool to obtain a corresponding low-resolution image, and forming a high-low resolution (HR-LR) image dataset with the low-resolution image.
The traditional CNN neural network structure improves the overall performance of the network by stacking deep networks, but the size of the model is exponentially increased in the mode, so that the difficulty of algorithm deployment is increased, the difficulty of network training is increased, and the training process is unstable. Therefore, a residual distillation network of a characteristic distillation structure is adopted, and the parameter number and the calculation complexity can be greatly reduced through the network structure.
More specifically, in the residual error distillation network, firstly, features of an input image are extracted through a residual error structure, distillation residual error branches are respectively led out from each residual error structure, the distillation residual error branches are realized through a convolution layer of 1 × 1, and finally, feature information of each distillation residual error branch is spliced to complete information fusion operation, which specifically comprises the following steps:
Fdistilled_1，F1＝fconv1×1(xin)，fconv3×3(xin)+xin
Fdistilled_2，F2＝fconv1×1(F1)，fconv3×3(F1)+F1
Fdistilled_3，F3＝fconv1×1(F2)，fconv3×3(F2)+F2
Fdistilled_4＝fconv1×1(F3)
Fout＝fconcat(Fdistilled_1，Fdistilled_2，Fdistilled_3，Fdistilled_4)
wherein, Fdistilled_1、Fdistilled_2、Fdistilled_3、Fdistilled_4Each represents a characteristic graph of the residual after distillation by 1X 1 convolution, F1、F2、F3Respectively representing the feature maps output by the first, second and third residual structureconv1×1Denotes a 1 × 1 convolution operation, fconv3×3Denotes a 3 × 3 convolution operation, xinRepresenting an input image, FoutRepresenting the output of the residual distillation network, fconcatThe splicing operation of the characteristic diagrams is shown.
More specifically, in the training process of the residual distillation network, the method further comprises the following steps: and obtaining fuzzy kernel characteristics by mapping transformation operation on the fuzzy kernels input into the residual distillation network, and splicing the fuzzy kernel characteristics with each residual characteristic graph.
In a specific implementation process, the blurred low-resolution image after the blurring process is directly used for training, so that the super-resolution and deblurring effects are poor due to further reduction of picture information. Therefore, in the embodiment, the fuzzy kernel feature and each residual feature map are spliced, so that the final reconstruction effect can be improved to the maximum extent.
More specifically, it is assumed that the information input to the residual distillation network includes a blur kernel with a size p × p and a noise level δ, and a low-resolution image with a size W × H × C, where p denotes the size of the blur kernel, and W, H, C denotes the width, height, and number of channels of the low-resolution image, respectively;
firstly, vectorizing a fuzzy core to obtain a normalized size p2The vector of x 1 is mapped into a linear space with the dimension of t by a principal component analysis technology, and finally the vector of the dimension of t and low-latitude characteristics, noise and the like of a low-resolution imageAnd (5) splicing the level features to form a degradation feature map with the size of W multiplied by H multiplied by (C + 1).
In a specific implementation process, the fuzzy kernel is mapped into a linear space with the dimension t through mapping transformation, so that the low-latitude characteristic, the noise level characteristic and the fuzzy kernel characteristic of the down-sampling low-resolution image are processed simultaneously on the same dimension.
More specifically, in the residual distillation network, two upsampling reconstructions are included.
In the specific implementation process, the connection of the high-frequency information and the low-frequency information is very effective for the restoration and reconstruction of the image, and based on this point, the present embodiment uses a new dense upsampling scheme instead of the conventional single-connection upsampling scheme to achieve a better reconstruction effect. By performing the first upsampling reconstruction in the shallow network and fusing with the second upsampling reconstruction performed at the tail of the network, high-frequency and low-frequency image information can be effectively utilized.
More specifically, the first upsampled reconstruction is:
wherein, Oupsample_1Representing the output of the first up-sampled reconstruction, FiRepresenting the output of the ith distillation residual branch in the residual distillation network, F0Representing the output of the first convolution block of the input low resolution image in the residual distillation network, R (-) represents a 3 × 3 convolution and a ReLU activation function operation in order to convert the extracted features into non-linear features.
In the implementation process, the outputs of the four characteristic distillation residual structure blocks RFDB are fused together by residual connection to perform the first upsampling reconstruction. The first upsampling reconstruction aims to combine different low-frequency and high-frequency features extracted by RFDB blocks with different depths through dense residual connection, so that the loss of feature information in continuous convolution is reduced, and the stability of network training can be effectively improved.
More specifically, the second upsampling is reconstructed into;
Oupsample_2＝R(F0)
wherein, Oupsample_2Representing the output of the second upsampled reconstruction, F0Represents the output of the first convolution block of the input low resolution image in the residual distillation network, and R (-) represents a 3 × 3 convolution and a ReLU activation function operation.
In a specific implementation, the second upsampling operation uses only F0As input characteristic information, richer low-frequency image characteristics can be provided for final reconstruction. The details of pictures can be lost in an excessively deep network in a complex environment and a gorgeous environment, and the low-frequency image information can be effectively utilized by performing second up-sampling reconstruction on a shallow network and fusing the second up-sampling reconstruction with the first up-sampling reconstruction performed on the tail part of the network.
More specifically, the method further comprises the following steps: and evaluating the reconstruction effect through two evaluation indexes of the peak signal-to-noise ratio and the structural similarity of the reconstructed super-resolution image and the original high-resolution aerial image.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. The aerial image blind super-resolution reconstruction method based on the residual distillation network is characterized by comprising the following steps of:
s1: acquiring a degraded unmanned aerial vehicle aerial image data set and a high-resolution aerial image data set;
s2: constructing a fuzzy kernel pool according to the degraded unmanned aerial vehicle aerial image data set;
s3: constructing an image degradation model according to the fuzzy kernel pool;
s4: carrying out degradation processing on the high-resolution aerial image data set through the image degradation model to generate a high-low resolution image data set;
s5: constructing a residual distillation network, and training the residual distillation network by using the fuzzy kernel pool and the high-low resolution image to a data set to obtain a trained residual distillation network;
s6: and reconstructing an ultra-resolution image through the trained residual distillation network.
2. The blind super-resolution reconstruction method for aerial images based on residual distillation network as claimed in claim 1, wherein in step S2, extracting each image in said degraded unmanned aerial vehicle aerial image dataset by generating a countermeasure network to obtain a corresponding blur kernel, thereby constructing a blur kernel pool suitable for the unmanned aerial vehicle aerial environment.
3. The blind super-resolution reconstruction method for aerial images based on residual distillation network as claimed in claim 1, wherein the image degradation model is:
xdataset＝(y*kpool)↓s+n
wherein x isdatasetRepresenting degraded data, y representing data of a degradation model of the input image, kpoolRepresent the fuzzy core pool, ↓sRepresenting a down-sampling operation with a scale factor s and n representing noise.
4. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 1, wherein in the residual distillation network, the input image is firstly characterized by extracting features through a residual structure, and each residual structure is led out a distillation residual branch, the distillation residual branches are realized through a 1 x 1 convolutional layer, and finally the feature information of each distillation residual branch is spliced to complete the information fusion operation, specifically as follows:
Fdistilled_1，F1＝fconv1×1(xin)，fconv3×3(xin)+xin
Fdistilled_2，F2＝fconv1×1(F1)，fconv3×3(F1)+F1
Fdistilled_3，F3＝fconv1×1(F2)，fconv3×3(F2)+F2
Fdistilled_4＝fconv1×1(F3)
Fout＝fconcat(Fdistilled_1，Fdistilled_2，Fdistilled_3，Fdistilled_4)
wherein, Fdistilled_1、Fdistilled_2、Fdistilled_3、Fdistilled_4Each represents a characteristic graph of the residual after distillation by 1X 1 convolution, F1、F2、F3Respectively representing the feature maps output by the first, second and third residual structureconv1×1Denotes a 1 × 1 convolution operation, fconv3×3Denotes a 3 × 3 convolution operation, xinRepresenting an input image, FoutRepresenting the output of the residual distillation network, fconcatThe splicing operation of the characteristic diagrams is shown.
5. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 4, wherein in the process of training the residual distillation network, the method further comprises: and obtaining fuzzy kernel characteristics by mapping transformation operation on the fuzzy kernels input into the residual distillation network, and splicing the fuzzy kernel characteristics with each residual characteristic graph.
6. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 5, wherein the information input into the residual distillation network is assumed to include a blur kernel with size p × p and noise level δ, and a low resolution image with size W × H × C, wherein p represents the size of the blur kernel, and W, H, C represents the width, height and number of channels of the low resolution image respectively;
firstly, vectorizing a fuzzy core to obtain a normalized size p2And the vector of x 1 is mapped into a linear space with the dimension of t by a principal component analysis technology, and finally the vector of the dimension of t is spliced with the low-latitude characteristic and the noise level characteristic of the low-resolution image to form a degradation characteristic diagram with the size of W x H x (C + 1).
7. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 1, wherein the residual distillation network comprises two times of up-sampling reconstruction.
8. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 7, wherein the first upsampling reconstruction is:
wherein, Oupsample_1Representing the output of the first up-sampled reconstruction, FiRepresenting the output of the ith distillation residual branch in the residual distillation network, F0Represents the output of the first convolution block of the input low resolution image in the residual distillation network, and R (-) represents a 3 × 3 convolution and a ReLU activation function operation.
9. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 7, wherein the second upsampling reconstruction is;
Oupsample_2＝R(F0)
wherein, Oupsample_2Representing the output of the second upsampled reconstruction, F0Indicating low score of inputResolution image output at the first convolution block in the residual distillation network, R (.) represents a 3 × 3 convolution and a ReLU activation function operation.
10. The blind super-resolution reconstruction method for aerial images based on the residual distillation network as claimed in claim 1, further comprising: and evaluating the reconstruction effect through two evaluation indexes of the peak signal-to-noise ratio and the structural similarity of the reconstructed super-resolution image and the original high-resolution aerial image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110963671.8A CN113724134A (en) | 2021-08-20 | 2021-08-20 | Aerial image blind super-resolution reconstruction method based on residual distillation network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110963671.8A CN113724134A (en) | 2021-08-20 | 2021-08-20 | Aerial image blind super-resolution reconstruction method based on residual distillation network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113724134A true CN113724134A (en) | 2021-11-30 |
Family
ID=78677143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110963671.8A Pending CN113724134A (en) | 2021-08-20 | 2021-08-20 | Aerial image blind super-resolution reconstruction method based on residual distillation network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113724134A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115965609A (en) * | 2023-01-03 | 2023-04-14 | 江南大学 | Intelligent detection method for ceramic substrate defects by knowledge distillation |
| CN117274067A (en) * | 2023-11-22 | 2023-12-22 | 浙江优众新材料科技有限公司 | Light field image blind super-resolution processing method and system based on reinforcement learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112927137A (en) * | 2021-03-08 | 2021-06-08 | 大连理工大学 | Method, device and storage medium for acquiring blind super-resolution image |
| CN113096013A (en) * | 2021-03-31 | 2021-07-09 | 南京理工大学 | Blind image super-resolution reconstruction method and system based on imaging modeling and knowledge distillation |
| CN113139904A (en) * | 2021-04-29 | 2021-07-20 | 厦门大学 | Image blind super-resolution method and system |
| CN113240580A (en) * | 2021-04-09 | 2021-08-10 | 暨南大学 | Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation |
-
2021
- 2021-08-20 CN CN202110963671.8A patent/CN113724134A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112927137A (en) * | 2021-03-08 | 2021-06-08 | 大连理工大学 | Method, device and storage medium for acquiring blind super-resolution image |
| CN113096013A (en) * | 2021-03-31 | 2021-07-09 | 南京理工大学 | Blind image super-resolution reconstruction method and system based on imaging modeling and knowledge distillation |
| CN113240580A (en) * | 2021-04-09 | 2021-08-10 | 暨南大学 | Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation |
| CN113139904A (en) * | 2021-04-29 | 2021-07-20 | 厦门大学 | Image blind super-resolution method and system |
Non-Patent Citations (2)
| Title |
|---|
| JIE LIU 等: "Residual Feature Distillation Network for Lightweight Image Super-Resolution", 30 January 2021, IMAGE AND VIDEO PROCESSING, pages: 1 - 14 * |
| RUOFAN ZHOU 等: "Kernel Modeling Super-Resolution on Real Low-Resolution Images", 27 February 2020, 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), pages: 2433 - 2434 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115965609A (en) * | 2023-01-03 | 2023-04-14 | 江南大学 | Intelligent detection method for ceramic substrate defects by knowledge distillation |
| CN115965609B (en) * | 2023-01-03 | 2023-08-04 | 江南大学 | Intelligent detection method for flaws of ceramic substrate by utilizing knowledge distillation |
| CN117274067A (en) * | 2023-11-22 | 2023-12-22 | 浙江优众新材料科技有限公司 | Light field image blind super-resolution processing method and system based on reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111598778B (en) | Super-resolution reconstruction method for insulator image | |
| Zhang et al. | One-two-one networks for compression artifacts reduction in remote sensing | |
| CN110136062B (en) | Super-resolution reconstruction method combining semantic segmentation | |
| CN112529776B (en) | Training method of image processing model, image processing method and device | |
| CN113837938A (en) | Super-resolution method for reconstructing potential image based on dynamic vision sensor | |
| CN110942424A (en) | Composite network single image super-resolution reconstruction method based on deep learning | |
| Li et al. | Underwater image high definition display using the multilayer perceptron and color feature-based SRCNN | |
| CN113724134A (en) | Aerial image blind super-resolution reconstruction method based on residual distillation network | |
| CN112614061A (en) | Low-illumination image brightness enhancement and super-resolution method based on double-channel coder-decoder | |
| CN113592726A (en) | High dynamic range imaging method, device, electronic equipment and storage medium | |
| CN115170915A (en) | Infrared and visible light image fusion method based on end-to-end attention network | |
| CN111951164A (en) | Image super-resolution reconstruction network structure and image reconstruction effect analysis method | |
| Liu et al. | Research on super-resolution reconstruction of remote sensing images: A comprehensive review | |
| CN113284061A (en) | Underwater image enhancement method based on gradient network | |
| CN110570375B (en) | Image processing method, device, electronic device and storage medium | |
| CN115713462A (en) | Super-resolution model training method, image recognition method, device and equipment | |
| CN113298744B (en) | End-to-end infrared and visible light image fusion method | |
| CN116977651B (en) | Image denoising method based on double-branch and multi-scale feature extraction | |
| CN114022356A (en) | River course flow water level remote sensing image super-resolution method and system based on wavelet domain | |
| CN113379606A (en) | Face super-resolution method based on pre-training generation model | |
| CN116452469B (en) | Image defogging processing method and device based on deep learning | |
| CN112686830A (en) | Super-resolution method of single depth map based on image decomposition | |
| CN116934592A (en) | Image stitching method, system, equipment and medium based on deep learning | |
| CN113808039A (en) | Transfer learning defogging method and system based on Gaussian process mapping | |
| Wang et al. | MSF-Net: Multi-Scale Feedback Reconstruction for Guided Depth Map Super-Resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |