CN113298718A - Single image super-resolution reconstruction method and system - Google Patents
Single image super-resolution reconstruction method and system Download PDFInfo
- Publication number
- CN113298718A CN113298718A CN202110689245.XA CN202110689245A CN113298718A CN 113298718 A CN113298718 A CN 113298718A CN 202110689245 A CN202110689245 A CN 202110689245A CN 113298718 A CN113298718 A CN 113298718A
- Authority
- CN
- China
- Prior art keywords
- resolution
- module
- image
- super
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000012549 training Methods 0.000 claims abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 14
- 230000006872 improvement Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 26
- 230000004913 activation Effects 0.000 claims description 15
- 230000008447 perception Effects 0.000 claims description 6
- 238000007670 refining Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a single image super-resolution reconstruction method and a single image super-resolution reconstruction system. The method comprises the following steps: improving a generator based on an image super-resolution reconstruction model for generating a countermeasure network by using the low-resolution image dataset; training the improved image super-resolution reconstruction model based on the generation countermeasure network by using a low-resolution image data set; reconstructing the high-resolution image of the low-resolution image by using the trained reconstruction model; the improvement of the generator comprises: constructing a header module of the generator using a layer of convolution; adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image. The method and the device can improve the expression performance of the network, and further improve the quality of the reconstructed super-resolution image.
Description
Technical Field
The invention relates to the technical field of information, in particular to a single image super-resolution reconstruction method and a single image super-resolution reconstruction system,
background
Single Image Super-Resolution (SISR) reconstruction refers to a technique for constructing a High-Resolution (HR) Image from an existing Low-Resolution (LR) Image by using Image processing and machine learning techniques, and aims to solve the problems of blurring and Low Resolution of an obtained Image due to limitation of an imaging sensor or Image acquisition equipment. Because the high-resolution image can provide rich texture detail information, the super-resolution image reconstruction has application and research values, for example, the super-resolution reconstruction of the medical image can be realized, a doctor can carry out more accurate disease diagnosis through the restored clear medical image, the super-resolution reconstruction of a video acquired by monitoring equipment in a public place can be realized, and people can find an object to be identified more easily. In addition, Super-Resolution (SR) reconstruction techniques can also help improve the performance of other computer vision tasks, such as image classification, object recognition, semantic segmentation, and the like.
With the development of computer hardware and software technology, the deep learning technology has been widely applied to single image super-resolution reconstruction, and the development of SISR is promoted. In the known technology of single image super-resolution reconstruction, Dong and the like firstly use a convolutional neural network in the super-resolution reconstruction problem, and the image super-resolution (SRCNN) based on the convolutional neural network is provided, so that the end-to-end super-resolution reconstruction is realized; kim and the like deepen the network depth to enlarge the receptive field, obtain enough detail information, and introduce residual learning to provide an image super-resolution reconstruction method based on a convolutional neural network, so that the image reconstruction effect is improved; shi et al propose an effective sub-pixel convolution layer to achieve the amplification of the image, which reduces computational complexity due to the direct convolution operation on the low resolution image; kim et al propose an image super-resolution reconstruction method based on a recursive network, which introduces recursive supervised learning to reduce model parameters; to prevent the reconstructed image from being excessively smooth, Ledig et al perform super-resolution image reconstruction (SRGAN) by generating a countermeasure network, proposing a perceptual loss optimization model; an image super-resolution reconstruction algorithm of a multi-scale intensive residual error network is provided by a self-heating furnace and the like so as to solve the problem that the utilization of detail information in a low-resolution image is not sufficient in the current mainstream algorithm; the Wang Dong winter and the like provide an improved image super-resolution algorithm for generating the confrontation network aiming at the factors of unstable training, difficult convergence and the like of a network model for improving the visual effect of generating the confrontation network reconstruction image.
However, in the above-described technology, the expression performance of the network is to be improved. Therefore, a method or a system for reconstructing super-resolution of a single image is needed to improve the expression performance of a network and further improve the quality of the reconstructed super-resolution image.
Disclosure of Invention
The invention aims to provide a single image super-resolution reconstruction method and a single image super-resolution reconstruction system, which can improve the expression performance of a network and further improve the quality of a reconstructed super-resolution image.
In order to achieve the purpose, the invention provides the following scheme:
a single image super-resolution reconstruction method comprises the following steps:
acquiring a high-resolution image data set, and determining a low-resolution image data set corresponding to the high-resolution image data set by utilizing a downsampling mode;
acquiring an image super-resolution reconstruction model based on a generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
refining the generator with the low resolution image dataset;
determining an improved image super-resolution reconstruction model based on a generation countermeasure network by utilizing an improved generator and a discriminator;
training the improved image super-resolution reconstruction model based on the generation countermeasure network by using a low-resolution image data set;
reconstructing a high-resolution image of a low-resolution image by using a trained image super-resolution reconstruction model based on a generated countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
Optionally, the training of the improved image super-resolution reconstruction model based on the generation countermeasure network by using the low-resolution image dataset specifically includes:
and training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
Optionally, the building a header module of the generator by using a layer of convolution specifically includes:
shallow features of the low resolution image are extracted using convolution kernels of size 3 x 64 and the PReLU activation function.
Optionally, the building an enhanced residual module by adding one deconvolution layer before convolution of each layer of the basic residual module specifically includes:
deconvolution and convolutional layers use convolution kernels of size 8 × 8 × 64.
Optionally, the constructing a backbone module of the generator according to the fusion of the multiple enhanced residual modules and the multi-level residual features specifically includes:
and performing multi-level residual error feature fusion by adopting a convolution kernel with the size of 1 × 1 × 64 and the PReLU activation function.
Optionally, the reconstructing the deep features with the reconstruction module to perform high-resolution image reconstruction specifically includes:
refining the fused features by using a convolution kernel with the size of 3 multiplied by 64 and a PReLU activation function;
utilizing the sub-pixel convolution layer to up-sample the image of the thinned features to a target size; the target size is a high resolution image size;
the upsampled image is restored to an image with more detailed information using a 33 convolution kernel.
A single image super-resolution reconstruction system, comprising:
the low-resolution image data set determining module is used for acquiring a high-resolution image data set and determining a low-resolution image data set corresponding to the high-resolution image data set by utilizing a down-sampling mode;
the reconstruction model acquisition module is used for acquiring an image super-resolution reconstruction model based on the generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
a generator refinement module to refine the generator with the low resolution image dataset;
the improved reconstruction model determining module is used for determining an improved image super-resolution reconstruction model based on the generation countermeasure network by utilizing the improved generator and the discriminator;
the reconstruction model training module is used for training the improved image super-resolution reconstruction model based on the generation countermeasure network by utilizing the low-resolution image data set;
the high-resolution image reconstruction module is used for reconstructing a high-resolution image of a low-resolution image by utilizing a trained image super-resolution reconstruction model based on a generation countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
Optionally, the reconstruction model training module specifically includes:
and the reconstruction model training unit is used for training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the single-image super-resolution reconstruction method and the single-image super-resolution reconstruction system, a multi-stage feature fusion structure is added for a generator in a generation countermeasure network, and a deconvolution layer is introduced into a basic residual module, so that the feature size can be enlarged, the resolution is improved, and the expression performance of the network is improved. The invention fully extracts the characteristic information of the input low-resolution image, enlarges the receptive field of the network, obtains more context information and improves the network reconstruction performance. And performing a local feature fusion method between different convolutional layers of the enhanced residual error module, and combining a local residual error learning method and a global residual error learning method to obtain more high-frequency detail information of the input low-resolution image, thereby reconstructing a high-quality super-resolution image. The method has obvious improvement on the peak signal-to-noise ratio (PSNR) of objective indexes, Structural Similarity (SSIM) and visual effect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of a single image super-resolution reconstruction method provided by the present invention;
FIG. 2 is a schematic overall flow chart of a single image super-resolution reconstruction method provided by the present invention;
FIG. 3 is a schematic diagram of a generator according to the present invention;
FIG. 4 is a schematic diagram of an enhanced residual error module according to the present invention;
FIG. 5 is a schematic diagram of an arbiter provided in the present invention;
FIG. 6 is a comparison graph of subjective effects of a single image super-resolution reconstruction method and an existing method provided by the present invention;
fig. 7 is a schematic structural diagram of a single image super-resolution reconstruction system provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a single image super-resolution reconstruction method and a single image super-resolution reconstruction system, which can improve the expression performance of a network and further improve the quality of a reconstructed super-resolution image.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a single image super-resolution reconstruction method provided by the present invention, fig. 2 is a schematic flow chart of an entire single image super-resolution reconstruction method provided by the present invention, and as shown in fig. 1 and fig. 2, the single image super-resolution reconstruction method provided by the present invention includes:
s101, acquiring a high-resolution image data set, and determining a low-resolution image data set corresponding to the high-resolution image data set by utilizing a down-sampling mode;
s102, acquiring an image super-resolution reconstruction model based on the generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
s103, improving the generator by utilizing the low-resolution image data set;
s104, determining an improved image super-resolution reconstruction model based on a generation countermeasure network by using the improved generator and the discriminator;
s105, training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the low-resolution image data set;
s105 specifically comprises the following steps:
and training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
S106, reconstructing a high-resolution image of the low-resolution image by using the trained image super-resolution reconstruction model based on the generation countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
The head module for constructing the generator by using one layer of convolution specifically comprises:
shallow features of the low resolution image are extracted using convolution kernels of size 3 x 64 and the PReLU activation function.
Adding a deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module, specifically comprising:
deconvolution and convolutional layers use convolution kernels of size 8 × 8 × 64.
The constructing of the backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual features specifically comprises:
and performing multi-level residual error feature fusion by adopting a convolution kernel with the size of 1 × 1 × 64 and the PReLU activation function.
The reconstructing the deep features with the reconstruction module to perform high-resolution image reconstruction specifically includes:
refining the fused features by using a convolution kernel with the size of 3 multiplied by 64 and a PReLU activation function;
utilizing the sub-pixel convolution layer to up-sample the image of the thinned features to a target size; the target size is a high resolution image size;
the upsampled image is restored to an image with more detailed information using a 33 convolution kernel.
As shown in fig. 3, the improvement of the generator by using the low resolution image dataset specifically includes:
step 1: shallow feature extraction for low resolution images using a layer of convolution, assume ILRRepresenting an input low resolution image, in particular using a convolution kernel of size 3 × 3 × 64 and a PReLU activation function, the extracted shallow feature E0Represented by the formula:
E0＝δ(H(ILR))；
h and delta respectively represent convolution operation and an activation function, and the obtained low-resolution characteristic diagram is input to a backbone module to extract more detailed information;
step 2: extracting deep features of the image, wherein the proposed backbone module is formed by fusing N enhanced residual modules and multi-level residual features; the enhancement residual module introduces a deconvolution operation before the convolution layer of the base residual module, so that the network can extract more high-frequency detail information by using more context information, and the deconvolution layer and the convolution layer in the enhancement residual module adopt convolution kernels with the size of 8 × 8 × 64, as shown in fig. 4, and the operation process is represented by the following formula:
wherein E isn-1Is an input feature of the nth enhancement residual module,
representing a feature map, DP, extracted by a first-level deconvolution and convolution operation1Representing layer 1 deconvolution and PReLU Activate function operations, HP1Activating function operations for first-layer convolution and PReLU, DP2Representing the layer 2 deconvolution and PReLU activation function operations, H2Representing convolution operations of the second layer, EnThe output of the nth enhancement residual module;
shallow feature E extracted from head module0Extracting more representative features as input of a backbone module, and then performing convolution on shallow features E through a layer of convolution0And features extracted by the N enhanced residual modules are fused, so that the extracted hierarchical features are fully utilized, and the expression capacity of the network is improved. Particularly adopt the size of
1 × 1 × 64 convolution kernel and the PReLU activation function, the operation process of the backbone module is represented by the following formula:
En＝ERBn(En-1)＝ERBn(ERBn-1(...ERB1(E0)))；
EUP＝δ(H[E0，E1，E2，...，En])；
wherein ERBnRepresenting the nth enhancement residual block function, En-1Represents the input of the nth enhancement residual module, EnRepresenting their respective output characteristics, δ representing the PReLU activation function, H representing a 1 × 1 convolutional layer, Ei(i ═ 1, 2, 3.. n; i ≠ 0) denotes the output characteristics of the ith enhancement residual module, E0Shallow features extracted for the head module, [ E ]0，E1，E2，...，En]Indicating the connection of the hierarchical features extracted by the enhanced residual module, EUPIs the output characteristic of the backbone module;
and step 3: during the reconstruction phase of the model, the fused features E are first paired by a convolution kernel of size 3X 64 and the PReLU activation functionUPThinning, and then up-sampling the thinned feature map to a target size through a sub-pixel convolution layer, namely, the feature map is matched with a high-resolution image IHRThe size is the same, and finally, the upsampled features are restored into a high-resolution image I with more detailed information through a 3 multiplied by 3 convolution kernelSR：
ISR＝R(EUP)＝F(ILR)；
Wherein R represents the image reconstruction function of the model and F is the low resolution image ILRImage and high resolution image IHRAn end-to-end mapping function between images;
the method comprises the following steps of determining an improved image super-resolution reconstruction model based on a generation countermeasure network by using an improved generator and a discriminator, and specifically comprises the following steps:
inputting the image generated by the generator into a discriminator network, and judging the probability that the received data is a real sample by a discriminator according to the data distribution of the discriminator, wherein the structure of the discriminator is shown in FIG. 5; calculating the difference between the high-resolution image reconstructed by the network and the original high-resolution image through a loss function, minimizing the difference, seeking the optimal solution of model parameters, optimizing the network model by using the perceptual loss, and updating the parameters, wherein the perceptual loss is the weighted sum of content loss and countervailing loss;
content loss is the calculation of semantic features between the reconstructed super-resolution image and the high-resolution image, first, a pre-trained VGG19 network is loaded and represented as
Generating a high-resolution image I by a generator modelSRSecondly, the high resolution image I generatedSRAnd its corresponding high resolution image IHRIs sent into
Network and semantic feature extraction at layer I
The feature sizes extracted from the two images are the same, and finally, the mean square error between the two image feature maps is calculated as follows:
wherein the content of the first and second substances,
which is indicative of a loss of content,
indicating the Loading of the pretrained VGG19 network, ILRFor input low resolution images, ISRRepresenting the generation of a high resolution image by a generator model,
and
respectively representing the high resolution image I to be reconstructedSRAnd its corresponding original high resolution image IHRIs sent into
The network extracts the obtained semantic features at the layer I, and N represents the number of training samples;
to combat the loss
Calculated using the formula:
wherein the content of the first and second substances,
representing a super-resolution image reconstructed by inputting a low-resolution image into a generator,
the probability that the nth generated image is input into a discriminator to be judged to be a real high-resolution image is shown, and N represents the number of training samples;
loss of perception
Calculated using the formula:
to verify the effectiveness of the present invention, the present example sets the magnification scale factor to 4, and a comparison experiment was performed on three test images, and the result of reconstruction is shown in fig. 6.
By comparing the image generated by the invention with the image generated by the SRGAN model, the result of SRGAN reconstruction lacks high-frequency detail information, and the image is smoother.
Fig. 7 is a schematic structural diagram of a single image super-resolution reconstruction system provided by the present invention, and as shown in fig. 7, the single image super-resolution reconstruction system provided by the present invention includes:
a low resolution image data set determining module 701, configured to obtain a high resolution image data set, and determine a low resolution image data set corresponding to the high resolution image data set by using a downsampling manner;
a reconstruction model obtaining module 702, configured to obtain a super-resolution image reconstruction model based on the generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
a generator refinement module 703 for refining the generator with the low resolution image dataset;
an improved reconstruction model determination module 704 for determining an improved super-resolution image reconstruction model based on a generation countermeasure network using an improved generator and a discriminator;
a reconstruction model training module 705, configured to train an improved super-resolution image reconstruction model based on the generated countermeasure network with a low-resolution image dataset;
a high resolution image reconstruction module 706, configured to reconstruct a high resolution image of the low resolution image by using the trained image super-resolution reconstruction model based on the generated countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
The reconstruction model training module 705 specifically includes:
and the reconstruction model training unit is used for training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (8)
1. A single image super-resolution reconstruction method is characterized by comprising the following steps:
acquiring a high-resolution image data set, and determining a low-resolution image data set corresponding to the high-resolution image data set by utilizing a downsampling mode;
acquiring an image super-resolution reconstruction model based on a generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
refining the generator with the low resolution image dataset;
determining an improved image super-resolution reconstruction model based on a generation countermeasure network by utilizing an improved generator and a discriminator;
training the improved image super-resolution reconstruction model based on the generation countermeasure network by using a low-resolution image data set;
reconstructing a high-resolution image of a low-resolution image by using a trained image super-resolution reconstruction model based on a generated countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
2. The single image super-resolution reconstruction method according to claim 1, wherein the training of the improved image super-resolution reconstruction model based on the generative countermeasure network using the low-resolution image dataset specifically comprises:
and training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
3. The single image super-resolution reconstruction method according to claim 1, wherein the constructing the head module of the generator by using a layer of convolution specifically comprises:
shallow features of the low resolution image are extracted using convolution kernels of size 3 x 64 and the PReLU activation function.
4. The single image super-resolution reconstruction method according to claim 1, wherein the building of the enhanced residual module by adding a deconvolution layer before each convolution of the basic residual module specifically comprises:
deconvolution and convolutional layers use convolution kernels of size 8 × 8 × 64.
5. The single image super-resolution reconstruction method according to claim 1, wherein the constructing the backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual features specifically comprises:
and performing multi-level residual error feature fusion by adopting a convolution kernel with the size of 1 × 1 × 64 and the PReLU activation function.
6. The single image super-resolution reconstruction method according to claim 1, wherein the reconstructing the deep features with the reconstruction module to obtain the high-resolution image specifically comprises:
refining the fused features by using a convolution kernel with the size of 3 multiplied by 64 and a PReLU activation function;
utilizing the sub-pixel convolution layer to up-sample the image of the thinned features to a target size; the target size is a high resolution image size;
the upsampled image is restored to an image with more detailed information using a 33 convolution kernel.
7. A single image super-resolution reconstruction system is characterized by comprising:
the low-resolution image data set determining module is used for acquiring a high-resolution image data set and determining a low-resolution image data set corresponding to the high-resolution image data set by utilizing a down-sampling mode;
the reconstruction model acquisition module is used for acquiring an image super-resolution reconstruction model based on the generated countermeasure network; the image super-resolution reconstruction model based on the generation countermeasure network comprises the following steps: a generator and a discriminator;
a generator refinement module to refine the generator with the low resolution image dataset;
the improved reconstruction model determining module is used for determining an improved image super-resolution reconstruction model based on the generation countermeasure network by utilizing the improved generator and the discriminator;
the reconstruction model training module is used for training the improved image super-resolution reconstruction model based on the generation countermeasure network by utilizing the low-resolution image data set;
the high-resolution image reconstruction module is used for reconstructing a high-resolution image of a low-resolution image by utilizing a trained image super-resolution reconstruction model based on a generation countermeasure network;
the improvement of the generator comprises:
constructing a header module of the generator using a layer of convolution; the head module is used for extracting shallow features of the low-resolution image by utilizing a layer of convolution;
adding a layer of deconvolution layer before convolution of each layer of the basic residual error module to construct an enhanced residual error module; constructing a backbone module of the generator according to the fusion of the plurality of enhanced residual modules and the multi-level residual characteristics; the backbone module is used for fusing the shallow feature and the high-frequency information extracted by the residual module to extract a deep feature;
and reconstructing the deep features by using a reconstruction module to obtain a high-resolution image.
8. The single image super-resolution reconstruction system according to claim 7, wherein the reconstruction model training module specifically comprises:
and the reconstruction model training unit is used for training the improved image super-resolution reconstruction model based on the generation countermeasure network by using the perception loss function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110689245.XA CN113298718A (en) | 2021-06-22 | 2021-06-22 | Single image super-resolution reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110689245.XA CN113298718A (en) | 2021-06-22 | 2021-06-22 | Single image super-resolution reconstruction method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113298718A true CN113298718A (en) | 2021-08-24 |
Family
ID=77328990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110689245.XA Pending CN113298718A (en) | 2021-06-22 | 2021-06-22 | Single image super-resolution reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298718A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114037843A (en) * | 2021-11-10 | 2022-02-11 | 南宁桂电电子科技研究院有限公司 | Method for improving resolution of underwater image based on improved generation countermeasure network |
| CN114549308A (en) * | 2022-01-28 | 2022-05-27 | 大连大学 | Perception-oriented image super-resolution reconstruction method and system with large receptive field |
| CN114841961A (en) * | 2022-05-05 | 2022-08-02 | 扬州大学 | Wheat scab detection method based on image enhancement and improvement of YOLOv5 |
| CN115187463A (en) * | 2022-09-13 | 2022-10-14 | 云南省交通规划设计研究院有限公司 | Landslide remote sensing image set super-resolution reconstruction method and system |
| CN115239564A (en) * | 2022-08-18 | 2022-10-25 | 中国矿业大学 | Mine image super-resolution reconstruction method combining semantic information |
| WO2023060746A1 (en) * | 2021-10-14 | 2023-04-20 | 中国科学院深圳先进技术研究院 | Small image multi-object detection method based on super-resolution |
| CN115994857A (en) * | 2023-01-09 | 2023-04-21 | 深圳大学 | Video super-resolution method, device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189253A (en) * | 2019-04-16 | 2019-08-30 | 浙江工业大学 | A kind of image super-resolution rebuilding method generating confrontation network based on improvement |
| CN110211045A (en) * | 2019-05-29 | 2019-09-06 | 电子科技大学 | Super-resolution face image method based on SRGAN network |
| CN111429355A (en) * | 2020-03-30 | 2020-07-17 | 新疆大学 | Image super-resolution reconstruction method based on generation countermeasure network |
| CN111932456A (en) * | 2020-07-31 | 2020-11-13 | 浙江师范大学 | Single image super-resolution reconstruction method based on generation countermeasure network |
| CN112950480A (en) * | 2021-04-15 | 2021-06-11 | 辽宁工程技术大学 | Super-resolution reconstruction method integrating multiple receptive fields and dense residual attention |
-
2021
- 2021-06-22 CN CN202110689245.XA patent/CN113298718A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189253A (en) * | 2019-04-16 | 2019-08-30 | 浙江工业大学 | A kind of image super-resolution rebuilding method generating confrontation network based on improvement |
| CN110211045A (en) * | 2019-05-29 | 2019-09-06 | 电子科技大学 | Super-resolution face image method based on SRGAN network |
| CN111429355A (en) * | 2020-03-30 | 2020-07-17 | 新疆大学 | Image super-resolution reconstruction method based on generation countermeasure network |
| CN111932456A (en) * | 2020-07-31 | 2020-11-13 | 浙江师范大学 | Single image super-resolution reconstruction method based on generation countermeasure network |
| CN112950480A (en) * | 2021-04-15 | 2021-06-11 | 辽宁工程技术大学 | Super-resolution reconstruction method integrating multiple receptive fields and dense residual attention |
Non-Patent Citations (3)
| Title |
|---|
| CHRISTIAN LEDIG 等: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| LIU CHUNYU 等: "Single Image Super-resolution Reconstruction Based on Enhanced Residual Network", 《JOURNAL OF PHYSICS: CONFERENCE SERIES》 * |
| 张宁 等: "基于深度学习的单幅图片超分辨率重构研究进展", 《自动化学报》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023060746A1 (en) * | 2021-10-14 | 2023-04-20 | 中国科学院深圳先进技术研究院 | Small image multi-object detection method based on super-resolution |
| CN114037843A (en) * | 2021-11-10 | 2022-02-11 | 南宁桂电电子科技研究院有限公司 | Method for improving resolution of underwater image based on improved generation countermeasure network |
| CN114549308A (en) * | 2022-01-28 | 2022-05-27 | 大连大学 | Perception-oriented image super-resolution reconstruction method and system with large receptive field |
| CN114549308B (en) * | 2022-01-28 | 2024-03-22 | 大连大学 | Image super-resolution reconstruction method and system with large receptive field and oriented to perception |
| CN114841961A (en) * | 2022-05-05 | 2022-08-02 | 扬州大学 | Wheat scab detection method based on image enhancement and improvement of YOLOv5 |
| CN114841961B (en) * | 2022-05-05 | 2024-04-05 | 扬州大学 | Wheat scab detection method based on image enhancement and improved YOLOv5 |
| CN115239564A (en) * | 2022-08-18 | 2022-10-25 | 中国矿业大学 | Mine image super-resolution reconstruction method combining semantic information |
| CN115187463A (en) * | 2022-09-13 | 2022-10-14 | 云南省交通规划设计研究院有限公司 | Landslide remote sensing image set super-resolution reconstruction method and system |
| CN115994857A (en) * | 2023-01-09 | 2023-04-21 | 深圳大学 | Video super-resolution method, device, equipment and storage medium |
| CN115994857B (en) * | 2023-01-09 | 2023-10-13 | 深圳大学 | Video super-resolution method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113298718A (en) | Single image super-resolution reconstruction method and system | |
| CN110211045B (en) | Super-resolution face image reconstruction method based on SRGAN network | |
| CN112507997B (en) | Face super-resolution system based on multi-scale convolution and receptive field feature fusion | |
| CN111062872B (en) | Image super-resolution reconstruction method and system based on edge detection | |
| CN111768342B (en) | Human face super-resolution method based on attention mechanism and multi-stage feedback supervision | |
| CN109345456B (en) | Generation countermeasure network training method, image processing method, device, and storage medium | |
| CN110189255B (en) | Face detection method based on two-stage detection | |
| CN110648334A (en) | Multi-feature cyclic convolution saliency target detection method based on attention mechanism | |
| CN110570353A (en) | Dense connection generation countermeasure network single image super-resolution reconstruction method | |
| CN111681166A (en) | Image super-resolution reconstruction method of stacked attention mechanism coding and decoding unit | |
| CN111275686B (en) | Method and device for generating medical image data for artificial neural network training | |
| CN112949636B (en) | License plate super-resolution recognition method, system and computer readable medium | |
| CN116309648A (en) | Medical image segmentation model construction method based on multi-attention fusion | |
| CN113538246B (en) | Remote sensing image super-resolution reconstruction method based on unsupervised multi-stage fusion network | |
| CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
| CN112862689A (en) | Image super-resolution reconstruction method and system | |
| CN115578262A (en) | Polarization image super-resolution reconstruction method based on AFAN model | |
| CN116739899A (en) | Image super-resolution reconstruction method based on SAUGAN network | |
| CN112950480A (en) | Super-resolution reconstruction method integrating multiple receptive fields and dense residual attention | |
| CN115526777A (en) | Blind over-separation network establishing method, blind over-separation method and storage medium | |
| CN114187181B (en) | Dual-path lung CT image super-resolution method based on residual information refining | |
| Chen et al. | Attention-based broad self-guided network for low-light image enhancement | |
| CN112529908B (en) | Digital pathological image segmentation method based on cascade convolution network and model thereof | |
| CN117217997A (en) | Remote sensing image super-resolution method based on context perception edge enhancement | |
| CN116862765A (en) | Medical image super-resolution reconstruction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |