CN115564652B - Reconstruction method for super-resolution of image - Google Patents

Reconstruction method for super-resolution of image Download PDF

Info

Publication number
CN115564652B
CN115564652B CN202211206792.9A CN202211206792A CN115564652B CN 115564652 B CN115564652 B CN 115564652B CN 202211206792 A CN202211206792 A CN 202211206792A CN 115564652 B CN115564652 B CN 115564652B
Authority
CN
China
Prior art keywords
output
image
model
layer
generated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211206792.9A
Other languages
Chinese (zh)
Other versions
CN115564652A (en
Inventor
吴启航
杨欣
朱义天
李恒锐
樊江锋
周大可
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN202211206792.9A priority Critical patent/CN115564652B/en
Publication of CN115564652A publication Critical patent/CN115564652A/en
Application granted granted Critical
Publication of CN115564652B publication Critical patent/CN115564652B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a super-resolution reconstruction method for an image, which relates to the technical field of digital image enhancement, and can extract the characteristics of a low-resolution image to a greater extent so as to restore the texture details of a high-resolution image. The invention comprises the following steps: establishing a generation countermeasure network and a feature extractor, the generation countermeasure network consisting of at least two neural networks including: generating a model and a judging model; inputting an image to be processed into the generation model of the generation countermeasure network to obtain a first output; inputting the first output into a feature extractor, and obtaining a second output after feature extraction; inputting the first output into the discrimination model of the generated countermeasure network to obtain a third output; and fixing parameters of the feature extractor, taking the antagonism loss, the perception loss and the space distance between the generated image and the original image as optimization targets, and alternately updating the discrimination model and the generation model of the antagonism network to achieve the aim of optimizing the generation model.

Description

Reconstruction method for super-resolution of image
Technical Field
The invention relates to the technical field of digital image enhancement, in particular to a super-resolution reconstruction method for an image.
Background
The high-resolution image is an important medium for clearly expressing information such as spatial structure, detail characteristics, edge textures and the like of the image, and has very wide practical value in the fields of medicine, criminal investigation, satellites and the like. However, in the actual image acquisition process, the obtained Low Resolution image (LR) or Resolution cannot meet the processing requirement, so how to reconstruct the Low Resolution image becomes the direction to be studied.
Methods adopted in the current super-resolution image reconstruction with more use include an image reconstruction method based on interpolation, an image reconstruction method based on reconstruction, a reconstruction method based on learning, and the like. The super-resolution image reconstruction method based on reconstruction is widely used in the field of image processing and is mainly divided into a frequency domain method and a spatial domain method. And extracting the required image characteristic information by utilizing the plurality of low-resolution images and the unknown high-resolution image, and reconstructing the high-resolution image after estimating the high-resolution image characteristic information.
However, in the current scheme, the extraction degree of the features of the low-resolution image is not high, and further, it is difficult to restore the texture details of the high-resolution image more.
Disclosure of Invention
The embodiment of the invention provides a super-resolution reconstruction method for an image, which can extract the characteristics of a low-resolution image to a greater extent and restore the texture details of a high-resolution image.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical scheme:
step one, establishing a generated countermeasure network and a feature extractor, wherein the generated countermeasure network consists of at least three neural networks and comprises: generating a model, a judging model and a characteristic extractor;
inputting an image to be processed into the generation model of the generation countermeasure network to obtain a first output, wherein the first output is an image with higher resolution than the image to be processed;
step three, inputting the first output into a feature extractor, and obtaining a second output after feature extraction, wherein the second output comprises: a feature map corresponding to different depths of the first output;
inputting the first output into the discrimination model for generating the countermeasure network to obtain a third output, wherein the third output is expressed as a two-dimensional matrix, each value in the matrix represents the fidelity degree of the detail texture of one area in the image of the first output, and the magnitude of the value positively correlates with the fidelity degree of the detail texture of the corresponding area in the image;
training the discrimination model of the generated countermeasure network, and optimizing the generation model of the generated countermeasure network.
The method for reconstructing the image super-resolution provided by the embodiment of the invention has the main tasks of constructing a generation model and a discrimination model, training the generation model and the discrimination model, and finally obtaining an algorithm model capable of recovering the texture details of the low-resolution image. The generated challenge model is improved based on the ESRGAN model. The generation model considers adopting the RRDB structure to avoid gradient dispersion problem caused by deepening the layer number of the nerve model as much as possible. The discriminant model is based on the VGG model, and references Relativistic Discriminator and Patch Discriminator are made to enhance the trainability of the model. Finally, combining the multi-scale perception loss, so that the generator can implicitly learn the semantic information of the image. Taking the complexity of an actual picture to be reconstructed into consideration, taking 4 times of BiCubic downsampling by using a clear image to obtain a low-resolution image as a training set, and training a model by using the training set to reconstruct a compressed low-resolution image with the same degree. Considering that PSNR and SSIM are selected as performance evaluation indexes, a loss function is constructed with these two indexes as references. Therefore, the characteristics of the low-resolution image can be extracted to a greater extent, and further, the texture details of the high-resolution image can be restored.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic block diagram of an improved RRDB structure provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a channel attention mechanism according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a generated model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a discriminant model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a subpixel convolution according to an embodiment of the present disclosure;
FIG. 6 is a diagram illustrating a multi-scale perceptual loss provided by an embodiment of the present invention;
fig. 7 is a schematic flow chart of a method according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail below with reference to the drawings and detailed description for the purpose of better understanding of the technical solution of the present invention to those skilled in the art. Embodiments of the present invention will hereinafter be described in detail, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention. As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The method for reconstructing the super-resolution image provided by the embodiment of the invention, as shown in fig. 7, may include the following links:
step one, establishing a generated countermeasure network and a feature extractor.
Wherein the generating an countermeasure network is comprised of at least three neural networks including: generating a model, discriminating the model and a feature extractor. The feature extractor consists of at least one neural network, including a pre-trained classification network for extracting image features, such as: the feature extractor can be pre-trained on ImageNet without further training.
Inputting the image to be processed into the generation model of the generation countermeasure network to obtain a first output.
Wherein the first output is an image having a higher resolution than the image to be processed.
And thirdly, inputting the first output into a feature extractor, and obtaining a second output after feature extraction.
Wherein the second output comprises: a feature map corresponding to different depths of the first output.
And step four, inputting the first output into the discrimination model for generating the countermeasure network to obtain a third output.
Wherein the third output is represented as a two-dimensional matrix, each value in the matrix representing a respective degree of realism of the detail texture of a region in the image of the first output, the magnitude of the value being positively correlated with the degree of realism of the detail texture of the corresponding region in the image.
Training the discrimination model of the generated countermeasure network, and optimizing the generation model of the generated countermeasure network.
The parameters of the feature extractor are fixed, the space distance between the countermeasure loss, the perception loss and the generated image and the original image is used as an optimization target, and the discrimination model and the generation model of the countermeasure network are alternately updated and generated, so that the purpose of optimizing the generation model is achieved.
In this embodiment, the generating network is composed of an initial and an end two-layer normal convolution, and an intermediate 23-layer RRDB module, where a LeakyRelu function is used as an activation function, where the structure of each neural network that composes the generating countermeasure network includes at least: the device comprises an initial layer and a tail layer, wherein the initial layer is formed by a convolution layer, the tail layer is formed by sub-pixel convolution, and a total of 23RRDB modules are arranged between the initial layer and the tail layer;
the activation function of the generated countermeasure network is LeakyReLU
Where i represents the index of the vector or matrix, x i Represents the i-th value, y of the input i The i-th value representing the output, a being a constant for correcting x i A value less than 0.
Specifically, corresponding linear loss functions are designed according to the PSNR and SSIM values so as to promote index convergence. And PSNR and SSIM are used as indexes for evaluating the performance of the model, and PSNR is used as peak signal-to-noise ratio for measuring the difference between pixel values of the generated high-resolution image and the real high-resolution image; SSIM is a structural similarity that measures the degree of similarity in brightness, contrast, and structure of two images.
In this embodiment, in the first step, the method includes:
constructing a generation model using a dense residual block (RRDB), the generation model in a generation countermeasure network; the generation network adopts an RRDB structure to avoid gradient dispersion problem caused by deepening the layer number of the neural network as much as possible. Specifically, in a generated model Dense residual Block (RRDB), outputting features corresponding to the image to be processed through a Dense Block (Dense Block) module; in the generated model dense residual block (RRDB), each 3 residual blocks in the res net superimpose the outputs of the blocks with the outputs of the previous 3 layers to form a residual. As shown in fig. 1, to further improve the gradient vanishing problem, every 3 residual blocks superimpose their outputs with the outputs of the previous 3 layers to form a residual. The generation of the countermeasure network is improved based on the ESRGAN network. For example: channel attention mechanisms, sub-pixel convolution upsampling, patch Discriminator, and multi-scale perceptual loss are used.
In this embodiment, the DenseBlock module is the core of DenseNet, which is a convolutional neural network with tight connection, where any two layers are directly connected, i.e. the input of each layer in the network is the union of the outputs of all the previous layers, and the learned features of this layer are also directly transferred to all the following layers as inputs. As shown in fig. 1, the dense block consists of several convolutional layers and an active layer. Specifically, in each Dense Block: x is x l ＝H l ([x 0 ,x 1 ,…,x l-1 ]) Wherein l represents the number of layers, x l Characteristic diagram representing l-layer output, [ x ] 0 ，x 1 ，…，x l-1 ]Representing the concatenation of the layer 0,1, …, l-1 feature maps, H l Indicating that the dense block contains the convolution of 3*3 and the leakrele. The output of the convolution layer is a three-dimensional matrix, which can be thought of as a cuboid (three dimensions of which are long-by-wide-by-high), and the channel combination is to splice the two matrices together in that dimension. ResNet is then constructed from Residual blocks (Residual blocks) expressed as: x is x l ＝H l (x l-1 )+ x l-1 Wherein H is l Convolution and LeakyReLU, x representing 3*3 l-1 The output of the first layer is the output of the l-1 layer after the rolling sum of the output of the l-1 layer and the LeakyReLU. As in fig. 1, the result of the Dense Block is added to the result of the non-Dense Block and finally output together through the Dense Block.
In a preferred aspect of this embodiment, in the generating an antagonizing network, a channel attention mechanism is employed, the channel attention mechanism including: as shown in fig. 2, the formula is expressed as:wherein x is c Feature map representing output c-th channel of RRDB structure block,/>A feature map representing the c-th channel after passing through the channel attention mechanism, s representing x calculated by the channel attention mechanism c And s=f (W U δ(W D z), wherein f (·) and δ (·) represent Sigmoid and ReLU activation functions, respectively, z represents the global features of the input feature map, i.e. the input feature map x is obtained by a global averaging pooling layer. W (W) D 、W U The parameters respectively represent two different convolution kernels, the number of channels is reduced to 1/r by the convolution layer, r is the super parameter, and after the ReLU activation, the pass parameter is W U The number of channels is restored to the original size, and finally the weight s of each channel is obtained through a sigmoid function of a gating unit. Thereby enabling the residual output in the RRDB structure block to adaptively adjust the scaling.
And the last convolution layer of the generated model adopts sub-pixel convolution as an up-sampling module, the sub-pixel convolution is used for combining and splicing the values of the corresponding spatial positions of the corresponding plurality of feature images, and the image is filled under the condition of not influencing the spatial information of the image, and the schematic diagram is shown in fig. 5. The entire generative model is then constructed, for example as shown in fig. 3.
In this embodiment, the fifth step includes:
optimizing the generation model by taking the output distance between the real picture and the first output as a loss function:
in the method, in the process of the invention,representing a true high resolution image, y representing the generated image, ε being a constant 10 -3 Is a constant of (c).
Inputting the real picture and the picture generated by the generation model into the discrimination model, and constructing a loss function of the discrimination model and a counterloss of the generation model:
wherein x is r Representing a true high resolution image, x f Representing the generated high resolution image; distinguishing device D Ra Is defined asC (·) is the output of the arbiter, σ (·) is the Sigmoid function, ++>Representing an arithmetic operation that averages all of the generated data in the min-batch. Wherein a real picture can be understood as a real high resolution image.
Further, inputting the pictures generated by the generation model and the real pictures into the feature extractor to obtain feature graphs with different scalesAnd->Characteristic diagram representing the ith layer obtained by inputting the real image into the characteristic extraction network,/for>A feature map representing the ith layer obtained by inputting a high resolution image generated by the generation model into a feature extraction network. The data used in this embodiment are all true high-resolution images, and the corresponding low-resolution images are obtained by downsampling technique, which is designed to pass throughThe generated high-resolution image obtained by the generation model is as close as possible to a real high-resolution image.
Further calculate the distance between feature graphs of different scales, i.e. perceived lossSuperposing the perception losses of different scales to obtain a multi-scale perception loss function, wherein the formula is +.>Lambda in i Is->The weight in the multi-scale perceptual penalty is used to control the importance of the penalty function without scale. Wherein the parameters of the feature extractor are frozen during the training process described above, thereby constructing a multi-scale perceptual penalty, the schematic diagram being shown in fig. 6. The feature extraction network takes VGG19 as a main body and outputs different depths [ conv3_4, conv4_4 and conv5_4 ]]Con3_4, representing the fourth convolutional layer in the third block, conv4_4, conv5_4 can be understood in the same way. Wherein the feature extraction network is pre-trained on the ImageNet dataset such that the network is able to extract high-level semantic information of the image. Respectively inputting a picture generated by a generated model generation model and a real picture into a pre-training feature extraction network to obtain feature graphs with different scales, and further calculating L1 norms between the corresponding feature graphs>The perception loss is constructed by the method, wherein feature images with different scales have different receptive fields, a low network layer has smaller receptive fields, and the extraction capability of image texture details is better; the high network layer is more focused on the overall structure of the image. Superimposing the multi-scale feature map can make the image focus on texture details and preserve the overall structure, making the resulting image organoleptically close to a real image. The superposition method comprises the following steps: />Freezing features during trainingParameters of the network are extracted.
In this embodiment, the discriminant model is composed of 7 convolution layers with BN and LeakyRelu, as shown in fig. 4, the output of the model is a two-dimensional matrix, each value in the matrix represents the fidelity of the detail texture of the corresponding region of the high-resolution image, the higher the value is, the more vivid the region of the image, and the discriminant model adopts relative discriminant loss:
wherein x is r Representing a true high resolution image, x f Representing the generated high resolution image; distinguishing device D Ra Is defined asC (·) is the output of the arbiter, σ (·) is the Sigmoid function, ++>Representing an arithmetic operation that averages all of the generated data in the min-batch. Wherein a real picture can be understood as a real high resolution image.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points. The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (8)

1. A reconstruction method for super resolution of an image, comprising:
step one, establishing a generated countermeasure network and a feature extractor, wherein the generated countermeasure network consists of at least three neural networks and comprises: generating a model, a judging model and a characteristic extractor;
inputting an image to be processed into the generation model of the generation countermeasure network to obtain a first output, wherein the first output is an image with higher resolution than the image to be processed;
step three, inputting the first output into a feature extractor, and obtaining a second output after feature extraction, wherein the second output comprises: a feature map corresponding to different depths of the first output;
inputting the first output into the discrimination model for generating the countermeasure network to obtain a third output, wherein the third output is expressed as a two-dimensional matrix, each value in the matrix represents the fidelity degree of the detail texture of one area in the image of the first output, and the magnitude of the value positively correlates with the fidelity degree of the detail texture of the corresponding area in the image;
training the discrimination model of the generated countermeasure network, and optimizing the generation model of the generated countermeasure network;
the fifth step comprises the following steps:
optimizing the generation model by taking the output distance between the real picture and the first output as a loss function:
in the method, in the process of the invention,representing a true high resolution image, y representing the generated image, ε being a constantIs 10 -3 Constant of (2);
inputting the real picture and the picture generated by the generation model into the discrimination model, and constructing a loss function of the discrimination model and a counterloss of the generation model:
wherein x is r Representing a true high resolution image, x f Representing the generated high resolution image; distinguishing device D Ra Is defined asC (·) is the output of the arbiter, σ (·) is the Sigmoid function, ++>Representing an arithmetic operation that averages all generated data in the min-batch;
further comprises: inputting the pictures and the real pictures generated by the generated model into the feature extractor to obtain feature graphs with different scalesAnd-> Characteristic diagram representing the ith layer obtained by inputting the real image into the characteristic extraction network,/for>The representation is composed ofInputting the high-resolution image generated by the generated model into a feature map of an ith layer obtained by a feature extraction network;
further calculate the distance between feature graphs of different scales, i.e. perceived lossSuperposing the perception losses of different scales to obtain a multi-scale perception loss function, wherein the formula is +.>Lambda in i Is->The weight in the multi-scale perceptual penalty is used to control the importance of the penalty function without scale.
2. The method of claim 1, wherein the feature extractor consists of at least one neural network, including a pre-trained classification network, which is used to extract image features.
3. The method according to claim 1, characterized in that in the structure of each neural network constituting the generation countermeasure network, at least: the device comprises an initial layer and a tail layer, wherein the initial layer is formed by a convolution layer, the tail layer is formed by sub-pixel convolution, and a total of 23RRDB modules are arranged between the initial layer and the tail layer;
the activation function for generating the countermeasure network is a LeakyReLU:where i represents the index of the vector or matrix, x i Represents the i-th value, y of the input i The i-th value representing the output, a being a constant for correcting x i A value less than 0.
4. The method according to claim 2, wherein in the first step, comprising:
constructing a generation model using a dense residual block (RRDB), the generation model in a generation countermeasure network;
in the generation model Dense residual Block (RRDB), outputting the characteristics corresponding to the image to be processed through a Dense Block (Dense Block) module;
in the generated model dense residual block (RRDB), each 3 residual blocks in the res net superimpose the outputs of the blocks with the outputs of the previous 3 layers to form a residual.
5. The method of claim 4, wherein in each Dense Block:
x l ＝H l ([x 0 ,x 1 ,…,x l-1 ])
wherein l represents the number of layers, x l Characteristic diagram representing l-layer output, [ x ] 0 ,x 1 ,…,x l-1 ]Representing the concatenation of the layer 0,1, …, l-1 feature maps, H l Indicating that the dense block contains the convolution of 3*3 and the leakrele.
6. The method of claim 5, wherein res net is represented as:
x l ＝H l (x l-1 )+x l-1
wherein H is l Convolution and LeakyReLU, x representing 3*3 l-1 The output of the first layer is the output of the l-1 layer after the rolling sum of the output of the l-1 layer and the LeakyReLU.
7. The method according to any of claims 1-6, wherein in the generating an antagonizing network, a channel attention mechanism is employed, the channel attention mechanism comprising:
wherein x is c Feature map representing output c-th channel of RRDB structure block,/>A feature map representing the c-th channel after passing through the channel attention mechanism, s representing x calculated by the channel attention mechanism c And s=f (W U δ(W D z), where f (·) and δ (·) represent Sigmoid and ReLU activation functions, respectively, z represents the global features of the input feature map, W D 、W U The parameters respectively represent two different convolution kernels, the number of channels is reduced to 1/r by the convolution layer, r is the super parameter, and after the ReLU activation, the pass parameter is W U The number of channels is restored to the original size, and finally the weight s of each channel is obtained through a sigmoid function of a gating unit.
8. The method of claim 7, wherein a sub-pixel convolution is employed as an upsampling module at a last convolution layer of the generative model, the sub-pixel convolution being used to combine and splice values of corresponding spatial locations of the corresponding plurality of feature maps.
CN202211206792.9A 2022-09-30 2022-09-30 Reconstruction method for super-resolution of image Active CN115564652B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211206792.9A CN115564652B (en) 2022-09-30 2022-09-30 Reconstruction method for super-resolution of image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211206792.9A CN115564652B (en) 2022-09-30 2022-09-30 Reconstruction method for super-resolution of image

Publications (2)

Publication Number Publication Date
CN115564652A CN115564652A (en) 2023-01-03
CN115564652B true CN115564652B (en) 2023-12-01

Family

ID=84743183

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211206792.9A Active CN115564652B (en) 2022-09-30 2022-09-30 Reconstruction method for super-resolution of image

Country Status (1)

Country Link
CN (1) CN115564652B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110189308A (en) * 2019-05-17 2019-08-30 山东财经大学 A kind of lesion detection approach and device based on BM3D and the fusion of dense convolutional network
CN110706157A (en) * 2019-09-18 2020-01-17 中国科学技术大学 Face super-resolution reconstruction method for generating confrontation network based on identity prior
CN112001847A (en) * 2020-08-28 2020-11-27 徐州工程学院 Method for generating high-quality image by relatively generating antagonistic super-resolution reconstruction model
CN115018705A (en) * 2022-05-27 2022-09-06 南京航空航天大学 Image super-resolution method based on enhanced generation countermeasure network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110189308A (en) * 2019-05-17 2019-08-30 山东财经大学 A kind of lesion detection approach and device based on BM3D and the fusion of dense convolutional network
CN110706157A (en) * 2019-09-18 2020-01-17 中国科学技术大学 Face super-resolution reconstruction method for generating confrontation network based on identity prior
CN112001847A (en) * 2020-08-28 2020-11-27 徐州工程学院 Method for generating high-quality image by relatively generating antagonistic super-resolution reconstruction model
CN115018705A (en) * 2022-05-27 2022-09-06 南京航空航天大学 Image super-resolution method based on enhanced generation countermeasure network

Also Published As

Publication number Publication date
CN115564652A (en) 2023-01-03

Similar Documents

Publication Publication Date Title
CN110033410B (en) Image reconstruction model training method, image super-resolution reconstruction method and device
CN111047515B (en) Attention mechanism-based cavity convolutional neural network image super-resolution reconstruction method
CN110119780B (en) Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network
CN111369440B (en) Model training and image super-resolution processing method, device, terminal and storage medium
CN110570353B (en) Super-resolution reconstruction method for generating single image of countermeasure network by dense connection
CN112507997B (en) Face super-resolution system based on multi-scale convolution and receptive field feature fusion
CN110599401A (en) Remote sensing image super-resolution reconstruction method, processing device and readable storage medium
CN110717857A (en) Super-resolution image reconstruction method and device
CN112396645B (en) Monocular image depth estimation method and system based on convolution residual learning
CN112037131A (en) Single-image super-resolution reconstruction method based on generation countermeasure network
Cheng et al. Zero-shot image super-resolution with depth guided internal degradation learning
CN110136062B (en) Super-resolution reconstruction method combining semantic segmentation
CN108921783B (en) Satellite image super-resolution reconstruction method based on mixed loss function constraint
CN112446835B (en) Image restoration method, image restoration network training method, device and storage medium
CN110136122B (en) Brain MR image segmentation method based on attention depth feature reconstruction
CN112949636B (en) License plate super-resolution recognition method, system and computer readable medium
CN109214989A (en) Single image super resolution ratio reconstruction method based on Orientation Features prediction priori
CN105513033B (en) A kind of super resolution ratio reconstruction method that non local joint sparse indicates
CN109949222A (en) Image super-resolution rebuilding method based on grapheme
CN115170915A (en) Infrared and visible light image fusion method based on end-to-end attention network
CN117575915B (en) Image super-resolution reconstruction method, terminal equipment and storage medium
CN114332625A (en) Remote sensing image colorizing and super-resolution method and system based on neural network
CN117474764B (en) High-resolution reconstruction method for remote sensing image under complex degradation model
Deng et al. Multiple frame splicing and degradation learning for hyperspectral imagery super-resolution
CN114881856A (en) Human body image super-resolution reconstruction method, system, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant