CN113850721A - Single image super-resolution reconstruction method, device and equipment and readable storage medium - Google Patents
Single image super-resolution reconstruction method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN113850721A CN113850721A CN202111129980.1A CN202111129980A CN113850721A CN 113850721 A CN113850721 A CN 113850721A CN 202111129980 A CN202111129980 A CN 202111129980A CN 113850721 A CN113850721 A CN 113850721A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution reconstruction
- resolution
- image super
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000003860 storage Methods 0.000 title claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 83
- 230000006870 function Effects 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 230000015654 memory Effects 0.000 claims description 10
- 238000005520 cutting process Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 7
- 238000004821 distillation Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and equipment for reconstructing super-resolution of a single image and a readable storage medium, wherein a training data set is obtained, and images in the training data set are cut to obtain an image block training data set; constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set; setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting training termination conditions to obtain a trained end-to-end image super-resolution reconstruction model; and inputting the low-resolution image into the trained end-to-end image super-resolution reconstruction model to obtain a high-resolution image output by the trained end-to-end image super-resolution reconstruction model. The invention aims to reduce the network parameter quantity and improve the performance of image super-resolution reconstruction.
Description
Technical Field
The invention belongs to the technical field of deep learning computer vision and image super-resolution reconstruction, and particularly relates to a single image super-resolution reconstruction method, a single image super-resolution reconstruction device, single image super-resolution reconstruction equipment and a readable storage medium.
Background
With the development of the information age, the way in which information is presented is also diversified, such as various entities like images, videos, audios, characters, etc. The image is used as a carrier and a medium for information storage and transmission, is also a main information source for human beings to obtain cognition and communication, and plays an extremely important role in daily life. Information carried by images depends greatly on image quality, and thus improving image quality has become a hot topic and research field. The image resolution represents the number of pixels contained in one image unit inch, and is one of important indexes for evaluating the image quality. The resolution of the image is closely related to the quality of the image, and the higher resolution corresponds to the denser pixel distribution and the richer detail information, and therefore the higher image quality is represented. However, there are various obstacles to obtaining the HR image, such as the influence of the image capturing device, external environment noise, transmission, storage and display mode, and the image may undergo a degradation process, resulting in a quality loss and an LR image. Therefore, reconstructing and restoring the image resolution is a research focus and difficulty in the field of image processing.
In recent years, due to the rise of artificial intelligence and the progress of computer hardware technology, image processing technology based on deep learning has also been rapidly developed. The method for reconstructing the single image super-resolution comprises an interpolation-based algorithm, a reconstruction-based algorithm and a learning-based algorithm. Compared with the traditional image super-resolution reconstruction algorithm, the single image super-resolution reconstruction algorithm based on deep learning has the advantages of greatly improved performance and relatively simple algorithm design. The current popular methods include EDSR, RCAN, etc., although these algorithms achieve good reconstruction effects, these models have large parameter quantities and are not flexible enough in use, which brings certain difficulties to practical application, and therefore how to efficiently perform image super-resolution reconstruction is an urgent problem to be solved.
Disclosure of Invention
The invention provides a method, a device, equipment and a readable storage medium for reconstructing super-resolution of a single image, aiming at reducing network parameters and improving the performance of reconstructing super-resolution of the image.
In order to solve the technical problems, the invention is realized by the following technical scheme:
a single image super-resolution reconstruction method comprises the following steps:
acquiring a training data set, and cutting images in the training data set to obtain an image block training data set;
constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set;
setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting training termination conditions to obtain a trained end-to-end image super-resolution reconstruction model;
and inputting the low-resolution image into the trained end-to-end image super-resolution reconstruction model to obtain a high-resolution image output by the trained end-to-end image super-resolution reconstruction model.
Further, after the cropping the image in the training data set, the method further includes:
and turning, rotating and randomly disordering the cut image to obtain an image block training data set.
Further, the setting of the hyper-parameters of the end-to-end image super-resolution reconstruction model, the training of the end-to-end image super-resolution reconstruction model after the loss function and the optimizer are adopted and the training termination condition is set, so as to obtain the trained end-to-end image super-resolution reconstruction model, specifically comprising:
taking a certain number of image blocks in the image block training data set as a batch as input of the end-to-end image super-resolution reconstruction model;
setting a hyper-parameter of the end-to-end image super-resolution reconstruction model, setting an initialization mode of the end-to-end image super-resolution reconstruction model, and setting a loss function and an optimizer of the end-to-end image super-resolution reconstruction model;
and training the end-to-end image super-resolution reconstruction model by using a random gradient descent algorithm and a reverse propagation algorithm to obtain the trained end-to-end image super-resolution reconstruction model.
Further, the loss function is defined as follows:
in the formula, theta represents a hyper-parameter of an end-to-end image super-resolution reconstruction model;
and
respectively representing a low-resolution image and a high-resolution image corresponding to the low-resolution image in the training image block;
representing a super-resolution image output after the end-to-end image super-resolution reconstruction model is processed; m denotes the number of image blocks.
Further, the optimizer is an Adam optimizer.
Further, the hyper-parameters of the end-to-end image super-resolution reconstruction model comprise: learning rate, decay rate, and training round.
A single image super-resolution reconstruction apparatus includes:
the acquisition module is used for acquiring a training data set and cutting images in the training data set to obtain an image block training data set;
the model construction module is used for constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set;
the model training module is used for setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting a training termination condition, and obtaining the trained end-to-end image super-resolution reconstruction model;
and the output module is used for inputting the low-resolution images into the trained end-to-end image super-resolution reconstruction model to obtain the high-resolution images output by the trained end-to-end image super-resolution reconstruction model.
An apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the single image super-resolution reconstruction method when executing the computer program.
A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a method for super-resolution reconstruction of a single image.
Compared with the prior art, the invention has at least the following beneficial effects:
when image super-resolution reconstruction is required to be carried out on an image, firstly, a training data set is obtained, and images in the training data set are cut to obtain an image block training data set; then, constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set; setting hyper-parameters of an end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting training termination conditions to obtain a trained end-to-end image super-resolution reconstruction model; and inputting the low-resolution images into the trained end-to-end image super-resolution reconstruction model to obtain high-resolution images output by the trained end-to-end image super-resolution reconstruction model. The invention extracts the characteristics of the image by utilizing convolution and inner convolution, and learns the mapping relation between the low-resolution image and the high-resolution image by utilizing the residual characteristic distillation module, thereby better reconstructing the image while greatly reducing the parameter quantity of the model.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a single image super-resolution reconstruction method of the present invention;
FIG. 2 is a network structure diagram of an end-to-end image super-resolution reconstruction model;
FIG. 3 is a structure diagram of a residual error feature distillation module (RMFDB) in an end-to-end image super-resolution reconstruction model;
FIG. 4 is an ESA structure diagram in an end-to-end image super-resolution reconstruction model;
FIG. 5 is a diagram of an inner convolution (convolution) structure in an end-to-end image super-resolution reconstruction model;
FIG. 6 shows the 2-fold enlargement of the super-resolution reconstruction model of the main flow model and the end-to-end image in the present invention on the Set5 data Set "Woman" image;
FIG. 7 shows the 2-fold enlargement reconstruction of the mainstream model and the end-to-end image super-resolution reconstruction model of the invention on the image of the Urban100 data set "img 011".
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, as a certain embodiment of the present invention, a single image super-resolution reconstruction method specifically includes the following steps:
Preferably, in order to increase the number of training images, after the images in the training data set are cropped, the cropped images are inverted, rotated, and subjected to random data augmentation to obtain more training images.
And 2, constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution (convolution) and inner convolution (convolution) by adopting a mode of combining residual learning and dense connection and introducing an convolution mechanism according to the image block training data set.
The end-to-end image super-resolution reconstruction model can better represent and learn the characteristics at different levels, and the image characteristics in the end-to-end image super-resolution reconstruction model constructed by the invention mainly comprise two types: one is low-level features obtained by means of a layer of 3 x 3 convolutional feature extraction modules, and the other is high-level features obtained by residual feature distillation block combined residual learning and dense concatenation.
Specifically, the end-to-end image super-resolution reconstruction model structure in the present invention is shown in fig. 2, and includes:
extracting low-level features of the image from the image block training data set by using a convolutional layer; extracting high-level features of the image in the image block training data set by using four residual feature distillation modules consisting of convolution layers and inner convolution layers; controlling the dimension of a feature map of an image needing to be reserved by using a 1 x 1 convolutional layer; recovering the low-level features and the high-level features by using an image reconstruction module consisting of the convolutional layer and the inner convolutional layer so as to obtain a high-resolution residual image; and amplifying and reconstructing the image by using the extracted characteristics of the image to obtain an end-to-end image super-resolution reconstruction model based on the combination of the common convolution and the internal convolution.
The residual characteristic distillation block consists of a 3 × 3 convolutional layer, a 3 × 3 inner convolutional layer, a 1 × 1 convolutional layer, and a spatial attention module. The image reconstruction module includes a nearest neighbor interpolation operation, two 3 x 3 convolutional layers, and a 3 x 3 inner convolutional layer sandwiched therebetween.
The inner convolution differs from convolution in that convolution operation has both spatial independence and channel specificity, while inner convolution operation has both spatial specificity and channel independence. The convolution kernel of the inner convolution has a special generation mode, and the parameter value corresponding to a space position of the convolution kernel is related to the input feature vector corresponding to the position. In addition, according to the design principle of channel sharing, when the inner convolution operation is performed, the feature maps are subjected to grouped convolution, the feature maps in each group share the parameters of one convolution kernel, but different convolution kernels are used at different spatial positions in the same group. And after the characteristic graphs of each group are processed, splicing the results of each group together again. Therefore, the computation complexity of the inner convolution is linear to the number of characteristic channels, and the computation complexity of the inner convolution is square to the number of characteristic channels. Therefore, the inner convolution operation greatly reduces the amount of calculation compared to the convolution operation.
And 3, setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting training termination conditions, and obtaining the trained end-to-end image super-resolution reconstruction model.
Specifically, the hyper-parameters of the end-to-end image super-resolution reconstruction model comprise: and the learning rate, the attenuation rate and the training turn of the end-to-end image super-resolution reconstruction model.
As a preferred embodiment, the method includes setting a hyper-parameter of an end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after a loss function and an optimizer are adopted and a training termination condition is set, and obtaining the trained end-to-end image super-resolution reconstruction model, and specifically includes:
taking a certain number of image blocks in the image block training data set as a batch as input of an end-to-end image super-resolution reconstruction model;
setting a hyper-parameter of an end-to-end image super-resolution reconstruction model, setting an initialization mode of the end-to-end image super-resolution reconstruction model, and setting a loss function and an optimizer of the end-to-end image super-resolution reconstruction model;
and training the end-to-end image super-resolution reconstruction model by using a random gradient descent algorithm and a reverse propagation algorithm to obtain the trained end-to-end image super-resolution reconstruction model.
That is to say, at the beginning of each training, randomly selecting image blocks in a batch of image block training sets as the input of an image super-resolution reconstruction model, sending the image blocks into an end-to-end image super-resolution reconstruction model for training, setting the initial learning rate, the attenuation rate and the training round of the end-to-end image super-resolution reconstruction model, setting the loss function and the optimizer of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model by adopting a Stochastic Gradient Descent (SGD) algorithm and a Back Propagation (BP), and keeping the parameters of the image super-resolution reconstruction model after the training is finished to obtain the trained end-to-end image super-resolution reconstruction model.
In this embodiment, the number of a batch of image blocks selected at the beginning of each training is 32, the training round is set to 1000, the initial learning rate of the end-to-end image super-resolution reconstruction model is set to 0.001, and the learning rate is changed to the original learning rate when the training round is 250, 500, and 750
Therefore, the parameters of the end-to-end image super-resolution reconstruction model can not fluctuate too much in the later training period.
The loss function employed is defined as follows:
wherein, theta represents a hyper-parameter of the end-to-end image super-resolution reconstruction model;
and
respectively representing a low-resolution image and a high-resolution image corresponding to the low-resolution image in the training image block;
representing a super-resolution image output after the end-to-end image super-resolution reconstruction model is processed; m denotes the number of image blocks.
In the training process, an adam (adaptive motion estimation) optimizer is selected to train the model, and the trained model parameters are stored when the training is finished.
And 4, inputting the image to be super-resolution reconstructed (low-resolution image) into the trained end-to-end image super-resolution reconstruction model to obtain a high-resolution image output by the super-resolution reconstruction model.
The embodiment selects a common Set5 data Set as a test data Set to verify the performance of the trained end-to-end image super-resolution reconstruction model.
And analyzing results, and evaluating the reconstruction effect of the trained end-to-end image super-resolution reconstruction model by adopting a peak signal-to-noise ratio (PSNR), wherein the PSNR is defined as follows:
wherein, IHRThe high-definition original image is obtained; i isSRImages generated for trained end-to-end image super-resolution reconstruction model；MAXIIs the maximum pixel value possible for an image, which in the present invention is 255; MSE (I)HR,ISR) Is represented byHRAnd ISRMean square error between, defined as follows:
where H and W are the height and width of the image, respectively.
Table 1 shows the parameter amounts of the present invention and the current super-resolution reconstruction method for main stream images, and it can be seen that the parameter amount of the present invention is less than that of other methods and the reconstruction objective evaluation index is higher, so that the present invention is more suitable for deployment and application in practice.
TABLE 1 parameter number and Performance of the super-resolution reconstruction model of the images
FIG. 3 is a structure diagram of a residual error feature distillation module (RMFDB) in an end-to-end image super-resolution reconstruction model, wherein an ALB module consists of 1 convolution layer or 1 inner convolution layer; FIG. 4 is a structure diagram of an ESA module in an end-to-end image super-resolution reconstruction model, wherein the ESA module is composed of a convolutional layer, a pooling layer, an upper sampling layer and a sigmoid function; fig. 5 is a diagram of an internal convolution (convolution) structure in an end-to-end image super-resolution reconstruction model, wherein feature maps in each group share the parameters of a convolution kernel, but different convolution kernels are used at different spatial positions in the same group. Fig. 6 shows the reconstruction results of the mainstream model and the end-to-end image super-resolution reconstruction model in the present invention on the Set5 data Set "wman" image, and fig. 7 shows the reconstruction results of the mainstream model and the end-to-end image super-resolution reconstruction model in the present invention on the Urban100 data Set "img 011" image. In fig. 6 and 7, the HR image is the high-resolution original image, the Bicubic image is the image generated by the high-resolution original image through Bicubic interpolation, and the rest of the images are the images generated by the SRCNN algorithm, the FSRCNN algorithm, the PAN algorithm, the A2F algorithm, the RFDN algorithm, and the image generated by the end-to-end image super-resolution model trained by the present invention. It can be seen that the method of the present invention is superior to other methods in not only the evaluation index (PSNR) but also the visual perception, and the image detail and the image texture are well preserved.
The present invention provides, in one embodiment, a computer device comprising a processor and a memory for storing a computer program comprising program instructions, the processor for executing the program instructions stored by the computer storage medium. The Processor may be a Central Processing Unit (CPU), or may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., which is a computing core and a control core of the terminal, and is adapted to implement one or more instructions, and is specifically adapted to load and execute one or more instructions to implement a corresponding method flow or a corresponding function; the processor provided by the embodiment of the invention can be used for the operation of a single image super-resolution reconstruction method.
In one embodiment of the invention, a single image super-resolution reconstruction method can be stored in a computer readable storage medium if the method is implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. Computer-readable storage media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data.
The computer storage media may be any available media or data storage device that can be accessed by a computer, including but not limited to magnetic memory (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical memory (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (e.g., ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), Solid State Disks (SSDs)), etc.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A single image super-resolution reconstruction method is characterized by comprising the following steps:
acquiring a training data set, and cutting images in the training data set to obtain an image block training data set;
constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set;
setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting training termination conditions to obtain a trained end-to-end image super-resolution reconstruction model;
and inputting the low-resolution image into the trained end-to-end image super-resolution reconstruction model to obtain a high-resolution image output by the trained end-to-end image super-resolution reconstruction model.
2. The single image super-resolution reconstruction method according to claim 1, wherein after the cropping the image in the training data set, the method further comprises:
and turning, rotating and randomly disordering the cut image to obtain an image block training data set.
3. The single-image super-resolution reconstruction method according to claim 1, wherein the setting of the super-parameters of the end-to-end image super-resolution reconstruction model, the training of the end-to-end image super-resolution reconstruction model after the adoption of a loss function and an optimizer and the setting of a training termination condition, and the obtaining of the trained end-to-end image super-resolution reconstruction model specifically comprises:
taking a certain number of image blocks in the image block training data set as a batch as input of the end-to-end image super-resolution reconstruction model;
setting a hyper-parameter of the end-to-end image super-resolution reconstruction model, setting an initialization mode of the end-to-end image super-resolution reconstruction model, and setting a loss function and an optimizer of the end-to-end image super-resolution reconstruction model;
and training the end-to-end image super-resolution reconstruction model by using a random gradient descent algorithm and a reverse propagation algorithm to obtain the trained end-to-end image super-resolution reconstruction model.
4. The single image super-resolution reconstruction method according to claim 1, wherein the loss function is defined as follows:
in the formula, theta represents a hyper-parameter of an end-to-end image super-resolution reconstruction model;
and
respectively representing a low-resolution image and a high-resolution image corresponding to the low-resolution image in the training image block;
representing a super-resolution image output after the end-to-end image super-resolution reconstruction model is processed; m denotes the number of image blocks.
5. The single image super-resolution reconstruction method according to claim 1, wherein the optimizer is an Adam optimizer.
6. The single image super-resolution reconstruction method of claim 1, wherein the hyper-parameters of the end-to-end image super-resolution reconstruction model comprise: learning rate, decay rate, and training round.
7. A single image super-resolution reconstruction device is characterized by comprising:
the acquisition module is used for acquiring a training data set and cutting images in the training data set to obtain an image block training data set;
the model construction module is used for constructing an end-to-end image super-resolution reconstruction model based on combination of common convolution and internal convolution according to the image block training data set;
the model training module is used for setting hyper-parameters of the end-to-end image super-resolution reconstruction model, training the end-to-end image super-resolution reconstruction model after adopting a loss function and an optimizer and setting a training termination condition, and obtaining the trained end-to-end image super-resolution reconstruction model;
and the output module is used for inputting the low-resolution images into the trained end-to-end image super-resolution reconstruction model to obtain the high-resolution images output by the trained end-to-end image super-resolution reconstruction model.
8. An apparatus comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a single image super resolution reconstruction method according to any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of a method for super-resolution reconstruction of a single image as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111129980.1A CN113850721A (en) | 2021-09-26 | 2021-09-26 | Single image super-resolution reconstruction method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111129980.1A CN113850721A (en) | 2021-09-26 | 2021-09-26 | Single image super-resolution reconstruction method, device and equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113850721A true CN113850721A (en) | 2021-12-28 |
Family
ID=78979819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111129980.1A Pending CN113850721A (en) | 2021-09-26 | 2021-09-26 | Single image super-resolution reconstruction method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113850721A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115472140A (en) * | 2022-09-09 | 2022-12-13 | 维沃移动通信有限公司 | Display method, display device, electronic apparatus, and readable storage medium |
| CN116342393A (en) * | 2023-04-11 | 2023-06-27 | 广州极点三维信息科技有限公司 | Image super-resolution method and system based on image noise prediction mechanism |
-
2021
- 2021-09-26 CN CN202111129980.1A patent/CN113850721A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115472140A (en) * | 2022-09-09 | 2022-12-13 | 维沃移动通信有限公司 | Display method, display device, electronic apparatus, and readable storage medium |
| CN116342393A (en) * | 2023-04-11 | 2023-06-27 | 广州极点三维信息科技有限公司 | Image super-resolution method and system based on image noise prediction mechanism |
| CN116342393B (en) * | 2023-04-11 | 2023-09-26 | 广州极点三维信息科技有限公司 | Image super-resolution method and system based on image noise prediction mechanism |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765296B (en) | Image super-resolution reconstruction method based on recursive residual attention network | |
| CN109118432B (en) | Image super-resolution reconstruction method based on rapid cyclic convolution network | |
| CN110136062B (en) | Super-resolution reconstruction method combining semantic segmentation | |
| CN111815516B (en) | Super-resolution reconstruction method for weak supervision infrared remote sensing image | |
| CN113658051A (en) | Image defogging method and system based on cyclic generation countermeasure network | |
| CN110675321A (en) | Super-resolution image reconstruction method based on progressive depth residual error network | |
| CN113850721A (en) | Single image super-resolution reconstruction method, device and equipment and readable storage medium | |
| CN111932461A (en) | Convolutional neural network-based self-learning image super-resolution reconstruction method and system | |
| CN112288632B (en) | Single image super-resolution method and system based on simplified ESRGAN | |
| CN114677304A (en) | Image deblurring algorithm based on knowledge distillation and deep neural network | |
| Singla et al. | A review on Single Image Super Resolution techniques using generative adversarial network | |
| CN112884650B (en) | Image mixing super-resolution method based on self-adaptive texture distillation | |
| CN115063297A (en) | Image super-resolution reconstruction method and system based on parameter reconstruction | |
| CN111754400B (en) | Efficient picture super-resolution reconstruction method | |
| CN113673675A (en) | Model training method and device, computer equipment and storage medium | |
| CN117788295B (en) | Super-resolution reconstruction method, system and medium for remote sensing image | |
| CN117058160B (en) | Three-dimensional medical image segmentation method and system based on self-adaptive feature fusion network | |
| CN112365422A (en) | Irregular missing image restoration method and system based on deep aggregation network | |
| CN114627035A (en) | Multi-focus image fusion method, system, device and storage medium | |
| CN113469884A (en) | Video super-resolution method, system, equipment and storage medium based on data simulation | |
| CN115115514A (en) | Image super-resolution reconstruction method based on high-frequency information feature fusion | |
| CN115526779A (en) | Infrared image super-resolution reconstruction method based on dynamic attention mechanism | |
| CN110570375B (en) | Image processing method, device, electronic device and storage medium | |
| CN113379606B (en) | Face super-resolution method based on pre-training generation model | |
| Liu et al. | Facial image inpainting using multi-level generative network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |