WO2021056969A1 - Super-resolution image reconstruction method and device - Google Patents

Abstract

A super-resolution image reconstruction method, comprising: constructing a generative adversarial network; constructing an original resolution image training set; training the generative adversarial network on the basis of the original resolution image training set; and reconstructing an input image by using a generator of the trained generative adversarial network. In the super-resolution image reconstruction method and device in embodiments of the present invention, the constructed generative adversarial network comprises a generator, a discriminator, and a loss function calculator. The generative adversarial network is trained by adopting an original resolution image training set, so that the original resolution image can be used as a reference image to reconstruct a super-resolution image, and even if the original resolution is relatively low, a relatively high-resolution image can still be reconstructed.

Description

Super-resolution image reconstruction method and device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 29, 2019, with application number 201910935280.8 and titled "Super-resolution image reconstruction method and device", the entire content of which is incorporated into this application by reference in.

Technical field

The invention belongs to the fields of computer vision, deep learning, and image processing, and particularly relates to a super-resolution image reconstruction method and device.

Background technique

High-resolution remote sensing images play an important role in resource exploration, environmental monitoring, natural disaster prevention, and military reconnaissance. However, due to the limitation of signal transmission bandwidth and imaging sensor storage, remote sensing imaging equipment usually can only obtain remote sensing images with low spatial resolution. The improvement of the accuracy of imaging equipment, the reduction of image size, and the increase of the number of pixels collected per unit area are all restricted by the current manufacturing level. It is difficult to improve the image resolution from the hardware, and it takes a long time and is extremely costly.

The super-resolution reconstruction method based on deep learning can learn the end-to-end mapping relationship between low-resolution images and high-resolution images through a large number of sample training, which can more fully learn the prior information in low-resolution images and reconstruct The speed is fast and the effect is good. However, there are currently few data sets containing a large number of high-resolution remote sensing images. How to use a small number of high-resolution remote sensing image training samples to use deep learning methods to improve the spatial resolution of remote sensing images is still an urgent problem to be solved.

Summary of the invention

In view of this, the embodiments of the present invention provide a super-resolution image reconstruction method and device, which can effectively solve the problem of difficulty in reconstruction caused by the unsatisfactory resolution of the reference image in the prior art.

The first aspect of the embodiments of the present invention provides a super-resolution image reconstruction method, including:

Construct a generative confrontation network, wherein the generative confrontation network includes at least a generator, a discriminator, and a loss function calculator; construct an original resolution image training set; train the generative confrontation network based on the original resolution image training set, wherein, The original resolution image training set is preprocessed as the first input of the discriminator, and the output of the generator is used as the second input of the discriminator; using the trained generator of the generated confrontation network, Reconstruct the input image.

A second aspect of the embodiments of the present invention provides a super-resolution image reconstruction device, which is characterized by comprising: a first building module to construct a generative confrontation network, wherein the generative confrontation network at least includes: a generator and a discriminator And loss function calculator; a second construction module, which constructs an original resolution image training set; a training module, which trains the generative confrontation network based on the original resolution image training set, wherein the original resolution image training set is pre-processed Processing is used as the first input of the discriminator, and the output of the generator is used as the second input of the discriminator; the reconstruction module uses the trained generator of the generated confrontation network to reconstruct the input image .

In the super-resolution image reconstruction method and device of the embodiment of the present invention, the constructed generation confrontation network includes a generator, a discriminator, and a loss function calculator, and the generation confrontation network is trained using the original resolution image training set, Therefore, it is possible to use the original resolution image as a reference image to reconstruct a super-resolution image, so that even if the original resolution is a lower resolution, a higher resolution image can still be reconstructed.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are merely of the present invention. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative labor.

FIG. 1A is a schematic flowchart of a super-resolution image reconstruction method according to an embodiment of the present invention.

FIG. 1B is a flowchart of specific training steps for generating a confrontation network model according to an embodiment of the present invention.

Fig. 2 is the flow of super-resolution reconstruction performed by unsupervised learning in the present invention.

Fig. 3 is a generator structure in a generating confrontation network in an embodiment of the present invention.

Fig. 4 is the structure of the discriminator in the generated confrontation network in the embodiment of the present invention.

Fig. 5 is a schematic block diagram of a super-resolution image reconstruction device according to an embodiment of the present invention.

Figure 6 is an evaluation diagram of the result of ×2 multiple reconstruction on the test set.

Figure 7 is an evaluation diagram of the result of ×4 multiple reconstruction on the test set.

Figure 8 is a visual quality effect diagram of a ×2 reconstructed picture.

Figure 9 is a visual quality effect diagram of a ×4 reconstructed picture.

detailed description

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of the present invention.

In order to illustrate the technical solution of the present invention, specific embodiments are used for description below.

FIG. 1A is a schematic flowchart of a super-resolution image reconstruction method according to an embodiment of the present invention. The super-resolution image reconstruction method 100 of FIG. 1A includes:

110: Build a generative confrontation network, where the generative confrontation network at least includes a generator, a discriminator, and a loss function calculator.

120: Construct a training set of original resolution images.

It should be understood that the original resolution image in the embodiment of the present invention may include any resolution image. Preferably, when a low-resolution image exists, the technical effect of the embodiment of the present invention can be optimized. That is, a data set that includes a considerable number of high-resolution images is also within the scope of the embodiment of the present invention to achieve the purpose of reconstructing a higher-resolution image, which is not limited in the embodiment of the present invention. The images in the training set according to the embodiment of the present invention may be any images, preferably remote sensing images. As a specific embodiment, in order to achieve the optimal technical effect, for example, as shown in FIG. 1B, the training set is a low-resolution remote sensing image data set constructed by down-sampling of UCM, NWPU, and WHU-RS19. The test set is the data set of the super-resolution image evaluation MATLAB toolbox provided by Jaime I University in Spain.

130: Train and generate a confrontation network based on the original resolution image training set, where the original resolution image training set is preprocessed as the first input of the discriminator, and the output of the generator is used as the second input of the discriminator;

140: Use the trained generator of the generated confrontation network to reconstruct the input image.

It should be understood that the original resolution image training set is input into the generator after preprocessing, and may be input into the generator after interpolation processing, and the embodiment of the present invention does not limit various interpolation methods. As a preferred embodiment, bicubic interpolation processing can be performed. In addition, the convolutional neural network may include an average pooling layer for down-sampling processing.

It should be understood that the image reconstruction device that implements the image reconstruction method can be a server, can be used as a component in a remote sensing device, or can be used as an independent processing device connected to the remote sensing device, which is not limited in the embodiment of the present invention. As a specific embodiment, for example, as shown in FIG. 1B, a PyTorch development environment is built; the hardware platform of the present invention is based on the Intel E5-2690V3 processor, TITAN V GPU, and 64G memory. The experimental software platform is based on the Urbanu16.04 version, using CUDA 9.0, CUDNN 7 and PyTorch 1.0.0 environment.

It should be understood that the generator in the embodiment of the present invention may not include a down sampler, that is, during training, the output of the down sampler is input to the discriminator. Alternatively, the generator may include a down-sampler (for example, an average pooling layer), and the output of the generator is input to the discriminator. The training in the embodiment of the present invention adopts the idea of generating a confrontation network, but it is applicable to a variety of architectures of generating a confrontation network. Therefore, any scheme that uses the original resolution image training set as the label of the discriminator for reconstruction is described in the embodiment of the present invention. In the range.

Therefore, in the super-resolution image reconstruction method of the embodiment of the present invention, the constructed generation confrontation network includes a generator, a discriminator, and a loss function calculator, and the generation confrontation network is trained using the original resolution image training set, Therefore, it is possible to use the original resolution image as a reference image to reconstruct a super-resolution image, so that even if the original resolution is a lower resolution, a higher resolution image can still be reconstructed. In other words, for example, using the embodiment of the present invention to construct a generative confrontation network model with a down-sampling process, as a more preferred embodiment, by using a lower resolution image as a label, that is, using the down-sampled reconstructed image and The lower resolution picture seeks the loss to update the network parameters to achieve the purpose of unsupervised super-resolution reconstruction.

At present, there are usually only a few or no high-resolution remote sensing images in remote sensing images. To solve the problem of few training data sources, the following solutions can be adopted: (1) Use data set amplification to amplify data; (2) Use natural images to train the reconstruction network model, and then migrate it to the super-resolution reconstruction of remote sensing images; (3) Use unsupervised learning to reconstruct high-resolution remote sensing images. The use of data set augmentation can only alleviate the impact of this problem to a certain extent. Compared with the ImageNet large data set, the amount of augmented data is not enough to train an ideal reconstruction model; and remote sensing images It is very different from the degradation process of natural image imaging. The network model trained with natural image is not suitable for direct application to remote sensing image super-resolution reconstruction, and migration learning still requires high-resolution remote sensing image tags; therefore To fundamentally solve the problem of fewer high-resolution image sources for remote sensing images, unsupervised learning methods can be used.

The Generative Adversarial Network (GAN) can conduct adversarial training with the generator by adding a discriminator, so as to achieve the purpose of generating pictures with good quality. The discriminator can discriminate the generated picture and the label picture as much as possible through training, and the generator can generate pictures that the discriminator cannot discriminate as much as possible through the training process. In the training process of the two confrontation, the discriminator can indirectly serve as a loss function to improve the learning ability of the generator. Applying generative confrontation network to image super-resolution reconstruction (SRGAN model) can make the reconstructed image have more texture detail information, and the visual effect is better than other deep learning models. Remote sensing images are usually acquired from a long distance and contain complex scene and feature information. Therefore, in the process of remote sensing image reconstruction, the requirements for detail reconstruction are higher. The introduction of GAN into remote sensing image super-resolution reconstruction can effectively improve the reconstruction. effect.

According to the super-resolution image reconstruction method described in Figure 1, constructing a generative confrontation network includes: constructing a generator using a convolutional neural network. The convolutional neural network includes multiple convolutional layers with successively decreasing convolution kernels, each of which The size of the convolution kernel of each convolutional layer is larger than the step size. Specifically, the generator structure adopts a codec structure constructed by a convolutional layer and a deconvolutional layer, which can extract and restore features, and introduce skip links to integrate the low-level features into the high-level output, effectively retaining the original picture Low frequency information.

According to the super-resolution image reconstruction method described in Figure 1, the use of a convolutional neural network to construct a generator includes: adding an average pooling layer for downsampling after the output layer of the convolutional neural network, where the average pooling layer The output image of the transformation layer is used as the second input of the discriminator. Among them, the input image is reconstructed by the generator of the generated confrontation network after training, including: inputting the input image to the generator, and obtaining the reconstructed image from the output layer . Specifically, another layer of average pooling is added to the output layer of the network (×4 multiples are two layers of average pooling) to achieve the purpose of down-sampling the reconstructed image. Compared with maximum pooling, average pooling can retain as much neighborhood pixel information as possible in the down-sampling process, and it can increase the generalization ability of the model for degraded images in different processes. The final output of the generator is the reconstructed picture I ^SR and the down-sampled reconstructed picture I ^SR' .

According to the super-resolution image reconstruction method described in Fig. 1, the first input of the discriminator corresponds to the first output, and the second input corresponds to the second output. The construction of a generative confrontation network includes: using a generator loss function calculator , So as to feed back the generator loss function to the generator, where the generator loss function includes at least the cross-entropy loss between the first output and the second output. First, the generator loss function L _{G of the} embodiment of the present invention may include the following parts: image loss L _image , perceptual loss L _VGG , confrontation loss L _Adv and total variation loss L _TV . The calculation formula of the generator loss function is:

L _G ＝L _image +2×10 ^-3 ×L _VGG +L _Adv +2×10 ^-8 ×L _TV (1)

Among them, the image loss is the L1 norm ^{of I SR} and I ^SR':

Among them, r is the feature extraction ratio, W is the pixel-based image width, and H is the pixel-based image height.

The VGG (VisualGeometryGroup, visual geometry group) model loss is _{the L 1} norm ^{between the output features of I LR} and I ^SR' through the last layer of the convolutional layer (including the activation function) of the VGG16 feature extraction network:

Among them, W'is the width of the image based on pixels, and H'is the height of the image based on pixels.

Based on the output function of the discriminator of D(.), the confrontation loss is between D(I ^LR ) and real label 1, D(I ^SR' ) and false label 0, and D(I ^LR ) and D(I ^SR' ) The weighted sum of cross entropy loss, where w ₁ and w ₂ are adjustable weights:

The total variation loss is the mean square value of the difference between the horizontal and vertical directions of the image:

Among them, I _{x, y} represents the xth pixel on the abscissa, and the yth pixel on the ordinate.

In addition to the generator loss function, the discriminator loss in the embodiment of the present invention is:

According to the super-resolution image reconstruction method described in Fig. 1, constructing a generative confrontation network includes: constructing a discriminator using a batch normalization layer, and the batch normalization layer is placed between the dense layer of the discriminator and the excitation function layer. Specifically, the discriminator is composed of a series of non-synchronized convolutional layers. Each convolutional layer is followed by a Leaky ReLU (α=0.2) activation function and a BN (normalization) layer, and finally in two densely connected layers Then add a layer of BN and then input it to the Sigmoid function (an excitation function) to get the discriminator's score for a picture.

In addition, after the training is completed, and before the input image is reconstructed using the trained generator that generates the adversarial network, the network model can be trained and tested. For example, in the embodiment of the present invention, the Adam optimizer can be used to optimize the loss function. The input data adopts 128*128 randomly cropped image blocks, which can play a certain role in data enhancement. The learning rate is set to 5*10-4, and the Batch_size size is 32. Use the training set to train the constructed generative confrontation network, and use the test set to test the trained reconstruction network to complete the super-resolution reconstruction of remote sensing images.

In addition, an evaluation index can be used to evaluate the reconstructed image. For example, the embodiment of the present invention uses SAM (spectral angle mapping), RMSE (root mean square error), ERGAS (integrated dimensionless overall relative error), sCC (spatial correlation coefficient), Qindex (quality index), SSIM (structure similarity) ), PSNR (Peak Signal-to-Noise Ratio) is used as an evaluation index to comprehensively evaluate the reconstructed image. For example, Table 1 shows the calculation formulas for these evaluation indicators.

Table 1 Evaluation index calculation formula

Figure 2 is a flow chart of realizing super-resolution reconstruction of remote sensing images through unsupervised learning. As a specific embodiment, after inputting the interpolated low-resolution image and outputting it from the generator, the reconstructed image is obtained, and then down-sampled to the same size as the low-resolution image LR, and then input to the discriminator to make it discriminate between LR and SR', and find the loss between the two to update the network parameters.

Figure 3 is the generator structure for generating the confrontation network in the present invention. The generator has a codec structure of a convolution layer and a deconvolution layer, and the convolution layer and the deconvolution layer have a skip connection. Specifically, the generator includes a series of convolutional layers and deconvolutional layers. First, the three convolution kernels are 7, 5, and 3 convolutional layers (connected to ReLU (linear rectification unit)) for feature extraction. This is followed by 4 consecutive 3×3 convolutional layers for image feature coding; then 4 layers of deconvolution are used for feature decoding, and 3, 5, and 7 three-layer deconvolutions are used for feature restoration, and finally a reconstructed image is obtained . In order to solve the problem of the loss of low-frequency information in continuous convolution of the image, a jump connection is added between the input and the output to preserve the low-frequency information.

Fig. 4 is the discriminator in the generated confrontation network of the present invention. After inputting the image, the discriminator goes through a series of convolution, Leaky ReLU and BN layers to extract the features of the input image. The final part is densely connected and the sigmoid function is input to obtain the score value of the input image, and the score between the input images The difference.

Fig. 5 is a schematic block diagram of an image reconstruction device according to an embodiment of the present invention. The super-resolution image reconstruction device 500 of FIG. 5 includes:

The first building module 510 is to build a generative confrontation network, where the generative confrontation network at least includes a generator, a discriminator, and a loss function calculator;

The second construction module 520 constructs an original resolution image training set;

The training module 530 trains and generates a confrontation network based on the original resolution image training set, where the original resolution image training set is preprocessed as the first input of the discriminator, and the output of the generator is used as the second input of the discriminator;

The reconstruction module 540 uses the trained generator to generate the confrontation network to reconstruct the input image.

In the super-resolution image reconstruction device of the embodiment of the present invention, the constructed generation confrontation network includes a generator, a discriminator, and a loss function calculator, and the generation confrontation network uses the original resolution image training set for training, so it can The original resolution image is used as the reference image to reconstruct the super-resolution image, so that even if the original resolution is a lower resolution, a higher resolution image can still be reconstructed.

According to the super-resolution image reconstruction device of Fig. 5, the first building module is specifically used to: construct a generator using a convolutional neural network, the convolutional neural network including a plurality of convolutional layers with successively decreasing convolution kernels, each of which The size of the convolution kernel of the convolution layer is larger than the step size.

According to the super-resolution image reconstruction device of FIG. 5, the first building module is specifically used to add an average pooling layer for downsampling after the output layer of the convolutional neural network, wherein the image output from the average pooling layer As the second input of the discriminator, the reconstruction module is specifically used to input the input image to the generator, and obtain the reconstructed image from the output layer.

According to the super-resolution image reconstruction device of FIG. 5, the first input of the discriminator corresponds to the first output, and the second input corresponds to the second output. The first building module is specifically used for: using a generator loss function calculator, In order to feed back the generator loss function to the generator, the generator loss function includes at least the cross-entropy loss between the first output and the second output.

According to the super-resolution image reconstruction device of FIG. 5, the first construction module is specifically used to construct a discriminator using a batch normalization layer, and the batch normalization layer is placed between the dense layer of the discriminator and the excitation function layer.

Fig. 6 is an average value of various evaluation indexes obtained by performing a ×2 multiple test on a test set in an embodiment of the present invention. The figure contains 12 unsupervised learning methods for image super-resolution reconstruction. The closer the SAM value is to 0, the better, the smaller the RMSE, the better, the smaller the ERGAS, the better, the closer the sCC is to 1, the larger the Q Well, the closer the SSIM is to 1, the better, and the larger the PSNR, the better. From the figures and the values in the table, it can be seen that, among the above indicators, the reconstructed image of the present invention can obtain the best evaluation result (ERGAS result ranks second), which proves that the network performs unsupervised learning to reconstruct the image. Effectiveness.

Fig. 7 is the average value of each evaluation index obtained by the ×4 multiple test on the test set. It can be seen from the figures and the numerical values in the table that, among the various evaluation indicators, the method achieves the best evaluation results, which proves the effectiveness of the unsupervised learning reconstruction results of the present invention at a larger magnification.

Figure 8 is a visual display of the results of the test set × 2 multiples in the embodiment of the present invention. Figure (a) is the original high-resolution image, Figure (b) is the image after bicubic interpolation of the input network, and Figure (c) The image reconstructed by this method can be seen from the figure that there are many details reconstructed by this method, which is close to a high-resolution image.

Figure 9 is a visual display of the results of the test set × 2 multiples in the embodiment of the present invention. Figure (a) is the original high-resolution image, Figure (b) is the image after bicubic interpolation of the input network, and Figure (c) For the image reconstructed by this method, it can be seen from the figure that the reconstructed image by this method can achieve good visual effects, which is closer to high-resolution images.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only used to facilitate distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered as going beyond the scope of the present invention.

In the embodiments provided by the present invention, it should be understood that the disclosed device/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still implement the foregoing various embodiments. The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in Within the protection scope of the present invention.

Claims (10)

1. A super-resolution image reconstruction method, characterized in that it comprises:

   Constructing a generative confrontation network, wherein the generative confrontation network at least includes a generator, a discriminator, and a loss function calculator;

   Construct the original resolution image training set;

   The generative confrontation network is trained based on the original resolution image training set, wherein the original resolution image training set is preprocessed as the first input of the discriminator, and the output of the generator is used as the discriminator The second input;

   The input image is reconstructed using the trained generator of the generated confrontation network.
2. The image reconstruction method according to claim 1, wherein said constructing a generative confrontation network comprises:

   The generator is constructed using a convolutional neural network. The convolutional neural network includes a plurality of convolutional layers with successively decreasing convolution kernel sizes, wherein the convolution kernel size of each convolutional layer is greater than the step size.
3. The image reconstruction method according to claim 2, wherein the construction of the generator using a convolutional neural network comprises:

   Add an average pooling layer for downsampling after the output layer of the convolutional neural network, where the image output from the average pooling layer is used as the second input of the discriminator,

   Wherein, the reconstructing the input image by using the trained generator for generating the confrontation network includes:

   The input image is input to the generator, and a reconstructed image is obtained from the output layer.
4. The image reconstruction method according to claim 1, wherein the first input of the discriminator corresponds to a first output, and the second input corresponds to a second output, wherein the constructing a confrontation network ,include:

   A generator loss function calculator is used to feed back the generator loss function to the generator, wherein the generator loss function includes at least a cross-entropy loss between the first output and the second output.
5. The image reconstruction method according to claim 1, wherein said constructing a generative confrontation network comprises:

   The discriminator is constructed using a batch normalization layer, and the batch normalization layer is placed between the dense layer and the excitation function layer of the discriminator.
6. A super-resolution image reconstruction device, characterized in that it comprises:

   The first building module is to build a generative confrontation network, wherein the generative confrontation network at least includes a generator, a discriminator, and a loss function calculator;

   The second building module is to build the original resolution image training set;

   The training module trains the generative confrontation network based on the original resolution image training set, wherein the original resolution image training set is preprocessed as the first input of the discriminator, and the generator output is used as the first input of the discriminator. The second input of the discriminator;

   The reconstruction module uses the trained generator for generating the confrontation network to reconstruct the input image.
7. The image reconstruction device according to claim 6, wherein the first building module is specifically configured to:

   The generator is constructed using a convolutional neural network. The convolutional neural network includes a plurality of convolutional layers with successively decreasing convolution kernel sizes, wherein the convolution kernel size of each convolutional layer is greater than the step size.
8. The image reconstruction device according to claim 7, wherein the first construction module is specifically configured to:

   Add an average pooling layer for downsampling after the output layer of the convolutional neural network, where the image output from the average pooling layer is used as the second input of the discriminator,

   Wherein, the reconstruction module is specifically used for:

   The input image is input to the generator, and a reconstructed image is obtained from the output layer.
9. The image reconstruction device according to claim 6, wherein the first input of the discriminator corresponds to a first output, and the second input corresponds to a second output, wherein the first building module Specifically used for:

   A generator loss function calculator is used to feed back the generator loss function to the generator, wherein the generator loss function includes at least a cross-entropy loss between the first output and the second output.
10. The image reconstruction device according to claim 6, wherein the first building module is specifically configured to:

    The discriminator is constructed using a batch normalization layer, and the batch normalization layer is placed between the dense layer and the excitation function layer of the discriminator.

Images