WO2020143513A1

WO2020143513A1 - Super-resolution image reconstruction method, apparatus and device

Info

Publication number: WO2020143513A1
Application number: PCT/CN2019/130826
Authority: WO
Inventors: 乔宇; 张文龙; 刘翼豪; 董超
Original assignee: 深圳先进技术研究院
Priority date: 2019-01-12
Filing date: 2019-12-31
Publication date: 2020-07-16
Also published as: CN109949219A; CN109949219B

Abstract

A super-resolution image reconstruction method, comprising: generating images of different perceptual qualities by means of different super-resolution image generation methods; calculating, by means of a perceptual quality evaluation index, perceptual quality evaluation scores of the images of different perceptual qualities; training a ranking estimation network according to the calculated perceptual quality evaluation scores; calculating, according to the trained ranking estimation network, a ranking content loss of an image generated by a generator of a generative adversarial network, and guiding, according to the ranking content loss, the generator of the generative adversarial network to train; performing super-resolution image reconstruction on a low-resolution image according to the trained generator. The present invention uses the ranking estimation network to directly perform optimization on the perceptual quality evaluation index, so as to obtain a better image perceptual quality, can flexibly perform expansion according to requirements, and can constrain the generator to generate super-resolution reconstructed images of different characteristics.

Description

Super-resolution image reconstruction method, device and equipment

Technical field

The present application belongs to the field of image processing, and particularly relates to a super-resolution image reconstruction method, device, and equipment.

Background technique

Super-resolution image reconstruction technology has important academic research and industrial application value in the field of computer vision and image processing. The purpose of super-resolution image reconstruction technology is to reconstruct the corresponding high-resolution image according to the given low-resolution image, and to obtain the best possible visual effect and small reconstruction error.

In the reconstruction of low-resolution images, currently generally include the learning method of convolutional neural network and the method of confrontation generation network. Among them, the learning method of convolutional neural network uses square error (Mean Square Error (MSE) to find the error between the high-resolution image reconstructed by the convolutional neural network and the real high-resolution image. Although the mean square error can be used to obtain a reconstruction result with little error between the mean square error and the true high-resolution image. However, the high-resolution image reconstructed by means of the mean square error shows an excessively smooth effect, which is not good for human eyes.

The existing cGAN-based super-resolution restoration and recovery technology is similar to the idea of super-resolution reconstruction using convolutional neural networks in the overall solution. The generator G is used to learn the mapping between low-resolution images and high-resolution images. Then, the discriminator D and the generator G are further introduced to conduct adversarial training. At the same time, the high-resolution image generated by the generator G is calculated and back-propagated with the real image in a specific feature space. Compared with the mean square error, although this method can get a better visual effect, compared with the real high-resolution image, the texture details are not real enough. On the other hand, the perceptual quality evaluation index uses feature extraction and further transformation operations, so it is not differentiable, and gradient solution and back propagation cannot be performed. This leads to high difficulty in optimizing the perceptual quality evaluation index, and visual The effect improvement is limited.

technical problem

In view of this, the embodiments of the present application provide a high-resolution image reconstruction method, device, and equipment to solve the problems in the prior art.

Technical solution

A first aspect of the embodiments of the present application provides a high-resolution image reconstruction method. The super-resolution image reconstruction method includes:

Generate images with different perceptual qualities through different super-resolution image generation methods;

Calculate the perceptual quality evaluation score of images with different perceptual quality through the perceptual quality evaluation index;

Train the ranking estimation network according to the calculated perceptual quality evaluation score;

Calculating the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and instructing the generator of generating the confrontation network according to the sequence content loss for training;

The low-resolution image is reconstructed according to the trained generator.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the step of calculating the perceptual quality evaluation score of images of different perceptual quality through the perceptual quality evaluation index includes:

Crop the images of different perceptual qualities into image blocks;

Image blocks with the same image content and different perceptual qualities are selected to form image pairs, and the perceptual quality evaluation scores of image pairs with different perceptual qualities are obtained through the non-reference image evaluation index.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the step of training the ranking estimation network according to the calculated perceptual quality evaluation score includes:

Selecting a convolutional neural network as the ranking estimation network to calculate the ranking score of the image;

According to the evaluation score calculated by the perceptual quality evaluation index and the ranking score, the ranking error of the convolutional neural network is obtained;

The parameters of the convolutional neural network are updated by a back propagation algorithm.

With reference to the first aspect, in a third possible implementation manner of the first aspect, the calculation based on the trained ranking estimation network calculates and generates the ranking content loss of the image generated by the generator against the network, and guides according to the ranking content loss guidance The steps to train a generator that generates an adversarial network include:

Generate high-resolution images through the generator;

Calculating the ranking content loss of the high-resolution image according to the trained ranking estimation network;

The generator is trained according to the sorted content loss.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the step of training the generator according to the sorted content loss further includes:

Determine the error between the generated high-resolution image and the real high-resolution image through the discriminator;

The perceptual loss error of the high-resolution image generated by the feature space calculation of the VGG network and the real image high-resolution image;

The generator is trained by combining the sorted content loss, the discriminator error and the perceptual loss error.

With reference to the first aspect, in a fifth possible implementation manner of the first aspect, before the step of training the ranking estimation network according to the calculated perceptual quality evaluation score, the method further includes:

Calculate the difference of images with different perceptual qualities, and select an image whose difference between the images is greater than a preset threshold.

A second aspect of an embodiment of the present application provides a super-resolution image reconstruction device, the super-resolution image reconstruction device includes:

Image generation unit for generating images of different perceptual qualities through different super-resolution image generation methods;

The evaluation score calculation unit is used to calculate the perception quality evaluation score of images with different perception qualities through the perception quality evaluation index;

The ranking estimation network training unit is used to train the ranking estimation network according to the calculated perceptual quality evaluation score;

The generator training unit is configured to calculate the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and guide the generator of the confrontation network generation training according to the sequence content loss;

The reconstruction unit is used to reconstruct the super-resolution image of the low-resolution image according to the trained generator.

With reference to the second aspect, in a first possible implementation manner of the second aspect, the evaluation score calculation unit includes:

A cropping subunit, configured to crop the images of different perceptual qualities into image blocks;

The evaluation subunit is used to select image blocks of the same image content and having different perceptual qualities to form an image pair, and to obtain the perceptual quality evaluation scores of image pairs of different perceptual qualities through a non-reference image evaluation index.

A third aspect of the embodiments of the present application provides a high-resolution image reconstruction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor When the computer program is executed, the steps of the high-resolution image reconstruction method according to any one of the first aspect are realized.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the computer program is implemented as described in any one of the first aspects. Resolution image reconstruction method steps.

Beneficial effect

Compared with the prior art, the embodiment of the present application has the following beneficial effects: the present application can use the ranking estimation network to directly optimize the perceptual quality evaluation index to obtain better image perceptual quality, which can reduce the cost of development time and can be significantly improved Reconstruct the perceptual quality of high-resolution images; and, through the order estimation network, you can learn training data with different a priori information, be able to flexibly expand according to needs, and constrain the generator to generate super-resolution reconstructed images with different characteristics .

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only for the application For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative labor.

1 is a schematic flowchart of an implementation method of a super-resolution image reconstruction method provided by an embodiment of the present application;

2 is a schematic diagram of an implementation process of a training method for ranking estimation network provided by an embodiment of the present application;

3 is a schematic structural diagram of a ranking estimation network provided by an embodiment of the present application;

4 is a schematic diagram of an implementation process of a generator training method provided by an embodiment of the present application;

5 is a schematic diagram of a super-resolution image reconstruction system provided by an embodiment of the present application;

FIG. 5a is a comparison schematic diagram of experimental effects provided by examples of the present application;

6 is a schematic diagram of a super-resolution image reconstruction device provided by an embodiment of the present application;

7 is a schematic diagram of a high-resolution image reconstruction device provided by an embodiment of the present application.

Embodiments of the invention

In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details hindering the description of the present application.

In order to explain the technical solutions described in the present application, the following will be described with specific embodiments.

FIG. 1 is a schematic diagram of an implementation process of a super-resolution image reconstruction method provided by an embodiment of the present application, and the details are as follows:

In step S101, images of different perceptual qualities are generated by different super-resolution image generation methods;

Specifically, one or more low-resolution images may be selected, and different super-resolution image generation methods may be used to generate images of different perceptual qualities for each image. The super-resolution image generation method may include a pixel-loss super-resolution method, a conventional generation-resistant super-resolution image generation method, or a super-resolution image generation method based on a convolutional neural network, and the like.

For the same image, when two different super-resolution image generation methods are used to generate images with different perceptual qualities, two images with different perceptual qualities can be selected as image pairs.

Among them, the training data set for generating low-resolution images of images with different perceptual qualities can be transformed and expanded according to requirements, that is, training data with different prior information can be selected.

In step S102, a perceptual quality evaluation score of images of different perceptual quality is calculated through a perceptual quality evaluation index;

In order to facilitate the calculation of the image through the perceptual quality evaluation index, images of different perceptual quality can be cropped into image blocks, image blocks with the same image content and different perceptual quality can be selected to form image pairs, and the image pair can be calculated through the perceptual quality evaluation index. Perceived quality evaluation score. Wherein, the perceptual quality evaluation index may include no reference image quality evaluation (English is referred to as No Reference Image Quality Assessment, referred to as NR-IQA in English, for example, natural image evaluation (English is called Natural image quality evaluator, referred to as NIQE in English), etc. Through the evaluation of the perceptual quality evaluation index, the perceptual quality evaluation score of the image pair with different perceptual quality on the same image content can be obtained, and the obtained perceptual quality evaluation score can be used as the label of the image in the image pair.

Of course, when using image pairs, you can also evaluate the difference of image pairs. For example, you can set a threshold to evaluate whether the difference between image pairs is large enough. If the difference between image pairs is greater than the preset threshold, it can be used in subsequent steps. The ranking of the estimated network training.

In step S103, train the ranking estimation network according to the calculated perceptual quality evaluation score;

For the generated images with different perceptual qualities, corresponding labels with different perceptual quality evaluation scores are corresponding. Use images with different perceptual qualities as input vectors, perceptual quality evaluation scores as labels as tags as output vectors, and train convolutional nerves as ranking estimation networks based on multiple input vectors and corresponding output vectors as training data Network, so that the trained ranking estimation network can perform ranking estimation on the super-resolution images, and output the ranking score of the super-resolution images. Wherein, the step of training the ranking estimation network may be specifically shown in FIG. 2 and includes:

In step S201, the convolutional neural network selected as the ranking estimation network calculates the ranking score of the image;

For the ranking estimation network, an appropriate convolutional neural network model can be selected to calculate the ranking score of the image. Since the sorting estimation task and the classification task have a certain degree of correlation, you can refer to the classic classification network for design, such as VGG-Net (Visual Geometry Group, Network). The ranking estimation network can be implemented by a twin structure of a basic convolutional neural network. As shown in FIG. 3, the convolutional neural network includes a feature extraction module with parameter sharing, a pooling module, a fully connected module, and a ranking loss function module , Where the feature extraction module can use VGG-16 network.

In step S202, according to the evaluation score calculated by the perceptual quality evaluation index and the ranking score, the ranking error of the convolutional neural network is obtained;

As can be seen from FIG. 3, images 1 and 2 with different perceptual qualities of the same content are input to the feature extraction module, and then through the pooling layer and the fully connected layer to obtain their corresponding ranking score 1 and ranking score 2, respectively, and then use the ranking Score 1 and ranking score 2 use the ranking loss function to update the parameters of the overall network.

In step S203, the parameters of the convolutional neural network are updated by a back propagation algorithm.

For two images of different perceptual quality under the same image content, the convolutional neural network can output their ranking scores. Generally, the perceptual quality evaluation index, for example, the smaller the NIQE, the better the perceptual quality of the image, and the smaller the ranking score under the same image content, the better the perceptual quality of the image. According to the perceptual quality evaluation index, the ranking loss (margin-ranking loss) can get the sorting error. Through reflection propagation algorithm (Backpropagation) Algorithm) can transfer the final sorting error of the network to each layer of the convolutional neural network, and realize the training and update of network parameters through error transfer, and finally minimize the sorting error.

In step S104, calculate the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and guide the generator of the generation of the confrontation network according to the sequence content loss for training;

The process of training the generator, as shown in Figure 4, includes:

In step S401, a high-resolution image is generated by the generator;

For the training of the adversarial network generator, you can use real super-resolution images and corresponding low-resolution images as training samples. To facilitate the calculation of the images, you can crop the same pair of images in the training sample into images with the same content. Block, through training calculation on the image block. In order to train the generator, the generator may generate a high-resolution image according to the low-resolution image or the image block of the low-resolution image in the sample.

In step S402, the ranked content loss of the high-resolution image is calculated according to the trained ranking estimation network;

Calculate the ranking score of the high-resolution image according to the previously trained ranking estimation network, and obtain the ranking content loss of the high-resolution image according to the ranking score.

In step S403, the generator is trained according to the sorted content loss.

Feedback the sorted content loss to the generator, and train and update the generator.

In a preferred embodiment, the step of training the generator according to the sorted content loss further includes:

Based on the loss of sorted content, combined with the discriminant error of the discriminator and the perceptual loss error of the feature space of the VGG network, the generator can be trained more effectively, the training efficiency of the generator can be improved, and the super-resolution can be improved. The realism of the rate image generation.

In step S105, super-resolution image reconstruction is performed on the low-resolution image according to the trained generator.

Using the trained generator, you can implement super-resolution on low-resolution images.

Because this application can use the ranking estimation network to directly optimize on specific perceptual quality evaluation standards to obtain better image perceptual quality. It is manifested that the existing super-resolution image methods mostly use the error in mean square error or feature space to optimize the super-resolution model. In the specific perceptual quality evaluation standard, the improvement of such methods is very time-consuming and laborious, because neither the mean square error nor the error optimization in the feature space can directly represent the improvement in specific perceptual quality. This application proposes to use a ranking estimation network to constrain the super-resolution model to directly optimize the specific perceptual quality, which can reduce the cost of development time and can significantly improve the perceptual quality of reconstructed high-resolution images.

Moreover, the present invention can have stronger expansibility for specific image perception quality. The performance is that the ranking estimation network can learn training data with different prior information, and can be flexibly expanded according to needs. Different prior information can be learned through the ordering of different perceptual quality data in training data and its own statistical characteristics. The ranking estimation network with different prior information can constrain the generator to generate super-resolution reconstruction images with different characteristics.

FIG. 5 is a schematic diagram of a super-rank generation anti-super-resolution image network provided by an embodiment of the present application, which is mainly composed of a generator, a discriminator, a feature extractor, and a rank estimator, where the feature extractor and rank estimator can For the trained model, only the loss calculation can be performed without updating the model parameters. It can be seen from the schematic diagram that the low-resolution image can be generated by the generator to generate a reconstructed high-resolution image, and then the discriminator can calculate the original high-resolution image and the reconstructed high-resolution image to combat loss, and the feature extractor can separately reconstruct The high-resolution image and the original high-resolution image calculate the error in the feature space to obtain the perceptual loss error. Furthermore, the reconstructed high-resolution image can be obtained by calculating the ranking score under the specific perceptual quality evaluation standard through the ranking estimator. Loss of sorted content. Finally, it uses joint loss, perceptual loss error and ordered content loss to jointly train the generator.

5a is a schematic diagram of a comparison of reconstruction effects of a super-resolution image provided by an embodiment of the application, where the first left is a low-resolution image, the second left is a super-resolution image generated by a super-resolution method based on pixel loss, and the right The second is the traditional generation of the super-resolution image generated by the super-resolution resolution method, and the first one is the super-resolution image generated by the reconstruction method of the super-resolution image described in this application. According to the experimental results, the super-resolution image reconstruction method described in this application can generate a more realistic image from a low-resolution image.

FIG. 6 is a schematic structural diagram of a super-resolution image reconstruction device provided by an embodiment of the present application, and the details are as follows:

The super-resolution image reconstruction device includes:

The image generating unit 601 is used to generate images with different perceptual qualities through different super-resolution image generating methods;

The evaluation score calculation unit 602 is used to calculate the perception quality evaluation scores of images with different perception qualities through the perception quality evaluation indicators;

The ranking estimation network training unit 603 is used to train the ranking estimation network according to the calculated perceptual quality evaluation score;

The generator training unit 604 is configured to calculate the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and guide the generator of the generation of the confrontation network according to the sequence content loss for training;

The reconstruction unit 605 is configured to reconstruct the super-resolution image of the low-resolution image according to the trained generator.

Preferably, the evaluation score calculation unit includes:

The high-resolution image reconstruction device described in FIG. 6 corresponds to the high-resolution image reconstruction method described in FIG. 1.

7 is a schematic diagram of a high-resolution image reconstruction device provided by an embodiment of the present application. As shown in FIG. 7, the high-resolution image reconstruction device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, For example, high-resolution image reconstruction procedures. When the processor 70 executes the computer program 72, the steps in the above embodiments of the method for reconstructing each high-resolution image are implemented. Alternatively, when the processor 70 executes the computer program 72, the functions of the modules/units in the foregoing device embodiments are realized.

Exemplarily, the computer program 72 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 71 and executed by the processor 70 to complete This application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 72 in the high-resolution image reconstruction device 7 . For example, the computer program 72 may be divided into:

The reconstruction device 7 of the high-resolution image may be a computing device such as a desktop computer, a notebook, a palmtop computer and a cloud server. The high-resolution image reconstruction device may include, but is not limited to, the processor 70 and the memory 71. Those skilled in the art may understand that FIG. 7 is only an example of the high-resolution image reconstruction device 7 and does not constitute a limitation on the high-resolution image reconstruction device 7, and may include more or less than the illustration. Components, or a combination of certain components, or different components, for example, the high-resolution image reconstruction device may further include an input and output device, a network access device, a bus, and the like.

The so-called processor 70 may be a central processing unit (Central Processing Unit (CPU), can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (Application Specific Integrated Circuit (ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 71 may be an internal storage unit of the high-resolution image reconstruction device 7, for example, a hard disk or a memory of the high-resolution image reconstruction device 7. The memory 71 may also be an external storage device of the high-resolution image reconstruction device 7, for example, a plug-in hard disk equipped on the high-resolution image reconstruction device 7, a smart memory card (Smart Media Card , SMC), Secure Digital (SD) card, flash memory card (Flash Card) etc. Further, the memory 71 may also include both the internal storage unit of the high-resolution image reconstruction device 7 and the external storage device. The memory 71 is used to store the computer program and other programs and data required by the high-resolution image reconstruction device. The memory 71 can also be used to temporarily store data that has been or will be output.

Those skilled in the art can clearly understand that, for convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example for illustration. In practical applications, the above-mentioned functions may be allocated by different functional units, Module completion means that the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above integrated unit may use hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the purpose of distinguishing each other, and are not intended to limit the protection scope of the present application. For the specific working processes of the units and modules in the above system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the above embodiments, the description of each embodiment has its own emphasis. For a part that is not detailed or recorded in an embodiment, you can refer to the related descriptions of other embodiments.

Those of ordinary skill in the art may realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed device/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are only schematic. For example, the division of the module or unit is only a logical function division, and in actual implementation, there may be another division manner, such as multiple units Or components can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software function unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the methods of the above embodiments, or it can be completed by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments may be implemented. . Wherein, the computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file, or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in jurisdictions. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media Excluded are electrical carrier signals and telecommunications signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they can still implement the foregoing The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Within the scope of protection of this application.

Claims

A super-resolution image reconstruction method, characterized in that the super-resolution image reconstruction method includes:

Generate images with different perceptual qualities through different super-resolution image generation methods;

Calculate the perceptual quality evaluation score of images with different perceptual quality through the perceptual quality evaluation index;

Train the ranking estimation network according to the calculated perceptual quality evaluation score;

Calculating the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and instructing the generator of generating the confrontation network according to the sequence content loss for training;

The low-resolution image is reconstructed according to the trained generator.
The method for reconstructing a super-resolution image according to claim 1, wherein the step of calculating a perceptual quality evaluation score of images of different perceptual quality through a perceptual quality evaluation index includes:

Crop the images of different perceptual qualities into image blocks;

Image blocks with the same image content and different perceptual qualities are selected to form image pairs, and the perceptual quality evaluation scores of image pairs with different perceptual qualities are obtained through the non-reference image evaluation index.
The method for reconstructing a super-resolution image according to claim 2, wherein the step of training the ranking estimation network according to the calculated perceptual quality evaluation score includes:

Selecting a convolutional neural network as the ranking estimation network to calculate the ranking score of the image;

According to the evaluation score calculated by the perceptual quality evaluation index and the ranking score, the ranking error of the convolutional neural network is obtained;

The parameters of the convolutional neural network are updated by a back propagation algorithm.
The method for reconstructing a super-resolution image according to claim 1, characterized in that, based on the trained ranking estimation network, the ranking content loss of the image generated by the generator against the network is calculated and calculated according to the ranking content The steps for the loss-guided generation adversarial network generator training include:

Generate high-resolution images through the generator;

Calculating the ranking content loss of the high-resolution image according to the trained ranking estimation network;

The generator is trained according to the sorted content loss.
The method for reconstructing a super-resolution image according to claim 4, wherein the step of training the generator according to the loss of sorted content further comprises:

Determine the error between the generated high-resolution image and the real high-resolution image through the discriminator;

The perceptual loss error of the high-resolution image generated by the feature space calculation of the VGG network and the real image high-resolution image;

The generator is trained by combining the sorted content loss, the discriminator error and the perceptual loss error.
The method for reconstructing a super-resolution image according to claim 1, wherein before the step of training the ranking estimation network according to the calculated perceptual quality evaluation score, the method further comprises:

Calculate the difference of images with different perceptual qualities, and select an image whose difference between the images is greater than a preset threshold.
A super-resolution image reconstruction device, characterized in that the super-resolution image reconstruction device includes:

Image generation unit for generating images of different perceptual qualities through different super-resolution image generation methods;

The evaluation score calculation unit is used to calculate the perception quality evaluation score of images with different perception qualities through the perception quality evaluation index;

The ranking estimation network training unit is used to train the ranking estimation network according to the calculated perceptual quality evaluation score;

The generator training unit is configured to calculate the sequence content loss of the image generated by the generator of the confrontation network according to the trained sequence estimation network, and guide the generator of the confrontation network generation training according to the sequence content loss;

The reconstruction unit is used to reconstruct the super-resolution image of the low-resolution image according to the trained generator.
The apparatus for reconstructing a super-resolution image according to claim 7, wherein the evaluation score calculation unit includes:

A cropping subunit, configured to crop the images of different perceptual qualities into image blocks;

The evaluation subunit is used to select image blocks of the same image content and having different perceptual qualities to form an image pair, and to obtain the perceptual quality evaluation scores of image pairs of different perceptual qualities through a non-reference image evaluation index.
A high-resolution image reconstruction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program The steps of implementing the high-resolution image reconstruction method according to any one of claims 1 to 6.
A computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, a high-resolution image according to any one of claims 1 to 6 is realized Refactoring method steps.