WO2019174522A1

WO2019174522A1 - Image generating method and device

Info

Publication number: WO2019174522A1
Application number: PCT/CN2019/077352
Authority: WO
Inventors: 谭文伟
Original assignee: 华为技术有限公司
Priority date: 2018-03-16
Filing date: 2019-03-07
Publication date: 2019-09-19
Also published as: CN110276720B; CN110276720A

Abstract

Embodiments of the present application relate to the technical field of image processing and provide an image generating method and device, capable of resolving the problem that an image processed using the ERM principle is excessively smooth and lack detailed information. The method comprises: determining at least one detail-oriented low-resolution feature map and at least one complementation-oriented low-resolution feature map corresponding to a low-resolution image; determining a detail-oriented super-resolution image corresponding to each of the at least one detail-oriented low-resolution feature map; determining a complementation-oriented super-resolution image corresponding to each of the at least one complementation-oriented low-resolution feature map; and obtaining a super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementation-oriented super-resolution image. The embodiments of the present application are applied in a super-resolution process of an image.

Description

Image generation method and device

The present application claims priority to Chinese Patent Application No. 201 810 222 290, filed on March 16, 2018, the entire disclosure of which is incorporated herein by reference. .

Technical field

The present application relates to the field of image processing technologies, and in particular, to an image generating method and apparatus.

Background technique

Super-Resolution refers to the recovery of high-resolution images from a low-resolution image or sequence of images. The current method of super-resolution processing of most single images is the use of the Empirical Risk Minimisation (ERM) principle. As shown in Fig. 1, the X column represents a low resolution image, and the Y column represents an image processed using the ERM principle.

However, images processed using the ERM principle tend to be excessively smooth over the pixels, resulting in blurred images, too smooth images, and lack of detailed information. The overall effect looks a lot different from the original image.

Summary of the invention

The embodiment of the present invention provides an image generation method and device, which can solve the problem that an image processed by the ERM principle is too smooth and lacks detailed information.

In a first aspect, an embodiment of the present application provides an image generating method, including: determining at least one detail-oriented low-resolution feature map corresponding to a low-resolution image and at least one complementary-oriented low-resolution feature map; determining that at least one detail orientation is low Determining, in each of the resolution maps, a detail-oriented super-resolution image corresponding to the low-resolution feature map; determining a complementary orientation super corresponding to each of the complementary-oriented low-resolution feature maps in the at least one complementary-oriented low-resolution feature map A resolution image; acquiring a super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary-oriented super-resolution image.

Thereby, the at least one detail-oriented low-resolution feature map and the at least one complementary-oriented low-resolution feature map determined according to the low-resolution image respectively record information such as real details of the low-resolution image and other feature information, and the detail orientation is super The resolution image and the complementary directional super-resolution image have more detailed information and other information such as local texture, that is, the weight of the information such as the detailed content is emphasized, so that the output super-resolution image has richer details and structure. At the same time, it can also suppress the sawtooth effect produced by some image processing. As shown in (a) of FIG. 5, the super-resolution image generated by the image generating method provided by the embodiment of the present application has richer detail content and structure, and the visual sense is more natural. As shown in (b) of FIG. 5 , the image generated by the ERM principle can solve the problem that the image is generated by using the ERM principle. The image is too smooth and lacks detailed information.

In a possible implementation, determining the at least one detail-oriented low-resolution feature map and the at least one complementary-oriented low-resolution feature map corresponding to the low-resolution image includes: determining at least one candidate feature map of the low-resolution image; Determining each candidate feature map into at least one candidate feature map, converting the candidate feature map into a grayscale image; dividing the grayscale image into N image blocks, determining that a median value of the gradient histogram corresponding to the N image blocks is greater than Or the number of image blocks D equal to the first preset threshold; if R (R=D/N) is greater than or equal to the second preset threshold, determining that the candidate feature map is a detail-oriented low-resolution feature map; wherein N is greater than Or an integer equal to 1, D is an integer greater than or equal to 0; the pixel value of the detail-oriented low-resolution feature map is subtracted from the pixel value of the gray-scale image of the low-resolution image to obtain a complementary directional low-resolution feature map.

Similarly, the detail-oriented separation module may determine the detail-oriented low-resolution feature map and the complementary-oriented low-resolution feature map according to the low-resolution image. Similarly, the detail-oriented separation module may determine the detail-oriented high-resolution feature according to the high-resolution image. Figure and complementary directional high resolution feature map.

In a possible implementation, determining the detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map comprises: orienting the low-resolution feature map for the at least one detail Each of the details is oriented to the low resolution feature map, and the detail oriented super resolution image is determined according to the detail oriented high resolution feature map corresponding to the detail oriented low resolution feature map and the detail oriented low resolution feature map.

Compared with the detail-oriented low-resolution feature map, the detail-oriented super-resolution image determined according to the detail-oriented low-resolution feature map and the detail-oriented high-resolution feature map has more details and structure, that is, the low-resolution image is emphasized The weight of the details and other information can make the visual sense of the subsequently generated super-resolution image more natural and realistic.

In a possible implementation, determining the complementary directional super-resolution image corresponding to each of the complementary directional low-resolution feature maps of the at least one complementary directional low-resolution feature map comprises: for at least one complementary directional low-resolution feature map Each of the complementary directional low-resolution feature maps determines a complementary directional super-resolution image according to the complementary directional low-resolution feature map and the complementary directional high-resolution feature map corresponding to the complementary directional low-resolution feature map.

Compared with the complementary directional low-resolution feature map, the complementary directional super-resolution image determined according to the complementary directional low-resolution feature map and the complementary directional high-resolution feature map has more content than the detail content, and can be subsequently generated. The visual sense of the super-resolution image is more natural and realistic.

In a possible implementation, acquiring the super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary-oriented super-resolution image comprises: locating each of the super-resolution feature maps for at least one detail Adding a pixel value of the detail-oriented super-resolution image corresponding to the detail-oriented super-resolution feature map and the complementary-oriented super-resolution feature map of the at least one complementary-oriented super-resolution feature map a pixel value of the detail-oriented super-resolution image corresponding to the complementary directional super-resolution feature map; wherein the detail-oriented super-resolution feature map corresponds to the complementary directional super-resolution feature map, ie, the complementary directional super-resolution feature map The corresponding complementary directional low-resolution feature map is obtained by subtracting the pixel value of the detail-oriented low-resolution feature map corresponding to the detail-oriented super-resolution feature map from the pixel value of the corresponding gray-scale image.

In a second aspect, an embodiment of the present application provides an image generating apparatus, including: a detail orientation separating module, configured to determine at least one detail oriented low resolution feature map corresponding to a low resolution image and at least one complementary oriented low resolution feature map a detail-oriented super-division module for determining a detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map; and a complementary directional super-segment module for determining at least one Complementary directional super-resolution image corresponding to each complementary directional low-resolution feature map in the complementary directional low-resolution feature map; super-image fusion module for directional super-resolution image and complementary directional super-resolution image acquisition A super-resolution image corresponding to a low-resolution image.

Thereby, the detail orientation separation module records information such as the real details of the low resolution image and other feature information respectively according to the at least one detail oriented low resolution feature map and the at least one complementary oriented low resolution feature map determined by the low resolution image. The detail-oriented super-resolution image determined by the detail-oriented super-division module and the complementary directional super-resolution image determined by the complementary directional super-division module emphasize the weight of the information such as the detailed content, so that the output super-resolution image has richer details. The content and structure can also suppress the jagged effect produced by some image processing. As shown in (a) of FIG. 5, the super-resolution image generated by the image generating method provided by the embodiment of the present application has richer detail content and structure, and the visual sense is more natural. As shown in (b) of FIG. 5 , the image generated by the ERM principle can solve the problem that the image is generated by using the ERM principle. The image is too smooth and lacks detailed information.

In a possible implementation, the detail orientation separation module is configured to: determine at least one candidate feature map of the low resolution image; and convert each candidate feature map into gray for each candidate feature map in the at least one candidate feature map The image is divided into N image blocks, and the number of image blocks D whose median value of the gradient histogram corresponding to the N image blocks is greater than or equal to the first preset threshold is determined; if R(R=D/N) Or greater than or equal to the second predetermined threshold, determining that the candidate feature map is a detail-oriented low-resolution feature map; wherein N is an integer greater than or equal to 1, and D is an integer greater than or equal to 0; directing the detail to a low resolution The pixel value of the feature map is subtracted from the pixel value of the gray image of the low resolution image to obtain a complementary directional low resolution feature map.

In a possible implementation, the detail oriented super-segment module is configured to: direct each low-resolution feature map for each detail in the at least one detail-oriented low-resolution feature map, and orient the low-resolution feature map and detail orientation according to the detail The detail-oriented high-resolution feature map corresponding to the low-resolution feature map determines the detail-oriented super-resolution image.

In a possible implementation, the complementary directional super-division module is configured to: for each complementary directional low-resolution feature map in the at least one complementary directional low-resolution feature map, according to the complementary directional low-resolution feature map and the complementary orientation The complementary directional high resolution feature map corresponding to the low resolution feature map determines the complementary directional super resolution image.

In a possible implementation, the super-image fusion module is configured to: in each of the at least one detail-oriented super-resolution feature map, the super-resolution feature map and the at least one complementary-oriented super-resolution feature map Each complementary directional super-resolution feature map, the pixel value of the detail-oriented super-resolution image corresponding to the detail-oriented super-resolution feature map and the pixel of the detail-oriented super-resolution image corresponding to the complementary directional super-resolution feature map a value; wherein the detail oriented super-resolution feature map corresponds to a complementary directional super-resolution feature map.

In a third aspect, an embodiment of the present application provides a device, which is in the form of a product of a chip. The device includes a processor and a memory, and the memory is coupled to the processor to save necessary program instructions of the device. And data for executing the program instructions stored in the memory such that the apparatus performs the functions of the image generating apparatus in the above method.

In a fourth aspect, an embodiment of the present application provides an image generating apparatus, which can implement the functions performed by the image generating apparatus in the foregoing method embodiments, and the functions can be implemented by hardware, or can be implemented by hardware. . The hardware or software includes one or more modules corresponding to the above functions.

In one possible design, the image generating apparatus includes a processor and a communication interface configured to support the image generating apparatus to perform a corresponding function in the above method. The communication interface is used to support communication between the image generating device and other network elements. The image generating device can also include a memory for coupling with the processor that holds program instructions and data necessary for the image generating device.

In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, including instructions, when executed on a computer, causing a computer to perform any one of the methods provided by the first aspect.

In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which when executed on a computer, cause the computer to perform any of the methods provided by the first aspect.

DRAWINGS

1 is a schematic diagram of comparison of an image processed by a low resolution image and an ERM principle according to an embodiment of the present application;

2 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present application;

3 is a schematic diagram of an end-to-end system framework provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart diagram of an image generating method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of comparison between an image processed by a super-resolution image and an ERM principle according to an embodiment of the present application; FIG.

FIG. 6 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present application.

detailed description

The embodiment of the present application provides an image generation method and apparatus, which can be applied to a process of image super-resolution, for example, a process of upgrading an SD image to a high-definition image.

FIG. 2 is a schematic diagram of an internal structure of an image generating apparatus according to an embodiment of the present disclosure. In the embodiment of the present application, the image generating apparatus may include a processing module 201, a communication module 202, and a storage module 203. The processing module 201 is used to control various parts of the image generating device, the hardware device, the application software, and the like. The processing module 201 may be a processor or a controller, for example, may be a central processing unit (CPU), a graphics processor. (Graphics Processing Unit, GPU), general purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), Field Programmable Gate Array (FPGA) Or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor can also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. The communication module 202 is configured to receive commands sent by other devices by using a communication method such as Long Term Evolution (LTE) or WIreless-Fidelity (WiFi), or send data of the image generating device to other devices. The communication module 202 can be a transceiver, a transceiver circuit, a communication interface, or the like. The storage module 203 is configured to execute storage of a software program of the image generation device, storage of data, operation of software, etc., and may be a read-only memory (ROM), other types of static storage that can store static information and instructions. A device, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), disk storage media or other magnetic storage devices, Or any other medium that can be used to carry or store desired program code in the form of an instruction or data structure and that can be accessed by a computer, but is not limited thereto.

The image generating device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, or the like structure in FIG. A device that supports image super-resolution technology.

In the embodiment of the present application, further, the processor of the image generating apparatus may implement an image generating method by running an end-to-end system framework, and the software modules included in the end-to-end system framework may be stored in a storage medium such as a memory. . As shown in FIG. 3 , it is a schematic diagram of a logical relationship of an end-to-end system framework provided by an embodiment of the present application. The system framework adopts a semantic network model, including a detail orientation separation module, a detail orientation super division module, and a complementary orientation super division module. And the super-division image fusion module, the input of the system frame is a low-resolution image, and the output is a super-resolution image.

The detail orientation separation module is configured to determine at least one detail-oriented low-resolution feature map and at least one complementary-oriented low-resolution feature map corresponding to the low-resolution image. The detail-oriented super-segment module is configured to determine a detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map, and the detail-oriented super-resolution image has a lower resolution than the detail-oriented super-resolution image The image has more edge structure, edge strength, detail content, and target feature type information. The complementary directional super-division module is operative to determine a complementary directional super-resolution image corresponding to each of the complementary directional low-resolution feature maps in the at least one complementary directional low-resolution feature map. A complementary directional super-resolution image has more other feature information, such as local texture information, than a complementary directional low-resolution image. The super-resolution image fusion module is configured to acquire a super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary directional super-resolution image.

For a clear and concise description of the following embodiments, a brief introduction of related concepts or techniques is first given:

RGB three-channel image: consists of three channels: R, G, and B. Wherein R represents red, G represents green, and B represents blue.

VGG-Net: is a Convolutional Neural Network (CNN). VGG-Net usually has 16-19 convolutional layers.

Mean Square Error (MSE) of the network: In simple terms, the sum of squared errors of a set of numbers is divided by the logarithm of the data.

Accelerating the Super-Resolution Convolutional Neural Network (FSRCNN): A convolutional neural network that typically has four convolutional layers that can be used to compute high-resolution convolutions of low-resolution images. image.

Efficient Sub-Pixel Convolutional Neural Network (ESPCN): Similar to FSRCNN, it is also a convolutional neural network that can compute convolutions of low-resolution images to obtain high-resolution images.

An embodiment of the present application provides an image generating method, as shown in FIG. 4, including:

401. Determine at least one detail-oriented low-resolution feature map and at least one complementary-oriented low-resolution feature map corresponding to the low-resolution image.

The detail directional separation module can be configured to receive the input low resolution image, determine at least one detail oriented low resolution feature map corresponding to the low resolution image, and the at least one complementary directional low resolution feature map.

First, the detail orientation separation module determines at least one candidate feature map of the low resolution image. For example, when the low-resolution image is an RGB three-channel image, VGG-net can be used to perform two consecutive convolutional layer operations on the RGB three-channel image, and each convolution layer operation can include N k*k Convolution kernel operation. Illustratively, N can be an integer between [20, 100], and k can be 3 or 5. Then, the detail orientation separation module may use at least one feature map obtained after the second convolution layer operation as at least one candidate feature map.

For each candidate feature map of the at least one candidate feature map, the detail orientation separation module converts the candidate feature map into a grayscale image; the grayscale image is segmented into N image blocks, and the gradient histogram corresponding to the N image blocks is determined The median value of the graph is greater than or equal to the number D of image blocks of the first predetermined threshold. Where N is an integer greater than or equal to 1, for example, N may be 9, 25 or 49 or the like. D is an integer greater than or equal to zero. If R(R=D/N) is greater than or equal to the second predetermined threshold, the detail orientation separation module determines that the candidate feature map is a detail oriented low resolution feature map. The detail-oriented low-resolution feature map contains edge information and detail content in different directions, and may also contain different image type feature information. It should be noted that R can be used to judge the richness of details, and the higher the R value, the more details, but the less details. Exemplarily, the second preset threshold may be 0.3.

For example, suppose N is 9, that is, the grayscale image obtained by the candidate feature map transformation is divided into nine image blocks, and the values of the edge gradient histograms corresponding to the nine image blocks are 0, 10, 20, 30, 40, respectively. 50, 60, 70, and 80, the first preset threshold is 30, and the number of image blocks D in which the median value of the gradient histogram corresponding to the nine image blocks is greater than or equal to the first preset threshold is 6, and the second preset threshold is 0.3, then R=D/N=0.67, R>0.3, so that the detail-oriented separation module can determine that the candidate feature map is a detail-oriented low-resolution feature map.

Then, the detail orientation separation module subtracts the pixel value of the detail-oriented low-resolution feature image from the pixel value of the gray-scale image of the low-resolution image according to the information complementation principle to obtain a complementary orientation low-resolution feature image.

It can be understood that the function of the detail directional separation module can be determined according to the model parameters. For example, a low-resolution image may be processed by a neural network such as FSRCNN or ESPCN to obtain a processing result, and then a weight back-propagation method is used to update the weight of the processing result according to an MSE error function. Training generates model parameters for the detail-oriented separation module.

It can be understood that the detail-oriented separation module can determine the detail-oriented low-resolution feature map and the complementary-oriented low-resolution feature map according to the low-resolution image. Similarly, the detail-oriented separation module can determine the detail-oriented high-resolution according to the high-resolution image. Rate feature map and complementary directional high resolution feature map. In one possible design, the detail orientation separation module may determine a high resolution image corresponding to the low resolution image based on the low resolution image.

402. Determine a detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map.

That is, the detail oriented super-segment module is configured to obtain at least one detail-oriented low-resolution feature map from the detail-oriented separation module, and determine a detail orientation super corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map. Resolution image.

For each detail-oriented low-resolution feature map in at least one detail-oriented low-resolution feature map, the detail-oriented super-segment module may be based on the detail-oriented low-resolution feature map and the detail-oriented low-resolution feature map corresponding to the detail-oriented high-resolution The rate feature map determines the detail-oriented super-resolution image.

It can be understood that the function of the detail oriented super-segment module can be determined according to the model parameters. For example, the high-resolution detail-oriented feature map corresponding to the low-resolution detail orientation feature map and the low-resolution detail orientation feature map may be processed by using a neural network such as FSRCNN or ESPCN, and then the processing result is obtained, and then according to the MSE error function, The error back propagation method is used to update the weight of the above processing result, and the model parameters of the detail oriented super-division module are trained.

403. Determine a complementary directional super-resolution image corresponding to each of the complementary directional low-resolution feature maps in the at least one complementary directional low-resolution feature map.

That is, the complementary directional super-division module is configured to obtain at least one complementary directional low-resolution feature map from the detail-oriented separation module, and determine a complementary orientation super corresponding to each complementary directional low-resolution feature map in the at least one complementary directional low-resolution feature map. Resolution image.

For each complementary directional low resolution feature map of the at least one complementary directional low resolution feature map, the complementary orientation is determined according to the complementary directional low resolution feature map and the complementary directional high resolution feature map corresponding to the complementary directional low resolution feature map Super resolution image.

It can be understood that the function of the complementary directional super-division module is determined according to the model parameters. For example, the high-resolution complementary orientation feature map corresponding to the low-resolution complementary orientation feature map and the low-resolution complementary orientation feature map may be processed by using a neural network such as FSRCNN or ESPCN, and then the processing result is obtained, and then according to the MSE error function, The error back propagation method is used to update the weight of the above processing result, and the model parameters of the complementary directional super-division module are trained.

404. Acquire a super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary directional super-resolution image.

The super-resolution image fusion module can be configured to acquire at least one detail-oriented super-resolution image from the detail-oriented super-segment module, and can acquire at least one complementary-oriented super-resolution image from the complementary orientation super-segment module, and super-resolution according to at least one detail orientation The rate image and the at least one complementary directional super-resolution image acquire a super-resolution image corresponding to the low-resolution image.

Adding the detail-oriented super-resolution to each of the at least one detail-oriented super-resolution feature map and each of the at least one complementary-oriented super-resolution feature map a pixel value of the detail-oriented super-resolution image corresponding to the feature map and a pixel value of the detail-oriented super-resolution image corresponding to the complementary-oriented super-resolution feature map; wherein the detail-oriented super-resolution feature map corresponds to the complementary orientation super The resolution feature map, that is, the complementary orientation low-resolution feature map corresponding to the complementary orientation super-resolution feature map is a pixel value and a corresponding gray level of the detail-oriented low-resolution feature map corresponding to the detail-oriented super-resolution feature map The pixel values of the image are subtracted.

The solution provided by the embodiment of the present application is mainly introduced from the perspective of the image generating apparatus. It can be understood that the image generating apparatus includes hardware structures and/or software modules corresponding to the execution of the respective functions in order to implement the above functions. Those skilled in the art will readily appreciate that the present application can be implemented in hardware or a combination of hardware and software in combination with the algorithm steps described in the embodiments disclosed herein. Whether a function is implemented in hardware or software-driven hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiment of the present application may divide the function module by the image generation device according to the above method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

FIG. 6 is a schematic diagram showing a possible structure of the image generating apparatus 6 involved in the above embodiment. The image generating apparatus includes: a detail orientation separating module 601, and a detail orientation. The super-division module 602, the complementary directional super-division module 603, and the super-sub-image fusion module 604. The detail orientation separation module 601 is configured to support the image generation apparatus to perform the process 401 in FIG. The detail oriented super-segment module 602 is configured to support the image generation device to perform the process 402 of FIG. The complementary directional super-division module 603 is for supporting the image generation device to perform the process 403 in FIG. The super-segment image fusion module 604 is configured to support the image generation device to perform 404. All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

The steps of a method or algorithm described in connection with the present disclosure may be implemented in a hardware or may be implemented by a processor executing software instructions. The software instructions may be comprised of corresponding software modules that may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable hard disk, read-only optical disk, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor to enable the processor to read information from, and write information to, the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. Additionally, the ASIC can be located in a core network interface device. Of course, the processor and the storage medium may also exist as discrete components in the core network interface device.

Those skilled in the art will appreciate that in one or more examples described above, the functions described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored in an image generation device readable medium or transmitted as one or more instructions or code on an image generation device readable medium. The image generating device readable medium includes an image generating device storage medium and a communication medium, wherein the communication medium includes any medium that facilitates transfer of an image generating device program from one place to another. The storage medium can be any available media that can be accessed by a general purpose or special purpose image generating device.

The specific embodiments of the present invention have been described in detail with reference to the specific embodiments of the present application. It is to be understood that the foregoing description is only The scope of protection, any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application are included in the scope of protection of the present application.

Claims

An image generating method, comprising:

Determining at least one detail-oriented low-resolution feature map and at least one complementary-oriented low-resolution feature map corresponding to the low-resolution image;

Determining a detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map;

Determining a complementary directional super-resolution image corresponding to each of the complementary directional low-resolution feature maps in the at least one complementary directional low-resolution feature map;

Acquiring the super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary directional super-resolution image.
The method according to claim 1, wherein the determining the at least one detail-oriented low-resolution feature map and the at least one complementary-oriented low-resolution feature map corresponding to the low-resolution image comprises:

Determining at least one candidate feature map of the low resolution image;

For each candidate feature map of the at least one candidate feature map, converting the candidate feature map into a grayscale image; dividing the grayscale image into N image blocks, and determining a gradient histogram corresponding to the N image blocks The median value of the graph is greater than or equal to the number of image blocks D of the first preset threshold; if R (R=D/N) is greater than or equal to the second preset threshold, determining that the candidate feature map is the detail-oriented low-resolution feature Figure; wherein N is an integer greater than or equal to 1, and D is an integer greater than or equal to 0;

And subtracting the pixel value of the detail-oriented low-resolution feature image from the pixel value of the gray-scale image of the low-resolution image to obtain the complementary directional low-resolution feature image.
The method according to claim 1 or 2, wherein the determining the detail-oriented super-resolution image corresponding to each of the detail-oriented low-resolution feature maps in the at least one detail-oriented low-resolution feature map comprises:

For each of the at least one detail-oriented low-resolution feature map, the low-resolution feature map is oriented according to the detail-oriented low-resolution feature map and the detail-oriented low-resolution feature map corresponding to the detail-oriented high-resolution The rate profile determines the detail oriented super-resolution image.
The method of any of claims 1-3, wherein the determining the complementary directional super-resolution corresponding to each of the complementary directional low-resolution feature maps in the at least one complementary directional low-resolution feature map The image includes:

For each of the complementary directional low resolution feature maps of the at least one complementary directional low resolution feature map, the complementary directional high resolution corresponding to the complementary directional low resolution feature map and the complementary directional low resolution feature map The rate profile determines the complementary directional super-resolution image.
The method according to any one of claims 1 to 4, wherein the acquiring the super resolution corresponding to the low resolution image according to the detail oriented super resolution image and the complementary directional super resolution image The image includes:

Adding the detail-oriented super-resolution to each of the at least one detail-oriented super-resolution feature map and each of the at least one complementary-oriented super-resolution feature map a pixel value corresponding to the detail-oriented super-resolution image corresponding to the feature map and a pixel value of the detail-oriented super-resolution image corresponding to the complementary directional super-resolution feature map;

Wherein, the detail-oriented super-resolution feature map corresponds to a complementary orientation super-resolution feature map.
An image generating apparatus, comprising:

a detail directional separation module, configured to determine at least one detail-oriented low-resolution feature map and at least one complementary directional low-resolution feature map corresponding to the low-resolution image;

a detail-oriented super-segment module, configured to determine a detail-oriented super-resolution image corresponding to each of the at least one detail-oriented low-resolution feature map;

a complementary directional super-resolution module for determining a complementary directional super-resolution image corresponding to each of the complementary directional low-resolution feature maps in the at least one complementary directional low-resolution feature map;

And a super-image fusion module, configured to acquire a super-resolution image corresponding to the low-resolution image according to the detail-oriented super-resolution image and the complementary directional super-resolution image.
The image generating apparatus according to claim 6, wherein the detail directional separation module is configured to:

Determining at least one candidate feature map of the low resolution image;

For each candidate feature map of the at least one candidate feature map, converting the candidate feature map into a grayscale image; dividing the grayscale image into N image blocks, and determining a gradient histogram corresponding to the N image blocks The median value of the graph is greater than or equal to the number of image blocks D of the first preset threshold; if R (R=D/N) is greater than or equal to the second preset threshold, determining that the candidate feature map is the detail-oriented low-resolution feature Figure; wherein N is an integer greater than or equal to 1, and D is an integer greater than or equal to 0;

And subtracting the pixel value of the detail-oriented low-resolution feature image from the pixel value of the gray-scale image of the low-resolution image to obtain the complementary directional low-resolution feature image.
The image generating apparatus according to claim 6 or 7, wherein the detail oriented super-segmenting module is configured to:

For each of the at least one detail-oriented low-resolution feature map, the low-resolution feature map is oriented according to the detail-oriented low-resolution feature map and the detail-oriented low-resolution feature map corresponding to the detail-oriented high-resolution The rate profile determines the detail oriented super-resolution image.
The image generating apparatus according to any one of claims 6 to 8, wherein the complementary directional super-division module is used to:

For each of the complementary directional low resolution feature maps of the at least one complementary directional low resolution feature map, the complementary directional high resolution corresponding to the complementary directional low resolution feature map and the complementary directional low resolution feature map The rate profile determines the complementary directional super-resolution image.
The image generating apparatus according to any one of claims 6 to 9, wherein the super-image fusion module is configured to:

Adding the detail-oriented super-resolution to each of the at least one detail-oriented super-resolution feature map and each of the at least one complementary-oriented super-resolution feature map a pixel value corresponding to the detail-oriented super-resolution image corresponding to the feature map and a pixel value of the detail-oriented super-resolution image corresponding to the complementary directional super-resolution feature map;

Wherein, the detail-oriented super-resolution feature map corresponds to a complementary orientation super-resolution feature map.