WO2021258529A1 - 图像降分辨率及复原方法、设备及可读存储介质 - Google Patents

图像降分辨率及复原方法、设备及可读存储介质 Download PDF

Info

Publication number
WO2021258529A1
WO2021258529A1 PCT/CN2020/111771 CN2020111771W WO2021258529A1 WO 2021258529 A1 WO2021258529 A1 WO 2021258529A1 CN 2020111771 W CN2020111771 W CN 2020111771W WO 2021258529 A1 WO2021258529 A1 WO 2021258529A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
resolution
restoration
low
target
Prior art date
Application number
PCT/CN2020/111771
Other languages
English (en)
French (fr)
Inventor
王荣刚
王振宇
韩冰杰
李旭峰
高文
Original Assignee
北京大学深圳研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学深圳研究生院 filed Critical 北京大学深圳研究生院
Publication of WO2021258529A1 publication Critical patent/WO2021258529A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4092—Image resolution transcoding, e.g. by using client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • This application relates to the field of artificial intelligence, and in particular to an image resolution reduction and restoration method, device, and readable storage medium.
  • the main purpose of this application is to provide an image resolution reduction and restoration method, device, and readable storage medium, aiming to solve the technical problem of low accuracy of restoration after image resolution reduction in the prior art.
  • the present application provides an image resolution reduction and restoration method, which is applied to an image resolution reduction and restoration method device, and the image resolution reduction and restoration method includes:
  • the target low-resolution image is input into a preset image restoration model to perform high-resolution restoration on the target low-resolution image based on the image gradient information to obtain a target restored image.
  • the present application also provides an image resolution reduction and restoration method device, the image resolution reduction and restoration method device is a virtual device, and the image resolution reduction and restoration method device is applied to the image Resolution reduction and restoration method equipment, the image resolution reduction and restoration method device includes:
  • the determining module is used to obtain the image to be processed and determine the image gradient information corresponding to the image to be processed;
  • the resolution reduction module is used to obtain the input of the image to be processed into a preset image reduction model, to reduce the resolution of the image to be processed, to obtain an initial low-resolution image;
  • a storage module configured to store the image gradient information in the initial low-resolution image to obtain a target low-resolution image
  • the restoration module is configured to input the target low-resolution image into a preset image restoration model to perform high-resolution restoration on the target low-resolution image based on the image gradient information to obtain a target restored image.
  • This application also provides an image resolution reduction and restoration method and equipment, the image resolution reduction and restoration method equipment is a physical device, and the image resolution reduction and restoration method equipment includes: a memory, a processor, and storage
  • the program of the image resolution reduction and restoration method that can be executed on the memory and can be run on the processor, and when the program of the image resolution reduction and restoration method is executed by the processor, the image resolution reduction and restoration method described above can be realized The steps of the recovery method.
  • the present application also provides a readable storage medium, the readable storage medium stores a program for implementing image resolution reduction and restoration method, and when the program for image resolution reduction and restoration method is executed by a processor, the implementation is as described above The steps of the image reduction and restoration method.
  • This application first calculates the image gradient information of the image to be processed before reducing the resolution of the image to be processed, and then stores the image gradient information in the low-resolution image after reducing the resolution of the image to be processed into a low-resolution image.
  • the image gradient information in the low-resolution image is extracted, and then based on the image gradient information, the low-resolution image is accurately restored to obtain the target restored image, where, Since the image gradient information has a guiding effect on the process of restoring the low-rate image to the original high-resolution image, the target restored image is closer to the original image to be processed, and the image restoration process is more accurate, and Because the image gradient information is stored in the low-resolution image, it does not occupy additional storage space, thereby achieving the purpose of accurate restoration of the low-resolution image without adding additional storage space. Therefore, the image The technical problem of low accuracy is restored after the resolution is reduced.
  • FIG. 1 is a schematic flowchart of a first embodiment of a method for image resolution reduction and restoration according to this application;
  • FIG. 2 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the application.
  • the embodiment of the present application provides an image resolution reduction and restoration method.
  • the image resolution reduction and restoration method includes:
  • Step A10 Obtain an image to be processed, and determine image gradient information corresponding to the image to be processed;
  • the image gradient information is a matrix composed of gradient values corresponding to each pixel in the image to be processed, and the image to be processed is a higher resolution image that needs to be reduced to a lower resolution.
  • Resolution images for example, assuming a lower resolution image is an 8bit image, the bit depth of the image to be processed should be greater than 8bit, for example, 10bit.
  • the image to be processed is acquired, and the pixel matrix of the image to be processed corresponding to the image to be processed is determined, where the pixel matrix of the image to be processed is a matrix composed of pixel coordinates corresponding to each pixel in the image to be processed, wherein the pixel coordinates
  • the corresponding amplitude is the pixel value, which is used to indicate the brightness of the image area of the pixel.
  • the pixel coordinate is (3, 4)
  • the corresponding pixel value is 5, etc.
  • calculate the pixel matrix of the image to be processed The horizontal gradient value and vertical gradient value corresponding to each pixel in the pixel, and then based on the horizontal gradient value and vertical gradient value corresponding to each pixel, the gradient value corresponding to each pixel is determined, and the image corresponding to the pixel matrix of the image to be processed is obtained
  • the gradient information matrix that is, to obtain the image gradient information, for example, suppose that the pixel coordinates corresponding to a certain pixel in the image to be processed are the vector X(i,j) and the coordinates corresponding to the adjacent pixel points are ( i+1, j) and (i, j+1), the horizontal gradient value The vertical gradient value is Then the gradient value corresponding to the pixel is
  • Step A20 input the to-be-processed image into a preset image resolution reduction model, and perform resolution-reduction on the to-be-processed image to obtain an initial low-resolution image;
  • the preset image resolution reduction model is a trained neural network model for image resolution reduction.
  • the image to be processed is input into a preset image reduction resolution model, the image to be processed is down-sampled to generate a thumbnail corresponding to the image to be processed, the down-sampled image is obtained, and the down-sampling is performed based on the preset bit depth gain factor
  • Each pixel corresponding to the image undergoes bit reduction and depth processing to map the pixel value corresponding to each pixel to a preset low-resolution value range to obtain an initial low-resolution image.
  • the process of reducing the resolution is as follows:
  • Y is the initial low-resolution image
  • X is the image to be processed
  • g is the preset bit depth gain factor
  • D R is the down-sampling method
  • round is the rounding.
  • the step of reducing the resolution of the image to be processed to obtain an initial low-resolution image includes:
  • Step A21 down-sampling the to-be-processed image to obtain a down-sampled image
  • the down-sampled image can be represented by a down-sampling matrix, where the down-sampling matrix is the pixel matrix of the down-sampled image, and the down-sampling methods include nearest neighbor sampling method, quadratic interpolation method, double Cubic convolution method and so on.
  • Step A22 based on a preset bit depth gain factor, perform bit depth reduction processing on the down-sampled image to obtain the initial low-resolution image.
  • the initial low-resolution image can be represented by the initial low-resolution image matrix
  • the initial low-resolution image matrix is the pixel matrix of the initial low-resolution image
  • the preset bit depth gain factor is based on the high-resolution image.
  • Image bit depth and image bit depth of low-resolution images determine the bit depth influence factor, used to convert the image bit depth of a high-resolution image to the image bit depth of a low-resolution image or convert the image bit depth of a low-resolution image Converted to the image bit depth of a high-resolution image, where the resolution of a high-resolution image is higher than that of a low-resolution image. For example, assuming that the image bit depth of a high-resolution image is Nbit, the image of a low-resolution image If the bit depth is Mbit, the calculation method of the preset bit depth gain factor g is as follows:
  • the down-sampling matrix corresponding to the down-sampled image is determined, and each pixel value in the down-sampled matrix image is divided by a preset bit depth gain factor and rounded to obtain the initial low-resolution image matrix.
  • Step A30 storing the image gradient information in the initial low-resolution image to obtain a target low-resolution image
  • the image gradient information includes an initial gradient map, where the initial gradient map is an information map of the gradient value corresponding to each pixel, and the initial gradient image corresponds to an initial gradient value matrix.
  • the matrix is a matrix used to store the gradient value corresponding to each pixel.
  • the visual saliency of each b-bit in the image bit is different.
  • the image bit with a lower visual saliency is an insignificant bit, and it is usually Adjusting the value of the insignificant bit usually does not affect the visual effect of the image. For example, if the image bit depth of the image is 10bit, it can be determined that the last 2bit of the 10bit is the insignificant bit.
  • the initial low-resolution image is shifted to the left to obtain the left-shifted vacant bit, and the left-shifted vacant bit is used as the target insignificant bit of the initial low-resolution image.
  • the initial gradient map is quantized and coded to obtain The target gradient map, where the target gradient map is matched with the target insignificant bit, and then the target gradient map is stored in the target insignificant bit to obtain the target low-resolution image, where the target low-resolution image and the image bit of the image to be processed
  • the depth is the same, and because the obvious position of the target has extremely low impact on the visual effect of the target low-resolution image, the visual effect of the target low-resolution image and the image to be processed should be consistent.
  • the initial low-resolution image has a depth of 8bit. Since the initial low-resolution image is an image obtained by reducing the resolution of the image to be processed, the visual effect of the initial low-resolution image is consistent with that of the image to be processed.
  • the rate image is shifted by 2bit to the left to obtain a 2bit left-shift vacancy bit, and the 2bit left-shift vacancy bit is used as the target insignificant bit.
  • the initial gradient map is quantized and encoded to obtain the target gradient of the 2bit low-resolution image Figure, and then write the target gradient map into the target inconspicuous bit to obtain a 10-bit target low-resolution image, and because the target inconspicuous bit has a very low impact on the visual effect of the image, the target low-resolution image and the initial low-resolution image
  • the visual effect of the target low-resolution image is consistent with the visual effect of the image to be processed.
  • this embodiment achieves the purpose of storing the gradient map in the image, thereby saving storage space for the storage of the gradient image, and when storing the gradient map in the image, it does not affect the vision of the image.
  • the effect that is, while realizing accurate resolution reduction and restoration of the image, the storage space is saved, thereby saving the storage cost.
  • the image gradient information is stored in the initial low-resolution image, and the process of obtaining the target low-resolution image can be expressed by the following formula:
  • X is the image to be processed
  • the image gradient information includes the initial gradient map, Grad represents gradient calculation, Quant represents quantization, C represents encoding, G is the target gradient image, Y is the initial low-resolution image, M is the bit value for left shift, and Y * is Target low-resolution image, bitshift stands for left shift.
  • the step of storing the image gradient information in the initial low-resolution image, and obtaining the target low-resolution image includes:
  • Step A31 quantizing and encoding the initial gradient map to obtain a target gradient map
  • the initial gradient map is quantized to obtain a quantized gradient map
  • the quantized gradient map is down-resolution encoded to reduce the resolution of the quantized gradient map from the high-resolution gradient information map to the low-resolution Rate gradient information map to obtain the target gradient map.
  • the step of quantizing and encoding the initial gradient map to obtain the target gradient map includes:
  • Step A311 Determine the number of difference bits between the image to be processed and the initial low-resolution image, and quantize the initial gradient map based on the number of difference bits to obtain a quantized gradient map;
  • the number of difference bits between the image to be processed and the initial low-resolution image is determined, and the initial gradient map is then quantized into a quantized gradient map corresponding to the number of difference bits.
  • the initial low-resolution image is an 8-bit image
  • the quantized gradient map is a 2-bit image.
  • Step A312 Perform down-resolution encoding on the quantized gradient map to obtain the target gradient map.
  • the quantized gradient map is de-resolution-encoded to reduce the resolution of the quantized gradient map to the low-resolution gradient information map corresponding to the initial low-resolution image to obtain Target gradient map.
  • Step A32 Shift the initial low-resolution image to the left to obtain a target storage bit
  • the number of difference bits between the image to be processed and the initial low-resolution image is determined, and then based on the number of difference bits, the initial low-resolution image is shifted to the left to obtain left-shift vacancies, Use the left-shift vacancy as the target storage bit corresponding to the target low-resolution image. For example, if the image to be processed is a 10-bit image and the initial low-resolution image is an 8-bit image, then the initial low-resolution image is shifted to the left by 2bit to obtain 2bit The left-shifted vacant bit position of 2bit is the target storage bit.
  • Step A33 Store the target gradient map in the target storage bit to obtain the target low-resolution image.
  • the target gradient map is stored in the target storage bit to obtain the target low-resolution image. Specifically, the target gradient map is written into the target storage bit to obtain the target low-resolution image. Rate image.
  • the data length of the target gradient map is not a fixed value
  • the target gradient map is written into the initial low-resolution image to obtain the target low-resolution image
  • the remaining storage capacity is to complement the missing bit values of the remaining pixels corresponding to the remaining storage capacity in the target low-resolution image.
  • the complement method includes ideal gain methods.
  • the target low-resolution image If the storage capacity is insufficient, the target gradient map needs to be down-sampled to reduce the data length of the resolution target gradient map, and then the reduced-resolution target gradient map is written into the initial low-resolution image.
  • Step A40 Input the target low-resolution image into a preset image restoration model to perform high-resolution restoration on the target low-resolution image based on the image gradient information to obtain a target restored image.
  • the preset image restoration model is a neural network model trained for image restoration, and the preset image restoration model includes gradient bit depth expansion network, image bit depth expansion and upsampling network and Image enhancement network, where the gradient bit depth extension network is used to decode and reconstruct the gradient map neural network, the image bit depth extension and upsampling network is used for image bit depth extension and upsampling neural network, image The enhancement network is a neural network that restores the high-frequency details of the image based on the gradient map.
  • extract the target gradient map and the initial low resolution image from the target low resolution image and then input the target gradient map into the gradient bit depth expansion network, perform image bit depth expansion and decoding on the target gradient map, and obtain the initial gradient map, and Input the initial low-resolution image into the image bit depth extension and up-sampling network, perform image bit-depth extension and up-sampling on the initial low-resolution image to obtain the initial restored image, and then input the initial gradient map and the initial restored image into the image enhancement network, Based on the initial gradient map, the high-frequency details of the initial restored image are promoted to obtain the target restored image.
  • the gradient bit-depth expansion network includes a bit-depth pre-expansion layer, a 3*3 input layer, N1 residual blocks, and a 3*3 output layer, where the bit-depth pre-expansion layer
  • the input layer is used to convert the image into 64 feature maps
  • the output layer is used to reconstruct the 64 feature maps into the output result
  • the difference block includes a 64-channel expanded convolutional layer, a ReLU activation function layer, a 64-channel 3*3 convolutional layer, and a residual connection connecting both ends.
  • the residual block is used to expand the image Convolution to increase the receptive field of the image.
  • the image bit depth expansion and up-sampling network includes a bit depth pre-expansion layer, a 3*3 input layer, N2 residual blocks, and a post-input layer.
  • the image enhancement network includes a 3*3 input layer, N3 convolutional layers, a residual connection from the input layer to the convolutional layer, and a 3*3 output layer.
  • N1, N2, and N3 are determined during the training of the preset image reduction model and the preset image restoration model.
  • the preset image restoration model includes a gradient bit depth extension network, an image bit depth extension and upsampling network, and an image enhancement network,
  • the step of inputting the target low-resolution image into a preset image restoration model to perform high-resolution restoration on the target low-resolution image based on the image gradient information, and obtaining the target restored image includes:
  • Step A41 extract the initial low-resolution image and the target gradient map from the target low-resolution image
  • the target gradient map is extracted from the target insignificant bits in the target low-resolution image
  • the initial image is extracted from other bits in the target low-resolution image except the target insignificant bits. Low resolution image.
  • Step A42 input the target gradient map to the gradient bit depth expansion network, and decode and reconstruct the target gradient map to obtain an initial gradient map;
  • the target gradient map is input to the gradient bit depth expansion network, the target gradient map is decoded to obtain the decoded gradient map, and then based on the bit depth pre-expansion layer of the gradient bit depth expansion network, the gradient map is decoded Multiply the preset bit depth gain factor to perform the initial bit depth enhancement of the decoded gradient map to obtain the preliminary bit depth enhancement gradient map, and then based on the gradient bit depth expansion network input layer, the preliminary bit depth enhancement gradient map is converted into a preset A number of gradient feature maps, and then based on each residual block of the gradient bit depth expansion network, perform expansion convolution on each gradient feature map to obtain each expanded convolution feature map. Further, based on the output layer of the gradient bit depth expansion network, Reconstruct each expanded convolution feature map to obtain an initial gradient map.
  • Step A43 input the initial low-resolution image to the image bit depth expansion and upsampling network, and perform depth expansion and super-resolution reconstruction on the initial low-resolution image to obtain an initial restored image;
  • the initial low-resolution image is input to the image bit depth extension and upsampling network, and based on the bit depth pre-expansion layer of the image bit depth extension and upsampling network, the initial low-resolution image is multiplied by the pre-expansion layer.
  • Set the bit depth gain factor to initially increase the bit depth of the initial low-resolution image to obtain a preliminary bit depth increase image.
  • the first residual block, upsampling Layer, the second residual block and the output layer perform depth expansion and super-resolution reconstruction on the preliminary bit depth enhancement image to obtain the initial restored image
  • the first residual block is used to perform bit depth information on the preliminary bit depth enhancement image Extension
  • the second residual block is used for feature extraction and reconstruction of the initial bit depth boosted image after up-sampling the initial bit depth boosted image.
  • Step A44 input the initial gradient map and the initial restored image to the image enhancement network, perform fusion reconstruction on the initial gradient map and the initial restored image, to obtain the target restored image.
  • the initial gradient map and the initial restored image are input to the image enhancement network to improve the high-frequency details of the initial restored image relative to the image to be processed based on the initial gradient map to obtain the target restored image, for example, for
  • the pixel value corresponding to a target pixel in the processed image is 355.
  • the pixel value corresponding to the target pixel is 88.
  • the pixel value corresponding to the target pixel is 353.
  • the image enhancement network based on the initial gradient map, the pixel value 353 can be accurately increased to 355, thereby improving the accuracy of image restoration.
  • the image gradient information of the image to be processed is first calculated, and then after reducing the resolution of the image to be processed to a low-resolution image, the image gradient information is stored in the low-resolution image, Then when the image needs to be restored, the image gradient information in the low-resolution image is extracted, and then based on the image gradient information, the low-resolution image is accurately restored to obtain the target restored image.
  • the process of restoring the original high-resolution image to the original high-resolution image has a guiding effect, which in turn makes the target restored image closer to the original image to be processed, thereby making the image restoration process more accurate, and because the image gradient information is stored in the low-resolution image, It does not take up additional storage space, and thus achieves the purpose of accurate restoration of low-resolution images without adding additional storage space. Therefore, the technical problem of low restoration accuracy after image resolution is solved is solved .
  • the image resolution reduction and restoration method includes:
  • Step A10 Obtain a training image, a real low-resolution image corresponding to the training image, a reduced-resolution model of the image to be trained, and a restoration model of the image to be trained;
  • both the to-be-trained image reduction model and the to-be-trained image restoration model are untrained neural network models.
  • Step A20 based on the training image and the real low-resolution image, perform iterative training and optimization on the to-be-trained image reduction model and the to-be-trained image restoration model until the to-be-trained image reduction model And the image restoration model to be trained reaches a preset second iterative training end condition jointly corresponding to the preset second iteration training end condition, and the preset image reduction model and the preset image restoration model are obtained.
  • the preset second iteration training end conditions include model loss convergence, reaching the maximum number of iteration training, and so on.
  • the training image is input to the image-to-be-trained image reduction model, the training image is reduced in resolution to obtain the training low-resolution image, and based on the training low-resolution image and the real low-resolution image, the first model training loss is calculated Further, the training gradient map corresponding to the training image is stored in the training low-resolution image, the target input low-resolution image is obtained, and the target low-resolution image is input to the image restoration model to be trained, and the target low-resolution image is high-resolution image.
  • Resolution restoration training high-resolution images are obtained, and based on the training images and training high-resolution images, the second model training loss is calculated, and then based on the first model training loss and the second model training loss, the total model loss is calculated, and further , Determine whether the total loss of the model converges, if the total loss of the model converges, the image reduction model to be trained is used as the preset image resolution reduction model, and the image restoration model to be trained is used as the preset image restoration model. If the total loss of the model is not Convergence, iterative training optimization is performed on the reduced-resolution model of the image to be trained and the image restoration model to be trained until the total loss of the model converges.
  • the preset second iterative training end condition includes the convergence of the total model training loss corresponding to the down-resolution model of the image to be trained and the image restoration model to be trained, and
  • the iterative training optimization is performed on the to-be-trained image reduction model and the to-be-trained image restoration model until the to-be-trained image reduction model and
  • the step of the image restoration model to be trained to reach a common corresponding preset second iteration training end condition, and the step of obtaining the preset image resolution reduction model and the preset image restoration model includes:
  • Step A21 Calculate the training image gradient information of the training image, and reduce the resolution of the training image based on the image-to-be-trained image resolution reduction model to obtain an initial training low-resolution image;
  • the training image gradient information of the training image is calculated, and the training image is input to the image reduction model to be trained, and the training is down-sampled to generate a thumbnail corresponding to the training image to obtain the training down-sampling Image, and based on the preset bit depth gain factor, perform down-bit depth processing on each pixel corresponding to the training down-sampled image to map the pixel value corresponding to each pixel of the training image to the preset low-resolution value range , To obtain the initial training low-resolution image.
  • Step A22 storing the training image gradient information in the initial training low-resolution image to obtain an output low-resolution image
  • the initial training low-resolution image is shifted left to obtain the training left shift vacancy, and the training left shift vacancy is used as the training target insignificant bit of the initial training low-resolution image, and then The gradient information of the training image is written into the insignificant bits of the training target, and the output low-resolution image is obtained.
  • Step A23 Calculate the training loss of the first model based on the output low-resolution image and the real low-resolution image;
  • the first distance between the output low-resolution image and the real low-resolution image is obtained, and the first distance is used as the first model training loss.
  • Step A24 based on the image restoration model to be trained, perform high-resolution restoration on the output low-resolution image to obtain an output high-resolution image;
  • the training image gradient information and the initial training low-resolution image are extracted from the output low-resolution image, and then the image bit depth extension and decoding are performed on the training image gradient information to obtain the initial training gradient map, and Perform image bit depth expansion and upsampling on the initial training low-resolution image to obtain the training initial restoration image, and then based on the initial training gradient map, improve the high-frequency details of the training initial restoration image, and obtain the output high-resolution image.
  • Step A25 Calculate the training loss of the second model based on the output high-resolution image and the training image;
  • the second distance between the output high-resolution image and the training image is calculated, and the second distance is used as the second model training loss.
  • Step A26 Calculate the total model training loss based on the first model training loss and the second model training loss
  • a weighted summation is performed on the first model training loss and the second model training loss to obtain the total model training loss.
  • the calculation process of the total model training loss is as follows:
  • ⁇ is the preset weight
  • L 1 is the training loss of the first model
  • L 2 is the training loss of the second model.
  • Step A27 Determine whether the total model training loss converges, and if the total model training loss converges, use the image resolution reduction model to be trained as the preset image resolution reduction model, and set the resolution reduction model of the image to be trained An image restoration model as the preset image restoration model;
  • step A28 if the total model training loss does not converge, re-train and optimize the to-be-trained image reduction model and the to-be-trained image restoration model until the total model training loss converges.
  • the training image and the real low-resolution image are re-acquired to re-train and optimize the reduced-resolution model of the training image and the restoration model of the image to be trained until The total model training loss converges.
  • the training image, the real low-resolution image corresponding to the training image, the image-to-be-trained image reduction model, and the image-to-be-trained image restoration model are acquired, and then based on the training image and the real low-resolution image, the image-to-be-trained image reduction model is Perform iterative training optimization with the image restoration model to be trained, until the image reduction model to be trained and the image restoration model to be trained reach the common corresponding preset second iteration training end condition, and the preset image reduction model and the preset image are obtained Restore the model.
  • this application provides an end-to-end method for training a preset image reduction model and a preset image restoration model, and then based on the trained preset image reduction model and the preset image restoration model, it can be realized
  • the resolution reduction and high resolution restoration of the image lays a foundation for the purpose of accurate restoration of the reduced resolution image without adding additional storage space, and then solves the problem of accurate restoration after the resolution of the image is reduced.
  • the preset image restoration model includes a gradient bit depth extension network
  • the image resolution reduction and restoration methods also include:
  • Step B10 Obtain the network to be trained, the training gradient information data, and the true gradient information data corresponding to the training gradient information data;
  • the network to be trained is an untrained neural network
  • the training gradient information data is the target training gradient image determined by quantizing and encoding the initial training gradient image of the training image
  • the true gradient information data Is the true gradient map corresponding to the training image.
  • Step B20 Based on the training gradient information data and the real gradient information data, iterative training optimization is performed on the network to be trained until the network to be trained reaches a preset first iterative training end condition, and the gradient position is obtained. Deeply expand the network.
  • the preset first iteration training end conditions include model loss convergence, reaching the threshold of the maximum number of iterations, and so on.
  • the target training gradient map is input to the network to be trained, the target training gradient map is decoded, and the training decoded gradient map is obtained, and then based on the preset bit depth gain factor, the training decoded gradient map is initially upgraded to obtain the training image Bit boost gradient map, and then expand the convolution of the training image bit gradient map to obtain the training output gradient map, and then calculate the third distance between the training output gradient map and the real gradient map, and use the third distance as the third model training If the training loss of the third model converges, the network to be trained will be used as a gradient bit depth extension network. If the training loss of the third model does not converge, the training network will be re-trained to optimize iterative training Until the training loss of the third model converges.
  • the network to be trained by acquiring the network to be trained, the training gradient information data, and the real gradient information data corresponding to the training gradient information data, and then based on the training gradient information data and the real gradient information data, the network to be trained is iteratively trained and optimized until the network to be trained The preset first iteration training end condition is reached, and the gradient bit depth expansion network is obtained.
  • this embodiment provides a method for training a gradient bit depth expansion network, and based on the gradient bit depth expansion network, the target gradient map stored in the initial low-resolution image can be reconstructed into the initial gradient map, and then based on The initial gradient map realizes the high-resolution restoration of the target low-resolution image, that is, it lays the foundation for the high-resolution restoration of the target low-resolution image, and then solves the problem of low-precision restoration after the image is reduced in resolution.
  • FIG. 2 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.
  • the image resolution reduction and restoration method device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002.
  • the communication bus 1002 is used to implement connection and communication between the processor 1001 and the memory 1005.
  • the memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory.
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
  • the image resolution reduction and restoration method equipment may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on.
  • the rectangular user interface may include a display screen (Display) and an input sub-module such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface.
  • the network interface can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).
  • image resolution reduction and restoration method device structure shown in FIG. 2 does not constitute a limitation on the image resolution reduction and restoration method device, and may include more or fewer components than shown in the figure. Or combine certain components, or different component arrangements.
  • the memory 1005 which is a computer storage medium, may include an operating system, a network communication module, and an image resolution reduction and restoration method program.
  • the operating system is a program that manages and controls the hardware and software resources of the image resolution reduction and restoration method equipment, and supports the operation of the image resolution reduction and restoration method and other software and/or programs.
  • the network communication module is used to realize the communication between the internal components of the memory 1005 and the communication with other hardware and software in the image resolution reduction and restoration method system.
  • the processor 1001 is used to execute the image resolution reduction and restoration method program stored in the memory 1005 to implement the image resolution reduction and restoration described in any of the above Method steps.
  • An embodiment of the present application also provides an image resolution reduction and restoration method device, the image resolution reduction and restoration method device is applied to an image resolution reduction and restoration method device, and the image resolution reduction and restoration method device includes:
  • the determining module is used to obtain the image to be processed and determine the image gradient information corresponding to the image to be processed;
  • the resolution reduction module is used to obtain the input of the image to be processed into a preset image reduction model, to reduce the resolution of the image to be processed, to obtain an initial low-resolution image;
  • a storage module configured to store the image gradient information in the initial low-resolution image to obtain a target low-resolution image
  • the restoration module is configured to input the target low-resolution image into a preset image restoration model to perform high-resolution restoration on the target low-resolution image based on the image gradient information to obtain a target restored image.
  • the storage module includes:
  • a quantization and encoding unit configured to quantize and encode the initial gradient map to obtain a target gradient map
  • the left shift unit is used to shift the initial low-resolution image to the left to obtain the target storage bit
  • the first storage unit is configured to store the target gradient map in the target storage bit to obtain the target low-resolution image.
  • the quantization and coding unit includes:
  • the quantization subunit is used to determine the number of difference bits between the image to be processed and the initial low-resolution image, and to quantize the initial gradient map based on the number of difference bits to obtain a quantized gradient map ;
  • the encoding subunit is used to perform down-resolution encoding on the quantized gradient map to obtain the target gradient map.
  • the restoration module includes:
  • An extraction unit configured to extract the initial low-resolution image and the target gradient map from the target low-resolution image
  • a decoding and reconstruction unit configured to input the target gradient map into the gradient bit depth expansion network, decode and reconstruct the target gradient map, and obtain an initial gradient map
  • a depth expansion and reconstruction unit configured to input the initial low-resolution image into the image bit depth expansion and upsampling network, and perform depth expansion and super-resolution reconstruction on the initial low-resolution image to obtain an initial restored image;
  • the fusion reconstruction unit is configured to input the initial gradient map and the initial restoration image into the image enhancement network, perform fusion reconstruction on the initial gradient map and the initial restoration image, to obtain the target restoration image.
  • the resolution reduction module includes:
  • a down-sampling unit configured to down-sample the to-be-processed image to obtain a down-sampled image
  • the down-bit depth processing unit is configured to perform down-bit depth processing on the down-sampled image based on a preset bit depth gain factor to obtain the initial low-resolution image.
  • the image resolution reduction and restoration device further includes:
  • the first acquisition module is used to acquire the network to be trained, the training gradient information data, and the real gradient information data corresponding to the training gradient information data;
  • the first iterative training optimization module is configured to perform iterative training optimization on the network to be trained based on the training gradient information data and the real gradient information data until the network to be trained reaches the preset first iterative training end condition , To obtain the gradient bit depth extension network.
  • the image resolution reduction and restoration device further includes:
  • the second acquisition module is used to acquire the training image, the real low-resolution image corresponding to the training image, the reduced-resolution model of the image to be trained, and the image restoration model to be trained;
  • the second iterative training optimization module is configured to perform iterative training and optimization on the reduced-resolution model of the image to be trained and the image restoration model to be trained based on the training image and the real low-resolution image, until the image to be trained
  • the training image reduced-resolution model and the image restoration model to be trained reach a common corresponding preset second iteration training end condition, and the preset image reduced-resolution model and the preset image restoration model are obtained.
  • the second iterative training optimization module includes:
  • the first calculation unit is configured to calculate the training image gradient information of the training image, and reduce the resolution of the training image based on the image-to-be-trained image reduction model to obtain an initial training low-resolution image;
  • the second storage unit is configured to store the training image gradient information in the initial training low-resolution image to obtain an output low-resolution image
  • the second calculation unit is configured to calculate the training loss of the first model based on the output low-resolution image and the real low-resolution image;
  • a restoration unit configured to perform high-resolution restoration on the output low-resolution image based on the image restoration model to be trained to obtain an output high-resolution image
  • the third calculation unit is configured to calculate the training loss of the second model based on the output high-resolution image and the training image;
  • a fourth calculation unit configured to calculate the total model training loss based on the first model training loss and the second model training loss
  • the first determination unit is used to determine whether the total model training loss converges, and if the total model training loss converges, the image to be trained resolution reduction model is used as the preset image resolution reduction model, and The image restoration model to be trained as the preset image restoration model;
  • the second determination unit is configured to, if the total model training loss does not converge, re-execute the iterative training optimization on the to-be-trained image reduction model and the to-be-trained image restoration model until the total model training loss converges .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了一种图像降分辨率及复原方法、设备及可读存储介质，所述图像降分辨率及复原方法包括：获取待处理图像，并确定所述待处理图像对应的图像梯度信息，进而将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像，进而将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像，将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。本申请解决了图像降分辨率后复原精确度低的技术问题。

Description

图像降分辨率及复原方法、设备及可读存储介质
本申请要求于2020年06月22日提交中国专利局、申请号为202010577359.0、发明名称为“图像降分辨率及复原方法、设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在申请中。
技术领域
本申请涉及人工智能领域，尤其涉及一种图像降分辨率及复原方法、设备及可读存储介质。
背景技术
随着计算机软件和人工智能的不断发展，神经网络模型的应用也越来越广泛，目前，在进行高分辨率图像传输或者存储时，由于高分辨率图像的图像位深度较深，占用的存储空间较大，进而需要基于神经网络将高分辨率图像降分辨率为图像位深度较低的低分辨率图像进行存储，进而在提取图像时，再把低分辨率图像复原为高分辨率图像，但是，低分辨率图像在复原为高分辨效率时，往往会无法精确复原高分辨率图像的高频信息，进而导致复原后的图像失真或者与原高分辨率图像相差较大，进而导致降分辨率图像复原的精确度极低。
发明内容
本申请的主要目的在于提供一种图像降分辨率及复原方法、设备及可读存储介质，旨在解决现有技术中图像降分辨率后复原精确度低的技术问题。
为实现上述目的，本申请提供一种图像降分辨率及复原方法，所述图像降分辨率及复原方法应用于图像降分辨率及复原方法设备，所述图像降分辨率及复原方法包括：
获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度 信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
此外，为实现上述目的，本申请还提供一种图像降分辨率及复原方法装置，所述图像降分辨率及复原方法装置为虚拟装置，且所述图像降分辨率及复原方法装置应用于图像降分辨率及复原方法设备，所述图像降分辨率及复原方法装置包括：
确定模块，用于获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
降分辨率模块，用于获将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
存储模块，用于将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
复原模块，用于将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
本申请还提供一种图像降分辨率及复原方法设备，所述图像降分辨率及复原方法设备为实体设备，所述图像降分辨率及复原方法设备包括：存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的所述图像降分辨率及复原方法的程序，所述图像降分辨率及复原方法的程序被处理器执行时可实现如上述的图像降分辨率及复原方法的步骤。
本申请还提供一种可读存储介质，所述可读存储介质上存储有实现图像降分辨率及复原方法的程序，所述图像降分辨率及复原方法的程序被处理器执行时实现如上述的图像降分辨率及复原方法的步骤。
本申请在降分辨率待处理图像之前，先计算所述待处理图像的图像梯度信息，进而在将所述待处理图像降分辨率为低分辨率图像后，将所述图像梯度信息存储于低分辨率图像中，进而在需要复原图像时，将低分辨率图像中的图像梯度信息提取出来，进而基于所述图像梯度信息，对低分辨率图像进行精确的复原，获得目标复原图像，其中，由于图像梯度信息对低分率图像复原为原来的高分辨率图像的过程具有指导作用，进而使得所述目标复原图像更加接近于原来的所述待处理图像，进而使得图像复原过程更加精确，且由于图像梯度信息存储低分辨率图像中，并不会占用额外的存储空间，进而 实现了在不增加额外的存储空间的前提下，对低分辨率图像的精确复原的目的，所以，解决了图像降分辨率后复原精确度低的技术问题。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请图像降分辨率及复原方法第一实施例的流程示意图；
图2为本申请实施例方案涉及的硬件运行环境的设备结构示意图。
本申请目的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
本申请实施例提供一种图像降分辨率及复原方法，在本申请图像降分辨率及复原方法的第一实施例中，参照图1，所述图像降分辨率及复原方法包括：
步骤A10，获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
在本实施例中，需要说明的是，图像梯度信息为待处理图像中的每一像素点对应的梯度值组成的矩阵，待处理图像为需要降分辨率为更低分辨率的图像的更高分辨率的图像，例如，假设更低分辨率的图像为8bit图像，则待处理图像的位深度应大于8bit，例如，10bit等。
具体地，获取待处理图像，并确定待处理图像对应的待处理图像像素矩阵，其中，待处理图像像素矩阵为待处理图像中各像素点对应的像素点坐标组成的矩阵，其中，像素点坐标对应的幅度为像素值，用于表示该像素点的图像区域的亮度，例如，假设像素点坐标为(3，4)，则对应的像素值为5等，进一步地，计算待处理图像像素矩阵中每一像素点对应的水平梯度值和垂直梯度值，进而基于每一像素点对应的水平梯度值和垂直梯度值，确定每 一像素点对应的梯度值，获得待处理图像像素矩阵对应的图像梯度信息矩阵，也即，获得图像梯度信息，例如，假设待处理图像中某一像素点对应的像素点坐标为向量X(i，j)对应的相邻像素点的坐标对应的向量分别为(i+1，j)和(i，j+1)，则水平梯度值
Figure PCTCN2020111771-appb-000001
垂直梯度值为
Figure PCTCN2020111771-appb-000002
进而像素点对应的梯度值为
Figure PCTCN2020111771-appb-000003
步骤A20，将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
在本实施例中，需要说明的是，预设图像降分辨率模型为训练好的用于进行图像降分辨率的神经网络模型。
具体地，将待处理图像输入预设图像降分辨率模型，对待处理图像进行降采样，以生成待处理图像对应的缩略图，获得降采样图像，并基于预设位深度增益因子，对降采样图像对应的每一像素点进行降位深度处理，以将每一像素点对应的像素值映射至预设低分辨率值域，获得初始低分辨率图像，其中，降分辨率过程如下所示：
Y＝round(D R(X)/g)
其中，Y为初始低分辨率图像，X为待处理图像，g为预设位深度增益因子，D R为为降采样方法，round为取整。
其中，所述对所述待处理图像进行降分辨率，获得初始低分辨率图像的步骤包括：
步骤A21，对所述待处理图像进行降采样，获得降采样图像；
在本实施例中，需要说明的是，降采样图像可用降采样矩阵进行表示，其中，降采样矩阵为降采样图像的像素矩阵，降采样的方法包括最邻近采样法、二次插值法、双三次卷积法等。
步骤A22，基于预设位深度增益因子，对所述降采样图像进行降位深度处理，获得所述初始低分辨率图像。
在本实施例中，初始低分辨率图像可用初始低分辨率图像矩阵进行表示，初始低分辨率图像矩阵为初始低分辨率图像的像素矩阵，预设位深度增益因子为基于高分辨率图像的图像位深度和低分辨率图像的图像位深度确定的位深度影响因子，用于将高分辨率图像的图像位深度转换为低分辨率图像的图 像位深度或者将低分辨率图像的图像位深度转换为高分辨率图像的图像位深度，其中，高分辨率图像的分辨率高于低分辨率图像的分辨率，例如，假设高分辨率图像的图像位深度为Nbit，低分辨率图像的图像位深度为Mbit，则预设位深度增益因子g的计算方法如下所示：
Figure PCTCN2020111771-appb-000004
具体地，确定降采样图像对应的降采样矩阵，并将降采样矩阵图像中的每一像素值均除以预设位深度增益因子并取整，获得初始低分辨率图像矩阵。
步骤A30，将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
在本实施例中，需要说明的是，图像梯度信息包括初始梯度图，其中，初始梯度图为由各像素点对应的梯度值的信息图，初始梯度图像对应一初始梯度值矩阵，初始梯度值矩阵为用于存储各像素点对应的梯度值的矩阵。
另外地，需要说明的是，随着图像的位深度的不断加深，图像位中的各b比特位的视觉显著程度不同，其中，视觉显著程度较低的图像位为不显著位，而通常对不显著位上的数值进行调整，通常不会影响图像的视觉效果，例如，假设图像的图像位深度为10bit，则可判定10bit中最末位的2bit位为不显著位。
具体地，将初始低分辨率图像进行左移，获得左移空缺位，并将左移空缺位作为初始低分辨率图像的目标不显著位，进一步地，将初始梯度图进行量化和编码，获得目标梯度图，其中，目标梯度图与目标不显著位相匹配，进而将目标梯度图存储与目标不显著位中，获得目标低分辨率图像，其中，目标低分辨率图像与待处理图像的图像位深度一致，且由于目标明显位对目标低分辨率图像的视觉效果影响极低，进而目标低分辨率图像与待处理图像的视觉效果应当一致，例如，假设待处理图像的图像位深度为10bit，初始低分辨图像的图像为深度为8bit，由于初始低分辨图像为对待处理图像进行降分辨率获得的图像，进而初始低分辨率图像与待处理图像的视觉效果一致，进一步地，将初始低分辨率图像左移2bit，获得2bit的左移空缺位，并将2bit的左移空缺位作为目标不明显位，进一步地，对初始梯度图进行量化和编码，获得2bit的低分辨率图像的目标梯度图，进而将目标梯度图写入目标不明显位中，获得10bit目标低分辨率图像，且由于目标不显著位对图像的视觉效果 影响极低，进而目标低分辨率图像与初始低分辨率图像的视觉效果一致，所以，目标低分辨率图像与待处理图像的视觉效果一致。
进一步地，需要说明的是，本实施例实现了将梯度图存储于图像中的目的，进而为梯度图像的存储节约了存储空间，且在将梯度图存储于图像中时，未影响图像的视觉效果，也即，在实现对图像进行精确的降分辨率及复原同时，节约了存储空间，进而节约的存储成本。
另外的，将图像梯度信息存储于初始低分辨率图像中，获得目标低分辨率图像的过程可用如下公式进行表示：
G＝C(Quant(Grad(X)))
Y *＝bitshift(Y,M)+G
其中，X为待处理图像，
其中，图像梯度信息包括初始梯度图，Grad代表梯度计算，Quant代表量化，C代表编码，G为目标梯度图，Y为初始低分辨率图像，M为进行左移的比特位数值，Y *为目标低分辨率图像，bitshift代表左移。
所述将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像的步骤包括：
步骤A31，对所述初始梯度图进行量化和编码，获得目标梯度图；
在本实施例中，具体地，对初始梯度图进行量化，获得量化梯度图，进而对量化梯度图进行降分辨率编码，以将量化梯度图由高分辨率梯度信息图降分辨率为低分辨率梯度信息图，获得目标梯度图。
其中，所述对所述初始梯度图进行量化和编码，获得目标梯度图的步骤包括：
步骤A311，确定所述待处理图像和所述初始低分辨率图像之间的差异比特位数量，并基于所述差异比特位数量，对所述初始梯度图进行量化，获得量化梯度图；
在本实施例中，具体地，确定待处理图像和初始低分辨率图像之间的差异比特位数量，进而将初始梯度图量化为差异比特位数量对应的量化梯度图，例如，假设待处理图像为10bit图像，初始低分辨率图像为8bit图像，进而量化梯度图为2bit图像。
步骤A312，对所述量化梯度图进行降分辨率编码，获得所述目标梯度图。
在本实施例中，具体地，基于霍夫曼编码方式，对量化梯度图进行降分辨率编码，以将量化梯度图降分辨率为初始低分辨率图像对应的低分辨率梯度信息图，获得目标梯度图。
步骤A32，将所述初始低分辨率图像进行左移，获得目标存储比特位；
在本实施例中，具体地，确定待处理图像和初始低分辨率图像之间的差异比特位数量，进而基于差异比特位数量，将初始低分辨率图像进行左移，获得左移空缺位，将左移空缺位作为目标低分辨率图像对应的目标存储比特位，例如，假设待处理图像为10bit图像，初始低分辨率图像为8bit图像，则将初始低分辨率图像左移2bit，获得2bit的左移空缺比特位，则这2bit的左移空缺比特位即为目标存储比特位。
步骤A33，将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像。
在本实施例中，将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像，具体地，将目标梯度图写入目标存储比特位中，获得目标低分辨率图像。
另外地，需要说明的是，由于目标梯度图的数据长度并非固定值，进而在将目标梯度图写入初始低分辨率图像中，获得目标低分辨率图像时，若目标低分辨率图像中存在剩余存储容量，则对目标低分辨率图像中的剩余存储容量对应的剩余像素点缺失的比特位数值进行补全，其中，补全方法包括理想增益方法等，相同地，若目标低分辨率图像中的存储容量不足，则需要对目标梯度图进行降采样，以降分辨率目标梯度图的数据长度，进而将降分辨率后的目标梯度图写入初始低分辨率图像中。
步骤A40，将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
在本实施例中，需要说明的是，预设图像复原模型为训练好用于进行图像复原的神经网络模型，预设图像复原模型包括梯度位深度扩展网络、图像位深度扩展及上采样网络和图像增强网络，其中，梯度位深度扩展网络用于对梯度图进行解码和重建的神经网络，图像位深度扩展及上采样网络为用于对图像进行图像位深度扩展可上采样的神经网络，图像增强网络为基于梯度 图，对图像的高频细节进行复原的神经网络。
具体地，从目标低分辨图像中提取目标梯度图和初始低分辨率图像，进而将目标梯度图输入梯度位深度扩展网络，对目标梯度图进行图像位深度扩展和解码，获得初始梯度图，并将初始低分辨率图像输入图像位深度扩展及上采样网络，对初始低分辨率图像进行图像位深度扩展和上采样，获得初始复原图像，进而将初始梯度图和初始复原图像输入图像增强网络，以基于初始梯度图，提升图初始复原图像的高频细节，获得目标复原图像。
在一种可实施的方案中，梯度位深度扩展网络包括一个位深度预扩展层、一个3*3的输入层、N1个残差块和3*3的输出层，其中，位深度预扩展层为基于预设位深度增益因子对图像进行初步位深度提升的神经网络层，输入层用于将图像转化为64张特征图，输出层用于将64张特征图重构为输出结果，一个残差块包括一个64通道的扩张卷积层、一个ReLU激活函数层、一个64通道的3*3的卷积层、以及一个连接两端的残差连接，其中，残差块用于对图像进行扩张卷积，以增大图像的感受野，另外地，图像位深度扩展及上采样网络包括一个位深度预扩展层、一个3*3的输入层、N2个的残差块、一个从输入层后到残差块输出后的残差连接、一个上采样层、一个从上采样层后到残差块输出后的残差连接、一个3*3的输出层，其中，上采样层用于对图像进行特征提取与重构，图像增强网络包括一个3*3的输入层、N3个卷积层、一个从输入层后到卷积层后的残差连接和一个3*3的输出层，其中，N1、N2和N3在进行预设图像降分辨率模型和预设图像复原模型的训练时进行确定。
其中，所述预设图像复原模型包括梯度位深度扩展网络、图像位深度扩展及上采样网络和图像增强网络，
所述将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像的步骤包括：
步骤A41，在所述目标低分辨率图像中提取所述初始低分辨率图像和目标梯度图；
在本实施例中，具体地，在目标低分辨率图像中的目标不显著位中提取目标梯度图，并在目标低分辨率图像中的除目标不显著位之外的其他比特位中提取初始低分辨率图像。
步骤A42，将所述目标梯度图输入所述梯度位深度扩展网络，对所述目标梯度图进行解码和重建，获得初始梯度图；
在本实施例中，具体地，将目标梯度图输入梯度位深度扩展网络，对目标梯度图进行解码，获得解码梯度图，进而基于梯度位深度扩展网络的位深度预扩展层，将解码梯度图乘以预设位深度增益因子，以对解码梯度图进行位深度初步提升，获得初步位深度提升梯度图，进而基于梯度位深度扩展网络的输入层，将初步位深度提升梯度图转化为预设数量的梯度特征图，进而基于梯度位深度扩展网络的各残差块，对各梯度特征图进行扩张卷积，获得各扩张卷积特征图，进一步地，基于梯度位深度扩展网络的输出层，对各扩张卷积特征图进行重构，获得初始梯度图。
步骤A43，将所述初始低分辨率图像输入所述图像位深度扩展及上采样网络，对所述初始低分辨率图像进行深度扩展和超分辨率重建，获得初始复原图像；
在本实施例中，具体地，将初始低分辨率图像输入图像位深度扩展及上采样网络，基于图像位深度扩展及上采样网络的位深度预扩展层，将初始低分辨率图像乘以预设位深度增益因子，以对初始低分辨率图像进行位深度初步提升，获得初步位深度提升图像，进一步地，基于图像位深度扩展及上采样网络的输入层、第一残差块、上采样层、第二残差块和输出层，对初步位深度提升图像进行深度扩展和超分辨率重建，获得初始复原图像，其中，第一残差块用于对初步位深度提升图像进行位深度信息扩展，第二残差块用于对初步位深度提升图像进行上采样后的初始位深度提升图像进行特征提取与重构。
步骤A44，将所述初始梯度图和所述初始复原图像输入所述图像增强网络，对所述初始梯度图和所述初始复原图像进行融合重建，获得所述目标复原图像。
在本实施例中，具体地，初始梯度图和初始复原图像输入图像增强网络，以基于初始梯度图，提升初始复原图像相对于待处理图像的高频细节，获得目标复原图像，例如，对于待处理图像中的一目标像素点对应的像素值为355，对图像进行降分辨率后，目标像素点对应的像素值为88，对图像进行初步复原后，目标像素点对应的像素值为353，进而通过图像增强网络，基于初始梯 度图，即可将像素值353精确提升至355，进而提高了图像复原的精确性。
本实施例在降分辨率待处理图像之前，先计算待处理图像的图像梯度信息，进而在将待处理图像降分辨率为低分辨率图像后，将图像梯度信息存储于低分辨率图像中，进而在需要复原图像时，将低分辨率图像中的图像梯度信息提取出来，进而基于图像梯度信息，对低分辨率图像进行精确的复原，获得目标复原图像，其中，由于图像梯度信息对低分率图像复原为原来的高分辨率图像的过程具有指导作用，进而使得目标复原图像更加接近于原来的待处理图像，进而使得图像复原过程更加精确，且由于图像梯度信息存储低分辨率图像中，并不会占用额外的存储空间，进而实现了在不增加额外的存储空间的前提下，对低分辨率图像的精确复原的目的，所以，解决了图像降分辨率后复原精确度低的技术问题。
进一步地，基于本申请中第一实施例，在本申请的另一实施例中，在所述将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像的步骤之前，所述图像降分辨率及复原方法包括：
步骤A10，获取训练图像、所述训练图像对应的真实低分辨率图像、待训练图像降分辨率模型和待训练图像复原模型；
在本实施例中，需要说明的是，待训练图像降分辨率模型和待训练图像复原模型均为未训练好的神经网络模型。
步骤A20，基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得所述预设图像降分辨率模型和所述预设图像复原模型。
在本实施例中，需要说明的是，预设第二迭代训练结束条件包括模型损失收敛、达到最大迭代训练次数等。
具体地，将训练图像输入待训练图像降分辨率模型，对训练图像进行降分辨率，获得训练低分辨率图像，并基于训练低分辨率图像和真实低分辨率图像，计算第一模型训练损失，进一步地，将训练图像对应的训练梯度图存储于训练低分辨率图像，获得目标输入低分辨率图像，并将目标低分辨率图 像输入待训练图像复原模型，对目标低分辨率图像进行高分辨率复原，获得训练高分辨率图像，并基于训练图像和训练高分辨率图像，计算第二模型训练损失，进而基于第一模型训练损失和第二模型训练损失，计算模型总损失，进一步地，判断模型总损失是否收敛，若模型总损失收敛，则将待训练图像降分辨率模型作为预设图像降分辨率模型，将待训练图像复原模型作为预设图像复原模型，若模型总损失未收敛，则重新对待训练图像降分辨率模型和待训练图像复原模型进行迭代训练优化，直至模型总损失收敛。
其中，所述预设第二迭代训练结束条件包括所述待训练图像降分辨率模型和所述待训练图像复原模型共同对应的总模型训练损失收敛，
所述基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得所述预设图像降分辨率模型和所述预设图像复原模型的步骤包括：
步骤A21，计算所述训练图像的训练图像梯度信息，并基于所述待训练图像降分辨率模型，对所述训练图像进行降分辨率，获得初始训练低分辨率图像；
在本实施例中，具体地，计算训练图像的训练图像梯度信息，并将训练图像输入待训练图像降分辨率模型，对训练进行降采样，以生成训练图像对应的缩略图，获得训练降采样图像，并基于预设位深度增益因子，对训练降采样图像对应的每一像素点进行降位深度处理，以将训练图像的每一像素点对应的像素值映射至预设低分辨率值域，获得初始训练低分辨率图像。
步骤A22，将所述训练图像梯度信息存储于所述初始训练低分辨率图像，获得输出低分辨率图像；
在本实施例中，具体地，将初始训练低分辨率图像进行左移，获得训练左移空缺位，并将训练左移空缺位作为初始训练低分辨率图像的训练目标不显著位，进而将训练图像梯度信息写入训练目标不显著位，获得输出低分辨率图像。
步骤A23，基于所述输出低分辨率图像和所述真实低分辨率图像，计算第一模型训练损失；
在本实施例中，具体地，求取输出低分辨率图像和真实低分辨率图像的第一距离，并将第一距离作为第一模型训练损失。
步骤A24，基于所述待训练图像复原模型，对所述输出低分辨率图像进行高分辨率复原，获得输出高分辨率图像；
在本实施例中，具体地，在输出低分辨率图像中提取训练图像梯度信息和初始训练低分辨率图像，进而对训练图像梯度信息进行图像位深度扩展和解码，获得初始训练梯度图，并对初始训练低分辨率图像进行图像位深度扩展和上采样，获得训练初始复原图像，进而基于初始训练梯度图，提升训练初始复原图像的高频细节，获得输出高分辨率图像。
步骤A25，基于所述输出高分辨率图像和所述训练图像，计算第二模型训练损失；
在本实施例中，具体地，计算输出高分辨率图像和训练图像之间的第二距离，并将第二距离作为第二模型训练损失。
步骤A26，基于所述第一模型训练损失和所述第二模型训练损失，计算所述总模型训练损失；
在本实施例中，具体地，基于预设权重，对第一模型训练损失和第二模型训练损失进行加权求和，获得总模型训练损失，其中，总模型训练损失的计算过程如下所示：
L＝λL 1+(1-λ)L 2
其中，λ为预设权重，L 1为第一模型训练损失，L 2为第二模型训练损失。
步骤A27，确定所述总模型训练损失是否收敛，若所述总模型训练损失收敛，则将所述待训练图像降分辨率模型作为所述预设图像降分辨率模型，并将所述待训练图像复原模型作为所述预设图像复原模型；
步骤A28，若所述总模型训练损失未收敛，则重新对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述总模型训练损失收敛。
在本实施例中，具体地，若总模型训练损失未收敛，则重新获取训练图像和真实低分辨率图像，以重新对待训练图像降分辨率模型和待训练图像复原模型进行迭代训练优化，直至总模型训练损失收敛。
本实施例通过获取训练图像、训练图像对应的真实低分辨率图像、待训 练图像降分辨率模型和待训练图像复原模型，进而基于训练图像和真实低分辨率图像，对待训练图像降分辨率模型和待训练图像复原模型进行迭代训练优化，直至待训练图像降分辨率模型和待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得预设图像降分辨率模型和预设图像复原模型。也即，本申请提供了一种端到端训练预设图像降分辨率模型和预设图像复原模型的方法，进而基于训练好的预设图像降分辨率模型和预设图像复原模型，可实现对图像的降分辨率和高分辨率复原，进而为实现在不增加额外的存储空间的前提下，对降分辨率图像的精确复原的目的奠定了基础，进而为解决图像降分辨率后复原精确度低的技术问题奠定了基础。
进一步地，基于本申请中第一实施例和第二实施例，在本申请的另一实施例中，所述预设图像复原模型包括梯度位深度扩展网络，
在所述将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像的步骤之前，所述图像降分辨率及复原方法还包括：
步骤B10，获取待训练网络和训练梯度信息数据以及所述训练梯度信息数据对应的的真实梯度信息数据；
在本实施例中，需要说明的是，待训练网络为未训练好的神经网络，训练梯度信息数据为对训练图像的初始训练梯度图进行量化和编码确定的目标训练梯度图，真实梯度信息数据为训练图像对应的真实梯度图。
步骤B20，基于所述训练梯度信息数据和所述真实梯度信息数据，对所述待训练网络进行迭代训练优化，直至所述待训练网络达到预设第一迭代训练结束条件，获得所述梯度位深度扩展网络。
在本实施例中，需要说明的是，预设第一迭代训练结束条件包括模型损失收敛、达到最大迭代次数阀值等。
具体地，将目标训练梯度图输入待训练网络，对目标训练梯度图进行解码，获得训练解码梯度图，进而基于预设位深度增益因子，对训练解码梯度图进行图像位初步提升，获得训练图像位提升梯度图，进而对训练图像位梯度图进行扩张卷积，获得训练输出梯度图，进而计算训练输出梯度图和真实梯度图之间的第三距离，并将第三距离作为第三模型训练损失，并判断第三模型训练损失是否收敛，若第三模型训练损失收敛，则将待训练网络作为梯 度位深度扩展网络，若第三模型训练损失未收敛，则重新对待训练网络进行迭代训练优化，直至第三模型训练损失收敛。
本实施例通过获取待训练网络和训练梯度信息数据以及训练梯度信息数据对应的的真实梯度信息数据，进而基于训练梯度信息数据和真实梯度信息数据，对待训练网络进行迭代训练优化，直至待训练网络达到预设第一迭代训练结束条件，获得梯度位深度扩展网络。也即，本实施例提供了一种梯度位深度扩展网络的训练方法，进而基于梯度位深度扩展网络，可将存储于初始低分辨率图像中的目标梯度图重构为初始梯度图，进而基于初始梯度图，实现对目标低分辨率图像的高分辨率复原，也即，为进行对目标低分辨率图像的高分辨率复原奠定了基础，进而为解决图像降分辨率后复原精确度低的技术问题奠定了基础。
参照图2，图2是本申请实施例方案涉及的硬件运行环境的设备结构示意图。
如图2所示，该图像降分辨率及复原方法设备可以包括：处理器1001，例如CPU，存储器1005，通信总线1002。其中，通信总线1002用于实现处理器1001和存储器1005之间的连接通信。存储器1005可以是高速RAM存储器，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储设备。
可选地，该图像降分辨率及复原方法设备还可以包括矩形用户接口、网络接口、摄像头、RF(Radio Frequency，射频)电路，传感器、音频电路、WiFi模块等等。矩形用户接口可以包括显示屏(Display)、输入子模块比如键盘(Keyboard)，可选矩形用户接口还可以包括标准的有线接口、无线接口。网络接口可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。
本领域技术人员可以理解，图2中示出的图像降分辨率及复原方法设备结构并不构成对图像降分辨率及复原方法设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图2所示，作为一种计算机存储介质的存储器1005中可以包括操作系统、网络通信模块以及图像降分辨率及复原方法程序。操作系统是管理和控制图像降分辨率及复原方法设备硬件和软件资源的程序，支持图像降分辨率及复原方法程序以及其它软件和/或程序的运行。网络通信模块用于实现存储 器1005内部各组件之间的通信，以及与图像降分辨率及复原方法系统中其它硬件和软件之间通信。
在图2所示的图像降分辨率及复原方法设备中，处理器1001用于执行存储器1005中存储的图像降分辨率及复原方法程序，实现上述任一项所述的图像降分辨率及复原方法的步骤。
本申请图像降分辨率及复原方法设备具体实施方式与上述图像降分辨率及复原方法各实施例基本相同，在此不再赘述。
本申请实施例还提供一种图像降分辨率及复原方法装置，所述图像降分辨率及复原方法装置应用于图像降分辨率及复原方法设备，所述图像降分辨率及复原方法装置包括：
确定模块，用于获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
降分辨率模块，用于获将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
存储模块，用于将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
复原模块，用于将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
可选地，所述存储模块包括：
量化和编码单元，用于对所述初始梯度图进行量化和编码，获得目标梯度图；
左移单元，用于将所述初始低分辨率图像进行左移，获得目标存储比特位；
第一存储单元，用于将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像。
可选地，所述量化和编码单元包括：
量化子单元，用于确定所述待处理图像和所述初始低分辨率图像之间的差异比特位数量，并基于所述差异比特位数量，对所述初始梯度图进行量化，获得量化梯度图；
编码子单元，用于对所述量化梯度图进行降分辨率编码，获得所述目标梯度图。
可选地，所述复原模块包括：
提取单元，用于在所述目标低分辨率图像中提取所述初始低分辨率图像和目标梯度图；
解码和重建单元，用于将所述目标梯度图输入所述梯度位深度扩展网络，对所述目标梯度图进行解码和重建，获得初始梯度图；
深度扩展和重建单元，用于将所述初始低分辨率图像输入所述图像位深度扩展及上采样网络，对所述初始低分辨率图像进行深度扩展和超分辨率重建，获得初始复原图像；
融合重建单元，用于将所述初始梯度图和所述初始复原图像输入所述图像增强网络，对所述初始梯度图和所述初始复原图像进行融合重建，获得所述目标复原图像。
可选地，所述降分辨率模块包括：
降采样单元，用于对所述待处理图像进行降采样，获得降采样图像；
降位深度处理单元，用于基于预设位深度增益因子，对所述降采样图像进行降位深度处理，获得所述初始低分辨率图像。
可选地，所述图像降分辨率及复原装置还包括：
第一获取模块，用于获取待训练网络和训练梯度信息数据以及所述训练梯度信息数据对应的的真实梯度信息数据；
第一迭代训练优化模块，用于基于所述训练梯度信息数据和所述真实梯度信息数据，对所述待训练网络进行迭代训练优化，直至所述待训练网络达到预设第一迭代训练结束条件，获得所述梯度位深度扩展网络。
可选地，所述图像降分辨率及复原装置还包括：
第二获取模块，用于获取训练图像、所述训练图像对应的真实低分辨率图像、待训练图像降分辨率模型和待训练图像复原模型；
第二迭代训练优化模块，用于基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得所述预设图像降分辨率模型和 所述预设图像复原模型。
可选地，所述第二迭代训练优化模块包括：
第一计算单元，用于计算所述训练图像的训练图像梯度信息，并基于所述待训练图像降分辨率模型，对所述训练图像进行降分辨率，获得初始训练低分辨率图像；
第二存储单元，用于将所述训练图像梯度信息存储于所述初始训练低分辨率图像，获得输出低分辨率图像；
第二计算单元，用于基于所述输出低分辨率图像和所述真实低分辨率图像，计算第一模型训练损失；
复原单元，用于基于所述待训练图像复原模型，对所述输出低分辨率图像进行高分辨率复原，获得输出高分辨率图像；
第三计算单元，用于基于所述输出高分辨率图像和所述训练图像，计算第二模型训练损失；
第四计算单元，用于基于所述第一模型训练损失和所述第二模型训练损失，计算所述总模型训练损失；
第一判定单元，用于确定所述总模型训练损失是否收敛，若所述总模型训练损失收敛，则将所述待训练图像降分辨率模型作为所述预设图像降分辨率模型，并将所述待训练图像复原模型作为所述预设图像复原模型；
第二判定单元，用于若所述总模型训练损失未收敛，则重新对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述总模型训练损失收敛。
本申请图像降分辨率及复原方法装置的具体实施方式与上述图像降分辨率及复原方法各实施例基本相同，在此不再赘述。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利处理范围内。

Claims (20)

  1. 一种图像降分辨率及复原方法，其特征在于，所述图像降分辨率及复原方法包括：
    获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
    将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
    将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
    将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
  2. 如权利要求1所述图像降分辨率及复原方法，其特征在于，所述图像梯度信息包括初始梯度图，
    所述将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像的步骤包括：
    对所述初始梯度图进行量化和编码，获得目标梯度图；
    将所述初始低分辨率图像进行左移，获得目标存储比特位；
    将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像。
  3. 如权利要求2所述图像降分辨率及复原方法，其特征在于，所述对所述初始梯度图进行量化和编码，获得目标梯度图的步骤包括：
    确定所述待处理图像和所述初始低分辨率图像之间的差异比特位数量，并基于所述差异比特位数量，对所述初始梯度图进行量化，获得量化梯度图；
    对所述量化梯度图进行降分辨率编码，获得所述目标梯度图。
  4. 如权利要求1所述图像降分辨率及复原方法，其特征在于，所述预设图像复原模型包括梯度位深度扩展网络、图像位深度扩展及上采样网络和图像增强网络，
    所述将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像的步骤包括：
    在所述目标低分辨率图像中提取所述初始低分辨率图像和目标梯度图；
    将所述目标梯度图输入所述梯度位深度扩展网络，对所述目标梯度图进行解码和重建，获得初始梯度图；
    将所述初始低分辨率图像输入所述图像位深度扩展及上采样网络，对所述初始低分辨率图像进行深度扩展和超分辨率重建，获得初始复原图像；
    将所述初始梯度图和所述初始复原图像输入所述图像增强网络，对所述初始梯度图和所述初始复原图像进行融合重建，获得所述目标复原图像。
  5. 如权利要求1所述图像降分辨率及复原方法，其特征在于，所述对所述待处理图像进行降分辨率，获得初始低分辨率图像的步骤包括：
    对所述待处理图像进行降采样，获得降采样图像；
    基于预设位深度增益因子，对所述降采样图像进行降位深度处理，获得所述初始低分辨率图像。
  6. 如权利要求1所述图像降分辨率及复原方法，其特征在于，所述预设图像复原模型包括梯度位深度扩展网络，
    在所述将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像的步骤之前，所述图像降分辨率及复原方法还包括：
    获取待训练网络和训练梯度信息数据以及所述训练梯度信息数据对应的的真实梯度信息数据；
    基于所述训练梯度信息数据和所述真实梯度信息数据，对所述待训练网络进行迭代训练优化，直至所述待训练网络达到预设第一迭代训练结束条件，获得所述梯度位深度扩展网络。
  7. 如权利要求1所述图像降分辨率及复原方法，其特征在于，在所述将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像的步骤之前，所述图像降分辨率及复原方法包括：
    获取训练图像、所述训练图像对应的真实低分辨率图像、待训练图像降分辨率模型和待训练图像复原模型；
    基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练 结束条件，获得所述预设图像降分辨率模型和所述预设图像复原模型。
  8. 如权利要求7所述图像降分辨率及复原方法，其特征在于，所述预设第二迭代训练结束条件包括所述待训练图像降分辨率模型和所述待训练图像复原模型共同对应的总模型训练损失收敛，
    所述基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得所述预设图像降分辨率模型和所述预设图像复原模型的步骤包括：
    计算所述训练图像的训练图像梯度信息，并基于所述待训练图像降分辨率模型，对所述训练图像进行降分辨率，获得初始训练低分辨率图像；
    将所述训练图像梯度信息存储于所述初始训练低分辨率图像，获得输出低分辨率图像；
    基于所述输出低分辨率图像和所述真实低分辨率图像，计算第一模型训练损失；
    基于所述待训练图像复原模型，对所述输出低分辨率图像进行高分辨率复原，获得输出高分辨率图像；
    基于所述输出高分辨率图像和所述训练图像，计算第二模型训练损失；
    基于所述第一模型训练损失和所述第二模型训练损失，计算所述总模型训练损失；
    确定所述总模型训练损失是否收敛，若所述总模型训练损失收敛，则将所述待训练图像降分辨率模型作为所述预设图像降分辨率模型，并将所述待训练图像复原模型作为所述预设图像复原模型；
    若所述总模型训练损失未收敛，则重新对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述总模型训练损失收敛。
  9. 一种图像降分辨率及复原方法装置，其特征在于，所述图像降分辨率及复原方法装置包括：
    确定模块，用于获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
    降分辨率模块，用于获将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
    存储模块，用于将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
    复原模块，用于将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
  10. 如权利要求9所述图像降分辨率及复原方法装置，其特征在于，所述存储模块包括：
    量化和编码单元，用于对所述初始梯度图进行量化和编码，获得目标梯度图；
    左移单元，用于将所述初始低分辨率图像进行左移，获得目标存储比特位；
    第一存储单元，用于将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像。
  11. 如权利要求10所述图像降分辨率及复原方法装置，其特征在于，所述量化和编码单元包括：
    量化子单元，用于确定所述待处理图像和所述初始低分辨率图像之间的差异比特位数量，并基于所述差异比特位数量，对所述初始梯度图进行量化，获得量化梯度图；
    编码子单元，用于对所述量化梯度图进行降分辨率编码，获得所述目标梯度图。
  12. 如权利要求9所述图像降分辨率及复原方法装置，其特征在于，所述复原模块包括：
    提取单元，用于在所述目标低分辨率图像中提取所述初始低分辨率图像和目标梯度图；
    解码和重建单元，用于将所述目标梯度图输入所述梯度位深度扩展网络，对所述目标梯度图进行解码和重建，获得初始梯度图；
    深度扩展和重建单元，用于将所述初始低分辨率图像输入所述图像位深度扩展及上采样网络，对所述初始低分辨率图像进行深度扩展和超分辨率重 建，获得初始复原图像；
    融合重建单元，用于将所述初始梯度图和所述初始复原图像输入所述图像增强网络，对所述初始梯度图和所述初始复原图像进行融合重建，获得所述目标复原图像。
  13. 如权利要求9所述图像降分辨率及复原方法装置，其特征在于，所述降分辨率模块包括：
    降采样单元，用于对所述待处理图像进行降采样，获得降采样图像；
    降位深度处理单元，用于基于预设位深度增益因子，对所述降采样图像进行降位深度处理，获得所述初始低分辨率图像。
  14. 如权利要求9所述图像降分辨率及复原方法装置，其特征在于，所述图像降分辨率及复原装置还包括：
    第一获取模块，用于获取待训练网络和训练梯度信息数据以及所述训练梯度信息数据对应的的真实梯度信息数据；
    第一迭代训练优化模块，用于基于所述训练梯度信息数据和所述真实梯度信息数据，对所述待训练网络进行迭代训练优化，直至所述待训练网络达到预设第一迭代训练结束条件，获得所述梯度位深度扩展网络。
  15. 如权利要求9所述图像降分辨率及复原方法装置，其特征在于，所述图像降分辨率及复原装置还包括：
    第二获取模块，用于获取训练图像、所述训练图像对应的真实低分辨率图像、待训练图像降分辨率模型和待训练图像复原模型；
    第二迭代训练优化模块，用于基于所述训练图像和所述真实低分辨率图像，对所述待训练图像降分辨率模型和所述待训练图像复原模型进行迭代训练优化，直至所述待训练图像降分辨率模型和所述待训练图像复原模型达到共同对应的预设第二迭代训练结束条件，获得所述预设图像降分辨率模型和所述预设图像复原模型。
  16. 一种图像降分辨率及复原设备，其中，所述图像降分辨率及复原设备包括存储器、处理器和存储在所述存储器上并可在所述处理器上运行的图像降分辨率及复原程序，所述图像降分辨率及复原程序被所述处理器执行时实现如下步骤：
    获取待处理图像，并确定所述待处理图像对应的图像梯度信息；
    将所述待处理图像输入预设图像降分辨率模型，对所述待处理图像进行降分辨率，获得初始低分辨率图像；
    将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像；
    将所述目标低分辨率图像输入预设图像复原模型，以基于所述图像梯度信息，对所述目标低分辨率图像进行高分辨率复原，获得目标复原图像。
  17. 如权利要求16所述图像降分辨率及复原设备，其特征在于，所述图像梯度信息包括初始梯度图，
    所述将所述图像梯度信息存储于所述初始低分辨率图像中，获得目标低分辨率图像的步骤包括：
    对所述初始梯度图进行量化和编码，获得目标梯度图；
    将所述初始低分辨率图像进行左移，获得目标存储比特位；
    将所述目标梯度图存储于所述目标存储比特位中，获得所述目标低分辨率图像。
  18. 如权利要求17所述图像降分辨率及复原设备，其特征在于，所述对所述初始梯度图进行量化和编码，获得目标梯度图的步骤包括：
    确定所述待处理图像和所述初始低分辨率图像之间的差异比特位数量，并基于所述差异比特位数量，对所述初始梯度图进行量化，获得量化梯度图；
    对所述量化梯度图进行降分辨率编码，获得所述目标梯度图。
  19. 如权利要求16所述图像降分辨率及复原设备，其特征在于，所述对所述待处理图像进行降分辨率，获得初始低分辨率图像的步骤包括：
    对所述待处理图像进行降采样，获得降采样图像；
    基于预设位深度增益因子，对所述降采样图像进行降位深度处理，获得所述初始低分辨率图像。
  20. 一种可读存储介质，其特征在于，所述可读存储介质上存储有实现图像降分辨率及复原方法的程序，所述实现图像降分辨率及复原方法的程序被处理器执行以实现如权利要求1至8中任一项所述图像降分辨率及复原方法的步骤。
PCT/CN2020/111771 2020-06-22 2020-08-27 图像降分辨率及复原方法、设备及可读存储介质 WO2021258529A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010577359.0 2020-06-22
CN202010577359.0A CN111754405B (zh) 2020-06-22 2020-06-22 图像降分辨率及复原方法、设备及可读存储介质

Publications (1)

Publication Number Publication Date
WO2021258529A1 true WO2021258529A1 (zh) 2021-12-30

Family

ID=72676421

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/111771 WO2021258529A1 (zh) 2020-06-22 2020-08-27 图像降分辨率及复原方法、设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN111754405B (zh)
WO (1) WO2021258529A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114972043A (zh) * 2022-08-03 2022-08-30 江西财经大学 基于联合三边特征滤波的图像超分辨率重建方法与系统
CN116029907A (zh) * 2023-02-14 2023-04-28 江汉大学 一种图像降分辨率模型的处理方法、装置以及处理设备
CN117425013A (zh) * 2023-12-19 2024-01-19 杭州靖安防务科技有限公司 一种基于可逆架构的视频传输方法和系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988311A (zh) * 2021-10-14 2023-04-18 荣耀终端有限公司 图像处理方法与电子设备
CN115631115B (zh) * 2022-12-08 2023-03-28 中国科学院自动化研究所 基于递归Transformer的动态图像复原方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204489A (zh) * 2016-07-12 2016-12-07 四川大学 结合深度学习与梯度转换的单幅图像超分辨率重建方法
CN106600553A (zh) * 2016-12-15 2017-04-26 华中科技大学 一种基于卷积神经网络的dem超分辨率方法
CN108133456A (zh) * 2016-11-30 2018-06-08 京东方科技集团股份有限公司 人脸超分辨率重建方法、重建设备以及计算机系统
CN108765343A (zh) * 2018-05-29 2018-11-06 Oppo（重庆）智能科技有限公司 图像处理的方法、装置、终端及计算机可读存储介质
US20190304063A1 (en) * 2018-03-29 2019-10-03 Mitsubishi Electric Research Laboratories, Inc. System and Method for Learning-Based Image Super-Resolution
CN110660021A (zh) * 2019-08-16 2020-01-07 西安理工大学 一种基于深度学习的脉冲涡流热成像高分辨率清晰化方法
CN110674862A (zh) * 2019-09-12 2020-01-10 浙江师范大学 基于内部样例的邻域回归的超分辨率方法
CN110956671A (zh) * 2019-12-12 2020-04-03 电子科技大学 一种基于多尺度特征编码的图像压缩方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5705391B1 (ja) * 2014-06-24 2015-04-22 三菱電機株式会社 画像処理装置及び画像処理方法
CN109903221B (zh) * 2018-04-04 2023-08-22 华为技术有限公司 图像超分方法及装置
CN111179177B (zh) * 2019-12-31 2024-03-26 深圳市联合视觉创新科技有限公司 图像重建模型训练方法、图像重建方法、设备及介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106204489A (zh) * 2016-07-12 2016-12-07 四川大学 结合深度学习与梯度转换的单幅图像超分辨率重建方法
CN108133456A (zh) * 2016-11-30 2018-06-08 京东方科技集团股份有限公司 人脸超分辨率重建方法、重建设备以及计算机系统
CN106600553A (zh) * 2016-12-15 2017-04-26 华中科技大学 一种基于卷积神经网络的dem超分辨率方法
US20190304063A1 (en) * 2018-03-29 2019-10-03 Mitsubishi Electric Research Laboratories, Inc. System and Method for Learning-Based Image Super-Resolution
CN108765343A (zh) * 2018-05-29 2018-11-06 Oppo（重庆）智能科技有限公司 图像处理的方法、装置、终端及计算机可读存储介质
CN110660021A (zh) * 2019-08-16 2020-01-07 西安理工大学 一种基于深度学习的脉冲涡流热成像高分辨率清晰化方法
CN110674862A (zh) * 2019-09-12 2020-01-10 浙江师范大学 基于内部样例的邻域回归的超分辨率方法
CN110956671A (zh) * 2019-12-12 2020-04-03 电子科技大学 一种基于多尺度特征编码的图像压缩方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114972043A (zh) * 2022-08-03 2022-08-30 江西财经大学 基于联合三边特征滤波的图像超分辨率重建方法与系统
CN114972043B (zh) * 2022-08-03 2022-10-25 江西财经大学 基于联合三边特征滤波的图像超分辨率重建方法与系统
CN116029907A (zh) * 2023-02-14 2023-04-28 江汉大学 一种图像降分辨率模型的处理方法、装置以及处理设备
CN116029907B (zh) * 2023-02-14 2023-08-08 江汉大学 一种图像降分辨率模型的处理方法、装置以及处理设备
CN117425013A (zh) * 2023-12-19 2024-01-19 杭州靖安防务科技有限公司 一种基于可逆架构的视频传输方法和系统
CN117425013B (zh) * 2023-12-19 2024-04-02 杭州靖安防务科技有限公司 一种基于可逆架构的视频传输方法和系统

Also Published As

Publication number Publication date
CN111754405B (zh) 2023-08-08
CN111754405A (zh) 2020-10-09

Similar Documents

Publication Publication Date Title
WO2021258529A1 (zh) 图像降分辨率及复原方法、设备及可读存储介质
CN108022212B (zh) 高分辨率图片生成方法、生成装置及存储介质
WO2019184823A1 (zh) 基于卷积神经网络模型的图像处理方法和装置
Jo et al. Practical single-image super-resolution using look-up table
CN111145090A (zh) 一种点云属性编码方法、解码方法、编码设备及解码设备
JP5076150B2 (ja) 画像符号化装置，画像符号化方法および画像符号化プログラム
US20130129246A1 (en) Memory Compression Technique with Low Latency per Pixel
CN112561792B (zh) 图像风格迁移方法、装置、电子设备及存储介质
CN105850129A (zh) 对高动态范围图像进行色调映射的方法及设备
CN104899835A (zh) 基于盲模糊估计与锚定空间映射的图像超分辨处理方法
CN113554058A (zh) 视觉目标图像分辨率增强的方法、系统、装置及存储介质
CN114331831A (zh) 一种轻量化的单张图像超分辨率重建方法
CN112699844A (zh) 一种基于多尺度残差层级密连接网络的图像超分辨率方法
CN113888410A (zh) 图像超分辨率方法、装置、设备、存储介质以及程序产品
Xing et al. Scale-arbitrary invertible image downscaling
CN112637604B (zh) 低时延视频压缩方法及装置
WO2019090876A1 (zh) 一种基于直线扩展收缩模式的图像缩放方法
US6501862B1 (en) Picture information processing method and apparatus and entertainment apparatus
CN100518323C (zh) 一种使用旋转压缩码书对图像进行匹配压缩的方法
CN113344786B (zh) 基于几何生成模型的视频转码方法、装置、介质和设备
CN110211037A (zh) 一种基于多级稀疏字典学习的图像超分辨率方法
CN115294222A (zh) 图像编码方法及图像处理方法、终端及介质
CN113096019B (zh) 图像重建方法、装置、图像处理设备及存储介质
US9911178B2 (en) System and method for content-adaptive super-resolution via cross-scale self-learning
CN114155152A (zh) 一种基于历史特征融合的实时超分辨率重建方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20942337

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20942337

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20942337

Country of ref document: EP

Kind code of ref document: A1