WO2023103715A1 - 图像处理方法、装置及电子设备 - Google Patents

图像处理方法、装置及电子设备 Download PDF

Info

Publication number
WO2023103715A1
WO2023103715A1 PCT/CN2022/131594 CN2022131594W WO2023103715A1 WO 2023103715 A1 WO2023103715 A1 WO 2023103715A1 CN 2022131594 W CN2022131594 W CN 2022131594W WO 2023103715 A1 WO2023103715 A1 WO 2023103715A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
texture
regions
enhanced
reconstructed
Prior art date
Application number
PCT/CN2022/131594
Other languages
English (en)
French (fr)
Inventor
孙宇乐
陈焕浜
杨海涛
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023103715A1 publication Critical patent/WO2023103715A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/41—Analysis of texture based on statistical description of texture
    • G06T7/45—Analysis of texture based on statistical description of texture using co-occurrence matrix computation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the embodiments of the present application relate to the technical field of image processing, and in particular, to an image processing method, device, and electronic equipment.
  • the video is compressed before the video is transmitted; wherein, the greater the video compression rate, the smaller the data volume of the compressed video.
  • the compression rate increases, visible visual distortion will appear in the video; in order to reduce the visual distortion of the reconstructed image, the reconstructed image can be post-processed to improve the quality while the bit rate remains the same.
  • the post-processing of the reconstructed image can significantly improve the subjective quality of the reconstructed image, it is easy to generate false texture in the non-textured area, which affects the subjective quality of the image.
  • the present application provides an image processing method, device and electronic equipment.
  • an embodiment of the present application provides an image processing method, the method includes: first, acquiring a reconstructed image; then, performing image enhancement on the reconstructed image to obtain an intermediate image. Next, determine the texture complexity weights corresponding to each region in the intermediate image, and the texture complexity weight is a number between 0 and 1; then, according to the texture complexity weights corresponding to each region in the intermediate image, each The texture intensity corresponding to the region is respectively attenuated to obtain the enhanced image corresponding to the reconstructed image.
  • the texture of the textured area in the intermediate image can be preserved, and the texture in the non-textured area can be attenuated, so as to enhance the texture of the textured area in the reconstructed image while avoiding false textures in the non-textured area, thereby reducing the texture of the reconstructed image.
  • Visual distortion increases the realism of the image and improves the subjective quality of the image.
  • the present application controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • the texture complexity weight is proportional to the texture complexity, that is, the greater the texture complexity, the greater the texture complexity weight. In this way, the texture effect of the textured area in the intermediate image can be kept as much as possible, and for the non-textured area, attenuation is performed according to the texture complexity.
  • the intermediate image is a texture enhanced image or a residual image.
  • the resolutions of the intermediate image and the enhanced image are the same as that of the reconstructed image.
  • GANEF Geneative Adversarial Network Enhancement Filter, Generative Adversarial Network Enhancement Filter
  • GANEF Geneative Adversarial Network Enhancement Filter, Generative Adversarial Network Enhancement Filter
  • the image output by GANEF may be a texture enhanced image, or a residual image.
  • the enhanced image has texture-enhanced attenuation for each area in the intermediate image, only a partial attenuation of the texture intensity is performed on the intermediate image. Compared with the reconstructed image, the texture intensity of some areas of the enhanced image is still is greater than that of the reconstructed image; that is, the enhanced image still has the effect of texture enhancement.
  • the intermediate image is a residual image; according to the texture complexity weights corresponding to each area in the intermediate image, the texture intensity corresponding to each area in the intermediate image is respectively attenuated to obtain an enhanced image corresponding to the reconstructed image, including: The pixel value of each pixel in the residual image is multiplied by the texture complexity weight corresponding to the area to which each pixel belongs to obtain a residual update image; an enhanced image is generated according to the residual update image and reconstructed image.
  • the residual image is obtained by subtracting the texture enhanced image from the reconstructed image.
  • generating an enhanced image according to the residual update image and the reconstructed image includes: adding the residual update image and the reconstructed image to obtain the enhanced image.
  • pixel-by-pixel addition may be performed on the residual update image and the reconstructed image to obtain an enhanced image.
  • the intermediate image is a texture-enhanced image
  • the method further includes: performing image fidelity on the reconstructed image to obtain a basic fidelity image.
  • the base fidelity image has the same resolution as the reconstructed image.
  • non-GANEF may be used to perform image anti-aliasing on the reconstructed image to obtain a basic anti-aliasing image.
  • determining the texture complexity weight corresponding to each region in the intermediate image includes: dividing the basic fidelity image and the texture enhanced image into two parts according to the preset partition rules N regions, where N is a positive integer; respectively determine the texture complexity of the N regions in the basic fidelity image; based on the texture complexity of the N regions in the basic fidelity image, determine the texture corresponding to the N regions in the texture enhanced image Complexity weight.
  • the texture intensity corresponding to each region in the intermediate image is respectively attenuated, and the corresponding texture intensity of the reconstructed image is obtained.
  • Enhancing the image including: according to the texture complexity weights corresponding to the N regions in the texture enhanced image, determining the weighted calculation weights corresponding to the N regions in the texture enhanced image and the weighted calculation weights corresponding to the N regions in the basic fidelity image; According to the weighted calculation weights corresponding to the N areas in the texture enhanced image and the weighted calculation weights corresponding to the N areas in the basic fidelity image, perform weighted calculations on the N areas in the texture enhanced image and the N areas in the basic fidelity image , to obtain the enhanced image corresponding to the reconstructed image.
  • the texture enhancement The N regions in the image and the N regions in the basic fidelity image are weighted and calculated to obtain the enhanced image corresponding to the reconstructed image, including: the weighted calculation weights corresponding to the N regions in the texture enhanced image and the N regions in the texture enhanced image The regions are multiplied to obtain the first product; the weighted calculation weights corresponding to the N regions in the basic fidelity image are respectively multiplied by the N regions in the basic fidelity image to obtain the second product; the first product and the first product The two products are added to obtain the enhanced image corresponding to the reconstructed image.
  • the texture complexity weights corresponding to the N regions in the texture enhanced image determine the weighted calculation weights and basic preservation weights corresponding to the N regions in the texture enhanced image respectively.
  • the weighted calculation weights corresponding to the N regions in the real image include: determining the texture complexity weights corresponding to the N regions in the texture enhanced image as the weighted calculation weights corresponding to the N regions in the texture enhanced image; Differences of texture complexity weights corresponding to N regions in the texture-enhanced image are determined as weighted calculation weights corresponding to N regions in the basic fidelity image.
  • determining the texture complexity weights corresponding to each region in the intermediate image includes: decoding the corresponding texture complexity weights of the N regions in the intermediate image from the received code stream Texture complexity weight, N is a positive integer. In this way, the accuracy of determining the texture intensity weights corresponding to each region in the intermediate image can be improved, and further, the attenuation of the texture intensity of the image can be controlled more accurately, thereby further improving the image quality.
  • determining the texture complexity weights corresponding to each region in the intermediate image includes: dividing the reconstructed image and the intermediate image into N parts according to the preset partition rules region, N is a positive integer; respectively determine the texture complexity of the N regions in the reconstructed image; based on the texture complexity of the N regions in the reconstructed image, determine the texture complexity weights corresponding to the N regions in the intermediate image.
  • an image processing device which includes:
  • An image acquisition module configured to acquire a reconstructed image
  • An image enhancement module is used to perform image enhancement on the reconstructed image to obtain an intermediate image
  • the texture weight determination module is used to determine the texture complexity weights corresponding to each region in the intermediate image, and the texture complexity weight is a number between 0 and 1;
  • the texture attenuation module is configured to respectively attenuate the texture intensity corresponding to each region in the intermediate image according to the texture complexity weights corresponding to each region in the intermediate image, so as to obtain an enhanced image corresponding to the reconstructed image.
  • the intermediate image is a residual image
  • the texture attenuation module includes:
  • the residual update module is used to multiply the pixel value of each pixel in the residual image by the texture complexity weight corresponding to the region to which each pixel belongs to obtain the residual update image;
  • the image generation module is used to update and reconstruct the image according to the residual to generate the enhanced image.
  • the image generating module is specifically configured to add the residual update image and the reconstructed image to obtain the enhanced image.
  • the intermediate image is a texture-enhanced image
  • the device further includes: an image fidelity module, configured to perform image fidelity on the reconstructed image to obtain a basic fidelity image.
  • the texture weight determination module is specifically used to divide the basic fidelity image and the texture enhanced image into N regions according to the preset partition rules, and N is positive Integer; respectively determine the texture complexity of the N regions in the basic fidelity image; determine the texture complexity weights corresponding to the N regions in the texture enhanced image based on the texture complexity of the N regions in the basic fidelity image.
  • the texture attenuation module includes:
  • a weighted weight determination module configured to determine the weighted calculation weights corresponding to the N areas in the texture enhanced image and the weighted calculations corresponding to the N areas in the basic fidelity image according to the texture complexity weights corresponding to the N areas in the texture enhanced image Weights;
  • the weighted calculation module is used to calculate the weighted calculation weights corresponding to the N areas in the texture enhanced image and the weighted calculation weights corresponding to the N areas in the basic fidelity image respectively, for the N areas in the texture enhanced image and the weighted calculation weights in the basic fidelity image. N regions are weighted and calculated to obtain an enhanced image corresponding to the reconstructed image.
  • the weighted calculation module is specifically configured to multiply the weighted calculation weights corresponding to the N regions in the texture enhanced image by the N regions in the texture enhanced image respectively , to obtain the first product; multiply the weighted calculation weights corresponding to the N regions in the basic fidelity image with the N regions in the basic fidelity image to obtain the second product; add the first product and the second product , to obtain the enhanced image corresponding to the reconstructed image.
  • the weighted weight determination module is specifically configured to determine the texture complexity weights corresponding to N regions in the texture enhanced image as N regions in the texture enhanced image
  • the corresponding weighted calculation weights; the difference between 1 and the texture complexity weights corresponding to the N regions in the texture enhanced image is determined as the weighted calculation weights corresponding to the N regions in the basic fidelity image.
  • the texture weight determination module is specifically used to decode the texture complexity weights corresponding to the N regions in the intermediate image from the received code stream, where N is positive integer.
  • the texture weight determination module 803 is specifically configured to divide the reconstructed image and the intermediate image into N regions respectively according to preset partition rules, where N is a positive integer; Determine the texture complexity of the N regions in the reconstructed image respectively; based on the texture complexity of the N regions in the reconstructed image, determine the texture complexity weights corresponding to the N regions in the intermediate image.
  • the second aspect and any implementation manner of the second aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • technical effects corresponding to the second aspect and any implementation manner of the second aspect reference may be made to the technical effects corresponding to the above-mentioned first aspect and any implementation manner of the first aspect, and details are not repeated here.
  • an embodiment of the present application provides an electronic device, including: a memory and a processor, the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the first aspect or An image processing method in any possible implementation manner of the first aspect.
  • the third aspect and any implementation manner of the third aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • the technical effects corresponding to the third aspect and any one of the implementation manners of the third aspect refer to the above-mentioned first aspect and the technical effects corresponding to any one of the implementation manners of the first aspect, which will not be repeated here.
  • the embodiment of the present application provides a chip, including one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, and the signals include memory computer instructions stored in the computer; when the processor executes the computer instructions, the electronic device is made to execute the image processing method in the first aspect or any possible implementation manner of the first aspect.
  • the fourth aspect and any implementation manner of the fourth aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • the technical effects corresponding to the fourth aspect and any one of the implementation manners of the fourth aspect refer to the above-mentioned first aspect and the technical effects corresponding to any one of the implementation manners of the first aspect, and details are not repeated here.
  • the embodiment of the present application provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program runs on a computer or a processor, the computer or processor executes the first aspect or the first aspect.
  • An image processing method in any possible implementation of an aspect.
  • the fifth aspect and any implementation manner of the fifth aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • the technical effects corresponding to the fifth aspect and any one of the implementation manners of the fifth aspect refer to the technical effects corresponding to the above-mentioned first aspect and any one of the implementation manners of the first aspect, and details are not repeated here.
  • the embodiment of the present application provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program runs on a computer or a processor, the computer or processor executes the first aspect or the first aspect.
  • An image processing method in any possible implementation of an aspect.
  • the sixth aspect and any implementation manner of the sixth aspect correspond to the first aspect and any implementation manner of the first aspect respectively.
  • the technical effects corresponding to the sixth aspect and any one of the implementation manners of the sixth aspect refer to the technical effects corresponding to the above-mentioned first aspect and any one of the implementation manners of the first aspect, and details are not repeated here.
  • Figure 1a is a schematic diagram of an exemplary application scenario
  • Figure 1b is a schematic diagram of an exemplary processing process
  • FIG. 2 is a schematic diagram of an exemplary processing process
  • FIG. 3a is a schematic diagram of an exemplary image enhancement process
  • Fig. 3b is a schematic diagram of an exemplary processing process
  • Fig. 3c is a schematic diagram of an exemplary image enhancement effect
  • FIG. 4 is a schematic diagram of an exemplary processing process
  • FIG. 5 is a schematic diagram of an exemplary processing process
  • FIG. 6 is a schematic diagram of an exemplary image enhancement process
  • FIG. 7 is a schematic diagram of an exemplary processing process
  • FIG. 8 is a schematic diagram of an exemplary image processing device
  • Fig. 9 is a schematic structural diagram of the device shown exemplarily.
  • first and second in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects.
  • first target object, the second target object, etc. are used to distinguish different target objects, rather than describing a specific order of the target objects.
  • words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.
  • Fig. 1a is a schematic diagram of an exemplary application scenario.
  • the encoding end may encode the image to be encoded to obtain a code stream; and then transmit the code stream to the decoding end. After the decoding end obtains the code stream, it can decode the code stream to obtain the reconstructed image.
  • the decoding end may perform post-processing on the reconstructed image to enhance the reconstructed image to obtain an enhanced image, and then output the enhanced image to reduce visual distortion of the reconstructed image.
  • the present application proposes an image processing method, which can enhance the texture of the textured area in the reconstructed image, and avoid false textures in the non-textured area, so as to reduce the visual distortion of the reconstructed image, thereby increasing the texture of the image.
  • Realism which improves the subjective quality of images.
  • Fig. 1b is a schematic diagram of an exemplary processing process.
  • the decoding end may decode the code stream to obtain a reconstructed image.
  • image enhancement may be performed on the reconstructed image to obtain an intermediate image.
  • an image enhancement model may be used to perform image enhancement on the reconstructed image.
  • the image enhancement model may be GANEF (Generative Adversarial Network Enhancement Filter, an enhancement filter for generating an adversarial network), which is an enhancement filter implemented based on a generating adversarial network.
  • GAN generating confrontation network
  • GANEF Generic Adversarial Network Enhancement Filter
  • GAN generating confrontation network
  • the generator and the discriminator can be alternately trained iteratively.
  • the goal of the discriminator is to distinguish the authenticity of the image generated by the generator and the original image
  • the goal of the generator is to generate an image close to the original image to deceive the discriminator.
  • the generator can generate images as close as possible to Image of the original image.
  • the training process of GANEF may be as follows: multiple sets of training data are generated, and one set of training data includes: an image to be encoded and a reconstructed image obtained by decoding the code stream of the image to be encoded.
  • training data may be used to train the generator and the discriminator alternately, wherein, when training the generator, the weight parameters of the discriminator may be fixed; when training the discriminator, the weight parameters of the generator may be fixed.
  • the process of training the generator may be as follows: in advance, the image to be encoded in the training data is input to the discriminator, and the discriminator performs forward calculation on the image to be encoded, and outputs the identification result. Then the weight parameter of the discriminator is adjusted to make the probability that the discrimination result output by the discriminator is "true” close to the preset probability as the target.
  • the preset probability can be set according to requirements, which is not limited in this application. According to the input image to be coded by the discriminator, when the difference between the probability that the discrimination result is "true" and the preset probability is smaller than the probability threshold, the weight parameter of the discriminator can be fixed.
  • the probability threshold can be set according to requirements, which is not limited in this application.
  • the reconstructed image in the training data is input to the generator, and the generator performs forward calculation on the reconstructed image, and outputs the intermediate image to the discriminator.
  • the discriminator performs forward calculation on the intermediate image to output the identification result; on the other hand, based on the intermediate image and the image to be encoded, a loss function value is generated. Then, to maximize the probability that the discrimination result output by the discriminator is "true", and to minimize the value of the loss function, adjust the weight parameters of the generator.
  • the process of training the discriminator may be as follows: in advance, the image to be coded in the training data is input to the discriminator, and the discriminator performs forward calculation on the image to be coded, and outputs a discrimination result. Then the weight parameter of the discriminator is adjusted to make the probability that the discrimination result output by the discriminator is "true" close to the preset probability as the target.
  • the discriminator when the discriminator outputs the difference between the probability that the identification result is "true" and the preset probability is less than the probability threshold, then the reconstructed image in the training data is input to the generator, and the generator reconstructs The image is forward-calculated and the intermediate image is output to the discriminator. Then, the discriminator performs forward calculation on the intermediate image, outputs the discriminative result, and then adjusts the weight parameters of the discriminator with the goal of maximizing the probability that the discriminative result output by the discriminator is "false".
  • the generator and the discriminator in GANEF are alternately and iteratively trained in the above manner until the discriminator cannot make a good distinction between the intermediate image output by the generator and the image to be encoded.
  • the intermediate image may be a residual image, or a texture enhanced image.
  • the texture enhanced image can be generated based on the residual image and the reconstructed image; then the texture enhanced image is input to the discriminator, and the texture enhanced image Calculate the loss function value with the image to be encoded.
  • the texture-enhanced image can be directly input into the discriminator.
  • the residual image and the reconstructed image may be added to generate a texture-enhanced image.
  • the residual image and the reconstructed image may be added pixel by pixel, that is, the pixel values of the corresponding pixels of the residual image and the reconstructed image are added to obtain an enhanced image corresponding to the reconstructed image.
  • only the trained generator in the GANEF may be used to perform image enhancement on the reconstructed image to obtain an intermediate image.
  • the resolution of the intermediate image is the same as that of the reconstructed image.
  • texture complexity weights corresponding to each region in the intermediate image may be determined according to the texture complexity of each region in the reconstructed image.
  • texture complexity weights corresponding to each region in the intermediate image may be determined according to the texture complexity of each region in the reconstructed image corresponding to the image to be encoded.
  • the texture complexity weight is proportional to the texture complexity, that is, the higher the texture complexity, the larger the texture complexity weight; the lower the texture complexity, the smaller the texture complexity weight.
  • the texture complexity weight is a decimal between 0 and 1.
  • the pixel value of each pixel in the region may be attenuated based on the texture complexity weight corresponding to the region, so as to adjust the texture intensity corresponding to the region.
  • the texture complexity weight is proportional to the texture complexity, after the texture intensity is adjusted, the area with higher texture complexity in the intermediate image has a smaller texture intensity attenuation, and the area with lower texture complexity has a corresponding texture enhancement attenuation. bigger. In this way, while the texture of the textured area (higher texture complexity) can be enhanced, false stripes can be avoided in the non-textured area (lower texture complexity).
  • this application controls the texture intensity attenuation of the GANEF result based on the texture complexity of the local area of the image, that is, the texture area maintains the original GANEF filtering effect as much as possible, and the non-texture area performs the attenuation of the GANEF filtering effect according to the texture complexity.
  • the smaller the texture complexity of the local area the larger the attenuation, which can effectively remove the false texture of the non-textured area.
  • the present application controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • FIG. 1 b Exemplarily, for the specific implementation manner of FIG. 1 b , reference may be made to FIG. 2 , FIG. 4 , FIG. 5 and FIG. 7 , and corresponding descriptions.
  • Fig. 2 is a schematic diagram of an exemplary processing procedure.
  • GANEF is used to filter the reconstructed image to obtain a residual image; then, based on the texture complexity of each area in the reconstructed image, the texture complexity weights corresponding to each area in the residual image are determined; then The residual image is updated according to the texture complexity weights corresponding to each region, and finally the updated residual image is added to the reconstructed image to obtain the final enhanced image.
  • the decoding end may decode the code stream to obtain a reconstructed image; then, an image enhancement model may be used to perform image enhancement on the reconstructed image to obtain an intermediate image.
  • the intermediate image is a residual image.
  • Fig. 3a is a schematic diagram of an exemplary image enhancement process.
  • the output of the trained image augmentation model is a residual image.
  • the reconstructed image is input to the image enhancement model, and the residual image can be obtained directly, as shown in Fig. 3a(1).
  • the output of the trained image augmentation model is a texture augmented image.
  • the image enhancement model can output a texture-enhanced image; then, based on the texture-enhanced image and the reconstructed image, a residual image is determined.
  • pixel-by-pixel subtraction can be performed on the texture-enhanced image and the reconstructed image, that is, the pixel values of the corresponding pixels in the texture-enhanced image and the reconstructed image are subtracted to obtain a residual image, as shown in Figure 3a(2) .
  • the residual image has the same resolution as the reconstructed image.
  • the texture complexity weight corresponding to each region in the residual image may be determined according to the texture complexity corresponding to each region in the reconstructed image.
  • S202 may include S2021-S2023:
  • the reconstructed image may be divided into N regions according to a preset partition rule set in advance; and the residual image may be divided into N regions according to a preset partition rule. That is to say, the reconstructed image and the residual image are divided into regions in the same way. Since the reconstructed image and the residual image have the same resolution, the N regions in the reconstructed image correspond to the N regions in the residual image. .
  • the preset partition rules can be set according to requirements, which is not limited in the present application; wherein, N is a positive integer determined according to the preset partition rules.
  • each of the N regions may be the same or different, which is not limited in the present application.
  • shape of each of the N regions may be the same or different, which is not limited in the present application.
  • present application does not limit the shapes of the N regions.
  • the texture complexity of the i-th area may be determined according to the pixel values of the pixels contained in the i-th area.
  • the texture complexity of the i-th region of the reconstructed image may be determined based on the co-occurrence matrix of the i-th region in the reconstructed image.
  • the co-occurrence matrix of the i-th area in the reconstructed image may be determined according to the pixel values of the pixels contained in the i-th area in the reconstructed image. Then extract the characteristic quantity (such as energy, contrast, entropy, inverse variance, etc.) Image texture complexity for i regions.
  • the feature quantity of the co-occurrence matrix of the i-th region in the reconstructed image is used as the texture complexity of the i-th region in the reconstructed image.
  • the texture complexity of the i-th region in the reconstructed image may be determined based on the edge ratio of the i-th region in the reconstructed image.
  • the gradient intensity corresponding to each pixel in the i-th area in the reconstructed image may be calculated according to the pixel values of the pixels contained in the i-th area of the reconstructed image. Then, the proportion of pixels whose gradient strength is greater than the gradient strength threshold in the i-th region of the reconstructed image is used as the texture complexity of the i-th region of the reconstructed image.
  • the gradient strength threshold can be set according to requirements, which is not limited in this application.
  • the texture complexity of the N regions in the reconstructed image can be determined.
  • the texture complexity weights corresponding to the i-th (i is an integer between 1 and N, i can be equal to 1 and N) regions in the residual image are determined below for exemplary illustration.
  • the texture complexity of the i-th region in the reconstructed image may be used as the texture complexity weight corresponding to the i-th region in the residual image.
  • the texture complexity of the i-th region in the reconstructed image may be mapped according to a preset mapping rule to obtain the texture complexity weight corresponding to the i-th region in the residual image.
  • the preset mapping rules can be set according to requirements, such as normalization, which is not limited in the present application.
  • the texture complexity of the i-th region in the reconstructed image can be normalized to obtain the texture complexity weight corresponding to the i-th region in the residual image.
  • the texture complexity weight may be a decimal between 0 and 1.
  • the texture complexity weight is proportional to the texture complexity, that is, the higher the texture complexity, the larger the texture complexity weight; the lower the texture complexity, the smaller the texture complexity weight.
  • the texture intensity corresponding to each region in the residual image may be attenuated according to the texture complexity weights corresponding to each region in the residual image.
  • residual updates may be performed on N regions in the residual image respectively to obtain a residual updated image.
  • the residual image can be residually updated by multiplying the pixel value of each pixel in the residual image with the texture complexity weight corresponding to the region to which each pixel belongs, to obtain the residual update image.
  • the i-th (i is an integer between 1 and N, i can be equal to 1 and N) regions in the residual image is taken as an example to illustrate the residual update.
  • the texture enhancement weight corresponding to the i-th region in the residual image is ratio_i
  • (k, j) is the integer index of the pixel coordinates of the residual image
  • k, j represent the horizontal and vertical coordinate indices respectively
  • the pixel index of the upper left corner of the residual image is (0,0). That is to say, after the pixel value of each pixel in the residual image is updated, the residual updated image can be obtained, and the pixel value of the pixel in the residual updated image is R2(k,j).
  • the resolution of the residual update image and the reconstructed image are the same.
  • the texture complexity weight is directly proportional to the texture complexity, in this way, the region with higher texture complexity of the residual image has smaller texture intensity attenuation, and the region with lower texture complexity has greater texture enhancement attenuation.
  • the pixel values of the corresponding pixel points in the reconstructed image and the residual update image may be added to obtain an enhanced image corresponding to the reconstructed image.
  • Fig. 3b is a schematic diagram of an exemplary processing process.
  • A1 is a reconstructed image, and the reconstructed image is filtered by GANEF to obtain a texture-enhanced image, as shown in A2. Then the reconstructed image A1 can be subtracted from the texture-enhanced image A2 to obtain the residual image, as shown in A4
  • the gradient of each pixel in the reconstructed image A1 can be calculated, and then based on the gradient of each pixel in the reconstructed image A1, the reconstructed image can be binarized to obtain a binary image, as shown in A3.
  • the binary image A3 is divided into N regions, and for each region, according to the proportion of black pixels in the region, the corresponding texture complexity of the region is determined; and then the N regions in the binary image A3 respectively correspond to texture complexity.
  • the texture complexity weights corresponding to the N regions in the residual image A4 are determined.
  • the residual image A4 is residually updated to obtain a residual updated image, as shown in A5.
  • the reconstructed image A1 may be added to the residual update image A5 to obtain the enhanced image A6.
  • Fig. 3c is a schematic diagram of an exemplary image enhancement effect.
  • FIG. 3c(1) is a schematic diagram of a local area in the texture-enhanced image A2 of FIG. 3b
  • FIG. 3c(2) is a schematic diagram of a local area in the enhanced image A6 of FIG. 3b.
  • the road surface in Figure 3c(1) and Figure 3c(2) is a non-textured area, through the elliptical area in Figure 3c(1) and Figure 3c(2), and comparing Figure 3c(1) and Figure 3c It can be seen from the rectangular area in (2) that there are no false stripes in the non-textured area of the enhanced image obtained in the present application.
  • the texture complexity weight in the range of 0 to 1 controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • Fig. 4 is a schematic diagram of an exemplary processing procedure.
  • GANEF is firstly used to filter the reconstructed image to obtain a residual image; then, based on the texture complexity of the reconstructed image corresponding to each region in the image to be encoded, the texture complexity corresponding to each region of the residual image is determined weight; then update the residual image according to the texture complexity weights corresponding to each region, and finally add the updated residual image and the reconstructed image to obtain the final enhanced image.
  • the encoding end may generate texture complexity weights corresponding to each region in the image to be encoded based on the image to be encoded, which may include S4011-S4013:
  • Determining the texture complexity weights corresponding to the i-th (i is an integer between 1 and N, and the value of i can be 1 or N) regions in the image to be coded is exemplified as an example for illustration.
  • the texture complexity of the i-th region in the image to be encoded may be used as the texture complexity weight corresponding to the i-th region in the image to be encoded.
  • the texture complexity of the i-th region in the image to be coded can be mapped according to a preset mapping rule to obtain the texture complexity weight corresponding to the i-th region in the image to be coded.
  • the texture complexity of the i-th region in the image to be coded can be normalized to obtain the texture complexity weight corresponding to the i-th region in the image to be coded.
  • the texture complexity weight may be a decimal between 0 and 1.
  • the texture complexity weight is proportional to the texture complexity, that is, the higher the texture complexity, the larger the texture complexity weight; the lower the texture complexity, the smaller the texture complexity weight.
  • the encoding end may encode the image to be encoded to obtain a corresponding code stream; and may encode texture complexity weights corresponding to N regions in the image to be encoded to obtain a corresponding code stream. Then, the bit stream obtained by encoding the texture complexity weights respectively corresponding to the N regions in the image to be encoded and the bit stream obtained by encoding the image to be encoded may be sent to the decoding end. In this way, after obtaining the code stream, the decoding end decodes the texture complexity weights respectively corresponding to N regions in the reconstructed image and the image to be encoded from the code stream.
  • the decoding end may use an image enhancement model to perform image enhancement on the reconstructed image to obtain an intermediate image.
  • the intermediate image is a residual image.
  • the resolution of the residual image is the same as the resolution of the reconstructed image and the resolution of the image to be encoded.
  • the residual image may be divided into N regions according to a preset partition rule set in advance.
  • the region division method of the residual image is the same as that of the image to be coded. Since the resolution of the residual image is the same as that of the image to be coded, the N regions in the residual image are the same as the N regions in the image to be coded. Regions are in one-to-one correspondence.
  • the texture complexity weight corresponding to the i-th region of the image to be encoded may be used as the texture complexity weight corresponding to the i-th region of the residual image. In this way, the texture complexity weights respectively corresponding to the N regions in the residual image can be determined.
  • the texture complexity weight may be a decimal between 0 and 1.
  • the texture intensity corresponding to each region in the residual image may be adjusted according to the texture complexity weights corresponding to each region in the residual image,
  • the enhanced image corresponding to the reconstructed image refer to S404-S405:
  • the texture enhancement corresponding to each region in the reconstructed image by controlling the texture complexity weight in the range of 0 to 1 according to the texture complexity of the image to be encoded, it is possible to enhance the texture region (higher texture complexity) in the reconstructed image At the same time, it avoids false stripes in the non-textured area (lower texture complexity) in the reconstructed image; thereby reducing the visual distortion of the reconstructed image.
  • the present application controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • the texture complexity weight determined according to the image to be encoded is more accurate, and thus can more accurately control the attenuation of image texture intensity, thereby further improving image quality.
  • Fig. 5 is a schematic diagram of an exemplary processing procedure.
  • non-GANEF and GANEF are used to filter the reconstructed image to obtain the basic fidelity image and the texture-enhanced image respectively, and then based on the reconstructed image or the basic fidelity
  • the texture complexity corresponding to each area in the image determines the weighted calculation factors corresponding to the basic fidelity image and the texture enhanced image, and finally based on the weighted calculation factors, the basic fidelity image and the texture enhanced image are weighted and fused to obtain the final enhanced image.
  • the decoding end may decode the code stream to obtain a reconstructed image; then, an image enhancement model may be used to perform image enhancement on the reconstructed image to obtain an intermediate image.
  • the intermediate image is a texture enhanced image.
  • Fig. 6 is a schematic diagram of an exemplary image enhancement process.
  • the output of the trained image augmentation model is a residual image.
  • the reconstructed image is input to the image enhancement model, and the residual image output by the image enhancement model can be obtained, and then the texture enhanced image can be determined according to the residual image and the reconstructed image.
  • the pixel values of the corresponding pixels in the residual image and the reconstructed image may be added to obtain a texture-enhanced image, as shown in FIG. 6(1).
  • the output of the trained image augmentation model is a texture augmented image.
  • the texture enhanced image can be obtained directly, as shown in Fig. 6(2).
  • the texture-enhanced image has the same resolution as the reconstructed image.
  • a preset image anti-aliasing model can be used to reconstruct an image for image anti-aliasing to obtain a basic anti-aliasing image.
  • the network used in the image fidelity model may be a non-generative confrontation network, such as a convolutional neural network, which is not limited in this application.
  • the training process of the image fidelity model may be as follows: multiple sets of training data are collected, and one set of training data includes the image to be encoded and the reconstructed image obtained by decoding the code stream of the image to be encoded.
  • a set of training data For a set of training data, a set of training data is input into the image fidelity model, and the image fidelity model performs forward calculation on the reconstructed image, and outputs the basic fidelity image.
  • Calculate the loss function value based on the basic fidelity image output by the image fidelity model and the image to be encoded in the training data, and adjust the weight parameters of the image fidelity model with the goal of minimizing the loss function value.
  • the image fidelity model can be trained using multiple sets of collected training data according to the above method, until the number of training times of the image fidelity model is equal to the preset number of training times, or the loss function value of the image fidelity model is less than or equal to the loss function threshold , or the effect of the image anti-aliasing model satisfies the preset effect condition, the training of the image anti-aliasing model is stopped, and the trained image anti-aliasing model is obtained.
  • the preset training times, the loss function threshold and the preset effect condition can all be set according to requirements, which is not limited in this application.
  • both the texture-enhanced image and the basic fidelity image have the same resolution as the reconstructed image.
  • S503 may include S5031a-S5034a:
  • the texture complexity weights corresponding to each region in the texture enhanced image can be generated, which can refer to S5031b-S5033b:
  • the regions of the texture-enhanced image and the basic fidelity image have a one-to-one correspondence.
  • each region in the texture-enhanced image can be weighted and fused with the corresponding region in the basic fidelity image to obtain an enhanced image.
  • S504 may include S5041-S5042:
  • the texture complexity weights corresponding to the N regions in the texture enhanced image determine the weighted calculation weights corresponding to the N regions in the texture enhanced image and the weighted calculation weights corresponding to the N regions in the basic fidelity image.
  • the texture complexity weights corresponding to the N regions in the texture enhanced image may be used as weighted calculation weights for the N regions in the texture enhanced image.
  • the difference between 1 and the texture complexity weights corresponding to the N regions in the texture-enhanced image is used as the weighted calculation weight of the N regions in the basic fidelity image.
  • the texture complexity weight corresponding to the i-th area of the texture-enhanced image is ratio_i
  • the weighted calculation weight of the i-th area of the texture-enhanced image can be ratio_i
  • the i-th area of the basic fidelity image The weighting calculation weight of the region is 1-ratio_i.
  • weighted calculation weights corresponding to the N regions in the texture enhanced image and the weighted calculation weights corresponding to the N regions in the basic fidelity image perform N regions in the texture enhanced image and N regions in the basic fidelity image Weighted calculation to obtain an enhanced image.
  • each pixel in the i-th area in the texture-enhanced image Points and each pixel of the i-th area in the basic fidelity image are weighted to obtain the enhanced image of the i-th area.
  • the weighted calculation weights corresponding to the N regions in the texture enhanced image can be multiplied by the N regions in the texture enhanced image to obtain the first product; the weighted weights corresponding to the N regions in the basic fidelity image Calculate the weight and multiply it with the N regions in the basic fidelity image to obtain the second product; add the first product and the second product to obtain the enhanced image corresponding to the reconstructed image.
  • the weighted calculation weight of the i-th region in the texture-enhanced image is ratio_i
  • the pixel value of a pixel (j,k) in the i-th region in the texture-enhanced image is E1(j,k); in the basic fidelity image
  • the weighted calculation weight of the i-th area is 1-ratio_i
  • the pixel value of a pixel point (j, k) in the i-th area in the basic fidelity image is E2(j, k)
  • Pixel value R(i,j) ratio_i*E1(i,j)+(1-ratio_i)*E2(i,j) of pixel point (j,k) in the region after image enhancement.
  • the texture enhancement corresponding to each region in the reconstructed image by controlling the texture complexity weight in the range of 0 to 1 according to the texture complexity of the reconstructed image, it is possible to enhance the texture area (higher texture complexity) in the reconstructed image. At the same time, it avoids false stripes in the non-textured area (lower texture complexity) in the reconstructed image; thereby reducing the visual distortion of the reconstructed image.
  • the present application controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • Fig. 7 is a schematic diagram of an exemplary processing procedure.
  • non-GANEF and GANEF are used to filter the reconstructed image to obtain the basic fidelity image and texture-enhanced image respectively, and then based on the corresponding to-be-encoded image of the reconstructed image
  • the texture complexity corresponding to each area in the image or the basic fidelity image determine the weighted calculation factors corresponding to the basic fidelity image and the texture enhanced image respectively, and finally carry out weighted fusion of the basic fidelity image and the texture enhanced image based on the weighted calculation factors, and obtain The final enhanced image.
  • the encoding end may generate texture complexity weights based on the image to be encoded, which may include S7011-S7013:
  • the texture-enhanced image may be divided into N regions according to a preset partition rule set in advance.
  • the region division method of the texture-enhanced image is the same as that of the image to be coded, and the resolution of the texture-enhanced image and the image to be coded is the same, so the regions in the texture-enhanced image and the regions in the image to be coded are one by one corresponding.
  • the texture complexity weight corresponding to the i-th region of the image to be encoded may be used as the texture complexity weight corresponding to the i-th region of the texture-enhanced image. In this way, the texture complexity weights corresponding to the N regions in the texture enhanced image can be determined.
  • the texture complexity weight may be a decimal between 0 and 1.
  • the texture intensity corresponding to each region in the texture-enhanced image can be adjusted according to the texture complexity weights corresponding to each region in the texture-enhanced image to obtain the reconstructed
  • the texture complexity weights corresponding to each region in the texture-enhanced image For the enhanced image corresponding to the image, refer to S704-S705:
  • the present application does not limit the execution order of S704 and S702.
  • the texture enhancement corresponding to each region in the reconstructed image by controlling the texture complexity weight in the range of 0 to 1 according to the texture complexity of the image to be encoded, it is possible to enhance the texture region (higher texture complexity) in the reconstructed image At the same time, it avoids false stripes in the non-textured area (lower texture complexity) in the reconstructed image; thereby reducing the visual distortion of the reconstructed image.
  • the present application controls the texture complexity weight in the range of 0 to 1, so that the texture corresponding to the adjacent area is softer, and the problem of inconsistency in the enhanced texture effect of the adjacent area can be avoided.
  • the texture complexity weight determined according to the image to be encoded is more accurate, and thus can more accurately control the attenuation of image texture intensity, thereby further improving image quality.
  • Fig. 8 is a schematic diagram of an exemplary image processing device.
  • the image processing device includes: an image acquisition module 801, an image enhancement module 802, a texture weight determination module 803, and a texture attenuation module 804, wherein:
  • An image acquisition module 801 configured to acquire a reconstructed image
  • An image enhancement module 802 configured to perform image enhancement on the reconstructed image to obtain an intermediate image
  • a texture weight determination module 803, configured to determine texture complexity weights corresponding to each region in the intermediate image, where the texture complexity weight is a number between 0 and 1;
  • the texture attenuation module 804 is configured to respectively attenuate the texture intensity corresponding to each region in the intermediate image according to the texture complexity weights corresponding to each region in the intermediate image, so as to obtain an enhanced image corresponding to the reconstructed image.
  • the intermediate image is a residual image
  • Texture attenuation module 804 including:
  • the residual update module is used to multiply the pixel value of each pixel in the residual image by the texture complexity weight corresponding to the region to which each pixel belongs to obtain the residual update image;
  • the image generation module is used to update and reconstruct the image according to the residual to generate the enhanced image.
  • the image generating module is specifically configured to add the residual update image and the reconstructed image to obtain an enhanced image.
  • the intermediate image is a texture enhanced image
  • the device further includes:
  • the image fidelity module is used to perform image fidelity on the reconstructed image to obtain a basic fidelity image.
  • the texture weight determination module 803 is specifically configured to divide the basic fidelity image and the texture enhanced image into N regions respectively according to preset partition rules, where N is a positive integer; respectively determine the N regions in the basic fidelity image The texture complexity of N regions; based on the texture complexity of the N regions in the basic fidelity image, determine the texture complexity weights corresponding to the N regions in the texture enhanced image.
  • the texture attenuation module 804 includes:
  • a weighted weight determination module configured to determine the weighted calculation weights corresponding to the N areas in the texture enhanced image and the weighted calculations corresponding to the N areas in the basic fidelity image according to the texture complexity weights corresponding to the N areas in the texture enhanced image Weights;
  • the weighted calculation module is used to calculate the weighted calculation weights corresponding to the N areas in the texture enhanced image and the weighted calculation weights corresponding to the N areas in the basic fidelity image respectively, for the N areas in the texture enhanced image and the weighted calculation weights in the basic fidelity image. N regions are weighted and calculated to obtain an enhanced image corresponding to the reconstructed image.
  • the weighting calculation module is specifically configured to multiply the weighted calculation weights respectively corresponding to the N regions in the texture enhanced image by the N regions in the texture enhanced image to obtain the first product;
  • the weighted calculation weights corresponding to the regions are respectively multiplied by the N regions in the basic fidelity image to obtain a second product; the first product and the second product are added to obtain an enhanced image corresponding to the reconstructed image.
  • the weighted weight determination module is specifically configured to determine the texture complexity weights corresponding to the N regions in the texture enhanced image as the weighted calculation weights corresponding to the N regions in the texture enhanced image; The differences of the texture complexity weights corresponding to the N regions in , are determined as the weighted calculation weights corresponding to the N regions in the basic fidelity image.
  • the texture weight determination module 803 is specifically configured to decode the texture complexity weights respectively corresponding to N regions in the intermediate image from the received code stream, where N is a positive integer.
  • the texture weight determination module 803 is specifically configured to divide the reconstructed image and the intermediate image into N regions respectively according to preset partition rules, where N is a positive integer; respectively determine the texture complexity of the N regions in the reconstructed image; Based on the texture complexity of the N regions in the reconstructed image, determine the texture complexity weights corresponding to the N regions in the intermediate image.
  • FIG. 9 shows a schematic block diagram of an apparatus 900 according to an embodiment of the present application.
  • the apparatus 900 may include: a processor 901 and a transceiver/transceiving pin 902 , and optionally, a memory 903 .
  • bus 904 includes a power bus, a control bus, and a status signal bus in addition to a data bus.
  • the various buses are referred to as bus 904 in the figure.
  • the memory 903 may be used for the instructions in the foregoing method embodiments.
  • the processor 901 can be used to execute instructions in the memory 903, and control the receiving pin to receive signals, and control the sending pin to send signals.
  • the apparatus 900 may be the electronic device or the chip of the electronic device in the foregoing method embodiments.
  • This embodiment also provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions are run on the electronic device, the electronic device executes the above-mentioned relevant method steps to realize the steps in the above-mentioned embodiments. image processing method.
  • This embodiment also provides a computer program product, which, when running on a computer, causes the computer to execute the above-mentioned related steps, so as to realize the image processing method in the above-mentioned embodiment.
  • an embodiment of the present application also provides a device, which may specifically be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, The processor can execute the computer-executable instructions stored in the memory, so that the chip executes the image processing methods in the above method embodiments.
  • the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment is all used to execute the corresponding method provided above, therefore, the beneficial effects it can achieve can refer to the above-mentioned The beneficial effects of the corresponding method will not be repeated here.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or It may be integrated into another device, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component shown as a unit may be one physical unit or multiple physical units, which may be located in one place or distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • an integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk.
  • the steps of the methods or algorithms described in connection with the disclosure of the embodiments of the present application may be implemented in the form of hardware, or may be implemented in the form of a processor executing software instructions.
  • the software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read-only memory (Read Only Memory, ROM), erasable programmable read-only memory ( Erasable Programmable ROM, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically EPROM, EEPROM), registers, hard disk, removable hard disk, CD-ROM, or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
  • the storage medium may also be a component of the processor.
  • the processor and storage medium can be located in the ASIC.
  • the functions described in the embodiments of the present application may be implemented by hardware, software, firmware or any combination thereof.
  • the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
  • Computer-readable media includes both computer-readable storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例提供了一种图像处理方法、装置及电子设备。该方法包括：首先，获取重建图像，然后，对重建图像进行图像增强，得到中间图像；接着，确定中间图像中各区域分别对应的纹理复杂度权重，纹理复杂度权重为0至1之间的数，随后，依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像。这样，能够保留中间图像中纹理区域的纹理，同时对非纹理区域中的纹理进行衰减，进而实现增强重建图像中纹理区域的纹理的同时，避免非纹理区域产生虚假纹理，从而减小重建图像的视觉失真，增加图像的真实感，提高图像的主观质量。

Description

图像处理方法、装置及电子设备
本申请要求于2021年12月10日提交中国国家知识产权局、申请号为202111511051.7、名称为“图像处理方法、装置及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及图像处理技术领域，尤其涉及一种图像处理方法、装置及电子设备。
背景技术
通常，在传输视频之前，会对视频进行压缩，以提高视频传输的效率；其中，视频压缩率越大，压缩后的视频的数据量越小。但随着压缩率的增加，视频会出现可见的视觉失真；为了减小重建图像的视觉失真，可以对重建图像进行后处理，以实现码率不变情况下的质量提升。
虽然对重建图像进行后处理，能够显著提高重建图像的主观质量，但在非纹理区域容易生成虚假纹理，影响图像的主观质量。
发明内容
为了解决上述技术问题，本申请提供一种图像处理方法、装置及电子设备。
第一方面，本申请实施例提供一种图像处理方法，该方法包括：首先，获取重建图像；然后，对重建图像进行图像增强，得到中间图像。接着，确定中间图像中各区域分别对应的纹理复杂度权重，纹理复杂度权重为0至1之间的数；随后，依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像。这样，能够保留中间图像中纹理区域的纹理，同时对非纹理区域中的纹理进行衰减，进而实现增强重建图像中纹理区域的纹理的同时，避免非纹理区域产生虚假纹理，从而减小重建图像的视觉失真，增加图像的真实感，提高图像的主观质量。
此外，本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
示例性的，纹理复杂度权重与纹理复杂度成正比，也就是说，纹理复杂度越大，纹理复杂度权重越大。这样，能够尽可能保持中间图像中纹理区域的纹理效果，而针对非纹理区域，根据纹理复杂度进行衰减。
示例性的，中间图像为纹理增强图像或残差图像。
示例性的，中间图像和增强图像的分辨率，均与重建图像的分辨率相同。
示例性的，可以采用GANEF(Generative Adversarial Network Enhancement Filter，生成对抗网络的增强滤波器)对重建图像进行图像增强，得到中间图像。
示例性的，GANEF输出的图像可以为纹理增强图像，也可以为残差图像。
需要说明的，虽然增强图像是对中间图像中各区域进行了纹理增强的衰减，但是仅对中间图像进行了纹理强度的部分衰减，相对于重建图像而言，增强图像部分区域的纹理强度依然是大于重建图像的；也就是说，增强图像依然是具有纹理增强效果的。
根据第一方面，中间图像为残差图像；依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像，包括：将残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，得到残差更新图像；根据残差更新图像和重建图像，生成增强图像。
示例性的，残差图像通过将纹理增强图像与重建图像相减得到。
根据第一方面，或者以上第一方面的任意一种实现方式，根据残差更新图像和重建图像，生成增强图像，包括：将残差更新图像和重建图像相加，得到增强图像。
示例性的，可以对残差更新图像和重建图像进行逐像素点相加，得到增强图像。
根据第一方面，或者以上第一方面的任意一种实现方式，中间图像为纹理增强图像，方法还包括：对重建图像进行图像保真，得到基础保真图像。
示例性的，基础保真图像与重建图像的分辨率相同。
示例性的，可以采用非GANEF对重建图像进行图像保真，得到基础保真图像。
根据第一方面，或者以上第一方面的任意一种实现方式，确定中间图像中各区域对应的纹理复杂度权重，包括：按照预设分区规则，将基础保真图像和纹理增强图像分别划分为N个区域，N为正整数；分别确定基础保真图像中N个区域的纹理复杂度；基于基础保真图像中N个区域的纹理复杂度，确定纹理增强图像中N个区域分别对应的纹理复杂度权重。
根据第一方面，或者以上第一方面的任意一种实现方式，依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像，包括：依据纹理增强图像中N个区域对应的纹理复杂度权重，确定纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重；依据纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，对纹理增强图像中N个区域和基础保真图像中N个区域进行加权计算，得到重建图像对应的增强图像。
根据第一方面，或者以上第一方面的任意一种实现方式，依据纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，对纹理增强图像中N个区域和基础保真图像中N个区域进行加权计算，得到重建图像对应的增强图像，包括：将纹理增强图像中N个区域分别对应的加权计算权重，与纹理增强图像中N个区域分别相乘，得到第一乘积；将基础保真图像中N个区域分别对应的加权 计算权重，与基础保真图像中N个区域分别相乘，得到第二乘积；将第一乘积和第二乘积相加，得到重建图像对应的增强图像。
根据第一方面，或者以上第一方面的任意一种实现方式，依据纹理增强图像中N个区域分别对应的纹理复杂度权重，确定纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，包括：将纹理增强图像中N个区域分别对应的纹理复杂度权重，确定为纹理增强图像中N个区域分别对应的加权计算权重；将1与纹理增强图像中N个区域分别对应的纹理复杂度权重的差值，确定为基础保真图像中N个区域分别对应的加权计算权重。
根据第一方面，或者以上第一方面的任意一种实现方式，确定中间图像中各区域分别对应的纹理复杂度权重，包括：从接收的码流中解码出中间图像中N个区域分别对应的纹理复杂度权重，N为正整数。这样，可以提高确定中间图像中各区域分别对应的纹理强度权重的准确性，进而能够更准确的对控制图像纹理强度的衰减，从而进一步提高图像质量。
根据第一方面，或者以上第一方面的任意一种实现方式，确定中间图像中各区域分别对应的纹理复杂度权重，包括：按照预设分区规则，将重建图像和中间图像分别划分为N个区域，N为正整数；分别确定重建图像中N个区域的纹理复杂度；基于重建图像中N个区域的纹理复杂度，确定中间图像中N个区域分别对应的纹理复杂度权重。
第二方面，本申请实施例提供一种图像处理装置，该装置包括：
图像获取模块，用于获取重建图像；
图像增强模块，用于对重建图像进行图像增强，得到中间图像；
纹理权重确定模块，用于确定中间图像中各区域分别对应的纹理复杂度权重，纹理复杂度权重为0至1之间的数；
纹理衰减模块，用于依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像。
根据第二方面，中间图像为残差图像；纹理衰减模块，包括：
残差更新模块，用于将残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，得到残差更新图像；
图像生成模块，用于根据残差更新图像和重建图像，生成增强图像。
根据第二方面，或者以上第二方面的任意一种实现方式，图像生成模块，具体用于将残差更新图像和重建图像相加，得到增强图像。
根据第二方面，或者以上第二方面的任意一种实现方式，中间图像为纹理增强图像， 装置还包括：图像保真模块，用于对重建图像进行图像保真，得到基础保真图像。
根据第二方面，或者以上第二方面的任意一种实现方式，纹理权重确定模块，具体用于按照预设分区规则，将基础保真图像和纹理增强图像分别划分为N个区域，N为正整数；分别确定基础保真图像中N个区域的纹理复杂度；基于基础保真图像中N个区域的纹理复杂度，确定纹理增强图像中N个区域分别对应的纹理复杂度权重。
根据第二方面，或者以上第二方面的任意一种实现方式，纹理衰减模块，包括：
加权权重确定模块，用于依据纹理增强图像中N个区域对应的纹理复杂度权重，确定纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重；
加权计算模块，用于依据纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，对纹理增强图像中N个区域和基础保真图像中N个区域进行加权计算，得到重建图像对应的增强图像。
根据第二方面，或者以上第二方面的任意一种实现方式，加权计算模块，具体用于将纹理增强图像中N个区域分别对应的加权计算权重，与纹理增强图像中N个区域分别相乘，得到第一乘积；将基础保真图像中N个区域分别对应的加权计算权重，与基础保真图像中N个区域分别相乘，得到第二乘积；将第一乘积和第二乘积相加，得到重建图像对应的增强图像。
根据第二方面，或者以上第二方面的任意一种实现方式，加权权重确定模块，具体用于将纹理增强图像中N个区域分别对应的纹理复杂度权重，确定为纹理增强图像中N个区域分别对应的加权计算权重；将1与纹理增强图像中N个区域分别对应的纹理复杂度权重的差值，确定为基础保真图像中N个区域分别对应的加权计算权重。
根据第二方面，或者以上第二方面的任意一种实现方式，纹理权重确定模块，具体用于从接收的码流中解码出中间图像中N个区域分别对应的纹理复杂度权重，N为正整数。
根据第二方面，或者以上第二方面的任意一种实现方式，纹理权重确定模块803，具体用于按照预设分区规则，将重建图像和中间图像分别划分为N个区域，N为正整数；分别确定重建图像中N个区域的纹理复杂度；基于重建图像中N个区域的纹理复杂度，确定中间图像中N个区域分别对应的纹理复杂度权重。
第二方面以及第二方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第二方面以及第二方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第三方面，本申请实施例提供一种电子设备，包括：存储器和处理器，存储器与处理器耦合；存储器存储有程序指令，当程序指令由处理器执行时，使得电子设备执行第一方面或第一方面的任意可能的实现方式中的图像处理方法。
第三方面以及第三方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第三方面以及第三方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第四方面，本申请实施例提供一种芯片，包括一个或多个接口电路和一个或多个处理器；接口电路用于从电子设备的存储器接收信号，并向处理器发送信号，信号包括存储器中存储的计算机指令；当处理器执行计算机指令时，使得电子设备执行第一方面或第一方面的任意可能的实现方式中的图像处理方法。
第四方面以及第四方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第四方面以及第四方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第五方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第一方面或第一方面的任意可能的实现方式中的图像处理方法。
第五方面以及第五方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第五方面以及第五方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第六方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第一方面或第一方面的任意可能的实现方式中的图像处理方法。
第六方面以及第六方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第六方面以及第六方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
附图说明
图1a为示例性示出的应用场景示意图；
图1b为示例性示出的处理过程示意图；
图2为示例性示出的处理过程示意图；
图3a为示例性示出的图像增强处理过程示意图；
图3b为示例性示出的处理过程示意图；
图3c为示例性示出的图像增强效果示意图；
图4为示例性示出的处理过程示意图；
图5为示例性示出的处理过程示意图；
图6为示例性示出的图像增强处理过程示意图；
图7为示例性示出的处理过程示意图；
图8为示例性示出的图像处理装置示意图；
图9为示例性示出的装置的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请实施例的说明书和权利要求书中的术语“第一”和“第二”等是用于区别不同的对象，而不是用于描述对象的特定顺序。例如，第一目标对象和第二目标对象等是用于区别不同的目标对象，而不是用于描述目标对象的特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请实施例的描述中，除非另有说明，“多个”的含义是指两个或两个以上。例如，多个处理单元是指两个或两个以上的处理单元；多个系统是指两个或两个以上的系统。
图1a为示例性示出的应用场景示意图。
参照图1a，示例性的，编码端可以对待编码图像进行编码，得到码流；然后将码流传输至解码端。解码端获取到码流后，可以对码流进行解码，得到重建图像。
示例性的，解码端在得到重建图像后，可以对重建图像进行后处理，来对重建图像进行增强，得到增强图像，然后输出增强图像，以减小重建图像的视觉失真。
示例性的，针对后处理阶段，本申请提出一种图像处理方法，能够增强重建图像中纹理区域的纹理，且避免非纹理区域产生虚假纹理，以减小重建图像的视觉失真，进而增加图像的真实感，提高图像的主观质量。
图1b为示例性示出的处理过程示意图。
S101，获取重建图像。
示例性的，解码端获取码流后，可以对码流进行解码，得到重建图像。
S102，对重建图像进行图像增强，得到中间图像。
示例性的，为了减少重建图像的视觉失真，可以对重建图像进行图像增强，得到中间图像。
示例性的，可以采用图像增强模型对重建图像进行图像增强。示例性的，图像增强模型可以是GANEF(Generative Adversarial Network Enhancement Filter，生成对抗网络的增强滤波器)，也就是基于生成对抗网络实现的增强滤波器。其中，GAN(生成对抗网络)可以包括生成器和鉴别器；这样，GANEF也包括生成器和鉴别器，在训练GANEF时，可以交替迭代训练生成器和鉴别器。其中，鉴别器的目标是判别出生成器生成的图像和原始图像的真假，生成器的目标是生成接近原始图像的图像，欺骗判别器，通过有效对抗训练，可以让生成器生成尽可能接近原始图像的图像。
示例性的，GANEF的训练过程可以如下：生成多组训练数据，一组训练数据包括：待编码图像和对待编码图像的码流进行解码得到的重建图像。示例性的，可以采用训练数据，轮流交替训练生成器和鉴别器，其中，在训练生成器时，可以固定鉴别器的权重参数；在训练鉴别器时，可以固定生成器的权重参数。
示例性的，训练生成器的过程可以如下：预先，将训练数据中的待编码图像输入至鉴别器，由鉴别器对待编码图像进行前向计算，输出鉴别结果。然后以使鉴别器输出的鉴别结果为“真”的概率接近预设概率为目标，调整鉴别器的权重参数。其中，预设概率可以按照需求设置，本申请对此不作限制。待鉴别器根据输入的待编码图像，输出鉴别结果为“真”的概率与预设概率的差值小于概率阈值时，可以固定鉴别器的权重参数。其中，概率阈值可以根据需求设置，本申请对此不作限制。此时，再将训练数据中的重建图像输入至生成器中，由生成器对重建图像进行前向计算，输出中间图像至鉴别器。接着，一方面，由鉴别器对中间图像进行前向计算，输出鉴别结果；另一方面，基于中间图像和待编码图像，生成损失函数值。随后，再以最大化鉴别器输出的鉴别结果为“真”的概率，且以最小化损失函数值为目标，调整生成器的权重参数。
示例性的，训练鉴别器的过程可以如下：预先，将训练数据中的待编码图像输入至鉴别器，由鉴别器对待编码图像进行前向计算，输出鉴别结果。然后以使鉴别器输出的鉴别结果为“真”的概率接近预设概率为目标，调整鉴别器的权重参数。待鉴别器根据输入的待编码图像，输出鉴别结果为“真”的概率与预设概率的差值小于概率阈值时，再将训练数据中的重建图像输入至生成器中，由生成器对重建图像进行前向计算，输出中间图像至鉴别器。然后，鉴别器对中间图像进行前向计算，输出鉴别结果，再以最大化鉴别器输出的鉴别结果为“假”的概率为目标，调整鉴别器的权重参数。
这样，按照上述方式交替迭代训练GANEF中的生成器和鉴别器，直至鉴别器无法对生成器输出的中间图像和待编码图像进行很好的判别为止。
示例性的，中间图像可以为残差图像，也可以为纹理增强图像。其中，在训练过程中，当生成器输出的中间图像是残差图像时，可以基于残差图像和重建图像，生成纹理增强图像；然后再将纹理增强图像输入至鉴别器，以及根据纹理增强图像和待编码图像计算损失函数值。当生成器输出的中间图像是纹理增强图像时，可以直接将纹理增强图像输入至鉴别器中。
示例性的，可以将残差图像和重建图像相加，生成纹理增强图像。示例性的，可以将残差图像和重建图像进行逐像素点相加，也就将残差图像和重建图像对应位置像素点的像素值进行相加，得到重建图像对应的增强图像。
示例性的，GANEF训练完成后，可以仅采用GANEF中训练后的生成器对重建图像进行图像增强，以得到中间图像。
其中，中间图像的分辨率与重建图像的分辨率相同。
S103，确定中间图像中各区域分别对应的纹理复杂度权重。
一种可能的方式中，在确定中间图像后，可以根据重建图像中各区域的纹理复杂度，来确定中间图像中各区域分别对应的纹理复杂度权重。
一种可能的方式中，在确定中间图像后，可以根据重建图像对应待编码图像中各区域的纹理复杂度，来确定中间图像中各区域分别对应的纹理复杂度权重。
示例性的，纹理复杂度权重与纹理复杂度成正比，也就是说，纹理复杂度越高，纹理复杂度权重越大；纹理复杂度越低，纹理复杂度权重越小。
示例性的，纹理复杂度权重为0到1之间的小数。
S104，依据所述中间图像中各区域分别对应的纹理复杂度权重，对所述中间图像中各区域对应的纹理强度分别进行衰减，得到所述重建图像对应的增强图像。
示例性的，针对中间图像中的每个区域，可以基于该区域对应的纹理复杂度权重，对该区域中各像素点的像素值进行衰减，以调整该区域对应的纹理强度。由于纹理复杂度权重与纹理复杂度成正比，因此纹理强度调整后，中间图像中纹理复杂度越高的区域，对应的纹理强度衰减越小，纹理复杂度越低的区域，对应的纹理增强衰减越大。这样，能够在增强纹理区域(纹理复杂度较高)纹理的同时，避免非纹理区域(纹理复杂度较低)产生虚假条纹。也就是说，本申请依据图像局部区域的纹理复杂程度对GANEF结果进行纹理强度衰减控制，即纹理区域尽可能保持原有GANEF的滤波效果，非纹理区域根据纹理复杂度进行GANEF滤波效果的衰减，局部区域的纹理复杂度越小衰减量越大，从而可以有效去除非纹理区域的虚假纹理。此外，本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
示例性的，图1b的具体实现方式，可以参照图2、图4、图5和图7，以及对应的描述。
图2为示例性示出的处理过程示意图。在图2的实施例中，首先采用GANEF对重建图像进行滤波，得到残差图像；然后基于重建图像中各区域的纹理复杂度，确定残差图像中各区域分别对应的纹理复杂度权重；再根据各区域分别对应的纹理复杂度权重对残差图像进行更新，最后将更新后的残差图像和重建图像相加，得到最终的增强图像。
S201，对重建图像进行图像增强，得到残差图像。
示例性的，解码端获取到码流后，可以对码流进行解码，得到重建图像；然后可以采用图像增强模型对重建图像进行图像增强，得到中间图像。
示例性的，中间图像为残差图像。
图3a为示例性示出的图像增强处理过程示意图。
一种可能的方式中，训练的图像增强模型的输出是残差图像。这样，将重建图像输入至图像增强模型，可以直接得到残差图像，如图3a(1)所示。
一种可能的方式中，训练的图像增强模型的输出是纹理增强图像。这样，将重建图像输入至图像增强模型后，可以得到图像增强模型输出纹理增强图像；然后基于纹理增强图像和重建图像，确定残差图像。可选地，可以对纹理增强图像和重建图像进行逐像素相减，也就说将纹理增强图像和重建图像对应像素点的像素值相减，得到残差图像，如图3a(2)所示。
示例性的，残差图像与重建图像的分辨率相同。
S202，基于重建图像，确定残差图像中各区域分别对应的纹理复杂度权重。
示例性的，可以根据重建图像中各个区域对应的纹理复杂度，来确定残差图像中各区域对应的纹理复杂度权重。示例性的，S202可以包括S2021～S2023：
S2021，按照预设分区规则，将重建图像和残差图像分别划分为N个区域。
示例性的，可以按照预先设置的预设分区规则，将重建图像划分为N个区域；以及按照预设分区规则，将残差图像划分为N个区域。也就是说，重建图像和残差图像的区域划分方式相同，由于重建图像和残差图像的分辨率相同，因此重建图像中的N个区域与残差图像中的N个区域是一一对应的。
示例性的，预设分区规则可以按照需求设置，本申请对此不作限制；其中，N为正整数，根据预设分区规则确定。例如，预设分区规则为：将图像划分为尺寸为w*h的N个区域。其中，w＝W/N1，h＝H/N2，N＝N1*N2，其中，W和H为图像的宽和高，N1和N2为正整数。
需要说明的是，N个区域中的每个区域的尺寸，可以相同，也可以不同，本申请对此不作限制。此外，N个区域中每个区域的形状可以相同，也可以不同，本申请对此不作限制。且本申请对N个区域的形状均不作限制。
S2022，分别确定重建图像中N个区域的纹理复杂度。
以下确定重建图像中第i(i为1～N之间的整数，i的取值可以为于1、N)个区域的纹理复杂度为例进行示例性说明。
示例性的，针对重建图像的第i个区域，可以根据第i个区域所包含像素点的像素值，确定第i个区域的纹理复杂度。
一种可能的方式中，可以基于重建图像中第i个区域的共生矩阵，确定重建图像的第i个区域的纹理复杂度。示例性的，可以根据重建图像中第i个区域所包含像素点的像素值，确定重建图像中第i个区域的共生矩阵。然后提取重建图像中第i个区域的共生矩阵对应的特征量(如能量、对比度、熵、逆方差等)，再依据重建图像中第i个区域的共生矩阵的特征量，确定重建图像中第i个区域的图像纹理复杂度。例如，将重建图像中第i个区域的共生矩阵的特征量，作为重建图像中第i个区域的纹理复杂度。
一种可能方式中，可以基于重建图像中第i个区域的边缘比例，确定重建图像中第i个区域的纹理复杂度。示例性的，可以根据重建图像的第i个区域所包含像素点的像素值，计算重建图像中第i个区域内每个像素点对应的梯度强度。然后将重建图像中第i个区域内梯度强度大于梯度强度阈值的像素点的占比，作为重建图像的第i个区域的纹理复杂度。其中，梯度强度阈值可以按照需求设置，本申请对此不作限制。
应当理解的是，还可以采用其他方式如根据重建图像中第i个区域的灰度直方图分 布，确定重建图像中第i个区域的纹理复杂度，本申请对此不作限制。
这样，按照上述方式，可以确定重建图像中N个区域的纹理复杂度。
S2023，基于重建图像中N个区域的纹理复杂度，确定残差图像中N个区域分别对应的纹理复杂度权重。
以下确定残差图像中第i(i为1～N之间的整数，i可以等于1和N)个区域对应的纹理复杂度权重为了进行示例性说明。
一种可能的方式中，可以将重建图像中第i个区域的纹理复杂度，作为残差图像中第i个区域对应的纹理复杂度权重。
一种可能的方式中，可以按照预设映射规则，将重建图像中第i个区域的纹理复杂度进行映射，得到残差图像中第i个区域对应的纹理复杂度权重。示例性的，预设映射规则可以按照需求设置，如归一化，本申请对此不作限制。例如，可以对重建图像中第i个区域的纹理复杂度进行归一化处理，得到残差图像中第i个区域对应的纹理复杂度权重。
示例性的，纹理复杂度权重可以是0至1之间的小数。
示例性的，纹理复杂度权重与纹理复杂度成正比，也就是说，纹理复杂度越高，纹理复杂度权重越大；纹理复杂度越低，纹理复杂度权重越小。
示例性的，在得到残差图像中N个区域对应的纹理复杂度权重后，可以依据残差图像中各区域对应的纹理复杂度权重，对残差图像中各区域对应的纹理强度进行衰减，来对重建图像进行图像增强，得到重建图像对应的增强图像，可以参照S203～S204：
S203，基于残差图像中各个区域对应的纹理复杂度权重，对残差图像中各个区域进行残差更新，得到残差更新图像。
示例性的，可以基于残差图像的N个区域对应的纹理复杂度权重，分别对残差图像中N个区域进行残差更新，得到残差更新图像。
一种可能方式中，可以通过将残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，来对残差图像进行残差更新，得到残差更新图像。
以下对残差图像中第i(i为1～N之间的整数，i可以等于1和N)个区域进行残差更新为例进行示例性说明。
示例性的，假设残差图像中第i个区域对应的纹理增强权重为ratio_i，针对第i个区域的一个像素点(k,j)，可以采用该像素点(k,j)对应的像素值R1(k,j)与ratio_i相乘，对该像素点(k,j)的像素值进行更新，得到新的像素值R2(k,j)＝R1(k,j)*ratio_i。其中，(k，j)为残差图像的像素点坐标整数索引，k，j分别表示水平、竖直方向坐标索引，残差图像左上角像素索引为(0,0)。也就是说，残差图像中各像素点的像素值均更新后，即可以得到残差更新图像，残差更新图像中像素点的像素值为R2(k,j)。
示例性的，残差更新图像和重建图像的分辨率相同。
由于，纹理复杂度权重与纹理复杂度成正比，这样，残差图像纹理复杂度越高的区域，对应的纹理强度衰减越小，纹理复杂度越低的区域，对应的纹理增强衰减越大。
S204，将重建图像和残差更新图像进行相加，得到增强图像。
示例性的，可以将重建图像和残差更新图像对应位置像素点的像素值进行相加，得 到重建图像对应的增强图像。
图3b为示例性示出的处理过程示意图。
参照图3b，示例性的，A1为重建图像，采用GANEF对重建图像进行滤波后，得到纹理增强图像，如A2所示。然后可以采用纹理增强图像A2减去重建图像A1，得到残差图像，如A4所示
示例性的，可以计算重建图像A1中各像素点的梯度，然后基于重建图像A1中各像素点的梯度，对重建图像进行二值化，得到二值图像，如A3所示。然后将二值图像A3划分为N个区域，针对每个区域，根据该区域中黑色像素点的占比，确定该区域对应的纹理复杂度；进而可以得到二值图像A3中N个区域分别对应的纹理复杂度。然后根据二值图像中N个区域分别对应的纹理复杂度，确定残差图像A4中N个区域分别对应的纹理复杂度权重。再基于残差图像A4中N个区域分别对应的纹理复杂度权重，对残差图像A4进行残差更新，得到残差更新图像，如A5所示。
示例性的，可以将重建图像A1与残差更新图像A5相加，得到增强图像A6。
图3c为示例性示出的图像增强效果示意图。
参照图3c，示例性的，图3c(1)是图3b的纹理增强图像A2中局部区域的示意图，图3c(2)是图3b的增强图像A6中局部区域示意图。
示例性的，图3c(1)和图3c(2)中的路面为非纹理区域，通过图3c(1)和图3c(2)中椭圆区域，以及比对图3c(1)和图3c(2)中矩形区域可知，本申请得到的增强图像中非纹理区域没有虚假条纹。
这样，通过根据重建图像纹理复杂度，将纹理复杂度权重控制在0到1范围，来对重建图像中各个区域对应的纹理强度进行衰减，能够保留中间图像中纹理区域的纹理，同时对非纹理区域中的纹理进行衰减，进而在增强重建图像中纹理区域(纹理复杂度较高)纹理的同时，避免重建图像中非纹理区域(纹理复杂度较低)产生虚假条纹；从而降低重建图像的视觉失真。且本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
此外，相对于现有技术采用两个模型来进行后处理而言，本申请仅使用一个模型，降低了计算复杂度。
图4为示例性示出的处理过程示意图。在图4的实施例中，首先采用GANEF对重建图像进行滤波，得到残差图像；然后基于重建图像对应待编码图像中各区域的纹理复杂度，确定残差图像各区域分别对应的纹理复杂度权重；再根据各区域分别对应的纹理复杂度权重对残差图像进行更新，最后将更新后的残差图像和重建图像相加，得到最终的增强图像。
S401，对码流进行解码，得到重建图像和待编码图像中各区域对应的纹理复杂度权重。
示例性的，编码端可以基于待编码图像，生成待编码图像中各区域对应的纹理复杂度权重，可以包括S4011～S4013：
S4011，按照预设分区规则，将待编码图像划分为N个区域。
S4012，分别确定待编码图像中N个区域的纹理复杂度。
示例性的，S4011～S4012，可以参照上述S2021～S2022的描述，在此不再赘述。
S4013，基于待编码图像中N个区域的纹理复杂度，确定待编码图像中N个区域分别对应的纹理复杂度权重。
以下确定待编码图像中第i(i为1～N之间的整数，i的取值可以为1、N)个区域对应的纹理复杂度权重为例进行示例性说明。
一种可能的方式中，可以将待编码图像中第i个区域的纹理复杂度，作为待编码图像中第i个区域对应的纹理复杂度权重。
一种可能的方式中，可以按照预设映射规则，将待编码图像中第i个区域的纹理复杂度进行映射，得到待编码图像中第i个区域对应的纹理复杂度权重。例如，可以对待编码图像中第i个区域的纹理复杂度进行归一化处理，得到待编码图像中第i个区域对应的纹理复杂度权重。
示例性的，纹理复杂度权重可以是0至1之间的小数。
示例性的，纹理复杂度权重与纹理复杂度成正比，也就是说，纹理复杂度越高，纹理复杂度权重越大；纹理复杂度越低，纹理复杂度权重越小。
示例性的，编码端可以对待编码图像进行编码，得到对应的码流；以及可以对待编码图像中N个区域对应的纹理复杂度权重进行编码，得到对应的码流。然后可以将对待编码图像中N个区域分别对应的纹理复杂度权重进行编码得到的码流和对待编码图像进行编码得到的码流，发送至解码端。这样，解码端获取到码流后，从码流中解码出重建图像和待编码图像中N个区域分别对应的纹理复杂度权重。
S402，对重建图像进行图像增强，得到残差图像。
示例性的，解码端可以采用图像增强模型对重建图像进行图像增强，得到中间图像。示例性的，中间图像为残差图像。
其中，S402可以参照上述S201的描述，在此不再赘述。
示例性的，残差图像的分辨率，与重建图像的分辨率以及待编码图像的分辨率均相同。
S403，依据待编码图像中各区域对应的纹理复杂度权重，生成残差图像中各区域分别对应的纹理复杂度权重。
示例性的，在得到残差图像后，可以按照预先设置的预设分区规则，将残差图像划分为N个区域。其中，对残差图像的区域划分方式与对待编码图像的区域划分方式相同，由于残差图像与待编码图像的分辨率相同，因此残差图像中的N个区域与待编码图像中的N个区域是一一对应的。进而，可以将待编码图像的第i个区域对应的纹理复杂度权重，作为残差图像的第i个区域对应的纹理复杂度权重。这样，可以确定残差图像中N个区域分别对应的纹理复杂度权重。
示例性的，纹理复杂度权重可以是0至1之间的小数。
示例性的，在得到残差图像中N个区域对应的纹理复杂度权重后，可以依据残差图像中各区域分别对应的纹理复杂度权重，调整残差图像中各区域分别对应的纹理强度， 得到重建图像对应的增强图像，可以参照S404～S405：
S404，基于残差图像中各区域分别对应的纹理复杂度权重对残差图像进行残差更新，得到残差更新图像。
S405，将重建图像和残差更新图像进行相加，得到增强图像。
示例性的，S404～S405，可以参照上文S203～S204的描述，在此不再赘述。
这样，通过根据待编码图像纹理复杂度，将纹理复杂度权重控制在0到1范围，来调整重建图像中各个区域对应的纹理增强，能够在增强重建图像中纹理区域(纹理复杂度较高)纹理的同时，避免重建图像中非纹理区域(纹理复杂度较低)产生虚假条纹；进而降低重建图像的视觉失真。且本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
此外，相对于现有技术采用两个模型来进行后处理而言，本申请仅使用一个模型，降低了计算复杂度。
再次，相对于根据重建图像确定的纹理复杂度权重而言，根据待编码图像确定的纹理复杂度权重更准确，进而能够更准确的对控制图像纹理强度的衰减，从而进一步提高图像质量。
图5为示例性示出的处理过程示意图。在图5的实施例中，首先采用非生成对抗网络(非GANEF)和生成对抗网络(GANEF)对重建图像进行滤波，分别得到基础保真图像和纹理增强图像，再基于重建图像或基础保真图像中各区域对应的纹理复杂度，确定基础保真图像和纹理增强图像分别对应的加权计算因子，最后基于加权计算因子将基础保真图像和纹理增强图像进行加权融合，得到最终的增强图像。
S501，对重建图像进行图像增强，得到纹理增强图像。
示例性的，解码端获取到码流后，可以对码流进行解码，得到重建图像；然后可以采用图像增强模型对重建图像进行图像增强，得到中间图像。示例性的，中间图像为纹理增强图像。
图6为示例性示出的图像增强处理过程示意图。
一种可能的方式中，训练的图像增强模型的输出是残差图像。这样，将重建图像输入至图像增强模型，可以得到图像增强模型输出的残差图像，然后可以根据残差图像和重建图像，确定纹理增强图像。可选的，可以将残差图像和重建图像对应像素点的像素值相加，得到纹理增强图像，如图6(1)所示。
一种可能的方式中，训练的图像增强模型的输出是纹理增强图像。这样，将重建图像输入至图像增强模型后，可以直接得到纹理增强图像，如图6(2)所示。
示例性的，纹理增强图像与重建图像的分辨率相同。
S502，对重建图像进行图像保真，得到基础保真图像。
示例性的，可以采用预设的图像保真模型可以重建图像进行图像保真，得到基础保真图像。
示例性的，图像保真模型采用的网络可以是非生成对抗网络，如卷积神经网络等，本申请对此不作限制。
示例性的，图像保真模型的训练过程可以如下：收集多组训练数据，一组训练数据包括待编码图像和对待编码图像的码流进行解码得到的重建图像。针对一组训练数据，将一组训练数据输入至图像保真模型中，由图像保真模型对重建图像进行前向计算，输出基础保真图像。基于图像保真模型输出的基础保真图像和训练数据中的待编码图像计算损失函数值，以最小化损失函数值为目标，调整图像保真模型的权重参数。然后可以根据上述方式，采用收集的多组训练数据对图像保真模型进行训练，直至图像保真模型的训练次数等于预设训练次数，或图像保真模型的损失函数值小于或等于损失函数阈值，或图像保真模型的效果满足预设效果条件，停止对图像保真模型的训练，得到训练后的图像保真模型。示例性的，预设训练次数、损失函数阈值和预设效果条件，均可以按照需求设置，本申请对此不作限制。
需要说明的是，本申请不限制S501与S502的执行顺序。
示例性的，纹理增强图像和基础保真图像，均与重建图像的分辨率相同。
S503，基于基础保真图像，生成纹理增强图像中各区域分别对应的纹理复杂度权重。
示例性的，S503可以包括S5031a～S5034a：
S5031a，按照预设分区规则，将基础保真图像和纹理增强图像分别划分为N个区域。
S5032a，分别确定基础保真图像中N个区域的纹理复杂度。
S5033a，基于基础保真图像中N个区域的纹理复杂度，确定纹理增强图像中N个区域分别对应的纹理复杂度权重。
示例性的，S5031a～S5033a可以参照上文S2021～S2023的描述，在此不再赘述。
一种可能的方式中，可以基于重建图像，生成纹理增强图像中各区域对应的纹理复杂度权重，可以参照S5031b～S5033b：
S5031b，按照预设分区规则，将重建图像和纹理增强图像分别划分为N个区域。
S5032b，分别确定重建图像中N个区域的纹理复杂度。
S5033b，基于重建图像中N个区域的纹理复杂度，确定纹理增强图像中N个区域分别对应的纹理复杂度权重。
示例性的，S5031b～S5033b可以参照上文S2021～S2023的描述，在此不再赘述。
S504，根据纹理增强图像中各区域分别对应的纹理复杂度权重，对基础保真图像和纹理增强图像进行加权融合，得到增强图像。
示例性的，将纹理增强图像和基础保真图像均划分为N个区域后，纹理增强图像与基础保真图像的区域是一一对应的。这样，可以将纹理增强图像中各区域，与基础保真图像中对应区域进行加权融合，可以得到增强图像。
示例性的，S504可以包括S5041～S5042：
S5041，依据纹理增强图像中N个区域分别对应的纹理复杂度权重，确定纹理增强图像中N个区域分别对应加权计算权重和基础保真图像中N个区域分别对应的加权计算权重。
示例性的，可以将纹理增强图像中N个区域对应的纹理复杂度权重，作为纹理增强图像中N个区域的加权计算权重。以及将1与纹理增强图像中N个区域对应的纹理复杂度权重之间的差值，作为基础保真图像中N个区域的加权计算权重。
例如，针对第i个区域，纹理增强图像的第i个区域对应的纹理复杂度权重为ratio_i，则纹理增强图像的第i个区域的加权计算权重可以为ratio_i，基础保真图像的第i个区域的加权计算权重为1-ratio_i。
S5042，依据纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，对纹理增强图像中N个区域和基础保真图像中N个区进行加权计算，得到增强图像。
示例性的，针对第i个区域，根据纹理增强图像中第i个区域的加权计算权重和基础保真图像中第i个区域的加权计算权重，对纹理增强图像中第i个区域的各像素点和基础保真图像中第i个区域的各像素点进行加权计算，得到第i个区域的增强图像。
示例性的，可以将纹理增强图像中N个区域分别对应的加权计算权重，与纹理增强图像中N个区域分别相乘，得到第一乘积；将基础保真图像中N个区域分别对应的加权计算权重，与基础保真图像中N个区域分别相乘，得到第二乘积；将第一乘积和第二乘积相加，得到重建图像对应的增强图像。
例如，纹理增强图像中第i个区域的加权计算权重为ratio_i，纹理增强图像中第i个区域的一个像素点(j,k)的像素值为E1(j,k)；基础保真图像中第i个区域的加权计算权重为1-ratio_i，基础保真图像中第i个区域的一个像素点(j,k)的像素值为E2(j,k)，则对重建图像的第i个区域的像素点(j,k)进行图像增强后的像素值R(i,j)＝ratio_i*E1(i,j)+(1-ratio_i)*E2(i,j)。
这样，通过根据重建图像纹理复杂度，将纹理复杂度权重控制在0到1范围来，调整重建图像中各个区域对应的纹理增强，能够在增强重建图像中纹理区域(纹理复杂度较高)纹理的同时，避免重建图像中非纹理区域(纹理复杂度较低)产生虚假条纹；进而降低重建图像的视觉失真。且本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
图7为示例性示出的处理过程示意图。在图7的实施例中，首先采用非生成对抗网络(非GANEF)和生成对抗网络(GANEF)对重建图像进行滤波，分别得到基础保真图像和纹理增强图像，再基于重建图像对应的待编码图像或基础保真图像中各区域对应的纹理复杂度，确定基础保真图像和纹理增强图像分别对应的加权计算因子，最后基于加权计算因子将基础保真图像和纹理增强图像进行加权融合，得到最终的增强图像。
S701，对码流进行解码，得到重建图像和待编码图像中各区域对应的纹理复杂度权重。
示例性的，编码端可以基于待编码图像，生成纹理复杂度权重，可以包括S7011～S7013：
S7011，按照预设分区规则，将待编码图像划分为N个区域。
S7012，分别确定待编码图像中N个区域的纹理复杂度。
S7013，基于待编码图像中N个区域的纹理复杂度，确定待编码图像中N个区域分别对应的纹理复杂度权重。
示例性的，S7011～S7013，可以参照上述S2021～S2023的描述，在此不再赘述。
S702，对重建图像进行图像增强，得到纹理增强图像。
示例性的，S702可以参照上述S502的描述，在此不再赘述。
S703，依据待编码图像中各区域对应的纹理复杂度权重，确定纹理增强图像中各区域分别对应的纹理增强权重。
示例性的，在得到纹理增强图像后，可以按照预先设置的预设分区规则，将纹理增强图像划分为N个区域。其中，对纹理增强图像的区域划分方式与对待编码图像的区域划分方式相同，且纹理增强图像与待编码图像的分辨率相同，因此纹理增强图像中的区域与待编码图像中的区域是一一对应的。进而，可以将待编码图像的第i个区域对应的纹理复杂度权重，作为纹理增强图像的第i个区域对应的纹理复杂度权重。这样，可以确定纹理增强图像中N个区域对应的纹理复杂度权重。
示例性的，纹理复杂度权重可以是0至1之间的小数。
示例性的，在得到纹理增强图像中N个区域对应的纹理复杂度权重后，可以依据纹理增强图像中各区域对应的纹理复杂度权重，调整纹理增强图像中各区域对应的纹理强度，得到重建图像对应的增强图像，可以参照S704～S705：
S704，对重建图像进行图像保真，得到基础保真图像。
示例性的，S704可以参照上述S502的描述，在此不再赘述。
示例性的，本申请不限制S704和S702的执行顺序。
S705，根据纹理增强图像中各区域对应的纹理复杂度权重，对基础保真图像和纹理增强图像进行加权融合，得到增强图像。
示例性的，S705可以参照上述S504的描述，在此不再赘述。
这样，通过根据待编码图像纹理复杂度，将纹理复杂度权重控制在0到1范围，来调整重建图像中各个区域对应的纹理增强，能够在增强重建图像中纹理区域(纹理复杂度较高)纹理的同时，避免重建图像中非纹理区域(纹理复杂度较低)产生虚假条纹；进而降低重建图像的视觉失真。且本申请将纹理复杂度权重控制在0到1范围，使得相邻区域对应的纹理更柔和，能够避免相邻区域增强的纹理效果不一致的问题。
再次，相对于根据重建图像确定的纹理复杂度权重而言，根据待编码图像确定的纹理复杂度权重更准确，进而能够更准确的对控制图像纹理强度的衰减，从而进一步提高图像质量。
图8为示例性示出的图像处理装置示意图。
参照图8，示例性的，该图像处理装置包括：图像获取模块801、图像增强模块802、纹理权重确定模块803和纹理衰减模块804，其中：
图像获取模块801，用于获取重建图像；
图像增强模块802，用于对重建图像进行图像增强，得到中间图像；
纹理权重确定模块803，用于确定中间图像中各区域分别对应的纹理复杂度权重，纹理复杂度权重为0至1之间的数；
纹理衰减模块804，用于依据中间图像中各区域分别对应的纹理复杂度权重，对中间图像中各区域对应的纹理强度分别进行衰减，得到重建图像对应的增强图像。
示例性的，中间图像为残差图像；
纹理衰减模块804，包括：
残差更新模块，用于将残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，得到残差更新图像；
图像生成模块，用于根据残差更新图像和重建图像，生成增强图像。
示例性的，图像生成模块，具体用于将残差更新图像和重建图像相加，得到增强图像。
示例性的，中间图像为纹理增强图像，装置还包括：
图像保真模块，用于对重建图像进行图像保真，得到基础保真图像。
示例性的，纹理权重确定模块803，具体用于按照预设分区规则，将基础保真图像和纹理增强图像分别划分为N个区域，N为正整数；分别确定基础保真图像中N个区域的纹理复杂度；基于基础保真图像中N个区域的纹理复杂度，确定纹理增强图像中N个区域分别对应的纹理复杂度权重。
示例性的，纹理衰减模块804，包括：
加权权重确定模块，用于依据纹理增强图像中N个区域对应的纹理复杂度权重，确定纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重；
加权计算模块，用于依据纹理增强图像中N个区域分别对应的加权计算权重和基础保真图像中N个区域分别对应的加权计算权重，对纹理增强图像中N个区域和基础保真图像中N个区域进行加权计算，得到重建图像对应的增强图像。
示例性的，加权计算模块，具体用于将纹理增强图像中N个区域分别对应的加权计算权重，与纹理增强图像中N个区域分别相乘，得到第一乘积；将基础保真图像中N个区域分别对应的加权计算权重，与基础保真图像中N个区域分别相乘，得到第二乘积；将第一乘积和第二乘积相加，得到重建图像对应的增强图像。
示例性的，加权权重确定模块，具体用于将纹理增强图像中N个区域分别对应的纹理复杂度权重，确定为纹理增强图像中N个区域分别对应的加权计算权重；将1与纹理增强图像中N个区域分别对应的纹理复杂度权重的差值，确定为基础保真图像中N个区域分别对应的加权计算权重。
示例性的，纹理权重确定模块803，具体用于从接收的码流中解码出中间图像中N个区域分别对应的纹理复杂度权重，N为正整数。
示例性的，纹理权重确定模块803，具体用于按照预设分区规则，将重建图像和中间图像分别划分为N个区域，N为正整数；分别确定重建图像中N个区域的纹理复杂度；基于重建图像中N个区域的纹理复杂度，确定中间图像中N个区域分别对应的纹理复杂度权重。
一个示例中，图9示出了本申请实施例的一种装置900的示意性框图装置900可包括：处理器901和收发器/收发管脚902，可选地，还包括存储器903。
装置900的各个组件通过总线904耦合在一起，其中总线904除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图 中将各种总线都称为总线904。
可选地，存储器903可以用于前述方法实施例中的指令。该处理器901可用于执行存储器903中的指令，并控制接收管脚接收信号，以及控制发送管脚发送信号。
装置900可以是上述方法实施例中的电子设备或电子设备的芯片。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例还提供一种计算机可读存储介质，该计算机可读存储介质中存储有计算机指令，当该计算机指令在电子设备上运行时，使得电子设备执行上述相关方法步骤实现上述实施例中的图像处理方法。
本实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的图像处理方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的图像处理方法。
其中，本实施例提供的电子设备、计算机可读存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
本申请各个实施例的任意内容，以及同一实施例的任意内容，均可以自由组合。对上述内容的任意组合均在本申请的范围之内。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。
结合本申请实施例公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机可读存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (24)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    获取重建图像；
    对所述重建图像进行图像增强，得到中间图像；
    确定所述中间图像中各区域分别对应的纹理复杂度权重，所述纹理复杂度权重为0至1之间的数；
    依据所述中间图像中各区域分别对应的纹理复杂度权重，对所述中间图像中各区域对应的纹理强度分别进行衰减，得到所述重建图像对应的增强图像。
  2. 根据权利要求1所述的方法，其特征在于，所述中间图像为残差图像；
    所述依据所述中间图像中各区域分别对应的纹理复杂度权重，对所述中间图像中各区域对应的纹理强度分别进行衰减，得到所述重建图像对应的增强图像，包括：
    将所述残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，得到残差更新图像；
    根据所述残差更新图像和所述重建图像，生成所述增强图像。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述残差更新图像和所述重建图像，生成所述增强图像，包括：
    将所述残差更新图像和所述重建图像相加，得到所述增强图像。
  4. 根据权利要求1所述的方法，其特征在于，所述中间图像为纹理增强图像，所述方法还包括：
    对所述重建图像进行图像保真，得到基础保真图像。
  5. 根据权利要求4所述的方法，其特征在于，所述确定所述中间图像中各区域对应的纹理复杂度权重，包括：
    按照预设分区规则，将所述基础保真图像和所述纹理增强图像分别划分为N个区域，N为正整数；
    分别确定所述基础保真图像中N个区域的纹理复杂度；
    基于所述基础保真图像中N个区域的纹理复杂度，确定所述纹理增强图像中N个区域分别对应的纹理复杂度权重。
  6. 根据权利要求5所述的方法，其特征在于，所述依据所述中间图像中各区域分别对应的纹理复杂度权重，对所述中间图像中各区域对应的纹理强度分别进行衰减，得到所述重建图像对应的增强图像，包括：
    依据所述纹理增强图像中N个区域对应的纹理复杂度权重，确定所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计 算权重；
    依据所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计算权重，对所述纹理增强图像中N个区域和所述基础保真图像中N个区域进行加权计算，得到所述重建图像对应的增强图像。
  7. 根据权利要求6所述的方法，其特征在于，所述依据所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计算权重，对所述纹理增强图像中N个区域和所述基础保真图像中N个区域进行加权计算，得到所述重建图像对应的增强图像，包括：
    将所述纹理增强图像中N个区域分别对应的加权计算权重，与所述纹理增强图像中N个区域分别相乘，得到第一乘积；
    将所述基础保真图像中N个区域分别对应的加权计算权重，与所述基础保真图像中N个区域分别相乘，得到第二乘积；
    将所述第一乘积和所述第二乘积相加，得到所述重建图像对应的增强图像。
  8. 根据权利要求6所述的方法，其特征在于，所述依据所述纹理增强图像中N个区域分别对应的纹理复杂度权重，确定所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计算权重，包括：
    将所述纹理增强图像中N个区域分别对应的纹理复杂度权重，确定为所述纹理增强图像中N个区域分别对应的加权计算权重；
    将1与所述纹理增强图像中N个区域分别对应的纹理复杂度权重的差值，确定为所述基础保真图像中N个区域分别对应的加权计算权重。
  9. 根据权利要求1至8任一项所述的方法，其特征在于，所述确定所述中间图像中各区域分别对应的纹理复杂度权重，包括：
    从接收的码流中解码出所述中间图像中N个区域分别对应的纹理复杂度权重，N为正整数。
  10. 根据权利要求1或2或3或4或6或7或8或9所述的方法，其特征在于，所述确定所述中间图像中各区域分别对应的纹理复杂度权重，包括：
    按照预设分区规则，将所述重建图像和所述中间图像分别划分为N个区域，N为正整数；
    分别确定所述重建图像中N个区域的纹理复杂度；
    基于所述重建图像中N个区域的纹理复杂度，确定所述中间图像中N个区域分别对应的纹理复杂度权重。
  11. 一种图像处理装置，其特征在于，所述装置包括：
    图像获取模块，用于获取重建图像；
    图像增强模块，用于对所述重建图像进行图像增强，得到中间图像；
    纹理权重确定模块，用于确定所述中间图像中各区域分别对应的纹理复杂度权重，所述纹理复杂度权重为0至1之间的数；
    纹理衰减模块，用于依据所述中间图像中各区域分别对应的纹理复杂度权重，对所述中间图像中各区域对应的纹理强度分别进行衰减，得到所述重建图像对应的增强图像。
  12. 根据权利要求11所述的装置，其特征在于，所述中间图像为残差图像；
    所述纹理衰减模块，包括：
    残差更新模块，用于将所述残差图像中各像素点的像素值，分别与各像素点所属区域对应的纹理复杂度权重相乘，得到残差更新图像；
    图像生成模块，用于根据所述残差更新图像和所述重建图像，生成所述增强图像。
  13. 根据权利要求12所述的装置，其特征在于，
    所述图像生成模块，具体用于将所述残差更新图像和所述重建图像相加，得到所述增强图像。
  14. 根据权利要求11所述的装置，其特征在于，所述中间图像为纹理增强图像，所述装置还包括：
    图像保真模块，用于对所述重建图像进行图像保真，得到基础保真图像。
  15. 根据权利要求14所述的装置，其特征在于，
    所述纹理权重确定模块，具体用于按照预设分区规则，将所述基础保真图像和所述纹理增强图像分别划分为N个区域，N为正整数；分别确定所述基础保真图像中N个区域的纹理复杂度；基于所述基础保真图像中N个区域的纹理复杂度，确定所述纹理增强图像中N个区域分别对应的纹理复杂度权重。
  16. 根据权利要求15所述的装置，其特征在于，所述纹理衰减模块，包括：
    加权权重确定模块，用于依据所述纹理增强图像中N个区域对应的纹理复杂度权重，确定所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计算权重；
    加权计算模块，用于依据所述纹理增强图像中N个区域分别对应的加权计算权重和所述基础保真图像中N个区域分别对应的加权计算权重，对所述纹理增强图像中N个区域和所述基础保真图像中N个区域进行加权计算，得到所述重建图像对应的增强图像。
  17. 根据权利要求16所述的装置，其特征在于，
    所述加权计算模块，具体用于将所述纹理增强图像中N个区域分别对应的加权计算权重，与所述纹理增强图像中N个区域分别相乘，得到第一乘积；将所述基础保真图像中N个区域分别对应的加权计算权重，与所述基础保真图像中N个区域分别相乘，得到 第二乘积；将所述第一乘积和所述第二乘积相加，得到所述重建图像对应的增强图像。
  18. 根据权利要求16所述的装置，其特征在于，
    所述加权权重确定模块，具体用于将所述纹理增强图像中N个区域分别对应的纹理复杂度权重，确定为所述纹理增强图像中N个区域分别对应的加权计算权重；将1与所述纹理增强图像中N个区域分别对应的纹理复杂度权重的差值，确定为所述基础保真图像中N个区域分别对应的加权计算权重。
  19. 根据权利要求11至18任一项所述的装置，其特征在于，
    所述纹理权重确定模块，具体用于从接收的码流中解码出所述中间图像中N个区域分别对应的纹理复杂度权重，N为正整数。
  20. 根据权利要求11或12或13或14或16或17或18或19所述的装置，其特征在于，
    所述纹理权重确定模块，具体用于按照预设分区规则，将所述重建图像和所述中间图像分别划分为N个区域，N为正整数；分别确定所述重建图像中N个区域的纹理复杂度；基于所述重建图像中N个区域的纹理复杂度，确定所述中间图像中N个区域分别对应的纹理复杂度权重。
  21. 一种电子设备，其特征在于，包括：
    存储器和处理器，所述存储器与所述处理器耦合；
    所述存储器存储有程序指令，当所述程序指令由所述处理器执行时，使得所述电子设备执行权利要求1至权利要求10中任一项所述的图像处理方法。
  22. 一种芯片，其特征在于，包括一个或多个接口电路和一个或多个处理器；所述接口电路用于从电子设备的存储器接收信号，并向所述处理器发送所述信号，所述信号包括存储器中存储的计算机指令；当所述处理器执行所述计算机指令时，使得所述电子设备执行权利要求1至权利要求10中任一项所述的图像处理方法。
  23. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，当所述计算机程序运行在计算机或处理器上时，使得所述计算机或所述处理器执行如权利要求1至权利要求10中任一项所述的图像处理方法。
  24. 一种计算机程序产品，其特征在于，所述计算机程序产品包含软件程序，当所述软件程序被计算机或处理器执行时，使得权利要求1至权利要求10任一项所述的方法的步骤被执行。
PCT/CN2022/131594 2021-12-10 2022-11-14 图像处理方法、装置及电子设备 WO2023103715A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111511051.7A CN114298922A (zh) 2021-12-10 2021-12-10 图像处理方法、装置及电子设备
CN202111511051.7 2021-12-10

Publications (1)

Publication Number Publication Date
WO2023103715A1 true WO2023103715A1 (zh) 2023-06-15

Family

ID=80967411

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/131594 WO2023103715A1 (zh) 2021-12-10 2022-11-14 图像处理方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN114298922A (zh)
WO (1) WO2023103715A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116567286A (zh) * 2023-07-10 2023-08-08 武汉幻忆信息科技有限公司 一种基于人工智能的在线直播视频处理方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114298922A (zh) * 2021-12-10 2022-04-08 华为技术有限公司 图像处理方法、装置及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109982082A (zh) * 2019-05-05 2019-07-05 山东大学深圳研究院 一种基于局部纹理特性的hevc多失真准则率失真优化方法
CN110246093A (zh) * 2019-05-05 2019-09-17 北京大学 一种解码图像增强方法
CN112634278A (zh) * 2020-10-30 2021-04-09 上海大学 基于超像素的恰可察觉失真模型
CN113744235A (zh) * 2021-08-30 2021-12-03 河南工业大学 一种基于sc-gan的膝部mri重建技术
CN114298922A (zh) * 2021-12-10 2022-04-08 华为技术有限公司 图像处理方法、装置及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109982082A (zh) * 2019-05-05 2019-07-05 山东大学深圳研究院 一种基于局部纹理特性的hevc多失真准则率失真优化方法
CN110246093A (zh) * 2019-05-05 2019-09-17 北京大学 一种解码图像增强方法
CN112634278A (zh) * 2020-10-30 2021-04-09 上海大学 基于超像素的恰可察觉失真模型
CN113744235A (zh) * 2021-08-30 2021-12-03 河南工业大学 一种基于sc-gan的膝部mri重建技术
CN114298922A (zh) * 2021-12-10 2022-04-08 华为技术有限公司 图像处理方法、装置及电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116567286A (zh) * 2023-07-10 2023-08-08 武汉幻忆信息科技有限公司 一种基于人工智能的在线直播视频处理方法及系统
CN116567286B (zh) * 2023-07-10 2023-09-22 武汉幻忆信息科技有限公司 一种基于人工智能的在线直播视频处理方法及系统

Also Published As

Publication number Publication date
CN114298922A (zh) 2022-04-08

Similar Documents

Publication Publication Date Title
WO2023103715A1 (zh) 图像处理方法、装置及电子设备
US9736455B2 (en) Method and apparatus for downscaling depth data for view plus depth data compression
CN110689599B (zh) 基于非局部增强的生成对抗网络的3d视觉显著性预测方法
CN116109798B (zh) 图像数据处理方法、装置、设备及介质
CN111611934A (zh) 一种人脸检测模型生成和人脸检测方法、装置及设备
WO2022183500A1 (zh) 基于投影的点云质量评价方法、装置、设备和存储介质
WO2021248966A1 (zh) 点云质量评估方法、编码器、解码器及存储介质
CN111340866A (zh) 深度图像生成方法、装置及存储介质
CN113763269B (zh) 一种用于双目图像的立体匹配方法
CN112672168B (zh) 基于图卷积的点云压缩方法及装置
CN115330940B (zh) 一种三维重建方法、装置、设备和介质
CN111046893B (zh) 图像相似性确定方法和装置、图像处理方法和装置
CN112017228A (zh) 一种对物体三维重建的方法及相关设备
CN111488810A (zh) 人脸识别方法、装置、终端设备及计算机可读介质
CN114782564B (zh) 一种点云的压缩方法、装置、电子设备及存储介质
CN116612468A (zh) 基于多模态融合与深度注意力机制的三维目标检测方法
CN113038123A (zh) 无参考全景视频质量评价方法、系统、终端及介质
CN112927348B (zh) 一种基于多视点rgbd相机高分辨率人体三维重建方法
CN115965961B (zh) 局部到全局的多模态融合方法、系统、设备及存储介质
CN107045713A (zh) 基于census立体匹配的低照度图像增强方法
CN108810511B (zh) 一种基于视点一致性的多视点压缩深度视频增强方法
WO2022219384A1 (en) Method and apparatus for generating point cloud encoder,method and apparatus for generating point cloud data, electronic device and computer storage medium
WO2023179710A1 (zh) 编码方法及终端
CN115171030B (zh) 基于多级特征融合的多模态图像分割方法、系统及器件
CN113643421B (zh) 图像的三维重建方法和三维重建装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22903138

Country of ref document: EP

Kind code of ref document: A1