WO2021082819A1 - 一种图像生成方法、装置及电子设备 - Google Patents

一种图像生成方法、装置及电子设备 Download PDF

Info

Publication number
WO2021082819A1
WO2021082819A1 PCT/CN2020/117213 CN2020117213W WO2021082819A1 WO 2021082819 A1 WO2021082819 A1 WO 2021082819A1 CN 2020117213 W CN2020117213 W CN 2020117213W WO 2021082819 A1 WO2021082819 A1 WO 2021082819A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image block
quality classification
classification result
enhanced
Prior art date
Application number
PCT/CN2020/117213
Other languages
English (en)
French (fr)
Inventor
贺沁雯
樊鸿飞
蔡媛
Original Assignee
北京金山云网络技术有限公司
北京金山云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京金山云网络技术有限公司, 北京金山云科技有限公司 filed Critical 北京金山云网络技术有限公司
Priority to US17/259,907 priority Critical patent/US11836898B2/en
Priority to SG11202100123VA priority patent/SG11202100123VA/en
Publication of WO2021082819A1 publication Critical patent/WO2021082819A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Definitions

  • This application relates to the field of image processing technology, and in particular to an image generation method, device and electronic equipment.
  • the purpose of the embodiments of the present application is to provide an image generation method, device, and electronic equipment to improve the effect of image enhancement.
  • the specific technical solutions are as follows:
  • An embodiment of the present application provides an image generation method, including: performing multiple types of image enhancement processing on an image to be processed to obtain multiple pre-enhanced images, wherein each of the pre-enhanced images corresponds to one type of the Image enhancement processing, each type of image enhancement processing corresponds to an influencing factor that affects image quality; image quality classification is performed on the first image block in the image to be processed to obtain the first image block corresponding to the first image block.
  • a quality classification result wherein the first quality classification result is used to indicate the influencing factors that affect the image quality in the first image block; based on the first quality classification result corresponding to the first image block, A target pre-enhanced image is determined from one of the pre-enhanced images, wherein the type of image enhancement processing corresponding to the target pre-enhanced image and the first image block indicated by the first quality classification result The influencing factors affecting the image quality are matched; the second image block is determined in the target pre-enhanced image, wherein the area to which the second image block belongs in the pre-enhanced image is in the same area as the first image block The regions belonging to the images to be processed are the same; and the target image is generated according to the second image block.
  • An embodiment of the present application also provides an image generation device.
  • the device includes: an image enhancement processing module configured to perform multiple types of image enhancement processing on the image to be processed to obtain multiple pre-enhanced images, wherein each image The pre-enhanced image corresponds to one type of the image enhancement processing, and each type of the image enhancement processing corresponds to an influencing factor that affects the image quality; the image quality classification module is set to determine the first image in the image to be processed.
  • An image determination module configured to determine a target pre-enhanced image among the plurality of pre-enhanced images based on the first quality classification result corresponding to the first image block, wherein the image corresponding to the target pre-enhanced image
  • the type of enhancement processing matches the influencing factors that affect the image quality in the first image block indicated by the first quality classification result;
  • the second image block determination module is set to be in the target pre-enhanced image
  • a second image block is determined, wherein the area to which the second image block belongs in the pre-enhanced image is the same as the area to which the first image block belongs in the image to be processed;
  • a target image generation module is set To generate a target image based on the second image block.
  • An embodiment of the present application further provides an electronic device including a processor and a memory; the memory is stored with a computer program, and the computer program executes the above-mentioned image generation method when run by the processor.
  • the present application provides a computer-readable storage medium with a computer program stored on the computer-readable storage medium, and the computer program executes the above-mentioned image generation method when the computer program is run by a processor.
  • An embodiment of the present application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus.
  • the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is configured to store computer programs; processing The device is set to implement the steps of any of the above-mentioned image generation methods when the program stored in the memory is executed.
  • the embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to execute any of the above-mentioned image generation methods.
  • the embodiments of the present application provide an image generation method, device, and electronic equipment.
  • multiple types of image enhancement processing can be performed on the image to be processed to obtain multiple pre-enhanced images, where each pre-enhanced image corresponds to one type of image enhancement processing, and each type of image enhancement Processing corresponds to an influencing factor that affects image quality.
  • image quality classification may be performed on the first image block in the image to be processed to obtain the first quality classification result corresponding to the first image block, where the first quality classification result is used to indicate the presence of the first image block that affects the image quality Influencing factors.
  • the target pre-enhanced image can be determined from the multiple pre-enhanced images, where the type of image enhancement processing corresponding to the target pre-enhanced image and the first quality classification result Match the influencing factors that affect the image quality existing in the indicated first image block. Furthermore, a second image block is determined in the target pre-enhanced image, where the area to which the second image block belongs in the pre-enhanced image is the same as the area to which the first image block belongs in the image to be processed, and is based on the second image block To generate the target image.
  • the influencing factors that affect the image quality of the first image block can be determined, and the second image block is determined through the pre-enhanced image to generate the target image, that is, the image in the image to be processed can be determined
  • Different image blocks perform corresponding enhancement processing according to their own image quality, which solves the problem of poor effect of single image processing on the entire image, thereby improving the effect of image enhancement.
  • FIG. 1 is a flowchart of an image generation method provided by an embodiment of the application
  • FIG. 2 is a flowchart of an image generation method provided by another embodiment of the application.
  • FIG. 3 is a schematic diagram of cropping a to-be-processed image provided by an embodiment of the application
  • FIG. 4 is a schematic structural diagram of an image generation device provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • an image generation method is provided. As shown in FIG. 1, the method may include the following steps:
  • S101 Perform multiple types of image enhancement processing on the image to be processed to obtain multiple pre-enhanced images.
  • each pre-enhanced image corresponds to one type of image enhancement processing
  • each type of image enhancement processing corresponds to an influencing factor that affects image quality.
  • the influencing factors affecting image quality include at least noise-free low resolution, compression noise, acquisition noise and blur.
  • image enhancement can be performed on it through image enhancement algorithms, and at least noise-free low-resolution enhancement processing, compressed noise enhancement processing, acquisition noise enhancement processing, and blur enhancement processing can be performed respectively.
  • noise-free low-resolution enhancement processing is used to enhance the resolution of the image to be processed.
  • the compression noise enhancement process is used to remove the compression noise contained in the image to be processed.
  • the acquisition noise enhancement processing is used to remove the acquisition noise contained in the image to be processed.
  • Blur enhancement processing is used to eliminate the blur contained in the image to be processed.
  • the image enhancement process may be performed on the image to be processed through a pre-established model, including:
  • the image to be processed can be input into the pre-established first model to obtain a noise-free low-resolution pre-enhanced image, where the first model is used to improve the resolution of the image to be processed .
  • the pre-established first model may be a super-resolution image reconstruction (SRIR, Super Resolution Image Reconstruction) model
  • the training sample of the super-resolution image reconstruction SRIR model may be a high-definition image and the high-definition image is obtained by down-sampling and then up-sampling Low-resolution image.
  • the high-definition image can be down-sampled to obtain the down-sampled image, and then the down-sampled image can be up-sampled to obtain the corresponding low-resolution image.
  • the image to be processed may be input into a pre-established second model to obtain a pre-enhanced image corresponding to the compressed noise, where the second model is used to reduce the compressed noise contained in the image to be processed.
  • the second model may be a decompression noise model
  • the training samples of the decompression noise model may be a high-definition image and a low-resolution image containing compression noise in which the high-definition image is compressed by JPEG.
  • the image to be processed can be input into a pre-established third model to obtain a pre-enhanced image corresponding to the collected noise, where the third model is used to reduce the collected noise contained in the image to be processed;
  • the third model may be to collect a noise model
  • the training samples to collect the noise model may be a high-definition image and a low-resolution image containing collected noise corresponding to the high-definition image.
  • the image to be processed is input into a pre-established fourth model to obtain a pre-enhanced image corresponding to the blur, where the fourth model is used to reduce the blur contained in the image to be processed.
  • the fourth model may be a deblurring model
  • the training samples of the deblurring model may be high-definition images and low-resolution images with blurring corresponding to the high-definition images.
  • the aforementioned image enhancement processing for different influencing factors may be executed simultaneously in different threads or processes, thereby further improving efficiency and saving time.
  • S102 Perform image quality classification on the first image block in the image to be processed, to obtain a first quality classification result corresponding to the first image block.
  • the first quality classification result is used to indicate the influencing factors that affect the image quality existing in the first image block.
  • the image block in the image to be processed is called the first image block, that is, the first image block may be any image block in the image to be processed.
  • the image to be processed may be clipped based on a preset clipping rule to obtain the first image block after clipping.
  • the preset tailoring rules can be determined according to actual needs and experience.
  • the size of the image block after cropping can be adjusted by adjusting the batch_size (batch-size) and patch_size (patch-size) parameters of the image block. If in actual use, a higher precision is required, set the trimmed image block to be smaller.
  • the preset cropping rule may be to crop the image to be processed into a first preset number of first image blocks with the same size.
  • the first preset number may be determined based on actual needs and experience, and the actual size of the image to be processed.
  • the image to be processed can be equally divided and cropped to obtain the first preset number of image blocks.
  • the image to be processed is equally divided and cropped into 10 first image blocks of the same size.
  • the preset cropping rule may also be to randomly crop the image to be processed to obtain a second preset number of image blocks.
  • the second preset number may be determined based on actual needs and experience, and the actual size of the image to be processed, or may be determined randomly. For example, if the image to be processed is randomly cropped to obtain n first image blocks, the second preset number is n.
  • the first image block may be input to a pre-established quality classification model, and the output result of the quality classification model is used as the first image block
  • the first quality classification result may be a convolutional neural network model for classification.
  • the quality classification model may include a convolutional layer and a softmax function.
  • the first quality classification result may be the probability of the existence of the influencing factor in the first image block, that is, the quality classification model may output the probability of the existence of the influencing factor in the first image block.
  • the first quality classification result of the first image block a is obtained: the probability of noise-free and low-resolution existence is 10%, and the probability of acquisition noise existence is 50% , The probability of compression noise is 15%, and the probability of blurring is 25%.
  • the output of the quality classification model may also be the influencing factor that has the greatest impact on the image quality of the first image block. For example, after image quality classification is performed on the first image block a, it is determined that the result of the first quality classification of the first image block a is to collect a noise image.
  • the type of image enhancement processing corresponding to the target pre-enhanced image matches the influencing factors that affect the image quality existing in the first image block indicated by the first quality classification result.
  • the pre-enhanced image corresponding to the first quality classification result of the first image block may be determined as the corresponding first image block.
  • Target pre-enhanced image based on the first quality classification result of the first image block, in each pre-enhanced image, the pre-enhanced image corresponding to the first quality classification result of the first image block may be determined as the corresponding first image block.
  • there are four pre-enhanced images which are respectively a noise-free low-resolution enhanced image, a decompressed noise enhanced image, a de-acquired noise enhanced image, and a deblurred enhanced image.
  • the first quality classification of the first image block may be used.
  • the pre-enhanced image obtained by processing the influencing factor is determined as the target pre-enhanced image corresponding to the first image block.
  • the noise-free low-resolution enhanced image is the target pre-enhancement corresponding to the first image block a. Enhance the image.
  • the first quality classification result indicates the probability of the existence of each influencing factor
  • the influencing factors with the probability of being greater than the first threshold among the influencing factors are used as the first influencing factor corresponding to the first image block.
  • the pre-enhanced image obtained by processing the first influencing factor corresponding to the first image block may also be determined as the target pre-enhanced image corresponding to the first image block.
  • the first threshold may be determined according to actual use scenarios and experience, for example, the first threshold may be zero.
  • the first quality classification result of the first image block a is: the probability of noise-free and low-resolution is 0%, the probability of acquisition noise is 55%, the probability of compressed noise is 10%, and the probability of existence of blur is 0%. The probability is 35%. Where the first threshold is 0, it can be determined that the first influencing factors of the first image block a include: acquisition noise, compression noise, and blur.
  • the pre-enhanced images obtained by processing the first influencing factor corresponding to the first image block a are: to acquire a noise enhanced image, a decompressed noise enhanced image, and a deblurred enhanced image, and determine to acquire a noise enhanced image .
  • the decompressed noise enhanced image and the deblurred enhanced image are the target pre-enhanced images of the first image block a.
  • S104 Determine a second image block in the target pre-enhanced image.
  • the area to which the second image block belongs in the pre-enhanced image is the same as the area to which the first image block belongs in the image to be processed;
  • the pixel value of the target pre-enhanced image corresponding to the first image block may be adjusted, and further, based on the adjusted pixel value, an area image corresponding to the first image block is obtained as the second image block.
  • the corresponding first image in the target pre-enhanced image can be obtained The area of the block is used as the second image block.
  • the image to be processed is clipped into a first image block a and a first image block b, where the target pre-enhanced image corresponding to the first image block a is a noise-free low-resolution enhanced image.
  • the area corresponding to the first image block a in the noise-free low-resolution enhanced image can be acquired as the second image block corresponding to the first image block a.
  • S105 Generate a target image according to the second image block.
  • the target image can be generated based on the second image blocks with the same number of image blocks in the image to be processed, where the second image block corresponds to the first image block in the image to be processed one-to-one.
  • the second image blocks corresponding to each first image block in the image to be processed may be spliced to generate the target image.
  • each second image block may be spliced according to the position of the first image block corresponding to each second image block in the image to be processed to obtain the target image.
  • the image to be processed is cropped into a first image block a and a first image block b, then the second image block corresponding to the first image block a and the second image block corresponding to the first image block b Make splicing.
  • the second image block corresponding to the first image block a and the second image block corresponding to the first image block b The blocks are stitched together to generate the target image.
  • each type of image enhancement processing corresponds to an influencing factor that affects image quality.
  • image quality classification may be performed on the first image block in the image to be processed to obtain the first quality classification result corresponding to the first image block, where the first quality classification result is used to indicate the presence of the first image block that affects the image quality Influencing factors.
  • the target pre-enhanced image can be determined from the multiple pre-enhanced images, where the type of image enhancement processing corresponding to the target pre-enhanced image and the first quality classification result Match the influencing factors that affect the image quality existing in the indicated first image block.
  • a second image block is determined in the target pre-enhanced image, where the area to which the second image block belongs in the pre-enhanced image is the same as the area to which the first image block belongs in the image to be processed, and is based on the second image block , Generate the target map. Since the quality of the first image block is classified, the factors affecting the image quality of the first image block can be determined, and the second image block can be determined through the pre-enhanced image to generate the target image, thereby improving the effect of image enhancement.
  • another image generation method is provided. As shown in FIG. 2, the method includes the following steps:
  • S201 Perform multiple types of image enhancement processing on the image to be processed to obtain multiple pre-enhanced images.
  • step S201 is the same as or similar to that of step S101, and will not be repeated here.
  • S202 Input the first image block in the image to be processed into a pre-established quality classification model to obtain an initial quality classification result corresponding to the first image block.
  • the pre-established quality classification model may be obtained by adopting an image quality classification scheme based on deep learning, constructing a convolutional neural network model, and inputting a large number of training samples (image blocks and their quality classification labels) for training.
  • the quality classification model may be a convolutional neural network model.
  • the quality classification model may include a convolutional layer and a softmax function.
  • the method of obtaining noise-free low-resolution images, low-resolution images with compression noise, low-resolution images with acquisition noise, and low-resolution images with blurring in the foregoing embodiment can be used to generate Training samples of the quality classification model.
  • the quality classification labels of the generated training samples respectively indicate noise-free low resolution, low resolution with compression noise, low resolution with acquisition noise, and low resolution with blur.
  • the quality classification result output by the quality classification model can be used as the initial quality classification result corresponding to the first image block, and further, based on the initial quality classification result and the preset image processing strategy, The final quality classification result.
  • the initial quality classification result includes the probability of the existence of each influencing factor.
  • the initial quality classification result of the first image block a output by the softmax function of the last layer of the network to construct the convolutional neural network model after training the first image block a is: the probability of existence of noise-free low resolution is 10% , The probability of acquisition noise is 50%, the probability of compression noise is 15%, and the probability of blur is 25%. That is, the first image block a may be input to the trained convolutional neural network model, and the output result of the softmax function of the network final layer of the convolutional neural network model can be used as the initial quality classification result of the first image block a.
  • S203 Determine the first quality classification result corresponding to the first image block based on the preset image processing strategy and the initial quality classification result corresponding to the first image block.
  • different image processing strategies can determine different first quality classification results.
  • the initial quality classification result includes the probability of the existence of each influencing factor
  • the above step S203 can be implemented in the following manner:
  • the first quality classification result is the influencing factor with the largest probability of the initial quality classification result, and the following method is adopted:
  • the image processing strategy (which can be called the second image processing strategy) represents a comprehensive consideration of the initial quality classification results
  • the above steps can be implemented in the following manner:
  • Step A Based on the position of the first image block in the image to be processed, an image block corresponding to the position of the first image block is determined as a reference image block corresponding to the first image block.
  • the image block that has a position correspondence with the first image block may be an image block whose position is associated with the first image block.
  • it may be an image block whose position is adjacent to the first image block, or an image block whose position is adjacent to the first image block.
  • the image block corresponding to the first image block in position may be another first image block adjacent to the first image block.
  • first image blocks adjacent to the first image block E are respectively B, D, F, and H
  • first image block A the first image adjacent to the first image block A
  • the blocks are B and D respectively.
  • the image block that has a positional correspondence with the first image block may also be another first image block that is connected to the first image block.
  • the first image blocks connected to the first image block E are A, B, C, D, F, G, H, and I, respectively.
  • the first image blocks adjacent to the first image block A are B, D, and E, respectively.
  • Step B According to the initial quality classification result corresponding to the first image block and the initial quality classification result corresponding to the reference image block corresponding to the first image block, the comprehensive quality classification result corresponding to the first image block is determined as the first image The first quality classification result corresponding to the block.
  • step B can be implemented in the following manner:
  • the first method is to determine whether the initial quality classification results corresponding to the reference image blocks corresponding to the first image block are the same. If they are the same, the initial quality classification result corresponding to the reference image block corresponding to the first image block is determined as the comprehensive quality classification result corresponding to the first image block; if they are different, the initial quality classification result corresponding to the first image block is determined Is the comprehensive quality classification result corresponding to the first image block.
  • the first image block A shown in FIG. 3 its adjacent first image blocks are B and D respectively. If the initial quality classification results of the first image block B and the first image block D are the same, for example, the initial quality classification results of the first image block B and the first image block D are both collected noise images, then the first image block A is determined The comprehensive classification result is the acquisition of noisy images. If the initial quality classification result of the first image block A is a noise-free low-resolution image, the initial quality classification result of the first image block B is a compressed noise low-resolution image, and the initial quality classification result of the first image block D is acquisition noise Image, it is determined that the comprehensive classification result of the first image block A is a noise-free low-resolution image.
  • the second method according to the probability of each influencing factor in the reference image block corresponding to the first image block, adjust the probability of each influencing factor in the first image block, and adjust each influencing factor in the first image block
  • the subsequent existence probability is used as the existence probability of each influencing factor included in the comprehensive quality classification result of the first image block.
  • the average value of the probability that the influencing factor exists in the first image block and the corresponding reference image blocks can be calculated as the integrated quality classification result of the first image block.
  • the probability of the existence of the influencing factor can be calculated as the integrated quality classification result of the first image block.
  • the first image block A shown in FIG. 3 its adjacent first image blocks are B and D respectively.
  • the initial quality classification result of the first image block A is that the probability of noise-free and low-resolution is 10%, the probability of acquisition noise is 50%, the probability of compression noise is 15%, and the probability of blurring is 25%.
  • the initial quality classification result of the first image block B is: the probability of noise-free and low-resolution is 0%, the probability of acquisition noise is 20%, the probability of compression noise is 55%, and the probability of blurring is 25% .
  • the initial quality classification result of the first image block D is: the probability of noise-free low resolution is 20%, the probability of acquisition noise is 20%, the probability of compression noise is 20%, and the probability of blur is 40%.
  • the comprehensive quality classification result of the first image block A (that is, the first quality classification result of the first image block A) is: the probability of noise-free low resolution is 10%, the probability of acquisition noise is 30%, and the compression noise The probability of existence is 30%, and the probability of existence of ambiguity is 30%. It can be understood that the above-mentioned numerical values are merely examples of this application for illustration, and this application is not limited to the above-mentioned numerical values.
  • the target pre-enhanced image may be determined from the multiple pre-enhanced images based on the target influencing factor corresponding to the first image block.
  • the above-mentioned first image processing strategy may be adopted, or the above-mentioned second image processing strategy may also be adopted.
  • Different image processing strategies can be used to determine different first quality classification results, thereby determining different target pre-enhanced images.
  • the influencing factor represented by the first quality classification result can be determined
  • the corresponding pre-enhanced image is used as the target pre-enhanced image corresponding to the first image block.
  • the first quality classification result indicates the probability of each influencing factor
  • the determined target pre-enhanced image may be one or multiple.
  • the determination of the target pre-enhanced image is similar to step S103, which will not be repeated here.
  • the first quality classification result indicates the probability of the existence of each influencing factor, it can be implemented in the following manner:
  • the determined target pre-enhanced image is one, it can be determined that an image block in the target pre-enhanced image that belongs to the same area as the first image block in the to-be-processed image can be determined as the second image block.
  • the pixel value of each pixel in each target pre-enhanced image can be obtained, and the pixel value of the pixel corresponding to the target pixel in each target pre-enhanced image can be weighted and summed to Get the comprehensive pixel value corresponding to the target pixel.
  • the target pixel is a pixel in the first image block
  • the weight of each pixel in the target pre-enhanced image is the probability that the influencing factor corresponding to each target pre-enhanced image exists in the first image block.
  • the target The pixel value of the pixel is adjusted to the integrated pixel value to generate a second image block corresponding to the first image block.
  • the noise-free low-resolution enhanced image, the noise-enhanced image, the decompressed noise-enhanced image, and the deblurred-enhanced image can be obtained separately.
  • the obtained pixel values are x1, x2, x3, and x4, and 15%, 35%, 25%, and 25% are used as the weights of x1, x2, x3, and x4, and then the weighted sum is performed and the sum is summed The result is the pixel value of the pixel.
  • each second image block may be spliced according to the position of the first image block corresponding to each second image block in the image to be processed to obtain the target image.
  • step S105 the specific implementation manner is the same as or similar to step S105, and details are not described herein again.
  • multiple types of image enhancement processing may be performed on the image to be processed to obtain multiple pre-enhanced images.
  • the first image block in the image to be processed may be input to a pre-established quality classification model to obtain an initial quality classification result corresponding to the first image block. Then, based on the preset image processing strategy and the initial quality classification result corresponding to the first image block, the first quality classification result corresponding to the first image block is determined, and based on the target influencing factor corresponding to the first image block, The target pre-enhanced image is determined from the pre-enhanced image.
  • the second image block is determined in the target pre-enhanced image, and the target image is generated according to the second image block. Since the quality of the first image block is classified, the factors affecting the image quality of the first image block can be determined, and the second image block can be determined through the pre-enhanced image to generate the target image, thereby improving the effect of image enhancement.
  • the embodiment of the application also provides an image generation device. As shown in FIG. 4, the device includes:
  • the image enhancement processing module 401 is configured to perform multiple types of image enhancement processing on the image to be processed to obtain multiple pre-enhanced images, wherein each pre-enhanced image corresponds to one type of image enhancement processing, and each type of image
  • the enhancement processing corresponds to an influencing factor that affects the image quality
  • the image quality classification module 402 is configured to perform image quality classification on the first image block in the image to be processed to obtain a first quality classification result corresponding to the first image block, where the first quality classification result is used to indicate the Existing influencing factors affecting image quality;
  • the image determining module 403 is configured to determine the target pre-enhanced image among multiple pre-enhanced images based on the first quality classification result corresponding to the first image block, wherein the type of image enhancement processing corresponding to the target pre-enhanced image is the same as the first Match the influencing factors that affect the image quality existing in the first image block indicated by the quality classification result;
  • the second image block determining module 404 is configured to determine the second image block in the target pre-enhanced image, where the area to which the second image block belongs in the pre-enhanced image and the area to which the first image block belongs in the image to be processed the same;
  • the target image generating module 405 is configured to generate a target image according to the second image block.
  • the image quality classification module 402 is specifically configured to input the first image block in the image to be processed into a pre-established quality classification model to obtain an initial quality classification result corresponding to the first image block, where the quality classification model Used to classify the image quality, and determine the first quality classification result corresponding to the first image block based on the preset image processing strategy and the initial quality classification result corresponding to the first image block.
  • the initial quality classification result includes the probability of the existence of each influencing factor
  • the image quality classification module 402 is specifically configured to determine the most probable influencing factor in the initial quality classification result corresponding to the first image block as the target influencing factor corresponding to the first image block, and according to the target influencing factor corresponding to the first image block The influencing factor determines the first quality classification result corresponding to the first image block.
  • the image determining module 403 is specifically configured to determine the target pre-enhanced image among multiple pre-enhanced images based on the target influencing factor corresponding to the first image block.
  • the image quality classification module 402 is specifically configured to determine the image block corresponding to the first image block in position based on the position of the first image block in the image to be processed, as the first image block Corresponding reference image block, and according to the initial quality classification result corresponding to the first image block and the initial quality classification result corresponding to the reference image block corresponding to the first image block, the comprehensive quality classification result corresponding to the first image block is determined, As the first quality classification result corresponding to the first image block.
  • the image quality classification module 402 is specifically configured to determine whether the initial quality classification results corresponding to the reference image blocks corresponding to the first image block are the same, and if they are the same, the reference image block corresponding to the first image block is The corresponding initial quality classification result is determined as the comprehensive quality classification result corresponding to the first image block, and if different, the initial quality classification result corresponding to the first image block is determined as the comprehensive quality classification result corresponding to the first image block.
  • the initial quality classification result includes the probability of the existence of each influencing factor
  • the image quality classification module 402 is specifically configured to adjust the existence probability of each influencing factor in the first image block according to the existence probability of each influencing factor in the reference image block corresponding to the first image block, and to adjust the existence probability of each influencing factor in the first image block.
  • the adjusted existence probability of each influencing factor is used as the existence probability of each influencing factor included in the comprehensive quality classification result of the first image block.
  • the second image block determination module 404 is configured to obtain the pixel value of each pixel in each target pre-enhanced image, and perform a weighted summation of the pixel values of the pixels corresponding to the target pixel in each target pre-enhanced image , In order to obtain the comprehensive pixel value of the corresponding target pixel, where the target pixel is a pixel in the first image block, and the weight of each pixel in the target pre-enhanced image is the influencing factor corresponding to each target pre-enhanced image in the first image The probability of existence in the block, and the pixel value of the target pixel is adjusted to the integrated pixel value to generate a second image block corresponding to the first image block.
  • the influencing factors include at least noise-free low resolution, compression noise, acquisition noise, and blur;
  • the image enhancement processing module 401 is specifically configured to input the image to be processed into a pre-established first model to obtain a corresponding noise-free low-resolution pre-enhanced image, where the first model is used to improve the resolution of the image to be processed, and The image to be processed is input into the pre-established second model to obtain a pre-enhanced image corresponding to the compression noise, where the second model is used to remove the compression noise contained in the image to be processed, and the image to be processed is input into the pre-established third model, Obtain a pre-enhanced image corresponding to the acquisition noise, where the third model is used to remove the acquisition noise contained in the image to be processed, and the image to be processed is input into the pre-established fourth model to obtain the corresponding blurred pre-enhanced image, where the first The four model is used to eliminate the blur contained in the image to be processed.
  • the target image generation module 405 is specifically configured to stitch the second image blocks according to the positions of the first image blocks corresponding to the second image blocks in the image to be processed. , Get the target image.
  • the embodiment of the present application also provides an electronic device, as shown in FIG. 5, including a processor 501, a communication interface 502, a memory 503, and a communication bus 504.
  • the processor 501, the communication interface 502, and the memory 503 pass through the communication bus 504. Complete the communication between each other,
  • the memory 503 is set to store computer programs
  • the image to be processed is subjected to multiple types of image enhancement processing to obtain multiple pre-enhanced images, wherein each of the pre-enhanced images corresponds to one type of the image enhancement processing, and each type of the image enhancement processing Correspond to an influencing factor that affects image quality;
  • a target pre-enhanced image is determined among a plurality of the pre-enhanced images, wherein the type of image enhancement processing corresponding to the target pre-enhanced image is the same as that of the Match the influencing factors that affect the image quality existing in the first image block indicated by the first quality classification result;
  • a second image block is determined in the target pre-enhanced image, where the area to which the second image block belongs in the pre-enhanced image is the same as the area to which the first image block belongs in the image to be processed the same;
  • a target image is generated.
  • the communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used for communication between the above-mentioned electronic device and other devices.
  • the memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
  • NVM non-Volatile Memory
  • the memory may also be at least one storage device located far away from the foregoing processor.
  • the above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU central processing unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a computer-readable storage medium stores a computer program, and the computer program implements any of the above-mentioned image generation methods when executed by a processor. A step of.
  • the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the factors affecting the image quality of the first image block can be determined, and then the second image block is determined through the pre-enhanced image to generate the target
  • the image that is, different image blocks in the image to be processed can be correspondingly enhanced according to their own image quality, which solves the problem of poor effect of single image processing on the entire image, thereby improving the effect of image enhancement.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

一种图像生成方法、装置及电子设备，包括：对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，并且对待处理图像中的第一图像块进行图像质量分类，得到第一图像块对应的第一质量分类结果，以及基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像，以及在目标预增强图像中确定出第二图像块，以及根据第二图像块，生成目标图像，由于对第一图像块进行质量分类，从而可以确定影响第一图像块图像质量的影响因素，再结合通过预增强图像确定出第二图像块，生成目标图像从而提高了图像增强的效果。

Description

一种图像生成方法、装置及电子设备
本申请要求于2019年10月31日提交中国专利局、申请号为201911055646.9发明名称为“一种图像生成方法、装置及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，特别是涉及一种图像生成方法、装置及电子设备。
背景技术
随着社会的不断发展，以及图像拍摄技术的不断进步，越来越多人愿意通过电子设备拍摄图像来记录生活。然而，由于设备、环境等因素的影响，使得图像在采集和生成的过程中，会产生各种噪声，从而影响图像的图像质量。例如，当拍摄环境的光线不足，亮度过暗时，拍摄的图像会产生采集噪声，以及在拍摄的过程中，拍摄对象运动、电子设备抖动或者拍摄对象失焦，都会造成拍摄的图像变得模糊。相关技术中存在一些图像增强处理方法，但是采用相关技术进行图像增强，只能粗略的针对一种影响因素对图像进行图像增强，图像增强的效果一般。
发明内容
本申请实施例的目的在于提供一种图像生成方法、装置及电子设备，以提高图像增强的效果。具体技术方案如下：
本申请实施例提供一种图像生成方法，包括：对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张所述预增强图像对应一种类型的所述图像增强处理，每种类型的所述图像增强处理对应一种影响图像质量的影响因素；对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，其中，所述第一质量分类结果用于指示所述第一图像块中存在的影响图像质量的影响因素；基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，其中，所述目标预增强图像所对应的图像增强处理的类型与所述第一质量分类结果所指示的所述第一图像块中存在的影响图像质量 的影响因素相匹配；在所述目标预增强图像中确定出第二图像块，其中，所述第二图像块在所述预增强图像中所属的区域与所述第一图像块在所述待处理图像中所属的区域相同；根据所述第二图像块，生成目标图像。
本申请实施例还提供一种图像生成装置，所述装置包括：图像增强处理模块，设置为对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张所述预增强图像对应一种类型的所述图像增强处理，每种类型的所述图像增强处理对应一种影响图像质量的影响因素；图像质量分类模块，设置为对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，其中，所述第一质量分类结果用于指示所述第一图像块中存在的影响图像质量的影响因素；图像确定模块，设置为基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，其中，所述目标预增强图像所对应的图像增强处理的类型与所述第一质量分类结果所指示的所述第一图像块中存在的影响图像质量的影响因素相匹配；第二图像块确定模块，设置为在所述目标预增强图像中确定出第二图像块，其中，所述第二图像块在所述预增强图像中所属的区域与所述第一图像块在所述待处理图像中所属的区域相同；目标图像生成模块，设置为根据所述第二图像块，生成目标图像。
本申请实施例还提供一种电子设备，包括处理器和存储器；所述存储器上存储有计算机程序，所述计算机程序在被所述处理器运行时执行上述图像生成方法。
本申请提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器运行时执行上述图像生成方法。
本申请实施例还提供一种电子设备，包括处理器、通信接口、存储器和通信总线，其中，处理器，通信接口，存储器通过通信总线完成相互间的通信；存储器，设置为存放计算机程序；处理器，设置为执行存储器上所存放的程序时，实现上述任一图像生成方法的步骤。
本申请实施例还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述任一图像生成方法。
本申请实施例提供了一种图像生成方法、装置及电子设备。本申请的方案中，可以对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张预增强图像对应一种类型的图像增强处理，每种类型的图像增强处理对应一种影响图像质量的影响因素。进而，可以对待处理图像中的第一图像块进行图像质量分类，得到第一图像块对应的第一质量分类结果，其中，第一质量分类结果用于指示第一图像块中存在的影响图像质量的影响因素。然后，可以基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像，其中，目标预增强图像所对应的图像增强处理的类型与第一质量分类结果所指示的第一图像块中存在的影响图像质量的影响因素相匹配。进而，在目标预增强图像中确定出第二图像块，其中，第二图像块在预增强图像中所属的区域与第一图像块在待处理图像中所属的区域相同，并根据第二图像块，生成目标图像。由于对第一图像块进行质量分类，从而可以确定影响第一图像块图像质量的影响因素，再结合通过预增强图像确定出第二图像块，生成目标图像，也就是，可以对待处理图像中的不同的图像块根据自身的图像质量进行对应的增强处理，解决了对整张图像进行的单一图像处理效果差的问题，从而提高了图像增强的效果。
当然，实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。
附图说明
为了更清楚地说明本申请实施例和相关技术的技术方案，下面对实施例和相关技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请一个实施例提供的图像生成方法的流程图；
图2为本申请另一个实施例提供的图像生成方法的流程图；
图3为本申请一个实施例提供的待处理图像剪裁示意图；
图4为本申请一个实施例提供的图像生成装置的结构示意图；
图5为本申请实施例提供的电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
在本申请的一个实施例中，提供一种图像生成方法，如图1所示，该方法可以包括以下步骤：
S101：对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像。
本申请实施例中，每张预增强图像对应一种类型的图像增强处理，每种类型的图像增强处理对应一种影响图像质量的影响因素。其中，影响图像质量的影响因素至少包括无噪声低分辨率、压缩噪声、采集噪声和模糊。
对于待处理图像，可以分别通过图像增强算法对其进行图像增强处理，且至少分别进行无噪声低分辨率增强处理、压缩噪声增强处理、采集噪声增强处理和模糊增强处理。其中，无噪声低分辨率增强处理用来提升待处理图像的分辨率。压缩噪声增强处理用于去除待处理图像所包含的压缩噪音。采集噪声增强处理用于去除待处理图像所包含的采集噪音。模糊增强处理用于消除待处理图像所包含的模糊。
在一个实施例中，可以通过预先建立的模型对待处理图像进行图像增强处理，包括：
针对无噪声低分辨率的图像增强处理，可以将待处理图像输入预先建立的第一模型，得到对应无噪声低分辨率的预增强图像，其中，第一模型用于提高待处理图像的分辨率。
其中，预先建立的第一模型可以为超分辨图像重建(SRIR，Super Resolution Image Reconstruction)模型，而超分辨图像重建SRIR模型的训练样 本可以为高清图像和对于该高清图像经过下采样再上采样得到的低分辨率图像。也就是说，可以对该高清图像进行下采样，得到下采样图像，然后再对下采样图像进行上采样，得到对应的低分辨图像。通过将待处理图像输入超分辨图像重建SRIR模型，得到提高分辨率的无噪声低分辨率增强图像。
针对压缩噪声的图像增强处理，可以将待处理图像输入预先建立的第二模型，得到对应压缩噪声的预增强图像，其中，第二模型用于减少待处理图像所包含的压缩噪音。
其中，第二模型可以为去压缩噪声模型，去压缩噪声模型的训练样本可以为高清图像和该高清图像经过JPEG压缩的含压缩噪声的低分辨率图像。通过将待处理图像输入去压缩噪声模型，得到去除压缩噪声的去压缩噪声增强图像。
针对采集噪声的图像增强处理，可以将待处理图像输入预先建立的第三模型，得到对应采集噪声的预增强图像，其中，第三模型用于降低待处理图像所包含的采集噪音；
其中，第三模型可以为去采集噪声模型，去采集噪声模型的训练样本可以为高清图像和对应该高清图像的含采集噪声的低分辨率图像。通过将待处理图像输入去采集噪声模型，得到去除采集噪声的去采集噪声增强图像。
针对模糊的图像增强处理，将待处理图像输入预先建立的第四模型，得到对应模糊的预增强图像，其中，第四模型用于减少待处理图像所包含的模糊。
其中，第四模型可以为去模糊模型，而去模糊模型的训练样本可以为高清图像和对应该高清图像的含模糊的低分辨率图像。通过将待处理图像输入去模糊模型，得到去除模糊的去模糊增强图像。
在一个实施例中，上述针对不同影响因素的图像增强处理可以是在不同线程或进程中同时执行的，从而进一步的提高了效率，节约了时间。
S102：对待处理图像中的第一图像块进行图像质量分类，得到第一图像块对应的第一质量分类结果。
本申请实施例中，第一质量分类结果用于指示第一图像块中存在的影响图像质量的影响因素。
在一实施方式中，待处理图像中的图像块被称为第一图像块，也就是第一图像块可以是待处理图像中的任意一个图像块。
在一个实施例中，可以基于预先设置的剪裁规则，对待处理图像进行剪裁，得到剪裁后的第一图像块。
其中，预设的剪裁规则可以是根据实际需求及经验进行确定的。本领域的技术人员所知的，可以通过调整图像块的batch_size(批处理-大小)和patch_size(补丁-大小)参数来调整剪裁后的图像块大小。如果在实际使用过程中，需要精度高一点就将剪裁后的图像块设置得小一点。
在一个实施例中，预设的剪裁规则可以是将待处理图像剪裁成大小相同的第一预设数量个第一图像块。其中，第一预设数量可以基于实际需求和经验，以及该待处理图像的实际大小确定。
本实施例中，确定第一预设数量后，可以将该待处理图像均等分的剪裁，得到第一预设数量个图像块。示例性的，第一预设数量为10，则将待处理图像均等分的剪裁为10个大小相同的第一图像块。
在一个实施例中，预设的剪裁规则还可以是对待处理图像进行随机剪裁，得到第二预设数量个图像块。其中，第二预设数量可以基于实际需求和经验，以及该待处理图像的实际大小确定，也可以是随机确定的。例如，随机的对待处理图像进行剪裁，得到n张第一图像块，则该第二预设数量为n。
在一个实施例中，针对待处理图像的每一张第一图像块，可以将该张第一图像块输入预先建立的质量分类模型，并且将质量分类模型输出的结果作为该张第一图像块的第一质量分类结果。例如，质量分类模型可以为用于分类的卷积神经网络模型，具体的，质量分类模型可以包括卷积层和softmax函数。
在一个实施例中，第一质量分类结果可以是第一图像块中影响因素存在的概率，也就是说，质量分类模型可以输出第一图像块中影响因素存在的概率。示例性的，对第一图像块a进行图像质量分类后，得到第一图像块a的第一 质量分类结果为：无噪声低分辨率存在的概率为10％，采集噪声存在的概率为50％，压缩噪声存在的概率为15％，模糊存在的概率为25％。
在一个实施例中，质量分类模型输出的也可以是对第一图像块的图像质量影响最大的影响因素。例如，对第一图像块a进行图像质量分类后，确定第一图像块a的第一质量分类结果为采集噪声图像。
S103：基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像。
本申请实施例中，目标预增强图像所对应的图像增强处理的类型，与第一质量分类结果所指示的第一图像块中存在的影响图像质量的影响因素相匹配。
在一实施方式中，可以基于第一图像块的第一质量分类结果，在各预增强图像中，确定对应第一图像块的第一质量分类结果的预增强图像，作为第一图像块对应的目标预增强图像。
在一个实施例中，存在四张预增强图像，分别为无噪声低分辨率增强图像、去压缩噪声增强图像、去采集噪声增强图像和去模糊增强图像。
在一个实施例中，若第一图像块的第一质量分类结果表示影响第一图像块图像质量的各影响因素中存在的概率最大的影响因素，则可以基于第一图像块的第一质量分类结果表示的影响因素，在各预增强图像中，确定针对该影响因素进行处理得到的预增强图像，作为该第一图像块对应的目标预增强图像。
示例性的，第一图像块a的第一质量分类结果为无噪声低分辨率，则在上述四张预增强图像中，确定无噪声低分辨率增强图像为第一图像块a对应的目标预增强图像。
在一个实施例中，若第一质量分类结果表示各影响因素存在的概率，则可以基于第一图像块的第一质量分类结果表示的各影响因素存在的概率，确定影响第一图像块图像质量的各影响因素中存在的概率大于第一阈值的影响因素，作为第一图像块对应的第一影响因素。还可以在各预增强图像中，确定针对第一图像块对应的第一影响因素进行处理得到的预增强图像，作为第 一图像块对应的目标预增强图像。其中，第一阈值可以是根据实际使用的场景和经验确定的，例如第一阈值可以为0。
示例性的，第一图像块a的第一质量分类结果为：无噪声低分辨率存在的概率为0％，采集噪声存在的概率为55％，压缩噪声存在的概率为10％，模糊存在的概率为35％。其中，第一阈值为0，可以确定第一图像块a的第一影响因素包括：采集噪声、压缩噪声和模糊。进一步的，确定针对第一图像块a对应的第一影响因素进行处理得到的预增强图像分别为：去采集噪声增强图像、去压缩噪声增强图像和去模糊增强图像，及确定去采集噪声增强图像、去压缩噪声增强图像和去模糊增强图像为第一图像块a的目标预增强图像。
S104：在目标预增强图像中确定出第二图像块。
本申请实施例中，第二图像块在预增强图像中所属的区域与第一图像块在待处理图像中所属的区域相同；
在一实施方式中，可以对第一图像块对应的目标预增强图像的像素值进行调整，进而，基于调整后的像素值，得到与第一图像块对应的区域图像，作为第二图像块。具体的，可以参考后续实施例中的详细介绍。
或者，也可以直接将目标预增强图像中对应对第一图像块的区域，确定为第二图像块。
在一个实施例中，若第一图像块的第一质量分类结果表示影响第一图像块图像质量的各影响因素中存在的概率最大的影响因素，则可以获取目标预增强图像中对应第一图像块的区域，作为第二图像块。
示例性的，待处理图像被剪裁为第一图像块a和第一图像块b，其中，第一图像块a对应的目标预增强图像为无噪声低分辨率增强图像。对于第一图像块a，则可以获取无噪声低分辨率增强图像中对应第一图像块a的区域，作为与第一图像块a对应的第二图像块。
S105：根据第二图像块，生成目标图像。
本申请实施例中，可以根据与待处理图像中的图像块数量相同的第二图像块生成目标图像，其中，第二图像块与待处理图像中的第一图像块一一对 应。
在一个实施例中，可以将与待处理图像中各第一图像块一一对应的第二图像块进行拼接，生成目标图像。
例如，可以按照各个第二图像块各自对应的第一图像块在待处理图像中的位置，对各个第二图像块进行拼接，得到目标图像。
在一个实施例中，待处理图像被剪裁为第一图像块a和第一图像块b，则可以将第一图像块a对应的第二图像块和第一图像块b对应的第二图像块进行拼接。
在一个实施例中，可以按照第一图像块a和第一图像块b在待处理图像中的位置，对第一图像块a对应的第二图像块和第一图像块b对应的第二图像块进行拼接，生成目标图像。
本申请实施例提供的上述如图1所示的图像生成方法中，对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张预增强图像对应一种类型的图像增强处理，每种类型的图像增强处理对应一种影响图像质量的影响因素。进而，可以对待处理图像中的第一图像块进行图像质量分类，得到第一图像块对应的第一质量分类结果，其中，第一质量分类结果用于指示第一图像块中存在的影响图像质量的影响因素。然后，可以基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像，其中，目标预增强图像所对应的图像增强处理的类型与第一质量分类结果所指示的第一图像块中存在的影响图像质量的影响因素相匹配。进而，在目标预增强图像中确定出第二图像块，其中，第二图像块在预增强图像中所属的区域与第一图像块在待处理图像中所属的区域相同，并根据第二图像块，生成目标图。由于对第一图像块进行质量分类，从而可以确定影响第一图像块图像质量的影响因素，再结合通过预增强图像确定出第二图像块，生成目标图像从而提高了图像增强的效果。
在本申请的一个实施例中，提供另一种图像生成方法，如图2所示，该方法包括以下步骤：
S201：对待处理图像分别进行多个类型的图像增强处理，以得到多张预 增强图像。
本申请实施例中，S201的具体实现方式与步骤S101相同或相似，在此不再赘述。
S202：将待处理图像中的第一图像块输入预先建立的质量分类模型，得到第一图像块对应的初始质量分类结果。
本申请实施例中，预先建立的质量分类模型可以是采用基于深度学习的图像质量分类方案，构建卷积神经网络模型，并投入大量训练样本(图像块及其质量分类标签)进行训练得到的。例如，质量分类模型可以为卷积神经网络模型，具体的，质量分类模型可以包括卷积层和softmax函数。
一种实现方式中，可以基于上述实施例中获取无噪声的低分辨率图像、含压缩噪声的低分辨率图像、含采集噪声的低分辨率图像和含模糊的低分辨率图像的方法，生成质量分类模型的训练样本。相应的，生成的训练样本的质量分类标签分别表示无噪声低分辨率、含压缩噪声的低分辨率、含采集噪声的低分辨率和含模糊的低分辨率。
为了提高第一图像块质量分类的准确性，可以将质量分类模型输出的质量分类结果作为第一图像块对应的初始质量分类结果，进而，基于初始质量分类结果和预设的图像处理策略，得到最终的质量分类结果。
在一个实施例中，初始质量分类结果包括各影响因素存在的概率。示例性的，将第一图像块a通过训练后构建卷积神经网络模型的网络末层softmax函数输出的第一图像块a的初始质量分类结果为：无噪声低分辨率存在的概率为10％，采集噪声存在的概率为50％，压缩噪声存在的概率为15％，模糊存在的概率为25％。也就是，可以将第一图像块a输入训练得到的卷积神经网络模型，将该卷积神经网络模型的网络末层softmax函数的输出结果，作为第一图像块a的初始质量分类结果。
S203：基于预设的图像处理策略和第一图像块对应的初始质量分类结果，确定出第一图像块对应的第一质量分类结果。
本申请实施例中，不同的图像处理策略可以确定出不同的第一质量分类结果。当初始质量分类结果包括各影响因素存在的概率，可选的，可以通过 如下方式实现上述S203的步骤：
当图像处理策略(可以称为第一图像处理策略)表示：第一质量分类结果为初始质量分类结果中，存在的概率最大的影响因素，采用如下方式：
确定出第一图像块对应的初始质量分类结果中存在的概率最大的影响因素，作为第一图像块对应的目标影响因素，并且根据第一图像块对应的目标影响因素，确定出第一图像块对应的第一质量分类结果。
当图像处理策略(可以称为第二图像处理策略)表示综合考虑初始质量分类结果，则可以采用如下方式实现上述步骤：
步骤A：基于第一图像块在待处理图像中的位置，确定出与第一图像块在位置上存在对应关系的图像块，作为第一图像块对应的参考图像块。
与第一图像块在位置上存在对应关系的图像块，可以为位置与第一图像块相关联的图像块。例如，可以为位置与第一图像块相邻的图像块，也可以为位置与第一图像块相接的图像块。
本申请实施例中，与该第一图像块在位置上存在对应关系的图像块可以是与该第一图像块相邻的其他第一图像块。
示例性的，如图3所示的待处理图像中存在9个第一图像块，分别用A、B、C、D、E、F、G、H和I表示。则对于第一图像块E，与第一图像块E相邻的第一图像块分别为B、D、F和H，对于第一图像块A，与第一图像块A相邻的第一图像块分别为B和D。
与该第一图像块在位置上存在对应关系的图像块还可以是与该第一图像块相接的其他第一图像块。
示例性的，对于如图3所示的第一图像块E，第一图像块E相接的第一图像块分别为A、B、C、D、F、G、H和I。对于第一图像块A，与第一图像块A相接的第一图像块分别为B、D和E。
步骤B：根据第一图像块对应的初始质量分类结果，以及第一图像块对应的参考图像块所对应的初始质量分类结果，确定出第一图像块对应的综合质量分类结果，作为第一图像块对应的第一质量分类结果。
在一实施方式中，步骤B可以通过以下方式实现：
第一种方式：判断第一图像块对应的参考图像块所对应的初始质量分类结果是否相同。若相同，则将第一图像块对应的参考图像块所对应的初始质量分类结果确定为第一图像块对应的综合质量分类结果；若不同，则将第一图像块对应的初始质量分类结果确定为第一图像块对应的综合质量分类结果。
示例性的，对于如图3所示的第一图像块A，其相邻的第一图像块分别为B和D。若第一图像块B和第一图像块D的初始质量分类结果相同，例如，第一图像块B和第一图像块D的初始质量分类结果都为采集噪声图像，则确定第一图像块A的综合分类结果为采集噪声图像。若第一图像块A的初始质量分类结果为无噪声低分辨率图像、第一图像块B的初始质量分类结果为压缩噪声低分辨率图像、第一图像块D的初始质量分类结果为采集噪声图像，则确定第一图像块A的综合分类结果为无噪声低分辨率图像。
第二种方式：根据第一图像块对应的参考图像块中每个影响因素存在的概率，调整第一图像块中每个影响因素存在的概率，并将第一图像块中每个影响因素调整后的存在的概率作为第一图像块的综合质量分类结果所包含的每个影响因素存在的概率。
在一个实施例中，针对每一影响因素，可以计算第一图像块和对应的各参考图像块中该影响因素存在的概率的平均值，作为第一图像块的综合质量分类结果所包含的该影响因素存在的概率。
对于如图3所示的第一图像块A，其相邻的第一图像块分别为B和D。若第一图像块A的初始类质量分类结果为无噪声低分辨率存在的概率为10％，采集噪声存在的概率为50％，压缩噪声存在的概率为15％，模糊存在的概率为25％，第一图像块B的初始质量分类结果为：无噪声低分辨率存在的概率为0％，采集噪声存在的概率为20％，压缩噪声存在的概率为55％，模糊存在的概率为25％。第一图像块D的初始质量分类结果为：无噪声低分辨率存在的概率为20％，采集噪声存在的概率为20％，压缩噪声存在的概率为20％，模糊存在的概率为40％。计算得到第一图像块A、B和D的无噪声低分辨率存在的平均概率为(10％+0％+20％)/3＝10％，采集噪声存在的平均概率为(50％+20％+20％)/3＝30％，压缩噪声存在的平均概率为(15％+55％+20％)/3＝30％，模糊存在 的平均概率为(25％+25％+40％)/3＝30％。则第一图像块A的综合质量分类结果(即第一图像块A的第一质量分类结果)为：无噪声低分辨率存在的概率为10％，采集噪声存在的概率为30％，压缩噪声存在的概率为30％，模糊存在的概率为30％。可以理解的是，上述数值仅是本申请为了说明所作的举例，本申请并不限于上述数值。
S204：基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像。
本申请实施例中，当第一质量分类结果根据目标影响因素确定时，可以基于第一图像块对应的目标影响因素，在多张预增强图像中确定出目标预增强图像。
在一个实施例中，在图像处理过程中，可以采取上述第一图像处理策略，或者，也可以采取上述第二图像处理策略。采取不同的图像处理策略可以确定出不同的第一质量分类结果，从而确定出不同的目标预增强图像。
在一个实施例中，若第一图像块的第一质量分类结果表示影响第一图像块图像质量的各影响因素中存在的概率最大的影响因素，则可以确定第一质量分类结果表示的影响因素对应的预增强图像，作为该第一图像块对应的目标预增强图像。
在一个实施例中，若第一质量分类结果表示各影响因素存在的概率，则可以确定影响第一图像块图像质量的各影响因素中存在的概率大于第一阈值的影响因素，进而，确定该影响因素对应的预增强图像，作为第一图像块对应的目标预增强图像。在该情况下，确定出的目标预增强图像可以为一个，也可以为多个。
在一个实施例中，其确定目标预增强图像与步骤S103类似，在此不再赘述。
S205：在目标预增强图像中确定出第二图像块。
在一个实施例中，若第一质量分类结果表示各影响因素存在的概率可以通过以下方式实现：
如果确定出的目标预增强图像是一个，则可以确定该目标预增强图像中所属的区域与第一图像块在待处理图像中所属的区域相同的图像块，作为第二图像块。
如果确定出的目标预增强图像是多个，则可以获取各目标预增强图像中每个像素的像素值，并且将各目标预增强图像中对应目标像素的像素的像素值进行加权求和，以得到对应目标像素的综合像素值。其中，目标像素为第一图像块中的一个像素，每个目标预增强图像中像素的权重为每个目标预增强图像对应的影响因素在第一图像块中存在的概率，然后，可以将目标像素的像素值调整为综合像素值，以生成对应第一图像块的第二图像块。
其中，对于第一图像块的每一个像素来说，可以分别获取无噪声低分辨率增强图像、去采集噪声增强图像、去压缩噪声增强图像和去模糊增强图像中对应第一图像块的像素的像素值。例如，获取的像素值分别为x1、x2、x3和x4，并分别将15％、35％、25％和25％作为x1、x2、x3和x4的权重，进行加权求和，并将求和结果作为像素的像素值。
S206：根据第二图像块，生成目标图像。
在得到每个第二图像块后，可以按照各个第二图像块各自对应的第一图像块在待处理图像中的的位置，对各个第二图像块进行拼接，得到目标图像。
本申请实施例中，具体实现方式与步骤S105相同或相似，在此不再赘述。
本申请实施例提供的上述如图2所示的图像生成方法中，可以对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像。进而，可以将待处理图像中的第一图像块输入预先建立的质量分类模型，得到第一图像块对应的初始质量分类结果。然后，基于预设的图像处理策略和第一图像块对应的初始质量分类结果，确定出第一图像块对应的第一质量分类结果，并基于第一图像块对应的目标影响因素，在多张预增强图像中确定出目标预增强图像。进而，在目标预增强图像中确定出第二图像块，以及根据第二图像块，生成目标图像。由于对第一图像块进行质量分类，从而可以确定影响第一图像块图像质量的影响因素，再结合通过预增强图像确定出第二图像块，生成目标图像从而提高了图像增强的效果。
基于同一发明构思，根据本申请实施例提供的图像生成方法，本申请实施例还提供了一种图像生成装置，如图4所示，该装置包括：
图像增强处理模块401，设置为对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张预增强图像对应一种类型的图像增强处理，每种类型的图像增强处理对应一种影响图像质量的影响因素；
图像质量分类模块402，设置为对待处理图像中的第一图像块进行图像质量分类，得到第一图像块对应的第一质量分类结果，其中，第一质量分类结果用于指示第一图像块中存在的影响图像质量的影响因素；
图像确定模块403，设置为基于第一图像块对应的第一质量分类结果，在多张预增强图像中确定出目标预增强图像，其中，目标预增强图像所对应的图像增强处理的类型与第一质量分类结果所指示的第一图像块中存在的影响图像质量的影响因素相匹配；
第二图像块确定模块404，设置为在目标预增强图像中确定出第二图像块，其中，第二图像块在预增强图像中所属的区域与第一图像块在待处理图像中所属的区域相同；
目标图像生成模块405，设置为根据第二图像块，生成目标图像。
在一实施方式中，图像质量分类模块402，具体设置为将待处理图像中的第一图像块输入预先建立的质量分类模型，得到第一图像块对应的初始质量分类结果，其中，质量分类模型用于对图像进行质量分类，并且基于预设的图像处理策略和第一图像块对应的初始质量分类结果，确定出第一图像块对应的第一质量分类结果。
在一实施方式中，初始质量分类结果包括各影响因素存在的概率；
图像质量分类模块402，具体设置为确定出第一图像块对应的初始质量分类结果中存在的概率最大的影响因素，作为第一图像块对应的目标影响因素，并且根据第一图像块对应的目标影响因素，确定出第一图像块对应的第一质量分类结果。
在一实施方式中，图像确定模块403，具体设置为基于第一图像块对应的 目标影响因素，在多张预增强图像中确定出目标预增强图像。
在一实施方式中，图像质量分类模块402，具体设置为基于第一图像块在待处理图像中的位置，确定出与第一图像块在位置上存在对应关系的图像块，作为第一图像块对应的参考图像块，并且根据第一图像块对应的初始质量分类结果，以及第一图像块对应的参考图像块所对应的初始质量分类结果，确定出第一图像块对应的综合质量分类结果，作为第一图像块对应的第一质量分类结果。
在一实施方式中，图像质量分类模块402，具体设置为判断第一图像块对应的参考图像块所对应的初始质量分类结果是否相同，并且若相同，则将第一图像块对应的参考图像块所对应的初始质量分类结果确定为第一图像块对应的综合质量分类结果，以及若不同，则将第一图像块对应的初始质量分类结果确定为第一图像块对应的综合质量分类结果。
在一实施方式中，初始质量分类结果包括各影响因素存在的概率；
图像质量分类模块402，具体设置为根据第一图像块对应的参考图像块中每个影响因素存在的概率，调整第一图像块中每个影响因素存在的概率，并将第一图像块中每个影响因素调整后的存在的概率作为第一图像块的综合质量分类结果所包含的每个影响因素存在的概率。
在一实施方式中，第二图像块确定模块404，设置为获取各目标预增强图像中每个像素的像素值，并且将各目标预增强图像中对应目标像素的像素的像素值进行加权求和，以得到对应目标像素的综合像素值，其中，目标像素为第一图像块中的一个像素，每个目标预增强图像中像素的权重为每个目标预增强图像对应的影响因素在第一图像块中存在的概率，以及将目标像素的像素值调整为综合像素值，以生成对应第一图像块的第二图像块。
在一实施方式中，影响因素至少包括无噪声低分辨率、压缩噪声、采集噪声和模糊；
图像增强处理模块401，具体设置为将待处理图像输入预先建立的第一模型，得到对应无噪声低分辨率的预增强图像，其中，第一模型用于提高待处理图像的分辨率，并且将待处理图像输入预先建立的第二模型，得到对应压 缩噪声的预增强图像，其中，第二模型用于去除待处理图像所包含的压缩噪音，并且将待处理图像输入预先建立的第三模型，得到对应采集噪声的预增强图像，其中，第三模型用于去除待处理图像所包含的采集噪音，并且将待处理图像输入预先建立的第四模型，得到对应模糊的预增强图像，其中，第四模型用于消除待处理图像所包含的模糊。
在一实施方式中，所述目标图像生成模块405，具体设置为按照所述第二图像块各自对应的第一图像块在所述待处理图像中的位置，对所述第二图像块进行拼接，得到目标图像。
本申请实施例还提供了一种电子设备，如图5所示，包括处理器501、通信接口502、存储器503和通信总线504，其中，处理器501，通信接口502，存储器503通过通信总线504完成相互间的通信，
存储器503，设置为存放计算机程序；
处理器501，设置为执行存储器503上所存放的程序时，实现如下步骤：
对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张所述预增强图像对应一种类型的所述图像增强处理，每种类型的所述图像增强处理对应一种影响图像质量的影响因素；
对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，其中，所述第一质量分类结果用于指示所述第一图像块中存在的影响图像质量的影响因素；
基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，其中，所述目标预增强图像所对应的图像增强处理的类型与所述第一质量分类结果所指示的所述第一图像块中存在的影响图像质量的影响因素相匹配；
在所述目标预增强图像中确定出第二图像块，其中，所述第二图像块在所述预增强图像中所属的区域与所述第一图像块在所述待处理图像中所属的区域相同；
根据所述第二图像块，生成目标图像。
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
通信接口用于上述电子设备与其他设备之间的通信。
存储器可以包括随机存取存储器(Random Access Memory，RAM)，也可以包括非易失性存储器(Non-Volatile Memory，NVM)，例如至少一个磁盘存储器。可选的，存储器还可以是至少一个位于远离前述处理器的存储装置。
上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，CPU)、网络处理器(Network Processor，NP)等；还可以是数字信号处理器(Digital Signal Processing，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
在本申请提供的又一实施例中，还提供了一种计算机可读存储介质，该计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一图像生成方法的步骤。
在本申请提供的又一实施例中，还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中任一图像生成方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所 述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、电子设备、计算机可读存储介质、计算机程序产品而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。
工业实用性
基于本申请实施例提供的上述技术方案，由于对第一图像块进行质量分类，从而可以确定影响第一图像块图像质量的影响因素，再结合通过预增强图像确定出第二图像块，生成目标图像，也就是可以对待处理图像中的不同的图像块根据自身的图像质量进行对应的增强处理，解决了对整张图像进行的单一图像处理效果差的问题，从而提高了图像增强的效果。

Claims (13)

  1. 一种图像生成方法，包括：
    对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张所述预增强图像对应一种类型的所述图像增强处理，每种类型的所述图像增强处理对应一种影响图像质量的影响因素；
    对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，其中，所述第一质量分类结果用于指示所述第一图像块中存在的影响图像质量的影响因素；
    基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，其中，所述目标预增强图像所对应的图像增强处理的类型与所述第一质量分类结果所指示的所述第一图像块中存在的影响图像质量的影响因素相匹配；
    在所述目标预增强图像中确定出第二图像块，其中，所述第二图像块在所述预增强图像中所属的区域与所述第一图像块在所述待处理图像中所属的区域相同；
    根据所述第二图像块，生成目标图像。
  2. 根据权利要求1所述的方法，其中，所述对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，包括：
    将所述待处理图像中的第一图像块输入预先建立的质量分类模型，得到所述第一图像块对应的初始质量分类结果，其中，所述质量分类模型用于对图像进行质量分类；
    基于预设的图像处理策略和所述第一图像块对应的初始质量分类结果，确定出所述第一图像块对应的第一质量分类结果。
  3. 根据权利要求2所述的方法，其中，所述初始质量分类结果包括各所述影响因素存在的概率；
    所述基于预设的图像处理策略和所述第一图像块对应的初始质量分类结 果，确定出所述第一图像块对应的第一质量分类结果，包括：
    确定出所述第一图像块对应的初始质量分类结果中存在的概率最大的影响因素，作为所述第一图像块对应的目标影响因素；
    根据所述第一图像块对应的目标影响因素，确定出所述第一图像块对应的第一质量分类结果。
  4. 根据权利要求3所述的方法，其中，所述基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，包括：
    基于所述第一图像块对应的目标影响因素，在多张所述预增强图像中确定出目标预增强图像。
  5. 根据权利要求2所述的方法，其中，所述基于预设的图像处理策略和所述第一图像块对应的初始质量分类结果，确定出所述第一图像块对应的第一质量分类结果，包括：
    基于所述第一图像块在所述待处理图像中的位置，确定出与所述第一图像块在位置上存在对应关系的图像块，作为所述第一图像块对应的参考图像块；
    根据所述第一图像块对应的初始质量分类结果，以及所述第一图像块对应的参考图像块所对应的初始质量分类结果，确定出所述第一图像块对应的综合质量分类结果，作为所述第一图像块对应的第一质量分类结果。
  6. 根据权利要求5所述的方法，其中，所述根据所述第一图像块对应的初始质量分类结果，以及所述第一图像块对应的参考图像块所对应的初始质量分类结果，确定出所述第一图像块对应的综合质量分类结果，作为所述第一图像块对应的第一质量分类结果，包括：
    判断所述第一图像块对应的参考图像块所对应的初始质量分类结果是否相同；
    若相同，则将所述第一图像块对应的参考图像块所对应的初始质量分类结果确定为所述第一图像块对应的综合质量分类结果；
    若不同，则将所述第一图像块对应的初始质量分类结果确定为所述第一 图像块对应的综合质量分类结果。
  7. 根据权利要求5所述的方法，其中，所述初始质量分类结果包括各所述影响因素存在的概率；
    所述根据所述第一图像块对应的初始质量分类结果，以及所述第一图像块对应的参考图像块所对应的初始质量分类结果，确定出所述第一图像块对应的综合质量分类结果，包括：
    根据所述第一图像块对应的参考图像块中每个所述影响因素存在的概率，调整所述第一图像块中每个所述影响因素存在的概率，并将所述第一图像块中每个所述影响因素调整后的存在的概率作为所述第一图像块的综合质量分类结果所包含的每个所述影响因素存在的概率。
  8. 根据权利要求6或7所述的方法，其中，所述在所述目标预增强图像中确定出第二图像块，包括：
    获取各所述目标预增强图像中每个像素的像素值；
    将各所述目标预增强图像中对应目标像素的像素的像素值进行加权求和，以得到对应所述目标像素的综合像素值，其中，所述目标像素为所述第一图像块中的一个像素，每个所述目标预增强图像中像素的权重为每个所述目标预增强图像对应的影响因素在所述第一图像块中存在的概率；
    将所述目标像素的像素值调整为所述综合像素值，以生成对应所述第一图像块的第二图像块。
  9. 根据权利要求1所述的方法，其中，所述影响因素至少包括无噪声低分辨率、压缩噪声、采集噪声和模糊；
    所述对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，包括：
    将所述待处理图像输入预先建立的第一模型，得到对应所述无噪声低分辨率的预增强图像，其中，所述第一模型用于提高所述待处理图像的分辨率；
    将所述待处理图像输入预先建立的第二模型，得到对应所述压缩噪声的预增强图像，其中，所述第二模型用于去除所述待处理图像所包含的所述压 缩噪音；
    将所述待处理图像输入预先建立的第三模型，得到对应所述采集噪声的预增强图像，其中，所述第三模型用于去除所述待处理图像所包含的所述采集噪音；
    将所述待处理图像输入预先建立的第四模型，得到对应所述模糊的预增强图像，其中，所述第四模型用于消除所述待处理图像所包含的所述模糊。
  10. 根据权利要求1所述的方法，其中，所述根据所述第二图像块，生成目标图像，包括：
    按照所述第二图像块各自对应的第一图像块在所述待处理图像中的位置，对所述第二图像块进行拼接，得到目标图像。
  11. 一种图像生成装置，包括：
    图像增强处理模块，设置为对待处理图像分别进行多个类型的图像增强处理，以得到多张预增强图像，其中，每张所述预增强图像对应一种类型的所述图像增强处理，每种类型的所述图像增强处理对应一种影响图像质量的影响因素；
    图像质量分类模块，设置为对所述待处理图像中的第一图像块进行图像质量分类，得到所述第一图像块对应的第一质量分类结果，其中，所述第一质量分类结果用于指示所述第一图像块中存在的影响图像质量的影响因素；
    图像确定模块，设置为基于所述第一图像块对应的第一质量分类结果，在多张所述预增强图像中确定出目标预增强图像，其中，所述目标预增强图像所对应的图像增强处理的类型与所述第一质量分类结果所指示的所述第一图像块中存在的影响图像质量的影响因素相匹配；
    第二图像块确定模块，设置为在所述目标预增强图像中确定出第二图像块，其中，所述第二图像块在所述预增强图像中所属的区域与所述第一图像块在所述待处理图像中所属的区域相同；
    目标图像生成模块，设置为根据所述第二图像块，生成目标图像。
  12. 一种电子设备，包括处理器和存储器；
    所述存储器上存储有计算机程序，所述计算机程序在被所述处理器运行时执行如权利要求1至10任一项所述的方法。
  13. 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器运行时执行权利要求1至10中任一项所述的方法。
PCT/CN2020/117213 2019-10-31 2020-09-23 一种图像生成方法、装置及电子设备 WO2021082819A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/259,907 US11836898B2 (en) 2019-10-31 2020-09-23 Method and apparatus for generating image, and electronic device
SG11202100123VA SG11202100123VA (en) 2019-10-31 2020-09-23 Method and apparatus for generating image, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911055646.9 2019-10-31
CN201911055646.9A CN110796624B (zh) 2019-10-31 2019-10-31 一种图像生成方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2021082819A1 true WO2021082819A1 (zh) 2021-05-06

Family

ID=69440666

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/117213 WO2021082819A1 (zh) 2019-10-31 2020-09-23 一种图像生成方法、装置及电子设备

Country Status (4)

Country Link
US (1) US11836898B2 (zh)
CN (1) CN110796624B (zh)
SG (1) SG11202100123VA (zh)
WO (1) WO2021082819A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110796624B (zh) * 2019-10-31 2022-07-05 北京金山云网络技术有限公司 一种图像生成方法、装置及电子设备
US20220067879A1 (en) * 2020-09-03 2022-03-03 Nvidia Corporation Image enhancement using one or more neural networks
CN112070744B (zh) * 2020-09-08 2022-11-22 济南博观智能科技有限公司 一种人脸识别的方法、系统、设备及可读存储介质
CN112967273B (zh) * 2021-03-25 2021-11-16 北京的卢深视科技有限公司 图像处理方法、电子设备及存储介质
CN114677578B (zh) * 2022-03-30 2024-09-13 腾讯科技（深圳）有限公司 确定训练样本数据的方法和装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150043810A1 (en) * 2011-09-23 2015-02-12 Alibaba Group Holding Limited Image quality analysis for searches
CN105894507A (zh) * 2016-03-31 2016-08-24 西安电子科技大学 基于图像信息量自然场景统计特征的图像质量评价方法
CN107481238A (zh) * 2017-09-20 2017-12-15 众安信息技术服务有限公司 图像质量评估方法及装置
CN109685785A (zh) * 2018-12-20 2019-04-26 上海众源网络有限公司 一种图像质量评估方法、装置及电子设备
CN109949264A (zh) * 2017-12-20 2019-06-28 深圳先进技术研究院 一种图像质量评价方法、设备及存储设备
WO2019144469A1 (zh) * 2018-01-24 2019-08-01 华讯方舟科技有限公司 一种图像质量分类方法、系统及终端设备
CN110363753A (zh) * 2019-07-11 2019-10-22 北京字节跳动网络技术有限公司 图像质量评估方法、装置及电子设备
CN110796624A (zh) * 2019-10-31 2020-02-14 北京金山云网络技术有限公司 一种图像生成方法、装置及电子设备

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8717456B2 (en) * 2002-02-27 2014-05-06 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
CN1177298C (zh) * 2002-09-19 2004-11-24 上海交通大学 基于块分割的多聚焦图像融合方法
JP4123356B2 (ja) * 2002-11-13 2008-07-23 富士ゼロックス株式会社 画像処理装置、画像処理プログラム、記憶媒体
US8682097B2 (en) * 2006-02-14 2014-03-25 DigitalOptics Corporation Europe Limited Digital image enhancement with reference images
US7822286B2 (en) * 2003-11-07 2010-10-26 Mitsubishi Electric Research Laboratories, Inc. Filtering artifacts in images with 3D spatio-temporal fuzzy filters
US7346224B2 (en) * 2003-11-07 2008-03-18 Mitsubishi Electric Research Laboratories, Inc. System and method for classifying pixels in images
JP4659036B2 (ja) 2004-08-20 2011-03-30 シリコン オプティックス インコーポレイテッド エッジ適応画像拡大強化システム及び方法
US7933454B2 (en) * 2007-06-25 2011-04-26 Xerox Corporation Class-based image enhancement system
US8472716B2 (en) * 2007-12-05 2013-06-25 Canon Kabushiki Kaisha Block-based noise detection and reduction method with pixel level classification granularity
US20120015825A1 (en) * 2010-07-06 2012-01-19 Pacific Biosciences Of California, Inc. Analytical systems and methods with software mask
US10742475B2 (en) * 2012-12-05 2020-08-11 Origin Wireless, Inc. Method, apparatus, and system for object tracking sensing using broadcasting
US8724919B2 (en) * 2012-09-21 2014-05-13 Eastman Kodak Company Adjusting the sharpness of a digital image
JP6254938B2 (ja) * 2012-10-04 2017-12-27 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 画像ノイズ除去装置、および画像ノイズ除去方法
TWI486906B (zh) * 2012-12-14 2015-06-01 Univ Nat Central Using Image Classification to Strengthen Image Matching
CN103905822A (zh) * 2014-03-25 2014-07-02 公安部第一研究所 一种基于视觉敏感度和空域滤波的低比特压缩图像去块效应方法
US9186909B1 (en) * 2014-09-26 2015-11-17 Intel Corporation Method and system of lens shading color correction using block matching
US9538126B2 (en) * 2014-12-03 2017-01-03 King Abdulaziz City For Science And Technology Super-resolution of dynamic scenes using sampling rate diversity
US10235608B2 (en) * 2015-12-22 2019-03-19 The Nielsen Company (Us), Llc Image quality assessment using adaptive non-overlapping mean estimation
CN107545544A (zh) * 2016-06-29 2018-01-05 诺基亚技术有限公司 去除图像中的混浊对象的方法和装置
CN106251318B (zh) * 2016-09-29 2023-05-23 杭州雄迈集成电路技术股份有限公司 一种序列图像的去噪装置及方法
CN107224359A (zh) * 2017-07-20 2017-10-03 李良 自动控制的轮椅
CN108629747B (zh) 2018-04-25 2019-12-10 腾讯科技（深圳）有限公司 图像增强方法、装置、电子设备及存储介质
CN109697705B (zh) 2018-12-24 2019-09-03 北京天睿空间科技股份有限公司 适于视频拼接的色差矫正方法
CN109859152B (zh) 2018-12-25 2021-05-25 北京奇艺世纪科技有限公司 模型生成方法、图像增强方法、装置及计算机可读存储介质
US11195060B2 (en) * 2019-07-05 2021-12-07 Art Eye-D Associates Llc Visualization of subimage classifications

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150043810A1 (en) * 2011-09-23 2015-02-12 Alibaba Group Holding Limited Image quality analysis for searches
CN105894507A (zh) * 2016-03-31 2016-08-24 西安电子科技大学 基于图像信息量自然场景统计特征的图像质量评价方法
CN107481238A (zh) * 2017-09-20 2017-12-15 众安信息技术服务有限公司 图像质量评估方法及装置
CN109949264A (zh) * 2017-12-20 2019-06-28 深圳先进技术研究院 一种图像质量评价方法、设备及存储设备
WO2019144469A1 (zh) * 2018-01-24 2019-08-01 华讯方舟科技有限公司 一种图像质量分类方法、系统及终端设备
CN109685785A (zh) * 2018-12-20 2019-04-26 上海众源网络有限公司 一种图像质量评估方法、装置及电子设备
CN110363753A (zh) * 2019-07-11 2019-10-22 北京字节跳动网络技术有限公司 图像质量评估方法、装置及电子设备
CN110796624A (zh) * 2019-10-31 2020-02-14 北京金山云网络技术有限公司 一种图像生成方法、装置及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI YANMEI: "Research on Key Techniques of Image Enhancement and Its Applications", CHINESE DOCTORAL DISSERTATIONS FULL-TEXT DATABASE, 29 June 2013 (2013-06-29), XP055808943 *

Also Published As

Publication number Publication date
US20210295475A1 (en) 2021-09-23
US11836898B2 (en) 2023-12-05
SG11202100123VA (en) 2021-06-29
CN110796624A (zh) 2020-02-14
CN110796624B (zh) 2022-07-05

Similar Documents

Publication Publication Date Title
WO2021082819A1 (zh) 一种图像生成方法、装置及电子设备
CN109872304B (zh) 图像缺陷检测方法及装置、电子设备、存储介质
WO2021057848A1 (zh) 网络的训练方法、图像处理方法、网络、终端设备及介质
CN110705583B (zh) 细胞检测模型训练方法、装置、计算机设备及存储介质
CN113168684B (zh) 提升低亮度图像的质量的方法、系统和计算机可读介质
US11354797B2 (en) Method, device, and system for testing an image
WO2020168699A1 (en) Neural network for enhancing original image, and computer-implemented method for enhancing original image using neural network
US20200082508A1 (en) Information processing method, information processing apparatus, and recording medium
CN110213479B (zh) 一种视频拍摄防抖方法和装置
WO2023125503A1 (zh) 一种暗光图像降噪方法及装置
CN111882565B (zh) 一种图像二值化方法、装置、设备及存储介质
WO2023125440A1 (zh) 一种降噪方法、装置、电子设备及介质
CN111325671B (zh) 网络训练方法、装置、图像处理方法及电子设备
CN110689496A (zh) 降噪模型的确定方法、装置、电子设备和计算机存储介质
Vasconcelos et al. Cuf: Continuous upsampling filters
CN110717864A (zh) 一种图像增强方法、装置、终端设备及计算机可读介质
CN111222446B (zh) 人脸识别方法、人脸识别装置及移动终端
CN110136061B (zh) 一种基于深度卷积预测与插值的分辨率提升方法及系统
CN111489289B (zh) 一种图像处理方法、图像处理装置及终端设备
JP2015179426A (ja) 情報処理装置、パラメータの決定方法、及びプログラム
CN115937121A (zh) 基于多维度特征融合的无参考图像质量评价方法及系统
CN114119377B (zh) 一种图像处理方法及装置
CN113239738B (zh) 一种图像的模糊检测方法及模糊检测装置
CN114861904A (zh) 图像训练数据的生成方法、装置、终端设备及存储介质
Hwang et al. Source camera identification based on interpolation via lens distortion correction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20881136

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20881136

Country of ref document: EP

Kind code of ref document: A1