WO2022105779A1 - 图像处理方法、模型训练方法、装置、介质及设备 - Google Patents

图像处理方法、模型训练方法、装置、介质及设备 Download PDF

Info

Publication number
WO2022105779A1
WO2022105779A1 PCT/CN2021/131155 CN2021131155W WO2022105779A1 WO 2022105779 A1 WO2022105779 A1 WO 2022105779A1 CN 2021131155 W CN2021131155 W CN 2021131155W WO 2022105779 A1 WO2022105779 A1 WO 2022105779A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target object
difference information
resolution
model
Prior art date
Application number
PCT/CN2021/131155
Other languages
English (en)
French (fr)
Inventor
孙佳
袁泽寰
王长虎
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Priority to US18/252,979 priority Critical patent/US20240013359A1/en
Publication of WO2022105779A1 publication Critical patent/WO2022105779A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present application is based on the CN application number 202011298847.4 and the filing date is Nov. 18, 2020, and claims its priority.
  • the disclosure of the CN application is hereby incorporated into the present application as a whole.
  • the present disclosure relates to the technical field of image processing, and in particular, to an image processing method, a model training method, an apparatus, a medium, and a device.
  • the quality of an image is affected by many factors, such as blurring, ripples, and dense noise due to noise, image compression, and other reasons.
  • the restoration and processing of images are mainly performed manually by technicians.
  • the manual processing method is time-consuming and labor-intensive, especially for the processing of a large number of image frames in videos or movies, and the efficiency is low.
  • the current methods of image restoration usually directly process the entire image as a whole, and the processing effect on specific objects in the image is not good.
  • the present disclosure provides an image processing method, the method comprising: extracting a first target object image from an image to be processed; inputting the first target object image into a target object image processing model to obtain the The second target object image output by the target object image processing model, wherein the resolution of the second target object image is higher than that of the first target object image; Image fusion to get the target image.
  • the target object image processing model is a generative adversarial network model including a generator, and the target object image processing model is obtained by training in the following way: using a low-resolution image of the original training sample image as the generated
  • the input of the generator is to obtain the high-resolution image output after the generator processes the low-resolution image; according to the target difference information between the high-resolution image and the original training sample image, it is determined whether the model is trained Completed, wherein the target difference information includes at least one of the following: feature point difference information between the high-resolution image and the original training sample image, the high-resolution image and the original training sample
  • the difference information of the specified feature area in the image in response to the completion of the model training, the image processing model of the target object is obtained.
  • the present disclosure provides a training method for a target object image processing model
  • the target object image processing model is a generative adversarial network model including a generator
  • the method includes: The image is used as the input of the generator, and the high-resolution image output by the generator after processing the low-resolution image is obtained; according to the target difference information between the high-resolution image and the original training sample image , determine whether the training of the model is completed, wherein the target difference information includes at least one of the following: feature point difference information between the high-resolution image and the original training sample image, the high-resolution image and the The specified feature area difference information in the original training sample image; in response to the completion of model training, the target object image processing model is obtained.
  • the present disclosure provides an image processing apparatus, the apparatus comprising: an extraction module for extracting a first target object image from an image to be processed; an input module for inputting the first target object image into a In the target object image processing model, a second target object image output by the target object image processing model is obtained, wherein the resolution of the second target object image is higher than that of the first target object image; the image fusion module, using performing image fusion according to the second target object image and the to-be-processed image to obtain a target image.
  • the target object image processing model is a generative adversarial network model including a generator, and the target object image processing model is obtained by training a training device for the target object image processing model.
  • the training of the target object image processing model The device includes: an image obtaining module, configured to use the low-resolution image of the original training sample image as the input of the generator to obtain a high-resolution image output by the generator after processing the low-resolution image; a determining module , for determining whether the model is trained according to the target difference information between the high-resolution image and the original training sample image, wherein the target difference information includes at least one of the following: the high-resolution The feature point difference information between the image and the original training sample image, and the specified feature area difference information in the high-resolution image and the original training sample image; the model obtaining module is used to respond to the completion of the model training, obtain The target object image processing model.
  • the present disclosure provides a training device for a target object image processing model
  • the target object image processing model is a generative adversarial network model including a generator
  • the device includes: an image acquisition module for converting the original training The low-resolution image of the sample image is used as the input of the generator to obtain a high-resolution image outputted by the generator after processing the low-resolution image; the determining module is configured to match the high-resolution image with the high-resolution image.
  • the target difference information between the original training sample images is used to determine whether the training of the model is completed, wherein the target difference information includes at least one of the following: a feature between the high-resolution image and the original training sample image point difference information, and the difference information of the designated feature area in the high-resolution image and the original training sample image; a model obtaining module, configured to obtain the target object image processing model in response to the completion of model training.
  • the present disclosure provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, implements the steps of the method provided in the first aspect of the present disclosure.
  • the present disclosure provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing apparatus, implements the steps of the method provided in the second aspect of the present disclosure.
  • the present disclosure provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device, so as to implement the computer program provided in the first aspect of the present disclosure. the steps of the method.
  • the present disclosure provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device, so as to implement the computer program provided in the second aspect of the present disclosure. the steps of the method.
  • the first target object image extracted from the image to be processed is input into the target object image processing model to obtain a second target object image with a higher resolution, and the first target object image in the to-be-processed image is individually After processing, the processed second target object image with higher resolution and the to-be-processed image are image-fused to obtain a target image, which can make the target object in the obtained target image clearer and the details more realistic.
  • the model is trained through the difference information of feature points or the difference information of the specified feature area, without comparing the entire image, the model training speed is faster, or the target difference information can include both of these two.
  • the difference information considered is more comprehensive, so that the difference between the high-resolution image and the original training sample image can be more accurately characterized according to the target difference information.
  • Training the model according to the target difference information can make the difference between the high-resolution image generated by the generator and the original training sample image smaller, that is, the image is more accurate.
  • FIG. 1 is a flow chart of a training method of a target object image processing model according to an exemplary embodiment.
  • Fig. 2 is a flowchart of an image processing method according to an exemplary embodiment.
  • Fig. 3 is a block diagram of an image processing apparatus according to an exemplary embodiment.
  • Fig. 4 is a block diagram of an apparatus for training a target object image processing model according to an exemplary embodiment.
  • Fig. 5 is a schematic structural diagram of an electronic device according to an exemplary embodiment.
  • the term “including” and variations thereof are open-ended inclusions, ie, "including but not limited to”.
  • the term “based on” is “based at least in part on.”
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; the term “some embodiments” means “at least some embodiments”. Relevant definitions of other terms will be given in the description below.
  • the target object can be objects such as people, faces, buildings, plants, animals, etc.
  • the present disclosure does not make specific restrictions on the target object.
  • the model can be used to perform resolution enhancement processing on the target object image.
  • FIG. 1 is a flow chart of a training method of a target object image processing model according to an exemplary embodiment, and the method can be applied to an electronic device with processing capability.
  • the target object image processing model may be a generative adversarial network model including a generator, such as enhanced super-resolution generative adversarial networks (ESRGAN, Enhanced Super-Resolution Generative Adversarial Networks), super-resolution generative adversarial networks (SRGAN, Super- Resolution Generative Adversarial Networks).
  • ESRGAN enhanced super-resolution generative adversarial networks
  • SRGAN Super- Resolution Generative Adversarial Networks
  • the method may include S101 to S103.
  • the low-resolution image of the original training sample image is used as the input of the generator, and the high-resolution image output by the generator after processing the low-resolution image is obtained.
  • the original training sample image may be any preset image, and the original training sample image may be an individual image or an image frame in a video file.
  • an image including the target object in the image may be used as the original training sample image, for example, if the target object is a human face, the image including the human face may be used as the original training sample image.
  • the low-resolution image of the original training sample image can be obtained in various ways, for example, by down-sampling the original training sample image, or by blurring the original training sample image to obtain the corresponding low-resolution image.
  • Resolution The resolution of the image is not specifically limited in the present disclosure.
  • the generator in the generative adversarial network model can be used to perform super-resolution processing on the image, that is, to improve the resolution of the image, so that the image is clearer and more realistic. After the low-resolution image of the original training sample image is input to the generator, the high-resolution image output by the generator can be obtained.
  • target difference information between the high-resolution image and the original training sample image can be obtained.
  • the target difference information may include at least one of the following: feature point difference information between the high-resolution image and the original training sample image, and specified feature area difference information between the high-resolution image and the original training sample image.
  • the difference information may include, for example, pixel difference information, color difference information, and the like.
  • the feature points in the image can be key points in the image, and the feature points can be obtained by performing feature point detection on the image.
  • the feature points in the image may include key points in the human face.
  • the feature points can be extracted from the high-resolution image, and the feature points can be extracted from the original training sample image, so as to determine the difference between the feature points extracted from the high-resolution image and the feature points extracted from the original training sample image.
  • Feature point difference information can be used to determine the difference between the feature points extracted from the high-resolution image and the feature points extracted from the original training sample image.
  • the designated feature region in the image may be a feature region in the image.
  • the designated feature regions of the image may be regions such as eyes, mouth, nose, and the like.
  • the designated feature area of the image can be leaves or flowers.
  • the specified feature region can be extracted from the high-resolution image, and the specified feature region can also be extracted from the original training sample image. Taking a face image as an example, for example, the region where the eyes are located in the image are both extracted, and the difference between the two is determined. The difference information is used as the difference information of the specified feature area.
  • the target difference information may include feature point difference information or specified feature area difference information, and the model is trained by the feature point difference information or the specified feature area difference information, without the need to compare the entire image, and the model training speed is faster.
  • the target difference information may include both, and the considered difference information is more comprehensive, so that the difference between the high-resolution image and the original training sample image can be more accurately represented according to the target difference information.
  • the generative adversarial network also includes a discriminator, which is used to distinguish the authenticity of the high-resolution images generated by the generator.
  • Training the generator and the discriminator according to the target difference information can make the difference between the high-resolution image generated by the generator and the original training sample image smaller, that is, the image is more accurate, the image details are more realistic, and the discrimination can be improved.
  • whether the model is trained can be determined according to the target difference information between the high-resolution image and the original training sample image. Among them, if the difference between the high-resolution image and the original training sample image is large, it can indicate that the image generated by the generator is not accurate enough and realistic enough, and the model needs to be trained continuously.
  • the conditions for the completion of model training may include that the difference between the high-resolution image and the original training sample image is small, and the discriminator judges that the high-resolution image generated by the generator is real.
  • Target object image processing model may include that the difference between the high-resolution image and the original training sample image is small, and the discriminator judges that the high-resolution image generated by the generator is real.
  • the target difference information may include at least one of the following: feature point difference information between the high-resolution image and the original training sample image, and specified feature area difference information between the high-resolution image and the original training sample image.
  • the model is trained through the feature point difference information or the specified feature area difference information, without the need to compare the entire image, the model training speed is faster, or the target difference information can include both, the difference information considered is more comprehensive, Therefore, the difference between the high-resolution image and the original training sample image can be more accurately characterized according to the target difference information. Training the model according to the target difference information can make the difference between the high-resolution image generated by the generator and the original training sample image smaller, that is, the image is more accurate and the image details are more realistic.
  • the target difference information between the high-resolution image and the original training sample image may further include: between the high-resolution image adjusted to the preset resolution and the original training sample image adjusted to the preset resolution overall difference information.
  • the high-resolution image after obtaining the high-resolution image generated by the generator, the high-resolution image can be adjusted to the preset resolution, and the original training sample image can also be adjusted to the preset resolution, that is, the resolution of the two can be adjusted. In order to be consistent, the overall difference information between the two after adjustment is compared.
  • the present disclosure does not specifically limit the value of the preset resolution.
  • the target difference information in the present disclosure may include at least one of feature point difference information and specified feature area difference information and the overall difference information.
  • the target difference information is more comprehensive, and the model can not only be trained from the perspective of the overall difference, but also can be trained according to the difference information between the feature points and/or the specified feature regions, so that the high-resolution generated by the generator can be trained.
  • the image is closer to the original training sample image, and the image details are more realistic.
  • the target object image processing model may further include a discriminator, and the above-mentioned S102 may include: the degree of difference represented by each type of difference information included in the target difference information is smaller than the corresponding difference degree threshold, and the discriminator pair When the true-false judgment result of the high-resolution image is true, it is determined that the model training is completed.
  • the target difference information including the overall difference information, the feature point difference information and the specified feature area difference information
  • the difference degree represented by the overall difference information is less than the corresponding first difference degree threshold
  • the feature point difference information The degree of difference is less than the corresponding second degree of difference threshold
  • the degree of difference represented by the difference information of the specified feature area is less than the corresponding third degree of difference threshold, that is, it is considered that the difference between the high-resolution image and the original training sample image is small
  • the high-resolution images generated by the generator are more accurate, and if the discriminator's true and false determination results of the high-resolution images are true at this time, that is, the discriminator cannot distinguish the true and false of the high-resolution images generated by the generator, it can be determined.
  • the image processing model of the target object is obtained.
  • the first difference degree threshold, the second difference degree threshold, and the third difference degree threshold may be pre-calibrated, and may be the same or different.
  • the difference information included in the target difference information contains difference information whose degree of difference is greater than or equal to the corresponding threshold of difference degree, or the discriminator’s true or false determination result of the high-resolution image is false, it can indicate that the model has not been trained yet. , another original training sample image can be obtained at this time to continue training the model.
  • the target object image processing model further includes a discriminator; the above S102 may include: fusing each type of difference information included in the target difference information to obtain fusion difference information; When it is less than the preset fusion difference degree threshold, and the discriminator's true or false determination result of the high-resolution image is true, it is determined that the model training is completed.
  • the manner of merging each kind of difference information may be to perform weighting processing on the difference degrees represented by each kind of difference information, and there is no specific limitation on the weight occupied by each kind of difference information. If the degree of difference represented by the fusion difference information is less than the preset fusion difference degree threshold, it can indicate that the high-resolution image generated by the generator is more accurate, and at this time, if the discriminator's true or false determination result of the high-resolution image is true , it can be determined that the model training is completed, that is, the image processing model of the target object is obtained.
  • the discriminator's true or false determination result for the high-resolution image is false, or, the degree of difference represented by the fusion difference information is greater than or equal to the preset fusion degree of difference threshold, that is, the difference between the high-resolution image and the original training sample image
  • the preset fusion degree of difference threshold that is, the difference between the high-resolution image and the original training sample image
  • the condition for whether the model is trained can include that the degree of difference represented by each type of difference information included in the target difference information is smaller than the corresponding difference degree threshold, or the degree of difference represented by the fusion difference information is smaller than the preset difference.
  • the difference threshold is fused, so that the target difference information can include various difference information between the high-resolution image and the original training sample image, and training the model according to the various difference information can make the obtained target object image processing model more accurate .
  • Fig. 2 is a flowchart of an image processing method according to an exemplary embodiment. As shown in Fig. 2 , the image processing method may include S201-S203.
  • a first target object image is extracted from the image to be processed.
  • the image to be processed may be a pre-stored image or an image captured by a user in real time, and the image to be processed may also be an image frame in a video file. If the image to be processed is an image frame of a video file, multiple Image frames are processed separately.
  • the first target object image may be an image of a target object detected from an image to be processed, for example, the target object is a human face, and the first target object image may be an extracted face image, for example, the target object is a building, and the first target object image may be an extracted human face image.
  • a target object image may be an extracted building image.
  • the first target object image is input into the target object image processing model to obtain the second target object image output by the target object image processing model.
  • the resolution of the second target object image is higher than that of the first target object image.
  • the first target object image After the first target object image is extracted from the image to be processed, the first target object image can be input into the target object image processing model, and the target object image processing model can be used to enhance the resolution of the first target object image process, and output a higher resolution and clearer image of the second target object.
  • the target object image processing model is a generative adversarial network model including a generator, and the target object image processing model may be obtained by training in the manner shown in FIG. 1:
  • the low-resolution image of the original training sample is The image is used as the input of the generator, and the high-resolution image output by the generator after processing the low-resolution image is obtained.
  • the target difference information may include at least one of the following: the high-resolution image and the original training sample image The difference information between the feature points, the high-resolution image and the specified feature area difference information in the original training sample image.
  • a target object image processing model is obtained in response to the completion of the model training.
  • image fusion is performed according to the second target object image and the to-be-processed image to obtain a target image.
  • the target object may be a key part in the image to be processed, that is, a relatively significant part.
  • the first target object image in the image to be processed is separately processed, and then the processed second target object with higher resolution is processed.
  • Image fusion of the image and the image to be processed can make the target object in the obtained target image clearer, and improve the problem that the target object is not clear caused by directly processing the entire image of the image to be processed.
  • the image to be processed may be an image frame in a video, and each image frame in the video may be processed separately to obtain a video file with higher resolution and higher definition.
  • the first target object image extracted from the image to be processed is input into the target object image processing model to obtain a second target object image with a higher resolution, and the first target object image in the to-be-processed image is individually After processing, the processed second target object image with higher resolution and the to-be-processed image are image-fused to obtain a target image, which can make the target object in the obtained target image clearer and the details more realistic.
  • the model is trained through the difference information of feature points or the difference information of the specified feature area, without comparing the entire image, the model training speed is faster, or the target difference information can include both of these two.
  • the difference information considered is more comprehensive, so that the difference between the high-resolution image and the original training sample image can be more accurately characterized according to the target difference information.
  • Training the model according to the target difference information can make the difference between the high-resolution image generated by the generator and the original training sample image smaller, that is, the image is more accurate.
  • performing image fusion according to the second target object image and the to-be-processed image to obtain the target image may include: performing resolution enhancement processing on the to-be-processed image to obtain the target to-be-processed image; The object image and the target to-be-processed image are image-fused to obtain the target image.
  • the image to be processed can be input into an image processing model to obtain the target image to be processed output by the image processing model.
  • the image processing model may be different from the above-mentioned target object image processing model, and the image processing model may be any network model for performing full image processing on the image to be processed, that is, improving the resolution of the image to be processed.
  • the to-be-processed image may be up-sampled to obtain a target to-be-processed image with a higher resolution.
  • the second target object image processed by the target object image processing model can be image-fused with the target to-be-processed image to obtain a target image with a resolution of the target image.
  • the target object details in the image are more realistic.
  • Poisson fusion may be used for image fusion, and the edge transition of the fusion part may be made more natural by using Poisson fusion.
  • FIG. 3 is a block diagram of an image processing apparatus according to an exemplary embodiment. As shown in FIG. 3 , the image processing apparatus 300 may include:
  • the extraction module 301 is used for extracting the first target object image from the image to be processed
  • the input module 302 is configured to input the first target object image into the target object image processing model to obtain a second target object image output by the target object image processing model, wherein the resolution of the second target object image higher than the first target object image;
  • An image fusion module 303 configured to perform image fusion according to the second target object image and the to-be-processed image to obtain a target image
  • the target object image processing model is a generative adversarial network model including a generator
  • the target object image processing model may be obtained by training the training device 400 of the target object image processing model shown in FIG. 4 , as shown in FIG.
  • the training device 400 of the target object image processing model may include:
  • the image obtaining module 401 is configured to use the low-resolution image of the original training sample image as the input of the generator, and obtain the high-resolution image output by the generator after processing the low-resolution image;
  • a determination module 402 configured to determine whether the model is trained according to target difference information between the high-resolution image and the original training sample image, wherein the target difference information includes at least one of the following: the Feature point difference information between the high-resolution image and the original training sample image, and specified feature area difference information between the high-resolution image and the original training sample image;
  • the model obtaining module 403 is configured to obtain the target object image processing model in response to the completion of model training.
  • the first target object image extracted from the image to be processed is input into the target object image processing model to obtain a second target object image with a higher resolution, and the first target object image in the to-be-processed image is individually After processing, the processed second target object image with higher resolution and the to-be-processed image are image-fused to obtain a target image, which can make the target object in the obtained target image clearer and the details more realistic.
  • the model is trained through the difference information of feature points or the difference information of the specified feature area, without comparing the entire image, the model training speed is faster, or the target difference information can include both of these two.
  • the difference information considered is more comprehensive, so that the difference between the high-resolution image and the original training sample image can be more accurately characterized according to the target difference information.
  • Training the model according to the target difference information can make the difference between the high-resolution image generated by the generator and the original training sample image smaller, that is, the image is more accurate.
  • the image fusion module 303 may include: a resolution enhancement processing sub-module for performing resolution enhancement processing on the to-be-processed image to obtain the target to-be-processed image; an image fusion sub-module for performing resolution enhancement processing on the to-be-processed image;
  • the second target object image and the target to-be-processed image are image-fused to obtain the target image.
  • FIG. 4 is a block diagram of an apparatus for training a target object image processing model according to an exemplary embodiment, where the target object image processing model is a generative adversarial network model including a generator, and the apparatus 400 may include:
  • the image obtaining module 401 is used for taking the low-resolution image of the original training sample image as the input of the generator, and obtaining the high-resolution image output by the generator after processing the low-resolution image;
  • a determination module 402 configured to determine whether the model is trained according to target difference information between the high-resolution image and the original training sample image, wherein the target difference information includes at least one of the following: the Feature point difference information between the high-resolution image and the original training sample image, and specified feature area difference information between the high-resolution image and the original training sample image;
  • the model obtaining module 403 is configured to obtain the target object image processing model in response to the completion of model training.
  • the target difference information further includes overall difference information between the high-resolution image adjusted to the preset resolution and the original training sample image adjusted to the preset resolution.
  • the target object image processing model further includes a discriminator; the determining module 402 may include: a first determining sub-module for determining the degree of difference represented by each type of difference information included in the target difference information If all are smaller than the respective corresponding difference thresholds, and the discriminator's true or false determination result of the high-resolution image is true, it is determined that the model training is completed.
  • the target object image processing model further includes a discriminator;
  • the determining module 402 may include: a fusion sub-module, configured to fuse each type of difference information included in the target difference information to obtain a fusion difference information; a second determination sub-module, used for when the degree of difference represented by the fusion difference information is less than a preset fusion degree of difference threshold, and the discriminator’s true or false determination result of the high-resolution image is true In this case, it is determined that the model training is completed.
  • Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, and the like.
  • the electronic device shown in FIG. 5 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.
  • an electronic device 500 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 501 that may be loaded into random access according to a program stored in a read only memory (ROM) 502 or from a storage device 508 Various appropriate actions and processes are executed by the programs in the memory (RAM) 503 . In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored.
  • the processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504.
  • An input/output (I/O) interface 505 is also connected to bus 504 .
  • I/O interface 505 input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration
  • An output device 507 such as a computer
  • a storage device 508 including, for example, a magnetic tape, a hard disk, etc.
  • Communication means 509 may allow electronic device 500 to communicate wirelessly or by wire with other devices to exchange data. While FIG. 5 shows electronic device 500 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via the communication device 509, or from the storage device 508, or from the ROM 502.
  • the processing apparatus 501 When the computer program is executed by the processing apparatus 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.
  • the client and server can use any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) to communicate, and can communicate with digital data in any form or medium Communication (eg, a communication network) interconnects.
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.
  • the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: extracts a first target object image from an image to be processed; The object image is input into the target object image processing model, and a second target object image output by the target object image processing model is obtained, wherein the resolution of the second target object image is higher than that of the first target object image; according to The second target object image and the to-be-processed image are subjected to image fusion to obtain a target image; wherein, the target object image processing model is a generative adversarial network model including a generator, and the target object image processing model is obtained by It is obtained by training in the following way: taking the low-resolution image of the original training sample image as the input of the generator, and obtaining the high-resolution image output by the generator after processing the low-resolution image; according to the high-resolution image The target difference information between the high-resolution image and the original training sample image is used to determine whether the model is trained, wherein the target
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device causes the electronic device to: use the low-resolution image of the original training sample image as the input, obtain the high-resolution image output after the generator processes the low-resolution image; determine whether the model is trained according to the target difference information between the high-resolution image and the original training sample image,
  • the target difference information includes at least one of the following: feature point difference information between the high-resolution image and the original training sample image, information on the difference between the high-resolution image and the original training sample image
  • the specified feature area difference information in response to the completion of model training, the target object image processing model is obtained.
  • Computer program code for performing operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and This includes conventional procedural programming languages - such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider to via Internet connection).
  • LAN local area network
  • WAN wide area network
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.
  • the modules involved in the embodiments of the present disclosure may be implemented in software or hardware. Wherein, the name of the module does not constitute a limitation of the module itself under certain circumstances, for example, the extraction module may also be described as a "target object image extraction module”.
  • exemplary types of hardware logic components include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical Devices (CPLDs) and more.
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs Systems on Chips
  • CPLDs Complex Programmable Logical Devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device.
  • the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • Example 1 provides an image processing method, the method comprising: extracting a first target object image from an image to be processed; inputting the first target object image into a target object In the image processing model, a second target object image output by the target object image processing model is obtained, wherein the resolution of the second target object image is higher than that of the first target object image; according to the second target object image
  • the image and the image to be processed are image-fused to obtain the target image;
  • the target object image processing model is a generative confrontation network model including a generator, and the target object image processing model is obtained by training in the following manner:
  • the low-resolution image of the original training sample image is used as the input of the generator to obtain a high-resolution image output after the generator processes the low-resolution image; according to the high-resolution image and the original Target difference information between training sample images, to determine whether the model is trained, wherein the target difference information includes at least one of the following: the feature point difference between the high-resolution image and the
  • Example 2 provides the method of Example 1, and the target difference information further includes a high-resolution image adjusted to a preset resolution and a high-resolution image adjusted to the preset resolution Overall difference information between the original training sample images.
  • Example 3 provides the method of Example 1, the target object image processing model further includes a discriminator; The target difference information of the target difference information, and determining whether the training of the model is completed, including: the degree of difference represented by each type of difference information included in the target difference information is smaller than the corresponding difference degree threshold, and the discriminator In the case where the true-false determination result of the image is true, it is determined that the model training is completed.
  • Example 4 provides the method of Example 1, the target object image processing model further includes a discriminator; The target difference information of the target difference information, and determining whether the training of the model is completed, including: fusing each kind of difference information included in the target difference information to obtain fusion difference information; the degree of difference represented by the fusion difference information is less than the preset fusion.
  • the difference threshold is set, and the discriminator's true-false determination result on the high-resolution image is true, it is determined that the model training is completed.
  • Example 5 provides the method according to any one of Examples 1-4, wherein image fusion is performed according to the second target object image and the to-be-processed image to obtain a target image , comprising: performing resolution enhancement processing on the to-be-processed image to obtain a target to-be-processed image; and performing image fusion according to the second target object image and the target to-be-processed image to obtain the target image.
  • Example 6 provides a method for training a target object image processing model, where the target object image processing model is a generative adversarial network model including a generator, and the method includes: The low-resolution image of the original training sample image is used as the input of the generator, and the high-resolution image output by the generator after processing the low-resolution image is obtained; according to the high-resolution image and the original training Target difference information between sample images, to determine whether the model is trained, wherein the target difference information includes at least one of the following: feature point difference information between the high-resolution image and the original training sample image , the difference information of the designated feature area in the high-resolution image and the original training sample image; in response to the completion of model training, the target object image processing model is obtained.
  • the target object image processing model is a generative adversarial network model including a generator
  • the method includes: The low-resolution image of the original training sample image is used as the input of the generator, and the high-resolution image output by the generator after processing the low-resolution image
  • Example 7 provides the method of Example 6, the target difference information further includes a high-resolution image adjusted to a preset resolution and a high-resolution image adjusted to the preset resolution Overall difference information between the original training sample images.
  • Example 8 provides the method of Example 6, the target object image processing model further includes a discriminator; The target difference information of the target difference information, and determining whether the training of the model is completed, including: the degree of difference represented by each type of difference information included in the target difference information is smaller than the corresponding difference degree threshold, and the discriminator In the case where the true-false determination result of the image is true, it is determined that the model training is completed.
  • Example 9 provides the method of Example 6, the target object image processing model further includes a discriminator; The target difference information of the target difference information, and determining whether the training of the model is completed, including: fusing each kind of difference information included in the target difference information to obtain fusion difference information; the degree of difference represented by the fusion difference information is less than the preset fusion.
  • the difference threshold is set, and the discriminator's true-false determination result on the high-resolution image is true, it is determined that the model training is completed.
  • Example 10 provides an image processing apparatus, the apparatus includes: an extraction module for extracting a first target object image from an image to be processed; an input module for The first target object image is input into the target object image processing model, and the second target object image output by the target object image processing model is obtained, wherein the resolution of the second target object image is higher than that of the first target object.
  • the target object image processing model is a generative confrontation network model including a generator
  • the target object image processing model is obtained by training a training device for the target object image processing model
  • the training device for the target object image processing model includes: an image acquisition module for using the low-resolution image of the original training sample image as a The input of the generator is to obtain the high-resolution image outputted by the generator after processing the low-resolution image; the determination module is used for determining according to the difference between the high-resolution image and the original training sample image.
  • Target difference information to determine whether the model is trained, wherein the target difference information includes at least one of the following: feature point difference information between the high-resolution image and the original training sample image, the high-resolution image The difference information of the designated feature area in the rate image and the original training sample image; the model obtaining module is used for obtaining the target object image processing model in response to the completion of the model training.
  • Example 11 provides an apparatus for training a target object image processing model, where the target object image processing model is a generative adversarial network model including a generator, and the apparatus includes: an image The obtaining module is used to take the low-resolution image of the original training sample image as the input of the generator, and obtain the high-resolution image output after the low-resolution image is processed by the generator; the determining module is used for according to The target difference information between the high-resolution image and the original training sample image, to determine whether the model is trained, wherein the target difference information includes at least one of the following: the high-resolution image and the feature point difference information between the original training sample images, and the specified feature area difference information in the high-resolution image and the original training sample image; a model obtaining module, used for obtaining the target object in response to the completion of model training Image processing model.
  • the target object image processing model is a generative adversarial network model including a generator
  • the apparatus includes: an image The obtaining module is used to take the low-resolution image of the
  • Example 12 provides a computer-readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the steps of the method described in any one of Examples 1-5 .
  • Example 13 provides a computer-readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the steps of the method described in any one of Examples 6-9 .
  • Example 14 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device, to The steps of implementing the method of any of Examples 1-5.
  • Example 15 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to The steps of implementing the method of any of Examples 6-9.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及一种图像处理方法、模型训练方法、装置、介质及设备，该图像处理方法包括：从待处理图像中提取第一目标对象图像；将第一目标对象图像输入到目标对象图像处理模型中，得到目标对象图像处理模型输出的第二目标对象图像；根据第二目标对象图像和待处理图像进行图像融合，得到目标图像。

Description

图像处理方法、模型训练方法、装置、介质及设备
本申请是以CN申请号为202011298847.4，申请日为2020年11月18日的申请为基础，并主张其优先权，该CN申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及图像处理技术领域，具体地，涉及一种图像处理方法、模型训练方法、装置、介质及设备。
背景技术
图像的质量受到很多因素的影响，例如由于噪声、图像压缩等原因导致图像出现模糊、波纹、噪点密集等现象。目前对图像的修复和处理工作主要通过技术人员进行人工处理，然而人工处理的方式耗时耗力，特别是对于视频或电影中大量的图像帧的处理，效率较低。而且目前对图像进行修复的方式通常直接对整张图像进行整体处理，对图像中特定对象的处理效果不佳。
发明内容
提供该发明内容部分以便以简要的形式介绍构思，这些构思将在后面的具体实施方式部分被详细描述。该发明内容部分并不旨在标识要求保护的技术方案的关键特征或必要特征，也不旨在用于限制所要求的保护的技术方案的范围。
第一方面，本公开提供一种图像处理方法，所述方法包括：从待处理图像中提取第一目标对象图像；将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像。
优选地，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过如下方式训练得到的：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定 特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
第二方面，本公开提供一种目标对象图像处理模型的训练方法，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述方法包括：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
第三方面，本公开提供一种图像处理装置，所述装置包括：提取模块，用于从待处理图像中提取第一目标对象图像；输入模块，用于将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；图像融合模块，用于根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像。
优选地，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过目标对象图像处理模型的训练装置训练得到的，该目标对象图像处理模型的训练装置包括：图像获得模块，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；确定模块，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；模型获得模块，用于响应于模型训练完成，得到所述目标对象图像处理模型。
第四方面，本公开提供一种目标对象图像处理模型的训练装置，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述装置包括：图像获得模块，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；确定模块，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；模型获得模块，用于响应于模型训练完成，得到所述目标对象图像处理模型。
第五方面，本公开提供一种计算机可读介质，其上存储有计算机程序，该程序被处理 装置执行时实现本公开第一方面提供的所述方法的步骤。
第六方面，本公开提供一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现本公开第二方面提供的所述方法的步骤。
第七方面，本公开提供一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开第一方面提供的所述方法的步骤。
第八方面，本公开提供一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现本公开第二方面提供的所述方法的步骤。
通过上述技术方案，将从待处理图像中提取的第一目标对象图像输入到目标对象图像处理模型中，得到分辨率更高的第二目标对象图像，对待处理图像中的第一目标对象图像单独进行处理，再将处理之后的分辨率更高的第二目标对象图像和待处理图像进行图像融合，得到目标图像，可以使得得到的目标图像中目标对象更为清晰，细节更加真实。在目标对象图像处理模型的训练阶段，通过特征点差异信息或指定特征区域差异信息对模型进行训练，无需将整个图像进行比对，模型训练速度更快，或者，目标差异信息可同时包括这两者，考虑的差异信息更为全面，从而根据该目标差异信息可以更加准确的表征高分辨率图像与原始训练样本图像之间的差异。根据目标差异信息对模型进行训练，可以使得生成器生成的高分辨率图像与原始训练样本图像之间的差异更小，即图像更准确。
本公开的其他特征和优点将在随后的具体实施方式部分予以详细说明。
附图说明
结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。在附图中：
图1是根据一示例性实施例示出的一种目标对象图像处理模型的训练方法的流程图。
图2是根据一示例性实施例示出的一种图像处理方法的流程图。
图3是根据一示例性实施例示出的一种图像处理装置的框图。
图4是根据一示例性实施例示出的一种目标对象图像处理模型的训练装置的框图。
图5是根据一示例性实施例示出的一种电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
首先介绍本公开实施例中目标对象图像处理模型的训练方法，目标对象可以是人物、人脸、建筑物、植物、动物等等对象，本公开对目标对象不做具体限制，该目标对象图像处理模型可用于对目标对象图像进行分辨率增强处理。
图1是根据一示例性实施例示出的一种目标对象图像处理模型的训练方法的流程图，该方法可应用于具有处理能力的电子设备中。该目标对象图像处理模型可以是包括生成器的生成式对抗网络模型，例如增强型超分辨率生成对抗网络(ESRGAN，Enhanced Super-Resolution Generative Adversarial Networks)、超分辨率生成对抗网络(SRGAN，Super-Resolution Generative Adversarial Networks)。如图1所示，该方法可包括S101～S103。
在S101中，将原始训练样本图像的低分辨率图像作为生成器的输入，得到生成器对低分辨率图像处理之后输出的高分辨率图像。
其中，原始训练样本图像可以是预先设置的任一图像，该原始训练样本图像可以是单独的图像，也可以是视频文件中的图像帧。在一示例中，可将图像中包括目标对象的图像作为原始训练样本图像，例如目标对象为人脸，可将包括人脸的图像作为原始训练样本图像。
原始训练样本图像的低分辨率图像可以通过多种方式获得，例如，对原始训练样本图像进行降采样处理，或者，对原始训练样本图像进行模糊化处理，得到对应的低分辨率图像，对于低分辨率图像的分辨率，本公开不做具体限制。
其中，生成式对抗网络模型中的生成器可用于对图像进行超分辨率处理，即提高图像的分辨率，使得图像的清晰度更高，更加逼真。将原始训练样本图像的低分辨率图像输入到生成器之后，可得到生成器处理之后输出的高分辨图像。
在S102中，根据高分辨率图像与原始训练样本图像之间的目标差异信息，确定模型是否训练完成。
其中，得到生成器输出的高分辨率图像之后，可获取该高分辨率图像与原始训练样本图像之间的目标差异信息。本公开中，目标差异信息可包括以下中的至少一者：高分辨率图像与原始训练样本图像之间的特征点差异信息、高分辨率图像与原始训练样本图像中的指定特征区域差异信息。差异信息例如可包括像素差异信息、色彩差异信息等。
图像中的特征点可以是图像中的关键点，该特征点可通过对图像进行特征点检测获得。示例地，以原始训练样本图像中包括人脸为例，图像中的特征点可包括人脸中的关键点。其中，可从高分辨率图像中提取特征点，并从原始训练样本图像中提取特征点，从而确定从高分辨率图像中提取的特征点和从原始训练样本图像中提取的特征点之间的特征点差异信息。
图像中的指定特征区域可以是图像中具有特征性的区域。示例地，若原始训练样本图像中包括人脸，图像的指定特征区域可以是眼睛、嘴巴、鼻子等区域。若原始训练样本图像中包括植物，图像的指定特征区域可以是叶片或花朵。其中，可从高分辨图像中提取指定特征区域，并从原始训练样本图像中也提取该指定特征区域，以人脸图像为例，例如均提取图像中眼睛所在的区域，并确定二者之间的差异信息，该差异信息作为指定特征区域差异信息。
示例地，目标差异信息可包括特征点差异信息或指定特征区域差异信息，通过特征点差异信息或指定特征区域差异信息对模型进行训练，无需将整个图像进行比对，模型训练速度更快。或者，目标差异信息可同时包括这两者，考虑的差异信息更为全面，从而根据该目标差异信息可以更加准确的表征高分辨率图像与原始训练样本图像之间的差异。其中，生成式对抗网络还包括判别器，用于分辨生成器生成的高分辨率图像的真实性。根据目标差异信息对生成器和判别器进行训练，可以使得生成器生成的高分辨率图像与原始训练样本图像之间的差异更小，即图像更准确，图像细节更真实，并且，可以提高判别器进行真假判定的分辨能力。
本公开中，可根据高分辨率图像与原始训练样本图像之间的目标差异信息，确定模型是否训练完成。其中，如果高分辨率图像与原始训练样本图像之间的差异较大，可表明生成器生成的图像不够准确不够逼真，需要继续对模型进行训练。
在S103中，响应于模型训练完成，得到目标对象图像处理模型。
模型训练完成的条件可以包括高分辨率图像与原始训练样本图像之间的差异较小，以及，判别器判断生成器生成的高分辨率图像是真实的，此时可表征模型训练完成，可得到目标对象图像处理模型。
如果高分辨率图像与原始训练样本图像之间的差异较大，或者，判别器判断高分辨率图像为假，可表征模型未训练完成，可继续获取另一原始训练样本图像，并返回S101，继续对模型进行训练。
通过上述技术方案，根据高分辨率图像与原始训练样本图像之间的目标差异信息，确定模型是否训练完成，响应于模型训练完成，可得到目标对象图像处理模型。其中，目标差异信息可包括以下中的至少一者：高分辨率图像与原始训练样本图像之间的特征点差异信息、高分辨率图像与原始训练样本图像中的指定特征区域差异信息。通过特征点差异信息或指定特征区域差异信息对模型进行训练，无需将整个图像进行比对，模型训练速度更快，或者，目标差异信息可同时包括这两者，考虑的差异信息更为全面，从而根据该目标差异信息可以更加准确的表征高分辨率图像与原始训练样本图像之间的差异。根据目标差异信息对模型进行训练，可以使得生成器生成的高分辨率图像与原始训练样本图像之间的差异更小，即图像更准确，图像细节更真实。
本公开中，高分辨率图像与原始训练样本图像之间的目标差异信息还可包括：调整至预设分辨率之后的高分辨率图像和调整至预设分辨率之后的原始训练样本图像之间的整体差异信息。
其中，在获取到生成器生成的高分辨率图像之后，可将高分辨率图像调整至预设分辨率，并将原始训练样本图像也调整至该预设分辨率，即将二者的分辨率调整为一致，再对比调整之后的二者之间的整体差异信息。对于预设分辨率的取值本公开不做具体限制。
在一实施例中，本公开中目标差异信息可包括特征点差异信息和指定特征区域差异信息中的至少一者以及该整体差异信息。这样，目标差异信息更为全面，不但可以从整体差异的角度对模型进行训练，而且可以根据特征点和/或指定特征区域之间的差异信息对模型进行训练，使得生成器生成的高分辨率图像与原始训练样本图像更接近，图像细节更加真实。
在一实施方式中，目标对象图像处理模型还可包括判别器，上述S102可包括：在目 标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值、且判别器对高分辨率图像的真假判定结果为真实的情况下，确定模型训练完成。
示例地，以目标差异信息同时包括整体差异信息、特征点差异信息和指定特征区域差异信息为例，若整体差异信息所表征的差异度小于对应的第一差异度阈值、特征点差异信息所表征的差异度小于对应的第二差异度阈值、且指定特征区域差异信息所表征的差异度小于对应的第三差异度阈值，即认为高分辨率图像与原始训练样本图像之间的差异较小，生成器生成的高分辨率图像较为准确，并且，如果此时判别器对高分辨率图像的真假判定结果为真实，即判别器无法分辨生成器生成的高分辨率图像的真假，可确定模型训练完成，即获得目标对象图像处理模型。其中，第一差异度阈值、第二差异度阈值和第三差异度阈值均可预先标定出，可以相同也可以不同。
如果目标差异信息包括的差异信息中存在所表征的差异度大于或等于对应的差异度阈值的差异信息，或者，判别器对高分辨率图像的真假判定结果为假，可表征模型尚未训练完成，此时可获取另一原始训练样本图像，继续对模型进行训练。
在另一实施方式中，目标对象图像处理模型还包括判别器；上述S102可包括：将目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；在融合差异信息所表征的差异度小于预设的融合差异度阈值、且判别器对高分辨率图像的真假判定结果为真实的情况下，确定模型训练完成。
示例地，将每种差异信息进行融合的方式可以为将每种差异信息所表征的差异度进行加权处理，对于各个差异信息所占的权重不做具体限制。如果该融合差异信息所表征的差异度小于预设的融合差异度阈值，可表征生成器生成的高分辨率图像较为准确，并且此时如果判别器对高分辨率图像的真假判定结果为真实，可确定模型训练完成，即得到目标对象图像处理模型。
如果判别器对高分辨率图像的真假判定结果为假，或者，该融合差异信息所表征的差异度大于或等于预设的融合差异度阈值，即高分辨率图像与原始训练样本图像之间的差异较大，生成器生成的高分辨率图像不够准确，则需要对模型继续进行训练。
通过上述技术方案，模型是否训练完成的条件可包括目标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值，或者，融合差异信息所表征的差异度小于预设的融合差异度阈值，这样，目标差异信息可包括高分辨率图像与原始训练样本图像之间的多种差异信息，根据多种差异信息对模型进行训练，可以使得得到的目标对象图像处理模型更准确。
本公开还提供一种图像处理方法，图2是根据一示例性实施例示出的一种图像处理方 法的流程图，如图2所示，该图像处理方法可包括S201～S203。
在S201中，从待处理图像中提取第一目标对象图像。
其中，待处理图像可以是预先存储的图像，或者用户实时拍摄的图像，该待处理图像也可以是视频文件中的图像帧，如果待处理图像是视频文件的图像帧，可以对视频中多个图像帧分别进行处理。
第一目标对象图像可以是从待处理图像中检测出的目标对象的图像，例如目标对象为人脸，该第一目标对象图像可以是提取出的人脸图像，例如目标对象为建筑物，该第一目标对象图像可以是提取出的建筑物图像。
在S202中，将第一目标对象图像输入到目标对象图像处理模型中，得到目标对象图像处理模型输出的第二目标对象图像。其中，该第二目标对象图像的分辨率高于第一目标对象图像。
在从待处理图像中提取出第一目标对象图像后，可将该第一目标对象图像输入到目标对象图像处理模型中，该目标对象图像处理模型可用于对第一目标对象图像进行分辨率增强处理，并输出分辨率更高更清晰的第二目标对象图像。
其中，目标对象图像处理模型为包括生成器的生成式对抗网络模型，该目标对象图像处理模型可以是通过图1所示的方式训练得到的：在S101中，将原始训练样本图像的低分辨率图像作为生成器的输入，得到生成器对低分辨率图像处理之后输出的高分辨率图像。在S102中，根据高分辨率图像与原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，目标差异信息可包括以下中的至少一者：高分辨率图像与原始训练样本图像之间的特征点差异信息、高分辨率图像与原始训练样本图像中的指定特征区域差异信息。在S103中，响应于模型训练完成，得到目标对象图像处理模型。
其中，目标对象图像处理模型的具体训练过程已在上文详细说明，此处不再赘述。
在S203中，根据第二目标对象图像和待处理图像进行图像融合，得到目标图像。
目标对象可以是待处理图像中的重点部分，即较为显著的部分，本公开中，对待处理图像中的第一目标对象图像单独进行处理，再将处理之后的分辨率更高的第二目标对象图像和待处理图像进行图像融合，可以使得得到的目标图像中目标对象更为清晰，改善直接对待处理图像进行全图整体处理导致的目标对象不清晰的问题。
在一实施例中，待处理图像可以为视频中的图像帧，可对视频中每一图像帧分别进行处理，则可得到分辨率更高、清晰度更高的视频文件。
通过上述技术方案，将从待处理图像中提取的第一目标对象图像输入到目标对象图像处理模型中，得到分辨率更高的第二目标对象图像，对待处理图像中的第一目标对象图像 单独进行处理，再将处理之后的分辨率更高的第二目标对象图像和待处理图像进行图像融合，得到目标图像，可以使得得到的目标图像中目标对象更为清晰，细节更加真实。在目标对象图像处理模型的训练阶段，通过特征点差异信息或指定特征区域差异信息对模型进行训练，无需将整个图像进行比对，模型训练速度更快，或者，目标差异信息可同时包括这两者，考虑的差异信息更为全面，从而根据该目标差异信息可以更加准确的表征高分辨率图像与原始训练样本图像之间的差异。根据目标差异信息对模型进行训练，可以使得生成器生成的高分辨率图像与原始训练样本图像之间的差异更小，即图像更准确。
在一可选实施例中，S203中根据第二目标对象图像和待处理图像进行图像融合，得到目标图像，可包括：对待处理图像进行分辨率增强处理，得到目标待处理图像；根据第二目标对象图像和目标待处理图像进行图像融合，得到目标图像。
其中，对待处理图像进行分辨率增强处理的方式可以有多种，示例地，可以将待处理图像输入至图像处理模型中，以得到该图像处理模型输出的目标待处理图像。该图像处理模型与上述的目标对象图像处理模型可以不同，该图像处理模型可以是任一种网络模型，用于对待处理图像进行全图处理，即提高待处理图像的分辨率。再示例地，可以对待处理图像进行上采样处理，以得到分辨率更高的目标待处理图像。
在得到分辨率更高的目标待处理图像后，可将经目标对象图像处理模型处理之后的第二目标对象图像与该目标待处理图像进行图像融合，从而得到目标图像，该目标图像的分辨率更高，并且图像中的目标对象细节更加真实。示例地，图像融合的方式可以采用泊松融合，采用泊松融合的方式可以使得融合部分的边缘过渡更加自然。
基于同一发明构思，本公开还提供一种图像处理装置，图3是根据一示例性实施例示出的一种图像处理装置的框图，如图3所示，该图像处理装置300可包括：
提取模块301，用于从待处理图像中提取第一目标对象图像；
输入模块302，用于将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；
图像融合模块303，用于根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像；
其中，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型可以是通过图4所示的目标对象图像处理模型的训练装置400训练得到的，如图4所示，该目标对象图像处理模型的训练装置400可包括：
图像获得模块401，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入， 得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；
确定模块402，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
模型获得模块403，用于响应于模型训练完成，得到所述目标对象图像处理模型。
通过上述技术方案，将从待处理图像中提取的第一目标对象图像输入到目标对象图像处理模型中，得到分辨率更高的第二目标对象图像，对待处理图像中的第一目标对象图像单独进行处理，再将处理之后的分辨率更高的第二目标对象图像和待处理图像进行图像融合，得到目标图像，可以使得得到的目标图像中目标对象更为清晰，细节更加真实。在目标对象图像处理模型的训练阶段，通过特征点差异信息或指定特征区域差异信息对模型进行训练，无需将整个图像进行比对，模型训练速度更快，或者，目标差异信息可同时包括这两者，考虑的差异信息更为全面，从而根据该目标差异信息可以更加准确的表征高分辨率图像与原始训练样本图像之间的差异。根据目标差异信息对模型进行训练，可以使得生成器生成的高分辨率图像与原始训练样本图像之间的差异更小，即图像更准确。
可选地，所述图像融合模块303，可包括：分辨率增强处理子模块，用于对所述待处理图像进行分辨率增强处理，得到目标待处理图像；图像融合子模块，用于根据所述第二目标对象图像和所述目标待处理图像进行图像融合，得到所述目标图像。
图4是根据一示例性实施例示出的一种目标对象图像处理模型的训练装置的框图，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述装置400可包括：
图像获得模块401，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；
确定模块402，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
模型获得模块403，用于响应于模型训练完成，得到所述目标对象图像处理模型。
可选地，所述目标差异信息还包括调整至预设分辨率之后的高分辨率图像和调整至所述预设分辨率之后的原始训练样本图像之间的整体差异信息。
可选地，所述目标对象图像处理模型还包括判别器；所述确定模块402，可包括：第一确定子模块，用于在所述目标差异信息包括的每种差异信息所表征的差异度均小于各自 对应的差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
可选地，所述目标对象图像处理模型还包括判别器；所述确定模块402，可包括：融合子模块，用于将所述目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；第二确定子模块，用于在所述融合差异信息所表征的差异度小于预设的融合差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关方法的实施例中进行了详细描述，此处将不做详细阐述说明。
下面参考图5，其示出了适于用来实现本公开实施例的电子设备500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(ROM)502中的程序或者从存储装置508加载到随机访问存储器(RAM)503中的程序而执行各种适当的动作和处理。在RAM 503中，还存储有电子设备500操作所需的各种程序和数据。处理装置501、ROM 502以及RAM 503通过总线504彼此相连。输入/输出(I/O)接口505也连接至总线504。
通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM 502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：从待处理图像中提取第一目标对象图像；将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像；其中，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过如下方式训练得到的：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异 信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
或者，上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言——诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块的名称在某种情况下并不构成对该模块本身的限定，例如，提取模块还可以被描述为“目标对象图像提取模块”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非 限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，示例1提供了一种图像处理方法，所述方法包括：从待处理图像中提取第一目标对象图像；将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像；其中，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过如下方式训练得到的：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
根据本公开的一个或多个实施例，示例2提供了示例1的方法，所述目标差异信息还包括调整至预设分辨率之后的高分辨率图像和调整至所述预设分辨率之后的原始训练样本图像之间的整体差异信息。
根据本公开的一个或多个实施例，示例3提供了示例1的方法，所述目标对象图像处理模型还包括判别器；所述根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：在所述目标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
根据本公开的一个或多个实施例，示例4提供了示例1的方法，所述目标对象图像处理模型还包括判别器；所述根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：将所述目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；在所述融合差异信息所表征的差异度小于预设的融合差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
根据本公开的一个或多个实施例，示例5提供了示例1-4任一项所述的方法，所述根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像，包括：对所述待处理图像进行分辨率增强处理，得到目标待处理图像；根据所述第二目标对象图像和所述目标待处理图像进行图像融合，得到所述目标图像。
根据本公开的一个或多个实施例，示例6提供了一种目标对象图像处理模型的训练方法，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述方法包括：将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；响应于模型训练完成，得到所述目标对象图像处理模型。
根据本公开的一个或多个实施例，示例7提供了示例6的方法，所述目标差异信息还包括调整至预设分辨率之后的高分辨率图像和调整至所述预设分辨率之后的原始训练样本图像之间的整体差异信息。
根据本公开的一个或多个实施例，示例8提供了示例6的方法，所述目标对象图像处理模型还包括判别器；所述根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：在所述目标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
根据本公开的一个或多个实施例，示例9提供了示例6的方法，所述目标对象图像处理模型还包括判别器；所述根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：将所述目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；在所述融合差异信息所表征的差异度小于预设的融合差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训 练完成。
根据本公开的一个或多个实施例，示例10提供了一种图像处理装置，所述装置包括：提取模块，用于从待处理图像中提取第一目标对象图像；输入模块，用于将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；图像融合模块，用于根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像；其中，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过目标对象图像处理模型的训练装置训练得到的，该目标对象图像处理模型的训练装置包括：图像获得模块，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；确定模块，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；模型获得模块，用于响应于模型训练完成，得到所述目标对象图像处理模型。
根据本公开的一个或多个实施例，示例11提供了一种目标对象图像处理模型的训练装置，所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述装置包括：图像获得模块，用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；确定模块，用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；模型获得模块，用于响应于模型训练完成，得到所述目标对象图像处理模型。
根据本公开的一个或多个实施例，示例12提供了一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现示例1-5中任一项所述方法的步骤。
根据本公开的一个或多个实施例，示例13提供了一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现示例6-9中任一项所述方法的步骤。
根据本公开的一个或多个实施例，示例14提供了一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现示例1-5中任一项所述方法的步骤。
根据本公开的一个或多个实施例，示例15提供了一种电子设备，包括：存储装置， 其上存储有计算机程序；处理装置，用于执行所述存储装置中的所述计算机程序，以实现示例6-9中任一项所述方法的步骤。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。

Claims (17)

  1. 一种图像处理方法，所述方法包括：
    从待处理图像中提取第一目标对象图像；
    将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；以及
    根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像。
  2. 根据权利要求1所述的方法，其中
    所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型是通过如下方式训练得到的：
    将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
    响应于模型训练完成，得到所述目标对象图像处理模型。
  3. 根据权利要求1或2所述的方法，其中所述目标差异信息还包括调整至预设分辨率之后的高分辨率图像和调整至所述预设分辨率之后的原始训练样本图像之间的整体差异信息。
  4. 根据权利要求1或2所述的方法，其中所述目标对象图像处理模型还包括判别器；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：
    在所述目标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
  5. 根据权利要求1或2所述的方法，其中所述目标对象图像处理模型还包括判别器；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：
    将所述目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；
    在所述融合差异信息所表征的差异度小于预设的融合差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
  6. 根据权利要求1-5中任一项所述的方法，其中，所述根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像，包括：
    对所述待处理图像进行分辨率增强处理，得到目标待处理图像；
    根据所述第二目标对象图像和所述目标待处理图像进行图像融合，得到所述目标图像。
  7. 一种目标对象图像处理模型的训练方法，其中，所述方法包括：
    将原始训练样本图像的低分辨率图像作为生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像，其中所述目标对象图像处理模型为包括所述生成器的生成式对抗网络模型，；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
    响应于模型训练完成，得到所述目标对象图像处理模型。
  8. 根据权利要求7所述的方法，其中所述目标差异信息还包括调整至预设分辨率之后的高分辨率图像和调整至所述预设分辨率之后的原始训练样本图像之间的整体差异信息。
  9. 根据权利要求7所述的方法，其中所述目标对象图像处理模型还包括判别器；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：
    在所述目标差异信息包括的每种差异信息所表征的差异度均小于各自对应的差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型 训练完成。
  10. 根据权利要求7所述的方法，其中所述目标对象图像处理模型还包括判别器；
    根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，包括：
    将所述目标差异信息中包括的每种差异信息进行融合，得到融合差异信息；
    在所述融合差异信息所表征的差异度小于预设的融合差异度阈值、且所述判别器对所述高分辨率图像的真假判定结果为真实的情况下，确定所述模型训练完成。
  11. 一种图像处理装置，所述装置包括：
    提取模块，被配置用于从待处理图像中提取第一目标对象图像；
    输入模块，被配置用于将所述第一目标对象图像输入到目标对象图像处理模型中，得到所述目标对象图像处理模型输出的第二目标对象图像，其中，所述第二目标对象图像的分辨率高于所述第一目标对象图像；
    图像融合模块，被配置用于根据所述第二目标对象图像和所述待处理图像进行图像融合，得到目标图像。
  12. 根据权利要求11所述的图像处理装置，其中
    所述目标对象图像处理模型为包括生成器的生成式对抗网络模型，所述目标对象图像处理模型被配置为通过目标对象图像处理模型的训练装置训练得到，该目标对象图像处理模型的训练装置包括：
    图像获得模块，被配置用于将原始训练样本图像的低分辨率图像作为所述生成器的输入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像；
    确定模块，被配置用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
    模型获得模块，被配置用于响应于模型训练完成，得到所述目标对象图像处理模型。
  13. 一种目标对象图像处理模型的训练装置，所述装置包括：
    图像获得模块，被配置用于将原始训练样本图像的低分辨率图像作为所述生成器的输 入，得到所述生成器对所述低分辨率图像处理之后输出的高分辨率图像，其中所述目标对象图像处理模型为包括生成器的生成式对抗网络模型；
    确定模块，被配置用于根据所述高分辨率图像与所述原始训练样本图像之间的目标差异信息，确定模型是否训练完成，其中，所述目标差异信息包括以下中的至少一者：所述高分辨率图像与所述原始训练样本图像之间的特征点差异信息、所述高分辨率图像与所述原始训练样本图像中的指定特征区域差异信息；
    模型获得模块，被配置用于响应于模型训练完成，得到所述目标对象图像处理模型。
  14. 一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现权利要求1-6中任一项所述方法的步骤。
  15. 一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现权利要求7-10中任一项所述方法的步骤。
  16. 一种电子设备，包括：
    存储装置，其上存储有计算机程序；
    处理装置，用于执行所述存储装置中的所述计算机程序，以实现权利要求1-6中任一项所述方法的步骤。
  17. 一种电子设备，包括：
    存储装置，其上存储有计算机程序；
    处理装置，用于执行所述存储装置中的所述计算机程序，以实现权利要求7-10中任一项所述方法的步骤。
PCT/CN2021/131155 2020-11-18 2021-11-17 图像处理方法、模型训练方法、装置、介质及设备 WO2022105779A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/252,979 US20240013359A1 (en) 2020-11-18 2021-11-17 Image processing method, model training method, apparatus, medium and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011298847.4A CN112381717A (zh) 2020-11-18 2020-11-18 图像处理方法、模型训练方法、装置、介质及设备
CN202011298847.4 2020-11-18

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/252,979 Continuation US20240013359A1 (en) 2020-11-18 2021-11-17 Image processing method, model training method, apparatus, medium and device

Publications (1)

Publication Number Publication Date
WO2022105779A1 true WO2022105779A1 (zh) 2022-05-27

Family

ID=74584392

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131155 WO2022105779A1 (zh) 2020-11-18 2021-11-17 图像处理方法、模型训练方法、装置、介质及设备

Country Status (3)

Country Link
US (1) US20240013359A1 (zh)
CN (1) CN112381717A (zh)
WO (1) WO2022105779A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116362972A (zh) * 2023-05-22 2023-06-30 飞狐信息技术（天津）有限公司 图像处理方法、装置、电子设备及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112381717A (zh) * 2020-11-18 2021-02-19 北京字节跳动网络技术有限公司 图像处理方法、模型训练方法、装置、介质及设备
CN113344776B (zh) * 2021-06-30 2023-06-27 北京字跳网络技术有限公司 图像处理方法、模型训练方法、装置、电子设备及介质
CN113688832B (zh) * 2021-08-27 2023-02-03 北京三快在线科技有限公司 一种模型训练及图像处理方法、装置
CN117196957B (zh) * 2023-11-03 2024-03-22 广东省电信规划设计院有限公司 基于人工智能的图像分辨率转换方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921782A (zh) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质
CN110310229A (zh) * 2019-06-28 2019-10-08 Oppo广东移动通信有限公司 图像处理方法、图像处理装置、终端设备及可读存储介质
CN110428366A (zh) * 2019-07-26 2019-11-08 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN111353929A (zh) * 2018-12-21 2020-06-30 北京字节跳动网络技术有限公司 图像处理方法、装置和电子设备
CN111626932A (zh) * 2020-05-07 2020-09-04 Tcl华星光电技术有限公司 图像的超分辨率重建方法及装置
CN112381717A (zh) * 2020-11-18 2021-02-19 北京字节跳动网络技术有限公司 图像处理方法、模型训练方法、装置、介质及设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921782A (zh) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 一种图像处理方法、装置及存储介质
CN111353929A (zh) * 2018-12-21 2020-06-30 北京字节跳动网络技术有限公司 图像处理方法、装置和电子设备
CN110310229A (zh) * 2019-06-28 2019-10-08 Oppo广东移动通信有限公司 图像处理方法、图像处理装置、终端设备及可读存储介质
CN110428366A (zh) * 2019-07-26 2019-11-08 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN111626932A (zh) * 2020-05-07 2020-09-04 Tcl华星光电技术有限公司 图像的超分辨率重建方法及装置
CN112381717A (zh) * 2020-11-18 2021-02-19 北京字节跳动网络技术有限公司 图像处理方法、模型训练方法、装置、介质及设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116362972A (zh) * 2023-05-22 2023-06-30 飞狐信息技术（天津）有限公司 图像处理方法、装置、电子设备及存储介质
CN116362972B (zh) * 2023-05-22 2023-08-08 飞狐信息技术（天津）有限公司 图像处理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN112381717A (zh) 2021-02-19
US20240013359A1 (en) 2024-01-11

Similar Documents

Publication Publication Date Title
WO2022105779A1 (zh) 图像处理方法、模型训练方法、装置、介质及设备
US20230394671A1 (en) Image segmentation method and apparatus, and device, and storage medium
WO2022252881A1 (zh) 图像处理方法、装置、可读介质和电子设备
CN111325704B (zh) 图像修复方法、装置、电子设备及计算机可读存储介质
CN111784712B (zh) 图像处理方法、装置、设备和计算机可读介质
WO2022171036A1 (zh) 视频目标追踪方法、视频目标追踪装置、存储介质及电子设备
CN110991373A (zh) 图像处理方法、装置、电子设备及介质
CN112800276B (zh) 视频封面确定方法、装置、介质及设备
WO2021197161A1 (zh) 图标更新方法、装置和电子设备
WO2023143222A1 (zh) 图像处理方法、装置、设备及存储介质
WO2022105622A1 (zh) 图像分割方法、装置、可读介质及电子设备
CN112418249A (zh) 掩膜图像生成方法、装置、电子设备和计算机可读介质
WO2022116990A1 (zh) 视频裁剪方法、装置、存储介质及电子设备
CN113038176B (zh) 视频抽帧方法、装置和电子设备
CN112257598B (zh) 图像中四边形的识别方法、装置、可读介质和电子设备
CN111783632B (zh) 针对视频流的人脸检测方法、装置、电子设备及存储介质
CN111311609B (zh) 一种图像分割方法、装置、电子设备及存储介质
CN116664849B (zh) 数据处理方法、装置、电子设备和计算机可读介质
WO2023098576A1 (zh) 图像处理方法、装置、设备及介质
WO2023138441A1 (zh) 视频生成方法、装置、设备及存储介质
WO2023016290A1 (zh) 视频分类方法、装置、可读介质和电子设备
CN116596748A (zh) 图像风格化处理方法、装置、设备、存储介质和程序产品
WO2022052889A1 (zh) 图像识别方法、装置、电子设备和计算机可读介质
CN113033552B (zh) 文本识别方法、装置和电子设备
CN111737575B (zh) 内容分发方法、装置、可读介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21893926

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18252979

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.09.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21893926

Country of ref document: EP

Kind code of ref document: A1