WO2022237633A1 - 一种图像处理方法、装置、设备及介质 - Google Patents

一种图像处理方法、装置、设备及介质 Download PDF

Info

Publication number
WO2022237633A1
WO2022237633A1 PCT/CN2022/091122 CN2022091122W WO2022237633A1 WO 2022237633 A1 WO2022237633 A1 WO 2022237633A1 CN 2022091122 W CN2022091122 W CN 2022091122W WO 2022237633 A1 WO2022237633 A1 WO 2022237633A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
model
sample
beautification
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/091122
Other languages
English (en)
French (fr)
Inventor
周思宇
何茜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to US18/289,855 priority Critical patent/US20240242406A1/en
Publication of WO2022237633A1 publication Critical patent/WO2022237633A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to an image processing method, device, equipment, and medium.
  • image beautification mainly relies on strategy-driven, which refers to using a large amount of data to train the face processing model, and then using the trained model to identify the key points of the face in the image, and then beautify the key points of the face to output the final beautified image.
  • strategy-driven refers to using a large amount of data to train the face processing model, and then using the trained model to identify the key points of the face in the image, and then beautify the key points of the face to output the final beautified image.
  • using the key points of the face as the beautification object will cause the facial features to be out of balance, reduce the naturalness of the face, deteriorate the integrity of the image, and the beautification effect is blunt; moreover, this method does not support targeting the nose. , mouth and other parts of the adjustment.
  • the present disclosure provides an image processing method, device, equipment and medium.
  • An embodiment of the present disclosure provides an image processing method, including: acquiring a user image; using an image beautification model to perform a beautification operation on the user image to obtain a beautified user image; wherein the image beautification model is based on the first sample
  • the training sample pair determined by the image and the second sample image is obtained through training; the second sample image is an image obtained after performing overall beautification and adjustment of image elements on the first sample image through an image processing model based on the target image;
  • the image elements are image style features and/or target parts of the target image.
  • the overall beautification and image element adjustment of the first sample image through an image processing model based on the target image includes: acquiring the first sample image and the target image; using the image processing model to generate the A second sample image corresponding to the first sample image; wherein, the image processing model includes a first sub-model and a second sub-model, and the first sub-model is used for overall image beautification, and the second sub-model The model is used to adjust image elements based on the target image.
  • the training process of the image beautification model includes: determining the first sample image and the second sample image as the training sample pair; performing training on the image beautification model based on the training sample pair Training, so that the trained image beautification model performs the beautification operation on the user image; wherein the beautification operation includes: adjustment of image style features and/or target parts of the target image.
  • using an image processing model to generate a second sample image corresponding to the first sample image includes: using the first sub-model to beautify the first sample image as a whole to obtain an image to be edited ; Using the second sub-model to adjust image elements of the image to be edited based on the target image to obtain a second sample image.
  • using the second sub-model to adjust image elements of the image to be edited based on the target image to obtain a second sample image includes: combining the target image and the image to be edited input to the second sub-model; through the second sub-model based on the target image, the image style of the image to be edited is converted to obtain a second sample image; wherein, the second sample image also includes The style feature of the image to be edited and the image style feature of the target image.
  • the method further includes: obtaining a first segmented image based on the target image; obtaining a second segmented image based on the image to be edited; combining the first segmented image, the second segmented image, the target image, and the The image to be edited is input to the second sub-model to obtain a second sample image.
  • using the second sub-model to adjust image elements of the image to be edited based on the target image to obtain a second sample image includes: performing face segmentation on the target image to obtain An initial segmented image; according to the segmented parts on the initial segmented image, different marks are added to the initial segmented image to obtain a plurality of first segmented images with different marks; wherein, the marks are used to indicate that the first Segmenting the part of the beautification target on the image; performing face segmentation on the image to be edited to obtain a second segmented image; using the second sub-model to combine the target part of the second segmented image with the first segmented image The first part indicated by the mark in the image is processed to obtain a second sample image; wherein, the target part and the first part are the same part of a human face.
  • the method further includes: according to the identification of the first segmented image, determining a first part as a beautification target on the first segmented image; a second part with the same part, and determine the second part as the target part of the image to be edited.
  • the acquiring the first sample image and target image includes: acquiring a first image collection and a second image collection; wherein, the first image collection includes unbeautified images, and the second The image collection includes beautified images, and the number of images in the second image collection is less than the number of images in the first image collection; any unbeautified image is obtained from the first image collection, As a first sample image; acquiring any beautified image from the second image set as a target image; wherein, the target image is used as a beautification target of the first sample image.
  • the method further includes: using the first sub-model to generate generated images corresponding to unbeautified images in the first image set, and determining a third image set from each of the generated images;
  • the images in the third image set are combined arbitrarily with the beautified images in the second image set to obtain a plurality of different image combinations; wherein, in the image combination, the images belonging to the third image set The image is the third sample image, and the image belonging to the second image set is the fourth sample image; using different image combinations to repeat the following training operations until the preset conditions are met: input the image combination to the to-be-trained
  • the second sub-model generates a fifth sample image; determines a loss function between the fifth sample image and the fourth sample image; converges the second sub-model to be trained based on the loss function until the When the above loss function converges to the preset value, it is determined that the preset condition is met and the training is stopped; the second sub-model obtained when the training is stopped is used as the trained second sub-model.
  • the image beautification model is applied to a mobile terminal.
  • An embodiment of the present disclosure also provides an image processing device, including: an image acquisition module, configured to acquire a user image; an image generation module, configured to perform a beautification operation on the user image by using an image beautification model, to obtain a beautified user image; Wherein, the image beautification model is obtained by training the training sample pair determined based on the first sample image and the second sample image; An image obtained after overall beautification and adjustment of image elements; the image elements are image style features and/or target parts of the target image.
  • An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; and the processor, for reading the instruction from the memory.
  • the instructions can be executed, and the instructions are executed to implement the image processing method provided by the embodiment of the present disclosure.
  • the embodiment of the present disclosure also provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the image processing method provided by the embodiment of the present disclosure.
  • Embodiments of the present disclosure provide an image processing method, device, device, and medium.
  • the technical solution uses an image beautification model to perform beautification operations on user images to obtain beautified user images; wherein, the image beautification model is based on the first This image and the training sample pair determined by the second sample image are obtained through training; the second sample image is an image obtained after performing overall beautification and adjustment of image elements on the first sample image through an image processing model based on the target image, And the image element is an image style feature and/or a target part of the target image.
  • the overall beautification can better maintain the proportion of facial features before and after image beautification, and improve the naturalness and integrity of the user's image after beautification; the adjustment of image elements can improve the image style diversity and local parts of the user's image after beautification.
  • the beautification effect enriches the image editing function and provides users with a personalized beautification solution; based on this, this solution can meet the user's multiple beautification needs for the overall, partial and image style of the image through the image beautification model, which significantly reduces user creation. Cost, improve the image beautification effect.
  • this embodiment only needs to train the image beautification model.
  • the data volume of training sample pairs required by an image beautification model is far The amount of data is less than the training data required for multiple beautification models.
  • FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of an image processing process provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a first segmented image provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the image integrity is poor, and the beautification effect is blunt and unnatural.
  • this method does not support the adjustment of local parts such as the nose and mouth. In order to meet the user's local adjustment needs, it is necessary to design additional beautification models specifically for each part, which will increase the user's creation cost in practical applications.
  • the amount of training sample data used for face processing model training is large, which affects the iterative update speed of the model, and the quality of the training samples is difficult to guarantee, which affects the use effect of the trained model.
  • embodiments of the present disclosure provide an image processing method, device, device, and medium, which can be used for generating training sample pairs in the model training phase, and for training the model in the application phase Image beautification.
  • image processing method device, device, and medium, which can be used for generating training sample pairs in the model training phase, and for training the model in the application phase Image beautification.
  • FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure.
  • the method can be executed by an image processing device, wherein the device can be implemented by software and/or hardware, and generally can be integrated in electronic equipment, for example, it can be applied to various off-screen camera products, such as mobile phones, computers, and cameras.
  • the method includes:
  • Step S102 acquiring user images.
  • the user image may be an image to be beautified including a human face obtained by the user through an image selection operation, an image capture operation, or an image upload operation in the terminal.
  • Step S104 using the image beautification model to perform a beautification operation on the user image to obtain a beautified user image.
  • the image beautification model is trained based on the training sample pair determined by the first sample image and the second sample image; the second sample image is based on the target image through the image processing model to perform overall beautification and image elements on the first sample image The image obtained after adjustment; the image elements are image style features and/or target parts of the target image.
  • a large number of training sample pairs with rich and diverse beautification effects can be used to train a variety of image beautification models with different beautification effects; for specific training methods of image beautification models, reference can be made to existing training methods.
  • the second sample image in the training sample pair can ensure high integrity and naturalness, and at the same time has the beautification effect of adjusting image elements (ie, image style features and/or target parts); based on this, use the training sample pair to train
  • the obtained image beautification model can perform a beautification operation on the user image, and the beautification operation includes beautification of the entire image and adjustment of image elements, and the beautified user image thus obtained can meet the beautification needs of the user.
  • the image processing method provided in this embodiment uses an image beautification model to perform a beautification operation on a user image to obtain a beautified user image; wherein, the image beautification model is obtained by training a pair of training samples determined based on the first sample image and the second sample image
  • the second sample image is an image obtained by overall beautifying the first sample image and adjusting image elements based on the target image through an image processing model, and the image elements are image style features and/or target parts of the target image.
  • the image beautification model can perform overall beautification and adjustment of image elements on user images; among them, the overall beautification can better maintain the proportion of facial features before and after image beautification, and improve the naturalness and integrity of user images after beautification;
  • the adjustment of elements can improve the diversity of image styles and the beautification effect of local parts of the user's image after beautification, enrich the image editing function, and provide users with a personalized beautification solution; based on this, this solution can satisfy users through the image beautification model
  • the multiple beautification requirements for the overall image, partial image and image style significantly reduce the user's creation cost and improve the image beautification effect.
  • this embodiment only needs to train the image beautification model.
  • the data volume of training sample pairs required by an image beautification model is far The amount of data is less than the training data required for multiple beautification models.
  • This embodiment provides a method for obtaining the second sample image, refer to the following two steps:
  • Step 1 acquire the first sample image and the target image.
  • the first image collection and the second image collection are acquired first; wherein, the first image collection includes unbeautified images, the second image collection includes beautified images, and the second image collection includes The number of images is less than the number of images in the first image collection; then any unbeautified image is obtained from the first image collection as the first sample image; and any one of the beautified images is obtained from the second image collection
  • the image of is used as the target image; wherein, the target image is used as the beautification target of the first sample image.
  • the image pair of the first sample image and the target image obtained above is the result of any combination of the images in the first image set and the images in the second image set, and the combined image pair is There are a large number of rich and diverse images, and this embodiment only uses any image pair among them as a description object.
  • the first sample image in this embodiment is an image that has not undergone any beautification processing such as skin smoothing, style conversion, or beautification; for example, the first sample image may be collected from a real person by an image collection device such as a camera
  • the image can also be an image rendered by a virtual character, or it can also be an image generated by an adversarial network.
  • the target image is an image with beautification effects such as filter, makeup, and style after beautification processing, and the target image is used as the beautification target of the first sample image; for example, the target image can be retrieved from the cloud or locally
  • the image may also be an image obtained by performing beautification processing on part of the first sample image.
  • Step 2 using the image processing model to generate a second sample image corresponding to the first sample image; wherein, the image processing model includes a first sub-model and a second sub-model, and the first sub-model is used for overall image beautification, and the second The sub-model is used to adjust the image elements based on the target image; the first sub-model and the second sub-model can both be GAN (Generative Adversarial Networks, Generative Adversarial Networks).
  • GAN Generic Adversarial Networks, Generative Adversarial Networks
  • the overall beautification effect of the image is realized by the first sub-model, and the integrity and naturalness of the beautified image are improved; at the same time, the second sub-model is combined with the image style special effect and/or the target part of the human face to realize the image enhancement. Adjustment, the obtained second sample image can also have a personalized image style and a local beautification effect, showing a high image beautification effect.
  • this embodiment provides an image processing process as shown in FIG. 2 .
  • the first sub-model is denoted as G1
  • the second sub-model is denoted as G2.
  • a good model can be directly applied to the overall beautification of the image.
  • the second sub-model G2 is used to adjust the image elements of the image to be edited based on the target image to obtain a second sample image.
  • feature extraction is performed on the target image through the second sub-model G2 to obtain style semantic features with the style of the target image; image feature extraction is performed on the image to be edited to obtain image semantic features; style adjustment is generated according to the style semantic features and image semantic features After the second sample image.
  • the style-adjusted second sample image and the first sample image can be used as a training sample pair for training the preset image beautification model, so that the trained image beautification model performs style adjustment on the user image.
  • this embodiment uses Before the second sub-model G2 obtains the second sample image, the face segmentation process can be performed first, and the implementation method refers to the following steps 1 to 3:
  • Step 1 Obtain a first segmented image based on the target image.
  • key points of the face on the target image can be extracted through a preset face segmentation model, and the target image can be segmented according to the key points of the face to obtain an initial segmented image.
  • different marks are added to the initial segmented image to obtain multiple first segmented images with different marks.
  • the identification is used to indicate the part as the beautification target on the first segmented image; the identification is, for example, the position parameter or the bounding box of the corresponding part. Taking the bounding box as an example, the first segmented image with the eye part identification added is shown in Figure 3 Show.
  • a target image corresponds to a plurality of first segmented images with different identifiers, for example, including: the first segmented image identifying the eyebrow part, the first segmented image identifying the eye part, the first segmented image identifying the nose part, and the first segmented image identifying the nose part.
  • the first segmented image of the zygomatic region will not be listed one by one.
  • For each first segmented image corresponding to a target image it can be arbitrarily combined with each different image to be edited; based on the first segmented image and the image to be edited, the diversity of image combinations is further improved, and accordingly, the follow-up can A large number of rich pairs of second sample images and training samples are further obtained.
  • Step 2 obtaining a second segmented image based on the image to be edited.
  • the key points of the face on the image to be edited can be extracted through the preset face segmentation model, and the image to be edited is segmented according to the key points of the face to obtain the second segmented image.
  • Step 3 Input the first segmented image, the second segmented image, the target image and the image to be edited into the second sub-model to obtain a second sample image.
  • the first segmented image and the second segmented image are obtained, as shown in FIG. 2 , the first segmented image, the second segmented image and the image to be edited are input to the second sub-model G2.
  • the target part of the second segmented image and the first part indicated by the logo in the first segmented image are processed to obtain the second sample image after adjusting the target part; wherein, the target part and the first part is the same part of the human face, the second sample image includes both the content features of the target part and the content features of the first part.
  • the target part that needs local beautification on the image to be edited its specific determination method is: (1) according to the identification of the first segmented image, determine the first part as the beautification target on the first segmented image; The eye part shown in 3 is an example; (2) determine the same second part as the first part on the second segmented image, and determine that the second part is the target part of the image to be edited; that is, the target part is the target part to be edited; The eye part on the image. Thereafter, fusion processing is performed on the eye part of the image to be edited and the eye part in the first segmented image to obtain a second sample image after adjusting the target part.
  • the second sample image after adjusting the target part and the first sample image can be used as a training sample pair for training a preset image beautification model, so that the trained image beautification model can locally beautify the user image.
  • the second sub-model when the image element is the image style feature and the target part, can be used to adjust the image style feature of the target part in the image to be edited based on the target image, and the style of the adjusted target part can be obtained accordingly. After the second sample image.
  • the first segmented image corresponding to the target image and the second segmented image corresponding to the image to be edited can be input into the second sub-model; using the second sub-model, based on the first part of the first segmented image, the The image style of the target part of the two-segmented image is converted to obtain the second sample image after adjusting the style of the target part; the partial part of the second sample image includes the image style feature of the target part in the image to be edited and the second sample image in the target image.
  • An image style feature of a part, the above-mentioned partial part refers to a part on the second sample image that is the same as the first part and the target part.
  • the second sample image and the first sample image in this embodiment are used as a training sample pair for training the preset image beautification model, so that the trained image beautification model performs overall beautification and local part beautification on the user image.
  • Style conversion enrich the image editing function of the image beautification model in the terminal.
  • the second sub-model In order to make the second sub-model in the above embodiment directly applicable to the adjustment of image elements, the second sub-model needs to be trained in advance, and the parameters of the second sub-model need to be obtained through training.
  • the purpose of training the second sub-model is the parameter that finally determines that the requirements can be met.
  • the second sub-model can obtain the expected beautification effect.
  • This embodiment provides a training method for the second sub-model, including the following four steps:
  • the first sub-model is used to generate generated images corresponding to the unbeautified images in the first image set, and the third image set is determined from each generated image.
  • the second step is to arbitrarily combine the images in the third image collection with the beautified images in the second image collection to obtain multiple different image combinations; wherein, in the image combination, the image belonging to the third image collection is the first There are three sample images, and the image belonging to the second image set is the fourth sample image.
  • this embodiment considering that the images in the first image set are easy to obtain and the number is large, although the number of target images in the second image set is small, the number of corresponding first segmented images is large, and the target image and All the first segmented images can be combined with the images in the first image set in a variety of ways, which can meet the adjustment requirements of image style features and target parts. Based on this, in order to reduce the cost of data acquisition, this embodiment can directly use the first image set to generate the third image set, and obtain the third sample image from it, and directly obtain the fourth sample image from the target image included in the second image set .
  • the image combination is input to the second sub-model to be trained to generate the fifth sample image; in specific implementation, the third sample image and the fourth sample image are input to the second sub-model to be trained, and the second sub-model to be trained is passed The second sub-model utilizes the fourth sample image to generate a fifth sample image after adjusting image elements corresponding to the third sample image.
  • the fourth step is to use the second sub-model obtained when the training is stopped as the trained second sub-model.
  • the second sub-model trained in the above way can better process the input user image in terms of the overall and local beautification effect and the image style conversion effect, so as to generate an image that meets the user's beautification needs, effectively improving the Image beautification effects and user experience.
  • the training method determined based on the first sample image and the second sample image can be Sample pair training image beautification model, present embodiment provides the following a kind of training method of image beautification model, comprising:
  • the first sample image and the second sample image are determined as a pair of training samples.
  • the second sample image in this embodiment is obtained based on the first sample image and the target image; thus, in practical applications, a large number of different first sample images and different target images can be used to obtain a large number of different first sample images.
  • Two sample images, the second sample image has rich and diverse beautification effects on image style features and partial parts of the face.
  • a large number of training sample pairs with rich and diverse beautification effects can be constructed from different groups of first sample images and second sample images.
  • the image beautification model is trained based on the training samples, so that the trained image beautification model performs a beautification operation on the user image; wherein, the beautification operation includes: adjustment of image style features and/or target parts of the target image.
  • the training sample pairs used in the training image beautification model can be generated based on only a small number of target images and any first sample image, the training sample pairs are easy to update and iterate, and the corresponding trained images
  • the beautification model has a fast iterative update speed, which can quickly respond to the user's new image beautification needs.
  • the image beautification model can be trained in the server, and then the trained image beautification model is applied to the mobile terminal, that is, the image beautification model is sent to the mobile phone, computer and other terminals for the terminal to call and treat the beautified The user image is beautified.
  • Fig. 4 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure, and the device is used to implement an image processing method.
  • the device can be realized by software and/or hardware, and generally can be integrated in electronic equipment, for example, can be applied to various off-screen camera products, such as mobile phones, computers, and video cameras.
  • the device includes:
  • An image generation module 404 configured to perform a beautification operation on the user image by using the image beautification model to obtain the beautified user image
  • the image beautification model is trained based on the training sample pair determined by the first sample image and the second sample image; the second sample image is based on the target image through the image processing model to perform overall beautification and image elements on the first sample image The image obtained after adjustment; the image elements are image style features and/or target parts of the target image.
  • the above-mentioned image processing device also includes an image processing module, which is used for:
  • the image processing model includes a first sub-model and a second sub-model, and the first sub-model is used for The overall image beautification is performed, and the second sub-model is used to adjust image elements based on the target image.
  • the above-mentioned image processing device also includes a training module of an image beautification model, which is used for:
  • the image beautification model is trained based on the training samples, so that the trained image beautification model performs a beautification operation on the user image; wherein, the beautification operation includes: adjustment of image style features and/or target parts of the target image.
  • the above-mentioned image processing module is specifically used for:
  • the image elements of the image to be edited are adjusted based on the target image to obtain a second sample image.
  • the above-mentioned image processing module is specifically used for:
  • the image style of the image to be edited is converted through the second sub-model based on the target image to obtain a second sample image; wherein, the second sample image includes both the style feature of the image to be edited and the image style feature of the target image.
  • the above-mentioned image processing module is specifically used for:
  • the above-mentioned image processing module is specifically used for:
  • the second sub-model uses the second sub-model to process the target part of the second segmented image and the first part represented by the logo in the first segmented image to obtain a second sample image; wherein, the target part and the first part are the same part of the human face.
  • the above-mentioned image processing module is specifically used for:
  • the identification of the first segmented image determine the first part as the beautification target on the first segmented image
  • a second part identical to the first part is determined on the second segmented image, and the second part is determined as a target part of the image to be edited.
  • the above-mentioned image processing module is specifically used for:
  • the first image collection includes unbeautified images
  • the second image collection includes beautified images
  • the number of images in the second image collection is less than the first image collection The number of images in ;
  • An arbitrary beautified image is acquired from the second image set as a target image; wherein, the target image is used as a beautification target of the first sample image.
  • the above-mentioned image processing device also includes a training module of the second sub-model, which is used for:
  • the image combination is input to the second sub-model to be trained to generate the fifth sample image
  • the second sub-model obtained when the training is stopped is used as the trained second sub-model.
  • the image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • an electronic device 500 includes one or more processors 501 and a memory 502 for storing instructions executable by the processors 501 .
  • the processor 501 is configured to read the executable instruction from the memory 502, and execute the instruction to implement the image processing method in the above embodiment.
  • the processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.
  • Memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache).
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • One or more computer program instructions can be stored on the computer-readable storage medium, and the processor 501 can execute the program instructions to realize the image processing method of the above-mentioned embodiments of the present disclosure and/or other desired function.
  • Various contents such as input signal, signal component, noise component, etc. may also be stored in the computer-readable storage medium.
  • the electronic device 500 may further include: an input device 503 and an output device 504, and these components are interconnected through a bus system and/or other forms of connection mechanisms (not shown).
  • the input device 503 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 504 can output various information to the outside, including determined distance information, direction information, and the like.
  • the output device 504 may include, for example, a display, a speaker, a printer, a communication network and remote output devices connected thereto, and the like.
  • the electronic device 500 may further include any other appropriate components.
  • the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the image processing described in the embodiments of the present disclosure. method.
  • the computer program product can be written in any combination of one or more programming languages to execute the program codes for performing the operations of the embodiments of the present disclosure, and the programming languages include object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.
  • embodiments of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the processor executes the image processing provided by the embodiments of the present disclosure. method.
  • the computer readable storage medium may employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • the embodiments of the present disclosure also provide a computer program product, including computer programs/instructions, and when the computer programs/instructions are executed by a processor, the methods in the embodiments of the present disclosure are implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例涉及一种图像处理方法、装置、设备及介质，其中该方法包括：获取用户图像；采用图像美化模型对用户图像执行美化操作，得到美化后用户图像；其中，图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；第二样本图像是基于目标图像通过图像处理模型对第一样本图像进行整体美化和图像元素的调整后得到的图像；图像元素为目标图像的图像风格特征和/或目标部位。本公开能够降低用户创作成本，提高图像美化效果。

Description

一种图像处理方法、装置、设备及介质
相关申请的交叉引用
本申请要求于2021年05月08日提交的，申请号为202110500411.7、发明名称为“一种图像处理方法、装置、设备及介质”的中国专利申请的优先权，该申请的全文通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其涉及一种图像处理方法、装置、设备及介质。
背景技术
随着科技的发展，针对图像中人脸的美化功能，已经成为影像应用中必不可少的一项。目前，图像美化主要依赖策略驱动，策略驱动是指先利用大量数据训练人脸处理模型，而后用训练好的模型识别图像中的人脸关键点，再对人脸关键点进行美化处理，以输出最终的美化图像。然而，在策略驱动的图像美化方式中，以人脸关键点作为美化对象，会导致五官比例失调而降低人脸自然度，图像整体性变差，美化效果生硬；而且，该方式不支持针对鼻子、嘴巴等部位的调整。为了满足用户对局部调整的个性化需求，还需要针对每个部位额外提 供一一对应的美化模型，额外提供的多个美化模型不但需要大量的训练数据，还会在实际应用中增加用户的创作成本。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开提供了一种图像处理方法、装置、设备及介质。
本公开实施例提供了一种图像处理方法，包括：获取用户图像；采用图像美化模型对所述用户图像执行美化操作，得到美化后用户图像；其中，所述图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；所述第二样本图像是基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整后得到的图像；所述图像元素为目标图像的图像风格特征和/或目标部位。
可选的，所述基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整，包括：获取所述第一样本图像和目标图像；利用图像处理模型生成所述第一样本图像对应的第二样本图像；其中，所述图像处理模型包括第一子模型和第二子模型，且所述第一子模型用于进行图像整体美化，所述第二子模型用于基于所述目标图像进行图像元素的调整。
可选的，所述图像美化模型的训练过程包括：将所述第一样本图像和所述第二样本图像确定为所述训练样本对；基于所述训练样本对对所述图像美化模型进行训练，以使训练好的图像美化模型对所述用 户图像执行所述美化操作；其中，所述美化操作包括：目标图像的图像风格特征和/或目标部位的调整。
可选的，所述利用图像处理模型生成所述第一样本图像对应的第二样本图像，包括：利用所述第一子模型对所述第一样本图像进行整体美化，得到待编辑图像；利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整，得到第二样本图像。
可选的，所述利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整，得到第二样本图像，包括：将所述目标图像和所述待编辑图像输入至所述第二子模型；通过所述第二子模型基于所述目标图像，对所述待编辑图像的图像风格进行转换，得到第二样本图像；其中，所述第二样本图像同时包括所述待编辑图像的风格特征和所述目标图像的图像风格特征。
可选的，所述方法还包括：基于所述目标图像得到第一分割图像；基于待编辑图像得到第二分割图像；将所述第一分割图像、第二分割图像、所述目标图像和所述待编辑图像输入至第二子模型得到第二样本图像。
可选的，所述利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整，得到第二样本图像，包括：对所述目标图像进行人脸分割，得到初始分割图像；按照所述初始分割图像上的分割部位，对所述初始分割图像添加不同的标识，得到具有不同标识的多张第一分割图像；其中，所述标识用于表示所述第一分割图像上作为美化目标的部位；对所述待编辑图像进行人脸分割，得到第二 分割图像；利用所述第二子模型，将所述第二分割图像的目标部位与所述第一分割图像中标识所表示的第一部位进行处理，得到第二样本图像；其中，所述目标部位与所述第一部位为人脸的相同部位。
可选的，所述方法还包括：根据所述第一分割图像的标识，确定所述第一分割图像上作为美化目标的第一部位；在所述第二分割图像上确定与所述第一部位相同的第二部位，并确定所述第二部位为所述待编辑图像的目标部位。
可选的，所述获取所述第一样本图像和目标图像，包括：获取第一图像集合和第二图像集合；其中，所述第一图像集合中包括未美化的图像，所述第二图像集合中包括已美化的图像，且所述第二图像集合中的图像数量少于所述第一图像集合中的图像数量；从所述第一图像集合中获取任意一张未美化的图像，作为第一样本图像；从所述第二图像集合中获取任意一张已美化的图像，作为目标图像；其中，所述目标图像作为所述第一样本图像的美化目标。
可选的，所述方法还包括：利用所述第一子模型生成与所述第一图像集合中未美化的图像相对应的生成图像，并由各个所述生成图像确定第三图像集合；将所述第三图像集合中的图像与所述第二图像集合中已美化的图像进行任意组合，得到不同的多个图像组合；其中，在所述图像组合中，属于所述第三图像集合的图像为第三样本图像，属于所述第二图像集合的图像为第四样本图像；利用不同的图像组合重复如下训练操作，直至满足预设条件时停止：将所述图像组合输入至待训练的第二子模型生成第五样本图像；确定所述第五样本图像与 所述第四样本图像之间的损失函数；基于所述损失函数对所述待训练的第二子模型进行收敛，直至所述损失函数收敛至预设值时确定满足预设条件并停止训练；将停止训练时得到的第二子模型作为训练好的第二子模型。
可选的，所述图像美化模型应用于移动终端。
本公开实施例还提供了一种图像处理装置，包括：图像获取模块，用于获取用户图像；图像生成模块，用于采用图像美化模型对所述用户图像执行美化操作，得到美化后用户图像；其中，所述图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；所述第二样本图像是基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整后得到的图像；所述图像元素为目标图像的图像风格特征和/或目标部位。
本公开实施例还提供了一种电子设备，所述电子设备包括：处理器；用于存储所述处理器可执行指令的存储器；所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现如本公开实施例提供的图像处理方法。
本公开实施例还提供了一种计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序用于执行如本公开实施例提供的图像处理方法。
本公开实施例提供的技术方案与现有技术相比具有如下优点：
本公开实施例提供了一种图像处理方法、装置、设备及介质，该技术方案采用图像美化模型对用户图像执行美化操作，得到美化后用 户图像；其中，所述图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；该第二样本图像是基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整后得到的图像，且图像元素为目标图像的图像风格特征和/或目标部位。在该方案中，整体美化能够较好地保持图像美化前后的五官比例，提高美化后用户图像的自然度、整体性；图像元素的调整能够提高美化后用户图像的图像风格多样性和局部部位的美化效果，丰富了图像编辑功能，为用户提供了个性化的美化方案；基于此，本方案通过图像美化模型即可满足用户对图像整体、局部和图像风格的多重美化需求，明显降低了用户创作成本，提高了图像美化效果。同时，相比于现有方案中所需的多个美化模型，本实施例只需训练图像美化模型这一个模型即可，显然，一个图像美化模型所需的训练样本对的数据量，要远少于多个美化模型所需的训练数据的数据量。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种图像处理方法的流程示意图；
图2为本公开实施例提供的一种图像处理过程示意图；
图3为本公开实施例提供的一种第一分割图像的示意图；
图4为本公开实施例提供的一种图像处理装置的结构示意图；
图5为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。
目前，依赖策略驱动的图像美化方式，以人脸关键点作为美化对象，导致图像整体性较差，美化效果生硬、不自然，而且，该方式不支持针对鼻子、嘴巴等局部部位的调整。为了满足用户的局部调整需求，还需要额外设计其它专门针对各个部位的美化模型，在实际应用中会增加用户的创作成本。此外，在该方式中，人脸处理模型训练使用的训练样本数据量较大，影响模型的迭代更新速度，训练样本的质量难以保证，影响训练后模型的使用效果。为了改善该上述问题至少之一，本公开实施例提供了一种图像处理方法、装置、设备及介质，该技术可用于模型训练阶段中训练样本对的生成，以及训练后模型在应用阶段中的图像美化。为便于理解，以下对本发明实施例进行详细介绍。
图1为本公开实施例提供的一种图像处理方法的流程示意图。该方法可以由图像处理装置执行，其中该装置可以采用软件和/或硬件实现，一般可集成在电子设备中，例如可以应用于各种屏下摄像产品中，诸如手机、电脑和摄像机等。如图1所示，该方法包括：
步骤S102，获取用户图像。该用户图像可以是用户通过在终端中的图像选择操作、图像拍摄操作或图像上传操作等方式获取的包含人脸的待美化图像。
步骤S104，采用图像美化模型对用户图像执行美化操作，得到美化后用户图像。
其中，图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；第二样本图像是基于目标图像通过图像处理模型对第一样本图像进行整体美化和图像元素的调整后得到的图像；图像元素为目标图像的图像风格特征和/或目标部位。
本实施例利用美化效果丰富多样的大量训练样本对，能够训练出多种具有不同美化效果的图像美化模型；关于图像美化模型的具体训练方式可参照现有训练方式。训练样本对中的第二样本图像能够保证较高的整体性和自然度，同时兼具调整图像元素(即图像风格特征和/或目标部位)的美化效果；基于此，利用该训练样本对训练得到的图像美化模型，能够对用户图像执行美化操作，该美化操作包括图像整体的美化和图像元素的调整，由此得到的美化后用户图像能够满足用户的美化需求。
本实施例提供的图像处理方法，采用图像美化模型对用户图像执 行美化操作，得到美化后用户图像；其中，图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；该第二样本图像是基于目标图像通过图像处理模型对第一样本图像进行整体美化和图像元素的调整后得到的图像，且图像元素为目标图像的图像风格特征和/或目标部位。在该方案中，图像美化模型能够对用户图像进行整体美化和图像元素的调整；其中，整体美化能够较好地保持图像美化前后的五官比例，提高美化后用户图像的自然度、整体性；图像元素的调整能够提高美化后用户图像的图像风格多样性和局部部位的美化效果，丰富了图像编辑功能，为用户提供了个性化的美化方案；基于此，本方案通过图像美化模型即可满足用户对图像整体、局部和图像风格的多重美化需求，明显降低了用户创作成本，提高了图像美化效果。同时，相比于现有方案中所需的多个美化模型，本实施例只需训练图像美化模型这一个模型即可，显然，一个图像美化模型所需的训练样本对的数据量，要远少于多个美化模型所需的训练数据的数据量。
本实施例给出了一种第二样本图像的得到方式，参考如下两个步骤：
步骤一，获取第一样本图像和目标图像。
在一种实现方式中，首先获取第一图像集合和第二图像集合；其中，第一图像集合中包括未美化的图像，第二图像集合中包括已美化的图像，且第二图像集合中的图像数量少于第一图像集合中的图像数量；然后从第一图像集合中获取任意一张未美化的图像，作为第一样 本图像；以及，从第二图像集合中获取任意一张已美化的图像，作为目标图像；其中，目标图像作为第一样本图像的美化目标。可以理解的是，上述获取的第一样本图像和目标图像这一图像对，是第一图像集合中的图像与第二图像集合中的图像进行任意组合的结果，由此组合的图像对是大量且丰富多样的，本实施例仅以其中的任意一个图像对作为描述对象。
本实施例中的第一样本图像是未经磨皮、风格转换或美颜等任何美化处理过的图像；示例性的，第一样本图像可以是通过摄像头等图像采集设备对真实人物采集的图像，还可以是通过虚拟人物渲染得到的图像，或者，还可以是通过对抗网络生成的图像。目标图像是经美化处理后，具有滤镜、妆容和风格等美化特效的图像，目标图像用于作为第一样本图像的美化目标；示例性的，目标图像可以是从云端或本地调取的图像，还可以是对部分第一样本图像进行美化处理后的图像。
步骤二，利用图像处理模型生成第一样本图像对应的第二样本图像；其中，图像处理模型包括第一子模型和第二子模型，且第一子模型用于进行图像整体美化，第二子模型用于基于目标图像进行图像元素的调整；第一子模型和第二子模型诸如均可以为GAN(Generative Adversarial Networks，生成式对抗网络)。
本实施例通过第一子模型实现图像的整体美化效果，提高美化后图像的整体性、自然度；同时结合第二子模型在图像风格特效和/或人脸局部的目标部位上实现对图像的调整，由此得到的第二样本图像 还能够兼具个性化的图像风格和局部美化效果，呈现出了较高的图像美化效果。
为了便于理解如何利用图像处理模型生成第二样本图像，本实施例提供一种如图2所示的图像处理过程。在本实施例中，第一子模型表示为G1，第二子模型表示为G2。将任意一张第一样本图像输入至第一子模型G1，利用第一子模型G1对第一样本图像进行整体美化，得到待编辑图像；本实施例中的第一子模型是预先训练好的模型，能够直接应用于对图像的整体美化。
而后，利用第二子模型G2，基于目标图像对待编辑图像的图像元素进行调整，得到第二样本图像。
当图像元素为图像风格特征时，在此提供一种利用第二子模型得到第二样本图像的实施例，参照如下所示：
将目标图像和待编辑图像输入至第二子模型G2；通过第二子模型G2基于目标图像，对待编辑图像的图像风格进行转换，得到风格调整后的第二样本图像，该第二样本图像同时包括待编辑图像的图像风格特征和目标图像的图像风格特征。
具体的，通过第二子模型G2对目标图像进行特征提取，得到具有目标图像风格的风格语义特征；对待编辑图像进行图像特征提取，得到图像语义特征；根据风格语义特征和图像语义特征生成风格调整后的第二样本图像。
该风格调整后的第二样本图像可以与第一样本图像作为训练样本对，用于对预设的图像美化模型进行训练，以使训练好的图像美化 模型对用户图像进行风格调整。
当图像元素为人脸的目标部位时，为了将目标图像上作为美化目标的人脸部位与待编辑图像上待美化的目标部位进行准确地匹配，以对目标部位进行调整，本实施例在利用第二子模型G2得到第二样本图像之前，可以先执行人脸分割过程，实现方式参照如下步骤1至步骤3：
步骤1，基于目标图像得到第一分割图像。具体可以通过预设的人脸分割模型提取目标图像上的人脸关键点，根据人脸关键点对目标图像进行人脸分割，得到初始分割图像。按照初始分割图像上的分割部位，对初始分割图像添加不同的标识，得到具有不同标识的多张第一分割图像。其中，标识用于表示第一分割图像上作为美化目标的部位；该标识诸如为相应部位的位置参数或包围框，以包围框为例，添加有眼睛部位标识的第一分割图像如图3所示。
可以理解，一张目标图像对应有标识不同的多张第一分割图像，比如包括：标识眉毛部位的第一分割图像、标识眼睛部位的第一分割图像、标识鼻子部位的第一分割图像、标识颧骨部位的第一分割图像，不再一一列举。对于一张目标图像对应的每张第一分割图像，均可以与各个不同的待编辑图像进行任意组合；基于第一分割图像与待编辑图像进一步提升了图像组合的多样性，相应地，后续能够进一步得到大量丰富的第二样本图像和训练样本对。
步骤2，基于待编辑图像得到第二分割图像。具体可以通过预设的人脸分割模型提取待编辑图像上的人脸关键点，根据人脸关键点对 待编辑图像进行人脸分割，得到第二分割图像。
步骤3，将第一分割图像、第二分割图像、目标图像和待编辑图像输入至第二子模型得到第二样本图像。
在得到第一分割图像和第二分割图像之后，如图2所示，将第一分割图像、第二分割图像和待编辑图输入至第二子模型G2。利用第二子模型G2，将第二分割图像的目标部位与第一分割图像中标识所表示的第一部位进行处理，得到调整目标部位后的第二样本图像；其中，目标部位与第一部位为人脸的相同部位，该第二样本图像同时包括目标部位的内容特征和第一部位的内容特征。
针对待编辑图像上需要局部美化的目标部位，其具体确定方式为：(1)根据第一分割图像的标识，确定第一分割图像上作为美化目标的第一部位；该第一部位可以以图3所示的眼睛部位为例；(2)在第二分割图像上确定与第一部位相同的第二部位，并确定第二部位为待编辑图像的目标部位；也即，目标部位是待编辑图像上的眼睛部位。此后，将待编辑图像的眼睛部位与第一分割图像中的眼睛部位进行融合处理，得到调整目标部位后的第二样本图像。
该调整目标部位后的第二样本图像可以与第一样本图像作为训练样本对，用于对预设的图像美化模型进行训练，以使训练好的图像美化模型对用户图像进行局部美化。
在一种实施例中，当图像元素为图像风格特征和目标部位时，可以利用第二子模型基于目标图像，对待编辑图像中目标部位的图像风格特征进行调整，相应地得到调整目标部位的风格后的第二样本图像。 在具体实现时，可以将目标图像对应的第一分割图像、待编辑图像对应的第二分割图像输入至第二子模型；利用第二子模型，基于第一分割图像的第一部位，对第二分割图像的目标部位的图像风格进行转换，得到调整目标部位的风格后的第二样本图像；该第二样本图像的局部部位同时包括待编辑图像中目标部位的图像风格特征和目标图像中第一部位的图像风格特征，上述局部部位是指第二样本图像上与第一部位、目标部位相同的部位。本实施例中的第二样本图像与第一样本图像作为训练样本对，用于对预设的图像美化模型进行训练，以使训练好的图像美化模型对用户图像进行整体美化和局部部位的风格转换，丰富终端中的图像美化模型的图像编辑功能。
为了使上述实施例中的第二子模型可以直接应用于图像元素的调整，需要事先训练该第二子模型，第二子模型的参数需要经过训练得到，对第二子模型进行训练的目的，是最终确定可满足要求的参数。利用已训练得到的参数，第二子模型能够得到预期的美化效果。本实施例给出了一种第二子模型的训练方法，包括如下四个步骤：
第一步，利用第一子模型生成与第一图像集合中未美化的图像相对应的生成图像，并由各个生成图像确定第三图像集合。
第二步，将第三图像集合中的图像与第二图像集合中已美化的图像进行任意组合，得到不同的多个图像组合；其中，在图像组合中，属于第三图像集合的图像为第三样本图像，属于第二图像集合的图像为第四样本图像。
在本实施例中，考虑到第一图像集合中的图像易于获取且数量较 大，第二图像集合中的目标图像虽然数量少，但其对应的第一分割图像数量较多，且目标图像及其第一分割图像均可以与第一图像集合中的图像进行丰富多样的组合，能够满足图像风格特征和目标部位的调整需求。基于此，为了降低数据获取成本，本实施例可以直接利用第一图像集合生成第三图像集合，并从中获取第三样本图像，以及直接从第二图像集合包括的目标图像中获取第四样本图像。
第三步，利用不同的图像组合重复如下训练操作，直至满足预设条件时停止：
(i)将图像组合输入至待训练的第二子模型生成第五样本图像；在具体实现时，将第三样本图像和第四样本图像输入至待训练的第二子模型，通过待训练的第二子模型利用第四样本图像，生成第三样本图像对应的图像元素调整后的第五样本图像。
(ii)确定第五样本图像与第四样本图像之间的损失函数。
(iii)基于损失函数对待训练的第二子模型进行收敛，直至损失函数收敛至预设值时确定满足预设条件并停止训练。
第四步，将停止训练时得到的第二子模型作为训练好的第二子模型。
通过上述方式训练好的第二子模型，能够在整体与局部的美化效果方面、图像风格的转换效果方面，对输入的用户图像进行较好处理，以生成满足用户美化需求的图像，有效提升了图像美化效果和用户体验。
在根据以上实施例得到第一样本图像，以及利用包括第一子模型 和第二子模型的图像处理模型生成第二样本图像之后，可基于第一样本图像和第二样本图像确定的训练样本对训练图像美化模型，本实施例提供如下一种图像美化模型的训练方法，包括：
首先将第一样本图像和第二样本图像确定为训练样本对。本实施例中的第二样本图像，是基于第一样本图像和目标图像得到的；从而，在实际应用中，利用不同的第一样本图像和不同的目标图像便能够得到大量不同的第二样本图像，第二样本图像在图像风格特征和人脸局部的部位上具有丰富多样的美化效果。相应地，由不同的多组第一样本图像和第二样本图像，能够构造美化效果丰富多样的大量训练样本对。
然后基于训练样本对对图像美化模型进行训练，以使训练好的图像美化模型对用户图像执行美化操作；其中，美化操作包括：目标图像的图像风格特征和/或目标部位的调整。
在本实施例中，由于训练图像美化模型所采用的训练样本对，仅需基于少量的目标图像和任意第一样本图像即可生成，因此，训练样本对易于更新迭代，相应训练出的图像美化模型迭代更新速度较快，能够快速响应用户新的图像美化需求。在实际应用中，可以在服务器中训练图像美化模型，然后将训练好的图像美化模型应用于移动终端，也即将图像美化模型下发至手机、电脑等终端中，以供终端调用并对待美化的用户图像进行美化操作。
图4为本公开实施例提供的一种图像处理装置的结构示意图，该 装置用于实现图像处理方法。该装置可以采用软件和/或硬件实现，一般可集成在电子设备中，例如可以应用于各种屏下摄像产品中，诸如手机、电脑和摄像机等。如图4所示，该装置包括：
图像获取模块402，用于获取用户图像；
图像生成模块404，用于采用图像美化模型对用户图像执行美化操作，得到美化后用户图像；
其中，图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；第二样本图像是基于目标图像通过图像处理模型对第一样本图像进行整体美化和图像元素的调整后得到的图像；图像元素为目标图像的图像风格特征和/或目标部位。
在一些实施例中，上述图像处理装置还包括图像处理模块，其用于：
获取第一样本图像和目标图像；利用图像处理模型生成第一样本图像对应的第二样本图像；其中，图像处理模型包括第一子模型和第二子模型，且第一子模型用于进行图像整体美化，第二子模型用于基于目标图像进行图像元素的调整。
在一些实施例中，上述图像处理装置还包括图像美化模型的训练模块，其用于：
将第一样本图像和第二样本图像确定为训练样本对；
基于训练样本对对图像美化模型进行训练，以使训练好的图像美化模型对用户图像执行美化操作；其中，美化操作包括：目标图像的图像风格特征和/或目标部位的调整。
在一些实施例中，上述图像处理模块具体用于：
利用第一子模型对第一样本图像进行整体美化，得到待编辑图像；
利用第二子模型，基于目标图像对待编辑图像的图像元素进行调整，得到第二样本图像。
在一些实施例中，上述图像处理模块具体用于：
将目标图像和待编辑图像输入至第二子模型；
通过第二子模型基于目标图像，对待编辑图像的图像风格进行转换，得到第二样本图像；其中，第二样本图像同时包括待编辑图像的风格特征和目标图像的图像风格特征。
在一些实施例中，上述图像处理模块具体用于：
基于目标图像得到第一分割图像；基于待编辑图像得到第二分割图像；将第一分割图像、第二分割图像、目标图像和待编辑图像输入至第二子模型得到第二样本图像。
在一些实施例中，上述图像处理模块具体用于：
对目标图像进行人脸分割，得到初始分割图像；按照初始分割图像上的分割部位，对初始分割图像添加不同的标识，得到具有不同标识的多张第一分割图像；其中，标识用于表示第一分割图像上作为美化目标的部位；
对待编辑图像进行人脸分割，得到第二分割图像；
利用第二子模型，将第二分割图像的目标部位与第一分割图像中标识所表示的第一部位进行处理，得到第二样本图像；其中，目标部位与第一部位为人脸的相同部位。
在一些实施例中，上述图像处理模块具体用于：
根据第一分割图像的标识，确定第一分割图像上作为美化目标的第一部位；
在第二分割图像上确定与第一部位相同的第二部位，并确定第二部位为待编辑图像的目标部位。
在一些实施例中，上述图像处理模块具体用于：
获取第一图像集合和第二图像集合；其中，第一图像集合中包括未美化的图像，第二图像集合中包括已美化的图像，且第二图像集合中的图像数量少于第一图像集合中的图像数量；
从第一图像集合中获取任意一张未美化的图像，作为第一样本图像；
从第二图像集合中获取任意一张已美化的图像，作为目标图像；其中，目标图像作为第一样本图像的美化目标。
在一些实施例中，上述图像处理装置还包括第二子模型的训练模块，其用于：
利用第一子模型生成与第一图像集合中未美化的图像相对应的生成图像，并由各个生成图像确定第三图像集合；
将第三图像集合中的图像与第二图像集合中已美化的图像进行任意组合，得到不同的多个图像组合；其中，在图像组合中，属于第三图像集合的图像为第三样本图像，属于第二图像集合的图像为第四样本图像；
利用不同的图像组合重复如下训练操作，直至满足预设条件时停 止：
将图像组合输入至待训练的第二子模型生成第五样本图像；
确定第五样本图像与第四样本图像之间的损失函数；
基于损失函数对待训练的第二子模型进行收敛，直至损失函数收敛至预设值时确定满足预设条件并停止训练；
将停止训练时得到的第二子模型作为训练好的第二子模型。
本公开实施例所提供的图像处理装置可执行本发明任意实施例所提供的图像处理方法，具备执行方法相应的功能模块和有益效果。
图5为本公开实施例提供的一种电子设备的结构示意图。如图5所示，电子设备500包括一个或多个处理器501和用于存储所述处理器501可执行指令的存储器502。
处理器501，用于从所述存储器502中读取所述可执行指令，并执行所述指令以实现上述实施例中的图像处理方法。处理器501可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备500中的其他组件以执行期望的功能。
存储器502可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器501可 以运行所述程序指令，以实现上文所述的本公开的实施例的图像处理方法以及/或者其他期望的功能。在所述计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
在一个示例中，电子设备500还可以包括：输入装置503和输出装置504，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
此外，该输入装置503还可以包括例如键盘、鼠标等等。
该输出装置504可以向外部输出各种信息，包括确定出的距离信息、方向信息等。该输出装置504可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
当然，为了简化，图5中仅示出了该电子设备500中与本公开有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备500还可以包括任何其他适当的组件。
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本公开实施例所述图像处理方法。
所述计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作 为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本公开实施例所提供的图像处理方法。
所述计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
本公开实施例还提供了一种计算机程序产品，包括计算机程序/指令，该计算机程序/指令被处理器执行时实现本公开实施例中的方法。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素， 或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所述的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (14)

  1. 一种图像处理方法，其特征在于，包括：
    获取用户图像；
    采用图像美化模型对所述用户图像执行美化操作，得到美化后用户图像；
    其中，所述图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；所述第二样本图像是基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整后得到的图像；所述图像元素为目标图像的图像风格特征和/或目标部位。
  2. 根据权利要求1所述的方法，其特征在于，所述基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整，包括：
    获取第一样本图像和目标图像；
    利用图像处理模型生成所述第一样本图像对应的第二样本图像；其中，所述图像处理模型包括第一子模型和第二子模型，且所述第一子模型用于进行图像整体美化，所述第二子模型用于基于所述目标图像进行图像元素的调整。
  3. 根据权利要求2所述的方法，其特征在于，所述图像美化模型的训练过程包括：
    将所述第一样本图像和所述第二样本图像确定为所述训练样本对；
    基于所述训练样本对对所述图像美化模型进行训练，以使训练好的图像美化模型对所述用户图像执行所述美化操作；其中，所述美化操作包括：目标图像的图像风格特征和/或目标部位的调整。
  4. 根据权利要求2所述的方法，其特征在于，所述利用图像处理模型生成所述第一样本图像对应的第二样本图像，包括：
    利用所述第一子模型对所述第一样本图像进行整体美化，得到待编辑图像；
    利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整，得到第二样本图像。
  5. 根据权利要求4所述的方法，其特征在于，所述利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整，得到第二样本图像，包括：
    将所述目标图像和所述待编辑图像输入至所述第二子模型；
    通过所述第二子模型基于所述目标图像，对所述待编辑图像的图像风格进行转换，得到第二样本图像；其中，所述第二样本图像同时包括所述待编辑图像的风格特征和所述目标图像的图像风格特征。
  6. 根据权利要求4所述的方法，其特征在于，所述方法还包括：
    基于所述目标图像得到第一分割图像；
    基于待编辑图像得到第二分割图像；
    将所述第一分割图像、第二分割图像、所述目标图像和所述待编辑图像输入至第二子模型得到第二样本图像。
  7. 根据权利要求4所述的方法，其特征在于，所述利用所述第二子模型，基于所述目标图像对所述待编辑图像的图像元素进行调整， 得到第二样本图像，包括：
    对所述目标图像进行人脸分割，得到初始分割图像；按照所述初始分割图像上的分割部位，对所述初始分割图像添加不同的标识，得到具有不同标识的多张第一分割图像；其中，所述标识用于表示所述第一分割图像上作为美化目标的部位；
    对所述待编辑图像进行人脸分割，得到第二分割图像；
    利用所述第二子模型，将所述第二分割图像的目标部位与所述第一分割图像中标识所表示的第一部位进行处理，得到第二样本图像；其中，所述目标部位与所述第一部位为人脸的相同部位。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    根据所述第一分割图像的标识，确定所述第一分割图像上作为美化目标的第一部位；
    在所述第二分割图像上确定与所述第一部位相同的第二部位，并确定所述第二部位为所述待编辑图像的目标部位。
  9. 根据权利要求2所述的方法，其特征在于，所述获取第一样本图像和目标图像，包括：
    获取第一图像集合和第二图像集合；其中，所述第一图像集合中包括未美化的图像，所述第二图像集合中包括已美化的图像，且所述第二图像集合中的图像数量少于所述第一图像集合中的图像数量；
    从所述第一图像集合中获取任意一张未美化的图像，作为第一样本图像；
    从所述第二图像集合中获取任意一张已美化的图像，作为目标图像；其中，所述目标图像作为所述第一样本图像的美化目标。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    利用所述第一子模型生成与所述第一图像集合中未美化的图像相对应的生成图像，并由各个所述生成图像确定第三图像集合；
    将所述第三图像集合中的图像与所述第二图像集合中已美化的图像进行任意组合，得到不同的多个图像组合；其中，在所述图像组合中，属于所述第三图像集合的图像为第三样本图像，属于所述第二图像集合的图像为第四样本图像；
    利用不同的图像组合重复如下训练操作，直至满足预设条件时停止：
    将所述图像组合输入至待训练的第二子模型生成第五样本图像；
    确定所述第五样本图像与所述第四样本图像之间的损失函数；
    基于所述损失函数对所述待训练的第二子模型进行收敛，直至所述损失函数收敛至预设值时确定满足预设条件并停止训练；
    将停止训练时得到的第二子模型作为训练好的第二子模型。
  11. 根据权利要求1所述的方法，其特征在于，所述图像美化模型应用于移动终端。
  12. 一种图像处理装置，其特征在于，包括：
    图像获取模块，用于获取用户图像；
    图像生成模块，用于采用图像美化模型对所述用户图像执行美化操作，得到美化后用户图像；
    其中，所述图像美化模型是基于第一样本图像和第二样本图像确定的训练样本对训练得到的；所述第二样本图像是基于目标图像通过图像处理模型对所述第一样本图像进行整体美化和图像元素的调整 后得到的图像；所述图像元素为目标图像的图像风格特征和/或目标部位。
  13. 一种电子设备，其特征在于，所述电子设备包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    所述处理器，用于从所述存储器中读取所述可执行指令，并执行所述指令以实现上述权利要求1-11中任一所述的图像处理方法。
  14. 一种计算机可读存储介质，其特征在于，所述存储介质存储有计算机程序，所述计算机程序用于执行上述权利要求1-11中任一所述的图像处理方法。
PCT/CN2022/091122 2021-05-08 2022-05-06 一种图像处理方法、装置、设备及介质 Ceased WO2022237633A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/289,855 US20240242406A1 (en) 2021-05-08 2022-05-06 A method, apparatus, device, and medium for image processing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110500411.7A CN113222841A (zh) 2021-05-08 2021-05-08 一种图像处理方法、装置、设备及介质
CN202110500411.7 2021-05-08

Publications (1)

Publication Number Publication Date
WO2022237633A1 true WO2022237633A1 (zh) 2022-11-17

Family

ID=77093864

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/091122 Ceased WO2022237633A1 (zh) 2021-05-08 2022-05-06 一种图像处理方法、装置、设备及介质

Country Status (3)

Country Link
US (1) US20240242406A1 (zh)
CN (1) CN113222841A (zh)
WO (1) WO2022237633A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113222841A (zh) * 2021-05-08 2021-08-06 北京字跳网络技术有限公司 一种图像处理方法、装置、设备及介质
CN113822245B (zh) * 2021-11-22 2022-03-04 杭州魔点科技有限公司 人脸识别方法、电子设备和介质
CN114387158B (zh) * 2022-01-10 2025-09-19 北京字跳网络技术有限公司 特效图像生成方法、装置、电子设备及存储介质
JP2023175441A (ja) * 2022-05-30 2023-12-12 キヤノン株式会社 画像処理装置とその制御方法、及びプログラム
CN115456195B (zh) * 2022-08-31 2025-12-19 Oppo广东移动通信有限公司 用于增量学习的模型训练方法、装置、介质与电子设备

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610042A (zh) * 2017-08-23 2018-01-19 维沃移动通信有限公司 一种图像美化方法及移动终端
US20180357800A1 (en) * 2017-06-09 2018-12-13 Adobe Systems Incorporated Multimodal style-transfer network for applying style features from multi-resolution style exemplars to input images
CN109410131A (zh) * 2018-09-28 2019-03-01 杭州格像科技有限公司 基于条件生成对抗神经网络的人脸美颜方法及系统
CN110070484A (zh) * 2019-04-02 2019-07-30 腾讯科技（深圳）有限公司 图像处理、图像美化方法、装置和存储介质
CN110516545A (zh) * 2019-07-22 2019-11-29 北京迈格威科技有限公司 模型训练、图像处理方法和设备、图像处理器及介质
CN111275650A (zh) * 2020-02-25 2020-06-12 北京字节跳动网络技术有限公司 美颜处理方法及装置
CN111292262A (zh) * 2020-01-19 2020-06-16 腾讯科技（深圳）有限公司 图像处理方法、装置、电子设备以及存储介质
CN112686820A (zh) * 2020-12-29 2021-04-20 北京旷视科技有限公司 虚拟美妆方法、装置和电子设备
CN113222841A (zh) * 2021-05-08 2021-08-06 北京字跳网络技术有限公司 一种图像处理方法、装置、设备及介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107993216B (zh) * 2017-11-22 2022-12-20 腾讯科技（深圳）有限公司 一种图像融合方法及其设备、存储介质、终端
CN107995428B (zh) * 2017-12-21 2020-02-07 Oppo广东移动通信有限公司 图像处理方法、装置及存储介质和移动终端
CN111127378A (zh) * 2019-12-23 2020-05-08 Oppo广东移动通信有限公司 图像处理方法、装置、计算机设备及存储介质

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180357800A1 (en) * 2017-06-09 2018-12-13 Adobe Systems Incorporated Multimodal style-transfer network for applying style features from multi-resolution style exemplars to input images
CN107610042A (zh) * 2017-08-23 2018-01-19 维沃移动通信有限公司 一种图像美化方法及移动终端
CN109410131A (zh) * 2018-09-28 2019-03-01 杭州格像科技有限公司 基于条件生成对抗神经网络的人脸美颜方法及系统
CN110070484A (zh) * 2019-04-02 2019-07-30 腾讯科技（深圳）有限公司 图像处理、图像美化方法、装置和存储介质
CN110516545A (zh) * 2019-07-22 2019-11-29 北京迈格威科技有限公司 模型训练、图像处理方法和设备、图像处理器及介质
CN111292262A (zh) * 2020-01-19 2020-06-16 腾讯科技（深圳）有限公司 图像处理方法、装置、电子设备以及存储介质
CN111275650A (zh) * 2020-02-25 2020-06-12 北京字节跳动网络技术有限公司 美颜处理方法及装置
CN112686820A (zh) * 2020-12-29 2021-04-20 北京旷视科技有限公司 虚拟美妆方法、装置和电子设备
CN113222841A (zh) * 2021-05-08 2021-08-06 北京字跳网络技术有限公司 一种图像处理方法、装置、设备及介质

Also Published As

Publication number Publication date
CN113222841A (zh) 2021-08-06
US20240242406A1 (en) 2024-07-18

Similar Documents

Publication Publication Date Title
WO2022237633A1 (zh) 一种图像处理方法、装置、设备及介质
JP7596533B2 (ja) 動物顔スタイル画像の生成方法、モデルのトレーニング方法、装置及び機器
CN117252791A (zh) 图像处理方法、装置、电子设备及存储介质
CN113536007A (zh) 一种虚拟形象生成方法、装置、设备以及存储介质
US20240046538A1 (en) Method for generating face shape adjustment image, model training method, apparatus and device
CN111240669A (zh) 界面生成方法、装置、电子设备及计算机存储介质
WO2022100680A1 (zh) 混血人脸图像生成方法、模型训练方法、装置和设备
WO2019227429A1 (zh) 多媒体内容生成方法、装置和设备/终端/服务器
CN118644596A (zh) 一种人脸关键点运动图像生成方法以及相关设备
CN115423936A (zh) Ai虚拟人物和影像处理方法、系统、电子设备及存储介质
KR20220024227A (ko) 데이터 주석화 방법, 관련 장치 및 컴퓨터 프로그램
CN116528015A (zh) 数字人视频的生成方法、装置、电子设备和存储介质
CN117201837A (zh) 视频生成方法、装置、电子设备及存储介质
CN114630190A (zh) 关节姿态参数的确定方法、模型训练方法及装置
CN116166125B (zh) 虚拟形象构建方法、装置、设备和存储介质
CN115766971A (zh) 演示视频生成方法、装置、电子设备和可读存储介质
CN115082298A (zh) 图像生成方法、装置、电子设备以及存储介质
CN118447131B (zh) 绘本生成方法、装置、电子设备、存储介质及产品
CN118864283A (zh) 图像处理方法及装置、计算机程序产品和电子设备
CN118197305A (zh) 语音控制方法、装置和电子设备
CN121436624A (zh) 一种智能工作流生成方法、装置、设备及存储介质
WO2025199881A1 (zh) 一种生成媒体内容的方法、装置、设备和存储介质
CN121746537A (zh) 基于人脸自适应替换的定制化人物图像生成方法及装置
CN119990081A (zh) 图表生成大模型的训练方法、图表生成方法及相关装置
CN117351121A (zh) 数字人编辑控制方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22806599

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18289855

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12-03-2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22806599

Country of ref document: EP

Kind code of ref document: A1