WO2022100690A1 - 动物脸风格图像生成方法、模型训练方法、装置和设备 - Google Patents

动物脸风格图像生成方法、模型训练方法、装置和设备 Download PDF

Info

Publication number
WO2022100690A1
WO2022100690A1 PCT/CN2021/130301 CN2021130301W WO2022100690A1 WO 2022100690 A1 WO2022100690 A1 WO 2022100690A1 CN 2021130301 W CN2021130301 W CN 2021130301W WO 2022100690 A1 WO2022100690 A1 WO 2022100690A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
image
animal
style
animal face
Prior art date
Application number
PCT/CN2021/130301
Other languages
English (en)
French (fr)
Inventor
何茜
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Priority to US18/252,855 priority Critical patent/US20240005466A1/en
Priority to JP2023528414A priority patent/JP2023549810A/ja
Priority to EP21891210.3A priority patent/EP4246425A4/en
Publication of WO2022100690A1 publication Critical patent/WO2022100690A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/02—Affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a method for generating an animal face style image, a method for training a model, an apparatus and a device.
  • Transforming image styles refers to transforming one or more images from one style to another.
  • the types of style transformation supported in current video interaction applications are still limited and less interesting, which leads to poor user experience and is difficult to meet the user's personalized image style transformation needs.
  • the embodiments of the present disclosure provide a method for generating an animal face style image, a method for training a model, an apparatus and a device.
  • an embodiment of the present disclosure provides a method for generating an animal face style image, including:
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face, and the animal face style image generation model is based on the first face sample image and the first animal face style Sample image training is obtained, the first animal face style sample image is generated by a pre-trained animal face generation model based on the first human face sample image, and the animal face generation model is based on the second human face sample image and the first animal Face sample images are trained.
  • an embodiment of the present disclosure also provides a method for training an animal face style image generation model, including:
  • the image generation model is trained based on the second human face sample image and the first animal face sample image to obtain an animal face generation model
  • a first animal face style sample image corresponding to the first human face sample image is obtained based on the animal face generation model; wherein, the first animal face style sample image refers to the human face on the first face sample image.
  • the image after the face is transformed into an animal face;
  • the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face. image.
  • an embodiment of the present disclosure further provides a device for generating an animal face style image, including:
  • the original face image acquisition module is used to obtain the original face image
  • a style image generation module used for generating a model of the pre-trained animal face style image to obtain an animal face style image corresponding to the original face image
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face, and the animal face style image generation model is based on the first face sample image and the first animal face style Sample image training is obtained, the first animal face style sample image is generated by a pre-trained animal face generation model based on the first human face sample image, and the animal face generation model is based on the second human face sample image and the first animal Face sample images are trained.
  • an embodiment of the present disclosure further provides an apparatus for training an animal face style image generation model, including:
  • the animal face generation model training module is used to train the image generation model based on the second human face sample image and the first animal face sample image to obtain the animal face generation model;
  • a style sample image generation module configured to obtain a first animal face style sample image corresponding to the first human face sample image based on the animal face generation model; wherein, the first animal face style sample image refers to the The image of the human face on the face sample image transformed into the animal face;
  • a style image generation model training module configured to train a style image generation model based on the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model
  • the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face. image.
  • embodiments of the present disclosure further provide an electronic device, including a memory and a processor, wherein: a computer program is stored in the memory, and when the computer program is executed by the processor, the processor Execute any of the animal face style image generation methods or the animal face style image generation model training methods provided in the embodiments of the present disclosure.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the processor executes the computer program provided by the embodiment of the present disclosure. Either an animal face style image generation method or an animal face style image generation model training method.
  • the animal face style image generation model can be pre-trained in the server, and then sent to the terminal for the terminal to call and generate the animal face style image corresponding to the original face image, which can enrich the information in the terminal.
  • the video interactive application as an example, call the animal face style image generation model to get the animal face style image corresponding to the original face image, which can not only enrich the image editing function of the application, but also improve the video
  • the fun of interactive applications provides users with more novel special effects and gameplay, thereby improving the user experience.
  • animal face style image generation model it is possible to dynamically generate animal face style images adapted to the original face images of users according to the original face images of different users, so as to improve the intelligence of generating animal face style images, and to present them. Better image effects, such as getting more realistic animal face style images.
  • FIG. 1 is a flowchart of a method for generating an animal face style image according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of another method for generating an animal face style image according to an embodiment of the present disclosure
  • FIG. 3 is a flowchart of a method for training an animal face style image generation model according to an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of an apparatus for generating an animal face style image according to an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of an apparatus for training an animal face style image generation model according to an embodiment of the present disclosure
  • FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a method for generating an animal face style image according to an embodiment of the present disclosure.
  • the animal face style image generation method can be executed by an animal face style image generation device, which can be implemented by software and/or hardware, and can be integrated on any electronic device with computing capabilities, such as a smart phone, tablet computer, laptop computer Wait for the terminal.
  • the animal face style image generation device can be implemented in the form of an independent application program or a small program integrated on the public platform, and can also be implemented as an application program with a style image generation function or a functional module integrated in the applet.
  • Functional applications may include, but are not limited to, video interaction applications, and the applet may include, but is not limited to, video interaction applets, and the like.
  • the method for generating an animal face style image can be applied to a scene in which an animal face style image is obtained.
  • the animal face style images or the animal face style sample images both refer to images obtained after transforming a human face into an animal face, for example, transforming a human face into a cat's face or a dog's face, etc.
  • the faces of other animals get an image of the animal face style class.
  • the expression on the human face can be consistent with the expression on the animal face
  • the facial features on the human face can also be consistent with the facial features on the animal face, such as a smile on the human face.
  • the face of the corresponding animal also shows a smiling expression; the eyes on the human face are in the open state, and the eyes on the corresponding animal face are also in the open state, etc.
  • the method for generating an animal face style image may include:
  • the image stored in the terminal may be acquired or an image or video may be captured in real time by an image capturing device of the terminal.
  • the animal face style image generating apparatus acquires the original face image to be processed according to the user's image selection operation, image capturing operation or image uploading operation in the terminal.
  • a photo-taking prompt can be displayed on the image capture interface.
  • the photographing prompt information can be used to prompt the user to place the face of the face image in the image acquisition interface at a preset position on the terminal screen (for example, the middle position of the screen, etc.), adjust the distance between the face and the terminal screen (adjust the The distance can be used to obtain the appropriate size of the face area in the image acquisition interface, to avoid the face area being too large or too small, etc.) and adjust the rotation angle of the face (different rotation angles correspond to different face orientations, such as frontal or sideways.
  • the user shoots an image according to the photographing prompt information, so that the video interaction application can conveniently obtain the original face image that meets the input requirements of the animal face-style image generation model.
  • the input requirements of the animal face style image generation model may refer to the constraints on the input image, such as the position of the face on the input image, the size of the input image, etc.
  • the video interaction application can also pre-store a photo template according to the input requirements of the animal face style image generation model, and the photo template predefines the position of the user's face on the image, the size of the face area on the image, the Angle, image size and other information, the video interaction application can obtain the required original face image by using the photographing template according to the user's photographing operation.
  • the images captured by the user can be cropped, zoomed, rotated, etc. process to obtain the original face image that matches the model input.
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face
  • the animal face style image generation model has the function of transforming a human face into an animal face.
  • the animal face style image generation model is trained based on the first human face sample image and the first animal face style sample image, and the first animal face style sample image is generated by the pre-trained animal face generation model based on the first face sample image, that is, the animal
  • the face generation model has the function of generating a corresponding animal face style image for any face image, and the corresponding first animal face style sample image is obtained by transforming the face on the first face sample image into an animal face.
  • the animal face generation model is trained based on the second human face sample image and the first animal face sample image.
  • the first animal face sample image refers to an animal face image showing real animal face features.
  • the second human face sample image and the first The face sample images may be the same face images or different face images, which are not specifically limited in the embodiment of the present disclosure.
  • the plurality of first animal face sample images participating in the training of the animal face generation model correspond to the same animal type.
  • the plurality of first animal face sample images participating in the training of the animal face generation model are all animal face images corresponding to cats or dogs.
  • the multiple first animal face sample images participating in the training of the animal face generation model can also correspond to the animal face images belonging to the same species under the same animal type, such as multiple first animal face samples participating in the training of the animal face generation model
  • the images are all animal face images corresponding to the raccoon cat breed or the Persian cat breed, that is, in the embodiment of the present disclosure, a plurality of animal face generation models can be separately trained for different animal types or different animal breeds under the same animal type, so that Each animal face generation model has the function of generating a specific type or specific breed of animal face images.
  • the first animal face sample image may be obtained by collecting animal images photographed for animals on the Internet.
  • the above model training process may include: first, training an image generation model based on the second human face sample image and the first animal face sample image to obtain an animal face generation model, wherein the available image generation model may be: Including but not limited to Generative Adversarial Networks (GAN, Generative Adversarial Networks) models, Style-Based Generative Adversarial Networks (Stylegan, Style-Based Generator Architecture for Generative Adversarial Networks) models, etc.
  • GAN Generative Adversarial Networks
  • Style-Based Generative Adversarial Networks Style-Based Generative Adversarial Networks
  • Stylegan Style-Based Generator Architecture for Generative Adversarial Networks
  • a first animal face style sample image corresponding to the first human face sample image is obtained based on the animal face generation model, where the first animal face style sample image refers to a face after transforming the face on the first face sample image into an animal face Image.
  • the style image generation model is trained based on the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model, wherein the available style image generation model may include conditions such as conditional generative adversarial networks ( CGAN, Conditional Generative Adversarial Networks) model, Cycle Consistent Generative Adversarial Networks (Cycle-GAN, Cycle Consistent Adversarial Networks) model, etc.
  • CGAN conditional generative adversarial networks
  • Cycle-GAN Cycle Consistent Generative Adversarial Networks
  • the first animal face style sample image corresponding to the first face sample image is obtained by using the animal face generation model, and then the first face sample image and the first animal face style sample image are used as paired training samples for training Obtaining the animal face style image generation model can ensure the training effect of the animal face style image generation model, and then can ensure that the generated animal face style image corresponding to the original face image has a better display effect, such as obtaining a more realistic animal face. style image.
  • the first face sample image is based on the difference between the face key points on the first original face sample image and the animal face key points on the first original animal face sample image
  • the first correspondence is obtained after adjusting the face position of the first original face sample image
  • the second face sample image is based on the second correspondence between the face key points on the second original face sample image and the animal face key points on the first original animal face sample image.
  • the image is obtained after adjusting the face position;
  • the first animal face sample image is obtained by adjusting the position of the animal face on the first original animal face sample image based on the first correspondence or the second correspondence.
  • the first correspondence between the face key points and the animal face key points on the first original animal face sample image so as to adjust the face position of the first original face sample image based on the first correspondence
  • the first face sample image of the input requirements of the face generation model or the animal face style image generation model (for example, the face position on the image, the image size, etc.); similarly, the first animal face sample image can also be pre-based on the first face sample image
  • the correspondence relationship is obtained by adjusting the position of the animal face on the first original animal face sample image, and the first animal face sample image also meets the input requirements of the model.
  • an affine transformation matrix for adjusting the position of the face on the first original face sample image may be constructed based on the face key points participating in the first correspondence, and based on this
  • the affine transformation matrix is used to adjust the face position of the first original face sample image to obtain the first face sample image; based on the animal face key points participating in the first correspondence, it is used to adjust the animal on the first original animal face sample image.
  • the affine transformation matrix of the face position, and based on the affine transformation matrix, the animal face position adjustment is performed on the first original animal face sample image to obtain the first animal face sample image.
  • the specific construction of the affine transformation matrix can refer to the principle of affine transformation.
  • the affine transformation matrix may be related to parameters such as scaling parameters and cropping ratios of the first original human face sample image or the first original animal face sample image, that is, in the process of adjusting the position of the human face or the position of the animal face, the Image processing operations may include cropping, scaling, rotation, etc., which may be determined according to image processing requirements.
  • Image adjustment is performed based on the same keypoint correspondence, and the final first human face sample image and the first animal face sample image have the same image size, and the face area on the first human face sample image and the first animal face
  • the animal face area on the sample image corresponds to the same image position.
  • the human face area is located in the center area of the first human face sample image, and the animal face area is also located in the center area of the first animal face sample image.
  • the difference between the area of the face area is less than the area threshold (the value can be set flexibly), that is, the area of the face area matches the area of the animal face area, so as to ensure that the first animal face style with better display effect can be generated based on the animal face generation model.
  • animal face style image generation model can ensure a better model training effect and avoid the animal face style images generated by using the animal face style image generation model.
  • Mismatch affects the display effect of animal face style images, for example, the animal face area is too large or too small compared to the human face area.
  • the animal face generation model Before training to obtain the animal face generation model, it is also possible to first determine the second correspondence between the face key points in the second original face sample image and the animal face key points in the first original animal face sample image. Then based on the second correspondence, the second original face sample image is adjusted to the face position, and the involved image processing operations can include cropping, scaling, rotation, etc., to obtain a second face that meets the input image conditions of the image generation model Sample image.
  • the first animal that meets the input requirements of the image generation model after adjusting the position of the animal face on the first original animal face sample image in advance based on the second correspondence. face sample image.
  • an affine transformation matrix for adjusting the position of the face on the second original face sample image may also be constructed based on the face key points participating in the second correspondence, based on The animal face key points participating in the second correspondence construct an affine transformation matrix for adjusting the position of the animal face on the first original animal face sample image.
  • the finally obtained second human face sample image and the first animal face sample image have the same image size, and the human face area on the second human face sample image and the animal face area on the first animal face sample image correspond to the same image position , for example, the human face area is located in the center area of the second human face sample image, and the animal face area is also located in the center area of the first animal face sample image, etc.
  • the difference between the area of the human face area and the area of the animal face area is less than the area threshold (take The value can be set flexibly), that is, the area of the face area matches the area of the animal face area, so as to ensure a better model training effect based on high-quality training samples.
  • the animal face style image generation model is obtained by training based on the first human face sample image and the second animal face style sample image, and the second animal face style sample image is replaced by the background area in the first animal face style sample image. obtained after the background region in a sample face image.
  • background replacement in the process of training to obtain the animal face style image generation model, the influence of the background area on the animal face style sample image on the model training effect can be minimized, so as to ensure a better model training effect, and then ensure the generated Animal face style images have a better display effect.
  • the second animal face style sample image is obtained by fusing the first animal face style sample image and the first human face sample image based on the second animal face mask image; the second animal face mask image is obtained by fusing the first animal face style sample image and the first human face sample image;
  • the pre-trained animal face segmentation model is obtained based on the first animal face style sample image, and the second animal face mask image is used to determine the animal face area on the first animal face style sample image as the second animal face style sample image. animal face area.
  • the animal face segmentation model can be obtained by training based on the second animal face sample image and the position labeling result of the animal face region on the second animal face sample image. On the basis of ensuring that the animal face segmentation model has the function of generating a mask image corresponding to the animal face region on the image, those skilled in the art can use any available training method to implement, which is not specifically limited in the embodiment of the present disclosure.
  • the animal face style image generation model can be pre-trained in the server, and then sent to the terminal for the terminal to call and generate the animal face style image corresponding to the original face image, which can enrich the information in the terminal.
  • the video interactive application as an example, call the animal face style image generation model to get the animal face style image, which can not only enrich the image editing function of the application, but also enhance the interest of the application and provide users with a comparison Novel special effects gameplay, thereby improving the user experience.
  • using the animal face style image generation model can realize the original face images of different users, dynamically generate animal face style images adapted to the user's original face images, improve the intelligence of generating animal face style images, and present a better image. good image effect.
  • FIG. 2 is a flowchart of another method for generating an animal face style image according to an embodiment of the present disclosure, which is further optimized and expanded based on the above technical solution, and can be combined with each of the above optional embodiments.
  • the method for generating an animal face style image may include:
  • the application or applet can display the animal feature type selection interface to the user, and the animal feature types can be distinguished according to different animal types, such as Cat face effects or dog face effects can also be distinguished according to different animal species, such as raccoon cat face effects or Persian cat face effects;
  • the terminal determines the type of animal effects that the user currently needs to generate corresponding to which animal according to the type of animal effects selected by the user.
  • the animal face style image is used to determine the correspondence between the key points of the animal's face and the key points of the human face. This correspondence can be pre-stored in the terminal for the terminal to call according to the type of animal special effects.
  • the terminal may also establish a correspondence between the animal face key points and the human face key points after determining the animal face corresponding to the animal special effect type selected by the user and recognizing the human face key points on the user image.
  • the user image may be an image obtained by the terminal according to an image selection operation, an image capturing operation or an image uploading operation performed by the user in the terminal.
  • the face position of the user image is adjusted to obtain the original face image.
  • the original face image meets the input requirements of the animal face style image generation model.
  • the input requirements corresponding to the model (such as the face position on the image, image size, etc.) are also determined at the same time. Therefore, the terminal uses the key point recognition technology to identify the face key points on the user image. Then, adjust the face position of the user image based on the determined corresponding relationship. For example, the terminal can use the face key points on the user image that belong to the corresponding relationship to construct an affine transformation matrix for adjusting the face position on the user image. , using the affine transformation matrix to adjust the face position on the user image, and the involved image processing operations may include cropping, scaling, rotation, etc., to obtain the original face image that meets the input requirements of the animal face style image generation model.
  • the background area on the user image refers to the remaining image area on the user image except the face area.
  • image processing technology can be used to extract the animal face area from the animal face style image, and the background area from the user image, and then according to the position of the background area and the position of the face area on the user image, the two Fusion (or blending). That is, on the target animal face style image finally displayed to the user, except that the user face features become animal face features, the image background retains the background area on the user image, which avoids the need for the user image in the process of generating the animal face style image. Changes to the upper background area.
  • the animal face area on the animal face style image is fused with the background area on the user image to obtain a target animal face style image corresponding to the user image, including:
  • an intermediate result image with the same image size as the user image is obtained; wherein, the position of the animal face area on the intermediate result image is the same as the position of the human face area on the user image;
  • the corresponding relationship between the animal face key points and the human face key points on the user image, the animal face style image is mapped to the image coordinates corresponding to the user image, and the intermediate result image is obtained.
  • a first animal face mask image corresponding to the animal effect type is determined.
  • the user image and the intermediate result image are fused to obtain the target animal face style image corresponding to the user image; wherein, the first animal face mask image is used to fuse the image on the intermediate result image
  • the animal face region is determined as the animal face region on the target animal face style image.
  • the first animal face mask image By using the first animal face mask image to realize the fusion of the user image and the intermediate result image, on the basis of ensuring that the target animal face style image is successfully obtained, it helps to improve the efficiency of image fusion processing.
  • the user image and the intermediate result image are fused to obtain a target animal face style image corresponding to the user image, which may include:
  • the smooth transition between the background area on the user image and the animal face area on the intermediate result image can be achieved, and the image fusion effect can be optimized. , to ensure the final presentation of the target animal face style image.
  • the special effect logo selected by the user can also be determined according to the special effect selection operation of the user on the image editing interface. , adding a special effect corresponding to the special effect identifier selected by the user to the aforementioned target animal face style image or the aforementioned animal face style image, so as to further enhance the interest of image editing.
  • the special effects selectable by the user may include any type of props or stickers, etc., which are not specifically limited in this embodiment of the present disclosure.
  • the user image after the user image is obtained, first adjust the face position of the user image according to the corresponding relationship between the animal face key points corresponding to the animal face special effect type selected by the user and the human face key points to obtain the original face image, and then use the animal face style image generation model to obtain the animal face style image corresponding to the original face image, and finally fuse the animal face area on the animal face style image with the background area on the user image to display it to the user
  • the target animal face style image of the user's face while animalizing the user's facial features, retains the original background on the user's image, and enriches the image editing function in the terminal.
  • calling the animal face style image generation model to obtain animal face style images not only enriches the image editing functions of the application, but also improves the fun of the application, providing users with relatively novel special effects gameplay. , thereby improving the user experience.
  • FIG. 3 is a flowchart of a method for training an animal face style image generation model provided by an embodiment of the present disclosure, which is applied to the situation of how to train an animal face style image generation model with the function of transforming a human face into an animal face.
  • the animal face style image generation model training method can be executed by an animal face style image generation model training device, which can be implemented by software and/or hardware, and can be integrated in a server.
  • the animal face style image generation model training method provided by the embodiments of the present disclosure is executed in cooperation with the animal face style image generation method provided by the embodiments of the present disclosure.
  • the method for training an animal face style image generation model may include:
  • the first animal face style sample image refers to an image obtained by transforming the human face on the first human face sample image into an animal face.
  • S303 Train a style image generation model based on the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model.
  • the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image, and the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face.
  • the model training method provided by the embodiment of the present disclosure further includes:
  • the model training method provided by the embodiment of the present disclosure further includes: determining a face key on the first original face sample image The first correspondence between the points and the animal face key points on the first original animal face sample image; based on the first correspondence, the animal face position adjustment is performed on the first original face sample image to obtain the first face sample image .
  • the model training method provided by the embodiment of the present disclosure further includes: The background area of is replaced with the background area in the first face sample image, and the second animal face style sample image is obtained.
  • the style image generation model is trained based on the first human face sample image and the first animal face style sample image to obtain an animal face style image generation model, including: based on the first face sample image and the second animal face style sample
  • the image-to-style image generation model is trained to obtain an animal face style image generation model.
  • replacing the background area in the first animal face style sample image with the background area in the first human face sample image to obtain a second animal face style sample image including: based on a pre-trained animal face segmentation model, obtaining an animal face mask image corresponding to the first animal face style sample image; based on the animal face mask image, the first animal face style sample image and the first human face sample image are fused to obtain a second animal face style sample image ; wherein, the animal face mask image is used to determine the animal face area on the first animal face style sample image as the animal face area on the second animal face style sample image.
  • the model training method provided by the embodiment of the present disclosure further includes: acquiring a second animal face sample image and a position labeling result of the animal face region on the second animal face sample image; based on the second animal face sample image and the animal face The position labeling result of the face area is trained to obtain an animal face segmentation model.
  • the animal face style image generation model can be pre-trained in the server, and then sent to the terminal for the terminal to call and generate the animal face style image corresponding to the original face image, which can enrich the information in the terminal.
  • the video interactive application as an example, call the animal face style image generation model to get the animal face style image, which can not only enrich the image editing function of the application, but also enhance the interest of the application and provide users with a comparison Novel special effects gameplay, thereby improving the user experience.
  • FIG. 4 is a schematic structural diagram of an apparatus for generating an animal face style image according to an embodiment of the present disclosure, which is applied to the situation of how to transform a user's face into an animal face.
  • the animal face style image generating apparatus can be implemented by software and/or hardware, and can be integrated on any electronic device with computing capabilities, such as user terminals such as smart phones, tablet computers, and notebook computers.
  • the animal face style image generating apparatus 400 provided by the embodiment of the present disclosure includes an original face image obtaining module 401 and a style image generating module 402, wherein:
  • an original face image acquisition module 401 configured to acquire an original face image
  • a style image generation module 402 configured to use a pre-trained animal face style image generation model to obtain an animal face style image corresponding to the original face image;
  • the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face, and the animal face style image generation model is trained based on the first face sample image and the first animal face style sample image.
  • the first animal face style sample image is generated by a pre-trained animal face generation model based on the first human face sample image, and the animal face generation model is trained based on the second human face sample image and the first animal face sample image.
  • the apparatus 400 provided in this embodiment of the present disclosure further includes:
  • the corresponding relationship determination module is used to determine the corresponding relationship between the animal face key points corresponding to the animal special effect type and the human face key points according to the animal special effect type selected by the user;
  • the face position adjustment module is used to adjust the face position of the user image based on the corresponding relationship between the animal face key points corresponding to the animal special effects type and the human face key points, so as to obtain the original face image;
  • the face image meets the input requirements of the animal face style image generation model;
  • the image fusion module is configured to fuse the animal face area on the animal face style image with the background area on the user image to obtain the target animal face style image corresponding to the user image.
  • the image fusion module includes:
  • an intermediate result image determining unit used for obtaining an intermediate result image with the same image size as the user image based on the animal face style image; wherein, the position of the animal face region on the intermediate result image is the same as the position of the human face region on the user image;
  • a first animal face mask image determining unit for determining the first animal face mask image corresponding to the animal special effect type
  • the image fusion unit is used to fuse the user image and the intermediate result image based on the first animal face mask image to obtain the target animal face style image corresponding to the user image; wherein, the first animal face mask image is used for The animal face region on the intermediate result image is determined as the animal face region on the target animal face style image.
  • the first face sample image is based on the first correspondence between the face key points on the first original face sample image and the animal face key points on the first original animal face sample image.
  • the original face sample image is obtained after adjusting the face position;
  • the second face sample image is based on the second correspondence between the face key points on the second original face sample image and the animal face key points on the first original animal face sample image.
  • the image is obtained after adjusting the face position;
  • the first animal face sample image is obtained by adjusting the position of the animal face on the first original animal face sample image based on the first correspondence or the second correspondence.
  • the animal face style image generation model is obtained by training based on the first human face sample image and the second animal face style sample image, and the second animal face style sample image is replaced by the background area in the first animal face style sample image. obtained after the background region in a sample face image.
  • the second animal face style sample image is obtained by fusing the first animal face style sample image and the first human face sample image based on the second animal face mask image;
  • the second animal face mask image is obtained by the pre-trained animal face segmentation model based on the first animal face style sample image, and the second animal face mask image is used to determine the animal face area on the first animal face style sample image is the animal face region on the second animal face style sample image.
  • the apparatus for generating an animal face style image provided by the embodiment of the present disclosure can execute any of the methods for generating an animal face style image provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • the description in any method embodiment of the present disclosure for the content that is not described in detail in the apparatus embodiment of the present disclosure, reference may be made to the description in any method embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an apparatus for training an animal face style image generation model provided by an embodiment of the present disclosure, which is applied to the situation of how to train an animal face style image generation model with the function of transforming a human face into an animal face.
  • the animal face style image generation model training device can be implemented by software and/or hardware, and can be integrated in a server.
  • the animal face style image generation model training apparatus 500 may include an animal face generation model training module 501, a style sample image generation module 502, and a style image generation model training module 503, wherein:
  • the animal face generation model training module 501 is used for training the image generation model based on the second human face sample image and the first animal face sample image to obtain the animal face generation model;
  • the style sample image generation module 502 is used to obtain a first animal face style sample image corresponding to the first face sample image based on the animal face generation model; wherein, the first animal face style sample image refers to the first face sample image The image on which the human face is transformed into an animal face;
  • the style image generation model training module 503 is used for training the style image generation model based on the first human face sample image and the first animal face style sample image to obtain the animal face style image generation model;
  • the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image, and the animal face style image refers to an image obtained by transforming the human face on the original face image into an animal face.
  • the apparatus 500 provided in this embodiment of the present disclosure further includes:
  • the second correspondence determination module is used to determine the second correspondence between the face key points in the second original face sample image and the animal face key points in the first original animal face sample image;
  • a face position adjustment module for performing face position adjustment on the second original face sample image based on the second correspondence to obtain a second face sample image
  • An animal face position adjustment module configured to perform an animal face position adjustment on the first original animal face sample image based on the second correspondence to obtain a first animal face sample image
  • a first correspondence determination module configured to determine the first correspondence between the face key points on the first original face sample image and the animal face key points on the first original animal face sample image
  • the face position adjustment module is used to adjust the position of the animal face on the first original face sample image based on the first correspondence, so as to obtain the first face sample image.
  • the apparatus 500 provided in this embodiment of the present disclosure further includes:
  • the background area replacement module is used to replace the background area in the first animal face style sample image with the background area in the first face sample image to obtain the second animal face style sample image;
  • style image generation model training module 503 is specifically used for:
  • the style image generation model is trained based on the first human face sample image and the second animal face style sample image to obtain an animal face style image generation model.
  • the background area replacement module includes:
  • the animal face mask image determining unit is configured to obtain the animal face mask image corresponding to the first animal face style sample image based on the pre-trained animal face segmentation model;
  • the image fusion unit is used for fusing the first animal face style sample image and the first human face sample image based on the animal face mask image to obtain the second animal face style sample image;
  • the animal face region on the first animal face style sample image is determined as the animal face region on the second animal face style sample image.
  • the apparatus 500 provided in this embodiment of the present disclosure further includes:
  • a sample image and labeling result obtaining module used for obtaining the second animal face sample image and the position labeling result of the animal face region on the second animal face sample image
  • the animal face segmentation model training module is used for training an animal face segmentation model based on the second animal face sample image and the position labeling result of the animal face region.
  • the animal face style image generation model training device provided by the embodiment of the present disclosure can execute any animal face style image generation model training method provided by the embodiment of the present disclosure, and has corresponding functional modules and beneficial effects of the execution method.
  • any animal face style image generation model training method provided by the embodiment of the present disclosure, and has corresponding functional modules and beneficial effects of the execution method.
  • FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, which is used to exemplarily illustrate an electronic device that implements the animal face style image generation method or the animal face style image generation model training method provided by the embodiment of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, Mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, servers, and the like.
  • the electronic device shown in FIG. 6 is only an example, and should not impose any limitation on the functions and occupancy scope of the embodiments of the present disclosure.
  • electronic device 600 includes one or more processors 601 and memory 602 .
  • Processor 601 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 600 to perform desired functions.
  • CPU central processing unit
  • Processor 601 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 600 to perform desired functions.
  • Memory 602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Volatile memory may include, for example, random access memory (RAM) and/or cache memory, among others.
  • Non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may execute the program instructions to implement the animal face style image generation method or the animal face style image generation model training method provided by the embodiments of the present disclosure, and further Other desired functions can be implemented.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the method for generating an animal face style image may include: obtaining an original face image; using a pre-trained animal face style image generation model to obtain an animal face style image corresponding to the original face image; wherein, the animal face style image refers to a The face on the original face image is transformed into an image of an animal face.
  • the animal face style image generation model is trained based on the first face sample image and the first animal face style sample image.
  • the first animal face style sample image is pre-trained
  • the animal face generation model is generated based on the first human face sample image, and the animal face generation model is trained based on the second human face sample image and the first animal face sample image.
  • the training method for the animal face style image generation model may include: training the image generation model based on the second human face sample image and the first animal face sample image to obtain the animal face generation model; The first animal face style sample image corresponding to the face sample image; wherein, the first animal face style sample image refers to the image after transforming the human face on the first face sample image into an animal face; based on the first face sample image Train the style image generation model with the first animal face style sample image to obtain an animal face style image generation model; wherein, the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image, and the animal face style The image refers to the image after transforming the human face on the original face image into an animal face.
  • the electronic device 600 may also perform other optional implementations provided by the method embodiments of the present disclosure.
  • the electronic device 600 may also include an input device 603 and an output device 604 interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the input device 603 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 604 can output various information to the outside, including the determined distance information, direction information, and the like.
  • the output device 604 may include, for example, displays, speakers, printers, and communication networks and their connected remote output devices, among others.
  • the electronic device 600 may also include any other suitable components according to the specific application.
  • the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when executed by the processor, cause the processor to perform the animal face style image generation provided by the embodiments of the present disclosure method or animal face style image generation model training method.
  • the computer program product may write program code for performing operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., as well as conventional procedural programming language, such as "C" language or similar programming language.
  • the program code may execute entirely on the user electronic device, partly on the user electronic device, as a stand-alone software package, partly on the user electronic device and partly on the remote electronic device, or entirely on the remote electronic device execute on.
  • embodiments of the present disclosure may further provide a computer-readable storage medium on which computer program instructions are stored, and when executed by the processor, the computer program instructions cause the processor to execute the animal face style image generation provided by the embodiments of the present disclosure method or animal face style image generation model training method.
  • the method for generating an animal face style image may include: obtaining an original face image; using a pre-trained animal face style image generation model to obtain an animal face style image corresponding to the original face image; wherein, the animal face style image refers to a
  • the face on the original face image is transformed into an image of an animal face, and the animal face style image generation model is trained based on the first face sample image and the first animal face style sample image, and the first animal face style sample image is pre-trained
  • the animal face generation model is generated based on the first human face sample image, and the animal face generation model is trained based on the second human face sample image and the first animal face sample image.
  • the training method for the animal face style image generation model may include: training the image generation model based on the second human face sample image and the first animal face sample image to obtain the animal face generation model; The first animal face style sample image corresponding to the face sample image; wherein, the first animal face style sample image refers to the image after transforming the human face on the first face sample image into an animal face; based on the first face sample image Train the style image generation model with the first animal face style sample image to obtain an animal face style image generation model; wherein, the animal face style image generation model is used to obtain an animal face style image corresponding to the original face image, and the animal face style The image refers to the image after transforming the human face on the original face image into an animal face.
  • the processor may also cause the processor to execute other optional implementations provided by the method embodiments of the present disclosure.
  • a computer-readable storage medium can employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种动物脸风格图像生成方法、模型训练方法、装置和设备，该生成方法包括：获取原始人脸图像（S101）；利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像（S102）；其中，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像，动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，第一动物脸风格样本图像由预先训练的动物脸生成模型基于第一人脸样本图像生成，动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。所述方法可以丰富终端中的图像编辑功能，提升视频交互类应用程序的趣味性，为用户提供特效玩法。

Description

动物脸风格图像生成方法、模型训练方法、装置和设备
本申请要求于2020年11月13日提交国家知识产权局、申请号为202011269334.0、申请名称为“动物脸风格图像生成方法、模型训练方法、装置和设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其涉及一种动物脸风格图像生成方法、模型训练方法、装置和设备。
背景技术
随着图像处理技术的发展，视频交互类应用程序的功能逐渐丰富化，变换图像成为了一种新的趣味性玩法。变换图像风格是指将一幅或者多幅图像由一种风格变换为另一种风格。然而，目前的视频交互类应用程序中支持的风格变换类型仍然有限、趣味性较差，进而导致用户体验较差，难以满足用户的个性化图像风格变换需求。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开实施例提供了一种动物脸风格图像生成方法、模型训练方法、装置和设备。
第一方面，本公开实施例提供了一种动物脸风格图像生成方法，包括：
获取原始人脸图像；
利用预先训练的动物脸风格图像生成模型，得到与所述原始人脸图像对应的动物脸风格图像；
其中，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像，所述动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，所述第一动物脸风格样本图像由预先训练的动物脸生成模型基于所述第一人脸样本图像生成，所述动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
第二方面，本公开实施例还提供了一种动物脸风格图像生成模型训练方法，包括：
基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；
基于所述动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，所述第一动物脸风格样本图像是指将所述第一人脸样本图像上的人脸变换为动物脸后的图像；
基于所述第一人脸样本图像和所述第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；
其中，所述动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像。
第三方面，本公开实施例还提供了一种动物脸风格图像生成装置，包括：
原始人脸图像获取模块，用于获取原始人脸图像；
风格图像生成模块，用于利用预先训练的动物脸风格图像生成模型，得到与所述原始人脸图像对应的动物脸风格图像；
其中，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像，所述动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，所述第一动物脸风格样本图像由预先训练的动物脸生成模型基于所述第一人脸样本图像生成，所述动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
第四方面，本公开实施例还提供了一种动物脸风格图像生成模型训练装置，包括：
动物脸生成模型训练模块，用于基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；
风格样本图像生成模块，用于基于所述动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，所述第一动物脸风格样本图像是指将所述第一人脸样本图像上的人脸变换为动物脸后的图像；
风格图像生成模型训练模块，用于基于所述第一人脸样本图像和所述第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；
其中，所述动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像。
第五方面，本公开实施例还提供了一种电子设备，包括存储器和处理器，其中：所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行本公开实施例提供的任一动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法。
第六方面，本公开实施例还提供了一种计算机可读存储介质，所述存储介质中存储有计算机程序，当所述计算机程序被处理器执行时，所述处理器执行本公开实施例提供的任一动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法。
本公开实施例提供的技术方案与现有技术相比至少具有如下优点：
在本公开实施例中，可以在服务器中预先训练得到动物脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的动物脸风格图像，可以丰富终端中的图像编辑功能，以视频交互类应用程序为例，调用该动物脸风格图像生成模型得到与原始人脸图像对应的动物脸风格图像，不仅可以丰富应用程序的图像编辑功能，还能提升该视频交互类应用程序的趣味性，为用户提供更加新颖的特效玩法，进而提高用户的使用体验。并且，采用该动物脸风格图像生成模型，可以实现针对不同用户的原始人脸图像，动态生成与用户原始人脸图像相适应的动物脸风格图像，提高生成动物脸风格图像的智能化，并呈现较好的图像效果，如得到更加真实的动物脸风格图像。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种动物脸风格图像生成方法的流程图；
图2为本公开实施例提供的另一种动物脸风格图像生成方法的流程图；
图3为本公开实施例提供的一种动物脸风格图像生成模型训练方法的流程图；
图4为本公开实施例提供的一种动物脸风格图像生成装置的结构示意图；
图5为本公开实施例提供的一种动物脸风格图像生成模型训练装置的结构示意图；
图6为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。
图1为本公开实施例提供的一种动物脸风格图像生成方法的流程图。该动物脸风格图像生成方法可以由动物脸风格图像生成装置执行，该装置可以采用软件和/或硬件实现，并可集成在任意具有计算能力的电子设备上，例如智能手机、平板电脑、笔记本电脑等终端。
动物脸风格图像生成装置可以采用独立的应用程序或者公众平台上集成的小程序的形式实现，还可以作为具有风格图像生成功能的应用程序或者小程序中集成的功能模块实现，该具有风格图像生成功能的应用程序可以包括但不限于视频交互类应用程序，该小程序可以包括但不限于视频交互类小程序等。
本公开实施例提供的动物脸风格图像生成方法可以应用于获得动物脸风格图像的场景。在本公开实施例中，动物脸风格图像或者动物脸风格样本图像均是指代将人脸变换为动物脸之后得到的图像，例如，将人脸变换为猫的脸部或者狗的脸部等其他动物的脸部，得到动物脸风格类的图像。并且，将人脸变换为动物脸之后，人脸上的表情与动物脸上的表情可以保持一致，人脸上的五官状态与动物脸上的五官状态也可以保持一致，例如人脸上呈现微笑的表情，对应的动物脸上也呈现微笑表情；人脸上的眼部处于睁眼状态，对应的动物脸上的眼部也处于睁眼状态等。
如图1所示，本公开实施例提供的动物脸风格图像生成方法可以包括：
S101、获取原始人脸图像。
示例性的，当用户存在生成动物脸风格图像的需求时，可以获取存储在终端中的图像或者通过终端的图像拍摄装置实时拍摄图像或者视频。动物脸风格图像生成装置根据用户在终端中的图像选择操作、图像拍摄操作或图像上传操作，获取待处理的原始人脸图像。
以用户通过在视频交互类应用程序中调用终端的图像拍摄装置(例如摄像头)进行实施拍摄图像为例，该视频交互类应用程序跳转到图像采集界面后，可以在图像采集界面上展示拍照提示信息，该拍照提示信息可以用于提示用户将图像采集界面中人脸图像的脸部置于终端屏幕上的预设位置(例如屏幕中间位置等)、调整脸部距离终端屏幕的距离(调整该距离可以在图像采集界面中得到合适尺寸的脸部区域，避免脸部区域过大或者过小等)以及调整脸部的旋转角度(不同的旋转角度对应不同的脸部朝向，例如正脸或者侧脸等)等信息中的至少一种；用户根据拍照提示信息，进行拍摄图像，从而使得视频交互类应用程序可以便捷得到符合动物脸风格图像生成模型输入要求的原始人脸图像。其中，动物脸风格图像生成模型输入要求可以是指对输入图像的限制条件，例如输入图像上的人脸位 置、输入图像的尺寸等。
进一步的，视频交互类应用程序还可以根据动物脸风格图像生成模型输入要求，预先存储拍照模板，该拍照模板预先定义了用户脸部在图像上的位置、图像上脸部区域的大小、脸部角度、图像尺寸等信息，视频交互类应用程序可以根据用户的拍照操作，利用该拍照模板获得所需的原始人脸图像。
当然，当用户拍摄的图像与动物脸风格图像生成模型输入要求的图像条件(例如图像上的人脸位置、图像尺寸等)存在差异时，可以对用户拍摄的图像进行裁剪、缩放、旋转等操作处理，以得到符合模型输入的原始人脸图像。
S102、利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像。
其中，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像，动物脸风格图像生成模型具有将人脸变换为动物脸的功能。动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，第一动物脸风格样本图像由预先训练的动物脸生成模型基于第一人脸样本图像生成，即动物脸生成模型具有为任意人脸图像生成对应的动物脸风格图像的功能，将第一人脸样本图像上的人脸变换为动物脸即得到对应的第一动物脸风格样本图像。动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到，第一动物脸样本图像是指展示有真实动物脸部特征的动物脸图像，第二人脸样本图像和第一人脸样本图像可以是相同的人脸图像，也可以是不同的人脸图像，本公开实施例不作具体限定。
并且，参与动物脸生成模型训练的多个第一动物脸样本图像对应相同的动物类型，例如，参与动物脸生成模型训练的多个第一动物脸样本图像均是对应猫或狗的动物脸图像；进一步细分，参与动物脸生成模型训练的多个第一动物脸样本图像还可以对应相同动物类型下属于相同品种的动物脸图像，例如参与动物脸生成模型训练的多个第一动物脸样本图像均是对应狸花猫品种或者波斯猫品种的动物脸图像，即在本公开实施例中可以针对不同的动物类型或者相同动物类型下的不同动物品种，分别训练得到多个动物脸生成模型，使得每个动物脸生成模型具有生成特定类型或者特定品种的动物脸图像的功能。第一动物脸样本图像可以通过收集互联网中为动物拍摄的动物图像而得到。
关于上述模型的具体训练过程，本公开实施例不作具体限定，本领域技术人员可以根据模型的功能采用任意可用的训练方式实现。示例性的，关于上述模型训练过程可以包括：首先，基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型，其中，可以利用的图像生成模型可以包括但不限于生成对抗网络(GAN，Generative Adversarial Networks)模型、基于样式的生成对抗网络(Stylegan，Style-Based Generator Architecture for Generative Adversarial Networks)模型等。然后，基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像，第一动物脸风格样本图像是指将第一人脸样本图像上的人脸变换为动物脸后的图像。最后，基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型，其中，可以利用的风格图像生成模型可以包括诸如条件生成对抗网络(CGAN，Conditional  Generative Adversarial Networks)模型、循环一致性生成对抗网络(Cycle-GAN，Cycle Consistent Adversarial Networks)模型等。
通过利用动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像，然后将第一人脸样本图像和第一动物脸风格样本图像作为成对的训练样本，用于训练得到动物脸风格图像生成模型，可以确保动物脸风格图像生成模型的训练效果，进而可以确保生成的与原始人脸图像对应的动物脸风格图像具有较好的展示效果，如得到更加真实的动物脸风格图像。
在上述技术方案的基础上，可选的，第一人脸样本图像是基于第一原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系，对第一原始人脸样本图像进行人脸位置调整后得到；
第二人脸样本图像是基于第二原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第二对应关系，对第二原始人脸样本图像进行人脸位置调整后得到；
第一动物脸样本图像是基于第一对应关系或第二对应关系，对第一原始动物脸样本图像进行动物脸位置调整后得到。
即考虑动物脸部和人脸之间的差异，在基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像之前，需要确定第一原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系，从而基于该第一对应关系对第一原始人脸样本图像进行人脸位置调整，以得到符合动物脸生成模型或者动物脸风格图像生成模型的输入要求(例如图像上的人脸位置、图像尺寸等)的第一人脸样本图像；同样地，第一动物脸样本图像也可以预先基于该第一对应关系对第一原始动物脸样本图像进行动物脸位置调整后得到，第一动物脸样本图像同样符合模型的输入要求。
示例性的，在确定上述第一对应关系后，可以基于参与第一对应关系中的人脸关键点构建用于调整第一原始人脸样本图像上人脸位置的仿射变换矩阵，并基于该仿射变换矩阵对第一原始人脸样本图像进行人脸位置调整，得到第一人脸样本图像；基于参与第一对应关系的动物脸关键点构建用于调整第一原始动物脸样本图像上动物脸位置的仿射变换矩阵，并基于该仿射变换矩阵对第一原始动物脸样本图像进行动物脸位置调整，得到第一动物脸样本图像。仿射变换矩阵的具体构建可以参考仿射变换原理。并且，仿射变换矩阵可以与第一原始人脸样本图像或者第一原始动物脸样本图像的缩放参数、裁剪比例等参数有关，即在进行人脸位置调整或者动物脸位置调整过程中，涉及的图像处理操作可以包括裁剪、缩放、旋转等，具体可以根据图像处理需求而定。
基于相同的关键点对应关系进行图像调整，最终得到的第一人脸样本图像和第一动物脸样本图像具有相同的图像尺寸，并且第一人脸样本图像上的人脸区域和第一动物脸样本图像上动物脸区域对应相同的图像位置，例如人脸区域位于第一人脸样本图像的中心区域，动物脸区域同样位于第一动物脸样本图像的中心区域等，此外人脸区域面积和动物脸区域面积的差值小于面积阈值(取值可以灵活设置)，即人脸区域面积和动物脸区域面积相匹配，从而可以确保基于动物脸生成模型生成具有较好展示效果的第一动物脸风格样本图像，进 而基于优质的训练样本训练得到动物脸风格图像生成模型，可以保证较好的模型训练效果，避免利用动物脸风格图像生成模型生成的动物脸风格图像上由于动物脸区域与人脸区域不匹配，影响动物脸风格图像的展示效果，例如相比于人脸区域，动物脸区域过大或者过小等。
同理，在训练得到动物脸生成模型之前，也可以首先确定第二原始人脸样本图像中的人脸关键点和第一原始动物脸样本图像中的动物脸关键点之间的第二对应关系；然后基于第二对应关系对第二原始人脸样本图像进行人脸位置调整，涉及的图像处理操作可以包括裁剪、缩放、旋转等，以得到符合图像生成模型的输入图像条件的第二人脸样本图像。当然，按照对第一动物脸样本图像的需求顺序，也可以基于该第二对应关系，预先对第一原始动物脸样本图像进行动物脸位置调整后，得到符合图像生成模型输入要求的第一动物脸样本图像。
示例性的，在确定上述第二对应关系后，也可以基于参与第二对应关系中的人脸关键点构建用于调整第二原始人脸样本图像上人脸位置调整的仿射变换矩阵，基于参与第二对应关系的动物脸关键点构建用于调整第一原始动物脸样本图像上动物脸位置的仿射变换矩阵。最终得到的第二人脸样本图像和第一动物脸样本图像具有相同的图像尺寸，并且第二人脸样本图像上的人脸区域和第一动物脸样本图像上动物脸区域对应相同的图像位置，例如人脸区域位于第二人脸样本图像的中心区域，动物脸区域同样位于第一动物脸样本图像的中心区域等，此外人脸区域面积和动物脸区域面积的差值小于面积阈值(取值可以灵活设置)，即人脸区域面积和动物脸区域面积相匹配，从而基于优质的训练样本，保证较好的模型训练效果。
可选的，动物脸风格图像生成模型基于第一人脸样本图像和第二动物脸风格样本图像训练得到，第二动物脸风格样本图像由第一动物脸风格样本图像中的背景区域替换为第一人脸样本图像中的背景区域后得到。通过背景替换，可以在训练得到动物脸风格图像生成模型的过程中，将动物脸风格样本图像上的背景区域对模型训练效果的影响降到最小，确保较好的模型训练效果，进而确保生成的动物脸风格图像具有较好的展示效果。
进一步的，第二动物脸风格样本图像是基于第二动物脸部蒙版图像，对第一动物脸风格样本图像和第一人脸样本图像进行融合后得到；第二动物脸部蒙版图像由预先训练动物脸部分割模型基于第一动物脸风格样本图像得到，第二动物脸部蒙版图像用于将第一动物脸风格样本图像上的动物脸区域确定为第二动物脸风格样本图像上的动物脸区域。动物脸部分割模型可以基于第二动物脸样本图像以及与第二动物脸样本图像上动物脸部区域的位置标注结果进行训练得到。在确保动物脸部分割模型具有生成与图像上动物脸区域对应的蒙版图像的功能基础上，本领域技术人员可以采用任意可用的训练方式实现，本公开实施例不作具体限定。
在本公开实施例中，可以在服务器中预先训练得到动物脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的动物脸风格图像，可以丰富终端中的图像编辑功能，以视频交互类应用程序为例，调用动物脸风格图像生成模型得到动物脸风格图像，不仅可以丰富应用程序的图像编辑功能，还能提升应用程序的趣味性，为用 户提供比较新颖的特效玩法，进而提高用户的使用体验。并且，采用动物脸风格图像生成模型，可以实现针对不同用户的原始人脸图像，动态生成与用户原始人脸图像相适应的动物脸风格图像，提高生成动物脸风格图像的智能化，并呈现较好的图像效果。
图2为本公开实施例提供的另一种动物脸风格图像生成方法的流程图，基于上述技术方案进一步优化与扩展，并可以与上述各个可选实施方式进行结合。
如图2所示，本公开实施例提供的动物脸风格图像生成方法可以包括：
S201、根据用户选择的动物特效类型，确定动物特效类型对应的动物脸关键点与人脸关键点之间的对应关系。
示例性的，当用户启用终端上具有风格图像生成功能的应用程序或者小程序之后，应用程序或者小程序可以向用户展示动物特征类型选择界面，动物特征类型可以按照不同的动物类型进行区分，例如猫脸特效或者狗脸特效，也可以按照不同的动物品种进行区分，例如狸花猫脸特效或者波斯猫脸特效等；终端根据用户选择的动物特效类型，确定用户当前需求生成与何种动物对应的动物脸风格图像，进而确定该动物的脸部关键点与人脸关键点之间的对应关系，这个对应关系可以预先存储在终端中，以供终端根据动物特效类型进行调用。当然，终端也可以在确定用户选择的动物特效类型对应的动物脸以及识别用户图像上的人脸关键点之后，建立动物脸关键点与人脸关键点之间的对应关系。用户图像可以是终端根据用户在终端中的图像选择操作、图像拍摄操作或图像上传操作所得到的图像。
S202、基于确定的对应关系对用户图像进行人脸位置调整，以得到原始人脸图像。
基于确定的动物脸关键点与人脸关键点之间的对应关系对用户图像进行人脸位置调整，以得到原始人脸图像。原始人脸图像符合动物脸风格图像生成模型输入要求。动物脸风格图像生成模型被训练完成后，模型对应的输入要求(例如图像上的人脸位置、图像尺寸等)也同时确定，因此，终端利用关键点识别技术识别用户图像上的人脸关键点后，基于确定的对应关系对用户图像进行人脸位置调整，例如，终端可以利用用户图像上属于该对应关系中的人脸关键点，构建用于调整用户图像上人脸位置的仿射变换矩阵，利用该仿射变换矩阵调整用户图像上的人脸位置，其中涉及的图像处理操作可以包括裁剪、缩放、旋转等，以得到符合动物脸风格图像生成模型的输入要求的原始人脸图像。
S203、获取原始人脸图像。
S204、利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像。
S205、将动物脸风格图像上的动物脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标动物脸风格图像。
用户图像上的背景区域指用户图像上除去人脸区域之外的剩余图像区域。示例性的，可以利用图像处理技术，从动物脸风格图像上提取出动物脸区域，从用户图像上提取出背景区域，然后按照用户图像上背景区域的位置和人脸区域的位置，将两者融合(或称为混合)。即最终展示给用户的目标动物脸风格图像上，除了用户脸部特征变为动物脸特征外，图像背景保留了用户图像上的背景区域，避免了在生成动物脸风格的图像过程中对用户图像上背景区域的改变。
可选的，将动物脸风格图像上的动物脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标动物脸风格图像，包括：
基于动物脸风格图像，得到与用户图像具有相同图像尺寸的中间结果图像；其中，中间结果图像上的动物脸区域位置与用户图像上的人脸区域位置相同；例如，可以按照动物脸风格图像上动物脸关键点和用户图像上人脸关键点的对应关系，将动物脸风格图像映射至用户图像对应的图像坐标，得到中间结果图像。
确定与动物特效类型对应的第一动物脸部蒙版图像。
基于第一动物脸部蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标动物脸风格图像；其中，第一动物脸部蒙版图像用于将中间结果图像上的动物脸区域确定为目标动物脸风格图像上的动物脸区域。
通过利用第一动物脸部蒙版图像实现用户图像与中间结果图像的融合，在确保成功得到目标动物脸风格图像的基础上，有助于提高图像融合处理的效率。
进一步的，基于第一动物脸部蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标动物脸风格图像，可以包括：
对第一动物脸部蒙版图像中的动物脸边界进行平滑处理，例如进行高斯模糊处理等；基于平滑处理后的动物脸部蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标动物脸风格图像。
通过对第一动物脸部蒙版图像中的动物脸边界进行平滑处理后，再参与图像融合，可以实现用户图像上背景区域与中间结果图像上动物脸区域之间的平滑过度，优化图像融合效果，确保目标动物脸风格图像的最终展示效果。
并且，在得到与用户图像对应的目标动物脸风格图像或者在得到与原始人脸图像对应的动物脸风格图像之后，还可以根据用户在图像编辑界面上的特效选择操作，确定用户选择的特效标识，将与用户选择的特效标识对应特效添加至前述目标动物脸风格图像或者前述动物脸风格图像，以进一步提升图像编辑的趣味性。用户可选择的特效可以包括任意类型的道具或贴纸等，本公开实施例不作具体限定。
在本公开实施例中，得到用户图像后，首先根据与用户选择的动物脸特效类型对应的动物脸关键点与人脸关键点之间的对应关系，对用户图像进行人脸位置调整，得到原始人脸图像，然后利用动物脸风格图像生成模型得到与原始人脸图像对应的动物脸风格图像，最后将动物脸风格图像上的动物脸区域与用户图像上的背景区域进行融合，得到展示给用户的目标动物脸风格图像，在将用户脸部特征进行动物化处理的同时，保留了用户图像上的原始背景，丰富了终端中的图像编辑功能。以视频交互类应用程序为例，调用动物脸风格图像生成模型得到动物脸风格图像，不仅丰富了应用程序的图像编辑功能，还提升了应用程序的趣味性，为用户提供了比较新颖的特效玩法，进而提高了用户的使用体验。
图3为本公开实施例提供的一种动物脸风格图像生成模型训练方法的流程图，应用于如何训练得到具有将人脸变换为动物脸功能的动物脸风格图像生成模型的情况。该动物脸风格图像生成模型训练方法可以由动物脸风格图像生成模型训练装置执行，该装置可以采用软件和/或硬件实现，并可集成在服务器中。
本公开实施例提供的动物脸风格图像生成模型训练方法与本公开实施例提供的动物脸风格图像生成方法配合执行，以下实施例中未详细解释的内容，可以参考上述实施例的描述。
如图3所示，本公开实施例提供的动物脸风格图像生成模型训练方法可以包括：
S301、基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型。
S302、基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像。
其中，第一动物脸风格样本图像是指将第一人脸样本图像上的人脸变换为动物脸后的图像。
S303、基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型。
其中，动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像。
可选的，在基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型之前，本公开实施例提供的模型训练方法还包括：
确定第二原始人脸样本图像中的人脸关键点和第一原始动物脸样本图像中的动物脸关键点之间的第二对应关系；基于第二对应关系对第二原始人脸样本图像进行人脸位置调整，以得到第二人脸样本图像；基于第二对应关系对第一原始动物脸样本图像进行动物脸位置调整，以得到第一动物脸样本图像。
在基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像之前，本公开实施例提供的模型训练方法还包括：确定第一原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系；基于第一对应关系对第一原始人脸样本图像进行动物脸位置调整，以得到第一人脸样本图像。
可选的，在基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像之后，本公开实施例提供的模型训练方法还包括：将第一动物脸风格样本图像中的背景区域替换为第一人脸样本图像中的背景区域，得到第二动物脸风格样本图像。
相应的，基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型，包括：基于第一人脸样本图像和第二动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型。
可选的，将第一动物脸风格样本图像中的背景区域替换为第一人脸样本图像中的背景区域，得到第二动物脸风格样本图像，包括：基于预先训练动物脸部分割模型，得到第一动物脸风格样本图像对应的动物脸部蒙版图像；基于动物脸部蒙版图像，将第一动物脸风格样本图像和第一人脸样本图像进行融合，得到第二动物脸风格样本图像；其中，动物脸部蒙版图像用于将第一动物脸风格样本图像上的动物脸区域确定为第二动物脸风格样本图像上的动物脸区域。
可选的，本公开实施例提供的模型训练方法还包括：获取第二动物脸样本图像以及与第二动物脸样本图像上动物脸部区域的位置标注结果；基于第二动物脸样本图像和动物脸 部区域的位置标注结果，训练得到动物脸部分割模型。
在本公开实施例中，可以在服务器中预先训练得到动物脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的动物脸风格图像，可以丰富终端中的图像编辑功能，以视频交互类应用程序为例，调用动物脸风格图像生成模型得到动物脸风格图像，不仅可以丰富应用程序的图像编辑功能，还能提升应用程序的趣味性，为用户提供比较新颖的特效玩法，进而提高用户的使用体验。
图4为本公开实施例提供的一种动物脸风格图像生成装置的结构示意图，应用于如何将用户人脸变换为动物脸的情况。该动物脸风格图像生成装置可以采用软件和/或硬件实现，并可集成在任意具有计算能力的电子设备上，例如智能手机、平板电脑、笔记本电脑等用户终端。
如图4所示，本公开实施例提供的动物脸风格图像生成装置400包括原始人脸图像获取模块401和风格图像生成模块402，其中：
原始人脸图像获取模块401，用于获取原始人脸图像；
风格图像生成模块402，用于利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像；
其中，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像，动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到。
可选的，第一动物脸风格样本图像由预先训练的动物脸生成模型基于第一人脸样本图像生成，动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
可选的，本公开实施例提供的装置400还包括：
对应关系确定模块，用于根据用户选择的动物特效类型，确定动物特效类型对应的动物脸关键点与人脸关键点之间的对应关系；
人脸位置调整模块，用于基于所述动物特效类型对应的动物脸关键点与人脸关键点之间的对应关系对用户图像进行人脸位置调整，以得到原始人脸图像；其中，原始人脸图像符合动物脸风格图像生成模型的输入要求；
可选的，图像融合模块，用于将动物脸风格图像上的动物脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标动物脸风格图像。
可选的，图像融合模块包括：
中间结果图像确定单元，用于基于动物脸风格图像，得到与用户图像具有相同图像尺寸的中间结果图像；其中，中间结果图像上的动物脸区域位置与用户图像上的人脸区域位置相同；
第一动物脸部蒙版图像确定单元，用于确定与动物特效类型对应的第一动物脸部蒙版图像；
图像融合单元，用于基于第一动物脸部蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标动物脸风格图像；其中，第一动物脸部蒙版图像用于将中间结果图像上的动物脸区域确定为目标动物脸风格图像上的动物脸区域。
可选的，第一人脸样本图像是基于第一原始人脸样本图像上的人脸关键点和第一原始 动物脸样本图像上的动物脸关键点之间的第一对应关系，对第一原始人脸样本图像进行人脸位置调整后得到；
第二人脸样本图像是基于第二原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第二对应关系，对第二原始人脸样本图像进行人脸位置调整后得到；
第一动物脸样本图像是基于第一对应关系或第二对应关系，对第一原始动物脸样本图像进行动物脸位置调整后得到。
可选的，动物脸风格图像生成模型基于第一人脸样本图像和第二动物脸风格样本图像训练得到，第二动物脸风格样本图像由第一动物脸风格样本图像中的背景区域替换为第一人脸样本图像中的背景区域后得到。
可选的，第二动物脸风格样本图像是基于第二动物脸部蒙版图像，对第一动物脸风格样本图像和第一人脸样本图像进行融合后得到；
第二动物脸部蒙版图像由预先训练动物脸部分割模型基于第一动物脸风格样本图像得到，第二动物脸部蒙版图像用于将第一动物脸风格样本图像上的动物脸区域确定为第二动物脸风格样本图像上的动物脸区域。
本公开实施例所提供的动物脸风格图像生成装置可执行本公开实施例所提供的任意动物脸风格图像生成方法，具备执行方法相应的功能模块和有益效果。本公开装置实施例中未详尽描述的内容可以参考本公开任意方法实施例中的描述。
图5为本公开实施例提供的一种动物脸风格图像生成模型训练装置的结构示意图，应用于如何训练得到具有将人脸变换为动物脸功能的动物脸风格图像生成模型的情况。该动物脸风格图像生成模型训练装置可以采用软件和/或硬件实现，并可集成在服务器中。
如图5所示，本公开实施例提供的动物脸风格图像生成模型训练装置500可以包括动物脸生成模型训练模块501、风格样本图像生成模块502和风格图像生成模型训练模块503，其中：
动物脸生成模型训练模块501，用于基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；
风格样本图像生成模块502，用于基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，第一动物脸风格样本图像是指将第一人脸样本图像上的人脸变换为动物脸后的图像；
风格图像生成模型训练模块503，用于基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；
其中，动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像。
可选的，本公开实施例提供的装置500还包括：
第二对应关系确定模块，用于确定第二原始人脸样本图像中的人脸关键点和第一原始动物脸样本图像中的动物脸关键点之间的第二对应关系；
人脸位置调整模块，用于基于第二对应关系对第二原始人脸样本图像进行人脸位置调 整，以得到第二人脸样本图像；
动物脸位置调整模块，用于基于第二对应关系对第一原始动物脸样本图像进行动物脸位置调整，以得到第一动物脸样本图像；
第一对应关系确定模块，用于确定第一原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系；
人脸位置调整模块，用于基于第一对应关系对第一原始人脸样本图像进行动物脸位置调整，以得到第一人脸样本图像。
可选的，本公开实施例提供的装置500还包括：
背景区域替换模块，用于将第一动物脸风格样本图像中的背景区域替换为第一人脸样本图像中的背景区域，得到第二动物脸风格样本图像；
可选的，风格图像生成模型训练模块503具体用于：
基于第一人脸样本图像和第二动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型。
可选的，背景区域替换模块包括：
动物脸部蒙版图像确定单元，用于基于预先训练动物脸部分割模型，得到第一动物脸风格样本图像对应的动物脸部蒙版图像；
图像融合单元，用于基于动物脸部蒙版图像，将第一动物脸风格样本图像和第一人脸样本图像进行融合，得到第二动物脸风格样本图像；其中，动物脸部蒙版图像用于将第一动物脸风格样本图像上的动物脸区域确定为第二动物脸风格样本图像上的动物脸区域。
可选的，本公开实施例提供的装置500还包括：
样本图像及标注结果获取模块，用于获取第二动物脸样本图像以及与第二动物脸样本图像上动物脸部区域的位置标注结果；
动物脸部分割模型训练模块，用于基于第二动物脸样本图像和动物脸部区域的位置标注结果，训练得到动物脸部分割模型。
本公开实施例所提供的动物脸风格图像生成模型训练装置可执行本公开实施例所提供的任意动物脸风格图像生成模型训练方法，具备执行方法相应的功能模块和有益效果。本公开装置实施例中未详尽描述的内容可以参考本公开任意方法实施例中的描述。
图6为本公开实施例提供的一种电子设备的结构示意图，用于对实现本公开实施例提供的动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法的电子设备进行示例性说明。本公开实施例中的电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机、服务器等等的固定终端。图6示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和占用范围带来任何限制。
如图6所示，电子设备600包括一个或多个处理器601和存储器602。
处理器601可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备600中的其他组件以执行期望的功能。
存储器602可以包括一个或多个计算机程序产品，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器601可以运行程序指令，以实现本公开实施例提供的动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法，还可以实现其他期望的功能。在计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
其中，动物脸风格图像生成方法可以包括：获取原始人脸图像；利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像；其中，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像，动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，第一动物脸风格样本图像由预先训练的动物脸生成模型基于第一人脸样本图像生成，动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
其中，动物脸风格图像生成模型训练方法可以包括：基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，第一动物脸风格样本图像是指将第一人脸样本图像上的人脸变换为动物脸后的图像；基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；其中，动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像。
应当理解，电子设备600还可以执行本公开方法实施例提供的其他可选实施方案。
在一个示例中，电子设备600还可以包括：输入装置603和输出装置604，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
此外，该输入装置603还可以包括例如键盘、鼠标等等。
该输出装置604可以向外部输出各种信息，包括确定出的距离信息、方向信息等。该输出装置604可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
当然，为了简化，图6中仅示出了该电子设备600中与本公开有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备600还可以包括任何其他适当的组件。
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本公开实施例所提供的动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法。
计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户电子设备上执行、部分地在用户电子设备上执行、作为一个独立的软件包执行、 部分在用户电子设备上且部分在远程电子设备上执行、或者完全在远程电子设备上执行。
此外，本公开实施例还可以提供一种计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本公开实施例所提供的动物脸风格图像生成方法或者动物脸风格图像生成模型训练方法。
其中，动物脸风格图像生成方法可以包括：获取原始人脸图像；利用预先训练的动物脸风格图像生成模型，得到与原始人脸图像对应的动物脸风格图像；其中，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像，动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，第一动物脸风格样本图像由预先训练的动物脸生成模型基于第一人脸样本图像生成，动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
其中，动物脸风格图像生成模型训练方法可以包括：基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；基于动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，第一动物脸风格样本图像是指将第一人脸样本图像上的人脸变换为动物脸后的图像；基于第一人脸样本图像和第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；其中，动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，动物脸风格图像是指将原始人脸图像上的人脸变换为动物脸后的图像。
应当理解，计算机程序指令在被处理器运行时，还可以使得处理器执行本公开方法实施例提供的其他可选实施方案。
计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范 围。

Claims (17)

  1. 一种动物脸风格图像生成方法，其特征在于，包括：
    获取原始人脸图像；
    利用预先训练的动物脸风格图像生成模型，得到与所述原始人脸图像对应的动物脸风格图像；
    其中，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像，所述动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，所述第一动物脸风格样本图像由预先训练的动物脸生成模型基于所述第一人脸样本图像生成，所述动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    根据用户选择的动物特效类型，确定所述动物特效类型对应的动物脸关键点与人脸关键点之间的对应关系；
    基于所述动物特效类型对应的动物脸关键点与人脸关键点之间的对应关系对用户图像进行人脸位置调整，以得到原始人脸图像；其中，所述原始人脸图像符合所述动物脸风格图像生成模型的输入要求；
  3. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    将所述动物脸风格图像上的动物脸区域与所述用户图像上的背景区域进行融合，得到与所述用户图像对应的目标动物脸风格图像。
  4. 根据权利要求3所述的方法，其特征在于，将所述动物脸风格图像上的动物脸区域与所述用户图像上的背景区域进行融合，得到与所述用户图像对应的目标动物脸风格图像，包括：
    基于所述动物脸风格图像，得到与所述用户图像具有相同图像尺寸的中间结果图像；其中，所述中间结果图像上的动物脸区域位置与所述用户图像上的人脸区域位置相同；
    确定与所述动物特效类型对应的第一动物脸部蒙版图像；
    基于所述第一动物脸部蒙版图像，将所述用户图像与所述中间结果图像进行融合，得到与所述用户图像对应的目标动物脸风格图像；其中，所述第一动物脸部蒙版图像用于将所述中间结果图像上的动物脸区域确定为所述目标动物脸风格图像上的动物脸区域。
  5. 根据权利要求1所述的方法，其特征在于：
    所述第一人脸样本图像是基于第一原始人脸样本图像上的人脸关键点和第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系，对所述第一原始人脸样本图像进行人脸位置调整后得到；
    所述第二人脸样本图像是基于第二原始人脸样本图像上的人脸关键点和所述第一原始动物脸样本图像上的动物脸关键点之间的第二对应关系，对所述第二原始人脸样本图像进行人脸位置调整后得到；
    所述第一动物脸样本图像是基于所述第一对应关系或所述第二对应关系，对所述第一 原始动物脸样本图像进行动物脸位置调整后得到。
  6. 根据权利要求1所述的方法，其特征在于：
    所述动物脸风格图像生成模型基于所述第一人脸样本图像和第二动物脸风格样本图像训练得到，所述第二动物脸风格样本图像由所述第一动物脸风格样本图像中的背景区域替换为所述第一人脸样本图像中的背景区域后得到。
  7. 根据权利要求6所述的方法，其特征在于：
    所述第二动物脸风格样本图像是基于第二动物脸部蒙版图像，对所述第一动物脸风格样本图像和所述第一人脸样本图像进行融合后得到；
    所述第二动物脸部蒙版图像由预先训练动物脸部分割模型基于所述第一动物脸风格样本图像得到，所述第二动物脸部蒙版图像用于将所述第一动物脸风格样本图像上的动物脸区域确定为所述第二动物脸风格样本图像上的动物脸区域。
  8. 一种动物脸风格图像生成模型训练方法，其特征在于，包括：
    基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；
    基于所述动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，所述第一动物脸风格样本图像是指将所述第一人脸样本图像上的人脸变换为动物脸后的图像；
    基于所述第一人脸样本图像和所述第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；
    其中，所述动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    确定第二原始人脸样本图像中的人脸关键点和第一原始动物脸样本图像中的动物脸关键点之间的第二对应关系；
    基于所述第二对应关系对所述第二原始人脸样本图像进行人脸位置调整，以得到所述第二人脸样本图像；基于所述第二对应关系对所述第一原始动物脸样本图像进行动物脸位置调整，以得到所述第一动物脸样本图像。
  10. 根据权利要求9所述的方法，其特征在于，所述方法还包括：
    确定第一原始人脸样本图像上的人脸关键点和所述第一原始动物脸样本图像上的动物脸关键点之间的第一对应关系；
    基于所述第一对应关系对所述第一原始人脸样本图像进行动物脸位置调整，以得到所述第一人脸样本图像。
  11. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    将所述第一动物脸风格样本图像中的背景区域替换为所述第一人脸样本图像中的背景区域，得到第二动物脸风格样本图像；
    所述基于所述第一人脸样本图像和所述第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型，包括：
    基于所述第一人脸样本图像和所述第二动物脸风格样本图像对所述风格图像生成模型进行训练，得到所述动物脸风格图像生成模型。
  12. 根据权利要求11所述的方法，其特征在于，所述将所述第一动物脸风格样本图像中的背景区域替换为所述第一人脸样本图像中的背景区域，得到第二动物脸风格样本图像，包括：
    基于预先训练动物脸部分割模型，得到所述第一动物脸风格样本图像对应的动物脸部蒙版图像；
    基于所述动物脸部蒙版图像，将所述第一动物脸风格样本图像和所述第一人脸样本图像进行融合，得到所述第二动物脸风格样本图像；其中，所述动物脸部蒙版图像用于将所述第一动物脸风格样本图像上的动物脸区域确定为所述第二动物脸风格样本图像上的动物脸区域。
  13. 根据权利要求12所述的方法，其特征在于，所述方法还包括：
    获取第二动物脸样本图像以及与所述第二动物脸样本图像上动物脸部区域的位置标注结果；
    基于所述第二动物脸样本图像和所述动物脸部区域的位置标注结果，训练得到所述动物脸部分割模型。
  14. 一种动物脸风格图像生成装置，其特征在于，包括：
    原始人脸图像获取模块，用于获取原始人脸图像；
    风格图像生成模块，用于利用预先训练的动物脸风格图像生成模型，得到与所述原始人脸图像对应的动物脸风格图像；
    其中，所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像，所述动物脸风格图像生成模型基于第一人脸样本图像和第一动物脸风格样本图像训练得到，所述第一动物脸风格样本图像由预先训练的动物脸生成模型基于所述第一人脸样本图像生成，所述动物脸生成模型基于第二人脸样本图像和第一动物脸样本图像训练得到。
  15. 一种动物脸风格图像生成模型训练装置，其特征在于，包括：
    动物脸生成模型训练模块，用于基于第二人脸样本图像和第一动物脸样本图像对图像生成模型进行训练，得到动物脸生成模型；
    风格样本图像生成模块，用于基于所述动物脸生成模型得到与第一人脸样本图像对应的第一动物脸风格样本图像；其中，所述第一动物脸风格样本图像是指将所述第一人脸样本图像上的人脸变换为动物脸后的图像；
    风格图像生成模型训练模块，用于基于所述第一人脸样本图像和所述第一动物脸风格样本图像对风格图像生成模型进行训练，得到动物脸风格图像生成模型；
    其中，所述动物脸风格图像生成模型用于得到与原始人脸图像对应的动物脸风格图像， 所述动物脸风格图像是指将所述原始人脸图像上的人脸变换为动物脸后的图像。
  16. 一种电子设备，其特征在于，包括存储器和处理器，其中：
    所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行权利要求1-7中任一项所述的动物脸风格图像生成方法，或者执行权利要求8-13中任一项所述的动物脸风格图像生成模型训练方法。
  17. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有计算机程序，当所述计算机程序被处理器执行时，所述处理器执行权利要求1-7中任一项所述的动物脸风格图像生成方法，或者执行权利要求8-13中任一项所述的动物脸风格图像生成模型训练方法。
PCT/CN2021/130301 2020-11-13 2021-11-12 动物脸风格图像生成方法、模型训练方法、装置和设备 WO2022100690A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/252,855 US20240005466A1 (en) 2020-11-13 2021-11-12 Animal face style image generation method and apparatus, model training method and apparatus, and device
JP2023528414A JP2023549810A (ja) 2020-11-13 2021-11-12 動物顔スタイル画像の生成方法、モデルのトレーニング方法、装置及び機器
EP21891210.3A EP4246425A4 (en) 2020-11-13 2021-11-12 ANIMAL HEAD STYLE IMAGE GENERATION METHOD AND APPARATUS, MODEL LEARNING METHOD AND APPARATUS, AND DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011269334.0 2020-11-13
CN202011269334.0A CN112330534A (zh) 2020-11-13 2020-11-13 动物脸风格图像生成方法、模型训练方法、装置和设备

Publications (1)

Publication Number Publication Date
WO2022100690A1 true WO2022100690A1 (zh) 2022-05-19

Family

ID=74318655

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/130301 WO2022100690A1 (zh) 2020-11-13 2021-11-12 动物脸风格图像生成方法、模型训练方法、装置和设备

Country Status (5)

Country Link
US (1) US20240005466A1 (zh)
EP (1) EP4246425A4 (zh)
JP (1) JP2023549810A (zh)
CN (1) CN112330534A (zh)
WO (1) WO2022100690A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112330534A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 动物脸风格图像生成方法、模型训练方法、装置和设备
CN113673422A (zh) * 2021-08-19 2021-11-19 苏州中科先进技术研究院有限公司 一种宠物种类识别方法及识别系统
CN113850890A (zh) * 2021-09-29 2021-12-28 北京字跳网络技术有限公司 动物形象的生成方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110930297A (zh) * 2019-11-20 2020-03-27 咪咕动漫有限公司 人脸图像的风格迁移方法、装置、电子设备及存储介质
CN111783647A (zh) * 2020-06-30 2020-10-16 北京百度网讯科技有限公司 人脸融合模型的训练方法、人脸融合方法、装置及设备
CN111968029A (zh) * 2020-08-19 2020-11-20 北京字节跳动网络技术有限公司 表情变换方法、装置、电子设备和计算机可读介质
CN112330534A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 动物脸风格图像生成方法、模型训练方法、装置和设备
CN112989904A (zh) * 2020-09-30 2021-06-18 北京字节跳动网络技术有限公司 风格图像生成方法、模型训练方法、装置、设备和介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108846793B (zh) * 2018-05-25 2022-04-22 深圳市商汤科技有限公司 基于图像风格转换模型的图像处理方法和终端设备
US10489683B1 (en) * 2018-12-17 2019-11-26 Bodygram, Inc. Methods and systems for automatic generation of massive training data sets from 3D models for training deep learning networks
CN109816589B (zh) * 2019-01-30 2020-07-17 北京字节跳动网络技术有限公司 用于生成漫画风格转换模型的方法和装置
CN109800732B (zh) * 2019-01-30 2021-01-15 北京字节跳动网络技术有限公司 用于生成漫画头像生成模型的方法和装置
CN111340865B (zh) * 2020-02-24 2023-04-07 北京百度网讯科技有限公司 用于生成图像的方法和装置
CN111833242A (zh) * 2020-07-17 2020-10-27 北京字节跳动网络技术有限公司 人脸变换方法、装置、电子设备和计算机可读介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110930297A (zh) * 2019-11-20 2020-03-27 咪咕动漫有限公司 人脸图像的风格迁移方法、装置、电子设备及存储介质
CN111783647A (zh) * 2020-06-30 2020-10-16 北京百度网讯科技有限公司 人脸融合模型的训练方法、人脸融合方法、装置及设备
CN111968029A (zh) * 2020-08-19 2020-11-20 北京字节跳动网络技术有限公司 表情变换方法、装置、电子设备和计算机可读介质
CN112989904A (zh) * 2020-09-30 2021-06-18 北京字节跳动网络技术有限公司 风格图像生成方法、模型训练方法、装置、设备和介质
CN112330534A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 动物脸风格图像生成方法、模型训练方法、装置和设备

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CHOI YUNJEY; UH YOUNGJUNG; YOO JAEJUN; HA JUNG-WOO: "StarGAN v2: Diverse Image Synthesis for Multiple Domains", 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE, 13 June 2020 (2020-06-13), pages 8185 - 8194, XP033803575, DOI: 10.1109/CVPR42600.2020.00821 *
LIANG ZI WEI: "Generating the Cat and Dog Versions of Trump, and Breaking the Face Editing Tool StarGANv2", 28 April 2020 (2020-04-28), XP055929385, Retrieved from the Internet <URL:https://ishare.ifeng.com/c/s/7w2keWci1Dq> *

Also Published As

Publication number Publication date
EP4246425A1 (en) 2023-09-20
EP4246425A4 (en) 2024-06-05
CN112330534A (zh) 2021-02-05
JP2023549810A (ja) 2023-11-29
US20240005466A1 (en) 2024-01-04

Similar Documents

Publication Publication Date Title
WO2022100690A1 (zh) 动物脸风格图像生成方法、模型训练方法、装置和设备
WO2022100680A1 (zh) 混血人脸图像生成方法、模型训练方法、装置和设备
CN105981368B (zh) 在成像装置中的照片构图和位置引导
WO2021082760A1 (zh) 虚拟形象的生成方法、装置、终端及存储介质
WO2022083383A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
WO2021109678A1 (zh) 视频生成方法、装置、电子设备及存储介质
WO2022166897A1 (zh) 脸型调整图像生成方法、模型训练方法、装置和设备
TWI255141B (en) Method and system for real-time interactive video
JP7209851B2 (ja) 画像変形の制御方法、装置およびハードウェア装置
WO2023051185A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN111400518A (zh) 作品生成和编辑方法、装置、终端、服务器和系统
US11367196B2 (en) Image processing method, apparatus, and storage medium
WO2019020061A1 (zh) 视频台词处理方法、客户端、服务器及存储介质
WO2019227429A1 (zh) 多媒体内容生成方法、装置和设备/终端/服务器
CN111612842A (zh) 生成位姿估计模型的方法和装置
US20240119082A1 (en) Method, apparatus, device, readable storage medium and product for media content processing
WO2021190625A1 (zh) 拍摄方法和设备
WO2022166908A1 (zh) 风格图像生成方法、模型训练方法、装置和设备
US20230057963A1 (en) Video playing method, apparatus and device, storage medium, and program product
WO2022237633A1 (zh) 一种图像处理方法、装置、设备及介质
WO2022170982A1 (zh) 图像处理方法、图像生成方法、装置、设备和介质
JP2023545052A (ja) 画像処理モデルの訓練方法及び装置、画像処理方法及び装置、電子機器並びにコンピュータプログラム
CN112785669B (zh) 一种虚拟形象合成方法、装置、设备及存储介质
WO2023246823A1 (zh) 一种视频播放方法、装置、设备及存储介质
US20230215296A1 (en) Method, computing device, and non-transitory computer-readable recording medium to translate audio of video into sign language through avatar

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21891210

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18252855

Country of ref document: US

Ref document number: 2023528414

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021891210

Country of ref document: EP

Effective date: 20230612