WO2022100680A1 - 混血人脸图像生成方法、模型训练方法、装置和设备 - Google Patents

混血人脸图像生成方法、模型训练方法、装置和设备 Download PDF

Info

Publication number
WO2022100680A1
WO2022100680A1 PCT/CN2021/130214 CN2021130214W WO2022100680A1 WO 2022100680 A1 WO2022100680 A1 WO 2022100680A1 CN 2021130214 W CN2021130214 W CN 2021130214W WO 2022100680 A1 WO2022100680 A1 WO 2022100680A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
image
mixed
race
degree parameter
Prior art date
Application number
PCT/CN2021/130214
Other languages
English (en)
French (fr)
Inventor
何茜
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2022100680A1 publication Critical patent/WO2022100680A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/02—Affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a method for generating a mixed-race face image, a method for training a model, an apparatus and a device.
  • Converting image styles refers to converting one or more images from one style to another.
  • the types of style transfer supported in current video interactive applications are still limited and less interesting, which in turn leads to poor user experience and is difficult to meet users' needs for personalized image style transfer.
  • the embodiments of the present disclosure provide a method for generating a mixed-race face image, a method for training a model, an apparatus and a device.
  • an embodiment of the present disclosure provides a method for generating a mixed-race face image, including:
  • the mixed-race face style image generation model is obtained by training based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the two face sample images belong to different groups of people respectively, the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter, and the mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter.
  • the mixed-race degree parameter, the mixed-race face image includes both the face features on the original face image and the face features on the face images that belong to different groups from the original face image.
  • an embodiment of the present disclosure also provides a method for training a mixed-race face-style image generation model, including:
  • a hybrid face style image generation model is obtained by training based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter;
  • the first face sample image and the second face sample image belong to different crowd classification attributes respectively
  • the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter
  • the The mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter
  • the mixed blood face style image generation model is used to obtain a mixed blood face image corresponding to the original face image, and the mixed blood face image simultaneously It includes the face features on the original face image and the face features on the face images classified by different groups of people from the original face image.
  • an embodiment of the present disclosure further provides an apparatus for generating a mixed-race face image, including:
  • the original face image acquisition module is used to obtain the original face image
  • a mixed-race face image generation module used for generating a model with a pre-trained mixed-race face style image to obtain a mixed-race face image corresponding to the original face image
  • the mixed-race face style image generation model is obtained by training based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the two face sample images belong to different groups of people respectively, the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter, and the mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter.
  • the mixed-race degree parameter, the mixed-race face image includes both the face features on the original face image and the face features on the face images that belong to different groups from the original face image.
  • an embodiment of the present disclosure further provides a hybrid face style image generation model training device, including:
  • a style image generation model determination module used for training a mixed-race face style image generation model based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter;
  • the first face sample image and the second face sample image belong to different crowd classification attributes respectively
  • the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter
  • the The mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter
  • the mixed blood face style image generation model is used to obtain a mixed blood face image corresponding to the original face image, and the mixed blood face image simultaneously It includes the face features on the original face image and the face features on the face images classified by different groups of people from the original face image.
  • embodiments of the present disclosure further provide an electronic device, including a memory and a processor, wherein: a computer program is stored in the memory, and when the computer program is executed by the processor, the processor Execute any method for generating a mixed-race face image or a method for training a model for generating a mixed-race face style image provided by the embodiments of the present disclosure.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the processor executes the computer program provided by the embodiment of the present disclosure. Either the hybrid face image generation method or the hybrid face style image generation model training method.
  • a mixed-race face style image generation model can be pre-trained in the server, and then sent to the terminal for use in the terminal.
  • the terminal calls and generates a mixed-race face image corresponding to the original face image, which can enrich the image editing function in the terminal.
  • calling the mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image can not only enrich the image editing function of the application, but also improve the video interactive application. It provides users with more novel special effects and gameplay, thereby improving the user's experience and meeting the user's personalized image style conversion needs.
  • the hybrid face style image generation model it is possible to dynamically generate a hybrid face image adapted to the original face image of the user according to the original face image of different users, so as to improve the intelligence of generating a hybrid face image, and It presents better image effects, such as getting a more realistic mixed-race face image.
  • FIG. 1 is a flowchart of a method for generating a mixed-race face image according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of another method for generating a mixed-race face image provided by an embodiment of the present disclosure
  • FIG. 3 is a flowchart of a method for training a hybrid face style image generation model according to an embodiment of the present disclosure
  • FIG. 4 is a flowchart of another method for training a mixed-race face style image generation model provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of an apparatus for generating a mixed-race face image according to an embodiment of the present disclosure
  • FIG. 6 is a schematic structural diagram of an apparatus for training a model for generating a mixed-race face style image according to an embodiment of the present disclosure
  • FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a method for generating a mixed-race face image according to an embodiment of the present disclosure.
  • the method for generating a mixed-race face image can be performed by a mixed-race face image generating device, which can be implemented by software and/or hardware, and can be integrated on any electronic device with computing capabilities, such as a smart phone, a tablet computer, a notebook computer and other terminals.
  • the mixed-race face image generation device can be implemented in the form of an independent application program or a small program integrated on a public platform, and can also be implemented as an application program with a style image generation function or a functional module integrated in the applet, which has a style image generation function.
  • Functional applications or applets may include, but are not limited to, video interactive applications or video interactive applets, and the like.
  • the method for generating a mixed-race face image can be applied to a scene in which a mixed-race face image is obtained.
  • the mixed-race face image refers to a face image that simultaneously includes face features classified by different groups of people. From a biological point of view, the descendants of human races with certain genetic differences can be called hybrids. Crowd classification can be distinguished in terms of skin color, face shape, facial features, hair color and hair shape. For example, a face image that includes both the facial features of the first skin color and the facial features of the second skin color belongs to a mixed-race face image, and includes both the facial features of the first facial features and the facial features of the second facial features.
  • the face image with facial features belongs to a kind of mixed-race face image. Usually, there are certain genetic differences between the races of different countries. Based on this, the mixed-race face image may also be a face image that includes both the facial features of the race of country A and the facial features of the race of country B.
  • the expression of the original face shown in the original face image and the expression of the mixed-race face shown in the mixed-race face image can be kept consistent , for example, a primitive face shows a smiling expression, and the corresponding mixed-race face also shows a smiling expression; the facial features on the primitive face and the facial features on the hybrid face can also be consistent, such as the eyes on the primitive face.
  • the eyes of the corresponding mixed-race person are also in the state of open eyes, etc.
  • the method for generating a mixed-race face image may include:
  • the image stored in the terminal may be acquired or an image or video may be captured in real time by an image capturing device of the terminal.
  • the mixed-race face image generating device acquires the original face image to be processed according to the user's image selection operation, image capturing operation or image uploading operation in the terminal.
  • a photo-taking prompt can be displayed on the image capture interface.
  • the photographing prompt information can be used to prompt the user to place the face of the face image in the image acquisition interface at a preset position on the terminal screen (for example, the middle position of the screen, etc.), adjust the distance between the face and the terminal screen (adjust the The distance can be used to obtain the appropriate size of the face area in the image acquisition interface, to avoid the face area being too large or too small, etc.) and adjust the rotation angle of the face (different rotation angles correspond to different face orientations, such as frontal or sideways.
  • the user takes an image according to the photographing prompt information, so that the video interactive application can conveniently obtain the original face image that meets the input requirements of the mixed-race face-style image generation model.
  • the input requirements of the mixed-race face style image generation model may refer to constraints on the input image, such as the position of the face on the input image, the size of the input image, and the like.
  • the video interaction application can also pre-store a photo template according to the input requirements of the mixed-race face-style image generation model, and the photo template pre-defines the position of the user's face on the image, the size of the face area on the image, and the size of the face. According to the user's photographing operation, the video interactive application can use the photographing template to obtain the required original face image.
  • the images taken by the user can be cropped, scaled, rotated, etc., to obtain the original image that meets the input requirements of the mixed-race face-style image generation model. face image.
  • the mixed-race face style image generation model has the function of generating a mixed-race face image
  • the mixed-race face image refers to a face image with a mixed-race style.
  • the mixed-race face image includes both the face features on the original face image and the face features on the face images that belong to different groups from the original face image.
  • the mixed-race face style image generation model is trained based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the specific implementation process of model training is not specifically limited in the embodiments of the present disclosure.
  • those skilled in the art can adopt any available training methods. As well as the flexibility to choose the available model structures.
  • the first face sample image and the second face sample image belong to different groups of people, for example, the first face sample image belongs to the first group category, the second face sample image belongs to the second group category, and the first face sample image belongs to the second group category.
  • the mixed blood degree parameter corresponding to the sample image is the minimum mixed blood degree parameter, and the minimum mixed blood degree parameter can take a value of 0; the mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter, and the maximum mixed blood degree parameter can take a value of 1.
  • the mixed blood degree parameter used can be any value between the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the mixed blood degree parameter corresponding to the mixed blood face image obtained by using the mixed blood face style image generation model can also be any value between the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the mixed blood degree parameter corresponding to the mixed blood face image is closer.
  • the minimum mixed blood degree parameter means that the face features on the generated mixed blood face image are closer to those on the original face image; the mixed blood degree parameter corresponding to the mixed blood face image is closer to the maximum mixed blood degree parameter, which means The face features on the generated mixed-race face image are closer to the face features on the face images classified by different groups of the original face image, or it means that the face features on the generated mixed-race face image are similar to the original face image. The farther away the facial features on the face image are.
  • the mixed-race degree parameter corresponding to the mixed-race face image obtained by using the mixed-race face style image generation model is related to the mixed-race degree parameter preset in the training process of the mixed-race face style image generation model. That is, in the embodiment of the present disclosure, different mixed-race degree parameters can be preset, and a plurality of mixed-race face style image generation models can be obtained by corresponding training, and then sent to the terminal, so that the user can determine the required image according to the mixed-race face image generation requirements.
  • the mixed-race level parameter so that the terminal calls the corresponding mixed-race face style image generation model according to the mixed-race level parameter selected by the user, generates a mixed-race face image for the user, and displays it.
  • the hybrid face style image generation model is obtained by training based on the third face sample image and the hybrid face sample image, and the hybrid degree parameter corresponding to the third face sample image is the minimum hybrid degree. parameter; the mixed-race face sample image is generated by the pre-trained face generation model based on the third face sample image and the first preset mixed-race degree parameter.
  • the face generation model is obtained by training based on the first face sample image, the second face sample image and the second preset mixing degree parameter; wherein, the first face sample image and the third face sample image may be the same face
  • the sample images may also be different face sample images, which are not specifically limited in this embodiment of the present disclosure.
  • the minimum mixed blood degree parameter is the lower limit of the first preset mixed blood degree parameter and the second preset mixed blood degree parameter
  • the maximum mixed blood degree parameter is the upper limit of the first preset mixed blood degree parameter and the second preset mixed blood degree parameter.
  • the specific values of the first preset mixed blood degree parameter and the second preset mixed blood degree parameter can be set adaptively during the model training process.
  • the above-mentioned model training process may include: first, using a second preset mixed blood degree parameter (for example, a value of 0 or 1) as a constraint parameter of the image generation model, based on the first face sample image and the second The face sample images are used to train the image generation model to obtain a face generation model.
  • a second preset mixed blood degree parameter for example, a value of 0 or 1
  • the available image generation models may include but are not limited to generative adversarial network (GAN, Generative Adversarial Networks) models, style-based generative adversarial network (Stylegan, Style-Based Generator Architecture for Generative Adversarial Networks) models, etc.; then, the The first preset mixed-race degree parameter (for example, the value is 0.5) is used as a constraint parameter of the face generation model, and a mixed-race face sample image corresponding to the third face sample image is obtained based on the face generation model, that is, the third face sample.
  • GAN Generative Adversarial Networks
  • style-based generative adversarial network Stylegan, Style-Based Generator Architecture for Generative Adversarial Networks
  • the image and the corresponding mixed-race face sample image can be used as paired training data, and the paired training data is used in the subsequent training process of the style image generation model; finally, based on the third face sample image and the mixed-race face sample image , train the style image generation model to obtain a mixed-race face style image generation model, wherein the style image generation models that can be used can include such as Conditional Generative Adversarial Networks (CGAN, Conditional Generative Adversarial Networks) model, cyclic consistency generative adversarial network (Cycle-GAN, Cycle Consistent Adversarial Networks) model, etc.
  • CGAN Conditional Generative Adversarial Networks
  • Cycle-GAN Cycle Consistent Adversarial Networks
  • the loss function of the face generation model is related to the similarity between the input image and the output image during the training of the face generation model, and the similarity is used to control the input image and the output image during the training of the face generation model. correlation between.
  • the similarity between the input image and the output image of the image generation model can be calculated as a constraint parameter for the training of the image generation model.
  • the loss function of the image generation model is also the loss function of the face generation model.
  • the process of obtaining the face generation model also includes: determining the input image of the image generation model
  • the similarity between the output image (such as the first face sample image) and the image generation model, the similarity calculation can be implemented by any available image similarity calculation method, such as cosine similarity calculation, etc.; add the similarity to
  • the loss function of the image generation model the implementation form of the loss function is not specifically limited, so as to control the correlation between the input image of the image generation model and the output image of the image generation model according to the loss function after adding the similarity.
  • a mixed-race face style image generation model can be pre-trained in the server, and then sent to the terminal for the terminal to call and generate a mixed-race face image corresponding to the original face image, which can enrich the terminal image editing function in .
  • calling the mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image can not only enrich the image editing function of the application, but also improve the video interactive application. It provides users with more novel special effects gameplay, thereby improving the user's experience and meeting the user's needs for personalized image style conversion.
  • the hybrid face style image generation model it is possible to dynamically generate a hybrid face image adapted to the original face image of the user according to the original face image of different users, so as to improve the intelligence of generating a hybrid face image, and It presents better image effects, such as getting a more realistic mixed-race face image.
  • FIG. 2 is a flowchart of another method for generating a mixed-race face image provided by an embodiment of the present disclosure, which is further optimized and expanded based on the above-mentioned technical solution, and can be combined with each of the above-mentioned optional embodiments.
  • the method for generating a mixed-race face image may include:
  • the original face image meets the input requirements of the hybrid face style image generation model.
  • the user image may be an image obtained by the device for generating a mixed-race face image according to an image selection operation, an image capturing operation, or an image uploading operation performed by the user in the terminal.
  • the mixed-race face image generating device uses the face key point recognition technology to identify the face key points on the user image, and adjusts the face position on the user image based on the face key points, so as to obtain a mixed-race face style image.
  • Generate the original face image required by the model input, and the involved image processing operations can include cropping, scaling, rotation, etc.
  • face key points can be used to obtain a transformation matrix for adjusting the position of the face on the user image based on the principle of affine transformation. The transformation matrix adjusts the face position on the user image to get the desired original face image.
  • the background area on the user image refers to the remaining image area on the user image except the face area.
  • image processing technology can be used to extract a face region from a mixed-race face image, and a background region can be extracted from the user image, and then according to the position of the background region and the position of the face region on the user image, the two Fusion (or blending).
  • the background area of the target mixed-race image retains the background area on the user's image, which avoids the background area on the user's image in the process of generating the target mixed-race image. change.
  • the face area on the mixed-race face image is fused with the background area on the user image to obtain a target mixed-race image corresponding to the user image, including:
  • an intermediate result image with the same image size as the user image is obtained; wherein, the position of the face region on the intermediate result image is the same as the position of the face region on the user image;
  • the corresponding relationship between the face key points and the face key points on the user image, the mixed-race face image is mapped to the image coordinates corresponding to the user image, and the intermediate result image is obtained.
  • the user image and the intermediate result image are fused to obtain the target mixed blood image corresponding to the user image; wherein the preset face mask image is used to determine the face region on the intermediate result image It is the face area on the target mixed-blood image, or the preset face mask image is used to determine the face area on the user image as an invalid face area in the image fusion process, that is, it is necessary to use the person on the intermediate result image.
  • the face area is used as the effective face area.
  • the specific size of the preset face mask image can be set as required.
  • the user image and the intermediate result image are fused to obtain a target mixed-blood image corresponding to the user image, which may include:
  • the smooth transition between the background area on the user image and the face area on the intermediate result image can be achieved, and the image fusion effect can be optimized. Ensure the final presentation of the target hybrid image.
  • the face position of the user image is first adjusted to obtain the original face image, and then the mixed-race face image corresponding to the original face image is obtained by using the mixed-race face style image generation model, Finally, the face area on the mixed-race face image is fused with the background area on the user image to obtain the target mixed-race image displayed to the user. While mixing the user's facial features, the original background on the user's image is preserved.
  • the image editing functions in the terminal are enriched. Taking a video interactive application as an example, calling the mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image not only enriches the image editing function of the application, but also improves the video interactive application. It provides users with more novel special effects and gameplay, thereby improving the user's experience and meeting the user's needs for personalized image style conversion.
  • the special effect identifier selected by the user can also be determined according to the special effect selection operation by the user on the image editing interface, which will be shared with the user.
  • the special effect corresponding to the selected special effect identifier is added to the target mixed-race image or mixed-race face image to further enhance the fun of image editing.
  • the special effects selectable by the user may include any type of props or stickers, etc., which are not specifically limited in this embodiment of the present disclosure.
  • FIG. 3 is a flowchart of a method for training a mixed-race face style image generation model according to an embodiment of the present disclosure.
  • the model training method can be performed by a hybrid face style image generation model training device, which can be implemented by software and/or hardware, and can be integrated in a server.
  • the mixed-race face style image generation model training method provided by the embodiment of the present disclosure can be applied to a scene in which a mixed-race face style image generation model is obtained by training.
  • the model training method includes the model training method shown in FIG. 3 and FIG.
  • the method for generating a face image is performed in coordination, and for the content not described in detail in the following embodiments, reference may be made to the explanations in the above-mentioned embodiments.
  • the method for training a mixed-race face style image generation model may include:
  • a hybrid face style image generation model is obtained by training based on the first face sample image, the second face sample image, the minimum hybrid degree parameter and the maximum hybrid degree parameter.
  • the first face sample image and the second face sample image belong to different crowd classification attributes respectively
  • the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter
  • the mixed blood degree corresponding to the second face sample image The parameter is the maximum hybrid degree parameter.
  • the hybrid face style image generation model is used to obtain a hybrid face image corresponding to the original face image.
  • the hybrid face image includes both the face features on the original face image and the original face image. Facial features on face images classified by different groups of people.
  • model training is not specifically limited in the embodiments of the present disclosure.
  • those skilled in the art can adopt any available training methods. As well as the flexibility to choose the available model structures.
  • FIG. 4 is a flowchart of another method for training a mixed-race face style image generation model provided by an embodiment of the present disclosure, but the specific limitations of the embodiment of the present disclosure should not be understood.
  • the method for training a mixed-race face style image generation model provided by an embodiment of the present disclosure may include:
  • the process of training the image generation model based on the first face sample image and the second face sample image to obtain the face generation model it also includes:
  • the mixed blood degree parameter corresponding to the third face sample image is the minimum mixed blood degree parameter.
  • a mixed-race face style image generation model can be pre-trained in the server, and then sent to the terminal for the terminal to call and generate a mixed-race face image corresponding to the original face image, which can enrich the terminal image editing function in .
  • calling the mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image can not only enrich the image editing function of the application, but also improve the video interactive application. It provides users with more novel special effects gameplay, thereby improving the user's experience and meeting the user's needs for personalized image style conversion.
  • FIG. 5 is a schematic structural diagram of an apparatus for generating a mixed-race face image provided by an embodiment of the present disclosure, which is applied to the situation of how to obtain a mixed-race face image.
  • the generating apparatus can be implemented by software and/or hardware, and can be integrated in any On electronic devices with computing power, such as smart phones, tablet computers, notebook computers and other user terminals.
  • the mixed-race face image generating apparatus 500 may include an original face image obtaining module 501 and a mixed-race face image generating module 502, wherein:
  • an original face image acquisition module 501 configured to acquire an original face image
  • the mixed-race face image generation module 502 is configured to use a pre-trained mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image;
  • the mixed-race face style image generation model is trained based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the first face sample image and the second face sample image belong to For different population classifications, the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter, and the mixed blood degree parameter corresponding to the second face sample image is the maximum mixed blood degree parameter.
  • the mixed blood face image also includes the original face image. face features and face features on face images classified by different groups from the original face image.
  • the hybrid face style image generation model is obtained by training based on the third face sample image and the hybrid face sample image, and the hybrid degree parameter corresponding to the third human face sample image is the minimum hybrid degree parameter;
  • the mixed-race face sample image is generated by a pre-trained face generation model based on the third face sample image and the first preset mixed-race degree parameter;
  • the face generation model is obtained by training based on the first face sample image, the second face sample image and the second preset mixing degree parameter;
  • the minimum mixed blood degree parameter is the lower limit of the first preset mixed blood degree parameter and the second preset mixed blood degree parameter
  • the maximum mixed blood degree parameter is the upper limit of the first preset mixed blood degree parameter and the second preset mixed blood degree parameter.
  • the loss function of the face generation model is related to the similarity between the input image and the output image during the training of the face generation model, and the similarity is used to control the input image and the output image during the training of the face generation model. correlation between.
  • the apparatus 500 for generating a mixed-race face image further includes:
  • the face position adjustment module is used to identify the face key points on the user image, and adjust the face position on the user image based on the face key points to obtain the original face image; wherein, the original face image
  • the face image meets the input requirements required by the mixed-race face style image generation model.
  • the apparatus 500 for generating a mixed-race face image further includes:
  • the target mixed-race image generation module is used to fuse the face area on the mixed-race face image with the background area on the user image to obtain the target mixed-race image corresponding to the user image.
  • the target mixed blood image generation module includes:
  • the intermediate result image determining unit is used to obtain the intermediate result image with the same image size as the user image based on the mixed-race face image; Wherein, the human face region position on the intermediate result image is identical with the human face region position on the user image;
  • the image fusion unit is used to fuse the user image and the intermediate result image based on the preset face mask image to obtain a target mixed-blood image corresponding to the user image; wherein the preset face mask image is used to fuse the intermediate result image
  • the face area on the target hybrid image is determined as the face area on the target hybrid image.
  • the mixed-race face image generating apparatus provided by the embodiment of the present disclosure can execute any mixed-race face image generating method provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • any mixed-race face image generating method provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 6 is a schematic structural diagram of a mixed-race face-style image generation model training apparatus provided by an embodiment of the present disclosure, which is applied to the situation of how to train a mixed-race face-style image generation model with a mixed-race face image generation function.
  • the training device can be implemented in software and/or hardware, and can be integrated in a server.
  • the mixed-race face style image generation model training apparatus 600 may include:
  • the style image generation model determination module 601 is used to obtain a mixed-race face style image generation model based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter;
  • the first face sample image and the second face sample image belong to different crowd classification attributes
  • the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter
  • the mixed blood degree parameter corresponding to the second face sample image is the maximum hybrid degree parameter.
  • the hybrid face style image generation model is used to obtain the hybrid face image corresponding to the original face image.
  • the hybrid face image includes both the face features on the original face image and the attribution of the original face image. Face features on face images classified by different groups of people.
  • the style image generation model determination module 601 includes:
  • the face generation model training unit is used to use the second preset mixed blood degree parameter as a constraint parameter of the image generation model, and train the image generation model based on the first face sample image and the second face sample image to obtain the face generation model.
  • Model
  • a mixed-race face sample image generation unit configured to use the first preset mixed-race degree parameter as a constraint parameter of the face generation model, and obtain a mixed-race face sample image corresponding to the third face sample image based on the face generation model; wherein, The mixed blood degree parameter corresponding to the third face sample image is the minimum mixed blood degree parameter;
  • the style image generation model training unit is used for training the style image generation model based on the third face sample image and the mixed-race face sample image to obtain the mixed-race face style image generation model.
  • the style image generation model determination module 601 further includes:
  • a similarity determination unit configured to determine the similarity between the input image of the image generation model and the output image of the image generation model; the input image of the image generation model is the first face sample image;
  • a loss function determination unit configured to add the determined similarity to the loss function of the image generation model, so as to control the correlation between the input image of the image generation model and the output image of the image generation model according to the loss function after adding the similarity sex.
  • the mixed-race face style image generation model training device provided by the embodiment of the present disclosure can execute any mixed-race face style image generation model training method provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • any mixed-race face style image generation model training method provided by the embodiment of the present disclosure has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, which is used to exemplarily illustrate an electronic device that implements the method for generating a mixed-race face image or a method for training a model for generating a mixed-race face style image provided by the embodiment of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, Mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, servers, and the like.
  • the electronic device shown in FIG. 7 is only an example, and should not impose any limitation on the functions and occupancy scope of the embodiments of the present disclosure.
  • electronic device 700 includes one or more processors 701 and memory 702 .
  • Processor 701 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 700 to perform desired functions.
  • CPU central processing unit
  • Processor 701 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 700 to perform desired functions.
  • Memory 702 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Volatile memory may include, for example, random access memory (RAM) and/or cache memory, among others.
  • Non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 701 may execute the program instructions to implement the method for generating a mixed-race face image or the method for training a model for generating a mixed-race face style image provided by the embodiments of the present disclosure, Other desired functions can also be implemented.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the method for generating a mixed-race face image may include: obtaining an original face image; using a pre-trained mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image; wherein, generating a mixed-race face style image
  • the model is trained based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the first face sample image and the second face sample image belong to different groups of people.
  • the hybridization degree parameter corresponding to the face sample image is the minimum hybridization degree parameter
  • the hybridization degree parameter corresponding to the second face sample image is the maximum hybridization degree parameter.
  • the face images belong to the face features on the face images classified by different groups of people.
  • the method for training a hybrid face style image generation model may include: training a hybrid face style image generation model based on the first face sample image, the second face sample image, the minimum hybrid degree parameter and the maximum hybrid degree parameter; wherein, The first face sample image and the second face sample image belong to different crowd classification attributes respectively, the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter, and the mixed blood degree parameter corresponding to the second face sample image is The maximum hybrid degree parameter.
  • the hybrid face style image generation model is used to obtain a hybrid face image corresponding to the original face image.
  • the hybrid face image includes both the face features on the original face image and the attribution different from the original face image. Face features on face images for crowd classification.
  • the electronic device 700 may also perform other optional implementations provided by the method embodiments of the present disclosure.
  • the electronic device 700 may also include an input device 703 and an output device 704 interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the input device 703 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 704 can output various information to the outside, including the determined distance information, direction information, and the like.
  • the output device 704 may include, for example, displays, speakers, printers, and communication networks and their connected remote output devices, among others.
  • the electronic device 700 may also include any other suitable components according to the specific application.
  • the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when executed by the processor, cause the processor to perform the hybrid face image generation provided by the embodiments of the present disclosure method or a method for training a hybrid face-style image generation model.
  • the computer program product may write program code for performing operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., as well as conventional procedural programming language, such as "C" language or similar programming language.
  • the program code may execute entirely on the user electronic device, partly on the user electronic device, as a stand-alone software package, partly on the user electronic device and partly on the remote electronic device, or entirely on the remote electronic device execute on.
  • embodiments of the present disclosure may further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor may perform the generation of the mixed-race face image provided by the embodiments of the present disclosure. method or a method for training a hybrid face-style image generation model.
  • the method for generating a mixed-race face image may include: obtaining an original face image; using a pre-trained mixed-race face style image generation model to obtain a mixed-race face image corresponding to the original face image; wherein, generating the mixed-race face style image
  • the model is trained based on the first face sample image, the second face sample image, the minimum mixed blood degree parameter and the maximum mixed blood degree parameter.
  • the first face sample image and the second face sample image belong to different groups of people.
  • the hybridization degree parameter corresponding to the face sample image is the minimum hybridization degree parameter
  • the hybridization degree parameter corresponding to the second face sample image is the maximum hybridization degree parameter.
  • the face images belong to the face features on the face images classified by different groups of people.
  • the method for training a hybrid face style image generation model may include: training a hybrid face style image generation model based on the first face sample image, the second face sample image, the minimum hybrid degree parameter and the maximum hybrid degree parameter; wherein, The first face sample image and the second face sample image belong to different crowd classification attributes respectively, the mixed blood degree parameter corresponding to the first face sample image is the minimum mixed blood degree parameter, and the mixed blood degree parameter corresponding to the second face sample image is The maximum hybridization degree parameter.
  • the hybrid face style image generation model is used to obtain a hybrid face image corresponding to the original face image.
  • the hybrid face image includes both the face features on the original face image and the attribution different from the original face image. Face features on face images for crowd classification.
  • the processor may also cause the processor to execute other optional implementations provided by the method embodiments of the present disclosure.
  • a computer-readable storage medium can employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种混血人脸图像生成方法、模型训练方法、装置和设备，其中，该图像生成方法可以包括：获取原始人脸图像（S101）；利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像（S102）；其中，混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。该方法可以丰富终端中的图像编辑功能，提升视频交互类应用程序的趣味性，为用户提供比较新颖的特效玩法。

Description

混血人脸图像生成方法、模型训练方法、装置和设备
本申请要求于2020年11月13日提交国家知识产权局、申请号为202011269305.4、申请名称为“混血人脸图像生成方法、模型训练方法、装置和设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其涉及一种混血人脸图像生成方法、模型训练方法、装置和设备。
背景技术
随着图像处理技术的发展，视频交互类应用程序的功能逐渐丰富化，转换图像风格成为了一种新的趣味性玩法。转换图像风格是指将一幅或者多幅图像由一种风格转换为另一种风格。然而，目前的视频交互类应用程序中支持的风格转换类型仍然有限、趣味性较差，进而导致用户体验较差，难以满足用户的个性化图像风格转换需求。
发明内容
为了解决上述技术问题或者至少部分地解决上述技术问题，本公开实施例提供了一种混血人脸图像生成方法、模型训练方法、装置和设备。
第一方面，本公开实施例提供了一种混血人脸图像生成方法，包括：
获取原始人脸图像；
利用预先训练的混血人脸风格图像生成模型，得到与所述原始人脸图像对应的混血人脸图像；
其中，所述混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，所述第一人脸样本图像和所述第二人脸样本图像分别归属不同的人群分类，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
第二方面，本公开实施例还提供了一种混血人脸风格图像生成模型训练方法，包括：
基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；
其中，所述第一人脸样本图像和所述第二人脸样本图像分别归属不同的人群分类属性，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
第三方面，本公开实施例还提供了一种混血人脸图像生成装置，包括：
原始人脸图像获取模块，用于获取原始人脸图像；
混血人脸图像生成模块，用于利用预先训练的混血人脸风格图像生成模型，得到与所述原始人脸图像对应的混血人脸图像；
其中，所述混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，所述第一人脸样本图像和所述第二人脸样本图像分别归属不同的人群分类，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
第四方面，本公开实施例还提供了一种混血人脸风格图像生成模型训练装置，包括：
风格图像生成模型确定模块，用于基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；
其中，所述第一人脸样本图像和所述第二人脸样本图像分别归属不同的人群分类属性，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
第五方面，本公开实施例还提供了一种电子设备，包括存储器和处理器，其中：所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行本公开实施例提供的任一混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法。
第六方面，本公开实施例还提供了一种计算机可读存储介质，所述存储介质中存储有计算机程序，当所述计算机程序被处理器执行时，所述处理器执行本公开实施例提供的任一混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法。
本公开实施例提供的技术方案与现有技术相比至少具有如下优点：在本公开实施例中，可以在服务器中预先训练得到混血人脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的混血人脸图像，可以丰富终端中的图像编辑功能。以视频交互类应用程序为例，调用该混血人脸风格图像生成模型得到与原始人脸图像对应的混血人脸图像，不仅可以丰富应用程序的图像编辑功能，还能提升该视频交互类应用程序的趣味性，为用户提供更加新颖的特效玩法，进而提高用户的使用体验，满足用户的个性化图像风格转换需求。并且，采用该混血人脸风格图像生成模型，可以实现针对不同用户的原始人脸图像，动态生成与用户原始人脸图像相适应的混血人脸图像，提高生成混血人脸图像的智能化，并呈现较好的图像效果，如得到更加真实的混血人脸图像。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种混血人脸图像生成方法的流程图；
图2为本公开实施例提供的另一种混血人脸图像生成方法的流程图；
图3为本公开实施例提供的一种混血人脸风格图像生成模型训练方法的流程图；
图4为本公开实施例提供的另一种混血人脸风格图像生成模型训练方法的流程图；
图5为本公开实施例提供的一种混血人脸图像生成装置的结构示意图；
图6为本公开实施例提供的一种混血人脸风格图像生成模型训练装置的结构示意图；
图7为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面将对本公开的方案进行进一步描述。需要说明的是，在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本公开，但本公开还可以采用其他不同于在此描述的方式来实施；显然，说明书中的实施例只是本公开的一部分实施例，而不是全部的实施例。
图1为本公开实施例提供的一种混血人脸图像生成方法的流程图。该混血人脸图像生成方法可以由混血人脸图像生成装置执行，该生成装置可以采用软件和/或硬件实现，并可集成在任意具有计算能力的电子设备上，例如智能手机、平板电脑、笔记本电脑等终端。
混血人脸图像生成装置可以采用独立的应用程序或者公众平台上集成的小程序的形式实现，还可以作为具有风格图像生成功能的应用程序或者小程序中集成的功能模块实现，该具有风格图像生成功能的应用程序或者小程序可以包括但不限于视频交互类应用程序或者视频交互类小程序等。
本公开实施例提供的混血人脸图像生成方法可以应用于获得混血人脸图像的场景。在本公开实施例中，混血人脸图像指代同时包括不同人群分类的人脸特征的人脸图像。从生物学角度而言，存在一定基因差异的人种的后代可以称为混血。人群分类可以从肤色、脸型、五官、头发颜色和头发形态等方面进行区分。例如，同时包括第一肤色的人脸特征和第二肤色的人脸特征的人脸图像属于一种混血人脸图像，同时包括第一五官特点的人脸特征和第二五官特点的人脸特征的人脸图像属于一种混血人脸图像。通常，不同国家的人种存在一定基因差异，基于此，混血人脸图像还可以是同时包括A国人种的人脸特征和B国人种的人脸特征的人脸图像。
在本公开实施例中，将原始人脸图像变换为混血人脸图像之后，原始人脸图像中所示的原始人脸的表情与混血人脸图像中所示的混血人脸的表情可以保持一致，例如原始人脸上呈现微笑的表情，对应的混血人脸上也呈现微笑表情；原始人脸上的五官 状态与混血人脸上的五官状态也可以保持一致，例如原始人脸上的眼部处于睁眼状态，对应的混血人脸上的眼部也处于睁眼状态等。
如图1所示，本公开实施例提供的混血人脸图像生成方法可以包括：
S101、获取原始人脸图像。
示例性的，当用户存在生成混血人脸图像的需求时，可以获取存储在终端中的图像或者通过终端的图像拍摄装置实时拍摄图像或者视频。混血人脸图像生成装置根据用户在终端中的图像选择操作、图像拍摄操作或图像上传操作，获取待处理的原始人脸图像。
以用户通过在视频交互类应用程序中调用终端的图像拍摄装置(例如摄像头)进行实时拍摄图像为例，该视频交互类应用程序跳转到图像采集界面后，可以在图像采集界面上展示拍照提示信息，该拍照提示信息可以用于提示用户将图像采集界面中人脸图像的脸部置于终端屏幕上的预设位置(例如屏幕中间位置等)、调整脸部距离终端屏幕的距离(调整该距离可以在图像采集界面中得到合适尺寸的脸部区域，避免脸部区域过大或者过小等)以及调整脸部的旋转角度(不同的旋转角度对应不同的脸部朝向，例如正脸或者侧脸等)等信息中的至少一种；用户根据拍照提示信息，进行拍摄图像，从而使得视频交互类应用程序可以便捷得到符合混血人脸风格图像生成模型输入要求的原始人脸图像。其中，混血人脸风格图像生成模型输入要求可以是指对输入图像的限制条件，例如输入图像上的人脸位置、输入图像的尺寸等。进一步的，视频交互类应用程序还可以根据混血人脸风格图像生成模型输入要求，预先存储拍照模板，该拍照模板预先定义了用户脸部在图像上的位置、图像上脸部区域的大小、脸部角度、图像尺寸等信息，视频交互类应用程序可以根据用户的拍照操作，利用该拍照模板获得所需的原始人脸图像。
当然，当用户拍摄的图像与混血人脸风格图像生成模型输入要求存在差异时，可以对用户拍摄的图像进行裁剪、缩放、旋转等处理，以得到符合混血人脸风格图像生成模型输入要求的原始人脸图像。
S102、利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像。
混血人脸风格图像生成模型具有生成混血人脸图像的功能，混血人脸图像指具有混血风格的人脸图像。例如混血人脸图像同时包括原始人脸图像上的人脸特征以及与 该原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到。关于模型训练的具体实现过程本公开实施例不作具体限定，在确保最终训练得到的混血人脸风格图像生成模型具有生成混血人脸图像功能的基础上，本领域技术人员可以采用任意可用的训练方式以及灵活选择可用的模型结构。其中，第一人脸样本图像和第二人脸样本图像归属不同的人群分类，例如第一人脸样本图像归属第一人群分类，第二人脸样本图像归属第二人群分类，第一人脸样本图像对应的混血程度参数为最小混血程度参数，最小混血程度参数可以取值为0，第二人脸样本图像对应的混血程度参数为最大混血程度参数，最大混血程度参数可以取值为1。在模型训练过程中，采用的混血程度参数可以是最小混血程度参数和最大混血程度参数之间的任意值。
利用混血人脸风格图像生成模型得到的混血人脸图像对应的混血程度参数，也可以是最小混血程度参数和最大混血程度参数之间的任意值，该混血人脸图像对应的混血程度参数越接近最小混血程度参数，意味着生成的混血人脸图像上的人脸特征与原始人脸图像上的人脸特征越接近；该混血人脸图像对应的混血程度参数越接近最大混血程度参数，意味着生成的混血人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征越接近，或者意味着生成的混血人脸图像上的人脸特征与原始人脸图像上的人脸特征越远离。
并且，利用混血人脸风格图像生成模型得到的混血人脸图像对应的混血程度参数与混血人脸风格图像生成模型训练过程中预设的混血程度参数有关。即在本公开实施例中，可以预先设置不同的混血程度参数，对应训练得到多个混血人脸风格图像生成模型，然后下发至终端，以供用户根据混血人脸图像生成需求，确定所需的混血程度参数，进而使得终端根据用户选择的混血程度参数调用相应的混血人脸风格图像生成模型，为用户生成混血人脸图像，并进行展示。
在上述技术方案的基础上，可选的，混血人脸风格图像生成模型基于第三人脸样本图像和混血人脸样本图像训练得到，第三人脸样本图像对应的混血程度参数为最小混血程度参数；混血人脸样本图像由预先训练的人脸生成模型基于第三人脸样本图像和第一预设混血程度参数生成。
人脸生成模型基于第一人脸样本图像、第二人脸样本图像和第二预设混血程度参 数训练得到；其中，第一人脸样本图像和第三人脸样本图像可以是相同的人脸样本图像，也可以是不同的人脸样本图像，本公开实施例不作具体限定。
最小混血程度参数为第一预设混血程度参数和第二预设混血程度参数的取值下限，最大混血程度参数为第一预设混血程度参数和第二预设混血程度参数的取值上限。第一预设混血程度参数和第二预设混血程度参数的具体取值，在模型训练过程中，均可以适应性设置。
示例性的，上述涉及的模型训练过程可以包括：首先，将第二预设混血程度参数(例如取值为0或者1)作为图像生成模型的约束参数，基于第一人脸样本图像和第二人脸样本图像对图像生成模型进行训练，得到人脸生成模型。其中，可以利用的图像生成模型可以包括但不限于生成对抗网络(GAN，Generative Adversarial Networks)模型、基于样式的生成对抗网络(Stylegan，Style-Based Generator Architecture for Generative Adversarial Networks)模型等；然后，将第一预设混血程度参数(例如取值为0.5)作为人脸生成模型的约束参数，基于人脸生成模型得到与第三人脸样本图像对应的混血人脸样本图像，即第三人脸样本图像和对应的混血人脸样本图像可以作为成对的训练数据，该成对的训练数据用于后续的风格图像生成模型训练过程中；最后，基于第三人脸样本图像和混血人脸样本图像，对风格图像生成模型进行训练，得到混血人脸风格图像生成模型，其中，可以利用的风格图像生成模型可以包括诸如条件生成对抗网络(CGAN，Conditional Generative Adversarial Networks)模型、循环一致性生成对抗网络(Cycle-GAN，Cycle Consistent Adversarial Networks)模型等。
可选的，人脸生成模型的损失函数与人脸生成模型训练过程中的输入图像和输出图像之间的相似度有关，相似度用于控制人脸生成模型训练过程中的输入图像和输出图像之间的相关性。为了优化人脸生成模型的模型效果，在训练得到人脸生成模型的过程中，可以计算图像生成模型的输入图像和输出图像之间的相似度，作为图像生成模型训练的一种约束参数。以基于图像生成模型训练得到人脸生成模型为例，图像生成模型的损失函数也即人脸生成模型的损失函数，在得到人脸生成模型的过程中，还包括：确定图像生成模型的输入图像(例如第一人脸样本图像)和图像生成模型的输出图像之间的相似度，该相似度计算可以采用任意可用的图像相似度计算方式实现，例如余弦相似度计算等；将相似度添加至图像生成模型的损失函数中，该损失函数的实现形式不具体限定，以根据添加相似度之后的损失函数，控制图像生成模型的输入 图像和图像生成模型的输出图像之间的相关性。
在本公开实施例中，可以在服务器中预先训练得到混血人脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的混血人脸图像，可以丰富终端中的图像编辑功能。以视频交互类应用程序为例，调用该混血人脸风格图像生成模型得到与原始人脸图像对应的混血人脸图像，不仅可以丰富应用程序的图像编辑功能，还能提升该视频交互类应用程序的趣味性，为用户提供更加新颖的特效玩法，进而提高用户的使用体验，满足用户对个性化图像风格进行转换的需求。并且，采用该混血人脸风格图像生成模型，可以实现针对不同用户的原始人脸图像，动态生成与用户原始人脸图像相适应的混血人脸图像，提高生成混血人脸图像的智能化，并呈现较好的图像效果，如得到更加真实的混血人脸图像。
图2为本公开实施例提供的另一种混血人脸图像生成方法的流程图，基于上述技术方案进一步优化与扩展，并可以与上述各个可选实施方式进行结合。
如图2所示，本公开实施例提供的混血人脸图像生成方法可以包括：
S201、识别用户图像上的人脸关键点，并基于人脸关键点对用户图像上的人脸位置进行调整，以得到原始人脸图像。
其中，原始人脸图像符合混血人脸风格图像生成模型的输入要求。用户图像可以是混血人脸图像生成装置根据用户在终端中的图像选择操作、图像拍摄操作或图像上传操作所得到的图像。然后，混血人脸图像生成装置利用人脸关键点识别技术，识别用户图像上的人脸关键点，并基于该人脸关键点调整用户图像上的人脸位置，以得到符合混血人脸风格图像生成模型输入要求的原始人脸图像，其中涉及的图像处理操作可以包括裁剪、缩放、旋转等。示例性的，可以利用人脸关键点，基于仿射变换原理，得到用于调整用户图像上人脸位置的变换矩阵，该变换矩阵可以与用户图像的缩放参数、裁剪比例等参数有关，然后基于变换矩阵调整用户图像上的人脸位置，以得到所需的原始人脸图像。
S202、获取原始人脸图像。
S203、利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像。
S204、将混血人脸图像上的人脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标混血图像。
用户图像上的背景区域指用户图像上除去人脸区域之外的剩余图像区域。示例性的，可以利用图像处理技术，从混血人脸图像上提取出人脸区域，从用户图像上提取出背景区域，然后按照用户图像上背景区域的位置和人脸区域的位置，将两者融合(或称为混合)。最终展示给用户的目标混血图像上，除了用户脸部特征发生变化外，该目标混血图像的背景区域保留了用户图像上的背景区域，避免了在生成目标混血图像过程中对用户图像上背景区域的改变。
可选的，将混血人脸图像上的人脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标混血图像，包括：
基于混血人脸图像，得到与用户图像具有相同图像尺寸的中间结果图像；其中，中间结果图像上的人脸区域位置与用户图像上的人脸区域位置相同；例如，可以按照混血人脸图像上人脸关键点和用户图像上人脸关键点的对应关系，将混血人脸图像映射至用户图像对应的图像坐标，得到中间结果图像。
基于预设人脸蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标混血图像；其中，预设人脸蒙版图像用于将中间结果图像上的人脸区域确定为目标混血图像上的人脸区域，或者，预设人脸蒙版图像用于将用户图像上的人脸区域确定为图像融合过程中的无效人脸区域，即需要使用中间结果图像上的人脸区域作为有效人脸区域。
预设人脸蒙版图像的具体尺寸可以根据需求进行设置。通过利用预设人脸蒙版图像实现用户图像与中间结果图像的融合，在确保成功得到目标混血图像的基础上，有助于提高图像融合的效率。
进一步的，基于预设人脸蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标混血图像，可以包括：
对预设人脸蒙版图像中的人脸边界进行平滑处理，例如进行高斯模糊处理等；基于平滑处理后的人脸蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标混血图像。
通过对预设人脸蒙版图像中的人脸边界进行平滑处理后，再参与图像融合，可以实现用户图像上背景区域与中间结果图像上人脸区域之间的平滑过度，优化图像融合效果，确保目标混血图像的最终展示效果。
在本公开实施例中，得到用户图像后，首先对用户图像进行人脸位置调整，得到 原始人脸图像，然后利用混血人脸风格图像生成模型得到与原始人脸图像对应的混血人脸图像，最后将混血人脸图像上的人脸区域与用户图像上的背景区域进行融合，得到展示给用户的目标混血图像，在将用户脸部特征混血化的同时，保留了用户图像上的原始背景，丰富了终端中的图像编辑功能。以视频交互类应用程序为例，调用该混血人脸风格图像生成模型得到与原始人脸图像对应的混血人脸图像，不仅丰富了应用程序的图像编辑功能，还提升了该视频交互类应用程序的趣味性，为用户提供了更加新颖的特效玩法，进而提高了用户的使用体验，满足用户对个性化图像风格进行转换的需求。
在得到与用户图像对应的目标混血图像或者在得到与原始人脸图像对应的混血人脸图像之后，还可以根据用户在图像编辑界面上的特效选择操作，确定用户选择的特效标识，将与用户选择的特效标识对应的特效添加至目标混血图像或混血人脸图像，以进一步提升图像编辑的趣味性。用户可选择的特效可以包括任意类型的道具或贴纸等，本公开实施例不作具体限定。
图3为本公开实施例提供的一种混血人脸风格图像生成模型训练方法的流程图。该模型训练方法可以由混血人脸风格图像生成模型训练装置执行，该装置可以采用软件和/或硬件实现，并可集成在服务器中。
本公开实施例提供的混血人脸风格图像生成模型训练方法可以应用于训练得到混血人脸风格图像生成模型的场景，该模型训练方法包括图3和图4所展示的模型训练方法，与上述混血人脸图像生成方法配合执行，以下实施例中未详细描述的内容，可以参考上述实施例中的解释。
如图3所示，本公开实施例提供的混血人脸风格图像生成模型训练方法可以包括：
S301、基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型。
其中，第一人脸样本图像和第二人脸样本图像分别归属不同的人群分类属性，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
关于模型训练的具体实现过程本公开实施例不作具体限定，在确保最终训练得到的混血人脸风格图像生成模型具有生成混血人脸图像功能的基础上，本领域技术人员 可以采用任意可用的训练方式以及灵活选择可用的模型结构。
作为对上述模型训练方案的进一步优化与扩展，图4为本公开实施例提供的另一种混血人脸风格图像生成模型训练方法的流程图，但不应理解对本公开实施例的具体限定。如图4所示，本公开实施例提供的混血人脸风格图像生成模型训练方法可以包括：
S401、将第二预设混血程度参数作为图像生成模型的约束参数，基于第一人脸样本图像和第二人脸样本图像对图像生成模型进行训练，得到人脸生成模型。
可选的，在基于第一人脸样本图像和第二人脸样本图像对图像生成模型进行训练，得到人脸生成模型的过程中，还包括：
确定图像生成模型的输入图像(例如第一人脸样本图像)和图像生成模型的输出图像之间的相似度；将相似度添加至图像生成模型的损失函数中，以根据添加相似度之后的损失函数，控制图像生成模型的输入图像和图像生成模型的输出图像之间的相关性。
S402、将第一预设混血程度参数作为人脸生成模型的约束参数，基于人脸生成模型得到与第三人脸样本图像对应的混血人脸样本图像。
其中，第三人脸样本图像对应的混血程度参数为最小混血程度参数。
S403、基于第三人脸样本图像和混血人脸样本图像，对风格图像生成模型进行训练，得到混血人脸风格图像生成模型。
在本公开实施例中，可以在服务器中预先训练得到混血人脸风格图像生成模型，然后下发至终端中，以供终端调用并生成与原始人脸图像对应的混血人脸图像，可以丰富终端中的图像编辑功能。以视频交互类应用程序为例，调用该混血人脸风格图像生成模型得到与原始人脸图像对应的混血人脸图像，不仅可以丰富应用程序的图像编辑功能，还能提升该视频交互类应用程序的趣味性，为用户提供更加新颖的特效玩法，进而提高用户的使用体验，满足用户对个性化图像风格进行转换的需求。
图5为本公开实施例提供的一种混血人脸图像生成装置的结构示意图，应用于如何得到混血人脸图像的情况，该生成装置可以采用软件和/或硬件实现，并可集成在任意具有计算能力的电子设备上，例如智能手机、平板电脑、笔记本电脑等用户终端。
如图5所示，本公开实施例提供的混血人脸图像生成装置500可以包括原始人脸图像获取模块501和混血人脸图像生成模块502，其中：
原始人脸图像获取模块501，用于获取原始人脸图像；
混血人脸图像生成模块502，用于利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像；
其中，混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，第一人脸样本图像和第二人脸样本图像归属不同的人群分类，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
可选的，混血人脸风格图像生成模型基于第三人脸样本图像和混血人脸样本图像训练得到，第三人脸样本图像对应的混血程度参数为最小混血程度参数；
混血人脸样本图像由预先训练的人脸生成模型基于第三人脸样本图像和第一预设混血程度参数生成；
人脸生成模型基于第一人脸样本图像、第二人脸样本图像和第二预设混血程度参数训练得到；
最小混血程度参数为第一预设混血程度参数和第二预设混血程度参数的取值下限，最大混血程度参数为第一预设混血程度参数和第二预设混血程度参数的取值上限。
可选的，人脸生成模型的损失函数与人脸生成模型训练过程中的输入图像和输出图像之间的相似度有关，相似度用于控制人脸生成模型训练过程中的输入图像和输出图像之间的相关性。
可选的，本公开实施例提供的混血人脸图像生成装置500还包括：
人脸位置调整模块，用于识别用户图像上的人脸关键点，基于所述人脸关键点对所述用户图像上的人脸位置进行调整，以得到原始人脸图像；其中，所述原始人脸图像符合所述混血人脸风格图像生成模型要求的输入要求。
可选的，本公开实施例提供的混血人脸图像生成装置500还包括：
目标混血图像生成模块，用于将混血人脸图像上的人脸区域与用户图像上的背景区域进行融合，得到与用户图像对应的目标混血图像。
可选的，目标混血图像生成模块包括：
中间结果图像确定单元，用于基于混血人脸图像，得到与用户图像具有相同图像 尺寸的中间结果图像；其中，中间结果图像上的人脸区域位置与用户图像上的人脸区域位置相同；
图像融合单元，用于基于预设人脸蒙版图像，将用户图像与中间结果图像进行融合，得到与用户图像对应的目标混血图像；其中，预设人脸蒙版图像用于将中间结果图像上的人脸区域确定为目标混血图像上的人脸区域。
本公开实施例所提供的混血人脸图像生成装置可执行本公开实施例所提供的任意混血人脸图像生成方法，具备执行方法相应的功能模块和有益效果。本公开装置实施例中未详尽描述的内容可以参考本公开任意方法实施例中的描述。
图6为本公开实施例提供的一种混血人脸风格图像生成模型训练装置的结构示意图，应用于如何训练得到具有混血人脸图像生成功能的混血人脸风格图像生成模型的情况。该训练装置可以采用软件和/或硬件实现，并可集成在服务器中。
如图6所示，本公开实施例提供的混血人脸风格图像生成模型训练装置600可以包括：
风格图像生成模型确定模块601，用于基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；
其中，第一人脸样本图像和第二人脸样本图像归属不同的人群分类属性，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
可选的，风格图像生成模型确定模块601包括：
人脸生成模型训练单元，用于将第二预设混血程度参数作为图像生成模型的约束参数，基于第一人脸样本图像和第二人脸样本图像对图像生成模型进行训练，得到人脸生成模型；
混血人脸样本图像生成单元，用于将第一预设混血程度参数作为人脸生成模型的约束参数，基于人脸生成模型得到与第三人脸样本图像对应的混血人脸样本图像；其中，第三人脸样本图像对应的混血程度参数为最小混血程度参数；
风格图像生成模型训练单元，用于基于第三人脸样本图像和混血人脸样本图像，对风格图像生成模型进行训练，得到混血人脸风格图像生成模型。
可选的，风格图像生成模型确定模块601还包括：
相似度确定单元，用于确定所述图像生成模型的输入图像和所述图像生成模型的输出图像之间的相似度；所述图像生成模型的输入图像为所述第一人脸样本图像；
损失函数确定单元，用于将确定的相似度添加至图像生成模型的损失函数中，以根据添加相似度之后的损失函数，控制图像生成模型的输入图像和图像生成模型的输出图像之间的相关性。
本公开实施例所提供的混血人脸风格图像生成模型训练装置可执行本公开实施例所提供的任意混血人脸风格图像生成模型训练方法，具备执行方法相应的功能模块和有益效果。本公开装置实施例中未详尽描述的内容可以参考本公开任意方法实施例中的描述。
图7为本公开实施例提供的一种电子设备的结构示意图，用于对实现本公开实施例提供的混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法的电子设备进行示例性说明。本公开实施例中的电子设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机、服务器等等的固定终端。图7示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和占用范围带来任何限制。
如图7所示，电子设备700包括一个或多个处理器701和存储器702。
处理器701可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备700中的其他组件以执行期望的功能。
存储器702可以包括一个或多个计算机程序产品，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器701可以运行程序指令，以实现本公开实施例提供的混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法，还可以实现其他期望的功能。在计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
其中，混血人脸图像生成方法可以包括：获取原始人脸图像；利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像；其中，混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，第一人脸样本图像和第二人脸样本图像分别归属不同的人群分类，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
其中，混血人脸风格图像生成模型训练方法可以包括：基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；其中，第一人脸样本图像和第二人脸样本图像分别归属不同的人群分类属性，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
应当理解，电子设备700还可以执行本公开方法实施例提供的其他可选实施方案。
在一个示例中，电子设备700还可以包括：输入装置703和输出装置704，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
此外，该输入装置703还可以包括例如键盘、鼠标等等。
该输出装置704可以向外部输出各种信息，包括确定出的距离信息、方向信息等。该输出装置704可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
当然，为了简化，图7中仅示出了该电子设备700中与本公开有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备700还可以包括任何其他适当的组件。
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本公开实施例所提供的混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法。
计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，程序设计语言包括面向对象的程序设计语言，诸如Java、 C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户电子设备上执行、部分地在用户电子设备上执行、作为一个独立的软件包执行、部分在用户电子设备上且部分在远程电子设备上执行、或者完全在远程电子设备上执行。
此外，本公开实施例还可以提供一种计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本公开实施例所提供的混血人脸图像生成方法或者混血人脸风格图像生成模型训练方法。
其中，混血人脸图像生成方法可以包括：获取原始人脸图像；利用预先训练的混血人脸风格图像生成模型，得到与原始人脸图像对应的混血人脸图像；其中，混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，第一人脸样本图像和第二人脸样本图像分别归属不同的人群分类，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
其中，混血人脸风格图像生成模型训练方法可以包括：基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；其中，第一人脸样本图像和第二人脸样本图像分别归属不同的人群分类属性，第一人脸样本图像对应的混血程度参数为最小混血程度参数，第二人脸样本图像对应的混血程度参数为最大混血程度参数，混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，混血人脸图像同时包括原始人脸图像上的人脸特征和与原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
应当理解，计算机程序指令在被处理器运行时，还可以使得处理器执行本公开方法实施例提供的其他可选实施方案。
计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或 者上述的任意合适的组合。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (13)

  1. 一种混血人脸图像生成方法，其特征在于，包括：
    获取原始人脸图像；
    利用预先训练的混血人脸风格图像生成模型，得到与所述原始人脸图像对应的混血人脸图像；
    其中，所述混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，所述第一人脸样本图像和所述第二人脸样本图像归属不同的人群分类，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
  2. 根据权利要求1所述的方法，其特征在于：
    所述混血人脸风格图像生成模型基于第三人脸样本图像和混血人脸样本图像训练得到，所述第三人脸样本图像对应的混血程度参数为所述最小混血程度参数；
    所述混血人脸样本图像由预先训练的人脸生成模型基于所述第三人脸样本图像和第一预设混血程度参数生成；
    所述人脸生成模型基于所述第一人脸样本图像、所述第二人脸样本图像和第二预设混血程度参数训练得到；
    所述最小混血程度参数为所述第一预设混血程度参数和所述第二预设混血程度参数的取值下限，所述最大混血程度参数为所述第一预设混血程度参数和所述第二预设混血程度参数的取值上限。
  3. 根据权利要求2所述的方法，其特征在于，所述人脸生成模型的损失函数与所述人脸生成模型训练过程中的输入图像和输出图像之间的相似度有关，所述相似度用于控制所述人脸生成模型训练过程中的输入图像和输出图像之间的相关性。
  4. 根据权利要求1所述的方法，其特征在于，在所述获取原始人脸图像之前，还包括：
    识别用户图像上的人脸关键点，基于所述人脸关键点对所述用户图像上的人脸位置进行调整，以得到原始人脸图像；其中，所述原始人脸图像符合所述混血人脸风格图像生成模型要求的输入要求。
  5. 根据权利要求4所述的方法，其特征在于，在所述利用预先训练的混血人脸风格图像生成模型，得到与所述原始人脸图像对应的混血人脸图像之后，还包括：
    将所述混血人脸图像上的人脸区域与所述用户图像上的背景区域进行融合，得到与所述用户图像对应的目标混血图像。
  6. 根据权利要求5所述的方法，其特征在于，将所述混血人脸图像上的人脸区域与所述用户图像上的背景区域进行融合，得到与所述用户图像对应的目标混血图像，包括：
    基于所述混血人脸图像，得到与所述用户图像具有相同图像尺寸的中间结果图像；其 中，所述中间结果图像上的人脸区域位置与所述用户图像上的人脸区域位置相同；
    基于预设人脸蒙版图像，将所述用户图像与所述中间结果图像进行融合，得到与所述用户图像对应的目标混血图像；其中，所述预设人脸蒙版图像用于将所述中间结果图像上的人脸区域确定为所述目标混血图像上的人脸区域。
  7. 一种混血人脸风格图像生成模型训练方法，其特征在于，包括：
    基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；
    其中，所述第一人脸样本图像和所述第二人脸样本图像归属不同的人群分类属性，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
  8. 根据权利要求7所述的方法，其特征在于，所述基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型，包括：
    将第二预设混血程度参数作为图像生成模型的约束参数，基于所述第一人脸样本图像和所述第二人脸样本图像对所述图像生成模型进行训练，得到人脸生成模型；
    将第一预设混血程度参数作为所述人脸生成模型的约束参数，基于所述人脸生成模型得到与第三人脸样本图像对应的混血人脸样本图像；其中，所述第三人脸样本图像对应的混血程度参数为所述最小混血程度参数；
    基于所述第三人脸样本图像和所述混血人脸样本图像，对风格图像生成模型进行训练，得到所述混血人脸风格图像生成模型。
  9. 根据权利要求8所述的方法，其特征在于，在所述基于第一人脸样本图像和第二人脸样本图像对所述图像生成模型进行训练，得到人脸生成模型的过程中，还包括：
    确定所述图像生成模型的输入图像和所述图像生成模型的输出图像之间的相似度；所述图像生成模型的输入图像为所述第一人脸样本图像；
    将所述相似度添加至所述图像生成模型的损失函数中，以根据添加所述相似度之后的损失函数，控制所述图像生成模型的输入图像和所述图像生成模型的输出图像之间的相关性。
  10. 一种混血人脸图像生成装置，其特征在于，包括：
    原始人脸图像获取模块，用于获取原始人脸图像；
    混血人脸图像生成模块，用于利用预先训练的混血人脸风格图像生成模型，得到与所述原始人脸图像对应的混血人脸图像；
    其中，所述混血人脸风格图像生成模型基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到，所述第一人脸样本图像和所述第二人脸 样本图像归属不同的人群分类，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
  11. 一种混血人脸风格图像生成模型训练装置，其特征在于，包括：
    风格图像生成模型确定模块，用于基于第一人脸样本图像、第二人脸样本图像、最小混血程度参数和最大混血程度参数训练得到混血人脸风格图像生成模型；
    其中，所述第一人脸样本图像和所述第二人脸样本图像归属不同的人群分类属性，所述第一人脸样本图像对应的混血程度参数为所述最小混血程度参数，所述第二人脸样本图像对应的混血程度参数为所述最大混血程度参数，所述混血人脸风格图像生成模型用于得到与原始人脸图像对应的混血人脸图像，所述混血人脸图像同时包括所述原始人脸图像上的人脸特征和与所述原始人脸图像归属不同人群分类的人脸图像上的人脸特征。
  12. 一种电子设备，其特征在于，包括存储器和处理器，其中：
    所述存储器中存储有计算机程序，当所述计算机程序被所述处理器执行时，所述处理器执行权利要求1-6中任一项所述的混血人脸图像生成方法，或者执行权利要求7-9中任一项所述的混血人脸风格图像生成模型训练方法。
  13. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有计算机程序，当所述计算机程序被处理器执行时，所述处理器执行权利要求1-6中任一项所述的混血人脸图像生成方法，或者执行权利要求7-9中任一项所述的混血人脸风格图像生成模型训练方法。
PCT/CN2021/130214 2020-11-13 2021-11-12 混血人脸图像生成方法、模型训练方法、装置和设备 WO2022100680A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011269305.4 2020-11-13
CN202011269305.4A CN112330533A (zh) 2020-11-13 2020-11-13 混血人脸图像生成方法、模型训练方法、装置和设备

Publications (1)

Publication Number Publication Date
WO2022100680A1 true WO2022100680A1 (zh) 2022-05-19

Family

ID=74317522

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/130214 WO2022100680A1 (zh) 2020-11-13 2021-11-12 混血人脸图像生成方法、模型训练方法、装置和设备

Country Status (2)

Country Link
CN (1) CN112330533A (zh)
WO (1) WO2022100680A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112330533A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 混血人脸图像生成方法、模型训练方法、装置和设备
CN112837213A (zh) * 2021-02-07 2021-05-25 北京字跳网络技术有限公司 脸型调整图像生成方法、模型训练方法、装置和设备
CN112991150A (zh) * 2021-02-08 2021-06-18 北京字跳网络技术有限公司 风格图像生成方法、模型训练方法、装置和设备
CN113449774A (zh) * 2021-06-02 2021-09-28 北京鹰瞳科技发展股份有限公司 眼底图像质量控制方法、装置、电子设备及存储介质
CN113613070B (zh) * 2021-10-08 2022-01-18 北京奇艺世纪科技有限公司 一种人脸视频处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019128508A1 (zh) * 2017-12-28 2019-07-04 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN109993716A (zh) * 2017-12-29 2019-07-09 微软技术许可有限责任公司 图像融合变换
CN110705625A (zh) * 2019-09-26 2020-01-17 北京奇艺世纪科技有限公司 一种图像处理方法、装置、电子设备及存储介质
CN110956079A (zh) * 2019-10-12 2020-04-03 深圳壹账通智能科技有限公司 人脸识别模型构建方法、装置、计算机设备和存储介质
CN111598818A (zh) * 2020-04-17 2020-08-28 北京百度网讯科技有限公司 人脸融合模型训练方法、装置及电子设备
CN112330533A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 混血人脸图像生成方法、模型训练方法、装置和设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609481B (zh) * 2017-08-14 2020-11-20 百度在线网络技术（北京）有限公司 为人脸识别生成训练数据的方法、装置和计算机存储介质
CN110135574A (zh) * 2018-02-09 2019-08-16 北京世纪好未来教育科技有限公司 神经网络训练方法、图像生成方法及计算机存储介质
US10762337B2 (en) * 2018-04-27 2020-09-01 Apple Inc. Face synthesis using generative adversarial networks
CN108846793B (zh) * 2018-05-25 2022-04-22 深圳市商汤科技有限公司 基于图像风格转换模型的图像处理方法和终端设备
CN111652828B (zh) * 2020-05-27 2023-08-08 北京百度网讯科技有限公司 人脸图像生成方法、装置、设备和介质
CN111833242A (zh) * 2020-07-17 2020-10-27 北京字节跳动网络技术有限公司 人脸变换方法、装置、电子设备和计算机可读介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019128508A1 (zh) * 2017-12-28 2019-07-04 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN109993716A (zh) * 2017-12-29 2019-07-09 微软技术许可有限责任公司 图像融合变换
CN110705625A (zh) * 2019-09-26 2020-01-17 北京奇艺世纪科技有限公司 一种图像处理方法、装置、电子设备及存储介质
CN110956079A (zh) * 2019-10-12 2020-04-03 深圳壹账通智能科技有限公司 人脸识别模型构建方法、装置、计算机设备和存储介质
CN111598818A (zh) * 2020-04-17 2020-08-28 北京百度网讯科技有限公司 人脸融合模型训练方法、装置及电子设备
CN112330533A (zh) * 2020-11-13 2021-02-05 北京字跳网络技术有限公司 混血人脸图像生成方法、模型训练方法、装置和设备

Also Published As

Publication number Publication date
CN112330533A (zh) 2021-02-05

Similar Documents

Publication Publication Date Title
WO2022100680A1 (zh) 混血人脸图像生成方法、模型训练方法、装置和设备
WO2021082760A1 (zh) 虚拟形象的生成方法、装置、终端及存储介质
JP7104683B2 (ja) 情報を生成する方法および装置
WO2022166897A1 (zh) 脸型调整图像生成方法、模型训练方法、装置和设备
WO2022100690A1 (zh) 动物脸风格图像生成方法、模型训练方法、装置和设备
TWI255141B (en) Method and system for real-time interactive video
EP3105921A1 (en) Photo composition and position guidance in an imaging device
WO2020244074A1 (zh) 表情交互方法、装置、计算机设备及可读存储介质
WO2023035531A1 (zh) 文本图像超分辨率重建方法及其相关设备
CN105430269B (zh) 一种应用于移动终端的拍照方法及装置
WO2023232056A1 (zh) 图像处理方法、装置、存储介质及电子设备
WO2022166908A1 (zh) 风格图像生成方法、模型训练方法、装置和设备
WO2021190625A1 (zh) 拍摄方法和设备
WO2020205003A1 (en) Techniques to capture and edit dynamic depth images
WO2023051244A1 (zh) 图像生成方法、装置、设备及存储介质
WO2022237633A1 (zh) 一种图像处理方法、装置、设备及介质
WO2023197648A1 (zh) 截图处理方法及装置、电子设备和计算机可读介质
JP2023515144A (ja) 美容処理方法及び装置
WO2022170982A1 (zh) 图像处理方法、图像生成方法、装置、设备和介质
WO2022016996A1 (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
WO2021057644A1 (zh) 拍摄方法和装置
CN108513090B (zh) 群组视频会话的方法及装置
CN116962848A (zh) 视频生成方法、装置、终端、存储介质及产品
CN112381709B (zh) 图像处理方法、模型训练方法、装置、设备和介质
KR20190002386A (ko) 캐릭터 서비스 시스템에서의 캐릭터 서비스를 제공하기 위한 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21891200

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21891200

Country of ref document: EP

Kind code of ref document: A1