WO2021051510A1 - Method and apparatus for generating face image, computer device, and storage medium - Google Patents

Method and apparatus for generating face image, computer device, and storage medium Download PDF

Info

Publication number
WO2021051510A1
WO2021051510A1 PCT/CN2019/116707 CN2019116707W WO2021051510A1 WO 2021051510 A1 WO2021051510 A1 WO 2021051510A1 CN 2019116707 W CN2019116707 W CN 2019116707W WO 2021051510 A1 WO2021051510 A1 WO 2021051510A1
Authority
WO
WIPO (PCT)
Prior art keywords
face image
angle
target face
target
image
Prior art date
Application number
PCT/CN2019/116707
Other languages
French (fr)
Chinese (zh)
Inventor
徐玲玲
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2021051510A1 publication Critical patent/WO2021051510A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification

Definitions

  • This application relates to the field of computers, in particular to methods, devices, computer equipment and storage media for generating facial images.
  • Face recognition has been widely used in robotics and other disciplines. It is of great significance for automatic identification and automatic discrimination of humans.
  • each user usually only collects a front view image.
  • the face recognition neural network can only generate one type of picture, and cannot generate pictures of multiple angle types at the same time. This makes the application of the real face recognition system have recognition defects, which leads to a sharp drop in the recognition rate and affects normal use.
  • the main purpose of this application is to provide a method for generating a face image, which aims to solve the technical problem that the existing face recognition neural network cannot generate pictures of multiple angle types at the same time.
  • This application proposes a method for generating a face image, including:
  • the original face image is converted into the first target face image corresponding to the first angle vector according to the preset conversion method, where the first angle vector is included in all target face images corresponding to In the angle vector, the first target face image is included in all target face images;
  • This application also provides a device for generating a face image, including:
  • the first input module is used to input the original face image and the angle vector of the target face image into the pre-trained face image generation network at the same time, wherein the angle vector of the target face image includes at least one;
  • the conversion module is used to convert the original face image into the first target face image corresponding to the first angle vector through the pre-trained face image generation network according to the preset conversion method, where the first angle vector is included in all targets In the angle vector corresponding to the face image, the first target face image is included in all target face images;
  • the evaluation module is used to evaluate whether the first angle vector matches the first target face image through a preset discrimination network
  • the output module is used to output the first target face image if it matches.
  • the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when the computer program is executed.
  • the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method are realized.
  • the original face image and the angle vector are input into the face image generation network together, and the angle vector is applied to the original face image to realize the synthesis of the target face image corresponding to the angle.
  • the face image generation network of the present application is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors to the face.
  • target face images of multiple angles corresponding to the original face image are generated at the same time.
  • the face image corresponding to the angle is classified into the correct angle domain by the discrimination network, it indicates The reliability of the generation network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction.
  • Fig. 1 is a schematic flowchart of a method for generating a face image according to an embodiment of the present application
  • Fig. 2 is a schematic structural diagram of an apparatus for generating a face image according to an embodiment of the present application
  • Fig. 3 is a schematic diagram of the internal structure of a computer device according to an embodiment of the present application.
  • a method for generating a face image includes:
  • the original face image is converted into the first target face image corresponding to the first angle vector according to the preset conversion method, where the first angle vector is included in all target face images In the corresponding angle vector, the first target face image is included in all target face images.
  • the original face image and the angle vector of the target face image are input into the face image generation network together, and the angle vector is applied to the original face image to realize the target face image corresponding to the angle vector Synthesis.
  • the face image generation network of this embodiment is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors together.
  • the target face images corresponding to the original face images after the conversion of each angle vector are generated and output at the same time.
  • step S2 of converting the original face image into the first target face image corresponding to the first angle vector according to the preset conversion method through the pre-trained face image generation network includes:
  • S22 Locate the feature points corresponding to the designated organs of the face according to the feature point positioning model, where the designated organs include at least one type.
  • S25 Form a first target face image according to the face area corresponding to the first angle vector.
  • the head image of the face is enlarged and displayed to form a face.
  • the head region image is recognized and located according to the positioning model, and the coordinate data of the feature point is mapped to the feature point template corresponding to the angle vector according to the angle vector to obtain the target face image.
  • the above-mentioned feature points of the human face include at least five feature points, the feature points corresponding to the two eyes, the feature points corresponding to the corners of the two mouths, and the feature point corresponding to the nose.
  • each coordinate is represented by (x, y), and then according to the opencv affine transformation, the coordinates of the five feature points obtained are The data is mapped to the feature point template corresponding to the pre-stored angle vector, and the face area corresponding to the angle vector is obtained to form the target face image.
  • the face detection model such as mtcnn
  • five feature points of the face can be obtained, and the coordinate data corresponding to the five feature points can be determined.
  • step S3 of evaluating whether the first angle vector matches the first target face image through a preset discrimination network includes:
  • S31 Input the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network.
  • S31 Generate a second target face image according to the inverse angle of the first angle vector and the first target face image.
  • This embodiment verifies the reliability of the pre-trained face image generation network by using the inverse angle of the corresponding angle of the target face image and the synthesized target face image as the input of the face image generation network, and whether to output the original face image If the inverse angle of the corresponding angle between the target face image and the synthesized target face image is used as the input of the face image generation network, and the original face image can be output, it indicates that the face image generation network is reliable.
  • the above-mentioned synthetic target face image corresponding angle is c, and the inverse angle of the synthetic target face image corresponding angle is -c.
  • step S33 of determining whether the second target face image is the same as the original face image the method includes:
  • , G represents the face image generation network, G(x,c) represents the input image x, When generating the corresponding face image when the angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, and c' -c.
  • the first correction loss function is Lg, and the face image generation network G is corrected by the first correction loss function.
  • G(x,c) means to generate a face image with an angle of c
  • Lfcls is used to optimize the generation network G, that is, the face image generated by the generation network G can be classified into the target angle domain c by the discrimination network D.
  • Lrec is to ensure that the original face image x is transformed in the angle domain c, and then transformed back to the face image corresponding to the original angle domain c', which is consistent with the original face image.
  • step S4 of outputting the first target face image the method includes:
  • the generation network is connected with the discrimination network, and the face image output by the generation network is classified by angle to further verify the reliability of the generation network. If the face image corresponding to the angle is classified to the correct angle by the discrimination network Domain indicates that the reliability of the generated network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction. In this embodiment, by comparing the magnitude of the angle vector of the first target face image relative to the original face, whether it is within the corresponding angle classification range, and if it is within the range, it indicates that the classification is correct.
  • step S1 inputting the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time before step S1 includes:
  • S11 Obtain face images of a specified number of users, where the face images of the same user include at least two angle images corresponding to the specified angles.
  • S14 Input the training samples into the neural network for training, and obtain a classification model that can classify the image domain corresponding to the specified angle.
  • the parameters of the angle vector are added at the same time to generate face image generation templates corresponding to different angle domains.
  • the face images are classified, the same angle is put into the same classification, and each angle is represented by an angle vector with a unique length.
  • the specified number is greater than or equal to 1000 users, and each user corresponds to face images of multiple angles.
  • the specified angles include, for example, -90, -60, -45, -30, 0, 30, 45, 60, 90 and other angles.
  • step S13 of combining the angle image and the vector value of the specified length corresponding to the specified angle into a training sample includes:
  • the training samples are corrected by positioning to improve the accuracy of the training samples.
  • the face image is centered by cropping, and then the pixel values of the feature points corresponding to the facial organs are corrected. For example, if the standard face image is 256*256, the standard pixel values of the eyes are 58*58, and 158*58.
  • the correction is aligned to The standard pixel value of the standard face image corresponding to the first specified angle, and the feature points corresponding to other face organs are correspondingly corrected in the above process to reduce the degree of sample difference, the impact on the training accuracy of the face image generation network, and improve the generation network Synchronously output the accuracy of face images corresponding to multiple angles.
  • the apparatus for generating a face image includes:
  • the first input module 1 is configured to simultaneously input the angle vectors of the original face image and the target face image into a pre-trained face image generation network, where the angle vector of the target face image includes at least one.
  • the conversion module 2 is used to convert the original face image into a first target face image corresponding to a first angle vector through a pre-trained face image generation network according to a preset conversion method, where the first angle vector is included in all In the angle vector corresponding to the target face image, the first target face image is included in all target face images.
  • the evaluation module 3 is used to evaluate whether the first angle vector matches the first target face image through a preset discrimination network.
  • the output module 4 is configured to output the first target face image if it matches.
  • the original face image and the angle vector of the target face image are input into the face image generation network together, and the angle vector is applied to the original face image to realize the target face image corresponding to the angle vector Synthesis.
  • the face image generation network of this embodiment is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors together.
  • the target face images corresponding to the original face images after the conversion of each angle vector are generated and output at the same time.
  • the conversion module 2 includes:
  • the cropping unit is used to crop the image of the head region of the face from the original face image.
  • the positioning unit is used for locating the feature points corresponding to the designated organs of the face according to the feature point positioning model, wherein the designated organs include at least one type.
  • the first acquiring unit is used to acquire the coordinate data of the characteristic points corresponding to each designated organ.
  • the mapping unit is used to map each coordinate data one-to-one to the feature point template corresponding to the first angle vector to obtain the face area corresponding to the first angle vector.
  • the forming unit is used to form the first target face image according to the face area corresponding to the first angle vector.
  • the head image of the face is enlarged and displayed to form a face.
  • the head region image is recognized and located according to the positioning model, and the coordinate data of the feature point is mapped to the feature point template corresponding to the angle vector according to the angle vector to obtain the target face image.
  • the above-mentioned feature points of the human face include at least five feature points, the feature points corresponding to the two eyes, the feature points corresponding to the corners of the two mouths, and the feature point corresponding to the nose.
  • each coordinate is represented by (x, y), and then according to the opencv affine transformation, the coordinates of the five feature points obtained are The data is mapped to the feature point template corresponding to the pre-stored angle vector, and the face area corresponding to the angle vector is obtained to form the target face image.
  • the face detection model such as mtcnn
  • five feature points of the face can be obtained, and the coordinate data corresponding to the five feature points can be determined.
  • evaluation module 3 includes:
  • the input unit is used to input the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network.
  • the first generating unit is configured to generate a second target face image according to the inverse angle of the first angle vector and the first target face image.
  • the judging unit is used to judge whether the second target face image is the same as the original face image.
  • the determining unit is configured to determine that the first angle vector matches the first target face image if they are the same.
  • This embodiment verifies the reliability of the pre-trained face image generation network by using the inverse angle of the corresponding angle of the target face image and the synthesized target face image as the input of the face image generation network, and whether to output the original face image If the inverse angle of the corresponding angle between the target face image and the synthesized target face image is used as the input of the face image generation network, and the original face image can be output, it indicates that the face image generation network is reliable.
  • the above-mentioned synthetic target face image corresponding angle is c, then the inverse angle of the synthetic target face image corresponding angle is -c.
  • evaluation module 3 includes:
  • the determining unit is configured to determine the difference information between the second target face image and the original face image if the second target face image is different from the original face image.
  • , G represents the face image generation network, G(x,c) Represents the input image x, the corresponding face image when the generated angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, c' -c.
  • Ladv logD(x)+log(1-D(G(x, c))
  • Lfcls -logD(G(x,c))
  • Lrec
  • the correction unit is configured to correct the parameters in the pre-trained face image generation network according to the first correction loss function.
  • the first correction loss function is Lg, and the face image generation network G is corrected by the first correction loss function.
  • G(x,c) means to generate a face image with an angle of c
  • Lfcls is used to optimize the generation network G, that is, the face image generated by the generation network G can be classified into the target angle domain c by the discrimination network D.
  • Lrec is to ensure that the original face image x is transformed in the angle domain c, and then transformed back to the face image corresponding to the original angle domain c', which is consistent with the original face image.
  • the device for generating a face image includes:
  • the judging module is used to judge whether the angle classification corresponding to each first target face image is correct.
  • the generation network is connected with the discrimination network, and the face image output by the generation network is classified by angle to further verify the reliability of the generation network. If the face image corresponding to the angle is classified to the correct angle by the discrimination network Domain indicates that the reliability of the generated network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction. In this embodiment, by comparing the magnitude of the angle vector of the first target face image relative to the original face, whether it is within the corresponding angle classification range, and if it is within the range, it indicates that the classification is correct.
  • the device for generating a face image includes:
  • the obtaining module is used to obtain face images of a specified number of users, where the face images of the same user include at least two angle images corresponding to the specified angles.
  • the mapping module is used to map the specified angle corresponding to the angle image to the vector value of the specified length.
  • the combination module is used to combine the angle image and the vector value of the specified length corresponding to the specified angle into a training sample.
  • the second input module is used to input training samples into the neural network for training to obtain a classification model that can classify the image domain corresponding to the specified angle.
  • the classification model As a module, it is used to use the classification model as a pre-trained face image generation network.
  • the parameters of the angle vector are added at the same time to generate face image generation templates corresponding to different angle domains.
  • the face images are classified, the same angle is put into the same classification, and each angle is represented by an angle vector with a unique length.
  • the specified number is greater than or equal to 1000 users, and each user corresponds to face images of multiple angles.
  • the specified angles include, for example, -90, -60, -45, -30, 0, 30, 45, 60, 90 and other angles.
  • the combination module includes:
  • the second acquiring unit is configured to acquire the first pixel value corresponding to the facial organ in the first angle image, where the first angle image is included in all angle images.
  • the third acquiring unit is used to acquire the standard face image corresponding to the first designated angle.
  • the correction unit is used to correct the first pixel value according to the standard pixel value corresponding to each facial organ in the standard face image to obtain the corrected first pixel value.
  • the combination unit is used to combine the second angle image carrying the corrected first pixel value and the first vector value corresponding to the first angle into a corrected training sample.
  • the training samples are corrected by positioning to improve the accuracy of the training samples.
  • the face image is centered by cropping, and then the pixel values of the feature points corresponding to the facial organs are corrected. For example, if the standard face image is 256*256, the standard pixel values of the eyes are 58*58, and 158*58.
  • the correction is aligned to The standard pixel value of the standard face image corresponding to the first specified angle, and the feature points corresponding to other face organs are correspondingly corrected in the above process to reduce the degree of sample difference, the impact on the training accuracy of the face image generation network, and improve the generation network Simultaneously output the accuracy of face images corresponding to multiple angles.
  • an embodiment of the present application also provides a computer device.
  • the computer device may be a server, and its internal structure may be as shown in FIG. 3.
  • the computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor designed by the computer is used to provide calculation and control capabilities.
  • the memory of the computer device includes a readable storage medium and an internal memory.
  • the readable storage medium stores an operating system, computer readable instructions, and a database.
  • the above-mentioned readable storage medium includes a non-volatile readable storage medium and a volatile readable storage medium.
  • the memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the database of the computer equipment is used to store the generated face images and other data.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer-readable instruction executes the process of the above-mentioned method embodiment.
  • An embodiment of the present application also provides a computer-readable storage medium on which computer-readable instructions are stored.
  • the processes of the foregoing method embodiments are executed.
  • the above-mentioned readable storage medium includes non-volatile readable storage medium and volatile readable storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed are a method and apparatus for generating a face image, a computer device, and a storage medium. The method for generating a face image comprises: simultaneously inputting an original face image and an angle vector of a target face image into a pre-trained face image generation network; converting, by the pre-trained face image generation network, the original face image into a first target face image corresponding to a first angle vector according to a preset conversion mode, the first angle vector being comprised in angle vectors corresponding to all target face images; the first target face image being comprised in all target face images; evaluating, by a preset determination network, whether the first angle vector matches the first target face image; and if the first angle vector matches the first target face image, outputting the first target face image. The synthesis of the target face image corresponding to an angle is achieved by simultaneously inputting the original face image and the angle vector into the face image generation network and acting the angle vector on the original face image.

Description

生成人脸图像的方法、装置、计算机设备及存储介质Method, device, computer equipment and storage medium for generating face image
本申请要求于2019年09月19日提交中国专利局、申请号为2019108866185,发明名称为“生成人脸图像的方法、装置、计算机设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on September 19, 2019, the application number is 2019108866185, and the invention title is "Methods, Apparatus, Computer Equipment and Storage Media for Generating Face Images", and its entire contents Incorporated in this application by reference.
技术领域Technical field
本申请涉及到计算机领域,特别是涉及到生成人脸图像的方法、装置、计算机设备及存储介质。This application relates to the field of computers, in particular to methods, devices, computer equipment and storage media for generating facial images.
背景技术Background technique
人脸识别已广泛应用在机器人等学科中,对自动鉴别和人类自动分辨有着重要的意义,然而在通过人脸识别验证时,通常每个用户只采集一副前视图图像,现有的普通人脸识别神经网络,只能生成一种类型的图片,不能同时生成多个角度类型的图片,使现实的人脸识别系统的应用具有识别缺陷,导致识别率大幅下降,影响正常使用。Face recognition has been widely used in robotics and other disciplines. It is of great significance for automatic identification and automatic discrimination of humans. However, when passing face recognition verification, each user usually only collects a front view image. The face recognition neural network can only generate one type of picture, and cannot generate pictures of multiple angle types at the same time. This makes the application of the real face recognition system have recognition defects, which leads to a sharp drop in the recognition rate and affects normal use.
技术问题technical problem
本申请的主要目的为提供生成人脸图像的方法,旨在解决现有人脸识别神经网络不能同时生成多个角度类型的图片的技术问题。The main purpose of this application is to provide a method for generating a face image, which aims to solve the technical problem that the existing face recognition neural network cannot generate pictures of multiple angle types at the same time.
技术解决方案Technical solutions
本申请提出一种生成人脸图像的方法,包括:This application proposes a method for generating a face image, including:
将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,目标人脸图像的角度向量至少包括一个;Input the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time, where the angle vector of the target face image includes at least one;
通过预训练的人脸图像生成网络,按照预设转换方式将原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,第一角度向量包含于所有目标人脸图像对应的角度向量中,第一目标人脸图像包含于所有目标人脸图像中;Through the pre-trained face image generation network, the original face image is converted into the first target face image corresponding to the first angle vector according to the preset conversion method, where the first angle vector is included in all target face images corresponding to In the angle vector, the first target face image is included in all target face images;
通过预设判别网络评价第一角度向量与第一目标人脸图像是否匹配;Evaluate whether the first angle vector matches the first target face image through a preset discrimination network;
若匹配,则输出第一目标人脸图像。If it matches, output the first target face image.
本申请还提供了一种生成人脸图像的装置,包括:This application also provides a device for generating a face image, including:
第一输入模块,用于将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,目标人脸图像的角度向量至少包括一个;The first input module is used to input the original face image and the angle vector of the target face image into the pre-trained face image generation network at the same time, wherein the angle vector of the target face image includes at least one;
转换模块,用于通过预训练的人脸图像生成网络,按照预设转换方式将原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,第一角度向 量包含于所有目标人脸图像对应的角度向量中,第一目标人脸图像包含于所有目标人脸图像中;The conversion module is used to convert the original face image into the first target face image corresponding to the first angle vector through the pre-trained face image generation network according to the preset conversion method, where the first angle vector is included in all targets In the angle vector corresponding to the face image, the first target face image is included in all target face images;
评价模块,用于通过预设判别网络评价第一角度向量与第一目标人脸图像是否匹配;The evaluation module is used to evaluate whether the first angle vector matches the first target face image through a preset discrimination network;
输出模块,用于若匹配,则输出第一目标人脸图像。The output module is used to output the first target face image if it matches.
本申请还提供了一种计算机设备,包括存储器和处理器,存储器存储有计算机程序,处理器执行计算机程序时实现上述方法的步骤。The present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when the computer program is executed.
本申请还提供了一种计算机可读存储介质,其上存储有计算机程序,计算机程序被处理器执行时实现上述的方法的步骤。The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method are realized.
有益效果Beneficial effect
本申请通过原始人脸图像和角度向量一并输入至人脸图像生成网络中,通过将角度向量作用于原始人脸图像,实现角度对应的目标人脸图像的合成。本申请的人脸图像生成网络通过多用户多角度人脸图像训练而成,携带多个角度图像分别对应的生成模板,可支持一张原始人脸图像,和多个角度向量一并输入至脸图像生成网络中,同时生成原始人脸图像对应的多个角度的目标人脸图像。通过将生成网络与判别网络进行连接,通过将生成网络输出的人脸图像进行角度分类,来进一步验证生成网络的可靠性,若角度对应的人脸图像被判别网络分类到正确的角度域,表明生成网络的可靠性高,判别网络的可靠性也高。若分类不正确,但经过第一校正loss函数分析,生成网络是可靠的,则表明判别网络需要进行校正,则通过第二校正loss函数Ld进行校正。In this application, the original face image and the angle vector are input into the face image generation network together, and the angle vector is applied to the original face image to realize the synthesis of the target face image corresponding to the angle. The face image generation network of the present application is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors to the face. In the image generation network, target face images of multiple angles corresponding to the original face image are generated at the same time. By connecting the generation network and the discrimination network, the reliability of the generation network is further verified by angle classification of the face images output by the generation network. If the face image corresponding to the angle is classified into the correct angle domain by the discrimination network, it indicates The reliability of the generation network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction.
附图说明Description of the drawings
图1本申请一实施例的生成人脸图像的方法流程示意图;Fig. 1 is a schematic flowchart of a method for generating a face image according to an embodiment of the present application;
图2本申请一实施例的生成人脸图像的装置结构示意图;Fig. 2 is a schematic structural diagram of an apparatus for generating a face image according to an embodiment of the present application;
图3本申请一实施例的计算机设备内部结构示意图。Fig. 3 is a schematic diagram of the internal structure of a computer device according to an embodiment of the present application.
本发明的最佳实施方式The best mode of the present invention
参照图1,本申请一实施例的生成人脸图像的方法,包括:1, a method for generating a face image according to an embodiment of the present application includes:
S1:将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,目标人脸图像的角度向量至少包括一个。S1: The angle vectors of the original face image and the target face image are simultaneously input into the pre-trained face image generation network, where the angle vector of the target face image includes at least one.
S2:通过预训练的人脸图像生成网络,按照预设转换方式将原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,第一角度向量包含于所有目标人脸图像对应的角度向量中,第一目标人脸图像包含于所有目标人脸图 像中。S2: Through the pre-trained face image generation network, the original face image is converted into the first target face image corresponding to the first angle vector according to the preset conversion method, where the first angle vector is included in all target face images In the corresponding angle vector, the first target face image is included in all target face images.
S3:通过预设判别网络评价第一角度向量与第一目标人脸图像是否匹配。S3: Evaluate whether the first angle vector matches the first target face image through a preset discrimination network.
S4:若匹配,则输出第一目标人脸图像。S4: If it matches, output the first target face image.
本实施例中,通过将原始人脸图像和目标人脸图像的角度向量一并输入至人脸图像生成网络中,通过将角度向量作用于原始人脸图像,实现角度向量对应的目标人脸图像的合成。本实施例的人脸图像生成网络通过多用户多角度人脸图像训练而成,携带多个角度图像分别对应的生成模板,可支持一张原始人脸图像,和多个角度向量一并输入至脸图像生成网络中,同时生成并输出原始人脸图像转化各角度向量后分别对应的目标人脸图像。In this embodiment, the original face image and the angle vector of the target face image are input into the face image generation network together, and the angle vector is applied to the original face image to realize the target face image corresponding to the angle vector Synthesis. The face image generation network of this embodiment is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors together. In the face image generation network, the target face images corresponding to the original face images after the conversion of each angle vector are generated and output at the same time.
进一步地,通过预训练的人脸图像生成网络,按照预设转换方式将原始人脸图像转换为第一角度向量对应的第一目标人脸图像的步骤S2,包括:Further, the step S2 of converting the original face image into the first target face image corresponding to the first angle vector according to the preset conversion method through the pre-trained face image generation network includes:
S21:从原始人脸图像中裁剪出人脸头部区域图像。S21: Cut out an image of the head region of the face from the original face image.
S22:根据特征点定位模型定位人脸的指定器官对应的特征点,其中,指定器官至少包括一种。S22: Locate the feature points corresponding to the designated organs of the face according to the feature point positioning model, where the designated organs include at least one type.
S23:获取各指定器官分别对应的特征点的坐标数据。S23: Obtain coordinate data of feature points corresponding to each designated organ.
S24:将各坐标数据一一对应映射到第一角度向量对应的特征点模板中,得到第一角度向量对应的人脸区域。S24: Map each coordinate data one-to-one to the feature point template corresponding to the first angle vector to obtain the face area corresponding to the first angle vector.
S25:根据第一角度向量对应的人脸区域,形成第一目标人脸图像。S25: Form a first target face image according to the face area corresponding to the first angle vector.
本实施例中在合成角度对应的目标人脸图像的过程中,为提高生成的目标人脸图像与原始人脸图像的准确匹配性,先通过裁剪使人脸头部图像放大显示,形成人脸头部区域图像,并根据定位模型识别并定位人脸器官特征点所在位置,并将特征点的坐标数据按照角度向量映射于角度向量对应的特征点模板中,得到目标人脸图像。上述人脸的特征点至少包括五个,两只眼睛分别对应的特征点、两个嘴角分别对应的特征点、鼻子对应的特征点。本实施例获取到上述五个特征点的在原始人脸图像中的坐标数据后,每个坐标按(x,y)表示,然后根据opencv的仿射变换,将获取的五个特征点的坐标数据映射到预存的角度向量对应的特征点模板中,获取角度向量对应的人脸区域,形成目标人脸图像。根据人脸检测模型(譬如mtcnn),可以获取人脸的五个特征点,并确定五个特征点分别对应的坐标数据。In this embodiment, in the process of synthesizing the target face image corresponding to the angle, in order to improve the accurate matching between the generated target face image and the original face image, the head image of the face is enlarged and displayed to form a face. The head region image is recognized and located according to the positioning model, and the coordinate data of the feature point is mapped to the feature point template corresponding to the angle vector according to the angle vector to obtain the target face image. The above-mentioned feature points of the human face include at least five feature points, the feature points corresponding to the two eyes, the feature points corresponding to the corners of the two mouths, and the feature point corresponding to the nose. In this embodiment, after obtaining the coordinate data of the five feature points in the original face image, each coordinate is represented by (x, y), and then according to the opencv affine transformation, the coordinates of the five feature points obtained are The data is mapped to the feature point template corresponding to the pre-stored angle vector, and the face area corresponding to the angle vector is obtained to form the target face image. According to the face detection model (such as mtcnn), five feature points of the face can be obtained, and the coordinate data corresponding to the five feature points can be determined.
进一步地,通过预设判别网络评价第一角度向量与第一目标人脸图像是否匹配的步骤S3,包括:Further, the step S3 of evaluating whether the first angle vector matches the first target face image through a preset discrimination network includes:
S31:将第一角度向量的逆角度与第一目标人脸图像,作为输入量输入至预训练的人脸图像生成网络。S31: Input the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network.
S31:根据第一角度向量的逆角度与第一目标人脸图像,生成第二目标人脸图像。S31: Generate a second target face image according to the inverse angle of the first angle vector and the first target face image.
S33:判断第二目标人脸图像是否与原始人脸图像相同。S33: Determine whether the second target face image is the same as the original face image.
S34:若相同,则判定第一角度向量与第一目标人脸图像匹配。S34: If they are the same, determine that the first angle vector matches the first target face image.
本实施例通过目标人脸图像和合成目标人脸图像对应角度的逆角度,作为人脸图像生成网络的输入量,是否输出原始人脸图像,来验证预训练的人脸图像生成网络的可靠性,若将目标人脸图像和合成目标人脸图像对应角度的逆角度,作为人脸图像生成网络的输入量,能输出原始人脸图像,则表明人脸图像生成网络是可靠的。上述合成目标人脸图像对应角度为c,则合成目标人脸图像对应角度的逆角度为-c。This embodiment verifies the reliability of the pre-trained face image generation network by using the inverse angle of the corresponding angle of the target face image and the synthesized target face image as the input of the face image generation network, and whether to output the original face image If the inverse angle of the corresponding angle between the target face image and the synthesized target face image is used as the input of the face image generation network, and the original face image can be output, it indicates that the face image generation network is reliable. The above-mentioned synthetic target face image corresponding angle is c, and the inverse angle of the synthetic target face image corresponding angle is -c.
进一步地,判断第二目标人脸图像是否与原始人脸图像相同的步骤S33之后,包括:Further, after step S33 of determining whether the second target face image is the same as the original face image, the method includes:
S35:若第二目标人脸图像与原始人脸图像不相同,则确定第二目标人脸图像与原始人脸图像的差异信息。S35: If the second target face image is different from the original face image, determine the difference information between the second target face image and the original face image.
S36:根据差异信息生成第一校正loss函数,其中,第一校正loss函数为Lg=Ladv+Lfcls+Lrec,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),Lrec=||x-G(G(x,c),c’)||,G表示人脸图像生成网络,G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数,c’=-c。S36: Generate a first correction loss function according to the difference information, where the first correction loss function is Lg=Ladv+Lfcls+Lrec, Ladv=logD(x)+log(1-D(G(x,c)), Lfcls =-logD(G(x,c)), Lrec=||xG(G(x,c),c')||, G represents the face image generation network, G(x,c) represents the input image x, When generating the corresponding face image when the angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, and c'=-c.
S37:根据第一校正loss函数校正预训练的人脸图像生成网络中的参量。S37: Correct the parameters in the pre-trained face image generation network according to the first correction loss function.
本实施例中若第二目标人脸图像与原始人脸图像不相同,或第二目标人脸图像与原始人脸图像的相似度小于预设阈值,则说明人脸图像生成网络的可靠性不能满足使用要求,第一校正loss函数为Lg,通过第一校正loss函数校正人脸图像生成网络G。G(x,c)表示生成角度为c的人脸图像,G(G(x,c),c')表示生成的角度为c的图像再次作为输入,生成自己原来的角度c’对应的人脸图像,c’=-c。Lfcls用来优化生成网络G,即让生成网络G生成的人脸图像能够被判别网络D分类成目标角度域c。Lrec是为了保证原始人脸图像x经过角度域c的变换后,再变换回原角度域c'对应的人脸图片,和原始人脸图像保持一致。In this embodiment, if the second target face image is different from the original face image, or the similarity between the second target face image and the original face image is less than the preset threshold, it means that the reliability of the face image generation network is not To meet the usage requirements, the first correction loss function is Lg, and the face image generation network G is corrected by the first correction loss function. G(x,c) means to generate a face image with an angle of c, G(G(x,c),c') means to generate an image with an angle of c as input again to generate the person corresponding to its original angle c' Face image, c'=-c. Lfcls is used to optimize the generation network G, that is, the face image generated by the generation network G can be classified into the target angle domain c by the discrimination network D. Lrec is to ensure that the original face image x is transformed in the angle domain c, and then transformed back to the face image corresponding to the original angle domain c', which is consistent with the original face image.
进一步地,输出第一目标人脸图像的步骤S4之后,包括:Further, after step S4 of outputting the first target face image, the method includes:
S41:判断对各第一目标人脸图像分别对应的角度分类是否分类正确。S41: Determine whether the angle classification corresponding to each first target face image is correct.
S42:若否,则通过第二校正loss函数校正判别网络,其中,第二校正loss函数Ld=-Ladv+Lfcls,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数。S42: If not, use the second correction loss function to correct the discrimination network, where the second correction loss function Ld=-Ladv+Lfcls, Ladv=logD(x)+log(1-D(G(x,c)) , Lfcls=-logD(G(x,c)), G(x,c) represents the input image x, when the generated angle is c, the corresponding face image, D(x) represents the domain calculated by D on the real image Label probability distribution, D represents the discriminant network function.
本实施例通过将生成网络与判别网络进行连接,通过将生成网络输出的人脸图像进行角度分类,来进一步验证生成网络的可靠性,若角度对应的人脸图像被判别网络分类到正确的角度域,表明生成网络的可靠性高,判别网络的可靠性也高。若分类不正确,但经过第一校正loss函数分析,生成网络是可靠的,则表明判别网络需要进行校正,则通过第二校正loss函数Ld进行校正。本实施例通过比较第一目标人脸图像相对于原始人脸的角度向量的大小,是否在对应的角度分类范围内,若在范围内,说明分类正确。In this embodiment, the generation network is connected with the discrimination network, and the face image output by the generation network is classified by angle to further verify the reliability of the generation network. If the face image corresponding to the angle is classified to the correct angle by the discrimination network Domain indicates that the reliability of the generated network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction. In this embodiment, by comparing the magnitude of the angle vector of the first target face image relative to the original face, whether it is within the corresponding angle classification range, and if it is within the range, it indicates that the classification is correct.
进一步地,将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中的步骤S1之前,包括:Further, inputting the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time before step S1 includes:
S11:获取指定数量用户的人脸图像,其中,同一用户的人脸图像至少包括两个指定角度对应的角度图像。S11: Obtain face images of a specified number of users, where the face images of the same user include at least two angle images corresponding to the specified angles.
S12:角度图像对应的指定角度映射为指定长度的向量值。S12: The specified angle corresponding to the angle image is mapped to a vector value of the specified length.
S13:将角度图像与指定角度对应的指定长度的向量值,组合成训练样本。S13: Combine the angle image and the specified length vector value corresponding to the specified angle to form a training sample.
S14:将训练样本输入至神经网络中进行训练,得到能分类指定角度对应图像域的分类模型。S14: Input the training samples into the neural network for training, and obtain a classification model that can classify the image domain corresponding to the specified angle.
S15:将分类模型作为预训练的人脸图像生成网络。S15: Use the classification model as a pre-trained face image generation network.
本实施例在训练人脸图像生成网络时,同时加入角度向量的参数,生成不同角度域分别对应的人脸图像生成模板。按照不同角度,将人脸图像进行分类,同一角度的放入相同分类,每一个角度使用一个长度唯一的角度向量表示。上述指定数量大于等于1000用户,每个用户对应多个角度的人脸图像,指定角度例如包括-90,-60,-45,-30,0,30,45,60,90等多个角度。In this embodiment, when training the face image generation network, the parameters of the angle vector are added at the same time to generate face image generation templates corresponding to different angle domains. According to different angles, the face images are classified, the same angle is put into the same classification, and each angle is represented by an angle vector with a unique length. The specified number is greater than or equal to 1000 users, and each user corresponds to face images of multiple angles. The specified angles include, for example, -90, -60, -45, -30, 0, 30, 45, 60, 90 and other angles.
进一步地,将角度图像与指定角度对应的指定长度的向量值,组合成训练样本的步骤S13,包括:Further, the step S13 of combining the angle image and the vector value of the specified length corresponding to the specified angle into a training sample includes:
S131:获取第一角度图像中人脸器官对应的第一像素值,其中,第一角度 图像包含于所有角度图像中。S131: Obtain the first pixel value corresponding to the facial organ in the first angle image, where the first angle image is included in all angle images.
S132:获取第一指定角度对应的标准人脸图像。S132: Obtain a standard face image corresponding to the first specified angle.
S133:根据标准人脸图像中各人脸器官对应的标准像素值,修正第一像素值,得到修正后的第一像素值。S133: Correct the first pixel value according to the standard pixel value corresponding to each face organ in the standard face image to obtain the corrected first pixel value.
S134:将携带修正后的第一像素值的第二角度图像,与第一角度对应的第一向量值,组合为修正训练样本。S134: Combine the second angle image carrying the corrected first pixel value and the first vector value corresponding to the first angle into a corrected training sample.
本实施例中,训练样本通过定位修正,以提高训练样本的正确性。本实施例通过裁剪,使人脸图像处于正中央,然后将人脸器官对应的特征点的像素值进行修正。比如标准人脸图像为256*256,则眼睛的标准像素值为58*58,和158*58,通过将当前用户的第一角度图像中人脸器官对应的第一像素值,矫正对准到第一指定角度对应的标准人脸图像的标准像素值,其他人脸器官对应的特征点如上过程对应校正,以减小样本差异度大,对人脸图像生成网络的训练精度影响,提高生成网络同步输出多个角度分别对应的人脸图像的精准性。In this embodiment, the training samples are corrected by positioning to improve the accuracy of the training samples. In this embodiment, the face image is centered by cropping, and then the pixel values of the feature points corresponding to the facial organs are corrected. For example, if the standard face image is 256*256, the standard pixel values of the eyes are 58*58, and 158*58. By correcting the first pixel value corresponding to the facial organ in the current user’s first angle image, the correction is aligned to The standard pixel value of the standard face image corresponding to the first specified angle, and the feature points corresponding to other face organs are correspondingly corrected in the above process to reduce the degree of sample difference, the impact on the training accuracy of the face image generation network, and improve the generation network Synchronously output the accuracy of face images corresponding to multiple angles.
参照图2,本申请一实施例的生成人脸图像的装置,包括:Referring to FIG. 2, the apparatus for generating a face image according to an embodiment of the present application includes:
第一输入模块1,用于将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,目标人脸图像的角度向量至少包括一个。The first input module 1 is configured to simultaneously input the angle vectors of the original face image and the target face image into a pre-trained face image generation network, where the angle vector of the target face image includes at least one.
转换模块2,用于通过预训练的人脸图像生成网络,按照预设转换方式将原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,第一角度向量包含于所有目标人脸图像对应的角度向量中,第一目标人脸图像包含于所有目标人脸图像中。The conversion module 2 is used to convert the original face image into a first target face image corresponding to a first angle vector through a pre-trained face image generation network according to a preset conversion method, where the first angle vector is included in all In the angle vector corresponding to the target face image, the first target face image is included in all target face images.
评价模块3,用于通过预设判别网络评价第一角度向量与第一目标人脸图像是否匹配。The evaluation module 3 is used to evaluate whether the first angle vector matches the first target face image through a preset discrimination network.
输出模块4,用于若匹配,则输出第一目标人脸图像。The output module 4 is configured to output the first target face image if it matches.
本实施例中,通过将原始人脸图像和目标人脸图像的角度向量一并输入至人脸图像生成网络中,通过将角度向量作用于原始人脸图像,实现角度向量对应的目标人脸图像的合成。本实施例的人脸图像生成网络通过多用户多角度人脸图像训练而成,携带多个角度图像分别对应的生成模板,可支持一张原始人脸图像,和多个角度向量一并输入至脸图像生成网络中,同时生成并输出原始人脸图像转化各角度向量后分别对应的目标人脸图像。In this embodiment, the original face image and the angle vector of the target face image are input into the face image generation network together, and the angle vector is applied to the original face image to realize the target face image corresponding to the angle vector Synthesis. The face image generation network of this embodiment is formed by multi-user and multi-angle face image training, and carries multiple angle images corresponding to the generation template, which can support an original face image and input multiple angle vectors together. In the face image generation network, the target face images corresponding to the original face images after the conversion of each angle vector are generated and output at the same time.
进一步地,转换模块2,包括:Further, the conversion module 2 includes:
裁剪单元,用于从原始人脸图像中裁剪出人脸头部区域图像。The cropping unit is used to crop the image of the head region of the face from the original face image.
定位单元,用于根据特征点定位模型定位人脸的指定器官对应的特征点,其中,指定器官至少包括一种。The positioning unit is used for locating the feature points corresponding to the designated organs of the face according to the feature point positioning model, wherein the designated organs include at least one type.
第一获取单元,用于获取各指定器官分别对应的特征点的坐标数据。The first acquiring unit is used to acquire the coordinate data of the characteristic points corresponding to each designated organ.
映射单元,用于将各坐标数据一一对应映射到第一角度向量对应的特征点模板中,得到第一角度向量对应的人脸区域。The mapping unit is used to map each coordinate data one-to-one to the feature point template corresponding to the first angle vector to obtain the face area corresponding to the first angle vector.
形成单元,用于根据第一角度向量对应的人脸区域,形成第一目标人脸图像。The forming unit is used to form the first target face image according to the face area corresponding to the first angle vector.
本实施例中在合成角度对应的目标人脸图像的过程中,为提高生成的目标人脸图像与原始人脸图像的准确匹配性,先通过裁剪使人脸头部图像放大显示,形成人脸头部区域图像,并根据定位模型识别并定位人脸器官特征点所在位置,并将特征点的坐标数据按照角度向量映射于角度向量对应的特征点模板中,得到目标人脸图像。上述人脸的特征点至少包括五个,两只眼睛分别对应的特征点、两个嘴角分别对应的特征点、鼻子对应的特征点。本实施例获取到上述五个特征点的在原始人脸图像中的坐标数据后,每个坐标按(x,y)表示,然后根据opencv的仿射变换,将获取的五个特征点的坐标数据映射到预存的角度向量对应的特征点模板中,获取角度向量对应的人脸区域,形成目标人脸图像。根据人脸检测模型(譬如mtcnn),可以获取人脸的五个特征点,并确定五个特征点分别对应的坐标数据。In this embodiment, in the process of synthesizing the target face image corresponding to the angle, in order to improve the accurate matching between the generated target face image and the original face image, the head image of the face is enlarged and displayed to form a face. The head region image is recognized and located according to the positioning model, and the coordinate data of the feature point is mapped to the feature point template corresponding to the angle vector according to the angle vector to obtain the target face image. The above-mentioned feature points of the human face include at least five feature points, the feature points corresponding to the two eyes, the feature points corresponding to the corners of the two mouths, and the feature point corresponding to the nose. In this embodiment, after obtaining the coordinate data of the five feature points in the original face image, each coordinate is represented by (x, y), and then according to the opencv affine transformation, the coordinates of the five feature points obtained are The data is mapped to the feature point template corresponding to the pre-stored angle vector, and the face area corresponding to the angle vector is obtained to form the target face image. According to the face detection model (such as mtcnn), five feature points of the face can be obtained, and the coordinate data corresponding to the five feature points can be determined.
进一步地,评价模块3,包括:Further, the evaluation module 3 includes:
输入单元,用于将第一角度向量的逆角度与第一目标人脸图像,作为输入量输入至预训练的人脸图像生成网络。The input unit is used to input the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network.
第一生成单元,用于根据第一角度向量的逆角度与第一目标人脸图像,生成第二目标人脸图像。The first generating unit is configured to generate a second target face image according to the inverse angle of the first angle vector and the first target face image.
判断单元,用于判断第二目标人脸图像是否与原始人脸图像相同。The judging unit is used to judge whether the second target face image is the same as the original face image.
判定单元,用于若相同,则判定第一角度向量与第一目标人脸图像匹配。The determining unit is configured to determine that the first angle vector matches the first target face image if they are the same.
本实施例通过目标人脸图像和合成目标人脸图像对应角度的逆角度,作为人脸图像生成网络的输入量,是否输出原始人脸图像,来验证预训练的人脸图像生成网络的可靠性,若将目标人脸图像和合成目标人脸图像对应角度的逆角度,作为人脸图像生成网络的输入量,能输出原始人脸图像,则表明人脸图像生成网络是可靠的。上述合成目标人脸图像对应角度为c,则合成目标人脸图 像对应角度的逆角度为-c。This embodiment verifies the reliability of the pre-trained face image generation network by using the inverse angle of the corresponding angle of the target face image and the synthesized target face image as the input of the face image generation network, and whether to output the original face image If the inverse angle of the corresponding angle between the target face image and the synthesized target face image is used as the input of the face image generation network, and the original face image can be output, it indicates that the face image generation network is reliable. The above-mentioned synthetic target face image corresponding angle is c, then the inverse angle of the synthetic target face image corresponding angle is -c.
进一步地,评价模块3,包括:Further, the evaluation module 3 includes:
确定单元,用于若第二目标人脸图像与原始人脸图像不相同,则确定第二目标人脸图像与原始人脸图像的差异信息。The determining unit is configured to determine the difference information between the second target face image and the original face image if the second target face image is different from the original face image.
第二生成单元,用于根据差异信息生成第一校正loss函数,其中,第一校正loss函数为Lg=Ladv+Lfcls+Lrec,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),Lrec=||x-G(G(x,c),c’)||,G表示人脸图像生成网络,G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数,c’=-c。The second generating unit is configured to generate the first correction loss function according to the difference information, where the first correction loss function is Lg=Ladv+Lfcls+Lrec, Ladv=logD(x)+log(1-D(G(x, c)), Lfcls=-logD(G(x,c)), Lrec=||xG(G(x,c),c')||, G represents the face image generation network, G(x,c) Represents the input image x, the corresponding face image when the generated angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, c'=-c.
校正单元,用于根据第一校正loss函数校正预训练的人脸图像生成网络中的参量。The correction unit is configured to correct the parameters in the pre-trained face image generation network according to the first correction loss function.
本实施例中若第二目标人脸图像与原始人脸图像不相同,或第二目标人脸图像与原始人脸图像的相似度小于预设阈值,则说明人脸图像生成网络的可靠性不能满足使用要求,第一校正loss函数为Lg,通过第一校正loss函数校正人脸图像生成网络G。G(x,c)表示生成角度为c的人脸图像,G(G(x,c),c')表示生成的角度为c的图像再次作为输入,生成自己原来的角度c’对应的人脸图像,c’=-c。Lfcls用来优化生成网络G,即让生成网络G生成的人脸图像能够被判别网络D分类成目标角度域c。Lrec是为了保证原始人脸图像x经过角度域c的变换后,再变换回原角度域c'对应的人脸图片,和原始人脸图像保持一致。In this embodiment, if the second target face image is different from the original face image, or the similarity between the second target face image and the original face image is less than the preset threshold, it means that the reliability of the face image generation network is not To meet the usage requirements, the first correction loss function is Lg, and the face image generation network G is corrected by the first correction loss function. G(x,c) means to generate a face image with an angle of c, G(G(x,c),c') means that the generated image with an angle of c is used as input again to generate the person corresponding to its original angle c' Face image, c'=-c. Lfcls is used to optimize the generation network G, that is, the face image generated by the generation network G can be classified into the target angle domain c by the discrimination network D. Lrec is to ensure that the original face image x is transformed in the angle domain c, and then transformed back to the face image corresponding to the original angle domain c', which is consistent with the original face image.
进一步地,生成人脸图像的装置,包括:Further, the device for generating a face image includes:
判断模块,用于判断对各第一目标人脸图像分别对应的角度分类是否分类正确。The judging module is used to judge whether the angle classification corresponding to each first target face image is correct.
校正模块,用于若否,则通过第二校正loss函数校正判别网络,其中,第二校正loss函数Ld=-Ladv+Lfcls,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数。The correction module is used to correct the discrimination network through the second correction loss function if not, where the second correction loss function Ld=-Ladv+Lfcls, Ladv=logD(x)+log(1-D(G(x, c)), Lfcls=-logD(G(x,c)), G(x,c) represents the input image x, when the generated angle is c, the corresponding face image, D(x) represents the calculation of the real image by D The probability distribution of the obtained domain label, D represents the discriminant network function.
本实施例通过将生成网络与判别网络进行连接,通过将生成网络输出的人脸图像进行角度分类,来进一步验证生成网络的可靠性,若角度对应的人脸图像被判别网络分类到正确的角度域,表明生成网络的可靠性高,判别网络的可 靠性也高。若分类不正确,但经过第一校正loss函数分析,生成网络是可靠的,则表明判别网络需要进行校正,则通过第二校正loss函数Ld进行校正。本实施例通过比较第一目标人脸图像相对于原始人脸的角度向量的大小,是否在对应的角度分类范围内,若在范围内,说明分类正确。In this embodiment, the generation network is connected with the discrimination network, and the face image output by the generation network is classified by angle to further verify the reliability of the generation network. If the face image corresponding to the angle is classified to the correct angle by the discrimination network Domain indicates that the reliability of the generated network is high, and the reliability of the discrimination network is also high. If the classification is incorrect, but the generation network is reliable after the first correction loss function analysis, it indicates that the discrimination network needs to be corrected, and the second correction loss function Ld is used for correction. In this embodiment, by comparing the magnitude of the angle vector of the first target face image relative to the original face, whether it is within the corresponding angle classification range, and if it is within the range, it indicates that the classification is correct.
进一步地,生成人脸图像的装置,包括:Further, the device for generating a face image includes:
获取模块,用于获取指定数量用户的人脸图像,其中,同一用户的人脸图像至少包括两个指定角度对应的角度图像。The obtaining module is used to obtain face images of a specified number of users, where the face images of the same user include at least two angle images corresponding to the specified angles.
映射模块,用于角度图像对应的指定角度映射为指定长度的向量值。The mapping module is used to map the specified angle corresponding to the angle image to the vector value of the specified length.
组合模块,用于将角度图像与指定角度对应的指定长度的向量值,组合成训练样本。The combination module is used to combine the angle image and the vector value of the specified length corresponding to the specified angle into a training sample.
第二输入模块,用于将训练样本输入至神经网络中进行训练,得到能分类指定角度对应图像域的分类模型。The second input module is used to input training samples into the neural network for training to obtain a classification model that can classify the image domain corresponding to the specified angle.
作为模块,用于将分类模型作为预训练的人脸图像生成网络。As a module, it is used to use the classification model as a pre-trained face image generation network.
本实施例在训练人脸图像生成网络时,同时加入角度向量的参数,生成不同角度域分别对应的人脸图像生成模板。按照不同角度,将人脸图像进行分类,同一角度的放入相同分类,每一个角度使用一个长度唯一的角度向量表示。上述指定数量大于等于1000用户,每个用户对应多个角度的人脸图像,指定角度例如包括-90,-60,-45,-30,0,30,45,60,90等多个角度。In this embodiment, when training the face image generation network, the parameters of the angle vector are added at the same time to generate face image generation templates corresponding to different angle domains. According to different angles, the face images are classified, the same angle is put into the same classification, and each angle is represented by an angle vector with a unique length. The specified number is greater than or equal to 1000 users, and each user corresponds to face images of multiple angles. The specified angles include, for example, -90, -60, -45, -30, 0, 30, 45, 60, 90 and other angles.
进一步地,组合模块,包括:Further, the combination module includes:
第二获取单元,用于获取第一角度图像中人脸器官对应的第一像素值,其中,第一角度图像包含于所有角度图像中。The second acquiring unit is configured to acquire the first pixel value corresponding to the facial organ in the first angle image, where the first angle image is included in all angle images.
第三获取单元,用于获取第一指定角度对应的标准人脸图像。The third acquiring unit is used to acquire the standard face image corresponding to the first designated angle.
修正单元,用于根据标准人脸图像中各人脸器官对应的标准像素值,修正第一像素值,得到修正后的第一像素值。The correction unit is used to correct the first pixel value according to the standard pixel value corresponding to each facial organ in the standard face image to obtain the corrected first pixel value.
组合单元,用于将携带修正后的第一像素值的第二角度图像,与第一角度对应的第一向量值,组合为修正训练样本。The combination unit is used to combine the second angle image carrying the corrected first pixel value and the first vector value corresponding to the first angle into a corrected training sample.
本实施例中,训练样本通过定位修正,以提高训练样本的正确性。本实施例通过裁剪,使人脸图像处于正中央,然后将人脸器官对应的特征点的像素值进行修正。比如标准人脸图像为256*256,则眼睛的标准像素值为58*58,和158*58,通过将当前用户的第一角度图像中人脸器官对应的第一像素值,矫正对准到第一指定角度对应的标准人脸图像的标准像素值,其他人脸器官对应的 特征点如上过程对应校正,以减小样本差异度大,对人脸图像生成网络的训练精度影响,提高生成网络同步输出多个角度分别对应的人脸图像的精准性。In this embodiment, the training samples are corrected by positioning to improve the accuracy of the training samples. In this embodiment, the face image is centered by cropping, and then the pixel values of the feature points corresponding to the facial organs are corrected. For example, if the standard face image is 256*256, the standard pixel values of the eyes are 58*58, and 158*58. By correcting the first pixel value corresponding to the facial organ in the current user’s first angle image, the correction is aligned to The standard pixel value of the standard face image corresponding to the first specified angle, and the feature points corresponding to other face organs are correspondingly corrected in the above process to reduce the degree of sample difference, the impact on the training accuracy of the face image generation network, and improve the generation network Simultaneously output the accuracy of face images corresponding to multiple angles.
参照图3,本申请实施例中还提供一种计算机设备,该计算机设备可以是服务器,其内部结构可以如图3所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中,该计算机设计的处理器用于提供计算和控制能力。该计算机设备的存储器包括可读存储介质、内存储器。该可读存储介质存储有操作系统、计算机可读指令和数据库,上述可读存储介质包括非易失性可读存储介质和易失性可读存储介质。该内存器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储生成的人脸图像等数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令在执行时,执行如上述各方法的实施例的流程。本领域技术人员可以理解,图3中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定。Referring to FIG. 3, an embodiment of the present application also provides a computer device. The computer device may be a server, and its internal structure may be as shown in FIG. 3. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor designed by the computer is used to provide calculation and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The above-mentioned readable storage medium includes a non-volatile readable storage medium and a volatile readable storage medium. The memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer equipment is used to store the generated face images and other data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instruction is executed, it executes the process of the above-mentioned method embodiment. Those skilled in the art can understand that the structure shown in FIG. 3 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
本申请一实施例还提供一种计算机可读存储介质,其上存储有计算机可读指令,该计算机可读指令在执行时,执行如上述各方法的实施例的流程。上述可读存储介质包括非易失性可读存储介质和易失性可读存储介质。以上所述仅为本申请的优选实施例,并非因此限制本申请的专利范围,凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本申请的专利保护范围内。An embodiment of the present application also provides a computer-readable storage medium on which computer-readable instructions are stored. When the computer-readable instructions are executed, the processes of the foregoing method embodiments are executed. The above-mentioned readable storage medium includes non-volatile readable storage medium and volatile readable storage medium. The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the specification and drawings of this application, or directly or indirectly applied to other related The technical field is equally included in the scope of patent protection of this application.

Claims (20)

  1. 一种生成人脸图像的方法,其特征在于,包括:A method for generating a face image, characterized in that it includes:
    将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,所述目标人脸图像的角度向量至少包括一个;Input the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time, wherein the angle vector of the target face image includes at least one;
    通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,所述第一角度向量包含于所有所述目标人脸图像对应的角度向量中,所述第一目标人脸图像包含于所有所述目标人脸图像中;Through the pre-trained face image generation network, the original face image is converted into a first target face image corresponding to a first angle vector according to a preset conversion method, wherein the first angle vector is included in all In the angle vector corresponding to the target face image, the first target face image is included in all the target face images;
    通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配;Evaluating whether the first angle vector matches the first target face image through a preset discrimination network;
    若匹配,则输出所述第一目标人脸图像。If it matches, output the first target face image.
  2. 根据权利要求1所述的生成人脸图像的方法,其特征在于,所述通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像的步骤,包括:The method for generating a face image according to claim 1, wherein the pre-trained face image generation network converts the original face image into a first angle vector according to a preset conversion method The corresponding steps of the first target face image include:
    从所述原始人脸图像中裁剪出人脸头部区域图像;Cropping out an image of a human face head region from the original human face image;
    根据特征点定位模型定位人脸的指定器官对应的特征点,其中,所述指定器官至少包括一种;Locating the feature points corresponding to the designated organs of the face according to the feature point positioning model, wherein the designated organs include at least one type;
    获取各所述指定器官分别对应的特征点的坐标数据;Acquiring coordinate data of feature points corresponding to each of the designated organs;
    将各所述坐标数据一一对应映射到所述第一角度向量对应的特征点模板中,得到所述第一角度向量对应的人脸区域;Mapping each of the coordinate data to the feature point template corresponding to the first angle vector in a one-to-one correspondence to obtain the face area corresponding to the first angle vector;
    根据所述第一角度向量对应的人脸区域,形成所述第一目标人脸图像。The first target face image is formed according to the face area corresponding to the first angle vector.
  3. 根据权利要求1所述的生成人脸图像的方法,其特征在于,所述通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配的步骤,包括:The method for generating a face image according to claim 1, wherein the step of evaluating whether the first angle vector matches the first target face image through a preset discrimination network comprises:
    将所述第一角度向量的逆角度与所述第一目标人脸图像,作为输入量输入至所述预训练的人脸图像生成网络;Inputting the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network;
    根据所述第一角度向量的逆角度与所述第一目标人脸图像,生成第二目标人脸图像;Generating a second target face image according to the inverse angle of the first angle vector and the first target face image;
    判断所述第二目标人脸图像是否与所述原始人脸图像相同;Judging whether the second target face image is the same as the original face image;
    若相同,则判定所述第一角度向量与所述第一目标人脸图像匹配。If they are the same, it is determined that the first angle vector matches the first target face image.
  4. 根据权利要求3所述的生成人脸图像的方法,其特征在于,所述判断 所述第二目标人脸图像是否与所述原始人脸图像相同的步骤之后,包括:The method for generating a face image according to claim 3, wherein after the step of determining whether the second target face image is the same as the original face image, the method comprises:
    若所述第二目标人脸图像与所述原始人脸图像不相同,则确定所述第二目标人脸图像与所述原始人脸图像的差异信息;If the second target face image is not the same as the original face image, determine the difference information between the second target face image and the original face image;
    根据所述差异信息生成第一校正loss函数,其中,所述第一校正loss函数为Lg=Ladv+Lfcls+Lrec,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),Lrec=||x-G(G(x,c),c’)||,G表示人脸图像生成网络,G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数,c’=-c;A first correction loss function is generated according to the difference information, where the first correction loss function is Lg=Ladv+Lfcls+Lrec, Ladv=logD(x)+log(1-D(G(x,c)) , Lfcls=-logD(G(x,c)), Lrec=||xG(G(x,c),c')||, G represents the face image generation network, G(x,c) represents the input image x, the corresponding face image when the generated angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, c'=-c;
    根据所述第一校正loss函数校正所述预训练的人脸图像生成网络中的参量。The parameters in the pre-trained face image generation network are corrected according to the first correction loss function.
  5. 根据权利要求1所述的生成人脸图像的方法,其特征在于,所述输出所述第一目标人脸图像的步骤之后,包括:The method for generating a face image according to claim 1, wherein after the step of outputting the first target face image, the method comprises:
    判断对各所述第一目标人脸图像分别对应的角度分类是否分类正确;Judging whether the angle classification corresponding to each of the first target face images is correct;
    若否,则通过第二校正loss函数校正所述判别网络,其中,第二校正loss函数Ld=-Ladv+Lfcls,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数。If not, the discrimination network is corrected by the second correction loss function, where the second correction loss function Ld=-Ladv+Lfcls, Ladv=logD(x)+log(1-D(G(x,c)) , Lfcls=-logD(G(x,c)), G(x,c) represents the input image x, when the generated angle is c, the corresponding face image, D(x) represents the domain calculated by D on the real image Label probability distribution, D represents the discriminant network function.
  6. 根据权利要求1所述的生成人脸图像的方法,其特征在于,所述将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中的步骤之前,包括:The method for generating a face image according to claim 1, wherein before the step of simultaneously inputting the angle vectors of the original face image and the target face image into the pre-trained face image generation network, include:
    获取指定数量用户的人脸图像,其中,同一所述用户的人脸图像至少包括两个指定角度对应的角度图像;Acquiring face images of a specified number of users, where the face images of the same user include at least two angle images corresponding to specified angles;
    所述角度图像对应的指定角度映射为指定长度的向量值;The specified angle corresponding to the angle image is mapped to a vector value of a specified length;
    将所述角度图像与所述指定角度对应的指定长度的向量值,组合成训练样本;Combining the angle image and the specified length vector value corresponding to the specified angle into a training sample;
    将所述训练样本输入至神经网络中进行训练,得到能分类所述指定角度对应图像域的分类模型;Inputting the training samples into a neural network for training to obtain a classification model that can classify the image domain corresponding to the specified angle;
    将所述分类模型作为所述预训练的人脸图像生成网络。The classification model is used as the pre-trained face image generation network.
  7. 根据权利要求6所述的生成人脸图像的方法,其特征在于,所述将所述角度图像与所述指定角度对应的指定长度的向量值,组合成训练样本的步骤, 包括:The method for generating a face image according to claim 6, wherein the step of combining the angle image and the specified length vector value corresponding to the specified angle into a training sample comprises:
    获取第一角度图像中人脸器官对应的第一像素值,其中,所述第一角度图像包含于所有所述角度图像中;Acquiring a first pixel value corresponding to a human face organ in a first angle image, where the first angle image is included in all the angle images;
    获取第一指定角度对应的标准人脸图像;Acquiring a standard face image corresponding to the first specified angle;
    根据所述标准人脸图像中各人脸器官对应的标准像素值,修正所述第一像素值,得到修正后的第一像素值;Correcting the first pixel value according to the standard pixel value corresponding to each facial organ in the standard face image to obtain the corrected first pixel value;
    将携带所述修正后的第一像素值的第二角度图像,与所述第一角度对应的第一向量值,组合为修正训练样本。The second angle image carrying the corrected first pixel value and the first vector value corresponding to the first angle are combined into a corrected training sample.
  8. 一种生成人脸图像的装置,其特征在于,包括:A device for generating a face image, characterized in that it comprises:
    第一输入模块,用于将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,所述目标人脸图像的角度向量至少包括一个;The first input module is configured to simultaneously input the angle vectors of the original face image and the target face image into the pre-trained face image generation network, wherein the angle vector of the target face image includes at least one;
    转换模块,用于通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,所述第一角度向量包含于所有所述目标人脸图像对应的角度向量中,所述第一目标人脸图像包含于所有所述目标人脸图像中;The conversion module is configured to use the pre-trained face image generation network to convert the original face image into a first target face image corresponding to a first angle vector according to a preset conversion method, wherein the first The angle vector is included in the angle vectors corresponding to all the target face images, and the first target face image is included in all the target face images;
    评价模块,用于通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配;An evaluation module, configured to evaluate whether the first angle vector matches the first target face image through a preset discrimination network;
    输出模块,用于若匹配,则输出所述第一目标人脸图像。The output module is configured to output the first target face image if it matches.
  9. 根据权利要求8所述的生成人脸图像的装置,其特征在于,所述转换模块,包括:The apparatus for generating a face image according to claim 8, wherein the conversion module comprises:
    裁剪单元,用于从所述原始人脸图像中裁剪出人脸头部区域图像;A cropping unit, configured to crop an image of the head region of the face from the original face image;
    定位单元,用于根据特征点定位模型定位人脸的指定器官对应的特征点,其中,所述指定器官至少包括一种;The positioning unit is configured to locate the characteristic points corresponding to the designated organs of the face according to the characteristic point positioning model, wherein the designated organs include at least one type;
    第一获取单元,用于获取各所述指定器官分别对应的特征点的坐标数据;The first acquiring unit is configured to acquire the coordinate data of the characteristic points corresponding to each of the designated organs;
    映射单元,用于将各所述坐标数据一一对应映射到所述第一角度向量对应的特征点模板中,得到所述第一角度向量对应的人脸区域;A mapping unit, configured to map each of the coordinate data to the feature point template corresponding to the first angle vector in a one-to-one correspondence to obtain the face area corresponding to the first angle vector;
    形成单元,用于根据所述第一角度向量对应的人脸区域,形成所述第一目标人脸图像。The forming unit is configured to form the first target face image according to the face area corresponding to the first angle vector.
  10. 根据权利要求8所述的生成人脸图像的装置,其特征在于,所述评价模块,包括:The apparatus for generating a face image according to claim 8, wherein the evaluation module comprises:
    输入单元,用于将所述第一角度向量的逆角度与所述第一目标人脸图像,作为输入量输入至所述预训练的人脸图像生成网络;An input unit, configured to input the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network;
    第一生成单元,用于根据所述第一角度向量的逆角度与所述第一目标人脸图像,生成第二目标人脸图像;A first generating unit, configured to generate a second target face image according to the inverse angle of the first angle vector and the first target face image;
    判断单元,用于判断所述第二目标人脸图像是否与所述原始人脸图像相同;A judging unit, configured to judge whether the second target face image is the same as the original face image;
    判定单元,用于若相同,则判定所述第一角度向量与所述第一目标人脸图像匹配。The determining unit is configured to determine that the first angle vector matches the first target face image if they are the same.
  11. 根据权利要求10所述的生成人脸图像的装置,其特征在于,所述评价模块,包括:The apparatus for generating a face image according to claim 10, wherein the evaluation module comprises:
    确定单元,用于若所述第二目标人脸图像与所述原始人脸图像不相同,则确定所述第二目标人脸图像与所述原始人脸图像的差异信息;A determining unit, configured to determine difference information between the second target face image and the original face image if the second target face image is different from the original face image;
    第二生成单元,用于根据所述差异信息生成第一校正loss函数,其中,所述第一校正loss函数为Lg=Ladv+Lfcls+Lrec,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),Lrec=||x-G(G(x,c),c’)||,G表示人脸图像生成网络,G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数,c’=-c;The second generating unit is configured to generate a first correction loss function according to the difference information, where the first correction loss function is Lg=Ladv+Lfcls+Lrec, Ladv=logD(x)+log(1-D( G(x,c)), Lfcls=-logD(G(x,c)), Lrec=||xG(G(x,c),c')||, G represents the face image generation network, G( x,c) represents the input image x, the corresponding face image when the generated angle is c, D(x) represents the probability distribution of the domain label calculated by D on the real image, D represents the discriminant network function, c'=-c;
    校正单元,用于根据所述第一校正loss函数校正所述预训练的人脸图像生成网络中的参量。The correction unit is configured to correct the parameters in the pre-trained face image generation network according to the first correction loss function.
  12. 根据权利要求8所述的生成人脸图像的装置,其特征在于,包括:The apparatus for generating a face image according to claim 8, characterized in that it comprises:
    判断模块,用于判断对各所述第一目标人脸图像分别对应的角度分类是否分类正确;A judging module for judging whether the angle classification corresponding to each of the first target face images is correct;
    校正模块,用于若否,则通过第二校正loss函数校正所述判别网络,其中,第二校正loss函数Ld=-Ladv+Lfcls,Ladv=logD(x)+log(1-D(G(x,c)),Lfcls=-logD(G(x,c)),G(x,c)表示输入图像x,在生成角度为c时对应的人脸图像,D(x)代表D对真实图像计算得到的域标签概率分布,D表示判别网络函数。The correction module is used to correct the discrimination network through a second correction loss function, if not, where the second correction loss function Ld=-Ladv+Lfcls, Ladv=logD(x)+log(1-D(G( x,c)), Lfcls=-logD(G(x,c)), G(x,c) represents the input image x, when the generated angle is c, the corresponding face image, D(x) represents the real The probability distribution of the domain label obtained by the image calculation, D represents the discriminant network function.
  13. 根据权利要求8所述的生成人脸图像的装置,其特征在于,包括:The apparatus for generating a face image according to claim 8, characterized in that it comprises:
    获取模块,用于获取指定数量用户的人脸图像,其中,同一所述用户的人脸图像至少包括两个指定角度对应的角度图像;An acquiring module for acquiring face images of a specified number of users, wherein the face images of the same user include at least two angle images corresponding to the specified angles;
    映射模块,用于所述角度图像对应的指定角度映射为指定长度的向量值;A mapping module for mapping a specified angle corresponding to the angle image to a vector value of a specified length;
    组合模块,用于将所述角度图像与所述指定角度对应的指定长度的向量值,组合成训练样本;A combination module, configured to combine the angle image and the specified length vector value corresponding to the specified angle into a training sample;
    第二输入模块,用于将所述训练样本输入至神经网络中进行训练,得到能分类所述指定角度对应图像域的分类模型;The second input module is configured to input the training samples into the neural network for training, and obtain a classification model that can classify the image domain corresponding to the specified angle;
    作为模块,用于将所述分类模型作为所述预训练的人脸图像生成网络。As a module, it is used to use the classification model as the pre-trained face image generation network.
  14. 根据权利要求13所述的生成人脸图像的装置,其特征在于,所述组合模块,包括:The apparatus for generating a face image according to claim 13, wherein the combination module comprises:
    第二获取单元,用于获取第一角度图像中人脸器官对应的第一像素值,其中,所述第一角度图像包含于所有所述角度图像中;The second acquiring unit is configured to acquire the first pixel value corresponding to the facial organ in the first angle image, wherein the first angle image is included in all the angle images;
    第三获取单元,用于获取第一指定角度对应的标准人脸图像;The third acquiring unit is configured to acquire a standard face image corresponding to the first designated angle;
    修正单元,用于根据所述标准人脸图像中各人脸器官对应的标准像素值,修正所述第一像素值,得到修正后的第一像素值;The correction unit is configured to correct the first pixel value according to the standard pixel value corresponding to each facial organ in the standard face image to obtain the corrected first pixel value;
    组合单元,用于将携带所述修正后的第一像素值的第二角度图像,与所述第一角度对应的第一向量值,组合为修正训练样本。The combination unit is configured to combine the second angle image carrying the corrected first pixel value and the first vector value corresponding to the first angle into a corrected training sample.
  15. 一种计算机设备,包括存储器和处理器,所述存储器存储有计算机程序,其特征在于,所述处理器执行所述计算机程序时实现生成人脸图像的方法,生成人脸图像的方法,包括:A computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that, when the processor executes the computer program, a method for generating a face image is realized, and the method for generating a face image includes:
    将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,所述目标人脸图像的角度向量至少包括一个;Input the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time, wherein the angle vector of the target face image includes at least one;
    通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,所述第一角度向量包含于所有所述目标人脸图像对应的角度向量中,所述第一目标人脸图像包含于所有所述目标人脸图像中;Through the pre-trained face image generation network, the original face image is converted into a first target face image corresponding to a first angle vector according to a preset conversion method, wherein the first angle vector is included in all In the angle vector corresponding to the target face image, the first target face image is included in all the target face images;
    通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配;Evaluating whether the first angle vector matches the first target face image through a preset discrimination network;
    若匹配,则输出所述第一目标人脸图像。If it matches, output the first target face image.
  16. 根据权利要求15所述的计算机设备,其特征在于,所述通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像的步骤,包括:The computer device according to claim 15, wherein the pre-trained face image generation network converts the original face image into a first angle vector corresponding to a first angle vector according to a preset conversion method. The steps of the target face image include:
    从所述原始人脸图像中裁剪出人脸头部区域图像;Cropping out an image of a human face head region from the original human face image;
    根据特征点定位模型定位人脸的指定器官对应的特征点,其中,所述指定 器官至少包括一种;Locating the feature points corresponding to the designated organs of the face according to the feature point positioning model, wherein the designated organs include at least one type;
    获取各所述指定器官分别对应的特征点的坐标数据;Acquiring coordinate data of feature points corresponding to each of the designated organs;
    将各所述坐标数据一一对应映射到所述第一角度向量对应的特征点模板中,得到所述第一角度向量对应的人脸区域;Mapping each of the coordinate data to the feature point template corresponding to the first angle vector in a one-to-one correspondence to obtain the face area corresponding to the first angle vector;
    根据所述第一角度向量对应的人脸区域,形成所述第一目标人脸图像。The first target face image is formed according to the face area corresponding to the first angle vector.
  17. 根据权利要求15所述的计算机设备,其特征在于,所述通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配的步骤,包括:15. The computer device according to claim 15, wherein the step of evaluating whether the first angle vector matches the first target face image through a preset discrimination network comprises:
    将所述第一角度向量的逆角度与所述第一目标人脸图像,作为输入量输入至所述预训练的人脸图像生成网络;Inputting the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network;
    根据所述第一角度向量的逆角度与所述第一目标人脸图像,生成第二目标人脸图像;Generating a second target face image according to the inverse angle of the first angle vector and the first target face image;
    判断所述第二目标人脸图像是否与所述原始人脸图像相同;Judging whether the second target face image is the same as the original face image;
    若相同,则判定所述第一角度向量与所述第一目标人脸图像匹配。If they are the same, it is determined that the first angle vector matches the first target face image.
  18. 一种计算机可读存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现生成人脸图像的方法,生成人脸图像的方法包括:A computer-readable storage medium with a computer program stored thereon, characterized in that, when the computer program is executed by a processor, a method for generating a face image is realized, and the method for generating a face image includes:
    将原始人脸图像和目标人脸图像的角度向量,同时输入至预训练的人脸图像生成网络中,其中,所述目标人脸图像的角度向量至少包括一个;Input the angle vectors of the original face image and the target face image into the pre-trained face image generation network at the same time, wherein the angle vector of the target face image includes at least one;
    通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像,其中,所述第一角度向量包含于所有所述目标人脸图像对应的角度向量中,所述第一目标人脸图像包含于所有所述目标人脸图像中;Through the pre-trained face image generation network, the original face image is converted into a first target face image corresponding to a first angle vector according to a preset conversion method, wherein the first angle vector is included in all In the angle vector corresponding to the target face image, the first target face image is included in all the target face images;
    通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配;Evaluating whether the first angle vector matches the first target face image through a preset discrimination network;
    若匹配,则输出所述第一目标人脸图像。If it matches, output the first target face image.
  19. 根据权利要求18所述的计算机可读存储介质,其特征在于,所述通过所述预训练的人脸图像生成网络,按照预设转换方式将所述原始人脸图像转换为第一角度向量对应的第一目标人脸图像的步骤,包括:The computer-readable storage medium of claim 18, wherein the pre-trained face image generation network converts the original face image into a first angle vector corresponding to a preset conversion method The steps of the first target face image include:
    从所述原始人脸图像中裁剪出人脸头部区域图像;Cropping out an image of a human face head region from the original human face image;
    根据特征点定位模型定位人脸的指定器官对应的特征点,其中,所述指定器官至少包括一种;Locating the feature points corresponding to the designated organs of the face according to the feature point positioning model, wherein the designated organs include at least one type;
    获取各所述指定器官分别对应的特征点的坐标数据;Acquiring coordinate data of feature points corresponding to each of the designated organs;
    将各所述坐标数据一一对应映射到所述第一角度向量对应的特征点模板中,得到所述第一角度向量对应的人脸区域;Mapping each of the coordinate data to the feature point template corresponding to the first angle vector in a one-to-one correspondence to obtain the face area corresponding to the first angle vector;
    根据所述第一角度向量对应的人脸区域,形成所述第一目标人脸图像。The first target face image is formed according to the face area corresponding to the first angle vector.
  20. 根据权利要求18所述的计算机可读存储介质,其特征在于,所述通过预设判别网络评价所述第一角度向量与所述第一目标人脸图像是否匹配的步骤,包括:18. The computer-readable storage medium according to claim 18, wherein the step of evaluating whether the first angle vector matches the first target face image through a preset discrimination network comprises:
    将所述第一角度向量的逆角度与所述第一目标人脸图像,作为输入量输入至所述预训练的人脸图像生成网络;Inputting the inverse angle of the first angle vector and the first target face image as input to the pre-trained face image generation network;
    根据所述第一角度向量的逆角度与所述第一目标人脸图像,生成第二目标人脸图像;Generating a second target face image according to the inverse angle of the first angle vector and the first target face image;
    判断所述第二目标人脸图像是否与所述原始人脸图像相同;Judging whether the second target face image is the same as the original face image;
    若相同,则判定所述第一角度向量与所述第一目标人脸图像匹配。If they are the same, it is determined that the first angle vector matches the first target face image.
PCT/CN2019/116707 2019-09-19 2019-11-08 Method and apparatus for generating face image, computer device, and storage medium WO2021051510A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910886618.5A CN110781738A (en) 2019-09-19 2019-09-19 Method and device for generating face image, computer equipment and storage medium
CN201910886618.5 2019-09-19

Publications (1)

Publication Number Publication Date
WO2021051510A1 true WO2021051510A1 (en) 2021-03-25

Family

ID=69383588

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/116707 WO2021051510A1 (en) 2019-09-19 2019-11-08 Method and apparatus for generating face image, computer device, and storage medium

Country Status (2)

Country Link
CN (1) CN110781738A (en)
WO (1) WO2021051510A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111401459A (en) * 2020-03-24 2020-07-10 谷元(上海)文化科技有限责任公司 Animation figure form change vision capture system
CN111652064B (en) * 2020-04-30 2024-06-07 平安科技(深圳)有限公司 Face image generation method, electronic device and readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160125572A1 (en) * 2014-11-05 2016-05-05 Korea Advanced Institute Of Science And Technology Device and method to generate image using image learning model
CN107292813A (en) * 2017-05-17 2017-10-24 浙江大学 A kind of multi-pose Face generation method based on generation confrontation network
CN107506702A (en) * 2017-08-08 2017-12-22 江西高创保安服务技术有限公司 Human face recognition model training and test system and method based on multi-angle
CN108776983A (en) * 2018-05-31 2018-11-09 北京市商汤科技开发有限公司 Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network
CN109635745A (en) * 2018-12-13 2019-04-16 广东工业大学 A method of Multi-angle human face image is generated based on confrontation network model is generated

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6751684B2 (en) * 2017-03-28 2020-09-09 株式会社Nttドコモ Similar image search device
CN108509915B (en) * 2018-04-03 2021-10-26 百度在线网络技术(北京)有限公司 Method and device for generating face recognition model

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160125572A1 (en) * 2014-11-05 2016-05-05 Korea Advanced Institute Of Science And Technology Device and method to generate image using image learning model
CN107292813A (en) * 2017-05-17 2017-10-24 浙江大学 A kind of multi-pose Face generation method based on generation confrontation network
CN107506702A (en) * 2017-08-08 2017-12-22 江西高创保安服务技术有限公司 Human face recognition model training and test system and method based on multi-angle
CN108776983A (en) * 2018-05-31 2018-11-09 北京市商汤科技开发有限公司 Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network
CN109635745A (en) * 2018-12-13 2019-04-16 广东工业大学 A method of Multi-angle human face image is generated based on confrontation network model is generated

Also Published As

Publication number Publication date
CN110781738A (en) 2020-02-11

Similar Documents

Publication Publication Date Title
EP1677250B1 (en) Image collation system and image collation method
EP3373202B1 (en) Verification method and system
US11403757B2 (en) Sight line detection method and sight line detection device
US7706601B2 (en) Object posture estimation/correlation system using weight information
US6430307B1 (en) Feature extraction system and face image recognition system
US7925060B2 (en) Authentication system and registration system related to facial feature information
Shan et al. Curse of mis-alignment in face recognition: problem and a novel mis-alignment learning solution
US20150302240A1 (en) Method and device for locating feature points on human face and storage medium
WO2019033574A1 (en) Electronic device, dynamic video face recognition method and system, and storage medium
WO2020006964A1 (en) Image detection method and device
CN112017212B (en) Training and tracking method and system of face key point tracking model
JPWO2018189795A1 (en) Recognition device, recognition method, and recognition program
WO2021051510A1 (en) Method and apparatus for generating face image, computer device, and storage medium
CN110448870A (en) A kind of human body attitude training method
WO2015165227A1 (en) Human face recognition method
WO2020223940A1 (en) Posture prediction method, computer device and storage medium
CN109344714A (en) One kind being based on the matched gaze estimation method of key point
WO2021212608A1 (en) Method and apparatus for positioning sound source user, and computer device
JP2007052575A (en) Metadata applying device and metadata applying method
CN112950759A (en) Three-dimensional house model construction method and device based on house panoramic image
WO2020237941A1 (en) Personnel state detection method and apparatus based on eyelid feature information
CN116597246A (en) Model training method, target detection method, electronic device and storage medium
CN115665361A (en) Video fusion method in virtual environment and online video conference communication method
WO2021084662A1 (en) Checking assistance device, checking assistance method, and computer-readable recording medium
CN117935369A (en) Examinee behavior detection auxiliary invigilation method and device based on cloud examination

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19945539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19945539

Country of ref document: EP

Kind code of ref document: A1