WO2021051543A1

WO2021051543A1 - Method for generating face rotation model, apparatus, computer device and storage medium

Info

Publication number: WO2021051543A1
Application number: PCT/CN2019/117299
Authority: WO
Inventors: 田笑; 陈嘉莉; 周超勇; 刘玉宇
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-09-18
Filing date: 2019-11-12
Publication date: 2021-03-25
Also published as: CN110826395A; CN110826395B

Abstract

A method for generating a face rotation model, an apparatus, a computer device and a storage medium. The method comprises: acquiring a face training image; acquiring a real position map according to three-dimensional face parameters and a preset average face model; extracting target features in the face training image, inputting the target features into a face rotation model, and acquiring three-dimensional position information of all second key points outputted by the face rotation model; using a mask to configure a weight value occupied by three-dimensional position information of each first key point and three-dimensional position information of each second key point in an average error loss function, and inputting the three-dimensional position information of each first key point and the three-dimensional position information of each second key point into the average error loss function to obtain a loss result; and when the loss result is less than or equal to a preset loss value, confirming that a face rotation model is completely trained. By using the described method, the angle of rotation required to rotate a face photograph to be a front-view face photograph can be accurately calculated.

Description

Method, device, computer equipment and storage medium for generating face rotation model

This application is based on the Chinese patent application filed on September 18, 2019, with the application number 201910882239.9 and titled "Generation Method, Device, Computer Equipment and Storage Medium of Face Rotation Model", and claims priority.

Technical field

This application relates to the field of model construction, and in particular to a method, device, computer equipment and storage medium for generating a face rotation model.

Background technique

For non-frontal faces, the face recognition technology in the field of artificial intelligence is not mature enough, and the face recognition technology for non-frontal faces has always been a problem in the field of face recognition. The face alignment technology of non-frontal faces in the existing face recognition system can usually only solve the rotation of the roll angle in the face pose, and the similar transformation is directly used for the large-angle yaw angle or the pitch angle. In this way, The face will be deformed greatly after the alignment, which is not conducive to subsequent face recognition; at the same time, the face alignment of a non-frontal face in the prior art cannot obtain accurate three-dimensional coordinate information of the face. Therefore, finding a technical solution that can solve the above-mentioned problems has become an urgent problem for those skilled in the art.

Summary of the invention

Based on this, it is necessary to provide a method, device, computer equipment, and storage medium for generating a face rotation model to solve the above-mentioned technical problems, so as to solve the current problem that the rotation angle required for a face photo to be rotated into a front face photo cannot be calculated more accurately. .

A method for generating a face rotation model, including:

Obtain face training pictures from the target set; three-dimensional face parameters have been marked in the face training pictures;

Obtain a preset average face model, and obtain a real position map according to the three-dimensional face parameters and the preset average face model; the real position map includes the face image in the face training picture The corresponding front face and the three-dimensional position information of all first key points of the front face;

Extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain all second key points output by the face rotation model 3D location information;

The three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function are set through a mask, and the value of each of the first key points is The three-dimensional position information and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

When the loss result is less than or equal to the preset loss value, it is confirmed that the face rotation model training is completed; the face rotation model is used to confirm the face when the face photo is input into the face rotation model The photo rotation is the required rotation angle for the front face photo.

A device for generating a face rotation model, including:

The first acquisition module is configured to acquire face training pictures from the target set; the face training pictures have already marked three-dimensional face parameters;

The second acquiring module is configured to acquire a preset average face model, and acquire a real position map according to the three-dimensional face parameters and the preset average face model; the real position map contains the same face as the face model. Three-dimensional position information of the front face corresponding to the face image in the training picture and all first key points of the front face;

The third acquisition module is configured to extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain the face rotation model The three-dimensional position information of all the output second key points;

The input module is used to set the three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function through a mask, and set each of the The three-dimensional position information of the first key point and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

The first confirmation module is used to confirm that the face rotation model training is completed when the loss result is less than or equal to the preset loss value; the face rotation model is used to input the face photo into the face rotation model , Confirm that the face photo is rotated to the required rotation angle for the front face photo.

A computer device includes a memory, a processor, and computer-readable instructions that are stored in the memory and can run on the processor, and the processor implements the following steps when the processor executes the computer-readable instructions:

One or more readable storage media storing computer readable instructions, when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

The details of one or more embodiments of the present application are set forth in the following drawings and description, and other features and advantages of the present application will become apparent from the description, drawings, and claims.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic diagram of an application environment of a method for generating a face rotation model in an embodiment of the present application;

2 is a schematic flowchart of a method for generating a face rotation model in an embodiment of the present application;

3 is a schematic flowchart of step S20 of the method for generating a face rotation model in an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an apparatus for generating a face rotation model in an embodiment of the present application;

Fig. 5 is a schematic diagram of a computer device in an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The method for generating a face rotation model provided in this application can be applied in an application environment as shown in FIG. 1, where the client communicates with the server through the network. Among them, the client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In an embodiment, as shown in FIG. 2, a method for generating a face rotation model is provided. Taking the method applied to the server in FIG. 1 as an example for description, the method includes the following steps:

S10: Obtain a face training picture from a target set; the face training picture has been marked with three-dimensional face parameters;

Understandably, the target set can be a large public face data set (such as the 300W-LP public set, which contains more than 60,000 human pictures, and each face training picture has annotated three-dimensional face parameters. Face parameters include face shape parameters, facial expression parameters, and face pose parameters). At this time, the face photos in the face data set can be used as face training photos; face training images include face rotation The 43867 face vertices required in the model modeling process (the three-dimensional coordinate information corresponding to the face vertices has not been determined yet).

S20. Obtain a preset average face model, and obtain a real position map according to the three-dimensional face parameters and the preset average face model; the real position map contains the person in the face training picture The front face corresponding to the face image and the three-dimensional position information of all first key points of the front face;

Understandably, the classic model Basel Face Model in the successfully trained and open source 3DMM model includes an average face model, and the average face model can represent a standard average face (and the average face model data comes from ：Facial features extracted from a preset number of ordinary faces. According to the measurement of facial features, each measurement data is obtained. Finally, each average value is calculated based on each measurement data, that is, each average value can be used as the data of the composite average face model ); The three-dimensional face parameters can be compared with the model parameters in the average face model; the real position map is a standard frontal face picture. The real position map is the same size as the face training picture, and both can be 256x256x3( 256x256 represents pixels in the horizontal and vertical directions respectively), this real position map will be used as the learning target in this training process; the three-dimensional position information refers to the three-dimensional coordinates in the three-dimensional coordinate system.

The real position map obtained in this embodiment can be used as a learning target in the training process of the face rotation model. Through the learning target, the initial parameters of the face rotation model can be adjusted in the subsequent training process, so that a computable face photo rotation can be obtained. The face rotation model of the rotation angle required for the front face photo.

Further, as shown in FIG. 3, the three-dimensional face parameters include face shape parameters, facial expression parameters, and face pose parameters; the obtaining a preset average face model is based on the three-dimensional face parameters and The preset average face model to obtain the real position map includes:

S201. Obtain the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to a shape composed of a preset number of three-dimensional position information of face vertices; The model parameters include model shape parameters and model expression parameters;

S202: Determine the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

S203: Determine the expression change of the face image in the face training picture according to the face expression parameter and the model expression parameter;

S204: Determine, according to the average face shape, the deformation of the face image, and the expression change of the face image, that the face image corresponds to each of the faces in the preset average face model Three-dimensional position information of vertices;

S205: According to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model, the face training pictures are similar Transform to the frontal position to obtain the real position map of the frontal face at the frontal position corresponding to the face image in the face training picture.

Understandably, the preset model parameters of the average face model include model shape parameters and model expression parameters. The model shape parameters and model expression parameters are used to reflect the appearance and state of the corresponding average face in the average face model. , That is, what the shape of the average face of most people should look like (the outline of the entire face and the size of the face are the standard average face), and what should the expression of the average face of most people look like (face keeping Slight smile or non-smiling face, etc.); the preset average face model corresponds to an average face shape, and the average face shape is an overall face shape (including the shape of various facial sense organs, such as mouth, eyes and nose, etc.) , Each vertex on the average face shape can be represented by three-dimensional position information, and the average face shape is first composed of each face vertex and line (the line between the face vertex and the face vertex) to form each two-dimensional Plane, and then assembled by each two-dimensional plane.

Specifically, the preset model parameters of the average face model can be used to determine the difference between the three-dimensional face parameters in the face training picture and the standard parameters. Firstly, by comparing the face shape parameters with the model shape parameters The difference determines the deformation of the face image in the face training picture (the face shape parameters in the face training picture can include multiple, such as the national character face and the melon face), by comparing the facial expression parameters and the model expression The parameters determine the expression changes of the face image (the facial expression parameters in the face training images can include multiple, such as angry and happy, etc. The expression changes will affect the shape of the various sense organs in the entire face). After determining these two changes, the average face shape and the deformation of the face image and the expression change of the face image can be used to determine the face image in the face training relative to the face in the preset average face model. The three-dimensional position information of the vertex, for example, the average face shape corresponds to a non-smiling facial expression, that is, the corners of the mouth will not rise or stretch out. If the face image in the face training image corresponds to a smiling expression at this time, that is The corners of the mouth have been raised or stretched. Since there are a preset number (43867) of face vertices on the average face shape, and each vertex has a corresponding three-dimensional position information, the training picture can be determined at this time The face vertices where the face image has been raised or stretched, so that the three-dimensional position information can be determined. Through the above example, it can be determined that each facial sensory in the face image corresponds to the face vertices in the preset average face model The three-dimensional location information. Finally, the face pose parameters in the three-dimensional face parameters (the face should be presented in the form of a front pose in the face image) and the face image corresponding to each face vertex in the preset average face model The three-dimensional position information determines the real position map of the frontal face corresponding to the face image in the face training picture at the frontal position, that is, by similarly transforming the face training picture to the frontal position, for example, as mentioned in the above example The face image of the face training picture is not in the front position (that is, there is a certain deviation from the front position). At this time, similar transformation can be used to determine the three-dimensional position information of the transformed first key point (located on the real position map). ), the real position map of the front position can also be obtained.

In this embodiment, the real position map obtained by the above method steps has accurate three-dimensional position information of the first key point, which provides an accurate reference for the subsequent steps to accurately calculate the rotation angle.

S30. Extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain all second features output by the face rotation model. Three-dimensional position information of key points;

Understandably, the face training image contains the data to be input to the face rotation model that contains the initial parameters. This data can be extracted and used as the target feature of the face image (the target feature still exists in the form of a picture) , The target feature can be the key points needed to construct the various sensory areas of the face (such as constructing the key points of the eye area, constructing the key points of the nose area, and constructing the key points of the mouth area, etc.); the face rotation model is The classic model in the 3DMM model, Basel Face Model.

Specifically, in the overall network structure (think of it as a Stacked Hourglass Network, stacked hourglass network structure), the input target feature is input in the form of a picture (the same size as the face training picture) into the overall network structure (where the target feature is) The pictures in the overall structure will become feature maps of different sizes between the base layers of the transposed rolls), there are 10 residual modules in the overall network structure, that is, the depth of the overall network structure can be deepened and the overall network structure can be deepened by the residual module first. There will be no problem of gradient disappearance, and then through the 17-layer transposed volume base layer existing in the overall network structure, the feature map is restored to the result position map of the same size as the face training image at the last transposed volume base layer (the overall network structure The output is a set of heatmaps, that is, the three-dimensional position information of all the second key points in the result location map), and there is also a loss layer (used to set the loss function) in the overall network structure.

In this embodiment, the function of feature extraction is to initially obtain data that is more convenient for the operation and calculation of the face rotation model containing the initial parameters, thereby reducing the complexity of the overall network structure operation process in the face rotation model, and also reducing The running time of the running process; and the face rotation model can be further used to capture the deep information contained in the picture where the target feature is located in different picture sizes.

In an embodiment, after inputting the target feature into the face rotation model including initial parameters, the method further includes:

Enabling the face rotation model to obtain a result position map equal in size to the real position map according to the target feature;

Let the face rotation model extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.

Understandably, the resulting position map is a picture of the same size as the real position map and the face training picture; the preset key points can be the key points in the front face, including the eyes, nose and mouth in the sensory area There are multiple key points in a sensory area.

In this embodiment, the extracted three-dimensional position information of the second key point will be used for comparison with the three-dimensional position information of the second key point, so that the rotation angle of the face rotation model can be calculated.

Further, the step of making the face rotation model to extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index includes:

Enabling the face rotation model to establish a three-dimensional coordinate system in the result position map;

Make the face rotation model determine the key points corresponding to the fixed sensory area in the result location map according to the preset key point index, and extract the three-dimensional coordinates of each of the key points from the three-dimensional coordinate system And record the three-dimensional coordinate information of each of the key points as the three-dimensional position information of each of the second key points.

Specifically, the result location map is located in a three-dimensional coordinate system, that is, each fixed sensory area (the area composed of the eyes, nose, and mouth) in the result location map. At this time, it is indexed from multiple fixed sensory areas. The preset key points determine the required key points, and extract the three-dimensional coordinate information of the key points, and the three-dimensional coordinate information of the last key point can be used as the three-dimensional position information of the second key point.

S40. Set the three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function through a mask, and set each of the first key points The three-dimensional position information of the point and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

Understandably, the three-dimensional position information of each first key point and the amount of the three-dimensional position information of each second key point in the mean square error loss function can be set in the numerical part of the mean square error loss function through the mask. Weight value, that is, set the weight value of the first key point and the second key point in the mean square error loss function, and the set first key point and second key point should be the key points corresponding to the same sensory area (For example, the first key point is the area where the eyes are located, that is, the second key point is also the area where the eyes are located), and the sensory area includes areas where the eyes, nose, and mouth are located (and there are a total of 68 key points in these three areas Composition, that is, the first key point must be greater than or equal to 68, and the second key point must be greater than or equal to 68) and other face areas. At this time, the areas where the eyes, nose, mouth and other face areas are in the mean square The ratio of the weight value in the error loss function is set to 16:4:3:0.

The mean square error loss function is

Wherein, n is the number of key or the second key point of the first point, y _i is the i-th vector set of three-dimensional position information of the second key is constituted, y _'i is the i th first key A set of vectors composed of three-dimensional position information.

In this embodiment, by calculating the loss result in the mean square error loss function, it can be compared with the preset loss value in the subsequent steps, so as to optimize the face rotation model accordingly; and in this embodiment, the masks are used to set each The weight value of the three-dimensional position information of a key point and the three-dimensional position information of each second key point in the mean square error loss function is helpful for the face rotation model to learn a more accurate real position map.

S50: When the loss result is less than or equal to a preset loss value, confirm that the face rotation model training is completed; the face rotation model is used to confirm that the face photo is input into the face rotation model. The face photo is rotated to the required rotation angle for the front face photo.

Understandably, when the loss result is less than or equal to the preset loss value, it can indicate that the face rotation model regression training process is close to the training target. At this time, it can be determined that the face rotation model training is completed.

Further, after the step S40, it further includes:

When the loss result is greater than the preset loss value, iteratively update the initial parameters of the face rotation model until the loss result is less than or equal to the preset loss value, confirm the face rotation The model training is complete.

Understandably, when the loss result is greater than the preset loss value, it can indicate that the initial parameters of the face rotation model are not suitable for this training process. Therefore, it is necessary to continuously update the initial parameters of the face rotation model until the condition is suitable for This training process.

Further, after the step S50, it further includes:

Input a face photo into the trained face rotation model, obtain the rotation angle output by the face rotation model, and rotate the face photo into the frontal photo according to the rotation angle; The rotation angle refers to a rotation angle required to rotate the face photo into a front face photo.

Understandably, this embodiment is an application of the face rotation model. Once the value of the rotation angle output by the face rotation model is not 0, the rotation angle needs to be rotated (the face rotation model can be used in the original training process). First set the positive and negative values represented by the rotation direction, that is, clockwise is a positive value, and counterclockwise is a negative value). And during the rotation process, the face photo and the front photo can accept a certain angle deviation (for example, within 0.5 degrees, etc.) within the preset deformation range.

In an embodiment, the rotating the face photo into the front face photo according to the rotation angle includes: extracting the target feature of the face photo, and inputting the target feature of the face photo to In the trained face rotation model, and obtain the three-dimensional position information of all third key points output by the trained face rotation model; according to the rotation angle and the three-dimensional position information of the third key point Rotate the face photo into the front face photo.

In summary, the foregoing provides a method for generating a face rotation model, which obtains face training images from a target set; the face training images have been marked with three-dimensional face parameters; and a preset average face model is obtained , Obtaining a real position map according to the three-dimensional face parameters and the preset average face model; the real position map includes the frontal face corresponding to the face image in the face training picture and the The three-dimensional position information of all the first key points of the frontal face; extract the target feature of the face image in the face training picture, input the target feature into the face rotation model containing initial parameters, and Obtain the three-dimensional position information of all the second key points output by the face rotation model; set the three-dimensional position information of each of the first key points and the three-dimensional position information of each of the second key points in the mean square error The weight value occupied in the loss function, and input the three-dimensional position information of each of the first key points and the three-dimensional position information of each of the second key points into the mean square error loss function to obtain a loss result; When the loss result is less than or equal to the preset loss value, it is confirmed that the face rotation model training is completed; the face rotation model is used to confirm the face when the face photo is input into the face rotation model The photo rotation is the required rotation angle for the front face photo. This application generates a face rotation model through the above training process to accurately and efficiently recognize each face photo input to the face rotation model, so as to accurately and efficiently calculate the face photo rotation as a positive face The desired rotation angle of the photo.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

In one embodiment, a device for generating a face rotation model is provided, and the device for generating a face rotation model corresponds to the method for generating a face rotation model in the above-mentioned embodiment in a one-to-one correspondence. As shown in FIG. 4, the device for generating the face rotation model includes a first acquisition module 11, a second acquisition module 12, a third acquisition module 13, an input module 14 and a first confirmation module 15. The detailed description of each functional module is as follows:

The first obtaining module 11 is configured to obtain face training pictures from a target set; the face training pictures have already marked three-dimensional face parameters;

The second acquiring module 12 is configured to acquire a preset average face model, and acquire a real position map according to the three-dimensional face parameters and the preset average face model; the real position map contains the same as the person The three-dimensional position information of the front face corresponding to the face image in the face training picture and all first key points of the front face;

The third acquisition module 13 is configured to extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain the face rotation 3D position information of all second key points output by the model;

The input module 14 is configured to set the three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function through a mask, and set Inputting the three-dimensional position information of the first key point and the three-dimensional position information of each of the second key points into the mean square error loss function to obtain a loss result;

The first confirmation module 15 is used to confirm that the face rotation model training is completed when the loss result is less than or equal to the preset loss value; the face rotation model is used to input a face photo into the face rotation When modeling, confirm that the face photo is rotated to the required rotation angle for the front face photo.

Further, the second acquisition module includes:

The first acquisition sub-module is used to acquire the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to the three-dimensional position of a preset number of face vertices The shape formed by information; the model parameters include model shape parameters and model expression parameters;

The first determining sub-module is configured to determine the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

The second determining submodule is configured to determine the facial expression change of the facial image in the facial training picture according to the facial facial expression parameters and the model facial expression parameters;

The third determining sub-module is configured to determine that the face image corresponds to the preset average face model according to the average face shape, the deformation of the face image, and the expression change of the face image The three-dimensional position information of each of the face vertices;

The transformation sub-module is configured to convert the person according to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model The face training picture is similarly transformed into the front face position, and the real position map of the front face corresponding to the face image in the face training picture at the front face position is obtained.

Further, the device for generating the face rotation model further includes:

A fourth acquisition module, configured to enable the face rotation model to acquire a result position map equal in size to the real position map according to the target feature;

The extraction module is configured to enable the face rotation model to extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.

Further, the extraction module includes:

The establishment sub-module is used to enable the face rotation model to establish a three-dimensional coordinate system in the result position map;

The fourth determining sub-module is used to make the face rotation model determine the key points corresponding to the fixed sensory area in the result position map according to the preset key point index, and extract them from the three-dimensional coordinate system The three-dimensional coordinate information of each key point, and the three-dimensional coordinate information of each key point is recorded as the three-dimensional position information of each second key point.

Further, the device for generating the face rotation model further includes:

The rotation module is used to input a face photo into the face rotation model that has been trained, obtain the rotation angle output by the face rotation model, and rotate the face photo according to the rotation angle. The front face photo; the rotation angle refers to the rotation angle required to rotate the face photo into a front face photo.

Further, the rotation module includes:

The second acquisition sub-module is used to extract the target feature of the face photo, input the target feature of the face photo into the face rotation model after training, and obtain the face rotation after training 3D position information of all third key points output by the model;

The rotation sub-module is configured to rotate the face photo into the front face photo according to the rotation angle and the three-dimensional position information of the third key point.

Further, the device for generating the face rotation model further includes:

The second confirmation module is configured to iteratively update the initial parameters of the face rotation model when the loss result is greater than the preset loss value, until the loss result is less than or equal to the preset loss value To confirm that the training of the face rotation model is completed.

For the specific limitation of the generation device of the face rotation model, please refer to the above limitation on the generation method of the face rotation model, which will not be repeated here. Each module in the device for generating a face rotation model described above can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 5. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer readable instructions in the readable storage medium. The database of the computer device is used to store the data involved in the generation method of the face rotation model. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a method for generating a face rotation model is realized. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor, and the processor implements the following steps when the processor executes the computer-readable instructions:

In one embodiment, one or more readable storage media storing computer readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage. A medium in which computer-readable instructions are stored on the readable storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors implement the following steps:

The three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function are set through a mask, and the value of each of the first key points The three-dimensional position information and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium or a volatile readable storage medium, when the computer readable instruction is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as required. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

A method for generating a face rotation model, which is characterized in that it includes:

Obtain face training pictures from the target set; three-dimensional face parameters have been marked in the face training pictures;

Obtain a preset average face model, and obtain a real position map according to the three-dimensional face parameters and the preset average face model; the real position map includes the face image in the face training picture The corresponding front face and the three-dimensional position information of all first key points of the front face;

Extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain all second key points output by the face rotation model 3D location information;

The three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function are set through a mask, and the value of each of the first key points The three-dimensional position information and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

When the loss result is less than or equal to the preset loss value, it is confirmed that the face rotation model training is completed; the face rotation model is used to confirm the face when the face photo is input into the face rotation model The photo rotation is the required rotation angle for the front face photo.
The method for generating a face rotation model according to claim 1, wherein the three-dimensional face parameters include face shape parameters, face expression parameters, and face pose parameters; and said obtaining a preset average face The model, which obtains a real position map according to the three-dimensional face parameters and the preset average face model, includes:

Obtain the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to a shape composed of a preset number of three-dimensional position information of face vertices; the model Parameters include model shape parameters and model expression parameters;

Determining the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

Determining the facial expression change of the facial image in the facial training picture according to the facial expression parameters and the model facial expression parameters;

According to the average face shape, the deformation of the face image, and the expression change of the face image, it is determined that the face image corresponds to each of the face vertices in the preset average face model. Three-dimensional position information;

According to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model, the face training image is similarly transformed to In the front face position, the real position map of the front face corresponding to the face image in the face training picture at the front face position is obtained.
The method for generating a face rotation model according to claim 1, wherein after inputting the target feature into the face rotation model containing initial parameters, the method further comprises:

Enabling the face rotation model to obtain a result position map equal in size to the real position map according to the target feature;

Let the face rotation model extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.
The method for generating a face rotation model according to claim 3, wherein said making said face rotation model extracts all said second key points from said result position map according to a preset key point index The three-dimensional location information, including:

Enabling the face rotation model to establish a three-dimensional coordinate system in the result position map;

Make the face rotation model determine the key points corresponding to the fixed sensory area in the result location map according to the preset key point index, and extract the three-dimensional coordinates of each of the key points from the three-dimensional coordinate system And record the three-dimensional coordinate information of each of the key points as the three-dimensional position information of each of the second key points.
The method for generating a face rotation model according to claim 1, wherein when the loss result is less than or equal to a preset loss value, after confirming that the training of the face rotation model is completed, the method further comprises:

Input a face photo into the trained face rotation model, obtain the rotation angle output by the face rotation model, and rotate the face photo into the frontal photo according to the rotation angle; The rotation angle refers to a rotation angle required to rotate the face photo into a front face photo.
The method for generating a face rotation model according to claim 5, wherein the rotating the face photo into the front face photo according to the rotation angle comprises:

Extract the target feature of the face photo, input the target feature of the face photo into the trained face rotation model, and obtain all third key points output by the trained face rotation model 3D location information;

Rotating the face photo into the front face photo according to the rotation angle and the three-dimensional position information of the third key point.
The method for generating a face rotation model according to claim 1, wherein after obtaining a loss result, the method further comprises:

When the loss result is greater than the preset loss value, iteratively update the initial parameters of the face rotation model until the loss result is less than or equal to the preset loss value, confirm the face rotation The model training is complete.
A device for generating a face rotation model, which is characterized in that it comprises:

The first acquisition module is configured to acquire face training pictures from the target set; the face training pictures have already marked three-dimensional face parameters;

The second acquiring module is configured to acquire a preset average face model, and acquire a real position map according to the three-dimensional face parameters and the preset average face model; the real position map contains the same face as the face model. Three-dimensional position information of the front face corresponding to the face image in the training picture and all first key points of the front face;

The third acquisition module is configured to extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain the face rotation model The three-dimensional position information of all the output second key points;

The input module is used to set the three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function through a mask, and set each of the The three-dimensional position information of the first key point and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

The first confirmation module is used to confirm that the face rotation model training is completed when the loss result is less than or equal to the preset loss value; the face rotation model is used to input the face photo into the face rotation model , Confirm that the face photo is rotated to the required rotation angle for the front face photo.
8. The device for generating a face rotation model according to claim 8, wherein the second acquisition module comprises:

The first acquisition sub-module is used to acquire the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to the three-dimensional position of a preset number of face vertices The shape formed by information; the model parameters include model shape parameters and model expression parameters;

The first determining sub-module is configured to determine the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

The second determining submodule is configured to determine the facial expression change of the facial image in the facial training picture according to the facial facial expression parameters and the model facial expression parameters;

The third determining sub-module is configured to determine that the face image corresponds to the preset average face model according to the average face shape, the deformation of the face image, and the expression change of the face image The three-dimensional position information of each of the face vertices;

The transformation sub-module is configured to convert the person according to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model The face training picture is similarly transformed into the front face position, and the real position map of the front face corresponding to the face image in the face training picture at the front face position is obtained.
The device for generating a face rotation model according to claim 8, wherein the device for generating a face rotation model further comprises:

A fourth acquisition module, configured to enable the face rotation model to acquire a result position map equal in size to the real position map according to the target feature;

The extraction module is configured to enable the face rotation model to extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.
The device for generating a face rotation model according to claim 10, wherein the extraction module comprises:

The establishment sub-module is used to enable the face rotation model to establish a three-dimensional coordinate system in the result position map;

The fourth determining sub-module is used to make the face rotation model determine the key points corresponding to the fixed sensory area in the result position map according to the preset key point index, and extract them from the three-dimensional coordinate system The three-dimensional coordinate information of each key point, and the three-dimensional coordinate information of each key point is recorded as the three-dimensional position information of each second key point.
The device for generating a face rotation model according to claim 10, wherein the device for generating a face rotation model further comprises:

The rotation module is used to input a face photo into the face rotation model that has been trained, obtain the rotation angle output by the face rotation model, and rotate the face photo according to the rotation angle. The front face photo; the rotation angle refers to the rotation angle required to rotate the face photo into a front face photo.
A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor, wherein the processor executes the computer-readable instructions as follows step:

Obtain face training pictures from the target set; three-dimensional face parameters have been marked in the face training pictures;

Obtain a preset average face model, and obtain a real position map according to the three-dimensional face parameters and the preset average face model; the real position map includes the face image in the face training picture The corresponding front face and the three-dimensional position information of all first key points of the front face;

Extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain all second key points output by the face rotation model 3D location information;

The three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function are set through a mask, and the value of each of the first key points The three-dimensional position information and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

When the loss result is less than or equal to the preset loss value, it is confirmed that the face rotation model training is completed; the face rotation model is used to confirm the face when the face photo is input into the face rotation model The photo rotation is the required rotation angle for the front face photo.
The computer device according to claim 13, wherein the three-dimensional face parameters include face shape parameters, face expression parameters, and face pose parameters; said acquiring a preset average face model is based on said The three-dimensional face parameters and the preset average face model to obtain the real position map include:

Obtain the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to a shape composed of a preset number of three-dimensional position information of face vertices; the model Parameters include model shape parameters and model expression parameters;

Determining the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

Determining the facial expression change of the facial image in the facial training picture according to the facial expression parameters and the model facial expression parameters;

According to the average face shape, the deformation of the face image, and the expression change of the face image, it is determined that the face image corresponds to each of the face vertices in the preset average face model. Three-dimensional position information;

According to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model, the face training image is similarly transformed to In the front face position, the real position map of the front face corresponding to the face image in the face training picture at the front face position is obtained.
The computer device according to claim 13, wherein after the target feature is input to the face rotation model including initial parameters, the processor further implements the following steps when executing the computer readable instruction:

Enabling the face rotation model to obtain a result position map equal in size to the real position map according to the target feature;

Let the face rotation model extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.
15. The computer device according to claim 15, wherein said making said face rotation model extracts three-dimensional position information of all said second key points from said result position map according to a preset key point index, include:

Enabling the face rotation model to establish a three-dimensional coordinate system in the result position map;

Make the face rotation model determine the key points corresponding to the fixed sensory area in the result location map according to the preset key point index, and extract the three-dimensional coordinates of each of the key points from the three-dimensional coordinate system And record the three-dimensional coordinate information of each of the key points as the three-dimensional position information of each of the second key points.
One or more readable storage media storing computer readable instructions, wherein when the computer readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Obtain face training pictures from the target set; three-dimensional face parameters have been marked in the face training pictures;

Obtain a preset average face model, and obtain a real position map according to the three-dimensional face parameters and the preset average face model; the real position map includes the face image in the face training picture The corresponding front face and the three-dimensional position information of all first key points of the front face;

Extract the target feature of the face image in the face training picture, input the target feature into a face rotation model containing initial parameters, and obtain all second key points output by the face rotation model 3D location information;

The three-dimensional position information of each of the first key points and the weight value of the three-dimensional position information of each of the second key points in the mean square error loss function are set through a mask, and the value of each of the first key points The three-dimensional position information and the three-dimensional position information of each of the second key points are input into the mean square error loss function to obtain a loss result;

When the loss result is less than or equal to the preset loss value, it is confirmed that the face rotation model training is completed; the face rotation model is used to confirm the face when the face photo is input into the face rotation model The photo rotation is the required rotation angle for the front face photo.
The readable storage medium according to claim 17, wherein the three-dimensional face parameters include face shape parameters, face expression parameters, and face pose parameters; said obtaining a preset average face model is based on The three-dimensional face parameters and the preset average face model to obtain a real position map includes:

Obtain the model parameters of the preset average face model and the average face shape that has been successfully trained; the average face shape refers to a shape composed of a preset number of three-dimensional position information of face vertices; the model Parameters include model shape parameters and model expression parameters;

Determining the deformation of the face image in the face training picture according to the face shape parameter and the model shape parameter;

Determining the facial expression change of the facial image in the facial training picture according to the facial expression parameters and the model facial expression parameters;

According to the average face shape, the deformation of the face image, and the expression change of the face image, it is determined that the face image corresponds to each of the face vertices in the preset average face model. Three-dimensional position information;

According to the face pose parameters and the determined three-dimensional position information of the face image corresponding to each of the face vertices in the preset average face model, the face training image is similarly transformed to In the front face position, the real position map of the front face corresponding to the face image in the face training picture at the front face position is obtained.
The readable storage medium of claim 17, wherein after the target feature is input to the face rotation model containing initial parameters, when the computer-readable instructions are executed by one or more processors , So that the one or more processors further execute the following steps:

Enabling the face rotation model to obtain a result position map equal in size to the real position map according to the target feature;

Let the face rotation model extract the three-dimensional position information of all the second key points from the result position map according to a preset key point index.
The readable storage medium of claim 19, wherein the face rotation model extracts the three-dimensional positions of all the second key points from the result position map according to a preset key point index Information, including:

Enabling the face rotation model to establish a three-dimensional coordinate system in the result position map;

Make the face rotation model determine the key points corresponding to the fixed sensory area in the result location map according to the preset key point index, and extract the three-dimensional coordinates of each of the key points from the three-dimensional coordinate system And record the three-dimensional coordinate information of each of the key points as the three-dimensional position information of each of the second key points.