CN111710035B

CN111710035B - Face reconstruction method, device, computer equipment and storage medium

Info

Publication number: CN111710035B
Application number: CN202010685179.4A
Authority: CN
Inventors: 林祥凯; 王盛; 暴林超
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-07-16
Filing date: 2020-07-16
Publication date: 2023-11-07
Anticipated expiration: 2040-07-16
Also published as: CN111710035A

Abstract

The application relates to a face reconstruction method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring a face point cloud of a target face; based on the face point cloud, carrying out iterative updating on the three-dimensional face model of the target face to obtain a reconstructed three-dimensional face model of the target face; the iterative updating steps are as follows: performing deformation treatment on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model; acquiring a point corresponding relation between the deformed three-dimensional face model and the face point cloud; performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies. According to the scheme, the face reconstruction is carried out on the face data obtained by scanning based on the two face bases with different accuracies, so that the accuracy of the face reconstruction is improved.

Description

Face reconstruction method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer vision, and in particular, to a face reconstruction method, apparatus, computer device, and storage medium.

Background

The 3DMM (3D Morphable Model, three-dimensional deformation model) library is used to provide a standard face model for three-dimensional (3 d) face reconstruction.

The three-dimensional face reconstruction method based on the 3DMM library is widely applied. In the related art, a scheme for reconstructing a three-dimensional face based on a 3DMM library is mainly to calculate a conversion coefficient, and a face base in the 3DMM library can be converted into a three-dimensional face model close to a target face under the action of the conversion coefficient.

In the above technical solution, the expression capability of the face base in the 3DMM library is insufficient, for example, details of the target face cannot be accurately reconstructed, resulting in poor face reconstruction effect.

Disclosure of Invention

The embodiment of the application provides a face reconstruction method, a device, computer equipment and a storage medium, which can reconstruct a face based on face data obtained by scanning based on two face bases with different precision, and improve the accuracy of the face reconstruction, and the technical scheme is as follows:

in one aspect, a face reconstruction method is provided, the method including:

acquiring a face point cloud of a target face;

based on the face point cloud, carrying out iterative updating on the three-dimensional face model of the target face until the iterative stopping condition is met, and obtaining a reconstructed three-dimensional face model of the target face;

The iterative updating steps are as follows:

performing deformation processing on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model;

acquiring point correspondence between the deformed three-dimensional face model and the face point cloud, wherein the point correspondence is correspondence between points of the two models;

performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies.

In yet another aspect, a face reconstruction apparatus is provided, the apparatus including:

the face point cloud acquisition module is used for acquiring face point clouds of the target face;

the iteration updating module is used for carrying out iteration updating on the three-dimensional face model of the target face based on the face point cloud until the iteration stopping condition is met, so as to obtain a reconstructed three-dimensional face model of the target face;

the iterative updating module comprises:

the face deformation unit is used for performing deformation treatment on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model;

The corresponding relation acquisition unit is used for acquiring the point corresponding relation between the deformed three-dimensional face model and the face point cloud, wherein the point corresponding relation is the corresponding relation between the points of the two models;

the face fitting unit is used for performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies.

In one possible implementation manner, the face fitting unit includes:

the first face fitting subunit is used for carrying out face fitting based on the point corresponding relation and the first face base to obtain a first fitting face model and a first fitting coefficient, wherein the first fitting coefficient is a coefficient of the first fitting face model obtained through reconstruction of the first face base;

the second face fitting subunit is used for carrying out face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of a face model obtained through reconstruction of the second face base; the fitting residual error is the residual error between the point corresponding relation and the first fitting face model;

And the face model updating subunit is used for constructing the updated three-dimensional face model based on the first face base, the second face base, the first fitting coefficient and the second fitting coefficient.

In one possible implementation, the first face fitting sub-unit is configured to, in use,

constructing a first loss function, wherein the first loss function takes the point corresponding relation, the first face base and the average value of the first face base as input, and takes a fitting coefficient as a variable;

obtaining a fitting coefficient which enables the value of the first loss function to be minimum as the first fitting coefficient;

and constructing the first fitting face model based on the first fitting coefficient, the first face base and the average value of the first face base.

In one possible implementation, the second face fitting subunit is configured to,

constructing a second loss function, wherein the second loss function takes the fitting residual error and the second face base as inputs, and takes a fitting coefficient as a variable;

and obtaining the fitting coefficient which enables the value of the second loss function to be minimum as the second fitting coefficient.

In a possible implementation manner, the face model sub-updating unit is configured to obtain, as the updated three-dimensional face model, a sum of the mean value of the first face base, the first product, and the second product;

wherein the first product is a product of the first face base and the first fitting coefficient, and the second product is a product of the second face base and the second fitting coefficient.

In one possible implementation, the second face base is a face base corresponding to a specified partial region in the face.

In a possible implementation manner, the face deformation unit is configured to iteratively deform the three-dimensional face model before updating according to a non-rigid registration manner, with the face point cloud as a target, to obtain the deformed three-dimensional face model;

the face deformation unit further includes:

a closest point pair acquisition subunit configured to acquire each closest point pair; any one target point pair of each nearest point pair is a point pair formed by the target point and the nearest point of the target point in the human face point cloud; the target point is any point in the three-dimensional face model before updating;

The nearest point pair filtering subunit is used for filtering each nearest point pair based on the distance information of each nearest point pair to obtain a filtered nearest point pair; the distance information is used for indicating the distance between two points in the corresponding point pair;

and the closest point pair registration subunit is used for registering based on the filtered closest point pair to obtain the deformed three-dimensional face model.

In one possible implementation, the closest point pair filtering subunit is configured to,

acquiring a distance threshold value based on the distance information of each nearest point pair;

and filtering out the point pairs with the distance indicated by the corresponding distance information being greater than the distance threshold value in each nearest point pair to obtain the filtered nearest point pair.

In one possible implementation manner, the nearest point pair filtering subunit is configured to obtain a distance threshold based on the distance information of the nearest point pairs, and further includes:

the distances indicated by the distance information of each nearest point are ordered in order from small to large;

acquiring a first distance threshold according to the sorting result, wherein the first distance threshold is a distance for dividing each nearest point according to a preset proportion in the distances indicated by the distance information of each nearest point;

And acquiring the larger value of the first distance threshold and the preset second distance threshold as the distance threshold.

In one possible implementation, the apparatus further includes:

the data acquisition module is used for acquiring RGBD data of the color depth of the target face;

the initial model building module is used for acquiring an initial three-dimensional face model of the target face based on the RGBD data; the initial three-dimensional face model is the three-dimensional face model before updating, which is used in the iterative updating process for the first time.

In one possible implementation manner, the initial model building module includes:

the key point acquisition unit is used for acquiring three-dimensional key points of the target face based on the RGBD data;

the initial model building unit is used for building an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

In yet another aspect, a computer device is provided, where the computer device includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored, where the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the face reconstruction method described above.

In yet another aspect, a computer readable storage medium is provided, where at least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-mentioned face reconstruction method.

In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned face reconstruction method.

The technical scheme provided by the embodiment of the application has the beneficial effects that at least:

after the three-dimensional face model before updating is deformed by taking the face point cloud as a target in each round of iteration process, the deformed three-dimensional face model is subjected to fitting updating based on two face bases with different precision (such as the low-frequency face base and the high-frequency face base), the updated three-dimensional face image is used as the three-dimensional face model before updating in the next round of iteration updating, that is, two face bases with different precision are introduced in each round of iteration updating process to perform face fitting, so that the accuracy of face reconstruction is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a face reconstruction method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a computer system provided in accordance with an exemplary embodiment of the present application;

FIG. 3 is a flowchart of a face reconstruction method according to an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of the NRICP effect involved in the embodiment of FIG. 3;

FIG. 5 is a graph of the effect of the closest point-to-filter involved in the embodiment of FIG. 3;

FIG. 6 is a schematic diagram of a face reconstruction process based on the NRICP algorithm according to the embodiment shown in FIG. 3;

FIG. 7 is a schematic diagram of a model in a single iteration update involved in the embodiment of FIG. 3;

FIG. 8 is a schematic view of a face reconstruction effect according to the embodiment shown in FIG. 3;

fig. 9 is a block diagram illustrating a structure of a face reconstruction apparatus according to an exemplary embodiment;

FIG. 10 is a block diagram of a computer device in accordance with an exemplary embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

First, terms according to the embodiments of the present application will be described.

Computer Vision (CV): the method is a science for researching how to make a machine "look at", and further means that a camera and a computer are used for replacing human eyes to recognize, track and measure targets and other machine vision, and further graphic processing is performed, so that the computer is processed into images which are more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (Optical Character Recognition, OCR), video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, and map construction, among others.

The first face base (also named low frequency face base): refers to a face base calculated based on first precision face data (also named high precision face data) and second precision face data (also named low precision face data). The first precision face data refers to face data with the face data precision higher than a threshold value, and the second precision face data refers to face data with the face data precision lower than the threshold value. And aligning the high-precision face grid data and the low-precision face grid data under the same coordinate system through data alignment (the number of vertexes of the high-precision face grid data and the low-precision face grid data are consistent, the topology is the same, but the faces are different in length, the precision of the high-precision face grid data is greater than that of the low-precision face grid data), and performing principal component analysis (Principal Component Analysis, PCA) on the aligned grid vertex coordinates to obtain a PCA base, wherein the PCA base is the low-frequency face base. Illustratively, face data obtained by scanning and processing a real face by using a scanning device with higher precision (such as a high-speed 3D camera) is used as first-precision face data, and face data obtained by scanning and processing a real face by using a device with lower precision (such as a smart phone) is used as second-precision face data.

The second face base (also named high frequency face base): the face base is constructed based on errors generated when the first face base fits the face data. Fitting the first face base to each piece of data in the face data to obtain an error vector corresponding to each piece of data, obtaining an error matrix according to the error vector, and calculating the error matrix through a PCA principal component analysis technology to obtain a second face base.

The face reconstruction method provided by the embodiment of the application is the application of the computer vision technology in the three-dimensional face reconstruction scene. Referring to fig. 1, a schematic diagram of a face reconstruction method according to an embodiment of the present application is shown. As shown in fig. 1, the face reconstruction method is performed as follows:

s11, acquiring a face point cloud of the target face.

The face point cloud of the target face is generated according to an actual face image of the target face.

In one possible implementation, the face point cloud of the target face is generated from a color depth image of the target face.

And S12, based on the face point cloud, carrying out iterative updating on the three-dimensional face model of the target face until the iterative stopping condition is met, and obtaining the reconstructed three-dimensional face model of the target face.

The steps of the iterative updating in each round are as follows:

and S12a, carrying out deformation treatment on the three-dimensional face model before updating by taking the face point cloud as a target to obtain the deformed three-dimensional face model.

The deformation processing is performed on the three-dimensional face model before updating by taking the face point cloud as a target, namely the deformation is performed on the three-dimensional face model before updating, so that the deformed three-dimensional face model is closer to the face point cloud corresponding to the actual target face than the three-dimensional face model before updating.

And S12b, acquiring the point corresponding relation between the deformed three-dimensional face model and the face point cloud, wherein the point corresponding relation is the corresponding relation between the points of the two models.

In the field of face reconstruction, the above-mentioned point correspondence is also called corespondance, and refers to a point-to-point correspondence between two models.

S12c, performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies.

In the embodiment of the application, in each round of iterative process, after the three-dimensional face model before updating is deformed by taking the face point cloud as a target, the deformed three-dimensional face model is subjected to fitting updating based on two face bases with different precision (such as the low-frequency face base and the high-frequency face base), the updated three-dimensional face image is used as the three-dimensional face model before updating in the next round of iterative updating, and two face bases with different precision are introduced in each round of iterative updating process to carry out face fitting, so that the accuracy of face reconstruction is improved.

The face reconstruction method provided by the embodiment of the application can be applied to the following scenes:

1. virtual character construction in gaming applications

Under the application scene, the method provided by the embodiment of the application can be applied to a background server of a game application program. When the virtual character is constructed, the user uses the terminal to collect face data of the user's face, and the collected face data is uploaded to the background server. The background server generates a face point cloud according to the face data, and performs face reconstruction according to the face point cloud, the first face base and the second face base, so as to construct a virtual character with the same face as the user. The virtual character is used for representing the identity of the user in the game application, and the user can control the virtual character to play the game. Or, the terminal may generate a face point cloud according to the face data, and reconstruct a face according to the face point cloud, the first face base, and the second face base.

2. Virtual object construction in virtual reality applications

Under the application scene, the method provided by the embodiment of the application can be applied to a terminal or a server running a virtual reality application program. And for face data acquired by a camera shooting component of a terminal, generating face point cloud based on the face data, reconstructing a three-dimensional face model similar to the length of the user according to the face point cloud and the first face base and the second face base, wherein the three-dimensional face model comprises some detail characteristics of the face of the user, such as double eyelid, head lifting lines, canthus lines and the like, namely, constructing the face of a virtual object similar to the real face of the user. The virtual object is used for representing the identity of the user in the virtual reality application program, and the user performs video call, scene simulation and the like in the virtual reality application program by utilizing the identity of the virtual object.

3. Virtual object construction in mixed reality applications

Under the application scene, the method provided by the embodiment of the application can be applied to a terminal or a server running a mixed reality application program. And for face data acquired by a user using a camera component of the terminal or face data input in advance, generating a face point cloud based on the face data, reconstructing a three-dimensional face model similar to the length of the user by the first face base and the second face base, and constructing a virtual object by combining other body data input by the user. The user can perform corresponding operations on the virtual object, for example, putting on clothes for the virtual object, realizing the effect of trying on clothes online, and the like.

The above description is given by taking three application scenarios as examples, and the method provided by the embodiment of the present application may also be applied to other scenarios (such as virtual person construction, face restoration, etc.) in which a three-dimensional face model with high accuracy needs to be reconstructed according to input high-accuracy face data.

The face reconstruction method provided by the embodiment of the application can be applied to computer equipment with stronger data processing capability. In a possible implementation manner, the face reconstruction method provided by the embodiment of the application can be applied to a personal computer, a workstation or a server, namely, the high-precision face shape library can be constructed through the personal computer, the workstation or the server.

Referring to FIG. 2, a schematic diagram of a computer system according to an exemplary embodiment of the application is shown. The computer system 200 includes a terminal 210 and a server 220, where data communication is performed between the terminal 210 and the server 220 through a communication network, alternatively, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.

An application having a virtual character face shape display requirement is installed in the terminal 210, and may be a virtual reality application, a game application, a dynamic face application, or an artificial intelligence (Artificial Intelligence, AI) application having a three-dimensional face generation function, to which the embodiment of the present application is not limited.

Alternatively, the computer device 210 may be a mobile terminal such as a smart phone, a smart watch, a tablet computer, a laptop portable notebook computer, or a terminal such as a desktop computer, a projection computer, which is not limited in the embodiment of the present application.

The server 220 may be implemented as a server or a server cluster formed by a group of servers, which may be a physical server or a cloud server. In one possible implementation, server 220 is a background server for applications in computer device 210.

In one possible implementation manner of this embodiment, in the face reconstruction stage, the terminal 210 obtains high-precision face data of the target face, and uploads the face data to the server 220. The terminal 210 may collect the face data through a built-in face data collection component (such as a depth camera), or may obtain face data collected and sent by other devices. After the server 220 obtains the input high-precision face data, a high-precision face point cloud is generated, then the three-dimensional face model is iteratively updated based on the face point cloud, in each iteration updating process, the face point cloud is firstly taken as a target, the three-dimensional face model is deformed, then two face bases with different preset precision are combined according to the corresponding relation between the deformation result and the face point cloud, the updated three-dimensional face model is obtained through fitting, the iteration updating process is repeatedly executed, and the reconstructed three-dimensional face model is obtained, so that the accuracy of face reconstruction is improved. After the server 220 generates the reconstructed three-dimensional face model, an avatar is generated based on the reconstructed three-dimensional face model and then transmitted to the terminal 210 for display.

In another possible implementation manner, after obtaining high-precision face data of the target face, the terminal 210 generates a high-precision face point cloud locally, and then iteratively updates the three-dimensional face model based on the face point cloud to obtain a reconstructed three-dimensional face model. Optionally, the terminal 210 transmits the reconstructed three-dimensional face model to the server 220, or generates an avatar based on the reconstructed three-dimensional face model and transmits the avatar to the server 220.

For convenience of description, the following embodiments are described by taking a face reconstruction method as an example to be executed by a computer device. The computer device is the terminal 210 or the server 220 described above.

Fig. 3 shows a flowchart of a face reconstruction method according to an exemplary embodiment of the present application. This embodiment is described by taking the method as an example in the computer system 200 shown in fig. 2, and the method includes the following steps:

step 301, acquiring face data of a target face.

In one possible implementation, the face data of the target face is a two-dimensional face image.

The two-dimensional face image is illustratively a color depth image of the target face.

The color depth image is also referred to as an R (red) G (green) B (blue) -D (depth) image, abbreviated as RGBD image.

In one possible implementation, the RGBD image of the target face is an image obtained by capturing the target face with a depth camera.

In one possible implementation, the face data of the target face is a high-precision color depth image, such as an image captured by a high-precision depth camera.

In one possible implementation, the face data of the target face is an RGBD image of the target face at different face poses.

Step 302, generating a face point cloud of the target face based on the face data of the target face.

In one possible implementation manner, when the face data of the target face is an RGBD image of the target face in different face poses, the computer device selects frames from the RGBD images in different poses by detecting the obtained face mark points, then calculates the camera pose corresponding to each frame, further cuts out the depth map point cloud, and finally synthesizes the complete face point cloud through the pose.

In other possible implementation manners, the computer device may also generate the face point cloud in other manners, for example, manually assist in selecting frames of RGBD images, perform point cloud matching on each RGBD image to obtain a pose corresponding to each RGBD image, or calculate the pose corresponding to each RGBD image through sparse points, and finally synthesize a complete face point cloud through the poses. The method for generating the face point cloud of the target face is not limited in the embodiment of the application.

Although the accuracy of the face point cloud generated by the steps is higher, the details are more similar to the real face, however, on one hand, the face point cloud generated by the face data usually has a defect, for example, due to the reasons of image acquisition shielding and frame selection, there may be a defect of the point cloud of part of the face area; on the other hand, the topological structure of the face point cloud generated by the face data is not fixed, and further driving cannot be performed on the basis, so that the application scene is limited. Therefore, in the subsequent steps of the embodiment of the application, the three-dimensional face model with fixed topological structure and complete face area is further reconstructed based on the face point cloud with higher accuracy.

Step 303, acquiring an initial three-dimensional face model of the target face based on the face data of the target face.

The initial three-dimensional face model is used in the subsequent first iteration updating process, and the three-dimensional face model is updated before updating.

In one possible implementation, when the face data of the target face is RGBD data, the computer device obtains three-dimensional key points of the target face based on the RGBD data, and constructs an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

For example, when the face data of the target face is an RGBD image of the target face in different face postures, the computer equipment extracts two-dimensional key points and three-dimensional key points in the RGBD image in different face postures, performs fitting by taking the two-dimensional key points, the three-dimensional key points and the regular terms as constraints to obtain a fitting coefficient on a three-dimensional deformation model library, and then obtains an initial three-dimensional face model through the fitting coefficient and the three-dimensional deformation model library.

In other possible implementation manners, the computer device may also generate the initial three-dimensional face model in other manners, for example, the computer device extracts two-dimensional key points in RGBD images under different face poses, performs fitting with the two-dimensional key points and the regular term as constraints, obtains a fitting coefficient on the three-dimensional deformation model library, and then obtains the initial three-dimensional face model through the fitting coefficient and the three-dimensional deformation model library. The method for generating the face point cloud of the target face is not limited in the embodiment of the application.

In one possible implementation, the computer device selects an algorithm that is computationally fast to generate an initial three-dimensional face model, e.g., one that is less accurate but faster to generate.

And step 304, based on the face point cloud and the initial three-dimensional face model, carrying out iterative updating on the three-dimensional face model of the target face.

In the embodiment of the present application, the steps of the iterative update are as follows:

s304a, performing deformation processing on the three-dimensional face model before updating by taking the face point cloud as a target to obtain the deformed three-dimensional face model.

In the embodiment of the application, the deformation processing is carried out on the three-dimensional face model before updating, so that the deformed three-dimensional face model is more similar to the face point cloud, and the face point cloud is generated through face data and is close to the real face, so that the deformed three-dimensional face model is also more close to the real face.

In one possible implementation, the computer device performs deformation processing on the three-dimensional face model before updating through an NRICP (Non-rigid Iterative Closest Point, non-rigid quasi-matching) algorithm.

Wherein NRICP can be deformed in addition to translational transformation of the model against the iteration closest point (Iterative Closest Point, ICP). In NRICP, the feature points of the initial model (i.e., the three-dimensional face model before updating) are used as source feature points, the feature points in the face point cloud are used as targets, and the purpose of NRICP is to find a transformation so that the source feature points can represent points in the face point cloud.

In this embodiment, the steps of deforming the three-dimensional face model before updating by the NRICP algorithm are as follows:

acquiring a three-dimensional face model before updating, wherein the three-dimensional face model before updating comprises a plurality of source characteristic points;

initializing a rotation translation matrix x0 of each source characteristic point, wherein the rotation translation matrix is used for performing rotation translation operation on each source characteristic point;

in each round of iteration process, carrying out rotation translation transformation on each source characteristic point through the rotation translation matrix, fitting the obtained new characteristic point with a corresponding point in the face point cloud, finding out the point in the face point cloud closest to each new characteristic point as a corresponding point, bringing all the new characteristic points and the corresponding points into a formula 1, optimizing the new rotation translation matrix, and optimizing the f value in the formula 1 to be as small as possible; and in the next iteration process, applying a new rotation translation matrix to the source characteristic points, and iterating the operation until the difference between the new rotation translation matrix and the rotation translation matrix of the last iteration is smaller than a certain threshold value.

The formula 1 is as follows:

（1）

where x1-xn is the rotational translation matrix corresponding to each source feature point, v 1-vn is each three-dimensional point in the source feature points, and u1 through un are the corresponding points of the new feature points. The formula (1) is divided into a left term and a right term, and the left term is that each three-dimensional point in the source characteristic points is rotated and translated by utilizing a rotation and translation matrix, and then a difference is made between a new characteristic point obtained by rotation and translation and a corresponding point, so that a variance value is obtained. The right is the amount for keeping the original topology unchanged as far as possible, and M is the topology of the model, namely, a feature point set capable of showing the structural features of the model with the least feature points.

Therefore, the right visual understanding of the formula (1) is that when x is 0, there is no rotation and translation, and the original shape is kept unchanged. The proportion of the two is regulated through a smoothing term a, the larger the smoothing term a is, the more the proportion of the topological structure in the energy function is represented, namely the smoothing term has the effect that the rotation translation matrix can be optimized to be attached to the face point cloud finally, the model topology result of the three-dimensional face model before updating is maintained, when a is larger, the proportion of the topological structure in the energy function is more, the original topology can be considered more in the optimization of the rotation translation matrix, and therefore the original topological structure can be reserved by utilizing the rotation translation matrix optimized through the larger smoothing term a, namely the characteristics of the three-dimensional face model before updating are reserved. And then optimizing the rotation translation matrix optimized by the larger smoothing term a through the smaller smoothing term a, wherein the smaller smoothing term a shows that the proportion of characteristic points of the face point cloud to new characteristic point errors obtained by rotation translation is larger, the model is easier to notice details in the face point cloud, namely, the face point cloud is utilized to optimize the rotation translation matrix in detail on the basis of the model structure of the three-dimensional face model before updating, so that the deformed three-dimensional face model obtained by the three-dimensional face model before updating is more attached to a real face by the final rotation translation matrix. For example, please refer to fig. 4, which illustrates a schematic diagram of NRICP effect according to an embodiment of the present application. As shown in fig. 4, the left side in fig. 4 is a three-dimensional face model before updating, the middle is a face point cloud, and the right side is a result after NRICP is applied to the three-dimensional face model before updating, that is, the three-dimensional face model after deformation.

In the execution process of the NRICP algorithm, corresponding points in the two models need to be determined to form each point pair so as to perform rotary translation according to a rotary translation matrix; however, because the face point cloud is generated through the face data, on one hand, the face point cloud is closer to the model of the real face, so that some details in the face point cloud are points which are not expressed by the three-dimensional face model before updating, such as the points corresponding to hairs in the face point cloud, on the other hand, the face point cloud may be incomplete, and therefore, in the process of searching for the corresponding points corresponding to the source feature points in the face cloud, the searched corresponding points may deviate far, and the situation of searching errors may occur. In one possible implementation manner, on the basis of the NRICP algorithm, the computer equipment performs iterative deformation on the three-dimensional face model before updating according to a non-rigid registration mode and with the face point cloud as a target to obtain the deformed three-dimensional face model; the steps of iterative deformation are as follows:

and S304a1, finding the nearest point as a corresponding point.

Acquiring each nearest point pair; any one target point pair of each nearest point pair is a point pair formed by the target point and the nearest point of the target point in the human face point cloud; the target point is any one point in the three-dimensional face model before updating.

The nearest point pair is a pair of characteristic points formed by one point in the three-dimensional face model before updating and a corresponding point in the face point cloud.

S304a2, screening out reliable corresponding points.

Filtering each nearest point pair based on the distance information of the nearest point pair to obtain a filtered nearest point pair; the distance information is used to indicate a distance between two points in the corresponding point pair.

In one possible implementation manner, filtering each nearest point pair based on the distance information of the nearest point pair to obtain a filtered nearest point pair, and acquiring a distance threshold value by the computer equipment based on the distance information of each nearest point pair; and filtering out the point pairs with the distance indicated by the corresponding distance information being greater than the distance threshold value in each nearest point pair to obtain the filtered nearest point pair.

In the embodiment of the application, the computer equipment dynamically determines a distance threshold value by combining the distance corresponding to the distance information of each nearest point pair, and deletes the nearest point pair with the distance between two points being greater than the distance threshold value.

In one possible implementation, when the distance threshold is obtained based on the distance information of the respective closest point pair, the computer device sorts the distances indicated by the distance information of the respective closest point in order from smaller to larger; acquiring a first distance threshold according to the sorting result, wherein the first distance threshold is the distance for cutting each nearest point according to a preset proportion in the distances indicated by the distance information of each nearest point; and acquiring the larger value of the first distance threshold and the preset second distance threshold as the distance threshold.

In the above scheme of the application, a screening process is newly added in the process of finding the closest point. The reason is that the face point cloud is incomplete, and there may be details which cannot be expressed by the three-dimensional face model, and the closest points found by a plurality of points may be far away, so that errors are generated, and the closest point pairs with large errors may affect the overall face reconstruction effect due to the existence of the smooth term (a), so that the error points need to be deleted. In the embodiment of the application, the distance is adopted as a threshold value for judgment. For example, the computer device calculates the distance between each point in the three-dimensional face model before updating and the closest point in the face point cloud, ranks all the obtained distances, and takes the distance value on the critical point of the preset proportion (such as the first 70%) before the error and the maximum value in the preset absolute threshold as the distance threshold, so that the inaccurate closest point pairs in the current iteration updating process can be screened out, the screened out closest point pairs are filled by the smooth term in the NRICP, and the errors are smaller and smaller along with the progress of the iteration and the reduction of the smooth term.

And S304a3, optimizing error items and smooth items of the corresponding points.

And registering based on the filtered nearest point pair, and obtaining the deformed three-dimensional face model after optimizing error items and smooth items of corresponding points.

For example, please refer to fig. 5, which illustrates a graph of the effect of the closest point-to-filter according to an embodiment of the present application. As shown in fig. 5, as the iteration proceeds, the number of the nearest point pairs with errors is smaller and smaller, so that the scheme can well avoid that a few error point pairs cause larger interference. As can be seen from fig. 5, the iteratively updated three-dimensional face model is closer to the real front point cloud (i.e., the face point cloud) than the initial three-dimensional face model.

In the embodiment of the application, after each closest point pair between the three-dimensional face model and the face point cloud before updating is acquired, the computer equipment further filters each closest point pair according to the distance information of each closest point pair so as to remove the wrong point pair as much as possible, thereby improving the robustness of the NRICP algorithm and further improving the accuracy of face reconstruction.

S304b, obtaining the point corresponding relation between the deformed three-dimensional face model and the face point cloud, wherein the point corresponding relation is the corresponding relation between the points of the two models.

The point correspondence between the deformed three-dimensional face model and the face point cloud is also called corruspore, or a matching relationship.

S304c, performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies.

The dense matching relationship obtained in the above steps 304a and 304b corresponds to a constraint of dense three-dimensional feature points (3 dkeypoints). However, due to insufficient expression capability of the common 3dmm library, in order to improve accuracy of face reconstruction, in the embodiment of the application, fitting is performed through face bases with different accuracies (namely, a high-frequency face base and a low-frequency face base), so that in each iteration process, fitting update is performed on the three-dimensional face model.

In the embodiment of the application, the precision of the first face base is lower than that of the second face base; when performing face fitting based on the point correspondence, the first face base, and the second face base, and obtaining an updated three-dimensional face model, the computer device may perform the following steps:

s304c1, fitting the basic face shape by using a low-frequency base.

And carrying out face fitting based on the point corresponding relation and the first face base to obtain a first fitting face model and a first fitting coefficient, wherein the first fitting coefficient is a coefficient of the first fitting face model obtained through reconstruction of the first face base.

In one possible implementation manner, when performing face fitting based on the point correspondence and the first face base to obtain the first fitting face model and a first fitting coefficient, the computer device constructs a first loss function, where the first loss function takes the average of the point correspondence, the first face base and the first face base as input, and takes the fitting coefficient as a function of a variable; obtaining a fitting coefficient which enables the value of the first loss function to be minimum as the first fitting coefficient; and constructing the first fitting face model based on the first fitting coefficient, the first face base and the mean value of the first face base.

In one possible implementation, the computer device optimizes the low frequency model by equation 2. Equation 2 is as follows:

F = W（MU_low+B_low*paras_low-keypoints3d）+ W（paras_low/sigma）（2）

wherein MU_Low is the average of the low frequency 3dmm basis (i.e., the first face basis), B_Low is the low frequency 3dmm basis (i.e., the first face basis), and parama_Low is the determined 3dmm coefficient (i.e., the first fitting coefficient). Sigma is the variance obtained on the low frequency 3dmm basis, W is a coefficient representing the weight of each dimension in the params_low. keypoints3d are the result of the nricp, i.e., the point correspondence. By optimizing equation 2 (the goal is that the value of F is as small as possible), a low frequency 3dmm coefficient can be obtained, and a low frequency model (vertex_low=mu_low+b_low). The vertex_low is the first fitting face model.

S304c2, fitting the residual with a high frequency basis.

Performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of a face model obtained through reconstruction of the second face base; the fitting residual is the residual between the point correspondence and the first fitting face model.

In one possible implementation manner, when performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient, the computer device constructs a second loss function, where the second loss function takes the fitting residual error and the second face base as inputs and takes the fitting coefficient as a function of a variable; and obtaining the fitting coefficient which enables the value of the second loss function to be minimum as the second fitting coefficient.

In the embodiment of the application, the computer equipment fits (fit) the residual error through the high-frequency base (namely the second face base) by using the residual error of the low-frequency 3dmm model (vertex_low) and the nricp result, so that a better face reconstruction effect can be obtained. Wherein the second fitting coefficient can be optimized by the following formula 3.

F = W（B_high*paras_high-（keypoints3d - vertex_low））+ W（paras_high /sigma）（3）

Wherein B_high is the high frequency 3dmm base (i.e., the second face base), and params_high is the determined 3dmm coefficient (i.e., the second fitting coefficient). Sigma is the variance obtained on the high frequency 3dmm basis, W is the coefficient, and represents the weight of each dimension in the params_high. The keypoints3d-vertex_low is the residual of the low frequency 3dmm model (vertex_low) and the nricp results described above. By optimizing equation 3 (the goal is to have a value of F as small as possible) a high frequency 3dmm coefficient can be obtained.

S304c3, outputting the updated three-dimensional face model.

And constructing the updated three-dimensional face model based on the first face base, the second face base, the first fitting coefficient and the second fitting coefficient.

In one possible implementation, when the updated three-dimensional face model is constructed based on the first face base, the second face base, the first fitting coefficient and the second fitting coefficient, the computer device obtains a sum of a mean value, a first product and a second product of the first face base as the updated three-dimensional face model; the first product is a product of the first face base and the first fitting coefficient, and the second product is a product of the second face base and the second fitting coefficient.

Through the scheme, the updated three-dimensional face model vertex_final obtained in one iteration updating process can be jointly determined by a low-frequency face base and a high-frequency face base, and is shown in the following formula 4:

vertex_final = MU_low + B_low * paras_low + B_high * paras_high（4）

the smoothness degree of the updated three-dimensional face model can be controlled by adjusting the proportion of the regular terms.

In one possible implementation, the second face base is a face base corresponding to a specified partial region in the face;

Alternatively, the second fitting coefficient is a fitting coefficient corresponding to a partial region specified in the face.

At this time, the second product, that is, the product of the second face base and the second fitting coefficient, only affects the designated partial area in the face, that is, the details of the high-frequency face base may only act on a certain portion (part). Such as only the fit cheek portion, thereby adding more details such as wrinkles (which are often not visible on the point cloud) so that the end result may appear more realistic.

And step 305, obtaining a three-dimensional face model of the reconstructed target face in response to the iteration stop condition being met.

In the embodiment of the application, the result of the dense fit (i.e., the updated three-dimensional face model) is sent to the NRICP process as an initial value (i.e., as a new pre-updated three-dimensional face model), and iterated multiple times until convergence.

In one possible implementation manner, the iteration stop condition may be that the number of iterations reaches a preset number threshold.

Alternatively, the iteration stop condition may be that a difference between the three-dimensional face model before updating and the three-dimensional face model after updating is smaller than a preset difference threshold.

Referring to fig. 6, a schematic diagram of a face reconstruction process based on an NRICP algorithm according to an embodiment of the present application is shown. As shown in fig. 6, the implementation of this scheme is as follows:

S61, a user collects high-precision face data, such as RGBD data of different angles of the face of the user, through a high-resolution depth camera in the terminal.

S62, the terminal sends the acquired face data to the server.

S63, the server builds a high-precision face point cloud based on the face data, for example, the face point cloud close to the real face is generated through the scheme shown in the step 302.

S64, the server builds a low-precision three-dimensional face model based on the face data, for example, by the sparse fitting scheme based on the key points shown in the step 303, and rapidly builds a low-precision initial three-dimensional face model.

S65, the server takes the face point cloud and the initial three-dimensional face model as input of an NRICP algorithm, and the following steps S66 to S68 are executed.

S66, for each point in the initial three-dimensional face model, the server finds the nearest point in the face point cloud as a corresponding point to obtain a plurality of point pairs.

S67, the server screens the distances among the points according to the points to obtain reliable point pairs.

And S68, the server optimizes error items and smooth items of the screened point pairs according to an NRICP algorithm to obtain the deformed three-dimensional face model.

And the server takes the three-dimensional face model obtained in the step S68 as the input of the step S66, repeatedly and iteratively executes the steps S66 to S68 until the difference value of the rotation offset matrix in the NRICP algorithm is smaller than a threshold value in the two previous and subsequent iteration processes, and outputs the matching relationship corruspore between the deformed three-dimensional face model and the face point cloud as a dense matching relationship.

And S69, the server carries out global fitting on the deformed three-dimensional face model through the low-frequency face base according to the dense matching relation obtained in the steps.

S610, the server fits through a high-frequency face base according to the residual error between the dense matching relation obtained in the steps and the global fitting result; and obtaining an updated three-dimensional face model after global fitting and residual fitting.

Referring to fig. 7, a schematic diagram of a model in a single iteration update according to an embodiment of the present application is shown. As shown in fig. 7, the left is the result of NRICP, the middle is the result of low frequency face-based fit, and the right is the result of high frequency face-based fit. As is apparent from fig. 7, after the high frequency face-based fit is added, there is more detail (more like a real person) and the five sense organs (nose shape, etc.) are more realistic.

S611, the server takes the updated three-dimensional face model as a new three-dimensional face model in S65, repeatedly and iteratively executes S65 to S610 until the stopping condition is met, and takes the finally output updated three-dimensional face model as a reconstructed face model.

And S612, the server generates a three-dimensional virtual image of the user according to the reconstructed face model and feeds the three-dimensional virtual image back to the terminal for display.

Please refer to fig. 8, which illustrates a schematic diagram of a face reconstruction effect according to an embodiment of the present application. The face reconstructed by the scheme shown in the embodiment of the application is more similar to the initial face point cloud compared with the sparse reconstructed low-mode (namely the initial three-dimensional face model), and is clearer compared with the model obtained by processing only through the NRICP algorithm, and excessive noise and burrs are avoided.

In summary, in the scheme provided by the embodiment of the application, after the three-dimensional face model before updating is deformed by taking the face point cloud as a target in each iteration process, the deformed three-dimensional face model is subjected to fitting updating based on two face bases with different precision (such as the low-frequency face base and the high-frequency face base), the updated three-dimensional face image is used as the three-dimensional face model before updating in the next iteration updating, and two face bases with different precision are introduced in each iteration updating process to perform face fitting, so that the accuracy of face reconstruction is improved.

In addition, in the scheme provided by the embodiment of the application, in each round of iterative process, the found nearest point pairs are filtered according to the respective distances when the nearest point pairs are found, so that the influence of a small number of wrong point pairs on the overall face reconstruction effect is avoided.

In addition, when the scheme provided by the embodiment of the application is used for fitting through the high-frequency face base, the scheme only acts on the appointed partial region in the face, so that the detail effect of the partial region in the face after fitting is further improved.

Fig. 9 is a block diagram illustrating a structure of a face reconstruction apparatus according to an exemplary embodiment. The face reconstruction device may implement all or part of the steps in the methods provided in the embodiments shown in fig. 1 or fig. 3. The face reconstruction device may include:

the face point cloud acquisition module 901 is used for acquiring a face point cloud of a target face;

the iteration updating module 902 is configured to iteratively update the three-dimensional face model of the target face based on the face point cloud until an iteration stopping condition is satisfied, thereby obtaining a reconstructed three-dimensional face model of the target face;

the iterative update module 902 includes:

the face deformation unit 902a is configured to perform deformation processing on the three-dimensional face model before updating with the face point cloud as a target, so as to obtain a deformed three-dimensional face model;

A correspondence acquiring unit 902b, configured to acquire a point correspondence between the deformed three-dimensional face model and the face point cloud, where the point correspondence is a correspondence between points of the two models;

a face fitting unit 902c, configured to perform face fitting based on the point correspondence, the first face base, and the second face base, to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies.

In one possible implementation manner, the face fitting unit 902c includes:

In a possible implementation manner, the face deformation unit 902a is configured to iteratively deform the three-dimensional face model before updating according to a non-rigid registration manner, with the face point cloud as a target, to obtain the deformed three-dimensional face model;

the face deformation unit 902a includes:

In one possible implementation, the apparatus further includes:

Fig. 10 shows a block diagram of a computer device 1000 in accordance with an exemplary embodiment of the present application. The computer device 1000 may be a user terminal or a server in the system shown in fig. 1.

In general, the computer device 1000 includes: a processor 1001 and a memory 1002.

The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 1001 may also include a main processor and a coprocessor. In some embodiments, the processor 1001 may integrate a GPU (Graphics Processing Unit, image processor), and the processor 1001 may further include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be non-transitory. Memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 1002 is used to store at least one instruction for execution by processor 1001 to implement all or part of the steps in the above-described method embodiments of the present application.

In some embodiments, when the computer device is implemented as a user terminal, the computer device 1000 may further optionally include: a peripheral interface 1003, and at least one peripheral. The processor 1001, the memory 1002, and the peripheral interface 1003 may be connected by a bus or signal line. The various peripheral devices may be connected to the peripheral device interface 1003 via a bus, signal wire, or circuit board. Optionally, the peripheral device includes: at least one of radio frequency circuitry 1004, touch display 1005, image acquisition component 1006, audio circuitry 1007, positioning component 1008, and power supply 1009.

Peripheral interface 1003 may be used to connect I/O (Input/Output) related at least one peripheral to processor 1001 and memory 1002.

Radio Frequency circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. Optionally, the radio frequency circuit 1004 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. Radio frequency circuitry 1004 may communicate with other computer devices via at least one wireless communication protocol. In some embodiments, the radio frequency circuitry 1004 may also include NFC (Near Field Communication ) related circuitry, which is not limiting of the application.

The display screen 1005 is used to display a UI (User Interface). When the display 1005 is a touch screen, the display 1005 also has the ability to capture touch signals at or above the surface of the display 1005.

The image acquisition component 1006 is used to acquire images or video. In some embodiments, camera assembly 1006 may also include a flash.

The audio circuit 1007 may include a microphone and a speaker. In some embodiments, audio circuit 1007 may also include a headphone jack.

The location component 1008 is used to locate the current geographic location of the computer device 1000 to enable navigation or LBS (Location Based Service, location-based services).

The power supply 1009 is used to power the various components in the computer device 1000.

In some embodiments, the computer device 1000 also includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: acceleration sensor 1011, gyroscope sensor 1012, pressure sensor 1013, fingerprint sensor 1014, optical sensor 1015, and proximity sensor 1016.

Those skilled in the art will appreciate that the architecture shown in fig. 10 is not limiting as to the computer device 1000, and may include more or fewer components than shown, or may combine certain components, or employ a different arrangement of components.

In exemplary embodiments, a non-transitory computer readable storage medium is also provided, such as a memory, including a computer program (instructions) executable by a processor of a computer device to perform the methods shown in the various embodiments of the application. For example, the non-transitory computer readable storage medium may be Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), compact disc Read-Only Memory (CD-ROM), magnetic tape, floppy disk, optical data storage device, and the like.

In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods shown in the above embodiments.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method of face reconstruction, the method comprising:

acquiring a face point cloud of a target face;

the iterative updating steps are as follows:

performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies;

The first face base is obtained by aligning first precision face data and second precision face data under the same coordinate system and performing principal component analysis on the aligned grid vertex coordinates, the first precision face data is face data with the precision of the face data higher than a threshold value, and the second precision face data is face data with the precision of the face data lower than the threshold value; the second face base is a face base obtained by fitting each piece of face data based on the first face base, obtaining an error vector corresponding to each piece of data, obtaining an error matrix according to the error vector, and calculating the error matrix through principal component analysis.

2. The method of claim 1, wherein the first face base is less accurate than the second face base;

the step of performing face fitting based on the point correspondence, the first face base and the second face base to obtain an updated three-dimensional face model comprises the following steps:

performing face fitting based on the point corresponding relation and the first face base to obtain a first fitting face model and a first fitting coefficient, wherein the first fitting coefficient is a coefficient of the first fitting face model obtained through reconstruction of the first face base;

Performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of a face model obtained through reconstruction of the second face base; the fitting residual error is the residual error between the point corresponding relation and the first fitting face model;

3. The method according to claim 2, wherein the performing face fitting based on the point correspondence and the first face base to obtain a first fitting face model and a first fitting coefficient includes:

4. The method according to claim 2, wherein the performing face fitting based on the fitting residual and the second face base to obtain a second fitting coefficient includes:

5. The method of claim 2, wherein the constructing the updated three-dimensional face model based on the first face base, the second face base, the first fitting coefficient, and the second fitting coefficient comprises:

obtaining the average value of the first face base, the sum of the first product and the second product as the updated three-dimensional face model;

6. The method of claim 2, wherein the second face base is a face base corresponding to a specified partial region in a face.

7. The method according to claim 1, wherein the deforming the three-dimensional face model before updating with the face point cloud as a target to obtain a deformed three-dimensional face model includes:

according to a non-rigid registration mode, taking the face point cloud as a target, carrying out iterative deformation on the three-dimensional face model before updating to obtain the deformed three-dimensional face model;

the iterative deformation steps are as follows:

acquiring each nearest point pair; any one target point pair of each nearest point pair is a point pair formed by the target point and the nearest point of the target point in the human face point cloud; the target point is any point in the three-dimensional face model before updating;

filtering each nearest point pair based on the distance information of each nearest point pair to obtain a filtered nearest point pair; the distance information is used for indicating the distance between two points in the corresponding point pair;

and registering based on the filtered closest point pair to obtain the three-dimensional face model after the deformation.

8. The method of claim 7, wherein filtering each closest point pair based on the distance information of the closest point pair to obtain a filtered closest point pair, comprises:

9. The method of claim 8, wherein the obtaining distance thresholds based on the distance information for the respective closest point pairs comprises:

10. The method according to claim 1, wherein the step of iteratively updating the three-dimensional face model of the target face based on the face point cloud until the iteration stop condition is satisfied, before obtaining the reconstructed three-dimensional face model of the target face, further comprises:

Acquiring RGBD data of the color depth of the target face;

acquiring an initial three-dimensional face model of the target face based on the RGBD data; the initial three-dimensional face model is the three-dimensional face model before updating, which is used in the iterative updating process for the first time.

11. The method of claim 10, wherein the obtaining an initial three-dimensional face model of the target face based on the RGBD data comprises:

based on the RGBD data, acquiring three-dimensional key points of the target face;

and constructing an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

12. A face reconstruction apparatus, the apparatus comprising:

the iterative updating module comprises:

the face fitting unit is used for performing face fitting based on the point corresponding relation, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different accuracies;

13. A computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the face reconstruction method of any one of claims 1 to 11.

14. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the face reconstruction method of any one of claims 1 to 11.