CN111710035A

CN111710035A - Face reconstruction method and device, computer equipment and storage medium

Info

Publication number: CN111710035A
Application number: CN202010685179.4A
Authority: CN
Inventors: 林祥凯; 王盛; 暴林超
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-07-16
Filing date: 2020-07-16
Publication date: 2020-09-25
Anticipated expiration: 2040-07-16
Also published as: CN111710035B

Abstract

The application relates to a face reconstruction method, a face reconstruction device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring a face point cloud of a target face; iteratively updating the three-dimensional face model of the target face based on the face point cloud to obtain a reconstructed three-dimensional face model of the target face; the steps of the iterative update are as follows: carrying out deformation processing on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model; acquiring the corresponding relation between the deformed three-dimensional face model and the face point cloud; performing face fitting based on the point correspondence, the first face basis and the second face basis to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions. According to the scheme, the face data obtained by scanning is subjected to face reconstruction based on two face bases with different precisions, so that the accuracy of face reconstruction is improved.

Description

Face reconstruction method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer vision, and in particular, to a method and an apparatus for reconstructing a human face, a computer device, and a storage medium.

Background

The 3D digital Model (3D) library is used to provide a standard face Model for three-dimensional (3 Dimensions, 3D) face reconstruction.

The three-dimensional face reconstruction method based on the 3DMM library is widely applied. In the related technology, a scheme for reconstructing a three-dimensional face based on a 3DMM library mainly calculates a conversion coefficient, and a face base in the 3DMM library can be converted into a three-dimensional face model close to a target face under the action of the conversion coefficient.

In the above technical solution, the expression capability of the face base in the 3DMM library is insufficient, for example, details of a target face cannot be accurately reconstructed, which results in a poor face reconstruction effect.

Disclosure of Invention

The embodiment of the application provides a face reconstruction method, a face reconstruction device, computer equipment and a storage medium, which can perform face reconstruction on face data obtained by scanning based on two face bases with different precisions, so that the accuracy of face reconstruction is improved, and the technical scheme is as follows:

in one aspect, a face reconstruction method is provided, and the method includes:

acquiring a face point cloud of a target face;

iteratively updating the three-dimensional face model of the target face based on the face point cloud until the three-dimensional face model of the target face is reconstructed when an iteration stop condition is met;

the steps of the iterative update are as follows:

carrying out deformation processing on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model;

acquiring a point correspondence between the deformed three-dimensional face model and the face point cloud, wherein the point correspondence is a correspondence between points of the two models;

performing face fitting based on the point correspondence, the first face basis and the second face basis to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions.

In yet another aspect, a face reconstruction apparatus is provided, the apparatus comprising:

the human face point cloud acquisition module is used for acquiring human face point cloud of a target human face;

the iteration updating module is used for performing iteration updating on the three-dimensional face model of the target face based on the face point cloud until the three-dimensional face model of the target face is reconstructed when an iteration stopping condition is met;

the iterative update module comprises:

the human face deformation unit is used for carrying out deformation processing on the three-dimensional human face model before updating by taking the human face point cloud as a target to obtain a deformed three-dimensional human face model;

a corresponding relation obtaining unit, configured to obtain a corresponding relation between the deformed three-dimensional face model and the face point cloud, where the corresponding relation is a corresponding relation between points of the two models;

the face fitting unit is used for carrying out face fitting on the basis of the point correspondence, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions.

In a possible implementation manner, the face fitting unit includes:

a first face fitting subunit, configured to perform face fitting based on the point correspondence and the first face basis to obtain the first fitted face model and a first fitting coefficient, where the first fitting coefficient is a coefficient of the first fitted face model obtained through reconstruction of the first face basis;

the second face fitting subunit is used for performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of a face model obtained through the second face base reconstruction; the fitting residual is a residual between the point correspondence and the first fitted face model;

and the face model updating subunit is configured to construct the updated three-dimensional face model based on the first face model, the second face model, the first fitting coefficient, and the second fitting coefficient.

In one possible implementation manner, the first face fitting subunit is configured to,

constructing a first loss function, wherein the first loss function is a function which takes the point correspondence, the first face base and the mean value of the first face base as input and takes a fitting coefficient as a variable;

obtaining a fitting coefficient which enables the value of the first loss function to be minimum as the first fitting coefficient;

and constructing the first fitting human face model based on the first fitting coefficient, the first human face base and the mean value of the first human face base.

In a possible implementation manner, the second face fitting subunit is configured to,

constructing a second loss function, wherein the second loss function is a function which takes the fitting residual error and the second face basis as input and takes a fitting coefficient as a variable;

and obtaining the fitting coefficient which enables the value of the second loss function to be minimum as the second fitting coefficient.

In a possible implementation manner, the face model sub-updating unit is configured to obtain a sum of a mean value of the first face base, a first product, and a second product as the updated three-dimensional face model;

wherein the first product is a product of the first face base and the first fitting coefficient, and the second product is a product of the second face base and the second fitting coefficient.

In one possible implementation, the second face base is a face base corresponding to a specified partial region in a face.

In a possible implementation manner, the face deformation unit is configured to perform iterative deformation on the three-dimensional face model before updating by taking the face point cloud as a target in a non-rigid registration manner, so as to obtain a deformed three-dimensional face model;

the face deformation unit further comprises:

a nearest point pair obtaining subunit, configured to obtain each nearest point pair; any one target point pair in each closest point pair is a point pair formed by a target point and the closest point of the target point in the human face point cloud; the target point is any one point in the three-dimensional face model before updating;

a nearest point pair filtering subunit, configured to filter each nearest point pair based on the distance information of each nearest point pair, to obtain a filtered nearest point pair; the distance information is used for indicating the distance between two points in the corresponding point pair;

and the nearest point pair registration subunit is used for registering based on the filtered nearest point pairs to obtain the deformed three-dimensional face model.

In one possible implementation, the nearest point pair filtering subunit is configured to,

acquiring a distance threshold value based on the distance information of each nearest point pair;

and filtering out the point pairs of which the corresponding distance indicated by the distance information is greater than the distance threshold value in each nearest point pair to obtain the filtered nearest point pairs.

In a possible implementation manner, the nearest point pair filtering subunit is configured to obtain a distance threshold based on distance information of the respective nearest point pairs, and further includes:

sorting the distances indicated by the distance information of each closest point according to a descending order;

obtaining a first distance threshold according to the sorting result, wherein the first distance threshold is the distance for segmenting each closest point according to a preset proportion in the distances indicated by the distance information of each closest point;

and acquiring the larger value of the first distance threshold value and a preset second distance threshold value as the distance threshold value.

In one possible implementation, the apparatus further includes:

the data acquisition module is used for acquiring the RGBD data of the color depth of the target face;

the initial model construction module is used for acquiring an initial three-dimensional face model of the target face based on the RGBD data; the initial three-dimensional face model is the three-dimensional face model before updating used in the iterative updating process for the first time.

In one possible implementation, the initial model building module includes:

a key point obtaining unit, configured to obtain three-dimensional key points of the target face based on the RGBD data;

and the initial model building unit is used for building an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

In yet another aspect, a computer device is provided, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the above-mentioned face reconstruction method.

In yet another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the storage medium, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by a processor to implement the above-mentioned face reconstruction method.

In yet another aspect, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the above-mentioned face reconstruction method.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

in each iteration process, after the three-dimensional face model before updating is deformed by taking the face point cloud as a target, the deformed three-dimensional face model is fitted and updated based on two face bases with different precisions (such as the low-frequency face base and the high-frequency face base), and the updated three-dimensional face image is used as the three-dimensional face model before updating in the next iteration update, namely, two face bases with different precisions are introduced into each iteration update process to perform face fitting, so that the accuracy of face reconstruction is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram illustrating a face reconstruction method according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a computer system provided in an exemplary embodiment of the present application;

FIG. 3 is a flowchart of a face reconstruction method according to an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of the effect of NRICP as related to the embodiment of FIG. 3;

FIG. 5 is a graph illustrating the effect of the nearest point on filtering according to the embodiment shown in FIG. 3;

FIG. 6 is a schematic diagram of a face reconstruction process based on the NRICP algorithm according to the embodiment shown in FIG. 3;

FIG. 7 is a schematic diagram of a model in a single iteration update involved in the embodiment shown in FIG. 3;

FIG. 8 is a schematic diagram illustrating the effect of face reconstruction according to the embodiment shown in FIG. 3;

FIG. 9 is a block diagram illustrating the structure of a face reconstruction apparatus according to an exemplary embodiment;

FIG. 10 is a block diagram illustrating a computer device according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

First, terms related to embodiments of the present application will be described.

Computer Vision (CV): the method is a science for researching how to make a machine see, and particularly refers to that a camera and a computer are used for replacing human eyes to perform machine vision such as identification, tracking and measurement on a target, and further graphics processing is performed, so that the computer processing becomes an image more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. The computer vision technology generally includes image processing, image Recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content/behavior Recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also includes common biometric technologies such as face Recognition and fingerprint Recognition.

First face base (also named low frequency face base): refers to a face base calculated based on first precision face data (also named high precision face data) and second precision face data (also named low precision face data). The first precision face data refers to face data with the precision higher than a threshold value, and the second precision face data refers to face data with the precision lower than the threshold value. The high-precision face grid data and the low-precision face grid data are aligned to the same coordinate system through data alignment (the high-precision face grid data and the low-precision face grid data have the same number of vertexes and the same topology, but the faces have different lengths, and the precision of the high-precision face grid data is higher than that of the low-precision face grid data), Principal Component Analysis (PCA) is carried out on the aligned grid vertex coordinates to obtain a PCA base, and the PCA base is the low-frequency face base. Illustratively, the face data obtained by scanning and processing a real face by using a scanning device with higher precision (such as a high-speed 3D camera) is used as the first-precision face data, and the face data obtained by scanning and processing a real face by using a device with lower precision (such as a smart phone) is used as the second-precision face data.

Second face base (also named high frequency face base): the method is a face base constructed based on errors generated when the first face base is fitted to face data. The first face basis is fitted to each data in the face data to obtain an error vector corresponding to each data, an error matrix is obtained according to the error vectors, and the error matrix is calculated through a Principal Component Analysis (PCA) technology to obtain a second face basis.

The face reconstruction method provided by the embodiment of the application is the application of the computer vision technology in the scene of three-dimensional face reconstruction. Please refer to fig. 1, which shows a schematic diagram of a human face reconstruction method according to an embodiment of the present application. As shown in fig. 1, the face reconstruction method is executed as follows:

and S11, acquiring a face point cloud of the target face.

The face point cloud of the target face is generated according to the actual face image of the target face.

In one possible implementation, the face point cloud of the target face is generated by a color depth image of the target face.

And S12, iteratively updating the three-dimensional face model of the target face based on the face point cloud until the three-dimensional face model of the reconstructed target face is obtained when the iteration stop condition is met.

Wherein, the iterative updating steps in each round are as follows:

and S12a, carrying out deformation processing on the three-dimensional face model before updating by taking the face point cloud as a target to obtain a deformed three-dimensional face model.

The method comprises the steps of taking a human face point cloud as a target, and carrying out deformation processing on a three-dimensional human face model before updating, namely, deforming the three-dimensional human face model before updating, so that the deformed three-dimensional human face model is closer to the human face point cloud corresponding to an actual target human face compared with the three-dimensional human face model before updating.

And S12b, acquiring the point corresponding relation between the deformed three-dimensional face model and the face point cloud, wherein the point corresponding relation is the corresponding relation between the points of the two models.

In the field of face reconstruction, the point correspondence relationship is also called correspondence, and refers to the point correspondence relationship between two models.

S12c, performing face fitting based on the point correspondence, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions.

In the embodiment of the application, in each iteration process, after the three-dimensional face model before updating is deformed by taking the face point cloud as a target, the deformed three-dimensional face model is fitted and updated based on two face bases with different precisions (such as the low-frequency face base and the high-frequency face base), the updated three-dimensional face image is used as the three-dimensional face model before updating in the next iteration update, and two face bases with different precisions are introduced into each iteration update process to perform face fitting, so that the accuracy of face reconstruction is improved.

The face reconstruction method provided by the embodiment of the application can be applied to the following scenes:

virtual character construction in game application

In the application scene, the method provided by the embodiment of the application can be applied to a background server of a game application program. When the virtual role is constructed, a user uses the terminal to collect face data of the user, and the collected face data is uploaded to the background server. The background server generates a face point cloud according to the face data, and carries out face reconstruction according to the face point cloud and the first face base and the second face base to construct a virtual role with the same face as the user. The virtual character is used for representing the identity of the user in the game application, and the user can also control the virtual character to play the game. Or, a terminal may generate a face point cloud according to the face data, and perform face reconstruction according to the face point cloud and the first face base and the second face base.

Virtual object construction in virtual reality application program

In the application scenario, the method provided by the embodiment of the application can be applied to a terminal or a server running a virtual reality application program. The method comprises the steps of acquiring face data of a user by using a camera component of a terminal, generating face point cloud based on the face data, and reconstructing a three-dimensional face model similar to the user in length according to the face point cloud and the first face base and the second face base, wherein the three-dimensional face model comprises some detail features of the user face, such as double-edged eyelids, raised lines, eye angle lines and the like, namely constructing the face of a virtual object similar to the real face of the user. The virtual object represents the identity of the user in the virtual reality application program, and the user performs video call, scene simulation and the like in the virtual reality application program by using the identity of the virtual object.

Third, virtual object construction in mixed reality application program

In the application scenario, the method provided by the embodiment of the application can be applied to a terminal or a server running a mixed reality application program. The method comprises the steps of acquiring face data of a user by using a camera component of a terminal or face data input in advance, generating face point cloud based on the face data, reconstructing a three-dimensional face model similar to the user length based on a first face base and a second face base, and constructing a virtual object by combining the rest body data input by the user. The user can perform corresponding operations on the virtual object, for example, wearing clothes on the virtual object, and realizing the effect of trying on the clothes on line.

The above description is given by taking only three application scenarios as an example, and the method provided in the embodiment of the present application may also be applied to other scenarios (such as virtual human construction, face restoration, and the like) that require reconstructing a high-accuracy three-dimensional face model according to input high-accuracy face data.

The face reconstruction method provided by the embodiment of the application can be applied to computer equipment with strong data processing capacity. In a possible implementation manner, the face reconstruction method provided by the embodiment of the present application may be applied to a personal computer, a workstation, or a server, that is, a high-precision face shape library may be constructed by the personal computer, the workstation, or the server.

Referring to FIG. 2, a schematic diagram of a computer system provided by an exemplary embodiment of the present application is shown. The computer system 200 includes a terminal 210 and a server 220, wherein the terminal 210 and the server 220 perform data communication through a communication network, optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.

The terminal 210 has an application program with a virtual character face shape display requirement installed therein, and the application program may be a virtual reality application program, a game application program, a dynamic face application program, or an Artificial Intelligence (AI) application program with a three-dimensional face generation function, which is not limited in this embodiment.

Optionally, the computer device 210 may be a mobile terminal such as a smart phone, a smart watch, a tablet computer, a laptop portable notebook computer, or may also be a terminal such as a desktop computer, a projection computer, and the like, which is not limited in this embodiment of the application.

The server 220 may be implemented as one server, or may be implemented as a server cluster formed by a group of servers, which may be physical servers or cloud servers. In one possible implementation, server 220 is a backend server for applications in computer device 210.

In a possible implementation manner of this embodiment, in the face reconstruction stage, the terminal 210 obtains high-precision face data of the target face and uploads the face data to the server 220. The terminal 210 may acquire the face data through a built-in face data acquisition component (such as a depth camera), or may acquire face data acquired and transmitted by other devices. The server 220 generates high-precision face point cloud after acquiring input high-precision face data, then iteratively updates the three-dimensional face model based on the face point cloud, in each iterative updating process, firstly takes the face point cloud as a target, deforms the three-dimensional face model, then fits according to a corresponding relation between a deformation result and the face point cloud and combines two face bases with different preset precisions to obtain an updated three-dimensional face model, and repeatedly executes the iterative updating process to obtain a reconstructed three-dimensional face model, so that the face reconstruction accuracy is improved. After the server 220 generates the reconstructed three-dimensional face model, an avatar is generated based on the reconstructed three-dimensional face model, and then the avatar is sent to the terminal 210 for display.

In another possible implementation manner, after the terminal 210 obtains high-precision face data of the target face, high-precision face point cloud is locally generated, and then the three-dimensional face model is iteratively updated based on the face point cloud to obtain a reconstructed three-dimensional face model. Optionally, the terminal 210 sends the reconstructed three-dimensional face model to the server 220, or generates an avatar based on the reconstructed three-dimensional face model and sends the avatar to the server 220.

For convenience of description, the following embodiments are described as examples in which the face reconstruction method is executed by a computer device. The computer device is the above-mentioned terminal 210 or the server 220.

Fig. 3 shows a flowchart of a face reconstruction method according to an exemplary embodiment of the present application. The embodiment is described by taking as an example that the method is used in the computer system 200 shown in fig. 2, and the method comprises the following steps:

step 301, obtaining face data of a target face.

In one possible implementation, the face data of the target face is a two-dimensional face image.

Illustratively, the two-dimensional face image is a color depth image of the target face.

The color depth image is also referred to as an R (red ) G (green, green) B (blue) -D (deep) image, which is abbreviated as an RGBD image.

In a possible implementation manner, the RGBD image of the target face is an image obtained by shooting the target face through a depth camera.

In one possible implementation, the face data of the target face is a high-precision color depth image, for example, an image captured by a high-precision depth camera.

In a possible implementation manner, the face data of the target face is an RGBD image of the target face in different face poses.

Step 302, generating a face point cloud of the target face based on the face data of the target face.

In a possible implementation manner, when the face data of the target face is RGBD images of the target face in different face poses, the computer device selects frames from the RGBD images in different poses through the face mark points obtained by detection, then calculates the camera pose corresponding to each frame, further cuts out the depth map point cloud, and finally synthesizes a complete face point cloud through the poses.

In other possible implementation manners, the computer device may also generate the face point cloud in other manners, for example, frame selection of the RGBD images is performed by manual assistance, point cloud matching is performed on each selected RGBD image to obtain a corresponding pose of each RGBD image, or the corresponding poses of each RGBD image are calculated by sparse points, and finally the complete face point cloud is synthesized by the poses. The embodiment of the application does not limit the way of generating the face point cloud of the target face.

Although the accuracy of the face point cloud generated in the above steps is higher, the details are closer to the real face, on one hand, the face point cloud generated by the face data usually has a deletion, for example, the point cloud of a part of face area may have a deletion due to image acquisition shielding and frame selection; on the other hand, the topological structure of the face point cloud generated by the face data is not fixed, and further driving cannot be performed on the basis, so that the application scene is limited. Therefore, in the subsequent steps of the embodiment of the application, based on the human face point cloud with higher accuracy, a three-dimensional human face model with a fixed topological structure and a complete human face area is further reconstructed.

Step 303, obtaining an initial three-dimensional face model of the target face based on the face data of the target face.

The initial three-dimensional face model is a three-dimensional face model before updating, which is used in the subsequent first iteration updating process.

In a possible implementation manner, when the face data of the target face is RGBD data, the computer device obtains three-dimensional key points of the target face based on the RGBD data, and constructs an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

For example, when the face data of the target face is RGBD images of the target face in different face poses, the computer device extracts two-dimensional key points and three-dimensional key points in the RGBD images in different face poses, performs fitting by using the two-dimensional key points, the three-dimensional key points, and the regular terms as constraints to obtain fitting coefficients on a three-dimensional deformation model base, and then obtains an initial three-dimensional face model through the fitting coefficients and the three-dimensional deformation model base.

In other possible implementation manners, the computer device may also generate the initial three-dimensional face model in other manners, for example, the computer device extracts two-dimensional key points in the RGBD images in different face poses, performs fitting using the two-dimensional key points and the regular terms as constraints to obtain fitting coefficients on the three-dimensional deformation model library, and then obtains the initial three-dimensional face model through the fitting coefficients and the three-dimensional deformation model library. The embodiment of the application does not limit the way of generating the face point cloud of the target face.

In one possible implementation, the computer device selects a fast-computing algorithm to generate the initial three-dimensional face model, for example, the initial three-dimensional face model is a model with low accuracy but fast generation speed.

And 304, iteratively updating the three-dimensional face model of the target face based on the face point cloud and the initial three-dimensional face model.

In the embodiment of the present application, the above-mentioned iterative updating steps are as follows:

s304a, with the face point cloud as a target, carrying out deformation processing on the three-dimensional face model before updating to obtain a deformed three-dimensional face model.

In the embodiment of the application, the three-dimensional face model before updating is subjected to deformation processing, so that the deformed three-dimensional face model is closer to the face point cloud, and the face point cloud is generated through face data and is closer to a real face, so that the deformed three-dimensional face model is also closer to the real face.

In a possible implementation manner, the computer device performs deformation processing on the three-dimensional face model before updating through a Non-rigid iterative closed Point (NRICP) algorithm.

In comparison with the Iterative Closest Point (ICP), NRICP can generate deformation in addition to performing translation transformation on the model. In NRICP, the feature points of an initial model (i.e., a three-dimensional face model before update) are used as source feature points, the feature points in the above-mentioned face point cloud are used as targets, and the purpose of NRICP is to find a transformation such that the source feature points can represent the points in the face point cloud.

In this embodiment, the step of deforming the three-dimensional face model before updating by the NRICP algorithm is as follows:

acquiring a three-dimensional face model before updating, wherein the three-dimensional face model before updating comprises a plurality of source characteristic points;

initializing a rotation and translation matrix x0 of each source characteristic point, wherein the rotation and translation matrix is used for carrying out rotation and translation operation on each source characteristic point;

in each iteration process, performing rotational translation transformation on each source characteristic point through the rotational translation matrix to obtain new characteristic points, fitting the obtained new characteristic points with corresponding points in the face point cloud, finding out the point in the face point cloud closest to each new characteristic point as a corresponding point, substituting all the new characteristic points and the corresponding points into a formula 1, and optimizing the new rotational translation matrix with the aim of minimizing the f value in the formula 1; and in the next iteration process, applying a new rotation and translation matrix to the source feature point, and iterating the operation until the difference between the new rotation and translation matrix and the rotation and translation matrix of the previous iteration is less than a certain threshold value, and ending the iteration.

The equation 1 is as follows:

wherein x1-xn is the rotational-translation matrix corresponding to each source feature point, v1-vn is each three-dimensional point in the source feature points, and u1 to un are the corresponding points of the new feature points. The formula (1) is divided into a left item and a right item, wherein the left item is that each three-dimensional point in the source characteristic points is subjected to rotational translation by using a rotational translation matrix, and then a new characteristic point obtained by the rotational translation is subtracted from a corresponding point to obtain a variance value. The right side is a quantity for keeping the original topology unchanged as much as possible, and M is the topology of the model, namely a feature point set capable of showing the structural features of the model by the minimum number of feature points.

Therefore, the intuitive understanding on the right side of equation (1) is that when x is 0, there is no rotational translation, and the original shape is kept unchanged. The proportion of the two is adjusted through the smoothing item a, the larger the smoothing item a is, the more the proportion of the representing topological structure in the energy function is, namely, the smoothing item has the effect that the rotating translation matrix can optimize the result of not only fitting the face point cloud, but also maintaining the model topology of the three-dimensional face model before updating, when a is larger, the proportion of the topological structure in the energy function is larger, the original topology can be considered more by optimizing the rotating translation matrix, and therefore the original topological structure can be reserved by utilizing the rotating translation matrix optimized through the larger smoothing item a, namely, the characteristics of the three-dimensional face model before updating are reserved. And then optimizing a rotation translation matrix optimized by a larger smooth item a through a smaller smooth item a, wherein the smaller smooth item a indicates that the proportion of errors of the feature points of the face point cloud and new feature points obtained by rotation translation is larger, and the model can more easily notice details in the face point cloud, namely, on the basis of keeping the model structure of the three-dimensional face model before updating, the face point cloud is utilized to optimize the rotation translation matrix in details, so that the deformed three-dimensional face model obtained by the three-dimensional face model before updating is acted by the final rotation translation matrix is more fit with a real face. For example, please refer to fig. 4, which shows a schematic diagram of the effect of NRICP according to an embodiment of the present application. As shown in fig. 4, the three-dimensional face model before updating is shown on the left side in fig. 4, the face point cloud is shown in the middle, and the result after applying NRICP to the three-dimensional face model before updating, that is, the deformed three-dimensional face model is shown on the right side.

In the execution process of the NRICP algorithm, corresponding points in the two models need to be determined to form each point pair so as to carry out rotation translation according to a rotation translation matrix; however, since the face point cloud is generated by the face data, on one hand, the face point cloud is closer to the model of the real face, and therefore, some details in the face point cloud cannot be expressed by the three-dimensional face model before updating, such as points corresponding to hairs in the face point cloud, and on the other hand, the face point cloud may be incomplete, and therefore, in the process of searching for corresponding points in the face cloud corresponding to the source feature points, the searched corresponding points may deviate far, resulting in a search error. In a possible implementation manner, the computer device performs iterative deformation on the three-dimensional face model before updating by taking the face point cloud as a target in a non-rigid registration manner on the basis of the NRICP algorithm to obtain a deformed three-dimensional face model; the steps of iterative morphing are as follows:

s304a1, finding the closest point as the corresponding point.

Obtaining each nearest point pair; any one target point pair in each closest point pair is a point pair formed by a target point and the closest point of the target point in the human face point cloud; the target point is any one point in the three-dimensional face model before updating.

The nearest point pair is a pair of feature points formed by one point in the three-dimensional face model before updating and corresponding points in the face point cloud.

S304a2, screening out reliable corresponding points.

Filtering each nearest point pair based on the distance information of each nearest point pair to obtain the filtered nearest point pair; the distance information is used to indicate the distance between two points in the corresponding pair of points.

In a possible implementation manner, when filtering each nearest point pair based on the distance information of each nearest point pair to obtain the filtered nearest point pair, the computer device obtains a distance threshold based on the distance information of each nearest point pair; and filtering the point pairs of which the corresponding distance indicated by the distance information is greater than the distance threshold value in each closest point pair to obtain the filtered closest point pair.

In the embodiment of the application, the computer device dynamically determines the distance threshold value by combining the distance corresponding to the distance information of each nearest point pair, and deletes the nearest point pair of which the distance between the two points is greater than the distance threshold value.

In one possible implementation manner, when the distance threshold is obtained based on the distance information of each closest point pair, the computer device sorts the distances indicated by the distance information of each closest point in the order from small to large; obtaining a first distance threshold value according to the sorting result, wherein the first distance threshold value is the distance for segmenting each closest point according to a preset proportion in the distances indicated by the distance information of each closest point; and acquiring the larger value of the first distance threshold value and a preset second distance threshold value as the distance threshold value.

In the scheme of the application, a screening process is newly added in the process of finding the closest point. The reason is that the point cloud of the face is incomplete, and there may exist details that the three-dimensional face model cannot express, and the closest points found by many points are actually far away, so that errors are generated, and due to the existence of the smoothing term (a), the closest points with large errors may affect the overall face reconstruction effect, so that these error points need to be deleted. In the embodiment of the present application, the distance is used as a threshold value for determination. For example, the computer device calculates each point in the three-dimensional face model before updating and the distance between the closest point in the face point cloud, sorts all the obtained distances, takes the distance value on the critical point with a preset proportion (for example, the top 70%) of the error and the maximum value in the preset absolute threshold as the distance threshold, i.e., can screen the inaccurate closest point pairs in the current iteration updating process, and the screened closest point pairs are filled by the smoothing term in the NRICP, and the error is less and less as the iteration progresses and the smoothing term is reduced.

And S304a3, optimizing the error term and the smoothing term of the corresponding point.

And registering based on the filtered nearest point pair, and optimizing an error term and a smooth term of the corresponding point to obtain the deformed three-dimensional face model.

For example, please refer to fig. 5, which illustrates a diagram of the effect of the nearest point on filtering according to an embodiment of the present application. As shown in fig. 5, as the iteration progresses, the number of wrong nearest point pairs is less, and the scheme can well avoid that a few wrong point pairs cause large interference. As can be seen from fig. 5, compared to the initial three-dimensional face model, the three-dimensional face model after iterative update is closer to the real front-end point cloud (i.e., the above-mentioned face point cloud).

In the embodiment of the application, after the computer device obtains each nearest point pair between the three-dimensional face model before updating and the face point cloud, the computer device further filters each nearest point pair according to the distance information of each nearest point pair to remove wrong point pairs as much as possible, so that the robustness of the NRICP algorithm is improved, and the accuracy of face reconstruction is further improved.

S304b, acquiring the point corresponding relation between the deformed three-dimensional face model and the face point cloud, wherein the point corresponding relation is the corresponding relation between the points of the two models.

The point correspondence between the deformed three-dimensional face model and the face point cloud also becomes correspondance or is called matching relationship.

S304c, performing face fitting based on the point correspondence, the first face base and the second face base to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions.

The dense matching relationship obtained in the above steps 304a and 304b is equivalent to a constraint of dense three-dimensional feature points (3 dkeypoints). However, due to the insufficient expression capability of the commonly used 3dmm base table, in order to improve the accuracy of face reconstruction, in the embodiment of the present application, fitting is performed through face bases with different precisions (i.e., a high-frequency face base and a low-frequency face base), so that in each iteration process, fitting updating is performed on the three-dimensional face model.

In the embodiment of the application, the precision of the first face base is lower than that of the second face base; when performing face fitting based on the point correspondence, the first face basis, and the second face basis to obtain an updated three-dimensional face model, the computer device may perform the following steps:

s304c1, fitting the basic facial form with the low frequency basis.

And carrying out face fitting based on the point correspondence and the first face basis to obtain a first fitted face model and a first fitting coefficient, wherein the first fitting coefficient is a coefficient of the first fitted face model obtained by reconstructing the first face basis.

In a possible implementation manner, when performing face fitting based on the point correspondence and the first face basis to obtain the first fitted face model and a first fitting coefficient, the computer device constructs a first loss function, where the first loss function is a function in which the point correspondence, the first face basis, and an average value of the first face basis are input, and the fitting coefficient is a variable; obtaining a fitting coefficient which enables the value of the first loss function to be minimum as the first fitting coefficient; and constructing the first fitting human face model based on the first fitting coefficient, the first human face base and the mean value of the first human face base.

In one possible implementation, the computer device optimizes the low frequency model by equation 2. Equation 2 is as follows:

F＝W(MU_low+B_low*paras_low-keypoints3d)+W(paras_low/sigma) (2)

where MU _ low is the mean of the low frequency 3dmm basis (i.e., the first face basis), B _ low is the low frequency 3dmm basis (i.e., the first face basis), and param _ low is the calculated 3dmm coefficient (i.e., the first fitting coefficient). Sigma is the variance obtained on the low frequency 3dmm basis, and W is a coefficient representing the weight for each dimension in the param _ low. keypoints3d is the result of nrichp, the point correspondence described above. By optimizing equation 2 (the goal is to make the value of F as small as possible), a low frequency 3dmm coefficient and thus a low frequency model can be obtained (vertex _ low ═ MU _ low + B _ low ═ paras _ low). The vertex _ low is the first fitted face model.

S304c2, fitting the residuals with high frequency bases.

Performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of the face model obtained through the second face base reconstruction; the fitting residual is a residual between the point correspondence and the first fitted face model.

In a possible implementation manner, when performing face fitting based on the fitting residual and the second face basis to obtain a second fitting coefficient, the computer device constructs a second loss function, where the second loss function is a function in which the fitting residual and the second face basis are used as inputs, and the fitting coefficient is used as a variable; and obtaining the fitting coefficient which enables the value of the second loss function to be minimum as the second fitting coefficient.

In the embodiment of the present application, the computer device fits (fit) the residual error through the residual error between the low-frequency 3dmm model (vertex _ low) and the nricp result and through the high-frequency basis (i.e., the second face basis), so as to obtain a better face reconstruction effect. Wherein, the second fitting coefficient can be optimized by the following formula 3.

F＝W(B_high*paras_high-(keypoints3d-vertex_low))+W(paras_high/sigma)(3)

Where B _ high is the high frequency 3dmm basis (i.e., the second face basis), and para _ high is the calculated 3dmm coefficient (i.e., the second fitting coefficient). Sigma is the variance obtained on the high frequency 3dmm basis, and W is a coefficient representing the weight for each dimension in the para _ high. keypoints3d-vertex _ low is the residual of the low frequency 3dmm model (vertex _ low) and nricp results described above. A high frequency 3dmm coefficient can be obtained by optimizing equation 3 (the goal is that the value of F is as small as possible).

And S304c3, outputting the updated three-dimensional face model.

And constructing the updated three-dimensional face model based on the first face model, the second face model, the first fitting coefficient and the second fitting coefficient.

In a possible implementation manner, when the updated three-dimensional face model is constructed based on the first face model, the second face model, the first fitting coefficient and the second fitting coefficient, the computer device obtains a sum of a mean value of the first face base, a first product and a second product as the updated three-dimensional face model; wherein the first product is a product of the first face base and the first fitting coefficient, and the second product is a product of the second face base and the second fitting coefficient.

By the above scheme, the updated three-dimensional face model vertex _ final obtained in the one-time iterative update process can be determined by the low-frequency face basis and the high-frequency face basis together, as shown in the following formula 4:

vertex_final＝MU_low+B_low*paras_low+B_high*paras_high (4)

the smoothness degree of the updated three-dimensional face model can be controlled by adjusting the proportion of the regular term.

In one possible implementation, the second face base is a face base corresponding to a specified partial region in the face;

alternatively, the second fitting coefficient is a fitting coefficient corresponding to a partial region specified in the face.

In this case, the second product, i.e. the product of the second face basis and the second fitting coefficient, only affects the specified partial region of the face, i.e. the details of the high-frequency face basis can only be applied to a certain part (part). For example, only fit the cheek portion, thereby adding more details such as wrinkles (which are often not visible in the point cloud), and thus making the final result more realistic.

Step 305, in response to the iteration stop condition being met, a three-dimensional face model of the reconstructed target face is obtained.

In the embodiment of the present application, the result of the dense fit (i.e., the updated three-dimensional face model) is sent to the NRICP process as an initial value (i.e., as a new three-dimensional face model before updating), and iteration is performed for multiple times until convergence.

In a possible implementation manner, the iteration stop condition may be that the number of iterations reaches a preset number threshold.

Alternatively, the iteration stop condition may be that a difference between the three-dimensional face model before updating and the three-dimensional face model after updating is smaller than a preset difference threshold.

Please refer to fig. 6, which shows a schematic diagram of a face reconstruction process based on NRICP algorithm according to an embodiment of the present application. As shown in fig. 6, the implementation of this scheme is as follows:

and S61, the user acquires high-precision face data, such as RGBD data of different angles of the face of the user, through a high-resolution depth camera in the terminal.

And S62, the terminal sends the collected face data to a server.

S63, the server constructs a high-precision point cloud of the face based on the face data, for example, by the scheme shown in step 302, a point cloud of the face close to the real face is generated.

S64, the server constructs a low-precision three-dimensional face model based on the face data, for example, by using the sparse fitting scheme based on the key points as shown in step 303, an initial low-precision three-dimensional face model is quickly constructed.

S65, the server takes the face point cloud and the initial three-dimensional face model as input to the NRICP algorithm, and performs the subsequent steps S66 to S68.

And S66, the server finds the closest point in the face point cloud as the corresponding point for each point in the initial three-dimensional face model to obtain a plurality of point pairs.

And S67, the server screens the distances between the points of each point pair to obtain reliable point pairs.

And S68, the server optimizes the error term and the smoothing term of the screened point pairs according to the NRICP algorithm to obtain a deformed three-dimensional face model.

And the server takes the three-dimensional face model obtained in the step S68 as the input of the step S66, and repeatedly iterates the steps S66 to S68 until the difference value of the rotation offset matrix in the NRICP algorithm is smaller than the threshold value in the two iteration processes, and outputs the matching relationship correspondence between the deformed three-dimensional face model and the face point cloud as a dense matching relationship.

And S69, the server performs global fitting on the deformed three-dimensional face model through the low-frequency face base according to the dense matching relation obtained in the step.

S610, the server performs fitting through a high-frequency face base according to the residual error between the dense matching relation and the global fitting result obtained in the step; and obtaining an updated three-dimensional face model after the global fitting and the residual fitting.

Please refer to fig. 7, which shows a schematic diagram of a model in a single iteration update according to an embodiment of the present application. As shown in fig. 7, the left is the result of NRICP, the middle is the result of low frequency face basis fit, and the right is the result of high frequency face basis fit. As is apparent from fig. 7, after the high-frequency face basis fit is added, richer details (more like a real person) are obtained, and the five sense organs (nose shape and the like) are more real.

And S611, the server takes the updated three-dimensional face model as a new three-dimensional face model in S65, repeatedly and iteratively executes S65-S610 until a stop condition is met, and finally outputs the updated three-dimensional face model as a reconstructed face model.

And S612, the server generates a three-dimensional virtual image of the user according to the reconstructed face model and feeds the three-dimensional virtual image back to the terminal for display.

Please refer to fig. 8, which illustrates a schematic diagram of a face reconstruction effect according to an embodiment of the present application. Compared with a sparsely reconstructed low-mode (namely, an initial three-dimensional face model), the face reconstructed by the scheme shown in the embodiment of the application is more like initial face point cloud, and compared with a model obtained by processing through an NRICP algorithm, the face reconstructed by the scheme shown in the embodiment of the application is clearer and does not have excessive noise and burrs.

To sum up, in the scheme provided in the embodiment of the present application, in each iteration, after the three-dimensional face model before updating is deformed by taking a face point cloud as a target, the deformed three-dimensional face model is fitted and updated based on two face bases with different precisions (such as the low-frequency face base and the high-frequency face base), the updated three-dimensional face image is then used as the three-dimensional face model before updating in the next iteration, and two face bases with different precisions are introduced into each iteration updating to perform face fitting, so that the accuracy of face reconstruction is improved.

In addition, according to the scheme provided by the embodiment of the application, in each iteration process, when the closest point pair is searched, the searched closest point pair is filtered according to respective distance, so that the influence of a small number of wrong point pairs on the overall face reconstruction effect is avoided.

In addition, the scheme provided by the embodiment of the application only acts on the specified partial region in the face when the fitting is performed through the high-frequency face base, so that the detail effect of the partial region in the face after fitting is further improved.

Fig. 9 is a block diagram illustrating a structure of a face reconstruction apparatus according to an exemplary embodiment. The face reconstruction device can implement all or part of the steps in the method provided by the embodiment shown in fig. 1 or fig. 3. The face reconstruction device may include:

a face point cloud obtaining module 901, configured to obtain a face point cloud of a target face;

an iteration update module 902, configured to perform iteration update on the three-dimensional face model of the target face based on the face point cloud, until an iteration stop condition is met, to obtain a reconstructed three-dimensional face model of the target face;

the iterative update module 902 includes:

a face deformation unit 902a, configured to perform deformation processing on the three-dimensional face model before updating by using the face point cloud as a target, so as to obtain a deformed three-dimensional face model;

a correspondence obtaining unit 902b, configured to obtain a correspondence between the deformed three-dimensional face model and the face point cloud, where the correspondence is a correspondence between points of the two models;

a face fitting unit 902c, configured to perform face fitting based on the point correspondence, the first face basis, and the second face basis to obtain an updated three-dimensional face model; the first face base and the second face base have different precisions.

In a possible implementation manner, the face fitting unit 902c includes:

In a possible implementation manner, the face deformation unit 902a is configured to perform iterative deformation on the three-dimensional face model before updating by taking the face point cloud as a target in a non-rigid registration manner, so as to obtain a deformed three-dimensional face model;

the face morphing unit 902a includes:

In one possible implementation, the apparatus further includes:

In one possible implementation, the initial model building module includes:

Fig. 10 shows a block diagram of a computer device 1000 according to an exemplary embodiment of the present application. The computer device 1000 may be a user terminal or a server in the system shown in fig. 1.

Generally, the computer device 1000 includes: a processor 1001 and a memory 1002.

Processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 1001 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 1001 may also include a main processor and a coprocessor. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit), and the processor 1001 may further include an AI (Artificial Intelligence) processor for Processing a computing operation related to machine learning.

Memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 1002 is used to store at least one instruction for execution by the processor 1001 to implement all or part of the steps of the above-described method embodiments of the present application.

In some embodiments, when the computer device is implemented as a user terminal, the computer device 1000 further optionally includes: a peripheral interface 1003 and at least one peripheral. The processor 1001, memory 1002 and peripheral interface 1003 may be connected by a bus or signal line. Various peripheral devices may be connected to peripheral interface 1003 via a bus, signal line, or circuit board. Optionally, the peripheral device includes: at least one of radio frequency circuitry 1004, touch screen display 1005, image capture assembly 1006, audio circuitry 1007, positioning assembly 1008, and power supply 1009.

The peripheral interface 1003 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 1001 and the memory 1002.

The Radio Frequency circuit 1004 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. Optionally, the radio frequency circuit 1004 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 1004 may communicate with other computer devices via at least one wireless communication protocol. In some embodiments, the rf circuit 1004 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 1005 is used to display a UI (User Interface). When the display screen 1005 is a touch display screen, the display screen 1005 also has the ability to capture touch signals on or over the surface of the display screen 1005.

The image capture component 1006 is used to capture images or video. In some embodiments, camera assembly 1006 may also include a flash.

The audio circuit 1007 may include a microphone and a speaker. In some embodiments, the audio circuit 1007 may also include a headphone jack.

The Location component 1008 is used to locate the current geographic Location of the computer device 1000 for navigation or LBS (Location Based Service).

The power supply 1009 is used to supply power to the various components in the computer device 1000.

In some embodiments, the computer device 1000 also includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: acceleration sensor 1011, gyro sensor 1012, pressure sensor 1013, fingerprint sensor 1014, optical sensor 1015, and proximity sensor 1016.

Those skilled in the art will appreciate that the configuration shown in FIG. 10 is not intended to be limiting of the computer device 1000, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as a memory comprising computer programs (instructions), executable by a processor of a computer device to perform the methods shown in the various embodiments of the present application, is also provided. For example, the non-transitory computer readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods shown in the various embodiments described above.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for reconstructing a human face, the method comprising:

acquiring a face point cloud of a target face;

the steps of the iterative update are as follows:

2. The method of claim 1, wherein the first face base is less accurate than the second face base;

performing face fitting based on the point correspondence, the first face basis and the second face basis to obtain an updated three-dimensional face model, including:

performing face fitting based on the point correspondence and the first face basis to obtain a first fitted face model and a first fitting coefficient, wherein the first fitting coefficient is a coefficient of the first fitted face model obtained through reconstruction of the first face basis;

performing face fitting based on the fitting residual error and the second face base to obtain a second fitting coefficient; the second fitting coefficient is a coefficient of a face model obtained through the second face base reconstruction; the fitting residual is a residual between the point correspondence and the first fitted face model;

3. The method of claim 2, wherein the performing face fitting based on the point correspondences and the first face basis to obtain the first fitted face model and a first fitting coefficient comprises:

4. The method of claim 2, wherein the performing face fitting based on the fitting residuals and the second face bases to obtain second fitting coefficients comprises:

5. The method of claim 2, wherein constructing the updated three-dimensional face model based on the first face model, the second face model, the first fitting coefficients, and the second fitting coefficients comprises:

acquiring the sum of the mean value of the first face base, the first product and the second product as the updated three-dimensional face model;

6. The method of claim 2, wherein the second face base is a face base corresponding to a specified partial region in a face.

7. The method of claim 1, wherein the deforming the three-dimensional face model before updating with the face point cloud as a target to obtain a deformed three-dimensional face model comprises:

performing iterative deformation on the three-dimensional face model before updating by taking the face point cloud as a target according to a non-rigid registration mode to obtain a deformed three-dimensional face model;

the steps of iterative deformation are as follows:

obtaining each nearest point pair; any one target point pair in each closest point pair is a point pair formed by a target point and the closest point of the target point in the human face point cloud; the target point is any one point in the three-dimensional face model before updating;

filtering each nearest point pair based on the distance information of each nearest point pair to obtain the filtered nearest point pair; the distance information is used for indicating the distance between two points in the corresponding point pair;

and registering based on the filtered nearest point pair to obtain the deformed three-dimensional face model.

8. The method according to claim 7, wherein the filtering the nearest point pairs based on the distance information of the nearest point pairs to obtain filtered nearest point pairs comprises:

9. The method according to claim 8, wherein the obtaining a distance threshold based on the distance information of the respective closest point pairs comprises:

10. The method of claim 1, wherein the iteratively updating the three-dimensional face model of the target face based on the face point cloud until a reconstructed three-dimensional face model of the target face is obtained when an iteration stop condition is satisfied, further comprising:

acquiring RGBD data of the color depth of the target face;

acquiring an initial three-dimensional face model of the target face based on the RGBD data; the initial three-dimensional face model is the three-dimensional face model before updating used in the iterative updating process for the first time.

11. The method of claim 10, wherein obtaining the initial three-dimensional face model of the target face based on the RGBD data comprises:

acquiring three-dimensional key points of the target face based on the RGBD data;

and constructing an initial three-dimensional face model of the target face based on the three-dimensional key points of the target face.

12. An apparatus for face reconstruction, the apparatus comprising:

the iterative update module comprises:

13. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement a face reconstruction method according to any one of claims 1 to 11.

14. A computer-readable storage medium, having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement a method of face reconstruction according to any one of claims 1 to 11.