CN111581411A

CN111581411A - Method, device and equipment for constructing high-precision face shape library and storage medium

Info

Publication number: CN111581411A
Application number: CN202010519649.XA
Authority: CN
Inventors: 王盛; 林祥凯
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-06-09
Filing date: 2020-06-09
Publication date: 2020-08-25
Anticipated expiration: 2040-06-09
Also published as: CN111581411B

Abstract

The application discloses a method, a device, equipment and a storage medium for constructing a high-precision face shape library, and relates to the field of computer vision. The method comprises the following steps: acquiring a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; constructing a second facial contour base through the first facial contour base and the first fitting error; fitting the disturbance data through the first face base to obtain a second fitting error; fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base; and iteratively updating the expanded first fitting error according to the second fitting error and the third fitting error, and constructing a high-precision face shape library according to the second face shape base and the first face shape base obtained when the iteration is finished. The data demand of the first-precision face data is reduced, so that the construction cost of the high-precision face shape library is reduced, and the construction efficiency of the high-precision face shape library is improved.

Description

Method, device and equipment for constructing high-precision face shape library and storage medium

Technical Field

The present application relates to the field of computer vision, and in particular, to a method, an apparatus, a device, and a storage medium for constructing a high-precision face shape library.

Background

The 3D digital Model (3D) library is used to provide a standard face Model for three-dimensional (3Dimensions, 3D) face reconstruction.

Taking a Basel Face Model (BFM) as an example, original Face data is collected by a high-precision three-dimensional scanner, after the original Face data is obtained, a Face mesh similar to the original Face data under a specific Model can be obtained by registering the original Face data corresponding to non-rigid registration (which is also named as non-rigid icp or nrichp), and then the obtained Face mesh is processed by a principal component analysis technology to construct a BFM shape library.

In the technical scheme, the BFM shape library needs to collect a large amount of high-precision face data, so that the process of constructing the 3DMM face shape library is difficult.

Disclosure of Invention

The embodiment of the application provides a method, a device, equipment and a storage medium for constructing a high-precision face shape library, and the high-precision face shape library is constructed by combining a first face shape base and a second face shape base, so that the construction difficulty of the 3DMM face shape library is reduced. The technical scheme is as follows:

according to an aspect of the present application, there is provided a method for constructing a high-precision face shape library, the method including:

acquiring a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; the precision of the first precision face data is higher than that of the second precision face data;

constructing a second face base through the first face base and a first fitting error, wherein the first fitting error is error information generated when the first face base is fitted with extended data, and the extended data is obtained by extending the first precision face data and the second precision face data;

fitting disturbance data through the first facial form base to obtain a second fitting error, wherein the disturbance data are obtained after disturbance processing is carried out on the basis of the expansion data; fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base;

and iteratively updating the expanded first fitting error according to the second fitting error and the third fitting error, and constructing the high-precision face shape library according to the second face shape base and the first face shape base obtained when iteration is finished.

According to another aspect of the present application, there is provided an apparatus for constructing a high-precision face shape library, the apparatus including:

the data acquisition module is used for acquiring a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; the precision of the first precision face data is higher than that of the second precision face data;

the construction module is used for constructing a second face base through the first face base and a first fitting error, the first fitting error is error information generated when the first face base is fitted with extended data, and the extended data is obtained after the extended data is extended based on the first precision face data and the second precision face data;

the data fitting module is used for fitting disturbance data through the first facial form base to obtain a second fitting error, and the disturbance data are obtained after disturbance processing is carried out on the basis of the expansion data; fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base;

and the iteration updating module is used for performing iteration updating on the expanded first fitting error according to the second fitting error and the third fitting error, and constructing the high-precision human face shape library according to the second facial form base and the first facial form base obtained when the iteration is finished.

According to another aspect of the present application, there is provided a computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the method of constructing a high-precision face shape library as described in the above aspect.

According to another aspect of the present application, there is provided a computer-readable storage medium having at least one instruction, at least one program, a set of codes, or a set of instructions stored therein, which is loaded and executed by a processor to implement the method for constructing a high-precision face shape library as described in the above aspect.

According to another aspect of the present application, there is provided a computer program product which, when run on a computer, causes the computer to execute the method of constructing a high-precision face shape library as described in the above aspect.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

the embodiment of the application is based on the idea of residual error data, a second face base is constructed through a first face base, the expanded first fitting error is iteratively updated according to a second fitting error corresponding to the first face base and a third fitting error corresponding to the second face base, and a high-precision face shape library is constructed according to the second face base and the first face base obtained at the end of iteration. According to the face shape library constructed by combining the second face shape base and the first face shape base, on the premise that the constructed face shape library has high precision, the demand for the first precision face data (with the face data with higher precision) is reduced, the construction difficulty of the high-precision face shape library is reduced, and the construction efficiency of the high-precision face shape library is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram illustrating a method for constructing a high-precision face shape library according to an exemplary embodiment of the present application;

FIG. 2 illustrates a block diagram of a computer system provided by an exemplary embodiment of the present application;

FIG. 3 is a flow chart illustrating a method for constructing a high-precision face shape library according to an exemplary embodiment of the present application;

FIG. 4 is a flow chart illustrating a method for building a high-precision face shape library according to another exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of a fitting process of third party face data provided by an exemplary embodiment;

FIG. 6 is a schematic diagram of a data augmentation process for face data provided by an exemplary embodiment;

FIG. 7 is a flowchart of a method for computing a target face based on a high-precision face shape library according to an exemplary embodiment of the present application;

FIG. 8 is a comparison of human face shape fit effects provided by an exemplary embodiment;

FIG. 9 is a flow chart of the construction of a high-precision face shape library provided by an exemplary embodiment of the present application;

fig. 10 is a block diagram illustrating a construction apparatus of a high-precision face shape library according to an exemplary embodiment of the present application;

fig. 11 is a schematic diagram illustrating an apparatus structure of a server according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

First, terms related to embodiments of the present application will be described.

Computer Vision (CV): the method is a science for researching how to make a machine see, and particularly refers to that a camera and a computer are used for replacing human eyes to perform machine vision such as identification, tracking and measurement on a target, and further graphics processing is performed, so that the computer processing becomes an image more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. The computer vision technology generally includes image processing, image Recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content/behavior Recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also includes common biometric technologies such as face Recognition and fingerprint Recognition.

First faceting (also named low frequency faceting): refers to a face shape base calculated based on first precision face data (also named high precision face data) and second precision face data (also named low precision face data). The first precision face data refers to face data with the precision higher than a threshold value, and the second precision face data refers to face data with the precision lower than the threshold value. The high-precision face grid data and the low-precision face grid data are aligned to the same coordinate system through data alignment (the high-precision face grid data and the low-precision face grid data have the same number of vertexes and the same topology, but have different face lengths, and the precision of the high-precision face grid data is higher than that of the low-precision face grid data), Principal Component Analysis (PCA) is carried out on the aligned grid vertex coordinates to obtain a PCA base, and the PCA base is a low-frequency shape base. Illustratively, the face data obtained by scanning and processing a real face by using a scanning device with higher precision (such as a high-speed 3D camera) is used as the first-precision face data, and the face data obtained by scanning and processing a real face by using a device with lower precision (such as a smart phone) is used as the second-precision face data.

Second facer (also named high frequency facer): the face shape base is constructed based on errors generated when the first face shape base is fitted to the face data. And fitting the first face base to each data in the face data to obtain an error vector corresponding to each data, obtaining an error matrix according to the error vector, and calculating the error matrix by a Principal Component Analysis (PCA) technology to obtain a second face base.

AI (Artificial Intelligence) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, face recognition, three-dimensional face model reconstruction, and the like.

The method for constructing the high-precision face shape library provided by the embodiment of the application is the application of the computer vision technology in the scene of three-dimensional face reconstruction. With the method provided by the embodiment of the present application, as shown in fig. 1, a computer device first performs Principal Component Analysis (PCA) calculation on input high-precision face data 11 (the total amount of data is much smaller than that of high-precision face data used for constructing a high-precision face shape library in the related art) and low-precision face data 12, respectively, to obtain a low-frequency face shape base 13 (the low-frequency face shape base is a low-frequency face shape base that has been iteratively updated). The computer device calculates the error of the low frequency face shape base 13 to obtain an error vector, obtains an error matrix (first fitting error) according to the expanded data, and expands the error matrix along the negative direction to obtain an expanded error matrix. And PCA computation is performed on the extended error matrix to construct the high frequency face shape bases 14. And calculating a low-frequency base error of the low-frequency facial form base 13 to obtain a low-frequency base error (a second fitting error), and calculating a high-frequency base error of the high-frequency facial form base 14 by combining the low-frequency base error to obtain a high-frequency base error (a third fitting error). And obtaining error data with the maximum previous N error values from the high-frequency base error. And recording the position of each error data in the high-frequency base error, determining corresponding error data from the low-frequency base error according to the positions, and obtaining a screening error matrix according to the error data in the low-frequency base error. And expanding the screening error matrix along the negative direction to obtain an expanded screening error matrix, and adding the expanded screening error matrix into the error matrix to obtain another expanded error matrix. A high frequency base error is calculated from the further extended error matrix. For the high-frequency face shape base 14, the computer device iteratively executes the process of calculating the high-frequency base error, and iteratively updates the high-frequency base error until the high-frequency base error meets the iteration convergence condition, and determines the high-frequency face shape base and the low-frequency face shape base (the iteratively updated low-frequency face shape base) obtained at the end of the iteration as the high-precision face shape library 15.

In the using stage of the high-precision face shape library, the computer equipment acquires low-precision face shape acquisition data of a user, so that the low-precision face shape acquisition data is processed by the high-precision face shape library to obtain high-precision face generation data, and then the high-precision face generation data is used for three-dimensional face reconstruction.

The construction method of the high-precision face shape library provided by the embodiment of the application can be applied to the following scenes:

virtual character construction in game application

In the application scenario, the high-precision face shape library constructed by the method provided by the embodiment of the application can be applied to a background server of a game application program. When the virtual role is constructed, a user uses the terminal to collect face data of the user, and the collected face data is uploaded to the background server. The background server generates a first face base and a second face base according to the face data and the high-precision face shape library, the first face base and the second face base are fed back to the game application program, the game application program carries out face reconstruction on the virtual role according to the first face base and the second face base, and finally the virtual role with the same face as the user is constructed in the game application. The virtual character is used for representing the identity of the user in the game application, and the user can also control the virtual character to play the game.

Virtual object construction in virtual reality application program

In the application scenario, the high-precision face shape library constructed by the method provided by the embodiment of the application can be applied to a terminal or a server running a virtual reality application program. The method comprises the steps that face shape collected data (usually second-precision face data, namely low-precision face data) obtained by collecting face data of a user by using a camera component of a terminal is fitted by using a high-precision face shape library, a first face shape base and a second face shape base are output, and therefore a face similar to the user in long term is reconstructed on the basis of the first face shape base and the second face shape base, the face comprises some detail features of the user face, such as double eyelids, raised lines, eye lines and the like, and the face of a virtual object similar to the real face of the user is constructed. The virtual object represents the identity of the user in the virtual reality application program, and the user performs video call, scene simulation and the like in the virtual reality application program by using the identity of the virtual object.

The above description is only given by taking two application scenarios as examples, and the method provided in the embodiment of the present application may also be applied to other scenarios (such as virtual human construction, face restoration, and the like) that require to output a high-precision face shape base according to input low-precision face data.

The method for constructing the high-precision human face shape library can be applied to computer equipment with high data processing capacity. In a possible implementation manner, the method for constructing the high-precision face shape library provided by the embodiment of the application can be applied to a personal computer, a workstation or a server, that is, the high-precision face shape library can be constructed through the personal computer, the workstation or the server.

The constructed high-precision face shape library can be realized as a part of an application program and is installed in the terminal, so that the terminal has the function of generating a face shape base according to low-precision face data; or the high-precision face shape library is arranged in a background server of the application program, so that the terminal provided with the application program can realize related functions (such as three-dimensional face reconstruction and driving of facial actions) based on the face shape base by means of the background server.

Referring to FIG. 2, a schematic diagram of a computer system provided by an exemplary embodiment of the present application is shown. The computer system 200 includes a computer device 210 and a server 220, wherein the computer device 210 and the server 220 perform data communication via a communication network, optionally, the communication network may be a wired network or a wireless network, and the communication network may be at least one of a local area network, a metropolitan area network, and a wide area network.

The computer device 210 has installed therein an application program with a virtual character face shape display requirement, which may be a virtual reality application program, a game application program, a dynamic face application program, or an Artificial Intelligence (AI) application program with a three-dimensional face generation function, and the embodiment of the present application is not limited thereto.

Optionally, the computer device 210 may be a mobile terminal such as a smart phone, a smart watch, a tablet computer, a laptop portable notebook computer, or may also be a terminal such as a desktop computer, a projection computer, and the like, which is not limited in this embodiment of the application.

The server 220 may be implemented as one server, or may be implemented as a server cluster formed by a group of servers, which may be physical servers or cloud servers. In one possible implementation, server 220 is a backend server for applications in computer device 210.

As shown in fig. 2, in the present embodiment, in the shape library construction stage, the server 220 acquires the input high-precision face data 11 and low-precision face data 12 in advance to obtain the low-frequency face bases 13, constructs the high-frequency face bases 14 from the low-frequency face bases 13, performs error calculation on the low-frequency face bases 13 and the high-frequency face bases 14 to obtain the error data 21 of the low-frequency face bases and the error data 22 of the high-frequency face bases, and constructs the high-precision face shape library 15 based on the error data of the two.

When receiving the low-precision face acquisition data (acquired when the user uses the shooting component of the computer device 210 to shoot the front face) sent by the computer device 210, the server 220 performs data fitting on the low-precision face acquisition data through the high-precision face shape library 15 to obtain high-precision face generation data 23, feeds the high-precision face generation data 23 back to the computer device 210, and constructs the face of the virtual character according to the high-precision face generation data 23 by the application program in the computer device 210 and displays the face.

In other possible embodiments, the high-precision face shape library may also be set in an application program, and the terminal locally outputs high-precision face generation data according to the input low-precision face acquisition data without using the server 220, which is not limited in this embodiment.

For convenience of description, the following embodiments are described as examples in which the construction method of the high-precision face shape library is performed by a computer device.

Fig. 3 is a flowchart illustrating a method for constructing a high-precision face shape library according to an exemplary embodiment of the present application. The embodiment is described by taking as an example that the method is used in the computer system 200 shown in fig. 2, and the method comprises the following steps:

step 301, obtaining a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; the accuracy of the first accuracy face data is higher than the second accuracy face data.

The first-precision face data (also named high-precision face data) and the second-precision face data (also named low-precision face data) are face data that are neutral faces, which means faces that do not have any expression.

In some embodiments, the first precision face data and the second precision face data include neutral face data of a plurality of objects, the data amount of the second precision face data is greater than that of the first precision face data, and the plurality of objects belong to the same or different races. For example, the first precision face data includes face data of 20 objects, the face data of 20 objects is face data of asian people, the second precision face data includes face data of 200 objects, and the face data of 200 objects is face data of euro and america people. The face data includes the shape of the face of the subject and some detail features of the face.

In some embodiments, the first-precision face data is high-precision face mesh (mesh) data, the second-precision face data is low-precision face mesh data, and the 3D faces corresponding to the high-precision face mesh data and the low-precision face mesh data have the same 3D vertex sequence number and semantic meaning. For example, the high-precision face mesh data and the low-precision face mesh data are data of a three-dimensional deformation Model (3D deformable Model, 3 DMM).

The first precision face data refers to face data with the precision higher than a threshold value, and the second precision face data refers to face data with the precision lower than the threshold value. The face detail degree corresponding to the first precision face data is higher than the face detail degree corresponding to the second precision face data. Therefore, compared with the face constructed by the face data of the first precision, the face constructed by the face data of the second precision lacks details, and accordingly, the three-dimensional face constructed by the face data of the second precision is difficult to show the detail characteristics of the real face, such as the simple eyelid, the raised line, the eye corner line and the like.

In some embodiments, the first-precision face data is face data obtained by scanning and processing a real face with a high-precision scanning device (such as a high-speed 3D camera), and the second-precision face data is face data obtained by scanning and processing a real face with a consumer-level device (such as a kinectrbd device or a smart phone), or the face data is face data in an existing face shape library, such as face data in fwh (faceware house), face data in BFM, and the like. The embodiment of the application does not limit the sources of the first precision face data and the second precision face data.

The first face shape base is a face shape base calculated based on the first precision face data and the second precision face data. Schematically, the first facial form base is obtained by the following method: the high-precision face grid data and the low-precision face grid data are aligned to the same coordinate system through data alignment (the high-precision face grid data and the low-precision face grid data are consistent in number and topology, but different in face length, and the precision of the high-precision face grid data is higher than that of the low-precision face grid data), principal component analysis is conducted on the aligned grid vertex coordinates, and a PCA base is obtained and is the first face type base. The embodiment of the present application does not limit the manner of obtaining the first facial form base.

In some embodiments, the first face shape base is a face shape base that is iteratively updated in advance, that is, in a subsequent step, the fitting error of the first face shape base and the face mesh data does not need to be iteratively updated.

And 302, constructing a second face base through the first face base and a first fitting error, wherein the first fitting error is error information generated when the first face base is fitted with extended data, and the extended data is obtained by extending based on the first-precision face data and the second-precision face data.

Because the acquisition cost of the face data is high, the production time is long, and after some face data (especially the first-precision face data) are obtained, the number of high-quality face data needs to be increased by expanding the data. Illustratively, the data expansion method is as follows: randomly selecting a pair of face data, wherein the faces of the pair can be the same race or different races; randomly selecting the organs for replacing the face, such as randomly selecting and replacing three organs of eyes, nose and mouth, 7 replacement modes (eye replacement, nose replacement, mouth replacement, two-part eye and nose replacement, two-part eye and mouth replacement, two-part nose and mouth replacement, and three-part eye, nose and mouth replacement) can be obtained.

The error of the first face shape base when fitting with the extended human face data is the first fitting error. Illustratively, the first fitting error is an extended error matrix, the extended error matrix is subjected to PCA analysis, and a second face base is constructed by using a mean value, principal component coefficients and principal component variances obtained after the PCA analysis. The principal component coefficient is an eigenvector matrix of the covariance matrix, wherein each column is an eigenvector arranged from small to large.

The computer device constructs a second face base according to the first face base and the first fitting error, wherein the second face base is used for fitting the residual error of the expanded face data relative to the first face base. In some embodiments, the shape base is formed by a variability network (blendshape) representing different face shapes, and each blendshape is formed by the same 3D face model changing under different face shapes, and vertex sequence numbers and meanings corresponding to different blendshapes are consistent.

Step 303, fitting the disturbance data through the first facial form base to obtain a second fitting error, wherein the disturbance data is obtained after disturbance processing is carried out on the basis of the expansion data; and fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base.

The disturbance data refers to face data subjected to disturbance processing, and the disturbance processing is used for enhancing the capability of the first face base to fit the face grid data with certain deflection characteristics.

Illustratively, the disturbance processing mode includes at least one of a rotation disturbance mode, a translation disturbance mode and a scaling disturbance mode.

1. A rotational perturbation mode.

And the computer equipment rotates at least one vertex in the face grid data by any angle around a reference point in any direction to obtain at least two disturbance data, wherein the reference point is any point in the face grid data.

The computer device may use any one vertex in the face mesh data as a reference point, and rotate another vertex in the face mesh data around the reference point. The direction and the angle of rotation are arbitrary, so that the spatial position of the point is changed, and a group of disturbance data is obtained after at least one vertex in the face grid data is rotated. For example, the reference point is a first vertex, a second vertex in the face mesh data is rotated around the first vertex from a first position to a second position, and distances from the second vertex to the first vertex before and after the rotation are unchanged.

Illustratively, the reference point may be a vertex representing a nose tip in the face mesh data, and in order to avoid an excessively large rotation angle, the rotation angle is controlled within ± 5 ° to obtain face distortion corresponding to the disturbance data.

Illustratively, the rotation changes the degree of relief of the vertices in the face mesh data.

2. And (4) a translational disturbance mode.

The computer equipment moves at least one vertex in the face grid data to a first direction by a first distance to obtain at least two disturbance data.

And the computer equipment translates at least one vertex in the face grid data for a certain distance along any direction to obtain disturbance data. Illustratively, the first direction is a direction pointing along the posterior brain-scoop portion of the face toward the tip of the nose. In order to avoid the face distortion corresponding to the disturbance data, the maximum translation distance is determined according to the face grid data of the face shape library. For example, the diameter distance from the central point of the posterior brain area to the nose tip of each group of face mesh data is calculated, and the difference between the maximum diameter distance and the minimum diameter distance is determined as the maximum distance of translation. In some embodiments, the maximum distance of translation may also be a preset value.

For example, the computer device moves the vertices representing the tips of the noses in the face mesh data by 1cm in the first direction, increasing the height of the nose, so that the nose with the perturbation data is more stereoscopic.

3. The perturbation mode is scaled.

The computer device uses the midpoint of the connecting line of the two ears in the face grid data as the origin of coordinates, and amplifies (or reduces) the face grid data in the same proportion to obtain at least two groups of disturbance data. Illustratively, the ratio is between 0.8 and 1.2 or between 0.9 and 1.1.

Illustratively, in order to make the scaling capable of preserving the original shape of the face, it is necessary to align the face mesh data first, and the midpoint of the line connecting the two ears is used as the origin of the coordinate system. The computer device re-determines the coordinates of each vertex in the face mesh data and then enlarges (or reduces) the new coordinates in the same scale. For example, multiplying the new coordinates by 0.8 yields a reduced set of perturbation data. Illustratively, the coordinate of each vertex in the face mesh data may be re-determined by using the average value of all vertices in the face mesh data as the origin of the coordinate system, and then scaled proportionally. In some embodiments, the computer device directly perturbs the unexpanded face mesh data.

In some embodiments, the fitting of the first face base and the first fitting error is also performed by fitting the first face base to the disturbance data, and the computer device calculates a second fitting error obtained by fitting the first face base and the first fitting error. The fitting mode of the first face base and the first fitting error can also be used for fitting the second face base and the second fitting error, and the computer equipment calculates a third fitting error obtained after the second face base and the second fitting error are fitted.

And 304, iteratively updating the expanded first fitting error according to the second fitting error and the third fitting error, and constructing a high-precision face shape library according to the second face shape base and the first face shape base obtained when iteration is finished.

And combining the previous step to obtain a second fitting error and a third fitting error through calculation, expanding the first fitting error by the computer equipment by using the second fitting error and the third fitting error, and performing iterative updating on the expanded first fitting error. And through the expansion of the first fitting error and the iterative update of the second face shape base, the fitting effect of the second face shape base on the face grid data is continuously improved. And when the third fitting error meets the iteration end condition, the computer equipment obtains a second face base updated by the last iteration. It can be understood that, by using the above method, a plurality of sets of second face bases after iteration can be obtained, and a high-precision face shape library is constructed according to the plurality of sets of second face bases and the first face bases which are updated in advance through iteration.

In the subsequent use process, the computer equipment can utilize the high-precision human face shape library to output high-precision human face generation data according to the input low-precision human face acquisition data, and then the high-precision human face generation data is utilized to restore a human face model which has higher similarity with the user and richer details.

In summary, the method provided in this embodiment is based on the idea of residual data, a second face base is constructed by using the first face base, the expanded first fitting error is iteratively updated according to a second fitting error corresponding to the first face base and a third fitting error corresponding to the second face base, and a high-precision face shape library is constructed according to the second face base and the first face base obtained at the end of the iteration. According to the face shape library constructed by combining the second face shape base and the first face shape base, on the premise that the constructed face shape library has high precision, the demand for the first precision face data (with the face data with higher precision) is reduced, the construction difficulty of the high-precision face shape library is reduced, and the construction efficiency of the high-precision face shape library is improved.

Fig. 4 is a flowchart illustrating a method for constructing a high-precision face shape library according to another exemplary embodiment of the present application. The embodiment is described by taking as an example that the method is used in the computer system 200 shown in fig. 2, and the method comprises the following steps:

step 401, obtaining a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; the accuracy of the first accuracy face data is higher than the second accuracy face data.

The face detail degree corresponding to the first precision face data is higher than the face detail degree corresponding to the second precision face data. The first precision face data is generally obtained by adopting scanning equipment with higher precision, and the second face precision is generally obtained by adopting scanning equipment with lower precision. The first precision face data refers to face data with the precision higher than a threshold value, and the second precision face data refers to face data with the precision lower than the threshold value. The manner of obtaining the first face base will be described. Illustratively, the obtaining manner of the first facial form base includes:

firstly, the acquired first-precision face data and the second-precision face data are processed separately, and each type of face data (the first precision and the second precision, namely high precision and low precision) is aligned to a unified coordinate system, and the high-precision face data can also use third-party high-precision face data (the existing public face grid data). In some embodiments, because the mesh topology of the third-party high-precision face data is different from the mesh topology in the embodiments of the present application, face mesh data having the same mesh topology as that in the embodiments of the present application, such as wrap software, can be obtained by using professional mesh overlay software. As shown in fig. 5 (a), which shows a model 51 corresponding to the third-party face mesh data, the third-party face mesh data is attached to the face mesh in the embodiment of the present application by wrap software, so as to obtain a model 52 corresponding to the face mesh data shown in fig. 5 (b). The fitted face grid data also needs to be aligned to a uniform coordinate system.

In some embodiments, the computer device performs an averaging process on the high-precision neutral face data (also named as first-precision neutral face data) and the high-precision neutral face mirror image data (also named as first-precision neutral face mirror image data) to obtain high-precision symmetrical neutral face data with a face left-right symmetrical shape (also named as first-precision symmetrical neutral face data); and carrying out mean value processing on the low-precision neutral face data (named as second-precision neutral face data) and the low-precision neutral face mirror image data (named as second-precision neutral face mirror image data) to obtain low-precision symmetrical neutral face data (named as second-precision symmetrical neutral face data).

Illustratively, mirror symmetry is performed on the aligned face mesh data once, as shown in fig. 6, a computer device performs mean processing on the face data 61 and the mirrored face data 62 to obtain left-right face symmetric neutral face data 63. The data expansion is performed on the face mesh data, and the data expansion mode is as in the embodiment of step 302 shown in fig. 3, which is not described herein again.

The vertex coordinates of the topological network in the three-dimensional space are recorded by calculating the face grid data, and the three-dimensional coordinates of the face grid data are expanded to obtain the column vector of the face grid data. For example, if a topological network of face grid data has N vertices, three-dimensional coordinates are expanded to obtain a column vector of [3N × 1], and if M face grid data correspond to a face grid matrix of [3N × M ], PCA calculation is performed on the face grid matrix to obtain a mean value mu, a principal component coefficient pc and a principal component variance ev _ f, where the principal component coefficient pc is a feature vector matrix of a covariance matrix, and a diagonal matrix ev formed by principal component standard deviations is calculated from the principal component variance ev _ f. The mean mu, principal component coefficient pc, and principal component variance ev constitute a first face base.

In some embodiments, the computer device performs iterative updating according to the fitting error when the first face base is fitted to the first precision face data and the second precision face data, so as to obtain an iteratively updated first face base. And performing subsequent steps by using the first facial form base after iterative update.

It is understood that the first facial form base may be obtained in other manners, and the manner of obtaining the first facial form base is not limited in the embodiments of the present application.

Step 402, a second face base is constructed through the first face base and a first fitting error, the first fitting error is error information generated when the first face base is fitted with extended data, and the extended data is obtained after the extended data is based on the first precision face data and the second precision face data.

Illustratively, the first fitting error comprises an error matrix. In some embodiments, step 402 may be replaced with the following steps:

step 4021, calculating a fitting error when the first facial form base is fitted to the extended data.

The first facetted base obtained in step 401 is denoted by mu0, pc0 and ev 0. The data1 represents the augmented data after the face data is augmented, and for example, the augmented data is also a face mesh matrix of [3N × M ], for one face mesh data face1 in the data1, a fitting coefficient id1 is corresponding to the data, the fitting coefficient id1 is used to represent the face shape, and the fitting coefficient id1 can be calculated by the following formula one:

the face _ rot is a face grid (a column vector of [3N × 1] after three-dimensional expansion) in the disturbance data, the face _ rot (x) represents the xth number in the disturbance data, the (mu0+ pc0 ev0 × α) (x) represents the xth number in the fitting error obtained after the first face base and the extended data are fitted, the α is a fitting coefficient id1, namely a face shape coefficient, and the λ is a regular coefficient.

After obtaining the fitting coefficient id1, calculating the fitting error when the first face shape base is fitted to the extended data by formula two:

loss1＝face1-(mu0+pc0*ev0*id1)

the face1 is one face mesh data in the extended data 1. mu0, pc0 and ev0 are first face bases, and id1 is a fitting coefficient, namely a face shape coefficient.

And step 4022, obtaining an error matrix according to the fitting error.

For each face mesh data in data1, the face mesh data is calculated to correspond to a fitting error loss 1. Then there is a first fitting error corresponding to data1, which is an error matrix ([3N × M ] matrix, M being the number of faces), denoted by loss _ all.

In some embodiments, the error matrix is extended by: and performing data expansion on the error matrix along the direction to obtain an expanded error matrix, wherein the expanded error matrix has symmetry. Namely, the extended error matrix [ loss _ all, -loss _ all ] is obtained from the error matrix loss _ all. The extended error matrix is a [3 Nx 2M ] error matrix.

And step 4023, performing principal component analysis on the error matrix to obtain a mean value, a principal component eigenvector matrix and a principal component variance corresponding to the error matrix, wherein the principal component variance is an eigenvalue corresponding to the eigenvector in the principal component eigenvector matrix.

The data volume and the data dimensionality generated by the steps are large, and in order to enable the constructed second facial form base to have a good fitting effect, the data volume and the dimensionality of the error matrix are reduced, so that the subsequent calculation amount is reduced. In some embodiments, the computer device processes the error matrix by a PCA technique, and reduces data dimensionality, redundant or interfering data, and the likelihood of subsequent overfitting by using a dimensionality reduction concept.

Regarding the principal component analysis method, schematically, the computer device first calculates the mean value of error data in the error matrix, then calculates the eigen covariance matrix corresponding to the error matrix, so as to obtain eigenvectors and eigenvalues of covariance according to the eigen covariance matrix, further selects a plurality of eigenvectors with the largest eigenvalue from the eigenvectors according to descending order of the eigenvalues to form a principal component eigenvector matrix, and generates principal component variance (vector) for the eigenvalues corresponding to the eigenvectors in the principal component eigenvector matrix.

Illustratively, PCA calculation is performed on the extended error matrix to obtain a mean value h _ mu, a principal component coefficient h _ pc (principal component eigenvector matrix) and a principal component variance h _ ev _ f of the extended error matrix, where the mean value h _ mu is a [3N × 1] vector, h _ pc is a [3N × t ] matrix, h _ ev _ f is a [ t × 1] vector, t is the number of selected eigenvectors, and N is the number of vertices of the face grid. The number t of the feature vectors is the number t of the selected principal component variances with the weight higher than a certain threshold (0.9).

And step 4024, constructing a second face base according to the mean value, the principal component eigenvector matrix and the principal component variance.

In some embodiments, the principal component variance comprises a column vector. Illustratively, the computer device performs matrix change on the principal component variance to obtain a principal component diagonal matrix, wherein the principal component diagonal matrix comprises a principal component standard deviation; the computer device determines the mean, the principal component eigenvector matrix, and the principal component diagonal matrix as a second face base.

In step 4023, the computer device performs an extraction process on the column vector h _ ev _ f (i.e., obtains a principal component standard deviation from the principal component variance) to obtain a principal component diagonal matrix h _ ev (h _ ev is a [ t × 1] vector). Accordingly, the second face base is composed of the mean h _ mu ([3N × 1] vector), principal component coefficients h _ pc ([3N × t ] matrix), and principal component diagonal matrices h _ ev ([ t × 1] vector).

Step 403, fitting the disturbance data through the first facial form base to obtain a second fitting error, wherein the disturbance data is obtained after disturbance processing is performed on the basis of the expansion data; and fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base.

In some embodiments, the disturbance data is obtained by one of a rotational disturbance mode, a translational disturbance mode and a scaling disturbance mode.

The computer device fits the second face base and the perturbation data, and since the second face base is constructed based on the first face base, the first face base and the second face base in the current iteration need to be combined in the fitting.

Schematically, a first face base is represented by mu0, pc0 and ev0, and the computer device fits the perturbation data through the first face base to obtain a first face shape coefficient. And the computer equipment calculates to obtain a second fitting error according to the first face base, the disturbance data and the first face shape coefficient, and a third calculation formula of the second fitting error is as follows:

l_loss＝face_rot-(mu0+pc0*ev0*l_id)

wherein l _ loss is the second fitting error, face _ rot is a face mesh (a column vector of [3N × 1] after three-dimensional expansion) in the perturbation data, and l _ id is the first face shape coefficient (calculated by the above formula one).

The second fitting error is similar to the error loss1 in equation two, where the error loss1 is the error when the first facial form base fits the augmented face data, and the second fitting error l _ loss is the error when the first facial form base fits the disturbance data, and the second fitting error corresponds to the first fitting error.

Schematically, a second face base of the current iteration is represented by h _ mu, h _ pc and h _ ev, and the computer device performs fitting through the second face base and disturbance data to obtain a second face shape coefficient. And the computer equipment calculates a third fitting error according to the second face shape base, the second fitting error and the second face shape coefficient, and a calculation formula of the third fitting error is as follows:

wherein h _ loss is the third fitting error, l _ loss (x) is the x-th number in the second fitting error, and h _ id is the second face shape coefficient (calculated by the first formula).

The computer device may obtain a second fitting error and a third fitting error based on equation three and equation four, respectively.

And step 404, obtaining a screening error according to the second fitting error and the third fitting error.

In some implementations, the screening error includes a screening error matrix. Step 404 may be replaced by the following steps:

step 4041, sorting the error values according to the third fitting error in a descending order to obtain m error positions corresponding to the first m error values, where m is a positive integer.

And for each group of face data in the disturbance data, a second fitting error l _ loss and a third fitting error h _ loss can be correspondingly calculated. Illustratively, if there are R sets of disturbance data, l _ loss is [3 NxR ], the matrix, and h _ loss is the row vector of [1 xR ].

In the process of the third fitting error h _ loss of the iteration, the error values in the h _ loss vector are arranged from large to small, and the first m error values are taken.

Step 4042, determining an error screening matrix corresponding to the first m error values from the second fitting errors according to the m error positions.

After step 4041 is executed, m error values are determined, and the sequential positions of the m error values in h _ loss are denoted as id _ h _ max (error position), because the l _ loss matrix and the h _ loss vector have the same column number, and the face data (the originally acquired face data) corresponding to each column is the same. Then the error screening matrix l _ loss _ select corresponding to the m error values can be determined in l _ loss according to id _ h _ max. Illustratively, id _ h _ max is a column vector of dimension S, and the error screening matrix l _ loss _ select is a [3 NxS ] matrix.

Step 405, adding the screening error to the first fitting error to obtain an expanded first fitting error.

Illustratively, the error screening matrix l _ loss _ select obtained in step 4042 is symmetric along the negative direction, and a symmetric error screening matrix [ l _ loss _ select, -l _ loss _ select ] is obtained, where the screening error matrix is a matrix of [3N × 2S ], and the screening error matrix has symmetry. And adding the screening error matrix into the first fitting error loss _ all, and expanding the first fitting error loss _ all to obtain the expanded first fitting error loss _ all.

And 406, iteratively updating the expanded first fitting error to obtain a high-precision face shape library.

In some embodiments, step 406 may be replaced with the following step:

step 4061, iteratively calculating a second face base based on the extended first fitting error.

And iteratively updating the expanded first fitting error loss _ all, wherein the expanded loss _ all is obtained by an error screening matrix l _ loss _ select, and the error screening matrix is obtained by a third fitting error h _ loss, so that the iteratively updating the loss _ all is the iteratively updating the third fitting error h _ loss.

Step 4062, in response to the third fitting error satisfying the iteration convergence condition, constructing a high-precision face shape library according to the second face shape base and the first face shape base when the iteration is finished.

And the computer equipment obtains the maximum fitting error corresponding to the iteration based on the fitting result of the expanded first fitting error to the third fitting error. And if the error value between the maximum fitting error corresponding to the current iteration and the maximum fitting error corresponding to the last iteration is smaller than a threshold value (for example, the threshold value is 0.01, namely the maximum fitting error corresponding to the current iteration is almost equal to the maximum fitting error corresponding to the last iteration), determining that an iteration convergence condition is met, and constructing a high-precision face shape library according to the second face base and the first face base when the iteration is finished.

In some embodiments, the computer device calculates a maximum fitting error for each iterative update process and determines whether the fitting error converges by comparing the maximum fitting errors for two adjacent iterative processes, thereby completing the iterative update when the fitting error converges.

It can be understood that, with the method of this embodiment, a plurality of sets of second face bases after iteration can be obtained, and a high-precision face shape library is constructed according to the plurality of sets of second face bases and the first face bases that have been updated iteratively in advance.

By performing principal component analysis on the error matrix, a second face shape base is constructed according to a principal component analysis result, and on the premise of ensuring that the face shape base has a good fitting effect, the data dimension of the generated shape base is reduced, so that the calculation amount during subsequent data fitting is reduced, and the construction efficiency of a high-precision face shape library is improved.

The error screening matrix is obtained by utilizing the internal relation between the second fitting error and the third fitting error, so that the computer equipment can expand the first fitting error by utilizing the error screening matrix, more favorable data are provided for the iterative updating process of the subsequent computer equipment, and the constructed human face shape library has better fitting effect.

The error values in the third fitting error are sorted in a descending order, and the first m error values with larger error values are selected, so that the computer equipment can accurately screen out the error screening matrix according to the m error values, and accurate data are provided for the subsequent data expansion of the first fitting error.

By establishing the internal relation between the first fitting error and the third fitting error, the computer device performs iterative update on the third fitting error, namely the second facial form base, when performing iterative update on the first fitting error. And when the third fitting error meets the convergence condition, constructing a high-precision face shape library according to the corresponding second face shape base and the corresponding first face shape base (which are updated in advance) when the iteration is finished. The high-precision face shape library constructed by combining the second face shape base and the first face shape base has a better fitting effect when fitting low-precision face data. And determining that the third fitting error meets the convergence condition by using the difference value between the maximum fitting error corresponding to the iteration and the maximum fitting error corresponding to the last iteration to be smaller than the threshold value, so that the computer equipment can accurately calculate the second face base after the iteration is updated.

The first face shape coefficient is obtained by fitting the first face base and the disturbance data, so that the computer equipment can accurately calculate the second fitting error by combining the first face shape coefficient, and accurate data is provided for the calculation of the subsequent third fitting error. And fitting the second face shape base with the disturbance data to obtain a second face shape coefficient, so that the computer equipment can accurately calculate a third fitting error by combining the second face shape coefficient and the second fitting error.

Based on the high-precision human face shape library constructed by the embodiment, when the computer equipment constructs the three-dimensional human face model according to the obtained human face data, the method comprises the following steps:

and 701, acquiring a low-frequency fitting coefficient, a high-frequency fitting coefficient and a high-precision face shape library, wherein the low-frequency fitting coefficient is used for representing the shape of the first face base, and the high-frequency fitting coefficient is used for representing the shape of the second face base.

The high-precision face shape library constructed based on the above embodiment needs to be fitted twice when in use, namely, the first face shape base is fitted once, and then the second face shape base is fitted once. The low-frequency fitting coefficient and the high-frequency fitting coefficient can be obtained through calculation according to the formula I, and the obtaining mode of the low-frequency fitting coefficient and the high-frequency fitting coefficient is not limited in the embodiment of the application.

And step 702, calculating to obtain the face shape of the target face according to the low-frequency fitting coefficient, the high-frequency fitting coefficient and the high-precision face shape library.

Schematically, the first face bases are represented by mu0, pc0 and ev0, the second face bases are represented by h _ mu, h _ pc and h _ ev, the low-frequency fitting coefficients are represented by l _ id, and the high-frequency fitting coefficients are represented by h _ id. The face shape of the target face is obtained by calculating according to a formula five:

face＝mu0+pc0*ev0*l_id+h_mu+h_pc*h_ev*h_id

wherein, the face represents the face shape of the target face.

According to the formula five, when the computer device fits the face shape base according to the acquired face data, the face shape base needs to be fitted with the second face shape base and the first face shape base respectively.

Schematically, as shown in fig. 8, a human face 81 is real face data of a collected human face, a human face 82 in fig. 8 is constructed based on a first face base, and a human face 83 in fig. 8 is constructed by combining the first face base and a second face base. Therefore, the face shapes of the face 83 and the face 81 are completely consistent, and compared with the face 82, the face 83 retains more face details (such as wrinkles of the eye corners, and statute lines of the mouth corners), so that the face restored by the face shape library obtained based on the embodiment of the present application has higher accuracy.

In summary, according to the method provided by this embodiment, the high-precision face shape library constructed based on the first face shape base and the second face shape base enables the computer device to output a high-quality face similar to the target face even if low-precision face data is collected, so that the face precision after fitting is improved, and a task of driving the face to make an action or an expression is conveniently performed subsequently.

In the embodiment of the application, the construction of the high-precision face shape library can be divided into a low-frequency face shape base acquisition stage and a high-frequency face shape base iteration updating stage. As shown in fig. 9, the stage of acquiring the low frequency face bases includes data preparation 901, data expansion 902 and iterative updating the low frequency face bases 903, and the iterative updating the high frequency face bases 904 further includes constructing a high frequency face base 9041, data perturbation 9042 and iterative updating errors 9043.

In the stage of obtaining the low-frequency face base, the computer device completes data preparation 901 by collecting low-precision face data and high-precision face data. The face data is obtained by scanning and processing real faces of a plurality of objects through a high-precision scanning device (such as a high-speed 3D camera), and the low-precision face data is obtained by scanning and processing real faces of a plurality of objects through a consumer-grade device (such as a kinectrbd device, a smart phone, and the like), or by using face data in an existing face shape library, such as face data in fwh (faceware house), face data in BFM, and the like. And removing interference noise from the collected face data, and then registering the point cloud into a specific face grid through a non-rigid ICP algorithm. For a large amount of collected data, a series of face mesh data with consistent vertex number and same topology but different appearance are obtained. And aligning the face grids under the same coordinate system.

The computer device then performs data augmentation 902 based on the existing face data, increasing the amount of high quality face data. Data augmentation 902 is achieved by replacing the organs of a pair of faces. In some embodiments, the computer device performs mirror symmetry on the extended data, and performs mean processing on the face data and the face mirror image data to obtain high-precision symmetrical neutral face data (first-precision symmetrical neutral face data) in which the face is left-right symmetrical.

The computer device constructs a low frequency face shape base from the augmented data. The low-frequency face shape base is obtained by principal component analysis of a [3 NxM ] matrix formed by M pieces of face grid data. The computer equipment firstly calculates the mean value mu ([3N multiplied by 1] vector) of error data in the matrix, then calculates the characteristic covariance matrix corresponding to the matrix, so as to obtain the eigenvector and the eigenvalue pc ([3N multiplied by t ] matrix, t is the quantity of the selected eigenvector) of covariance according to the characteristic covariance matrix, further selects a plurality of eigenvectors with the largest eigenvalue from the eigenvectors according to the descending order sorting of the eigenvalue to form a principal component eigenvector matrix, and generates the principal component variance ev _ f ([ t multiplied by 1] vector) for the eigenvalue corresponding to the eigenvector in the principal component eigenvector matrix. And (3) the computer equipment opens the root of the principal component variance ev _ f to obtain a diagonal matrix ev, and then the low-frequency face base is constructed based on the mean mu, the principal component coefficient pc and the diagonal matrix ev.

The computer device updates the low frequency face shape base 903 for iteration, and data perturbation is needed to be carried out on the low frequency face shape base before the iteration is updated, so that the expression capability of the low frequency face shape base is improved. And fitting the low-frequency face shape base by the computer equipment based on the face shape coefficient id and the disturbance data, and obtaining the low-frequency face shape base when the iteration is finished when the low-frequency face shape base error (second fitting error) meets the convergence condition.

The computer device builds a high frequency face base 9041 based on the low frequency face base at the end of the iteration. The error matrix loss _ all is obtained from the error loss1 generated when the low frequency face shape base is fitted to the augmented data face 1. And expanding the error matrix loss _ all to obtain an expanded error matrix [ loss _ all, -loss _ all ]. Similarly, similar to the process of constructing the low-frequency face base, the computer device first calculates the mean value h _ mu ([3N × 1] vector) of the error data in the extended error matrix [ loss _ all, -loss _ all ], then calculates the feature covariance matrix corresponding to the error matrix [ loss _ all, -loss _ all ], so as to obtain the feature vector and the feature value h _ pc ([3N × t ] matrix, t is the number of the selected feature vectors) of the covariance according to the feature covariance matrix, further selects a plurality of feature vectors with the largest feature values from the feature vectors according to the descending order sorting of the feature values, forms a principal component feature vector matrix, and generates the principal component variance h _ ev _ f ([ t × 1] vector) for the feature value corresponding to the feature vector in the principal component feature vector matrix. And (3) the computer equipment performs root cutting on the principal component variance h _ ev _ f to obtain a diagonal matrix h _ ev, and then a high-frequency face base is constructed based on the mean value h _ mu, the principal component coefficient h _ pc and the diagonal matrix h _ ev.

The computer device performs face data perturbation 9042, where the perturbation manner refers to step 303 shown in fig. 3, and is not described here again.

The computer device iteratively updates the high frequency base error 9043. The computer equipment firstly uses the low-frequency facial form base to fit the disturbance data to obtain a low-frequency base error (a second fitting error, l _ loss), and then the computer equipment uses the low-frequency base error and the high-frequency facial form base to fit to obtain a high-frequency base error (a third fitting error, h _ loss). And selecting the first m error values (m is a positive integer according to the descending order of the error values) from the high-frequency base error, and recording the error positions id _ h _ max corresponding to the m error values. And the computer equipment determines an error screening matrix l _ loss _ select ([3 NxS ] matrix, S is the dimension of the error position id _ h _ max) corresponding to the m error values from the low-frequency base error according to the error position id _ h _ max. And expanding the error screening matrix along the negative direction to obtain an expanded screening error matrix [ l _ loss _ select, -l _ loss _ select ] ([3 Nx 2S ] matrix). And adding the expanded error matrix [ l _ loss _ select, -l _ loss _ select ] to the error matrix loss _ all, and expanding the error matrix loss _ all.

And the computer equipment carries out iterative updating based on the expanded error matrix loss _ all, and finishes the iteration when the high-frequency basis meets the convergence condition. And the computer equipment takes the maximum fitting error obtained by fitting the high-frequency basis error and the error matrix as a convergence condition, and determines that the iteration convergence condition is met when the error difference value between the maximum fitting error corresponding to the iteration and the maximum fitting error of the previous iteration is less than a threshold value. The computer device obtains a high frequency face shape base at the end of the iteration.

The computer device outputs the high frequency face shape base and the low frequency face shape base 905, and a high precision face shape library is constructed according to the low frequency face shape base and the high frequency face shape base when the iteration is finished.

By the method, even if the computer equipment collects low-precision face data, the computer equipment can output a high-quality face similar to the target face, so that the face precision after fitting is improved, and the task of driving the face to make an action or an expression can be conveniently executed subsequently.

By establishing the internal relation between the first fitting error and the third fitting error, the computer device performs iterative update on the third fitting error, namely the second facial form base, when performing iterative update on the first fitting error. And when the third fitting error meets the convergence condition, constructing a high-precision face shape library according to the corresponding second face shape base and the corresponding first face shape base (which are updated in advance) when the iteration is finished. And the high-precision face shape library constructed by combining the second face shape base and the first face shape base has a better fitting effect when fitting the second-precision face data. And determining that the third fitting error meets the convergence condition by using the difference value between the maximum fitting error corresponding to the iteration and the maximum fitting error corresponding to the last iteration to be smaller than the threshold value, so that the computer equipment can accurately calculate the second face base after the iteration is updated.

Fig. 10 is a block diagram of a construction apparatus for constructing a high-precision face shape library according to an exemplary embodiment of the present application, where the apparatus includes:

the data acquisition module 1010 is used for acquiring a first face shape base, wherein the first face shape base is a face shape base obtained through first precision face data and second precision face data; the precision of the first precision face data is higher than that of the second precision face data;

a constructing module 1020, configured to construct a second face shape base according to the first face shape base and a first fitting error, where the first fitting error is error information generated when the first face shape base is fitted to the extended data, and the extended data is obtained by extending the first precision face data and the second precision face data;

the data fitting module 1030 is configured to fit the disturbance data through the first facial form base to obtain a second fitting error, where the disturbance data is obtained after disturbance processing is performed on the basis of the expansion data; fitting the second fitting error through the second facial form base to obtain a third fitting error of the second facial form base;

and the iteration updating module 1040 is configured to iteratively update the expanded first fitting error according to the second fitting error and the third fitting error, and construct a high-precision face shape library according to the second face shape base and the first face shape base obtained when the iteration is ended.

In an optional embodiment, the data fitting module 1030 is configured to obtain a screening error according to the second fitting error and the third fitting error; adding the screening error to the first fitting error to obtain an expanded first fitting error; and the iteration updating module 1040 is configured to perform iteration updating on the expanded first fitting error to obtain a high-precision face shape library.

In an alternative embodiment, the screening error comprises an error screening matrix;

the data fitting module 1030 is configured to perform descending order sorting on error values in the third fitting error, to obtain m error positions corresponding to m previous error values, where m is a positive integer; and determining an error screening matrix corresponding to the previous m error values from the second fitting errors according to the m error positions.

In an alternative embodiment, the iterative update module 1040 is configured to iteratively calculate a second face base based on the extended first fitting error; and in response to the third fitting error meeting the iteration convergence condition, constructing a high-precision face shape library according to the second face shape base and the first face shape base when the iteration is ended.

In an optional embodiment, the iteration updating module 1040 is configured to obtain a maximum fitting error corresponding to the current iteration based on a fitting result of the expanded first fitting error to the third fitting error; and determining that an iteration convergence condition is met in response to the fact that the error difference value between the maximum fitting error corresponding to the iteration and the maximum fitting error corresponding to the last iteration is smaller than a threshold value, and constructing a high-precision face shape library according to the second face shape base and the first face shape base when the iteration is finished.

In an alternative embodiment, the first fitting error comprises an error matrix;

the building module 1020 is configured to calculate a fitting error when the first face shape base is fitted to the extended data; obtaining an error matrix according to the fitting error; performing principal component analysis on the error matrix to obtain a mean value, a principal component eigenvector matrix and a principal component variance corresponding to the error matrix, wherein the principal component variance is an eigenvalue corresponding to the eigenvector in the principal component eigenvector matrix; and constructing a second face base according to the mean value, the principal component eigenvector matrix and the principal component variance.

In an alternative embodiment, the principal component variance comprises a column vector;

the constructing module 1020 is configured to perform matrix change on the principal component variance to obtain a principal component diagonal matrix, where the principal component diagonal matrix includes a principal component standard deviation; and determining the mean value, the principal component eigenvector matrix and the principal component diagonal matrix as a second face base.

In an optional embodiment, the data fitting module 1030 is configured to fit the second face shape base and the disturbance data to obtain a second face shape coefficient; and calculating to obtain a third fitting error according to the second face shape base, the second fitting error and the second face shape coefficient, wherein the second fitting error corresponds to the first fitting error.

In an alternative embodiment, the data obtaining module 1010 is configured to obtain a low frequency fitting coefficient, a high frequency fitting coefficient and a high precision face shape library, where the low frequency fitting coefficient is used to represent the shape of the first face base, and the high frequency fitting coefficient is used to represent the shape of the second face base; and calculating to obtain the face shape of the target face according to the low-frequency fitting coefficient, the high-frequency fitting coefficient and the high-precision face shape library.

In summary, the apparatus provided in this embodiment constructs a second face base from the first face base based on the idea of residual data, iteratively updates the expanded first fitting error according to the second fitting error corresponding to the first face base and the third fitting error corresponding to the second face base, and constructs a high-precision face shape library according to the second face base and the first face base obtained at the end of the iteration. According to the face shape library constructed by combining the second face shape base and the first face shape base, on the premise that the constructed face shape library has high precision, the demand for the first precision face data (with the face data with higher precision) is reduced, the construction difficulty of the high-precision face shape library is reduced, and the construction efficiency of the high-precision face shape library is improved.

Through the high-precision human face shape library constructed based on the first face shape base and the second face shape base, even if low-precision human face data are collected by computer equipment, a high-quality human face similar to a target human face can be output, the human face precision after fitting is improved, and a task of driving the human face to make an action or an expression is conveniently executed subsequently.

It should be noted that: the construction apparatus for a high-precision face shape library provided in the above embodiment is only illustrated by dividing each functional module, and in practical applications, the above function allocation may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus for constructing the high-precision face shape library and the method for constructing the high-precision face shape library provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in detail in the method embodiments and are not described herein again.

Fig. 11 shows a schematic structural diagram of a server according to an exemplary embodiment of the present application. The server may be the server 220 in the computer system 100 shown in fig. 2.

The server 1100 includes a Central Processing Unit (CPU) 1101, a system Memory 1104 including a Random Access Memory (RAM) 1102 and a Read Only Memory (ROM) 1103, and a system bus 1105 connecting the system Memory 1104 and the Central Processing Unit 1101. The server 1100 also includes a basic Input/Output System (I/O) 1106, which facilitates information transfer between devices within the computer, and a mass storage device 1107 for storing an operating System 1113, application programs 1114, and other program modules 1115.

The basic input/output system 1106 includes a display 1108 for displaying information and an input device 1109 such as a mouse, keyboard, etc. for user input of information. Wherein the display 1108 and the input device 1109 are connected to the central processing unit 1101 through an input output controller 1110 connected to the system bus 1105. The basic input/output system 1106 may also include an input/output controller 1110 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 1110 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 1107 is connected to the central processing unit 1101 through a mass storage controller (not shown) that is connected to the system bus 1105. The mass storage device 1107 and its associated computer-readable media provide non-volatile storage for the server 1100. That is, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or Compact disk Read Only Memory (CD-ROM) drive.

Computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other Solid State Memory technology, CD-ROM, Digital Versatile Disks (DVD), or Solid State Drives (SSD), other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM). Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 1104 and mass storage device 1107 described above may be collectively referred to as memory.

The server 1100 may also operate in accordance with various embodiments of the application through remote computers connected to a network, such as the internet. That is, the server 1100 may connect to the network 1112 through the network interface unit 1111 that is coupled to the system bus 1105, or may connect to other types of networks or remote computer systems (not shown) using the network interface unit 1111.

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

In an alternative embodiment, a computer device is provided, which includes a processor and a memory, wherein at least one instruction, at least one program, code set, or instruction set is stored in the memory, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the method for constructing the high-precision face shape library as described above.

In an alternative embodiment, a computer-readable storage medium is provided, in which at least one instruction, at least one program, code set, or instruction set is stored, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method for constructing a high-precision face shape library as described above.

Optionally, the computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM). The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the merits of the embodiments.

The embodiment of the present application further provides a computer program product, which, when running on a computer, causes the computer to execute the method for constructing the high-precision face shape library provided by the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is intended to be exemplary only, and not to limit the present application, and any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application are intended to be included therein.

Claims

1. A method for constructing a high-precision face shape library, which is characterized by comprising the following steps:

2. The method according to claim 1, wherein the iteratively updating the extended first fitting error according to the second fitting error and the third fitting error, and constructing the high-precision face shape library according to the second face shape base and the first face shape base obtained at the end of the iteration comprises:

obtaining a screening error according to the second fitting error and the third fitting error;

adding the screening error to the first fitting error to obtain the expanded first fitting error;

and iteratively updating the expanded first fitting error to obtain the high-precision face shape library.

3. The method of claim 2, wherein the screening error comprises an error screening matrix;

obtaining a screening error according to the second fitting error and the third fitting error, including:

sorting error values in the third fitting error in a descending order to obtain m error positions corresponding to the first m error values, wherein m is a positive integer;

and determining the error screening matrix corresponding to the first m error values from the second fitting errors according to the m error positions.

4. The method of claim 2, wherein iteratively updating the extended first fitting error to obtain the high-precision face shape library comprises:

iteratively calculating the second face base based on the augmented first fit error;

determining the second face shape base and the first face shape base at the end of the iteration as the high-precision face shape library in response to the third fitting error satisfying an iteration convergence condition.

5. The method of claim 4, wherein determining the second face base and the first face base at the end of the iteration as the high-precision face shape library in response to the third fitting error satisfying an iteration convergence condition comprises:

obtaining a maximum fitting error corresponding to the iteration based on a fitting result of the expanded first fitting error to the third fitting error;

and determining that the iteration convergence condition is met in response to the fact that the error difference value between the maximum fitting error corresponding to the current iteration and the maximum fitting error corresponding to the last iteration is smaller than a threshold value, and determining the second facial form base and the first facial form base at the time of finishing the iteration as the high-precision facial form library.

6. The method of any of claims 1 to 5, wherein the first fitting error comprises an error matrix;

the constructing a second face base by the first face base and the first fitting error comprises:

calculating a fitting error when the first face base is fitted with the extended data;

obtaining the error matrix according to the fitting error;

performing principal component analysis on the error matrix to obtain a mean value, a principal component eigenvector matrix and a principal component variance corresponding to the error matrix, wherein the principal component variance is an eigenvalue corresponding to an eigenvector in the principal component eigenvector matrix;

and constructing the second face base according to the mean value, the principal component eigenvector matrix and the principal component variance.

7. The method of claim 6, wherein the principal component variance comprises a column vector;

the constructing the second face shape base according to the mean, the principal component eigenvector matrix, and the principal component variance comprises:

performing matrix change on the principal component variance to obtain a principal component diagonal matrix, wherein the principal component diagonal matrix comprises a principal component standard deviation;

determining the mean, the principal component feature vector matrix, and the principal component diagonal matrix as the second face base.

8. The method of any of claims 1 to 5, wherein fitting the perturbation data with the first facial base to obtain a second fitting error comprises:

fitting the first facial form base and the disturbance data to obtain a first facial form coefficient;

and calculating to obtain the second fitting error according to the low-frequency face shape base, the disturbance data and the first face shape coefficient.

9. The method of any of claims 1 to 5, wherein said fitting the second fitting error with the second facetted portion to obtain a third fitting error for the second facetted portion comprises:

fitting the second face base and the disturbance data to obtain a second face shape coefficient;

and calculating to obtain the third fitting error according to the second facial form base, the second fitting error and the second face shape coefficient, wherein the second fitting error corresponds to the first fitting error.

10. The method of any of claims 1 to 5, further comprising:

obtaining a low-frequency fitting coefficient, a high-frequency fitting coefficient and the high-precision face shape library, wherein the low-frequency fitting coefficient is used for representing the shape of the first face base, and the high-frequency fitting coefficient is used for representing the shape of the second face base;

and calculating to obtain the face shape of the target face according to the low-frequency fitting coefficient, the high-frequency fitting coefficient and the high-precision face shape library.

11. An apparatus for constructing a high-precision face shape library, the apparatus comprising:

12. A computer device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the method of constructing a high accuracy library of human face shapes according to any one of claims 1 to 10.

13. A computer-readable storage medium, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the computer-readable storage medium, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by a processor to implement the method for constructing the high-precision face shape library according to any one of claims 1 to 10.