WO2022110791A1 - Method and apparatus for face reconstruction, and computer device, and storage medium - Google Patents

Abstract

The present invention provides a method and apparatus for face reconstruction, and a computer device, and a storage medium. The method comprises: generating a first real face model on the basis of a target image; performing, by using multiple pre-generated second real face models, fitting processing on the first real face model to obtain fitting coefficients respectively corresponding to the multiple second real face models; generating target bone data and a target skin morph coefficient on the basis of the fitting coefficients respectively corresponding to the multiple second real face models, and virtual face models having preset styles and respectively corresponding to the multiple second real face models; and generating, on the basis of the target bone data and the target skin morph coefficient, a target virtual face model corresponding to the first real face model.

Description

Method, device, computer equipment and storage medium for reconstructing human face

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims the priority of the Chinese patent application filed on November 25, 2020, with the application number of 202011337901.1 and the invention titled "A method, device, computer equipment and storage medium for face reconstruction", which is cited by reference manner is incorporated into this article.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to a method, an apparatus, a computer device and a storage medium for reconstructing a human face.

Background technique

Usually, a three-dimensional model of a virtual face can be established according to a real face or one's own preferences, so as to realize the reconstruction of the face, which has a wide range of applications in the fields of games, animation, and virtual social interaction. For example, in a game, the player can use the face reconstruction system provided by the game program to generate a virtual face 3D model according to the real face included in the image provided by the player, and use the generated virtual face 3D model to have a more sense of substitution participation in the game.

At present, when performing face reconstruction based on real faces included in portrait images, face contour features are usually extracted based on face images, and then the extracted face contour features and pre-generated virtual three-dimensional models are matched and fused. , to generate a 3D model of the virtual face corresponding to the real face; however, due to the low matching degree between the contour features of the face and the pre-generated virtual 3D model, the generated 3D model of the virtual face and the real face image are closely related. the similarity is low.

SUMMARY OF THE INVENTION

The embodiments of the present disclosure provide at least a method, an apparatus, a computer device, and a storage medium for reconstructing a human face.

In a first aspect, an embodiment of the present disclosure provides a method for reconstructing a face, including: generating a first real face model based on a target image; performing a fitting process on the face model to obtain fitting coefficients corresponding to the plurality of second real face models respectively; based on the fitting coefficients corresponding to the plurality of second real face models respectively, the plurality of second real face models The virtual face models with preset styles corresponding to the models respectively generate target bone data and target skin deformation coefficients; based on the target bone data and the target skin deformation coefficients, generate the first real face model The corresponding target virtual face model.

In this embodiment, using the fitting coefficient as a medium, a plurality of associations between the second real face models and the first real face models are established, and the associations can represent the virtual reality established based on the second real face models. The relationship between the face model and the target virtual face model established based on the first real face model; in addition, the target skin deformation coefficient can characterize the deformation characteristics of the face skin in the target image, such as the case of the same bones There is a difference in fat and thinness that can be represented by the skin; the target virtual face model determined based on the fitting coefficient and the target skin deformation coefficient has both the preset style and the original face corresponding to the first real face model. It can also reflect the fat and thin features of the original face, and the generated target virtual face model has a higher similarity with the original face corresponding to the first real face model.

In an optional implementation manner, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively , generating a target skin deformation coefficient, comprising: generating the target mask based on the fitting coefficients corresponding to the plurality of second real face models and the skin deformation coefficients respectively included in the plurality of virtual face models Skin deformation coefficient.

In this embodiment, the standard skin data of the standard virtual face model is used as a reference, and after the skin deformation coefficient of the virtual face model is determined, the correlation between the characterizing virtual face model and the target virtual face model can be based on The fitting coefficient of the target virtual face can determine the target skin deformation coefficient of the target virtual face, so that the skin data of the target virtual face can be more accurately determined based on the target skin deformation coefficient, so that the generated target virtual face model and the first The original face corresponding to the real face model has a higher similarity.

In an optional implementation manner, the generating the said multiple second real face models is based on the fitting coefficients corresponding to the multiple second real face models and the skin deformation coefficients respectively included in the multiple virtual face models. The target skin deformation coefficient includes: normalizing the fitting coefficients corresponding to the plurality of second real face models respectively; based on the normalized fitting coefficients and the virtual face models respectively The included skin deformation coefficients are obtained to obtain the target skin deformation coefficients.

In this embodiment, by performing normalization processing on the fitting coefficients corresponding to the plurality of second real face models respectively, the fitting coefficients based on the normalization processing and the virtual face models respectively include When the target skin deformation coefficient is obtained from the skin deformation coefficient of , the expression of the data is simpler, the processing process is simplified, and the processing speed of subsequent face reconstruction using the fitting results is improved.

In an optional implementation manner, the generating a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient includes: Describe the target skeleton data and the association relationship between the standard skeleton data and the standard skin data in the standard virtual face model, perform position transformation processing on the standard skin data, and generate intermediate skin data; based on the target skin data The deformation coefficient is used to deform the intermediate skin data to obtain target skin data; based on the target bone data and the target skin data, the target virtual face model is formed.

In this embodiment, after the intermediate skin data is generated, the intermediate skin data is deformed by using the target skin deformation coefficient, and the obtained target skin data can not only represent the appearance features of the first real face model, but also can express The degree of fatness and thinness of the first real face is obtained, and the generated target virtual face model not only has the difference in appearance, but also has the difference in the degree of fatness and thinness, so that when different target virtual faces are generated, they are different from those of the first real face. The original face corresponding to the model has a higher similarity.

In an optional implementation manner, the target skeleton data includes at least one of the following: target skeleton position data, target skeleton scaling data, and target skeleton rotation data; skeleton data corresponding to the plurality of virtual face models respectively It includes at least one of the following: bone rotation data, bone position data, and bone scaling data corresponding to each of the multiple face bones of the virtual face.

In this embodiment, the bone data corresponding to each of the multiple face bones can be more accurately represented by using the bone data, and the target virtual face model can be determined more accurately by using the target bone data.

In an optional implementation manner, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively , generating target skeleton data, comprising: performing interpolation processing on the skeleton position data corresponding to the plurality of virtual face models based on the fitting coefficients corresponding to the plurality of second real face models respectively, to obtain the target skeleton location data.

In an optional implementation manner, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively , generating target skeleton data, comprising: performing interpolation processing on the skeleton scaling data corresponding to the plurality of virtual face models based on the fitting coefficients corresponding to the plurality of second real face models respectively, to obtain the target skeleton Scale data.

In an optional implementation manner, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively , generating target skeleton data, including: converting the skeleton rotation data corresponding to the plurality of virtual face models into quaternions, and performing regularization processing on the quaternions corresponding to the plurality of virtual face models, respectively, obtaining a regularized quaternion; based on the fitting coefficients corresponding to the multiple second real face models respectively, perform interpolation processing on the regularized quaternions corresponding to the multiple virtual face models respectively, to obtain the obtained Describe the target bone rotation data.

In an optional implementation manner, the generating the first real face model based on the target image includes: acquiring a target image including an original face; performing a three-dimensional human face analysis on the original face included in the target image face reconstruction to obtain the first real face model.

In this embodiment, the face features of the original face in the target image can be more accurately and comprehensively represented by using the first real face model obtained by reconstructing the original face in three dimensions.

In an optional implementation manner, multiple second real face models are pre-generated according to the following methods: acquiring multiple reference images including reference faces; for each reference image in the multiple reference images , performing three-dimensional face reconstruction on the reference face included in the reference image to obtain the second real face model corresponding to the reference image.

In this embodiment, by using multiple reference images, it is possible to cover a wider range of face shape features as much as possible. Therefore, the second real face model obtained by performing 3D face reconstruction based on each reference image in the multiple reference images is also the same as It can cover as wide a range of facial features as possible.

In an optional implementation manner, the method further includes, for each second real face model in the plurality of second real face models, using the following method to obtain the corresponding A virtual face model with a preset style: generating an intermediate virtual face model with a preset style corresponding to the second real face model; based on multiple sets of preset skin deformation coefficients relative to the standard virtual face model, generating The skin deformation coefficient of the virtual face model corresponding to the second real face model relative to the standard virtual face model; using the skin deformation coefficient, the intermediate mask in the intermediate virtual face model is The skin data is adjusted; based on the adjusted intermediate skin data and the intermediate skeleton data of the intermediate virtual face model, a virtual face model of each of the second real face models is generated.

In this embodiment, the intermediate skin data of the intermediate virtual face model corresponding to the second real face model is adjusted by the skin deformation coefficient, so that the generated virtual face model not only has the preset style, but also the second real face model. The appearance features of the face model can also represent the fatness and thinness of the reference face corresponding to the second real face model, so that the virtual face model and the corresponding reference face have a higher similarity.

In an optional implementation manner, the first real face model is fitted by using a plurality of pre-generated second real face models to obtain corresponding corresponding second real face models respectively. The fitting coefficients include: performing least squares processing on the plurality of second real face models and the first real face models to obtain fitting coefficients corresponding to the plurality of second real face models respectively.

In this embodiment, by using the fitting coefficient, the fitting situation when the first real face model is fitted by using a plurality of second real face models can be accurately represented.

In a second aspect, an embodiment of the present disclosure further provides an apparatus for reconstructing a human face, including: a first generating module for generating a first real face model based on a target image; a processing module for using a plurality of pre-generated Two real face models perform fitting processing on the first real face model to obtain fitting coefficients corresponding to a plurality of second real face models respectively; The fitting coefficients corresponding to the face models, and the virtual face models with preset styles corresponding to the plurality of second real face models, respectively, generate target skeleton data and target skin deformation coefficients; a third generation module, for generating a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient.

In an optional embodiment, the second generation module is based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the preset values corresponding to the plurality of second real face models respectively. The style of the virtual face model, when generating the target skin deformation coefficient, is used for: based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the skins respectively included in the plurality of the virtual face models deformation coefficient to generate the target skin deformation coefficient.

In an optional embodiment, the second generation module is based on the fitting coefficients corresponding to the plurality of second real face models and the skin deformation coefficients respectively included in the plurality of virtual face models. , when generating the target skin deformation coefficient, it is used to: perform normalization processing on the fitting coefficients corresponding to the plurality of second real face models respectively; The skin deformation coefficients respectively included in the virtual face models are obtained to obtain the target skin deformation coefficients.

In an optional embodiment, when the third generation module generates the target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient is used to: perform position transformation processing on the standard skin data based on the target bone data and the association between the standard bone data and standard skin data in the standard virtual face model, and generate intermediate skin data; Based on the target skin deformation coefficient, the intermediate skin data is deformed to obtain target skin data; the target virtual face model is formed based on the target bone data and the target skin data.

In an optional implementation manner, the target skeleton data includes at least one of the following: target skeleton position data, target skeleton scaling data, and target skeleton rotation data; the skeleton data corresponding to the plurality of virtual face models respectively include: At least one of the following: bone rotation data, bone position data, and bone scaling data corresponding to each face bone in the multiple face bones of the virtual face.

In an optional embodiment, the second generation module is based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the preset values corresponding to the plurality of second real face models respectively. The style of the virtual face model, when generating the target skeleton data, is used for: based on the fitting coefficients corresponding to the plurality of second real face models, respectively, on the skeleton position data corresponding to the plurality of virtual face models. Interpolation processing is performed to obtain the target bone position data.

In an optional embodiment, the second generation module is based on the fitting coefficients corresponding to the plurality of second real face models and the preset styles corresponding to the plurality of second real face models respectively. The virtual face model, when generating target skeleton data, is used to: perform interpolation processing on the skeleton scaling data corresponding to the plurality of virtual face models based on the fitting coefficients corresponding to the plurality of second real face models respectively to obtain the target bone scaling data.

In an optional embodiment, the second generation module is based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the preset values corresponding to the plurality of second real face models respectively. The style of the virtual face model, when generating target skeleton data, is used to: convert the skeleton rotation data corresponding to the plurality of virtual face models respectively into quaternions, and to the corresponding virtual face models of the plurality of virtual face models. Perform regularization processing on the quaternion to obtain a regularized quaternion; based on the fitting coefficients corresponding to the plurality of second real face models respectively, the regularization quaternions corresponding to the plurality of virtual face models The arity is interpolated to obtain the rotation data of the target bone.

In an optional embodiment, when generating the first real face model based on the target image, the first generation module is used to: acquire a target image including the original face; Three-dimensional face reconstruction is performed on the original face to obtain the first real face model.

In an optional implementation manner, the processing module pre-generates the plurality of second real face models according to the following methods: acquiring a plurality of reference images including a reference face; for each of the plurality of reference images; A reference image is taken, and three-dimensional face reconstruction is performed on the reference face included in the reference image to obtain the second real face model corresponding to the reference image.

In an optional implementation manner, an acquisition module is also included for acquiring the second real human face in the following manner for each second real human face model in the plurality of second real human face models. A virtual face model with a preset style corresponding to the model: generating an intermediate virtual face model with a preset style corresponding to the second real face model; based on multiple sets of preset skins relative to the standard virtual face model deformation coefficient, to generate the skin deformation coefficient of the virtual face model corresponding to the second real face model relative to the standard virtual face model; using the skin deformation coefficient, to the intermediate virtual face model The middle skin data in the middle skin is adjusted; based on the adjusted middle skin data and the middle skeleton data of the middle virtual face model, a virtual face model of the second real face model is generated.

In an optional implementation manner, the processing module uses a plurality of pre-generated second real face models to perform fitting processing on the first real face model, and obtains a plurality of second real face models corresponding to When the fitting coefficient is , it is used to: perform least squares processing on the plurality of second real face models and the first real face models to obtain the corresponding fitting coefficients of the plurality of second real face models respectively. Combined coefficient.

In a third aspect, an optional implementation manner of the present disclosure further provides a computer device, a processor, and a memory, where the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the instructions stored in the memory. machine-readable instructions, when the machine-readable instructions are executed by the processor, when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or any possible implementation of the first aspect, is executed steps in the method.

In a fourth aspect, an optional implementation manner of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program executes the first aspect, or any of the first aspect, when the computer program is run. steps in one possible implementation.

For a description of the effects of the above-mentioned apparatus for reconstructing a human face, computer equipment, and a computer-readable storage medium, please refer to the description of the above-mentioned method for reconstructing a human face, which will not be repeated here. In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. These drawings illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1 shows a flowchart of a method for reconstructing a human face provided by an embodiment of the present disclosure;

FIG. 2 shows a flowchart of a method for reconstructing a human face provided by another embodiment of the present disclosure;

FIG. 3 shows a flowchart of a specific method for obtaining a target skin deformation coefficient provided by an embodiment of the present disclosure;

4 shows a flowchart of a specific method for generating a target virtual face model corresponding to a first real face model based on target bone data and target skin deformation coefficients provided by an embodiment of the present disclosure;

FIG. 5 shows an example of multiple faces and face models involved in a method for reconstructing a face provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of an apparatus for reconstructing a human face provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

After research, it is found that the method of face reconstruction can establish a three-dimensional model of virtual face according to the real face or one's own preferences. Among them, in the case of face reconstruction based on the real face in the portrait image, the feature extraction is usually performed on the real face in the portrait image to obtain the face contour features, and then the face contour features are combined with the pre-generated face contour features. The features in the virtual 3D model are matched, and based on the matching results, the facial contour features are fused with the virtual 3D model to obtain a virtual 3D model of the face corresponding to the real face in the portrait image. Since the matching accuracy is low when the face contour features are matched with the features in the pre-generated virtual three-dimensional model, the matching error between the virtual three-dimensional model and the face contour features is relatively large, and it is easy to cause the matching results based on the matching results. The problem of low similarity between the virtual face 3D model obtained by fusing the face contour feature and the face virtual 3D model and the face in the portrait image.

In view of the defects of the above solutions, the embodiments of the present disclosure provide a method for reconstructing a face, which can generate a target virtual image with a preset style and features of the original face corresponding to the first real face model The face model, the target virtual face model can reflect the fat and thin features of the original face, and has a high similarity with the original face corresponding to the first real face model.

In order to facilitate the understanding of this embodiment, a method for reconstructing a face disclosed by an embodiment of the present disclosure is first introduced in detail. The execution subject of the method for reconstructing a face provided by the embodiment of the present disclosure is generally a computer with a certain computing capability. equipment, the computer equipment for example includes: terminal equipment or server or other processing equipment, the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementations, the method for reconstructing a human face may be implemented by the processor calling computer-readable instructions stored in the memory.

The following describes the method for reconstructing a human face provided by the embodiments of the present disclosure.

FIG. 1 is a flowchart of a method for reconstructing a face provided by an embodiment of the present disclosure. As shown in FIG. 1 , the method includes steps S101 to S104, wherein:

S101: Generate a first real face model based on the target image.

S102: Perform fitting processing on the first real face model by using multiple pre-generated second real face models to obtain fitting coefficients corresponding to the multiple second real face models respectively.

S103: Generate target skeleton data and target skin deformation coefficients based on fitting coefficients corresponding to the plurality of second real face models and virtual face models with preset styles corresponding to the plurality of second real face models respectively . The target skin deformation coefficient may represent the deformation of the skin data of the target face model to be generated relative to the standard skin data of the pre-generated standard virtual face model.

S104: Generate a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient.

The embodiment of the present disclosure provides a method for reconstructing a face, which uses the fitting coefficient as a medium to establish a plurality of associations between the second real face models and the first real face model, and the associations can represent The relationship between the virtual face model established by the second real face model and the target virtual face model established based on the first real face model, at the same time, the face skin deformation in the target image is represented by the target skin deformation coefficient characteristics, such as the difference in fat and thinness when the bones are the same, so that the target virtual face model is generated based on the fitting coefficient and the virtual face model. The target virtual face model has both a preset style and a first The features of the original face corresponding to the real face model can also reflect the fat and thin features of the original face. The generated target virtual face model and the original face corresponding to the first real face model have a higher difference. similarity.

The above steps S101 to S104 will be described in detail below.

For the above step S101, the target image is, for example, an acquired image including a human face, for example, an image including a human face acquired when a certain object is photographed with a photographing device such as a camera. At this time, for example, any face included in the image can be determined as the original face, and the original face can be used as the object of face reconstruction.

When the method for reconstructing a face provided by the embodiment of the present disclosure is applied to different scenarios, the method for acquiring the target image is also different.

For example, when the method for reconstructing a human face is applied to a game, an image including the face of the game player can be obtained through an image obtaining device installed in the game device, or an image including the face of the game player can be selected from the album of the game device. The image of the face of the game player, and the acquired image including the face of the game player is used as the target image.

For another example, when the method for reconstructing a face is applied to a terminal device such as a mobile phone, an image including the user's face can be collected by the camera of the terminal device, or an image including the user's face can be selected from the album of the terminal device, Or receive images including the user's face from other applications installed in the terminal device.

For another example, when the method for reconstructing a human face is applied to a live broadcast scene, a video frame image containing a human face can be obtained from multiple frames of video frame images included in a video stream obtained by a live broadcast device; image as the target image. Here, for example, the target image may have multiple frames; for example, the multiple-frame target image may be obtained by sampling a video stream.

When generating the first real face model based on the target image, for example, the following methods may be used: acquiring a target image including the original face; performing three-dimensional face reconstruction on the original face included in the target image to obtain the first real face Model.

Here, when performing three-dimensional face reconstruction on the original face included in the target image, for example, a three-dimensional deformable face model (3 Dimensions Morphable Models, 3DMM) can be used to obtain the first real face model corresponding to the original face. Wherein, the first real face model includes, for example, the position information of each key point in the preset camera coordinate system among the multiple key points of the original face in the target image.

For the above step S102, the second real face model is generated based on the reference image including the reference face. The reference faces in different reference images may be different; exemplarily, multiple people with different at least one of gender, age, skin color, degree of fatness and thinness, etc. may be selected, and for each of the multiple people, obtain the face image, and use the acquired face image as a reference image. In this way, the plurality of second real face models obtained based on the plurality of reference images can cover a relatively wide range of face shape features as much as possible.

The reference face includes, for example, faces corresponding to N different objects, (N is an integer greater than 1). Exemplarily, by shooting N different objects respectively, N photos corresponding to the N different objects can be obtained, and each photo corresponds to a reference face. At this time, the N photos may be used as N reference images; or, N reference images may be determined from a plurality of pre-shot images including different faces.

Exemplarily, the method for generating multiple second real face models includes: acquiring multiple reference images including a reference face; for each reference image in the multiple reference images, obtaining the reference face included in the reference image. Perform three-dimensional face reconstruction to obtain a second real face model corresponding to the reference image.

Wherein, the method for performing 3D face reconstruction on the reference face is similar to the above-mentioned method for performing 3D face reconstruction on the original face, and will not be repeated here. The obtained second real face model includes position information of each key point in the preset camera coordinate system among the multiple key points of the reference face in the reference image. At this time, the coordinate system of the second real face model and the coordinate system of the first real face model may be the same coordinate system.

Perform fitting processing on the first real face model by using multiple pre-generated second real face models to obtain multiple fitting coefficients corresponding to the multiple second real face models respectively. For example, the following methods can be used to achieve : Least square processing is performed on the multiple second real face models and the first real face model to obtain the fitting coefficients corresponding to the multiple second real face models respectively.

Exemplarily, the model data corresponding to the first real face model can be represented as D _a , and the model data corresponding to the second real face model can be represented as D _bi (i∈[1,N]), where D _bi Indicates the i-th second real face model among the N second real face models.

Using D _a to perform least squares processing on each of D _b1 to D _bN , N fitting values can be obtained, and the fitting values are expressed as α _i (i∈[1,N]). Among them, α _i represents the fitting value corresponding to the ith second real face model. Using the N fitted values, the fitting coefficient Alpha can be determined, for example, it can be represented by a coefficient matrix, that is,

Alpha=[α ₁ ,α ₂ ,...,α _N ].

Here, in the process of fitting the first real face model by using the plurality of second real face models, the data obtained after the weighted summation of the plurality of second real face models by using the plurality of fitting coefficients can be as close as possible to the data of the first real face model.

The fitting coefficient can also be regarded as an expression coefficient of each second real face model when the first real face model is expressed by using a plurality of second real face models. That is, the second real face model can be converted and fitted to the first real face model by using the respective fitting values corresponding to the expression coefficients of the multiple second real face models.

For the above step S103, the preset style may be, for example, a cartoon style, an ancient style, or an abstract style, and may be specifically set according to actual needs. Exemplarily, for the case where the preset style is a cartoon style, the virtual face model with the preset style may be, for example, a virtual face model with a certain cartoon style.

Wherein, the virtual face model may include bone data, and skin data and/or skin deformation coefficients. The skin deformation coefficient represents the deformation of the skin data of the virtual face model relative to the standard skin data of the pre-generated standard virtual face model.

Referring to FIG. 2 , an embodiment of the present disclosure provides that, for each second real face model in the plurality of second real face models, a virtual image with a preset style corresponding to the second real face model is generated. Specific methods of face model, including:

S201: Generate an intermediate virtual face model with a preset style corresponding to the second real face model.

Here, the method for generating an intermediate virtual face model with a preset style corresponding to the second real face model includes, for example, at least one of the following (a1) and (a2).

(a1) Based on the reference image, a virtual face image with reference face features and a preset style can be produced, and three-dimensional modeling of the virtual face in the virtual face image can be obtained to obtain a virtual face in the virtual face image. Skeletal data for the face as well as skin data.

The skeleton data includes skeleton rotation data, skeleton scaling data, and skeleton position data of multiple bones preset for the virtual face in a preset coordinate system. Here, for example, multiple bones can be divided into multiple levels; for example, they include root (root) bones, facial bones, and facial detail bones; wherein facial bones may include: eyebrow bones, nasal bones, zygomatic bones, mandibular bones, and mouth bones Etc.; the facial features detail bones, for example, can further divide different facial features bones into further details. Specific settings can be made according to different styles of virtual image requirements, which are not limited here.

The skinned data includes position information of multiple position points on the surface of the virtual face in the preset model coordinate system, and information about the association relationship between each position point and at least one bone among the multiple bones. The model coordinate system is a three-dimensional coordinate system established for the virtual face model.

The virtual model obtained by performing three-dimensional modeling on the virtual face in the virtual face image is used as an intermediate virtual face model corresponding to the second real face model.

(a2) Pre-generate a standard virtual face model with a preset style. The standard virtual face model also includes standard bone data, standard skin data, and an association relationship between the standard bone data and the standard skin data. Based on the face features of the reference face corresponding to each reference image in the multiple reference images, the standard skeleton data in the standard virtual face model is adjusted, so that the adjusted standard virtual face model has a preset style At the same time, it also includes the features of the reference face in the reference image; then, based on the relationship between the standard bone data and the standard skin data, the standard skin data is adjusted, and a reference can also be added to the standard skin data. The feature information of the human face is based on the modified standard skeleton data and the modified standard skin data to generate an intermediate virtual face model corresponding to the second real face model.

Here, for the specific data representation of the intermediate virtual face model, reference may be made to the description in (a1) above, which will not be repeated here.

S202: Generate skin deformation coefficients of the virtual face model corresponding to the second real face model relative to the standard virtual face model based on multiple sets of preset skin deformation coefficients relative to the standard virtual face model.

Here, for the standard virtual face model, the generated sets of skin deformation coefficients are only used to characterize the standard virtual human in the standard skin data of the standard virtual face model under the condition that the bones of the standard virtual face model have not changed. An adjustment coefficient for adjusting at least part of the position points corresponding to specific positions of the face model, such as cheekbones.

Wherein, each group of skin deformation coefficients represents the result of adjusting the position of at least part of the position points in the standard skin data in the model coordinate system, so that the parts corresponding to the adjusted position points in the standard virtual face model appear fat or slimming effect.

When using multiple sets of preset skin data to combine the skin deformation coefficients corresponding to the reference face, for example, multiple sets of preset skin data can be fitted, so that the fitting result is consistent with the face shape of the reference face. resemblance.

S203: Adjust the intermediate skin data in the intermediate virtual face model by using the skin deformation coefficient, and generate the second real person based on the adjusted intermediate skin data and the intermediate bone data of the intermediate virtual face model The virtual face model corresponding to the face model.

For example, in a possible implementation, R groups of preset skin deformation coefficients Blendshape may be obtained; here, each group of preset skin deformation coefficients includes points corresponding to multiple positions in the skin data respectively. deformation coefficient value. Exemplarily, if there are W position points in the skin data, and each position point corresponds to a deformation coefficient value, the dimension of each group of skin deformation coefficients in the R groups of preset skin deformation coefficients is W.

Among them, Blendshape _i (i∈[1,R]) is used to represent the i-th group of preset skin deformation coefficients. Using the R group of preset skin deformation coefficients, the fatness and thinness of the standard virtual face model can be modified to obtain R standard virtual face models after adjusting the fatness and thinness features.

When generating a virtual face model, the skin deformation coefficients of the virtual face model can be obtained by combining the preset skin deformation coefficients Blendshape of the R group. Here, for example, corresponding weights can be added to different preset skin deformation coefficients, and by using the weights, the R group of preset skin deformation coefficients can be weighted and summed to obtain the skin of a certain virtual face model. deformation factor.

Exemplarily, when N second real face models are generated in advance and R groups of preset skin deformation coefficients are acquired, the dimension of the skin deformation coefficient Blendshape _i of the i-th real face is R×W. The skin deformation coefficients corresponding to the N second real face models can form a matrix with a dimension of N×R×W; the matrix includes the virtual face models corresponding to the N second real face models respectively. Skin deformation factor.

In addition, when using the skin deformation coefficient to adjust the skin data in the intermediate virtual face model, the skeleton data of the intermediate virtual face model can also be fine-tuned to optimize the facial details of the generated virtual face model. , so that the generated virtual face model has a higher similarity with the reference face.

After the virtual face models corresponding to the N second real face models are obtained, the target virtual face model can be fitted by using the N virtual face models and the corresponding fitting coefficients, and the target skeleton data and the target mask can be generated. Skin deformation data.

Specifically, the target virtual face model includes: target bone data and target skin data; wherein the target skin data is determined based on the target bone data and target skin deformation data of the target virtual face model.

In the embodiment of the present disclosure, when the target skeleton data is obtained based on the fitting coefficients corresponding to the multiple second real face models and the skeleton data corresponding to the multiple virtual face models, for example, the method includes: based on the multiple second real face models According to the fitting coefficients corresponding to the models, the skeleton data corresponding to the multiple virtual face models are interpolated to obtain the target skeleton data.

The bone data corresponding to the virtual face model includes at least one of the following: bone rotation data, bone position data and bone scaling data corresponding to each face bone among the multiple face bones of the virtual face. The obtained target bone data includes at least one of the following: target bone position data, target bone scaling data, and target bone rotation data.

Exemplarily, based on the fitting coefficients corresponding to the plurality of second real face models, interpolation processing is performed on the skeleton data corresponding to the plurality of virtual face models respectively to obtain the target skeleton data, for example, the following (b1) can be used: To at least one of (b3):

(b1) Based on the fitting coefficients corresponding to the plurality of second real face models respectively, perform interpolation processing on the bone position data corresponding to the plurality of virtual face models respectively to obtain target bone position data.

(b2) Based on the fitting coefficients corresponding to the plurality of second real face models respectively, perform interpolation processing on the skeleton scaling data corresponding to the plurality of virtual face models respectively to obtain the target skeleton scaling data.

(b3) Converting the bone rotation data corresponding to multiple virtual face models into quaternions, and performing regularization processing on the obtained quaternions to obtain regularized quaternions; based on multiple second real face models According to the corresponding fitting coefficients, interpolation processing is performed on the regularization quaternions corresponding to the multiple virtual face models respectively to obtain the target bone rotation data.

In a specific implementation, for the above-mentioned method (b1) and method (b2), in the case of acquiring bone position data and bone scaling data, it also includes determining each level of bones and each level based on a plurality of second real face models. The local coordinate system corresponding to the bone. Among them, in the case of stratifying the skeleton level of the face model, for example, the skeleton level can be determined directly according to the biological skeleton stratification method, or the skeleton level can be determined according to the requirements of face reconstruction, and the specific stratification method can be determined according to the actual situation. The situation is determined and will not be repeated here.

After each bone level is determined, a bone coordinate system corresponding to each bone level can be established based on each bone level. Illustratively, each level of bone can be represented as Bone _i .

At this time, the bone position data may include the three-dimensional coordinate values of each level of bone Bone _i in the virtual face model in the corresponding bone coordinate system; the bone scaling data may include the corresponding bone Bone _i of each level in the virtual face model. In the bone coordinate system, the percentage used to represent the degree of bone scaling, such as 80%, 90% or 100%.

In a possible implementation manner, the bone position data corresponding to the ith virtual face model is represented as Pos _i , and the bone scaling data corresponding to the ith virtual face model is represented as Scaling _i . At this time, the bone position data Pos _i includes the bone position data corresponding to the plurality of hierarchical bones respectively, and the bone scaling data Scaling _i includes the bone scaling data corresponding to the plurality of hierarchical bones respectively.

At this time, the corresponding fitting coefficient is a _i . Based on the fitting coefficients corresponding to the M second real face models respectively, interpolation processing is performed on the bone position data Pos _i corresponding to the M virtual face models respectively to obtain the target bone position data.

Exemplarily, for example, the fitting coefficient can be used as the weight corresponding to each virtual face model, and the weighted summation processing is performed on the bone position data Pos _i corresponding to the M virtual face models respectively, so as to realize the process of interpolation processing. At this time, the target bone position data Pos _new satisfies the following formula (1):

Similarly, based on the fitting coefficients corresponding to the M second real face models, interpolation processing is performed on the bone scaling data corresponding to the M virtual face models respectively, so as to obtain the target bone scaling data, wherein the ith virtual human The bone scaling data corresponding to the face model is represented as Scaling _i , and the fitting coefficients corresponding to the M second real face models can be used as the weights of the corresponding virtual face models, and the bones corresponding to the M virtual face models can be scaled respectively. The data is weighted and summed to realize interpolation processing on the M virtual face models; in this case, the target bone scaling data Scaling _new satisfies the following formula (2):

For the above method (b3), the bone rotation data may include a vector value representing the degree of rotation coordinate transformation of the bones for each bone in the virtual face model in the corresponding bone coordinate system, for example, including a rotation axis and a rotation angle . In a possible implementation manner, the bone rotation data corresponding to the ith virtual face model is represented as Trans _i . Since the rotation angle contained in the bone rotation data has the problem of gimbal deadlock, the bone rotation data is converted into a quaternion, and the quaternion is regularized to obtain the regularized quaternion data, which is expressed as Trans' _i , in order to prevent overfitting when the quaternion is directly weighted and summed.

When performing interpolation processing on the regularization quaternion Trans' _i corresponding to the M second real face models based on the fitting coefficients corresponding to the M second real face models, the M second real face models may also be used for interpolation. The fitting coefficients corresponding to the models are used as weights, and the regularization quaternions corresponding to the M virtual face models are weighted and summed; in this case, the target bone rotation data Trans _new satisfies the following formula (3):

In addition, other interpolation methods can also be used to obtain target bone position data Pos _new , target bone scaling data Scaling _new , and target bone rotation data Trans _new , which can be determined according to actual needs, which are not limited in this disclosure.

Based on the target bone position data Pos _new , target bone scaling data Scaling _new , and target bone rotation data Trans _new obtained in (b1), (b2), and (b3) above, the target bone data can be determined, which is represented as Bone _new . Exemplarily, the target bone data can be expressed in vector form as:

(Pos _new , Scaling _new , Trans _new ).

When the fitting coefficients corresponding to the plurality of second real face models and the virtual face models with preset styles corresponding to the plurality of second real face models are determined, the target skin deformation coefficient is generated. For example, the following method may be adopted: generating the target skin based on the fitting coefficients corresponding to the multiple second real face models and the skin deformation coefficients respectively included in the multiple virtual face models deformation factor. The skin deformation coefficient of the virtual face model represents the deformation of the skin data of the virtual face model relative to the standard skin data of the pre-generated standard virtual face model.

Referring to FIG. 3 , an embodiment of the present disclosure further provides a specific method for obtaining a target skin deformation coefficient, including:

S301: Normalize the fitting coefficients corresponding to the plurality of second real face models respectively.

Wherein, in the case of normalizing the fitting coefficients corresponding to the multiple second real face models, for example, a normalization function (Softmax) may be used to obtain a probability value to represent the multiple second real faces The ratio of the corresponding fitting coefficients of the models among multiple fitting coefficients, and the normalized fitting coefficients are set as Alpha _Norm .

Exemplarily, when there are N second real face models, the dimension of the fitting coefficient Alpha _Norm obtained by performing the normalization process is N.

S302: Based on the normalized fitting coefficients, perform interpolation processing on the skin deformation coefficients respectively included in the multiple virtual face models to obtain the target skin deformation coefficients.

Here, the skin deformation coefficients included in the virtual face model are respectively fitted with the fitting coefficients corresponding to the plurality of second real face models, and the obtained fitting results can represent the pairs of the plurality of second real face models. The influence of the virtual face model to generate target skin deformation coefficients. The target skin deformation coefficient may, for example, adjust the fatness and thinness of the face, so that the obtained target virtual face model is consistent with the fatness and thinness features of the face in the target image.

Exemplarily, based on the normalized fitting coefficients, the skin deformation coefficients corresponding to the multiple virtual face models may be weighted and summed, so as to achieve the Corresponding skin deformation coefficients are subjected to interpolation processing to obtain the target skin deformation coefficients.

The fitting coefficient Alpha _Norm obtained by the normalization process can represent the first vector with dimension N, and the skin deformation coefficients corresponding to the R virtual face models respectively can form the second vector with dimension N×R; , the skin deformation coefficients corresponding to the multiple virtual face models are weighted and summed, for example, the first vector and the second vector may be directly multiplied to obtain the target skin deformation coefficient.

Exemplarily, for example, the following formula can be used to obtain the target skin deformation coefficient, which is expressed as Blendshape', and Blendshape' satisfies the following formula (4):

Blendshape'=Blendshape×Alpha _Norm (4).

For the above S104, referring to FIG. 4, an embodiment of the present disclosure further provides a specific method for generating a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient, include:

S401: Based on the target bone data and the relationship between the standard bone data and the standard skin data in the standard virtual face model, perform position transformation processing on the skin data to generate intermediate skin data.

Wherein, the association relationship between the standard skeleton data and the standard skin data in the standard virtual face model may be, for example, the association relationship between the standard skeleton data corresponding to each level of bones and the standard skin data. Based on this relationship, the skin can be bound to the bones in the virtual face model.

Using the target skeleton data and the relationship between the standard skeleton data and the standard skin data in the standard virtual face model, the skin data at the corresponding positions of multiple levels of bones can be subjected to position transformation, so that the generated target skin The position of the corresponding level bone in the data may be consistent with the position in the corresponding target bone data. In this case, for example, the skin data after the position transformation process may be used as the generated intermediate skin data.

S402: Perform deformation processing on the intermediate skin data based on the target skin deformation coefficient to obtain target skin data.

S403: Construct a target virtual face model based on the target bone data and the target skin data.

Here, using the target bone data, it is possible to determine the various levels of bones used to construct the target virtual face model; and using the target skin data, it is possible to determine the skin that binds the model to the bones, thereby forming the target virtual face model. .

The method for determining a target virtual face model includes at least one of the following: directly establishing a target virtual face model based on target bone data and target skin data; replacing the first real face with target bone data corresponding to each level of bones The corresponding skeleton data of each level in the model, and then use the target skin data to establish the target virtual face model. The specific method for establishing the target virtual face model can be determined according to the actual situation, and will not be repeated here.

The embodiments of the present disclosure also provide a method for reconstructing a face provided by the embodiments of the present disclosure, and describe the specific process of acquiring the target virtual face model Mod _Aim corresponding to the original face A in the target image Pic _A.

Determining the target virtual face model Mod _Aim includes the following steps (c1) to (c6):

(c1) Preparing materials; including: preparing materials for standard virtual face models; and preparing materials for virtual pictures.

When preparing the material of the standard virtual face model, take the cartoon style as the default style as an example, first set a cartoon style standard virtual face model Mod _Base .

Generate 9 groups of preset skin deformation coefficients; among them, use the 9 groups of skin deformation coefficients to change different parts and/or different degrees of the standard skin data of the standard virtual face model, and then the standard virtual face can be changed The fat and thin are adjusted to cover most face features.

When preparing the material of the virtual picture, collect 24 virtual pictures Pic ₁ to Pic ₂₄ ; the number of boys and girls corresponding to the virtual faces B ₁ to B ₂₄ in the collected 24 virtual pictures is balanced and includes as wide a range as possible. distribution of facial features.

(c2) face model reconstruction; including: using the original face A in the target image Pic _A to generate a first real face model Mod _fst ; and using the virtual faces B ₁ to B ₂₄ in the virtual picture to generate a second real face model Face model Mod _snd-1 ~ Mod _snd-24 .

When it is determined that the original face A generates the first real face model Mod _fst , firstly, the face in the target image is straightened and cropped, and then the pre-trained RGB is used to reconstruct the neural network to generate the first real face corresponding to the original face A. Face model Mod _fst . Similarly, by using the pre-trained RGB reconstruction neural network, the second real face models Mod _snd-1 to Mod _snd-24 corresponding to the virtual faces B ₁ to B ₂₄ respectively can be determined.

After determining the second real face models Mod _snd-1 to Mod _snd-24 , the method further includes: determining the second real face models Mod snd- ₁ to Mod _snd -24 by using a preset style and manual adjustment The corresponding virtual face models Mod _fic-1 to Mod _fic-24 with preset styles respectively.

In addition, based on 9 groups of preset skin deformation coefficients, the skin deformation coefficients of 24 virtual face models will be generated.

(c3) fitting processing; wherein, including: using a plurality of second real face models to perform fitting processing on the first real face model, to obtain the fitting coefficients corresponding to the plurality of second real face models respectively:

alpha=[alpha _snd-1 , alpha _snd-2 , . . . , alpha _snd-24 ].

When using a plurality of second real face models to fit the first real face model, the method of least squares is selected for fitting, and a 24-dimensional coefficient alpha is obtained.

(c4) Determining the target skin deformation coefficient; wherein, when determining the target skin deformation coefficient, the following (c4-1), (c4-2), and (c4-3) are also included.

(c4-1) reading the respective skin deformation coefficients blendshape _fic-1 to blendshape _fic-24 corresponding to the virtual face models Mod _fic-1 to Mod _fic-24 with preset styles;

(c4-2) normalizing the fitting coefficients alpha corresponding to the plurality of second real face models respectively;

(c4-3) Interpolate the skin deformation coefficients blendshape _fic-1 to blendshape _fic-24 respectively included in the multiple virtual face models by using the fitting coefficients alpha corresponding to the multiple second real face models to generate a target Skin deformation coefficient blendshape _Aim .

(c5) Determining target bone data; wherein, when determining the target bone data, the following (c5-1) and (c5-2) are also included.

(c5-1) Read the skeleton data; wherein, the skeleton data includes: the skeleton position data Pos _i , respectively corresponding to the virtual face models Mod _fic-1 to Mod _fic-24 with preset styles under the bones Bone _i of each level. Bone scaling data Scaling _i , and bone rotation data Trans _i .

(c5-2) Using the fitting coefficient alpha to perform interpolation processing on the skeleton data corresponding to the virtual face models Mod _fic-1 to Mod _fic-24 of the preset style respectively, to generate target bone data Bone _new , and the target bone data includes the target Bone position data Pos _new , target bone scaling data Scaling _new , and target bone rotation data Trans _new .

(c6) Generate the target virtual face model.

Based on the target bone data and the target skin deformation coefficient, replace the target bone data into the standard virtual face model Mod _Base , and use the target skin deformation coefficient blendshape _Aim to fit the skin and the bones to generate the first real person The target virtual face model corresponding to the face model.

Referring to Fig. 5, an example of specific data used in multiple processes included in the above-mentioned specific example provided by the embodiment of the present disclosure. Among them, in Figure 5 a represents the target image, 51 represents the original face A; in Figure 5 b represents a schematic diagram of a standard virtual face model with a cartoon style; After each position point in the data is adjusted, a schematic diagram of the relative positional relationship of each position point in the target skin data is obtained; d in Figure 5 represents the obtained schematic diagram corresponding to the target virtual face model generated by the original face A.

Here, it is worth noting that the above (c1) to (c6) are only a specific example of the method for reconstructing a human face, and do not limit the method for reconstructing a human face provided by the embodiments of the present disclosure.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, an apparatus for reconstructing a face corresponding to the method for reconstructing a face is also provided in the embodiment of the present disclosure, because the principle of solving the problem by the apparatus in the embodiment of the present disclosure is the same as the above-mentioned method for reconstructing a face in the embodiment of the present disclosure. Similar, therefore, the implementation of the apparatus may refer to the implementation of the method, and repeated descriptions will not be repeated.

Referring to FIG. 6 , an embodiment of the present disclosure provides an apparatus for reconstructing a human face. The apparatus includes: a first generation module 61 , a processing module 62 , a second generation module 63 , and a third generation module 64 .

The first generating module 61 is configured to generate a first real face model based on the target image.

The processing module 62 is configured to perform fitting processing on the first real face model by using a plurality of pre-generated second real face models to obtain fitting coefficients corresponding to the plurality of second real face models respectively.

The second generation module 63 is configured to, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively, Generate target bone data and target skin deformation coefficients.

The third generation module 64 is configured to generate a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient.

In an optional embodiment, the virtual face model includes a skin deformation coefficient representing the deformation of the skin data of the virtual face model relative to the standard skin data of the pre-generated standard virtual face model.

The second generation module 63 is based on the fitting coefficients corresponding to the plurality of second real face models and the virtual face models with preset styles corresponding to the plurality of second real face models, respectively, When generating the target skin deformation coefficient, it is used for: generating the target based on the fitting coefficients corresponding to the plurality of second real face models and the skin deformation coefficients respectively included in the plurality of virtual face models Skin deformation factor.

In an optional embodiment, the second generation module 63 is based on the fitting coefficients corresponding to the plurality of second real face models and the skin deformations respectively included in the plurality of virtual face models. coefficient, when generating the target skin deformation coefficient, it is used for: normalizing the fitting coefficients corresponding to the plurality of second real face models respectively; based on the normalized fitting coefficients, and The skin deformation coefficients respectively included in the virtual face models are used to obtain the target skin deformation coefficients.

In an optional embodiment, the third generation module 64 generates a target virtual face model corresponding to the first real face model based on the target skeleton data and the target skin deformation coefficient. is used to: perform position transformation processing on the standard skin data based on the target bone data and the association between the standard bone data and standard skin data in the standard virtual face model, and generate intermediate skin data ; Based on the target skin deformation coefficient, deform the intermediate skin data to obtain target skin data; Based on the target bone data and the target skin data, generate the target virtual face model .

In an optional implementation manner, the target bone data includes at least one of the following: target bone position data, target bone scaling data, and target bone rotation data.

The bone data corresponding to the virtual face model includes at least one of the following: bone rotation data, bone position data, and bone scaling data corresponding to each face bone among the multiple face bones of the virtual face.

In an optional implementation manner, the second generation module 63 is based on the fitting coefficients corresponding to the plurality of second real face models, and the corresponding corresponding to the plurality of second real face models. The virtual face model of the preset style, when generating the target skeleton data, is used for: based on the fitting coefficients corresponding to the plurality of second real face models respectively, for the skeleton positions corresponding to the plurality of virtual face models respectively The data is subjected to interpolation processing to obtain the target bone position data.

In an optional embodiment, the second generation module 63 is based on the fitting coefficients corresponding to the plurality of second real face models and the preset values corresponding to the plurality of second real face models respectively. The style of the virtual face model, when generating the target skeleton data, is used for: based on the fitting coefficients corresponding to the plurality of second real face models respectively, on the skeleton scaling data corresponding to the plurality of virtual face models respectively. Interpolation processing is performed to obtain the target bone scaling data.

In an optional implementation manner, the second generation module 63 is based on the fitting coefficients corresponding to the plurality of second real face models, and the corresponding corresponding to the plurality of second real face models. The virtual face model of the preset style, when generating target skeleton data, is used for: converting the skeleton rotation data corresponding to the plurality of virtual face models respectively into quaternions, and for the plurality of virtual face models respectively The corresponding quaternions are regularized to obtain a regularized quaternion; based on the fitting coefficients corresponding to the plurality of second real face models respectively, the regularization corresponding to the plurality of virtual face models is performed. Interpolate the quaternion to obtain the target bone rotation data.

In an optional embodiment, when generating the first real face model based on the target image, the first generation module 61 is used to: acquire a target image including the original face; 3D face reconstruction is performed on the original face to obtain the first real face model.

In an optional implementation manner, the processing module 62 pre-generates the multiple second real face models according to the following methods: acquiring multiple reference images including reference faces; For each reference image, three-dimensional face reconstruction is performed on the reference face included in the reference image to obtain a second real face model corresponding to the reference image.

In an optional embodiment, the device for reconstructing a human face further includes an acquisition module 65, which is used for acquiring each second real human face model in the plurality of second real human face models in the following manner: A virtual face model with a preset style corresponding to the second real face model: generating an intermediate virtual face model with a preset style corresponding to the second real face model; The multiple sets of preset skin deformation coefficients are generated, and the skin deformation coefficients of the virtual face model corresponding to the second real face model relative to the standard virtual face model are generated; using the skin deformation coefficients, The intermediate skin data in the intermediate virtual face model is adjusted; based on the adjusted intermediate skin data and the intermediate skeleton data of the intermediate virtual face model, a virtual person of the second real face model is generated face model.

In an optional implementation manner, the processing module 62 performs fitting processing on the first real face model by using a plurality of pre-generated second real face models to obtain a plurality of second real face models respectively. When the corresponding fitting coefficient is used, it is used to: perform least squares processing on the plurality of second real face models and the first real face models to obtain the corresponding corresponding fitting coefficients.

For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.

As shown in FIG. 7 , an embodiment of the present disclosure further provides a computer device, including: a processor 71 and a memory 72 .

The memory 72 stores machine-readable instructions executable by the processor 71, and the processor 71 is configured to execute the machine-readable instructions stored in the memory 72. When the machine-readable instructions are executed by the processor 71, the processor 71 executes the following: Steps: generating a first real face model based on the target image; using a plurality of pre-generated second real face models to perform fitting processing on the first real face model, and obtaining a plurality of second real face models corresponding to based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the virtual face models with preset styles corresponding to the plurality of second real face models respectively, generate target skeleton data and a target skin deformation coefficient; based on the target bone data and the target skin deformation coefficient, a target virtual face model corresponding to the first real face model is generated.

The above-mentioned memory 72 includes a memory 721 and an external memory 722; the memory 721 here is also called an internal memory, which is used to temporarily store the operation data in the processor 71 and the data exchanged with the external memory 722 such as the hard disk. The external memory 722 performs data exchange.

For the specific execution process of the above instruction, reference may be made to the method for reconstructing a human face described in the embodiments of the present disclosure, and details are not described herein again.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the method for reconstructing a face described in the foregoing method embodiments is executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the methods for reconstructing a face described in the foregoing method embodiments. For details, please refer to the foregoing methods. The embodiments are not repeated here.

Wherein, the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on this understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that make contributions to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, but not to limit them. The protection scope of the present disclosure is not limited to this, although the aforementioned The embodiments describe the present disclosure in detail, and those skilled in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (15)

1. A method of reconstructing a human face, comprising:

   generating a first real face model based on the target image;

   Perform fitting processing on the first real face model by using a plurality of pre-generated second real face models to obtain fitting coefficients corresponding to the plurality of second real face models respectively;

   Based on the fitting coefficients corresponding to the plurality of second real face models, and the virtual face models with preset styles corresponding to the plurality of second real face models, respectively, target skeleton data and target skin are generated deformation coefficient;

   Based on the target bone data and the target skin deformation coefficient, a target virtual face model corresponding to the first real face model is generated.
2. The method for reconstructing a human face according to claim 1, characterized in that, based on the fitting coefficients corresponding to the plurality of second real face models respectively, and the corresponding corresponding to the plurality of second real face models having The virtual face model of the preset style generates the target skin deformation coefficient, including:

   The target skin deformation coefficient is generated based on the fitting coefficients corresponding to the plurality of second real face models and the skin deformation coefficients respectively included in the plurality of virtual face models.
3. The method for reconstructing a human face according to claim 2, wherein the fitting coefficients corresponding to the plurality of second real face models and the masks respectively included in the plurality of virtual face models skin deformation coefficient, generating the target skin deformation coefficient, including:

   normalizing the fitting coefficients corresponding to the plurality of second real face models respectively;

   The target skin deformation coefficient is obtained based on the normalized fitting coefficient and the skin deformation coefficient respectively included in the plurality of virtual face models.
4. The method for reconstructing a human face according to any one of claims 1 to 3, characterized in that, based on the target skeleton data and the target skin deformation coefficient, generating a model corresponding to the first real face The target virtual face model, including:

   Based on the target skeleton data and the association between the standard skeleton data in the standard virtual face model and the standard skin data, perform position transformation processing on the standard skin data to generate intermediate skin data;

   Deformation processing is performed on the intermediate skin data based on the target skin deformation coefficient to obtain target skin data;

   The target virtual face model is constructed based on the target bone data and the target skin data.
5. The method for reconstructing a human face according to any one of claims 1 to 4, wherein,

   The target bone data includes at least one of the following: target bone position data, target bone scaling data, and target bone rotation data;

   The skeleton data corresponding to the plurality of virtual face models respectively include at least one of the following: skeleton rotation data, skeleton position data, and skeleton scaling data corresponding to each face skeleton in the plurality of face skeletons of the virtual face .
6. The method for reconstructing a face according to claim 5, wherein the fitting coefficients corresponding to the plurality of second real face models and the corresponding fitting coefficients of the plurality of second real face models respectively The virtual face model with preset style generates target skeleton data, including:

   Based on the fitting coefficients corresponding to the plurality of second real face models respectively, interpolation processing is performed on the bone position data corresponding to the plurality of virtual face models, to obtain the target bone position data.
7. The method for reconstructing a face according to claim 5 or 6, wherein the fitting coefficients corresponding to the plurality of second real face models respectively and the plurality of second real face models The corresponding virtual face models with preset styles are generated to generate target skeleton data, including:

   Based on the respective fitting coefficients corresponding to the multiple second real face models, interpolation processing is performed on the skeleton scaling data corresponding to the multiple virtual face models respectively, to obtain the target skeleton scaling data.
8. The method for reconstructing a face according to any one of claims 5 to 7, wherein the fitting coefficients corresponding to the multiple second real face models and the multiple second real face models The face models respectively correspond to virtual face models with preset styles, and generate target skeleton data, including:

   Converting the skeleton rotation data corresponding to the plurality of virtual face models respectively into quaternions,

   performing regularization processing on the quaternions corresponding to the plurality of virtual face models respectively to obtain the regularized quaternions;

   Based on the fitting coefficients corresponding to the plurality of second real face models respectively, interpolation processing is performed on the regularization quaternions corresponding to the plurality of virtual face models respectively, to obtain the target bone rotation data.
9. The method for reconstructing a human face according to any one of claims 1 to 8, wherein the generating the first real face model based on the target image comprises:

   Get the target image including the original face;

   Performing three-dimensional face reconstruction on the original face included in the target image to obtain the first real face model.
10. The method for reconstructing a human face according to any one of claims 1 to 9, wherein the plurality of second real face models are pre-generated according to the following manner:

    Obtain multiple reference images including reference faces;

    For each reference image in the plurality of reference images, three-dimensional face reconstruction is performed on the reference face included in the reference image to obtain the second real face model corresponding to the reference image.
11. The method for reconstructing a human face according to any one of claims 1 to 10, further comprising, for each second real face model in the plurality of second real face models, adopting the following manner Obtain a virtual face model with a preset style corresponding to the second real face model:

    generating an intermediate virtual face model with a preset style corresponding to the second real face model;

    generating skin deformation coefficients of the virtual face model corresponding to the second real face model relative to the standard virtual face model based on multiple sets of preset skin deformation coefficients relative to the standard virtual face model;

    adjusting the intermediate skin data in the intermediate virtual face model using the skin deformation coefficient; and

    Based on the adjusted intermediate skin data and the intermediate skeleton data of the intermediate virtual face model, a virtual face model of the second real face model is generated.
12. The method for reconstructing a face according to any one of claims 1 to 11, wherein the first real face model is fitted by using a plurality of pre-generated second real face models, The fitting coefficients corresponding to multiple second real face models are obtained, including:

    Least squares processing is performed on the plurality of second real face models and the first real face models to obtain fitting coefficients corresponding to the plurality of second real face models respectively.
13. A device for reconstructing a human face, comprising:

    a first generation module for generating a first real face model based on the target image;

    a processing module, configured to perform fitting processing on the first real face model by using a plurality of pre-generated second real face models to obtain fitting coefficients corresponding to the plurality of second real face models respectively;

    The second generation module is configured to generate a virtual face model with a preset style corresponding to the plurality of second real face models respectively based on the fitting coefficients corresponding to the plurality of second real face models and the corresponding virtual face models with the preset style respectively. Target bone data and target skin deformation coefficient;

    The third generation module is configured to generate a target virtual face model corresponding to the first real face model based on the target bone data and the target skin deformation coefficient.
14. A computer device comprising a processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor for executing the machine-readable instructions stored in the memory, the machine-readable instructions When the instructions are executed by the processor, the processor executes the method for reconstructing a human face according to any one of claims 1 to 12.
15. A computer-readable storage medium on which a computer program is stored, and when the computer program is run by a computer device, the computer device executes the method for reconstructing a human face according to any one of claims 1 to 12.

Images