TWI780919B - Method and apparatus for processing face image, electronic device and storage medium - Google Patents

Abstract

The present disclosure provides a method and apparatus for processing a face image, electronic device and storage medium, wherein the method includes: obtaining initial dense point cloud data of a target face, and generating an initial virtual face image of the target face based on the initial dense point cloud data; determining a deformation coefficient of the initial dense point cloud data relative to standard dense point cloud data corresponding to a standard virtual face image; adjusting the deformation coefficient in response to an adjustment operation for the initial virtual face image to obtain a target deformation coefficient; and generating a target virtual face image corresponding to the target face based on the target deformation coefficient and the standard dense point cloud data.

Description

Face image processing method, device, electronic equipment and storage medium

The present disclosure relates to the technical field of face reconstruction, in particular, to a face image processing method, device, electronic equipment and storage media.

In the 3D world, the shape of an object can be represented by a 3D point cloud. For example, the face shape can be represented by a dense point cloud of a face. It consists of tens of thousands of points. When it is necessary to adjust the corresponding face shape, the points need to be adjusted one by one. The process is cumbersome and the efficiency is low.

Embodiments of the present disclosure provide at least one face image processing solution.

In a first aspect, an embodiment of the present disclosure provides a method for processing a face image, including: acquiring initial dense point cloud data of a target face, and generating an initial virtual person of the target face based on the initial dense point cloud data Face image; determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual human face image; adjust the deformation coefficient in response to the adjustment operation for the initial virtual human face image , to obtain a target deformation coefficient; based on the target deformation coefficient and the standard dense point cloud data, generate a target virtual face image corresponding to the target face.

In the embodiment of the present disclosure, it is proposed to determine the deformation coefficient used to adjust the virtual face image of the target face through the dense point cloud data, so that the corresponding relationship between the dense point cloud data and the deformation coefficient can be established, so that it can be directly based on The deformation coefficient adjusts the virtual face image. Compared with adjusting the points in the dense point cloud data one by one, it can improve the adjustment efficiency and quickly generate the adjusted target virtual face image.

On the other hand, considering that the deformation coefficient here is determined according to the dense point cloud data, in the process of adjusting the initial virtual face image based on the deformation coefficient, the dense point cloud in the dense point cloud data can be directly based on the deformation coefficient. Adjustment, which can be directly accurate to the adjustment of each point in the dense point cloud that constitutes the virtual face image, and can improve the adjustment accuracy on the basis of improving the adjustment efficiency.

In a possible implementation manner, the deformation coefficient includes at least one of at least one bone coefficient and at least one mixed deformation coefficient; wherein each bone coefficient is used to construct the first dense point cloud associated with the bone coefficient Adjust the initial pose of the skeleton of the bone; each blend shape coefficient is used to adjust the initial position corresponding to the second dense point cloud associated with the blend shape coefficient.

In the embodiments of the present disclosure, the positions of different types of dense point clouds can be adjusted respectively based on the bone coefficient and/or the mixed deformation coefficient in the deformation coefficients, so as to achieve precise adjustment of the dense point clouds.

In a possible implementation manner, the determination of the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image includes: modifying the standard dense point cloud data based on the current deformation coefficient Adjust to obtain the adjusted dense point cloud data, the initial current deformation coefficient is preset; based on the adjusted dense point cloud data and the initial dense point cloud data, determine the first loss value; based on the The first loss value and the preset constraint range of the deformation coefficient adjust the current deformation coefficient; based on the adjusted current deformation coefficient, return to the step of adjusting the standard dense point cloud data until the When the adjustment operation of the current deformation coefficient meets the first adjustment cut-off condition, the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data is obtained based on the current deformation coefficient.

In the embodiment of the present disclosure, in the process of determining the deformation coefficient, it is determined by adjusting multiple points in the standard dense point cloud data, so the obtained deformation coefficient can represent the initial dense point cloud of the target face. The change amount of the standard dense point cloud, so that in the process of adjusting the initial virtual face image of the target face, the associated points in the dense point cloud data can be adjusted based on the deformation coefficient, thereby improving the adjustment accuracy.

On the other hand, in the process of determining the deformation coefficient, after adjusting all the dense point clouds, and then based on the adjusted dense point cloud data and the loss value determined by the initial dense point cloud data of the target face, the current deformation The optimization of the deformation coefficient fully considers the correlation between the deformation coefficient and the overall dense point cloud to improve the optimization efficiency; in addition, during the adjustment process, the adjustment constraints are adjusted through the preset deformation coefficient constraint range, which can effectively avoid the distortion of the deformation coefficient. Get deformation coefficients that cannot represent normal target faces.

In a possible implementation manner, the adjusting the deformation coefficient in response to the adjustment operation on the initial virtual human face image to obtain the target deformation coefficient includes: responding to the adjustment operation on the initial virtual human face image Operation, determine the target adjustment position for the initial virtual human face image, and the adjustment range for the target adjustment position; according to the adjustment range, adjust the deformation coefficient associated with the target adjustment position to obtain the Target deformation coefficient.

In the embodiment of the present disclosure, the target deformation coefficient can be determined according to the adjustment operation, so that the adjusted target virtual human face image can be determined later based on the target deformation coefficient. This method can adjust the deformation coefficient personalizedly based on user needs.

In a possible implementation manner, the generating the target virtual human face image corresponding to the target face based on the target deformation coefficient and the standard dense point cloud data includes: based on the target deformation coefficient, The standard dense point cloud data is adjusted to obtain the target dense point cloud data; based on the target dense point cloud data, the target virtual human face image is generated.

In the embodiment of the present disclosure, after the target deformation coefficient is determined, the standard dense point cloud data can be directly adjusted according to the target deformation coefficient, and the target dense point cloud data can be determined, so that the target face can be quickly obtained according to the target dense point cloud data Target virtual human face image.

In a possible implementation manner, the generating the target virtual human face image based on the target dense point cloud data includes: determining a virtual human face model corresponding to the target dense point cloud data; The face attribute feature and the virtual human face model are used to generate the target virtual human face image.

In the embodiment of the present disclosure, when adjusting the initial virtual human face image, it may also be adjusted in combination with the facial attribute characteristics selected by the user, so that the target virtual human face image is more suitable for the actual needs of the user.

In a possible implementation manner, the acquiring the initial dense point cloud data of the target face, and generating the initial virtual face image of the target face based on the initial dense point cloud data includes: acquiring the target face The first human face image corresponding to the face, and the dense point cloud data respectively corresponding to the multiple second human face images of the preset style; based on the first human face image and the multiple second human faces of the preset style The dense point cloud data corresponding to the images respectively, determine the initial dense point cloud data of the target face in the preset style; based on the initial dense point cloud data of the target face in the preset style, generate The initial virtual human face image of the target human face in the preset style.

In the embodiment of the present disclosure, the dense point cloud data of the first face image in the preset style can be determined according to the dense point cloud data corresponding to the pre-stored multiple base images in the preset style, so that it can be quickly displayed Generate a virtual face image of the target face in the preset style.

In a possible implementation manner, based on the dense point cloud data respectively corresponding to the first face image and the plurality of second face images of the preset style, it is determined that the target face is in the preset The initial dense point cloud data under the preset style includes: extracting the face parameter values of the first face image, and the face parameter values corresponding to a plurality of second face images of the preset style; wherein, The face parameter value includes a parameter value representing the shape of the face and a parameter value representing the expression of the face; a face parameter value based on the first face image and a plurality of second faces of the preset style The face parameter values and dense point cloud data corresponding to the images respectively determine the initial dense point cloud data of the target face in the preset style.

In the embodiment of the present disclosure, it is proposed that in the process of determining the dense point cloud data of the first face image in the preset style, the faces of the first face image and multiple second face images of the preset style can be combined Because the number of parameter values used to represent the face through the face parameter value is small, the dense point cloud data of the target face in the preset style can be determined more quickly.

In one embodiment, the face parameter values based on the first face image, and the face parameter values and dense point cloud data respectively corresponding to the plurality of second face images of the preset style are determined. The initial dense point cloud data of the target face in the preset style includes: face parameter values based on the first face image, and multiple second face images of the preset style corresponding to The face parameter value is used to determine the linear fitting coefficient between the first face image and multiple second face images of the preset style; according to the multiple second face images of the preset style, respectively Corresponding to the dense point cloud data and the linear fitting coefficient, determine the initial dense point cloud data of the target face in the preset style.

In the embodiment of the present disclosure, it can be proposed that the linear fitting coefficient representing the correlation between the first face image and multiple second face images can be quickly obtained through a small number of face parameter values, and further can be based on the linear fitting coefficient The combination coefficient adjusts the dense point cloud data of multiple second face images in the preset style, and can quickly obtain the dense point cloud data of the target face in the preset style.

In a possible implementation manner, the first human face image is determined based on the face parameter values of the first face image and the face parameter values corresponding to the plurality of second face images of the preset style. The linear fitting coefficient between a human face image and multiple second human face images of the preset style includes: obtaining the current linear fitting coefficient, the initial current linear fitting coefficient is preset; based on the The current linear fitting coefficient and the face parameter values corresponding to the plurality of second face images of the preset style respectively, predict the current face parameter value of the first face image; based on the predicted current face parameters value and the face parameter value of the first face image to determine a second loss value; based on the second loss value and the preset constraint range corresponding to the linear fitting coefficient, adjust the current linear fitting coefficient; based on the adjusted current linear fitting coefficient, return to the step of predicting the current face parameter value, until the adjustment operation to the current linear fitting coefficient meets the second adjustment cut-off condition, based on the current linear The fitting coefficient obtains a linear fitting coefficient between the first human face image and multiple second human face images of a preset style.

In the embodiment of the present disclosure, in the process of adjusting the linear fitting coefficients between the first face image and multiple second face images of the preset style, the linear fitting coefficient can be adjusted by the second loss value and/or the number of adjustments. The fitting coefficient can be adjusted multiple times, so that the accuracy of the linear fitting coefficient can be improved; on the other hand, in the adjustment process, the constraint range of the preset linear fitting coefficient can be adjusted to obtain the linear fitting coefficient, which can be more reasonable Determine the dense point cloud data corresponding to the target face.

In a possible implementation manner, the dense point cloud data includes the coordinate values of each point in the dense point cloud; the dense point cloud data and the corresponding dense point cloud data of the multiple second face images according to the preset style The linear fitting coefficient, determining the initial dense point cloud data of the target face in the preset style, including: the dense point cloud corresponding to a plurality of second face images based on the preset style The coordinate values of each point in the center, determine the coordinate value of the corresponding point of the average dense point cloud data; the coordinate values of each point in the dense point cloud data corresponding to the multiple second face images based on the preset style, and The coordinate values of the corresponding points in the average dense point cloud data determine the coordinate difference values corresponding to the plurality of second human face images in the preset style; the plurality of second human face images in the preset style correspond to The coordinate difference value corresponding to the first human face image and the linear fitting coefficient, determine the coordinate difference value corresponding to the first human face image; based on the coordinate difference value corresponding to the first human face image and the average dense point cloud data The coordinate values of the corresponding points are used to determine the initial dense point cloud data of the target face in the preset style.

In the embodiment of the present disclosure, when there are few second face images, the dense point cloud data of the diverse second face images can accurately represent the dense points of different target faces in the preset style cloud data.

In a possible implementation manner, the face parameter value is extracted by a pre-trained neural network, and the neural network is trained based on sample images with face parameter values marked in advance.

In the embodiment of the present disclosure, it is proposed to extract the face parameter values of the face image by using a pre-trained neural network, which can improve the extraction accuracy and extraction efficiency of the face parameter values.

In a possible implementation manner, the neural network is pre-trained in the following manner: a sample image set is obtained, and the sample image set includes a plurality of sample images and the labeled face parameter values corresponding to each sample image; Multiple sample images are input into the neural network to obtain the predicted face parameter values corresponding to each sample image; based on the predicted face parameter values and marked face parameter values corresponding to each sample image, the neural network network The parameter values are adjusted to obtain the trained neural network.

In the embodiment of the present disclosure, during the training process of the neural network used to extract the face parameter value, it is proposed to continuously adjust the network parameter value of the neural network by marking the face parameter value of each sample image, Thus, a neural network with higher accuracy can be obtained.

In a second aspect, an embodiment of the present disclosure provides a face image processing device, including: an acquisition module, configured to acquire the original dense point cloud data of a target face, and generate the target object based on the initial dense point cloud data. The initial virtual face image of the face; the determination module is used to determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image; the adjustment module is used to respond to the The adjustment operation of the initial virtual human face image is to adjust the deformation coefficient to obtain the target deformation coefficient; the generation module is used to generate the target person based on the target deformation coefficient and the standard dense point cloud data. The target virtual face image corresponding to the face.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor, a memory, and a bus bar. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the The processor communicates with the storage through a bus, and when the machine-readable instructions are executed by the processor, the steps of the processing method described in the first aspect are executed.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the processing method described in the first aspect are executed.

In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this article only describes an association relationship, which means that there may be three relationships, for example, A and/or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one or any combination of at least two of the plurality, for example, including at least one of A, B, and C, may mean including the composition consisting of A, B, and C Any one or more elements selected in the collection.

In the field of 3D modeling, a face can be represented by a dense point cloud collected for the face. The dense point cloud representing the face generally contains tens of thousands of points. When adjusting the shape of the image, it is necessary to adjust the positions of thousands of points one by one, which is a cumbersome process and low efficiency.

Based on the above research, the present disclosure provides a face image processing method. After obtaining the original dense point cloud data of the target face, it can be determined that the initial dense point cloud data of the target face corresponds to the standard face image. The deformation coefficient of the standard dense point cloud data, in this way, establishes the corresponding relationship between the dense point cloud data and the deformation coefficient, so that when the adjustment operation for the initial virtual face image is detected, it can directly target the deformation Coefficients are adjusted to complete the adjustment of the initial virtual face image. This method does not need to adjust the points in the dense point cloud data one by one, which improves the adjustment efficiency. In addition, this scheme is based on the deformation coefficient determined by the dense point cloud data. In the process of adjusting the initial virtual human face image, the adjustment accuracy is higher.

In order to facilitate the understanding of this embodiment, a face image processing method disclosed in the embodiments of the present disclosure is first introduced in detail. The processing method provided by the embodiments of the present disclosure is generally executed by a computer device with a certain computing power. The computer equipment includes, for example: terminal equipment or servers or other processing equipment, and the terminal equipment may be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, handheld equipment, computing equipment, wearable equipment, and the like. In some possible implementation manners, the processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.

Referring to FIG. 1 , an embodiment of the present disclosure provides a method for processing a human face image, and the processing method includes the following steps S101 to S104.

S101. Acquire original dense point cloud data of a target face, and generate an initial virtual face image of the target face based on the original dense point cloud data.

Exemplarily, the dense point cloud data can represent a 3D model of a human face. Specifically, the dense point cloud data can include the coordinate values of multiple points on the surface of the human face in a pre-built 3D coordinate system, and the multiple points are connected to form The 3D network (3Dmesh) and the coordinate values of multiple points can be used to represent the 3D model of the face, as shown in Figure 2, which represents the schematic diagram of the 3D model of the face represented by different dense point cloud data, dense point cloud The more points contained in , the finer the dense point cloud data will be when representing the 3D model of the face.

Exemplarily, the initial virtual face image can be a three-dimensional face image or a two-dimensional face image, which is related to a specific application scenario. Correspondingly, when the initial virtual face image is a three-dimensional face image, the following mentioned The face image is also a three-dimensional face image. When the initial virtual face image is a two-dimensional face image, the face image mentioned later is also a two-dimensional face image. The embodiment of the present disclosure will use the virtual face image A three-dimensional face image is taken as an example for illustration.

Exemplarily, when the acquired initial dense point cloud data of the target face is the dense point cloud data corresponding to the target face in the preset style, for example, when the dense point cloud data corresponding to the target face in the classical style, based on the The initial virtual face image of the target face displayed by the initial dense point cloud data is also a face image in the classical style. How to obtain the corresponding dense point cloud data of the target face in the preset style will be explained later.

S102. Determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image.

Exemplarily, the standard dense point cloud data corresponding to the standard virtual human face image here may be dense point cloud data corresponding to a preset virtual human face image, the preset virtual human face image has a preset face shape and facial features, Further, based on the standard virtual face image, the deformation coefficient of the initial dense point cloud data of the target face compared with the standard dense point cloud data can be determined.

Exemplarily, the deformation coefficient is associated with the dense point cloud data, and can represent the deformation amount of the dense point cloud data compared with the standard dense point cloud data, so that the deformation coefficient corresponding to the target face can represent the difference between the target face and the standard face. The amount of deformation, such as increased nose bridge, enlarged eyes, raised mouth corners, smaller cheeks, etc.

Specifically, the deformation coefficient includes at least one bone coefficient and/or at least one hybrid deformation coefficient;

Among them, each bone coefficient is used to adjust the initial pose of the bone composed of the first dense point cloud associated with the bone coefficient; each mixed deformation coefficient is used to adjust the second dense point cloud associated with the mixed deformation coefficient Adjust the corresponding initial position.

Exemplarily, the bone coefficient can contain multiple, which can be used to adjust the bones of the human face. In the specific adjustment, the bones can be adjusted in a pre-built three-dimensional coordinate system (it can be one of the points of the human face as the coordinate source) Points to construct the world coordinate system, which will be introduced later), to adjust the initial pose in), taking one of the bone coefficients corresponding to the nose bridge of the face as an example, by adjusting the bone coefficient, the first dense The initial pose of the point cloud is adjusted to complete the adjustment of the initial pose of the nose bridge of the face, for example, the nose bridge of the face is adjusted to be more straight.

The mixed deformation coefficient can also contain multiple, which is used to adjust the initial position of the associated second dense point cloud in the pre-built three-dimensional coordinate system, which can achieve the goal of adjusting the size and shape of the facial contour and facial features , taking one of the blending deformation coefficients corresponding to the face contour as an example, by adjusting the blending deformation coefficient, the initial position of the second dense point cloud that constitutes the face contour can be adjusted, thereby completing the size and/or adjustment of the face contour Adjust the shape, such as reducing the size of a round face, or adjusting it into an oval face.

Exemplarily, in response to different adjustment requirements, there may be at least a partial point overlap between the first dense point cloud associated with the bone coefficients and the second dense point cloud associated with the hybrid deformation coefficients, for example, to Take a bone coefficient for pose adjustment as an example. By adjusting the first dense point cloud associated with the bone coefficient, the pose of the face and nose can be adjusted. When the size of the face and nose needs to be adjusted, The second dense point cloud associated with the mixed deformation coefficient corresponding to the face and nose can be the same as the first dense point cloud associated with the bone coefficient used to adjust the pose of the face and nose; of course, the first dense point cloud associated with the bone coefficient The second dense point cloud associated with the point cloud and the mixed deformation coefficient can also be a different dense point cloud, for example, the first dense point cloud associated with the bone coefficient used to condition the pose of the face and nose, and the first dense point cloud used to The second dense point cloud associated with the blended deformation coefficients adjusted for the cheek size.

For example, in order to represent the deformation coefficient of the initial dense point cloud data of the target face relative to the standard dense point cloud data, one of the points in the dense point cloud contained in the target face can be used as the origin of the coordinate system in advance, and three Two mutually perpendicular directions are used as the three coordinate axes of the coordinate system to construct the world coordinate system. In this world coordinate system, the deformation coefficient and deformation coefficient of the initial dense point cloud data of the target face relative to the standard dense point cloud data can be determined. The specific determination process may be determined according to a machine learning algorithm, which will be described in detail later.

In the embodiment of the present disclosure, it is proposed that the deformation coefficient includes the bone coefficient used to adjust the initial pose of the skeleton, and the mixed deformation coefficient used to adjust the initial position of the dense point cloud, so that the target face can be adjusted based on the deformation coefficient. Make full adjustments.

S103, in response to the adjustment operation on the initial virtual human face image, adjust the deformation coefficient to obtain a target deformation coefficient.

Exemplarily, when showing the initial virtual human face image of the target face, an operation button for adjusting the initial virtual human face image may also be displayed, allowing the user to shape the displayed initial virtual human face image through the operation button. During the adjustment process, in order to facilitate the user to adjust the initial virtual face image intuitively, a variety of corresponding relationships between the position to be adjusted and the deformation coefficient can be established in advance, such as establishing the mouth, eyes, nose, eyebrows, face shape, etc. Waiting for the corresponding relationship between the adjusted position and the deformation coefficient, so that the user can directly adjust the position to be adjusted based on the displayed initial virtual face image, so as to achieve the purpose of adjusting the deformation coefficient.

S104. Based on the target deformation coefficient and the standard dense point cloud data, generate a target virtual face image corresponding to the target face.

After the target deformation coefficient is obtained, the standard dense point cloud data can be further adjusted based on the target deformation coefficient to obtain the target dense point cloud data corresponding to the target face, and then according to the target dense point cloud data, the target face corresponding to Target virtual human face image.

In the embodiment of the present disclosure, it is proposed to determine the deformation coefficient used to adjust the virtual face image of the target face through the dense point cloud data, so that the corresponding relationship between the dense point cloud data and the deformation coefficient can be established, so that it can be directly based on The deformation coefficient adjusts the virtual face image, which can improve the adjustment efficiency compared to adjusting the points in the dense point cloud data one by one.

On the other hand, considering that the deformation coefficient here is determined according to the dense point cloud data, in the process of adjusting the initial virtual face image based on the deformation coefficient, the points in the dense point cloud data can be adjusted directly based on the deformation coefficient. In this way, the adjustment of each point constituting the virtual human face image can be directly and accurately adjusted, and the adjustment accuracy can be improved on the basis of improving the adjustment efficiency.

The above steps S101 to S104 will be described in detail below in conjunction with specific embodiments.

Regarding the above S101, when acquiring the initial dense point cloud data of the target face and displaying the initial virtual face image of the target face based on the initial dense point cloud data, as shown in FIG. 3 , the following steps S201 to S203 may be included.

S201. Acquire a first face image corresponding to a target face, and dense point cloud data respectively corresponding to a plurality of second face images in a preset style.

Exemplarily, the first face image corresponding to the target face may be a color face image of the target face collected by an image collection device, or a grayscale face image of the target face, which is not specifically limited herein.

Exemplarily, the plurality of second face images are pre-selected images with some characteristics, through which different first face images can be characterized, for example, n second face images are selected, and for each For the first face images, the first face images can be characterized by the n second face images and the linear fitting coefficients. Exemplarily, in order to enable multiple second face images to fit most of the first face images, an image of a face with some prominent features compared to an average face may be selected as the second face image, for example , select a face image with a smaller face size than the average face as the second face image, or select a face image with a larger mouth size than the average face as the second face image face images, or select a face image of a face with a larger eye size than an average face as a second face image, by selecting a face image of a face with specific characteristics as the second face image, The first human face image can be represented by adjusting the linear fitting coefficient.

Exemplarily, the dense point cloud data corresponding to each second face image in multiple styles can be acquired and saved in advance, for example, the dense point cloud data corresponding to the classical style, and the dense point cloud data corresponding to the modern style , the dense point cloud data corresponding to the western style and the dense point cloud data corresponding to the Chinese style facilitate subsequent determination of the virtual face models corresponding to the first face image in different styles.

Exemplarily, for each second face image, the dense point cloud data corresponding to the second face image and the face parameter value of the second face image can be extracted, for example, the second face image can be extracted 3D Morphable Face Model (3DMM) parameter values of each face image, and then adjust the coordinate values of multiple points in the dense point cloud data according to the face parameter values to obtain each second face image at The dense point cloud data corresponding to various styles, for example, the dense point cloud data corresponding to each second face image in the classical style, the dense point cloud data corresponding to the cartoon style, and then for each second face image Face images are stored in dense point cloud data in different styles.

Exemplarily, the face parameter value includes a parameter value representing a face shape, and a parameter value representing a facial expression, for example, a face parameter value may include a K-dimensional parameter value representing a face shape, including M Dimensions are used to represent the parameter value of facial expression, wherein, the parameter value of K dimension is used to represent the shape of the face together reflects the facial shape of the second human face image, and the parameter value of M dimension is used to represent facial expression The parameter values together reflect the facial expression of the second human face image.

For example, the value range of the dimension of K is generally between 150 and 400, the smaller the dimension of K, the simpler the facial shape that can be represented, the larger the dimension of K, the more complex the facial shape that can be represented; The value range is generally between 10 and 40. The less the dimension of M, the simpler the facial expression that can be represented, and the more the dimension of M, the more complex the facial expression that can be represented. It can be seen that the embodiment of the present disclosure proposes that the facial expression can be represented by A face is represented by face parameter values with a relatively small number range, so as to facilitate subsequent determination of the initial virtual face model corresponding to the target face.

Exemplarily, in combination with the meaning of the face parameter value, the above-mentioned coordinate values of multiple points in the dense point cloud are adjusted according to the face parameter value to obtain the corresponding values of each second face image in multiple styles. Dense point cloud data can be understood as the point in the dense point cloud in the pre-established 3D The coordinate values in the coordinate system are adjusted to obtain the dense point cloud data corresponding to the second face image in various styles.

S202. Based on the dense point cloud data respectively corresponding to the first human face image and multiple second human face images of the preset style, determine the dense point cloud data of the target human face in the preset style.

Exemplarily, by finding the correlation between the first human face image and multiple second human face images, for example, by linear fitting, it is possible to determine the relationship between the multiple second human face images and the first human face image The linear fitting coefficient between them, and then according to the linear fitting coefficient and the dense point cloud data corresponding to multiple second face images of the preset style, the dense point of the target face under the preset style can be determined cloud data.

S203. Based on the dense point cloud data of the target face in the preset style, generate and display an initial virtual face image of the target face in the preset style.

After obtaining the dense point cloud data of the target face in the preset style, the initial virtual face image of the target face in the preset style can be generated and displayed according to the dense point cloud data corresponding to the target face, for example The initial virtual face image of the target face can be displayed based on the style set by default or the style set by the user.

In the embodiment of the present disclosure, the dense point cloud data of the first face image in the preset style can be determined according to the dense point cloud data corresponding to each base image in the pre-stored base image database in the preset style , so as to quickly display the virtual face image of the target face in the preset style.

For the above S202, the dense point cloud data includes the coordinate values of multiple points in the dense point cloud, and the target person is determined based on the dense point cloud data corresponding to the first face image and multiple second face images of the preset style. When the dense point cloud data of the face is in the preset style, as shown in FIG. 4 , the following steps S301 to S302 may be included.

S301, extracting the face parameter values of the first face image, and the face parameter values corresponding to the plurality of second face images of the preset style; wherein, the face parameter values include the parameter values and representations representing the shape of the face The parameter value of facial expression.

Exemplarily, here, the face parameter values of the first face image and the face parameter values corresponding to multiple second face images of the preset style can be respectively extracted through a pre-trained neural network, for example, the The first face image and each second face image are respectively input into the pre-trained neural network to obtain respective corresponding face parameter values.

S302, based on the face parameter values of the first face image, and the face parameter values and dense point cloud data respectively corresponding to multiple second face images of the preset style, determine the denseness of the target face in the preset style point cloud data.

Considering that the face parameter values and the dense point cloud data have a corresponding relationship when representing the same face, it is possible to use the face parameter values corresponding to the first face image and multiple second face images of the preset style, Determining the correlation between the first human face image and the multiple second human face images of the preset style, and then according to the correlation and the dense point cloud data respectively corresponding to the multiple second human face images of the preset style, Determine the dense point cloud data of the target face in the preset style.

In the embodiment of the present disclosure, it is proposed that in the process of determining the dense point cloud data of the target face image in the preset style, it can be determined by combining the face parameter values of the first face image and multiple second face images, Because the number of parameter values used to represent a human face is relatively small, the dense point cloud data of the target human face in a preset style can be determined more quickly.

Exemplarily, the aforementioned face parameter values are extracted by a pre-trained neural network, and the neural network is trained based on sample images with pre-marked face parameter values.

In the embodiment of the present disclosure, it is proposed to extract the face parameter value of the face image through the pre-trained neural network, which can improve the extraction efficiency of the face parameter value.

Specifically, the neural network may be pre-trained in the following manner, as shown in FIG. 5 , which may include the following steps S401 to S403.

S401. Obtain a sample image set, where the sample image set includes multiple sample images and labeled face parameter values corresponding to each sample image;

S402, inputting multiple sample images into the neural network to obtain predicted face parameter values corresponding to each sample image;

S403, based on the predicted face parameter value and the marked face parameter value corresponding to each sample image, adjust the network parameter value of the neural network to obtain a trained neural network.

Exemplarily, a large number of face images and the marked face parameter values corresponding to each face image can be collected as the sample image set here, and each sample image can be input into the neural network, and the output of the neural network can be obtained. The predicted face parameter value corresponding to the sample image can further determine the third loss value corresponding to the neural network based on the marked face parameter value and the predicted face parameter value corresponding to the sample image, and then according to the third loss value. Network parameter values are adjusted until the number of adjustments reaches a preset number of times and/or the third loss value is less than a third preset threshold, and a trained neural network is obtained.

In the embodiment of the present disclosure, during the training process of the neural network used to extract the face parameter value, it is proposed to continuously adjust the network parameter value of the neural network by marking the face parameter value of each sample image, Thus, a neural network with higher accuracy can be obtained.

Specifically, for the above S302, based on the face parameter values of the first face image, and the face parameter values and dense point cloud data respectively corresponding to multiple second face images of the preset style, it is determined that the target face is For dense point cloud data in the preset style, as shown in Figure 6, the following S3021~S3032 can be included:

S3021, based on the face parameter values of the first face image and the face parameter values respectively corresponding to the multiple second face images of the preset style, determine the first face image and the multiple second face images of the preset style Linear fit coefficient between face images;

S3022. Determine the dense point cloud data of the target face in the preset style according to the dense point cloud data and linear fitting coefficients respectively corresponding to the multiple second face images of the preset style.

（1）。 Exemplarily, taking the face parameter value as 3DMM parameter value as an example, considering that the 3DMM parameter value of the first face image can represent the face shape and expression corresponding to the first face image, each second face The 3DMM parameter value corresponding to the image can represent the face shape and expression corresponding to the second face image, and the correlation between the first face image and multiple second face images can be determined by the 3DMM parameter value, specifically Specifically, it is assumed that multiple second face images contain n second face images, so that the linear fitting coefficient between the first face image and multiple second face images also includes n linear fitting coefficient values, The relationship between the face parameter values of the first face image and the face parameter values corresponding to multiple second face images can be expressed according to the following formula (1):

(1).

表示第一人臉影像對應的3DMM參數值；

表示第一人臉影像和第x張第二人臉影像之間的線性擬合系數值；

表示第x張第二人臉影像對應的人臉參數值；L表示確定第一人臉影像對應的人臉參數值時使用到的第二人臉影像的數量；x用於指示第x張第二人臉影像，其中，

。 in,

Indicates the 3DMM parameter value corresponding to the first face image;

Indicates the linear fitting coefficient value between the first face image and the xth second face image;

Represents the face parameter value corresponding to the xth second face image; L represents the number of second face images used when determining the face parameter value corresponding to the first face image; x is used to indicate the xth face image Two face images, in which,

.

In the embodiment of the present disclosure, it can be proposed that the linear fitting coefficient representing the correlation between the first face image and multiple second face images can be quickly obtained through a small number of face parameter values, and further can be based on the linear fitting coefficient The combination coefficient adjusts the dense point cloud data of multiple second face images in the preset style, and can quickly obtain the dense point cloud data of the target face in the preset style.

Specifically, for the above S3021, based on the face parameter values of the first face image and the face parameter values corresponding to the plurality of second face images of the preset style, the first face image and the preset style are determined. When performing linear fitting coefficients between multiple second human face images, the following steps S30211 to S30214 are included.

S30211. Obtain the current linear fitting coefficient; wherein, the initial current linear fitting coefficient is preset.

The current linear fitting coefficient can be the linear fitting coefficient adjusted at least once according to the following steps S30212 to S30214, and can also be the initial linear fitting coefficient. In the case where the current linear fitting coefficient is the initial linear fitting coefficient , the initial linear fitting coefficient can be set in advance based on experience.

S30212. Predict the current face parameter value of the first face image based on the current linear fitting coefficient and the face parameter values respectively corresponding to the multiple second face images.

Exemplarily, the face parameter values corresponding to the multiple second face images can be extracted by the above-mentioned pre-trained neural network, and then the current linear fitting coefficient and the multiple second face images can be respectively The corresponding face parameter value is input into the above formula (1), and the current face parameter value of the first face image is obtained by prediction.

S30213. Determine a second loss value based on the predicted current face parameter value and the face parameter value of the first face image.

In the process of adjusting the linear fitting coefficient, the difference between the current face parameter value of the predicted first face image and the face parameter value of the first face image extracted by the above-mentioned pre-trained neural network There is a certain gap, and based on the gap, a second loss value between the extracted face parameter value of the first face image and the predicted face parameter value of the first face image can be determined.

S30214, based on the second loss value and the constraint range corresponding to the preset linear fitting coefficient, adjust the current linear fitting coefficient, and return to the step of predicting the current face parameter value based on the adjusted current linear fitting coefficient until the When the adjustment operation of the current linear fitting coefficient meets the second adjustment cut-off condition, the linear fitting coefficient between the first human face image and multiple second human face images of the preset style is obtained based on the current linear fitting coefficient.

For example, considering that the face parameter values are used to represent the shape and size of the face, in order to avoid the occurrence of dense point cloud data of the first face image determined by the linear fitting coefficient in the later stage when characterizing the face Distortion, it is proposed here that in the process of adjusting the current linear fitting coefficient based on the second loss value, it is necessary to adjust the current linear fitting coefficient together with the preset constraint range of the linear fitting coefficient. For example, a large amount of data can be used here Statistics, determine that the constraint range corresponding to the preset linear fitting coefficient is set between -0.5 and 0.5, so that in the process of adjusting the current linear fitting coefficient based on the second loss value, each adjusted linear fitting can be made The coefficient is between -0.5 and 0.5.

Exemplarily, within the constraint range corresponding to the second loss value and the preset linear fitting coefficient, the current linear fitting coefficient is adjusted so that the predicted current face parameter value of the first face image and the neural The face parameter values of the first face image extracted by the network are closer, and then return to S30212 based on the adjusted current linear fitting coefficient until the adjustment operation of the current linear fitting coefficient meets the second adjustment cut-off condition. In some cases, for example, the linear fitting coefficient is obtained after the second loss value is smaller than the second preset threshold and/or the number of adjustments for the current linear fitting coefficient reaches the preset number.

In the embodiment of the present disclosure, in the process of adjusting the linear fitting coefficients between the first face image and multiple second face images of the preset style, the linear fitting is performed through the second loss value and/or the number of adjustments Adjusting the coefficients multiple times can improve the accuracy of the linear fitting coefficients; on the other hand, during the adjustment process, the constraints are adjusted through the preset constraints of the linear fitting coefficients, so that the linear fitting coefficients can be obtained, which can be more reasonably determined The dense point cloud data corresponding to the target face.

Specifically, the dense point cloud data includes the coordinate values of each point in the dense point cloud. For the above S3022, determine the target When the dense point cloud data of the face is in the preset style, the following steps S30221 to S30224 are included.

S30221, based on the coordinate values of each point in the dense point cloud corresponding to the multiple second face images of the preset style, determine the coordinate value of the corresponding point in the average dense point cloud data;

For example, when determining the coordinate values of each point in the average dense point cloud data corresponding to multiple second face images of the preset style, based on the coordinate values of each point corresponding to the multiple second face images, And the number of multiple second human face images is determined. For example, multiple second face images contain 20 pieces, and the dense point cloud data corresponding to each second face image contains 3D coordinate values of 100 points. For the first point, the first point can be placed in the 20th The corresponding three-dimensional coordinate values in the two face images are summed, and then the value obtained by dividing the summation result by 20 is used as the coordinate value of the corresponding first point in the average dense point cloud data. In the same manner, the coordinate value of each point in the three-dimensional coordinate system in the average dense point cloud data corresponding to multiple second face images can be obtained. In other words, the mean values of the coordinates of the corresponding points in the dense point cloud data of the multiple second face images constitute the coordinate values of the corresponding points in the average dense point cloud data here.

S30222. Based on the coordinate values of each point in the dense point cloud corresponding to the plurality of second human face images of the preset style and the coordinate values of the corresponding points in the average dense point cloud data, determine the plurality of second human face images of the preset style The coordinate difference values corresponding to the face images respectively.

Exemplarily, the coordinate value of each point in the average dense point cloud data can represent the average virtual face model corresponding to multiple second face images, for example, the facial features size represented by the coordinate value of each point in the average dense point cloud data can be The average size of facial features corresponding to the plurality of second facial images, and the facial size represented by the coordinate values of each point in the average dense point cloud data may be the average facial size corresponding to the plurality of second facial images.

Exemplarily, by making a difference between the coordinate values of the dense point clouds corresponding to the multiple second face images and the coordinate values of the corresponding points in the average dense point cloud data, the dense point cloud values corresponding to the multiple second face images can be obtained respectively. The coordinate difference value of the coordinate value of each point in the point cloud relative to the coordinate value of the corresponding point in the average dense point cloud data (this article can also be referred to as "the coordinate difference value corresponding to the second face image"), which can characterize the The difference between the virtual human face model corresponding to the second human face image and the aforementioned average human face model.

S30223. Determine the coordinate difference value corresponding to the first human face image based on the coordinate difference values and linear fitting coefficients respectively corresponding to the plurality of second human face images in the preset style.

Exemplarily, the linear fitting coefficient can represent the relationship between the face parameter values corresponding to the first face image and the face parameter values corresponding to multiple second face images respectively, and the face values corresponding to the face images There is a corresponding relationship between the parameter value and the dense point cloud data corresponding to the face image, so the linear fitting coefficient can also represent the dense point cloud data corresponding to the first face image and the dense point cloud data corresponding to multiple second face images. The relationship between point cloud data.

In the case of corresponding to the same average dense point cloud data, the linear fitting coefficient can also represent the relationship between the coordinate difference values corresponding to the first face image and the coordinate difference values corresponding to multiple second face images , therefore, based on the coordinate difference values and linear fitting coefficients corresponding to the multiple second face images, the coordinate difference value of the dense point cloud data corresponding to the first face image relative to the average dense point cloud data can be determined.

S30224. Based on the coordinate difference value corresponding to the first face image and the coordinate value of the corresponding point in the average dense point cloud data, determine the dense point cloud data of the target face in a preset style.

Summing the coordinate difference value corresponding to the first face image and the coordinate value of the corresponding point in the average dense point cloud data, the dense point cloud data corresponding to the first face image can be obtained, based on the dense point cloud data can represent the A virtual face model corresponding to the first face image.

表示，具體可以根據以下公式（2）進行確定：

（2）。 Specifically, when determining the dense point cloud data corresponding to the target face, considering the relationship between the dense point cloud data and 3DMM, the dense point cloud data corresponding to the target face (first face image) can be obtained by

Specifically, it can be determined according to the following formula (2):

(2).

表示第x張第二人臉影像對應的稠密點雲的坐標值；

表示根據多張第二人臉影像確定的平均稠密點雲數據中對應點的坐標值；

可以表示第一人臉影像對應的點的坐標值相對於平均稠密點雲數據中對應點的坐標值的坐標差異值。 in,

Indicates the coordinate value of the dense point cloud corresponding to the xth second face image;

Indicates the coordinate value of the corresponding point in the average dense point cloud data determined according to multiple second face images;

It may represent the coordinate difference value of the coordinate value of the point corresponding to the first face image relative to the coordinate value of the corresponding point in the average dense point cloud data.

Here, when determining the dense point cloud data of the first face image, the method of steps S30221 to S30224 is used for determination, that is, the determination is made by the method of the above formula (2), compared with the method of multiple second face images corresponding to The method of using dense point cloud data and linear fitting coefficients to determine the dense point cloud data corresponding to the target face can include the following benefits.

In the embodiment of the present disclosure, it is considered that the linear fitting coefficient is used to linearly fit the coordinate difference values corresponding to multiple second face images, so that the coordinate values of the points corresponding to the first face images are relatively The coordinate difference value of the coordinate value of the corresponding point in the average dense point cloud data (this article can also be referred to as "the coordinate difference value corresponding to the first face image"), so it is not necessary to make the sum of these linear fitting coefficients equal to 1. It is limited that after the coordinate difference value corresponding to the first face image is added to the coordinate value of the corresponding point in the average dense point cloud data, the obtained dense point cloud data can also represent a normal face image.

In addition, in the case of fewer second human face images, according to the method provided by the embodiment of the present disclosure, the linear fitting coefficient can be reasonably adjusted to use a small number of second human face images to determine the target human face. The purpose of the dense point cloud data corresponding to the preset style, for example, the eye size of the first face image is small eyes, through the above method, there is no need to limit the eye size of multiple second face images, but can be linearly The fitting coefficient adjusts the coordinate difference value, so that after the adjusted coordinate difference value and the coordinate value of the corresponding point in the average dense point cloud data are superimposed, dense point cloud data representing ommatidia can be obtained. Specifically, even when multiple second face images have big eyes, the eyes represented by the corresponding average dense point cloud data are also big eyes, and the linear fitting coefficient can still be adjusted, so that the adjusted coordinate difference value The dense point cloud data representing ommatidia can be obtained by summing the coordinate values of the corresponding points in the average dense point cloud data.

It can be seen that, for different first facial images, the embodiment of the present disclosure does not need to select a second facial image similar to the facial features of the first facial image to determine the dense point cloud data corresponding to the first facial image. The method can accurately represent the dense point cloud data of different target faces in the preset style through the dense point cloud data of the diverse second face images when there are few second human face images.

According to the above method, the dense point cloud data of the target face in the preset style can be obtained, for example, the dense point cloud data of the target face in the classical style can be obtained, and further based on the dense point cloud data to show the target face in the classical style initial virtual face image.

Regarding the above S102, when determining the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image, as shown in FIG. 7 , the following steps S501 to S504 are included.

S501. Adjust the standard dense point cloud data based on the current deformation coefficient to obtain the current dense point cloud data. The initial current deformation coefficient is preset.

Exemplarily, when the deformation coefficient includes a bone coefficient, the change matrix when adjusting the first dense point cloud associated with the bone coefficient can be jointly determined based on the current bone coefficient and the initial bone change matrix; In the case of the deformation coefficient, the change amount when adjusting the second dense point cloud associated with the mixed deformation coefficient can be determined based on the current mixed deformation coefficient and the unit mixed deformation amount. See below for details.

Exemplarily, in order to explain the process of adjusting standard dense point cloud data, the bone coordinate system and the conversion relationship between the bone coordinate system and the world coordinate system can be introduced here, where the bone coordinate system is established for each bone in three dimensions The coordinate system is the local coordinate system corresponding to each bone. The world coordinate system is a three-dimensional coordinate system established for the entire face. There is a conversion relationship between the local coordinate system corresponding to each bone and the world coordinate system. According to the conversion relationship , which can convert the position of the dense point cloud in the bone coordinate system to the position in the world coordinate system.

In particular, in the process of adjusting the standard dense point cloud data based on the current deformation coefficient, it can be divided into two cases. The first case is when adjusting the corresponding points in the standard dense point cloud data based on the mixed deformation coefficient , will be affected by the bone coefficient, which will be explained in conjunction with formula (3) below; the second case is that when adjusting the corresponding points in the standard dense point cloud data based on the mixed deformation coefficient, it will not be affected by the bone coefficient The situation of the influence will be described in conjunction with formula (4) below.

（3）。 Specifically, in the first case, the adjusted dense point cloud data can be determined according to the following formula (3):

(3).

為針對標準稠密點雲數據中第m個點進行調整過程中，在預先以人臉建立的世界坐標系下的坐標值；

表示第i個骨骼對應的骨骼坐標系向世界坐標系進行變換的變換矩陣；

表示預先設定的第i個骨骼在該骨骼對應的骨骼坐標系下的初始骨骼變換矩陣；

表示在第i個骨骼在第i個骨骼對應的骨骼坐標系下的值；

表示標準稠密點雲數據中的第m個點在第i個骨骼對應的骨骼坐標系下的初始坐標值（當該第m個點不在第i個骨骼中時，該初始坐標值為0）；

表示預先設定與第m個點關聯的混合形變係數在第i個骨骼對應的骨骼坐標系下的單位形變量；

表示與第m個點關聯的混合形變係數在第i個骨骼對應的骨骼坐標系下的坐標值；i用於指示第i個骨骼，

；n表示標準虛擬人臉影像對應的骨骼個數；m表示稠密點雲數據中的第m個點。 in,

In the process of adjusting the mth point in the standard dense point cloud data, the coordinate value in the world coordinate system pre-established with the face;

Indicates the transformation matrix that transforms the bone coordinate system corresponding to the i-th bone to the world coordinate system;

Indicates the initial bone transformation matrix of the preset i-th bone in the bone coordinate system corresponding to the bone;

Indicates the value of the i-th bone in the bone coordinate system corresponding to the i-th bone;

Indicates the initial coordinate value of the m-th point in the standard dense point cloud data in the bone coordinate system corresponding to the i-th bone (when the m-th point is not in the i-th bone, the initial coordinate value is 0);

Indicates the unit deformation of the blended deformation coefficient associated with the mth point in the bone coordinate system corresponding to the i bone;

Indicates the coordinate value of the mixed deformation coefficient associated with the m-th point in the bone coordinate system corresponding to the i-th bone; i is used to indicate the i-th bone,

; n represents the number of bones corresponding to the standard virtual face image; m represents the mth point in the dense point cloud data.

It can be seen that for the first case above, after adjusting the coordinate values of the points in the standard dense point cloud data in the bone coordinate system based on the mixed deformation coefficient, it is necessary to combine the bone deformation coefficient to finally determine the standard dense point cloud data. The coordinate value of the dense point cloud in the world coordinate system, which is mentioned above, will be affected by the bone coefficient when adjusting the dense point cloud in the standard dense point cloud data based on the mixed deformation coefficient.

（4）。 For the second case, the adjusted dense point cloud data can be determined according to the following formula (4):

(4).

針對標準稠密點雲數據中第m個點進行調整過程中，在世界坐標系下的坐標值；

表示第i個骨骼對應的骨骼坐標系向世界坐標系進行變換的變換矩陣；

表示預先設定的第i個骨骼在該骨骼對應的骨骼坐標系下的初始骨骼變換矩陣；

表示在第i個骨骼在第i個骨骼對應的骨骼坐標系下的值；

表示標準稠密點雲數據中的第m個點在第i個骨骼對應的骨骼坐標系下的初始位置（當該第m個點不在第i個骨骼中時，該初始位置為0）；

表示預先設定與第m個點關聯的混合形變係數在世界坐標系下的單位形變量；

表示與第m個點關聯的混合形變係數在世界坐標系下的值；i用於指示第i個骨骼，

；n表示需要調整的骨骼個數。 in,

The coordinate value in the world coordinate system during the adjustment process for the mth point in the standard dense point cloud data;

Indicates the transformation matrix that transforms the bone coordinate system corresponding to the i-th bone to the world coordinate system;

Indicates the initial bone transformation matrix of the preset i-th bone in the bone coordinate system corresponding to the bone;

Indicates the value of the i-th bone in the bone coordinate system corresponding to the i-th bone;

Indicates the initial position of the m-th point in the standard dense point cloud data in the bone coordinate system corresponding to the i-th bone (when the m-th point is not in the i-th bone, the initial position is 0);

Indicates the pre-set unit deformation of the mixed deformation coefficient associated with the mth point in the world coordinate system;

Indicates the value of the blended deformation coefficient associated with the mth point in the world coordinate system; i is used to indicate the ith bone,

; n represents the number of bones that need to be adjusted.

It can be seen that for the second case above, the coordinate values of the points in the standard dense point cloud data in the world coordinate system can be adjusted directly based on the mixed deformation coefficient, that is, the above-mentioned standard dense point cloud data based on the mixed deformation coefficient When adjusting the points in , it will not be affected by the bone coefficient.

The above formula (3) or formula (4) is the process of adjusting one of the points in the standard dense point cloud data. In the same way, it can be adjusted for other points in the standard dense point cloud data in turn, so as to complete An adjustment to standard dense point cloud data based on the current deformation coefficient.

S502. Determine a first loss value based on the adjusted dense point cloud data and the initial dense point cloud data of the target face.

Exemplarily, the first loss value may be represented by a difference between the initial dense point cloud data of the target face and the adjusted dense point cloud data.

（5）。 Specifically, the first loss value can be expressed by the following formula (5):

(5).

表示調整後的稠密點雲數據相比目標人臉的初始稠密點雲數據的第一損失值；

表示目標人臉的初始稠密點雲數據中的第m個點在世界坐標系下的坐標值；

表示調整後的稠密點雲數據中的第m個點在世界坐標系下的坐標值；m表示稠密點雲數據中的第m個點；M表示稠密點雲數據中的點的數量。 in,

Represents the first loss value of the adjusted dense point cloud data compared to the initial dense point cloud data of the target face;

Represents the coordinate value of the mth point in the world coordinate system in the initial dense point cloud data of the target face;

Represents the coordinate value of the mth point in the adjusted dense point cloud data in the world coordinate system; m represents the mth point in the dense point cloud data; M represents the number of points in the dense point cloud data.

S503. Adjust the current deformation coefficient based on the first loss value and the preset constraint range of the deformation coefficient. Based on the adjusted current deformation coefficient, return to the step of adjusting the standard dense point cloud data until the current deformation coefficient is adjusted. The operation meets the first adjustment cut-off condition, and the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data is obtained.

Exemplarily, considering that the current deformation coefficient is the deformation coefficient of the target face relative to the standard face, that is, the current deformation coefficient is used to represent the normal facial appearance, in order to avoid excessive adjustment of the current deformation coefficient, make The facial shape it represents is distorted. Here, it is proposed that in the process of adjusting the current deformation coefficient based on the loss function value, it is necessary to adjust the current linear fitting coefficient together with the constraint range of the preset linear fitting coefficient. Specifically, The preset deformation coefficient here is a mixed deformation coefficient, for example, the value of the restricted mixed deformation coefficient is between 0 and 1.

Exemplarily, the current deformation coefficient is adjusted based on the first loss value and the constraint range corresponding to the preset deformation coefficient, so that the initial dense point cloud data of the target face and the adjusted dense point cloud data are closer. approach, and then return to S501 based on the adjusted current deformation coefficient, until the adjustment operation on the current deformation coefficient meets the first adjustment cut-off condition, for example, when the first loss value is less than the first preset threshold and/or the current deformation coefficient After the number of adjustments reaches the preset number of times, the deformation coefficient corresponding to the target face is obtained.

In the embodiment of the present disclosure, in the process of determining the deformation coefficient, it is determined by adjusting multiple points in the standard dense point cloud data, so the obtained deformation coefficient can represent the initial dense point cloud of the target face. The accurate change amount of the standard dense point cloud, so that in the process of adjusting the initial virtual face image of the target face, the associated points in the dense point cloud data can be adjusted based on the deformation coefficient, thereby improving the adjustment accuracy.

On the other hand, in the process of determining the deformation coefficient, the loss value is determined based on the adjusted dense point cloud data and the initial dense point cloud data of the target face after adjusting all the dense point clouds. The optimization of the deformation coefficient fully considers the correlation between the deformation coefficient and the overall dense point cloud to improve the optimization efficiency; in addition, during the adjustment process, the adjustment constraints are adjusted through the preset deformation coefficient constraint range, which can prevent the deformation coefficient from being distorted. Get deformation coefficients that cannot represent normal target faces.

For the above S103, when adjusting the deformation coefficient in response to the adjustment operation for the initial virtual face image to obtain the target deformation coefficient, as shown in Figure 8, the following steps S601 to S602 may be included:

S601. In response to the adjustment operation on the initial virtual face image, determine a target adjustment position for the initial virtual face image and an adjustment range for the target adjustment position;

S602. Adjust the deformation coefficient associated with the target adjustment position according to the adjustment range to obtain the target deformation coefficient.

For example, in the process of adjusting the initial virtual face image, considering that the initial virtual face image contains many positions that can be adjusted, when presenting these adjustable positions to the user, it is possible to pre-select the positions that can be adjusted. For example, it can be classified according to different areas of the face, for example, it can be divided into the chin area, eyebrow area, eye area, etc. Correspondingly, the adjustment operation buttons corresponding to the chin area, eyebrow area, and eye area can be displayed , the user can select the target adjustment area based on the adjustment operation buttons corresponding to different areas; or, the user can be shown each time adjustment buttons corresponding to a set number of adjustment positions, and an indication button for changing the adjustment position. For example, as shown in Figure 9, the left picture in Figure 9 shows six adjustment interfaces for adjusting positions, including the amplitude bars corresponding to the upper and lower sides of the nose, the height of the bridge of the nose, the size of the nose, the orientation of the nose, the size of the mouth, and the upper and lower parts of the mouth. , the user can adjust the adjustment position by dragging the amplitude bar, or after selecting the adjustment position, adjust it through the adjustment buttons above the adjustment position, such as the "minus one" button and "plus one" button, the right side of the adjustment interface The lower corner also shows an arrow button used to indicate the replacement of the adjustment position. The user can trigger the arrow button to change to the 6 adjustment positions shown in the right picture in Figure 9.

Specifically, for each adjustment position, the adjustment range for the adjustment position can be determined according to the range bar corresponding to the adjustment position, and when the user adjusts the range bar corresponding to one of the adjustment positions, the adjustment position can be used as the target adjustment position , determine the adjustment range for the target adjustment position based on the change data of the range bar, and further carry out the deformation coefficient associated with the target adjustment position according to the adjustment range and the preset relationship between each adjustment position and the deformation coefficient Adjust to get the target deformation coefficient.

In the embodiment of the present disclosure, the target deformation coefficient can be determined according to the adjustment operation, so that the target virtual human face image can be determined later based on the target deformation coefficient. This method can adjust the deformation coefficient based on personalized user needs.

Regarding the above S104, when generating a target virtual face image corresponding to the target face based on the target deformation coefficient and standard dense point cloud data, as shown in FIG. 10 , the following steps S801 to S802 may be included:

S801, based on the target deformation coefficient, adjust the standard dense point cloud data to obtain the target dense point cloud data;

S802. Generate a virtual face image of the target based on the dense point cloud data of the target.

Exemplarily, the target deformation coefficient may include a changed deformation coefficient associated with the target adjusted position, or may include an unchanged deformation coefficient that has not been adjusted. Considering that the deformation coefficient is the initial dense point cloud data of the target face Therefore, in the process of adjusting the initial virtual face image based on the target deformation coefficient, the target dense point cloud data corresponding to the target face can be obtained based on the target deformation coefficient and the standard dense point cloud data , and further based on the dense point cloud data of the target, a target virtual face image is generated.

Exemplarily, for example, as shown in FIG. 9 above, the user clicks to adjust the height of the bridge of the nose to increase the height of the bridge of the nose. Compared with the initial virtual face image, the bridge of the nose of the target virtual face image becomes higher.

In the embodiment of the present disclosure, after the target deformation coefficient is determined, the standard dense point cloud data can be directly adjusted according to the target deformation coefficient to determine the target dense point cloud data, so that the target face correspondence can be quickly obtained according to the adjusted target dense point cloud data. target virtual face image.

Specifically, when generating the target virtual human face image based on the target dense point cloud data, the following steps S8021 to S8022 are included:

S8021, determining a virtual face model corresponding to the target dense point cloud data;

S8022. Generate a target virtual face image based on the preselected face attributes and virtual face model.

Exemplarily, the virtual face model may be a three-dimensional face model or a two-dimensional face model, which is related to a specific application scenario and is not limited here.

Exemplarily, the face attribute features may include skin color, hairstyle and other features, and the face attribute features may be determined according to the user's selection, for example, the user may choose to set the skin color to be white, and the hairstyle to be brown curly hair.

After obtaining the target dense point cloud data, the target virtual face model can be generated based on the target dense point cloud data. The target virtual face model can include the shape and expression features of the target face, and then combined with the face attribute features, it can generate The target virtual face image that meets the user's individual needs.

In the embodiment of the present disclosure, when adjusting the initial virtual human face image, it may also be adjusted in combination with the facial attribute characteristics selected by the user, so that the target virtual human face image is more suitable for the actual needs of the user.

The processing process of the face image will be described below with a specific embodiment, including the following steps S901~S904:

（其中

表示稠密點雲中M個點的坐標值），再獲取標準虛擬人臉影像對應的標準稠密點雲數據和預設的初始形變係數（包括初始骨骼形變係數和初始混合形變係數）； S901, for the input target face, using a computer to read the initial dense point cloud data of the input target face

(in

represent the coordinate values of M points in the dense point cloud), and then obtain the standard dense point cloud data corresponding to the standard virtual face image and the preset initial deformation coefficient (including the initial bone deformation coefficient and the initial mixed deformation coefficient);

（其中

表示稠密點雲中M個點調整後的坐標值），具體可以通過上述公式（3）或者上述公式（4）進行調整； S902. Adjust the standard dense point cloud data according to the initial bone deformation coefficient and the initial mixed deformation coefficient to obtain adjusted dense point cloud data

(in

Indicates the adjusted coordinate values of M points in the dense point cloud), which can be adjusted by the above formula (3) or the above formula (4);

和調整後的稠密點雲數據

之間的差異值

，並通過該差異值以及針對初始混合形變係數的約束項，對初始骨骼形變係數和初始混合形變係數進行調整； S903, calculate the initial dense point cloud data of the target face

and adjusted dense point cloud data

difference between

, and adjust the initial bone deformation coefficient and the initial mixed deformation coefficient through the difference value and the constraint item for the initial mixed deformation coefficient;

和調整後的稠密點雲數據

的差異值小於第一預設閾值，或者迭代次數超過預設次數。 S904, replace the initial bone deformation coefficient according to the adjusted bone deformation coefficient, and replace the initial mixed deformation coefficient according to the adjusted mixed deformation coefficient, and return to step S902 to continue adjusting the bone coefficient and the mixed deformation coefficient until the initial denseness of the target face point cloud data

and adjusted dense point cloud data

The difference value of is smaller than the first preset threshold, or the number of iterations exceeds the preset number.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same technical concept, the embodiment of the present disclosure also provides a processing device corresponding to the processing method of the human face image. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned processing method of the embodiment of the present disclosure, the device's For the implementation, please refer to the implementation of the method, and repeated descriptions will not be repeated.

Referring to FIG. 11 , an embodiment of the present disclosure provides a face image processing device 1000, the processing device includes: an acquisition module 1001, which is used to acquire the initial dense point cloud data of the target face, and based on the initial dense point cloud data Generate the initial virtual face image of the target face; determine the module 1002 for determining the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image; adjust the module 1003 for responding For the adjustment operation of the initial virtual face image, the deformation coefficient is adjusted to obtain the target deformation coefficient; the generation module 1004 is used to generate the target virtual face corresponding to the target face based on the target deformation coefficient and standard dense point cloud data image.

In a possible implementation manner, the deformation coefficient includes at least one of at least one bone coefficient and at least one mixed deformation coefficient; wherein each bone coefficient is used for the bone formed by the first dense point cloud associated with the bone coefficient The initial pose of the mixture is adjusted; each mixture deformation coefficient is used to adjust the initial position corresponding to the second dense point cloud associated with the mixture deformation coefficient.

In a possible implementation, when the determination module 1002 is used to determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data, it includes: adjusting the standard dense point cloud data based on the current deformation coefficient, and obtaining the adjusted The dense point cloud data, the initial current deformation coefficient is preset; based on the adjusted dense point cloud data and the initial dense point cloud data, determine the first loss value; based on the first loss value and the constraints of the preset deformation coefficient Range, adjust the current deformation coefficient; based on the adjusted current deformation coefficient, return to the step of adjusting the standard dense point cloud data until the adjustment operation of the current deformation coefficient meets the first adjustment cut-off condition, according to the current deformation coefficient Obtain the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data.

In a possible implementation manner, when the adjustment module 1003 adjusts the deformation coefficient in response to the adjustment operation on the initial virtual human face image to obtain the target deformation coefficient, it includes: responding to the adjustment operation on the initial virtual human face image The operation is to determine the target adjustment position for the initial virtual face image and the adjustment range for the target adjustment position; according to the adjustment range, adjust the deformation coefficient associated with the target adjustment position to obtain the target deformation coefficient.

In a possible implementation, when the generation module 1004 is used to generate the target virtual human face image corresponding to the target face based on the target deformation coefficient and standard dense point cloud data, it includes: based on the target deformation coefficient, standard dense point cloud data The cloud data is adjusted to obtain the target dense point cloud data; based on the target dense point cloud data, the target virtual face image is generated.

In a possible implementation, when the generation module 1004 is used to generate a target virtual human face image based on the target dense point cloud data, it includes: determining a virtual human face model corresponding to the target dense point cloud data; Face attribute features and a virtual face model to generate a target virtual face image.

In a possible implementation manner, when the acquiring module 1001 is used to acquire the initial dense point cloud data of the target face and display the initial virtual face image of the target face based on the initial dense point cloud data, it includes: acquiring the target face The first face image corresponding to the face, and the dense point cloud data corresponding to the multiple second face images of the preset style; the multiple second face images based on the first face image and the preset style respectively Dense point cloud data, determine the dense point cloud data of the target face in the preset style; based on the dense point cloud data of the target face in the preset style, generate and display the initial virtual person of the target face in the preset style face image.

In a possible implementation, the acquisition module 1001 is used to determine the target face in the preset style based on the dense point cloud data corresponding to the first face image and multiple second face images of the preset style. When using the dense point cloud data, it includes: extracting the face parameter values of the first face image, and the face parameter values corresponding to multiple second face images of the preset style; wherein, the face parameter values include Face shape parameter values and parameter values representing facial expressions; face parameter values based on the first face image, and face parameter values and dense point cloud data corresponding to multiple second face images of preset styles , to determine the dense point cloud data of the target face in the preset style.

In a possible implementation manner, the acquisition module 1001 uses the face parameter values based on the first face image and the face parameter values and dense point clouds respectively corresponding to multiple second face images of preset styles. When determining the dense point cloud data of the target face in the preset style, it includes: face parameter values based on the first face image, and face parameters corresponding to multiple second face images of the preset style value to determine the linear fitting coefficient between the first face image and multiple second face images of the preset style; the dense point cloud data and linear fitting coefficients corresponding to the multiple second face images of the preset style Combined coefficient to determine the dense point cloud data of the target face in the preset style.

In a possible implementation manner, the acquisition module 1001 is used to determine the first face parameter value based on the face parameter value of the first face image and the face parameter values corresponding to multiple second face images of a preset style. When the linear fitting coefficient between the face image and multiple second face images of the preset style includes: obtaining the current linear fitting coefficient, the initial current linear fitting coefficient is preset; based on the current linear fitting coefficient Predict the current face parameter values of the first face image based on face parameter values corresponding to multiple second face images of the preset style; based on the predicted current face parameter values and the face of the first face image Parameter value, determine the second loss value; based on the second loss value and the constraint range corresponding to the preset linear fitting coefficient, adjust the current linear fitting coefficient; based on the adjusted current linear fitting coefficient, return to perform prediction of the current face The step of parameter value, until the adjustment operation to the current linear fitting coefficient meets the second adjustment cut-off condition, obtain the first human face image and the plurality of second human face images of the preset style based on the current linear fitting coefficient. The linear fit coefficient between.

In a possible implementation, the dense point cloud data includes the coordinate values of each point in the dense point cloud; the dense point cloud data and linear Fitting coefficient, determine the dense point cloud data of the target face in the preset style, including: based on the coordinate values of each point in the dense point cloud corresponding to multiple second face images of the preset style, determine the average dense point The coordinate value of the corresponding point in the cloud data; based on the coordinate value of each point in the dense point cloud corresponding to the multiple second face images of the preset style, and the coordinate value of the corresponding point in the average dense point cloud data, determine the preset The coordinate difference values corresponding to the multiple second face images of the style; based on the coordinate difference values and linear fitting coefficients corresponding to the multiple second face images of the preset style, determine the coordinate difference corresponding to the first face image value; based on the coordinate difference value corresponding to the first face image and the coordinate value of the corresponding point in the average dense point cloud data, determine the dense point cloud data of the target face in the preset style.

In a possible implementation manner, the face parameter values are extracted by a pre-trained neural network, and the neural network is trained based on sample images with pre-marked face parameter values.

In a possible implementation manner, the processing device further includes a training module 1005, and the training module 1005 is used to pre-train the neural network in the following manner: obtain a sample image set, the sample image set includes multiple sample images and each sample image The corresponding tagged face parameter values; input multiple sample images into the neural network to obtain the predicted face parameter values corresponding to each sample image; based on the predicted face parameter values and tagged face parameter values corresponding to each sample image, Adjust the network parameter values of the neural network to obtain a trained neural network.

For the description of the processing flow of each module in the device and the interaction flow between the modules, reference may be made to the relevant description in the above method embodiment, and details will not be described here.

Corresponding to the processing method of the face image in FIG. 1 , the embodiment of the present disclosure also provides a kind of communication, so that the processor 111 executes the following instructions: acquire the initial dense point cloud data of the target face, and based on the initial dense point cloud The initial virtual face image of the target face is generated from the data; the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image is determined; in response to the adjustment operation for the initial virtual face image, the deformation The coefficient is adjusted to obtain the target deformation coefficient; based on the target deformation coefficient and standard dense point cloud data, the target virtual face image corresponding to the target face is generated.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method for processing a human face image described in the above-mentioned method embodiments are executed. . Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

An embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the method for processing a human face image described in the method embodiment above, for details, please refer to The foregoing method embodiments are not described in detail here.

Wherein, the above-mentioned computer program product can be realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may also be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium, including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: flash drive, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. media.

Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than to limit them, and the protection scope of the present disclosure is not limited thereto. The present disclosure has been described in detail in this example, and those skilled in the art should understand that any person familiar with the technical field can still modify or modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure. Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of the technical solutions of the embodiments of the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

S101: Obtain the original dense point cloud data of the target face, and generate an initial virtual face image of the target face based on the original dense point cloud data S102: Determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image S103: In response to the adjustment operation for the initial virtual face image, adjust the deformation coefficient to obtain the target deformation coefficient S104: Based on the target deformation coefficient and standard dense point cloud data, generate a target virtual face image corresponding to the target face S201: Obtain the first face image corresponding to the target face, and the dense point cloud data respectively corresponding to multiple second face images of the preset style S202: Based on the dense point cloud data corresponding to the first face image and multiple second face images of the preset style, determine the dense point cloud data of the target face in the preset style. S203: Based on the dense point cloud data of the target face in the preset style, generate and display the initial virtual face image of the target face in the preset style S301: extract the face parameter value of the first face image, and the face parameter values corresponding to the plurality of second face images of the preset style; wherein, the face parameter value includes a parameter value and a representation characterizing the shape of the face Parameter values of facial expressions S302: Based on the face parameter values of the first face image, and the face parameter values and dense point cloud data respectively corresponding to a plurality of second face images of the preset style, determine the denseness of the target face under the preset style point cloud data S401: Obtain a sample image set, the sample image set includes multiple sample images and the labeled face parameter values corresponding to each sample image S402: Input multiple sample images into the neural network to obtain the predicted face parameter value corresponding to each sample image S403: Based on the predicted face parameter value and marked face parameter value corresponding to each sample image, the network parameter value of the neural network is adjusted to obtain the trained neural network S3021: Based on the face parameter values of the first face image and the face parameter values corresponding to the multiple second face images of the preset style, determine the first face image and the multiple second face images of the preset style Linear fit coefficient between face images S3022: Determine the dense point cloud data of the target face in the preset style according to the dense point cloud data and linear fitting coefficients respectively corresponding to the multiple second face images of the preset style. S501: Adjust the standard dense point cloud data based on the current deformation coefficient to obtain the current dense point cloud data, the initial current deformation coefficient is preset S502: Based on the adjusted dense point cloud data and the initial dense point cloud data of the target face, determine a first loss value S503: Adjust the current deformation coefficient based on the first loss value and the constraint range of the preset deformation coefficient, and return to the step of adjusting the standard dense point cloud data based on the adjusted current deformation coefficient until the adjustment of the current deformation coefficient The operation meets the first adjustment cut-off condition, and the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data is obtained S601: In response to the adjustment operation on the initial virtual face image, determine the target adjustment position for the initial virtual face image, and the adjustment range for the target adjustment position S602: Adjust the deformation coefficient associated with the target adjustment position according to the adjustment range to obtain the target deformation coefficient S801: Based on the target deformation coefficient, adjust the standard dense point cloud data to obtain the target dense point cloud data S802: Based on the dense point cloud data of the target, generate a virtual human face image of the target 1000: Face image processing device 1001: Get the module 1002: Determine the module 1003: adjust the module 1004: generate module 1005: training module 1100: Electronic equipment 111: Processor 112: Storage 1121: Memory 1122: external memory 113: busbar

Fig. 1 shows a flow chart of a method for processing a face image provided by an embodiment of the present disclosure. Fig. 2 shows a schematic diagram of a three-dimensional model of a human face represented by dense point cloud data provided by an embodiment of the present disclosure. Fig. 3 shows a flow chart of a method for generating an initial virtual face image provided by an embodiment of the present disclosure. Fig. 4 shows a flow chart of a method for determining dense point cloud data of a target face in a preset style provided by an embodiment of the present disclosure. Fig. 5 shows a flowchart of a method for training a neural network provided by an embodiment of the present disclosure. Fig. 6 shows a flow chart of a method for specifically determining the dense point cloud data of a target face in a preset style provided by an embodiment of the present disclosure. Fig. 7 shows a flowchart of a method for determining deformation coefficient provided by an embodiment of the present disclosure. Fig. 8 shows a flowchart of a method for adjusting deformation coefficient provided by an embodiment of the present disclosure. FIG. 9 shows a schematic diagram of an adjustment interface for a virtual face image provided by an embodiment of the present disclosure. Fig. 10 shows a flowchart of a method for generating a target virtual face image of a target face provided by an embodiment of the present disclosure. FIG. 11 shows a schematic structural diagram of an apparatus for processing a human face image provided by an embodiment of the present disclosure. Fig. 12 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

S101: Obtain the original dense point cloud data of the target face, and generate an initial virtual face image of the target face based on the original dense point cloud data

S102: Determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual face image

S103: In response to the adjustment operation for the initial virtual face image, adjust the deformation coefficient to obtain the target deformation coefficient

S104: Based on the target deformation coefficient and standard dense point cloud data, generate a target virtual face image corresponding to the target face

Claims (15)

A method for processing a human face image, comprising: acquiring initial dense point cloud data of a target human face, and generating an initial virtual human face image of the target human face based on the initial dense point cloud data; determining the initial dense point cloud The data is relative to the deformation coefficient of the standard dense point cloud data corresponding to the standard virtual face image; in response to the adjustment operation for the initial virtual face image, the deformation coefficient is adjusted to obtain the target deformation coefficient; based on the target The deformation coefficient and the standard dense point cloud data generate a target virtual face image corresponding to the target face; wherein the deformation coefficient includes at least one of at least one bone coefficient and at least one hybrid deformation coefficient; wherein, Each bone coefficient is used to adjust the initial pose of the bone composed of the first dense point cloud associated with the bone coefficient; each mixed deformation coefficient is used to adjust the corresponding second dense point cloud associated with the mixed deformation coefficient Adjust the initial position. The processing method according to claim 1, wherein the determining the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data includes: modifying the standard dense point cloud based on the current deformation coefficient The data is adjusted to obtain adjusted dense point cloud data, and the initial current deformation coefficient is preset; based on the adjusted dense point cloud data and the initial dense point cloud data, a first loss value is determined; Adjust the current deformation coefficient based on the first loss value and the preset constraint range of the deformation coefficient; return to the step of adjusting the standard dense point cloud data based on the adjusted current deformation coefficient, until If the adjustment operation on the current deformation coefficient meets the first adjustment cut-off condition, the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data is obtained based on the current deformation coefficient. The processing method according to claim 1, wherein, in response to the adjustment operation on the initial virtual human face image, adjusting the deformation coefficient to obtain the target deformation coefficient includes: responding to the initial virtual face image The adjustment operation of the virtual human face image is to determine the target adjustment position for the initial virtual human face image, and the adjustment range for the target adjustment position; according to the adjustment range, the The deformation coefficient is adjusted to obtain the target deformation coefficient. The processing method according to claim 1, wherein generating a target virtual face image corresponding to the target face based on the target deformation coefficient and the standard dense point cloud data includes: based on the The target deformation coefficient is adjusted to the standard dense point cloud data to obtain the target dense point cloud data; based on the target dense point cloud data, the target virtual human face image is generated. The processing method as described in claim 4, wherein generating the target virtual human face image based on the target dense point cloud data includes: Determine a virtual human face model corresponding to the target dense point cloud data; generate the target virtual human face image based on the preselected human face attributes and the virtual human face model. The processing method as described in claim 1, wherein the acquisition of initial dense point cloud data of the target face, and generating an initial virtual face image of the target face based on the initial dense point cloud data includes : Obtain the first human face image corresponding to the target human face, and the dense point cloud data respectively corresponding to a plurality of second human face images of the preset style; based on the first human face image and the preset style The dense point cloud data corresponding to a plurality of second face images respectively, determine the initial dense point cloud data of the target face in the preset style; based on the initial dense point cloud data of the target face in the preset style Dense point cloud data to generate an initial virtual face image of the target face in the preset style. The processing method according to claim 6, wherein the target is determined based on the dense point cloud data respectively corresponding to the first face image and multiple second face images of the preset style The initial dense point cloud data of the face in the preset style includes: extracting the face parameter values of the first face image, and the faces corresponding to the plurality of second face images in the preset style A face parameter value; wherein, the face parameter value includes a parameter value characterizing the shape of a face and a parameter value characterizing a facial expression; based on the face parameter value of the first face image and the preset style The face parameter values and dense point cloud data corresponding to multiple second face images are determined The initial dense point cloud data of the target face in the preset style. The processing method as described in claim 7, wherein the face parameter values based on the first face image and the face parameters respectively corresponding to the plurality of second face images of the preset style Value and dense point cloud data, determine the initial dense point cloud data of the target face in the preset style, including: based on the face parameter value of the first face image, and the preset style The face parameter values corresponding to the plurality of second face images respectively determine the linear fitting coefficients between the first face image and the plurality of second face images of the preset style; according to the preset The dense point cloud data and the linear fitting coefficients respectively corresponding to the plurality of second face images of the style are used to determine the initial dense point cloud data of the target face in the preset style. The processing method as described in claim 8, wherein the face parameter value based on the first face image and the face parameters respectively corresponding to a plurality of second face images of the preset style Value, determining the linear fitting coefficient between the first face image and the plurality of second face images of the preset style, including: obtaining the current linear fitting coefficient, the initial current linear fitting coefficient It is preset; based on the current linear fitting coefficient and the face parameter values corresponding to the plurality of second face images of the preset style, predicting the current face parameter value of the first face image; based on The predicted current face parameter value and the face parameter value of the first face image determine a second loss value; Based on the second loss value and the preset constraint range corresponding to the linear fitting coefficient, adjust the current linear fitting coefficient; based on the adjusted current linear fitting coefficient, return to perform prediction of current face parameters value, until the adjustment operation to the current linear fitting coefficient meets the second adjustment cut-off condition, based on the current linear fitting coefficient, the first human face image and multiple second person images of the preset style are obtained Linear fit coefficient between face images. The processing method as described in claim 8, wherein the dense point cloud data includes the coordinate values of each point in the dense point cloud; the plurality of second human face images corresponding to the preset style respectively The dense point cloud data and the linear fitting coefficient determine the initial dense point cloud data of the target face in the preset style, including: corresponding to multiple second face images based on the preset style The coordinate values of each point in the dense point cloud data, determine the coordinate values of the corresponding points in the average dense point cloud data; the dense point cloud data respectively corresponding to a plurality of second face images based on the preset style Coordinate values of each point in and the coordinate values of corresponding points in the average dense point cloud data determine the coordinate difference values corresponding to the plurality of second face images of the preset style; based on the preset style The coordinate difference values corresponding to the plurality of second face images and the linear fitting coefficients are respectively determined to determine the coordinate difference values corresponding to the first face images; based on the coordinate difference values corresponding to the first face images and the average dense point The coordinate values of the corresponding points in the cloud data determine the initial dense point cloud data of the target face in the preset style. The processing method according to claim 7, wherein the face parameter values are extracted by a pre-trained neural network, and the neural network is trained based on sample images with pre-marked face parameter values. The processing method as described in claim 11, wherein the neural network is pre-trained in the following manner: a sample image set is obtained, and the sample image set includes multiple sample images and labeled human faces corresponding to each sample image Parameter value; the multiple sample images are input into the neural network to obtain the predicted face parameter value corresponding to each sample image; based on the predicted face parameter value and the marked face parameter value corresponding to each sample image, the The network parameter values of the neural network are adjusted to obtain a trained neural network. A processing device for a human face image, characterized in that it includes: an acquisition module for acquiring initial dense point cloud data of a target human face, and generating an initial virtual person of the target human face based on the initial dense point cloud data Face image; determination module, used to determine the deformation coefficient of the initial dense point cloud data relative to the standard dense point cloud data corresponding to the standard virtual human face image; adjustment module, used to respond to the initial virtual human face The adjustment operation of the image is to adjust the deformation coefficient to obtain the target deformation coefficient; A generating module, configured to generate a target virtual human face image corresponding to the target face based on the target deformation coefficient and the standard dense point cloud data; wherein the deformation coefficient includes at least one bone coefficient and at least one blending At least one of the deformation coefficients; wherein, each bone coefficient is used to adjust the initial pose of the bone formed by the first dense point cloud associated with the bone coefficient; each mixed deformation coefficient is used to adjust the The initial position corresponding to the second dense point cloud associated with the coefficient is adjusted. An electronic device, including a processor, a memory, and a bus bar, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory pass through Bus communication, when the machine-readable instructions are executed by the processor, the steps of the processing method described in any one of claims 1 to 12 are executed. A computer-readable storage medium, on which a computer program is stored, and the computer program executes the steps of the processing method described in any one of claims 1 to 12 when the computer program is run by a processor.

Images