CN116843838A - Human body grid model face changing method and device - Google Patents
Human body grid model face changing method and device Download PDFInfo
- Publication number
- CN116843838A CN116843838A CN202310835294.9A CN202310835294A CN116843838A CN 116843838 A CN116843838 A CN 116843838A CN 202310835294 A CN202310835294 A CN 202310835294A CN 116843838 A CN116843838 A CN 116843838A
- Authority
- CN
- China
- Prior art keywords
- face
- grid model
- human body
- model
- reconstructed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 44
- 238000005457 optimization Methods 0.000 claims abstract description 14
- 238000013507 mapping Methods 0.000 claims description 31
- 230000009466 transformation Effects 0.000 claims description 26
- 239000011159 matrix material Substances 0.000 claims description 16
- 239000003086 colorant Substances 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 238000001218 confocal laser scanning microscopy Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 230000003993 interaction Effects 0.000 abstract description 5
- 230000008859 change Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 230000001131 transforming effect Effects 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Abstract
The application provides a face changing method and device for a human body grid model, wherein the method comprises the following steps: acquiring a figure face picture and a preconfigured human body grid model; constructing a reconstructed face grid model according to the face picture of the person; replacing the face area of the human body grid model with a preset standard human body grid model to obtain an initial human body grid model; aligning the reconstructed face grid to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model; and performing iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model. Therefore, the method and the device can quickly change the face of the human body grid model, do not need manual interaction operation, can automatically operate, reduce the operation time, and further improve the processing efficiency of the human body grid model.
Description
Technical Field
The application relates to the technical field of computers, in particular to a human body grid model face changing method and device.
Background
Currently, with the popularity of virtual technology, digital mannequins are applied to different scenarios. Some intelligent scenes require fast generation of drivable manikins similar to the input face images so that animators can animate using the drivable manikins or drive the manikins in real time by collecting human motion data of the real person. In the existing method, a human body model with skeleton binding and texture mapping is obtained by inputting pictures into software through a Metahuman software generation method. However, in practice, it is found that the existing method requires manual interaction operation, cannot realize automatic operation, and has long operation time, thereby reducing the human body grid model processing efficiency.
Disclosure of Invention
The embodiment of the application aims to provide a human body grid model face changing method and device, which can quickly change the face of a human body grid model, do not need manual interaction operation, can automatically operate, reduce the operation time and further improve the human body grid model processing efficiency.
The first aspect of the embodiment of the application provides a human body grid model face changing method, which comprises the following steps:
acquiring a figure face picture and a preconfigured human body grid model;
constructing a reconstructed face grid model according to the figure face picture;
replacing the face area of the human body grid model with a preset standard human face grid model to obtain an initial human body grid model;
aligning the reconstructed face grid model to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model;
and carrying out iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model.
In the implementation process, the method can obtain the face picture of the person and the preconfigured human body grid model preferentially; then, constructing a reconstructed face grid model according to the face picture of the person; then, the face area of the human body grid model is replaced by a preset standard human face grid model, and an initial human body grid model is obtained; thirdly, aligning the reconstructed face grid to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model; and finally, carrying out iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model. Therefore, the method can quickly change the face of the human body grid model, does not need manual interaction operation, can automatically operate, reduces the operation time, and accordingly improves the human body grid model processing efficiency.
Further, the constructing a reconstructed face mesh model according to the face picture of the person includes:
processing the figure face picture through a preset depth network model to obtain an initial reconstructed face grid model comprising vertex colors;
performing two-dimensional expansion on the initial reconstructed face grid model by using a preset LSCM algorithm to obtain a mapping two-dimensional coordinate corresponding to the initial reconstructed face grid model; the mapping two-dimensional coordinates are corresponding two-dimensional coordinates after the three-dimensional vertex coordinates of the initial reconstructed face grid model are mapped to a two-dimensional plane;
generating a texture map according to the vertex colors and the mapping two-dimensional coordinates;
and carrying out texture mapping interpolation processing on the initial reconstructed face grid model according to the vertex colors and the texture mapping to obtain an interpolated reconstructed face grid model.
Further, the replacing the face area of the human body mesh model with a preset standard human face mesh model to obtain an initial human body mesh model includes:
determining boundary point data of the human body grid model and the reconstructed human face grid model according to boundary point selection instructions input by a user;
deleting a face area of the human body grid model according to the boundary point data;
obtaining a standard face grid model with the same topological structure as the reconstructed face grid model;
placing the standard face mesh model in a face region of the body mesh model;
merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model;
and re-binding the whole body skeleton of the single human body grid model to obtain an initial human body grid model.
Further, the aligning the reconstructed face mesh to the face region of the initial human body mesh model according to a preset ICP algorithm to obtain an aligned reconstructed face mesh model, including:
determining the initial human body mesh model as a target mesh of a preset ICP algorithm, and taking the reconstructed human face mesh model as a transformation mesh of the preset ICP algorithm;
acquiring a transformation matrix when the reconstructed human face grid model is transformed to a face area of the initial human body grid model through the preset ICP algorithm, the target grid and the transformation grid;
and generating an aligned reconstructed face grid model according to the transformation matrix and the reconstructed face grid model.
Further, the iterative optimization processing is performed on the initial human body mesh model and the aligned reconstruction human face mesh model to obtain an optimized target human body model, which comprises the following steps:
acquiring a vertex index mapping relation between the aligned reconstruction face mesh model and the initial human mesh model, a face mesh vertex set of the aligned reconstruction face mesh model and a human mesh vertex set of the initial human mesh model;
performing deformation processing on the initial human body grid model according to a preset iterative deformation algorithm and the aligned reconstruction human body grid model to obtain a deformed human body grid model so as to minimize the distance between the aligned reconstruction human body grid model and corresponding points of the initial human body grid model;
acquiring deformation vertex information of the deformation human body grid model and alignment vertex information of the alignment reconstruction human face grid model; the deformation vertex information comprises deformation vertex coordinates, deformation vertex texture coordinates and deformation vertex normal; the aligned vertex information comprises vertex coordinates and texture coordinates;
and replacing the vertex information of the initial human body grid model according to the deformation vertex information and the alignment vertex information to obtain an optimized target human body model.
A second aspect of an embodiment of the present application provides a face-changing device for a human body mesh model, the face-changing device for a human body mesh model including:
an acquisition unit for acquiring a figure face picture and a preconfigured human body grid model;
the construction unit is used for constructing a reconstructed face grid model according to the figure face picture;
the replacing unit is used for replacing the face area of the human body grid model with a preset standard human face grid model to obtain an initial human body grid model;
the alignment unit is used for aligning the reconstructed face grid model to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model;
and the optimization unit is used for carrying out iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model.
In the implementation process, the device can acquire the figure face picture and a preconfigured human body grid model through an acquisition unit; a reconstruction face grid model is constructed according to the face picture of the person through a construction unit; the face area of the human body grid model is replaced by a preset standard human face grid model through a replacement unit, and an initial human body grid model is obtained; the method comprises the steps that an alignment unit is used for aligning a reconstructed face grid model to an area where the face of an initial human body grid model is located according to a preset ICP algorithm, so that an aligned reconstructed face grid model is obtained; and performing iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model through an optimization unit to obtain an optimized target human body model. Therefore, the device can quickly change the face of the human body grid model, does not need manual interaction operation, can automatically operate, reduces the operation time, and accordingly improves the human body grid model processing efficiency.
Further, the construction unit includes:
the first processing subunit is used for processing the figure face picture through a preset depth network model to obtain an initial reconstructed face grid model comprising vertex colors;
the two-dimensional unfolding subunit is used for carrying out two-dimensional unfolding on the initial reconstruction face grid model by using a preset LSCM algorithm to obtain a mapping two-dimensional coordinate corresponding to the initial reconstruction face grid model; the mapping two-dimensional coordinates are corresponding two-dimensional coordinates after the three-dimensional vertex coordinates of the initial reconstructed face grid model are mapped to a two-dimensional plane;
a first generating subunit, configured to generate a texture map according to the vertex color and the mapped two-dimensional coordinate;
and the interpolation subunit is used for carrying out texture mapping interpolation processing on the initial reconstructed face grid model according to the vertex color and the texture mapping to obtain an interpolated reconstructed face grid model.
Further, the replacing unit includes:
the first determining subunit is used for determining the boundary point data of the human body grid model and the reconstructed human face grid model according to the boundary point selection instruction input by the user;
a deleting subunit, configured to delete a face area of the human body mesh model according to the boundary point data;
the first acquisition subunit is used for acquiring a standard face grid model with the same topological structure as the reconstructed face grid model;
a merging subunit, configured to place the standard face mesh model in a face area of the human mesh model; merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model; and re-binding the whole body skeleton of the single human body grid model to obtain an initial human body grid model.
Further, the alignment unit includes:
a second determining subunit, configured to determine the initial human body mesh model as a target mesh of a preset ICP algorithm, and use the reconstructed human face mesh model as a transformation mesh of the preset ICP algorithm;
an alignment subunit, configured to obtain, through the preset ICP algorithm, the target mesh, and the transformation mesh, a transformation matrix when transforming the reconstructed face mesh model to a face area of the initial human mesh model;
and the second generation subunit is used for generating an aligned reconstruction face grid model according to the transformation matrix and the reconstruction face grid model.
Further, the optimizing unit includes:
a second obtaining subunit, configured to obtain a vertex index mapping relationship between the aligned reconstructed face mesh model and the initial human mesh model, a face mesh vertex set of the aligned reconstructed face mesh model, and a human mesh vertex set of the initial human mesh model;
the second processing subunit is used for carrying out deformation processing on the initial human body grid model according to a preset iterative deformation algorithm and the alignment reconstruction human body grid model to obtain a deformed human body grid model so as to minimize the distance between corresponding points of the alignment reconstruction human body grid model and the initial human body grid model;
the second obtaining subunit is further configured to obtain deformation vertex information of the deformed human body mesh model and alignment vertex information of the aligned reconstructed human face mesh model; the deformation vertex information comprises deformation vertex coordinates, deformation vertex texture coordinates and deformation vertex normal; the aligned vertex information comprises vertex coordinates and texture coordinates;
and the optimizing subunit is used for replacing the vertex information of the initial human body grid model according to the deformation vertex information and the alignment vertex information to obtain an optimized target human body model.
A third aspect of the embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to execute the computer program to cause the electronic device to execute the face-changing method of the human body mesh model according to any one of the first aspect of the embodiment of the present application.
A fourth aspect of the embodiments of the present application provides a computer readable storage medium storing computer program instructions which, when read and executed by a processor, perform the face-changing method of the human body mesh model according to any one of the first aspect of the embodiments of the present application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a face-changing method of a human body mesh model according to an embodiment of the present application;
fig. 2 is a schematic flow chart of another face-changing method of a human body mesh model according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a face-changing device for a human body mesh model according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
Example 1
Referring to fig. 1, fig. 1 is a schematic flow chart of a face-changing method of a human body mesh model according to the present embodiment. The human body grid model face changing method comprises the following steps:
s101, acquiring a figure face picture and a preconfigured human body grid model.
S102, constructing a reconstructed face grid model according to the face picture of the person.
S103, replacing the face area of the human body grid model with a preset standard human face grid model to obtain an initial human body grid model.
And S104, aligning the reconstructed face grid model to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model.
S105, performing iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model.
In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, which is not limited in this embodiment.
In this embodiment, the execution body of the method may be an intelligent device such as a smart phone or a tablet computer, which is not limited in this embodiment.
Therefore, by implementing the human body grid model face changing method described in the embodiment, the human body grid model with skeleton binding and texture map information, the face area of which is similar to that of the input picture, can be quickly generated.
Example 2
Referring to fig. 2, fig. 2 is a schematic flow chart of a face-changing method of a human body mesh model according to the present embodiment. The human body grid model face changing method comprises the following steps:
s201, acquiring a figure face picture and a preconfigured human body grid model.
In this embodiment, the method may receive a single Zhang Wanzheng face picture.
S202, processing the face picture of the person through a preset depth network model to obtain an initial reconstructed face grid model comprising vertex colors.
In this embodiment, the method may output the corresponding reconstructed face mesh model with the vertex color through the depth network.
In this embodiment, the method may preferentially obtain a clear front face photo without shielding, and start subsequent work after outputting a reconstructed face mesh model with vertex color similar to the input picture through a depth network reconstructed by a functional non-face.
S203, performing two-dimensional expansion on the initial reconstructed face grid model by using a preset LSCM algorithm to obtain mapping two-dimensional coordinates corresponding to the initial reconstructed face grid model.
In this embodiment, the mapped two-dimensional coordinates are two-dimensional coordinates corresponding to the three-dimensional vertex coordinates of the initial reconstructed face mesh model after being mapped to the two-dimensional plane.
S204, generating a texture map according to the vertex colors and the mapped two-dimensional coordinates.
In this embodiment, the method may use an LSCM algorithm to perform two-dimensional expansion on the initial reconstructed face mesh model, obtain two-dimensional coordinates corresponding to when the three-dimensional vertex coordinates of the initial reconstructed face mesh model are mapped to the two-dimensional plane, and fill the color of the vertex into the two-dimensional coordinates corresponding to the vertex, so as to generate the texture map.
In this embodiment, the method may use an LSCM algorithm to perform two-dimensional plane expansion on the mesh model, so as to obtain two-dimensional coordinates corresponding to each vertex of the mesh model after the three-dimensional coordinates of each vertex are mapped to the two-dimensional plane, that is, texture coordinates, and store the texture coordinates of each vertex in the triangular mesh model, and output a reconstructed face mesh with texture coordinate information. And calculating the position of the coordinate on the full-white two-dimensional picture with the specified size according to the texture coordinate of each vertex, assigning the vertex color to the corresponding position of the full-white two-dimensional picture, and outputting the mapped two-dimensional picture, namely the unprocessed texture map.
And S205, performing texture mapping interpolation processing on the initial reconstructed face mesh model according to the vertex colors and the texture mapping to obtain the reconstructed face mesh model after interpolation.
In this embodiment, the method may interpolate the unprocessed texture map according to the vertex color of the initial reconstructed face mesh model to obtain an interpolated texture map, where the interpolated texture map is the texture map of the reconstructed face mesh model.
In this embodiment, the method may traverse each triangular patch of the initial reconstructed face mesh model, for the kth triangular patch t k Three vertices A, B, C are at corresponding pixels on the texture map. Extracting a triangle t 'surrounded by three pixel points' k Taking all pixel points with white color values as pixel point sets to be processed, and calculating the triangle t 'of each pixel point i to be processed' k Center of gravity coordinates (w) A ，w B ，w C ) Then, the triangle t 'is paired according to the value of the barycentric coordinates' k Vertex color C of three pixel vertices A ，C B ，C C Weighted summation is carried out to obtain a new color c of the pixel point i i The calculation formula is as follows:
c i ＝w A c A +w B c B +w C c C ；
and processing each triangular patch of the initial reconstructed face grid model according to the same steps to obtain the interpolated texture map.
S206, determining boundary point data of the human body grid model and the reconstructed human face grid model according to boundary point selection instructions input by a user.
In this embodiment, the method may select the corresponding points of the reconstructed face mesh model and the human mesh model by manual work.
In this embodiment, boundary points between the human body mesh model and the reconstructed human face mesh model obtained in the first step are manually specified, and index information of the boundary points in the model vertex set is stored. After the first manual assignment, the subsequent processing does not need to assign again, and the first data is directly used.
S207, deleting the face area of the human body grid model according to the boundary point data.
S208, acquiring a standard face grid model with the same topological structure as the reconstructed face grid model.
S209, placing the standard human face grid model in a face area of the human body grid model.
S210, merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model.
S211, rebinding the whole skeleton of the single human body grid model to obtain an initial human body grid model.
In this embodiment, after deleting the face area of the human body mesh model, the method may place the standard human face mesh model with the same topology as the reconstructed human face mesh model in the face area of the human body mesh model, and re-perform skeleton binding of the whole body.
In this embodiment, the method may delete an original face area from the human body mesh model according to the boundary points, and place a standard human face mesh model having the same topology structure as the reconstructed human face mesh model in the face area of the human body mesh model; then merging the standard human face grid model into a head grid model of the human body grid model to form a single grid model, re-binding the whole body skeleton of the human body grid model, and outputting the re-bound human body grid model; after the first manual assignment, the subsequent processing does not need to assign again, and the first data is directly used.
S212, determining the initial human body mesh model as a target mesh of a preset ICP algorithm, and taking the reconstructed human face mesh model as a transformation mesh of the preset ICP algorithm.
S213, obtaining a transformation matrix when transforming the reconstructed human face grid model to the face area of the initial human body grid model through a preset ICP algorithm, a target grid and a transformation grid.
S214, generating an aligned reconstructed face grid model according to the transformation matrix and the reconstructed face grid model.
In this embodiment, the method may use an ICP algorithm to align the reconstructed face mesh to the region of the face of the body mesh model.
In this embodiment, the method may use the rebind human body mesh model obtained in the previous step as an ICP algorithm target mesh, use the reconstructed human face mesh model as a transformation mesh of the ICP algorithm, transform the reconstructed human face mesh model to a face area of the human body mesh model through ICP iteration, record a transformation matrix of the reconstructed human face mesh, multiply the transformation matrix by vertices of the reconstructed human face mesh model to obtain an aligned reconstructed human face mesh model, and output the aligned reconstructed human face mesh model.
S215, obtaining the vertex index mapping relation of the aligned reconstruction face grid model and the initial human body grid model, a face grid vertex set of the aligned reconstruction face grid model and a human body grid vertex set of the initial human body grid model.
S216, performing deformation processing on the initial human body grid model according to a preset iterative deformation algorithm and the alignment reconstruction human body grid model to obtain a deformed human body grid model so as to minimize the distance between corresponding points of the alignment reconstruction human body grid model and the initial human body grid model.
In this embodiment, the method may minimize the distance between the reconstructed face mesh and the corresponding points of the mesh model by an iterative deformation algorithm.
In this embodiment, the method may be applied to vertex set v= { V in the human mesh model 1 ，v 2 ，……，v n Each vertex v of } i All have a corresponding transformation matrix R i So that the new transformed set of vertices V' = { R 1 v 1 ，R 2 v 2 ，......，R n v n The loss function formula can be minimized;
L＝L arap +L chamfer ；
wherein, the liquid crystal display device comprises a liquid crystal display device,
L arap representing deformation loss functions before and after matrix transformation, L chamfer Representing the minimum distance of the two model boundary points. I N (i) i represents the number of all neighboring points of the pixel point i, i B b The number of vertexes of the boundary point set of the human body grid model is the number of vertexes of the boundary point set of the human body grid model, and B is the number of vertexes of the boundary point set f The number of vertices of the boundary point set of the human mesh model is.
In this embodiment, the method may iteratively minimize the loss function by the SGD gradient descent method and update the transformation matrix R for each vertex by back propagation i Until the loss function is less than a threshold or the maximum number of iterations is reached; multiplying the transformation matrix by the vertexes of the reconstructed face mesh model to obtain an aligned reconstructed face mesh model, and outputting the aligned reconstructed face mesh model.
S217, obtaining deformation vertex information of the deformation human body grid model and alignment vertex information of the alignment reconstruction human face grid model.
In this embodiment, the deformation vertex information includes deformation vertex coordinates, deformation vertex texture coordinates, and deformation vertex normal.
In this embodiment, the aligned vertex information includes vertex coordinates and texture coordinates.
And S218, replacing the vertex information of the initial human body grid model according to the deformation vertex information and the alignment vertex information to obtain the optimized target human body model.
In this embodiment, the method may update the vertex coordinates and texture coordinates of the reconstructed face mesh model to the vertex positions corresponding to the human mesh model according to the vertex index mapping relationship between the reconstructed face mesh model and the initial human mesh model, and update other vertex positions of the human mesh model.
In this embodiment, the method may replace the vertex coordinates, vertex texture coordinates and vertex normal of the original human body mesh model with the human body mesh model after the deformation obtained in the previous step; and replacing the vertex coordinates and the texture coordinates of the face area of the original human body grid model by using the vertex coordinates and the texture coordinates of the reconstructed human body grid model, and outputting the human body grid model with updated information.
By implementing the implementation mode, the loss of binding information can be avoided, the human body model with high similarity is generated, and the algorithm has the advantages of full automation and high speed.
In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, which is not limited in this embodiment.
In this embodiment, the execution body of the method may be an intelligent device such as a smart phone or a tablet computer, which is not limited in this embodiment.
Therefore, by implementing the human body grid model face changing method described in the embodiment, the human body grid model with skeleton binding and texture map information, the face area of which is similar to that of the input picture, can be quickly generated.
Example 3
Referring to fig. 3, fig. 3 is a schematic structural diagram of a face-changing device for a human body mesh model according to the present embodiment. As shown in fig. 3, the human body mesh model face changing device includes:
an acquiring unit 310, configured to acquire a face picture of a person and a preconfigured human body mesh model;
a construction unit 320, configured to construct a reconstructed face mesh model according to the face picture of the person;
a replacing unit 330, configured to replace a face area of the human body mesh model with a preset standard human face mesh model, to obtain an initial human body mesh model;
an alignment unit 340, configured to align the reconstructed face mesh model to an area where the face of the initial human mesh model is located according to a preset ICP algorithm, so as to obtain an aligned reconstructed face mesh model;
and the optimizing unit 350 is configured to perform iterative optimization processing on the initial human body mesh model and the aligned reconstructed human face mesh model, so as to obtain an optimized target human body model.
As an alternative embodiment, the construction unit 320 includes:
the first processing subunit 321 is configured to process the face picture of the person through a preset depth network model to obtain an initial reconstructed face mesh model including vertex colors;
a two-dimensional unfolding subunit 322, configured to use a preset LSCM algorithm to perform two-dimensional unfolding on the initial reconstructed face mesh model, so as to obtain a mapped two-dimensional coordinate corresponding to the initial reconstructed face mesh model; the mapping two-dimensional coordinates are corresponding two-dimensional coordinates after the three-dimensional vertex coordinates of the initial reconstructed face grid model are mapped to the two-dimensional plane;
a first generating subunit 323, configured to generate a texture map according to the vertex colors and the mapped two-dimensional coordinates;
the interpolation subunit 324 is configured to perform texture mapping interpolation on the initial reconstructed face mesh model according to the vertex color and the texture mapping, so as to obtain an interpolated reconstructed face mesh model.
As an alternative embodiment, the replacement unit 330 includes:
a first determining subunit 331, configured to determine, according to a boundary point selection instruction input by a user, boundary point data of the human body mesh model and the reconstructed human face mesh model;
a deleting subunit 332, configured to delete the face area of the human body mesh model according to the boundary point data;
a first obtaining subunit 333, configured to obtain a standard face mesh model with the same topology structure as the reconstructed face mesh model;
a merging subunit 334, configured to place the standard face mesh model in a face area of the human mesh model; merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model; and rebinding the whole body skeleton of the single body grid model to obtain an initial body grid model.
As an alternative embodiment, the alignment unit 340 includes:
a second determining subunit 341, configured to determine the initial human body mesh model as a target mesh of a preset ICP algorithm, and use the reconstructed human face mesh model as a transformed mesh of the preset ICP algorithm;
an alignment subunit 342, configured to obtain a transformation matrix when transforming the reconstructed face mesh model to the face area of the initial human mesh model by presetting an ICP algorithm, a target mesh, and a transformation mesh;
the second generating subunit 343 is configured to generate an aligned reconstructed face mesh model according to the transformation matrix and the reconstructed face mesh model.
As an alternative embodiment, the optimizing unit 350 includes:
a second obtaining subunit 351, configured to obtain a vertex index mapping relationship between the aligned reconstructed face mesh model and the initial human mesh model, a human mesh vertex set of the aligned reconstructed face mesh model, and a human mesh vertex set of the initial human mesh model;
the second processing subunit 352 is configured to perform deformation processing on the initial human body mesh model according to a preset iterative deformation algorithm and an alignment reconstruction human face mesh model, so as to obtain a deformed human body mesh model, so as to minimize a distance between corresponding points of the alignment reconstruction human face mesh model and the initial human body mesh model;
the second obtaining subunit 351 is further configured to obtain deformation vertex information of the deformed human body mesh model and alignment vertex information of the aligned reconstructed human face mesh model; the deformation vertex information comprises deformation vertex coordinates, deformation vertex texture coordinates and deformation vertex normal; the aligned vertex information includes vertex coordinates and texture coordinates;
and the optimizing subunit 353 is configured to replace vertex information of the initial human body mesh model according to the deformation vertex information and the alignment vertex information, so as to obtain an optimized target human body model.
In this embodiment, the explanation of the face-changing device of the human body mesh model may refer to the description in embodiment 1 or embodiment 2, and no redundant description is given in this embodiment.
Therefore, by implementing the human body grid model face changing device described in the embodiment, the human body grid model with skeleton binding and texture map information, the face area of which is similar to that of the input picture, can be quickly generated.
An embodiment of the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a computer program, and the processor runs the computer program to cause the electronic device to execute a face-changing method of a human body mesh model in embodiment 1 or embodiment 2 of the present application.
An embodiment of the present application provides a computer readable storage medium storing computer program instructions that, when read and executed by a processor, perform the face-changing method of the human body mesh model in embodiment 1 or embodiment 2 of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, of the flowcharts and block diagrams in the figures that illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and variations will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A human body mesh model face changing method, comprising:
acquiring a figure face picture and a preconfigured human body grid model;
constructing a reconstructed face grid model according to the figure face picture;
replacing the face area of the human body grid model with a preset standard human face grid model to obtain an initial human body grid model;
aligning the reconstructed face grid model to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model;
and carrying out iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model.
2. The face changing method of a human body grid model according to claim 1, wherein the constructing a reconstructed human face grid model according to the human face picture comprises:
processing the figure face picture through a preset depth network model to obtain an initial reconstructed face grid model comprising vertex colors;
performing two-dimensional expansion on the initial reconstructed face grid model by using a preset LSCM algorithm to obtain a mapping two-dimensional coordinate corresponding to the initial reconstructed face grid model; the mapping two-dimensional coordinates are corresponding two-dimensional coordinates after the three-dimensional vertex coordinates of the initial reconstructed face grid model are mapped to a two-dimensional plane;
generating a texture map according to the vertex colors and the mapping two-dimensional coordinates;
and carrying out texture mapping interpolation processing on the initial reconstructed face grid model according to the vertex colors and the texture mapping to obtain an interpolated reconstructed face grid model.
3. The face changing method of a human body mesh model according to claim 1, wherein the replacing the face area of the human body mesh model with a preset standard human face mesh model to obtain an initial human body mesh model comprises:
determining boundary point data of the human body grid model and the reconstructed human face grid model according to boundary point selection instructions input by a user;
deleting a face area of the human body grid model according to the boundary point data;
obtaining a standard face grid model with the same topological structure as the reconstructed face grid model;
placing the standard face mesh model in a face region of the body mesh model;
merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model;
and re-binding the whole body skeleton of the single human body grid model to obtain an initial human body grid model.
4. The face changing method of a human body mesh model according to claim 1, wherein the aligning the reconstructed human face mesh to the face region of the initial human body mesh model according to a preset ICP algorithm to obtain an aligned reconstructed human face mesh model includes:
determining the initial human body mesh model as a target mesh of a preset ICP algorithm, and taking the reconstructed human face mesh model as a transformation mesh of the preset ICP algorithm;
acquiring a transformation matrix when the reconstructed human face grid model is transformed to a face area of the initial human body grid model through the preset ICP algorithm, the target grid and the transformation grid;
and generating an aligned reconstructed face grid model according to the transformation matrix and the reconstructed face grid model.
5. The face changing method of human body mesh model according to claim 4, wherein the iterative optimization processing is performed on the initial human body mesh model and the aligned reconstructed human face mesh model to obtain an optimized target human body model, and the method comprises:
acquiring a vertex index mapping relation between the aligned reconstruction face mesh model and the initial human mesh model, a face mesh vertex set of the aligned reconstruction face mesh model and a human mesh vertex set of the initial human mesh model;
performing deformation processing on the initial human body grid model according to a preset iterative deformation algorithm and the aligned reconstruction human body grid model to obtain a deformed human body grid model so as to minimize the distance between the aligned reconstruction human body grid model and corresponding points of the initial human body grid model;
acquiring deformation vertex information of the deformation human body grid model and alignment vertex information of the alignment reconstruction human face grid model; the deformation vertex information comprises deformation vertex coordinates, deformation vertex texture coordinates and deformation vertex normal; the aligned vertex information comprises vertex coordinates and texture coordinates;
and replacing the vertex information of the initial human body grid model according to the deformation vertex information and the alignment vertex information to obtain an optimized target human body model.
6. A human mesh model face changing device, characterized in that the human mesh model face changing device comprises:
an acquisition unit for acquiring a figure face picture and a preconfigured human body grid model;
the construction unit is used for constructing a reconstructed face grid model according to the figure face picture;
the replacing unit is used for replacing the face area of the human body grid model with a preset standard human face grid model to obtain an initial human body grid model;
the alignment unit is used for aligning the reconstructed face grid model to the area where the face of the initial human body grid model is positioned according to a preset ICP algorithm to obtain an aligned reconstructed face grid model;
and the optimization unit is used for carrying out iterative optimization processing on the initial human body grid model and the aligned reconstruction human face grid model to obtain an optimized target human body model.
7. The human mesh model face changing apparatus according to claim 6, wherein the construction unit includes:
the first processing subunit is used for processing the figure face picture through a preset depth network model to obtain an initial reconstructed face grid model comprising vertex colors;
the two-dimensional unfolding subunit is used for carrying out two-dimensional unfolding on the initial reconstruction face grid model by using a preset LSCM algorithm to obtain a mapping two-dimensional coordinate corresponding to the initial reconstruction face grid model; the mapping two-dimensional coordinates are corresponding two-dimensional coordinates after the three-dimensional vertex coordinates of the initial reconstructed face grid model are mapped to a two-dimensional plane;
a first generating subunit, configured to generate a texture map according to the vertex color and the mapped two-dimensional coordinate;
and the interpolation subunit is used for carrying out texture mapping interpolation processing on the initial reconstructed face grid model according to the vertex color and the texture mapping to obtain an interpolated reconstructed face grid model.
8. The human mesh model face changing apparatus according to claim 6, wherein the replacing unit includes:
the first determining subunit is used for determining the boundary point data of the human body grid model and the reconstructed human face grid model according to the boundary point selection instruction input by the user;
a deleting subunit, configured to delete a face area of the human body mesh model according to the boundary point data;
the first acquisition subunit is used for acquiring a standard face grid model with the same topological structure as the reconstructed face grid model;
a merging subunit, configured to place the standard face mesh model in a face area of the human mesh model; merging the standard human face grid model into a head grid model of the human body grid model to obtain a single human body grid model; and re-binding the whole body skeleton of the single human body grid model to obtain an initial human body grid model.
9. An electronic device comprising a memory for storing a computer program and a processor that runs the computer program to cause the electronic device to perform the human mesh model face-changing method of any one of claims 1 to 5.
10. A readable storage medium having stored therein computer program instructions which, when read and executed by a processor, perform the human mesh model face-changing method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310835294.9A CN116843838A (en) | 2023-07-07 | 2023-07-07 | Human body grid model face changing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310835294.9A CN116843838A (en) | 2023-07-07 | 2023-07-07 | Human body grid model face changing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116843838A true CN116843838A (en) | 2023-10-03 |
Family
ID=88170396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310835294.9A Pending CN116843838A (en) | 2023-07-07 | 2023-07-07 | Human body grid model face changing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116843838A (en) |
-
2023
- 2023-07-07 CN CN202310835294.9A patent/CN116843838A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3279803B1 (en) | Picture display method and device | |
| Li et al. | Detail-preserving and content-aware variational multi-view stereo reconstruction | |
| CN101529468B (en) | System and method for recovering three-dimensional particle systems from two-dimensional images | |
| Perra et al. | An analysis of 3D point cloud reconstruction from light field images | |
| EP3326156B1 (en) | Consistent tessellation via topology-aware surface tracking | |
| KR102169243B1 (en) | Semantic segmentation method of 3D reconstructed model using incremental fusion of 2D semantic predictions | |
| CN111612882B (en) | Image processing method, image processing device, computer storage medium and electronic equipment | |
| CN115409933B (en) | Multi-style texture mapping generation method and device | |
| CN113112581A (en) | Texture map generation method, device and equipment for three-dimensional model and storage medium | |
| CN115546371A (en) | Point cloud optimization method and system, electronic device and storage medium | |
| CN114981845A (en) | Image scanning method and device, equipment and storage medium | |
| Guichard et al. | Contrast invariant image analysis and PDE’s | |
| CN115375847B (en) | Material recovery method, three-dimensional model generation method and model training method | |
| CN109166176B (en) | Three-dimensional face image generation method and device | |
| CN116758219A (en) | Region-aware multi-view stereo matching three-dimensional reconstruction method based on neural network | |
| CN116843838A (en) | Human body grid model face changing method and device | |
| CN115082640A (en) | Single image-based 3D face model texture reconstruction method and equipment | |
| Zhu et al. | Textured mesh surface reconstruction of large buildings with multi-view stereo | |
| TW202312100A (en) | Grid generation method, electronic device and computer-readable storage medium | |
| CN108921908B (en) | Surface light field acquisition method and device and electronic equipment | |
| Chu et al. | Hole-filling framework by combining structural and textural information for the 3D Terracotta Warriors | |
| CN113496468A (en) | Method and device for restoring depth image and storage medium | |
| Ling et al. | Large-scale and efficient texture mapping algorithm via loopy belief propagation | |
| KR100684558B1 (en) | Texture mipmapping device and the same method | |
| CN116993924B (en) | Three-dimensional scene modeling method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |