CN111476901B - Three-dimensional human body shape representation method - Google Patents

Three-dimensional human body shape representation method Download PDF

Info

Publication number
CN111476901B
CN111476901B CN202010279674.5A CN202010279674A CN111476901B CN 111476901 B CN111476901 B CN 111476901B CN 202010279674 A CN202010279674 A CN 202010279674A CN 111476901 B CN111476901 B CN 111476901B
Authority
CN
China
Prior art keywords
human body
deformation
grid
reconstruction
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010279674.5A
Other languages
Chinese (zh)
Other versions
CN111476901A (en
Inventor
张举勇
江博艺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202010279674.5A priority Critical patent/CN111476901B/en
Publication of CN111476901A publication Critical patent/CN111476901A/en
Application granted granted Critical
Publication of CN111476901B publication Critical patent/CN111476901B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/44—Morphing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a three-dimensional human body shape representation method, on one hand, a large amount of labeled training data can be obtained by processing an acquired human body grid data set, and the robustness of a model is increased; on the other hand, by using the deformation representation and defining the deformation representation based on the human body approximate rigid block, higher precision and robustness can be achieved compared with the mode that the Euclidean distance is directly used. Meanwhile, a hierarchical reconstruction network is designed aiming at the hinge type deformation of the human body, the shape prior of the human body is fully utilized, and the accuracy of the model is improved.

Description

Three-dimensional human body shape representation method
Technical Field
The invention relates to the technical field of human body three-dimensional reconstruction, in particular to a three-dimensional human body shape representation method.
Background
The parameterized human body model has wide application in the fields of computer graphics and computer vision, including three-dimensional human body tracking, three-dimensional human body reconstruction and attitude estimation. Since the human body has various attributes including gender, race, posture, body type, and the like, which contain abundant geometric deformation, high-precision reconstruction of the human body is a challenging task. In recent years, with the development of deep learning, geometric reconstruction using a neural network has become a trend.
In the past, human reconstruction was primarily based on the skeletal skinning method. The method expresses the identity deformation space of a human body by a group of simple linear basis deformations, and then deforms the human body to a specific posture by using a skeleton. The method is simple and high in efficiency in calculation, decoupling of the identity and the posture is achieved, high-frequency change of a human identity deformation space is ignored, and geometric reconstruction accuracy is limited. In addition, motion has a relative motion representation of skeletal joint points, without encoding a prior distribution of human motion, and thus may produce an abnormal human model. The other main method is based on deformation expression, typically, a three-dimensional point coordinate is replaced by triangular deformation, and decoupling parametric modeling of a human body is realized by decomposing the deformation into parts related to various attributes such as identity, posture and the like. The method has higher geometric reconstruction accuracy, but the calculation process is more complex, the parameter quantity is large, and the speed is slower.
Recently, learning some shapes like human faces using neural networks has been developed. Through end-to-end training, the network maps the shape manifold to a low-dimensional nonlinear space, which shows higher precision. But the human body has significant large-scale deformation caused by different poses compared to the human face. A network architecture which achieves good accuracy in face modeling is directly applied to a human body grid, and a good result cannot be obtained. This is because the european coordinate is not easy to maintain the original deformation mode for large-scale deformation, and is easy to cause distortion. On the other hand, the network architecture is not designed for a special skeleton hinge structure of a human body, and the precision is not improved by using the prior of the shape of the human body.
Disclosure of Invention
The invention aims to provide a three-dimensional human body shape representation method which can improve the reconstruction precision of a three-dimensional human body model.
The purpose of the invention is realized by the following technical scheme:
a three-dimensional human body shape representation method comprising:
preprocessing the collected human body grid data set, deforming based on a standard posture, and calculating ACAP deformation representing and describing human body deformation characteristics of rigid block deformation to form a training data set;
constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain an identity attribute and an action attribute, a three-dimensional human body model is reconstructed through the reconstruction network by utilizing the identity attribute and the action attribute, and the coder network and a hierarchical reconstruction network are trained by utilizing an error between a reconstruction result and input training data;
and after the training is finished, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result.
According to the technical scheme provided by the invention, 1) a large amount of labeled training data can be obtained by processing the acquired human body grid data set, so that the robustness of the model is improved. 2) By using the deformation representation and defining a deformation representation based on human body approximate rigid blocks, higher precision and robustness can be achieved compared with the mode that Euclidean distance is directly used. 3) Aiming at the hinge type deformation of the human body, a hierarchical reconstruction network is designed, the shape prior of the human body is fully utilized, and the precision of the model is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a method for representing a three-dimensional human body shape according to an embodiment of the present invention;
FIG. 2 is a reference body grid for calculating ACAP (consistent deformation) features and rigid blocks defined by calculating large-scale features g of a body according to an embodiment of the present invention;
fig. 3 is a schematic visualization diagram of reconstruction of each phase of a neutral posture and an original posture generated by a reconstruction network according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
In the field of human body parametric representation, the traditional skeleton skin-based method has high model speed and strong action expression capability, but the reconstruction precision is limited because simple linear dimension reduction is only carried out on the deformation space of the identity; on the other hand, the representation of the motion is directly based on the relative motion of the joint points and does not limit the rationality of the human motion. To this end, an embodiment of the present invention provides a method for representing a three-dimensional human body shape, as shown in fig. 1, the method mainly includes:
and 11, preprocessing the collected human body grid data set, deforming based on the standard posture, and calculating the ACAP deformation expression and the human body deformation characteristics describing the deformation of the rigid blocks to form a training data set.
The preferred embodiment of this step is as follows:
1) And carrying out standardization processing on the collected human body grid data set to obtain human body grid data with unified topology, and deforming to obtain a neutral human body grid corresponding to each human body grid data through the defined standard posture.
In the embodiment of the invention, the original human body grid data can be obtained from the network, including SCAPE, FAUST, dyna, MANO and the like. For collected human mesh data sets of different sources, the mesh representations of the collected human mesh data sets are inconsistent and need to be converted into a standard topology G = { V, E }, where V is a vertex set and E is an edge set. In this embodiment, a source data is used as a standard topology (e.g., SCAPE); calculating corresponding ACAP (consistent deformation) deformation representation by using action grids in standard topology (for example, seventy action grids in SCAPE), obtaining a group of priori deformation representation bases C of human body actions, and recovering the deformation representation of a human body by using a group of parameters w; and (3) using the linear space of Cw as a priori space of human body deformation, then optimizing a group of vertex coordinates p and rigid transformation parameters of standard topology, namely a rotation parameter R and a translation parameter t, and standardizing human body grid data sets from different sources.
The normalization process is to solve the following optimization problem:
Figure BDA0002446089520000031
wherein λ is 1 、λ 2 、λ 3 Are all set weights; | w | charging 1 Is a sparse regularization constraint on parameter w;
E prior is a human body deformation prior term determined by a prior deformation representation base C, so that the optimized grid vertex conforms to the human body shape as much as possible, and is represented as follows:
Figure BDA0002446089520000032
wherein, T i (w) is the prior deformation of a neighborhood of the ith vertex in the standard topology, the prior deformation represents that the base C is multiplied by w to obtain an ACAP characteristic, and then the ith vertex component of the ACAP characteristic is converted to obtain T i (w)；q i Is the position of the ith vertex in the deformed reference grid under the standard topology; relative to the deformation reference grid, the position of the ith vertex of the grid to be optimized under the standard topology is p i (ii) a N (i) refers to a neighborhood vertex index set of the ith vertex under the standard topology, j refers to the jth vertex in the N (i), and the positions of the corresponding vertices in the deformation reference grid and the grid to be optimized are respectively represented as q j 、p j ；c ij Is an edge weight value calculated on the deformation reference grid, called cotangent weight, and is specifically q j And q is i The edge weight value of (2). In the embodiment of the invention, the deformation reference grid is a human body grid required for calculating ACAP deformation expression, and can be selected according to actual conditions or experience; as shown in fig. 2, the pre-selected warped reference grid is on the left. Those skilled in the art will appreciate that the deformed reference mesh corresponds one-to-one to all vertices of the standard topology, except for the different locations of the vertices; in the embodiment of the invention, the deformation reference gridFor reference, optimizing a grid under a standard topology; that is, q i Is stationary, p i Is optimized and the value is changed.
E icp Is a point-to-plane registration energy term with the nearest neighbor of the target mesh such that the optimization result is close to the target, expressed as:
Figure BDA0002446089520000041
d is an index set of the corresponding point pair selected by dynamic calculation under the standard topology; v. of l(i) Is and p i The corresponding point on the corresponding target grid,
Figure BDA0002446089520000042
representing point v l(i) Normal direction of (2); the target grid refers to human body grid data sets of different sources to be optimized;
E lan the L2 loss energy terms of a group of manually marked sparse corresponding points of the standard topology and the target grid are expressed as follows to reduce sliding errors and avoid local minimum values:
Figure BDA0002446089520000043
where L is a pre-selected set of corresponding points in a standard topology, the above equation limits the spatial location of the points in the pre-selected set of corresponding points L to be as small as possible.
In the embodiment of the invention, the human body grid data fitted based on the mode has consistent standard topology, and has tolerable registration error with the original data.
The human body model data constructed on the basis also needs to define the corresponding neutral human body grids. Illustratively, a consistent a-gesture in the SPRING dataset may be used as the neutral gesture. And for all the human body data of each identity, selecting a human body with the minimum error with the SPRING average grid as a candidate human body grid, and then converting the candidate human body grid into the posture A by using an ARAP (as rigid as possible) deformation method to be used as a neutral human body grid of the human body of the identity.
2) Calculating ACAP deformation representation of each human body grid data and corresponding neutral human body grid, and recording as f and f s (ii) a And respectively calculating human body deformation characteristics g and g of the human body grid and the corresponding neutral human body grid describing rigid block deformation according to the hinge type deformation characteristics of the human body based on the skeleton s To obtain a set of training data { f, f } s ,g,g s And correspondingly processing the collected data to obtain a training data set.
By the method, the human body grid data with the uniform topology are obtained, and each identity human body has a neutral posture grid. In the embodiment of the invention, the deformation expression of the geometric shape of the ACAP is used for replacing the original Euclidean coordinate shape expression to enhance the modeling precision of large-scale deformation and obtain better performance compared with a general linear model.
The formula for calculating the human body grid data and the ACAP deformation expression of the corresponding neutral human body grid is as follows:
Figure BDA0002446089520000051
similarly, i represents a vertex in the standard topology, and the above is standardized, so the standard topology can be any standard topology after being standardized or a previously selected standard topology. The other parameters have the same meanings as above and are not described in detail.
T in the above formula i Refers to the affine transformation matrix of the ith vertex in the standard topology, the affine transformation matrix T i The method comprises the steps of transforming a local umbrella-shaped structure of a neighborhood of a deformation reference grid into various deformation information of a structure of a computational grid; by polar decomposition, T i Decomposition to rigid R i And a non-rigid part S i (determined by 3 and 6 degrees of freedom, respectively), after disambiguation of the rigid deformation part of each vertex, the ACAP deformation of the computational mesh is obtainedRepresenting; the dimensions of the representation are 9 times the number of vertices, each vertex having 3 and 6 parametric record rigid and non-rigid deformations, respectively.
Taking each human body grid data and corresponding neutral human body grid as calculation grid to be substituted into the formula to obtain corresponding ACAP deformation expressions f and f s 。
In addition, some approximately rigid parts, such as the lower arm, the head, etc., are defined on the human body in consideration of the skeleton-based hinged deformation characteristics of the human body. On the basis, a deformation characteristic g of a large scale of a human body for describing the deformation of the rigid block is defined, and the calculation formula is as follows:
Figure BDA0002446089520000052
wherein v is k Is the set of vertices of the kth rigid block, q i' 、p i' Respectively representing the positions of the ith' vertex on the kth rigid block in the deformation reference grid and the calculation grid;
Figure BDA0002446089520000053
and &>
Figure BDA0002446089520000054
Respectively averaging the k-th rigid block on the deformation reference grid and the calculation grid; by means of radiation deformation of a rigid block>
Figure BDA0002446089520000055
And performing polar decomposition and parameterization to obtain the human body deformation characteristic, wherein the dimension of the human body deformation characteristic is 9 times of the number of approximate rigid blocks of the human body.
Similarly, each human body grid data and corresponding neutral human body grid are taken as calculation grids and are substituted into the formula to obtain deformation characteristics g and g s 。
As shown in fig. 2, the rigid blocks defined for calculating the deformation reference grid (left side of fig. 2) and the large-scale features of the human body (right side of fig. 2) are represented for ACAP deformation.
Step 12, constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain identity attributes and action attributes, the identity attributes and the action attributes are utilized to reconstruct a three-dimensional human body model through a reconstruction network, and errors between a reconstruction result and input training data are utilized to train the coder network and the hierarchical reconstruction network.
In the embodiment of the invention, the encoder network can adopt a standard variational self-encoder structure to learn the identity attribute e from the ACAP deformation expression f of the human body grid data s And attitude attribute e p Then, an end-to-end network structure is formed by combining the reconstructed network for training. Training data { f, f s ，g，g s Only f is used as the input of the network and the rest of the data is used to calculate the reconstruction error.
In the embodiment of the invention, the hierarchical reconstruction network is based on human body geometric prior and comprises two parts, wherein the first part is used for hierarchically reconstructing a main part of a three-dimensional human body model, the second part is used for reconstructing a difference part, and the reconstructed main part and the reconstructed difference part are added to obtain a reconstruction result; the reconstructed network is described as:
Figure BDA0002446089520000061
Figure BDA0002446089520000062
wherein the content of the first and second substances,
Figure BDA0002446089520000063
representation utilization identity attribute e s And attitude attribute e p In conjunction with the reconstruction result of (a), based on the result of (b)>
Figure BDA0002446089520000064
Representation utilization identity attribute e s The reconstruction result of (2); />
Figure BDA0002446089520000065
Indicates a reconstruction result->
Figure BDA0002446089520000066
The main part b in (1), W represents a skin layer; />
Figure BDA0002446089520000067
Represents the result of a reconstruction>
Figure BDA0002446089520000068
The difference part d in (1); />
Figure BDA0002446089520000069
Indicates a reconstruction result->
Figure BDA00024460895200000610
Main part b of (1) s ，/>
Figure BDA00024460895200000611
Representing a skin layer;
Figure BDA00024460895200000612
represents the result of a reconstruction>
Figure BDA00024460895200000613
The difference part d in s ；
Reconstruction of the main parts b and b s The following equation is used to reconstruct the human deformation characteristics
Figure BDA00024460895200000623
And &>
Figure BDA00024460895200000624
Figure BDA00024460895200000614
Figure BDA00024460895200000615
Wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00024460895200000616
are independent mapping transformations, which may be modeled, for example, using a multi-tier perceptron.
Then, the skin layer is utilized
Figure BDA00024460895200000617
Reconstruction of the main parts b and b s ：
Figure BDA00024460895200000618
Figure BDA00024460895200000619
Wherein the skin layer
Figure BDA00024460895200000620
In the form of a matrix, is selected>
Figure BDA00024460895200000621
Represents->
Figure BDA00024460895200000622
The xth row and the y column; y is the number of rigid blocks, e.g., Y =16; the above principle is that the deformation of each vertex is obtained by linear convex combination of the relative rigid block deformation of the vertex, and the coefficient of the convex combination is determined by learning training and is not set manually.
Difference parts d and d s Expressed as:
Figure BDA0002446089520000071
Figure BDA0002446089520000072
wherein the content of the first and second substances,
Figure BDA0002446089520000073
independent mapping transformations, illustratively, these mapping transformations may be modeled using a multi-tier perceptron.
As shown in fig. 3, an example of the visualization of the various computed portions of the reconstructed network over one reconstructed instance is given. B (e) in FIG. 3 s ，e p )、B(e s 0) i.e. main parts b and b as mentioned herein s 。
In the embodiment of the invention, the network can be trained end to end through the training data set, and after the training is finished, the decoupled low-dimensional hidden layer can be used for representing e s ，e p The method can be used for reconstructing a human body (or applied to aspects of human body editing, motion migration and the like), can also be applied to aspects of human body editing, motion migration and the like, and has wide application prospects in the fields of video live broadcast, virtual fitting, somatosensory games and the like.
During training, L1 mode loss between reconstruction and input features and distribution regularization loss of hidden layer variables can be adopted as loss functions.
The L1 mode loss can be expressed as:
Figure BDA0002446089520000074
Figure BDA0002446089520000075
in the above-mentioned formula, the compound has the following structure,
Figure BDA0002446089520000076
all are the reconstruction results of the relevant data in the training data, and the specific values 9 (number of deformation features) and 16 (number of rigid blocks) involved in the above formula) Are by way of example only and are not limiting.
The regular partial loss of the hidden layer parameter distribution is:
E sKL ＝D KL (q(e s |f)||p(e s ))
E pKL ＝D KL (q(e p |f)||p(e p ))
the two losses are KL divergence losses of which the distribution of the standard constraint hidden layer in the variational self-encoder meets the prior distribution.
And step 13, after training, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result.
After the network training is completed through the step 12, the encoder network can be abandoned, and the reconstructed network is directly used as a decoupled low-dimensional human body parameterized model. The model reconstructs a complete human body grid from two groups of decoupling parameters respectively representing identity and posture. Specifically, the trained hierarchical reconstruction network may reconstruct the identity attribute and the posture attribute of the input data according to the method introduced in the training phase
Figure BDA0002446089520000081
And/or>
Figure BDA0002446089520000082
Both are represented by ACAP deformation, and the corresponding human body grid and the neutral human body grid can be obtained through a simple conversion. The conversion method referred to herein can be referred to in the prior art, and is not described in detail.
Compared with the traditional human body parameterized model representation method, the scheme of the embodiment of the invention mainly has the following advantages:
1) The characteristics of input and output are represented by nonlinear deformation, the traditional Euclidean coordinates are replaced, the precision is higher, and the deformation with large scale is more robust.
2) The reconstruction accuracy of the model is further improved by utilizing the strong fitting capability of the neural network and combining the framework design of human body deformation prior.
3) The obtained posture hidden layer representation has certain semantics by utilizing the learning of a large number of human body models with various postures, namely embedding reasonable actions of the human body into a low-dimensional space. However, the posture parameters of the conventional model often have no semantics and may generate unreasonable human body actions.
Through the description of the above embodiments, it is clear to those skilled in the art that the above embodiments may be implemented by software, or by software plus a necessary general hardware platform. With this understanding, the technical solutions of the embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (3)

1. A three-dimensional human body shape representation method, comprising:
preprocessing the collected human body grid data set, deforming based on a standard posture, and calculating ACAP deformation representing and describing human body deformation characteristics of rigid block deformation to form a training data set;
constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain an identity attribute and an action attribute, a three-dimensional human body model is reconstructed through the reconstruction network by utilizing the identity attribute and the action attribute, and the coder network and a hierarchical reconstruction network are trained by utilizing an error between a reconstruction result and input training data;
after training, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result;
the preprocessing is performed on the collected human body mesh data set, and the training data set formed by the ACAP deformation expression and the human body deformation characteristic is obtained by the method comprising the following steps:
firstly, carrying out standardization processing on a collected human body grid data set to obtain human body grid data with unified topology, and deforming to obtain a neutral human body grid corresponding to each human body grid data through a defined standard posture;
then, calculating ACAP deformation representation of each human body grid data and corresponding neutral human body grid, and recording the ACAP deformation representation as f and f s (ii) a And respectively calculating human body deformation characteristics g and g of the human body grid and the corresponding neutral human body grid describing rigid block deformation according to the hinge type deformation characteristics of the human body based on the skeleton s To obtain a set of training data f, f s ,g,g s Correspondingly processing the collected data to obtain a training data set;
the formula for calculating the human body deformation characteristic g describing the deformation of the rigid block is as follows:
Figure FDA0004059770490000011
wherein v is k Is the set of vertices of the kth rigid block, q i' 、p i' Respectively representing the ith' vertex on the k rigid blocks of the deformation reference grid and the calculation grid;
Figure FDA0004059770490000012
and &>
Figure FDA0004059770490000013
Respectively averaging the k-th rigid block on the deformation reference grid and the calculation grid; by means of radiation deformation of a rigid block>
Figure FDA0004059770490000014
Performing polar decomposition and parameterization to obtain human body deformation characteristics;
taking each human body grid data and corresponding neutral human body grid as a calculation grid to be substituted into the formula to obtain deformation characteristics g and g s ；
When a network structure is trained by utilizing a training data set, L1 mode loss between reconstruction and input characteristics and distribution regularization loss of hidden layer variables are used as loss functions;
the hierarchical reconstruction network is based on human body geometric prior and comprises two parts, wherein the first part is used for hierarchically reconstructing a main part of the three-dimensional human body model, the second part is used for reconstructing a difference part, and the reconstructed main part and the reconstructed difference part are added to obtain a reconstruction result; the reconstructed network is described as:
Figure FDA0004059770490000021
Figure FDA0004059770490000022
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0004059770490000023
representation utilization identity attribute e s And attitude attribute e p Based on the result of the reconstruction of>
Figure FDA0004059770490000024
Representation utilization identity attribute e s The reconstruction result of (2); />
Figure FDA0004059770490000025
Indicates a reconstruction result->
Figure FDA0004059770490000026
Is selected, is based on the main part b, <' > is selected>
Figure FDA0004059770490000027
Representing a skin layer; />
Figure FDA0004059770490000028
Represents the result of a reconstruction>
Figure FDA0004059770490000029
The difference portion d in (a); />
Figure FDA00040597704900000210
Represents the result of a reconstruction>
Figure FDA00040597704900000211
Main part b of (1) s ，/>
Figure FDA00040597704900000212
Representing a skin layer; />
Figure FDA00040597704900000213
Indicates a reconstruction result->
Figure FDA00040597704900000214
Part of difference d in (1) s ；
Reconstruction of the main parts b and b s The following formula is used to reconstruct the human deformation characteristics
Figure FDA00040597704900000215
And/or>
Figure FDA00040597704900000216
Figure FDA00040597704900000217
Figure FDA00040597704900000218
Wherein the content of the first and second substances,
Figure FDA00040597704900000219
is an independent mapping transformation;
thereafter, the skin layer is utilized
Figure FDA00040597704900000220
Reconstruction of the main parts b and b s ：
Figure FDA00040597704900000221
Figure FDA00040597704900000222
Wherein the skin layer
Figure FDA00040597704900000223
In the form of a matrix, is selected>
Figure FDA00040597704900000224
Represents->
Figure FDA00040597704900000225
The xth row and the y column; y is the number of rigid blocks;
difference parts d and d s Expressed as:
Figure FDA00040597704900000226
Figure FDA00040597704900000227
wherein the content of the first and second substances,
Figure FDA00040597704900000228
independent mapping transformation.
2. A method of representing a three-dimensional body shape according to claim 1, wherein said normalizing the collected mesh data set comprises:
regarding the collected human body grid data sets of different sources, taking certain source data as standard topology; calculating corresponding ACAP deformation expression by utilizing an action grid in a standard topology to obtain a group of priori deformation expression bases C of human body actions, and recovering the deformation expression of a human body by utilizing a group of parameters w; and (3) using the linear space of Cw as a priori space of human body deformation, then optimizing a group of vertex coordinates p and rigid transformation parameters of standard topology, namely a rotation parameter R and a translation parameter t, and standardizing human body grid data sets from different sources.
3. A method as claimed in claim 2, wherein the normalization process is to solve the following optimization problem:
Figure FDA0004059770490000031
wherein λ is 1 、λ 2 、λ 3 Are all set weights; | w | non-woven phosphor 1 Is a sparse regularization constraint on parameter w;
E prior is a human body deformation prior term determined by a prior deformation representation base C, and is represented as:
Figure FDA0004059770490000032
wherein, T i (w) is a prior deformation of a neighborhood of the ith vertex in the standard topology, q i The position of the ith vertex in a deformation reference grid under the standard topology is selected in advance; for the mesh to be optimized under the standard topology relative to the deformed reference mesh, the position of the ith vertex is p i (ii) a N (i) refers to a neighborhood vertex index set of the ith vertex under the standard topology, j refers to the jth vertex in the N (i), and the positions of the corresponding vertices in the deformation reference grid and the grid to be optimized are respectively expressed as q j 、p j ；c ij Is an edge weight value calculated on the deformed reference grid;
E icp is the registration energy term for the point to plane with the nearest neighbor of the target mesh, expressed as:
Figure FDA0004059770490000033
d is an index set of the corresponding point selected in the dynamic calculation under the standard topology; v. of l(i) Is and p i The corresponding point on the corresponding target grid,
Figure FDA0004059770490000034
representing point v l(i) Normal direction of (2); the target grid refers to human body grid data sets of different sources to be optimized;
E lan the L2 loss energy terms for a set of manually labeled sparse corresponding points for a standard topology and a target mesh are expressed as:
Figure FDA0004059770490000035
wherein, L is a corresponding point set selected in the standard topology in advance.
CN202010279674.5A 2020-04-10 2020-04-10 Three-dimensional human body shape representation method Active CN111476901B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010279674.5A CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010279674.5A CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Publications (2)

Publication Number Publication Date
CN111476901A CN111476901A (en) 2020-07-31
CN111476901B true CN111476901B (en) 2023-04-07

Family

ID=71751818

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010279674.5A Active CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Country Status (1)

Country Link
CN (1) CN111476901B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805977A (en) * 2018-06-06 2018-11-13 浙江大学 A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks
CN110363833A (en) * 2019-06-11 2019-10-22 华南理工大学 A kind of complete human body sport parameter representation method based on local rarefaction representation
CN110428493A (en) * 2019-07-12 2019-11-08 清华大学 Single image human body three-dimensional method for reconstructing and system based on grid deformation
CN110619681A (en) * 2019-07-05 2019-12-27 杭州同绘科技有限公司 Human body geometric reconstruction method based on Euler field deformation constraint
CN110827342A (en) * 2019-10-21 2020-02-21 中国科学院自动化研究所 Three-dimensional human body model reconstruction method, storage device and control device

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101194604B1 (en) * 2008-12-22 2012-10-25 한국전자통신연구원 Method and apparatus for shape deforming surface based 3d human model
EP3314577A1 (en) * 2015-06-24 2018-05-02 Max-Planck-Gesellschaft zur Förderung der Wissenschaften Skinned multi-person linear model
US9898858B2 (en) * 2016-05-18 2018-02-20 Siemens Healthcare Gmbh Human body representation with non-rigid parts in an imaging system
EP3579196A1 (en) * 2018-06-05 2019-12-11 Cristian Sminchisescu Human clothing transfer method, system and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805977A (en) * 2018-06-06 2018-11-13 浙江大学 A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks
CN110363833A (en) * 2019-06-11 2019-10-22 华南理工大学 A kind of complete human body sport parameter representation method based on local rarefaction representation
CN110619681A (en) * 2019-07-05 2019-12-27 杭州同绘科技有限公司 Human body geometric reconstruction method based on Euler field deformation constraint
CN110428493A (en) * 2019-07-12 2019-11-08 清华大学 Single image human body three-dimensional method for reconstructing and system based on grid deformation
CN110827342A (en) * 2019-10-21 2020-02-21 中国科学院自动化研究所 Three-dimensional human body model reconstruction method, storage device and control device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
袁仁奇 ; 徐增波 ; .基于Kinect的人体模板化三维模型拟合重建.丝绸.2017,(10),全文. *
高丽伟 ; 张元 ; 韩燮 ; .基于Kinect的三维人体建模研究.中国科技论文.2018,(02),全文. *

Also Published As

Publication number Publication date
CN111476901A (en) 2020-07-31

Similar Documents

Publication Publication Date Title
Wang et al. Hf-neus: Improved surface reconstruction using high-frequency details
Gao et al. Sparse data driven mesh deformation
Deng et al. A Survey of Non‐Rigid 3D Registration
Li et al. Global correspondence optimization for non‐rigid registration of depth scans
Zhu et al. Adafit: Rethinking learning-based normal estimation on point clouds
CN111524226B (en) Method for detecting key point and three-dimensional reconstruction of ironic portrait painting
CN113111861A (en) Face texture feature extraction method, 3D face reconstruction method, device and storage medium
Fan et al. Dual neural networks coupling data regression with explicit priors for monocular 3D face reconstruction
CN110889893B (en) Three-dimensional model representation method and system for expressing geometric details and complex topology
CN111797692B (en) Depth image gesture estimation method based on semi-supervised learning
Wan et al. Data-driven facial expression synthesis via Laplacian deformation
Yang et al. Multiscale mesh deformation component analysis with attention-based autoencoders
Wang et al. Rethinking point cloud filtering: A non-local position based approach
CN110717978A (en) Three-dimensional head reconstruction method based on single image
Sun et al. Cgof++: Controllable 3d face synthesis with conditional generative occupancy fields
WO2022222091A1 (en) Method for generating character bas-relief model on basis of single photo
Madadi et al. Deep unsupervised 3D human body reconstruction from a sparse set of landmarks
CN111476901B (en) Three-dimensional human body shape representation method
Zheng et al. Deformation representation based convolutional mesh autoencoder for 3D hand generation
Zhu et al. CED-Net: contextual encoder–decoder network for 3D face reconstruction
CN110544309A (en) Real-time sparse editing method and system based on large-scale grid model representation
Spurek et al. General hypernetwork framework for creating 3d point clouds
CN113379890B (en) Character bas-relief model generation method based on single photo
CN115457171A (en) Efficient expression migration method adopting base expression space transformation
Yoshiyasu et al. Learning body shape and pose from dense correspondences

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant