CN111476901A - Three-dimensional human body shape representation method - Google Patents

Three-dimensional human body shape representation method Download PDF

Info

Publication number
CN111476901A
CN111476901A CN202010279674.5A CN202010279674A CN111476901A CN 111476901 A CN111476901 A CN 111476901A CN 202010279674 A CN202010279674 A CN 202010279674A CN 111476901 A CN111476901 A CN 111476901A
Authority
CN
China
Prior art keywords
human body
deformation
grid
reconstruction
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010279674.5A
Other languages
Chinese (zh)
Other versions
CN111476901B (en
Inventor
张举勇
江博艺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202010279674.5A priority Critical patent/CN111476901B/en
Publication of CN111476901A publication Critical patent/CN111476901A/en
Application granted granted Critical
Publication of CN111476901B publication Critical patent/CN111476901B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/44—Morphing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a three-dimensional human body shape representation method, on one hand, a large amount of labeled training data can be obtained by processing an acquired human body grid data set, and the robustness of a model is increased; on the other hand, by using the deformation representation and defining the deformation representation based on the human body approximate rigid block, higher precision and robustness can be achieved compared with the mode that the Euclidean distance is directly used. Meanwhile, a hierarchical reconstruction network is designed aiming at the hinge type deformation of the human body, the shape prior of the human body is fully utilized, and the accuracy of the model is improved.

Description

Three-dimensional human body shape representation method
Technical Field
The invention relates to the technical field of human body three-dimensional reconstruction, in particular to a three-dimensional human body shape representation method.
Background
The parameterized human body model has wide application in the fields of computer graphics and computer vision, including three-dimensional human body tracking, three-dimensional human body reconstruction and attitude estimation. Since the human body has various attributes including gender, race, posture, body type, and the like, which contain abundant geometric deformation, high-precision reconstruction of the human body is a challenging task. In recent years, with the development of deep learning, geometric reconstruction using a neural network has become a trend.
In the past, human reconstruction was primarily based on the skeletal skinning method. The method expresses the identity deformation space of a human body by a group of simple linear basis deformations, and then deforms the human body to a specific posture by using a skeleton. The method is simple and high in efficiency in calculation, decoupling of the identity and the posture is achieved, high-frequency change of a human identity deformation space is ignored, and geometric reconstruction accuracy is limited. In addition, motion has a relative motion representation of skeletal joint points, without encoding a prior distribution of human motion, and thus may produce an abnormal human model. The other main method is based on deformation expression, typically, a three-dimensional point coordinate is replaced by triangular deformation, and decoupling parametric modeling of a human body is realized by decomposing the deformation into parts related to various attributes such as identity, posture and the like. The method has higher geometric reconstruction accuracy, but the calculation process is more complex, the parameter quantity is large, and the speed is slower.
Recently, learning some shapes like human faces using neural networks has been developed. Through end-to-end training, the network maps the shape manifold to a low-dimensional nonlinear space, which shows higher precision. But the human body has significant large-scale deformation caused by different poses compared to the human face. A network architecture with good precision in face modeling is directly applied to a human body grid, and a good result cannot be obtained. This is because the euclidean coordinates are not easy to maintain the original deformation mode for large-scale deformation, and distortion is likely to occur. On the other hand, the network architecture is not designed for a special skeleton hinge structure of a human body, and the precision is not improved by using the prior of the shape of the human body.
Disclosure of Invention
The invention aims to provide a three-dimensional human body shape representation method which can improve the reconstruction precision of a three-dimensional human body model.
The purpose of the invention is realized by the following technical scheme:
a three-dimensional human body shape representation method comprising:
preprocessing the collected human body grid data set, deforming based on a standard posture, and calculating ACAP deformation representing and describing human body deformation characteristics of rigid block deformation to form a training data set;
constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain an identity attribute and an action attribute, a three-dimensional human body model is reconstructed through the reconstruction network by utilizing the identity attribute and the action attribute, and the coder network and a hierarchical reconstruction network are trained by utilizing an error between a reconstruction result and input training data;
and after the training is finished, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result.
According to the technical scheme provided by the invention, 1) a large amount of labeled training data can be obtained by processing the acquired human body grid data set, so that the robustness of the model is improved. 2) By using the deformation representation and defining a deformation representation based on human body approximate rigid blocks, higher precision and robustness can be achieved compared with the mode that Euclidean distance is directly used. 3) Aiming at the hinge type deformation of the human body, a hierarchical reconstruction network is designed, the shape prior of the human body is fully utilized, and the precision of the model is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a method for representing a three-dimensional human body shape according to an embodiment of the present invention;
FIG. 2 is a reference body grid for calculating ACAP (consistent deformation) features and rigid blocks defined by calculating large-scale features g of a body according to an embodiment of the present invention;
fig. 3 is a schematic visualization diagram of reconstruction of each phase of a neutral posture and an original posture generated by a reconstruction network according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
In the field of human body parametric representation, the traditional model based on the skeleton skin method has high speed and strong action expression capability, but the reconstruction precision is limited because only simple linear dimension reduction is carried out on the deformation space of the identity; on the other hand, the representation of the motion is directly based on the relative motion of the joint points and does not limit the rationality of the human motion. To this end, an embodiment of the present invention provides a method for representing a three-dimensional human body shape, as shown in fig. 1, which mainly includes:
and 11, preprocessing the collected human body grid data set, deforming based on the standard posture, and calculating the ACAP deformation expression and the human body deformation characteristics describing the deformation of the rigid block to form a training data set.
The preferred embodiment of this step is as follows:
1) and carrying out standardization processing on the collected human body grid data set to obtain human body grid data with unified topology, and deforming to obtain a neutral human body grid corresponding to each human body grid data through the defined standard posture.
In the embodiment of the invention, the original human body grid data can be obtained from the network, including SCAPE, FAUST, Dyna, MANO and the like. For collected human body mesh data sets from different sources, the mesh representations of the collected human body mesh data sets are inconsistent, and the collected human body mesh data sets need to be converted into a standard topology G { V, E }, wherein V is a vertex set and E is an edge set. In this embodiment, a source data is used as a standard topology (e.g., SCAPE); calculating corresponding ACAP (consistent deformation) deformation representation by using action grids in standard topology (for example, seventy action grids in SCAPE), obtaining a group of priori deformation representation bases C of human body actions, and recovering the deformation representation of a human body by using a group of parameters w; and (3) using the linear space of Cw as a priori space of human body deformation, then optimizing a group of vertex coordinates p and rigid transformation parameters of standard topology, namely a rotation parameter R and a translation parameter t, and standardizing human body grid data sets from different sources.
The normalization process is to solve the following optimization problem:
Figure BDA0002446089520000031
wherein λ is1、λ2、λ3Are all set weights; | w | non-woven phosphor1Is a sparse regularization constraint on parameter w;
Eprioris a human body deformation prior term determined by a prior deformation representation base C, so that the optimized grid vertex conforms to the human body shape as much as possible, and is represented as follows:
Figure BDA0002446089520000032
wherein, Ti(w) is the prior deformation of a neighborhood of the ith vertex in the standard topology, the prior deformation represents that the base C is multiplied by w to obtain an ACAP characteristic, and then the ith vertex component of the ACAP characteristic is converted to obtain Ti(w)；qiIs the position of the ith vertex in the deformed reference grid under the standard topology; for the mesh to be optimized under the standard topology relative to the deformed reference mesh, the position of the ith vertex is pi(ii) a N (i) refers to a neighborhood vertex index set of the ith vertex under the standard topology, j refers to the jth vertex in N (i), and the positions of the corresponding vertices in the deformation reference grid and the grid to be optimized are respectively represented as qj、pj；cijIs an edge weight value calculated on the deformation reference grid, called cotangent weight, and is specifically qjAnd q isiThe edge weight value of (2). In the embodiment of the invention, the deformation reference grid is a human body grid required for calculating ACAP deformation expression, and can be selected according to actual conditions or experience; as shown in fig. 2, the pre-selected warped reference grid is on the left. Those skilled in the art will appreciate that the deformed reference mesh corresponds one-to-one to all vertices of the standard topology, except for the different locations of the vertices; in the embodiment of the invention, a deformation reference grid is used as a reference to optimize a grid under a standard topology; that is, qiIs stationary, piAre optimized and the values are changed.
EicpIs a point-to-plane registration energy term with the nearest neighbor of the target mesh such that the optimization result is close to the target, expressed as:
Figure BDA0002446089520000041
d is an index set of the corresponding point pair selected by dynamic calculation under the standard topology; v. ofl(i)Is and piThe corresponding point on the corresponding target grid,
Figure BDA0002446089520000042
representing point vl(i)Normal direction of (2); the target grid refers to human body grid data sets of different sources to be optimized;
Elanl2 loss energy terms, which are a set of manually labeled sparse corresponding points of the standard topology and the target mesh, to reduce slip errors and avoid local minima, are expressed as:
Figure BDA0002446089520000043
l is a pre-selected corresponding point set in a standard topology, and the above formula limits the spatial location of the points in the pre-selected corresponding point set L to be as small as possible.
In the embodiment of the invention, the human body grid data fitted based on the mode has consistent standard topology, and has tolerable registration error with the original data.
The human body model data constructed on the basis also needs to define the corresponding neutral human body grids. Illustratively, a consistent a-gesture in the SPRING dataset may be used as the neutral gesture. For all the human body data of each identity, selecting a human body with the minimum error with the SPRING average grid as a candidate human body grid, and then converting the candidate human body grid into an A posture by using an ARAP (as rigid as possible) deformation method to be used as a neutral human body grid of the human body of the identity.
2) Calculating ACAP deformation representation of each human body grid data and corresponding neutral human body grid, and recording as f and fs(ii) a And respectively calculating human body deformation characteristics g and g of the human body grid and the corresponding neutral human body grid describing rigid block deformation according to the hinge type deformation characteristics of the human body based on the skeletonsTo obtain a set of training data f, fs,g,gsAnd correspondingly processing the collected data to obtain a training data set.
By the method, the human body grid data with the uniform topology are obtained, and each identity human body has a neutral posture grid. In the embodiment of the invention, the deformation expression of the geometric shape of the ACAP is used for replacing the original Euclidean coordinate shape expression to enhance the modeling precision of large-scale deformation and obtain better performance compared with a general linear model.
The formula for calculating the human body grid data and the ACAP deformation expression of the corresponding neutral human body grid is as follows:
Figure BDA0002446089520000051
similarly, i represents a vertex in the standard topology, and the above is normalized, so the standard topology can be any standard topology after normalization or a previously selected standard topology. The other parameters have the same meanings as above and are not described in detail.
T in the above formulaiRefers to the affine transformation matrix of the ith vertex in the standard topology, the affine transformation matrix TiThe method comprises the steps of transforming a local umbrella-shaped structure of a neighborhood of a deformation reference grid into various deformation information of a structure of a computational grid; by polar decomposition, TiDecomposition to rigid RiAnd a non-rigid part Si(determined by 3 and 6 degrees of freedom, respectively), after disambiguation of the rigidly deformed part of each vertex, an ACAP deformation representation of the computational mesh is obtained; the dimensions of the representation are 9 times the number of vertices with 3 and 6 parameters for rigid and non-rigid deformations, respectively, per vertex.
Taking each human body grid data and corresponding neutral human body grid as calculation grid to be substituted into the formula to obtain corresponding ACAP deformation expressions f and fs。
In addition, some approximately rigid parts, such as the lower arm, the head, etc., are defined on the human body in consideration of the skeleton-based hinged deformation characteristics of the human body. On the basis, a deformation characteristic g of a large scale of a human body for describing the deformation of the rigid block is defined, and the calculation formula is as follows:
Figure BDA0002446089520000052
wherein v iskIs the set of vertices of the kth rigid block, qi'、pi'Respectively representing the positions of the ith' vertex on the kth rigid block in the deformation reference grid and the calculation grid;
Figure BDA0002446089520000053
and
Figure BDA0002446089520000054
respectively averaging the k-th rigid block on the deformation reference grid and the calculation grid; by radial deformation of rigid masses
Figure BDA0002446089520000055
Performing polar decomposition and parameterization to obtain human body deformation characteristic with dimension of human bodyApproximately 9 times the number of rigid blocks.
Similarly, each human body grid data and corresponding neutral human body grid are taken as calculation grids to be substituted into the formula to obtain deformation characteristics g and gs。
As shown in fig. 2, the rigid blocks defined for calculating the deformation reference grid (left side of fig. 2) and calculating the human body large-scale features (right side of fig. 2) of the ACAP deformation representation.
Step 12, constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain an identity attribute and an action attribute, a three-dimensional human body model is reconstructed through the identity attribute and the action attribute through the reconstruction network, and the coder network and the hierarchical reconstruction network are trained through errors between the reconstruction result and input training data.
In the embodiment of the invention, the encoder network can adopt a standard variational self-encoder structure to learn the identity attribute e from the ACAP deformation expression f of the human body grid datasAnd attitude attribute epAnd then combining the reconstructed network to form an end-to-end network structure for training. Training data { f, fs，g，gsOnly f is used as the input of the network and the rest of the data is used to calculate the reconstruction error.
In the embodiment of the invention, the hierarchical reconstruction network is based on human body geometric prior and comprises two parts, wherein the first part is used for hierarchically reconstructing a main part of a three-dimensional human body model, the second part is used for reconstructing a difference part, and the reconstructed main part and the reconstructed difference part are added to obtain a reconstruction result; the reconstructed network is described as:
Figure BDA0002446089520000061
Figure BDA0002446089520000062
wherein the content of the first and second substances,
Figure BDA0002446089520000063
representation utilization identity attribute esAnd attitude attribute epAs a result of the reconstruction of (a),
Figure BDA0002446089520000064
representation utilization identity attribute esThe reconstruction result of (2);
Figure BDA0002446089520000065
indicating the result of the reconstruction
Figure BDA0002446089520000066
The main portion b in (1), W represents a skin layer;
Figure BDA0002446089520000067
indicating the result of the reconstruction
Figure BDA0002446089520000068
The difference portion d in (a);
Figure BDA0002446089520000069
indicating the result of the reconstruction
Figure BDA00024460895200000610
Main part b of (1)s，
Figure BDA00024460895200000611
Representing a skin layer;
Figure BDA00024460895200000612
indicating the result of the reconstruction
Figure BDA00024460895200000613
The difference part d ins；
Reconstruction of the main parts b and bsThe following equation is used to reconstruct the human deformation characteristics
Figure BDA00024460895200000623
And
Figure BDA00024460895200000624
Figure BDA00024460895200000614
Figure BDA00024460895200000615
wherein the content of the first and second substances,
Figure BDA00024460895200000616
are independent mapping transformations, which may be modeled, for example, using a multi-tier perceptron.
Thereafter, the skin layer is utilized
Figure BDA00024460895200000617
Reconstruction of the main parts b and bs：
Figure BDA00024460895200000618
Figure BDA00024460895200000619
Wherein the skin layer
Figure BDA00024460895200000620
In the form of a matrix, the matrix is,
Figure BDA00024460895200000621
to represent
Figure BDA00024460895200000622
The x-th row and the y-th column; y is the number of rigid blocks, e.g., Y ═ 16; the above principle is that the deformation of each vertex is obtained by linear convex combination of the relative rigid block deformation of the vertex, and the coefficient of the convex combination is determined by learning training and is not set manually.
Difference of constructionHetero moieties d and dsExpressed as:
Figure BDA0002446089520000071
Figure BDA0002446089520000072
wherein the content of the first and second substances,
Figure BDA0002446089520000073
independent mapping transformations, illustratively, these mapping transformations may be modeled using a multi-tier perceptron.
As shown in fig. 3, an example of the visualization of the various computed portions of the reconstructed network over one reconstructed instance is given. B (e) in FIG. 3s，ep)、B(es0) i.e. the main parts b and b mentioned hereins。
In the embodiment of the invention, the network can be trained end to end through the training data set, and after the training is finished, the decoupled low-dimensional hidden layer can be used for representing es，epThe method can be used for reconstructing a human body (or applied to aspects of human body editing, motion migration and the like), can also be applied to aspects of human body editing, motion migration and the like, and has wide application prospects in the fields of video live broadcast, virtual fitting, somatosensory games and the like.
During training, L1 mode loss between reconstruction and input features and distribution regularization loss of hidden layer variables can be adopted as loss functions.
L1 the mode loss can be expressed as:
Figure BDA0002446089520000074
Figure BDA0002446089520000075
in the above formula, the first and second carbon atoms are,
Figure BDA0002446089520000076
are the reconstruction results of the relevant data in the training data, and the specific values 9 (the number of deformation features) and 16 (the number of rigid blocks) referred to in the above formula are all examples and are not limited.
The regular partial loss of the hidden layer parameter distribution is:
EsKL＝DKL(q(es|f)||p(es))
EpKL＝DKL(q(ep|f)||p(ep))
the two losses are K L divergence losses of the invariant self-coder, in which the standard constrained hidden layer distribution conforms to the prior distribution.
And step 13, after training, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result.
After the network training is completed through the step 12, the encoder network can be abandoned, and the reconstructed network is directly used as a decoupled low-dimensional human body parameterized model. The model reconstructs a complete human body grid from two groups of decoupling parameters respectively representing identity and posture. Specifically, the trained hierarchical reconstruction network may reconstruct the identity attribute and the posture attribute of the input data according to the method introduced in the training phase
Figure BDA0002446089520000081
And
Figure BDA0002446089520000082
both are ACAP deformation representations, and the corresponding body mesh and the neutral body mesh can be obtained through a simple conversion. The conversion method referred to herein can be referred to in the prior art, and is not described in detail.
Compared with the traditional human body parameterized model representation method, the scheme of the embodiment of the invention mainly has the following advantages:
1) the characteristics of input and output are represented by nonlinear deformation, the traditional Euclidean coordinates are replaced, the precision is higher, and the deformation with large scale is more robust.
2) The reconstruction accuracy of the model is further improved by utilizing the strong fitting capability of the neural network and combining the framework design of human body deformation prior.
3) The obtained posture hidden layer representation has certain semantics by utilizing the learning of a large number of human body models with various postures, namely embedding reasonable actions of the human body into a low-dimensional space. However, the posture parameters of the conventional model often have no semantics and may generate unreasonable human body actions.
Through the above description of the embodiments, it is clear to those skilled in the art that the above embodiments can be implemented by software, and can also be implemented by software plus a necessary general hardware platform. With this understanding, the technical solutions of the embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (7)

1. A three-dimensional human body shape representation method, comprising:
preprocessing the collected human body grid data set, deforming based on a standard posture, and calculating ACAP deformation representing and describing human body deformation characteristics of rigid block deformation to form a training data set;
constructing an encoder network and a hierarchical reconstruction network to form an end-to-end network structure, and training the network structure by using a training data set; in the training process, the ACAP deformation expression is coded through a coder network to obtain an identity attribute and an action attribute, a three-dimensional human body model is reconstructed through the reconstruction network by utilizing the identity attribute and the action attribute, and the coder network and a hierarchical reconstruction network are trained by utilizing an error between a reconstruction result and input training data;
and after the training is finished, inputting the identity attribute and the action attribute into a trained hierarchical reconstruction network to obtain a three-dimensional human body model reconstruction result.
2. The method as claimed in claim 1, wherein the pre-processing the collected mesh data set of human body to obtain the training data set composed of ACAP deformation representation and human body deformation features comprises:
firstly, carrying out standardization processing on a collected human body grid data set to obtain human body grid data with unified topology, and deforming to obtain a neutral human body grid corresponding to each human body grid data through a defined standard posture;
then, calculating ACAP deformation representation of each human body grid data and corresponding neutral human body grid, and recording the ACAP deformation representation as f and fs(ii) a And respectively calculating human body deformation characteristics g and g of the human body grid and the corresponding neutral human body grid describing rigid block deformation according to the hinge type deformation characteristics of the human body based on the skeletonsTo obtain a set of training data f, fs,g,gsAnd correspondingly processing the collected data to obtain a training data set.
3. A method of representing a three-dimensional body shape according to claim 2, wherein said normalizing the collected mesh data set comprises:
regarding the collected human body grid data sets of different sources, taking certain source data as standard topology; calculating corresponding ACAP deformation representation by utilizing an action grid in a standard topology to obtain a group of prior deformation representation bases C of human body actions, and recovering the deformation representation of a human body by utilizing a group of parameters w; and (3) using the linear space of Cw as a priori space of human body deformation, then optimizing a group of vertex coordinates p and rigid transformation parameters of standard topology, namely a rotation parameter R and a translation parameter t, and standardizing human body grid data sets from different sources.
4. A method as claimed in claim 3, wherein the normalization process is to solve the following optimization problem:
Figure FDA0002446089510000011
wherein λ is1、λ2、λ3Are all set weights; | w | non-woven phosphor1Is a sparse regularization constraint on parameter w;
Eprioris a human body deformation prior term determined by a prior deformation representation base C, and is represented as:
Figure FDA0002446089510000021
wherein, Ti(w) is a prior deformation of a neighborhood of the ith vertex in the standard topology, qiThe position of the ith vertex in a deformation reference grid under the standard topology is selected in advance; for the mesh to be optimized under the standard topology relative to the deformed reference mesh, the position of the ith vertex is pi(ii) a N (i) refers to a neighborhood vertex index set of the ith vertex under the standard topology, j refers to the jth vertex in N (i), and the positions of the corresponding vertices in the deformation reference grid and the grid to be optimized are respectively represented as qj、pj；cijIs an edge weight value calculated on the deformed reference grid;
Eicpis the registration energy term for the point to plane with the nearest neighbor of the target mesh, expressed as:
Figure FDA0002446089510000022
d is an index set of the corresponding point selected in the dynamic calculation under the standard topology;vl(i)is and piThe corresponding point on the corresponding target grid,
Figure FDA0002446089510000023
representing point vl(i)Normal direction of (2); the target grid refers to human body grid data sets of different sources to be optimized;
Elanthe L2 loss energy term, which is a set of manually labeled sparse correspondences of the standard topology and the target mesh, is expressed as:
Figure FDA0002446089510000024
l is a corresponding point set selected in the standard topology in advance.
5. A method according to claim 2, wherein the formula for calculating the body deformation characteristic g describing the deformation of the rigid blocks is:
Figure FDA0002446089510000025
wherein v iskIs the set of vertices of the kth rigid block, qi'、pi'denotes the ith' vertex on the k-th rigid blocks of the deformed reference grid and the computation grid, respectively;
Figure FDA0002446089510000026
and
Figure FDA0002446089510000027
respectively averaging the k-th rigid block on the deformation reference grid and the calculation grid; by radial deformation of rigid masses
Figure FDA0002446089510000028
Performing polar decomposition and parameterization to obtain human body deformation characteristics;
each human body grid data and the corresponding middleThe sexual body grid is taken as a calculation grid and is substituted into the formula to obtain deformation characteristics g and gs。
6. The method of claim 1, wherein L1 mode loss between reconstructed and input features and distribution regularization loss of hidden layer variables are used as loss functions when training the network structure with the training data set.
7. The method according to claim 1, wherein the hierarchical reconstruction network is a human geometry prior-based hierarchical reconstruction network, and comprises two parts, a first part hierarchically reconstructs a main part of the three-dimensional human model, a second part reconstructs a difference part, and the reconstructed main part and the reconstructed difference part are added to obtain a reconstruction result; the reconstructed network is described as:
Figure FDA0002446089510000031
Figure FDA0002446089510000032
wherein the content of the first and second substances,
Figure FDA0002446089510000033
representation utilization identity attribute esAnd attitude attribute epAs a result of the reconstruction of (a),
Figure FDA0002446089510000034
representation utilization identity attribute esThe reconstruction result of (2);
Figure FDA0002446089510000035
indicating the result of the reconstruction
Figure FDA0002446089510000036
The main part b of (a) is,
Figure FDA0002446089510000037
representing a skin layer;
Figure FDA0002446089510000038
indicating the result of the reconstruction
Figure FDA0002446089510000039
The difference portion d in (a);
Figure FDA00024460895100000310
indicating the result of the reconstruction
Figure FDA00024460895100000311
Main part b of (1)s，
Figure FDA00024460895100000312
Representing a skin layer;
Figure FDA00024460895100000313
indicating the result of the reconstruction
Figure FDA00024460895100000314
The difference part d ins；
Reconstruction of the main parts b and bsThe following equation is used to reconstruct the human deformation characteristics
Figure FDA00024460895100000315
And
Figure FDA00024460895100000316
Figure FDA00024460895100000317
Figure FDA00024460895100000318
wherein the content of the first and second substances,
Figure FDA00024460895100000319
is an independent mapping transformation;
thereafter, the skin layer is utilized
Figure FDA00024460895100000320
Reconstruction of the main parts b and bs：
Figure FDA00024460895100000321
Figure FDA00024460895100000322
Wherein the skin layer
Figure FDA00024460895100000323
In the form of a matrix, the matrix is,
Figure FDA00024460895100000324
to represent
Figure FDA00024460895100000325
The x-th row and the y-th column; y is the number of rigid blocks;
difference parts d and dsExpressed as:
Figure FDA00024460895100000326
Figure FDA00024460895100000327
wherein the content of the first and second substances,
Figure FDA00024460895100000328
independent mapping transformation.
CN202010279674.5A 2020-04-10 2020-04-10 Three-dimensional human body shape representation method Active CN111476901B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010279674.5A CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010279674.5A CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Publications (2)

Publication Number Publication Date
CN111476901A true CN111476901A (en) 2020-07-31
CN111476901B CN111476901B (en) 2023-04-07

Family

ID=71751818

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010279674.5A Active CN111476901B (en) 2020-04-10 2020-04-10 Three-dimensional human body shape representation method

Country Status (1)

Country Link
CN (1) CN111476901B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100156935A1 (en) * 2008-12-22 2010-06-24 Electronics And Telecommunications Research Institute Method and apparatus for deforming shape of three dimensional human body model
US20170337732A1 (en) * 2016-05-18 2017-11-23 Siemens Healthcare Gmbh Human Body Representation With Non-Rigid Parts In An Imaging System
US20180315230A1 (en) * 2015-06-24 2018-11-01 Max-Planck-Gesellschaft Zur Förderung Dur Wissenschaften E.V. Skinned multi-person linear model
CN108805977A (en) * 2018-06-06 2018-11-13 浙江大学 A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks
CN110363833A (en) * 2019-06-11 2019-10-22 华南理工大学 A kind of complete human body sport parameter representation method based on local rarefaction representation
CN110428493A (en) * 2019-07-12 2019-11-08 清华大学 Single image human body three-dimensional method for reconstructing and system based on grid deformation
US20190371080A1 (en) * 2018-06-05 2019-12-05 Cristian SMINCHISESCU Image processing method, system and device
CN110619681A (en) * 2019-07-05 2019-12-27 杭州同绘科技有限公司 Human body geometric reconstruction method based on Euler field deformation constraint
CN110827342A (en) * 2019-10-21 2020-02-21 中国科学院自动化研究所 Three-dimensional human body model reconstruction method, storage device and control device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100156935A1 (en) * 2008-12-22 2010-06-24 Electronics And Telecommunications Research Institute Method and apparatus for deforming shape of three dimensional human body model
US20180315230A1 (en) * 2015-06-24 2018-11-01 Max-Planck-Gesellschaft Zur Förderung Dur Wissenschaften E.V. Skinned multi-person linear model
US20170337732A1 (en) * 2016-05-18 2017-11-23 Siemens Healthcare Gmbh Human Body Representation With Non-Rigid Parts In An Imaging System
US20190371080A1 (en) * 2018-06-05 2019-12-05 Cristian SMINCHISESCU Image processing method, system and device
CN108805977A (en) * 2018-06-06 2018-11-13 浙江大学 A kind of face three-dimensional rebuilding method based on end-to-end convolutional neural networks
CN110363833A (en) * 2019-06-11 2019-10-22 华南理工大学 A kind of complete human body sport parameter representation method based on local rarefaction representation
CN110619681A (en) * 2019-07-05 2019-12-27 杭州同绘科技有限公司 Human body geometric reconstruction method based on Euler field deformation constraint
CN110428493A (en) * 2019-07-12 2019-11-08 清华大学 Single image human body three-dimensional method for reconstructing and system based on grid deformation
CN110827342A (en) * 2019-10-21 2020-02-21 中国科学院自动化研究所 Three-dimensional human body model reconstruction method, storage device and control device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
袁仁奇;徐增波;: "基于Kinect的人体模板化三维模型拟合重建" *
高丽伟;张元;韩燮;: "基于Kinect的三维人体建模研究" *

Also Published As

Publication number Publication date
CN111476901B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
Qian et al. PUGeo-Net: A geometry-centric network for 3D point cloud upsampling
Gao et al. Sparse data driven mesh deformation
Eisenberger et al. Smooth shells: Multi-scale shape registration with functional maps
Li et al. Global correspondence optimization for non‐rigid registration of depth scans
Zhu et al. Adafit: Rethinking learning-based normal estimation on point clouds
Fan et al. Dual neural networks coupling data regression with explicit priors for monocular 3D face reconstruction
Tang et al. Skeletonnet: A topology-preserving solution for learning mesh reconstruction of object surfaces from rgb images
CN113111861A (en) Face texture feature extraction method, 3D face reconstruction method, device and storage medium
CN111524226B (en) Method for detecting key point and three-dimensional reconstruction of ironic portrait painting
CN111797692B (en) Depth image gesture estimation method based on semi-supervised learning
Yang et al. Multiscale mesh deformation component analysis with attention-based autoencoders
Wei et al. GeoDualCNN: Geometry-supporting dual convolutional neural network for noisy point clouds
Wang et al. Rethinking point cloud filtering: A non-local position based approach
Sun et al. Cgof++: Controllable 3d face synthesis with conditional generative occupancy fields
WO2022222091A1 (en) Method for generating character bas-relief model on basis of single photo
Madadi et al. Deep unsupervised 3D human body reconstruction from a sparse set of landmarks
Li et al. Remodeling of mannequins based on automatic binding of mesh to anthropometric parameters
Zheng et al. Deformation representation based convolutional mesh autoencoder for 3D hand generation
CN111476901B (en) Three-dimensional human body shape representation method
Zhu et al. CED-Net: contextual encoder–decoder network for 3D face reconstruction
CN110544309A (en) Real-time sparse editing method and system based on large-scale grid model representation
Chen et al. Adaptive skeleton‐driven cages for mesh sequences
CN113379890B (en) Character bas-relief model generation method based on single photo
Yoshiyasu et al. Learning body shape and pose from dense correspondences
CN112837420A (en) Method and system for completing shape of terracotta warriors point cloud based on multi-scale and folding structure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant