CN110619681B - Human body geometric reconstruction method based on Euler field deformation constraint - Google Patents

Human body geometric reconstruction method based on Euler field deformation constraint Download PDF

Info

Publication number
CN110619681B
CN110619681B CN201910605065.1A CN201910605065A CN110619681B CN 110619681 B CN110619681 B CN 110619681B CN 201910605065 A CN201910605065 A CN 201910605065A CN 110619681 B CN110619681 B CN 110619681B
Authority
CN
China
Prior art keywords
field
target
human body
local
deformation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910605065.1A
Other languages
Chinese (zh)
Other versions
CN110619681A (en
Inventor
郝爱民
李帅
梁晓辉
杨文军
高阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Tonghui Technology Co ltd
Original Assignee
Hangzhou Tonghui Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Tonghui Technology Co ltd filed Critical Hangzhou Tonghui Technology Co ltd
Priority to CN201910605065.1A priority Critical patent/CN110619681B/en
Publication of CN110619681A publication Critical patent/CN110619681A/en
Application granted granted Critical
Publication of CN110619681B publication Critical patent/CN110619681B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a human body geometric reconstruction method based on Euler field deformation constraint, which comprises the following steps: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using an LBS (location based service) method to roughly align the postures of the template model and the target model; performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; performing local shape drive-based region expansion and contraction; and optimizing a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field based on the SDF, calculating a deformation vector field to obtain an SDF deformation result, and driving the template grid to be aligned with the target model according to the shape to realize human body reconstruction. The invention improves the robustness and efficiency of three-dimensional human body structural reconstruction and meets the requirement of human body reconstruction on rough data.

Description

Human body geometric reconstruction method based on Euler field deformation constraint
Technical Field
The invention particularly relates to a human body geometric reconstruction method based on Euler field deformation constraint.
Background
In recent years, three-dimensional modeling of objects in virtual scenes, particularly human body three-dimensional modeling, can be applied to not only creation of roles required in movies, games and the like, but also wide application in aspects of virtual fitting, medical assistance, vocational education and the like, but a human body modeling method has certain limitations, and accurate, rapid and vivid reconstruction of a specific human body model is difficult and complicated in calculation. In recent years, three-dimensional human reconstruction meeting engineering and scientific research requirements is receiving attention from scholars and scientific research institutions at home and abroad. Three-dimensional human reconstruction studies based on computer graphics related techniques are undoubtedly facing enormous challenges and attractive to fluid-related researchers and computer graphics researchers.
Three-dimensional human body reconstruction methods are mainly divided into two types: a template-free human body surface reconstruction method and a template-based human body structural reconstruction method. In order to perform the three-dimensional human body structural reconstruction, the general method is to establish the human body shape space or perform the internal anatomy simulation, which are relatively complex and have certain limitations.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a human body geometric reconstruction method based on Euler field deformation constraint, which improves the effect of grid deformation in three-dimensional human body reconstruction, improves the robustness and efficiency of three-dimensional human body structural reconstruction, and meets the requirement of human body reconstruction on rough data.
In order to achieve the purpose, the invention adopts the following technical scheme:
a human body geometric reconstruction method based on Euler field deformation constraint comprises the following three steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating through the feature points to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using a linear hybrid skin LBS method, so that the postures of the template model and the target model are roughly aligned;
(2) posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
(3) human body reconstruction based on Euler field target energy function drive evolution: based on the SDF generated by the template grid aligned in posture, a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field is optimized, a deformation vector field is calculated, an SDF deformation result is obtained, and the template grid and a target model are driven to be aligned in shape so as to realize three-dimensional human body reconstruction.
Further, in the step (1), the template model is in a T-shaped static posture.
Further, the specific process of the step (2) is as follows: the local posture adjustment adopts the rigid transformation of the estimated local HRBF field to estimate the local surface of the target human body, and carries out space discretization on the target data to obtain a symbol distance field phi of the target datatarAnd obtaining the gradient of the field
Figure BDA0002120439740000021
Driving a central point set by using a spatial field, and indirectly driving the deformation of the HRBF local field to be close to the local shape of target data; after the target SDF is obtained, the drive vector for the HRBF center point is expressed as
Figure BDA0002120439740000022
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and enabling the central point to finally fall on the surface of the target data through iteration; at the central point to the end position
Figure BDA0002120439740000023
Then, the normal direction is set to
Figure BDA0002120439740000024
Figure BDA0002120439740000025
Thus, obtaining a surface alignment result with the HRBF field central point as a core; to make the local HRBF field align with the local part of the target as much as possible, the energy equation of the attitude adjustment is expressed as
Figure BDA0002120439740000026
TpRepresenting the rotation variation of the template skeleton, and when a rotation axis is estimated by using the rotation transformation of a central point motion set about local nodes, expressing the average by using quaternion, and estimating the rotation angle, iteratively solving by using a local linearization method; here the normal energy constraint of the target surface is added
Figure BDA0002120439740000027
And is
Figure BDA0002120439740000028
Represents and vcAfter each iteration is finished, updating the target matching point of each central point according to the normal and the position relation by an ICP method, and then performing the next iteration; the distortion vector is consistent with the rough alignment stage, and after the iteration end point is reached, the adjustment result of the human body posture is obtained;
providing a state of a local scale by utilizing expansion and contraction of an HRBF field as initialization input of a fitting stage; when the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, estimating a deformation end point state of HRBF in advance, and estimating a proper gamma value of local field expansion and contraction through center point drive fitting after each iteration; in knowing the motion endpoint set of HRBF center point
Figure BDA0002120439740000029
After, the energy can be expressed as:
Figure BDA00021204397400000210
since the unknown is only a constant, the solution to the target energy function can be written directly
Figure BDA00021204397400000211
Wherein N iscRepresenting the number of the central points except the additional points, when gamma is used as an expansion and contraction coefficient to be applied to a local field, the central points reach new positions through movement, then the state of an Euler vector field is obtained at the new positions where the central points reach, and the central points are moved to the surface of a target through the Euler field driving in the state to obtain a new end point set; through iteration, the expansion and contraction coefficients of the region can be obtained.
Further, the specific process of the step (3) is as follows: driving evolution by using a spatially discretized Euler field target energy function, and reconstructing the shape; discretizing the space where the template and the target human body grid are located to generate an SDF, then establishing a target energy function based on an Euler space by using target data constraint, human body deformation constraint and level set constraint, and estimating a vector field aligning the template and the target SDF by using a gradient descent method; and after the SDFs are aligned, driving the template grids to a target shape by using the SDFs, and finishing the structural human body reconstruction of the target data.
Further, assume the template model SDF is φtmpThe target model SDF is phitarSDF after being driven by vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field is represented as
Figure BDA0002120439740000031
The summation operation refers to summing the results of all the nodes participating in the operation in the SDF;
with deformation driving of the HRBF field, a smooth energy constraint based on body deformation is provided for the vector field Ψ, written as:
Figure BDA0002120439740000032
wherein, JΨI.e., the Jacobian matrix of Ψ, and
Figure BDA0002120439740000039
then represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics are reflected in
Figure BDA0002120439740000033
Thus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic constraint of human body can be obtained, so that phi is driventmpThe human body deformation characteristics are better embodied during evolution;
to ensure the level set characteristics of the SDF field, satisfied after each iteration
Figure BDA0002120439740000034
I.e. to ensure that its iso-surface dimension is 1.
Further, while generating the SDF, the triangular grid points are recorded to correspond to φtmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Figure BDA0002120439740000035
Wherein v isiWhich represents the grid points,
Figure BDA0002120439740000036
indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmp(Ψ) and a new value in
Figure BDA0002120439740000037
Then represents the corresponding position in the vector fieldtmpGradient of (Ψ)
Figure BDA0002120439740000038
Driven using this formula, v is driven by the step length ηiMoving to the original iso-surface position is equivalent to following the SDF surface.
Further, in the SDF driving process, a local tangential relaxation method is used, and the grid points are moved to the weighted centers of the 1-ring neighborhood points of the grid points according to the weights in each iteration; in the initial state, a mean coordinate method is used for the grids to obtain a projection set q of 1-ring neighborhood points of the grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
Figure BDA0002120439740000041
wherein,
Figure BDA0002120439740000042
representing the tangentially modified vertex position, the weight μ is expressed as a function of ψ (v)i) Varying smoothing function
Figure BDA0002120439740000043
Figure BDA0002120439740000044
Representing the initial value, ψ, after the grid points have not been driven and the SDF has been changed0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d); a relatively large tangential relaxation will be performed at the beginning of the drive, and a gradual tapering relaxation up to 0 is chosen as the target isosurface is approached.
Further, when some regions generate sharp distortion, Laplacian smoothing (Laplacian smoothing) processing is used at the corresponding region points.
The invention has the beneficial effects that:
(1) the invention adopts a top-down template deformation method to align the postures first and then realize local shape alignment, thereby improving the efficiency of the algorithm.
(2) The target energy functions of the space discrete nodes can be operated simultaneously in parallel, so that the operation speed in the process of optimizing alignment is improved.
(3) The method uses HRBF to fit the deformation characteristic of the local human body, realizes human body reconstruction by using a mode of constructing a local field, and reduces the calculation complexity of deformation.
(4) The invention adopts a method for optimally driving the alignment of the human body template model and the target data by using an Euler field energy function containing local field deformation constraint, on one hand, a smooth implicit surface is used for driving the template grid to ensure the reconstruction quality, and on the other hand, the calculation efficiency is improved by using the characteristic that the Euler field drive can be parallelized, thereby solving the reconstruction problem of rough human body data and reconstructing a three-dimensional human body structural geometric model which accords with the shape and the posture of the target.
Drawings
FIG. 1 is a process diagram of a human body geometric reconstruction method based on Euler field deformation constraint.
Fig. 2 is a schematic view of human body feature points.
FIG. 3 is a schematic view of a human body skeleton.
FIG. 4 is a schematic diagram of the HRBF local field of human body.
Fig. 5 is a schematic diagram of a three-dimensional human body reconstruction result when different data are input.
Detailed Description
The general process flow of the geometric reconstruction of the human body based on the Euler field deformation constraint is shown in FIG. 1, and the invention is further explained in the following with reference to the accompanying drawings.
The invention provides a human body geometric reconstruction method based on Euler field deformation constraint, in particular to skeleton deformation driven by characteristic points, local deformation driven by HRBF and shape deformation driven by Euler field, comprising the following steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: taking a target human body grid as a data source, extracting predefined human body characteristic points by using a random forest regression method, and calculating through the characteristic points to obtain target human body skeleton nodes; driving skeleton deformation by using the characteristic points, obtaining skeleton weight distribution on template model grid points by using a Linear Blend skin Skinning (LBS) method, driving skeleton deformation, and driving the grid to deform simultaneously by using corresponding weight so as to roughly align the postures of the template model and the target model;
the method uses a target function nonlinear optimization mode to carry out skeleton alignment and posture alignment; the distribution of the human body characteristic points is shown in fig. 2, the skeleton structure is shown in fig. 3, and skeleton nodes are shown in fig. 3; use of
Figure BDA0002120439740000051
And
Figure BDA0002120439740000052
respectively representing the human body characteristic points l marked in advance on the template modeltmpSet of (2)
Figure BDA0002120439740000053
Extracting human body characteristic points l on a target human body grid by using a pre-trained random forest regression model expressing the position relation of shape characteristics and characteristic pointstarSet of (2)
Figure BDA0002120439740000054
Use of
Figure BDA0002120439740000055
Representing a set of scaling ratios of the skeleton, s representing a scaling ratio of the skeleton, using
Figure BDA0002120439740000056
(Here [. cndot.)]TRepresenting a corresponding column vector) represents a rotation state of the skeleton, wherein
Figure BDA0002120439740000057
Representing the rotation and translation of rigid transformation of a human body in space, and theta represents a rotation angle set of all skeleton nodes; the rest state, i.e. the state of the framework in T-shape, is expressed by theta0And (4) showing.
By using feature point sets on a target body mesh
Figure BDA0002120439740000058
Locally calculating the average position to obtain the skeleton node set of the target human body grid
Figure BDA0002120439740000059
jtarThe state change of theta is estimated through a target skeleton node and a feature point set; the local scale of the template and the target needs to be consistent firstly, and for a certain skeleton, the father and son nodes of the template are assumed to be j respectivelya,jb(ii) a The parent and child nodes of the target are j 'respectively'a,j′bThen the scaling ratio can be expressed as:
Figure BDA00021204397400000510
assigning a set of expansion ratios to a template framework, and applying framework expansion transformation to a template grid to obtain a template model consistent with a target local scale; after the scales are consistent, the bone rotation transformation estimation based on the distortion representation is carried out. Unlike continuous frame data used in animation or motion estimation scenarios, where the pose difference of the template and the target generally has a large magnitude; for iterative estimation of a rotation angle, firstly, initializing a template framework by using the obtained target framework node information, and reducing the posture difference between the template and a target; for this purpose, the template skeleton is pre-estimated from the parent to child rotation angles top-down.
The rigid transformation of human body is estimated by the state in the root node space, and for a certain bone, the rotation angle can be estimated by the rotation angle of the vector corresponding to the template and the target bone, and the rotation angle theta and the rotation axis omega can be respectively expressed as:
Figure BDA00021204397400000511
Figure BDA0002120439740000061
according to
Figure BDA0002120439740000062
And
Figure BDA0002120439740000063
the initialized skeleton rotation and translation vector can be obtained
Figure BDA0002120439740000064
Can change theta0Is initialized to
Figure BDA0002120439740000065
To substantially align the template skeleton with the target skeleton; here, it should be noted that
Figure BDA0002120439740000066
And
Figure BDA0002120439740000067
after all the father skeletons corresponding to the current skeleton layer by layer are rotated, the new position of the current skeleton node
Figure BDA0002120439740000068
And
Figure BDA0002120439740000069
the applied rotation angle and the rotation axis are required. In addition, in the process of scaling and rotation angle estimation, some skeleton nodes are father nodes of a plurality of skeletons, and for this purpose, corresponding directions of all skeletons are adoptedThe average value of the quantity calculation results is used as the rotation and scaling parameters of the father node, wherein in order to ensure the smoothness, a double-quaternion method is used for the average of the rotation angles; at the moment, the template skeleton is approximately aligned with the target skeleton globally, but the two groups of feature points
Figure BDA00021204397400000610
And
Figure BDA00021204397400000611
there is also some local rotational difference between them.
For detail adjustment, the rotation adjustment of the framework driven by the characteristic points is needed; wherein the feature points at different positions are restricted to the local parts and corresponding skeleton weights are used, so that the difference energy of the two can be expressed as the whole skeleton
Figure BDA00021204397400000612
Figure BDA00021204397400000613
Where Δ Θ represents a rotation parameter of the bone, and Δ ΘkRepresenting the amount of change, ξ, of the skeleton kjRepresenting initialized skeleton transform vectors
Figure BDA00021204397400000614
Figure BDA00021204397400000615
Is xijIn the form of a matrix; t isa(ΔΘk) In the form of a linear mixture of skeletal transformations, wiRepresents the weight of bone i, A (i) represents the set of adjacent bones of bone i, Δ θjThe variation of the rotation angle of the skeleton j is shown, and the initialized distortion vector is used to keep the rotation and the initialization direction consistent in the optimization process; due to the initialized alignment, from
Figure BDA00021204397400000616
The state starts, changes are small in the process, so that local linearization of the warped representation can be performed, e.g. for the matrix Ta(ΔΘk) At Δ Θ — 0, the taylor expansion can be approximated as:
Figure BDA00021204397400000617
where I represents a unit array, so a linearized equation can be solved:
H·ΔΘ＝b
in the linear equation, H and b represent a coefficient matrix and a target value vector, respectively, which are generated by the energy-constrained objective function calculation. In order to minimize the energy function, the linear equation is solved, new linearization is performed near each iteration result, new iteration is performed, and finally the locally adjusted Δ Θ is obtained.
(2) Posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
a) the Hermite Radial Basis Function (HRBF) local field pose adjustment algorithm is as follows:
the human HRBF local field is shown in FIG. 4; the local posture adjustment is carried out in a mode of estimating the rigid transformation of a local HRBF field to enable the rigid transformation to be as close to the local posture of a target human body as possible; to estimate the local surface of the target human body, the target data is discretized spatially to obtain a Signed Distance Field (SDF) phi of the target datatarAnd obtaining the gradient of the field
Figure BDA0002120439740000071
And driving the central point set by using the spatial field, and indirectly driving the deformation of the HRBF local field so as to be close to the local shape of the target data. After the target SDF is obtained, the drive vector for the HRBF center point may be expressed as
Figure BDA0002120439740000072
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and finally enabling the central point to fall on the surface of the target data through proper iteration times; at the central point to the end position
Figure BDA0002120439740000073
Then, in order to acquire information on the target surface, the normal direction thereof is set to
Figure BDA0002120439740000074
Thus, obtaining a surface alignment result with the HRBF field central point as a core; to align the local HRBF field as closely as possible to the local part of the target, the energy equation for the pose adjustment may be expressed as
Figure BDA0002120439740000075
Here TpAnd T in the aboveaSimilarly, when the variation of the template skeleton rotation is expressed, and the rotation axis is estimated by using the rotation transformation of the central point motion set about the local node, the rotation angle can be estimated by using the quaternion expression average, and the iterative solution can be performed by using the similar local linearization method. Here the normal energy constraint of the target surface is added
Figure BDA0002120439740000076
And is
Figure BDA0002120439740000077
Represents and vcThe matched central point, that is, after each iteration is finished, the target matching point of each central point is updated according to the normal and the position relation by the ICP methodAnd then the next iteration is carried out. It should be noted that the twist vector is consistent with the rough alignment stage to ensure the continuity of rotation; and obtaining the adjustment result of the human body posture after the iteration end point is reached.
b) The HRBF local field region adjustment algorithm is as follows:
the expansion and contraction of the HRBF field are utilized to provide a more appropriate local scale state as an initialization input of the fitting stage. When the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, estimating a deformation end point state of HRBF in advance, and then estimating a proper gamma value of local field expansion and contraction through center point drive fitting after each iteration; in knowing the motion endpoint set of HRBF center point
Figure BDA0002120439740000078
After, the energy can be expressed as:
Figure BDA0002120439740000079
it was observed that the solution to the target energy function can in fact be written directly, since the unknowns are only a constant quantity
Figure BDA00021204397400000710
Wherein N iscRepresenting the number of the central points (except additional points), when gamma is used as an expansion and contraction coefficient to be applied to a local field, the central points reach new positions through movement, then obtaining the state of an Euler vector field at the new positions where the central points reach, and enabling the central points to move to the target surface through the Euler field driving in the state to obtain a new end point set; through a few iterations, the expansion and contraction coefficients of the region can be obtained.
(3) Human body reconstruction based on Euler field target energy function drive evolution: based on the SDF generated by the template grid aligned in posture, a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in the Euler field is optimized, a deformation vector field is calculated, an SDF deformation result is obtained, and the template grid and a target model are driven to be aligned in shape so as to realize three-dimensional human body reconstruction.
Energy constraints constructed based on the Euler field SDF are divided into three parts, namely target data energy constraints, human body deformation energy constraints and energy constraints of a level set; the overall energy equation can be expressed as:
Eshape＝wdEdata+wbEbody+wlElsm
wherein E isdata、Ebody、ElsmRespectively representing the target data energy, the human body deformation energy and the energy of the level set, wd， wb，wlRespectively representing the weight coefficients of the target data energy, the human body deformation energy and the energy of the level set, and respectively introducing the three energy terms;
first, to align the template model with the target model surface as much as possible, in implicitly expressed SDF, it means that the value difference of each node of the SDF field for the two surfaces is as small as possible. The change of the node value is realized by vector field driving, so that the problem is converted into energy constraint on the vector field; assume template model SDF as phitmpThe target model SDF is phitarSDF driven by a vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field can be expressed as
Figure BDA0002120439740000081
The summation operation here means that the results of all the nodes participating in the operation in the SDF are summed.
Secondly, since the target data may have noise, affecting the quality of its SDF, if only data item constraints are used, it will likely cause uncontrollable local deformation in the noisy region; the local field generated by the Hermite Radial Basis Function (HRBF) has a good constraint effect in regional deformation, and the combined field generated by the local field has the characteristics of smooth deformation and human body deformation with different regions, so that the vector field Ψ can be provided with smooth energy constraint based on human body deformation by means of deformation driving of the HRBF field, and the constraint can be written as:
Figure BDA0002120439740000082
here JΨI.e., the Jacobian matrix of Ψ, and
Figure BDA0002120439740000084
then represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics can be reflected in
Figure BDA0002120439740000083
Thus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic restriction of human body can be obtained, so that phi is driventmpThe human body deformation characteristic is better embodied during evolution.
In addition, to ensure the level set characteristics of the SDF field, it is also necessary to satisfy the requirements after each iteration
Figure BDA0002120439740000091
I.e. to ensure that its iso-surface dimension is 1.
Here, a gradient descent method is used to find a solution that minimizes the energy function through a first order iterative update. Because the optimization object is a function in a three-dimensional space, a variational method is required to be used for derivation of an energy function by using an Euler-Lagrange equation; the derivative of each energy term is expressed as follows:
Figure BDA0002120439740000092
Figure BDA0002120439740000093
Figure BDA0002120439740000094
wherein,
Figure BDA0002120439740000095
represents phi after vector field drivetmpThe field gradient of (a) is determined,
Figure BDA0002120439740000096
is indicative of phitmpHessian matrix (Hessian matrix), which is composed of its second order partial derivatives about three coordinate axis directions x, y, z; in a similar manner, the first and second substrates are,
Figure BDA0002120439740000097
Figure BDA0002120439740000098
wherein
Figure BDA0002120439740000099
Then is the hessian matrix representing the respective components u, v, w of the vector field Ψ in the x, y, z directions; note that the modulus of the field gradient at the denominator position uses a subscript e, expressed as a constant plus a small quantity e 10-5The divisor is avoided to be 0.
After respective derivative expressions are obtained, iteration is carried out on the vector field under a gradient descent framework, and the expressions are recurred
Ψnew＝Ψ-τE′shape(Ψ)
Where π represents the step size, E ', of the gradient descent iteration'shapeIn one update of the representationDerivative expression of total energy, ΨnewRepresenting a new state of the vector field after one iteration; when the update scale of Ψ is smaller than 0.01, the update stops.
Driven by vector field, corresponding phi of template modeltmpPhi can be generated with the target datatarLocal shape alignment; however, the triangular mesh of the template model needs to be aligned with the target shape, so that the template model depending on skeleton driving has a complete reconstruction form, and local shape details of a human body may be locally lost due to the spatial discretization corresponding to the SDF, which requires the deformation of the triangular mesh to migrate the details to the reconstruction result.
First, due to phitmpIt is only one shape estimate of the triangular mesh, which means that the SDF value corresponding to the spatial position of the vertex of the triangular mesh is not necessarily 0; it is therefore necessary to record that the triangular grid points correspond to phi while generating the SDFtmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Figure BDA00021204397400000910
Wherein v isiWhich represents the grid points,
Figure BDA00021204397400000911
indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmp(Ψ) and a new value in
Figure BDA00021204397400000912
Then represents the corresponding position in the vector fieldtmpGradient of (Ψ)
Figure BDA00021204397400000913
Driven using this formula, v can then be scaled by the appropriate step size ηiMove to the original isosurface position, phaseWhile following the SDF surface.
It should be noted that in global SDF, areas of the body where different areas are closely spaced from each other tend to occur, such as gaps between the arms and the body; at this time, the iteration of the grid points should be stopped in time according to the change of the field gradient, so as to avoid the problem of self-intersection which may occur.
Secondly, when the grid points are driven by the SDF in an unconstrained state, tangential distortion can be caused, unnecessary noise is introduced, and the original grid quality is even reduced; for this purpose, a local tangential relaxation method is used, and the grid points are moved to the weighted centers of the 1-ring neighborhood points of the grid points according to the weights in each iteration; to achieve the purpose, in an initial state, a Mean value coordinate (Mean coordinate) method is used for grids to obtain a projection set q of 1-ring neighborhood points of grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
Figure BDA0002120439740000101
Figure BDA0002120439740000102
denotes the position of the vertex after tangential correction, where the weight μ can be expressed as a function of ψ (v)i) Varying smoothing function
Figure BDA0002120439740000103
Here, the
Figure BDA0002120439740000104
Representing the initial values after the SDF has been changed without the grid points being driven, as already mentioned above, ψ0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d). It can be seen that a relatively large drive will be performed at the beginning of the driveAnd (4) tangential relaxation, and gradually reducing the relaxation to 0 when the target isosurface is approached.
Furthermore, since the driving process is not a global energy constraint, some regions may generate sharp distortions, in which case Laplacian smoothing (Laplacian smoothing) is used at the corresponding region points. The triangular meshes before and after the SDF evolution are driven by proper step length, so that the triangular meshes are deformed into target shapes.
Fig. 5 shows human body reconstruction output results of different data inputs, and fig. 5(a), (b), (c), and (d) respectively show a target data model, a reconstruction result, and an error thermogram from left to right, where the range of the error thermogram is 0 to 4cm, and the errors sequentially correspond to colors with different shades from small to large, so that the size of the error can be determined by the shade of the color, and it can be known from the figure that the lighter the color is, the larger the error is. The target data in fig. 5(a) is from the student's Kinect scan fusion data, the target data in fig. 5(b) is from the cause dataset, the target data in fig. 5(c) is from the human body dataset created by Shahu et al, and fig. 5(d) is the MIT multi-view reconstruction dataset.
From the reconstruction results, fig. 5(a) and (b) show that the model only worn by underpants can better recover the shape of the human body, and the reconstruction result has less deviation when the noise of the target data is small, and of course, when the target itself is a high-precision scanning model, the accuracy after reconstruction is lost compared with the FAUST, which is because the method of the present invention depends more on the details of the human body of the template model, and thus the method focuses on dealing with rough data.
Fig. 5(c) and (d) show the dressing model, for the reconstruction of the dressing model, the HRBF local field in the shape alignment must use a smaller sampling rate, and the weight of the target energy constraint needs to be reduced to avoid causing some distortion introduced by the clothes. However, if the reconstruction result is simply compared with the target data, the grid errors in fig. 5(c) and (d) are larger, but it can be seen that fig. 5(c) and (d) have good estimation of the human body shape. Therefore, the method can reconstruct a good human body shape model for different types of human body data, particularly rough human body surface data.
Technical contents not described in detail in the present invention belong to the well-known techniques of those skilled in the art.
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all inventions utilizing the inventive concept are protected.

Claims (6)

1. A human body geometric reconstruction method based on Euler field deformation constraint is characterized by comprising the following three steps:
(1) roughly aligning the human body postures based on the characteristic point driven skeleton deformation: extracting human body feature points by using a random forest regression method by taking a target human body grid as a data source, and calculating through the feature points to obtain target human body skeleton nodes; driving the skeleton to deform by using the characteristic points, and driving the grid to deform by using a linear hybrid skin LBS method, so that the postures of the template model and the target model are roughly aligned;
(2) posture adjustment and region deformation based on HRBF local field constraint: performing rigid transformation driven by local shape difference on the basis of a local field generated by HRBF; and performing region expansion and contraction based on local shape driving;
(3) human body reconstruction based on Euler field target energy function drive evolution: optimizing a target energy function based on target data constraint, human body deformation constraint and level set characteristic constraint in an Euler field based on the SDF generated by the template grid aligned in posture, calculating a deformation vector field and obtaining an SDF deformation result, and driving the template grid to be aligned with the target model according to the shape to realize three-dimensional human body reconstruction;
the specific process of the step (2) is as follows: the local posture adjustment adopts the rigid transformation of the estimated local HRBF field to estimate the local surface of the target human body, carries out space discretization on the target data,obtaining a symbol distance field phi of target datatarAnd obtaining the gradient of the field
Figure FDA0003488472020000011
Driving a central point set by using a spatial field, and indirectly driving the deformation of the HRBF local field to be close to the local shape of target data; after the target SDF is obtained, the drive vector for the HRBF center point is expressed as
Figure FDA0003488472020000012
Wherein phi istar(vc) Is indicated at point vcValue of field function of (n)cDenotes vcThe direction of the field gradient, where a particular field function value at the center point is obtained using a tri-linear interpolation of the spatial node values; with u (v)c) For vcDriving, and enabling the central point to finally fall on the surface of the target data through iteration; at the central point to the end position
Figure FDA0003488472020000013
Then, the normal direction is set to
Figure FDA0003488472020000014
Figure FDA0003488472020000015
Thus, obtaining a surface alignment result with the HRBF field central point as a core; to align the local HRBF field as closely as possible to the local part of the target, the energy equation for the pose adjustment is expressed as
Figure FDA0003488472020000016
Wherein E ispose(Δ Θ) represents the energy of the pose adjustment; delta thetakRepresents the variation of a skeleton k, k represents the skeleton, T represents a T-type,
Figure FDA0003488472020000017
representative point, TpRepresenting the rotation variation of the template skeleton, and when a rotation axis is estimated by using the rotation transformation of a central point motion set about local nodes, expressing the average by using quaternion, and estimating the rotation angle, iteratively solving by using a local linearization method; here the normal energy constraint of the target surface is added
Figure FDA0003488472020000018
And is
Figure FDA0003488472020000019
Represents and vcAfter each iteration is finished, updating the target matching point of each central point according to the normal and the position relation by an ICP method, and then performing the next iteration; the distortion vector is consistent with the rough alignment stage, and after the iteration end point is reached, the adjustment result of the human body posture is obtained;
providing a state of a local scale by utilizing expansion and contraction of an HRBF field as initialization input of a fitting stage; when the local field is used for attitude adjustment, the HRBF field is used as the estimation of the local shape, so that the central point set is driven to the target curved surface; firstly, estimating a deformation end point state of HRBF in advance, and estimating a proper gamma value of local field expansion and contraction through center point drive fitting after each iteration; in knowing the motion endpoint set of HRBF center point
Figure FDA0003488472020000021
After, the energy can be expressed as:
Figure FDA0003488472020000022
Emodify(y) represents the adjusted energy, and since the unknowns are only a constant, the solution to the target energy function can be written directly
Figure FDA0003488472020000023
Wherein N iscRepresenting the number of the central points except the additional points, when gamma is used as an expansion and contraction coefficient to be applied to a local field, the central points reach new positions through movement, then the state of an Euler vector field is obtained at the new positions where the central points reach, and the central points are moved to the surface of a target through the Euler field driving in the state to obtain a new end point set; through iteration, the expansion and contraction coefficients of the region can be obtained;
the specific process of the step (3) is as follows: driving evolution by using a spatially discretized Euler field target energy function, and reconstructing the shape; discretizing the space where the template and the target human body grid are located to generate an SDF, then establishing a target energy function based on an Euler space by using target data constraint, human body deformation constraint and level set constraint, and estimating a vector field of the alignment template and the target SDF by using a gradient descent method; and after the SDFs are aligned, driving the template grids to a target shape by using the SDFs, and finishing the structured human body reconstruction of the target data.
2. The geometric reconstruction method of human body based on Euler field deformation constraint of claim 1, wherein in step (1), the template model is in T-shaped static posture.
3. The Euler field deformation constraint-based human geometry reconstruction method of claim 1, wherein the template model SDF is assumed to be φtmpThe target model SDF is phitarSDF after being driven by vector field is phitmp(Ψ), then the energy that constrains the alignment of the template model vector field to the target model vector field is represented as:
Figure FDA0003488472020000024
wherein E isdata(Ψ) represents the target data energy, Ψ is a vector field, and the summation operation herein represents the summation of the results of all the participating operational nodes in the SDF;
with deformation driving of the HRBF field, a smooth energy constraint based on body deformation is provided for the vector field Ψ, written as:
Figure FDA0003488472020000025
wherein E isbody(Ψ) represents the body deformation energy, JΨI.e., the Jacobian matrix of Ψ, and
Figure FDA0003488472020000026
then represents ΨbodyA Jacobian matrix of; here ΨbodyMeans that phi can be adjustedtmpPhi and phiHRBFAligned vector field of phiHRBFGenerated by a combined field of local HRBF fields deformed to an end point driven by the target SDF, so psibodyAlso has the characteristics of smoothness and local deformation of human body, and the characteristics are reflected in
Figure FDA00034884720200000310
Thus when Ψ vector field local characteristics and ΨbodyWhen approaching, local deformation characteristic constraint of human body can be obtained, so that phi is driventmpThe human body deformation characteristics are better embodied during evolution;
to ensure the level set characteristics of the SDF field, satisfied after each iteration
Figure FDA0003488472020000031
Etsm(Ψ) represents the energy of the level set, i.e., ensuring that its iso-surface scale is 1.
4. The Euler field deformation constraint-based human body geometric reconstruction method according to claim 1, whereinIn that, while generating the SDF, the recording triangular grid points correspond to phitmpTrue value psi0At phitmpWhen the driving deformation is carried out in the vector field, the grid points are also driven along with the SDF change as follows
Figure FDA0003488472020000032
Wherein v isiWhich represents the grid points,
Figure FDA0003488472020000033
indicating the new position after driving of the grid points, # (v)i) Representing grid points viAfter the vector field changes, in the vector field phitmpNew value in (Ψ), Ψ0(vi) Representing triangular grid points viCorresponds to phitmpTrue value of
Figure FDA0003488472020000034
Then represents the corresponding position in the vector fieldtmpGradient of (Ψ)
Figure FDA0003488472020000035
Driven using this formula, v is driven by the step length ηiMoving to the original iso-surface position is equivalent to following the SDF surface.
5. The Euler field deformation constraint-based human body geometric reconstruction method according to claim 1, wherein in the SDF driving process, a local tangential relaxation method is used, and in each iteration, a grid point is moved to a weighted center of a 1-ring neighborhood point of the grid point according to weight; in the initial state, a mean coordinate method is used for the grids to obtain a projection set q of 1-ring neighborhood points of the grid points on a tangent plane of the grid pointsijAnd the coordinate of the center of mass is bijIf v ═ Σ is satisfied in the initial statejbijqijAnd in iteration, the following formula is used for correction:
Figure FDA0003488472020000036
wherein,
Figure FDA0003488472020000037
representing the tangentially modified vertex position, the weight μ is expressed as a function of ψ (v)i) Varying smoothing function
Figure FDA0003488472020000038
Figure FDA0003488472020000039
Representing the initial value, ψ, after the grid points have not been driven and the SDF has been changed0(vi) Indicates that the triangular mesh points correspond to phitmpThe true value of (d); a relatively large tangential relaxation will be performed at the beginning of the drive, and a gradual reduction of the relaxation up to 0 is chosen as the target isosurface is approached.
6. The euler field deformation constraint-based human geometry reconstruction method according to claim 5, wherein when some regions generate sharp distortion, Laplacian smoothing (Laplacian smoothing) processing is used at the corresponding region points.
CN201910605065.1A 2019-07-05 2019-07-05 Human body geometric reconstruction method based on Euler field deformation constraint Active CN110619681B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910605065.1A CN110619681B (en) 2019-07-05 2019-07-05 Human body geometric reconstruction method based on Euler field deformation constraint

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910605065.1A CN110619681B (en) 2019-07-05 2019-07-05 Human body geometric reconstruction method based on Euler field deformation constraint

Publications (2)

Publication Number Publication Date
CN110619681A CN110619681A (en) 2019-12-27
CN110619681B true CN110619681B (en) 2022-04-05

Family

ID=68921459

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910605065.1A Active CN110619681B (en) 2019-07-05 2019-07-05 Human body geometric reconstruction method based on Euler field deformation constraint

Country Status (1)

Country Link
CN (1) CN110619681B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111476901B (en) * 2020-04-10 2023-04-07 中国科学技术大学 Three-dimensional human body shape representation method
CN111311732B (en) * 2020-04-26 2023-06-20 中国人民解放军国防科技大学 3D human body grid acquisition method and device
CN111862318A (en) * 2020-07-28 2020-10-30 杭州优链时代科技有限公司 Digital human body fitting method and system
CN112785712B (en) * 2021-01-25 2024-09-17 新东方教育科技集团有限公司 Three-dimensional model generation method and device and electronic equipment
CN112991444B (en) * 2021-02-10 2023-06-20 北京字跳网络技术有限公司 Position determination method and device
CN113034690A (en) * 2021-03-04 2021-06-25 广州虎牙科技有限公司 Human body model fitting method and device and electronic equipment
CN116051696B (en) * 2023-01-10 2023-12-22 之江实验室 Reconstruction method and device of human body implicit model capable of being re-illuminated
CN117519150B (en) * 2023-11-02 2024-06-04 浙江大学 Autonomous implicit indoor scene reconstruction method combined with boundary exploration

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108665537A (en) * 2018-05-15 2018-10-16 清华大学 The three-dimensional rebuilding method and system of combined optimization human body figure and display model
CN109840940A (en) * 2019-02-11 2019-06-04 清华-伯克利深圳学院筹备办公室 Dynamic three-dimensional reconstruction method, device, equipment, medium and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9928663B2 (en) * 2015-07-27 2018-03-27 Technische Universiteit Delft Skeletal joint optimization for linear blend skinning deformations utilizing skeletal pose sampling

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108665537A (en) * 2018-05-15 2018-10-16 清华大学 The three-dimensional rebuilding method and system of combined optimization human body figure and display model
CN109840940A (en) * 2019-02-11 2019-06-04 清华-伯克利深圳学院筹备办公室 Dynamic three-dimensional reconstruction method, device, equipment, medium and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Reconstruction of body geometry on unstructured meshes when using immersed boundary method;Abalakin I V等;《Matematicheskoe modelirovanie》;20161231;全文 *
Surface reconstruction from unorganized points with l0 gradient minimization;Li H等;《Computer Vision and Image Understanding》;20181231;全文 *

Also Published As

Publication number Publication date
CN110619681A (en) 2019-12-27

Similar Documents

Publication Publication Date Title
CN110619681B (en) Human body geometric reconstruction method based on Euler field deformation constraint
Jiang et al. Simplicial complex augmentation framework for bijective maps
Sumner et al. Mesh-based inverse kinematics
US10586371B2 (en) Motion retargeting method for character animation and apparatus thererof
CN109711048B (en) Wing design method considering aerodynamic and structural multi-type design variables
Nealen et al. Fibermesh: designing freeform surfaces with 3d curves
CN102982578B (en) Estimation method for dressed body 3D model in single character image
US7843456B2 (en) Gradient domain editing of animated meshes
CN103854306A (en) High-reality dynamic expression modeling method
CN114429538B (en) Method for interactively editing nerve radiation field geometry
JP2023524252A (en) Generative nonlinear human shape model
CN105427364B (en) A kind of production method of multi-point touch 2 D animation
CN106484511A (en) A kind of spectrum attitude moving method
CN105957154B (en) A kind of the threedimensional model edit methods and system of data-driven
CN111274732B (en) Grid repairing method based on 'connection relation-position' iterative optimization
Dey et al. Eigen deformation of 3d models
Cheng et al. Design and analysis of optimization methods for subdivision surface fitting
Yang et al. Finite element mesh deformation with the skeleton-section template
CN108876922A (en) A kind of mesh amending method based on the regularization of interior dihedral angle supplementary angle
Salval et al. Multiresolution surface blending for detail reconstruction
CN109801367B (en) Grid model characteristic editing method based on compressed manifold mode
Wu et al. Freescup: A novel platform for assisting sculpture pose design
Au et al. Mesh editing with curvature flow laplacian operator
Pang et al. Developing an image-based 3D model editing method
Morigi Geometric surface evolution with tangential contribution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant