KR20030040701A - Estimation method of deformation of three dimension polyhedral surfaces - Google Patents

Abstract

PURPOSE: A method for estimating transformation of a three-dimensional polyhedral surface is provided to reliably and accurately estimate a complex and minute motion of a mesh through a multi-resolution. CONSTITUTION: An input mesh and a target mesh are simplified to generate a basic input mesh and a basic reference mesh. A motion is estimated from the generated basic input mesh and the basic reference mesh. Resolutions of the basic input mesh and the basic reference mesh are increased by referring the estimated motion and a new motion is estimated. The basic input mesh is transformed by referring the estimated motion. It is judged whether the transformed mesh satisfies preliminarily set conditions.

Description

Estimation method of three-dimensional polyhedral surface {ESTIMATION METHOD OF DEFORMATION OF THREE DIMENSION POLYHEDRAL SURFACES}

The present invention relates to a polyhedral representation technique, and more particularly, to a method for estimating deformation of a three-dimensional polyhedron surface.

In general, a two-dimensional image frame or frame sequence may represent a static or dynamic three-dimensional object.

However, the two-dimensional image frame shows only one cross section of the object and lacks depth information.

Therefore, polyhedral surfaces are widely used to describe three-dimensional objects in computer vision and computer animation.

In addition, in order to transmit visual characteristics of a moving 3D object, information about the shape and motion of the object is required.

One of these objects is the human face with various expressions.

When modeling a human face using polyhedral surfaces, it is necessary to precisely modify the face surface in order to express various facial expression changes, and sometimes it is necessary to manually modify the surface.

However, when a moving object is represented by the surface of a polyhedron, the surface must be manually deformed, which is not precise, and there is a problem that only a limited burden is used.

Therefore, it is necessary to automatically estimate the deformation of the polyhedral surface.

Accordingly, an object of the present invention is to provide a method for estimating deformation of a three-dimensional polyhedron, which is designed to automatically estimate free-form motion of a polyhedral surface to accurately represent a changing surface of an object having a specific motion.

1 is an exemplary view showing distortion of a projection due to a large displacement.

2 is an illustration showing separation of vertices.

3 to 7 are exemplary diagrams illustrating deformation estimation when a 'Za' is pronounced from a normal state in an embodiment of the present invention.

3A-3D show displacement field area estimates at the base level with a simplified input mesh having 300 faces and 168 vertices,

4A-4D show displacement field area estimates at a level where the simplified input mesh has 500 faces and 275 vertices,

5A-5D show the displacement field area estimation at a level where the simplified input mesh has 1000 faces and 534 vertices,

6A-6D show the displacement field area estimation at a level where the simplified input mesh has 2000 faces and 1047 vertices,

7A-7D show displacement field area estimation at a level with finer level 5000 faces and 2566 vertices from the result of displacement field area estimation at a level where the simplified input mesh has 2000 faces and 1047 vertices.

8 to 12 show the deformation estimation of the shape when the "Zui" pronunciation in the "Ze" pronunciation in the embodiment of the present invention,

8A-8D show the displacement field area estimation at the level where the simplified input mesh has 300 faces and 168 vertices,

9A-9D show the displacement field area estimation at a level where the simplified input mesh has 500 faces and 275 vertices,

10A-10D show the displacement field area estimation at a level where the simplified input mesh has 1000 faces and 534 vertices,

11A-11D show the displacement field area estimation at a level where the simplified input mesh has 2000 faces and 1047 vertices,

12A-12F show displacement field area estimation at a level with finer level 5000 faces and 2566 vertices from the result of displacement field area estimation at a level where the simplified input mesh has 2000 faces and 1047 vertices.

FIG. 13 is an operational flowchart showing deformation of an input mesh according to a specific movement in an embodiment of the present invention. FIG.

In order to achieve the above object, the present invention uses a multiresolution technique for effectively estimating complex surface changes, and uses a new energy function for effectively estimating motion at each resolution.

That is, in order to achieve the object of the present invention, a basic input mesh is subjected to a simplification process for a complex input mesh to be deformed and a target mesh having a form to be finally deformed. And generating the base reference mesh as a coarsest mesh, and a motion vector field represented by three-dimensional movement of each vertex of the base input mesh from the base input mesh and the base reference mesh. Estimating the three-dimensional motion by detecting the three-dimensional motion, and predicting the new motion by referring to the three-dimensional motion estimated by increasing the resolution of the basic input mesh and the basic reference mesh and using the local geometric structure. Minimizing the stiffness constant to control the energy function and the fairness constant to smooth the mesh surface A step of correcting, projecting various points on the base reference mesh surface to the surface of the base input mesh with reference to the predicted movement corrected in the above step, and deforming to a high resolution mesh; Repeating the above steps until the condition is satisfied.

When the estimated deformation projection to the high resolution mesh is large and the mesh is complex at the same time, the normal vector, which is a vector perpendicular to the surface at two points between coincidence points, is generally similar in direction. A point on the mesh ( Tangent at) ) And the match point in the input message ( ) Is any point on the input mesh ( ) Is configured to satisfy the following conditions.

In the above formula Is Is the tangent to Is the given threshold.

In addition, in order to prevent inaccurate deformation in planes having some projection points by the same weight in the estimation projection of the vertex displacement, a weighting factor such as the following expression is applied in the estimation deformation projection to improve the projection effect. It features.

In addition, the motion estimation in the subdivided mesh may be expressed as a total energy function in consideration of the stiffness constant and the fairness constant, which may be expressed by the following equation.

Where stiffness constant ( ) Decreases toward higher resolutions and the fairness constant ( ) Increases as the resolution increases.

The fairness constant ( ) Is the stiffness constant ( ) Is characterized by having a smaller value.

Hereinafter, the present invention will be described in detail with reference to the drawings.

First, a mesh simplification process is performed for each input mesh to be deformed and the target mesh to be finally deformed to generate a coarse mesh as maximum as possible.

In this case, the roughest input mesh generated by the mesh simplification process is called a basic input mesh, and the roughest target mesh is called a basic reference mesh.

In general, the base reference mesh should have slightly more faces and vertices than the base input mesh.

An edge-collapse method may be applied to the mesh simplification process.

The mesh simplification process using the edge-collapse method calculates positions of all vertices, selects valid pairs for all vertices, calculates new positions for the selected valid pairs, and heaps the valid pairs. Repeat the step of aligning (heap) to reduce the edge.

Then, by comparing the base input mesh and the base reference mesh to estimate the deformation displacement for any movement.

However, various deformations for expressing a 3D object may be expressed as a freeform deformation using a polyhedron mesh, which is closely related to each vertex of the mesh.

Closely related to each vertex means that the vertices of one mesh are moved to their corresponding points in the other mesh.

That is, when two meshes have different phase structures, the vertices of one mesh may correspond to one point on a surface instead of the vertices of another mesh.

Therefore, in the present invention, the deformation displacement estimation for minimizing the distance between the vertices associated with the vertices of the basic input mesh from the vertices of the basic input mesh is based on the affine transformation estimation and the weighted least square method. It is executed by dividing it into the following description.

First, the estimation of global affine transformation will be described.

When the two meshes are slightly diffrent, the maximum closest point on the two meshes is considered as the corresponding points.

In order to deform one mesh as closely as possible to another mesh, it is necessary to repeatedly minimize the distance between many related points.

In an embodiment of the present invention, an ICP algorithm is applied to accurately estimate a rigid body motion.

The ICP algorithm is executed by the following process.

(a). Find the closest spot on each base input mesh for each point on the base reference mesh.

(b). Compute the affine transform matrix.

(c). The base matrix is modified by using the affine transformation matrix calculated above.

(d). Calculate the value of the objective function.

(e). If the change of the calculated value is larger than the set threshold, the process is repeated. On the contrary, if the change of the calculated value is smaller than the set threshold, the process is repeated.

The ICP algorithm is described in B.K.P. The method proposed by Horn will be briefly described as an example.

First, a rotation term (r) and a transition term (d) are estimated to minimize Equation 1 below.

Where xi and pi are the two closest points.

At this time, the multiple point set of the reference mesh ( Center of ) And a multipoint set of input meshes ( Center of ) Are defined as shown in [Equation 2] below.

Also, the point set of the input mesh ( ) And the point set of the reference mesh ( The covariance matrix between) is defined as in Equation 3 below.

here, Is the periodic component of the asymmetric matrix (A).

A column vector forming a symmetric matrix represented by Equation 4 below ).

The symmetric matrix ( The unit Eigen vector corresponding to the maximum Eigen value of) has information about the circulant matrix r.

The rotation matrix (r) is expressed by Equation 5 below.

Therefore, the translation vector (d) is expressed as Equation 6 below.

Next, an estimation of free-form motion using weighted least square method will be described.

Most mesh deformations can be represented using the global affine transform of the mesh and the local displacement of each vertex.

In this case, the global affine transformation represents a large rigid movement and the local displacement represents a small freeform deformation.

In expressing deformation of the mesh by using a displacement field defined at all vertices of the input mesh, the mesh to be deformed may be expressed as in Equation 7 below.

If we assume that the two meshes have different phase structures, then the distance between the point in the reference mesh and the maximum proximity point in its input mesh will be very small if the input mesh is correctly transformed into the reference mesh.

At this time, the distance between the point in the reference mesh and the closest point in the input mesh is difficult to be exactly '0', because the two meshes have different phase structures.

Therefore, the objective function for the estimation of the mesh deformation is defined as in Equation 8 below.

Where v _i is the vertex of the input mesh, d _i is the displacement of the vertex v _i of the input mesh, Is the vertex you want to project onto the input mesh, Is Is a spot located on one side of the projected input mesh.

And, Is the displacement field at each vertex of the defined input mesh.

This equation can be solved using the conjugated gradient method.

Then, when the deformation displacement is estimated by the above process, the input mesh is deformed by moving each vertex of the input mesh using the estimated motion vector at each vertex.

However, assuming that the maximum proximity point is the corresponding point during projection, the corresponding point ( ) Is selected as shown in Equation 9 below.

Where p _o is a point in one mesh and pi, pj is a point in another mesh.

Most vertex displacement estimates then depend on whether many points on the target mesh are projected onto the plane connected to the vertices.

If a portion of the mesh has a large movement, the maximum proximity point is not the corresponding point and the estimation cannot be made accurately.

Therefore, the present invention uses a normal vector of two corresponding points having similarities to solve the above problem.

That is, [Equation 9] is modified to have the same characteristics as [Equation 10] below.

In other words, in finding the matching point by projecting several points on the reference mesh surface onto the surface of the input mesh, a closest point is assumed to be the matching point when the difference between the two meshes is small. However, if the difference between the two meshes is large and the shape of the mesh is complicated at the same time, it is not accurate to set the closest point as the coincidence point.

In this case, we use this information because a normal vector, a vector perpendicular to the surface at two points between coincidence points, is generally similar in direction.

That is, one point of the reference mesh ( Tangent at) ) And the match point in the input message ( ) Is any point on the input mesh ( ), The following Equation 10 must be satisfied.

From above Is Is the tangent to Is the given threshold.

On the other hand, since the projection may occur incorrectly during the projection as described above by applying a weighting factor (weighting factor) to ensure accurate projection.

If the displacement of the vector is very large, most points are projected to the corresponding point and the remaining points are not projected to the corresponding point because of incorrect projection.

That is, because of incorrect projection, most points are projected onto some faces connected to the vertices, while some of the other points are projected onto other faces connected to the same vertices.

This is as shown in the exemplary diagram of FIG.

Therefore, if all projections are equally weighted in estimating vertex displacement, the faces with some projection points cannot be precisely deformed, so the weighting factor as shown in Equation 11 below is used.

In addition, when deforming the surface of an elastic object, the mesh surface has the property of maintaining the geometric shape in the local area.

This property is represented by similar values in the motion vectors of neighboring vertices.

By adding these conditions, the total energy function is expressed as Equation 12 below.

From above Silver vertex ( Motion vector at) Is the stiffness constant.

However, when the deformation of a complex mesh is very large, the total energy minimization still converges to a local mimimum even when using a stiffness energy function.

To solve this problem, a multiresolution method is used.

The mesh can be deformed into any shape by gradually deforming at multiple resolution levels.

In the same principle, the multi-resolution technique is used in the present invention to reliably and precisely estimate various changes of the mesh.

First, it looks for large movements in the coarsest base mesh, and then gradually finds the correct movements with increasing resolution.

The motion vector of each vertex is newly estimated by subdividing the low resolution mesh into a high resolution mesh.

At this time, the initial value of the motion vector of the newly generated high resolution vertex is predicted from the motion vector of each correction estimated at the low resolution.

In this process, the following problems should be solved.

First, since low resolution meshes are generated through a simplification process, high resolution meshes are created through a reversal processing of the simplification process.

In other words, the segmentation for the transformation from the low resolution mesh to the high resolution mesh is performed in the reverse order of the simplification process. For this purpose, the edge-split method for subdividing the midpoint with the coarse base mesh is applied. Can be.

However, a problem in this process is that the high resolution mesh is not deformed by the motion estimation process but has the original shape.

Although we try to maintain a local geometric structure using stiffness constraints, they are somewhat deformed.

For this reason, high-resolution meshes created in the reverse order of the simplification procedure may have uneven surfaces.

To solve this problem, we added a fairness constraint.

In other words, the way to add a condition is to add fairness energy to the total energy function in the same way as the stiffness constraint.

The total energy function that adds stiffness energy and fairness energy is expressed as in [Equation 13] below.

At this time, the state of the mesh surface and the accuracy of the motion estimation are determined by the stiffness constant ( ) And fairness constant ( Determine by adjusting the value of).

The two values were adjusted according to the following meanings.

First, movement at low resolution estimates a wide range of motion rather than subtle regional changes.

In the high resolution, the error of the motion found in the low resolution and the motion within the narrow range are found.

In order to satisfy these conditions, the stiffness constant should decrease from low resolution to high resolution.

If the stiffness constant is too large, the mesh structure is hardly deformed, so a much smaller motion value is estimated. On the contrary, if the stiffness constant is too small, it is difficult to find a wide range of motion and the surface is flipped locally. It is difficult to estimate reliable movement such as face-flip phenomenon.

In addition, in the case of the fairness constant, when a vertex is predicted from the surrounding vertex, the error is very large at low resolution, and conversely, the error is very small at high resolution.

For this reason, the fairness constant should increase from low resolution to high resolution.

In addition, when the value of the fairness constant is very large, the fairness constant should be smaller than the stiffness constant since excessively burring of the surface shape of the mesh rapidly changing from the vicinity of the nose, mouth, and eyes may occur. .

The process of refining the mesh by separating the vertices is shown in the example of FIG. 2, where vertices v _c in FIG. 2 (a) are separated into vertices v _l and v _r . It is shown.

The motion vector of the newly created vertex is then predicted using the previously estimated motion of the surrounding vertices.

At this time, the displacement vector at vertex v _c and neighboring vertices in the coarse mesh are used to estimate the displacement vector at vertex v _l , v _r .

In this case, two separate fragment vertices v _l and v _r are mapped onto the coarse mesh.

The spots mapped through this process are represented by Equation 14 below.

here, Is the center coordinate.

Thus, the displacement at vertex v _l ( ) Is estimated as in Equation 15 below.

Accordingly, a high resolution input mesh is generated from the low resolution input mesh with reference to the estimated displacement as described above. In this case, the generated high resolution input mesh becomes an initial value when a new motion is estimated at a higher resolution.

Then, the motion is continuously estimated while gradually increasing the resolution of the mesh until a given condition is satisfied.

Meanwhile, an embodiment of the present invention will be described with reference to the exemplary diagrams of FIGS. 3 to 12.

3 to 12 illustrate a case in which a polyhedral target model is deformed into a face shape when a variety of sounds “Za”, “Ze”, and “Zui” are pronounced in a normal face shape.

That is, FIGS. 3A to 7D show deformation estimates of shapes from the normal state until they are pronounced as "Za" sounds, and the respective reference numerals operate in order of increasing order.

First, FIGS. 3A to 3D show the displacement field area estimation of the simplified input mesh at 300 faces and 168 vertices coarest levels. FIG. 3A is normal and FIG. 3B is pronounced Za. 3C shows an estimated displacement field, and FIG. 3D shows a deformed normal with reference to the displacement field.

In other words, when the 'Za' is pronounced in the normal state of FIG. 3A, the strain is transformed into a target mesh as shown in FIG. 3B, the meshes of FIG. 3A and FIG. Reference will be made to the target mesh as shown in FIG. 3d.

4A to 4D show displacement field area estimations at 500 faces and 275 vertices levels, and FIG. 4A is a shape estimated using the results in FIGS. 3A to 3D. 4b is a shape when Za is pronounced, FIG. 4c is an estimated displacement field, and FIG. 4d is a deformed normal.

5a to 5d show displacement field area estimates at 1000 faces and 534 vertices levels, FIG. 5a is a shape estimated using the results in FIGS. 4a to 4d, FIG. 5b is a shape when pronounced Za, Fig. 5C shows the estimated displacement field, and Fig. 5D shows the modified state.

6a to 6d show displacement field area estimates at 2000 faces and 1047 vertices levels, FIG. 6a is a shape estimated using the results in FIGS. 5a to 5d, FIG. 6b is a shape when Za is pronounced, FIG. 6C shows an estimated displacement field, and FIG. 6D shows a modified state.

7A-7D show the displacement field area estimate at a finer level 5000 faces and 2566 vertices from the displacement field area estimation results at the 2000 faces and 1047 vertices levels, and FIG. 7A shows the results in FIGS. 6A-6D. Fig. 7B shows a shape when Za is pronounced, Fig. 7C shows an estimated displacement field, and Fig. 7D shows a right shape in a deformed state.

The process of FIGS. 4 to 7 has been described sequentially after the process of FIG. 3, and each process is the same as the process of FIG. 3.

8A to 12D show deformation estimation of shapes when "Zui" is pronounced in "Ze" pronunciation.

8A-8D show displacement field area estimates at 300 faces and 168 vertices levels, FIG. 8A is a shape when pronounced Ze, FIG. 8B is a shape when pronounced Zui, and FIG. 8C is an estimated displacement field (Estimated displacement field), FIG. 8D shows a deformed Ze state.

9A-9D show displacement field area estimates at 500 faces and 275 vertices levels. FIG. 9A is a representation using results in the previous level in FIGS. 8A-8D, FIG. 9B. Is the shape when pronounced Zui, FIG. 9C shows the estimated displacement field, and FIG. 9D shows the deformed Ze.

10A to 10D show displacement field area estimates at 1000 faces and 534 vertices levels, FIG. 10A is a shape estimated using the results in FIGS. 9A-9D, FIG. 10B is a shape when pronunciation of Zui, Fig. 10C shows an estimated displacement field, and Fig. 10D shows a modified state.

11A-11D show displacement field area estimates at 2000 faces and 1047 vertices levels, FIG. 11A is a shape estimated using the results in FIGS. 10A-10D, FIG. 11B is a shape when pronunciation of Zui, Fig. 11C shows the estimated displacement field, and Fig. 11D shows the deformed state.

12A-12F show the displacement field area estimates at a finer level 5000 faces and 2566 vertices from the displacement field area estimation results at the 2000 faces and 1047 vertices levels, and FIG. 12A shows the results in FIGS. 11A-11D. Fig. 12B shows the right shape, Fig. 12C shows the front shape when pronounced Zui, Fig. 12D shows the right shape, Fig. 12E shows the estimated displacement field front region, and Fig. 12F shows the displacement. Indicates the right side of the field.

As described in detail above, the present invention can reliably and accurately estimate the motion of a complicated and fine mesh through a multi-resolution technique.

In particular, the accuracy and reliability of motion estimation are improved by using the stiffness and fairness constants to control the stiffness energy and fairness energy.

Therefore, the present invention has an effect that can accurately represent the finally deformed mesh without distortion of the surface.

Claims (16)

A first step of generating a base input mesh and a base reference mesh by simplifying the input mesh and the target mesh, a second step of predicting motion from the generated base input mesh and the base reference mesh, and estimating at the second step A third step of newly predicting the motion while increasing the resolutions of the base input mesh and the base reference mesh with reference to the moved motion; and a fourth step of deforming the base input mesh with reference to the motion predicted in the third step; In the fifth step of determining whether the deformed mesh satisfies a preset condition, steps 2 through 5 are repeated until the deformed mesh satisfies the preset condition. Deformation estimation method. A first step of generating a base input mesh and a base reference mesh by simplifying the input mesh and the target mesh, a second step of estimating motion from the generated base input mesh and the base reference mesh, and estimating at the second step A third step of deforming the basic input mesh with reference to the corrected motion, a fourth step of predicting the motion after subdividing the deformed mesh in the third step, and referring to the motion predicted in the fourth step, A fifth step of deforming the fine-grained input mesh, a sixth step of determining whether the deformed mesh satisfies a preset condition in the fifth step, and the mesh deformed in the fifth step does not satisfy the preset condition In the fifth step, the deformed mesh is set as the primary input mesh, and at the same time, the reference mesh is subdivided to generate the primary reference mesh. A method for estimating deformation of a three-dimensional polyhedron surface, wherein the second to seventh steps are repeated until the deformed mesh satisfies a preset condition. The method for estimating deformation of a three-dimensional polyhedron surface according to claim 1 or 2, wherein the simplification process applies an edge-collapse method. The method for estimating deformation of a three-dimensional polyhedron surface according to claim 1 or 2, wherein the estimated motion is a motion vector field represented by three-dimensional movement of each vertex of the input mesh. The method of claim 1 or 2, wherein in the case of the low resolution mesh during motion estimation, the first process of finding the maximum close point on each of the points on the target mesh on the base mesh, and the affine transform matrix, A second process of calculating, a third process of transforming a base matrix using the affine transformation matrix calculated above, a fourth process of calculating a value of an objective function, and And a fifth process of determining whether the change in the calculated value is smaller than the preset threshold is repeated until the change in the calculated value is less than or equal to the preset threshold. Method for estimating strain on dimensional polyhedron surfaces. The method for estimating deformation of a three-dimensional polyhedron surface according to claim 1 or 2, wherein the segmentation process for increasing resolution is applied by an edge-split method performed in the reverse order of the simplification process. The method of claim 1 or 2, wherein the initial value of the predicted motion vector is predicted from a previous estimated motion vector. The method according to claim 1 or 2, wherein in the motion estimation, a point of the reference mesh is used to find a matching point by projecting several points on the surface of the reference mesh to the surface of the input mesh. Tangent at) ) And the match point (p_ {i}) on the input mesh are arbitrary points on the input mesh ( A method for estimating deformation of a three-dimensional polyhedron surface, characterized in that the following conditions are satisfied. From above Is Is the tangent to Is the given threshold. The method for estimating deformation of a three-dimensional polyhedron surface according to claim 1 or 2, wherein the deformation of the input mesh is performed by the following equation. Where v _i is the vertex of the base mesh, d _i is the displacement of the vertex v _i of the base mesh, Is the vertex you want to project to the base mesh, Is Is a spot on one side of the projected base mesh, Is the displacement field at each vertex of the defined base mesh. 10. The method for estimating deformation of a three-dimensional polyhedron surface according to claim 9, wherein a weight is given to the following equation for deformation of the input mesh. The method of claim 1 or 2, wherein the stiffness constant A method of estimating deformation of a three-dimensional polyhedron surface, characterized in that to perform a calculation such as From above Silver vertex ( Motion vector 12. The method of claim 11, wherein the stiffness constant decreases with every mesh deformation from low resolution to high resolution. The method according to claim 1 or 2, further comprising: A method for estimating the deformation of a three-dimensional polyhedron surface, characterized in that to perform a calculation as shown in the following equation to equalize the surface of the mesh. 15. The method of claim 13, wherein the fairness constant increases with every mesh deformation from low resolution to high resolution. The method for estimating deformation of a three-dimensional polyhedron according to claim 1 or 2, wherein the calculation is performed to correct the input mesh by the stiffness constant and the fairness constant. 16. The method of claim 15, wherein the fairness constant value is set to be less than the stiffness constant value.