KR100512565B1 - Method for automatic animation of three dimensions scan face data - Google Patents

Abstract

The present invention uses the range scan data of a three-dimensional face model to find feature points in a face, perform feature-maintaining simplification, and perform an animation of a three-dimensional scan face data using the simplified face model. The present invention relates to detecting contours from a texture image using three-dimensional face scan data as input data, and extracting feature points from a geometric model through the detected contours and texture coordinates of the geometric model. Then, we weight the extracted feature points, obtain quadric errors in the geometric model, keep the feature points as many as the number of inputs, and simplify the face model. Also, in order to animate a simplified face model, expression retargeting is applied to perform a desired animation.As a scan model does not open a mouth during animation, the lips are separated and an animation is applied to make an animation similar to the reality.

Description

Method for automatic animation of three dimensions scan face data}

The present invention relates to an automatic animation method of three-dimensional scan face data, and in particular, finds and maintains feature points on a face using range scan data of a three-dimensional face model, and performs face animation through the simplified face model. It relates to an automatic animation method of the three-dimensional scan face data to perform.

In general, animation refers to a technique for continuously displaying a series of images generated inside a computer on a display screen so that the image looks similar to moving, or such an image. Recently, in computer aided learning (CAI), such animation techniques have been used to enhance the understanding of learners, giving visual effects to the explanation of complex and difficult problems.

The process of modeling a human face using a computer requires a high degree of expertise, a lot of time, and a lot of manual work to avoid creating an unnatural face. Difficulties with real-time facial animation include the problem of reducing the data of many face models, and reducing the number of data to the desired level while maintaining the eyes, eyebrows, nose, and mouth that characterize the face in the process of reducing the data. This is to ensure that the texture image is not distorted when the data is reduced.

Natural facial animation requires a long time by a skilled designer. Whenever a new model is created or a new facial expression is added, the designer has to repeat creating the facial expression.

Conventional three-dimensional animation technology has a problem that the actual quality of the model is reduced by using the deformed model instead of directly using the scanned input face model, and the number of feature points is about 200.

There is also a way to use the new model directly, in which case the lip boundary must exist and the location and number of boundary points must be the same.

Therefore, the present invention takes a lot of time to create a new face that occurs in the conventional face animation as described above, takes a lot of time to create a new expression, to solve all the problems such that the animation is possible only when there is a border of the lips. As proposed for

An object of the present invention is to automatically find and maintain feature points on a face using range scan data of a three-dimensional face model, and to perform face animation through the simplified face model. To provide a way to animate.

The "automatic animation method of the three-dimensional scan face data" according to the present invention for achieving the above object,

Simplify range scan data to reduce modeling time, use motion capture data to reduce animation time, and automatically generate lip boundaries to enable natural animation.

That is, the contour is detected from the texture image using the three-dimensional face scan data as input data, and feature points of the geometric model are extracted from the detected contour and the texture coordinates of the geometric model. After that, the extracted feature points are weighted, quadric errors are obtained from the geometric model, and the feature points are kept as many as the number of inputs and the face model is simplified. To animate the simplified face model, apply expression retargeting to produce the desired animation. However, the scan model doesn't open, so the lips are separated and animate, enabling realistic animation.

Hereinafter, described in detail with reference to the accompanying drawings, preferred embodiments of the present invention according to the technical spirit as described above.

The automatic animation generation technique from the scan face model according to the present invention requires a long time manual work by an experienced designer for the animation of the conventional natural face, and each time a new model is created or a new expression is added, Overcome the problem of having to repeat the process of making facial expressions and quickly apply the desired animation to a new face. The problem with this process is that it takes some trial and error to capture the feature points.

The feature points are divided into two types, one for the feature of the face and the other for the coincidence of the deformation model. The feature points of the face are not simplified and the vertices are preserved. This prevents the appearance of unnaturalness in animating. The facial features are mainly focused on the moving vertices such as the tip of the nose, the ends of the ears, and the edges of the lips. About 20 points are assigned. Match points are used to transform an animated source model into a simplified scan model. RBF (Radial Basis Function) is used, which is a method of interpolation around coincidence points. That is, the coincidence point is exactly set in place, and the remaining points between the coincidence points are moved to an appropriate position through interpolation. This is about 20 points.

1 is a flowchart illustrating an automatic animation method of 3D scan face data according to the present invention.

As shown therein, step S10 of receiving three-dimensional scan face data; Determining whether the received scan data is a texture image (S20); Extracting a feature point from the texture image when the scan data inputted as a result of the determination is a texture image (S30); Extracting geometry information from geometry when the input scan data is not a texture image (S40); Measuring a discrete curvature from the extracted feature point data and the geometry information (S50); (S60) simplifying the face data by removing the points outside the feature by creating a bounding area and removing the points outside the feature area of the face; Generating a boundary line at a lip line for a process of moving the lips (S70); Generating target face data using the simplified face data and the generated lip boundary line (S80); Generating a matching point (S90); Generating modified face data by using the generated coincidence point and the source face data (S100); Generating source animation data from the source face data (S110); The target animation is performed by using the generated target face data, modified face data, and source animation data to generate target animation data (S120).

The automatic animation method of the three-dimensional scan face data according to the present invention as described above will be described in detail with reference to the accompanying drawings.

First, in step S10, the 3D scan face data is received. 7A to 7D are exemplary views of three-dimensional face scan data. 7A illustrates a mesh, FIG. 7B illustrates a texture image, FIG. 7C illustrates scan data, and FIG. 7D illustrates a texture mapping state.

The input 3D scan data is composed of geometry data and texture image. In the present invention, scan data obtained by directly photographing a human face using a 3D scanner is used. The image data is scanned directly and scanning is performed with just one shot in front of the face.

Next, in step S20, it is determined whether the input scan data is a texture image. If the input scan data is not a texture image as a result of the determination, the process moves to step S40 to extract geometric information from the input scan data. On the other hand, if the input scan data is a texture image, the process moves to step S3 to extract feature points from the texture image.

2 is a flowchart illustrating the feature point extraction process in more detail.

As shown therein, detecting the contour from the texture image of the face model (S31, S32); Extracting feature points by mapping texture coordinates of a geometric model to the texture image (S33 and S34).

In more detail, the feature extraction process performed in this way, in step S31 and S32 to apply a Sobel edge detection method to the texture image of the face model obtained from the three-dimensional scanner to detect the contour, and in step S33 and S34 texture and geometry The texture coordinates of the model are mapped to extract feature points (geometric information) from the geometric model. 8 is a diagram illustrating a process of extracting a specific point from the texture image.

In operation S50, discrete curvatures are measured from the texture image and the geometry data, and in operation S60, the face data is simplified.

3 is a flowchart illustrating an embodiment of the face data simplification process.

As shown therein, step S61 of mapping the local coordinates to the world coordinates; Generating a boundary area (S62); Performing noise removal and filtering to remove feature points of regions other than a specific portion of the face (S63 and S64); Weighting the remaining feature points from which the feature points are removed (S65); Generating a quadric error (S66); Performing edge collapse (S67); In step S68, it is checked whether the user repeats the steps S66 and S67 as many times as the user wants.

Looking at the face data simplification process in this way in more detail as follows.

First, as in step S61, the local coordinates are mapped to the world coordinates.

Thereafter, in step S62, a boundary area is created.

That is, among the points extracted from the texture image, it is possible to distinguish between the points of important parts representing noise and facial features and the points not. Among the points obtained by using Sobel edge detection, there are the feature points and noise. Therefore, a bounding area is generated for points outside the feature part of the face.

In steps S63 and S64, noise removal and filtering are performed to remove noise other than the feature points in the bounding area. To remove the noise, use curvature filtering to remove the noise.

9A to 9B illustrate the discrete curvature filtering process. 9A illustrates extracted feature point data, FIG. 9B illustrates a state in which a bounding region is generated, and FIG. 9C illustrates a result of discrete curvature filtering.

Next, in step S65, the feature points are weighted, in step S66 a quadric error is generated, and in step S67, edge collapse is performed. In step S68, the user checks whether the steps S66 and S67 have been repeatedly performed as many times as desired by the user. If the desired number is not reached, the process moves to the step S66. If the desired number is reached, the process goes to step S70. Move.

10A to 10D illustrate a process of simplifying face data by performing feature-maintenance simplification from the discrete curvature filtered feature points. FIG. 10A shows the original model 107,696 faces extracted, FIG. 10B shows a case where the data of the original model is simplified to 3,000 through the feature-maintenance simplification process, and FIG. 10C shows the original model through the feature-maintenance simplification process. The data of the model is simplified to 1,500, and FIG. 10D illustrates a case of 600 data of the original model through the feature-maintenance simplification process.

Next, in step S70, the boundary line of the lip line is generated for the process of moving the lips during the facial animation.

4 is a flowchart illustrating an embodiment of the lip boundary point generation process.

As shown here, aligning the lip boundary points in a line (S71); Generating the same number of vertices at the same position as the lip boundary point (S72); Forming a boundary inside the mouth (S73).

Referring to the lip boundary point generation process according to the present invention made in more detail as follows.

First, align the lips for lip animation as in step S71. Next, in step S72, the same number of vertices are generated at the same position as the lip boundary line, and in step S73, a boundary line is created inside the lip from the generated vertices to enable the operation of opening the lips during animation.

Next, in step S80, target face data is generated using the simplified face data and the generated lip boundary data. In operation S90, a coincidence point is generated, and in operation S100, the modified face data is generated using the generated target face data, the coincidence point data, and the source face data.

FIG. 5 is a flowchart illustrating an embodiment of the process of generating the modified face data. In step S101, volume morphing is performed, and in step S102, deformation face data is generated by performing a Cylindrical Projection.

11 illustrates an embodiment of a process of generating a deformation model.

Next, in step 110, source animation data is generated, and in step S120, final target animation data is generated.

6 is a flowchart illustrating an embodiment of the target animation generation process.

As shown therein, generating a motion vector (S121); Converting the direction of the generated motion vector (S122); Converting the magnitude of the generated motion vector (S123); Performing Cylindrical Projection (S124); Calculating a target motion vector (S125); Steps S126 and S127 generate target animation data by arranging the generated motion vectors in size order.

A more detailed description of the target animation generation process is as follows.

First, in step S121, a motion vector is generated. The animation of the model is performed by the motion vectors present in each vertex. The motion vector means a direction and a distance for moving from the vertex of the first frame to the position of the vertex of the current frame. Each face has a different shape, so the motion vector is inevitably different. Therefore, it is necessary to replace the motion vector existing in the standard model with the motion vector of the target model. Vectors generally have size and direction. Therefore, to change the motion vector to match the target model, the size and direction must be changed.

Next, in step S122, the direction of the motion vector is changed.

To convert the direction of the motion vector, the coordinate system existing in each vertex of the standard model is obtained, and a matrix for fitting it to the X, Y, and Z axes of the world coordinate system is obtained. To find the coordinate system that exists in a vertex, we use the normal vector as the X axis, find a plane perpendicular to that axis, and project any neighboring vertices onto that plane. The original vertex and the projected vertex are then specified on the Y axis. The Z axis is obtained by calculating the cross products of the X and Y axes. In addition, after obtaining a matrix matching the world coordinate system to the coordinate system existing in the vertex of the standard model (hereinafter, referred to as the "deformation model") transformed into the target model, multiplying these two matrices and multiplying the vertex coordinate system of the standard model by the vertex of the deformation model. Get a matrix that transforms into a coordinate system.

Next, in step S123, the length of the motion vector is converted.

Create a bounding box (BB) with each vertex and its neighbors, and compare the X, Y, and Z lengths of the corresponding BBs between the standard and deformed models to find the scaling values. This transformed the Mosen vector of the standard model into the motion vector of the deformation model. That is, the animation of the standard model can be used in the deformation model. 12A and 12B show a process of converting a motion vector. FIG. 12A shows a process of converting a standard model into a transform model, and FIG. 12B shows a process of converting a transform model into a scan model.

Next, in step S124, Cylindrical Projection is performed. That is, the motion vector of the deformation model is transferred to the target model. To do this, first, the target model is transformed into a cylindrical projection. Then all the vertices of the target model are on each side of the deformation model. Therefore, by finding the triangle of the deformation model including the vertex of the target model and combining the motion vectors of the three vertices of the triangle at an appropriate ratio using Barycentric Coordinates, the motion vector of the scan model is created.

Next, in step S125, the target motion vector is calculated.

The expression application from the control mesh to the multi-level DAM is made in the order of steps, in which the control mesh is applied to the primary DSM and the primary DSM to the secondary DSM. In subdivision, the position of newly created point is determined by the mask value. At this time, the motion vector of the newly created point is calculated by applying the mask value to the motion vector of the previous model, as if the location of the point is found. The resulting animation gives facial expressions to the DSM at each stage. Here Subdivision uses a generic loop subdivision.

Next, the motion vectors generated in steps S126 and S127 are sorted in size order to adjust the number of motion vectors to be used according to the distance. That is, the temporal gain is obtained by using a large number of motion vectors at a close distance and a motion vector existing at a relatively large distance at a long distance.

According to the present invention described above, after generating a deformation model by applying volume morphing and cylindrical projection to the model that simplified the three-dimensional scan face data, using this deformation model to move the animation to the simplified scan model almost accurately In addition, since arbitrary animation can be used for any model, various expressions can be produced on various models.

In addition, after the feature-maintenance simplification of the 3D scan face data is automatically performed, the user may obtain various animations of various faces by simply drawing a lip boundary point for the face animation.

In addition, since the designer's manual work can be drastically reduced in face modeling and animation, it is possible to save a lot of time and money in games and animations.

1 is a flowchart illustrating an automatic animation method of three-dimensional scan face data according to the present invention;

2 is a flowchart illustrating an embodiment of a feature point extraction process of FIG. 1;

3 is a flowchart illustrating an embodiment of a face data simplification process of FIG. 1;

4 is a flowchart illustrating an embodiment of a process of generating a lip boundary point of FIG. 1;

FIG. 5 is a flowchart illustrating an embodiment of a process of generating modified face data of FIG. 1;

6 is a flowchart illustrating an embodiment of a process of generating a target animation of FIG. 1;

7A to 7D are exemplary views of 3D face scan data.

FIG. 8 is a diagram illustrating a process of extracting a specific point from a texture image in FIG. 2;

FIG. 9A illustrates extracted feature point data, FIG. 9B illustrates a state of generating a bounding region, FIG. 9C illustrates a result of discrete curvature filtering,

10A to 10D illustrate a process of simplifying face data by performing feature-maintenance simplification from discrete curvature filtered feature points,

11 is a view showing an embodiment of a process of generating a deformation model,

12A and 12B illustrate a process of converting a motion vector.

Claims (7)

In the method to automatically animate any face used in games, movies, etc., A first step of receiving 3D scan face data; Determining whether the received scan data is a texture image, and extracting feature points from the texture image when the input scan data is a texture image; Extracting geometry information from geometry when the input scan data is not a texture image; A fourth step of measuring discrete curvature from the extracted feature point data and the geometry information; A fifth step of simplifying facial data by removing a point outside the feature by forming a bounding area and removing noise outside the feature point in the bounding area after the fourth step; A sixth step of generating a boundary line from the lip line for the process of moving the lips after the fifth step; A seventh step of generating target face data using the simplified face data generated in the fifth step and the generated lip boundary line; An eighth step of generating a matching point after the seventh step and generating modified face data by using the generated matching point data, the generated target face data and source face data; And generating a target animation data by generating source animation data from the source face data, and performing target animation using the generated source animation data, the generated target face data, and the modified face data. An automatic animation method of the three-dimensional scan face data, characterized in that. The method of claim 1, wherein the second step, Detecting an outline by applying a Sobel edge detection technique to the texture image of the face model; And extracting feature points by mapping texture coordinates of a geometric model to the texture image. The method of claim 1, wherein the fifth step, Mapping the local coordinates to the world coordinates (S61); Generating a boundary area (S62); Performing noise removal and filtering to remove feature points of regions other than a specific portion of the face (S63 and S64); Weighting the remaining feature points from which the feature points are removed (S65); Generating a quadric error (S66); Performing edge collapse (S67); And (S68) checking whether the steps (S66) and (S67) have been repeatedly performed as many times as desired by the user. The method of claim 1, wherein the sixth step is Aligning the lip boundary points in a line (S71); Generating the same number of vertices at the same position as the lip boundary point (S72); Forming a boundary inside the mouth (S73), characterized in that the automatic animation method of the three-dimensional scan face data. The method of claim 1, wherein the eighth step is Performing volume morphing (S101); And generating a modified face data by performing a cylindrical projection (S102). The method of claim 1, wherein the ninth step is Generating a motion vector (S121); Converting the direction of the generated motion vector (S122); Converting the magnitude of the generated motion vector (S123); Performing Cylindrical Projection after the step S123; Calculating a target motion vector after the step S124; And generating (S126, S127) target animation data by arranging the generated motion vectors in order of size (S126, S127). The method of claim 6, wherein the use rate of the motion vector is adjusted according to the distance when the target animation data is generated.