KR20070061252A - A method of retargeting a facial animation based on wire curves and example expression models - Google Patents

Abstract

A method for retargeting 3-dimensional face animation based on a wire curve and an example expression model is provided to effectively re-use facial expression animation of previously produced characters to produce realistic expression animation of 3-dimensional virtual characters, thereby reducing time required to produce 3-dimensional animation. A method for retargeting 3-dimensional face animation based on a wire curve and an example expression model includes a step of forming a correspondence relation between an original expression mode and an object expression model(S100), a step of respectively defining an original example expression model and an object example expression model for the original expression model and the object expression model(S200), and a step of blending object example expression models by using a weight extracted from the original example expression model to create a new expression(S300).

Description

A method of retargeting a facial animation based on wire curves and example expression models

1 is a flowchart showing a method of reapplying a 3D face animation according to the present invention;

2 is a diagram showing a mutual relationship between an original and a target facial expression model using a wire curve;

3 is a diagram showing an emotion space diagram in which various emotions are briefly displayed in a two-dimensional space;

4 is a diagram illustrating extracting deformation parameter values (? Vsi) of a wire curve from original example facial expression models and an original facial expression model (base model);

FIG. 5 is a diagram illustrating target example facial expression models generated by applying a deformation parameter value (? Vsi) of an extracted original wire curve to a control pointer of a wire curve of a target facial expression model;

FIG. 6 is a diagram illustrating a method of generating a new facial expression by obtaining a weight value of original facial expression models using a predefined weighting function and applying the blending to a target facial expression model.

<Description of Significant Symbols in Drawings>

10. Original animation 20. Original facial expression model

21. Original example facial expression model 30. Target facial expression model

31. Target Example Facial Expression Model 40. Result Animation

50. Feature Point 60. Wire Curve

The present invention relates to a method for generating a three-dimensional face animation, using a wire curve and an example expression model to effectively apply facial expression animation data of an existing three-dimensional virtual character to a new face model. A method of retargeting facial expressions.

Many methods have been proposed to produce more realistic three-dimensional face animations, especially in front of the camera, a physical-based approach that simulates the physical characteristics of real human skin and muscles. Performance-driven approach based on data extracted from the movement of a marker attached to a real face, image-based approach based on data extracted from contour and texture information of a two-dimensional face photograph ). In order to generate an animation by the conventional method, there is a disadvantage in that the same operation must be repeated for each new model.

Recently, a retargeting and cloning approach has been proposed to overcome these drawbacks by reusing existing animation data into new models. For example, Korean Patent No. 2003-0070578 (“Automatic Animation Method of 3D Scan Face Data”) simplifies the face model with the feature points of the geometric model extracted from the contour and texture information of the range scan data of the 3D face model. We propose a method of generating the resulting animation through post-motion vectors, but the animation of the resulting animation is generated because the user needs to specify the lip contour of the scanned input face model and relies on geometric calculations such as motion vectors. The facial expressions appearing in the constraints cannot be generated monotonically or in real time.

In addition, Japanese Patent Application No. 2001-0091219 (a method of retargeting a sampled facial expression to a new face) extracts behavioral characteristics from three-dimensional facial expression data captured by a face tracking means, and extracts the extracted behavioral characteristics from different facial models. Although we propose a method of reapplying the models of different faces through motion vectors, but the expressions are generated based on geometric calculations such as motion vectors, the constraints that appear in the resulting animation cannot be monotonous or in real time. There is this.

Accordingly, an object of the present invention is to solve the above-described problems of the prior art, and an object of the present invention is to capture various facial expression changes contained in a given original animation for reapplying an existing animation to a target model. It is to provide a method in which animations are generated in real time as a result of even the unique facial expressions of the target facial expression model.

On the other hand, another object of the present invention is to provide a method for reducing the production time of the three-dimensional animation by effectively reusing the facial expression animation of another character already produced in producing a realistic facial expression animation of the three-dimensional virtual character It is.

Meanwhile, another object of the present invention is not to rely on mathematical calculation of a motion vector, which is difficult to consider the characteristics of the target model when reapplying facial expressions, and directly using a wire curve as an example facial expression model that the user can intuitively use for directing facial expressions. It provides a way to create more satisfactory result animation through blending technique while manipulating.

Reapplying the three-dimensional face animation based on the wire curve and the example facial expression model of the present invention for achieving the above object, in the method for generating a new animation by reapplying existing facial animation to a new model A first step of establishing a corresponding relationship between the original facial expression model and the target facial expression model, a second step of defining each original sample facial expression model and a target sample facial expression model for the original facial expression model and the target facial expression model, and the original sample And a third step of blending the target example facial expression models with the weight values extracted from the facial expression model to create a new facial expression.

In the method of reapplying the 3D facial animation based on the wire curve and the example facial expression model of the present invention, the first step includes assigning a feature point to the original facial expression model, and applying a feature point of the original facial expression model to a target facial expression model. Assigning a feature point to a specific portion of the object; and assigning wire curves to a model surface using the feature points as a control point, wherein the second step includes generating the original example facial expression models, and the original example facial expression model Extracting deformation parameter values between the first and second facial expression models, and applying the deformation parameter values to a wire curve of the target facial expression model corresponding to the original facial expression model to generate object example facial expression models. The step previews the weight function for each original example facial expression model. Defining, extracting a weight value through the weight function when expressions are input every frame in the original animation, and blending target example facial expression models with the weight value to generate a new facial expression.

In particular, the present invention does not rely on the mathematical calculation of the motion vector difficult to consider the characteristics of the target model when reapplying facial expressions, while blending techniques while directly manipulating the example facial expression model that can be used for directing facial expressions by wire curves. Produces a more satisfactory result animation.

In addition, the wire curve deformation parameter values required to define the example facial expression model or blending weight values required to interpolate the sample facial expression models are precomputed and stored as matrix values, and these weight values are used to create a new facial expression. To generate the resulting animation in real time.

Hereinafter, a method of reapplying 3D facial animation based on a wire curve and an example facial expression model of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a method of reapplying a three-dimensional face animation according to the present invention, wherein the relationship between the original animation 10 is established (S100), the example expression model definition (S200), and the expression blending (S300). After the process of steps, the final result animation 40 is generated.

In the first step of establishing a correspondence relationship (S100), a user designates wire curves 60 on the model surface and defines an example facial expression model (S200) in order to give a correspondence between the original and target face models. In the second step, original sample expression models 21 are generated by sampling the input original animation 10, and the original sample expression models 21 and the original expression model 20 are generated. After extracting the deformation parameter values, the deformation parameter values are applied to the wire curve 60 of the target facial expression model 30 to generate the target example facial expression models 31. Finally, in the third step of blending the facial expressions (S300), blending weights are obtained from the facial expressions coming from every frame in the original animation 10 and the original example facial expression models 21, and then the weights are obtained. Using the value, the target example facial expression models 31 are blended into a radial basis function (RBF) to produce the final result animation 40.

Therefore, each step of the present invention will be described in more detail with reference to FIGS. 2 to 6.

1. Establishment of the correspondence relationship (S100)

2 illustrates a correspondence relationship between the original facial expression model 20 and the target facial expression model 30. First, among the vertices on the surface of the original model, a feature point 50 that plays an important role in facial expression change is selected and selected, and the corresponding feature point 50 is also selected in the target expression model 30. The feature point 50 is mainly selected around the lip, eyebrows, chin, forehead, cheeks, etc., where the movement of the vertices is large, and about 20 to 40 points are designated according to the geographical complexity of the model. The number of feature points 50 here will later affect the accuracy and time of face model deformation or blending techniques.

The feature points 50 to which a similar facial expression change is applied in the vicinity of the specific points 50 may be control points of the wire curve 60. For example, in FIG. 2, the three feature points 50 designated on the left eyebrow surfaces of the two facial expression models 20 and 30 are control pointers forming a wire curve that controls the movement of the left eyebrow.

In order to apply the wire curve 60 made of the control pointer to the face model, a reference wire is first set at a portion to be transformed in the facial expression model in which the vertices are displayed. Moving this set reference curve causes deformation at the vertices around it, and the vertices near the reference curve are affected by the deformation of the curve depending on the distance. That is, vertices close to the reference curve deform to a degree similar to the reference curve, and distant vertices hardly deform. Wires that connect vertices that are similarly affected by the deformation of the reference curve are similar. A more detailed description of the wire can be found in detail by referring to a paper entitled “Wires: A Geometric Deformation Technique” published by SIGGRAPH in 1998 by Karan Singh et al.

In this way, a wire curve 60 is formed on the face surfaces of the two facial expression models 20 and 30, and a facial expression change is made to the original facial expression model 20 by designating a corresponding wire curve 60 between the two facial expression models 20 and 30. Whenever there is a parameter value of the original wire curve is extracted, and the parameter value extracted from the control pointer of the corresponding target wire curve is applied so that the facial expression of the target facial expression model 30 can be changed like the original facial expression model 20. do.

2. Example expression model definition step (S200)

Since the expression of the final result animation 40 is generated by blending the target example facial expression models 31, before the example facial expressions 21 and 31 representing the main facial expressions of the original facial expression model 20 and the target facial expression model 30 are generated. You must first create it. The original example facial expression models 21 generated for the original facial expression model 20 are weighted for blending and parameter values of the wire curve 60 required for later generation of the target facial expression models 31 of the target facial expression model 30. Used to calculate the value.

2-1. Create original example facial expression model (21) of original facial expression model (20)

In a given original animation 10, it is possible to select the important example facial expressions 21 necessary for reapplying facial expressions. However, in this case, there is a disadvantage in that the user needs to view and decide every frame of the animation. In order to efficiently generate example facial expressions, as shown in FIG. 3, important facial expressions are selected by referring to an emotion space diagram defined in psychology, and a corresponding example facial expression model 21 is directly generated or a corresponding relationship is generated. In the establishment step S100, a frame having a large change may be selected as the original example facial expression model 21 by tracking the movement of the feature point 50 designated by the user.

The selected original example facial expression models 21 are used to obtain deformation parameter values required for manipulating control points of the wire curve 60 from a user-specified base model, that is, the original facial expression model 20.

4 shows deformation parameter values (ΔVsi: deformation parameter values between the original facial expression model and the i-th original facial expression model) extracted from the original example facial expression models 21 and the original facial expression model (base model) 20. It is shown. The process of extracting the deformation parameter value ΔVsi between the original facial expression model (base model) 20 and the example facial expression model 21 is a process of inversely solving the deformation of the wire curve 60 and M (M is a natural number) wire curves. Given (60) and the example facial expression model (21), the deformation parameter values required to transform m (m is a natural number) from the base model (20) to the example facial expression model (21) by means of the least-squares minimization technique. You can extract them. A more detailed description of the process of extracting these deformation parameter values can be found in H. Pyun et al. In a paper entitled “On Extracting Wire Curves from Multiple Face Models for Facial Animation” published in 2004 by Computers and Graphics. In the present invention, the description is omitted.

2-2. Generation of the target example facial expression model 31 of the target facial expression model 30

FIG. 5 applies the deformation parameter value (ΔVsi: deformation parameter value between the original facial expression model and the i-th original example facial expression model) of the extracted original wire curve 60 to the control pointer of the wire curve 60 of the target facial expression model 30. The example facial expression models 31 generated by using the above-described example are shown. The deformation parameter value ΔVsi extracted from the original example facial expression models 21 is a wire curve in the target facial expression model (base model) 30 as shown in FIG. 5. The target example facial expression model 31 corresponding to each original sample facial expression model 21 is generated by being applied to the control pointer of 60.

The object example facial expression models 31 generated as described above are used for blending into the result animation 40 in the facial blending step S300 which is the next step. In particular, in the facial expression blending step (S300), when the user wants to emphasize a specific facial expression in the result animation 40, the weight value is multiplied by the control pointer of the wire curve 60 on the surface of the corresponding target example facial expression model 31. You can also adjust the look of your choice.

3. Expression blending step (S300)

In the facial expression blending step S300 of the present invention, a new facial expression is generated by blending the target example facial expression models 31 generated in the previous step S200 by the weight values obtained from the original facial expression models 21.

3-1. Extraction of Weighted Values

In order to create a new facial expression by blending technique, it is necessary to find out the effect of each original example facial expression model 21 on the new blended facial expression. Therefore, in the present invention, the influence is defined as a weight value multiplied by the target example facial expression model 31, and the weight value is obtained through a weight function predefined for each original example facial expression model 21.

FIG. 6 illustrates a result of a new facial expression (Voi: i-th input original example facial expression model by blending the target facial expression models 31 by obtaining weight values from the original sample target models 21 using a predefined weighting function. Animation), and as shown in FIG. 6, when a new facial expression ΔVi comes in every frame from the original animation 10, a weight function predefined in each original example facial expression model 21 is used. Weight values (w1, w2) needed to create a new expression (Voi = w1 * Vt1 + w2 * Vt2 +… + wi * Vti, where Vti is the weight value of the i th target example facial expression model and wi is the weight value of the i th example facial expression model. , wi ... can be saved. Then, each target example facial expression model 31 receives a weight value (w1, w2,..., Wi) from the corresponding original example facial expression model 21 and generates a new facial expression (Voi) through facial blending. do.

Here, the weight values used for facial expression blending should be recalculated every time a new facial expression comes in. However, the present invention uses a predefined weighting function for each facial expression model. Do. The weight function used in the present invention consists of a cardinal basis function defined by P. P. Sloan et al. In a paper entitled “Shape by Example” published by Syposium on Interactive 3D Graphics in 2001. This cardinal basis function is a linear-basis function that defines the approximate but overall linear graph shape of the weights of the example expressions in the m-dimensional space and the rest of the linear graph. ), Which consists of a curve-based function that defines a curve graph shape to measure more accurate values. That is, a new facial expression is generated by using the overall weight value obtained from the linear basis function, and the detailed expression of the facial expression of the example facial expression model using the detailed weight value obtained by the radial basis function. The movement of the feature point can be generated to blend minute expression changes. As described above, it is possible to more accurately generate the expressions of the original animation 10 as the expressions of the result animation 40 through the blending technique using the above-described two kinds of weight functions.

3-2. Blending

As a result of the facial expression inputted from the original animation 10, the new facial expression of the animation 40 is multiplied by the weight values of each target example facial expression model 31, and added as one facial expression to generate a new facial expression Voi (Voi = w1). * Vt1 + w2 * Vt2 +… + wi * Vti). At this time, each target example facial expression model 31, which is multiplied by the weight value, has a feature point difference value with the target facial expression model (base model; 30) in order to prevent distortion in the facial expression of the resulting animation 40 due to the weight value error. Consider only.

Therefore, when the facial expressions coming from the original animation 10 are severely different from the facial expression examples 31 with the target example facial expression models, it is difficult to generate the facial expressions of the resulting animation 40 only by simple blending. By blending the target example facial expression model 31 through, it is possible to generate the facial expression of the result animation 40 as close as possible to the facial expression coming from the original animation 10. In addition, when the user sees the input facial expression and includes the corresponding target facial expression, an accurate result animation 40 closer to the original animation 10 can be generated.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

As described above, the method of reapplying the 3D facial animation based on the wire curve and the example facial expression model according to the present invention includes the characteristics of the target facial expression model, which is difficult to consider in the conventional motion vector based reapplying method. By re-applying the original animation to the new model in real time, reflecting it in the resulting animation as intended, it is possible to effectively reuse facial expression animations of other characters that have already been produced in producing realistic facial expression animations of 3D virtual characters. It can contribute to shortening the production time of the animation, and can also be used for the motion animation of the carrytuck in addition to the facial animation.

Claims (10)

In a method of generating a new animation by reapplying an existing face animation to a new model, A first step of establishing a corresponding relationship between the original facial expression model and the target facial expression model; A second step of defining each original example facial expression model and a target example facial expression model for the original facial expression model and the target facial expression model; And A third step of blending the target example facial expression models with a weight value extracted from the original sample facial expression model to create a new facial expression Reapply method of 3D facial animation based on a wire curve and an example facial expression model comprising a. The method of claim 1, wherein the first step, Designating a feature point on a surface of the original facial expression model; Assigning the feature point to a specific portion corresponding to the original expression model by applying the feature point to a target expression model; And Designating wire curves to surfaces of the two facial expression models using the feature points of the two facial expression models as control points Reapply method of 3D facial animation based on a wire curve and an example facial expression model comprising a. The method of claim 1, wherein the second step, Generating original example facial expression models for the original facial expression model; Extracting deformation parameter values between the original example facial expression models and the original facial expression model; And Generating the target example facial expression models by applying the deformation parameter values to a wire curve of the target facial expression model corresponding to the original facial expression model Reapply method of 3D facial animation based on a wire curve and an example facial expression model comprising a. The method of claim 1, wherein the third step, Defining a weighting function in advance for each original example facial expression model; Extracting a weight value through the weight function when facial expressions are input every frame in the original animation; And Generating a new facial expression by blending the target example facial expression models with the weight value Reapply method of 3D facial animation based on a wire curve and an example facial expression model comprising a. The method of claim 3, wherein the generation of the original example facial expression model, A method of reapplying three-dimensional face animation based on a wire curve and an example facial model, characterized by generating facial expressions by selecting facial expressions with reference to an emotion space diagram. The method of claim 3, wherein the generation of the original example facial expression model, Reapply the 3D face animation based on the wire curve and the example facial expression model, by sampling the input original animation and tracking the movement of the feature points specified in the first step to designate a large change frame as the original facial expression model. Way. The method of claim 3, wherein the extraction of the deformation parameter value, Reapplying a three-dimensional face animation based on the wire curve and the example facial expression model, characterized in that the deformation parameter values are extracted through the motion vector of the control pointer of the wire curve between the original facial expression model and the original facial expression model. The weight function defined in claim 4, wherein 3-D facial animation based on a wire curve and an example facial expression model comprising a linear basis function to obtain an overall weight value and a radial basis function to obtain a detailed weight value Reapply method. The method of claim 4, wherein the blending generates a new facial expression. The new facial expression is reapplied to a three-dimensional facial animation based on a wire curve and an example facial expression model, characterized in that the product is generated by multiplying each target example facial expression model by the weighted value extracted from the original facial expression model and adding them as one facial expression. . The method of claim 9, The distortion of the new facial expression caused by the error of the weight value is prevented by considering only the movement difference value of the feature point between the target example facial expression model and the target facial expression model multiplied by the weight value. Reapplying 3D face animation based on facial expression model.

Images