KR101116838B1 - Generating Method for exaggerated 3D facial expressions with personal styles - Google Patents

Abstract

The present invention relates to a method for generating an exaggerated expression reflecting an individual tendency, the method for generating an exaggerated expression reflecting an individual tendency in a method for generating an exaggerated expression using a three-dimensional face model that can generate an expression through the movement of virtual muscle, (a) acquiring motion data according to facial expression change from a live image of a model to which a plurality of markers are attached, and calculating motion vector and exaggerated motion vector according to facial expression change from the obtained motion data; (b) Calculate the distance value between markers according to facial expression change from the live-action image of the model for each personality type with a plurality of markers divided according to the personality type test result, and use the calculated distance value generating a rate mapping table); (c) calculating an exaggeration index for each personality type using the generated rate mapping table; (d) receiving a two-dimensional image of an individual and generating a three-dimensional face model for each individual using the inputted two-dimensional image of the individual and the three-dimensional face model; (e) calculating the exaggeration index for each personality type by receiving the personality type and the exaggeration rate information of the individual; And (f) applying the calculated exaggeration index for each personality type to the exaggerated motion vector, and applying the result to the virtual muscle to generate an exaggerated expression reflecting individual inclination in the individual three-dimensional face model. It is a method of generating exaggerated expressions reflecting personal tendencies.

According to the present invention, since the exaggerated expression is generated by analyzing / reflecting the results of the disposition of various subjects and the characteristics of the facial expressions of each inclination, the exaggerated expression may be more realistic and reflect the individual's inclination. Since it can be arbitrarily adjusted by the user, there is an advantage that an exaggerated expression of various scales can be generated.

Personal orientation, personalized face model, exaggeration, exaggeration, motion vector, personality type, virtual muscle

Description

Generating Method for exaggerated 3D facial expressions with personal styles}

The present invention relates to a method for generating an exaggerated expression reflecting an individual tendency, and more particularly, to a method for generating an exaggerated expression that can generate a personalized exaggerated expression reflecting an individual's inclination.

In recent years, a technique for generating a three-dimensional face image from an individual two-dimensional face image has been used in many fields. However, the early three-dimensional personalized face generation technique itself had a problem in that it merely provided a function of simply converting a two-dimensional image into a three-dimensional image.

In order to solve this problem, research into a technology for generating a three-dimensional image capable of generating a variety of expressions, rather than a static three-dimensional image is increasing.

However, even in the related art, there is a problem in that the expression of the three-dimensional image is simply changed by a user's mouse manipulation or the three-dimensional image of the expression generated and stored in advance according to the user's selection is displayed.

In addition, in the case of a three-dimensional image in which the emotion according to the prior art is expressed, there is a problem in that it is impossible to realize a realistic facial expression by merely modifying a face model of the three-dimensional image.

In addition, although the expression of the same emotion can be expressed differently according to the inclination of the individual, there was a problem that it is impossible to generate an expression reflecting the inclination of the individual.

The present invention is derived based on the three-dimensional face model to map the expression, the texture to be used for the basic face model, the motion capture data of each expression, anatomical analysis and motion data division to solve the problems of the prior art as described above. The purpose of the present invention is to provide a method for generating personalized exaggerated facial expressions using propensity classification results of various subjects and characteristics of facial expressions of each propensity based on the muscle data for each facial expression.

In order to achieve the above object, according to a preferred embodiment of the present invention, in the exaggerated expression generating method using a three-dimensional face model that can generate an expression through the movement of the virtual muscle, (a) a plurality of Acquiring motion data according to facial expression change from the actual image of the model to which the marker is attached, and calculating motion vectors and exaggerated motion vectors according to facial expression change from the obtained motion data; (b) Calculate the distance value between markers according to facial expression change from the live-action image of the model for each personality type with a plurality of markers divided according to the personality type test result, and use the calculated distance value generating a rate mapping table); (c) A method of generating an exaggerated expression reflecting personality is provided, comprising calculating an exaggeration index for each personality type by using the generated lattice mapping table.

In addition, the method for generating an exaggerated expression reflecting the above-described individual tendencies, (d) receives a two-dimensional image of the individual, and generates a three-dimensional face model for each individual by using the input two-dimensional image and the three-dimensional face model Making; (e) calculating the exaggeration index for each personality type by receiving the personality type and the exaggeration rate information of the individual; And (f) applying the calculated exaggeration index for each personality type to the exaggerated motion vector, and applying the result to the virtual muscle to generate an exaggerated expression reflecting individual tendencies in the individual three-dimensional face model. have.

Here, in step (a), (a1) the six basic facial expressions of the actor with a plurality of markers are continuously photographed, and the captured images are analyzed to three-dimensional of each marker according to the expression change for each frame. Calculating motion data for each facial expression using coordinates; (a2) obtaining the optimized approximation matrix BE by decomposing the motion data matrix M for each facial expression into a non-negative matrix, and then decomposing the characteristic matrix B and the weight matrix E using non-negative matrix factorization (NMF); And (a3) decomposing each column of the weight matrix E into an average and a deviation, adding an ingot travel matrix R to the approximate matrix BE, and calculating a motion vector and an exaggerated motion vector therefrom.

In addition, the motion vector calculated in step (a3) described above is

이고,

ego,

The exaggerated motion vector calculated in step (a3) is

으로 도출될 수 있다.

Can be derived.

On the other hand, the six basic facial expressions of step (a1) described above may be surprise, fear, anger, disgust, happiness, sadness.

In addition, the personality type test of step (b) described above is an MBTI test, and the personality type test results may be classified into IF tendency, IT tendency, EF tendency, and ET tendency.

More preferably, the step (b) described above, (b1) a plurality of markers are attached to the face of the model having the corresponding personality type according to the personality type test result based on the feature points, and the expression, surprise, fear, Photographing seven facial expressions of anger, disgust, happiness, and sadness; And (b2) comparing the positions of the markers of the six expression images and the expressionless images photographed by the personality type, respectively, to calculate the average moving distances of all markers between the expressionless facial images and the emotion expression facial images, and proportionate them. Generating a mapping table.

Here, in the above-described step (b2), the calculation formula of the average moving distance of the marker is

(s: 마커 인덱스 , m_s: 표정의 마커 벡터, m_Ns: 무표정의 마커 벡터)일 수 있다.

(s: marker index, m _s : marker vector of expression, m _Ns : marker vector of expressionless).

In addition, the proportioning in the step (b2) described above

,

,

scale parameter)식을 이용하여 수행될 수 있다. (

,

,

scale parameter) expression.

On the other hand, the exaggerated index calculation formula for each personality type calculated in step (c) described above is

(s _i : style index, n: exponent parameter, k: scale parameter, w (0-1): exaggeration rate, C: convergent value).

On the other hand, step (d) described above, (d1) receiving a two-dimensional image of the individual; (d2) automatically detecting and outputting a feature point from an input two-dimensional image using an AAM (Active Appearance Model) algorithm; (d3) changing the position of the feature point detected by the rubber band method according to a user's manipulation; (d4) transforming the three-dimensional face model into a personal three-dimensional face model by using the finally determined position values and 3D warping techniques; And (d5) performing texture mapping on the individual three-dimensional face model.

Here, in the exaggerated expression generation method reflecting the above-described individual tendencies, in applying the exaggerated motion vector to the virtual muscle in the step (f), each part of the three-dimensional face model or the individual three-dimensional face model is positioned at the position of the marker. The feature box is divided into a plurality of hexahedral shape boxes, and the ratios of x, y, and z extraction distances for each segmented feature box of the marker and the three-dimensional face model or the individual three-dimensional face model are calculated. It can be configured to multiply and apply the amount of change of the marker.

In this case, the above-described three-dimensional face model or individual three-dimensional face model may be divided into 12 feature boxes in consideration of the muscle movement according to the expression.

As described above, the exaggerated expression generating method reflecting the personal tendency according to the present invention has an advantage of generating a realistic exaggerated expression.

In addition, according to the present invention, since the exaggerated expression is generated by analyzing / reflecting the characteristics of the inclination of each subject and the characteristics of the facial expressions of each inclination, an exaggerated expression that reflects the individual's personal tendency can be generated. There is this.

In addition, according to the present invention, since the exaggeration rate can be arbitrarily adjusted by the user, there is an advantage that an exaggeration expression of various scales can be generated.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the method for generating an exaggerated expression reflecting the personal tendency according to the present invention.

The present invention has developed a facial expression generation system that exaggerates the motion data of a facial expression obtained through motion capture in consideration of a user's face shape and personal tendency.

To this end, we first constructed a database for the system, and each data was based on a three-dimensional face model to map facial expressions, a texture to be used for the basic face model, motion capture data for each facial expression, and anatomical and motion data analysis. Muscle data for each facial expression.

The system basically outputs a three-dimensional face model with texture mapping. After that, when the user photographs his or her face using a web camera or a digital camera, each feature information of the face is automatically extracted and analyzed from the user's photo, and the model is modified based on this. The input photo is automatically edited, and the photo is mapped to a texture to output the user's face model.

Also, in order to generate an exaggerated expression, the motion capture data of each expression is used to find the feature of the marker movement and exaggerate it. At this time, a mapping method for the position of the muscle to generate the facial expression in the face model and an exaggerated facial expression conversion function considering the facial features are presented.

Lastly, in order to reflect the personal style on the facial expressions, the results of the classification of the various subjects and the characteristics of the facial expressions of each inclination were analyzed.

1. Create 3D personalized face

end. Facial feature analysis

Techniques for facial feature extraction have been invented by a variety of methods and approaches. Recently, techniques using wavelet analysis and image filters such as templates have been introduced. In the present invention, the feature point of the face is found by using an AAM (Active Appearance Model) algorithm. AAM is an algorithm that fits a statistical model of the shape and appearance of an object to an input image, and is divided into a modeling step and a matching step. This is very convenient and quick compared to the existing method of specifying the feature point position directly by the user.

1) Execution Process and Result of Feature Point Input Module

Users can model with a personalized 3D face model by shooting with a PC Cam camera or using a ready-made front view. The input picture is based on the frontal expressionless picture, and various facial expressions are also available depending on the AAM statistical model database. The user loads the input photo to check that the face is correct and executes the AAM algorithm to automatically generate the feature points primarily. In order to execute AAM algorithm, statistical model database should be built. In this system, the following process was performed.

① The Nikon D80 camera was used to take pictures of the subject (subject), and in order to maintain the same conditions and environment, the resolution of the photograph was 800 × 600 pixels, and the distance from the focal plane of the camera to the eyes of the subject (subject) was 2.0. m, take a picture with the tripod fixed.

② A slight inclination is generated for each photograph taken to correct the inclination, and a coordinate value is input by inputting LandMark of 69 predefined feature points.

③ In order to model the shape change of LandMark of each model, it is necessary to normalize it. Through the analysis of procrustes analysis, the feature points of the face are arranged in a similar position.

④ Procrustes analysis Analyze PCAs for 10 images using coordinates of normalized feature points to generate an average shape model.

Warp model images and generate an appearance model using the Delaunay trigonometry algorithm so that the feature points of each shape model are matched to the average shape model (FIG. 3).

⑥ Construct a combination model of shape and appearance and save it as a file.

Using the statistical model (combined model) processed above, calculate the difference between the input images and modify the model appropriately so that the difference is minimized, and repeat the transformation process so that the combined model is most similar to the area to be detected by the combined model in the input image. To obtain the desired area.

Automatic feature detection using AAM (Fig. 4) After that, the rubber band method is used for accurate feature point control, which is formed by increasing or decreasing the point and line like a rubber band when the user drags the feature point with the mouse. Say the way. In this system, both Active Appearance Model and Rubber band method are used to input facial feature points. This minimizes the user's input time because the user automatically finds the predefined feature location by using the AAM algorithm, and the rubber band method improves the accuracy by randomly moving the feature point found first. Height plays a role.

I. Create personalized 3D faces

In order to convert the 3D normalized model into a personalized face model using the position values of the 69 feature points defined, the normalized model must be transformed into the shape of the input photo. In general, the proportional deformation method has been used, but in this system, the normalized model is modified using a 3D warping technique. Warping techniques can be used to transform the overall shape, eyes, nose and mouth of an existing face.

By the warping technique, the appearance of the normalization model can be transformed to be similar to that of the user. However, there is a limit in expressing the skin, lips, and eyebrows realistically like the live-action image, as it is a deformation of the primary model appearance. In a custom model, texture mapping plays an important role in making the input image almost like that. 5 shows the result of a personalized 3D face to which texture mapping is applied.

2. Create exaggerated facial expressions

The state of overexpression of universal emotional expressions is said to be 'exaggerated'. This chapter introduces how to exaggerate and simulate six basic human expressions.

The process of creating exaggerated facial expressions can be divided into three main categories. First, using a virtual muscle based on anatomical analysis, a three-dimensional face model is produced to show the change in the facial surface during muscle contraction and relaxation. Second, in order to give an expression to a model, motion data corresponding to six basic expressions of humans are collected using optical motion capture equipment, and analyzed using data analysis algorithms to characterize facial surface movements when realizing facial expressions. Find it. Third, the motion data obtained above is exaggerated using an exaggeration algorithm and then applied to the muscles of the three-dimensional face model to simulate the exaggeration of the six basic expressions.

end. Create and exaggerate facial expressions

Based on the analysis of facial expressions extracted from motion capture data, parameters for specific muscles in each facial expression are defined. This parameter can be applied to the muscle algorithm of the system to reflect each muscle movement to generate an expression.

In addition, it is possible to form an exaggerated expression by extracting the feature of motion data for each expression and exaggerating the value of the parameter.

In order to generate a three-dimensional facial expression model, data about the actual human face movement is required. In the present invention, the motion data of the face surface according to each expression was obtained through motion capture. At this time, the position definition of the marker is very important because the movement of the marker is the most basic source for generating an expression.

Therefore, after determining the position of the marker in consideration of the position of the muscle, the inventors captured the facial changes for 700 to 1000 frames at 120 frames / sec for six basic expressions. A static number of markers were attached to the areas that greatly affected facial expression changes, and five markers were attached to the upper part of the head to obtain the overall movement of the face.

The motions of the markers according to the actor's expression in the motion capture are stored as 3D coordinate data through the optical sensor, recording the absolute position of the entire marker every frame. Changes and features could be analyzed.

1) Extraction and exaggeration of features in facial expression

The motion data of the facial expression is expressed as a three-dimensional movement change of the marker set over time, and the motion data can be exaggerated by finding relatively large movement values and scaling the input exaggeration constant. As a result, exaggerating the movement of the markers that make up the facial expression can be said to be the core of generating the exaggerated facial expression.

A) Decomposition of motion data

Motion data was decomposed using NMF to represent the movement pattern and its weight. Non-negative matrix factorization (NMF) is a multivariate data analysis algorithm used to extract meaningful representations from datasets such as documents, images, spectrograms, and so on. Expressed in the form of a product of weight matrices indicating the degree of activation of the bases.

In the present invention, after decomposing the motion data of each of the six basic facial expressions, an exaggerated facial expression is generated by multiplying a differential exaggeration constant when an activation value corresponding to a characteristic of each facial expression is equal to or greater than a reference value.

Since the measured motion data also contains negative numbers, the motion data is made non-negative by first adding the absolute value of the minimum value of the data for matrix decomposition.

When the motion data is a non-negative matrix M (motion data) having a size of n × m having a three-dimensional moving value of each frame of the markers, M satisfies r (n + m) <nm as in Equation 1. It is decomposed into a feature matrix B (basis or feature vector) having a size of n × r and a weight matrix E (encoding or weight value) having a size of r × m based on an arbitrary r (rank) value.

The decomposition process of M first initializes the characteristic matrix B and the weight matrix E with arbitrary non-negative values, and then repeats the update through Equation 2, while applying the approximation matrix BE of M using an optimization function such as Equation 3. Obtain

,

,

B) Motion data exaggeration

를 수학식 4와 같이 기저 벡터

와 잉여 벡터

의 선형 조합 형태로 재정의 할 수 있다.When the update is finished with an optimized approximation matrix, we decompose each column of the weighting matrix E back into the mean m _i and the deviation d _i to find the movement pattern to exaggerate. In addition, in order to remove the noise (error) caused by the matrix decomposition process, the in-travel matrix R (residual), which is the difference between the motion data M and the approximate matrix BE, is added.

The basis vector as

Surplus vector

Can be redefined as a linear combination of.

는 수학식 5와 같이 편차 d_i를 과장상수

로 곱해서 구하게 되는데, 이때 과장상수를 모든 편차에 곱하는 것이 아니라 먼저 과장시킬만한 이동 패턴을 찾는 것이 중요하다. 따라서 편차가 다음의 수학식 5와 같이 표준편차의 두 배 이상일 때만 과장상수를 곱해주도록 제약을 둔다. 또한 과장된 모션 데이터에 맞추어 잉여벡터

을 과장함으로써 과장모션벡터에서의 노이즈를 제거하여 준다.Finally exaggerated motion vector

Is an exaggerated constant for the deviation d _i as in Equation 5

It is important to find a shift pattern that can be exaggerated first, rather than multiplying all the deviations by an exaggeration constant. Therefore, it is limited to multiply the exaggerated constant only when the deviation is more than twice the standard deviation as shown in Equation 5 below. Also, surplus vectors to match exaggerated motion data

Exaggerate to remove the noise from the exaggerated motion vector.

only,

6 shows an exaggerated result of motion capture data. It can be seen that the degree of exaggeration is different depending on the exaggeration index of the motion data in the angry and surprise expressions.

I. Apply personality

The degree of exaggeration in facial expressions can vary depending on the individual's disposition. Therefore, individual styles are applied to the exaggeration algorithm defined above. The style referred to herein refers to a propensity (personality, sex, race) that has an individual, and in the present invention, a more individual expression can be generated by varying the range of application of parameters according to the style. To do this, the style must be defined as a representable abstract data type, and the data on personal disposition must be constructed first.

1) Detection of features in facial expressions according to personal preference

A personality type test was conducted using the MBTI-GS form for 30 college students living in the Seoul metropolitan area, in their early 20s to mid-20s.They are typical facial expressions of surprise, fear, anger, and disgust. Seven pictures were taken with the Nikon D80: happy, sad and expressionless. The resolution of the picture was 800 × 600 Pixel, and the distance from the camera's focal plane to the subject's eyes was fixed at 2m. At this time, the basic expressions other than the expressionless expressions were photographed by providing the basic model because the same emotions may be different according to individual differences of people.

As shown in the expression sample photograph of FIG. 7, 30 markers were positioned at each facial expression facial muscle position based on the facial model feature points defined in MPEG-4, and the distance value was calculated between the expressionless expression and the six facial expressions.

: 마커 인덱스

,

: 표정의 마커 벡터,

: 무표정의 마커 벡터)(

Marker Index

,

: Marker vector of expression,

: Marker vector without expression)

Using the above Equation 6, the pixel average moving distance of six expressions for each MBTI type (FIG. 8) can be expressed as shown in [Table 1]. Of the six expressions, the largest change in Surprise can be seen. It can be seen that there are the most facial changes when generating the Surprise expression. In addition, by comparing the average distance value in the suppression expression, the propensity of the largest change, the EF type and the smallest change, the IF type can be distinguished.

Expression / Style EF ET IF IT Happiness 102.99 85.29 95.88 80.07 Surprise 144.77 121.18 108.50 104.46 Sadness 90.15 71.25 88.89 69.15 Fear 104.46 95.66 118.63 92.75 Anger 127.13 83.40 85.10 103.46 Disgust 105.38 81.41 85.92 63.78

of Styles Average distance

of Styles

은 최종 SER(Styled Exaggeration rate)를 도출하는데 사용된다.As shown in [Table 1], it was divided into disposition by facial expression using distance value. In the present invention, the mapping table of the style rates is formed using the proportional change of Equation 7 for each propensity, and the style rates of the mapping table as shown in [Table 2].

Is used to derive the final Styled Exaggeration Rate (SER).

,

,

scale parameter)(

,

,

scale parameter)

Expression / Style EF ET IF IT Happiness 1.00 0.87 0.95 0.83 Surprise 1.00 0.90 0.85 0.83 Sadness 1.00 0.83 0.99 0.83 Fear 0.91 0.85 1.00 0.83 Anger 1.00 0.83 0.84 0.91 Disgust 1.00 0.90 0.92 0.83

Style rates

2) Derivation of SER (Styled-Exaggeration Rate)

의 맵핑 테이블을 이용하여 수학식 8과 같이 과장률에 따른 SER을 도출한다.SER is an exaggerated exponent assigned to the exaggerated expression generation algorithm.

Using the mapping table of to derive the SER according to the exaggeration rate as shown in Equation (8).

: style index,

: exponent parameter,

: scale parameter,

: exaggeration rate,

: convergent value)(

style index,

: exponent parameter,

: scale parameter,

exaggeration rate,

: convergent value)

)을 입력하면 이 값들이 과장 표정 생성 알고리즘에 대입되는데, 여기서

일 때

는

에 적용되고,

이면 수렴값(최대값)

로 정의된다.

일 때는 과장이 없는 기본 표정생성이고,

일 때에는 최대 과장 표정생성을 나타낸다. 이때 SER 값이

를 만족하도록 스케일 상수

값을 적용하였다. 도 10은 과장률에 따른 놀람표정에서의 MBTI 성향(EF, ET, IF, IT)들에 대한 SER 변화를 나타낸 것이다. 사용자는 0~1까지의 과장률를 선택하면 SER을 이용하여 최종적인 개인성향 과장지수(Styled Exaggeration Rate)가 결정된다.The user can select the MBTI type and exaggeration rate (

), These values are assigned to the exaggerated facial expression generation algorithm, where

when

Is

Applied to,

If convergence (maximum value)

Is defined as

When is the default expression without exaggeration,

, The maximum exaggerated facial expression. SER value is

Scale constant to satisfy

Value was applied. Figure 10 shows the SER change for MBTI propensity (EF, ET, IF, IT) in surprise expression according to the exaggeration rate. When the user selects an exaggeration rate from 0 to 1, the final styled exaggeration rate is determined by using the SER.

All. Motion cloning

Coordinates of the motion data acquired through motion capture were recorded based on the head and face of the performer. Therefore, it is necessary to fit motion data to the model in order to use it in the basic model of the system and personalized model. In this work, the positions of all markers should be properly positioned on the face of the model, and the change in intensity of the motion data of each facial expression must be maintained.

The initial start of the motion data is the initial state of the model since the expression has not been generated yet (neutral expression) state. Thus, when the model is in the absence of expression, the motion data at the starting point can be fitted. However, since the parts of the face that make up the facial expressions have independent movements with different intensities, the motions of the parts must be divided and applied to fit the marker movements in the frames to the model. Accordingly, in the present invention, by dividing each part of the face model into cubes according to the positions of the markers, the ratio of the x, y, and z axis distances to the respective parts of the marker and the face model is calculated, and the ratio is calculated. The amount of change in the marker was multiplied and applied to the face model. Each segment was defined as a feature box divided into 12 as shown in FIG. 11 in consideration of the muscle movement according to the expression, which is based on the position of the marker in the motion capture to fit the motion data to the marker. It was made.

)와 얼굴 모델의 특징상자(

)는 각각 가로(

,

), 세로(

,

), 높이(

,

)의 세 가지 요소를 가진다.As shown in Fig. 12, the feature box of motion data (

) And the feature box of the face model

) Is horizontal (

,

), Vertical(

,

), Height(

,

) Has three elements:

라고 할 때, 무표정에서의 마커 위치

은 기본 얼굴 모델 또는 개인 맞춤형 얼굴의 특징상자 인덱스에 맞추어 이동하게 되며, 개인화 얼굴 모델에 맞춘 모션 데이터의 마커 위치는

이며 최종적으로 모션클로닝 값은 수학식 9 내지 수학식 12에 의해 구해진다.The marker position in the feature box

, The marker position in the expressionless

Moves to the feature box index of the default face model or personalized face, and the marker position in the motion data for the personalized face model

Finally, the motion cloning value is obtained by equations (9) through (12).

la. Muscle mapping

Anatomically, a human facial expression is a physical action based on skin and muscle movements. Therefore, in order to realize a three-dimensional facial expression, it is necessary to produce a face model similar to the actual human face structure through structural analysis of skin tissue and muscles. It also allows various facial muscle movements to be synthesized.

Ekman showed that the FACS (Facial Action Coding System) can be used to generate all facial expressions by combining muscles, and Parke proposed a method to generate facial expressions by classifying muscle types. Accordingly, the present invention aims to generate muscle-based facial expressions, and will define virtual muscle mapping through anatomical analysis.

Based on the results of anatomical analysis of facial muscles, the present system was used to create a realistic facial expression by generating a three-dimensional face model and mapping virtual muscles that act like real muscles. The basic face model was modeled using 3D Studio Max with a total of 1876 vertices and 3668 faces, and as shown in FIG. 13, a realistic texture image was mapped to a Korean adult male. Figure 14 illustrates the structure and characteristics of the virtual muscle adopted in the present invention.

3. System interface

15 is a configuration diagram of a system interface according to the present invention. As shown in FIG. 15, the system according to the present invention is composed of a menu bar portion and a view portion, and uses a mapping method using a color texture of a vertex and a mapping method using a model texture map by setting a mode value. It is possible to map variously according to the developer's intention. Basically, the menu bar is divided into 6 parts and performs various commands for each menu division. In the view part, the face model is visible. It rotates 360 ° using the mouse and wheel movement and enlarges the model. , Can be reduced.

end. Exaggerated Expression Generation Result

The hyperbolic expression for each of the following six basic expressions was simulated by substituting the muscle contraction value (SER) reflecting the individual tendency and the muscle contraction value obtained from the motion data in the muscle algorithm and the exaggeration algorithm defined in the present invention.

FIG. 16 illustrates a change in motion data with respect to an exaggeration rate used in generating an expression, and it is possible to grasp the position of the marker showing a clear movement and the degree of change.

Next, FIGS. 17 to 22 show simulation results of exaggerated facial expressions for six basic facial expressions by facial expression, individual inclination, and exaggerated percentage.

I. Exaggerated Expression Generation Results in Personalized Face Models

23 to 26 show simulation results of an exaggerated facial expression applied to a personalized face model.

Preferred embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art will be able to make various modifications, changes and additions within the spirit and scope of the present invention. Additions should be considered to be within the scope of the following claims.

1 is a structural diagram of an exaggerated expression generating system reflecting personal tendencies according to the present invention.

2 is an exemplary view illustrating a sample image in which 69 feature points are input according to the present invention.

Figure 3 is an exemplary view showing a face warped in the combined shape and the average shape according to the present invention.

4 is an exemplary diagram illustrating an image AAM result screen not included in a training set.

5 is an exemplary view showing a personalized 3D face generation result according to the present invention.

6 is a diagram illustrating a motion data exaggeration result according to the present invention.

7 is an exemplary view showing a facial expression sample picture in which a marker according to the present invention is displayed.

8 shows four types of classification of MBTIs.

9 is a graph showing the average distance according to individual tendencies.

FIG. 10 shows SER change for MBTI propensities in surprise expression according to exaggeration rate. FIG.

11 is a view showing a feature box defined in a motion data and a face model.

12 is a diagram showing a feature box index of a motion data and a face model.

FIG. 13 illustrates 3D face modeling and realistic texture mapping results. FIG.

14 is a view showing the position and name of the virtual muscle mapped to the face model.

15 is an interface diagram of a system according to the present invention.

16 is a view showing a change in motion data according to the change of the exaggeration rate in accordance with the present invention.

17 to 22 are diagrams showing simulation results of exaggerated expressions for six basic expressions by facial expression, individual inclination, and exaggeration rate.

23 to 26 illustrate results of generating an exaggerated expression in a personalized face model.

Claims (13)

In the exaggerated expression generation method using a three-dimensional face model that can generate an expression through the movement of the virtual muscle, (a) acquiring motion data according to facial expression change from a live image of a model to which a plurality of markers are attached, and calculating motion vector and exaggerated motion vector according to facial expression change from the obtained motion data; (b) Calculate the distance value between markers according to facial expression change from the live-action image of the model for each personality type with a plurality of markers divided according to the personality type test result, and use the calculated distance value generating a rate mapping table); And (c) calculating an exaggeration index for each personality type using the generated lattice mapping table; The method of claim 1, Exaggerated expression generation method reflecting the personality tendency, (d) receiving a two-dimensional image of an individual and generating a three-dimensional face model for each individual using the inputted two-dimensional image of the individual and the three-dimensional face model; (e) calculating the exaggeration index for each personality type by receiving the personality type and the exaggeration rate information of the individual; And (f) applying the calculated exaggeration index for each personality type to the exaggerated motion vector, and applying the result to the virtual muscle to generate an exaggerated expression reflecting individual tendencies in the individual three-dimensional face model. How to create an exaggerated expression reflecting personal tendencies. 3. The method of claim 2, In step (a), (a1) Photographing six basic facial expressions of an actor with a plurality of markers in succession, analyzing the photographed images, and calculating motion data for each facial expression using three-dimensional coordinates of each marker according to facial expression change for each frame. step; (a2) obtaining the optimized approximation matrix BE by decomposing the motion data matrix M for each facial expression into a non-negative matrix, and then decomposing the characteristic matrix B and the weight matrix E using non-negative matrix factorization (NMF); And (a3) decomposing each column of the weighting matrix E into an average and a deviation, adding the in traveling matrix R to the approximate matrix BE, and calculating a motion vector and an exaggerated motion vector therefrom. How to create the exaggerated expression reflected. The method of claim 3, The motion vector calculated in the step (a3) is

ego, The exaggerated motion vector calculated in step (a3) is

Exaggerated expression generation method reflecting the personality, characterized in that the. The method of claim 3, The six basic facial expressions of step (a1) are surprise, fear, anger, disgust, happiness, sadness. The method of claim 1, The personality type test of step (b) is an MBTI test, and the personality type test result is classified into IF tendency, IT tendency, EF tendency, and ET tendency. The method of claim 1, In step (b), (b1) According to the personality type test result, a plurality of markers are attached to the face of the model having the corresponding personality type based on the feature points, and the seven facial expressions of expressionlessness, surprise, fear, anger, disgust, happiness, and sadness of the model. Photographing; And (b2) Comparing the positions of markers of the expressionless images and the six emotion-expression images taken by personality type, respectively, calculating the average moving distances of all markers between the expressionless face images and the emotion-expression face images, and scaling them. An exaggerated expression generating method reflecting personal tendencies, characterized in that it comprises the step of generating a mapping table. The method of claim 7, wherein Calculation formula of the average moving distance of the marker in the step (b2)

(s: marker index, m _s : marker vector of expression, m _Ns : marker vector of expressionless expression). The method of claim 7, wherein The ratio in the step (b2) is

(

,

,

Exaggerated expression generation method reflecting personality, characterized in that performed using the scale parameter) expression. 10. The method of claim 9, Exaggeration index calculation formula for each personality type calculated in step (c) is

(s _i : style index, n: exponent parameter, k: scale parameter, w (0-1): exaggeration rate, C: convergent value) 3. The method of claim 2, The step (d) (d1) receiving a two-dimensional image of an individual; (d2) automatically detecting and outputting a feature point from an input two-dimensional image using an AAM (Active Appearance Model) algorithm; (d3) According to the user's operation, when the user drags the feature point with the mouse for precise control of the feature point, the point and the line increase or decrease like a rubber band to form a shape. Changing the position of the detected feature point by moving the rubber band to increase accuracy; (d4) transforming the three-dimensional face model into a personal three-dimensional face model by using the finally determined position values and 3D warping techniques; And (d5) Exaggerated expression generation method reflecting personality, characterized in that it comprises the step of performing texture mapping on the individual three-dimensional face model. The method of claim 11, Exaggerated expression generation method reflecting the personality tendency, In the step (f), in applying the exaggerated motion vector to the virtual muscle, each part of the three-dimensional face model or the individual three-dimensional face model is divided into a plurality of cube-shaped feature boxes according to the position of the marker, and After calculating the ratio of x, y, z extraction distance for each divided feature box of the three-dimensional face model or the individual three-dimensional face model, this ratio is multiplied by the change amount of the corresponding marker to apply the personal tendency. How to create the exaggerated expression reflected. The method of claim 12, The three-dimensional face model or the individual three-dimensional face model is an exaggerated facial expression generation method that reflects personal tendencies, characterized in that divided into 12 feature boxes in consideration of muscle movement according to the expression.

Images