KR101436730B1 - 3d face fitting method of unseen face using active appearance model - Google Patents

Abstract

The present invention provides a method for 3D face fitting of an unlearned face using an active appearance model. The method comprises the steps of: (a) inputting an image; (b) extracting a face region including an internal region and an external region adjacent to the internal region by warping the input image to an average image; (c) applying a single scale retinex (SSR) to the face region, and extracting face texture from the face region by mapping the application result to the face region within a range from a predetermined maximum result value to a predetermined minimum result value; (d) extracting a texture error by applying the extracted face texture to an active appearance model; (e) extracting a face shape from the extracted face texture; (f) extracting a shape error by applying the extracted face shape to the active appearance model; and (g) giving a predetermined weight to the texture error and the shape error, aggregating the weighted errors, and fitting a face image in the input image to a model face by applying the aggregate result to the active appearance model. Therefore, the method can minimize a generalized error by considering the texture and shape of a face.

Description

TECHNICAL FIELD [0001] The present invention relates to a 3D facial fitting method of a non-learning face using an active appearance model,

The present invention relates to a 3D face fitting method of a non-learning face using an active appearance model, and more particularly, to a 3D face fitting method of a non-learning face using an active appearance model that minimizes a generalized error considering a texture and a shape of a face Fitting method.

As various image processing applications have been increasing in recent years, face model fitting techniques can be effectively used in various fields such as face recognition, photo correction, robot interaction, and the like. On the other hand, various studies for facial model fitting have been active in the meantime, but facial fitting for non-learning faces remains a major problem, and facial model fitting technology is becoming a major cause of difficulty in practical applications.

For accurate face model fitting, it is important to find the face shape from the input face while minimizing errors between the input face and the fitted face model. Generally, these face model fitting methods can be classified into two types.

The first method is a local feature-based method in which each major feature point of a face is searched using a learned local template or a regression function. We learn the change of each facial feature from learning face and construct whole facial feature point by using parameter model such as Active Appearance Model (AAM).

However, this method can cause an error in each feature point so that an inaccurate facial fitting can be performed. In addition, since the fitted face model changes depending on the parameter model used, it is not suitable to be applied to the fitting of the non-learning face.

The second method is a model-based method that minimizes the error between the input face and the fitted face model. This method determines the accuracy of the face fitting depending on the error function used and the degree of optimization of the function.

This model-based method shows more accurate face fitting results than the regional feature-based method, but has a limited disadvantage that the change of the fitable face model is also limited because of the limited change of the learned face, which is not effective for non-learning face fitting . In addition, the model-based method is sensitive to initial parameters and is prone to fall into local minima during optimization.

In order to perform effective facial fitting on more various faces by using the model-based method, it is necessary to use an error function that reflects a general face change rather than a limited learning face change, and develops a mechanism that does not fall into the local minimum in the optimization process Should be.

In order to construct a generalized error model for non - learning face fitting, we consider the texture and shape characteristics of face simultaneously. (The entire contents of the present invention have been deleted.) The main cause of the difference in the texture between the faces is the difference in the face texture between the different persons and the difference in illumination occurring in the face. The first cause is skin color between different people, difference in facial contour, presence or absence of hair black hair and beard, and it is difficult to minimize this difference because it varies from person to person. On the other hand, the texture difference caused by the illumination is very large because it gives a large change to the facial pixels in the input image, and minimizes the difference, thereby minimizing the texture difference.

Since face shapes are similar between different people, they can be a major factor for face fitting. Existing research that reflects the facial shape has learned the x-axis and y-axis gradient of each pixel in the face model. However, this additional learning does not correctly reflect the human face shape similarity because it has x-axis and y-axis changes that are limited to the learning face changes.

Meanwhile, various face fitting methods using a face model have been proposed to reflect the general characteristics of the face. For example, in TJ Cootes, GJ Edwards, and CJ Taylor, "Active Appearance Models (23 (6): 681-685, 2001.) Active appearance model (AAM).

The face shape and texture space is constructed by applying Principal Component Analysis method to the learning face. The AAM adjusts the texture and shape change of the face at once by adjusting the model parameter c. In order to fit the face model to the input face, the AAM estimates the distortion parameter such as the size of the model and the movement of the model together with the model parameter by minimizing the error between the input face and the face model as shown in Equation 1 .

[Equation 1]

로 와핑(Warping)시켜주는 함수이고, A는 질감 파라미터이고, g_i는 질감 고유 벡터이고,

는 학습 영상의 평균 얼굴 질감 벡터이다.Here, W (I) represents the input face image I as an average face shape

A is a texture parameter, g _i is a texture eigenvector,

Is the mean face texture vector of the training image.

AAM can perform relatively accurate facial fitting for various facial expression changes by learning the facial expression change of the input facial as learning facial variation. However, since it is difficult to learn all the changes due to the three-dimensional pose change of the face, it is generally useful only in front face images.

In order to solve these problems, TF Cootes, GV Wheeler, KN Walker, and CJ Taylor published a pose of a face in the article "View-based active appearance models (In Image and Vision Computing, vol. 20, 657-664, And a View-based AAM that constitutes an independent AAM for each pose. To perform an effective fitting on the input face with a three-dimensional pose change, the second article on T. F. Cootes selects the input image and the AAM with the smallest error. However, since it is difficult to learn all three-dimensional pose changes, it is difficult to obtain accurate fitting results.

M. Zhou, L. Liang, J. Sun, and Y. Wang in "AAM Based Face Tracking with Temporal Matching and Face Segmentation (CVPR, 2010.) We proposed an effective facial fitting method in video by combining additional methods such as temporal matching. However, this method is disadvantageous in that face fitting is not effective when the difference between the model and the input face is large, and a lot of calculation time is required through additional calculation.

Since a face of a real person has a three-dimensional shape, a three-dimensional model is more efficient than a two-dimensional model for face fitting. V. Blanz and T. Vetter proposed 3D Morphable Model (3DMM) as a method using 3D face model in the paper "A Morphable Model for the Synthesis of 3D Faces (In SIGGRAPH'99 Conference Proceedings, 1999.) Respectively.

3DMM expresses the face as a statistical model between 3D shape and texture by directly using 3D information of face using 3D scanner or depth camera. The shape space and texture space are created in the same way as AAM. However, it is not easy to use 3D methods because it requires high cost to directly acquire 3D information of face.

Recently, 3DAAM methods have been proposed to estimate face 3D information from 2D images based on AAM. C.-W. Chen and C.-C. Wang describes the depth information estimated using the AAM and stereo in the paper "3D Active Appearance Model for Aligning Faces in 2D Images (IEEE Proceedings of the IEEE / RS International Conference on Intelligent Robots and Systems, 3133-3139, 2008.) And proposed a combined 3DAAM method.

However, it is difficult to fit face in real - time because stereo computation requires a large amount of computation to estimate the face depth. In addition, since face has many similar face areas due to the same skin color, estimation of face depth using stereo can provide inaccurate 3-dimensional information.

The model-based methods described above are not suitable for non-learning face fittings because non-learning faces can have large differences in the shape and texture of learned faces. To address the differences between these non-learning and learning faces, C. Zhao, W.-K. Cham, and X. Wang proposed a model-based method that can arrange them at once using multiple images of non-learning faces in the paper "Face Alignment with a Generic Deformable Face Model (CVPR, 2011.)". This method minimizes the rank of matrices formed by face vectors while using AAM to perform model fitting for each face.

This rank minimization method is proposed by Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma in "Robust Alignment by Sparse and Low-rank Decomposition for Linearly Correlated Images (CVPR, 2010.) And the input face has an advantage that it is not dependent on the learning face. However, this method is not suitable for fitting a single face image or a limited number of face images because a large number of mutually related face vectors are required to construct a matrix for performing rank minimization.

For effective face fitting of a single facial image, a method of combining regional changes of each facial feature point with a model-based method has been proposed. J. Heo and M. Savvides proposed a two-dimensional face image from a face image in the paper "In Between 3D Active Appearance Models and 3D Morphable Models (In IEEE Conf. On Computer Vision and Pattern Recognition Workshops, 20-26, 2009.) 3DAAM, and proposed CASAAM which combines this with Active Shape Model (ASM).

ASM is more efficient for non-learning faces because it considers regional changes in each facial feature point. CASAAM performed an effective facial fitting on non-learning face by performing more fitting method among AAM and ASM according to reconstruction error and performing fitting.

Unlike this, methods combining regression functions are proposed. This method detects facial feature points using a regression function instead of an optimization method. The regression function learns from a very large number of data consisting of various samples separated by true and false and detects facial feature points with less errors than ASM by boosting. However, the method using ASM and regression function has limitation on morphological change because it uses parameter model of model - based method.

Therefore, for effective face fitting on non-learning faces, common errors on the entire face, such as model-based methods, should be considered, and error models should consider both local and global facial feature changes.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide an image processing method and an image processing method that do not depend on shape and texture changes in a face, Dimensional face fitting method using an active appearance model capable of performing various non-learning face fitting only with a limited number of face change learning.

According to an aspect of the present invention, there is provided a method for fitting a face of a non-learning face using an active appearance model, the method comprising the steps of: (a) inputting an input image; (b) extracting a face region including an inner region and an outer region adjacent to the inner region by warping the input image with an average image; (c) applying a single-scale retinex (SSR) to the face region, and extracting a face texture from the face region by mapping within a range of a preset maximum value and a predetermined result minimum value; (d) extracting the texture error by applying the extracted face texture to an active appearance model; (e) extracting a face shape from the extracted face texture; (f) applying the extracted face shape to an active appearance model to extract shape errors; (g) fitting the face image in the input image to the model face by applying the texture error and the predetermined weight to the shape error, summing the result, and applying the sum result to the active appearance model Dimensional face fitting method using an Active Appearance Model, which is a feature of the present invention.

delete

In step (b), the face area may be extracted as a rectangular area including a face inside area.

In addition, in the step (d), the texture error may be performed on an area inside the face.

In the step (f), the shape error may be calculated by applying an adjustment weight that extends the edge of the average face shape.

Here, the adjustment weight may be expressed by Equation

(Where, w _d (x, y) is an adjusting weight, and size of the weight parameter α is preset for each pixel, S _Intensified (x, y) is an edge of the extended video, S _mean (x, y) Is the average face shape of the learning face image, and G is the Gaussian function).

In the step (f), the shape error can be determined and extracted by a predetermined number of pixels from the face area.

According to the present invention, a three-dimensional face fitting method of a non-learning face using an active appearance model that minimizes a generalized error considering a texture and a shape of a face is provided.

FIG. 1 is a diagram schematically illustrating a process of extracting a texture error and a shape error in a 3D face fitting method of a non-learning face according to the present invention,
FIG. 2 is a view illustrating a fitting process of a 3D face fitting method of a non-learning face according to the present invention,
FIG. 3 is a view illustrating an example of a weight function generation process for shape errors using an average shape of learning face images,
FIGS. 4 to 6 are views for explaining the effect of the method of three-dimensional face fitting of a non-learning face according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In order to reflect general facial characteristics, the 3D facial fitting method of the non-learning face according to the present invention performs correction for the error function in the above-mentioned Active Appearance Model (hereinafter, referred to as 'AAM').

The error function corrected through the 3D face fitting method of the non-learning face according to the present invention includes both the texture error and the shape error of the face. FIG. 1 is a diagram schematically illustrating a process of extracting a texture error and a shape error in a 3D face fitting method of a non-learning face according to the present invention. FIG. 2 is a diagram illustrating a 3D face fitting method FIG.

Referring to FIGS. 1 and 2, an input image is input (S10). Here, when the input image is input, the feature points of the face region are extracted from the input image to learn the face model, as shown in FIG.

Here, the input image and the learning image are firstly learned separately from the feature points of the learning image by handwriting. The training image is used to learn the change of the appearance model, and the final face fitting is performed by repeatedly optimizing the input image from steps S20 to S80 at the initial position of the appearance model.

In this case, the 3D face fitting method of the non-learning face according to the present invention extracts the face region from which feature points are extracted by including the face inside region and the face outside region adjacent to the face inside region. In FIG. 2, feature points are extracted in a face region of a rectangular region. Here, in the extraction of the minutiae, the learning image is learned and performed based on the marked minutiae, and the input image is performed on the input image area for the model position of the current fitting process.

In this way, by using only the texture information of the inner area of the face by learning the inner area of the face and the outer area of the face together, it is possible to eliminate the error that the shape of the face extracted from the texture does not appear correctly in the jaw area.

Meanwhile, the face region is extracted by warping the input image, that is, the input image from which the feature points are extracted, as an average image (S20). Then, the facial texture is extracted by applying a single size Retinex Single Scale Retinex (SSR) to the extracted face region (S30). In the 3D face fitting method of a non-learning face according to the present invention, a single-size retinex (SSR) is applied to a face region. In the case of applying a single-scale retinex (SSR) So that the face texture is extracted from the face area. Then, the extracted face texture is applied to the Active Appearance Model to extract the texture error (S60).

Then, the face shape is extracted from the extracted face texture (S40), and the extracted face shape is applied to the active appearance model (S70).

More specifically, the error function of the 3D face fitting method of the non-learning face according to the present invention is formed by combining the texture error and the shape error. In order to minimize the texture difference caused by the illumination, the mass error is applied to the facial image with a single size retinex (SSR), and the range determined by the maximum result value and the minimum result value is set to a single size reticle Ranged Single Scale Retinex (RSSR).

In general, Single Scale Retinex (SSR) estimates the reflectance of each image using the ratio of the local average of each pixel. The Single Scale Retinex (SSR) can be defined as: " (2) "

&Quot; (2) "

는 콘볼루션(Convolution) 연산자이다. Here, F (x, y) is the input facial image, R (x, y) represents the application result in which Single Scale Retinex (SSR) is applied, G (x, y) is a Gaussian filter ,

Is a convolution operator.

Single Scale Retinex (SSR) maps the application results to image values of (0, 255) in the range of the maximum and minimum values in the result to image R (x, y). In this mapping process, the results of applying Single Scale Retinex (SSR) show different results for the same pixel depending on the maximum and minimum values.

Accordingly, in the 3D face fitting method of the non-learning face according to the present invention, as described above, the maximum value and the minimum value of the single-scale retinex (SSR) are adjusted to the preset maximum value and the maximum value do. This allows the shape extracted from the application result to be adjusted more or less weakly. In the present invention, log (1.6) is set as the maximum value of the result, and log (1.0) is set as the minimum value of the result.

On the other hand, in the calculation of the face shape error, the face shape is directly extracted from the texture information of the face, as described above, without learning the shape of the face. The shape information extracted from the texture information of the face is extracted from the image waved to the average face according to the facial expression or the three-dimensional face pose according to the present invention as in step S20 of FIG.

The shape F _s of the face is the magnitude of the pixel change along the x and y axes, and can be expressed as Equation (3).

&Quot; (3) "

이고,

이다.here,

ego,

to be.

As described above, the error model of the 3D face fitting method of the non-learning face according to the present invention reflects the texture and morphological change of the face. Here, in the case of a texture error, since the face region of a rectangular shape is used as described above in the present invention, both inside and outside of the face are learned.

In the case of the area outside the face, since the learning image and the input image are greatly different from each other, if the learned texture area is directly used as an error, the result of erroneous error can not be obtained. In order to solve this problem, in the 3D face fitting method of the non-learning face according to the present invention, only the face inside area is calculated when the texture error is calculated.

In the case of a shape error, if the shape of the face is the same as the jaw, it is generated adjacent to the area inside the face, so that it is expanded by a predetermined number of pixels in the inner area of the face.

In step S20, information on the face shape of the face image watermarked with the average face is similarly expressed because the positions of the eyes, nose, and mouth are the same regardless of the facial expression or human difference. FIG. 3 is a view showing an example of a face texture and an extracted face shape generated for various faces. As shown in FIG. 3, the face shape extracted through the 3D face fitting method of the non-learning face according to the present invention has a similar shape to the various input faces. Thus, it is possible to perform face fitting on various non-learning faces by combining texture error and shape error.

On the other hand, the texture error due to the difference in texture between the input image and the face model can be defined as in Equation (4) from Equation (1) for face fitting of the active appearance model.

&Quot; (4) "

Here, the RSSR () function is a function for applying Ranged Single Scale Retinex (RSSR) according to the present invention. Similarly, the shape error between the input face and the face model can be defined as: " (5) "

&Quot; (5) "

The Shape () function is a shape extraction function using [Equation 3]. Here, the type information of the input image and the face model is not learned, but is directly extracted from the texture information as described above. Accordingly, in order to fit the face model to the input face, texture errors and shape errors are added with predetermined weights as shown in Fig. 2 (S70). [Equation (6)] represents an error that is weighted and added.

&Quot; (6) "

Here, w _e is a weight, which adjusts the error difference between the texture error and the form error, and 1 is applied so that there is no difference in weight between the texture error and the form error.

As described above, when the sum of the texture error and the shape error is completed, the result of summation is applied to the active appearance model, as shown in FIG. 2, to fit the face image in the input image to the face model S80).

On the other hand, as described above, the form error is calculated as the difference between face shapes extracted using [Equation 3] from the face model fitting with the input face. For efficient face fitting using shape errors, the shape error should be small when the edge region of the input face exists in the edge region of the face model.

On the other hand, if the edge region is present in the face model where there is no edge, the shape error must be large. In addition, for effective error optimization, the form error must gradually decrease as the edge of the input face approaches the edge of the face model as the face model is fitted to the input face. However, since the edge of the face appears as a pointed shape, the shape error has a problem that it does not have such an error reduction.

Accordingly, the 3D face fitting method of the non-learning face according to the present invention uses the average face form of the learning face image to fit various non-learning faces using the shape error reflecting the general shape of the face, And the calculation is applied.

The average face shape of the learning face represents the general shape of the face shape. FIG. 3 is a diagram illustrating a process of generating an adjustment weight of a method for fitting a three-dimensional face of a non-learning face according to the present invention.

The adjustment weight is defined as a function, which is generated by expanding the pixels of the edge in the average face shape. The expanded image S _Intensified (x, y) of the edge is defined by taking the larger value of each pixel value in the average face shape image and the Gaussian filter applied image, as shown in Equation (7).

&Quot; (7) "

Where S _mean (x, y) is the average face shape of the learning face image and G is the Gaussian function. The adjustment weight for each pixel from the edge-averaged face averaged image is computed from equation (8).

&Quot; (8) "

Here, w _d (x, y) is an adjustment weight for each pixel and? Is a predetermined size weight parameter. As described above, the summed error reflecting the adjustment weight can be expressed as Equation (9).

&Quot; (9) "

On the other hand, one of the major problems for facial fitting to non-learning faces is the limited form of change. This problem occurs because the face model can change shape only within the change of learning face. On the other hand, the input face can be changed into a shape that is significantly different from the learning face. As a result, model-based methods can not perform fitting correctly on non-training faces.

However, the facial feature points corresponding to each part of the face exist in similar positions regardless of the change in facial shape. For example, the nose is located in the center of the face and the eye is located in the upper part of the nose at the bottom of the nose. Therefore, in order to have a change of facial features for expressing various changes of a face, the feature of each element of the facial shape is adjusted in the three-dimensional facial fitting method of the non-learning face according to the present invention.

More specifically, in order to adjust the change of the minutiae corresponding to each part of the face shape, an additional form parameter s = {s ₁ , s ₂ , ... , s _n }. Each shape parameter independently adjusts the feature points for each part of the face shape. The fitting error caused by the shape parameter uses the texture error of (9).

However, if shape parameters change significantly to minimize texture errors, it is difficult to fit the shape of the input face of the face model correctly. For example, if a parameter that adjusts the size of an eye makes the size of the eye too large, the problem arises that the area of the eye exceeds the area of the face. Therefore, there is a need for a change restriction on the form parameter. Equation (10) reflects the shape parameter in Equation (9).

&Quot; (10) "

Here, w _s and a _i are weight parameters for adjusting the shape parameter change error and each shape change error of the face.

One of the ways to optimize such shape parameters is to learn the error changes according to the parameters using the learning face as in the conventional Active Appearance Model (AAM). However, the optimization result is not accurate because it can not include all normal faces entered in the learning face, and it is easy to fall into the local minimum, so that the correct face fitting can not be performed.

In order to solve this problem, the Jacobian matrix according to the morphological parameter changes is calculated at each step of performing the face fitting in the 3-dimensional face fitting method of the non-learning face according to the present invention to calculate the correct optimal morphological parameter. This method requires more computation than pre-learning, but it can perform accurate face fitting to various non-learning face forms.

Hereinafter, experimental results for verifying the effect of the 3D face fitting method according to the present invention will be described in detail.

Four public databases were used to verify the effect. The first is the IMM DB, which is composed of two frontal facial images with no expression and joy feeling for each person from 40 people, two images rotated left and right at 30 degrees, and frontal images with focused illumination and arbitrary facial expressions It consists of a total of 240 images including frontal images. 4 shows an example of the IMM DB.

The second is the BioID DB, which consists of 1,521 images from 23 people with faces close to the front. The third is the FGNet Tralking Face video, which consists of 5,000 frames with one human interview. Finally, the Labeled Faces in the Wild (LFW) DB does not have the face minutia information of the ground-truth as the DB collected from the Web on various backgrounds, lighting face pose, random face image with facial expression change.

The experiment was evaluated in two ways. First, the accuracy of the non-learning face according to the present invention is evaluated by comparing the three-dimensional face fitting method with the conventional method using four DBs. At this time, all methods for evaluation of non - learning faces were evaluated by using IMM DB. Secondly, fitting evaluation for various non - learning faces was performed by applying the proposed method learned by IMM DB and conventional method to LFW DB.

The 2D AAM, the approximated 3D AAM, and the multi-band AAM are used as the conventional fitting method in which the non-learning face according to the present invention is compared with the 3D face fitting method. In order to perform the evaluation of each partial technique of the 3D face fitting method according to the present invention, the non-learning face according to the present invention may be partially or completely divided into the RSSR, the adjustment function, A comparison was made between the results.

In order to evaluate the accuracy of the face fitting results, [Equation 11] was used by using the distance difference between the fitting results and the ground-truth feature points. In this case, the distance difference is normalized to the eye distance so that the result is not different depending on the size of the face of the input image.

&Quot; (11) "

와

은 각각 ground-truth와 피팅 결과의 특징점의 위치를 나타낸다.here,

Wow

Represent the ground-truth and the position of the feature point of the fitting result, respectively.

First, we evaluated the IMM DB. In order to evaluate the IMM DB, the DB was divided into two groups, and the images obtained from 20 different persons were included in each group.

In the first group of images, all methods were learned by using facial expressions of expressionless face and smiling face. The IMM DB provides ground-truth for each image with 58 facial feature points. However, to provide only the location of each feature point, and to apply the AAM of each fitting method, a face mesh was created as shown in FIG. In order to generate the 3D AAM, the 3D shape of the face was reconstructed using two side images of each person in the DB.

The evaluation using the IMM DB was done in two ways. First, fitting accuracy for each method was evaluated on the focused face and face of arbitrary face of 20 people learned in the DB. Secondly, we evaluated the fitting accuracy of 20 face images not included in the learning in the DB. At this time, the image of each person was divided into three images and evaluated.

The first one is the face image of the same face as the learning face, the second is the focused image, and the third is the front face image of the arbitrary face.

Table 1 shows fitting errors for each method for the IMM DB. The three-dimensional face fitting method of the non-learning face according to the present invention performed face fitting with the smallest error, while other conventional fitting methods performed more inaccurate face fitting due to the difference in the texture between the in-image noise and the learning face.

[Table 1]

After performing the learning using the IMM DB to evaluate the non - learning face, the face fitting results for the BioID DB were evaluated. All the methods evaluated were 80 face images with 2 images for 40 IMM DB users. BioID DB provides fewer number of feature points as ground-truth than IMM DB, and the position of feature points common between two DBs has a slight difference in position. Therefore, in the evaluation, as shown in FIG. 5, errors were calculated for major feature points located in the eyes, nose, mouth, and chin line.

Table 2 shows the comparison results of fitting errors for each method. In the evaluation, since the positions of the common feature points are slightly different between the two DBs, the fitting error result is slightly larger than the evaluation using only the IMM DB. Since the face images of BioID DB are obtained by different environments and different facial pose and facial expressions from the facial images of the learned IMM DB, facial fitting results are often wrong for 2D AAM and 3D AAM in several images.

On the other hand, the 3D face fitting method of the non-learning face according to the present invention performs accurate face fitting with the smallest fitting error. In addition, it can be seen that the three-dimensional face fitting method RSSR and the shape parameter of the non-learning face according to the present invention improve the fitting accuracy of the non-learning face.

[Table 2]

On the other hand, for the evaluation of the talking face video DB, all the methods were performed using the IMM DB, and then face fitting was performed. To calculate the fitting error, 40 similar facial feature points between the two DBs were used, as shown in Fig. Since each frame of the video DB is a continuous face change, the initial position for face fitting is used as the fitting result in the previous frame.

Table 3 shows a comparison of fitting error results between the methods. Since Video DB does not include special lighting on the face as a solid background, most methods performed relatively correct face fitting. However, while the existing method shows erroneous fitting in some facial feature points due to the type restriction, the 3D facial fitting method of the non-learning face according to the present invention performs accurate face fitting with the smallest error.

[Table 3]

Although several embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various modifications may be made without departing from the principles and spirit of the invention . The scope of the invention will be determined by the appended claims and their equivalents.

Claims (7)

In a 3D face fitting method of a non-learning face using an Active Appearance Model,
(a) inputting an input image;
(b) extracting a face region including an inner region and an outer region adjacent to the inner region by warping the input image with an average image;
(c) applying a single-scale retinex (SSR) to the face region, and extracting a face texture from the face region by mapping within a range of a preset maximum value and a predetermined result minimum value;
(d) extracting the texture error by applying the extracted face texture to an active appearance model;
(e) extracting a face shape from the extracted face texture;
(f) applying the extracted face shape to an active appearance model to extract shape errors;
(g) fitting the face image in the input image to the model face by applying the texture error and the predetermined weight to the shape error, summing the result, and applying the sum result to the active appearance model A three - dimensional face fitting method of non - learning face using Active Appearance Model. delete The method according to claim 1,
Wherein the face region is extracted as a rectangular region including an inner region of the face in the step (b). The method according to claim 1,
Wherein the texture error is performed on an inner area of the face in step (d). The method according to claim 1,
Wherein the shape error in the step (f) is calculated by applying an adjustment weight that extends the edge of the average face shape to the 3D face fitting method of the non-learning face using the Active Appearance Model. 6. The method of claim 5,
The adjustment weight may be calculated using Equation

(Where, w _d (x, y) is an adjusting weight, and size of the weight parameter α is preset for each pixel, S _Intensified (x, y) is an edge of the extended video, S _mean (x, y) Is the average face shape of the learning face image, and G is the Gaussian function)
Dimensional face fitting method using an Active Appearance Model, which is characterized in that the 3D face fitting method of the present invention is calculated by the following method. The method according to claim 1,
Wherein in the step (f), the shape error is determined and extracted by a predetermined number of pixels from the face inside area.

Images