KR101734212B1 - Facial expression training system - Google Patents

Abstract

The present invention discloses a facial expression practice system which helps facial expression practice by using a photographing apparatus such as a camera. A facial expression practice system according to the present invention comprises: an interface unit for providing photographing information, presenting a practice face model to a user, and transmitting a first facial expression value of the practice face model; an analyzing unit for analyzing the photographing information to generate user face information; a comparison unit for generating a facial expression guide model corresponding to the first facial expression value and the user facial information, and providing comparison information obtained by comparing the facial expression guide model and the user facial information to the interface unit; and a database for storing the practice face model. The first facial expression value is a value obtained by digitizing a facial expression amount of the practice face model.

Description

Facial Exercise System {FACIAL EXPRESSION TRAINING SYSTEM}

The present invention relates to a facial expression exercising system, and more particularly, to a facial expression exercising system that assists a user in facial expression practice by analyzing information captured by a user.

The natural and charming facial expression plays an important role in making the first impression good. In addition, through various exercises of facial expression, it is possible to prevent the mouth tails from sagging through tautness and control of the facial muscles, and the face can be seen to be alive.

In addition, a variety of facial exercises are required to provide acting or favorable impression, such as those who require rehabilitation of facial muscles, such as patients with paralyzed facial muscles, aspiring learners, or stewardesses.

Therefore, a system that can recognize, analyze, and evaluate a user's facial expression in real time and practice various facial expressions is required.

In a system for photographing and analyzing a user's face using a photographing device such as a camera, a conventional facial expression exercising system provides the same facial expression exercising function to all users without considering the shape of a user's face.

In such a case, there is a problem that it is difficult to obtain the effect of the expression exercise practically by providing a collective expression exercise environment without consideration of various types of faces for each user.

In addition, the conventional technique has a problem in that it is difficult to practice various facial expressions that are not inputted in the database, since the facial expression to be practiced can be retrieved only in the facial expression information already inputted in the database.

In addition, the conventional art has a problem that it is difficult to practice the change of the expression according to the passage of time because the expression to be practiced is limited to the photograph which does not move.

An object of the present invention is to provide a facial expression exercise system for recognizing, analyzing and evaluating a user's facial expression in real time.

Another object of the present invention is to provide a facial expression exercising function that takes into account different facial types for each user.

Another object to be solved by the present invention is to make facial expression exercises possible for various facial expressions which are not inputted in the database.

Another object to be solved by the present invention is to provide a function of practicing a change of facial expression according to time.

The facial expression exercise system of the present invention provides photographing information photographed by a user, presents an exercise face model for user's facial expression exercise to a user, and transmits a first facial expression value of the practice face model selected by the user An interface unit; An analysis unit for analyzing the shooting information and generating user face information corresponding to the user's face; A comparison unit for generating a facial expression guide model corresponding to the first facial expression value and the user facial information and providing comparison information obtained by comparing the facial expression guide model and the user facial information to the interface unit; And a database for storing the practice face models and providing the practice face models corresponding to the request of the interface unit; And the first facial expression value is a value obtained by digitizing a facial expression of the practice face model.

In addition, the facial expression exercising system of the present invention includes an interface unit for providing photographing information photographed by a user and image information for comparing the photographing information; An analysis unit for receiving and analyzing the photographing information and the image information, providing user face information corresponding to the face of the user included in the photographing information, and a first facial expression value of the image information; And a comparison unit configured to generate a face guide model corresponding to the user face information and the first facial expression value and to provide comparison information obtained by comparing the facial expression guide model and the user facial information to the interface unit; And the first facial expression value is a value obtained by digitizing the facial expression.

As described above, the present invention can provide a facial expression exercising system for recognizing, analyzing and evaluating facial expressions of a user in real time.

In addition, the present invention provides a facial expression exercising function considering different facial types for each user.

In addition, the present invention automatically generates facial expression data through a facial expression image input by a user for various facial expressions that are not input in the database, and allows the user to perform facial expression exercises.

Further, the present invention can practice the change of the expression according to the passage of time.

1 is a block diagram showing an embodiment of a facial expression exercising system of the present invention.
2 is a block diagram showing another embodiment of the expression exercise system of the present invention.
FIG. 3 is a diagram of a photographing method for generating the photographing information of FIGS. 1 and 2. FIG.
4 is a diagram of various face models that can be classified according to facial identity values and facial expression values.
5 is a diagram illustrating a method by which the user of FIG. 1 modifies an exercise face model.
FIG. 6 is a diagram illustrating a case where the image information of FIG. 2 is a picture file.
FIG. 7 is a diagram illustrating a method of selecting an exercise face model to be subjected to a face exercise exercise among the exercise face models stored in the database 400 by the user of FIGS. 1 and 2. FIG.
FIG. 8 is a diagram for illustrating the rendering of the comparison guide model in FIGS. 1 and 2; FIG.
FIG. 9 is another view for illustrating that the comparison unit of FIGS. 1 and 2 renders a facial expression guide model.
FIG. 10 is a diagram for illustrating that the comparison unit of FIG. 1 and FIG. 2 displays a visual annotation.
Fig. 11 is a diagram illustrating that the user of Figs. 1 and 2 has completed the expression exercise according to the expression guide model.
FIG. 12 is a flowchart showing a process of exercising facial expression according to the facial expression exercise system of FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It is to be understood that the terminology used herein is for the purpose of description and should not be interpreted as limiting the scope of the present invention.

The embodiments described in the present specification and the configurations shown in the drawings are preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention and thus various equivalents and modifications Can be.

1 is a block diagram showing an embodiment of a facial expression exercising system of the present invention.

Referring to FIG. 1, an embodiment of the present invention includes an interface unit 100, an analysis unit 200, a comparison unit 300, and a database 400.

The user can use the facial expression exercise system of the present invention through the interface unit 100. [ Here, the user refers to a person who wants to exercise a facial expression using the facial expression exercise system of the present invention. The facial expression exercising system according to the present invention may be an application that can be installed in a smart device such as a smart system, or a program that can be installed in a desktop, a notebook, or the like.

The present invention can be applied not only to a mobile phone having a built-in camera but also to a camera capable of recognizing a face in the system, such as a portable multimedia player (PMP), a personal digital assistant (PDA) It is available if you have built in sensors that can recognize facial expressions.

The interface unit 100 provides shooting information to the analyzer 200, which captures an image of a user through a camera or a sensor for recognizing various facial expressions.

In addition, the interface unit 100 can induce the user's selection by presenting the exercise face model provided by the database 400 to the user.

In addition, the interface unit 100 can help the user exercise the facial expression by displaying the comparison information provided by the comparison unit 300 to the user.

Here, the photographing information may refer to a photographing device including a camera, a file of a user's photographed through a face recognizing sensor, or an image photographed in real time by a user.

In addition, the comparison information may include information on a process of comparing the user's face information and a facial expression guide model, a comparison screen, or a comparison result.

The interface unit 100 receives at least one practice face model for facial expression practice and presents the model to the user and transmits the facial expression value (first facial expression value) of the practiced face model selected by the user to the comparison unit 300 .

The practice face model may be a 3D face model made up of various expressions. In addition, it is possible to have different face expression values (first facial expression values) for each exercise model. The practice face model may be stored in the database 400.

The facial expression value means a value obtained by digitizing facial expression. That is, it can have a different value depending on the degree of change of expression of a face model such as an exercise face model or a user's face model to be performed. The facial expression value can be expressed by various variables. For example, the facial expression value associated with the eye shape of the face model may have a different value depending on the degree of change of the eye shape, and the facial expression value associated with the mouth shape of the face model may be different depending on the degree of change of the mouth shape Lt; / RTI > In addition, various facial expression values can be defined in consideration of the positions and orientations of facial parts and facial muscles. In this case, the facial expression value may have the same facial expression value if the facial expression represented by the facial model, that is, the facial identity value, is different but the same facial expression is expressed.

Here, the facial expression value of the practice face model may be regarded as the first facial expression value, and the facial expression value of the user facial model may be regarded as the second facial expression value.

In addition, the practice face model can be composed of a sequence in which the expression changes according to the passage of time, not the stationary facial expression. When the practice face model is composed of a sequence, the facial expression value (first facial expression value) possessed by the practice model may also be composed of a sequence that varies with the passage of time. When the user selects an exercise face model composed of a sequence, the user can practice the change of the expression together without being limited to a single expression.

The interface unit 100 may request a plurality of exercise face models out of the exercise face models stored in the database 400 and present them to the user. Herein, the practice face model requested by the interface unit 100 is a constant weight having a higher weight according to a weight (for example, a higher weight is assigned to a more practiced face model selected by the user) separately given to each practice face model A number of practice face models, and may be determined arbitrarily. In addition, the database 400 may provide all exercise face models stored in the database 400 according to the set values.

When the user selects one of the presented exercise face models, the interface unit 100 delivers the face expression values (first facial expression values) of the selected exercise face model to the comparison unit 300. [

In addition, the interface unit 100 can transmit the facial expression value (first facial expression value) of the modified practice face model when the user corrects the facial expression of the practicing facial model.

That is, the user can select the practice face model presented from the interface unit 100, and further modify the selected practice face model. When the user corrects the practice face model, the face expression value (first facial expression value) of the modified practice face model is provided to the comparison unit 300 by the interface unit 100.

When the user modifies the practice face model, the facial expression value (first facial expression value) can also be modified. More specifically, when the user modifies the practice face model, the expression of the practice face model can be changed by directly modifying the parameter or numerical value expressing the facial expression value (first facial expression value) of the practice face model. In addition, the user can indirectly modify the related facial expression value (first facial expression value) by clicking or touching the eyes or mouth of the practicing face model on the screen on which the practicing face model is displayed, There is also.

The analysis unit 200 receives and analyzes the photographing information and generates user face information corresponding to the face of the user.

Here, the user face information is information including various information about the user face model and the user face model. The user face model means that the current face of the user who is being photographed is recognized as a 3D model by recognizing the shape and facial expression of the user's face included in the photographing information. The user face information may further include a face expression value (second face expression value), a face identity value, a face size information, a face rotation information, a face position information, and a camera information of user face information to be set have.

The process in which the analyzer 200 generates the user's face information from the photographing information may be performed in real time every frame of the photographing information. That is, the analysis unit 200 can generate user face information by analyzing the shooting information in real time.

At this time, since the user's face information is generated by analyzing the shooting information in real time by the analyzing unit 200, the user's face model of the user's face information has the effect of following the user's face included in the shooting information in real time.

The analyzer 200 may generate a user's face model from the photographing information and extract a face identity value and a facial expression value (second facial expression value) from the user's facial model.

Here, the face identity value means a value obtained by digitizing the type of the face shape. For example, there may be various face shapes such as a wide forehead, a cheekbone protruding face, a small or large eye, and a long face with a corner or jaw, and a different face identity value may be set according to each shape.

Even if you practice the same expression, the shape of your face may vary from person to person. Therefore, when user face information is generated, a face identity value having a different value can be considered for each user.

The analyzer 200 can further extract face size information, face position information, face rotation information, and camera information from the generated user face model in addition to the face identity value and the facial expression value (second facial expression value).

Here, the face size information is information on the size of the face of the user included in the photographing information, and may include a parameter for scaling of the 3D face model.

The face position information may include a position or a translation matrix of the 3D face model as information on the position of the user's face included in the shooting information in the screen.

The face rotation information is information on how much the face of the user included in the photographing information is rotated toward which direction, and matrix information about the rotation of the 3D face model may be included.

The camera information is information on a focal length of a camera that photographs a user, and may include information for projecting the camera.

The comparison unit 300 then compares the facial identity value, the facial expression value (second facial expression value), the facial size information, the facial position information, the facial rotation information, and the camera information extracted from the user facial model by the analysis unit 200, It can be used when rendering the model. In addition, the facial expression value (second facial expression value) extracted from the user facial model can be used when the comparing unit 300 compares the facial expression value (first facial expression value) of the practice facial model.

The user face model or the practice face model can be expressed by the following equation.

&Quot; (1) "

Equation (1) is an expression of a 3D face model. When V is a 3D face model such as a user face information or an exercise face model, C _r corresponds to a core structure model. C _r is the value learned inside the system that is not directly visible to the user, and analysis of the 3D face model is started based on C _r . W _id is the face identity value, and W _exp is the face expression value. The function f () can be developed in various ways. That is, the 3D face model can generate various face models by changing the values of W _id and W _exp in C _r .

&Quot; (2) "

Equation (2) is a mathematical expression re-expressing Equation (1) by a tensor operation.

The tensor operation is an expression for parameterizing the face identity value, the facial expression value, and the face structure information (vertex) with each other.

The facial identity value, facial expression value, facial size information, facial position information, facial rotation information, and camera information extracted from the user facial information by the analyzing unit 200 can be used in the rendering process as follows.

&Quot; (3) "

Equation (3) corresponds to a mathematical expression when a 3D face model calculated by Equation (1) is displayed to a user on an actual screen through the interface unit 100 and is modeled.

In Equation (3), the 3D model corresponds to the 3D model shown to the user on the actual screen, s corresponds to face size information, P corresponds to camera information, R corresponds to face rotation information, and T corresponds to face position information.

That is, the 3D model shown to the user on the actual screen rotates the 3D face model of Equation (1) represented by Cr, W _id, and W _exp , and resizes (S) and camera projection (P) , And then adjusted by adjusting the position (T).

&Quot; (4) "

Equation (4) is an objective function used in optimizing the 3D face model of Equation (3) from the camera in the face exercise program of the present invention. That is, it means an objective function used when restoring the face shown in the image to the 3D face model.

Here, n is the number of feature points in the face image having the feature point of the 3D face model and the corresponding meaning, X _i ^3d is the feature point of the 3D face model, and X _i ^2d has the corresponding meaning (X) is the feature vector or descriptor of the feature point X, s is the 3D face size information, P is the camera information, R is the face rotation information, and T is the face position information .

Here, the minutia refers to a point at which the facial expression of the facial model is determined, which means that it has a feature that is easily recognizable visually. For example, an outline defining the shape of an eye or a mouth, a line point of an eyelid, and an outline of an eyebrow may be exemplified.

In addition, the feature vector means that the feature points are represented by information of dimensions other than image information.

In this case, dist [] represents the distance between two vectors, and the degree of similarity can be determined using at least one of a cosine distance, an Euclidean distance, and a Hamming distance.

Here, the cosine distance means a method of finding the similarity between two data by a cosine calculation. The Euclidean distance means a method of knowing the similarity between two data by L2 norm. The Hamming distance indicates a degree of similarity between the two data, The difference between the elements means the method of measuring the number of parts.

In addition, (X) can use feature vectors through Feature Extraction using Convolutional Neural Network or Deep Neural Network.

Here, the deep neural network refers to a learning technique using a network composed of a combination of a convolution layer, a pooling layer, a normalization layer, and a recurrent layer by extending an existing neural network.

In order to solve Equation (4), at least one of a gradient descent method, a regression method using a continuous regressor, and a method using a recurrent neural network can be used have.

The slope descent method is a method using the differential concept, which means an asymptotic algorithm that approaches the minimum or maximum of the function step by step in proportion to the gradient at the current position. That is, it means searching along the gradient starting from the initial estimate of the solution of the problem to find the minimum or maximum value of the given problem.

In addition, the regression method through the continuous regressor means an algorithm that continuously searches for a series of regressors by learning a solution to an existing problem in order to obtain a minimum or maximum value of a function. In this case, as a regressor, a continuous linear regressor that expresses a problem as a feature space of another dimension and obtains a solution of the problem by repetitive linear combination in the feature space, A tree-based regressor for solving a problem by successive combinations of characteristic vectors of a tree or FERN structure, and the like.

In addition, when a neural network is constructed, the neural network is characterized by a repetitive cyclic structure in which the direction of data is not forward feed forward but can proceed both forward and backward (Recurrent) it means.

The analysis unit 200 may use Adaboost (Adaptive Boosting) and Convolutional Neural Network to generate user face information from the photographing information.

Here, both Adaboost and Spiral Neural Network are algorithms used for Machine Learning. Adaboost means to create a stronger classifier by combining weak classifiers successively, and it can be increased by using it with other kinds of learning algorithms. In addition, spiral neural network is a type of artificial neural network that allows a certain pattern to be grasped in a large amount of data.

As a method that can be used when the analysis unit 200 analyzes shooting information, a method including at least one of a slope descent method and a regression method through a continuous regressor may be used.

In addition, the analysis unit 200 may use various techniques for representing the image of the image as a descriptor of another dimension as the corresponding information (Corresponding Information) used when analyzing the photographing information, rendering the facial expression guide model, , A convolution filter can be learned and a feature or a descriptor obtained by using a convolutional neural network expressing a feature in a multilayer layer including convolution can be used.

The comparison unit 300 receives the facial expression values (first facial expression value) and the user facial information of the practice face model and generates a facial expression guide model corresponding to the received facial expression values. In addition, the interface unit 100 provides comparison information in which the facial expression guide model is compared with the user face information.

Here, the expression guide model refers to a 3D model in which a user has created a facial expression that the user has practiced to match the face shape of the user. Accordingly, the facial expression guide model may include a facial expression value (first facial expression value) provided from the interface unit 100 and a facial identity value of the user facial information provided from the analysis unit 200, facial size information, facial rotation information, Information and camera information.

The comparing unit 300 may render the facial expression guide model in the entire area or a part of the area where the user's face is located in the photographing information.

That is, the comparison unit 300 may generate a facial expression guide model in which a face of the user is restored to the 3D model, and render the facial expression guide model in accordance with the face position, size, and degree of rotation of the user on the screen.

When the facial expression guide model is rendered, the comparing unit 300 may process the facial expression guide model in a transparent or translucent manner. The user can perform facial expression exercises along the enhanced facial expression guide model on his / her face appearing in the photographing information by rendering the facial expression guide model to be followed by the user on the face of the user appearing in the photographing information transparently or translucently.

Also, when the facial expression guide model is rendered, the comparing unit 300 may extract only the feature points of the facial expression guide model and render the feature points on the face of the user included in the photographing information.

That is, the user can exercise the facial expression in accordance with the expression of the facial expression guide model having his / her facial shape included in the photographing information. Also, the comparison unit 300 compares the facial expression guide model and the user's face information to provide the comparison information to the interface unit 100, and the interface unit 100 displays the comparison information to the user.

When comparing the facial expression guide model and the user's face information, the comparing unit 300 determines the similarity between the facial expression value (first facial expression value) of the facial expression guide model and the facial expression value (second facial expression value) of the user's facial information .

Also, when comparing the facial expression guide model and the user face information, the comparing unit 300 can determine and compare the position similarities of the vertexes representing the feature points of the facial expression guide model and the user face information.

At this time, the comparison unit 300 calculates the difference between the facial expression values and the positions of the minutiae points by using at least one of the cosine distance, the Euclidean distance, and the Hamming distance, The degree of similarity can be determined.

The comparing unit 300 may determine the degree of similarity between the user's face information and the facial expression guide model, and determine that the user's facial expression exercises have been successfully completed when the similarity degree is equal to or greater than the critical point.

Here, the critical point means whether the comparison unit 300 regards the user's facial expression practice to be complete as long as the facial expression of the user is similar to the facial expression of the user. The critical point can be arbitrarily set by the user.

The comparing unit 300 may provide the comparison information including a visual annotation for indicating the difference between the user's face information and the facial expression guide model. The visual annotation serves as a guide to how the user should change his facial expression. In other words, the visual annotation enhances the difference between the facial expression guide model and the user facial information at the same time, thereby giving feedback on the degree of the facial exercise and knowing which direction should be further moved in which direction. The visual annotation is intended to help intuitive understanding of the face part that the user needs to change in face expression exercise, and the visual annotation can be displayed by indicating media information such as symbols, pictures, moving pictures, characters, etc., .

2 is a block diagram showing another embodiment of the present invention.

Referring to FIG. 2, an embodiment of the present invention includes an interface unit 100, an analysis unit 200, and a comparison unit 300. In the embodiment of FIG. 2, the description of the structure and effect overlapping with the embodiment of FIG. 1 is omitted.

The embodiment of FIG. 2 is an embodiment in which a user provides shooting information and image information through the interface unit 100. FIG. Here, the image information may be an image or an image file including a face to be subjected to a face exercise.

Since the embodiment of FIG. 2 does not use the practice face model stored in the database 400, unlike the embodiment of FIG. 1, the face expression value (first facial expression value) is analyzed from the image information I have to pay.

If the image information is an image file, the analyzer 200 analyzes the face included in the image file and generates a facial expression value (first facial expression value) which is composed of a sequence of at least one facial expression, Can be provided.

If the image information is a photo file, the analyzer 200 may analyze the face included in the photo file to provide a facial expression value (first facial expression value).

In other words, unlike the case of selecting one of the practice face models already stored in the database, unlike in FIG. 1, the embodiment of FIG. 2 selects a video or photo file as separate image information As shown in Fig.

The analyzer 200 receives and analyzes the photographing information and the image information, generates user face information corresponding to the face of the user included in the photographing information, and provides a facial expression value (first facial expression value) of the image information .

1, the analyzing unit 200 analyzes the image information composed of video or photo files in the same manner as in the embodiment of FIG. 1 to generate an exercising face model and outputs the facial expression values of the exercising face model (E.g., a facial expression value) and provide the same to the comparison unit 300.

The comparison unit 300 generates a facial expression guide model corresponding to the user facial information and the facial expression value (first facial expression value) provided from the analysis unit 200, and compares the facial expression guide model with the user facial information. Can be provided to the interface unit 100.

The embodiment of FIG. 2 stores the facial expression values (first facial expression values) of the image information and the image information analyzed by the analysis unit 200, and stores corresponding image information and image information (First facial expression value) of the first facial expression value.

FIG. 3 is a diagram of a photographing method for generating photographing information provided by the interface unit 100 of FIG.

Referring to FIG. 3, the present invention can be implemented in a device (e.g., a smart device, a notebook, a desktop, or the like) having a camera or a face recognition sensor capable of photographing or recognizing a user.

4 is a diagram of various face models that can be classified according to facial identity values and facial expression values. Referring to FIG. 4, the horizontal axis represents the face identity value (W _id ), and the vertical axis represents the change in the facial expression value (W _exp ). Such as facial identity value (W _id) branches even facial value according to the change of (W _exp) can have a variety of expressions, such as facial expression values (W _exp) branches even face identity value of the face according to the (W _id) It can be seen that the shape can be changed.

5 is a diagram illustrating a method by which a user modifies an exercise face model. 5, the user can modify the exercise face model by directly modifying the face expression value by modifying the parameter representing the face expression value, such as 51 and 52, and the user can modify the exercise face model The facial expression value of the relevant site can be indirectly modified by directly touching or clicking the facial expression.

FIG. 6 is a diagram illustrating a case where the image information of FIG. 2 is a picture file. Referring to FIG. 6, a case where a user inputs a photo file including an image 61 through the interface unit 100 can be illustrated. The analysis unit 200 recognizes the face included in the image 61 and analyzes the face to generate the practice face models 63, 64 and 65, extracts facial expression values of the practice face model, 300).

FIG. 7 is a diagram illustrating a method for the user to select an exercise face model to be subjected to a face exercise exercise among the exercise face models stored in the database 400. FIG. Referring to FIG. 7, the interface unit 100 presents various exercise face models stored in the database 400 to a user so that the user can select an exercise face model.

8 is a diagram for illustrating that the comparison unit 300 renders a facial expression guide model. Referring to FIG. 8, it can be seen that the facial expression guide model 82 or 83 is rendered in consideration of face size, face rotation, camera information, etc. in an area where the user's face included in the photographing information is located 81 .

9 is another diagram for illustrating that the comparison unit 300 renders the facial expression guide model. Referring to FIG. 9, the comparing unit 300 may render the facial expression model 93 transparently or semi-transparently in an area where the user's face is located in the photographing information (91). The comparing unit 300 may extract the feature points of the facial expression guide model 93 and render (92) the feature points of the facial expression model 93 in the region where the user's face is located in the photographing information.

10 is a diagram for illustrating that the comparison unit 300 displays visual annotations. Referring to FIG. 10, it can be seen that arrows are displayed around the mouth of the user so that the user can easily follow the facial expression guide model.

Fig. 11 is a diagram illustrating that the user completes the expression exercise according to the facial expression guide model. Referring to FIG. 11, when the user follows the facial expression guide model, the comparing unit 300 determines whether the face information of the user and the facial expression guide model have a degree of similarity equal to or greater than a critical point. If it is determined that the degree of similarity is equal to or greater than the critical point, the comparison unit 300 considers that the user's facial expression exercises have been successfully completed.

FIG. 12 is a flowchart showing a process of exercising facial expression according to the facial expression exercise system of FIG.

Referring to FIG. 12, the interface unit 100 presents an exercise face model to a user, and the user selects an exercise face model as a target of face exercise (S1).

 The interface unit 100 provides the comparison unit 300 with a facial expression value (first facial expression value) of the practicing face model selected by the user (S2).

The analysis unit 200 generates user face information using the photographing information provided from the interface unit 100 (S3).

The comparing unit 300 generates a facial expression guide model using the facial expression value (first facial expression value) of the practicing face model selected by the user and the facial identity value of the user facial information (S4).

The comparing unit 300 compares the user's face information with the facial expression guide model to determine whether the similarity is greater than or equal to a threshold value (S5).

If it is determined that the similarity degree is lower than the threshold value as a result of the comparison unit 300, the user's facial expression practice has failed.

If it is determined that the similarity degree is equal to or greater than the threshold value as a result of the comparison, the user determines whether the facial expression guide model is composed of a sequence (S6)

If the facial expression guide model is a single facial expression rather than a sequence, the facial expression practice is terminated. If the facial expression guide model is composed of a sequence, the following sequence is used as a facial expression practice.

As described above, the present invention can provide a facial expression exercising system for recognizing, analyzing and evaluating facial expressions of a user in real time. In addition, the present invention provides a facial expression exercising function considering different facial types for each user. In addition, the present invention allows facial expression exercises for various facial expressions that are not input in the database. Further, the present invention can practice the change of the expression according to the passage of time.

100: interface unit 200: analysis unit
300: comparison unit 400: database

Claims (25)

An interface unit for providing photographing information photographed by a user, presenting a practice face model for user's facial expression exercises to a user, and transmitting a first facial expression value of the practice face model selected by the user;
An analysis unit for generating user face information including a user face model obtained by analyzing the photographing information and restoring the face of the user in 3D;
A comparison unit for generating a facial expression guide model corresponding to the first facial expression value and the user facial information and providing comparison information obtained by comparing the facial expression guide model and the user facial information to the interface unit; And
A database for storing the exercise face model and providing the exercise face model in response to a request of the interface unit; Lt; / RTI >
Wherein the first facial expression value is a value obtained by digitizing a facial expression of the practice face model. 2. The apparatus of claim 1, wherein the interface
And when the user corrects the face of the exercise face model, the first face expression value of the modified exercise face model is delivered. 3. The method of claim 2, wherein the database
And the user stores the modified exercise face model when the user corrects the expression of the practice face model. 2. The apparatus of claim 1, wherein the interface
Wherein the first facial expression value of the training face model is composed of at least one sequence of consecutive facial expressions. 5. The method according to claim 4,
And the second face is composed of a series of consecutive sequences corresponding to the first facial expression, and the facial expression changes with the passage of time. The apparatus of claim 1, wherein the analyzing unit
Wherein the photographing information is analyzed using at least one of a gradient descent method, a regression method using a continuous regressor, and a cascade regressor. The apparatus of claim 1, wherein the analyzing unit
Wherein the facial expression information is generated by analyzing the photographing information in real time. The method of claim 1,
The user's face model, the face identity value extracted from the user's face information, and the information about the second facial expression value,
The face identity value is a value obtained by digitizing the type of the face shape,
Wherein the second facial expression value is a value obtained by digitizing a facial expression of the user facial model. 9. The method according to claim 8,
And further comprising face size information, face position information, face rotation information, and camera information extracted from the user face information. 10. The apparatus of claim 9, wherein the comparing unit
Wherein the facial expression model generating unit generates the facial expression guide model using the facial identity value, the facial size information, the facial position information, the facial rotation information, and the camera information of the first facial expression value and the user facial information Exercise system. 11. The apparatus of claim 10, wherein the comparing unit
Wherein the degree of similarity between the first facial expression value and the second facial expression value is determined when comparing the facial expression guide model and the user facial information. 12. The apparatus of claim 11, wherein the comparing unit
Wherein at least one of a cosine distance, an Euclidean distance, and a Hamming distance is used when determining the degree of similarity. 9. The apparatus of claim 8, wherein the comparing unit
Wherein the degree of similarity between the facial expression guide model and a position of a vertex indicating a feature point of the user facial model is determined when comparing the facial expression guide model and the user facial information. 14. The apparatus of claim 13, wherein the comparing unit
Wherein at least one of a cosine distance, an Euclidean distance, and a Hamming distance is used when determining the degree of similarity. 9. The apparatus of claim 8, wherein the comparing unit
And providing the comparison information including a visual annotation for indicating a difference between the user's face model and the facial expression guide model. 9. The apparatus of claim 8, wherein the comparing unit
Determining a degree of similarity between the user's face model and the facial expression guide model and determining that the user's facial expression exercises have been successfully completed when the similarity degree is equal to or greater than a critical point. 17. The method of claim 16,
Wherein the user can be arbitrarily set by the user. The apparatus of claim 1, wherein the comparing unit
Wherein the facial expression model is rendered in an area where the face of the user is located in the photographing information. 19. The apparatus of claim 18, wherein the comparing unit
And extracting and rendering the feature points of the facial expression guide model when rendering the facial expression guide model. 19. The apparatus of claim 18, wherein the comparing unit
Wherein when the facial expression guide model is rendered, the facial expression model is rendered transparent or translucent. 19. The apparatus of claim 18, wherein the comparing unit
And when the facial expression model is rendered, the facial expression model is rendered in the entire area or a part of the area where the user's face is located in the photographing information. An interface unit for providing photographing information photographed by a user and image information for comparing the photographing information;
The user's face information including the user's face model in which the photographing information and the image information are received and analyzed and the user's face included in the photographing information is restored in 3D and a first facial expression value of the image information Analysis section; And
A comparison unit for generating a facial expression guide model corresponding to the user facial information and the first facial expression value and providing comparison information for comparing the facial expression guide model and the user facial information to the interface unit; Lt; / RTI >
Wherein the first facial expression value is a value obtained by digitizing a facial expression. 23. The method of claim 22,
Wherein the image information is an image file,
Wherein the analyzing unit analyzes the face included in the image file and provides the first facial expression value corresponding to a face composed of at least one sequence of consecutive facial expressions. 23. The method of claim 22,
The image information is a picture file,
Wherein the analysis unit analyzes the face included in the photo file to provide the first facial expression value of the photo file. 23. The method of claim 22,
A database for storing the image information analyzed by the analysis unit and the first facial expression values of the image information and providing the first facial expression values of the image information and the image information corresponding to a request of the interface unit; And a face exercise program.

Images