KR102497688B1 - Method for correcting facial asymmetry using expression training - Google Patents

Abstract

The present invention determines the user's facial asymmetry by digitizing facial information through image analysis of the user's face with various facial expressions, and can easily correct the user's facial asymmetry through facial expression training of a balance model most similar to the user's facial skeleton. It's about technology that makes it possible.
A facial asymmetry correction service method using facial expression training according to the present invention includes a first step of acquiring facial images of each user with different facial expressions including expressionless expressions as facial expression images in a service server; The second step of extracting a plurality of points of each part including the mouth, the first horizontal reference line extending the top of the eyebrow in the expressionless image in the service server and the first horizontal reference line from both tear gland points is 1:1 A third step of generating a facial reference line composed of a vertical reference line extending from the reference line position and the center point between both lacrimal gland points and a second horizontal reference line extending connecting both lip tail points, the vertical reference line and Gravity consisting of a vertical vertical gravity line mutually perpendicular to the point where the first horizontal reference lines meet, a horizontal first horizontal gravity line, and a second horizontal gravity line perpendicular to the vertical gravity line based on the lip tail point on the higher side. A fourth step of generating a reference line, the service server determines an asymmetric type based on whether or not the face reference line is inclined and the direction of the facial reference line with respect to each gravity reference line, and the asymmetric type is the first asymmetric type in which only the second horizontal reference line is in an inclined state. and a second asymmetric type in which the second horizontal reference line and the first horizontal reference line are inclined in different directions, and a third asymmetric type in which the second horizontal reference line and the first horizontal reference line are inclined in the same direction. The fifth step, the sixth step of determining the standard model of the facial structure most similar to the user's face from the standard models pre-registered in the service server, and analyzing the facial movement using the user's expression image, and the service server, the asymmetric type and It is characterized in that it comprises a seventh step of generating a facial expression training schedule consisting of training information to have a standard model facial expression image based on the facial motion analysis result and providing it to the user terminal.

Description

Facial asymmetry correction service method using expression training {Method for correcting facial asymmetry using expression training}

The present invention determines the user's facial asymmetry by digitizing facial information through image analysis of the user's face with various facial expressions, and can easily correct the user's facial asymmetry through facial expression training of a balance model most similar to the user's facial skeleton. It's about technology that makes it possible.

The movement of the face interacts with the numerous muscles that make up the face by being connected to 22 bone pieces. Facial asymmetry is a phenomenon that appears as a result of these complex causes of the musculoskeletal system.

The main cause of facial asymmetry is very closely related to an individual's lifestyle (mastication, sleeping posture, chin resting with hands, postural imbalance of the upper and lower body), and these habits affect the musculoskeletal system of the face for a long time, resulting in the loss of facial muscles. It appears as contraction relaxation and tension, whether or not it is used, and changes in the skeletal structure of the face and head due to the muscles.

Also, the consequences of facial asymmetry can have a very profound effect on the balance and movement of the temporomandibular joint.

The temporomandibular joint is a part that regulates the meridian system and brain activity, and the temporomandibular joint is also the basis of brain activity treatment. The reason is that numerous nerves and blood vessels pass around the temporomandibular joint, so if facial asymmetry occurs, the balance of the nervous system This is because breaking can cause many problems.

If left unattended, it can cause chronic headaches, edema, color changes, shoulder hangs, dizziness, vomiting, difficulty in chewing, and disorders of the nervous and muscular system, such as abnormal secretion of hormones. It is necessary to prevent its progress.

In addition, facial asymmetry can not only give a bad impression on interpersonal relationships, but also negatively affect self-efficacy in social relationships because it causes appearance stress and complexes.

This is a factor that lowers self-confidence, leading to psychological passivity in accurately conveying one's intentions to the other person or persuading the other person, affecting the area of leading emotional expression and self-reliance, and limiting or reducing one's social influence to subordinate to the environment. The easier it is to go in that direction.

As a result of these results, facial asymmetry is not limited to disease problems, but it is very closely related to psychological issues in life related to individual sociality and self-efficacy, so it is necessary to solve this problem.

To this end, facial asymmetry-related procedures or surgical methods have been used, but currently used facial asymmetry-related procedures or surgical methods use specialized equipment in hospitals, and thus have the disadvantage of requiring a lot of cost.

1. Japanese Patent Laid-open Publication No. 2019-524397 (Title: Method for realizing an orthodontic appliance for treating facial asymmetry)

Therefore, the present invention was created in view of the above circumstances, and determines the user's facial asymmetry by digitizing facial information through image analysis of the user's face with various facial expressions, and trains the facial expression of the balance model most similar to the user's facial skeleton. Its technical purpose is to provide a facial asymmetry correction service method using facial expression training that enables users to easily correct facial asymmetry through

According to one aspect of the present invention for achieving the above object, a first step of obtaining facial images of each user with different facial expressions including expressionless expressions in a service server as facial expression images, A second step of extracting a plurality of points of each part including a first horizontal reference line in which the distance between the first horizontal reference line extending the top of the eyebrow in the expressionless image in the service server and the first horizontal reference line at both lacrimal gland points is 1:1 A third step of generating a facial reference line consisting of a vertical reference line extending from the center point between the position and both lacrimal gland points and a second horizontal reference line extending connecting both lip tail points; 1 Gravity reference line consisting of a vertical vertical gravity line mutually perpendicular to the point where the horizontal reference lines meet, a first horizontal gravity line in the horizontal direction, and a second horizontal gravity line perpendicular to the vertical gravity line based on the upper lip tail point In the fourth step of generating, the service server determines an asymmetric type based on whether or not the facial reference line is inclined and the direction of the facial reference line with respect to each gravity reference line, the asymmetric type is the first asymmetric type in which only the second horizontal reference line is in an inclined state and , a second asymmetric type in which the second horizontal reference line and the first horizontal reference line are inclined in different directions, and a third asymmetric type in which the second horizontal reference line and the first horizontal reference line are inclined in the same direction. 5th step, a 6th step of determining a standard model of a facial structure most similar to the user's face from standard models pre-registered in the service server and analyzing facial movements using the user's facial expression image, and asymmetric type and face in the service server A facial asymmetry correction service using facial expression training, characterized in that it comprises a seventh step of generating a facial expression training schedule consisting of training information to have a facial expression image of a standard model based on the motion analysis result and providing it to the user terminal. A method is provided.

In addition, in the first step, the service server presents various induction sentences for inducing facial expressions of expressionless expression, joy, surprise, fear, sadness, anger, and disgust to the user terminal, and the reaction expression for each induction sentence is the user's basic Provided is a facial asymmetry correction service method using facial expression training, characterized in that it is acquired as a facial expression image.

delete

In addition, in the sixth step, the service server combines the user facial expression images acquired in the first step to additionally generate a user facial expression image, and then analyzes the facial movement using the user facial expression image including the additionally generated facial expression image. , In the seventh step, the service server generates training information to have a standard model facial expression image including an additionally generated facial expression image, and the additionally generated facial expression image includes a joy image, a surprise image, a fear image, A facial asymmetry correction service method using facial expression training, characterized by sequentially generating a satisfaction image, a boundary image, a worry image, a discouragement image, a hardened image, and a contempt image by sequentially combining sadness images, anger images, and disgust images. Provided.

In addition, in the fourth step, the service server calculates an asymmetric attribute value based on the distance between the gravity reference line and each part point on both sides and the distance between the part points, and in the seventh step, the service server calculates the facial movement related to the asymmetric attribute value. A facial asymmetry correction service method using facial expression training is provided, wherein training information is generated by selecting a certain number of facial expression images in a large order.

In addition, in the seventh step, the service server compares the user's facial expression image with the facial expression image of the standard model, calculates the facial movement difference for each facial expression image, selects a certain number of facial expression images in the order of the largest facial movement difference, and obtains training information. There is provided a facial asymmetry correction service method using facial expression training, characterized in that for generating.

According to the present invention, facial values are measured through facial image analysis of various facial expressions, facial asymmetry is determined through this, and facial asymmetry of the user is easily corrected through facial expression training of a balance model most similar to the user's facial skeleton. can do.

In addition, the user can recognize the asymmetry of his or her face and identify the cause to prevent bad habits and improve the wrong musculoskeletal condition by non-surgical and non-surgical methods.

In addition, according to the present invention, it is possible to give a better impression in interpersonal relationships through the practice of attractive facial expressions having facial features most similar to one's own.

1 is a diagram showing a schematic configuration of a facial asymmetry correction service system using facial expression training to which the present invention is applied.
Figure 2 is a flow chart for explaining a facial asymmetry correction service method using facial expression training according to the present invention.
3 is a flowchart for explaining a process (ST200) of extracting facial numerical information shown in FIG. 2;
4 and 5 illustrate the facial numerical information extracted in FIG. 3 in an expressionless image.
6 is a diagram for explaining an asymmetric type determination method (ST300) shown in FIG. 2;

The embodiments described in the present invention and the configurations shown in the drawings are only preferred embodiments of the present invention, and do not represent all the technical ideas of the present invention, so the scope of the present invention is limited to the embodiments and drawings described in the text. should not be construed as being limited by That is, since the embodiment can be changed in various ways and can have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, the scope of the present invention should not be construed as being limited thereto.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Terms defined in commonly used dictionaries should be interpreted as being consistent with meanings in the context of related art, and cannot be interpreted as having ideal or excessively formal meanings that are not clearly defined in the present invention.

1 is a diagram showing a schematic configuration of a facial asymmetry correction service system using facial expression training to which the present invention is applied.

Referring to FIG. 1, a facial asymmetry correction service system using facial expression training to which the present invention is applied includes a user terminal 100 and a service server 200, and the service server 200 includes an information acquisition unit 210. ), a facial state analysis unit 220, a training processing unit 230, and a data storage 240.

The user terminal 100 is a personal communication terminal having a camera 1, and may be a wireless terminal such as a smart phone or tablet PC, or a wired terminal such as a laptop or desktop.

The user terminal 100 transmits face images of each user with different facial expressions, including expressionless expressions, to the service server 200 . At this time, the user terminal 100 transmits to the service server 200 various information related to facial correction, including survey response information corresponding to a questionnaire related to the user's lifestyle, service selection information for facial correction service, and environmental information according to training. do.

The information acquisition unit 210 collects face images transmitted from the user terminal 100 . At this time, the information acquisition unit 210 provides the user terminal 100 with facial expression induction information for obtaining facial images of various facial expressions, that is, facial expression images, and photographs reaction facial expressions thereto to capture a plurality of different facial expression images. can be obtained.

The facial state analyzer 220 analyzes the facial expression image collected by the information acquisition unit 210 to digitize the facial information and calculates an asymmetric attribute value based on the facial numerical information. At this time, the facial state analyzer 220 measures and digitizes the overall contour of the face and the position, size, angle, interval, etc. of each part in the face image using Open CV (Open Source Computer Vision) algorithm.

In addition, the facial state analyzer 220 may additionally apply lifestyle-related questionnaire response information to the facial asymmetry state analysis.

The training processing unit 230 determines the asymmetric type of the user's face based on the asymmetric attribute value, determines the standard model most similar to the corresponding facial skeleton in the model big data, and determines each part of the face is trained CNN (synthesis The user's facial movement is analyzed by applying the user's facial expression image to the convolutional neural network (RNN) model and the recurrent neural network (RNN) model. Then, the training processing unit 230 compares the user's asymmetric attribute value with the balance facial numerical information of the standard model and analyzes the difference.

In addition, the training processing unit 230 determines the asymmetric type based on the facial numerical information, and creates a facial expression training schedule consisting of training information to have a facial expression image of the standard model based on the asymmetric type and the facial motion analysis result.

In addition, the training processor 230 transmits training information to the user terminal 100 and collects training feedback information input from the user terminal 100 in response thereto. In addition, the training processing unit 230 provides training feedback information to the facial state analysis unit 220 to request analysis, generates training evaluation information based on the analysis result, and provides it to the user terminal 100 .

The data storage 240 stores various types of information for providing facial correction services, including the Open CV algorithm, CNN and RNN models trained on each part of the face, standard models of balance for each facial type, survey information, and asymmetry. Information including reference information for each type is stored. Here, the CNN model is an AI algorithm for measuring movement of facial parts, and the RNN model is an AI algorithm for measuring interaction, vertical relaxation information and direction of facial muscles moving in conjunction with the CNN model. In addition, standard model information is stored together with a plurality of different facial expression images for each standard model.

Next, a facial asymmetry correction service method using facial expression training having the above configuration will be described.

2 is a flowchart illustrating a facial asymmetry correction service method using facial expression training according to the present invention.

First, a user accesses the service server 200 through the user terminal 100 and receives and installs a facial correction program for a facial asymmetry correction service according to the present invention. Hereinafter, the user terminal 100 performs the following operations according to the execution of the face correction program, and the case of a smart phone equipped with a camera will be described.

While the face correction program is running, the user terminal 100 accesses the service server 200, captures a face image of itself through a camera, and transmits it to the service server 200 (ST100).

In the step ST100, the service server 200 provides the user terminal 100 with various inducing sentences for inducing corresponding facial expressions in order to acquire face images having seven preset facial expressions, and different A user face image of a facial expression is obtained from the user terminal 100 .

That is, in the present invention, the service server 200 acquires seven basic facial expression images of expressionless expression, laughter, joy, surprise, fear, sadness, anger, and disgust by using the induction sentence. Table 1 below illustrates information provided from the service server 200 to the user terminal 100 to acquire 7 basic facial expression images.

expression
Sentence Features and Example Sentences
expressionless
Put your hands on your lap or on a table and make a look of indifference when you are alone with no one around.
laugh
Try to copy the tone and tempo of the voice you hear now.
'Fu hahahaha', 'Laughing brings blessings'
pleasure
This is the expression of a loved one after a long time, or a good friend or family member.
Think of it in your head and make a facial expression.
'Nice to meet you, friend hahaha', 'I love you daddy (mom)''My beloved son (daughter)'
surprised
Try to copy the tone and tempo of the voice you hear now.
'what? Is that true?', 'I forgot! Aren't you surprised?'
fear
Listen to the following story.
'A crocodile suddenly attacked a gazelle that was drinking water. The spray rose and the gazelle tried with all its might to escape, but the crocodile grabbed the gazelle with its strong teeth and tossed it around with unbearable force. '
Imagine this story happening to you and keep it in your head.
sadness
What was the saddest moment for you?
Have you ever been heartbroken the day you broke up with someone you loved?
'Mom, I think my heart is just blown open here. There is nothing, no matter how much I fill it, I can't put it in. I miss my dad so much. I miss you right now noooo
anger
Imagine a situation where you are angry and try to express your feelings when you see the picture below.
'A person demonstrating or protesting or in a standoff'
hate
What would your face look like if you saw something unpleasant or terrifying?
'You're dirty, who stepped on your ass? oh what a smell Wow! 'I'm about to throw up'

At this time, the user terminal 100 outputs a photographing point through the display window so that the user gazes at it, and the user terminal 100 photographs its front face and side face using a camera according to the user's manipulation. Here, the user performs face image capture in a state in which the sizes of both ears match when capturing the frontal face and the tails of the lips match the facial contour when capturing the side face.

Subsequently, the service server 200 calculates facial numerical information of the user based on the expressionless face image, that is, the expressionless image (ST200).

At this time, the service server 200 applies the expressionless image to the Open CV algorithm to measure the overall contour of the face and the position, size, angle, and spacing of each part, and extracts the user's facial numerical information necessary for correcting facial asymmetry. .

In the process of extracting facial numerical information, as shown in FIG. 3 , the service server 200 first extracts preset body part points from an expressionless image (ST210). As shown in Figure 4, the site points are the lacrimal glands (P1, P2) of each site, the white tips of the eyes (P3, P4), the tails of the lips (P5, P6), the tips of the ears (P7, P8), and the upper lip. It includes a groove (P9).

In addition, the service server 200 sets facial reference lines using points of each part in the expressionless image (ST220). As shown in FIG. 4, facial reference lines include a vertical reference line VL set on the center of the face in the vertical direction, a first horizontal reference line HL1 formed in the horizontal direction on the upper side of the face, and a horizontal reference line HL1 on the lower side of the face. It consists of a second horizontal reference line HL2 formed in the direction.

In the process of creating a facial reference line, first, an extension line connecting upper ends of both eyebrows is created as a first horizontal reference line HL1.

First, a first vertical point (VP1) is set at a position where the distance from each lacrimal point (P1, P2) of both eyes to the first horizontal reference line (HL1) is 1:1, and the lacrimal gland points (P1, P2) of both eyes are set. ) by setting the center point of the connection line between them as the second vertical point VP2, and passing through the first vertical point VP1 and the second vertical point VP2, an extension line meeting the first horizontal reference line HL1 at right angles It is created as a vertical reference line (VL).

In addition, an extension line connecting both lip tail points P5 and P6 is created as a second horizontal reference line HL2.

In the above state, the service server 200 creates a measurement line for determining a facial asymmetry state by connecting the points P1 to P8 and the facial reference line as shown in FIG. 4 (ST230). The measurement line is a line connecting both ear tip points (P7, P8) perpendicularly to the first horizontal reference line (HL1), a line connecting both tear glands (P1, P2) and the white tip (P3, P4) points of the eye, A line connecting the points of the tails of the lips P5 and P6 and a line connecting the points of the tails of the lips P5 and P6 and the grooves of the lips P9 are included.

Then, as shown in FIG. 5, the service server 200 sets a vertical gravity line GL1 and a first horizontal gravity line GL2 perpendicular to the point where the vertical reference line VL and the first horizontal reference line HL1 meet. Then, based on the upper lip tail point among the lip tails, a gravity reference line consisting of a vertical gravity line GL1 and a second horizontal gravity line GL3 perpendicular to the vertical gravity line GL3 is generated (ST240).

In addition, the service server 200 measures the distance between each part point on both sides of the gravity reference line, the distance between different part points, the distance between measurement lines, the angle, and the area of the surface formed by the measurement line connecting the points. An asymmetric attribute value is calculated based on the value (ST250). At this time, the service server 200 sets a comparison line by rotating all measurement lines of the face parallel to the horizontal gravity lines GL1 and GL2, and determines the asymmetric attribute value based on the difference between the comparison line and the user's face measurement line. yield In the present invention, the asymmetry attribute is an item for determining asymmetry, the height of both ear tips, the size of both eyes, the horizontal height of both eyes, the horizontal height of the lip tip, the left and right balance of the nose, the left and right balance of the mouth, the left and right eyebrows and the corners of the mouth length, and the left and right areas of the face.

Subsequently, the service server 200 determines the asymmetric type based on the facial numerical value information calculated in the step ST200 (ST300). At this time, the service server 200 compares the user's face reference line shown in FIG. 4 with the gravity reference line shown in FIG. Determine the asymmetric type according to

In Fig. 6, three types of asymmetry are illustrated.

Referring to FIG. 6 , in the present invention, three types of asymmetric types are classified according to whether the horizontal reference lines HL1 and HL2 are inclined and the direction of the inclination. The first asymmetric type (A) is a case where only the second horizontal reference line is inclined, the second asymmetric type is a case where the second horizontal reference line and the first horizontal reference line are inclined in different directions, and the third asymmetric type is a case where the second horizontal reference line and the first horizontal reference line are inclined in different directions. It is divided into a case where the horizontal reference line and the first horizontal reference line are inclined in the same direction.

The first asymmetry type A is the facial asymmetry state at the earliest stage, and only the second horizontal reference line HL2 is set to an inclined state. This is a state in which the lower jaw is distorted to the left or the right, and the asymmetry is not yet evident in areas other than the lower jaw.

The second asymmetric type (B) is a condition in which the distortion of the lower jaw becomes more severe in the first asymmetry type (A) state, and is the most common facial asymmetry type. In the second asymmetric type (B), both the first and second horizontal reference lines HL1 and HL2 are inclined, and are set to be inclined in different directions. This is analyzed as a type in which the externally rotated upper jaw (maxilla) is twisted in the same direction as the lower jaw as the temporal bone is pushed in the direction the lower jaw is distorted, and the eyes and cheekbones sink in the direction the lower jaw is distorted, causing the face to look distorted. The nose is often crooked. In this case, as aging progresses, wrinkles around the eyes and nasolabial folds are induced more deeply in a distorted direction.

The third asymmetry type (C) is the most severe type of facial asymmetry, and the first and second horizontal reference lines HL1 and HL2 are both inclined and set to be inclined in the same direction. This is a state in which the asymmetry has progressed further in the second asymmetric type (B) and the displacement of the skull, such as the temporal bone and the sphenoid bone, has progressed. This is the upward state, and the displacement of the facial bones and skull is analyzed as the most advanced type.

Subsequently, the service server 200 determines the user's face value, preferably, the most similar skeletal value among the symmetric face standard models registered in the data storage 240 based on the facial value information calculated in step ST200. Determine the model (ST400). At this time, the service server 200 transforms each part point and measurement line to be balanced based on the center line of gravity, and determines an optimal standard model based on the converted balance value.

Then, the service server 200 analyzes the user's facial movements based on the user's various facial expression images (ST500).

At this time, the service server 200 uses GANs (Generative Adversarial Networks, deep convolutional generative adversarial networks) learned from big data facial image data sets of 7 different facial expression images to add 6 facial images of facial expressions create with

Six new facial expression images are generated through the combination of facial expression images in the manner shown in Table 2 below.

default expression
synthetic expression
Facial expression generated by GAN
2. Joy
1. Expressionless
8. Satisfaction
3. Surprise
9. Boundary
4. Fear
10. Worry
5. Sorrow
11. Discouragement
6. Anger
12. Hardened
7. Dislike
13. Contempt

Referring to Table 2, the basic facial expression and synthesized facial expression are face images obtained from the user terminal 100 in step ST100, and the facial expressions generated through GAN are additionally generated face images. By sequentially combining images, fear images, sadness images, anger images, and disgust images, a satisfaction image, a caution image, a worry image, a discouragement image, a hardened image, and a contempt image are sequentially created. For example, in Table 2, a “satisfied” facial expression image is generated by combining a happy facial expression image and an expressionless expression image, and a “border” facial expression image is generated by combining a “surprise” facial expression image and an expressionless expression image.

That is, the service server 200 analyzes the user's facial movement by applying a total of 13 facial expression images to the CNN model and the RNN model in which each part of the face has been learned. At this time, the service server 200 presents a preset short sentence to measure the movement of the user's facial muscles during a conversation, acquires a face image reading the sentence from the user terminal 100, and displays the preset conversation content. A face image in a state of outputting and listening to a corresponding conversation may be obtained from the user terminal 100 and additionally reflected in the facial motion analysis.

Subsequently, the service server 200 generates a facial expression training schedule consisting of sequential training information for each date based on the asymmetric type determined in step ST300 and the facial movement analysis result determined in step ST500, and transmits it to the user terminal 100 (ST600, ST700).

At this time, the service server 200 creates a facial expression training schedule targeting the standard model facial expression images for 13 facial expressions including the user's standard facial expression and additionally generated facial expressions, that is, the standard image. That is, the service server 200 compares the user's facial expression image with the standard image and creates a facial expression training schedule for displaying the corresponding facial expression.

For example, the service server 200 may select a certain number of facial expression images in order of increasing facial movements related to the asymmetric attribute, and generate training information to make the corresponding facial expressions become facial expressions of the standard image.

In addition, the service server 200 compares each facial expression image of the corresponding user with the facial expression image of the standard model to calculate the facial movement difference for each facial expression image, selects a certain number of facial expression images in the order of the largest facial movement difference, and selects the corresponding facial expression. Training information to be the facial expression of this standard image can be generated.

This facial expression training schedule includes basic facial expression training consisting of muscle relaxation for each expression, facial movement, point acupressure, and massage, and practice of moving two or more facial parts simultaneously after practicing the movements of the facial parts individually, Step-by-step training consisting of making facial expressions step by step and then making facial expressions at once, deriving improvement points from expressionless state, generating and presenting images of soft and likeable facial expressions, and training on areas in need of improvement It may include liking training in which elements are derived and training is performed. Here, the standard model for liking training may be directly selected by the user through the user terminal 100 from among a plurality of models similar to the user's facial skeleton.

Meanwhile, the user terminal 100 transmits its own facial expression image according to the training schedule provided from the service server 200 to the service server 200 as training feedback information (ST800).

The service server 200 analyzes training feedback information received from the user terminal 100 to perform training evaluation, and transmits training result information for this to the user terminal 100 (ST900).

That is, according to the embodiment, facial values are measured through facial image analysis of various facial expressions to determine facial asymmetry, and the user's facial asymmetry can be easily corrected through facial expression training of a balance model most similar to the user's facial skeleton. can

100: user terminal, 200: service server,
210: information acquisition unit, 220: facial state analysis unit,
230: training processing unit, 240: data storage.

Claims (6)

A first step of acquiring facial images of each user with different facial expressions, including expressionless expressions, as facial expression images in a service server;
A second step of extracting a plurality of points of each part including the eyes, ears, and mouth from the expressionless image in the service server;
A first horizontal reference line extending the top of the eyebrow in the expressionless image in the service server, and a vertical reference line extending the center point between the first horizontal reference line position where the distance between the first horizontal reference line at both lacrimal gland points is 1:1 and both lacrimal gland points, and , a third step of generating a facial reference line consisting of a second horizontal reference line extending to connect both lip tail points;
In the expressionless image in the service server, the vertical gravity line in the vertical direction mutually perpendicular to the point where the vertical reference line and the first horizontal reference line meet, the first horizontal gravity line in the horizontal direction, and the vertical gravity line based on the lip tail point on the higher side A fourth step of generating a gravity reference line consisting of a second vertical horizontal gravity line;
The service server determines an asymmetric type based on whether or not the face reference line is inclined and the direction of the facial reference line with respect to each gravity reference line. The asymmetric type is a first asymmetric type in which only the second horizontal reference line is in an inclined state, A fifth step of dividing into a second asymmetric type in which one horizontal reference line is inclined in different directions and a third asymmetric type in which the second horizontal reference line and the first horizontal reference line are inclined in the same direction;
A sixth step of determining a standard model of a facial structure most similar to the user's face from standard models pre-registered in the service server and analyzing facial movements using the user's facial expression image;
A seventh step of creating a facial expression training schedule consisting of training information to have a facial expression image of a standard model based on the asymmetric type and facial movement analysis result in the service server and providing it to the user terminal Facial expression, characterized in that it is configured to include Facial asymmetry correction service method using training.
delete According to claim 1,
In the first step, the service server presents various induction sentences to the user terminal for inducing expressions of expressionless expression, joy, surprise, fear, sadness, anger, and disgust, and the reaction expression for each induction sentence is a basic facial expression image of the user. Facial asymmetry correction service method using facial expression training, characterized in that obtained by.
According to claim 1,
In the sixth step, the service server additionally generates a user expression image by combining the user expression images obtained in the first step, and then analyzes the facial movement using the user expression image including the additionally generated expression image,
In the seventh step, the service server generates training information to have a standard model facial expression image including an additionally generated facial expression image,
The additionally generated facial expression image is a satisfaction image, a caution image, a worry image, a discouragement image, a hardening image, and a contempt image by sequentially combining a joy image, a surprise image, a fear image, a sadness image, an anger image, and a disgust image in an expressionless image. A facial asymmetry correction service method using facial expression training, characterized in that sequentially additionally generated.
According to claim 1,
In the fourth step, the service server calculates an asymmetric attribute value based on the distance between the gravity reference line and each part point on both sides and the distance between the part points,
In the seventh step, the service server generates training information by selecting a certain number of facial expression images in order of increasing facial movements related to the asymmetry attribute value.
According to claim 1,
In the seventh step, the service server compares the user's expression image and the expression image of the standard model to calculate the facial movement difference for each facial expression image, and selects a certain number of facial expression images in order of increasing facial movement difference to generate training information Facial asymmetry correction service method using facial expression training, characterized in that.

Images