KR20210086905A - Landmarks Decomposition Apparatus, Method and Computer Readable Recording Medium Thereof - Google Patents

Abstract

Disclosed are a landmark decomposition apparatus, a method, and a computer-readable recording medium. A landmark decomposition method according to an embodiment of the present invention comprises the steps of: receiving the face image of a first person and landmark information corresponding to the face image; estimating a transformation matrix corresponding to the landmark information; calculating the expression landmark of the first person using the transformation matrix; and calculating the unique landmark of the first person by using the expression landmark. It is possible to perform landmark decomposition even for targets with only a small amount of data.

Description

Landmarks Decomposition Apparatus, Method and Computer Readable Recording Medium Thereof

The present invention relates to a landmark separation apparatus, method, and computer-readable recording medium, and more particularly, to a landmark separation apparatus, method, and computer-readable recording capable of separating a landmark from one frame or a small number of frames. It's about the medium.

A facial landmark is an analysis method that extracts the key points of the main elements of the face or extracts the outline drawn by connecting the key points. Facial landmarks are used at the very bottom of techniques such as analysis, synthesis, morphing, reenactment, and classification of facial images such as facial expression classification, pose analysis, synthesis and transformation.

Existing facial image analysis and utilization technologies based on facial landmarks do not distinguish between the physical characteristics of the target and the characteristics of emotions such as facial expressions when processing facial landmarks, which is accompanied by a decrease in performance. For example, when classifying the emotions of a person whose eyebrows are higher than others, even if they are actually expressionless, they may be incorrectly classified as making a surprised expression.

An object of the present invention is to provide a landmark separation apparatus, method, and computer-readable recording medium capable of performing landmark separation even for an object having only a small amount of data.

A landmark separation method according to an embodiment of the present invention includes the steps of: receiving a face image of a first person and landmark information corresponding to the face image; estimating a transformation matrix corresponding to the landmark information; Calculating an expression landmark of the first person by using a transformation matrix, and calculating a unique landmark of the first person by using the expression landmark.

In addition, in the step of estimating the transformation matrix, a learning model trained to estimate the PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary face image may be used.

Also, the learning model may classify a plurality of landmark information into a plurality of semantic groups and output PCA transform coefficients corresponding to each of the plurality of semantic groups.

Also, in the calculating of the expression landmark, the expression landmark of the first person may be calculated by calculating the estimated transformation matrix and the PCA unit vector.

In addition, the landmark information may be defined as the sum of the expression landmark, the unique landmark, and the average landmark of a sufficiently large number of people's faces.

In addition, in the step of calculating the unique landmark, it is possible to calculate the unique landmark of the first person by calculating the expression landmark and the average landmark from the landmark information.

On the other hand, a computer-readable recording medium in which a program for performing the landmark separation method according to the present invention is recorded may be provided.

On the other hand, the landmark separation apparatus according to an embodiment of the present invention, a receiving unit for receiving a face image of a first person and landmark information corresponding to the face image, a transformation for estimating a transformation matrix corresponding to the landmark information and a calculation unit for calculating an expression landmark of the first person by using a matrix estimator and the transformation matrix, and calculating a unique landmark of the first person by using the expression landmark.

In addition, the transformation matrix estimator may use a learning model trained to estimate a PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary face image.

Also, the learning model may classify the landmark information into a plurality of semantic groups and output PCA transform coefficients corresponding to each of the plurality of semantic groups.

Also, the calculator may calculate the expression landmark of the first person by calculating the estimated transformation matrix and the PCA unit vector.

In addition, the landmark information may be defined as the sum of the expression landmark, the unique landmark, and the average landmark of a sufficiently large number of people's faces.

In addition, in the step of calculating the unique landmark of the first person, it is possible to calculate the unique landmark of the first person by calculating the expression landmark and the average landmark from the landmark information.

The present invention can provide a landmark separation apparatus, method, and computer-readable recording medium capable of performing landmark separation even for an object having only a small amount of data.

1 is a diagram schematically illustrating an environment in which a landmark separation apparatus according to the present invention operates.
2 is a flowchart schematically illustrating a landmark separation method according to an embodiment of the present invention.
3 is a diagram schematically illustrating a method of calculating a transformation matrix according to an embodiment of the present invention.
4 is a diagram schematically showing the configuration of a landmark separation apparatus according to an embodiment of the present invention.
5 is a view exemplarily showing a method of reproducing a face using the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the spirit of the present invention.

The terminology used herein is for the purpose of describing the embodiment and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” or “comprising” implies that the stated component or step does not exclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted with meanings commonly understood by those of ordinary skill in the art to which the present invention pertains. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

1 is a diagram schematically illustrating an environment in which a landmark separation apparatus according to the present invention operates. Referring to FIG. 1 , an environment in which a first terminal 10 and a second terminal 200 operate is a server 100 and a first terminal 10 and a second terminal 20 connected to the server 100 . ) may be included. For convenience of explanation, only two terminals, ie, the first terminal 10 and the second terminal 20, are shown in FIG. 1, but more terminals than two may be included. With respect to the terminal that can be added, the description of the first terminal 10 and the second terminal 20 may be applied, except for the description to be specifically mentioned.

The server 100 may be connected to a communication network. The server 100 may be connected to other external devices through the communication network. The server 100 may transmit data to or receive data from other devices connected to each other.

The communication network connected to the server 100 may include a wired communication network, a wireless communication network, or a complex communication network. The communication network may include a mobile communication network such as 3G, LTE, or LTE-A. The communication network may include a wired or wireless communication network such as Wi-Fi, UMTS/GPRS, or Ethernet. The communication network is Magnetic Secure Transmission (MST), Radio Frequency Identification (RFID), Near Field Communication (NFC), ZigBee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE, Bluetooth Low Energy) , or may include a local area network such as infrared communication (IR, InfraRed communication). The communication network may include a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN).

The server 100 may receive data from at least one of the first terminal 10 and the second terminal 20 . The server 100 may perform an operation using data received from at least one of the first terminal 10 and the second terminal 20 . The server 100 may transmit the operation result to at least one of the first terminal 10 and the second terminal 20 .

The server 100 may receive a mediation request from at least one of the first terminal 10 and the second terminal 20 . The server 100 may select a terminal that has transmitted the mediation request. For example, the server 100 may select the first terminal 10 and the second terminal 20 .

The server 100 may mediate a communication connection between the selected first terminal 10 and the second terminal 20 . For example, the server 100 may mediate a video call connection between the first terminal 10 and the second terminal 20 or a text transmission/reception connection. The server 100 may transmit connection information on the first terminal 10 to the second terminal 20 , and may transmit connection information on the second terminal 20 to the first terminal 10 .

The connection information for the first terminal 10 may include, for example, an IP address and a port number of the first terminal 10 . Upon receiving the connection information for the second terminal 20 , the first terminal 10 may attempt to connect to the second terminal 20 using the received connection information.

When the connection attempt of the first terminal 10 to the second terminal 20 or the connection attempt of the second terminal 20 to the first terminal 10 is successful, the first terminal 10 and the second terminal 20 A video call session may be established between Through the video call session, the first terminal 10 may transmit a video or sound to the second terminal 20 . The first terminal 10 may encode an image or sound into a digital signal, and transmit the encoded result to the second terminal 20 .

Also, through the video call session, the first terminal 10 may receive a video or sound from the second terminal 20 . The first terminal 10 may receive an image or sound encoded as a digital signal, and decode the received image or sound.

Through the video call session, the second terminal 20 may transmit a video or sound to the first terminal 10 . Also, through the video call session, the second terminal 20 may receive a video or sound from the first terminal 10 . Accordingly, the user of the first terminal 10 and the user of the second terminal 20 can make a video call with each other.

The first terminal 10 and the second terminal 20 may be, for example, a desktop computer, a laptop computer, a smart phone, a smart tablet, a smart watch, a mobile terminal, a digital camera, a wearable device, or a portable device. It may be an electronic device or the like. The first terminal 10 and the second terminal 20 may execute programs or applications. Each of the first terminal 10 and the second terminal 200 may be a device of the same type or may be a device of a different type.

2 is a flowchart schematically illustrating a landmark separation method according to an embodiment of the present invention. Referring to FIG. 2 , the landmark separation method according to an embodiment of the present invention includes the steps of receiving a face image and landmark information (S110), estimating a transformation matrix (S120), and calculating an expression landmark (S130), and calculating a unique landmark (S140).

In step S110, a face image of the first person and landmark information corresponding to the face image are received. Here, the landmark may be understood as a landmark of the face. The landmark may mean major elements of the face, for example, eyes, eyebrows, nose, mouth, jaw line, and the like.

In addition, the landmark information may include information about the position, size, or shape of the main element of the face. In addition, the landmark information may include information about a color or texture of a major element of the face.

The first person means an arbitrary person, and in step S110, a face image of an arbitrary person and landmark information corresponding to the face image are received. The landmark information can be obtained through a known technique, and any of the known methods may be used. In addition, the present invention is not limited by the method of obtaining the landmark.

In step S120, a transformation matrix corresponding to the landmark information is estimated. The transformation matrix may constitute the landmark information together with a predetermined unit vector. For example, the first landmark information may be calculated as a product of the unit vector and a first transformation matrix. Also, the second landmark information may be calculated as a product of the unit vector and a second transformation matrix.

The transformation matrix is a matrix that transforms high-dimensional landmark information into low-dimensional data, and may be utilized in Principal Component Analysis (PCA). PCA is a dimensionality reduction technique that converts variables in high-dimensional space into variables in low-dimensional space by finding new axes orthogonal to each other while preserving data variance as much as possible. PCA reduces the dimension of data by first finding the hyperplane closest to the data, and then projecting the data onto a low-dimensional hyperplane.

In PCA, a unit vector defining the i-th axis is called an i-th principal component (PC), and these axes are linearly combined to convert high-dimensional data into low-dimensional data.

Here, X denotes high-dimensional landmark information, Y denotes a low-dimensional principal component, and α denotes a transformation matrix.

As described above, the unit vector, that is, the principal component may be predetermined. Accordingly, when new landmark information is received, a transformation matrix corresponding thereto may be determined. In this case, a plurality of transformation matrices may exist corresponding to one piece of landmark information.

Meanwhile, in step S120 , a learning model trained to estimate the transformation matrix may be used. The learning model may be understood as a model trained to estimate a PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary face image.

The learning model may be trained to estimate the transformation matrix from face images of different people and landmark information corresponding to each face image. A plurality of transformation matrices corresponding to one high-dimensional landmark information may exist, and the learning model may be trained to output only one transformation matrix from among several transformation matrices.

The landmark information used as an input to the learning model may be obtained through a known method of extracting landmark information from a face image and visualizing it.

Accordingly, in step S120, the face image of the first person and landmark information corresponding to the face image are received as inputs, and one transformation matrix is estimated and outputted therefrom.

On the other hand, the learning model classifies the landmark information into a plurality of semantic groups corresponding to each of the right eye, left eye, nose, and mouth, and is trained to output PCA transform coefficients corresponding to each of the plurality of semantic groups. can

In this case, the semantic group is not necessarily classified to correspond to the right eye, left eye, nose, and mouth, but is classified to correspond to the eyebrow, eye, nose, mouth, and jaw line, or the eyebrow, right eye, left eye, nose, mouth, jaw line, It is also possible to classify according to the ear. In step S120, the landmark information may be classified into a semantic group of a subdivided unit according to the learning model, and a PCA transform coefficient corresponding to the classified semantic group may be estimated.

In step S130, an expression landmark of the first person is calculated using the transformation matrix. The landmark information may be decomposed into a plurality of sub landmark information. In the present invention, it is assumed that the landmark information may be expressed as follows.

Here, l(c, t) is landmark information in the t-th frame of the video including person c, l _m is human mean facial landmark information, and l _id (c) is information of person c individual. The unique landmark (facial landmark of identity geometry) information, l _exp (c, t) means the facial landmark of expression geometry of the person c in the t-th frame of the video including the person c.

That is, landmark information in a specific frame of a specific person includes average landmark information of all faces, landmark information unique to the specific person, and expression and movement information of the specific person in the specific frame. can be expressed as a sum.

The average landmark information may be defined by the following equation, and may be calculated based on a large amount of video that can be collected in advance.

Here, T means the total number of frames in the video, and therefore l _m means the average of landmark l(c, t) of all persons appearing in the video collected in advance.

On the other hand, the expression landmark can be calculated using the following equation.

The above equation represents the PCA performance result for each semantic group of person c. n _exp is the sum of the number of expression basis of all semantic groups, b _exp is the expression basis that is the basis of PCA, and α is the coefficient of PCA.

In other words, b _exp means the eigenvector described above, and a high-dimensional expression landmark may be defined as a combination of low-dimensional eigenvectors. And, n _exp means the total number of facial expressions and movements that the person c can express through his right eye, left eye, nose, and mouth.

Accordingly, the expression landmark of the first person may be defined as a set of expression information for each of the main parts of the face, that is, the right eye, the left eye, the nose, and the mouth. And, α _k (c, t) may exist corresponding to each eigenvector.

As shown in Equation 2, the learning model described above uses the photo x(c, t) of the person c to separate the landmark information and the landmark information l(c, t) as inputs, and calculates the PCA coefficient α(c, t). can be taught to estimate. Through such learning, the learning model can estimate a PCA coefficient from an image of a specific person and corresponding landmark information, and can estimate the low-dimensional eigenvector.

When applying the learned neural network, the photograph x(c`, t) and landmark information l(c`, t) of the person c` who want to perform landmark separation are input to the neural network, Estimate the PCA transformation matrix. In this case, b _exp uses a value obtained from the training data, and the predicted (estimated) PCA coefficient and b _exp can be used to estimate the expression landmark as follows.

는 추정된 표현 랜드마크,

는 추정된 PCA 변환 행렬을 의미한다.here,

is the estimated expression landmark,

denotes the estimated PCA transformation matrix.

In step S140, an identity landmark of the first person is calculated using the expression landmark. As described with reference to Equation 2, the landmark information may be defined as the sum of the average landmark information, the unique landmark information, and the expression landmark information, and the expression landmark information is expressed by the equation in step S130. 5 can be estimated.

Therefore, the unique landmark can be calculated as follows.

The Equation may be derived from Equation 2, and when an expression landmark is calculated in step S130, a unique landmark may be calculated through Equation 6 in step S140. Average landmark information l _m can be calculated based on a large amount of video that can be collected in advance.

Accordingly, when a face image of an arbitrary person is given, landmark information can be obtained therefrom, and expression landmark information and unique landmark information can be calculated from the face image and landmark information.

3 is a diagram schematically illustrating a method of calculating a transformation matrix according to an embodiment of the present invention. Referring to FIG. 3 , an artificial neural network receives an input image of an arbitrary person as an input. The artificial neural network may apply some of known artificial neural networks, and in an embodiment, the artificial neural network may be ResNet. ResNet is a type of Convolution Neural Network (CNN), and the present invention is not limited to a specific type of artificial neural network.

MLP (Multi Layer Perceptron) is a type of artificial neural network in which multiple layers of perceptrons are stacked to overcome the limitations of single-layer perceptrons. Referring to FIG. 3 , the MLP receives the output of the artificial neural network and landmark information corresponding to the face image as inputs. Then, the MLP outputs a transformation matrix.

It can also be understood that the artificial neural network and the MLP in FIG. 3 constitute one learned artificial neural network as a whole.

When the transformation matrix is estimated through the learned artificial neural network, as described with reference to FIG. 2 , expression landmark information and unique landmark information can be calculated. The landmark separation method according to the present invention can be applied even when there are only a very small number of face images or only one frame of face images.

The learned artificial neural network is trained to estimate low-dimensional eigenvectors and transform coefficients from a large number of face images and corresponding landmark information. Vector and transform coefficients can be estimated.

In this way, when the expression landmark of an arbitrary person and the unique landmark are separated, the quality of facial image processing technology such as face reenactment, face classification, and face morphing based on the facial landmark can be improved.

Face reenactment technology is a technology for synthesizing face images and photos that follow the movement of the driver's face when given a target face and a driver's face, but have the identity of the target face.

Face morphing technology is a technology for synthesizing a face image or photo of a third person that follows the characteristics of person 1 and person 2 when face images or photos of person 1 and person 2 are given. A traditional morphing algorithm finds a face key point, and then divides the face into non-overlapping triangular or square pieces based on the face key point. After that, the photos of person 1 and person 2 are combined to synthesize a picture of a third person. Since the starting positions of person 1 and person 2 are different, the pictures of person 1 and person 2 are combined pixel-wise to form a third person. If you are making a photo of a person, you may feel a sense of incongruity. Since the existing face morphing technology does not distinguish between the physical characteristics of the object and the characteristics caused by emotions such as facial expressions, the quality of the morphing result may be low.

Since the landmark separation method according to the present invention can separate the expression landmark information and the unique landmark information from one landmark information, respectively, it can contribute to improving the results of facial image processing technology using facial landmarks. In particular, since the landmark separation method according to the present invention can separate the landmark even when only a very small amount of face image data is given, the utility is very high.

4 is a diagram schematically showing the configuration of a landmark separation apparatus according to an embodiment of the present invention. Referring to FIG. 4 , the landmark separation apparatus 100 according to an embodiment of the present invention includes a receiver 110 , a transformation matrix estimator 120 , and a calculator 130 .

The receiver 110 receives the face image of the first person and landmark information corresponding to the face image. Here, the landmark may be understood as a concept including major elements of the face, for example, eyes, eyebrows, nose, mouth, jaw line, etc. as a landmark of the face.

In addition, the landmark information may include information about the position, size, or shape of the main element of the face. In addition, the landmark information may include information about a color or texture of a major element of the face.

The first person means an arbitrary person, and the receiving unit 110 receives a face image of an arbitrary person and landmark information corresponding to the face image. The landmark information can be obtained through a known technique, and any of the known methods may be used. In addition, the present invention is not limited by the method of obtaining the landmark.

The transformation matrix estimator 120 estimates a transformation matrix corresponding to the landmark information. The transformation matrix may constitute the landmark information together with a predetermined unit vector. For example, the first landmark information may be calculated as a product of the unit vector and a first transformation matrix. Also, the second landmark information may be calculated as a product of the unit vector and a second transformation matrix.

The transformation matrix is a matrix that transforms high-dimensional landmark information into low-dimensional data, and may be utilized in Principal Component Analysis (PCA). PCA is a dimensionality reduction technique that converts variables in high-dimensional space into variables in low-dimensional space by finding new axes orthogonal to each other while preserving data variance as much as possible. PCA reduces the dimension of data by first finding the hyperplane closest to the data, and then projecting the data onto a low-dimensional hyperplane.

In PCA, a unit vector defining the i-th axis is called an i-th principal component (PC), and these axes are linearly combined to convert high-dimensional data into low-dimensional data.

As described above, the unit vector, that is, the principal component may be predetermined. Accordingly, when new landmark information is received, a transformation matrix corresponding thereto may be determined. In this case, a plurality of transformation matrices may exist corresponding to one piece of landmark information.

Meanwhile, the transformation matrix estimator 120 may use a learning model learned to estimate the transformation matrix. The learning model may be understood as a model trained to estimate a PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary face image.

The learning model may be trained to estimate the transformation matrix from face images of different people and landmark information corresponding to each face image. A plurality of transformation matrices corresponding to one high-dimensional landmark information may exist, and the learning model may be trained to output only one transformation matrix from among several transformation matrices.

The landmark information used as an input to the learning model may be obtained through a known method of extracting landmark information from a face image and visualizing it.

Accordingly, the transformation matrix estimator 120 receives the face image of the first person and landmark information corresponding to the face image as inputs, and estimates and outputs one transformation matrix therefrom.

On the other hand, the learning model classifies the landmark information into a plurality of semantic groups corresponding to each of the right eye, left eye, nose, and mouth, and is trained to output PCA transform coefficients corresponding to each of the plurality of semantic groups. can

In this case, the semantic group is not necessarily classified to correspond to the right eye, left eye, nose, and mouth, but is classified to correspond to the eyebrow, eye, nose, mouth, and jaw line, or the eyebrow, right eye, left eye, nose, mouth, jaw line, It is also possible to classify according to the ear. The transformation matrix estimator 120 may classify the landmark information into a subdivided semantic group according to the learning model, and estimate a PCA transform coefficient corresponding to the classified semantic group.

The calculator 130 calculates an expression landmark of the first person by using the transformation matrix, and calculates a unique landmark of the first person by using the expression landmark. The landmark information may be decomposed into a plurality of sub landmark information, for example, may be divided into average landmark information, unique landmark information, and expression landmark information.

That is, landmark information in a specific frame of a specific person includes average landmark information of all faces, landmark information unique to the specific person, and expression and movement information of the specific person in the specific frame. can be expressed as a sum.

The average landmark information may be defined by the following equation, and may be calculated based on a large amount of video that can be collected in advance.

As shown in Equation 2, the learning model described above uses the photo x(c, t) of the person c to separate the landmark information and the landmark information l(c, t) as inputs, and calculates the PCA coefficient α(c, t). can be taught to estimate. Through such learning, the learning model can estimate a PCA coefficient from an image of a specific person and corresponding landmark information, and can estimate the low-dimensional eigenvector.

When applying the learned neural network, the photograph x(c`, t) and landmark information l(c`, t) of the person c` who want to perform landmark separation are input to the neural network, Estimate the PCA transformation matrix. In this case, b _exp is a value obtained from the learning data, and the expression landmark can be estimated as in Equation 5 using the predicted (estimated) PCA coefficient and b _exp.

On the other hand, as described with reference to Equation 2, the landmark information may be defined as the sum of the average landmark information, the unique landmark information, and the expression landmark information, and the expression landmark information is obtained in step S130. It can be estimated through Equation 5.

Accordingly, the unique landmark can be calculated as in Equation 6, and when a face image of an arbitrary person is given, landmark information can be obtained from it, and expression landmark information and Unique landmark information can be calculated.

5 is a diagram exemplarily showing a method of reproducing a face using the present invention. Referring to FIG. 5 , a target image 41 and a driver image 42 are shown, and a shape corresponding to the target image 41 may be reproduced.

It can be seen that the reproduced image 43 has characteristics of the target image 41 , but its expression corresponds to the driver image 42 . That is, the reproduced image 43 has a unique landmark of the target image 41 , and the expression landmark has a characteristic corresponding to the driver image 42 .

Therefore, it can be seen that it is important to properly separate the unique landmark and the expression landmark in one landmark for natural face reproduction.

The embodiments described above may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and can include both volatile and non-volatile media, removable and non-removable media.

In addition, computer-readable media may include computer storage media. Computer storage media may include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: server, landmark separation device
110: receiver
120: transformation matrix estimator
130: arithmetic unit

Claims (13)

receiving a face image of a first person and landmark information corresponding to the face image;
estimating a transformation matrix corresponding to the landmark information;
calculating an expression landmark of the first person by using the transformation matrix; and
calculating a unique landmark of the first person by using the expression landmark;
A landmark separation method comprising a. According to claim 1,
In the step of estimating the transformation matrix, a landmark separation method using a learning model trained to estimate a PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary face image. 3. The method of claim 2,
The learning model classifies a plurality of landmark information into a plurality of semantic groups, and outputs a PCA transform coefficient corresponding to each of the plurality of semantic groups. 4. The method of claim 3,
In the step of calculating the expression landmark, the landmark separation method of calculating the expression landmark of the first person by calculating the estimated transformation matrix and the PCA unit vector. According to claim 1,
and the landmark information is defined as the sum of the expression landmark, the unique landmark, and the average landmark of a sufficiently large number of people's faces. 6. The method of claim 5,
In the step of calculating the unique landmark, the landmark separation method for calculating the unique landmark of the first person by calculating the expression landmark and the average landmark from the landmark information. A computer-readable recording medium in which a program for performing the method according to any one of claims 1 to 6 is recorded. a receiving unit configured to receive a face image of a first person and landmark information corresponding to the face image;
a transformation matrix estimator for estimating a transformation matrix corresponding to the landmark information; and
a calculator for calculating an expression landmark of the first person by using the transformation matrix, and calculating a unique landmark of the first person by using the expression landmark;
A landmark separation device comprising a. 9. The method of claim 8,
The transformation matrix estimating unit landmark separation apparatus using a learning model trained to estimate a PCA transformation matrix from an arbitrary face image and landmark information corresponding to the arbitrary facial image. 10. The method of claim 9,
The learning model classifies the landmark information into a plurality of semantic groups and outputs a PCA transform coefficient corresponding to each of the plurality of semantic groups. 11. The method of claim 10,
The operation unit is configured to calculate the expression landmark of the first person by calculating the estimated transformation matrix and the PCA unit vector. 9. The method of claim 8,
The landmark information is a landmark separation device defined as the sum of the expression landmark, the unique landmark, and the average landmark of a sufficiently large number of people's faces. 13. The method of claim 12,
In the step of calculating the unique landmark of the first person, the landmark separation device for calculating the unique landmark of the first person by calculating the expression landmark and the average landmark from the landmark information.

Images