KR20050019599A - lip region segmentation and feature extraction method for Speech Recognition - Google Patents

Abstract

PURPOSE: A method for detecting a lip region for speech recognition and acquiring lip shape information using the lip region is provided to detect a face region and a lip position using a pattern classifier employing a face color and lip color distribution model learning result and a support vector machine. CONSTITUTION: When an exemplary face image is input and a lip region on the face image is designated(S501,S503), a color distribution histogram of the lower part of the face image is calculated and a color distribution histogram of the designated lip region is calculated(S504,S505). Lip color probability for each color is obtained using the two color distribution histograms(S507). The lip color probability constructs a lip color similarity table(S509). A pattern classifier for lip shapes is trained using various examples of lip shapes(S510).

Description

Lip region detection for speech recognition and lip shape information acquisition method using the same {lip region segmentation and feature extraction method for Speech Recognition}

The present invention relates to a method for acquiring lip shape information, and more particularly, to detect lip regions for voice recognition, extract features to obtain lip shape information, and provide useful information for voice recognition from the corresponding lip shape information. It is about a method.

In general, a voice recognizer targets only a voice in a voice, but recently, a technique for performing voice recognition by processing a sound and an image at the same time has been proposed in order to improve performance under noise. As described above, in order to recognize the voice from the mouth image, a technique for obtaining lip shape information is essential.

In order to acquire the lip shape information, the position of the lips is detected from the mouth image, and then the lip shape information is obtained for the detected lips.

The conventional method of detecting the lip position from the image is to estimate the lip color distribution, and to classify the points corresponding to the lip in the image based on the lip color distribution, and to find the lip by finding the gathered points. However, due to the characteristics of the image, the lip color variation due to lighting or human differences is large, and in some cases, the lip color is not very different from the skin color. .

Another conventional method for detecting a lip position to solve this problem is to create a geometric lip shape model and calculate the degree of registration with a given image to find an area of the image most similar to the pre-made lip shape model. However, in this method, it is not easy to find an optimal lip shape model, and it is difficult to guarantee accurate lip position detection performance because a lot of effort is required to adjust parameters for model deformation.

Conventional methods of obtaining lip shape information include an image-based approach and a shape information-based approach. The first image-based approach is to use a given image as it is for extracting the lip shape as it is, or to transform the coordinate system and compress only the main information of the image and use it as lip shape information. This method provides more information than the shape information-based approach. However, since the image itself is used, accurate lip shape information cannot be obtained when there is an error in lip position detection. The second shape information-based approach is to analyze the geometric shape of the lips and use it as a lip feature, and the lip shape parameters obtained during lip detection become lip features. However, this method has a disadvantage that it is difficult to use for recognition due to the small amount of information.

An object of the present invention devised to solve the above problems of the prior art is to detect a face region and a lip position using a face classifier and a lip color distribution model learning result and a pattern classifier using a support vector machine. It is to provide a method.

Another object of the present invention is to provide a method of obtaining lip shape information using a Principle Component Analysis.

Lip position learning method according to the present invention for achieving the above object, if the example face image is input and the lip region of the example face image is specified, the color distribution histogram of the lower half region of the example face image is calculated and the specified A histogram calculation step of calculating a color distribution histogram of the lip region;

A lip color probability calculation step of obtaining a lip color probability of each color by using the color distribution histogram of the lower half of the face and the color distribution histogram of the lip region;

A lip color similarity table generating step of constructing the color and the lip color probability of each color into a lip color similarity table;

It characterized by including a pattern classifier learning step of learning a pattern classifier for the lip shape by using the examples and counterexamples of the various lip shapes.

According to the present invention, there is provided a computer-readable recording medium having recorded thereon a program for executing the lip position learning method as described above.

In addition, the lip shape information acquisition method according to an embodiment of the present invention, the lip color similarity table storing the color and color lip color probability value for each color, and in the state that the pattern classifier learning the pattern of the lip shape is built,

A lip color similarity image acquisition step of obtaining a lip color similarity image by applying the input face image to the lip color similarity table when a face image is input;

An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image;

A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position;

And a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for the image of the lip position set in the lip position setting step.

In addition, the method of obtaining lip shape information according to another embodiment of the present invention includes a face color similarity table storing color and color face color probability values, a lip color similarity table storing color and color lip color probability values, and a lip shape. With the pattern classifier that has learned the pattern of,

A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input;

Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape;

Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image;

An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image;

A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position;

And a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for the image of the lip position set in the lip position setting step.

In addition, according to another embodiment of the present invention, a method of acquiring lip shape information includes: learning a face color probability value of a specific color with respect to an example image and constructing a face color probability value for each color and color into a face color similarity table; Wow;

A lip color distribution model learning step of learning a lip color probability value of a specific color for an example face image and constructing a lip color probability value for each color and color into a lip color similarity table;

A pattern classifier learning step of learning a lip pattern for an example lip image;

A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input;

Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape;

Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image;

An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image;

A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position;

And a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for the image of the lip position set in the lip position setting step.

According to the present invention, there is provided a computer-readable recording medium having recorded thereon a program for executing the method of obtaining the lip shape information as described above.

Hereinafter, with reference to the accompanying drawings will be described in more detail the lip position learning method and lip shape information acquisition method according to an embodiment of the present invention.

1 is a block diagram illustrating a lip shape information acquisition system to which the present invention is applied. Referring to FIG. 1, the lip shape information acquisition system for the present invention includes an image acquisition device 110, a display device 120, and a signal processing device 130.

The image acquisition device 110 generates an image signal by photographing a subject and provides the image signal to the signal processing device 130. The image acquisition device 110 corresponds to a charge coupled device (CCD) camera, a digital camera, or a video processing device.

The signal processing device 130 is a device for obtaining lip shape information by signal processing an image signal provided from the image acquisition device 110, the image frame acquisition unit 131, the face color distribution model learning unit 132, Lip color distribution model learning unit 133, storage unit 134, face color similarity image acquisition unit 135, face area detection unit 136, lip color similarity image acquisition unit 137, and lip position tracking The unit 138, the lip feature extracting unit 139, and a control unit 140 are provided.

The image frame acquisition unit 131 separates the video signal input from the image acquisition device 110 by one frame. The image frame acquisition unit 131 provides an image signal input in the learning mode to the face color distribution model learning unit 132 or the lip color distribution model learning unit 133, and acquires the image color input image input in the execution mode. Provided to section 135.

The face color distribution model learner 132 grasps the RGB color of each pixel of each frame of the input image signal, and converts the RGB color of each pixel into a HS / Haturation (Hue / Saturation) color value of the HSV (Hue-Saturation-Value) color space. The color similarity for each color is calculated based on the converted HS color values, and the face color probability (face color similarity) for the calculated color is implemented as a face color similarity table and stored in the storage unit 134. This conversion of RGB color to HS color of HSV color space is because this HSV color space classifies face color and lip color relatively better than other color spaces.

The lip color distribution model learning unit 133 calculates the lip color similarity for each color of the input face image, and implements the lip color probability (lip color similarity) for the calculated color as a lip color similarity table. ).

The storage unit 134 stores a face color similarity table and a lip color similarity table.

The face color similarity image acquisition unit 135 calculates face color similarity for all pixels of the current frame with reference to the image frame input from the image frame acquisition unit 131 and the face color similarity table stored in the storage unit 134.

The face area detection unit 136 composes a gray image based on the face color similarity of each pixel obtained by the face color similarity image acquisition unit 135, and limits the position and size of the face through binarization and elliptic approximation. Extract the area.

The lip color similarity image acquisition unit 137 calculates the lip color similarity for all pixels of the current face area by referring to the face area image input from the face area detection unit 136 and the lip color similarity table stored in the storage unit 134. do.

The lip position tracking unit 138 calculates the center of gravity of the lip color similarity image of each pixel obtained by the lip color similarity image acquisition unit 137 and tracks the lip position.

The lip feature extracting unit 139 extracts the lip feature using a component analysis method for the lip position tracked by the lip position tracking unit 138.

The controller 140 controls the overall operation of the apparatus, and the display apparatus 120 displays the results processed by each component of the signal processing apparatus 130 on the screen.

Hereinafter, a method of obtaining lip shape information according to an embodiment of the present invention will be described with reference to FIG. 2. According to the present invention, a face image and a lip region are learned through color distribution learning using a sample image in a learning mode, and a face region is extracted from an input image and a lip feature is extracted in a run mode.

When the user selects the learning mode (S201), it is checked whether the face color or the lip color learning (S202), and the face color distribution model learning unit 132 or the lip color distribution model learning unit 133 is a face color distribution model learning subroutine ( S203) or the lip color distribution model learning subroutine (S204). Meanwhile, when the user selects an execution mode other than the learning mode (S201), the face region extraction subroutine S205 and the lip feature extraction subroutine S206 are performed on the input image.

3 is a detailed operation flowchart of the face color distribution model learning subroutine of the face color distribution model learning unit 132 according to an embodiment of the present invention. In order to segment the face area using colors, knowledge of what color constitutes the face area is required. This knowledge can be approximated by experience, but learning face color through statistical observations is most accurate. Whether or not a specific color is a face color can be indicated by a face color probability (face similarity), and a statistical observation result can be used to obtain a face color probability (face similarity) of a specific color. 3 is a process in which the face color distribution model learning unit 132 obtains a face color probability (face color similarity) of each color.

Referring to FIG. 3, when an example image is input (S301), the RGB color of each pixel is grasped for the input example image frame and the RGB color is converted into an HS color (S302). In operation S303, the face area is designated by the user. When the face area is designated by the user (S303), the color distribution histogram F _B (C) of the entire image is calculated and cumulatively summed with the previously obtained F _B (C) (S304), and the color distribution histogram of the face area specified by the user obtained prior to calculating a _{F F (C) F F (} C) and the cumulative sum (S305). The above steps S301 to S305 are repeatedly processed for all the example images (S306).

Next, the face color probability P _F (C) is calculated for all color values by applying the following Equation 1 (S307). The face color probability information for each color value becomes a face color distribution model.

In the case of the arbitrary color c1, the number of pixels F _B (c1) having the color c1 among the pixels constituting the entire image, and the number of pixels having the color c1 among the pixels constituting the face region of the frame F _F (c1) Is applied to Equation 1, the probability P _F (c1) that the color c1 is the face color is obtained. Here, F _B (c1) is called the face observation frequency of the color c1, and F _F (c1) is called the face classification frequency of the color c1.

Next, the face color model obtained in step S307 is Gaussian Convolution to generalize the face color distribution model (S308). To explain the reason for Gaussian convolution, the face color distribution model obtained in step S307 is discrete because there is a limitation in the observation experience. That is, the face color distribution model is discretely obtained because it is not observed or is observed but biased. Thus, by linearly converting the discrete face color distribution model by Gaussian convolution as shown in FIG. 4, the effect as if observed for all colors is obtained.

Next, the HS value is indexed for the generalized face color distribution model, and the color value and the face color probability (face color similarity) are matched and stored in the face color similarity table (S309).

The face color similarity table accumulates and retains color similarity information for the face color by a learning operation, which gradually learns which color is the face color. Therefore, as the training data accumulates in the face color similarity table, the color reflecting the illumination at the time of shooting or the color reflecting the different skin color is learned for each person. Therefore, the face color similarity table does not affect the lighting or skin color. It can be used as a useful source of information.

5 is a detailed operation flowchart of the lip color distribution model learning subroutine of the lip color distribution model learning unit 133 according to an embodiment of the present invention. The subroutine learning the lip color distribution model is similar to the face color distribution model learning subroutine. If there is a difference, in the face color distribution model training subroutine, the observation frequency means the number of pixels having any color among the pixels constituting the entire image, and the classification frequency is the number of pixels having any color among the pixels constituting the face area. In the lip color distribution model learning subroutine, the number of observations means the number of pixels having any color among the pixels constituting the lower half of the face, and the classification frequency has any color among the pixels constituting the lip region. It means the number of pixels.

A lip color distribution model learning subroutine will be described with reference to FIG. 5. When the example face image is input (S501), the RGB color of each pixel is identified for the input example face image frame and the RGB color is converted into the HS color (S502). Then, the user waits for the lip region to be designated by the user (S503). If a lip region is specified from the user (S503), the color distribution histogram F _lf (C) of the lower half of the face is calculated and cumulatively summed with the previously obtained F _lf (C) (S504), and the color distribution of the lip region designated by the user is calculated. The histogram F _lip (C) is calculated and cumulatively summed with the previously obtained _lip (C) (S505). Steps S501 to S505 are repeated for all the example face images (S506).

Next, a calculation by applying a lip color _lip probability P (C) for all the color values in the equation (2) below (S507). Lip color probability information for each color value becomes the lip color distribution model.

For an arbitrary color c1, the number of pixels F _lf (c1) having the color c1 among the pixels constituting the lower half of the face and the number of pixels F _lip (c1) having the color c1 among the pixels constituting the lip region Applying to 2, the probability P _lip (c1) is obtained that the color c1 is the lip color. Here, F _lf (c1) is referred to as the lip observation frequency of the color c1, and F _lip (c1) is referred to as the lip classification frequency of the color c1.

Next, the lip color model obtained in step S507 is Gaussian Convolution to generalize the lip color distribution model (S508). Here, the reason for Gaussian convolution is the same as the reason for generalizing the face color distribution model described above.

Next, the HS values are indexed for the generalized lip color distribution model, and color values and lip color probabilities (lip color similarity) are matched and stored in the lip color similarity table (S509).

Next, the SVM (Support Vector Machine) is learned using the lip shape example and inverse (S510). In this case, the kernel used for the SVM is a Gaussian kernel, which is represented by Equation 3 below. The learned SVM pattern classifier is used later to fine tune the lip position.

After learning about the face color and the lip color as described above, when the user executes the execution mode, the face region extraction subroutine and the lip feature extraction subroutine are performed. FIG. 6 illustrates a face region according to an embodiment of the present invention. An operational flow diagram illustrating an extraction subroutine.

In the present invention, a color similarity is obtained by applying a face color similarity table to each pixel of the image with respect to the input image, and a gray image is constructed as shown in FIG. 7 from the obtained color similarity values. Then, the position of the face is identified through this image, and the position and size of the face are obtained through binarization and elliptic approximation. This facial region extraction process will be described in more detail.

When an image is input (S601), the RGB color of the input image is converted into an HSV color (S602). Then, color similarity is obtained by applying the color similarity table to each pixel of the input image to obtain a face color similarity image (S603). Then, the obtained face color similarity image is binarized (S604), and the binarized face color similarity image is filtered into an ellipse shape to segment the face region (S605).

Face regions can be approximated by ellipses. Accordingly, the face region similarity image, which is a gray image, is divided through binarization, and the face region is detected by ellipsoid detection. At this time, since the binarized image contains a plurality of miscellaneous images, the face region is extracted by approximating the face region with an ellipse through the center of gravity and the moment calculation. The equation for extracting the elliptical region of the face through the moment calculation is as follows.

First, assuming that the secondary moment in the y direction of the image is yy, the secondary mixed moment is xy, and the secondary moment in the x direction is xx, the value is calculated as in Equation 4 below.

In the above formula, A means the number of pixels of the face area. When the equation representing the area of the ellipse is Equation 5, the coefficients d, e, and f constituting Equation 5 are calculated as in Equation 6.

After obtaining the equation of the ellipse, the point inside the ellipse is used as the face region, and the lip position detection and feature extraction are performed. To this end, the HSV image of the divided face region is extracted and provided as a lip feature extraction subroutine. The face region approximated by an ellipse is shown in FIG. 8.

A process of searching for a position of a lip and extracting a feature with respect to a face region obtained by the previous face region extraction subroutine is performed. to be.

Since the face detected by the face region extraction subroutine has a size difference according to the distance, size normalization is performed to offset the face (S901). Normalization is performed based on the width of the face, allowing the ratio of the original face width-length to remain constant.

Next, a lip color similarity image of the HSV image of the face region is obtained (S902). At this time, the lip color similarity image is obtained for the entire face region, and the lip color similarity image is obtained using the lip color similarity table for the lower half of the face only. The center of gravity of the obtained lip color similarity image is calculated and estimated as the initial position of the lip (S903). When the lip similarity value of each point of the similarity image is I (x, y), the initial position of the lip (Cx, Cy) is calculated as in Equation 7 below.

After that, the position of the lips is finely searched for a range from the initial position (Cx, Cy) of the previously found lips to x and y. The image is acquired by moving the matching center within a specified range, and the acquired image is input to the SVM pattern classifier to determine whether the image is a lip or not. That is, while all the points of the search range are moved, the lip matching degree value which is the output of the SVM pattern classifier is calculated (S904), and the position having the largest matching degree is set as the lip position (S905).

A process of extracting the lip position from the face image as shown above is shown in FIG. 10. The leftmost picture of FIG. 10 is a lip color similarity image, and the picture next to it is a position of the lip obtained by calculating the center of gravity of the similarity image. Also, the figure next to it is the SVM lip registration value calculated through a range of searches from the initial lip position, and finally the rightmost figure shows the position of the lip adjusted by the SVM lip registration value.

Next, a lip image is extracted from the lip position set in step S905 (S906), and a basis for expressing the lip image using the PCA is extracted from the extracted lip image. Using the extracted basis, the base and PCA coefficients c ₁ , c ₂ ,... , c _n can be obtained (S907), and the PCA coefficients (c ₁ , c ₂ ,..., c _n ) are output as lip features (S908). Principal Component Analysis (PCA) is a method of finding a component that constitutes a specific class of signals (eg, facial and lip images). How to analyze. This component is called a basis for convenience. A detailed description of this PCA is described in the book of Neural Networks (SimonHaykin, “Neural Networks”, Prenctice Hall).

When the input image is a moving picture consisting of M frames, M N-dimensional vectors are calculated and provided as lip features.

However, since the extracted features are different from one speaker to another, it is preferable to obtain an average vector of M images and then express the lip features by the difference therefrom. Therefore, the lip feature c 'is finally calculated as shown in Equation 8 below.

Although the technical spirit of the present invention has been described above with reference to the accompanying drawings, it is intended to exemplarily describe the best embodiment of the present invention, but not to limit the present invention. In addition, it is obvious that any person skilled in the art may make various modifications and imitations without departing from the scope of the technical idea of the present invention.

As described above, according to the present invention, since the lip position can be detected quickly and accurately, there is an effect that can be applied to character recognition using a mouth shape.

1 is a block diagram showing a lip shape information acquisition system to which the present invention is applied;

2 is an operation flowchart illustrating a method of obtaining lip shape information according to an embodiment of the present invention;

3 is a detailed operation flowchart of a face color distribution model learning subroutine of the face color distribution model learning unit according to one embodiment of the present invention;

4 is a diagram illustrating a process of linearly converting a discrete face color distribution model by Gaussian convolution;

5 is a detailed operation flowchart of a lip color distribution model learning subroutine of a lip color distribution model learning unit according to an embodiment of the present invention;

6 is an operation flowchart showing a face region extraction subroutine according to an embodiment of the present invention;

7 is a diagram illustrating a process of forming a face color similarity and a gray image by applying a face color similarity table to an input image;

8 is a view showing an approximate face region by an ellipse;

9 is an operation flowchart showing a lip feature extraction subroutine according to an embodiment of the present invention;

10 is a view illustrating a process of extracting a lip position from a face image;

11 is a diagram illustrating an image representation process using PCA coefficients.

<Brief description of symbols for the main parts of the drawings>

110; Image acquisition apparatus 120; Display device

130; Signal processing apparatus 131; Video frame acquisition unit

132; Face color distribution model learning unit 133; Lip Color Distribution Model

134; Storage 135; Face color similarity image acquisition unit

136; Face area detection unit 137; Lip Color Similarity Image Acquisition

138; Lip position tracking unit 139; Lip feature extraction

140; Control

Claims (23)

A histogram calculation step of calculating a color distribution histogram of the lower half region of the example face image and calculating a color distribution histogram of the designated lip region when an example face image is input and a lip region of the example face image is designated; A lip color probability calculation step of obtaining a lip color probability of each color by using the color distribution histogram of the lower half of the face and the color distribution histogram of the lip region; A lip color similarity table generating step of constructing the color and the lip color probability of each color into a lip color similarity table; Lip position learning method characterized in that it comprises a pattern classifier learning step of learning a pattern classifier for the lip shape using a variety of examples and counterexamples. The method of claim 1, wherein the histogram calculation step is performed after converting the example face image from the RGB color space to the HSV color space. The method of claim 1, wherein the lip color similarity table is constructed after generalizing the lip color distribution model by Gaussian convolving the lip color probability of each color obtained in the lip color probability calculation step. On your computer, A histogram calculation step of calculating a color distribution histogram of the lower half region of the example face image and calculating a color distribution histogram of the designated lip region when an example face image is input and a lip region of the example face image is designated; A lip color probability calculation step of obtaining a lip color probability of each color by using the color distribution histogram of the lower half of the face and the color distribution histogram of the lip region; A lip color similarity table generating step of constructing the color and the lip color probability value for each color into a lip color similarity table; A computer-readable recording medium having recorded thereon a program for executing a lip position learning method including a pattern classifier learning step of learning a pattern classifier for a lip shape using the various lip examples and counterexamples. With the lip color similarity table that stores the color and the lip color probability values for each color, and the pattern classifier learning the pattern of the lip shape, A lip color similarity image acquisition step of obtaining a lip color similarity image by applying the input face image to the lip color similarity table when a face image is input; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; And a lip feature extraction step of extracting a lip feature by extracting a principal component analysis (PCA) coefficient with respect to the image of the lip position set in the lip position setting step. With the face color similarity table that stores the color and face color probability values by color, the lip color similarity table that stores the color and lip color probability values by color, and the pattern classifier that has learned the shape of the lips, A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input; Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape; Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; And a lip feature extraction step of extracting a lip feature by extracting a principal component analysis (PCA) coefficient with respect to the image of the lip position set in the lip position setting step. A face color distribution model learning step of learning a face color probability value of a specific color for an example image and constructing a face color probability value for each color and color into a face color similarity table; A lip color distribution model learning step of learning a lip color probability value of a specific color for an example face image and constructing a lip color probability value for each color and color into a lip color similarity table; A pattern classifier learning step of learning a lip pattern for an example lip image; A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input; Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape; Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; And a lip feature extraction step of extracting a lip feature by extracting a principal component analysis (PCA) coefficient with respect to the image of the lip position set in the lip position setting step. The method according to any one of claims 5 to 7, wherein the lip color similarity image acquisition step is performed after standardizing the size of the input face image. The method of claim 5, wherein the lip color similarity image acquisition step is performed after converting the input face image from the RGB color space to the HSV color space. The method of claim 6 or 7, wherein the face color similarity image acquisition step is performed after converting the input image from the RGB color space to the HSV color space. The lip shape according to any one of claims 5 to 7, wherein the lip color similarity image acquisition step comprises applying a lip color similarity image to the face color similarity table with respect to the lower half region of the input face image. How to get information. The method of claim 5, wherein the initial stage lip estimation step obtains the initial position of the lips (Cx, Cy) by the following equation. [Equation] Here, I (x, y) is each pixel value of the lip color similarity image. The method according to any one of claims 5 to 7, wherein the lip color similarity image acquisition step to the lip feature extraction step is repeated for a plurality of consecutive lip images to extract the lip features for each lip image And displaying a feature vector, and obtaining a mean vector for the plurality of lip images, and then setting a difference between the feature vector and the mean vector as a lip feature. How to get information. 8. The method of claim 6 or 7, wherein the step of dividing the face region extracts an elliptic region of the face by applying the following formula to the binarized image of the face color similarity image. [Equation] here, ego, Is, yy is the secondary moment in the y direction of the image, xy is the secondary mixed moment, xx is the secondary moment in the x direction of the image, and A is the number of pixels in the face region. The method of claim 7, wherein the face color distribution model learning step, A histogram calculation step of calculating a color distribution histogram of the entire region of the example image and calculating a color distribution histogram of the specified face region when the face region of the example image is designated; A face color probability calculation step of obtaining a face color probability of each color by using a color distribution histogram of the entire image region and a color distribution histogram of the face region; And a face color similarity table generating step of constructing the color and the face color probability of each color into the face color similarity table. The method of claim 15, wherein the histogram calculation step is performed after converting the example image from the RGB color space to the HSV color space. 16. The method of claim 15, wherein the face color similarity table is constructed after generalizing a face color distribution model by Gaussian convolving the face color probability of each color obtained in the face color probability calculation step. The method of claim 7, wherein the lip color distribution model learning step, A histogram calculation step of calculating a color distribution histogram of the lower half region of the example face image and calculating a color distribution histogram of the designated lip region when the lip region of the example face image is designated; A lip color probability calculation step of obtaining a lip color probability of each color by using the color distribution histogram of the lower half of the face and the color distribution histogram of the lip region; And a lip color similarity table generating step of constructing the color and the lip color probability for each color as a lip color similarity table. 19. The method of claim 18, wherein the histogram calculation step is performed after converting the example face image from the RGB color space to the HSV color space. 19. The method of claim 18, wherein the lip color similarity table is constructed after generalizing the lip color distribution model by Gaussian convolving the lip color probability of each color obtained in the lip color probability calculation step. . On a computer with a lip color similarity table that stores color and color lip color probability values, and a pattern classifier that learns lip patterns, A lip color similarity image acquisition step of obtaining a lip color similarity image by applying the input face image to the lip color similarity table when a face image is input; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; A computer program recording a program for executing a lip shape information acquisition method including a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for an image of a lip position set in the lip position setting step Recordable media. On a computer with a face color similarity table that stores color and face color probability values by color, a lip color similarity table that stores color and color lip color probability values, and a pattern classifier that learns lip patterns, A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input; Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape; Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; A computer program recording a program for executing a lip shape information acquisition method including a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for an image of a lip position set in the lip position setting step Recordable media. On your computer, A face color distribution model learning step of learning a face color probability value of a specific color for an example image and constructing a face color probability value for each color and color into a face color similarity table; A lip color distribution model learning step of learning a lip color probability value of a specific color for an example face image and constructing a lip color probability value for each color and color into a lip color similarity table; A pattern classifier learning step of learning a lip pattern for an example lip image; A face color similarity image acquisition step of obtaining a face color similarity image by applying the input image to the face color similarity table when an image is input; Dividing the face region by binarizing the face color similarity image and filtering the ellipse into an ellipse shape; Obtaining a lip color similarity image by applying the face region image of the input image to the lip color similarity table to obtain a lip color similarity image; An initial lip position estimation step of estimating the initial position of the lip by calculating the center of gravity in the lip color similarity image; A lip position setting step of calculating a degree of registration in the pattern classifier for a predetermined range around the initial position of the lip, and setting the position of highest lip as the lip position; A computer program recording a program for executing a lip shape information acquisition method including a lip feature extraction step of extracting lip features by extracting Principal Component Analysis (PCA) coefficients for an image of a lip position set in the lip position setting step Recordable media.