KR100368743B1 - Method and Device for Face Compression and Reconstruction Using a Small Set of Feature Points - Google Patents

Abstract

The present invention relates to a method and apparatus for compressing and restoring a face image for storing and transmitting an input face image using a small number of feature points and for restoring an intact face image from a damaged input image.

According to the present invention, the feature image extracting unit 51 extracts feature points from the face image input unit 50, the eye region, the nose region, the mouth region, and the outer region of the face image received by A / D conversion of the face image through a camera or a scanner. A feature point information extracting unit 52 for extracting feature information necessary for reconstruction from the extracted feature points, a specific point information encoder 53 for compressing and encoding the extracted feature information into a form suitable for transmission or storage, and encoded Transmitter 54 for transmitting the feature information through the transmitting end, a storage unit 55 for storing the encoded feature information, a receiver 56 for receiving the feature point information from the transmitter 54, the storage unit 55 A loading unit 57 for loading the feature point information from the feature point, a feature point information decoding unit 58 for decoding the feature point identification code, the color / contrast information, the position information, etc. from the encoded feature point information, A face model database unit 59 configured based on the information, a full face image restoring unit 60 for restoring the color / contrast information and the shape of the entire face using the feature point information and the face model information; The face image output unit 61 outputs to a printer and a storage device.

Description

{Method and Device for Face Compression and Reconstruction Using a Small Set of Feature Points}

The present invention relates to a method and apparatus for compressing and restoring a face image for storing and transmitting an input face image using a small number of feature points and for restoring an intact face image from a damaged input image.

The conventional still face image compression technology has a limitation in increasing the compression rate because it compresses the entire face image, and even in the latest video face compression system such as MPEG4, the shape of the whole face image is used to transmit a realistic face. Since color and contrast information must be transmitted at least once or periodically, there was a problem in that the amount of information suddenly increased when transmitting the entire face image.

In addition, a conventional reconstruction method using feature points requires more than 150 feature points to restore the shape of the face, and color / contrast information of the entire face image is required to restore the color / contrast.

In the conventional method, when the input face image is partially damaged by miscellaneous images, overlap by other objects, shadows by lighting, and the like, there is a drawback that the damaged part cannot be restored.

The present invention is to solve such a conventional problem, an object of the present invention is to compress the entire face image using only a small number of feature point information of about 40 or less or to restore the entire face from some damaged face images The present invention provides a method and apparatus for compressing a face image using a small number of feature points and restoring a damaged face image.

1 is a block diagram showing a configuration according to the present invention

2 is an operation flowchart showing a feature point information extraction process according to the present invention.

3 is an operation flowchart showing a face image restoration process according to the present invention;

4 is an operational flow diagram showing the characteristic parts of the present invention.

5 illustrates facial feature points of the invention.

6 is a diagram illustrating an example of compressing a face image and restoring a damaged face image according to the present invention.

<Explanation of symbols for main parts of drawing>

50: face image input unit 51: feature point extraction unit

52: feature point information extracting unit 53: feature point information encoding unit

54: transmitter 55: storage

56: receiving unit 57: loading unit

58: feature point information decoding unit 59: face model database unit

60: full face image restoration unit 61: face image output unit

The present invention for achieving the above object is characterized by consisting of a feature point information extraction process and a face image restoration process using the feature point information.

In the method of the present invention, a first step of photographing or scanning using a camera or a scanner by selecting an input device, a second step of extracting feature points from a face image input through the first step, and the input image A third step of extracting feature point information such as location information and color / contrast information together with the identification code of each feature point if the feature point extraction is possible; a fourth step of encoding and transmitting or storing the feature point extraction for each region of the face A fifth step of receiving or loading the feature point information through a communication network, a sixth step of decoding the feature point information upon completion of receiving and loading the feature point information, and determining the location information and the color / contrast information, Restoration of shape information of entire face using location information and face shape model information, and feature point in case of color / contrast information The seventh step of restoring color / contrast information of the entire face using color / contrast information and the face color / contrast model information, and the eighth step of restoring and outputting the entire face image by synthesizing the shape and color / contrast information. It is characterized by.

Accordingly, in the full face image restoration step using the feature point information and the face model information of the present invention, the entire face may be restored at a time using the full face model as the face model information, and each partial region, eye region, nose region, Each model can be restored by using model information about the mouth area and the outside area of the face independently, and then the entire face can be combined with each area.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of the present invention, the feature image is extracted from the face image input unit 50 receiving the A / D conversion of the face image through a camera or a scanner, and the eye region, the nose region, the mouth region and the outer region of the face. A feature point extracting unit 51, a feature point information extracting unit 52 for extracting feature information necessary for reconstruction of the extracted feature point, and feature point information for compressing and encoding the extracted feature information into a form suitable for transmission or storage. The encoder 53, a transmitter 54 for transmitting the encoded feature information through the transmitting end, a storage unit 55 for storing the encoded feature information, and feature point information from the transmitter 54 are received. A receiver 56, a loading unit 57 for loading feature point information from the storage unit 55, and a feature point information decoding unit for decoding feature point identification codes, color / contrast information, position information, etc. from the encoded feature point information. (58) The face model database unit 59 is configured based on information obtained from several face images, and the full face image restoration unit 60 restores color / contrast information and shape of the entire face using the feature point information and the face model information. And a face image output unit 61 outputting the restored face image to a display, a printer, and a storage device.

Referring to the operation of the present invention configured as described above is as follows.

2 illustrates an embodiment of a feature point information extraction process in a method of compressing a face image using face feature point information and a method of restoring a damaged face image.

First, a desired input device is selected from a camera and a scanner (S1), and photographing and scanning are completed by using a camera or a scanner and converted into digital signals (S2-S5).

Next, feature points are extracted from the input face image (S6), and if feature extraction is possible by extracting feature point information from the input image, feature point information such as feature point identification code, brightness code, and location information code is extracted (S7). , S8)

If the feature points to be extracted remain, the process returns to the feature point extraction step of step S6 and repeats the steps S6-S9 until it is determined whether the feature points can be extracted for all the feature points.

When the feature point extraction is completed, the feature point information is encoded and the feature point is transmitted to determine whether to transmit or store the feature point, or the feature point is stored (S9-S13).

In the present invention, since only the feature points are transmitted or stored, the entire face image can be represented with a much smaller amount of information than the entire face image, and the feature points cannot be extracted accurately due to miscellaneous operations, overlapping by other objects, shadows by lighting, etc. As for the feature point extraction determination step of S7, the feature point extraction is not performed.

In this case, in this case, the feature information is extracted even in the region where the feature point extraction is impossible, and the wrong feature information is extracted to cause severe deformation of the reconstructed image or the face image reconstruction in the region is not performed.

However, in the present invention, the entire face image may be efficiently restored using only extracted feature point information.

3 illustrates an embodiment of a face image restoration process using feature point information in a method of compressing a face image using face feature point information and a method of restoring a damaged face image.

That is, the feature point information is received via the communication network or the feature point information is loaded from the storage device (S14 and S15), respectively, and the reception and loading of the feature point information are respectively confirmed (S16 and S17), and the feature point information receiving step or From the encoded feature point information obtained through feature point loading, feature point information such as a feature point identification code, position information, and color / contrast information is decoded (S18).

Each feature point information is obtained using a feature point identification code, and the position information and color / contrast information are separated (S19), and the shape information for the entire face is restored using the separated position information and the face shape model information. Meanwhile, using the separated color / contrast information and face color / contrast model information, color / contrast information of the entire face is restored (S20 and S21).

Next, the entire face image is restored by synthesizing the restored form information and the color / contrast information (S22), and selecting the output device suitable for the application to display the restored face image or to store it in a storage device or the like ( S23-S26).

4 illustrates a specific example of a method of restoring an entire face image using separated feature point location information and color / contrast information.

First, each feature point information is obtained using the feature point identification code, and the location information and the color / contrast information are separated (S30).

In the case of position information, a pseudoinverse is obtained by converting feature point position information into a matrix form, and coefficients for each shape eigenvector are calculated using the pseudo inverse matrix and shape eigenvectors, which are face model information stored in a database. Shape information is restored by linear superposition of each eigenvector using the obtained coefficients of each eigenvector (S31-S33).

In addition, in the restoration of color / contrast information, the separated color / contrast information is converted into a matrix to obtain a pseudoinverse, and each color using the pseudoinverse matrix and color / contrast eigenvectors, which are face model information stored in a database, is obtained. Calculate the coefficients for the eigenvectors, and reconstruct the color / contrast information by linear superposition of each eigenvector using the coefficients of the eigenvectors obtained (S34-S36).

Next, the shape information and color / contrast information restored after the steps S33 and S36 are synthesized through a warping process to restore the entire face image (S37).

Hereinafter, a method of restoring the entire face image using the separated feature point position information and color / contrast information described with reference to FIG. 3 will be described in detail.

This process finds an optimized approximation of the deformation required by each face, inputting only the location and color / contrast information at separate feature points, on the premise of modeling the face as a linear superposition of shape and color / contrast prototypes. The process can be summarized.

First, the process of modeling the face as a linear superposition of shape and color / contrast prototypes is as follows.

Assuming that the correspondence of the reference faces of all face images is already given, the shape information of the face is encoded into a deformation field of the reference face, and the color / contrast information is encoded into the color / contrast information of the mapped image. do.

That is, the shape information S (x) is a position difference between the point x on the reference image and the corresponding point of the input image, and the color / contrast information T (x) is defined as the color or contrast value of the corresponding point with respect to the point x on the image. .

Shape average with independently obtained shape and color / contrast information , Color / contrast average Form and color / contrast differences Wow Find the covariances C_S and C_T of.

Based on this, principal component analysis is used to obtain a face model based on the covariance shape eigenvectors s_i and color / contrast eigenvectors t_i for m faces.

here , IN R ^ m-1, the coefficient The probability of is defined by Equation 2.

Where sigma_i ^ 2 is the eigenvalue of the shape covariance C_S, and the probability of the coefficient for color / contrast in the same way Can be obtained.

The following is the process of finding the optimal approximation of the deformation required by each face using only the positional information and the color / contrast information at the feature points.

Since only the positional information and the color / contrast information of the feature point are given in the encoding process, only an approximation of the required deformation can be estimated in this condition.

Thus, the present invention seeks to find the optimal solution under these under-determined conditions.

First, the sum of normalized coefficients is defined as an energy function, and a condition is set that feature information at a given feature point must be completely recoverable.

Here, the energy function E (alpha) represents the degree of deformation from the average face. The problem to be solved here is to find the alpha that minimizes the energy function E (alpha) among the coefficients satisfying the condition. In other words,

Energy function E (alpha):

- Condition:

here, Are extracted feature points, and n is smaller than (m-1) because fewer feature points are extracted than the number of bases.

To make this simpler, the problem given by Equation 6 is simplified to a problem that can be solved by the least squares method.

Equation 5 may be equally represented by Equation 6.

In order to use orthogonality, Equation 6 is expressed as follows.

here,

,

,

ego, It is assumed that the row vectors of are linearly independent. At this time May be expressed as in Equation (9).

here, Is a matrix It is the pseudo inverse of, and can be easily obtained by the following singular value decomposition (SVD). This is performed in the coefficient calculation step S32 for the eigenbeck of FIG.

Assuming that SVD of Is

to be.

The column vector of U is Is the eigenvector of, and the column vector of V is Eigenvectors of.

The main diagonal component of W is filled with nonzero values of the eigenvalues of the two matrices, and W ^ + consists of the inverse of the nonzero components of W.

This is done in step S31 of FIG.

Finally, to obtain the shape information of the entire image, the following equations obtained from Equations 1 and 8 are used.

This is performed in the shape information reconstruction step S33 by linear superposition of the eigenvectors of FIG. 4.

By applying the same method to the color / contrast information, it is possible to recover the color / contrast information for the entire face from a small number of feature points (performed in steps S34, S35, and S36 of FIG. 4).

5 shows an example of feature points required for face image compression and restoration of a damaged face image.

35 black circles are examples of feature points needed to compress and restore face shape and color / contrast information, and four overlapping circles are examples of feature points used to compress and restore color / contrast information of facial skin color. to be.

The facial feature point information extraction apparatus shown in FIG. 1 includes a face image input unit 50, a feature point extractor 51, a feature point information extractor 52, a feature point information encoder 53, a transmitter 54, and a storage unit 55. It consists of.

The face image input unit 50 receives a face image from a camera or a scanner, converts the face image into a digital form, and extracts it from the eye region, the nose region, the mouth region, and the outer region of the face image input through the feature point extractor 51. Extract feature points deemed possible.

At this time, the feature point information extracting unit 52 assigns an identification code to the extracted feature point, extracts color / contrast information and position information of the feature point necessary for image compression and reconstruction, and the feature point information encoding unit 53 obtains the obtained feature point. The information is encoded in a form suitable for transmission and storage, and the transmission unit 54 and the storage unit 55 transmit or store the encoded feature point information, respectively.

In addition, the face image restoration apparatus includes a receiver 56, a loading unit 57, a feature point information decoding unit 58, a face model database unit 59, a full face image restoration unit 60, and a face image output unit 61. It is composed of

The receiver 56 receives the input electrical signal, converts it into encoded feature point information, or reads the information stored in the storage unit 55 in various forms such as a hard disk, a CD-ROM, a diskette, etc. from the loading unit 57. And the encoded feature information before being stored in the storage device.

The encoded feature information is decoded by the feature point information decoder 58 to unique feature codes of each feature point, and feature point information such as color / contrast information and position information of each feature point, and the entire face image reconstruction unit 60 is decoded. The entire face is restored based on the feature point information and the face model information stored in the face model database unit 59.

At this time, even if there are feature points that cannot be extracted by overlapping, shadowing, blurring, etc. in the feature point extraction process, the feature point information such as the decoded feature point identification code, color / contrast information, and location information, and the face of the face model database unit 59 The entire face image can be effectively restored based on the model.

6 briefly illustrates an example of facial image compression using facial feature point information and restoration of a damaged facial image.

That is, it is possible to determine whether feature points can be extracted from the input images (1), (2), and (3), and the position and color / contrast characteristic information as shown in (4), (5), and (6). Extract and transmit it, or store it, and then restore the original full face image from (7).

In the present invention, a method of compressing a face image using face feature point information and restoring a damaged face image is performed even if a part of the input image is damaged, such as (2) or (3). Only the original face information can be restored as shown in (7).

As described above, the present invention uses a small number of feature point information to efficiently compress the entire face image or to partially lose face information due to noise, overlapping by another object, shadow by lighting, etc. It has the effect of restoring the entire face.

Claims (3)

In the face image compression and face image encoding and decoding method using the facial feature point information, First step of selecting an input device to shoot or scan using a camera or scanner, A second step of extracting feature points from the face image inputted through the first step, A third step of extracting feature point information such as location information and color / contrast information together with an identification code of each feature point when the feature point extraction is possible from the input image; A fourth step of encoding and transmitting or storing the feature points for each region of the face when the extraction of the feature points is completed; A fifth step of receiving or loading the feature point information through a communication network, a sixth step of decoding the feature point information upon reception and loading of the feature point information, The location information is determined by determining the location information and the color / contrast information, and the shape information of the entire face is restored using the location information of the feature point and the face shape model information.In the case of the color / contrast information, the color / contrast information and the face color of the feature point The seventh step of restoring the color / contrast information of the entire face using the / contrast model information; 8. A method of compressing a face image using a small number of feature points and restoring a damaged face image, comprising: an eighth step of reconstructing and outputting an entire face image by synthesizing shape and color / contrast information. The method of claim 1, wherein when restoring an entire face using separated color / contrast information and face color / contrast model information, A first step of obtaining location information and color / contrast information by using the feature point identification code; A second step of obtaining a pseudoinverse by converting feature point position information into a matrix form; A third step of calculating coefficients for each shape eigenvector by using a pseudo-inverse matrix and shape eigenvectors stored in a database; A fourth step of restoring the shape information by linear superposition of each eigenvector using the obtained coefficients of each eigenvector, A fifth step of obtaining a pseudoinverse by converting the separated color / contrast information into a matrix form; A sixth step of calculating coefficients for each color / contrast eigenvector by using a pseudo-inverse matrix and color / contrast eigenvectors, which are face model information stored in a database, A seventh step of restoring the color / contrast information by linear superposition of each eigenvector using the obtained coefficients of each eigenvector, Characterized in that it is performed by the seventh step of reconstructing the entire face image by combining the reconstructed shape information and color / contrast information through a warping process A method of compressing a face image using a small number of feature points and restoring a damaged face image. An apparatus for compressing a face image using face feature point information and encoding and decoding a damaged face image, A face image input unit for receiving a face image from a camera or a scanner and converting the face image into a digital form; A feature point extracting unit for extracting feature points determined to be extracted from an eye region, a nose region, a mouth region, and an outer region of the face from the input face image; A feature point information extraction unit for assigning an identification code to the extracted feature points and extracting color / contrast information and position information of the feature points necessary for image compression and reconstruction; A feature point information encoder for encoding the obtained feature point information into a form suitable for transmission and storage; A transmitter and a storage unit for transmitting or storing encoded feature point information; A receiver which receives an input electrical signal and converts the encoded electrical signal into encoded feature point information; A loading unit for reading information stored in various types of storage devices such as hard disks, CD-ROMs, diskettes, etc., and converting the information into encoded feature point information before being stored in the storage devices; A feature point information decoder which decodes the encoded feature information into feature point information such as a unique identification code of each feature point, color / contrast information, and position information of each feature point; Database part storing face model, and ` Facial image compression using a small number of feature points and restoring a damaged face image, comprising a full face image restoration unit for restoring an entire face using the feature point information decoder and the face model information stored in the database unit. Device.