KR101326691B1 - Robust face recognition method through statistical learning of local features - Google Patents

Abstract

A robust face recognition method is disclosed through statistical learning of local features. The present invention disclosed above comprises the steps of: dividing a face image to be learned into M images, and obtaining a local feature descriptor of each segmented image through SIFT feature extraction; (B) calculating an average and a variance for the plurality of local feature descriptors obtained in step (a); Dividing the face images to be compared into M images, and obtaining local feature descriptors of each of the divided images through SIFT feature extraction; (D) calculating a distance between the plurality of local feature descriptors obtained in step (a) and the plurality of local feature descriptors obtained in step (c); (E) calculating each weight for M images of the face image to be learned using the average and variance calculated in step (b); And combining the distances between the plurality of local feature descriptors calculated in step (d) and the respective weights calculated in step (e), for each image divided into M pieces between the face image to be learned and the face image to be compared. (F) calculating a distance; characterized in that it comprises a.

Description

Robust Face Recognition Method through Statistical Study of Regional Features {ROBUST FACE RECOGNITION METHOD THROUGH STATISTICAL LEARNING OF LOCAL FEATURES}

The present invention relates to a face recognition method, and more particularly, to a face recognition method capable of robust face recognition through extraction of local feature descriptors for face images and statistical learning thereof.

In general, various types of signals can be captured by humans. Among them, face images have various changes in some cases, so face recognition technology is one of the most popular areas in pattern recognition and machine learning.

Conventional face recognition technology extracts a feature of a single image from an entire image or extracts only features of a specific part of the face image such as eyes, nose, and mouth, and reflects the extracted data in a recognition rate result. have.

As described above, the method of extracting features can be classified by various criteria, and representative examples include a global feature extraction method and a local feature extraction method according to a range of a region from which the feature is extracted.

The global feature extraction method uses statistical values, such as covariance, across the training data. It is common to apply eigenvalues, eigenvectors, etc. to satisfy specific objective functions. Therefore, features acquired from arbitrary learning data (face images) are expressed to enable a general description of the corresponding learning data. Representative methods include Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA).

The local feature extraction method extracts local features that can explain this well from arbitrary learning data (face images), and is extracted for each learning data, without considering the rest of the learning data. Do not consider other regions within the data. Representative methods for regional feature extraction include methods such as Scale Invariant Feature Transform (SIFT) and Dens-SIFT.

However, the global feature extraction method or the local feature extraction method is difficult to extract statistical features for changes in various face images such as lighting, facial expressions, poses, occlusion, etc. There was a problem that it was difficult to express a high recognition rate for the face image because it could not express properly, and as a result, it did not properly preserve the robust (statistical feature) according to various changes of the face image.

In order to solve the above problems, an object of the present invention is to divide each face image into a plurality of face images photographing various facial expressions of a person, extract local features from the divided images, and then statistically learn them. By providing a face recognition method that can maximize the face recognition rate.

In order to achieve the above object, the present invention comprises the steps of: (a) dividing a plurality of face images to be learned to each of the same M image, and to obtain a local feature descriptor of each segmented image through local feature descriptor extraction; (B) calculating an average and a variance for the plurality of local feature descriptors obtained in step (a); (C) dividing one face image to be compared into M images, and obtaining a local feature descriptor of each segmented image through local feature descriptor extraction; (D) calculating a distance between the plurality of local feature descriptors obtained in step (a) and the plurality of local feature descriptors obtained in step (c); (E) calculating respective weights of the M-segmented images of the face images to be learned, using the average and the variance calculated in the step (b); And (f) calculating a distance between the face image to be learned and the face image to be compared by combining the distances between the plurality of local feature descriptors calculated in step (d) and the respective weights calculated in step (e). It provides a face recognition method comprising a.

In the step (f), the distance distance calculation between the face image to be learned and the face image to be compared may be calculated by the following equation.

Local feature descriptor extraction in steps (a) and (c) is preferably performed through a scale invariant feature transform (SIFT) feature extraction method or a dens-SIFT feature extraction method.

As described above, in the present invention, a robust feature can be used for occlusion through statistical learning as well as an advantage of local features that are robust to lighting, poses, and facial expressions of a person, thereby improving face recognition rate.

1 is a flowchart sequentially illustrating a robust face recognition process through statistical learning of regional features of the present invention.
2 is a view showing a single face image (training image) to be learned in the present invention,
FIG. 3 is a diagram illustrating a state in which the grid is entirely divided in order to extract local features of the face image shown in FIG. 2.
4 is a view showing a closed face image (test image) in which part of the face image is hidden;
FIG. 5 is a diagram showing a state in which the grid is entirely divided in order to extract local features of the face image shown in FIG. 4.
6 is a graph comparing face recognition rates between the present invention and conventional PCA and LDA feature extraction methods;
7 is a diagram illustrating a face image (test image) in which part of the face image is artificially occluded.
8 is a graph comparing face recognition rates between the present invention and conventional PCA and LDA feature extraction methods.

With reference to the accompanying drawings, a robust face recognition method through the statistical learning of the regional features of the present invention will be described.

First, as shown in FIG. 2, one face image 1 of the entire face is divided into a plurality of (M) partial regions as shown in FIG. 3.

In this case, since the entire face image 1 must be used without using features of a specific part of one face image 1 such as eyes, nose, and mouth, all parts of the face image 1 must be used. It is preferable to divide into a grid pattern in order to divide into images of the same size. Each of the divided parts is referred to as divided images P1 to PM hereinafter.

For each of the divided images P1 to PM divided into a plurality of regions, local feature descriptors are acquired through SIFT, which is a local feature extraction method (S1). The process of obtaining a local feature descriptor is as follows.

SIFT goes through two computational steps to generate a set of image features. Step 1 determines how to select key points from the full face image 1. Here, the selected important pixel is referred to as a 'keypoint'. Step 2 defines an appropriate descriptor for the selected keypoints to represent meaningful local properties of the image 1. Here, the descriptor is referred to as a 'local feature descriptor'.

As described above, each of the divided images P1 to PM divided into M pieces in one face image 1 may be represented by a plurality of keypoint sets having the local feature descriptor.

Hereinafter, the local feature descriptors will be briefly described, and a method of applying the local feature descriptors to show each of the divided images P1 to PM divided into a plurality will be described.

SIFT uses a scale-space difference-of-Gaussian (DOG) function to detect a plurality of keypoints within each image. For one face image I (x, y) input, the scale-space is a function L (x, y) provided from the convolution of the variable-scale Gaussian G (x, y, σ) with the face image. , σ). Accordingly, the DOG function is defined as in Equation 1 below.

Where x is the x-axis coordinate, y is the y-axis coordinate, sigma is the scale, and k is the multiplicative factor.

In this case, the local maximum and minimum values of the D (x, y, σ) function are based on eight surrounding images that surround one segmented image within the current single image 1. In the case of extracting a feature through the conventional SIFT, a plurality of keypoints are selected based on the values of stability and local feature descriptors, in which case the plurality of keypoints and positions are different for each segmented image.

On the other hand, in case of face recognition, very few keypoints can be extracted because of lack of texture of face image. In this case, instead of SIFT, another local feature extraction method, Dense-SIFT extraction method, is applied. The problem can be solved.

Each local feature descriptor extracted through the SIFT extraction method is represented by a 128 dimensional vecter descriptor, which is a four-dimensional 128-dimensional vector. The four parts here are a locus, scale (σ), direction and slope indicating where the feature is selected. In this case, the degree of inclination (m (x, y)) and the direction (θ (x, y)) for each key point located in the two-dimensional coordinates (x, y) for each segmented image are represented by Equations 2 and Obtained through Equation 3.

Subsequently, in order to apply the SIFT extraction method to displaying the face image 1, first, the locations of the M feature descriptors and the local feature descriptors on a regular grid are determined. Here, since each local feature descriptor is represented by a descriptor vector κ, one face image I can be represented by a set of M descriptor vectors. In this case, the face image I may be expressed as in Equation 4 below.

Based on this equation, probability distribution learning is performed by calculating the mean and variance for the M descriptor vectors κ (S2).

The random training data for the face image is {I ⁱ } _{i = 1,...} Given by _{, N} , M training sets for the keypoint descriptor can be expressed as Equation 5 below.

The M training sets T _m have a plurality of local feature descriptors for a segmented image of a specific position among the segmented images of the face images obtained from all the face images divided into m regions.

By using the training set T _m , it is possible to estimate the probability density of the local feature descriptor κ _m for the segmented image of a particular location. We also use a multivariate Gaussian model for 128-dimensional random vectors, following a simple preliminary approach. Therefore, the local feature descriptor κ _m can be expressed as Equation 6 below.

In Equation 6, the two model parameters, mean (μ _m ) and covariance matrix (Σ _m ), can be estimated by the simple mean and simple covariance matrix of the training set (T _m ), respectively.

As described above, one face image 1 may be represented by a plurality of determined local feature descriptors κ _m (m = 1,…, M), and may be applied to a plurality of face images photographing various expressions of the same person. The average and the variance are calculated through the steps S1 and S2 described above. In addition, a plurality of different people are learned through the above-described steps S1 and S2.

Using the probability density estimated as above, it is possible to calculate the probability that each descriptor is found at a particular location in a prototype image of the human frontal faces.

Meanwhile, a face image 3 of a predetermined person (hereinafter referred to as a 'test image') as shown in FIG. 4 is selected for a face recognition test, and a segment is divided into M regions as shown in FIG. 5 with respect to the test image. Each local feature descriptor of the test image is obtained through the extraction method (S3). The test image is an image of the same person as the training image 1 and is a state in which the bottom part of the nose is covered with a shawl.

Subsequently, the distance for the local feature descriptor between the training image 1 and the test image 3 is calculated (S4). To this end, the acquired local feature descriptors can be used to find the weight of each descriptor.

In this case, I ^tst input test image (3) for comparison with training image (1). In this case, the set of keypoint descriptors for the test image 3 may be obtained by applying the SIFT feature extraction method as shown in Equation 7 below.

에 대하여, 확률 밀도

를 계산할 수 있고, 각 지역적 특징 기술자

에 대한 가중치 w_m를 획득할 수 있다(S5). 여기서 가중치 w_m는 하기 수학식 8과 같이 나타낼 수 있다.Then, local feature descriptors for each subarea

, Probability density

Can be calculated, and each local feature descriptor

A weight w _m may be obtained (S5). The weight w _m may be expressed as Equation 8 below.

Then, the test image I ^tst calculated as above And the distance of the local feature descriptors between the corresponding regions between the training image I ⁱ and the calculated weight w _m are calculated (S6). In this case, the distance may be calculated using Equation 9 below.

및

는 테스트 이미지와 i번째 트레이닝 이미지의 m 부분영역에 대한 지역적 특징 기술자를 각각 의미한다.Where m is each subregion index, w _m is an arbitrary subregion weight, d (·, ·) is a distance calculation function such as L ₁ norm and L ₂ norm,

And

Denotes local feature descriptors for the m subregion of the test image and the i th training image, respectively.

In this case, the weight w _m is determined by the m th local patch of the test image represented by the m th local feature descriptor, so that the local patch represents the weight in the measurement between the training image and the test images. Corresponds to an important factor.

Thus, if there is an occlusion in the test image, the regional patch containing the occlusion may not be a patch commonly seen in a pre-acquired set of training images and thus the weight is small. In view of this, the present invention can provide more robust results against regional changes by exposing a hidden portion of the measurements.

To confirm the robustness of the facial image for the present invention, comparative experiments were conducted on a sample database with regional variations. The present invention has been compared with conventional regional approaches and conventional statistical methods. The sample database consists of a number of frontal images (3600 colors or more) photographing the faces of 126 subjects (70 males and 56 females). Specifically, 26 images (images) were taken for each test, and all of these images were taken with different expressions. For each subject, these images were taken at two different time periods at two week intervals. Each period consists of 13 images with differences in facial expressions, lighting, and occlusion.

In this experiment, images of 100 images of the 126 images were selected and 13 images captured in the first period were used for each test subject. For reference, as a preprocessing step, an aligned image with eye position was obtained manually. After positioning, the face image was morphed and resized to 88 × 64 pixels.

As a result of confirming each face recognition rate using the present invention and the conventional PCA and LDA method using the image data, as shown in FIG. 6, the present invention shows a face recognition rate close to 90%, whereas the conventional PCA and LDA method is shown. All of them showed facial recognition rate less than 60%.

In addition, when artificial occlusion is applied to the test image as shown in FIG. 7, the face recognition rate of the present invention is compared with that of the prior art. As shown in FIG. 8, the present invention shows a face recognition rate of 95%. All of the methods showed a facial recognition rate of less than 70%.

As described above, although the present invention has been described by way of limited embodiments and drawings, the present invention is not limited thereto, and the technical idea of the present invention and the following by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalents of the claims to be described.

1: training image 3: test image

Claims (3)

(A) dividing a plurality of face images to be learned into M images in the same manner and acquiring local feature descriptors of each of the divided images by extracting local feature descriptors;
(B) calculating averages and variances among the plurality of regional feature descriptors for the divided images corresponding to each other among the plurality of face images to be learned obtained in step (a);
(C) dividing one face image to be compared into M images, and obtaining a local feature descriptor of each segmented image through local feature descriptor extraction;
(D) calculating a distance between the plurality of local feature descriptors obtained in step (a) and the plurality of local feature descriptors obtained in step (c);
(E) calculating respective weights of the M-segmented images of the face images to be learned, using the average and the variance calculated in the step (b); And
(F) calculating a distance between the face image to be learned and the face image to be compared by combining the distances between the plurality of local feature descriptors calculated in step (d) and the respective weights calculated in step (e). Facial recognition method comprising a. The method of claim 1,
In step (f), the distance recognition between the face image to be learned and the face image to be compared is calculated by the following equation.

Where I ^tst is the face image to be compared, I ⁱ is the i-th face image to learn, m is each subregion index, w _m is any subregion weight, and d (·, ·) is L ₁ norm and L ₂ norm Same distance calculation function,

And

Denotes local feature descriptors for the m subregion of the image to be compared and the image to be learned, respectively. The method of claim 1,
The local feature descriptor extraction in steps (a) and (c) is performed through a scale invariant feature transform (SIFT) feature extraction method or a dens-SIFT feature extraction method.

Images