KR101314293B1 - Face recognition system robust to illumination change - Google Patents

Abstract

PURPOSE: A face recognition system is provided to control misrecognition rates generated in an illumination change environment by applying a characteristic extracting technology based on image covariance. CONSTITUTION: A face database (DB) (150) stores face images and identification information for the face image. A face area detecting unit (110) extracts the face image from the inputted image, and a pre-processing unit (120) changes the pixel values of pixels constructing the face area into a binary pattern. A characteristic extraction unit (130) extracts characteristic information from the face image changed into the binary pattern. A face recognition unit (140) recognizes a face by using the characteristic information for the face images stored in the characteristic information and the face DB. [Reference numerals] (100) Image input unit; (110) Face area detecting unit; (120) Face area pre-processing unit; (130) Characteristic extraction unit; (140) Face recognition unit; (150) Face database

Description

Face recognition system robust to illumination change

The present invention relates to a face recognition system and method using a binary pattern, and more particularly, to a method for generating a binary pattern descriptor that is robust to external lighting changes and a face recognition system and method using the same.

With the development of the information society, as the interface technology between humans and computers has been highlighted, research on biometrics technology is being actively conducted. In addition, as the importance of personal information emerges, technologies for using biometric information such as a human face, voice, iris, fingerprint, signature, and vein have been studied as a method for security and management of information and identification of an individual. Among biometric technologies, face recognition has an advantage of providing biometric information in a non-contact manner and providing more convenience to users because the image is processed using a camera.

However, face recognition appears as a problem of deterioration in recognition performance even though the face of the same person is expressed as a very different image according to the change in external lighting.

In this regard, this patent constructs a face recognition system through an improved binary pattern descriptor and image covariance-based feature extraction technology, and attempts to solve the above problems by using the same.

United States Patent Application Publication US 2011 / 0293189A1 Korean Patent Publication No. KR 10-2012-0042101 Korea Patent Registration KR 10-0345245 United States Patent Application Publication US 2010/0150452

An object of the present invention for solving the above problems is to provide a face recognition system and method that can suppress the recognition rate and may improve the recognition rate that may occur when face recognition in a lighting change environment.

Another object of the present invention is to provide a face recognition system and method capable of reducing the amount of computation required for face recognition while improving the recognition rate even in a light changing environment.

According to an aspect of the present invention, there is provided a face recognition system, including: a face database storing a plurality of face images and identification information for each face image; A face region detector extracting a face region from an image input from the outside; A preprocessing unit converting pixel values of each pixel constituting the detected face region into a binary pattern; A feature extractor which extracts feature information from a face image converted into a binary pattern; And a face recognition unit configured to perform face recognition using the extracted feature information and feature information on face images stored in the face database.

The preprocessor converts the pixel value of each pixel into a binary pattern using the following equation.

A face recognition system according to a second aspect of the present invention includes a face database storing a plurality of face images and identification information for each face image; A face region detector extracting a face region from an image input from the outside; A preprocessor configured to calculate edge components of neighboring pixels for each pixel constituting the detected face region, and convert pixel values of each pixel into a binary pattern by comparing values between the center pixel and the edge components; A feature extractor which extracts feature information from a face image converted into a binary pattern; A face recognition unit performing face recognition using the extracted feature information and feature information on face images of the face database; The preprocessor converts the pixel value of each pixel into a binary pattern by using the following equation.

A face recognition system according to a third aspect of the present invention includes a face database storing a plurality of face images and identification information for each face image; A face region detector extracting a face region from an image input from the outside; A first preprocessing unit converting pixel values of each pixel constituting the detected face region into a first binary pattern; A second preprocessor configured to calculate edge components of neighboring pixels for each pixel constituting the detected face region, and compare pixel values of the center pixels with edge components to convert pixel values of the respective pixels into a second binary pattern; ; A third preprocessor configured to perform an AND operation on a pixel basis to convert the face image converted into the first binary pattern and the face image converted into the second binary pattern into a third binary pattern; A feature extractor configured to extract feature information from a face image converted into a third binary pattern; And a face recognition unit configured to perform face recognition using the extracted feature information and feature information on face images of the face database.

In the face recognition system according to the third aspect, the first preprocessor converts the pixel value of each pixel into a binary pattern by using the following equation,

The second preprocessor converts the pixel value of each pixel into a binary pattern by using the following equation.

In the face recognition system according to the first to third features described above, the feature extracting unit is an image covariance matrix of learning face images stored in the database according to an algorithm of 2D-PCA, A2D-PCA, and (2D) ² PCA. It is preferable to obtain the image information, detect the projection matrix using the image covariance matrices, and extract the feature vector from the transform image using the projection matrix.

The face recognition system according to the present invention performs preprocessing through the conversion of binary patterns such as MCS-LBP, MLDP, and CGP to face images, and applies feature extraction techniques based on image covariance, which can occur during face recognition in a lighting change environment. It can suppress the false recognition rate and improve the recognition rate.

In addition, in the facial recognition system according to the present invention, 2D-PCA, A2D-PCA, (2D) ² PCA, and the like use image covariance feature extraction techniques to reduce the amount of computation required for face recognition. Has the effect.

1 is a block diagram schematically showing the structure of a face recognition system according to a preferred embodiment of the present invention.
2 is a diagram illustrating LBP, which is a binary pattern conversion method according to the related art.
FIG. 3 is a diagram illustrating MCS-LBP, which is a binary pattern conversion method used in a preprocessor, in a face recognition system according to an exemplary embodiment of the present invention.
4 illustrates a kirsch mask for MLDP calculation, which is an embodiment of the face region preprocessor in the face recognition system according to the preferred embodiment of the present invention.
FIG. 5 is a diagram illustrating images preprocessed using various binary pattern conversion methods used in a face region preprocessor of a face recognition system according to an exemplary embodiment of the present invention.
6 is a graph showing the performance of the face recognition system according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the face recognition system and face recognition method that is robust to external lighting changes according to an embodiment of the present invention.

1 is a block diagram schematically showing the structure of a face recognition system according to a preferred embodiment of the present invention. Referring to FIG. 1, the face recognition system 10 according to the present invention includes an image input unit 100, a face region detector 110, a face region preprocessor 120, a feature extractor 130, and a face recognizer 140. ) And facial database 150. In the face recognition system according to the present invention having the above-described configuration, the face region preprocessing unit preprocesses the face region using a binary pattern conversion method, and the feature extractor extracts the feature vector using an image covariance based algorithm. Characterized in that. Hereinafter, each component of the face recognition system according to the present invention will be described in detail.

The image input unit 100 obtains an image including a face from the outside. The acquired image may include not only an image having almost no light change, but also an image having various light changes.

The face region detector 110 detects a face region from an image acquired through the image input unit. The face detection method for detecting the face area by the face area detection unit may be classified into a knowledge based method, a feature based method, a template matching method, and an appearance based method. The present invention is not limited to a specific face detection technique. However, in the exemplary embodiment of the present invention, the face region is detected from the input image by using the Haar-like feature based Adaboost algorithm. The AdaBoost learning algorithm is a classification method that groups strong classifiers with high detection performance through the linear combination of several weak classifiers.

The face region preprocessor 120 performs preprocessing to convert each pixel of the face regions acquired by the face region detector into a binary pattern. The binary pattern conversion method may use one of a Modified Center Symmetric Local Binary Pattern (MCS-LBP), a Modified Local Directional Pattern (MLDP), and a Centralized Gradient Pattern (CGP).

LBP, a conventional binary pattern conversion method, is a representative binary pattern conversion method that is applied to various fields such as image restoration, biometric image analysis, face image analysis and recognition because of its high discrimination ability and durability against simple lighting changes. LBP is represented by Equation 1, and represents the difference between the pixel value of the current position and the neighboring pixel value as values of 0 and 1. FIG. 2 is a diagram illustrating a calculation of LBP, which is a conventional binary pattern conversion method. Referring to FIG. 2, the LBP is represented by 8 bits because the LBP is configured to be compared with eight adjacent pixels by a method of converting the value to 1 if the value is larger than the central pixel value in the clockwise direction, or to 0 otherwise. In Equation 1, P and R respectively mean the number of adjacent pixels and the radius of the circle, g _c and g _p mean the pixel value of the center pixel and the pixel value of the neighboring pixel, respectively, LBP (P, R) Denotes an LBP transformed value with respect to the center pixel.

Here, p is sequentially rotated clockwise or counterclockwise with respect to one point of the neighboring pixel with respect to the center pixel and increases by 1 sequentially.

MCS-LBP is a binary pattern conversion method that emphasizes the gradient component of the diagonal component in the existing CS-LBP represented by Equation 2 is expressed as Equation 3. 3 is a diagram illustrating a center pixel and neighboring pixels surrounding the pixel to explain MCS-LBP, which is a binary pattern conversion method according to the present invention.

here, MCS - LBP represents a binary pattern value converted by MCS - LBP for each pixel, g _C is a pixel value of the center pixel, and g ₀ , g ₁ , g ₂ , g ₃ , g ₄ , g ₅ , g ₆ , g ₇ is a pixel value of neighboring pixels sequentially arranged in a clockwise or counterclockwise direction with respect to the center pixel, and g ₀ , g ₂ , g ₄ , and g ₆ are arranged at positions horizontal or vertical to the center pixel. The pixel values of neighboring pixels, and g ₁ , g ₃ , g ₅ , and g ₇ are pixel values of neighboring pixels disposed at diagonal positions with respect to the center pixel.

Unlike LBP or MCS-LBP, which generates binary patterns by considering only gray-level pixel values, MLDP calculates edge components of surrounding pixels and compares the values of the center pixel and edge components. How to create a value. Among various filter masks for obtaining the edge components, in the embodiment of the present invention, the Kirsch mask represented by FIG. 4 was used. The edge components filtered with the Kirsch mask are compared with the center pixel values to produce an 8-bit binary pattern. That is, assuming that the filtered peripheral edge component is m _p , the operation of MLDP is expressed as in Equation 4 below.

Here, MLDP represents a binary pattern value converted by MLDP for each pixel.

The CGP is calculated through the pixel level AND operation of the image converted by the MCS-LBP and the MLDP described above, and designed to reduce the influence of noise contained in the face image.

5 is a diagram illustrating images preprocessed using various binary pattern conversion methods used in a face region preprocessor of a face recognition system according to an exemplary embodiment of the present invention. Referring to FIG. 5, three original images captured in different lighting environments and LBP, MCS-LBP, MLDP, and CGP converted images are sequentially illustrated for each original image.

The feature extractor 130 is an image such as 2D-PCA (Two-Dimensional Principal Component Analysis), Alternative 2D-PCA (A2D-PCA), (2D) ² PCA, etc. Feature vectors are extracted by applying a covariance based algorithm. In general, the conventional PCA (Principal Component Analysis) algorithm obtains a covariance matrix by converting a two-dimensional image into a one-dimensional vector, so the dimension of the covariance matrix becomes very high, and thus a large amount of computation is required to calculate the eigenvector. In order to overcome this problem, the feature extraction unit of the face recognition system according to the present invention is a 2D-PCA, A2D-PCA, (2D) ² PCA that directly uses a two-dimensional image without converting the covariance matrix into a one-dimensional vector Use one method. In addition, image covariance-based feature extraction algorithms have much smaller covariance matrices than PCA, which is very efficient in computing the eigenvectors and improves face recognition performance over PCA.

라 하자. 이미지 행렬 A가 X 축에 투영된다면, m× d 의 행렬은 Y= AX 로 계산된다. Hereinafter, a calculation process of 2D-PCA, A2D-PCA, and (2D) ² PCA according to an embodiment of the present invention will be described in detail. An image matrix having a face image size of m × n is called A , and a matrix whose rows are orthogonal to each other.

Let's say. If the image matrix A is projected on the X axis, then the matrix of m × d is calculated as Y = AX .

라 하고, 학습 얼굴 영상들의 평균이미지를

라 하자. 학습 얼굴 영상들의 이미지 공분산 행렬 G 는 수학식 5로 계산된다. At this time, a calculation method of 2D-PCA will be described. In 2D-PCA, the variance of the image matrix samples is used to determine the optimal projection matrix X. N learning face spirits

The average image of the learning face images

Let's say. The image covariance matrix G of the learning face images is calculated by Equation 5.

Here, the optimal projection matrix X is d eigenvectors of the covariance matrix G having the maximum eigenvalues, where X = [ X ₁ , X ₂ , ..., X _d ] . Since the covariance matrix G has only n × n , the eigenvector projection matrix X can be computed very efficiently. Since the projection matrix X is a matrix of n × d dimension, the feature vector Y _k (m × d ) of the image matrix A (m × n ), which is a given face image, is a projection matrix that is an eigenvector as shown in Equation 6 below. Calculated by projecting onto the axis of X.

Hereinafter, the calculation process of A2D-PCA will be described. A2D-PCA is treated in a similar process as 2D-PCA. However, in A2D-PCA, the covariance matrix G ' is calculated using Equation 7. In this case, the optimal projection matrix Z is the d 'eigenvectors of the covariance matrix G' having the maximum eigenvalues, which are m x d 'matrices where Z = [ Z ₁ , Z ₂ , ..., Z _d' ]. It is composed.

Since the projection matrix Z is a matrix of m × d ' dimension, the feature vector Y _k ' ( n × d ' ) of the image matrix A (m × n ), which is a given face image, is the eigenvector of the projection matrix Z as shown in Equation 8. Calculated by projecting on the axis.

Hereinafter, the calculation method of (2D) ² PCA is demonstrated. (2D) ² PCA is a combination of 2D-PCA and A2D-PCA, the feature vector C is calculated by the equation (9).

Here, X (n × d) and Z (m × d ') represent projection matrices of 2D-PCA and A2D-PCA, respectively, and when the image matrix A of the face image is the size of m × n, the feature vector C is It has a dimension of d '× d. It can be seen that the feature vector of the (2D) ² PCA has a dimension of d '× d smaller than the feature vector dimension of the 2D-PCA or A2D-PCA, so that it is a more efficient feature descriptor at face recognition.

The face recognition unit 140 obtains feature information about face images stored in a face database by using face feature vectors in the form of a 2D matrix extracted by the feature extractor 130, and applies the face image to a face image to be recognized. The face recognition is performed by comparing the feature information. The face recognition unit performs face recognition on the feature information of the test image to be recognized and the feature information of the training image by applying a Euclidean distance scale and a Nearest Neighbor (NN) classifier. The Euclidean distance scale is represented by Equation 10.

과

는 각각 학습 및 테스트에 사용되는 2D-PCA, A2D-PCA, 또는 (2D)²PCA의 특징 벡터를 의미한다. 더불어, 얼굴인식을 위하여 본 발명은 NN 분류기에 국한하지 않으며, Bayesian classifier, Fisher classifier, support vector machine (SVM) classifier, K-NN(k-Nearest Neighbor), Random forest 등의 일반적인 분류 알고리즘을 적용할 수 있다. here,

and

Denotes a feature vector of 2D-PCA, A2D-PCA, or (2D) ² PCA used for learning and testing, respectively. In addition, the present invention is not limited to the NN classifier for face recognition, and general classification algorithms such as Bayesian classifier, Fisher classifier, support vector machine (SVM) classifier, K-NN ( k- Nearest Neighbor), Random forest, etc. may be applied. Can be.

6 is a graph showing the performance of the face recognition system according to an embodiment of the present invention. Referring to FIG. 6, the face recognition method combining 2D-PCA and A2D-PCA with pre-processed images of MCS-LBP, MLDP, CGP, etc. of the present invention is better than the conventional PCA method, LBP and 2D-PCA. It can be seen that the recognition performance is improved.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood that various changes and modifications may be made without departing from the spirit and scope of the invention. It is to be understood that the present invention may be embodied in many other specific forms without departing from the spirit or essential characteristics thereof.

The face recognition system according to the present invention can be widely used for security systems, personal identification systems, and the like.

10: face recognition system
100: video input unit
110: face area detection unit
120: face region preprocessor
130: feature extraction unit
140: face recognition unit

Claims (9)

A face database storing a plurality of face images and identification information for each face image;
A face region detector extracting a face region from an image input from the outside;
A preprocessing unit converting pixel values of each pixel constituting the detected face region into a binary pattern;
A feature extractor which extracts feature information from a face image converted into a binary pattern;
And a face recognition unit configured to perform face recognition using the extracted feature information and feature information on face images stored in the face database.
The preprocessing unit is a face recognition system robust to changes in illumination, characterized in that for converting the pixel value of each pixel into a binary pattern using the following equation.

Here, MCS-LBP represents a binary pattern converted value of each pixel, g _C is a pixel value of the center pixel, and g ₀ , g ₁ , g ₂ , g ₃ , g ₄ , g ₅ , g ₆ , g ₇ Is a pixel value of neighboring pixels sequentially arranged in a clockwise or counterclockwise direction with respect to the center pixel, and g ₀ , g ₂ , g ₄ , g ₆ are neighboring pixels arranged in a horizontal or vertical position with respect to the center pixel. Are pixel values, and g ₁ , g ₃ , g ₅ , and g ₇ are pixel values of neighboring pixels disposed at diagonal positions with respect to the center pixel.
A face database storing a plurality of face images and identification information for each face image;
A face region detector extracting a face region from an image input from the outside;
A preprocessor configured to calculate edge components of neighboring pixels for each pixel constituting the detected face region, and convert pixel values of each pixel into a binary pattern by comparing values between the center pixel and the edge components;
A feature extractor which extracts feature information from a face image converted into a binary pattern;
A face recognition unit performing face recognition using the extracted feature information and feature information on face images of the face database;
And the preprocessing unit converts the pixel value of each pixel into a binary pattern by using the following equation.

Here, MLDP represents a value converted into a binary pattern of each pixel, g _C represents a pixel value of the center pixel, m _p represents an edge component of neighboring pixels, and p represents a number of neighboring pixels. A face database storing a plurality of face images and identification information for each face image;
A face region detector extracting a face region from an image input from the outside;
A first preprocessing unit converting pixel values of each pixel constituting the detected face region into a first binary pattern;
A second preprocessor configured to calculate edge components of neighboring pixels for each pixel constituting the detected face region, and compare pixel values of the center pixels with edge components to convert pixel values of the respective pixels into a second binary pattern; ;
A third preprocessor configured to perform an AND operation on a pixel basis to convert the face image converted into the first binary pattern and the face image converted into the second binary pattern into a third binary pattern;
A feature extractor configured to extract feature information from a face image converted into a third binary pattern;
A face recognition unit performing face recognition using the extracted feature information and feature information on face images of the face database;
Face recognition system robust to changes in illumination, characterized in that it comprises a.
4. The face recognition system of claim 3, wherein the first preprocessor converts the pixel values of each pixel into a binary pattern by using the following equation.

Here, MCS - LBP represents a binary pattern converted value of each pixel, g _C is a pixel value of the center pixel, and g ₀ , g ₁ , g ₂ , g ₃ , g ₄ , g ₅ , g ₆ , g ₇ Is a pixel value of neighboring pixels sequentially arranged in a clockwise or counterclockwise direction with respect to the center pixel, and g ₀ , g ₂ , g ₄ , g ₆ are neighboring pixels arranged at a position horizontal or perpendicular to the center pixel. Are pixel values, and g ₁ , g ₃ , g ₅ , and g ₇ are pixel values of neighboring pixels disposed at diagonal positions with respect to the center pixel. 4. The face recognition system of claim 3, wherein the second preprocessor converts the pixel value of each pixel into a binary pattern by using the following equation.

Here, MLDP represents a value converted into a binary pattern of each pixel, g _C represents a pixel value of the center pixel, m _p represents an edge component of neighboring pixels, and p represents a number of neighboring pixels. The method of claim 1, wherein the feature extractor obtains an image covariance matrix of learning face images stored in the database, detects a projection matrix using the image covariance matrix, and uses the projection matrix. And extract a feature vector from the transformed image and provide the feature vector as feature information. The method of claim 6, wherein the image covariance matrix ( G ) is obtained by the following equation, and the projection matrix ( X ) is d eigenvectors ( X ₁ , X ₂ , ..., X _d And a feature vector ( Y _k ) of the transformed image ( A) is obtained by the following equation.

here, A _k is N learning face images

The average image of the learning face images is

being. The method of claim 6, wherein the image covariance matrix G ' is obtained by the following equation, and the projection matrix Z is d' eigenvectors Z ₁ , Z of the image covariance matrix having a maximum eigenvalue. ₂ , ..., Z _{d '} And a feature vector ( Y _k ') of the transformed image ( A) is obtained by the following equation.

here, A _k is N learning face images

The average image of the learning face images is

being. The feature extractor of any one of claims 1 to 5, wherein the feature extraction unit
The first image covariance matrix G of the learning face images stored in the database is obtained according to the following equation, and the d eigenvectors X ₁ , X ₂ , ... , X _d Find the first projection matrix X composed of
'Obtained according to the equation below, the second image covariance matrix of d having the maximum eigenvalue second image covariance matrix (G), of the learning face images stored in the database, the eigenvectors (Z _1, Z _2,. .., Z _{d '} Find the second projection matrix Z composed of
And a feature vector ( C ) is obtained from the transformed image ( A ) using the first and second projection matrices ( X, Z ) according to the following equation and provided as feature information.

here, A _k is N learning face images

The average image of the learning face images is

being.

Images