CN103971096B - A kind of Pose-varied face recognition method based on MB LBP features and face energy diagram - Google Patents

Abstract

The invention provides a multi-pose face recognition method based on MB-LBP features and face energy maps. In the present invention, by establishing a multi-pose face image training library, after the face image is subjected to size normalization processing, the face mean energy map and variance energy map of the training library are constructed; then the obtained face mean energy map and variance energy MB‑LBP feature extraction is performed on the image and stored as matching library information; when face detection is performed, the face image is detected and the face area is extracted, and the size of the face area image is normalized to obtain a standard face image; perform MB‑LBP feature extraction on the standard face image; finally use the nearest neighbor classifier to complete the classification and recognition of multi-pose faces. The present invention can better retain the inherent appearance features of multi-pose faces, and retains the microstructure and macrostructure of face image patterns, can remove the influence of single pixel noise, requires less storage space, and has excellent Recognition rate and recognition speed.

Description

A multi-pose face recognition method based on MB-LBP feature and face energy map

technical field

The invention belongs to the technical field of biological feature identification, and in particular relates to a multi-pose face recognition method based on MB-LBP features and a face energy map.

Background technique

Compared with other biometric identification methods such as fingerprints and irises, automatic face recognition technology has special advantages such as convenient collection and non-intrusiveness, so it has very broad application prospects and economic value. If face recognition is divided by pose, it can be divided into forward-looking face recognition and multi-pose face recognition. Among them, the technology of forward-looking face recognition is relatively mature. However, the multi-pose face recognition method still has many technical problems such as large storage capacity, complex calculation, and low recognition rate. The lag in research on multi-pose face recognition has become one of the main obstacles restricting the real application of face recognition technology. Therefore, the research on multi-pose face recognition is of great significance to the popularization of face recognition technology.

Existing multi-pose face recognition, the typical document is the patent "A Multi-pose Face Recognition Method Based on Face Mean and Variance Energy Map" (Wang Kejun, Zou Guofeng, etc. Chinese invention patent: 201310122161.3[P] .2013-07-24.), using face images of different poses to construct a face energy map for multi-pose face recognition. However, the face energy map does not have the periodicity of the gait energy map, and cannot represent the face posture changes at different pitch angles and different swing angles.

Liao et al. proposed (Liao S C, et al. Learning multi-scale block local binary patterns for face recognition. In Proceedings of the 2007 International Conference on Biometrics. Seoul, South Korea: Springer, 2007.828-837.) Based on the multi-scale local binary pattern ( Multi-scale Block Local Binary Pattens, MB-LBP) face recognition method. Although the MB-LBP feature extraction method has achieved good results in texture analysis and face recognition application experiments, under the influence of complex factors such as severe illumination changes, extreme changes in imaging conditions, posture, expression, age, etc., MB-LBP The representation ability and classification ability of features are also limited, and the recognition performance drops sharply.

The present invention combines the MB-LBP feature and the face energy map method, and uses the MB-LBP feature extraction method to perform secondary feature extraction on the face mean energy map and variance energy map, which reduces the computational complexity. Redundant information is removed, and then the extracted image texture features are used for classification and recognition of multi-pose faces.

Contents of the invention

The purpose of the present invention is to provide a kind of MB-based image recognition system that can effectively extract the key information of human face under the situation of pitch change and left-right swing change, and requires small storage space, low computational complexity, and high recognition rate and recognition speed. A multi-pose face recognition method based on LBP features and face energy maps.

The purpose of the present invention is achieved like this:

(1) set up a multi-pose human face image training library, and carry out size normalization to all face images in the multi-pose human face image training library;

(2) According to the different pitch angles of the face area image, the pitch change range of the face is divided, and the narrow face mean energy map and the narrow face face variance energy map are constructed in combination with the pitch change range, as the primary features of multi-pose face recognition;

The expression of the narrow-sense face mean energy map F _k (x, y) involved is:

In the formula, M _k represents the total number of images in the same pitch angle range and left and right swing angle changes, I _j (x, y) is a multi-pose grayscale face image, k represents different pitch angle ranges, k=1 represents looking up, k=2 means looking up, k=3 means looking down, j means the image of the jth left and right swing angle changes, and x, y represent the two-dimensional image plane coordinates;

The expression of the narrow-sense face variance energy map D _k (x, y) involved is:

(3) adopt MB-LBP algorithm to carry out secondary feature extraction to the narrow-sense face mean energy map and the narrow-sense face variance energy map obtained in step (2), store the MB-LBP feature information for classification recognition;

The involved MB-LBP features can be expressed as:

in

g _k represents the gray value of a single pixel; B represents the average gray value of the nth pixel block;

(4) Read the multi-pose face image to be detected, detect the face area and extract the face;

(5) carry out size normalization processing with the extracted face area, obtain standard face training image;

(6) Carry out MB-LBP feature extraction to standard face training image, and store the extracted MB-LBP feature information;

(7) For the MB-LBP feature information of the standard face image to be detected obtained in step (6) and the MB-LBP feature information of the training database face energy map obtained in step (3), through the Euclidean distance-based The nearest neighbor classifier performs classification and recognition, and finally outputs the face recognition result.

The beneficial effects of the present invention are:

The face energy map combines the comprehensive information of multiple face images, which can not only save storage space and reduce computational complexity, but also weaken the noise interference in a single frame image. The face energy map contains a variety of poses The face contour information under the condition is conducive to the realization of face recognition under large-angle pose changes. The present invention performs feature extraction on the human face energy map and the detected standard human face image through the MB-LBP operator for classification and recognition. The extracted MB-LBP feature can well retain the inherent appearance features of multi-pose faces, and includes the microstructure and macrostructure of the face image pattern, and can remove the influence of single pixel noise, which also improves the recognition rate. The recognition speed has been significantly improved, and the comprehensive performance of multi-pose face recognition has been improved.

Description of drawings

Figure 1 is a flow chart of multi-pose face recognition based on MB-LBP features and face energy maps;

Figure 2 is a diagram of the change of the human face in three-dimensional space;

Fig. 3 is a multi-pose face image and a face mean energy map;

Fig. 4 is a multi-pose face image and a face variance energy map;

Fig. 5 is the original image in the face database;

Figure 6 is a normalized face image;

Fig. 7 is a partial face image in the test face image database;

Fig. 8 is a representation method diagram of MB-LBP features;

Fig. 9 is the texture image after the face mean energy map and variance energy map MB-LBP;

Fig. 10 is the texture image after the standard face image MB-LBP;

Figure 11 is a comparison table of the recognition effect of the method combining MB-LBP features and face energy maps with other methods.

detailed description

The present invention is described in more detail below in conjunction with accompanying drawing example:

A multi-pose face recognition method based on MB-LBP features and face energy maps. Firstly, all face images in the multi-pose face image database are normalized in size, and then the mean energy map and variance energy of human faces are further constructed. Fig. 1, then perform MB-LBP feature extraction on the face mean energy map and variance energy map; then, read the multi-pose face image to be detected, and perform face area detection on the face image based on the Adaboost algorithm, and extract the face The size of the region is normalized to obtain a standard face image; then the feature extraction method of MB-LBP is used to extract the feature of the standard face image obtained from the test library; finally, the face recognition is completed through the nearest neighbor classifier.

1. Size normalization of multi-pose face training library images

The experiment of the present invention uses the CAS-PEAL-R1 shared face image database of the Institute of Computer Technology, Chinese Academy of Sciences, based on a subset of non-frontal face images in the face image database, which includes images of 1040 people, and 50 of them are selected in the experiment images, each containing 21 different pose variations. First, 6 faces are selected from the faces of each person under each pitch change, a total of 18 images, and a training face database is constructed, with a total of 50×18=900 images. Then, size normalization is performed on all images, and in the present invention, the size of the face area is uniformly normalized to 230×270 pixels.

2. Construct face mean energy map and variance energy map

The normalized training library images are used to construct the narrow-sense face mean energy map and variance energy map under the three pitch changes respectively.

2.1 Face mean energy map in the narrow sense

Taking the horizontal viewing angle of the face as zero degree, the maximum pitch angles that can appear on the face are -45° and 60°. Generally, the pitch angles of the face are distributed between [-30°, 30°]. In the present invention, the human face image whose pitch angle range is between [-5°, 5°] is defined as the human face image in the head-up range, and the human face image between [5°, 30°] is defined as the human face in the upward-looking range Image, the face image between [-30°, -5°] is defined as the face image in the overlooking range. According to different pitch angle ranges, there is a concept of narrow face mean energy map, as follows:

Narrow face mean energy image (NFMEI): refers to the mean image obtained by superimposing and summing the images of the same person in the same pitch angle range and different left and right sway angles, and then averaged, depending on the pitch angle range of the face Each person contains 3 average energy images, which are the upward average energy image, the horizontal average energy image and the downward average energy image.

Given a multi-pose grayscale face image I _j (x, y), the calculation formula of the narrow face mean energy map is shown in (1):

Among them, M _k represents the total number of images in the same pitch angle range and the left and right swing angle changes, k represents different pitch

Angle range, k=1 means looking up, k=2 means looking up, k=3 means looking down, j means the image of the jth left and right swing angle changes, x, y represent the two-dimensional image plane coordinates.

2.2 Narrow face variance energy map

According to the different pitch angle ranges, there is also the concept of narrow face variance energy map, as follows:

Narrow face variance energy image (NFVEI): refers to the average of the square sum of the difference between the image of the same person in the same pitch angle range and different left and right swing angles and the corresponding narrow face mean energy image. image.

For a multi-pose face grayscale image I _j (x, y), the definition of the narrow face variance energy map is shown in (2):

Wherein, M _k represents the total number of images in the same pitch angle range and left and right swing angle changes, k represents different pitch angle ranges, k=1 means looking up, k=2 means looking up, k=3 means looking down, F _k (x, y) represents the narrow mean energy map obtained when the left and right swing angles change within a certain pitch angle range, j represents the jth image of the left and right swing angle changes, and x, y represent the two-dimensional image plane coordinates.

Combining Figure 3 and Figure 4, it shows the images of a person with seven left and right swing changes in the three pitch situations of looking down, looking up, and looking up, and the corresponding narrow-sense face mean energy map and variance energy map.

3. Extract MB-LBP feature from face mean energy map and variance energy map

The face energy map is the primary feature obtained by superimposing the pose face image, which can be directly used for score recognition. However, since data redundancy still exists in the face energy map and the recognition effect is not good, the present invention uses MB-LBP to perform secondary feature extraction on the face energy map for face recognition.

A multi-scale local binary mode face recognition method, which replaces the comparison between the pixel values of the traditional LBP operator with the comparison of the average gray value between sub-blocks to realize MB-LBP calculation. Each pixel block is a square block containing adjacent pixels. If the side length L of the block is used as a parameter, 9×L×L represents the size of MB-LBP,

MB-LBP features can be expressed as:

in

g _k represents the gray value of a single pixel. B represents the average gray value of the nth pixel block.

Fig. 8 is a representation method of MB-LBP features. Fig. 9 is a texture image after extracting the MB-LBP feature of the mean energy map and the variance energy map of a part of the face respectively.

4. Read multi-pose face images and face area detection

4.1 Definition of Face Pose Change

Combined with Figure 2, the changes of the human face in the 3-dimensional space are the translation and rotation along the axis, in which the left and right translation along the axis, the up and down translation along the axis, the front and back translation along the axis, and the rotation of a certain angle around the axis are caused by Face image tilt can be effectively overcome by geometric normalization. However, several normalizations cannot overcome the up and down pitch changes caused by the rotation of the central axis of the face image, and the left and right sway changes caused by the rotation of the axis as the center. In the present invention, the change caused by the face image centering on the axis is called pitch change, which can be divided into looking up, looking up and looking down according to the different rotation angles; the change caused by the face image rotating around the axis is called swaying change.

4.2 Face area detection

With reference to FIG. 5 and FIG. 6 , the present invention needs to first read multi-pose face images with pitch changes and left-right swing changes from the multi-pose face database. Then the effective face area is obtained through the AdaBoost algorithm.

The Adaboost classifier is a cascade of multi-layer weak classifiers. The correct result obtained by the first layer classifier triggers the second layer classifier, and the correct result output by the second layer triggers the third layer classifier, and so on. . Conversely, a negated output from any layer causes detection to stop immediately. By setting the threshold of each layer, most of the faces can pass, and the non-faces cannot pass, so that the layer close to the back end of the cascade classifier rejects most of the non-faces. Experiments show that the AdaBoost algorithm can effectively detect the face area.

5. Normalize the image size of the face area

After obtaining the face area images, it is necessary to normalize the size of all images. In the present invention, the size of the face area is normalized to 230×270 pixels. Figure 7 is an image of the detected face area.

6. MB-LBP feature extraction for standard face images

The obtained standard face image is subjected to MB-LBP feature extraction, and the extracted feature value is stored for classification and recognition.

Fig. 10 is the texture image after standard face image MB-LBP.

7. Classification recognition

The nearest neighbor classification method based on Euclidean distance calculates the distance between the standard face feature matrix and the face energy map feature matrix, realizes classification, and finally outputs the result.

8. Test the classification and recognition process of faces

Testing process:

(1) First, the face image needs to be extracted from the test library, and the face area detection is performed on the image based on the Adaboost algorithm to obtain the test face area image, and the size normalization process is performed on it to obtain a 230×270 standard face image T.

(2) Then perform MB-LBP feature extraction on the standard face image T.

(3) Finally, perform nearest neighbor classification on the standard face image T and the features of the face mean energy map and variance energy map to obtain the classification result.

Claims (1)

1. a multi-pose face recognition method based on MB-LBP feature and face energy graph, is characterized in that, comprises the following steps: (1) Establish a multi-pose human face image training library, and carry out size normalization to all face images in the multi-pose human face image training library, and uniformly normalize the size of the human face area to 230 × 270 pixels; (2) According to the different pitch angles of the face area image, the pitch change range of the face is divided, and the narrow face mean energy map and the narrow face face variance energy map are constructed in combination with the pitch change range, as the primary features of multi-pose face recognition; The face image whose pitch angle range is between [-5°, 5°] is defined as the face image of the head-up range, and the face image between [5°, 30°] is defined as the face image of the look-up range, The face image between [-30 °, -5 °] is defined as the face image of the overlooking range; The expression of the narrow-sense face mean energy map F _k (x, y) involved is: <mrow><msub><mi>F</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>)</mo></mrow><mo>=</mo><mfrac><mn>1</mn><msub><mi>M</mi><mi>k</mi></msub></mfrac><munderover><mo>&amp;Sigma;</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>M</mi><mi>k</mi></msub></munderover><msub><mi>I</mi>mi><mi>j</mi></msub><mrow><mo>(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>)</mo></mrow><mo>,</mo><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn></mrow> In the formula, M _k represents the total number of images in the same pitch angle range and left and right swing angle changes, I _j (x, y) is a multi-pose grayscale face image, k represents different pitch angle ranges, k=1 represents looking up, k=2 means looking up, k=3 means looking down, j means the image of the jth left and right swing angle changes, and x, y represent the two-dimensional image plane coordinates; The expression of the narrow-sense face variance energy map D _k (x, y) involved is: <mrow><msub><mi>D</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>)</mo></mrow><mo>=</mo><mfrac><mn>1</mn><msub><mi>M</mi><mi>k</mi></msub></mfrac><munderover><mo>&amp;Sigma;</mo><mi>j</mi><msub><mi>M</mi><mi>k</mi></msub></munderover><msup><mrow><mo>(</mo><msub><mi>I</mi><mi>j</mi></msub><mrow><mo>(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>)</mo></mrow><mo>-</mo><msub><mi>F</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>)</mo></mrow><mo>)</mo></mrow><mn>2</mn></msup><mo>,</mo><mi>k</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn></mrow> (3) adopt MB-LBP algorithm to carry out secondary feature extraction to the narrow-sense face mean energy map and the narrow-sense face variance energy map obtained in step (2), store the MB-LBP feature information for classification recognition; The involved MB-LBP features can be expressed as: in g _k represents the gray value of a single pixel; B represents the average gray value of the nth pixel block; (4) Read the multi-pose face image to be detected, detect the face area and extract the face; the change caused by the face image centered on the axis is called pitch change, which can be divided according to the rotation angle For looking up, looking up and looking down; the change caused by the rotation of the face image on the axis is called the left and right swing change; the effective face area is obtained through the AdaBoost algorithm; (5) Normalize the size of the extracted face area to obtain a standard face training image; normalize the size of the face area to 230×270 pixels (6) Carry out MB-LBP feature extraction to standard face training image, and store the extracted MB-LBP feature information; (7) For the MB-LBP feature information of the standard face image to be detected obtained in step (6) and the MB-LBP feature information of the training database face energy map obtained in step (3), through the Euclidean distance-based The nearest neighbor classifier performs classification and recognition, and finally outputs the face recognition result; (8) Test the classification and recognition process of faces: The face image is extracted from the test database, and the face area detection is performed on the image based on the Adaboost algorithm, and the test face area image is obtained, and the size is normalized to obtain a 230×270 standard face image T: the standard face image Face image T is subjected to MB-LBP feature extraction: the standard face image T and the features of the face mean energy map and variance energy map are subjected to nearest neighbor classification to obtain the classification result.