KR100955255B1 - Face Recognition device and method, estimation method for face environment variation - Google Patents

Abstract

The present invention estimates internal and external environmental change variables (eg, pose and lighting changes) using three-dimensional face data as input data, and two-dimensional face image of a user, and data in a database for correction. Simultaneous estimation in real time with an appropriate learning-based estimation algorithm improves recognition performance (distinguishment between the person and others) while maintaining the advantages of face recognition technology in terms of user convenience.

Face Pose Illumination Estimation 3d

Description

Face Recognition Device and Method, Estimation Method for Face Environment Variation

The present invention relates to a face recognition apparatus and a method thereof, and more particularly, to a face recognition apparatus using the biometric information of a face corrected by estimating the degree of internal and external environmental changes (eg, pose change and illumination change) and a method thereof. It is about.

In general, personal authentication technology using biometric information of the face is most suitable in terms of user convenience and relatively easy to obtain biometric information compared to personal authentication technology using other biometric information (eg, fingerprint, iris, retina, etc.). to be.

In the conventional two-dimensional face recognition apparatus of FIG. 1 using such a technique, internal and external environmental changes (eg, poses, ambient lighting, facial expressions, jewelry, etc.) are estimated by using a face region of a person extracted from an input two-dimensional face image. The final recognition result is obtained by comparing the feature information obtained from the compensated image and the 2D face information of the corresponding user in the DB.

However, since it is sensitive to internal and external environmental changes, only robust algorithms can be applied to the actual use environment.

On the other hand, the three-dimensional face recognition technology has been studied a lot recently, there are many advantages over the existing two-dimensional face recognition technology, in particular, shows excellent performance even in the environment where the internal and external environment of the face changes.

However, due to the operation method and the price of the 3D data acquisition equipment, it is still difficult to apply it to actual application fields, and 3D data acquisition requires user cooperation such as operation time and user's specific behavior. There is a hassle to do.

Thus, unlike the two-dimensional face recognition technology, the maximum benefit of the face recognition technology of user convenience is faded.

The present invention has been developed to solve the above problems, while maintaining the advantages of the face recognition technology of the user convenience aspect, while simpler and better results than the internal and external environmental variable estimation method in the conventional two-dimensional face recognition technology (eg It is an object of the present invention to provide a face recognition device and a method thereof, which can achieve a recognition performance improvement.

Another object is to provide a method of estimating the degree of internal and external environmental change of a learning-based face suitable for an actual use environment.

The face recognition apparatus of the present invention according to the above object

A face detector for detecting a face region in an input two-dimensional image, a feature extractor for extracting feature information from the face region, and an environment change variable calculator for obtaining internal and external environment change variables of a face in the face region normalized using the feature information A face corrector configured to correct the estimated environment change using the environment change variable and the user's 3D face data in the DB to the user's 3D face data, face information resulting from the correction, and the feature extractor And a feature comparison unit for obtaining a recognition result by comparing the extracted feature information.

The face recognition method of the present invention according to the above object

Extracting feature information from the face region in the input 2D image; changing the inside and outside environment of the face using the 3D face data of the user and the internal and external environment change variables obtained from the face region normalized using the feature information Estimating the degree, and obtaining the recognition result by comparing the face information obtained by correcting the degree of change of the internal and external environment of the face to three-dimensional face data of the corresponding user and the feature information.

The estimating degree of the internal and external environmental change of the face may be performed by classifying various environmentally modified two-dimensional face images generated by applying the internal and external environmental change variables to the 3D face data of the corresponding user in the DB. It is characterized by estimating the degree of internal and external environmental changes in the unique face obtained using the images.

It is characterized in that the degree of internal and external environmental changes of the face is estimated in a class format that distinguishes the degree of change for each environmental variable.

Method for estimating the degree of internal and external environmental changes of the face of the present invention according to the above object

Using the feature information extracted from the face region in the input 2D image, internal and external environment change variables obtained from the normalized face region are applied to the 3D face data of the corresponding user in the DB to generate various 2D face images of various environments. And estimating the degree of environmental change in the unique face of the corresponding user obtained by classifying the plurality of two-dimensional face images by environment variables.

It is characterized in that the degree of internal and external environmental changes of the face is estimated in a class format that distinguishes the degree of change for each environmental variable.

The environmental variable is a face pose, ambient lighting.

According to the present invention, by estimating variables of internal and external environmental changes of a face using two-dimensional face images as input images and three-dimensional face data as data in a DB for correction, the environment while maintaining the advantages of the face recognition technology in terms of user convenience, It is strong against change, increasing recognition performance (the distinction between you and others).

In addition, by simultaneously estimating internal and external environmental changes (eg, pose and lighting changes) in real time, an estimation appropriate to the actual environment is made.

Hereinafter, with reference to the accompanying drawings will be described the present invention.

The face recognition apparatus of the present invention of FIG. 2 is a face recognition apparatus using two-dimensional face images as input images and three-dimensional face data as data for correction.

Specifically, a face detection unit 200 for detecting a face region in an input two-dimensional image, a feature extractor 201 for extracting feature information from the face region, and a change in the internal and external environment of the face in the face region normalized using the feature information. An environment change variable calculation unit 202 for obtaining a variable, and a face correction unit 203 for correcting an environment change degree estimated using the environment change variable and the user's 3D face data in the DB to the user's 3D face data. And a feature comparator 204 which obtains a recognition result by comparison between the face information resulting from the correction and the feature information extracted by the feature extractor.

The operation of such a device is as follows.

First, the feature extractor 201 directly extracts feature information from a face region in the input 2D image extracted by the face detector 200.

That is, the feature information is directly extracted from the face region before compensation, instead of extracting the feature information from the face region where the internal and external environment changes are compensated as in the related art.

Next, the environment change variable calculation unit 202 normalizes the face area using the extracted feature information.

Then, the variables of internal and external environment (eg, face pose, ambient light) are calculated in the normalized face region.

Next, the face correction unit 203 applies the calculated environment change variable to 3D face data of the corresponding user in the DB, and estimates the degree of internal and external environment change.

Specifically, it is as follows.

First, the calculated environment change variable is applied to 3D face data of a corresponding user in a DB to generate a plurality of 2D face images of various environment changes (eg, face pose change, ambient light change).

Subsequently, several 2D face images are classified by environment variables (eg, face pose, ambient light).

Next, the user's unique face is obtained through Principle Components Analysis (PCA) in the classified 2D image.

Then, the degree of change in the environment (eg, face pose, ambient light, etc.) is estimated from the user's unique face.

In particular, estimate in class form.

For example, it is estimated in the form of "face pose change third class".

Here, the "third class" represents a specific range such as 『10 ° ~ 15 ° pose change, left side lighting type』, and the like.

The class classifies the degree of change for each environment variable and is composed of a first class, a second class, a third class, ..., and the like.

For example, the pose change may be composed of a first class, a second class, a third class, ..., and the like by dividing the left 45 ° to the right 45 ° by 15 ° intervals.

In addition, the lighting change may be composed of three classes: left side, front side, and right side.

Next, the degree of environment change (eg, pose change, lighting change) estimated as described above is corrected to the 3D face data of the corresponding user in the DB.

Thereafter, the feature comparison unit 204 compares the face information resulting from the correction with the face feature information of the corresponding user directly extracted from the face region in the input 2D image.

Thus, the result of the comparison is used to obtain a face recognition result.

In the facial feature extraction process of the present invention of FIG. 2, internal and external environmental change compensation is performed in a database DB instead of a conventional input image, and needs to be performed only once.

For reference, conventionally, feature extraction for face normalization and feature extraction for face comparison should be performed. Feature extraction for face normalization is performed on an image in which internal and external environmental changes are not compensated, and feature extraction for face comparison is performed on an image in which internal and external environment changes are compensated.

Compensation for the change of the internal and external environment (eg, face pose, ambient light, facial expression, makeup, ornaments, etc.) of the face of the present invention of FIG. 2 uses three-dimensional face data, so that an error region does not occur in the compensation process.

For reference, an error region may occur when compensating for an internal or external environment change using a 2D image. In particular, since most of the structural information of the face is lost when the 3D object is used as the 2D image, an error region is likely to occur when the front image is generated by compensating for the pose change of the face.

3 is a simulation of a learning-based estimation algorithm for estimating a change in face pose and ambient light in real time, in detail, in and out of the environment according to the present invention.

The learning based estimation algorithm is composed of a learning part and an estimation part.

First, in the learning part, internal and external environmental change variables of the face obtained from the input image are applied to the 3D face data of the corresponding user in the DB. Next, classify the resulting multiple pose and lighting variations of two-dimensional face images into a pose class or lighting class, and use the classified images to classify two-dimensional to three-dimensional pose and lighting estimation classifiers (e.g., multi-layer classifiers) Multi-Layer Perceptron (MLP)).

In the estimation part, the degree of pose and lighting change in the face image learned in the learning part is estimated in a class format. For example, the pose and light changes are estimated in the form of "face pose and light change third class".

For reference, in the conventional art, the face pose change 13 ° and the lighting direction upper left 17 ° "

The difference from the conventional method is that the degree of pose and lighting change is estimated in a class form.

This estimation, although less accurate, is made in a learning manner rather than an analytical method, which enables simultaneous estimation of pose and lighting changes in real time.

The three-dimensional face data of FIG. 4 is an example of three-dimensional face data according to the present invention.

It is a database of BERC (Biometrics Engineering Research Center) obtained using a 3D laser scanner, and has both structural information and optical information.

The three-dimensional face data of FIG. 5 is another example of the three-dimensional face data according to the present invention, which is a face recognition grand challenge (FRGC) database including both structural information and optical information of a face obtained by a three-dimensional laser scanner.

The two-dimensional face image of FIG. 6 may have a pose change at intervals of 15 ° from 45 ° to 45 ° to the right of the BERC (Biometrics Engineering Research Center) image in various examples. In the case of a person wearing glasses, wearing glasses It may have a pose change for each case and one without wearing.

The two-dimensional face image of FIG. 7 is an example of a Face Recognition Grand Challenge (FRGC) image, and may be input to a high resolution camera in a controlled environment or a low resolution camera in a general environment.

The face recognition method of FIG. 8 is an example of the face recognition method of the present invention in which face pose change and ambient light change are considered as internal and external environment change variables.

Specifically, the face pose calculated from the face region normalized using the feature information extracted directly from the face region in the input 2D image, the ambient light change variable (S100 to S103), and the class using the 3D face data of the corresponding user in the DB. The face pose estimated in the form and the degree of change in ambient light are corrected (S104 to S106) on the user's three-dimensional face data (S107), and the recognition is performed by comparing the face information resulting from the correction with feature information extracted directly from the input image. The result is obtained (S108).

The difference from the conventional method is that compensation for internal and external environmental changes is made in the database, not the input image.

That is, the internal and external environmental changes are made in the database of the same environment as the input image generated by reflecting the internal and external environmental change variables obtained by using feature information extracted directly from the input two-dimensional face image before compensation.

* Experimental Results and Analysis

(Reference Paper: The 20th Workshop on Image Processing and Understanding 2008.2.20-2.22. Face Recognition System Robust to Posing and Lighting Variation Yang, Uk-Il, Kwang-Hun Son Department of Electrical and Electronic Engineering, Yonsei University)

Experiment and analysis of the face recognition system of the present invention is as follows.

In this experiment, we used BERC database.

First, a BERC three-dimensional face data and openGL programming are used to generate a two-dimensional face image with a pose change of 1 ° intervals in the range of 45 ° to 45 °, and each image is one of three lighting effects: left, front, and right. Has an effect.

Next, two-dimensional face images generated in this way are classified into appropriate classes, and unique faces are obtained through Principle Components Analysis (PCA).

Then, the face is converted into a feature vector using the unique face, the vector distance between the poses is calculated, and the average vector distance between poses is calculated using this.

Table 1 shows an example of the average vector distance between poses.

ID 001 Front ° Left 5 ° 10 ° left 15 ° left .. face 0 Left 5 ° 372.63 0 10 ° left 1103.82 468.37 0 15 ° left 2598.77 1273.40 668.31 0 ..

[Table 2] is a pose class classification result obtained using the average vector distance between poses, and the pose variation in the range of 45 ° to 45 ° from left is classified into 13 classes. The lighting change is divided into three classes: left, front, and right.

class range Representative value class range Representative value face Left 8 ° ~ Right 8 ° Left 1 ~ Left 20 ° 14 ° Right 1 To right 20 ° 14 ° Left 2 ~ 26 ° left 23 ° Right 2 To 26 ° 23 ° Left 3 ~ 31 ° left 29 ° Right 3 To 31 ° 29 ° Left 4 ~ Left 36 ° 33 ° Right 4 To right 36 ° 33 ° Left 5 ~ 41 ° left 39 ° Right 5 To 41 ° 39 ° Left 6 ~ 45 ° left 43 ° Right 6 To right 45 ° 43 °

For reference, classes are divided such that the vector distance between the representative vector and the boundary vector of each class is equal to or less than a predetermined threshold. Increasing the threshold reduces the number of pose classes, and decreasing the threshold increases the number of pose classes. In the case of the right pose change, the left result is symmetrical.

Next, learn the pose classifier and the light classifier according to the class classification of [Table 2]. The experimental results for evaluating the learning results are shown in [Table 3] and [Table 4]. Table 3 shows the pose estimation, and Table 4 shows the light estimation experiment results.

Experiment 1 Experiment 2 Learning data BERC 200 (optional) x 91 Pose Changes x 3 Lighting Changes BERC 300 x 91 Pose Changes x 3 Light Changes Test data BERC 100 (rest) x 91 Pose Change x 3 Light Change BERC stereo 300 x 7 pose change accuracy 91.0% (100 Iterations Result) 96.38% (4048/4200)

Experiment 1 Experiment 2 Learning data BERC 200 (optional) x 91 Pose Changes x 3 Lighting Changes BERC 300 x 91 Pose Changes x 3 Light Changes Test data BERC 100 (rest) x 91 Pose Change x 3 Light Change FRGC two-dimensional data accuracy 99.3% (100 iterations result) 98.79% (11197/11334)

[Table 5] generates three-dimensional face data to which the degree of environmental change is applied by using three-dimensional face data and openGL programming based on the estimated pose change and lighting change variables, and generates the three-dimensional face data and the input image. This is the final recognition result.

Experiment 1 Experiment 2 Learning data BERC 3D 300 FRGC 3D 285 people x 3 data Test data BERC stereo 300 x 7 pose change FRGC two-dimensional data accuracy 95.67% (2009/2100) 92.39% (10472/11334)

1 is a block diagram of a conventional two-dimensional face recognition device

2 is a block diagram of a face recognition apparatus of the present invention

Figure 3 is a schematic diagram of a method for estimating the degree of internal and external environmental changes of the face of the present invention

Figure 4 is an illustration of three-dimensional face data in the BERC (Biometrics Engineering Research Center) DB of the present invention

5 is an exemplary diagram of three-dimensional face data using a Face Recognition Grand Challenge (FRGC) DB according to the present invention.

Figure 6 is an illustration of a two-dimensional face image of the BERC DB of the present invention

Figure 7 is an illustration of a two-dimensional face image of the FRGC DB of the present invention

8 is a flowchart of a face recognition method according to the present invention.

Explanation of symbols on the main parts of the drawings

200: face detection unit 201: feature extraction unit

202: environment change variable calculation unit 203: face correction unit

204: feature comparison unit

Claims (7)

A face detector detecting a face region in the input 2D image; A feature extractor which extracts feature information from the face region; An environment change variable calculation unit for obtaining an internal and external environment change variable of a face in a face region normalized using the feature information; A face correction unit for correcting the degree of environmental change estimated using internal and external environment change variables of the face and 3D face data of the corresponding user in the DB to the 3D face data of the user; And a feature comparator for obtaining a recognition result by comparing the face information resulting from the correction with the feature information extracted by the feature extractor. Extracting feature information from a face region in the input two-dimensional image; Estimating the degree of internal and external environmental change of the face using internal and external environmental change variables obtained from the face region normalized using the feature information and three-dimensional face data of the corresponding user in a DB; And a step of obtaining a recognition result by comparing the face information obtained by correcting the degree of internal and external environmental changes of the face to three-dimensional face data of the user and the feature information. The method of claim 2, Estimating the degree of internal and external environmental changes of the face The degree of internal and external environmental change in the unique face obtained by using the image obtained by classifying various environmentally variable two-dimensional face images generated by applying the internal and external environment change variables to the 3D face data of the corresponding user in the DB. Face recognition method for estimating. The method of claim 3, wherein The degree of internal and external environmental changes Face recognition method estimated in class format that classifies the degree of change by environment variables. Using the feature information extracted from the face region in the input 2D image, internal and external environment change variables obtained from the normalized face region are applied to the 3D face data of the corresponding user in the DB to generate various 2D face images of various environments. step; Estimating the degree of environmental change in the user's own face obtained by using the two-dimensional image obtained by classifying the plurality of two-dimensional face images by environment variables. The method of claim 5, In the step of estimating the degree of environmental change in the unique face of the user obtained by using the two-dimensional image obtained by classifying the plurality of two-dimensional face images by environmental variables A method of estimating the degree of internal and external environmental change of a face that estimates the degree of environmental change in the user's own face in a class format that classifies the degree of change by environment variables. The method of claim 6, The environment variable A method of estimating the degree of internal or external environmental change of a face, such as a face pose or an ambient light.

Images