KR20060074410A - An adaptive realtime face detecting method based on training - Google Patents

Abstract

The present invention comprises the steps of learning the features of the correlation coefficient values of horizontal / vertical size projection of the training images on the face database through Bayesian learning, and constructing weak classifiers based on rectangular features using the training image set. Learning and classifying weak classifiers to obtain an optimal strong classifier, and when the image is input, the histogram is smoothed to improve the quality of the input image for face detection and minimize the lighting effect ( Preprocessing step), using the Bayesian-learned result to detect face candidates from the input image (pre-detection step), and checking whether the input image is a face using a strong classifier obtained through the boosting to determine the optimal face area. Detection (main detection step) and same during scanning process for real time face detection Prevent duplicate detection on the face and provides an adaptive real-time face detection method based on a study wherein (a single face candidate selection stage) to remove unnecessary detection process.

Description

Adaptive real-time face detection based on learning {AN ADAPTIVE REALTIME FACE DETECTING METHOD BASED ON TRAINING}             

1 is a flowchart of pre-processing, pre-detection, and main detection of a real-time adaptive face detection method based on learning according to the present invention;

2 is a diagram showing a histogram smoothing as a pretreatment process of the present invention;

3 is a view showing size projection as a first step of face detection according to the present invention;

4 is a view showing a Gaussian distribution modeling the size projection characteristics of the face / non-face in the present invention,

5 is a view showing a rectangular feature of the main detection process of the present invention,

6 is a view showing an integrated image representation used in the main detection process of the present invention;

7 is a view showing a process of calculating a rectangular feature value of the main detection process of the present invention,

8 is a diagram illustrating a Gaussian distribution of rectangular feature values for a training image set in a main detection process of the present invention;

Figure 9 shows a block diagram of the AdaBoost algorithm in the present invention,

10 is a view showing a single face candidate selection method in the present invention,

11 is an example of detection tested through a face detection method according to the present invention;

12 is a schematic diagram of a face detection method according to the present invention.

 The present invention relates to a method for detecting a face region in an image as a first step of developing a facial recognition method for detecting a face and recognizing a facial expression in a still image and a moving image. The face detection method using a correlation coefficient of size projection and A real-time face detection method based on learning using face detection method using AdaBoost (Adaptive Boosting) algorithm based on the rectangular feature proposed by Viola and Jones.

With the development of information and communication technology, the information digital society is coming due to the emergence of information appliances that can connect to the Internet and transmit and receive data. As home networks are being built in homes, research into the distribution of digital appliances such as digital door phones, facial expression recognition systems, and user recognition / adaptation HCI systems are being actively conducted. Accurate face detection is a prerequisite for facial recognition. However, detecting faces accurately from still images and moving images is no easy task. This is because various face sizes and the presence of structural elements such as beards and glasses, facial posture, changes in illumination, camera characteristics, and facial expressions are difficult to detect.

There are four types of face detection methods in such an image. The first method is a knowledge-based method, which is based on the general knowledge of a typical face. The position, distance, etc. between eyes, nose, and mouth constituting the face correspond to this method, which was widely used for initial face detection. However, this method has a disadvantage in that it is difficult to apply fixed rules to various face postures of humans.

The second method is an invariant feature approach, which is mainly used for face localization to find structural features that are easy to detect face even in posture or lighting.

The third method is a template matching method. The basic shape of some faces is stored according to the whole and part of the face, and it is commonly used as a method of judging faces when there is a certain level of correlation through comparison with the input image. Way.

Finally, there is an appearance-based method. Unlike the template matching method, a method of learning a template or a face model from training images having a representative variety of faces has recently been attracting the most attention, and the detection performance is also excellent.

The present invention is to overcome the limitations of the accuracy and detection time of the existing face detection methods, the face detection method using the correlation coefficient of the size projection in still / movie and the face using the AdaBoost algorithm based on the rectangular feature The object of the present invention is to provide an adaptive real-time face detection method based on the learning method.

The inventors of the present application have recognized that in producing a training face image for learning, it is important to obtain a selected face DB that expresses a variety of skin colors, facial shapes, or facial elements. Accordingly, the present invention builds a training face image from the CMU PIE database, the BioID database, and the Slovenian CVL database, so that the learning algorithm can discover common and unique characteristics of the face, and pre-process various lighting and camera performances. Accurate face detection was made regardless of changes in the environment.

In the present invention, a face is accurately detected in real time by using a discriminant using a correlation coefficient of size projection in a still image and an AdaBoost (Adaptive Boosting) algorithm based on a rectangular feature.

The face detection method according to the present invention uses a single face candidate selection method in the post-processing process to prevent the occurrence of duplicate detection results on the face and to minimize the increase in the detection time due to unnecessary scanning, thereby enabling real-time face detection. do.

The present invention includes a preprocessing process for an input image, a line detection process using a correlation coefficient value of size projection, and a main detection process using an AdaBoost algorithm based on a rectangular feature.

In the present invention, the preprocessing is a process of performing histogram smoothing to prevent detection errors due to illumination differences in the face image, and the predetection process is performed after projecting the preprocessed sub-image onto the size projection plane. The main detection process is a process of detecting a face by constructing a strong classifier by combining the rectangular features with a weak classifier and combining them through the AdaBoost algorithm.

In addition, the pre-detection process of the present invention obtains a correlation value between horizontal / vertical size projection of each training image set and horizontal / vertical size projection of an average image, and constructs a face detection discrimination function through Gaussian modeling of these values. Therefore, it is characterized by preventing excessive detection that may occur in the main detection process using AdaBoost.

In addition, the main detection process of the present invention is made through a strong classifier composed of a combination of weak classifier, characterized by achieving a high detection rate corresponding to 98% by accurate classification through reliability prediction.

(Example)

Hereinafter, with reference to the accompanying drawings will be described a real-time adaptive face detection method based on the learning of the present invention.

1 is a flowchart of a method for real-time adaptive face detection based on learning according to the present invention, which comprises a preprocessing process of converting an input image from a camera into a search window program by 24 × 24 pixels and smoothing histogram. The pre-detection step of mapping the processed image to the size projection plane and detecting using the correlation coefficient value, and the integrative image to efficiently calculate the total number of possible rectangular features in the training image of 24 × 24 pixel size The main detection step of detecting the image by performing boosting is shown.

Hereinafter, each process will be described in detail.

1. Pretreatment

2 shows a pretreatment process by smoothing. Reference numeral 201 denotes an original image, and reference numeral 202 denotes a result of performing histogram smoothing on the original image 201. One difficulty in face detection is the change in illumination. As the lighting changes, the face is markedly different, which directly affects the detection performance.

In the present invention, histogram smoothing is applied as a preprocessing process in order to minimize the effects of such illumination changes. Accordingly, as shown in FIG. 2, the face detection error factor in the subsequent process according to the light state of the original image is previously removed or It can be minimized.

2. Line Detection Process Using Amplitude Projection

In the present invention, Bayseian learning is performed using the correlation coefficient values of the size projection in the vertical and horizontal directions of the image, and the method of pre-detecting the face is used using the result. This method can be regarded as the preprocessing of the main detection process that can effectively compensate for the shortcomings of the simple rectangular features that constitute the weak classifier of the AdaBoost algorithm. When detecting the face, the correlation function of the size projection is applied to the discrimination function to check whether the face is a face or not. Although the accuracy is lower than the detection method using the AdaBoost algorithm, which is the main detection process, the training time and detection time are relatively fast. Using this point, the present invention uses the correlation coefficient value of size projection as the previous step of the main detection process to improve the speed and accuracy of the face detection system.

3 shows the size projection of the face image in the horizontal / vertical direction. Reference numeral 301 denotes an original image, 302 shows a horizontal size projection, and 303 shows a vertical size projection. It is understood from the picture projecting the image in the horizontal and vertical directions that the gray portion value of the nose part is larger than the other parts, which could not be measured by the face image itself.

Training of the pre-detection process using the size projection in the present invention is carried out as follows.

The horizontal and vertical size projections of each training image set are obtained, and the horizontal and vertical size projections of the mean face images are obtained. The mutual coefficient values ρ are calculated as shown in Equation 1.

In order to model the size projection of a typical face, the present invention assumes a two-dimensional Gaussian distribution of correlation values. That is, the correlation coefficients in the vertical direction and the horizontal direction form a two-dimensional vector, and Gaussian modeling of the general face size projection characteristics using the calculated correlation coefficients between the training image set and the input image. Gaussian modeling is the process of obtaining the mean and covariance matrix needed to construct a two-dimensional Gaussian function. 4 graphically illustrates this Gaussian distribution.

When the parameters for the Gaussian model are obtained, the horizontal and vertical size projection values of the input image are obtained, and the correlation coefficient with the average image is obtained. When the correlation value is applied to the Gaussian function for the face and non-face already modeled, it is classified as a higher value class. In this case, a discrimination function derived from a Gaussian function is used to facilitate the calculation. The discrimination function for each class is shown in Equation (2).

The discrimination function is applied in this way to remove images that are not likely to be faces and improve the accuracy and speed of face detection. The discrimination function is applied as follows.

Equation (3) is the discrimination function of the face class.

Equation (4) is the discriminant function of the non-face class.

Equation (5) indicates that the discrimination function value of the face class is larger than the discrimination function value of the non-face class.

If the discrimination function value g _{Negative of the} face class is larger than the discrimination function value g _Positive of the non-face class, it is determined as a face candidate.

In addition, the magnitude of the difference between the two expressions represents the confidence of the face, and the larger the value, the greater the probability of the face and the smaller the probability of the face. It is responsible for eliminating or minimizing the over-detection that may occur in the main detection stage based on the fast detection time.

3. Main Detection Process Based on Rectangular Features and Using AdaBoost Algorithm

We use the learning-based AdaBoost algorithm for accurate face detection. Weak classifiers for application to the AdaBoost algorithm can use facial subclassifiers. By combining the classifiers for parts of the face through the AdaBoost algorithm, we can create a strong classifier for the whole face. The rectangle applied in the present invention satisfies this requirement well.

The rectangular feature is simple and easy to calculate due to the difference of the sum of the pixel values of adjacent rectangular regions. The rectangular features applied in the face detection method of the main detection process are three as shown in Fig. 5, two rectangular features (Fig. 5 (a)) and three rectangular features (Fig. 5 (b)) Consists of horizontally and vertically adjacent rectangles with the same shape and the same area.

In the present invention, a 24 × 24 pixel size training image is used, and the total number of possible rectangular features in a 24 × 24 pixel size is very large, and an integrated image representation method is used to efficiently calculate each of these features. Integral images can be easily implemented with a small amount of computation per pixel, and the rectangular feature values can be quickly calculated when the integral image is obtained.

As shown in FIG. 3, the three features used in the present invention are two simple (vertical, vertical) rectangular features among the five simple rectangular features proposed by Viola and Jones (horizontal and vertical). ) And three rectangular features (Fig. 5 (b)). The two rectangular feature values are the difference of the sum of the pixels within the two rectangles. The three rectangular feature values are the difference of the sum of the pixel values of the outer rectangles in the sum of the pixel values within the center rectangle. Here, all feature values are kept absolute so that feature values do not have negative real numbers. In a 24 x 24 pixel image, two rectangular features consist of 86,400 horizontal and vertical combinations and 27,600 three rectangular features.

The integrated image is an image obtained by adding pixel values in the diagonal direction to the right with respect to the original pixel of the original image. The integrated image is shown in FIG. 6. Integral images are obtained from equation (7). FIG. 6 illustrates the conversion of an integral image of a Lena image, and FIG. 7 illustrates a calculation formula of rectangular feature values in the integrated image.

In the present invention, by applying the AdaBoost algorithm effectively to the weak classifier according to the rectangular feature, various types of crystal boundaries are required for the classification of complex classes in the face. The training image set that is input to the AdaBoost algorithm is expressed as Sm = (x1, y1), .. (xn, yn), and the image that is the face when yi = 1 for each xi, and when yi = 0. Represents an image rather than a face. The AdaBoost algorithm repeatedly calls the learner during t = 1, ..., T rounds to select a weak classifier and assigns a weight to the classification performance. In the face detection method of the present invention, the learner is configured to select a weak classifier having the least error for the training image set in each round. The weak classifier is constructed as shown in equation (8) using the rectangular feature.

If the j th classifier is h _j (x) for all possible weak classifiers for the training image, it is judged to be a face if the rectangular feature value f _j (x) is greater than or less than a certain value using the threshold θ _j and the extreme value p _j . Done. At this time, the sign and the threshold value can be obtained by assuming Gaussian distribution of the rectangular feature values for the training image set as shown in FIG. 8.

If the weak classifier is constructed using training image sets, the AdaBoost algorithm works as follows. First, a series of weights (distributions) are assigned to the entire training image. The weight of the i-th training image in the t-th round is defined as Wt, i. In the initial stage, the weights of all the training images are equally given. The value is determined by Eq. (9) by the size m of the face training image set and the size n of the non-face training image set constituting the class.

The input weight is normalized as shown in equation (10) every round.

0)

0)

The learner calculates the error ε for every weak classifier possible every round. The learner's task is to find a weak classifier that minimizes the error ε and weight it according to the classification performance. The error ε is determined as shown in equation (11).

If the weak classifier h with the minimum error ε is selected, the weight for the training image set is updated as shown in equation (12).

According to Equation (12), the new weight of the i th data is reduced when the weak classifier makes the right decision and remains the same when the wrong judgment is made. Since the updated weights are normalized again, the effect of this update process is to select a new weak classifier that best classifies the images that were difficult to classify by increasing the weights of the images that the classifier determined at the repetition t did not classify well.

Table 1 below shows the weighting methods of the two final classifiers.

Table 1

The strong classifier H for face detection depends on the weight determination method. The weight determination method can be divided into two cases as shown in Table 1. Case 1 is a case where the same weight is assigned to all classifiers of each training image set. Since the characteristics of the training image sets do not affect the composition of the classifier, performance can be guaranteed only when the characteristics of each training image set are similar. Case 2 is a method of determining weights according to the classification performance of the training image set. The method is characterized because it depends only on the weight classification performance of the training image set with an H _t selected from a training image category is easy to set a high weight.

In the present invention, by adopting the method as in case 2, we build a highly accurate face detection system based on reliability by combining the classifiers and generating the final classifier H based on the weighted majority voting method. .

Table 2 below summarizes the generalized AdaBoost algorithm presented by Schapire and Freund and FIG. 9 shows a block diagram of the AdaBoost algorithm.

TABLE 2

4. Single Face Candidate Selection Method

The present invention solves the problem of lowering the accuracy of the detector and slowing down the speed by detecting incorrectly around the face during face detection in the main detection process using the line detection process using the size projection and the strong classifier generated by the AdaBoost algorithm. To do this, the single face candidate selection method is used in the scanning process. The reliability of each sub-image is examined by moving the sub-window by 1 pixel size up to 7 pixels to the right and 7 pixels to the bottom from the point where the face is first determined to be a face. During the investigation, if there is a point with higher reliability than the point that was first identified as a face, the part is classified as a face. Otherwise, the face first identified as the face will be the face. That is, by searching for the sub-window with the highest reliability among the sub-windows, only one sub-window corresponding to the face is detected as the face among the sub-windows classified as the face around the face. Subsequently, the subwindow region detected as a face is skipped without performing scanning. After that, existing scanning is continued to the lower right corner of the input image to search for another face. If the part determined to be a face is detected again, the previous process is once again detected with the face having the highest reliability.

In the present invention, applying a single face candidate selection method in face detection prevents over-detection in the part where the face actually exists in the test image, thereby improving detection accuracy of the face detection method and not scanning the area detected as face. By not doing this, the scanning of all the pixels is not performed, which makes it possible to speed up the face detection system.

10 shows a single face candidate selection method.

The face detection system of the present invention can be divided into four processes.

First, as a preprocessing process, it is a part that performs histogram smoothing on the input image set. Histogram smoothing is performed to improve the image quality of the input image for face detection and training image set composed from DB with different lighting conditions and to minimize the lighting effect.

Second, as the first process of face detection, this study learns the features of the correlation coefficients of horizontal / vertical size projection of training image sets through Bayesian learning and uses this feature as an element of face candidate detection. The training is done offline and the Gaussian parameter values produced by the training are saved to a file and called by the detector.

Third, as a key step in face detection, face detection is performed by applying the AdaBoost algorithm. As a weak classifier with rectangular features, this is the part that finally constructs an optimal strong classifier that can detect the face through training and boosting the training image set and finally checks whether it is a face or not. As with size projection, boosting-related parameter values from training are stored in a file and called by the detector. Due to the nature of the AdaBoost algorithm, the classifier's training time is longer than other face detection algorithms, but once the training is completed, real-time face detection is possible.

Fourth, a single face candidate selection method is used in the scanning process in order to solve the problem of lowering the accuracy of the detector and delaying the speed by erroneous detection around the face during face detection in the two face detection processes. This makes it possible to increase the speed of the face detection system as an additional process for real time face detection.

11 is an example of detection tested through the face detection method described so far.

12 is a schematic diagram of a face detection method according to the present invention.

In order to train and test the face detection method presented in the present invention, the training image set is largely composed of the face training image set and the non-face training image set, respectively, obtained through the face database and the website, and the test images are 300 images in total. The experiment was performed by constructing a single face + multiple face into a set of test images.

In the case of the face training image set, the training image set was constructed based on the front face image. The face training image set was selected by selecting 175 from the CMU PIE database, 130 from the BioID database, and 171 from the Slovenian CVL database. The non-face training image set was extracted from natural images that were quite large in size without any face obtained from the website. We randomly constructed 10000 non-facial images from 30 large images, and 2000 non-face training images were selected from among them.

Since the file size and format of the three face databases are different, the size of both face and non-face image sets is limited to 24 x 24 pixel size to suit the learning conditions. Saved to file Experimental results show that the detection rate is significantly improved by detecting more than 98% in the proposed face detection system, which includes severely changed and partially covered faces, glasses and slanted faces.

The present invention accurately detects faces in real time using AdaBoost (Adaptive Boosting) algorithm based on a discriminant equation and a rectangular feature based on Bayesian learning using cross-correlation coefficient values of size projection in still images.

In the present invention, first, the AdaBoost algorithm based on rectangular features combines weak classifiers and creates a final classifier to build an accurate face detection system based on learning. Second, the face correlation system speed and accuracy are improved by using the cross-correlation coefficient value of the learning-based size projection in all stages of the main detection process. Third, by using a single face candidate selection method, real-time face detection is enabled, and histogram smoothing is used to eliminate or minimize face detection error factors in the subsequent process according to the light condition of the original image.

Therefore, the face detection method of the present invention is very useful for constructing a security system such as an unlocking function or performing entertainment such as four weeks, affinity, astrology, coronary and Avatar, especially in a mobile communication terminal environment such as a mobile phone or a PDA. Can be used.

Claims (11)

Learning through the Bayesian learning the characteristics of the correlation coefficient values of the horizontal / vertical size projection of the training image set on the database having the training image set consisting of face images and non-face images; It consists of training image sets to construct weak classifiers based on rectangular features and to boost and classify weak classifiers in order to obtain an optimal strong classifier. Once the video is input, Improve histogram quality for face detection and smooth histogram to minimize lighting effects (preprocessing step), Detecting a face candidate from the input image using the Bayesian learned result (pre-detection step), Using the strong classifier obtained through the boosting step to detect whether or not the input image is a face to detect an optimal face area (main detection step), An adaptive real-time face detection method based on learning, characterized by preventing duplicate detection of the same face and removing unnecessary detection processes (single face candidate selection step) for real time face detection. The method of claim 1, In the learning step, for the pre-detection step, to obtain the horizontal and vertical size projection of each training image, Obtain the horizontal and vertical size projection of the mean face image, The intercoefficient values (ρ) between them are

Face detection method characterized in that obtained. The method of claim 2, The detected correlation coefficient values are Gaussian modeled, And a discrimination function for pre-detecting a face candidate.  The method of claim 3, The determination function g (x) is,

Face detection method characterized in that.  The method of claim 4, wherein The determination function g (x) is,

Is determined to be a face, where g _Positive (x) is the discriminating function of the face class and g _Negative (x) is the discriminating function of the non-face class. The face detection method is characterized in that the larger the difference between the two expressions, the greater the reliability of face discrimination. The method of claim 1, In the boosting step, rectangular features are calculated using an integrated image representation method, Construct a weak classifier with the calculated rectangular features using the training image set, Learning the configured weak classifiers on the training image set to find the weak classifiers h that well reflect the feature of the face; Combining the found weak classifiers according to weighted majority voting and obtaining an optimal strong classifier H. The method of claim 6, wherein the rectangular feature value in the integral image,

Face detection method characterized in that is calculated as. The method of claim 6, wherein using the rectangular feature value, Weak classifier

(Where h _j (x) is the j th classifier for all possible weak classifiers for the training image, θ _j is the threshold value, p _j is the extreme value, f _j (x) is the feature value of the rectangle), A face detection method comprising determining that a rectangular feature value f _j (x) is larger or smaller than a predetermined value using a threshold value θ _j and an extreme value p _j . 7. The method of claim 6, wherein finding a weak classifier h that well reflects facial features, A series of weights are assigned to the entire training image according to the following equation (the weight for the i-th training image in the t-th round is Wt (i), m is the size of the face training image collection, and n is the size),

The input weight is normalized every round as the following formula,

For every possible weak classifier every round, the error ε is calculated according to

Find a weak classifier h that minimizes ε, If the weak classifier h with the minimum error ε is selected, the weight for the training image set is updated as shown below.

And the updated weights are normalized again. The method of claim 6, When the weak classifier is learned in the learning process, the ratio of the face training image set to the non-face training image set is 1: 2 and the number of boosting times is 200 times. The method of claim 1, In the single face candidate selection step, the reliability of each sub-image is examined while moving the sub-window by 1 pixel size up to 7 pixels to the right and 7 pixels to the bottom from the point where the face is first determined for the real-time face detection. A face detection method characterized by improving the detection speed and preventing duplicate detection of the same face by selecting a portion having the highest reliability as a face and not scanning the area selected as the face.

Images