KR101415325B1 - Apparatus for detecting face - Google Patents

Abstract

The present invention relates to a face detection apparatus for obtaining an integral image and an edge image from an input image, extracting Hal-like feature information and EOH feature information therefrom, and combining the extracted information to detect a face. The face detection apparatus generates an integral image and an edge image from an input image, extracts Haar-like feature information from the integral image using pre-stored training data, and extracts Edge of Orientation (EOH) from the edge image. Feature information extracting means for extracting feature information from the histogram feature information; And face detecting means for detecting a face by applying an AdaBoost algorithm to the combined no-like feature information and EOH feature information.

Hal-like feature extraction, EOH feature extraction, feature value combination, AdaBoost algorithm

Description

[0001] Apparatus for detecting face [0002]

The present invention relates to an image processing apparatus and method, which obtains an integral image and an edge image from an input image, extracts Haar-like feature information and Edge of Orientation Histogram (EOH) feature information therefrom, combines extracted information, ≪ / RTI >

In digital image processing technology, there are various methods of face recognition such as knowledge-based method, feature-based method, template matching method, and appearance-based method.

The knowledge-based method is based on the assumption that the human face includes eyebrows, eyes, nose, mouth, etc., and each face component has a certain distance and positional relationship with each other. to be.

The feature-based method is a method of detecting a face by using the size, shape, correlation, facial color and texture information of the facial feature components, or mixed type information of the facial feature components.

The template matching method is a method of manually creating a standard face pattern for all the target faces, and then comparing the face with the input image.

The appearance-based method is a method of detecting a face using a model created through a training process by receiving various images.

Such conventional face recognition methods detect a face using only one type of feature, for example, an intensity component or an edge component, in order to extract feature values for face detection. When the face is detected using the intensity component, if the illumination is non-uniform at the time of detecting the face, there is a problem that the detection performance is deteriorated because it affects the intensity of the image. In the case of detecting a face using an edge component, edge information is lost in a low-resolution environment, so that facial feature components are reduced and face detection performance is deteriorated.

The technical problem to be solved by the present invention is to obtain an integral image and an edge image from an input image, extract Ha-like feature information and Edge of Orientation Histogram (EOH) feature information from the input image, Thereby detecting a face.

According to an aspect of the present invention, there is provided a face detection apparatus for generating an integral image and an edge image from an input image, and generating Haar-like feature information from the integral image using pre- Extracting feature information extracting means for extracting Edge of Orientation Histogram (EOH) feature information from the edge image; And face detecting means for detecting a face by applying an AdaBoost algorithm to the combined no-like feature information and EOH feature information.

The apparatus of claim 1, wherein the feature information extracting unit comprises: an integral image generating unit for generating an integral image by integrating the input image; An edge image generating unit for generating an edge image from the input image and obtaining an intensity in each edge direction from the edge image; Likelihood feature extracting unit for calculating a difference value between a sum of a bright area pixel and a sum of a dark area pixel in a designated sub window from the input image integrated using the learning data; And an EOH feature extraction unit for extracting a ratio value between the intensities in the edge directions in the subwindow and a charge ratio value of the intensities in each edge direction using the learning data.

In the present invention, the above-mentioned do-by-like feature calculating section may classify the type of the prototype as a first type which is composed of four regions and in which bright regions and dark regions are symmetrically diagonal to each other, A second type in which a bright region is located on the upper and lower sides, a third type in which a bright region and a dark region are located in the left and right, and a fourth type in which a dark region, Like feature information of the first to fourth prototypes in the sub-window.

In the present invention, the edge image generation unit may include a conversion unit that converts the input image into a monochrome image; A first calculating unit for calculating a hardness for each coordinate in the horizontal / vertical direction by applying a Sobel mask to the monochrome image and calculating the size of the edge from the hardness; A second calculation unit for calculating an edge strength of each coordinate by summing the squares of the edge sizes in the horizontal / vertical directions of the respective coordinates; A noise eliminator for eliminating noise by setting an allowable value for the edge intensity; A third calculating unit for calculating an edge direction of each coordinate from a size of the edge in the horizontal / vertical direction; And a fourth calculation unit for quantizing the edge directions in K directions, collecting the edge values of the coordinates having the directionality in each direction, and calculating an edge image for the K edge directions.

In the present invention, the EOH characteristic calculating unit may set the edge direction to be horizontal, vertical, and first and second diagonal directions.

In the present invention, the EOH feature calculating section calculates the EOH feature value in the sub-window in the edge direction in the horizontal direction, the horizontal direction and the vertical direction, the horizontal direction problem 2 diagonal direction, the first diagonal direction and the vertical direction, The ratio values between the intensities in the diagonal direction and the vertical direction and the diagonal directions in the vertical direction can be extracted and the charge ratio values in the horizontal direction, the first diagonal direction, the vertical direction, and the second diagonal direction, which are the edge directions, can be extracted.

As described above, according to the present invention, by using the characteristics of intensity and edge at the time of face detection, the performance against the general environment is not lower than that of the conventional face detection technology, and at the same time, So that it has a robust performance.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a configuration of a face detection apparatus according to the present invention.

The face detection apparatus shown in FIG. 1 includes a feature information extraction unit 110, a storage unit 120, an information combination unit 130, and a face detection unit 140. The feature information extracting unit 110 includes an integral image generating unit 111, a likelihood feature extracting unit 112, an edge image generating unit 113, and an EOH feature extracting unit 114.

The feature information extraction means 110 extracts the do-or-like feature information from the integral image using the learning data stored in the storage means 120, and outputs the feature information to the storage means 120 EOH feature information is extracted from the edge image using the stored training data. The learning data stored in the storage means 120 will be described later.

The integral image generating unit 111 obtains and generates all the pixel sum of the input image. Likelihood feature values are obtained in a state in which the pixel sum of the image is obtained in advance. Therefore, the feature value can be obtained using only some pixel values without adding all the pixels in the rectangular area. Equation (1) is a formula for converting the input image i into the integral image ii. In Equation (1), x, y are the positions of the pixels, and x ', y' are the positions of the previous pixels.

When the image is transformed using the formula of the integral image, it can be obtained by the following equation (2). In the equation (2), s (x, y) is the sum of the pixel values and the initial value (the first value of the calculation) is s (x, -1) = 0 and ii (-1, y) = 0 in the calculation.

Assuming that there is a 4x4 image as shown in FIG. 2, if FIG. 2A is an intensity value of the original image, the intensity value of the transformed image is as shown in FIG. FIGS. 2C and 2D are actual images and results obtained by integrating them and normalizing them to 255. FIG. The sum of the pixel values within a desired interval after the image is integrated is a pixel value at a position where the coordinates of the corresponding rectangle are located, as shown in FIG.

Likelike feature of the subwindow from the integrated input image using the learning data (the position, size, and prototype type of the subwindow) stored in the storage unit 120, And subtracts the difference between the bright region pixel sum and the dark region pixel sum of the prototype in the corresponding sub window.

In order to extract Hal-like features, we next describe the features of Hal-like.

Face images have a unique pattern of faces, and classify differences between faces and other objects using unique patterns. As shown in Fig. 3A, the reason why human beings classify the contents of a video as a face is that the face of a real person, the shape of a toy face, the shape of a rock face, and a unique pattern of a face such as eyes, I have. When a pattern is obtained using a color image, the calculation complexity increases according to the color of the pattern. Therefore, the pattern can be converted into a monochrome image, and the pixels can be easily classified into luminance value moods.

The Hal-like features are proposed by Papageorgiou for Haar-wavelet detection, and P. Viola and M. Jones used features similar to wavelet-like features for object detection. The Hal-like has prototypes like Figure 3b. In this prototype, the lower part of the face image is regarded as a black rectangle and the higher part is regarded as a white rectangle, and the value obtained at that position is used as a pattern for classifying the face and the non-face (FIG.

The calculation of the Hal-like feature value is calculated by subtracting the sum (b) of the pixel values in the rectangle of the dark region from the sum (a) of the pixel values in the rectangle of the bright region in the prototype as shown in FIG. Equation (3) is an equation for calculating a hall-likelihood characteristic value.

Due to the simplicity of the calculation method, the feature value calculation speed is faster than other features. The object operation method is simple in concept, but it has a calculation amount of 0 (n2) (horizontal operation * vertical operation) since the values of all pixels of the subwindow must be added for the actual operation. This is a factor that has a critical effect on the detection speed, and in order to improve the operation speed, the image to be detected is converted and processed using the integral imaging method.

The likelihood feature extractor 112 extracts the highlight area of the prototype from the integrated input image using the learning data (the position, size, and prototype type of the sub window) stored in the storage unit 120, And a difference value between the pixel sum of the dark region and the pixel sum of the dark region.

As an example, when there is a prototype as shown in FIG. 3E, it is possible to easily perform calculation using the six pixel values of the portion converted into the integral image, and the equation for calculating the hall- .

The edge image generating unit 113 converts an input image into a monochrome image, obtains an edge image using a Sobel mask, and obtains an intensity in each edge direction from the obtained edge image.

Several steps of preprocessing are required to generate an edge image. A sobel mask (not shown) is used to simply and efficiently extract the edge information distributed in the image. First, since the color component of the image does not affect the edge information extraction, the color image is converted into a monochrome image as shown in FIG. 4A.

And the hardness of each coordinate of the original image is obtained by using the Sobel mask on the converted monochrome image. 4B is an edge image obtained by using a horizontal direction Sobel mask on a monochrome image, and FIG. 4C is an image obtained by using a vertical direction Sobel mask on a monochrome image. Equation 5-1 discloses a mathematical expression for obtaining the size of an edge using a horizontal direction Sobel mask, and Equation 5-2 discloses a mathematical expression for obtaining the size of an edge using a vertical direction Sobel mask.

In Equation 5 I is the original image, G _x, G _y is horizontal / the edge image in the vertical direction, Sobel _x, Sobel _y are (x, y) horizontal / vertical direction Sobel mask, I determined using a Sobel mask circle G _x (x, y) and G _y (x, y) represent the pixel values of the (x, y) point in the image obtained by the horizontal / , I.e., the size of the edge in the horizontal / vertical direction.

The edge intensity of each coordinate can be obtained by using the edge size in the horizontal / vertical direction in the (x, y) coordinates of the image obtained through Equation (5), and the edge intensity of each coordinate can be obtained as shown in Equation (6) Respectively. In Equation (6), G denotes an image having an edge strength value.

However, since the edge strength of each of the obtained coordinates is small, the contribution to the edge characteristic is degraded, so that the allowable value can be set and the lower values can be ignored. The allowable value to be used must be set manually by the user, for example, 80. In Equation (7), an equation for judging the edge strength with a small contribution to the edge characteristic as noise is shown. In FIG. 4E, the edge intensity with a low contribution to the edge characteristic is judged as noise and the edge intensity image from which the noise component is removed Respectively. In Equation (7), T denotes an allowable value, and G 'denotes an edge image from which noise components are removed.

The edge direction of each coordinate can be obtained by using the horizontal / vertical direction edge size of the (x, y) coordinates of the image obtained through Equation (7), and the edge direction of each coordinate as shown in Equation (8) Respectively. In Equation (8),? Denotes an image having an edge direction value.

An edge image along the K edge directions can be obtained by quantizing the edge directions obtained through Equation 8 into K directions and collecting edge values of the coordinates having the direction with respect to each direction, And an edge image along the edge direction is shown in Fig. 4G. In Equation (9), bin _k denotes an edge-size image having only a k-th edge direction component and ψ _k denotes an edge-direction image component having only a k-th edge component.

In the present invention, the edge direction is defined as a horizontal direction, a 45-degree direction, a vertical direction, and a 135-degree direction as shown in Figs. 4H to 4K.

The EOH feature extraction unit 114 extracts The edge intensity magnitudes for each edge direction in the subwindow are all summed up using the learning data (position, size, and prototype type of the subwindow) stored in the storage means 120 to obtain a scalar intensity value And the ratio between the intensities in the edge directions in the sub window and the charge ratios of the intensities in each edge direction are extracted.

The EOH feature is an edge-based feature that utilizes features for edge components that appear locally in the face. In general, people's faces have some edge-based common elements. 5A). The ratio of the horizontal component to the vertical component in the face is not infinitely divergent but has a certain degree of closed value (FIG. 5B) . In addition, edge directions are distributed locally according to positions in the face. FIG. 5C is a view showing the periphery of the eye in which a large number of edges are distributed, and FIG. 5D is a view of the periphery of the jaw where a large number of diagonal edges are distributed. Such an edge property does not change with the overall illumination change, and it is advantageous to extract a geometric characteristic that is hard to extract from a characteristic based on a pixel value.

The EOH feature extraction unit 114 can extract the EOH feature information using intensity values in each edge direction generated by the edge image generation unit 113. [ The edge intensity magnitudes for each edge direction in the sub-window of the image may be summed to obtain a scalar intensity value using equation (10). In Equation (10), R denotes a sub-window in an image, and E _k (R) denotes an intensity in a k-th edge direction with respect to a sub-window R. FIG. 5E shows that the eye periphery is taken as a subwindow, and FIG. 5F shows intensity values for 0 degree (horizontal), 45 degree, 90 degree (vertical), and 135 degree edge direction components in the sub window R.

The ratio between the intensities in the edge directions extracted in the sub window can be obtained as Equation (11) and used as the characteristic value A. A constant ε is added to the denominator and the numerator of the equation (11) so that the value of A can be distributed in a somewhat stable range. When the edge direction is K, the total _K C ₂ feature values can be obtained. FIG. 5G shows the ratio (relative ratio) between the intensities in the edge directions in the sub window R, with the feature value A obtained using the values in FIG. 5F.

The ratio of the intensity of each edge direction drawn in the sub window to the total edge intensity in the sub window can be calculated as shown in Equation (12) and used as the characteristic value B. Similarly, a constant ε is added to the denominator and the numerator of the equation so that the value of B can be distributed in a somewhat stable range. When the divided edge direction is K, the feature value of the total K can be obtained. FIG. 5H shows a characteristic ratio B obtained by using the values in FIG. 5F, and shows a charge ratio (absolute ratio) of intensity in each edge direction in the sub window R. FIG.

The information combination means 130 is a combination of the So-and-liked features (the difference between the pixel sum of the bright region and the pixel sum of the dark region in the prototype) extracted in the subwindow by the learning data and the EOH characteristic (A value) and the charge ratio (B value) of the intensity in each edge direction are combined. 6 shows information obtained from one subwindow. First, the EOH feature values (10 in total: r ₁ -r ₁₀ ) for intensity (1, 45, 90, 135) Like feature values (four in total: h ₁ -h ₄ ) for the _four prototypes in the corresponding subwindow.

The face detection unit 130 detects the face using the feature selected through the cascaded AdaBoost algorithm on the combined hash-like feature value h ₁ -h ₄ and the EOH feature value r ₁ -r ₁₀ do. FIG. 7 is a flowchart illustrating a method of classifying faces using a detector manufactured by a cascaded AdaBoost algorithm. Adaboost is an algorithm that selects weak classifiers with low error by iterative operation from data and creates a strong classifier with high accuracy through linear combination of these classifiers. The AdaBoost learning algorithm is the most well known boost algorithm and has the advantage of being simple and efficient. Referring to FIG. 7, the cascaded AdaBoost learning algorithm calculates the Hall-likelihood values h ₁ -h ₄ and the EOH feature values r ₁ -r ₁₀ extracted from the designated subwindows at step 720, 730, 740, 750) to determine whether the designated sub-window is a face. As each step passes, the sub window is classified as a face through a classifier with better performance. With this method, a faster detection algorithm can be implemented. In the first step 720, a classifier is manufactured with a small number of feature values. In the next step 730, a more accurate classifier is created than the previous step with a larger number of feature values than in the first step 720. Since most of the input image occupies the non-face part, most of the non-face is filtered at the initial stage as the area passes through each step of the detector, so that a high-speed detector can be manufactured.

8 is a diagram for explaining generation of learning data stored in the storage means 120. Fig.

As shown in FIG. 8, the learning data is generated by the learning classifier 800 and then stored in the storage means 120. FIG. The learning classifier 800 includes an integral image generating unit 810, an edge image generating unit 820, a likelike feature extracting unit 830, an EOH feature extracting unit 840, and an adaboard learning unit 850 .

The training data image input to the integral image generating unit 810 and the edge image generating unit 820 is a gray image having a predetermined size (for example, 16 × 16), which can determine whether or not the face and the face are non-face images.

The integral image generator 810 obtains and generates all the pixel sum of the input learning data image. Detailed description of the generation of the integral image is omitted since it is described above.

The edge image generating unit 820 obtains an edge image using the Sobel mask on the input learning data image, and obtains the intensity in each edge direction from the obtained edge image. The detailed description of the edge image generation has been given above and will be omitted here.

The likelike feature extraction unit 830 extracts difference values between the sum of the bright region pixels and the sum of the dark region pixels of the various types of prototypes in the subwindow from the integrated learning data image.

The EOH feature extraction unit 840 extracts Of the edge image The edge intensity magnitudes for each edge direction in the subwindow are all summed to extract intensity values in the form of a single scalar and the ratio between the intensities in each edge direction in the subwindow and the ratio of the intensity in each edge direction is extracted do.

The AdaBoost learning unit 850 applies the AdaBoost algorithm to the Hal-like feature value and the EOH feature value to determine the position, size, and feature type of the sub window for extracting the optimal do-like feature value and the EOH feature value And stores it in the storage means 120 as learning result data.

The learning result data stored in the storage means 120 indicates the positions, sizes, and types of sub windows for extracting the Hal-like feature and EOH feature extraction in FIG. 1, and constitutes a detector constituting each step of FIG. 7 .

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a block diagram showing a configuration of a face detection apparatus according to the present invention.

FIG. 2 is a view for explaining generation of an integral image in FIG.

FIG. 3 is a diagram for explaining Haar-Like feature extraction in FIG.

FIG. 4 is a view for explaining edge image generation in FIG.

FIG. 5 is a view for explaining EOH feature extraction in FIG. 1; FIG.

FIG. 6 is a diagram illustrating Haar-Like and EOH feature combinations extracted in FIG. 1; FIG.

FIG. 7 is a view for explaining the Adaboost algorithm for face detection in FIG.

FIG. 8 is a diagram for explaining generation of learning data stored in the storage means in FIG.

Claims (6)

Extracts haar-like feature information from the integral image using the pre-stored training data, and generates EOH (edge of histogram) feature information from the edge image Feature information extracting means for extracting feature information; Information combining means for combining the above-described Likelihood feature information and the EOH feature information and outputting a combination result; And And face detection means for detecting faces by applying an adaboost algorithm to the combination result. The apparatus according to claim 1, wherein the feature information extracting means An integral image generating unit for integrating the input image to generate an integral image; An edge image generating unit for generating an edge image from the input image and obtaining an intensity in each edge direction from the edge image; Likelihood feature extracting unit for calculating a difference value between a sum of a bright area pixel and a sum of a dark area pixel in a designated sub window from the input image integrated using the learning data; And And an EOH feature extraction unit for extracting a ratio value between intensities in each edge direction in the subwindow and a charge ratio value of intensity in each edge direction using the learning data. Claim 3 has been abandoned due to the setting registration fee. The type of the prototype is divided into a first type which is composed of four regions and a second type in which a bright region and a dark region are symmetrically diagonal to each other, a second type which is composed of two regions, And a fourth type in which a dark region, a bright region, and a dark region are sequentially located, each of which is composed of a third type and a third type in which a bright region and a dark region are located right and left, Likelike feature information of the fourth prototype is calculated. Claim 4 has been abandoned due to the setting registration fee. 3. The apparatus of claim 2, wherein the edge image generation unit A conversion unit for converting the input image into a monochrome image; A first calculating unit for calculating a hardness for each coordinate in the horizontal / vertical direction by applying a Sobel mask to the monochrome image and calculating the size of the edge from the hardness; A second calculation unit for calculating an edge strength of each coordinate by summing the squares of the edge sizes in the horizontal / vertical directions of the respective coordinates; A noise eliminator for eliminating noise by setting an allowable value for the edge intensity; A third calculating unit for calculating an edge direction of each coordinate from a size of the edge in the horizontal / vertical direction; And And a fourth calculation unit for quantizing the edge directions in K directions and collecting edge values of coordinates having directionality in each direction to calculate an edge image for K edge directions. Claim 5 has been abandoned due to the setting registration fee. 3. The apparatus according to claim 2, wherein the EOH characteristic calculating unit And the edge direction is set to the horizontal, vertical, and first and second diagonal directions. Claim 6 has been abandoned due to the setting registration fee. 6. The apparatus according to claim 5, wherein the EOH characteristic calculating unit In the sub-window, an edge direction horizontal directional task 1 diagonal direction, a horizontal direction and a vertical direction, a horizontal direction task 2 diagonal direction, a first diagonal direction and a vertical direction, a first diagonal direction task 2 diagonal direction, And extracts a ratio value between the intensities in the edge direction, the edge direction, the first diagonal direction, the vertical direction, and the second diagonal direction.

Images