KR101681233B1 - Method and apparatus for detecting face with low energy or low resolution - Google Patents

Abstract

According to the present invention, in low-energy and low-resolution face detection, a face is detected by sliding a window while reducing an image to a resolution corresponding to a window size for face detection with respect to an input image, For the windows that have passed through the prefiltering, the final block of interest is extracted by matching with the local block texture feature and feature template. Then, It is possible to more efficiently detect face and person having low energy or low resolution by detecting faces using a classifier having a relatively large amount of calculation.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for detecting a face having low energy / resolution,

The present invention relates to face and person detection, and more particularly, to detecting a face by sliding a window while reducing an image to a resolution corresponding to a window size for an input image, The window that does not contain the face is removed quickly by using the variance. For the window that passed the pre-filtering, the final block of interest is extracted by matching the local block texture feature and the feature template, And more particularly, to a low energy and resolution face detection method and apparatus that can more efficiently detect faces and persons having low energy or low resolution by detecting faces using a classifier having a large calculation amount.

In recent years, as the need for public places and personal security increases, a large amount of surveillance video analysis is required along with an increase in surveillance cameras. As a part of the intelligent surveillance system, the large-capacity surveillance video analysis method is one of the fields that have been continuously studied in the fields of image processing and computer vision, and requires intelligent functions such as automatic human detection and tracking.

Recently, despite various methods for object detection and tracking in the field of computer vision, there is a demand for a low energy face and person detecting device in an environment where energy supply is restricted by diversification of locations where surveillance cameras are installed. Face detection is usually known to require a large number of memories and logic gates in hardware configuration. The most commonly used face detection is adaptive boosting (AdaBoost) and cascade classifier method. It uses resources, memory, I get a lot.

Therefore, a face detection algorithm and hardware with a small memory and a number of logic gates are required while detecting a small resolution face for demanding low energy and for face and person detection in a surveillance camera environment.

(Patent Literature)

Korean Registered Patent No. 10-0795160 (Registered Date Jan. 09, 2008)

Accordingly, the present invention relates to a low-energy face and person detection method and an efficient hardware configuration, and more particularly, to a low-energy and resolution face detection method and apparatus capable of more efficiently detecting a face having low energy or low resolution in an intelligent surveillance camera ≪ / RTI >

The present invention relates to a low-energy and low-resolution face detecting apparatus, which performs prefiltering on an input image in a window sliding manner and refers to a result obtained by comparing a variance value calculated in each window region with a preset reference value A prefilter for removing a window not including a face; a local block texture feature value calculation unit for calculating a local block texture feature value for each window passed by the prefilter unit, and extracting a histogram of each window region using the local block texture feature value And comparing the feature vector values of the respective windows extracted by the feature extraction unit with templates of a predetermined reference feature vector to determine whether the corresponding window is a face region A feature template matching unit for determining the face region from the feature template matching unit, And an erroneous detection removal unit for removing a window of the window from which the face region is erroneously detected and finally detecting a face region in the input image.

In addition, the input image is down-scaled stepwise to a resolution corresponding to the window size capable of detecting the face, and is input to the pre-filtering unit.

The pre-filtering unit may compare the first variance value with the second variance value calculated for each window based on the first variance value calculated for the window including the face, And determines that the face is not included in the window if the first variance value is smaller than or greater than a preset threshold value.

The feature extracting unit may extract three histograms using the local block texture feature value in each region by dividing the region into three regions of eyes, nose, and mouth for each window region, connecting the three histograms to each other, And generates one feature vector value for the window.

The feature template matching unit may measure a feature vector value of the window and a distance between the template and determine a face area if the measured distance is less than a preset threshold value. And a distance between the template and the feature vector value for the window is measured for the three templates.

According to another aspect of the present invention, there is provided a low energy and low resolution face detection method comprising the steps of: prefiltering an input image in a window sliding manner to remove a window not including a face; Comparing a feature vector value of each extracted window with a template of a predetermined reference feature vector to determine whether the corresponding window is a face region; And removing a window of the determined window from which the face region is erroneously detected to finally detect a face region in the input image.

Also, the input image is downscaled stepwise to a resolution corresponding to the window size capable of detecting the face.

Also, the step of removing the window may include storing a first variance value calculated for a window containing a face as a reference value, performing a variance for each window, Comparing the second variance value with the first variance value; and removing the window if the second variance value is smaller or larger than a predetermined threshold value that is greater than the first variance value .

The step of calculating the feature vector value may further include the steps of: calculating the local block texture feature value in each region by dividing the region into three regions of eyes, nose, and mouth for each window region; Extracting three histograms for the three regions by using the three histograms, and generating one feature vector value for the window by concatenating the three histograms.

The step of determining whether the face region is the face region includes the steps of: constructing three templates from a face having a pose change in the left and right directions; measuring a distance between the feature vector values of the respective windows and the three templates; Comparing the measured distance with a threshold value set for determining a face area, and determining whether the corresponding window is a face area.

According to the present invention, in the low-energy and low-resolution face detection, prefiltering is performed using structural characteristics and dispersion of windows to quickly remove a window that does not include a face, After extracting the final region of interest by matching the texture feature and feature template, the face and the person having the low energy or the low resolution can be detected more efficiently by detecting the face using the classifier having a relatively large amount of calculation only for the region of interest.

In addition, for face detection, a plurality of templates composed of local block texture features (LBTF) are created, and templates are formed in characteristic vector space, but several are generated so as to be resistant to environmental changes. In other words, face template uses LBP (local binary pattern) which is robust against illumination and pose change within a certain range, and features based on block partitioning. By using several template robust against face change when testing, Detection is possible.

In order to efficiently detect faces in images having various sizes of faces, ROIs are quickly detected, and a plurality of ROIs detected after ROI detection are passed through a classifier By performing detection and removal, local block texture feature extraction and template matching, which have a relatively small amount of computation, are performed in advance, so that the total amount of calculation can be reduced. Then, face detection is performed using a classifier such as SVM (support vector machine) only for the detected region of interest, thereby enabling efficient energy management in a low energy face detection apparatus.

1 is a block diagram of a low energy and low resolution face detection apparatus according to an embodiment of the present invention;
FIG. 2 is a conceptual diagram of dispersion and average calculation for a face region according to an embodiment of the present invention;
3 is a flow chart of operation control for low energy and low resolution face detection according to an embodiment of the present invention;
4 is a block diagram of a hardware logic circuit of a low energy face detection apparatus according to an embodiment of the present invention.

Hereinafter, the operation principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

FIG. 1 shows a detailed block configuration of a low-energy and low-resolution face detection apparatus according to an embodiment of the present invention.

1, the low-energy and low-resolution face detection apparatus 100 slides a window in a sliding window manner while reducing an image from an original resolution of the input image to a resolution corresponding to a window size, Whether or not it is.

The operation of the low energy face detection apparatus may be largely composed of a region of interest (ROI) extraction unit 110 and a false positive detection removal unit 120.

The region of interest extraction unit 110 includes a pre-filtering unit 102, a feature extraction unit 104, and a feature template matching unit 106, In the step, non-interest areas can be quickly removed using the structural properties and dispersion of each window. In addition, local block texture features, which are texture information useful for expressing facial expressions in the window passed in the prefiltering step, are extracted, and the features and distances between the features and the three feature templates previously created are measured The final region of interest regions are extracted. In the next erroneous detection operation, it is finally determined whether or not the face region is efficiently detected by using a relatively large amount of calculator for the extracted region of interest, and merging of the candidate regions is performed to detect the face region.

Hereinafter, more detailed operation of each component of the low-energy and low-resolution face detection apparatus according to the embodiment of the present invention will be described in detail with reference to FIG.

First, the pre-filtering unit 102 quickly checks whether the face region is a face region using a structural feature and variance of each window for a window whose resolution is reduced from the input image to a window size for face detection, Removes a window that is not a region. Such a prefiltering operation is a method that can filter out whether or not a face region is obtained by a relatively simple operation. This is a part contributing to the improvement of the system speed, and at the same time, can reduce the false detection.

That is, the pre-filtering unit 102 performs an operation related to the variance in the window region and the geometric properties of the human face. The first variance value is compared with the second variance value calculated for each window based on the first variance value calculated for the window including the face, 1 < / RTI > variance value is greater than or equal to a predetermined threshold value, it is determined that the face is not included in the corresponding window, and the window is removed. In other words, by obtaining the dispersion in the window area, a background area having a small dispersion or a background can be very complex, which can help to eliminate a large dispersion area. In addition, in general, the face of a person has a characteristic that the brightness of the eye region is relatively dark compared to the brightness of the region below it. Using this characteristic, the average of the pixel values inside the white square, as shown in FIG. 3, If the value obtained by subtracting the average of the pixel values is not larger than a predetermined value, it can be determined that the face is not the face.

The feature extraction unit 104 extracts texture features that are effective for facial expression in order to evaluate whether or not the face region is the area of the window passed by the pre-filtering unit 102. At this time, a typical texture feature is LBP (local binary pattern). LBP is strong against illumination, and it is known that it is resistant to pose change within a certain range. In the present invention, a local block texture feature (LBTF) is used to effectively utilize not only texture but also contour and morphological information of a face.

That is, the feature extracting unit 104 divides the window area passed by the pre-filtering unit 102 into three areas, for example, the left eye, the right eye, and the mouth area. It may also include an area that includes the neck shoulder below the face. Then, an LBP histogram is extracted from each region, and three histograms of the result are connected to generate one feature vector. In this case, since the number of pixels in the two regions in the three regions is the same, the number of pixels in the third region corresponding to the input region are different, so that the normalization process is performed through the weights. In addition, although the embodiment of the present invention has exemplified the division of the three regions for the texture and the shape of the face region, it may include an additional region for representing a shape related to a person such as a neck or a shoulder.

The feature template matching unit 106 classifies whether the face region is a face region or not according to the distance between the features obtained for the input window and the pre-configured feature template. At this time, the method based on the template matching is relatively simple because the calculation is simple and the template of the feature vector is stored in the integer form. In addition, the distance measure uses L1-norm as distance measure to perform integer operation in hardware implementation.

)에 대해서 테스트(x)와의 L1-norm(d)이 측정되고, 이 거리가 임계치보다 작을 경우에 얼굴 영역인 것으로 판단할 수 있다. 위와 같은 본 발명의 실시예에서는 3가지 템플릿을 사용하는 것을 예를 들어 설명하였으나, 3개 이외의 탬플릿을 사용하는 것도 포함할 수 있으며 얼굴검출의 강인성을 위해 다수개의 템플릿을 활용하는 것도 포함할 수 있다.In addition, in the embodiment of the present invention, the template has three feature templates from the face having the front face and the face having the pose change in order to have tolerance to the pose change of the face, 1] as shown in Fig.

The L1-norm (d) with respect to the test (x) is measured with respect to the face region, and if the distance is smaller than the threshold value, it can be judged to be the face region. In the above embodiment of the present invention, three templates are used. However, it is possible to use a template other than three, and it is also possible to use a plurality of templates for robustness of face detection have.

[Equation 1]

The False Positive Reduction unit 120 determines whether or not the ROI area obtained by the feature template matching unit 106 is a face area using a relatively large computational amount classifier. At this time, various algorithms can be used in the erroneous detection removal unit 120 for eliminating the erroneous detection. In the embodiment of the present invention, a SVM (support vector machine) known to have excellent generalization capability is used.

)과 테스트 벡터(x) 사이의 내적을 수행한다. 이러한 [수학식 2]에서 볼 수 있듯이 모든 서포트 벡터들과의 내적은 서포트 벡터들의 합에 해당하는 벡터(w)와 테스트 벡터 사이의 한 번의 내적에 의해 간단히 수행될 수 있다. 마지막으로 아래 [수학식 2]를 계산한 후에 이 값이 미리 정한 임계값과 비교해서 얼굴영역인지 얼굴영역이 아닌지를 판단할 수 있다.An example of a hardware logic implementation of the low energy and low resolution face detection apparatus of FIG. 4, described below, shows the adoption of a linear SVM for computational and hardware architectural efficiency. The SVM is one of the binary classifiers, and when given a training set, it uses a hyperplane to minimize the classification error to distinguish the two classes, and a margin, which means the distance from the hyperplane to the training set closest to it, To select the hyperplane. At this time, for the test, support vectors existing around the hyperplane (Equation 2)

) And the test vector (x). As can be seen from this equation (2), the dot product with all the support vectors can be simply performed by one dot product between the test vector and the vector w corresponding to the sum of the support vectors. Finally, after calculating [Equation 2], it is possible to determine whether this value is a face region or not in comparison with a predetermined threshold value.

&Quot; (2) "

FIG. 3 shows an operation control flow in a low-energy and low-resolution face detection apparatus according to an embodiment of the present invention. Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 3. FIG.

First, the input image can be downscaled down to the resolution corresponding to the window size capable of face detection, and input to the low energy and low resolution face detection apparatus (S300).

The pre-filtering unit 102 of the low-energy and low-resolution face detection apparatus performs prefiltering on the input image in a window sliding manner. The result of comparing the variance value calculated in each window region with a predetermined reference value And performs a pre-filtering process for removing a window that does not include a face (S302). In this case, for example, the pre-filtering unit 102 compares the first variance value and the second variance value calculated for each window based on the first variance value calculated for the window including the face, If the variance value is smaller or larger than a predetermined threshold value or larger than the predetermined threshold value, it is determined that the face is not included in the window and can be removed.

When the pre-filtering is performed as described above, the feature extracting unit 104 calculates the local block texture feature value for each window that has passed the pre-filtering and extracts the histogram of each window region using the local block texture feature value A feature vector value for each window is calculated (S304).

At this time, the feature extracting unit 104 calculates the local block texture feature value in each region by dividing the region into three regions, for example, eyes, nose, and mouth for each window region, Three histograms can be extracted for each region. In addition, three histograms can be concatenated to generate one feature vector value for the window.

In this way, the feature vector value of each window calculated from the feature extraction unit 104 is applied to the feature template matching unit. Then, in the feature template matching unit 106, the feature vector value for each window and the template It is possible to determine whether the corresponding window is a face region through feature template matching (S306).

At this time, in determining whether the face region is a face region, the feature template matching unit 106 constructs three templates from, for example, a face having a pose change in the right and left directions, and calculates a distance between the feature vector value for each window and three templates It is possible to determine whether the corresponding window is a face area by comparing the measured distance with a threshold value set for judgment of the face area.

Then, after the face region is determined by the feature template matching unit 106 as described above, it is necessary to remove the window that is detected as a face region. That is, in the erroneous detection removal unit 120, the window of the window judged as the face region by the feature template matching unit 106 is erroneously detected, and finally the face region is detected in the input image ( S308).

Then, the face regions finally determined through the erroneous detection removal unit 120 are merged through the merging unit and the finally detected face region is output (S310).

4 illustrates a hardware logic circuit configuration for a low energy face detection apparatus according to an embodiment of the present invention.

Referring to FIG. 4, in the hardware logic circuit design of the low energy face detecting apparatus of the present invention, since memory is the memory that occupies the most energy consumption and logic most important factor in hardware configuration based on the algorithm, Design a hardware structure that can be used efficiently.

That is, in the embodiment of the present invention, the above-described algorithm is implemented through memory and logic optimization, and the overall hardware core core is composed of four cores as shown in FIG. The hardware structure includes a first module 400 for applying a face detection method on an 8 scale to detect faces of various sizes, a second module 402 for storing LBTF features and RAW data by calling data in units of lines, A third module 404 for calculating a cumulative histogram and an average and variance using the stored LBTF feature and RAW data, a template matching process for comparing the template with the template, and a fourth module 406 for performing FP reduction do. The finally detected face information is (X, Y, S) which is the position coordinate.

The first module 400 may be a logic circuit implementing part that performs downscaling to a window size for performing face detection on an input image. The first module 400 downscales an input image applied to the pre-filtering unit 102 to a window size do.

That is, the first module 400 receives 320 × 240 (QVGA resolution) images, generates 8 scaled images, and merges them into a single data stream. In this case, the resolution of the eight scaled images is 320x240, 224x171, 160x120, 112x85, 80x60, 56x42, 40x30, and 28x21 and the input data 76,800 (= 320x240) is 152,764, which is 1.989 (= 152,764 / 76,800) have. Thus, the first module can always operate at twice the operating frequency of the sensor clock. In the design of this module, a unified SRAM layout can be designed to reduce memory usage and reduce the number of gates.

The second module 402 and the third module 404 may be logic circuit implementation parts for operation of the pre-filtering unit 102 and feature extraction unit 104 of FIG. That is, the second module 402 receives data in units of lines and outputs a 16x1 local binary pattern (LBP) vector and a 16x1 RAW vector. The module has a built-in 510x136 single-ported static random access memory (SPSRAM) to generate 18x1 column vectors for each of the 8 scaled images, and then calculates and outputs 16x1 LBP with RAW data.

The third module 404 calculates and outputs a histogram for a 16x16 LBP detection window. The histogram for a 16x16 window can be generated by adding a histogram for a 16x1 column vector. In addition to the LBP histogram calculation, the variance calculation for the 6x6 region is performed as shown in FIG. 2, and an average corresponding to the 8x3 and 8x2 regions corresponding to the eyes and nose is calculated and pre-filtered as shown in FIG. 2 .

The fourth module 406 may be a logic circuit implementation portion for operation of the feature template matching unit 106 of FIG. That is, the fourth module 406 performs template matching on the calculated histogram. A maximum of 6 templates can be defined. If one or more of these templates has a cost that is considered to be a face, the face is considered to be detected.

As described above, according to the present invention, in the low energy and low resolution face detection, the face is detected by sliding the window while reducing the image to the resolution corresponding to the window size for the face detection on the input image, The window that does not contain the face is quickly removed by performing prefiltering using the structural characteristics and dispersion of each window, and for the window that has passed through the prefiltering, the local block texture feature and the feature template are matched with each other, It is possible to more efficiently detect faces and persons having low energy or low resolution by detecting faces using a classifier having a relatively large calculation amount only for the region of interest.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should not be limited by the described embodiments but should be defined by the appended claims.

102: pre-filtering unit 104: feature extracting unit
106: Feature template matching unit 110:
120: Error detection removal 130:

Claims (10)

A prefilter for performing prefiltering on an input image in a window sliding manner and removing a window not including a face with reference to a result of comparing a variance value calculated in each window region with a predetermined reference value;
A feature extraction unit for calculating a local block texture feature value for each window passed by the pre-filtering unit and a feature vector value for each window by extracting a histogram of each window region using the local block texture feature value, Wow,
A feature template matching unit for comparing a difference between a feature vector value of each window calculated by the feature extraction unit and a template of a predetermined reference feature vector to determine whether the corresponding window is a face region,
And an erroneous detection removal unit for removing a window of the window determined as the face region from the feature template matching unit and erroneously detecting the face region and finally detecting a face region in the input image,
The feature extraction unit may extract,
Three histograms using the local block texture feature value are extracted in each region, and three histograms are concatenated to form one feature vector for the window Wherein the low-energy and low-resolution face detection apparatus generates the low-energy and low-resolution face detection apparatus.
The method according to claim 1,
The input image may include:
Scaled down to a resolution corresponding to the window size capable of face detection, and input to the pre-filtering unit.
The method according to claim 1,
The pre-
Comparing the first variance value with a second variance value calculated for each window based on the first variance value calculated for the window including the face, and comparing the second variance value with the first variance value, And if it is smaller or larger than the set threshold value, it is judged that the face is not included in the window and is removed.
delete The method according to claim 1,
Wherein the feature template matching unit comprises:
Determining a face area when the measured distance is less than a predetermined threshold by measuring a feature vector value for the window and a distance between the template and constructing three templates from faces having a right and left pose change, And a distance between the template and the feature vector value for the window is measured with respect to the template.
Performing a pre-filtering operation on the input image in a window sliding manner to remove a window not including a face;
Computing a feature vector value for each window passed in the prefiltering;
Comparing the calculated feature vector value for each window with a template of a predetermined reference feature vector to determine whether the corresponding window is a face region,
Detecting a face region in the input image by removing a window in which the face region is erroneously detected among the windows determined as the face region,
Wherein the step of calculating the feature vector value comprises:
Calculating local block texture feature values in each region by dividing the region into three regions of eyes, nose, and mouth for each window region;
Extracting three histograms for the three regions using the local block texture feature value;
Connecting the three histograms to generate one feature vector value for the window
Wherein the low energy and low resolution face detection method comprises the steps of:
The method according to claim 6,
The input image may include:
Scaled down to a resolution corresponding to the window size capable of face detection.
The method according to claim 6,
The method of claim 1,
Storing a first variance value calculated for a window containing a face as a reference value;
Performing a variance for each window and comparing the first variance value with a second variance value calculated for each window;
If the second variance value is smaller than or greater than a predetermined threshold value than the first variance value, removing the corresponding window
Wherein the low energy and low resolution face detection method comprises the steps of:
delete The method according to claim 6,
The method of claim 1,
Comprising the steps of: constructing three templates from a face having a left-right pose change;
Measuring a feature vector value for each window and a distance between the three templates;
Comparing the measured distance with a threshold value set for determination of a face area and determining whether the window is a face area
Wherein the low energy and low resolution face detection method comprises the steps of:

Images