JP2007108990A - Face detecting method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress erroneous detection in processing for detecting a face included in an image. <P>SOLUTION: An index value computing means 34 computes an index value showing probability that a partial image W is a face image on each of partial images taken out in different positions on an image to be detected, and a face image candidate extracting means 34 extracts all of the partial images W whose index values are a predetermined threshold value or more, as candidates of the face image. A highly reliable face image determining means 31 determines the candidate on which a certain high index value is computed, as a highly reliable face image, and an index value increasing means 31 increases the index value to the specific candidate with almost the same face inclination as the highly reliable face image. A face image detecting means 31 performs threshold value determination of the index values on all the candidates to detect the face image. Only the specific candidate may be detected as the face image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a face detection method and apparatus for detecting a face image from a digital image, and a program therefor.

  Conventionally, it is possible to check the color distribution of the face area of a person in a snapshot photographed by a digital camera and correct the skin color, or to recognize a person in a digital image photographed by a digital video camera of a surveillance system. Has been done. In such a case, since it is necessary to detect a face area corresponding to a human face in the digital image, various methods for detecting a face image including a face in the digital image have been proposed so far.

For example, a method of extracting a partial image at a plurality of different positions on the detection target image, determining whether the partial image is an image including a face (face image), and detecting the face image on the detection target image Is mentioned. In order to determine whether or not a partial image is a face image, for example, a template matching method, a method using a discriminator module in which facial features are learned by a machine learning learning method, and the like can be considered (for example, In any of these methods, an index value indicating the probability that the partial image is a face image is calculated based on the image pattern of the partial image. However, when the index value exceeds a predetermined threshold, it is common to use a technique for discriminating the partial image from a face image.
"High-speed omnidirectional face detection", Shihong LAO et al., Image Recognition and Understanding Symposium (MIRU2004), July 2004, P.II-271-II-276 Japanese Patent Application No. 2003-316924 Japanese Patent Application No. 2003-316925 Japanese Patent Application No. 2003-316926

  However, in the face detection method using the discrimination method based on the index value threshold determination based on the image pattern as described above, the image quality of the detection target image is poor, such as unevenness in the brightness or contrast of the image in the detection target image. If the face on the detection target image is obstructed by another subject and part of the face is obstructed, the facial features cannot be captured sufficiently, and face image detection may be missed. If a non-face pattern similar to a face happens to be present on the detection target image, the non-face image is erroneously detected as a face image.

  In view of the above circumstances, an object of the present invention is to provide a face detection method and apparatus capable of further suppressing detection errors and erroneous detection of face images, and a program therefor.

  A first face detection method of the present invention is a face detection method for detecting a face image in an input image, the step of cutting out partial images at different positions on the input image, and a plurality of clips cut out at the different positions For each of the partial images, an index value indicating the probability that the partial image is a face image including a face having a predetermined inclination based on the feature amount on the image of the partial image, and the predetermined inclination is set to a plurality of different inclinations. Each of the plurality of partial images, and a partial image whose calculated index value is equal to or greater than a first threshold is included in the predetermined orientation when the index value is calculated. A step of extracting all as face image candidates, and a candidate having a calculated index value equal to or greater than a second threshold value greater than the first threshold value is determined as a highly reliable face image from among the face image candidates. Step Performing a process for increasing the index value of the candidate for candidates whose face inclination is substantially the same as the face inclination in the highly reliable face image among the face image candidates; Detecting the candidate whose index value is equal to or greater than a third threshold value, which is a value between the first threshold value and the second threshold value, as the face image. .

  A second face detection method according to the present invention is a face detection method for detecting a face image in an input image, the step of cutting out partial images at different positions on the input image, and a plurality of clips cut out at the different positions. For each of the partial images, an index value indicating the probability that the partial image is a face image including a face having a predetermined inclination based on the feature amount on the image of the partial image, and the predetermined inclination is set to a plurality of different inclinations. And calculating a partial image having a calculated index value equal to or greater than a first threshold among the plurality of partial images, the face having the predetermined inclination when the index value is calculated. A step of extracting all of the face image candidates as candidates, and a candidate having a calculated index value equal to or greater than a second threshold value greater than the first threshold value is determined as a highly reliable face image from the face image candidates. Step And detecting only candidates whose face inclination is substantially the same as the face inclination in the high-reliability face image among the face image candidates as the face image. .

  A first face detection device according to the present invention is a face detection device that detects a face image in an input image, and includes a partial image cutout unit that cuts out a partial image at a different position on the input image, and a cutout at the different position. For each of the plurality of partial images, an index value indicating the probability that the partial image is a face image including a face having a predetermined inclination is set on the basis of the feature amount on the image of the partial image. Index value calculating means for calculating each of them with different slopes, and the predetermined value when the index value is calculated for a partial image of the plurality of partial images whose calculated index value is greater than or equal to a first threshold value A face image candidate extracting unit that extracts all face image candidates including a face in the direction of the face, and a calculated index value from the face image candidates is equal to or greater than a second threshold value that is greater than the first threshold value. Candidate, high confidence face A high-reliability face image determination unit that determines the image; and a candidate for which the face inclination is substantially the same as the face inclination in the high-reliability face image among the face image candidates. Index value increasing means for performing an increase process, and the candidate whose index value after the process is greater than or equal to a third threshold value that is a value between the first threshold value and the second threshold value as the face image It comprises a face image detecting means for detecting.

  A second face detection device according to the present invention is a face detection device that detects a face image in an input image, and includes a partial image cutout unit that cuts out a partial image at a different position on the input image, and a cutout at the different position. For each of the plurality of partial images, an index value indicating the probability that the partial image is a face image including a face having a predetermined inclination is set on the basis of the feature amount on the image of the partial image. The index value calculating means for calculating each of them with different slopes, and the partial image of which the calculated index value is greater than or equal to a first threshold among the plurality of partial images, when the index value is calculated Face image candidate extraction means for extracting all face image candidates including a face having a predetermined inclination, and a calculated index value out of the face image candidates is greater than or equal to a second threshold value greater than the first threshold value. A candidate is highly reliable A highly reliable face image determining means for determining an image, and among the face image candidates, a face image in which only a candidate whose face inclination is substantially the same as the face inclination in the highly reliable face image is detected as the face image. And a detecting means.

  A first program according to the present invention is a program for causing a computer to function as a face detection device for detecting a face image in an input image, wherein the computer cuts out partial images at different positions on the input image. Image segmentation means, for each of the plurality of partial images cut out at the different positions, shows the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount of the partial image. Index value calculation means for calculating the index value by changing the predetermined inclination to a plurality of different inclinations, and a partial image having a calculated index value equal to or greater than a first threshold among the plurality of partial images. Face image candidate extraction means for extracting all face image candidates including the face of the predetermined orientation when the value is calculated, and the calculated index value is selected from the face image candidates. A highly reliable face image determining unit that determines a candidate that is equal to or greater than a second threshold value greater than the first threshold value as a highly reliable face image, and among the face image candidates, the face tilt in the highly reliable face image Index value increasing means for performing processing for increasing the index value of the candidate for a candidate that is substantially the same as the slope of the candidate, and the index value after the processing is between the first threshold value and the second threshold value The candidate having a value equal to or greater than a third threshold value is made to function as a face image detecting means for detecting the candidate as the face image.

  A second program of the present invention is a program for causing a computer to function as a face detection device for detecting a face image in an input image, wherein the computer cuts out partial images at different positions on the input image. Image segmentation means, for each of the plurality of partial images cut out at the different positions, shows the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount of the partial image. Index value calculation means for calculating the index value by changing the predetermined inclination to a plurality of different inclinations, and among the plurality of partial images, a partial image whose calculated index value is greater than or equal to a first threshold value, Face image candidate extraction means for extracting all face image candidates including the face having the predetermined inclination when the index value is calculated, and the calculated index value from the face image candidates Highly reliable face image determination means for determining a candidate that is equal to or greater than a second threshold value greater than the first threshold value as a highly reliable face image, and among the face image candidates, a face tilt in the highly reliable face image Only candidates that are substantially the same as the inclination of the face are made to function as face image detection means for detecting the face image.

  In the present invention, the “face image” refers to an image including an image constituting a face.

  Further, “face inclination” means an inclination in the in-plane (in-image plane) direction, and in other words means a rotation position on the face image.

  Further, “the face inclination is substantially the same as the face inclination in the highly reliable face image” means that, for example, the angle deviation between the face inclination in the face image candidate and the face inclination in the highly reliable face image is within a predetermined range. For example, the angle deviation may be within ± 45 degrees.

  According to the first face detection method, apparatus, and program of the present invention, a partial image whose index value indicating the probability of being a face image is not less than the first threshold is extracted as a face image candidate from the detection target image. Among the extracted face image candidates, a candidate having an index value equal to or higher than a second threshold value that is higher than the first threshold value is determined as a highly reliable face image, and the inclination of the face is determined as the highly reliable face image. Since the index value for a candidate that is substantially the same is increased and a candidate whose index value is equal to or greater than a third threshold value that is a value between the first threshold value and the second threshold value is detected as a face image, Based on an empirical rule that the inclinations of multiple faces are often approximately the same, the reliability of the candidate can be more appropriately reflected in the index value for the candidate for the face image, and the detection of the face image is omitted. And it is possible to suppress false detection

  Further, according to the second face detection method and apparatus of the present invention and the program therefor, a partial image whose index value indicating the probability of being a face image is greater than or equal to the first threshold is detected from the image to be detected. Among the extracted face image candidates, a candidate having an index value equal to or higher than a second threshold value that is higher than the first threshold value is determined as a highly reliable face image. Since only candidates that are substantially the same as the trusted face image are detected as face images, the reliability is considered to be low based on an empirical rule that the inclinations of multiple faces included in the same image are often substantially the same. Can be excluded, and erroneous detection of the face image can be suppressed.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a schematic block diagram showing a configuration of a face detection system 1 which is an embodiment (first embodiment) of a first face detection apparatus according to the present invention. This face detection system 1 converts an image including a face on an input digital image (hereinafter referred to as a face image) into a face position, size, tilt (rotation position in the image plane), and orientation (left and right swing). It is detected regardless of the direction). The face detection system 1 is a discriminator module (hereinafter simply referred to as a discriminator) generated by machine learning learning using a sample image as a face detection method that is particularly excellent in detection accuracy and robustness. The method using is adopted. This technique uses a plurality of different face sample images representing faces with a predetermined inclination and orientation and a plurality of different non-face sample images representing non-faces to learn facial features. A discriminator capable of discriminating whether or not the face image includes a face with an inclination and orientation is prepared and prepared, and partial images are displayed at different positions on an image to be detected (hereinafter referred to as a detection target image). Is sequentially extracted, and whether or not the partial image is a face image is determined using the above discriminator, thereby detecting a face image included in the detection target image.

  As shown in FIG. 1, the face detection system 1 includes a multi-resolution conversion unit 10, a normalization unit 20, a face detection unit 30, and an overlap detection determination unit 40.

  The multi-resolution converting unit 10 multi-resolutions the input detection target image S0 to obtain a resolution image group S1 including a plurality of images having different resolutions (S1_1, S1_2,..., S1_n; hereinafter referred to as resolution images). Is.

  By converting the image size of the detection target image S0, that is, the resolution, the resolution is normalized to a predetermined resolution, for example, a rectangular size image having a short side of 416 pixels, and the standardized detection target image S0 ′ is converted. obtain. Then, by further performing resolution conversion based on the standardized detection target image S0 ′, a plurality of resolution images having different resolutions are generated, and a resolution image group S1 is obtained. The reason why such a resolution image group is generated is that the size of the face included in the detection target image S0 is usually unknown, while the size of the face to be detected is determined by a method of generating a discriminator described later. In order to detect faces of different sizes, each partial image of a predetermined size is cut out while shifting the position on images with different resolutions. This is because it is necessary to determine whether the image is an image.

  FIG. 2 is a diagram illustrating a process of multi-resolution of the detection target image S0. Specifically, the multiresolution, that is, the generation of the resolution image group, is performed by setting the standardized detection target image S0 ′ as the basic resolution image S1_1 as shown in FIG. 2 and adding 2 to the resolution image S1_1. Resolution image S1_2 having a size of -1/3 times, and a resolution image having a size of -1/3 times 2 of the resolution image S1_2 (a size of -2/3 times the size of the basic image S1_1). S1_3 is generated first, and then a resolution image obtained by reducing each of the resolution images S1_1, S1_2, and S1_3 to 1/2 times size is generated, and the reduced resolution image is further reduced to 1/2 times size. A predetermined number of resolution images are generated by repeatedly performing processing such as image generation. In this way, the reduction processing of 1/2 times that does not require the interpolation processing of the pixel value representing the luminance is the main processing, and the size is reduced by 2−1 / 3 times from the basic resolution image. A plurality of images can be generated at high speed. For example, when the resolution image S1_1 has a rectangular size of 416 pixels on the short side, the resolution images S1_2, S1_3,... Have 330 pixels, 262 pixels, 208 pixels, 165 pixels, 131 pixels, and 104 pixels on the short sides, respectively. , 82 pixels, 65 pixels,..., And can generate a plurality of resolution images reduced by a factor of 2 to −1/3. Note that an image generated without interpolating pixel values in this way has a strong tendency to retain the characteristics of the original image pattern as it is, and is preferable in that an improvement in accuracy can be expected in face detection processing.

  The normalization unit 20 performs overall normalization processing and local normalization processing on each of the resolution images so that the contrast of the resolution image is suitable for the face detection processing, and a plurality of normalized resolution images A resolution image group S1 ′ composed of (S1′_1, S1′_2,..., S1′_n) is obtained.

  First, the overall normalization process will be described. In the overall normalization process, the contrast of the resolution image is brought close to a predetermined level suitable for the face detection process, that is, a level suitable for extracting the performance of the discriminator described later. This is a process of converting the pixel value of the entire resolution image according to a conversion curve that approximates the value representing the logarithm of the luminance of the subject in this image.

  FIG. 3 is a diagram showing an example of a conversion curve used for the overall normalization process. As the overall normalization process, for example, as shown in FIG. 3, a so-called inverse gamma conversion (= 2.2 power) in the sRGB space is performed, and then a conversion curve (look-up table) that takes a logarithm is used. Therefore, a process for converting pixel values in the entire image can be considered. This is due to the following reason.

  The light intensity I observed as an image is usually expressed as the product of the reflectance R of the subject and the intensity L of the light source (I = R × L). Therefore, when the intensity L of the light source changes, the light intensity I observed as an image also changes. However, if only the reflectance R of the subject can be evaluated, it does not depend on the intensity L of the light source. It is possible to perform highly accurate face discrimination that is not affected by the brightness of the image.

Here, when the intensity of the light source is L, the light intensity observed from the portion with the reflectance R1 on the subject is I1, and the light intensity observed from the portion with the reflectance R2 on the subject is I2. In the logarithmic space, the following equation holds.
log (I1) −log (I2) = log (R1 × L) −log (R2 × L) = log (R1) + log (L) − (log (R2) + log (L)) = log (R1) −log (R2) = log (R1 / R2)

  In other words, logarithmic conversion of pixel values in an image results in conversion into a space where the reflectance ratio is expressed as a difference. In such a space, only the reflectance of the subject that does not depend on the intensity L of the light source is evaluated. It becomes possible to do. In other words, it is possible to align different contrasts (here, the pixel value difference itself) depending on the brightness in the image.

  On the other hand, the color space of an image acquired by a device such as a general digital camera is sRGB. sRGB is an international standard color space that defines and unifies color, saturation, etc., in order to unify the differences in color reproduction between devices. In this color space, the gamma value (γout) is 2. The image pixel value is a value obtained by raising the input luminance to 1 / γout (= 0.45) in order to enable proper color reproduction in the .2 image output device.

  Therefore, the pixel value in the entire image is converted according to a so-called inverse gamma conversion, that is, according to a conversion curve that takes a logarithm after being raised to the power of 2.2, thereby evaluating only by the reflectance of the subject independent of the intensity of the light source. Can be performed properly.

  In other words, such an overall normalization process is a process of converting pixel values in the entire image according to a conversion curve for converting a specific color space into a color space having different characteristics. it can.

  By applying such processing to the detection target image, different contrasts can be provided depending on the brightness in the image, and the accuracy of the face detection processing is improved. The overall normalization process is characterized in that the processing result is easily influenced by the difference in oblique light, background, and input modality in the detection target image, but the processing time is short.

  Next, the local normalization process will be described. The local normalization process is a process for suppressing variation in contrast in a local region on the resolution image. That is, for each local region in the resolution image, for a local region in which the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level, the degree of dispersion is made closer to a certain level higher than the predetermined level. A second luminance gradation conversion that performs luminance gradation conversion processing and suppresses the degree of dispersion to a level lower than the predetermined level for a local region where the degree of dispersion of pixel values is less than the predetermined level. Processing is performed. This local normalization process has a long processing time, but has a feature that the influence on the determination result due to the difference in oblique light, background, and input modality in the detection target image is small.

FIG. 4 is a diagram showing the concept of local normalization processing, and FIG. 5 is a diagram showing the flow of local normalization processing. Expressions (1) and (2) are gradation conversion expressions for pixel values for the local normalization process.

Here, X is the pixel value of the pixel of interest, X ′ is the pixel value after conversion of the pixel of interest, mlocal is the average of the pixel values in the local region centered on the pixel of interest, Vlocal is the variance of the pixel values in this local region, SDlocal is a standard deviation of pixel values in this local area, (C1 × C1) is a reference value corresponding to the above-mentioned constant level, C2 is a threshold value corresponding to the above-mentioned predetermined level, and SDc is a predetermined constant. In the present embodiment, the number of gradations of luminance is 8 bits, and the possible pixel values are 0 to 255.

  As shown in FIG. 4, first, one pixel in the resolution image is set as a target pixel (step S1), and a pixel value in a local area having a predetermined size centered on the target pixel, for example, 11 × 11 pixel size. The variance Vlocal is calculated (step S2), and it is determined whether or not the variance Vlocal is equal to or greater than the threshold value C2 corresponding to the predetermined level (step S3). If it is determined in step S3 that the variance Vlocal is equal to or greater than the threshold C2, the variance Vlocal is larger than the reference value (C1 × C1) corresponding to the certain level as the first luminance gradation conversion process. The tone conversion that decreases the difference between the pixel value X of the target pixel and the average mlocal, and increases the difference between the pixel value X of the target pixel and the average mlocal as the variance mlocal is smaller than the reference value (C1 × C1). Is performed according to the equation (1) (step S4). On the other hand, if it is determined in step S3 that the variance Vlocal is less than the threshold value C2, linear tone conversion that does not depend on the variance Vlocal is performed as the second luminance tone conversion processing according to equation (2). This is performed (step S5). Then, it is determined whether or not the target pixel set in step S1 is the last pixel (step S6). If it is determined in step S6 that the target pixel is not the last pixel, the process returns to step S1, and the next pixel on the same resolution image is set as the target pixel. On the other hand, if it is determined in step S6 that the target pixel is the last pixel, the local normalization for the resolution image is terminated. As described above, by repeating the processes of steps S1 to S6, local normalization can be performed on the entire resolution image.

  Note that the predetermined level may be changed according to the whole or a part of luminance in the local region. For example, in the normalization process in which gradation conversion is performed for each target pixel, the threshold value C2 may be changed according to the pixel value of the target pixel. That is, the threshold value C2 corresponding to the predetermined level may be set higher when the luminance of the target pixel is relatively high, and may be set lower when the luminance is relatively low. In this way, it is possible to correctly normalize a face that exists in a low-brightness, so-called dark area with low contrast (a state in which the dispersion of pixel values is small).

  Here, the inclination of the face to be detected is a total of 12 types of inclinations that are set by rotating in 30 degree increments in the image plane of the detection target image S0 with reference to the vertical direction of the detection target image S0. Assume that the switching order of the face inclination to be determined is determined in advance. For example, assuming that the switching order is represented by a clockwise rotation angle with respect to the vertical direction of the detection target image S0, the upward three directions are 0 degrees, 330 degrees, 30 degrees (0 degree group), and the rightward three directions. 90 degrees, 60 degrees, 120 degrees (90 degree group), leftward three directions 270 degrees, 240 degrees, 300 degrees (270 degree group), and three downward directions 180 degrees, 150 degrees, 210 degrees (180 degrees) Degree group).

  The face detection unit 30 performs face detection processing on each resolution image of the resolution image group S1 ′ that has been normalized by the normalization unit 20 while changing the inclination of the face to be detected according to a preset order. By performing the detection, the face image S2 included in each resolution image is detected, and the image is further composed of a plurality of elements.

  FIG. 6 is a block diagram illustrating a configuration of the face detection unit 30. As shown in FIG. 6, the face detection unit 30 includes a detection control unit (highly reliable image determination unit, index value increase unit, face image detection unit) 31, a resolution image selection unit 32, and a sub-window setting unit (partial image extraction). Means) 33 and a classifier group (index value calculating means, face image candidate extracting means) 34.

  The detection control unit 31 mainly controls sequence control in the face detection process by controlling other units constituting the face detection unit 30. That is, the resolution image selection unit 32, the sub window setting unit 33, and the discriminator group 34 are controlled, and for each resolution image (input image) constituting the resolution image group S1 ′, partial images are sequentially cut out and cut out over the entire resolution image. Applying all kinds of discriminators with different face inclinations to be discriminated against the extracted partial images, the face image candidates on the resolution image are extracted regardless of the face inclination, and the extracted face image By determining whether each of the candidates is a true face image, the true face image S2 in each resolution image is detected. For example, as appropriate, the resolution image selection unit 32 is instructed to select a resolution image, the sub-window setting unit 33 is instructed to set the sub-window, and among the classifiers constituting the classifier group 34 Change the type of discriminator to be used. The sub-window setting condition includes a range on the image where the sub-window is set, a sub-window moving interval, that is, a detection roughness.

  Under the control of the detection control unit 31, the resolution image selection unit 32 selects resolution images to be subjected to face detection processing from the resolution image group S1 ′ in ascending order of size, that is, in order of coarse resolution. Note that the face detection method in this embodiment determines whether the partial image W is a face image with respect to the partial images W of the same size that are sequentially cut out on the respective resolution images. Since this is a method for detecting a face, it can be considered that the resolution image selection unit 32 sets the size of the face to be detected in the detection target image S0 while changing from large to small each time.

  The sub-window setting unit 33 selects a partial image W that is a determination target of whether or not it is a face image in the resolution image selected by the resolution image selection unit 32 based on the sub-window setting condition set by the detection control unit 31. The subwindow to be cut out is set while shifting its position by a predetermined width.

  For example, in the selected resolution image, a sub-window for cutting out a partial image W having a predetermined size, that is, a 32 × 32 pixel size, is sequentially set by moving a predetermined number of pixels, for example, by 2 pixels, and the cut-out portion The image W is input to the classifier group 34. As will be described later, each discriminator constituting the discriminator group 34 discriminates a face image including a face having a predetermined inclination and orientation. By doing so, it is possible to obtain any inclination and orientation. It becomes possible to discriminate a face image of a certain face.

  Based on the image pattern of the partial image W, the discriminator group 34 has a score (index value) SC indicating the probability that the partial image W is a face image including a face having a predetermined inclination and orientation. Are calculated by changing the predetermined inclination and orientation, and the partial image W whose calculated score SC is equal to or greater than the first threshold Th1 is a face having the predetermined inclination and orientation when the score is calculated. Is extracted as a candidate for the face image S2 including.

  FIG. 7 is a diagram showing the configuration of the classifier group 34. As shown in FIG. 7, the discriminator group 34 includes a plurality of types of discriminator groups having different face orientations to be discriminated, that is, a front face discriminator group 34_F that mainly discriminates a front face, and mainly a left side face. The left side face classifier group 34_L for discriminating and the right side face classifier group 34_R for mainly discriminating the right side face are connected in parallel. Further, each of these three types of classifier groups is a classifier corresponding to a total of 12 directions in which the inclination of the face to be distinguished differs by 30 degrees with respect to the vertical direction of the image, that is, the front face classifier group 34_F is a classifier. 34_F0, 34_F30,..., 34_F330, the left side face discriminator group 34_L is the discriminator 34_L0, 34_L30,. It is composed of

  As shown in FIG. 7, each of the classifiers includes a plurality of weak classifiers WC. The weak classifier WC calculates at least one feature amount related to the distribution of pixel values of the partial image W. The score SC indicating the probability that the partial image W is a face image including a face having a predetermined inclination and orientation is calculated using the feature amount.

  In each of the classifier groups 34 described above, the main face directions that can be discriminated are three types: front face, left side face, and right side face. A discriminator for discriminating each of the face and the left oblique face may be further provided.

  Here, the configuration of each classifier constituting the classifier group 34, the flow of processing in the classifier, and the classifier learning method will be described.

  The discriminator has a plurality of weak discriminators WC effective for discrimination selected from a large number of weak discriminators WC by learning described later. The weak discriminator WC calculates a feature amount from the partial image W according to a predetermined algorithm specific to each weak discriminator WC, and based on the feature amount and its own histogram described later as a predetermined score table. The score sc indicating the probability that the partial image W is a face image including a face having a predetermined inclination and orientation is obtained. The discriminator adds the individual scores sc obtained from the plurality of weak discriminators WC to obtain a score SC, and determines the partial image W having the score SC equal to or greater than the threshold Th1 as a face having a predetermined inclination and orientation. Extracted as a candidate for the included face image S2.

  FIG. 8 is a flowchart showing the flow of processing in one classifier. When the partial image W is input to the discriminator, different types of feature amounts x are calculated in the plurality of weak discriminators WC (step S11). For example, as shown in FIG. 9, for a partial image W having a predetermined size, for example, a 32 × 32 pixel size, an average of four neighboring pixels (the image is divided into a plurality of blocks for each 2 × 2 pixel size, By processing the average value of the pixel values of the four pixels of the block as the pixel value of one pixel corresponding to the block), a 16 × 16 pixel size image and an 8 × 8 pixel size reduction are performed. 2 points in each pair constituting one pair group consisting of a plurality of pairs, with a predetermined two points set in the plane of these three images including the original image as one pair A difference value between pixel values (brightness) is calculated, and a combination of these difference values is used as a feature amount. The predetermined two points of each pair are, for example, two predetermined points arranged in the vertical direction and two predetermined points arranged in the horizontal direction so as to reflect the characteristics of the facial shading on the image. Then, a value corresponding to a combination of difference values as feature amounts is calculated as x. Next, in accordance with the value x, the face to be identified by the partial image W from a predetermined score table (self histogram) (for example, in the case of the discriminator 34_F30, “the face direction is front and the tilt is 30 degrees rotation angle”). A score sc indicating the probability of being a face image including the face of “)” is calculated for each weak classifier (step S12). Then, the score SC is obtained by adding the individual scores sc calculated for each weak classifier (step S13), and it is determined whether or not the score SC is equal to or greater than the first threshold Th1 (step S14). If the result is affirmative, the partial image W is extracted as a candidate for a face image S2 including a face having a predetermined inclination and orientation to be discriminated by the discriminator (step S15).

  Next, a classifier learning (generation) method will be described.

  FIG. 10 is a flowchart showing a learning method of the classifier. For learning of the discriminator, a plurality of sample images that are standardized with a predetermined size, for example, 32 × 32 pixel size, and further subjected to the same processing as the normalization processing by the normalization unit 20 described above are used. Sample images include a plurality of different face sample images (face sample image group) known to be faces and a plurality of different non-face sample images (non-face sample image group) known to be non-faces. prepare.

  The face sample image group includes each sample image obtained by stepwise scaling in units of 0.1 times within a range of 0.7 to 1.2 times in length and / or width for one face sample image. On the other hand, a plurality of deformation variations obtained by rotating in steps of 3 degrees in a range of ± 15 degrees on the plane are used. At this time, the face sample image is standardized in size and position so that the eye position is at a predetermined position, and the above-described rotation and scaling on the plane are performed based on the eye position. For example, in the case of a sample image of d × d size, as shown in FIG. 11, the positions of both eyes are 1 / 4d on the inner side and the lower side from the uppermost vertex and the uppermost vertex of the sample image, respectively. The size and position of the face are standardized so as to come to each position moved by 1 / 4d, and the rotation and expansion / contraction on the plane is performed around the middle point of both eyes.

  Each of these sample images is assigned a weight or importance. First, the initial value of the weight of all sample images is set equal to 1 (step S21).

  Next, when a plurality of pairs of groups consisting of a plurality of pairs are set with a predetermined two points set in the plane of the sample image and the reduced image as one pair, each of the plurality of types of pairs is weak. A separate device is created (step S22). Here, each weak discriminator sets one pair group consisting of a plurality of pairs with a predetermined two points set in the plane of the partial image cut out in the sub-window W and the reduced image as one pair. This provides a reference for discriminating between a face image and a non-face image using a combination of difference values of pixel values (luminance) between two points in each pair constituting this one pair group. . In the present embodiment, a histogram for a combination of pixel value difference values between two points in each pair constituting one pair group is used as the basis of the score table of the weak classifier.

  FIG. 12 is a diagram showing how a histogram is generated from a sample image. As shown in the sample image on the left side of FIG. 12, two points of each pair constituting the pair group for creating this discriminator are a plurality of sample images that are known to be faces. The right eye on the reduced image of 16 × 16 pixel size in which the point in the center of the right eye is P1, the point in the right cheek part is P2, the point in the part between the eyebrows is P3, and the sample image is reduced by an average of four neighboring pixels The point at the center of P4, the point at the cheek on the right side is P5, and the point at the forehead part on the reduced image of 8 × 8 pixel size reduced by the average of 4 neighboring pixels is P6, the mouth part A certain point is P7, and there are five pairs of P1-P2, P1-P3, P4-P5, P4-P6, and P6-P7. Note that the coordinate positions of the two points of each pair constituting one pair group for creating a certain classifier are the same in all sample images. For all sample images that are known to be faces, combinations of pixel value difference values between two points of each of the five pairs are obtained, and a histogram thereof is created. Here, the value that can be taken as a combination of the difference values of the pixel values depends on the number of luminance gradations of the image, but if it is a 16-bit gradation, there are 65536 different values for the difference value of one pixel value, As a whole, the number of gradations is (the number of pairs), that is, 65536 to the fifth power, and a large number of samples, time, and memory are required for learning and detection. For this reason, in the present embodiment, the difference value of the pixel value is divided by an appropriate numerical value width and quantized to be n-valued (for example, n = 100). Thereby, since the number of combinations of the difference values of the pixel values is n to the fifth power, the number of data representing the combination of the difference values of the pixel values can be reduced.

  Similarly, a histogram is also created for the non-face sample image group. For the non-face sample image, positions corresponding to the positions of the two predetermined points of each pair on the face sample image (similarly, reference numerals P1 to P7 are used) are used. A histogram obtained by taking logarithm values of ratios of frequency values indicated by these two histograms and representing the histogram is used as the basis of the score table of the weak discriminator shown on the rightmost side of FIG. The value of each vertical axis indicated by the histogram of the weak classifier is hereinafter referred to as a discrimination point. According to this weak discriminator, an image showing the distribution of combinations of pixel value difference values corresponding to positive discrimination points is highly likely to be a face, and the possibility increases as the absolute value of the discrimination point increases. It can be said. Conversely, an image showing a distribution of combinations of difference values of pixel values corresponding to negative discrimination points is highly likely not to be a face, and the possibility increases as the absolute value of the discrimination point increases. In step S22, a plurality of weak discriminators in the above-described histogram format are created for combinations of pixel value difference values between predetermined two points of each pair constituting a plurality of types of pair groups that can be used for discrimination.

  Subsequently, the weak classifier that is most effective for determining whether or not the image is a face image is selected from the plurality of weak half-classifiers created in step S22. The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak classifiers are compared, and the weak classifier showing the highest weighted correct answer rate is selected (step S23). That is, in the first step S23, since the weight of each sample image is equal to 1, there is a sample having the largest number of sample images that can be correctly discriminated whether or not the image is a face image simply by the weak discriminator. , Selected as the most effective weak classifier. On the other hand, in the second step S23 after the weight of each sample image is updated in step S25, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S23 after the second time, more emphasis is placed on correctly determining a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the combination of weak classifiers selected so far, that is, using the weak classifiers selected so far in combination (in the learning stage, the weak classifiers do not necessarily need to be linearly combined. ) It is ascertained whether or not the result of determining whether or not each sample image is a face image has exceeded a predetermined threshold value that matches the answer whether or not it is actually a face image (step S24). . Here, the current weighted sample image group or the sample image group with equal weight may be used for evaluating the correct answer rate of the combination of weak classifiers. When the predetermined threshold value is exceeded, learning is terminated because it is possible to determine whether the image is a face with a sufficiently high probability by using the weak classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step S26 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far.

  In step S26, the weak discriminator selected in the most recent step S23 is excluded so as not to be selected again.

  Next, in the weak classifier selected in the most recent step S23, the weight of the sample image that could not correctly determine whether or not it is a face is increased, and the sample that can correctly determine whether or not the image is a face image The image weight is reduced (step S25). The reason for increasing or decreasing the weight in this way is that in selecting the next weak discriminator, importance is placed on images that have not been correctly discriminated by the already selected weak discriminator, and whether these images are face images or not. This is because a weak discriminator that can correctly discriminate is selected to enhance the effect of the combination of the weak discriminators.

  Subsequently, the process returns to step S23, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  As a weak discriminator suitable for discriminating whether or not it is a face image by repeating the above steps S23 to S26, the difference value of the pixel value between two predetermined points of each pair constituting a specific pair group When the weak discriminator corresponding to the combination is selected and the correct answer rate confirmed in step S24 exceeds the threshold, the type of the weak discriminator used for discriminating whether the image is a face image and the discriminating condition are determined. This is confirmed (step S27), thereby completing the learning. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to constitute one classifier. For each weak classifier, a score table for calculating a score according to a combination of pixel value difference values is generated based on the obtained histogram. Note that the histogram itself can also be used as a score table. In this case, the discrimination point of the histogram is directly used as a score.

  In this way, the discriminator is generated by learning using the face sample image group and the non-face sample image group. However, as described above, a plurality of different discriminators depending on the inclination and orientation of the face to be discriminated. In order to generate a discriminator, a plurality of types of face sample image groups corresponding to each inclination and direction of the face are prepared, and learning using the face sample image group and the non-face sample image group is performed. It will be done for each type.

  That is, in the present embodiment, the face orientation includes three types of front, left side, and right side, and the tilt includes a total of 36 types, a total of 12 types in increments of 30 degrees from 0 degrees to 330 degrees. A face sample image group is prepared.

  When the plurality of face sample image groups are obtained, the discriminator group 34 is obtained by performing the learning using the face sample image group and the non-face sample image group for each type of the face sample image group. Can be generated.

  As described above, by using a plurality of discriminators learned for each face orientation and each face inclination, it is possible to discriminate face images including faces having various inclinations and orientations.

  In the case of adopting the above learning method, the weak classifier uses a combination of difference values of pixel values between two predetermined points of each pair constituting a specific pair group, and a face image and a non-face image. Is not limited to the above-described histogram format, and may be anything, for example, binary data, a threshold value, a function, or the like. Further, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 12 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  When face image candidates are extracted by a plurality of types of discriminators, the detection control unit 31 determines whether the face image candidates are true face images S2, as mentioned above. At this time, the detection control unit 31 determines a candidate having a calculated score SC that is equal to or greater than a second threshold Th2 that is greater than the first threshold Th1 as a highly reliable face image from among the face image candidates, Among the image candidates, processing for increasing the score SC of the candidate, for example, adding a special score, is performed on a candidate whose face inclination is substantially the same as the face inclination in the highly reliable face image. . And the candidate whose score SC after this process is more than 3rd threshold value Th3 which is the value between 1st threshold value Th1 and 2nd threshold value Th2 is detected as true face image S2. Here, as a candidate whose face inclination is substantially the same as the face inclination in the highly reliable face image, for example, the face inclination is an inclination within a rotation angle of ± 30 degrees from the face inclination in the highly reliable face image. Can be a candidate.

  For each of the face images S2 detected on each resolution image, the overlap detection determination unit 40 is a face image that represents the same face on the detection target image S0 based on the positional relationship of the face images. It is determined whether or not the images are detected in duplicate on a plurality of resolution images adjacent to each other in resolution, and a process of combining a plurality of face images that are recognized to be detected in duplicate into one is performed. A true face image S3 without detection is output.

  When obtaining a plurality of resolution images by converting the detection target image S0 into multiple resolutions, the resolution gap between adjacent resolution images cannot be made very large in order to prevent detection errors of face images. The discriminator usually has a certain allowable range as the size of the face that can be discriminated. In such a case, the same face on the detection target image S0 may be detected redundantly in a plurality of adjacent resolution images. The above-described processing by the duplication detection determination unit 40 is processing performed to eliminate such duplication detection and obtain an accurate detection result.

  Next, the flow of processing in the face detection system 1 will be described.

  13a and 13b are flowcharts showing the flow of processing in the face detection system 1. As shown in these drawings, when the detection target image S0 is supplied to the multi-resolution conversion unit 10 (step S31), an image S0 ′ in which the image size of the detection target image S0 is converted into a predetermined size is generated. Then, a resolution image group S1 composed of a plurality of resolution images reduced in size (resolution) by 2 to a power of −1/3 is generated from the image S0 ′ (step S32). Then, in the normalizing unit 20, the above-described overall normalization processing and local normalization processing are performed on each resolution image of the resolution image group S1, and a normalized resolution image group S1 ′ is obtained (step S1). S33).

  In the face detection unit 30, the resolution image selection unit 32 that has received an instruction from the detection control unit 31 starts from the resolution image group S 1 ′ in ascending order of image size, that is, S 1 ′ _n, S 1 ′ _n−1, ..., a predetermined resolution image S1'_i is selected in the order of S1'_1 (step S34). Next, the sub window setting unit 33 sets the sub window on the resolution image S1′_i while moving the sub window at a predetermined pitch, for example, at an interval of two pixels, and sequentially cuts out the partial images W of a predetermined size (step S35). Is input to the classifier group 34 (step S36). Each discriminator constituting the discriminator group 34 uses a plurality of weak discriminators to calculate a score SC indicating a probability that the input partial image W is a face image including a face having a predetermined inclination and orientation ( In step S37), the partial image W having the calculated score SC equal to or greater than the first threshold Th1 is extracted as a face image candidate, and the detection control unit 51 acquires the extraction result R (step S38).

  Then, it is determined whether or not the current partial image W is the last partial image on the current resolution image (step S39). If it is determined that the current partial image W is not the last partial image, the process returns to step S35, a new partial image W is cut out on the current resolution image, and the detection process is continued. On the other hand, when it is determined that the current partial image W is the last partial image, the next determination process is performed. That is, it is determined whether or not the current resolution image is the last resolution image (step S40). If it is determined that the current resolution image is not the last resolution image, the process returns to step S34 to select a new resolution image and continue the process of extracting face image candidates. On the other hand, when it is determined that the current resolution image is the last resolution image, the detection control unit 31 determines whether the face image candidate is the true face image S2. That is, the detection control unit 31 determines a candidate whose calculated score SC is equal to or greater than the second threshold Th2 that is greater than the first threshold Th1 as a highly reliable face image from among the face image candidates (step S41). ) Among the face image candidates, candidates whose face inclination is substantially the same as the face inclination in the highly reliable face image, that is, the face inclination is within a rotation angle of ± 30 degrees from the face inclination in the highly reliable face image. A new score SC is calculated by performing a process of adding a special score to the candidate score SC with respect to a candidate having a slope of (step S42). Then, a candidate whose score SC after the addition processing is equal to or greater than a third threshold Th3 that is a value between the first threshold Th1 and the second threshold Th2 is detected as a true face image S2 (step S43). .

  Then, based on the positional relationship of the detected face images, the duplicate detection determination unit 40 is a face image representing the same face on the detection target image S0 for each of the face images. It is determined whether or not the images are detected in duplicate on a plurality of resolution images adjacent to each other in resolution, and a process of combining a plurality of face images recognized as being detected in duplicate is performed (step S43). .

  FIG. 14 shows how the resolution images are selected in ascending order of size by repeating the above steps S34 to S40, the partial images W are sequentially cut out on each resolution image, and face detection is performed. It is a figure.

  FIG. 15 is a diagram illustrating a snap photograph image as an example of the detection target image S0. In this snapshot image, faces C1 to C3 of three persons are shown, and a part of one face C3 is shielded by another subject. When face detection processing is performed on such a snapshot image, the facial features of two faces C1 and C2 that are not shielded can be captured sufficiently, and a relatively high score (for example, 70 points and 65 points) are calculated and become the first threshold Th1 (for example, 30 points) or more, extracted as face image candidates, and further, are the third threshold Th3 (for example, 50 points) or more. Detected as On the other hand, with respect to the other face C3 that is shielded, the facial features cannot be captured sufficiently, and usually a relatively low score (for example, 40 points) is calculated. Although it is extracted, it may not be detected as a face image because it is not equal to or greater than the third threshold Th3. However, since the inclinations of a plurality of faces included in the same image are often substantially the same, the calculated score is not less than the second threshold Th2 (for example, 70 points) as in the above embodiment. A candidate (C1) is determined as a highly reliable face image, and a special score (for example, for example, a candidate for a face image (C2, C3) including a face having substantially the same inclination as the face in the highly reliable face image (for example, (20 points) is added, a score exceeding the third threshold Th3 (60 points in this example) can be expected even for the face C3 that cannot sufficiently capture the facial features due to shielding or the like. it can.

  According to such a face detection system of the present embodiment, a partial image whose index value indicating the probability of being a face image is greater than or equal to the first threshold is extracted as a face image candidate from the detection target image and extracted. Among the candidates for the face image, a candidate having an index value equal to or higher than a second threshold value higher than the first threshold value is determined as a highly reliable face image, and the face inclination is substantially the same as the highly reliable face image. The index value for the candidate is increased, and candidates whose index value is greater than or equal to the third threshold value, which is a value between the first threshold value and the second threshold value, are detected as face images, so they are included in the same image Based on the empirical rule that the inclination of multiple faces is often approximately the same, the index value for the face image candidate can more appropriately reflect the reliability as a candidate, and face image detection omissions and errors can be reflected. Detection can be suppressed.

  In the first embodiment, a face image candidate whose calculated score is greater than or equal to the second threshold Th2 is determined as a highly reliable face image among the extracted face image candidates. As another technique, for example, among the extracted face image candidates, a candidate having the highest calculated score may be determined as a highly reliable face image.

  In the first embodiment, after determining a highly reliable face image with a certain high degree of reliability, a face image whose face inclination is substantially the same as the face inclination in the highly reliable face image is determined. The candidate score is increased with respect to the candidate. As another method, for example, after determining a highly reliable face image, the face inclination in the highly reliable face image is determined. It is also possible to detect only face image candidates that are substantially the same as the true face image (corresponding to the second embodiment of the face detection device according to the present invention).

  In this way, a partial image whose index value indicating the probability of being a face image is equal to or more than the first threshold is extracted as a face image candidate from the detection target image, and the extracted face image candidates are extracted. A candidate whose index value is equal to or higher than a second threshold value higher than the first threshold value is determined as a highly reliable face image, and only candidates whose face inclination is substantially the same as the highly reliable face image are detected as face images. Therefore, based on an empirical rule that the inclinations of a plurality of faces included in the same image are often substantially the same, it is possible to eliminate face image candidates that are considered to be low in reliability, and It becomes possible to suppress erroneous detection.

  Next, an embodiment (second embodiment) of a second face detection apparatus according to the present invention will be described. As in the case of the first embodiment, the face detection system according to the present embodiment is configured by a multi-resolution unit 10, a normalization unit 20, a face detection unit 30, and an overlap detection determination unit 40, as shown in FIG. Further, the face detection unit 30 includes a detection control unit 31 (highly reliable face image determination unit, face image detection unit), a resolution image selection unit 32, a sub window setting unit (partial image cutout unit) 33, a classifier group ( The index value calculation means and the face image candidate extraction means) 34 are configured, but the processing in the face detection unit 30 is different from that in the first embodiment.

  In the present embodiment, the discriminator group 34 has a total of 36 types (12 types of face inclinations in increments of 30 degrees from 0 degrees to 330 degrees, the face orientation is front, as in the first embodiment. These discriminators are divided into four groups according to the inclination of the face to be discriminated as shown in FIG. 17. That is, the discriminators whose face to be discriminated are 330 degrees, 0 degrees, and 30 degrees are classified into the first group (0 degree group), and the face inclinations to be discriminated are 60 degrees, 90 degrees, and 120 degrees. The classifier is the second group (90 degree group), the classifiers whose face to be discriminated are 240 degrees, 270 degrees and 300 degrees, the classifier is the third group (270 degree group), and the face tilt to be discriminated is 150. The classifiers having degrees, 180 degrees, and 210 degrees are set as a fourth group (180 degree group).

  Then, the detection control unit 31 applies these discriminators in order from the first group to the fourth group to all the partial images W cut out by the sub-window setting unit 33 on the resolution image. Face image candidates are detected.

  At this time, when a score higher than the second threshold Th2, which is a sufficiently high score to be reliable, is calculated by a classifier of a certain group and a highly reliable face image is detected, the detection control unit 31 Thereafter, it is not applied to the discriminator of the group in which the order of application to the partial image W is later. In other words, the inclination of the face to be discriminated is fixed to substantially the same inclination as that of the face in the highly reliable face image. The reason for this is that when multiple faces are included in one image, the order of use is based on the rule of thumb that the inclination of each face (rotation position on the image) is often aligned. This is because detection of a face image can be omitted with respect to the inclination of the face corresponding to a classifier of a certain group, thereby speeding up the face detection. However, in order to deal with a face that has a neck squeezed, a plurality of types of discriminators with close face inclinations to be discriminated as described above are grouped so that variations in inclination of about ± 30 degrees can be allowed. The classifier is used for each group.

  As a specific example, for example, after a face image candidate is detected using a first group discriminator, a face belonging to the second group and having a face inclination to be discriminated is 60 degrees. When an image candidate is detected, a score equal to or higher than the second threshold Th2 is calculated, and when a highly reliable face image is detected, the third and fourth groups that are used in the subsequent order are used. The discriminator is not used thereafter. Assuming that the average detection processing time for each group is 1, if the face inclination to be discriminated as described above is not fixed, it takes 1 × 4 direction = 4 time, whereas In the case of fixing the inclination of the face to be discriminated, it is assumed that the probability that face image candidates are detected by applying the discriminator of each group is equal regardless of the inclination of the face (that is, each group The probability that a face will be detected for 1/4) is 1 × 1/4 + 2 × 1/4 + 3 × 1/4 + 4 × 1/4 = 2.5. When the inclination of the face to be discriminated is fixed, the detection processing time is shorter and faster.

  The reason why the 180 degree group is detected lastly is based on an empirical rule that a face whose top and bottom are upside down has the lowest existence probability compared to the inclination of the face corresponding to the other three groups.

  When a highly reliable face image is detected, the face image candidates detected using the classifiers of the previous group are regarded as erroneous detection and are deleted. When fixing the inclination of the face to be discriminated as described above, for example, when the second group discriminator is used, a highly reliable face image is detected, and the face inclination to be discriminated is fixed. When performed, the discriminators of the third and fourth groups are not used, and detection of faces with a predetermined inclination corresponding to these groups is not performed, so that erroneous detection occurs for faces with these inclinations. Never do. However, when the first group discriminator is used, a face image candidate having a relatively low score may be detected. Since it can be considered that there is a high possibility of erroneous detection, the detection result by the discriminator of the first group is regarded as erroneous detection and deleted, so that erroneous detection suppression can be realized.

  Hereafter, the flow of the process in the face detection system which is 2nd Embodiment is demonstrated.

  FIGS. 16a and 16b are flowcharts showing the flow of processing of the face detection system according to the second embodiment. As shown in these drawings, when the detection target image S0 is supplied to the multi-resolution conversion unit 10 (step S51), an image S0 ′ in which the image size of the detection target image S0 is converted to a predetermined size is generated. Then, a resolution image group S1 composed of a plurality of resolution images reduced in size by −1/3 power of 2 is generated from the image S0 ′ (step S52). Then, in the normalizing unit 20, the above-described overall normalization processing and local normalization processing are performed on each resolution image of the resolution image group S1, and a normalized resolution image group S1 ′ is obtained (step S1). S53).

  In the face detection unit 30, the resolution image selection unit 32 that has received an instruction from the detection control unit 31 starts from the resolution image group S 1 ′ in ascending order of image size, that is, S 1 ′ _n, S 1 ′ _n−1, ..., a predetermined resolution image S1'_i is selected in the order of S1'_1 (step S54).

  Then, the detection control unit 31 selects a group of discriminators to be applied to the partial image W (Step S54). In step S54, each time this step is executed, the group of discriminators is selected in a predetermined order, but when a new resolution image is selected, the order is reset. In addition, when the group fixing described later is already performed, only that group is selected.

  Next, the sub-window setting unit 33 sets the sub-window on the resolution image S1′_i while moving the sub-window at a predetermined pitch, for example, at an interval of two pixels, and sequentially cuts out the partial images W of a predetermined size (step S56). W is input to the discriminator of the selected group in the discriminator group 34 (step S57). The discriminator to which the partial image W is input uses a plurality of weak discriminators to calculate a score SC indicating the probability that the partial image W is a face image including a face having a predetermined inclination and orientation corresponding to the discriminator. Then, the partial image W having the calculated score SC equal to or greater than the first threshold Th1 is extracted as a face image candidate, and the detection control unit 51 acquires the extraction result R (step S59).

  When the face image candidate is extracted, the detection control unit 31 determines whether or not the classifier group to be applied to the partial image W has already been fixed (step S60), and the group has already been fixed. If it is determined that it is, the process proceeds to step S63. On the other hand, if it is determined that the group has not yet been fixed, it is further determined whether or not the score SC of the face image candidate is equal to or greater than the second threshold Th2 (step S61). When it is determined that the calculated score SC is equal to or greater than the second threshold Th2, the face image candidate is determined as a highly reliable face image, and a group of discriminators to be used thereafter is currently selected. It fixes to the group currently performed (step S62).

  Then, it is determined whether or not the current partial image W is the last partial image on the current resolution image (step S63). If it is determined that the current partial image W is not the last partial image, the process returns to step S56, a new partial image W is cut out on the current resolution image, and the detection process is continued. On the other hand, when it is determined that the current partial image W is the last partial image, the next determination process is performed. That is, it is determined whether or not the currently selected classifier group is the last group (step S64). If it is determined that the currently selected classifier group is the last group, the process proceeds to step S65, while the currently selected classifier group is not the last group. If so, there is still a group to be selected next, so the process returns to step S55, and the next group of discriminators is selected.

  In step S65, it is determined whether or not the current resolution image is the last resolution image. If it is determined that the current resolution image is not the last resolution image, the process returns to step S54 to select a new resolution image and continue the process of extracting face image candidates. On the other hand, if it is determined that the current resolution image is the last resolution image, the detection control unit 31 deletes the candidates extracted by the classifier of the previous group that fixes the group of classifiers, The remaining face image candidates are determined as true face images (step S66). That is, when a highly reliable face image is detected, only face image candidates whose face inclination is substantially the same as that of the highly reliable face image are detected as face images.

  Then, based on the positional relationship of the detected face images, the duplication detection determination unit 40 is a face image representing the same face on the input image S0 for each of the face images, and has a resolution. It is determined whether or not the images are detected in duplicate on a plurality of resolution images adjacent to each other, and a plurality of face images that are recognized to be detected in duplicate are combined into one (step S67).

  Although the face detection system according to the embodiment of the present invention has been described above, a program for causing a computer to execute each process in a portion corresponding to the face detection device of the present invention in the face detection system is also included in the present invention. This is one of the embodiments. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

Block diagram showing the configuration of the face detection system 1 The figure which shows the process of multiresolution of a detection target image The figure which shows an example of the conversion curve used for a whole normalization process Diagram showing the concept of local normalization processing Diagram showing the flow of local normalization processing Block diagram showing the configuration of the face detection unit 30 Block diagram showing configuration of classifier group Diagram showing the processing flow in the classifier The figure for demonstrating calculation of the feature-value in a weak discriminator Flow chart showing the learning method of the classifier The figure which shows the sample image of the face standardized so that the position of eyes may be in a predetermined position The figure which shows the method of deriving the histogram of the weak classifier Flowchart showing the processing in the face detection system 1 according to the first embodiment (first half) A flowchart (second half) showing processing in face detection system 1 by a 1st embodiment. The figure for demonstrating the change of the resolution image used as a face detection object, and the movement of the subwindow on the image A diagram showing a snap photo image as an example of an input image Flowchart showing the processing in the face detection system 1 according to the second embodiment (first half) Flowchart showing the process in the face detection system 1 according to the second embodiment (second half) The figure which shows a mode that a classifier group is grouped according to the inclination of the face which should be discriminated.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face detection system 10 Multi-resolution part 20 Normalization part 30 Face detection part 31 Detection control part 32 Resolution image selection part 33 Subwindow setting part 34 Discriminator group 40 Duplication detection determination part

Claims (6)

A face detection method for detecting a face image in an input image,
Cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the partial image. Changing the predetermined inclination into a plurality of different inclinations, respectively,
A step of extracting all partial images having a calculated index value equal to or greater than a first threshold among the plurality of partial images as face image candidates including the face of the predetermined orientation when the index value is calculated. When,
Determining, from among the face image candidates, a candidate whose calculated index value is equal to or greater than a second threshold value greater than the first threshold value as a highly reliable face image;
Performing a process of increasing the index value of the candidate for the candidate of the face image with respect to a candidate whose face inclination is substantially the same as the inclination of the face in the highly reliable face image;
Detecting the candidate having a post-processing index value equal to or greater than a third threshold value, which is a value between the first threshold value and the second threshold value, as the face image. Face detection method. A face detection method for detecting a face image in an input image,
Cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the image of the partial image, Changing the predetermined inclination into a plurality of different inclinations, respectively,
Out of the plurality of partial images, all partial images whose calculated index value is equal to or greater than the first threshold are extracted as face image candidates including the face having the predetermined inclination when the index value is calculated. Steps,
Determining, from among the face image candidates, a candidate whose calculated index value is equal to or greater than a second threshold value greater than the first threshold value as a highly reliable face image;
And a step of detecting, as the face image, only candidates whose face inclination is substantially the same as the face inclination in the highly reliable face image among the face image candidates. A face detection device for detecting a face image in an input image,
Partial image cutout means for cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the image of the partial image, Index value calculating means for calculating the predetermined inclination by changing the predetermined inclination into a plurality of different inclinations;
A face for extracting all partial images having a calculated index value greater than or equal to the first threshold as candidate face images including the face in the predetermined direction when the index value is calculated from the plurality of partial images. Image candidate extraction means;
A highly reliable face image determining means for determining, as a highly reliable face image, a candidate whose calculated index value is greater than or equal to a second threshold value greater than the first threshold value from among the face image candidates;
Index value increasing means for performing processing for increasing the index value of the candidate for candidates whose face inclination is substantially the same as that of the face in the highly reliable face image among the face image candidates;
A face image detecting means for detecting the candidate whose detected index value is not less than a third threshold value that is a value between the first threshold value and the second threshold value as the face image. A face detection device characterized by the above. A face detection device for detecting a face image in an input image,
Partial image cutout means for cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the image of the partial image, Index value calculating means for calculating the predetermined inclination by changing the predetermined inclination into a plurality of different inclinations;
Out of the plurality of partial images, all partial images whose calculated index value is equal to or greater than the first threshold are extracted as face image candidates including the face having the predetermined inclination when the index value is calculated. Face image candidate extraction means;
A highly reliable face image determining means for determining, as a highly reliable face image, a candidate whose calculated index value is greater than or equal to a second threshold value greater than the first threshold value from among the face image candidates;
A face image detecting means for detecting only those candidates whose face inclination is substantially the same as the face inclination in the highly reliable face image among the face image candidates; Detection device. A program for causing a computer to function as a face detection device for detecting a face image in an input image,
The computer
Partial image cutout means for cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the partial image. Index value calculating means for calculating the predetermined inclination by changing to a plurality of different inclinations, respectively;
A face for extracting all partial images having a calculated index value greater than or equal to the first threshold as candidate face images including the face in the predetermined direction when the index value is calculated from the plurality of partial images. Image candidate extraction means,
Highly reliable face image determining means for determining, as a highly reliable face image, a candidate whose calculated index value is greater than or equal to a second threshold value greater than the first threshold value from among the face image candidates;
Index value increasing means for performing a process of increasing the index value of the candidate for the face image candidates, with respect to a candidate whose face inclination is substantially the same as the face inclination in the highly reliable face image;
Causing the candidate whose detected index value is equal to or greater than a third threshold value, which is a value between the first threshold value and the second threshold value, to function as a face image detection unit that detects the face image; A program characterized by A program for causing a computer to function as a face detection device for detecting a face image in an input image,
The computer
Partial image cutout means for cutting out partial images at different positions on the input image;
For each of the plurality of partial images cut out at the different positions, an index value indicating the probability that the partial image is a face image including a face with a predetermined inclination based on the feature amount on the partial image. Index value calculating means for calculating the predetermined inclination by changing to a plurality of different inclinations, respectively;
Out of the plurality of partial images, all partial images whose calculated index value is equal to or greater than the first threshold are extracted as face image candidates including the face having the predetermined inclination when the index value is calculated. Face image candidate extraction means,
Highly reliable face image determining means for determining, as a highly reliable face image, a candidate whose calculated index value is greater than or equal to a second threshold value greater than the first threshold value from among the face image candidates;
A program that functions as a face image detection unit that detects, as the face image, only a candidate whose face inclination is substantially the same as the face inclination in the highly reliable face image among the face image candidates. .

Images