JP2006309715A - Face discrimination method and device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress lowering of discrimination accuracy by suppressing noise components caused in a normalization process in the process of normalizing an image subjected to face image discrimination such that a uniform degree of variance of pixel values suppresses contrast variations and then discriminating the image. <P>SOLUTION: Out of local regions of a discrimination target image, local regions in which the degree of variance of pixel values is equal to or greater than a predetermined level are subjected to first brightness gradation conversion for approximating the degree of variance to a constant level, and local regions in which the degree is less than the predetermined level are subjected to second brightness gradation conversion for reducing the degree of variance to a level lower than the constant level or to no brightness gradation conversion. In regions of originally flat contrast such as the forehead and cheeks of a face and a background sky, the flatness is maintained to suppress noise. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a face discrimination method and apparatus for discriminating whether or not a target image is a face image, and a program therefor.

  Conventionally, the color distribution of a person's face area in a snapshot photographed by a digital camera is examined to correct the skin color, or a person in a digital image photographed by a digital video camera of a surveillance system is recognized. Has been done. In such a case, it is necessary to detect a face area corresponding to the face of a person in the digital image. For this reason, various methods for determining whether or not the target image is an image representing a face have been proposed. .

  For example, it is calculated from each of a number of sample image groups including a plurality of sample images known to be images representing a predetermined object and a plurality of sample images known to be images not representing the predetermined object. A reference value for discriminating between an image representing a predetermined object and an image not representing the predetermined object by inputting the feature amount, which is obtained by learning the obtained feature amount in advance by a machine learning method. This technique includes a plurality of discriminators, and discriminates that a discrimination target image is an image representing a predetermined target when a weighted sum of reference values output from the plurality of discriminators exceeds a predetermined threshold. It has been proposed by the applicant (see Patent Documents 1 to 3).

  In addition, the feature amount calculated from each of a large number of sample images including a plurality of sample images known to be images representing a face and a plurality of sample images known to be images not representing a face is obtained. A plurality of weak discriminators, which are obtained by learning in advance by a machine learning method, and that determine whether or not the discrimination target image is an image representing a face by input of a feature amount. A method has also been proposed in which when a weakly classifier is identified as an image representing a face, the image to be identified is an image representing a face when the images are represented in a cascade structure. Non-patent document 1).

  In these methods, the feature amount is often used as a feature amount related to the luminance distribution of the discrimination target image, and discrimination is often performed using this feature amount.

  By the way, in the snapshot image acquired by a digital camera or the like, the contrast (degree of light and darkness) varies due to, for example, the position of the light source or the shape of the subject due to the shooting scene, shooting conditions, and the like. Although the image may be an image, if discrimination based on the feature amount related to the luminance distribution is performed on an image with such contrast variations, the feature image will not properly reflect the likelihood of a face image. Will be reduced.

Therefore, in order to suppress such variation in contrast, as a pre-processing for face discrimination, a normalization process is performed uniformly to uniformize the degree of dispersion of pixel values representing luminance with respect to a discrimination target image to a certain level. Techniques are known.
Japanese Patent Application No. 2003-316924 Japanese Patent Application No. 2003-316925 Japanese Patent Application No. 2003-316926 "High-speed omnidirectional face detection", Shihong LAO et al., Image Recognition and Understanding Symposium (MIRU2004), July 2004, P.II-271-II-276

  However, with the above-described method of uniformly performing the normalization process for equalizing the degree of dispersion of the pixel values on the determination target image to a certain level, for example, a region representing the forehead and cheeks of the face, and the background sky For example, even in areas that originally have a flat contrast, such as areas that represent inevitably, the degree of dispersion of the pixel values is brought close to a certain level, so the brightness varies and the noise component for the face discrimination process is increased to increase the accuracy of face discrimination. There is a problem that it may be lowered.

  In view of the above circumstances, an object of the present invention is to provide a face discrimination method and apparatus capable of suppressing a reduction in face discrimination accuracy, and a program therefor.

  The face discrimination method of the present invention includes a normalization step of performing a normalization process for suppressing a variation in contrast in an image to be discriminated as a target for discriminating whether or not the image is a face image, and the normalization A face having a face determination step of calculating at least one feature amount related to the luminance distribution of the processed discrimination target image and discriminating whether or not the discrimination target image is a face image using the feature amount In the determination method, the normalization process may be configured to reduce the degree of dispersion from the predetermined level for each local region in the determination target image with respect to a local region in which the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level. A first luminance gradation conversion process is performed so as to approach a high constant level, and the degree of dispersion is lower than the certain level for a local region where the degree of dispersion of the pixel value is less than the predetermined level. A method which is a second luminance gradation conversion processing performed processing suppressed to have levels (first face detection method).

  The face discrimination method of the present invention includes a normalization step of performing a normalization process for suppressing a variation in contrast in an image to be discriminated as a target for discriminating whether or not the image is a face image, and the normalization A face having a face determination step of calculating at least one feature amount related to the luminance distribution of the processed discrimination target image and discriminating whether or not the discrimination target image is a face image using the feature amount In the determination method, the normalization process is configured to set the degree of dispersion for the local area in the determination target image only for the local area where the degree of dispersion of pixel values representing luminance is equal to or higher than the predetermined level. This is a method characterized by performing a first luminance gradation conversion process that approaches a higher constant level (second face discrimination method).

  The face discriminating apparatus of the present invention includes a normalizing unit that performs a normalization process for suppressing a variation in contrast in an image to be discriminated as a target for discriminating whether or not the image is a face image, and the normalization Face discrimination means for calculating at least one feature amount relating to the luminance distribution of the processed discrimination target image and discriminating whether or not the discrimination target image is a face image using the feature amount; In the face discrimination device, the normalization processing is performed on the local area where the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level for each local area in the discrimination target image. A first luminance gradation conversion process is performed to bring the pixel value closer to a higher constant level, and the degree of dispersion is reduced to a level lower than the predetermined level for a local region where the degree of dispersion of the pixel value is less than the predetermined level. Is characterized in that a second luminance gradation conversion processing performed processing to suppress the (first face detection device).

  The face discriminating apparatus of the present invention includes a normalizing unit that performs a normalization process for suppressing a variation in contrast in an image to be discriminated as a target for discriminating whether or not the image is a face image, and the normalization Face discrimination means for calculating at least one feature amount relating to the luminance distribution of the processed discrimination target image and discriminating whether or not the discrimination target image is a face image using the feature amount; In the face discriminating apparatus, the normalization process is configured to set the degree of dispersion only for a local region where the degree of dispersion of pixel values representing luminance is a predetermined level or more for each local region in the discrimination target image. This is a process for performing a first luminance gradation conversion process that approaches a certain level higher than the level (second face discrimination device).

  The program according to the present invention includes a normalizing unit that performs normalization processing for suppressing a variation in contrast in an image on a determination target image that is a target for determining whether or not the image is a face image; At least one feature amount related to the luminance distribution of the discrimination target image that has been subjected to the digitization process, and functions as a face discrimination unit that discriminates whether or not the discrimination target image is a face image using the feature amount In the program for the above, the normalization processing is performed for each local region in the determination target image, with respect to a local region where the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level. A first luminance gradation conversion process is performed to bring the pixel value closer to a higher constant level, and the degree of dispersion for a local region where the degree of dispersion of the pixel values is less than the predetermined level It is characterized in that a process for performing the second luminance gradation conversion processing suppressed to a level lower than the predetermined level (first program).

  The program according to the present invention includes a normalizing unit that performs normalization processing for suppressing a variation in contrast in an image on a determination target image that is a target for determining whether or not the image is a face image; At least one feature amount related to the luminance distribution of the discrimination target image that has been subjected to the digitization process, and functions as a face discrimination unit that discriminates whether or not the discrimination target image is a face image using the feature amount In the program for performing the normalization processing, the normalization processing is performed by setting the degree of dispersion only for a local area where the degree of dispersion of pixel values representing luminance is a predetermined level or more for each local area in the determination target image. This is a process for performing a first luminance gradation conversion process that approaches a certain level higher than the level (second program).

  In the present invention, the normalization processing sequentially sets each pixel in the discrimination target image as a target pixel, and for each target pixel, the pixel value in a local region having a predetermined size represented by the target pixel. When the variance is calculated, and at least when the variance is equal to or greater than a threshold corresponding to the predetermined level, as the first luminance gradation conversion process, the larger the variance is the reference value corresponding to the certain level, the more attention is paid The difference between the pixel value of the pixel and a predetermined statistical value of the pixel value in the local region represented by the pixel of interest is reduced, and the pixel value of the pixel of interest is reduced as the variance is smaller than the reference value. And gradation conversion for increasing the difference between the predetermined representative value and the predetermined representative value.

  The predetermined level may be changed according to the whole or a part of luminance in the local region. For example, in the above-described normalization process in which gradation conversion is performed for each target pixel, the threshold value may be changed according to the pixel value of the target pixel. That is, the threshold corresponding to the predetermined level may be set higher when the luminance of the pixel of interest is relatively high, and may be set lower when the luminance is relatively low. In this way, a face that exists in a low-luminance (dark) region with a low contrast (small dispersion value) can be correctly normalized.

  In the face discrimination method of the present invention, the face discrimination step includes a plurality of different weak discriminators (modules) that discriminate whether or not the discrimination target image is a face image, and the weak discriminator has high reliability. It can be a step of performing processing in the face discrimination means having a structure formed by linearly connecting in order. In the face discriminating apparatus and program according to the present invention, the face discriminating unit may include a plurality of different weak discriminators (modules) that discriminate whether or not the discrimination target image is a face image. It can have a structure that is linearly coupled in descending order of reliability.

  In this case, furthermore, in the face discrimination method of the present invention, the face discrimination step includes a plurality of different learning face images in which the face orientation is the same as the face orientation to be discriminated and the face orientations are aligned. By the learning using, the type of the weak discriminator used for discrimination and the process in the face discriminating means in which the order of combining the weak discriminators is determined can be set as a step. In the face discriminating apparatus and program according to the present invention, the face discriminating means uses a plurality of different learning face images having the same face orientation as the face orientation to be discriminated and having the face orientations aligned. According to the learning, the type of the weak classifier used for discrimination and the order of combining the weak classifiers can be determined.

  Here, the “degree of dispersion of pixel values” means the degree of dispersion of pixel values. For example, in addition to so-called mathematical dispersion values of pixel values, the maximum value and the minimum value of pixel values It can be a difference value or the like. In addition, the “predetermined level” is one of the values that the variance value or the difference value can take, such as an image that has a flat contrast between the face and cheeks, the background sky, and the like. The variance value or the difference value corresponding to a boundary that can be distinguished from an image can be used.

  The predetermined level should be set according to a noise level that differs for each image, and when the input source is known, it is preferable to change the predetermined level accordingly. For example, the image picked up by the image pickup means built in the mobile phone is relatively noisy, so the predetermined level is set high in order to suppress noise amplification. Further, for example, when the noise level of the image can be known from the analysis information obtained from the noise analysis means, the gain information at the time of image pickup obtained from the image pickup means, etc., the larger the noise level, the higher the predetermined level. Is set higher, and the predetermined level is preferably set lower as the noise level is lower.

  The “local region of a predetermined size typified by the pixel of interest” is, for example, a first region Z1 having the pixel of interest x approximately at the center, approximately the center of gravity, or approximately the center as shown in FIG. This means the second region Z2, which is a partial region within the first region, which is set according to the distribution of pixel values in the first region Z1. As the second region Z2, for example, as shown in FIG. 17, when the shape of the histogram of pixel values in the first region Z1 is a shape having a plurality of peaks, the pixel of interest among these peaks It can be set as the area | region which consists of a pixel corresponding to the peak containing the pixel value X of x. That is, a region obtained by removing a region Za composed of pixels corresponding to a mountain not including the pixel value X of the target pixel x from the first region Z1 can be set as the second region Z2. Such a region setting is performed only when the first region Z1 extends over regions having clearly different image densities in order to avoid an adverse effect due to the boundary of the density change. This is a method often used in the case of calculating a value.

  The “predetermined representative value in the pixel value statistics” is a central value of the distribution representing the characteristics of the distribution of the pixel value. For example, the statistical average value and the median value of the pixel value , Intermediate values, mode values, and the like.

  The “local region” is preferably a rectangular region for convenience, but may be a shape such as a perfect circle or an ellipse.

  In the present invention, it is sufficient that at least the learning face image is used as a face image used for learning. Of course, a learning non-face image may be used in addition to the learning face image as an image used for learning. Absent.

  “Learning face image” means a sample image used for learning that is known to be an image representing a face, and “Non-face image for learning” is used for learning that is not known to represent a face. A sample image.

  “Faces of the top and bottom of the face are aligned” is not limited to the state in which the top and bottom of the face are completely coincident, and a predetermined angle range on the image plane, for example, ± 15 degrees is allowed. To do.

  “Weak discriminator” is a discriminating means (module) with a correct answer rate exceeding 50%, and “a structure in which a plurality of weak discriminators are linearly combined” is such a weak discriminator connected in series. The weak classifier has a structure configured to proceed to the next weak classifier when the target image is determined to be a face image and to leave the determination process when it is determined to be a non-face image. That means. The target image determined to be a face image by the last weak classifier is finally determined to be a face image.

  According to the face discriminating method and apparatus and the program therefor according to the present invention, in the normalization processing for the face discrimination target image, the normalization processing is not performed uniformly, but the degree of dispersion of pixel values representing luminance for this image Is applied to a local area where the variance is higher than a predetermined level, and luminance gradation conversion processing is performed to bring the variance closer to a certain level. Since the luminance gradation conversion processing is performed or the luminance gradation conversion processing is not performed, the original contrast such as the forehead of the face, cheeks, background sky, etc., where the dispersion of pixel values is considered to be below the predetermined level In a flat area, the dispersion of pixel values can be prevented from increasing unnecessarily and the luminance can vary, and the noise component for face discrimination processing can be suppressed. It is possible to suppress a decrease in accuracy of the determination.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a schematic block diagram showing the configuration of a face detection system to which the face discrimination device of the present invention is applied. This face detection system detects a face included in a digital image regardless of the position, size, orientation, and rotation direction of the face. As shown in FIG. 1, the face detection system 1 multi-resolutions an input image S0 that is a target for detecting a face to obtain a plurality of images having different resolutions (hereinafter referred to as resolution images) S1_i (i = 1, 2, 3). ..)) And a pre-processing aimed at improving the accuracy of the face detection processing to be executed later, as a pre-process for the resolution image S1_i. A local normalization unit (normalization means) 20 that obtains a local normalized resolution image S1′_i by performing normalization that suppresses variation (hereinafter referred to as local normalization), and a local normalized resolution image S1 ′ First face detection unit 30 that extracts face candidate S2 by performing rough face detection processing on each of _i, and highly accurate face detection for an image in a predetermined neighborhood region including each face candidate processing The second face detection unit 40 that further narrows down the face candidates S2 and obtains the face S3 that seems to be a true face, and the face S3 detected on each resolution image overlaps the same face. It is provided with an overlap detection determination unit 50 that determines whether or not it is detected based on its positional relationship and arranges it, and obtains a face S3 ′ without overlap detection.

  The multi-resolution conversion unit 10 converts the resolution (image size) of the input image S0 to normalize the resolution to a predetermined resolution, for example, an image having a rectangular size with a short side of 416 pixels. An image S1 is obtained. Then, by further performing resolution conversion using this image S1 as a basic image, a plurality of resolution images S1_i having different resolutions are generated. The reason for generating such a plurality of resolution images is usually that the size of the face included in the input image is unknown, but the size of the face to be detected (image size) is determined by a discriminator described later. This is because it is necessary to cut out partial images of a predetermined size on images with different resolutions and determine whether the partial image is a face or a non-face. Specifically, as illustrated in FIG. 2, an image S1 is a basic image S1_1, an image S1_2 that is −1/3 times a power of 2 with respect to the image S1_1, and a −1/3 power of 2 with respect to the image S1_2. The image S1_3 that is doubled (2 to the power of 2/3 for the basic image S1_1) is generated first, and then a reduced image that is ½ times the size of each of the images S1_1, S1_2, and S1_3 Are generated, and a reduced image of 1/2 size is further generated with respect to the reduced images, so that a predetermined number of reduced images are generated. In this way, a plurality of images whose resolution is reduced by a factor of −1/3 times from the basic image mainly by a reduction process of 1/2 times that does not require an interpolation process of pixel values representing luminance. Can be generated at high speed. For example, when the image S1_1 has a rectangular size of 416 pixels on the short side, the images S1_2, S1_3,... Have 330 pixels, 262 pixels, 208 pixels, 165 pixels, 131 pixels, 104 pixels, and 82 on the short sides, respectively. A rectangular size of pixels, 65 pixels,... Can be generated, and a plurality of resolution images reduced by −1/3 times power can be generated. Note that an image generated without interpolating pixel values in this way has a strong tendency to carry the characteristics of the image pattern as they are, and is preferable in that an improvement in accuracy can be expected in face detection processing.

  For each of the resolution images S1_i, the local normalization unit 20 sets the degree of dispersion for each local region in the resolution image with respect to a local region in which the degree of dispersion of the pixel values representing luminance is a predetermined level or more. A first luminance gradation conversion process is performed to approach a certain level higher than the predetermined level, and the degree of dispersion is lower than the certain level for a local region where the degree of dispersion of the pixel values is less than the predetermined level. A second luminance gradation conversion process that suppresses the level is performed. Here, a specific process in the local normalization unit 20 will be described.

FIG. 12 is a diagram illustrating the concept of local normalization processing, and FIG. 13 is a diagram illustrating a processing flow in the local normalization unit 20. Expressions (1) and (2) are gradation conversion expressions for pixel values for the local normalization process.

Here, X is the pixel value of the pixel of interest, X ′ is the pixel value after conversion of the pixel of interest, mlocal is the average of the pixel values in the local region centered on the pixel of interest, Vlocal is the variance of the pixel values in this local region, SDlocal is a standard deviation of pixel values in this local area, (C1 × C1) is a reference value corresponding to the above-mentioned fixed level, C2 is a threshold corresponding to the above-mentioned predetermined level, and SDc is a predetermined constant. In the present embodiment, the number of gradations of luminance is 8 bits, and the possible pixel values are 0 to 255.

  As shown in FIG. 13, the local normalization unit 20 sets one pixel in the resolution image as a target pixel (step S21), and has a predetermined size centered on the target pixel, for example, an 11 × 11 pixel size. A variance Vlocal of pixel values in the local region is calculated (step S22), and it is determined whether or not the variance Vlocal is equal to or greater than a threshold C2 corresponding to the predetermined level (step S23). If it is determined in step S23 that the variance Vlocal is greater than or equal to the threshold value C2, the variance Vlocal is larger than the reference value (C1 × C1) corresponding to the certain level as the first luminance gradation conversion process. The tone conversion that decreases the difference between the pixel value X of the target pixel and the average mlocal, and increases the difference between the pixel value X of the target pixel and the average mlocal as the variance mlocal is smaller than the reference value (C1 × C1). Is performed according to the equation (1) (step S24). On the other hand, if it is determined in step S23 that the variance Vlocal is less than the threshold value C2, linear tone conversion that does not depend on the variance Vlocal is performed as the second luminance tone conversion processing according to equation (2). This is performed (step S25). Then, it is determined whether or not the target pixel set in step S21 is the last pixel (step S26). If it is determined in step S26 that the target pixel is not the last pixel, the process returns to step S21, and the next pixel on the same resolution image is set as the target pixel. On the other hand, if it is determined in step S26 that the target pixel is the last pixel, the local normalization for the resolution image is terminated. In this way, by repeating the processes of steps S21 to S26, a resolution image in which local normalization is performed on the entire resolution image is obtained. By performing this series of processing on each resolution image, a locally normalized resolution image S1′_i is obtained.

  Note that the predetermined level may be changed according to the whole or a part of luminance in the local region. For example, in the normalization process in which gradation conversion is performed for each target pixel, the threshold value C2 may be changed according to the pixel value of the target pixel. That is, the threshold value C2 corresponding to the predetermined level may be set higher when the luminance of the target pixel is relatively high, and may be set lower when the luminance is relatively low. In this way, it is possible to correctly normalize a face that exists in a low-brightness, so-called dark area with low contrast (a state in which the dispersion of pixel values is small).

  Further, here, a case where only local normalization is performed on the resolution image has been described, but normalization different from local normalization may be performed simultaneously. For example, after performing gradation conversion using a look-up table (LUT) set so as to increase the contrast of a so-called dark area with low luminance (corresponding to increasing the dispersion of pixel values), The local normalization may be performed. By doing this, the same effect as changing the threshold value C2 according to the pixel value of the target pixel as described above can be obtained, and a face existing in a dark region with low contrast can be correctly normalized. Can do.

  The first face detection unit 30 performs a relatively coarse and high-speed face detection process on each resolution image S1′_i that has been subjected to the local normalization process by the local normalization unit 20, and performs face detection from each resolution image S1′_i. Candidate S2 is provisionally extracted. FIG. 3 is a block diagram showing a configuration of the first face detection unit 30. As shown in FIG. 3, the first face detection unit 30 sequentially sets subwindows W for cutting out partial images (discrimination target images) to be discriminated as to whether or not each resolution image is a face image. Sub-window setting unit 31, a first front face discriminator (face discrimination means) 33 for discriminating whether or not the partial image is mainly a front face image, and this partial image is mainly a left side face image. A first left side profile discriminator (face discrimination means) 34 for discriminating whether or not there is, and a first right side profile discriminator (face discrimination) for discriminating whether or not this partial image is mainly a right side profile image Each discriminator 33 to 35 has a cascade structure in which a plurality of weak discriminators WCi (i = 1 to N) are linearly coupled.

  For each resolution image S1′_i, the first sub-window setting unit 31 rotates a resolution image 360 degrees on the plane, and a sub-window W for cutting out a 32 × 32 pixel size partial image on the resolution image. The predetermined partial number is sequentially set while moving, for example, 5 pixels, and the cut out partial images are set as the first front face discriminator 33, the first left side face discriminator 34, and the first right side face discriminator 35. Respectively.

  Each of the discriminators 33 to 35 determines whether or not each partial image is a face image representing a front face, a left side face, or a right side face with respect to the sequentially input partial images. In S1′_i, a front face, a left side face, and a right side face at every rotation angle on the plane are detected, and a face candidate S2 is output. In addition, in order to improve the detection accuracy of the oblique face, a discriminator for discriminating each of the right oblique face and the left oblique face may be further provided, but it is not particularly provided here.

  Each of the discriminators 33 to 35 calculates at least one feature amount related to the luminance distribution of the partial image, and determines whether or not the partial image is a face image using the feature amount. Here, specific processing in each of the discriminators 33 to 35 will be described. FIG. 5 shows a general processing flow in each discriminator, and FIG. 6 shows a processing flow by each weak discriminator therein.

  First, the first weak discriminator WC1 discriminates whether or not this partial image is a face with respect to the partial image of a predetermined size cut out on the resolution image S1′_i (step SS1). Specifically, as shown in FIG. 7, the weak discriminator WC1 performs four neighboring pixels on a partial image of a predetermined size cut out on the resolution image S1′_i, that is, an image of 32 × 32 pixel size. By performing an average (a process in which an image is divided into a plurality of blocks for each 2 × 2 pixel size and an average value of pixel values of four pixels of each block is set to a pixel value of one pixel corresponding to the block), A pair of a plurality of types is obtained by obtaining a 16 × 16 pixel size image and a reduced image of 8 × 8 pixel size as a pair of two predetermined points set in the plane of these three images. The difference value of the brightness | luminance between two points in each pair which comprises a group is each calculated, and the combination of these difference values is made into a feature-value (step SS1-1). The predetermined two points of each pair are, for example, two predetermined points arranged in the vertical direction and two predetermined points arranged in the horizontal direction so as to reflect the characteristics of the facial shading on the image. Then, a score is calculated by referring to a predetermined score table according to a combination of difference values as feature amounts (step SS1-2), and the score calculated by itself is added to the score calculated by the previous weak discriminator. The accumulated score is calculated (step SS1-3). However, since the first weak discriminator WC1 has no previous weak discriminator, the score calculated by itself is used as the cumulative score. It is determined whether or not the partial image is a face depending on whether or not the accumulated score is equal to or greater than a predetermined threshold (step SS1-4). Here, when the partial image is determined to be a face, the process proceeds to determination by the next weak classifier WC2 (step SS2). When the partial image is determined to be a non-face, the partial image is immediately determined to be a non-face. It is determined (step SSB), and the process ends.

  Also in step SS2, as in step SS1, the weak classifier WC2 calculates the above-described feature amount representing the feature on the image based on the partial image (step SS2-1), and refers to the score table for the feature. A score is calculated from the amount (step SS2-2). Then, the score calculated by itself is added to the cumulative score calculated by the previous weak discriminator WC1 to update the cumulative score (step SS2-3), and the partial image is determined depending on whether or not the cumulative score is equal to or greater than a predetermined threshold. Is a face (step SS2-4). Again, when the partial image is determined to be a face, the process proceeds to determination by the next weak classifier WC3 (step SS3). When the partial image is determined to be a non-face, the partial image is immediately determined to be a non-face. (Step SSB), and the process ends. In this way, when it is determined that the partial image is a face in all N weak classifiers, the partial image is finally extracted as a face candidate (step SSA).

  Each of the discriminators 33 to 35 is a discriminator composed of a plurality of weak discriminators determined by unique feature type, score table, and threshold value. A face, a left side profile, and a right side profile are identified.

  The second face detection unit 40 performs face detection processing with relatively high accuracy on each image in a predetermined area in which each of the face candidates S2 is included in all resolution images, and from the images near the face candidates. The true face S3 is detected. The second face detection unit 40 basically has the same configuration as that of the first face detection unit 30. As shown in FIG. 4, the second face detection unit 40 and the second front face are arranged. It comprises a discriminator 43, a second left side face detector 44, and a second right side face detector 45. Each of the discriminators 43 to 45 includes a plurality of weak discriminators WCi (i = 1). To N) are linearly coupled to form a cascade structure. However, it is preferable that these discriminators have higher discrimination accuracy than the discriminators in the first face detection unit 30. In the second face detection unit 40, the general processing flow in the classifier and the processing flow in the weak classifier are basically the same as those in the first face detection unit 30, but the sub window W is set. The position is limited to an image within a predetermined area including the face candidate S2 extracted by the first face detection unit 30, and the movement width of the subwindow W is finer than that of the first face detection unit 30, for example, One pixel at a time. As a result, the face candidates S2 roughly extracted by the first face detection unit 30 are further narrowed down and only the true face S3 is output.

  The duplication detection determination unit 50 detects duplicated faces detected on each resolution image based on the position information of the face S3 on each resolution image S1′_i detected by the second face detection unit 40. A process of grouping the same faces as one face is performed, and position information of the face S3 ′ detected in the input image S0 is output. Although the discriminator depends on the learning method, the size of the face that can be detected with respect to the size of the partial image generally has a certain width, and therefore, in a plurality of resolution images having adjacent resolution levels, the same face is detected. This is because it may be detected in duplicate.

  FIG. 9 is a flowchart showing the flow of processing in the face detection system. As shown in FIG. 9, when the input image S0 is supplied to the multi-resolution conversion unit 10 (step S1), an image S1 in which the image size of the input image S0 is converted to a predetermined size is generated, and the image S1 is generated. A plurality of resolution images S <b> 1 </ b> _i whose resolution is reduced by 2 to 1/3 times are generated (step S <b> 2). Then, the local normalization unit 20 performs local normalization processing for suppressing variation in contrast in a local region in the entire image of each resolution image S1_i, that is, for a region where the dispersion of pixel values is equal to or greater than a predetermined threshold. Local brightness gradation conversion is performed to reduce the dispersion to a level lower than the predetermined level for areas where the dispersion of pixel values falls below the predetermined threshold. Normalization is performed, and a locally normalized resolution image S1′_i is obtained (step S3). The first face detection unit 30 sequentially cuts out partial images of a predetermined size according to the setting of the subwindow W in each resolution image S1′_i that has been locally normalized by the first subwindow setting unit 31 (step S4). Using the front face, right side face, and left side face discriminators 33 to 35, face discrimination is performed on each of these partial images, and a face candidate S2 for each resolution image S1′_i is roughly detected (step S5). ). Further, the second face detection unit 40 cuts out a partial image based on the setting of the subwindow W (step S6), similar to the first face detection unit 30, with respect to the neighborhood image of the face candidate S2 extracted in step S5. ) Face detection corresponding to scrutiny is performed using the front face, right side face, and left side face discriminators 43 to 45, and the face is narrowed down to the true face S3 (step S7). Then, the same face detected redundantly in each resolution image S1′_i is determined (step S8), and these are collectively combined into one finally detected face S3 ′.

  Next, a learning method for the classifier will be described. FIG. 10 is a flowchart showing a learning method of the classifier. Note that learning is performed for each type of discriminator, that is, for each orientation of the face to be discriminated.

  The sample image group to be learned is a plurality of sample images that are known to be faces and a plurality of samples that are known to be non-faces, standardized at a predetermined size, for example, 32 × 32 pixel size. It consists of an image. As a sample image that is known to be a face, an image in which the face orientation is the same as the face orientation to be discriminated by the discriminator and the face orientations are aligned is used. A sample image that is known to be a face is obtained by scaling in steps of 0.1 times in the range of 0.7 to 1.2 times in length and / or width for each sample image. For each sample image to be obtained, a plurality of deformation variations obtained by rotating stepwise in units of 3 degrees within a range of ± 15 degrees on a plane is used. At this time, the face sample image is standardized in size and position so that the eye position is at a predetermined position, and the above-described rotation and scaling on the plane are performed based on the eye position. For example, in the case of a d × d size front face sample image, as shown in FIG. 14, the positions of both eyes are 1/4 d inward from the upper left vertex and the upper right vertex of the sample image, respectively. The face size and position are normalized so as to come to each position moved 1 / 4d downward, and the above-mentioned rotation and scaling on the plane are performed around the middle point of both eyes. Each sample image is assigned a weight or importance. First, the initial value of the weight of all the sample images is set equal to 1 (step S11).

  Next, when a plurality of pairs of groups consisting of a plurality of pairs are set with a predetermined two points set in the plane of the sample image and the reduced image as one pair, each of the plurality of types of pairs is weak. A separate device is created (step S12). Here, each weak discriminator sets one pair group consisting of a plurality of pairs with a predetermined two points set in the plane of the partial image cut out in the sub-window W and the reduced image as one pair. This provides a reference for discriminating between a face image and a non-face image using a combination of difference values of pixel values (luminance) between two points in each pair constituting this one pair group. . In the present embodiment, a histogram for a combination of pixel value difference values between two points in each pair constituting one pair group is used as the basis of the score table of the weak classifier.

  The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 11, two points of each pair constituting the pair group for creating this discriminator are a plurality of sample images that are known to be faces. The right eye on the reduced image of 16 × 16 pixel size in which the point in the center of the right eye is P1, the point in the right cheek part is P2, the point in the part between the eyebrows is P3, and the sample image is reduced by an average of four neighboring pixels The point at the center of P4, the point at the cheek on the right side is P5, and the point at the forehead part on the reduced image of 8 × 8 pixel size reduced by the average of 4 neighboring pixels is P6, the mouth part A certain point is P7, and there are five pairs of P1-P2, P1-P3, P4-P5, P4-P6, and P6-P7. Note that the coordinate positions of the two points of each pair constituting one pair group for creating a certain classifier are the same in all sample images. For all sample images that are known to be faces, combinations of pixel value difference values between two points of each of the five pairs are obtained, and a histogram thereof is created. Here, the value that can be taken as a combination of the difference values of the pixel values depends on the number of luminance gradations of the image, but if it is a 16-bit gradation, there are 65536 different values for the difference value of one pixel value, As a whole, the number of gradations is (the number of pairs), that is, 65536 to the fifth power, and a large number of samples, time, and memory are required for learning and detection. For this reason, in the present embodiment, the difference value of the pixel value is divided by an appropriate numerical value width and quantized to be n-valued (for example, n = 100).

  As a result, the number of combinations of difference values of pixel values is n to the fifth power, so that the number of data representing combinations of difference values of pixel values can be reduced.

  Similarly, histograms are created for a plurality of sample images that are known not to be faces. For sample images that are known not to be faces, positions corresponding to the positions of the two predetermined points of each pair on the sample image that is known to be a face (similarly, reference numerals P1 to P7 are used). ) Is used. A histogram obtained by taking the logarithm of the ratio of the frequency values indicated by these two histograms and representing the histogram is the histogram used as the basis of the score table of the weak discriminator shown on the rightmost side of FIG. The value of each vertical axis indicated by the histogram of the weak classifier is hereinafter referred to as a discrimination point. According to this weak discriminator, an image showing the distribution of combinations of pixel value difference values corresponding to positive discrimination points is highly likely to be a face, and the possibility increases as the absolute value of the discrimination point increases. It can be said. Conversely, an image showing a distribution of combinations of difference values of pixel values corresponding to negative discrimination points is highly likely not to be a face, and the possibility increases as the absolute value of the discrimination point increases. In step S12, a plurality of weak discriminators in the above-described histogram format are created for combinations of pixel value difference values between two predetermined points of each pair constituting a plurality of types of pair groups that can be used for discrimination.

  Subsequently, the most effective weak discriminator for discriminating whether or not the image is a face is selected from the plurality of weak semi-divided devices created in step S12. The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak classifiers are compared, and the weak classifier showing the highest weighted correct answer rate is selected (step S13). That is, in the first step S13, since the weight of each sample image is equal to 1, the one with the largest number of sample images for which it is simply determined correctly whether or not the image is a face by the weak classifier is as follows: Selected as the most effective weak classifier. On the other hand, in the second step S13 after the weight of each sample image is updated in step S15, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step S13 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the combination of weak classifiers selected so far, that is, using the weak classifiers selected so far in combination (in the learning stage, the weak classifiers do not necessarily need to be linearly combined. ) It is ascertained whether the result of determining whether or not each sample image is a face image has exceeded a predetermined threshold value at a rate that matches the answer of whether or not it is actually a face image (step) S14). Here, the current weighted sample image group or the sample image group with equal weight may be used for evaluating the correct answer rate of the combination of weak classifiers. When the predetermined threshold value is exceeded, learning is terminated because it is possible to determine whether the image is a face with a sufficiently high probability by using the weak classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step S16 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far.

  In step S16, the weak discriminator selected in the most recent step S13 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not be correctly determined whether or not it is a face in the weak classifier selected in the most recent step S13 is increased, and the sample image that can be correctly determined whether or not the image is a face. Is reduced (step S15). The reason for increasing or decreasing the weight in this way is that in the selection of the next weak classifier, importance is placed on images that could not be correctly determined by the already selected weak classifier, and whether or not those images are faces is correct. This is because a weak discriminator that can be discriminated is selected to enhance the effect of the combination of the weak discriminators.

  Subsequently, the process returns to step S13, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  As a weak discriminator suitable for discriminating whether or not a face is repeated by repeating the above steps S13 to S16, the difference value of the pixel value between two predetermined points of each pair constituting a specific pair group If the weak discriminator corresponding to the combination is selected and the correct answer rate confirmed in step S14 exceeds the threshold value, the type of the weak discriminator used for determining whether or not it is a face and the determination condition are determined. (Step S17), thereby completing the learning. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to constitute one classifier. For each weak classifier, a score table for calculating a score according to a combination of pixel value difference values is generated based on the obtained histogram. Note that the histogram itself can also be used as a score table. In this case, the discrimination point of the histogram is directly used as a score.

  In the case of adopting the above learning method, the weak classifier uses a combination of difference values of pixel values between two predetermined points of each pair constituting a specific pair group, and a face image and a non-face image. Is not limited to the above-described histogram format, and may be anything, for example, binary data, a threshold value, a function, or the like. Further, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 11 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  Here, the optimum size of the local region in the local normalization process by the local normalization unit 20 will be considered.

  As described above, the local normalization process is a process for suppressing local contrast variation in the partial image cut out in the sub-window W. Specifically, each pixel in the partial image is sequentially set as a target pixel. In addition, for each target pixel, a variance of pixel values representing luminance in a local region having a predetermined size centered on the target pixel is calculated, and the pixel value of the target pixel is calculated as the variance is larger than a predetermined reference value. The difference between the average value of pixel values in the local region (which may be other statistical representative values) is reduced, and the pixel value of the pixel of interest and the pixel value of the local region become smaller as the variance is smaller than the reference value. This is a normalization process that performs gradation conversion that increases the difference from the average value.

  In such a local normalization process, the local region size determines how much the variation in contrast is locally suppressed, but in general, the larger the local region size, the higher the luminance. While variation can be suppressed, it becomes difficult to suppress fine contrast fluctuations, and as the size of the local region is smaller, fine contrast fluctuations are more likely to be suppressed, while brightness tends to vary. Also, when all or some of the multiple types of parts constituting the face enter the local area at the same time, for example, when a part of the nose enters the local area centering on the eyes The dispersion of the pixel values is sensitive to the pixel value of the nose portion and the ratio of the nose portion to the local region, and the pixel value of the eye portion located at the center varies unnaturally. In some cases, however, if the pixel values in a predetermined part constituting the face vary unnaturally in this way, the feature value related to the luminance distribution on the discrimination target image is not properly reflected and the discrimination accuracy is not accurately reflected. There is a problem that decreases.

  Therefore, in consideration of the characteristics and problems of the local normalization process, the appropriate size of the local region has a contrast that can be suppressed in light of the fact that the local normalization process is a pre-process of the face discrimination process. The balance between the degree of variation and the variation in luminance is good, and the size is such that a plurality of types of parts constituting the face do not enter at the same time in view of the distance between the parts constituting the face. As a local region satisfying such requirements, for example, as shown in FIG. 15, other parts do not enter empirically based on the smallest “eye” among the constituent parts of the face representing facial features. An area having a size that includes only one face eye to be discriminated by the discriminator can be obtained. Note that the size of the face eye to be discriminated by the discriminator is based on the size of the face eye in the sample image representing the face used for the learning, so this local region is the longitudinal direction of this region. Can be a region having a length between 1.1 and 1.8 times the average value of the width in the longitudinal direction of the eyes in the sample image representing the face used for learning of the discriminator. The size of the local region in the local normalization process by the local normalization unit 20 in this embodiment, that is, the 11 × 11 pixel size is an example of a size set from the above viewpoint.

  As described above, according to the face discrimination method and apparatus according to the present embodiment, in the normalization process for the face discrimination target image, the normalization process is not performed uniformly, but the distribution of the pixel values representing the luminance of the image is not performed. Luminance gradation conversion processing is performed for areas whose degree is equal to or higher than a predetermined level, and the degree of dispersion is suppressed to a level lower than the predetermined level for areas below the predetermined level. Since the luminance gradation conversion process is performed or the luminance gradation conversion process is not performed, the original contrast such as the forehead of the face, cheeks, background sky, etc., where the variance of the pixel values is considered to be below a predetermined level. In a flat area, the dispersion of pixel values can be prevented from increasing unnecessarily and the brightness can be prevented, and noise components for face discrimination processing can be suppressed. It is possible to suppress the reduction.

  In the case where the discriminator is configured as a linear combination of a plurality of weak discriminators as described above, if normalization processing is uniformly performed on the discrimination target image as in the past, In a region where the contrast is originally flat, the dispersion of the pixel value is forcibly increased and the luminance is unnecessarily dispersed. In the determination based on the above, it is difficult to determine that the non-face image is a non-face at the initial determination stage, resulting in an excessive determination process. On the other hand, according to the present embodiment, since the dispersion of pixel values is not forcibly increased for such a flat region, the flatness is maintained in the region where the contrast is flat, and the non-face image is displayed. It becomes easy to be determined as a non-face at the initial determination stage, and there is an effect of suppressing an increase in calculation time.

  In the present embodiment, luminance gradation conversion processing is performed to suppress the degree of dispersion to an area where the degree of dispersion of pixel values is below a predetermined level. As another method, for example, It is also conceivable that no luminance gradation conversion processing is performed on the area.

  Although the face discrimination method and apparatus according to the embodiment of the present invention has been described above, a program for causing a computer to execute each process in the face discrimination apparatus (local normalization unit and discriminator) is also an embodiment of the present invention. It is one of. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

Block diagram showing the configuration of the face detection system 1 The figure which shows the process of multiresolution of a detection target image The block diagram which shows the structure of the 1st face detection part 30. The block diagram which shows the structure of the 2nd face detection part 40 Diagram showing the global processing flow in the classifier The figure which shows the processing flow in the weak classifier The figure for demonstrating calculation of the feature-value in a weak discriminator Diagram for explaining resolution image rotation and sub-window movement in multiple resolution images The flowchart which shows the process performed in the face detection system 1 Flow chart showing the learning method of the classifier The figure which shows the method of deriving the histogram of the weak classifier Diagram showing the concept of local normalization processing The figure which shows the processing flow in the local normalization part The figure which shows the sample image of the face standardized so that the position of eyes may be in a predetermined position The figure which shows the example of the local area of appropriate size on the basis of the size of the eye The figure which shows the example of the local area | region which represented the attention pixel The figure which shows the example of the local area | region represented by the attention pixel set based on distribution of a pixel value

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face detection system 10 Multi-resolution part 20 Local normalization part (normalization means)
30 1st face detection part 31 1st subwindow setting part 33 1st front face discriminator (face discrimination means)
34 First left side face classifier (face discrimination means)
35 First right side face classifier (face discrimination means)
40 Second face detection unit 41 Second sub window setting unit 43 Second front face discriminator (face discrimination means)
44 Second left side face classifier (face discrimination means)
45 Second right side face classifier (face discrimination means)
50 Duplicate detection judgment part

Claims (10)

A normalization step of performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
A face determination step of calculating at least one feature amount relating to a luminance distribution of the determination target image subjected to the normalization process, and determining whether the determination target image is a face image using the feature amount; In the face discrimination method having
For each local region in the discrimination target image, the normalization process sets the degree of dispersion to a constant level higher than the predetermined level for a local region where the degree of dispersion of pixel values representing luminance is a predetermined level or higher. A first luminance gradation conversion process is performed so as to make a second luminance scale that suppresses the degree of dispersion to a level lower than the predetermined level for a local region where the degree of dispersion of the pixel value is less than the predetermined level. A face discrimination method characterized by being a process for performing a tone conversion process. A normalization step of performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
A face determination step of calculating at least one feature amount relating to a luminance distribution of the determination target image subjected to the normalization process, and determining whether the determination target image is a face image using the feature amount; In the face discrimination method having
In the normalization process, for each local region in the determination target image, the degree of dispersion is set to a certain level higher than the predetermined level only for a local region where the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level. A face discrimination method characterized by being a process of performing a first luminance gradation conversion process approaching to. Normalization means for performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
Face discriminating means for calculating at least one feature amount relating to a luminance distribution of the discrimination target image subjected to the normalization processing and discriminating whether or not the discrimination target image is a face image using the feature amount; In the face discrimination device equipped with
For each local region in the discrimination target image, the normalization process sets the degree of dispersion to a constant level higher than the predetermined level for a local region where the degree of dispersion of pixel values representing luminance is a predetermined level or higher. A first luminance gradation conversion process is performed so as to make a second luminance scale that suppresses the degree of dispersion to a level lower than the predetermined level for a local region where the degree of dispersion of the pixel value is less than the predetermined level. A face discriminating apparatus characterized by being a process for performing a tone conversion process. Normalization means for performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
Face discriminating means for calculating at least one feature amount relating to a luminance distribution of the discrimination target image subjected to the normalization processing and discriminating whether or not the discrimination target image is a face image using the feature amount; In the face discrimination device equipped with
In the normalization process, for each local region in the determination target image, the degree of dispersion is set to a certain level higher than the predetermined level only for a local region where the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level. A face discriminating apparatus characterized in that it is a process for performing a first luminance gradation conversion process approaching to.   The normalization process sequentially sets each pixel in the discrimination target image as a target pixel, and calculates a variance of pixel values in a local area of a predetermined size represented by the target pixel for each target pixel. When at least the variance is equal to or greater than a threshold value corresponding to the predetermined level, as the first luminance gradation conversion process, the pixel value of the target pixel is larger as the variance is larger than a reference value corresponding to the certain level. Between the pixel value of the pixel of interest and the predetermined value as the variance is smaller than the reference value. 5. The face discriminating apparatus according to claim 3, wherein the face discriminating apparatus performs gradation conversion for increasing a difference from the representative value.   6. The face discrimination apparatus according to claim 5, wherein the threshold value changes according to a pixel value of the target pixel.   The face discriminating unit has a structure in which a plurality of different weak discriminators that discriminate whether or not the discrimination target image is a face image are linearly coupled in descending order of reliability of the weak discriminators. The face discrimination device according to claim 3, wherein   The weak discriminator used for discrimination by the face discriminating means by learning using a plurality of different learning face images in which the face orientation is the same as the face orientation to be discriminated and the top and bottom faces are aligned. 8. The face discriminating apparatus according to claim 7, wherein the type and the order of combining the weak discriminators are determined. Computer
Normalization means for performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
As face discrimination means for calculating at least one feature amount related to the luminance distribution of the discrimination target image subjected to the normalization process and discriminating whether the discrimination target image is a face image or not using the feature amount. In the program to make it function,
For each local region in the discrimination target image, the normalization process sets the degree of dispersion to a constant level higher than the predetermined level for a local region where the degree of dispersion of pixel values representing luminance is a predetermined level or higher. A first luminance gradation conversion process is performed so as to make a second luminance scale that suppresses the degree of dispersion to a level lower than the predetermined level for a local region where the degree of dispersion of the pixel value is less than the predetermined level. A program characterized by being a process for performing a tone conversion process. Computer
Normalization means for performing a normalization process for suppressing a variation in contrast in the image on a determination target image that is a target for determining whether or not the image is a face image;
As face discrimination means for calculating at least one feature amount related to the luminance distribution of the discrimination target image subjected to the normalization process and discriminating whether the discrimination target image is a face image or not using the feature amount. In the program to make it function,
In the normalization process, for each local region in the determination target image, the degree of dispersion is set to a certain level higher than the predetermined level only for a local region where the degree of dispersion of pixel values representing luminance is equal to or higher than a predetermined level. A program characterized in that it is a process for performing a first luminance gradation conversion process to be close to.

Images