JP5317934B2 - Object detection apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect an object of a specific type, such as a face from a detection object image at high speed. <P>SOLUTION: A plurality of resolution images of different resolutions are obtained by multiplex resolution of a face detection object image with a predetermined magnification. A partial image is formed by scanning a plurality of windows which have sizes to interpolate a predetermined magnification between resolution images on the plurality of resolution images. It is determined whether the partial image is a face by computing based on the partial image, the amount of characteristic concerning distribution of pixel values of the partial image, and using the amount of the characteristic. For determination, a discriminator is used, which discriminates whether the partial image is the image of the object by using the amount of characteristic concerning the partial image. The discriminator has learned the amount of characteristic concerning distribution of the face for each size of window by a plurality of sample images which are rendered the same illumination correction and have different sizes corresponding to sizes of the plurality of windows. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an object detection apparatus and method for detecting a specific type of object such as a human face from a detection target image, and a program for causing a computer to execute the object detection method.

  Conventionally, the color distribution of a person's face area in a snapshot photographed by a digital camera is examined to correct the skin color, or a person in a digital image photographed by a digital video camera of a surveillance system is recognized. Has been done. In such a case, since it is necessary to detect a face region corresponding to a person's face in the digital image, various techniques for detecting a face in the digital image have been proposed so far. Among them, a method using a discriminator module (hereinafter simply referred to as a discriminator) generated by machine learning learning using sample images is known as a face detection method that is considered to have excellent detection accuracy and robustness. It has been.

  This method uses a face sample image group composed of a plurality of different face sample images and a non-face sample image group composed of a plurality of different non-face sample images that are known not to be faces. A classifier that can learn whether or not an image is a face image is generated and prepared, and a partial image is detected in an image that is a face detection target (hereinafter referred to as a detection target image). A method of detecting a face on a detection target image by sequentially cutting out, determining whether or not the partial image is a face using the above discriminator, and extracting a region of the partial image determined to be a face It is.

  Here, the classifier extracts a plurality of feature amounts related to the distribution of pixel values from the face sample image and the non-face sample image, and learning is performed using the feature amounts. It is composed of a plurality of weak classifiers for determining whether or not a face is present. Therefore, whether or not the partial image is a face is determined by extracting a plurality of feature amounts related to the distribution of pixel values from the partial image as in learning, and determining the extracted feature amounts by each weak classifier. Is done.

  In this method, a face having the same size as the face sample image can be accurately detected by cutting out the partial image so as to be the same size as the sample image. However, since the size of the face that may be included in the detection target image is not constant, using only the discriminator learned using only the face sample image of the same size, all of the faces included in the detection target image The face cannot be detected. For this reason, the partial image is cut out while gradually changing the size of the window to cut out the partial image, a plurality of corresponding feature amounts are extracted from the partial image, and the extracted feature amount is determined by the corresponding weak discriminator, A method for determining whether or not a partial image is a face has been proposed (see Non-Patent Document 1).

  Also, instead of the size of the window, the detection target image is multi-resolution to obtain a plurality of resolution images, a partial image is cut out from each resolution image using a window of the same size as the sample image, and the partial image is a face A method of determining whether or not is also proposed (see Patent Document 1). As shown in FIG. 20, the technique described in Patent Document 1 uses a detection target image as a basic resolution image S1_1, and a resolution image S1_2 having a size that is −1/3 times the size of the resolution image S1_1. And a resolution image S1_3 having a size of 2−1 / 3 times the size of the resolution image S1_2 (2−2 / 3 times the size of the basic image S1_1), and then generating the resolution image S1_2. A process of repeatedly generating a resolution image obtained by reducing each of S1_1, S1_2, and S1_3 to a half size and generating a resolution image obtained by further reducing the reduced resolution image to a half size is performed. A predetermined number of resolution images having different values are generated.

  By the way, since the discriminator is generally learned using a sample image with relatively good image quality, it is basically made for an image with a good image quality. On the other hand, as the detection target image, images with various brightness and contrast of the shooting scene are assumed. Therefore, for example, when the detection target image is an image taken in a dark place, even if an attempt is made to search for a dark part of the eye and a bright part of the nose representing facial features in this image, the brightness of the image affects, Searching may be difficult.

  For this reason, normalization is performed so that the contrast of the image is set to a certain level as a pre-process for the image to be detected or discriminated so that the face can be detected even if the brightness and contrast of the detection target image are different. There has been proposed a technique for performing illumination correction by performing processing. The following three methods are mainly proposed as a method for performing this normalization process.

  The first method is a method of performing conversion according to a conversion curve (lookup table) that approximates the pixel value of the entire image of the detection target image to a value representing the logarithm of the luminance of the subject in the image. The second method is a method of converting the pixel values for each partial image cut out from the detection target image so that the degree of dispersion of the pixel values (luminance values) in the region is made constant. The third method scans the local area of a predetermined size within the area of the partial image cut out from the detection target image, and adjusts the degree of dispersion of the pixel values in the local area to a certain level. Is a method to convert. Of the first to third methods, the third method has the highest degree of illumination correction because it is not easily affected by differences in light shielding, background, and input modality in the detection target image. In addition, when the detection target image is multi-resolution, the pixel value is set so that the degree of dispersion of the pixel value in the local region is made constant while scanning the local region of a predetermined size for each of the plurality of resolution images. It is also possible to perform illumination correction by conversion.

Paul Viola and Michael Jones, Rapid object detection using a boosted cascade of features, IEEE CVPR, 2001 JP 2007-25766 Gazette

  However, in the method described in Non-Patent Document 1, since a plurality of windows are used, the size of the partial image differs depending on the size of the window. Thus, when performing illumination correction by the 3rd method with respect to the partial image from which size differs, it is necessary to change the size of a local area | region according to the size of a partial image. For this reason, the calculation for illumination correction becomes very complicated, and as a result, a long time is required for the calculation for illumination correction.

  In the method described in Non-Patent Document 1, a plurality of windows are used. However, preparing a discriminator composed of weak discriminators that perform learning according to all window sizes is difficult to learn. Is not realistic. For this reason, in the method described in Non-Patent Document 1, the position of the pixel value for acquiring the feature amount determined by the weak classifier corresponds to each other between the partial images having different sizes. In addition, the scale of the weak classifier is changed. However, if a discriminator consisting of a weak discriminator whose scale has been changed is used, the accuracy of face detection decreases. Further, since it is necessary to change the scale for each size of the partial image, the calculation takes a long time.

  On the other hand, in the method described in Patent Document 1, since it is possible to perform illumination correction on each of a plurality of resolution images, it is easier than in the method described in Non-Patent Document 1. Illumination correction can be performed. In addition, since a single size window is used, there is no possibility that the accuracy of face detection is lowered. However, in order to improve the accuracy of face detection, as shown in FIG. 20, it is necessary to obtain a resolution image with many resolutions by reducing the difference in resolution in the resolution image. It takes time.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to detect a specific type of object such as a face from a detection target image with high accuracy and high speed.

An object detection apparatus according to the present invention includes a multi-resolution processing unit that multi-resolutions a detection target image for detecting a specific type of object at a predetermined magnification to obtain a plurality of resolution images having different resolutions;
A partial image generating means for generating a partial image by scanning a plurality of windows having a size for interpolating the predetermined magnification between the resolution images on the plurality of resolution images;
A determination unit that calculates a feature amount related to a distribution of pixel values of the partial image based on the partial image and determines whether the partial image is an image of the object using the feature amount; ,
The determination unit is configured to learn, for each window size, a feature amount related to the object distribution by using a plurality of sample images having different sizes corresponding to the sizes of the plurality of windows and subjected to the same illumination correction. In addition, the image processing apparatus includes a discriminator that determines whether or not the partial image is an image of the object using the feature amount related to the partial image.

As the “predetermined magnification”, any magnification that reduces the detection target image can be used, but it is preferable to use 1/2 ⁿ times that is relatively easy to calculate as the predetermined magnification.

The “size for interpolating a predetermined magnification” means a size for interpolating a magnification difference between a plurality of resolution images. “A plurality of windows having a size for interpolating a predetermined magnification” means that a window having a predetermined size is set as a reference window, and when the reference window is reduced at a predetermined magnification, the size between the reference window size and the reduced window size is intermediate. It means a plurality of windows having a size. For example, when the predetermined magnification is 1/2 ⁿ times, the difference in magnification between the resolution images is 1/2, that is, 2 ^−1. Therefore, the “multiple windows having a size for interpolating the predetermined magnification” is used as the reference window. , ² -1/4 size of the reference ^window 2 ^{-2/4, 2 -3/4} window or ^{2 -1/3} having a size of, the use of windows and the like having a size of ^{2 -2/3} it can.

  As described above, the present invention provides a discriminator for determining whether or not a partial image is an image of a specific type of object using a feature amount related to the partial image, and has different sizes corresponding to the sizes of a plurality of windows. The feature amount related to the distribution of the object is learned for each window size by using a plurality of sample images subjected to the same illumination correction. For this reason, even if the window size is different, there is no need to perform illumination correction for each partial image with different sizes, so the amount of computation for object detection can be reduced, and as a result, objects can be detected quickly and accurately. can do.

  In addition, according to the present invention, a plurality of resolution images having different resolutions are generated by multiplying the detection target image at a predetermined magnification to generate a plurality of resolution images, and a plurality of sizes for interpolating a predetermined magnification between the resolution images for a size of a window for generating a partial image It is assumed to have a size of For this reason, unlike the technique described in Patent Document 1, it is not necessary to make a fine difference in resolution when multi-resolution, and as a result, the amount of calculation when multi-resolution detection target image can be reduced, Therefore, object detection can be performed at high speed.

  In addition, since the number of resolution images is reduced as compared with the method described in Patent Document 1, it is possible to reduce the amount of calculation when performing illumination correction on each resolution image.

  In addition, the size of the window for generating the partial image has a plurality of sizes for interpolating a predetermined magnification between the resolution images, while making the difference in resolution when the detection target image is multi-resolution relatively large. Therefore, objects of various sizes included in the detection target image can be detected with high accuracy.

  Further, unlike the method described in Non-Patent Document 1, it is not necessary to perform calculation for changing the scale of the discriminator, so that the object can be detected at high speed and with high accuracy.

In the object detection apparatus according to the present invention, each of the plurality of windows has at least one sub-window having a plurality of resolutions with different resolutions and having a plurality of resolutions.
The discriminator further learns the feature quantity related to the distribution of the object for each size of the sub-window by using the plurality of sample images having different sizes corresponding to the size of the sub-window and subjected to the same illumination correction. It is also good.

  As a result, the resolution of the plurality of resolution images can be associated with the resolutions of the plurality of windows and the subwindows of the plurality of windows.

In this case, in the object detection apparatus according to the present invention, a window having a predetermined size among the plurality of windows is set as a target pixel of the resolution image to be detected, and a resolution image having a lower resolution than the resolution image to be detected Set the sub-window of the window of the predetermined size in the order of resolution to the pixel corresponding to the target pixel of
Performing a first determination as to whether or not the partial image generated by the relatively low resolution sub-window is the image of the object;
Only when the first determination is affirmative, the generation of the partial image by the plurality of windows and / or a sub-window having a resolution higher than the relatively low resolution of the plurality of windows and the partial image is the object. The partial image generation means and a control means for controlling the determination means may be provided so as to determine whether or not the image is an image.

  A “target pixel” is a pixel that is a target for detection of a specific type of object.

  “Set subwindows of a given size window in order of resolution” means that the highest resolution, ie, the largest resolution image, has the highest resolution, ie, the largest size window, and the next largest resolution image. For example, a sub-window having a size smaller by one step than the size of the largest sub-window is set, and sub-windows having smaller sizes are sequentially set according to the size of the resolution image as pixels corresponding to the target pixel of each resolution image. Means to set.

  As a result, when the first determination is negative, the predetermined window can be moved to the next pixel of interest, and the process of the next stage can proceed. Here, the generation of the partial image and the first determination using the low resolution resolution image and the low resolution sub-window have a small amount of calculation because the number of pixels of the partial image is small. Therefore, the position of the target pixel can be changed efficiently, and as a result, it can be determined at high speed whether or not the partial image is an object image.

Further, in this case, in the object detection device according to the present invention, the control unit determines that the determination on the partial image generated by the sub-window having the second highest resolution among the sub-windows of the predetermined size window is affirmed. A plurality of sub-images with the highest resolution sub-window for all of the plurality of windows;
Performing a second determination as to whether the plurality of partial images are images of the object;
By the second determination, the partial image is generated only by a window corresponding to the sub-window that generated the partial image having the highest probability of being the image of the object,
The partial image generation unit and the determination unit may be controlled to perform a third determination as to whether or not the partial image is an image of the object.

  “Accuracy of being an image of a specific type of object” means whether or not the determination unit using the classifier is the specific type of object, whether or not the output of the classifier is equal to or greater than a predetermined threshold Therefore, the output of the discriminator for each window size according to the second determination can be used as the “accuracy of being an image of a specific type of object”.

  Thereby, an object can be detected efficiently and accurately at high speed.

  The object detection apparatus according to the present invention may further include illumination correction means for performing the illumination correction on the plurality of resolution images.

  Thereby, an object can be detected with high accuracy.

The object detection method according to the present invention obtains a plurality of resolution images having different resolutions by multi-resolution a detection target image to be a target for detecting a specific type of object at a predetermined magnification,
A partial image is generated by scanning a plurality of windows having a size for interpolating the predetermined magnification between the resolution images on the plurality of resolution images,
The partial image having a different size corresponding to the size of the plurality of windows and learning a feature amount related to the distribution of the object for each size of the window by using a plurality of sample images subjected to the same illumination correction. A feature relating to a distribution of pixel values of the partial image based on the partial image by a determination unit including a discriminator that determines whether the partial image is an image of the object using the feature amount related to the partial image. An amount is calculated, and it is determined whether or not the partial image is an image of the object using the feature amount.

  In addition, you may provide as a program for making a computer perform the object detection method by this invention.

  According to the present invention, the amount of calculation for object detection can be reduced, and an object can be detected at high speed and with high accuracy.

Block diagram showing the configuration of the face detection system The figure which shows the process of multiresolution of a detection target image Diagram showing the concept of local normalization processing Flowchart showing processing in the illumination correction unit Schematic block diagram showing the configuration of the face detection unit Diagram for explaining generation of multiple size windows Block diagram showing configuration of classifier group Flow chart showing global processing in discriminator Flow chart showing processing in weak classifier The figure for demonstrating calculation of the feature-value in a weak discriminator A flowchart showing processing performed in the face detection system Diagram for explaining window settings (1) Diagram for explaining window settings (2) Detailed detection process flowchart Illustration for explaining normalization of face size and position Flow chart showing the learning method of the classifier The figure which shows the method of deriving the histogram of the weak classifier A flowchart showing processing when a face is detected from a moving image in the present embodiment. Diagram for explaining moving object detection Diagram showing the conventional multi-resolution process

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a face detection system to which the object detection device of the present invention is applied. This face detection system detects a face included in a digital image. As shown in FIG. 1, the face detection system 1 has a resolution image group S1 (= resolution image) composed of a plurality of images with different resolutions (hereinafter referred to as resolution images) by converting a detection target image S0 that is a target for detecting a face into multiple resolutions. (S1_1, S1_2,..., S1_n), and illumination processing is performed by performing illumination correction processing on each resolution image as preprocessing for the purpose of improving the accuracy of face detection processing to be executed later. The illumination correction unit 20 for obtaining the corrected resolution image group S1 ′ (= S1′_1, S1′_2,..., S1′_n) and the face for each of the overall normalized resolution image group S1 ′ A face detection unit 30 that detects an image (hereinafter referred to as a face image) S2 representing a face included in each resolution image of the resolution image group S1 ′ by performing detection processing.

  The multi-resolution conversion unit 10 converts the resolution (image size) of the detection target image S0 to normalize the resolution into an image having a predetermined resolution, for example, a rectangular size of VGA size (640 × 480 pixels). A converted detection target image S0 ′ is obtained. Then, by further performing resolution conversion based on the standardized detection target image S0 ′, a plurality of resolution images having different resolutions are generated, and a resolution image group S1 is obtained. The reason for generating such a resolution image group is that the size of the face included in the detection target image is usually unknown, whereas the size of the face to be detected (image size) is determined by a discriminator described later. In order to detect a face with a different size, a partial image of various sizes described later is cut out while shifting the position on an image with a different resolution. This is because it is necessary to determine whether the partial image is a face or a non-face.

  Specifically, as shown in FIG. 2, a standardized detection target image S0 ′ is set as a basic resolution image S1_1, and a resolution image S1_2 having a size that is −1 times the size of the resolution image S1_1, A resolution image S1_3 having a size of 2 −1 times the size of the image S1_2 (a size of 2−2 times a size of the resolution image S1_1), and a size of 2 −1 times the size (resolution image of the resolution image S1_3 A resolution image S1_4 having a size of 2 −3 times the size of S1_1 is generated.

  Thus, when the resolution image S1_1 has a rectangular size of 640 × 480 pixels, the short sides of the resolution images S1_2, S1_3,... Are 320 × 240 pixels, 160 × 120 pixels, 80 × 60 pixels,. It is possible to generate a plurality of resolution images that have a rectangular size of 2 and are reduced by 2 to the power of −1. In the present embodiment, the resolution moving image S1_1 having the highest resolution is referred to as a first-layer resolution image S1_1. Hereinafter, as the resolution decreases, the second-layer resolution image S1_2 and the third-layer resolution image S1_1 are displayed. It shall be called resolution image S1_3.

  The illumination correction unit 20 performs illumination correction on each resolution image group S1 by performing local normalization processing for suppressing variations in contrast in local regions on the image. In other words, the illumination correction unit 20 converts the pixel value so as to align the degree of dispersion of the pixel value in the local region to a certain level while scanning the local region of a predetermined size in each resolution image. Specifically, for each local region in each resolution image, for a local region where the degree of dispersion of pixel values representing luminance is a predetermined level or higher, the degree of dispersion is set to a constant level higher than the predetermined level. A first luminance gradation conversion process is performed so as to reduce the variance of the pixel value to a level lower than the predetermined level for a local region where the variance of the pixel value is less than the predetermined level. A luminance gradation conversion process is performed.

FIG. 3 is a diagram showing the concept of local normalization processing, and FIG. 4 is a flowchart showing processing in the illumination correction unit 20. Expressions (1) and (2) are gradation conversion expressions for pixel values for the local normalization process.

  Here, X is the pixel value of the pixel of interest, X ′ is the pixel value after conversion of the pixel of interest, mlocal is the average of the pixel values in the local region centered on the pixel of interest, Vlocal is the variance of the pixel values in this local region, SDlocal is a standard deviation of pixel values in this local area, (C1 × C1) is a reference value corresponding to the above-mentioned constant level, C2 is a threshold value corresponding to the above-mentioned predetermined level, and SDc is a predetermined constant. In the present embodiment, the number of gradations of luminance is 8 bits, and the possible values of pixel values are 0 to 255.

  As shown in FIG. 4, the illumination correction unit 20 sets one pixel in each resolution image S1-i (i = 1 to n) as a target pixel (step ST1), and a predetermined center around this target pixel. A variance Vlocal of pixel values in a local area having a size, for example, a 9 × 9 pixel size is calculated (step ST2), and it is determined whether or not the variance Vlocal is equal to or greater than a threshold C2 corresponding to the predetermined level (step ST3). ). If it is determined in step ST3 that the variance Vlocal is greater than or equal to the threshold value C2, the variance Vlocal is larger than the reference value (C1 × C1) corresponding to the certain level as the first luminance gradation conversion process. The tone conversion that decreases the difference between the pixel value X of the target pixel and the average mlocal, and increases the difference between the pixel value X of the target pixel and the average mlocal as the variance mlocal is smaller than the reference value (C1 × C1). Is performed according to the equation (1) (step ST4).

  On the other hand, if it is determined in step ST3 that the variance Vlocal is less than the threshold value C2, linear tone conversion that does not depend on the variance Vlocal is performed according to equation (2) as the second luminance tone conversion processing. Perform (step ST5). Then, it is determined whether or not the target pixel set in step ST1 is the last pixel (step ST6). If it is determined in step ST6 that the target pixel is not the last pixel, the process returns to step ST1, and the next pixel on the same partial image is set as the target pixel. On the other hand, if it is determined in step ST6 that the target pixel is the last pixel, the local normalization for the partial image is terminated. In this way, by repeating the processes of steps ST1 to ST6, a resolution image S1′_i obtained by subjecting the entire resolution image S1_i to local normalization processing is obtained.

  Note that the predetermined level may be changed according to the whole or a part of luminance in the local region. For example, in the normalization process in which gradation conversion is performed for each target pixel, the threshold value C2 may be changed according to the pixel value of the target pixel. That is, the threshold value C2 corresponding to the predetermined level may be set higher when the luminance of the target pixel is relatively high, and may be set lower when the luminance is relatively low.

  The face detection unit 30 performs face detection processing on each of the resolution image groups S1 ′ that has been subjected to illumination correction by the illumination correction unit 20, and detects a face image S2 in each resolution image. FIG. 5 is a schematic block diagram showing the configuration of the face detection unit 30. As shown in FIG. 5, the face detection unit 30 controls each unit to be described later to mainly perform sequence control in the face detection process, and the resolution used for the face detection process from the resolution image group S1 ′. A resolution image selection unit 32 that sequentially selects images S1′_i in descending order of size, and a partial image B that is a determination target of whether or not the image is a face image in the resolution image selected by the resolution image selection unit 32. A window setting unit 33 that sequentially sets the window to be cut out while shifting the position thereof, and a classifier group 34 that determines whether or not the cut out partial image B is a face image.

  The detection control unit 31 controls the resolution image selection unit 32 and the window setting unit 33 so as to perform face detection processing for detecting the face image S2 for each image in the resolution image group S1 ′. For example, as appropriate, the resolution image selection unit 32 is instructed to select a resolution image, the window setting unit 33 is instructed of window setting conditions, and the obtained detection result is output to the discriminator group 34. Or Note that the window setting condition includes a range on the image where the window is set, a window movement interval (roughness of detection), and the like.

  Under the control of the detection control unit 31, the resolution image selection unit 32 sequentially selects resolution images to be used for face detection processing from the resolution image group S1 ′ in descending order of size (in order of fine resolution). Note that the face detection method in the present embodiment determines whether or not the partial image B sequentially cut out on each resolution image is a face image, and determines that the partial image B is a face. Therefore, the resolution image selection unit 32 sets the size of the face to be detected in the detection target image S0 while changing the size of the face in the detection target image S0. Therefore, it can be said to be equivalent to setting the face to be detected while changing the size of the face from small to large.

  Based on the window setting conditions set by the detection control unit 31, the window setting unit 33 sequentially sets while moving windows of various sizes on the resolution image selected by the resolution image selection unit 32. In the present embodiment, windows of various sizes are generated as follows. First, in the present embodiment, a 32 × 32 pixel size window is prepared. Here, since the difference in resolution between the resolution image groups S1 is ½, faces having different sizes cannot be detected with high accuracy only by using a single size window.

  For this reason, in the present embodiment, a plurality of sizes of windows are generated based on the 32 × 32 pixel size window. FIG. 6 is a diagram for explaining the generation of windows of a plurality of sizes. As shown in FIG. 6, in the present embodiment, three windows W2_1, W3_1, and W4_1 having a size for interpolating a difference in resolution between resolution image groups S1 are generated on the basis of a window W1_1 having a 32 × 32 pixel size. . Here, since the difference in magnification between the resolution images of the resolution image group S1 is ½, the window W2_1 has a size of −1/4 times the size of the window W1_1, and the window W3_1 W2_1 has a size of -1/4 times the size of 2 (a size of -2/4 times a size of 2 for the window W1_1), and the window W4_1 has a size of -1/4 times the size of the window W3_1 Size (2 −3/4 times the size for the window W1_1). Since the window W1_1 has a 32 × 32 pixel size, the window W2_1 has a 27 × 27 pixel size, the window W3_1 has a 23 × 23 pixel size, and the window W4_1 has a 19 × 19 pixel size.

  Further, in the present embodiment, a process in which each of the windows W1_1 to W4_1 is reduced to a ½ size and a window in which the reduced resolution image is further reduced to a ½ size is generated. To generate multiple windows. In the present embodiment, windows W1_2 to W4_2 in which each of the windows W1_1 to W4_1 is reduced to a power of 2 times 1 and windows W1_3 to W4_3 in which each of the windows W1_1 to W4_1 is reduced to a power of 2 to a power of 2 are provided. Generate.

  Here, since the window W1_1 has a 32 × 32 pixel size, the windows W1_2 and W1_3 have a 16 × 16 pixel size and an 8 × 8 pixel size, respectively. Further, since the window W2_1 has a size of 27 × 27 pixels, the windows W2_2 and W2_3 have a size of 13 × 13 pixels and a size of 6 × 6 pixels, respectively. Since the window W3_1 has a 23 × 23 pixel size, the windows W3_2 and W3_3 have an 11 × 11 pixel size and a 5 × 5 pixel size, respectively. Since the window W4_1 has a 19 × 19 pixel size, the windows W4_2 and W4_3 have a 9 × 9 pixel size and a 4 × 4 pixel size, respectively.

  Here, in the following description, the windows W1_1, W2_1, W3_1, and W4_1 are referred to as first to fourth windows, and the windows W1_2, W1_3 that are generated from the first window W1_1 and the first window W1_1 are small in size. , W1_4 includes a plurality of windows including a first window group W1, a second window W2_1 and a second window W2_1, and a plurality of windows including small windows W2_2, W2_3, and W2_4 that are generated from the second window group. A plurality of windows including small windows W3_2, W3_3, and W3_4 generated from W2, the first window W3_1, and the third window W3_1 are defined as a third window group W3, a fourth window W4_1, and a fourth window W. Window size is generated from _1 small W4_2, W4_3, it is assumed that a plurality of windows including W4_4 called fourth window groups W4. In the first to fourth window groups W1 to W4, the windows in the first hierarchy, the second hierarchy, and the third hierarchy are called in order from the window having the highest resolution (that is, the size is larger). Shall. Note that window setting and face discrimination processing will be described later.

  The discriminator group 34 is a discriminator group that discriminates whether or not the partial image B is a face image, and is used for detecting the face image in the resolution image. FIG. 7 is a diagram showing the configuration of the classifier group 34. As shown in FIG. 7, the discriminator group 34 includes a plurality of types of discriminator groups having different detectable face sizes, that is, first sizes corresponding to the sizes of 12 windows for generating the partial image B. To twelfth discriminator groups 34_1 to 34_12. Note that the first to third discriminator groups 34_1 to 34_3 have the size of the windows W1_1 to W1_3, the fourth to sixth discriminator groups 34_4 to 34_6 have the sizes of the windows W2_1 to W2_3, and the seventh to ninth. The classifier groups 34_7 to 34_9 correspond to the sizes of the windows W3_1 to W3_3, and the tenth to twelfth classifier groups 34_10 to 34_12 correspond to the sizes of the windows W4_1 to W4_3, respectively. The first to third discriminator groups 34_1 to 34_3 are serially arranged in order from the third discriminator group 34_3, and the fourth to sixth discriminator groups 34_4 to 34_6 are serially arranged in order from the sixth discriminator group 34_6. In addition, the seventh to ninth discriminator groups 34_7 to 34_9 are serially arranged in order from the ninth discriminator group 34_9, and the tenth to twelfth discriminator groups 34_10 to 34_12 are serially arranged in order from the twelfth discriminator group 34_12. Is bound to. Furthermore, the first to third discriminator groups 34_1 to 34_3, the fourth to sixth discriminator groups 34_4 to 34_6, the seventh to ninth discriminator groups 34_7 to 34_9, the tenth to the tenth, respectively coupled in series. The twelfth discriminator groups 34_10 to 34_12 are coupled in parallel. Each of the first to twelfth discriminator groups 34_1 to 34_12 discriminates a face having a size corresponding to the size of the input partial image B.

  Thus, the first to third discriminator groups 34_1 to 34_3, the fourth to sixth discriminator groups 34_4 to 34_6, the seventh to ninth discriminator groups 34_7 to 34_9, and the tenth to twelfth discriminators. For the device groups 34_10 to 34_12, by connecting in series sequentially from the discriminator group that determines whether or not the partial image B cut out by the small size window Wk_3 (k = 1 to 4) is a face image, As will be described later, it is possible to sequentially determine from the partial image B cut out by the small size window Wk_3 (k = 1 to 4) toward the large size window Wk_1 (k = 1 to 4). Become.

  Further, as shown in FIG. 7, the classifier group 34 has a cascade structure in which a plurality of weak classifiers WC are linearly coupled, and the weak classifier has a distribution of pixel values (luminance) of the partial image B. At least one feature amount related to is calculated, and using this feature amount, it is determined whether or not the partial image B is a face image.

  Here, specific processing in the classifier group 34 will be described. FIG. 8 is a flowchart showing a general process in each classifier group 34_n (n = 1 to 12) included in the classifier group 34, and FIG. 9 is a flowchart showing a process by each weak classifier in the classifier group 34_n. .

  First, the first weak discriminator WC discriminates whether or not the partial image B is a face with respect to the partial image B cut out on the resolution image S1′_i by the corresponding size window (step SS1). ). Specifically, as shown in FIG. 10, the first weak discriminator WC applies a partial image B (for example, 32 × 32 pixel size) having a predetermined size cut out from the resolution image S1′_i. 4-neighbor pixel average (processing that divides an image into a plurality of blocks for each 2 × 2 pixel size and sets an average value of pixel values of four pixels of each block to a pixel value of one pixel corresponding to the block) As a result, an image of 16 × 16 pixel size and a reduced image of 8 × 8 pixel size are obtained, and a predetermined pair of two points set in the plane of these three images is used as a pair, and a plurality of types of pairs are used. A difference value of pixel values (luminance) between two points in each pair constituting one pair group is calculated, and a combination of these difference values is used as a feature amount (step SS1-1). The predetermined two points of each pair are, for example, two predetermined points arranged in the vertical direction and two predetermined points arranged in the horizontal direction so as to reflect the characteristics of the facial shading on the image. Then, a score is calculated by referring to a predetermined score table according to a combination of difference values as feature amounts (step SS1-2), and the score calculated by itself is added to the score calculated by the previous weak discriminator. The cumulative score is calculated (step SS1-3). In the first weak classifier WC1, since there is no previous weak classifier, the self-calculated score is used as the cumulative score as it is. It is determined whether or not the partial image is a face depending on whether or not the accumulated score is equal to or greater than a predetermined threshold (step SS1-4). When the partial image B is determined to be a face, the process proceeds to determination by the next weak classifier WC2 (step SS2). When the partial image B is determined to be a non-face, the partial image is immediately non- The face is determined (step SSB), and the process ends.

  Also in step SS2, as in step SS1, the second weak classifier WC calculates the above-described feature amount representing the feature on the image based on the partial image (step SS2-1), and refers to the score table. Then, a score is calculated from the feature amount (step SS2-2). Then, the score calculated by itself is added to the cumulative score calculated by the immediately preceding first weak discriminator WC to update the cumulative score (step SS2-3), and whether or not the cumulative score is equal to or greater than a predetermined threshold value. To determine whether the partial image B is a face (step SS2-4). Again, when the partial image B is determined to be a face, the process proceeds to determination by the next third weak classifier WC (step SS3), and when the partial image B is determined to be a non-face, the partial image B is Immediately, the face is determined to be non-face (step SSB), and the process ends. In this way, when the partial image B is determined to be a face in all N weak classifiers WC constituting the classifier, the partial image B is finally determined to be a face image (step SSA).

  In the present embodiment, the detection control unit 31, the resolution image selection unit 32, the window setting unit 33, and the discriminator group 34 function as the determination unit of the present invention.

  Next, the flow of processing in the face detection system 1 will be described. FIG. 11 is a flowchart showing the flow of processing in the face detection system according to this embodiment. As shown in FIG. 11, when the detection target image S0 is input to the multi-resolution converting unit 10 (step ST11), the multi-resolution converting unit 10 multi-resolutions the detection target image S0, and a resolution image composed of a plurality of resolution images. A group S1 is generated (step ST12). Then, the illumination correction unit 20 performs illumination correction on each of the resolution image groups S1, and obtains an illumination-corrected resolution image group S1 ′ (step ST13).

  In response to an instruction from the detection control unit 31, the face detection unit 30 uses the resolution image selection unit 32 in descending order of the image size from the resolution image group S1 ′, that is, S1′_1, S1′_2,. , S1′_n in the order of resolution images S1′_i (step ST14). Next, the detection control unit 31 instructs the window setting unit 33 to set the window at the initial position, that is, the first target pixel on the selected resolution image (step ST15). In the present embodiment, among the first to fourth window groups W1 to W4, a third window group W3 having an intermediate size is first set on the resolution image. For this reason, in the present embodiment, a face having a 23 × 23 pixel size is first detected from the resolution image S1′_1 of the first hierarchy. Instead of the third window group W3, the second window group W2 may be set on the selected resolution image.

  FIG. 12 is a diagram for explaining window settings. FIG. 12 shows window settings for the resolution image S1′_1 in the first hierarchy. As shown in FIG. 12, with respect to the resolution image S1′_1 of the first hierarchy, the window W3_1 of the first hierarchy in the third window group W3 is replaced with the resolution image S1′_2 of the second hierarchy. Is set to the second layer window W3_2, and the third layer window W3_3 is set to the third layer resolution image S1'_3. Of the three windows set on the selected resolution image, the window setting unit 33 sets the resolution image of the hierarchy in which the lowest hierarchy window (here, window W3_3) is set (here, the resolution image of the third hierarchy). The partial image B is cut out from S1′_3) (step ST16), and the partial image is input to the classifier group 34 (step ST17).

  If the selected resolution image is the second-level resolution image S1′_2, no window is set for the first-level resolution image S1′_1 as shown in FIG. The first layer window W3_1 of the third window group W3 for the second layer resolution image S1'_2 and the second layer for the third layer resolution image S1'_2. The third layer window W3_3 is set for the fourth layer resolution image S1'_3. Similarly, the partial image B is cut out from the resolution image (fourth hierarchy resolution image S1′_4) in which the lowest-layer window W3_3 is set, and the partial image B is input to the discriminator group 34. .

  Here, if the hierarchy of the selected resolution image is lowered, the number of resolution images in the hierarchy lower than the hierarchy of the selected resolution image may be 1 or 0. It may not be possible to set the resolution image. In this case, in the process of step ST16, it is cut out by the window of the lowest hierarchy among the set windows. For example, when the selected resolution image is the lowest resolution image, only the first hierarchy window can be set as the selected resolution image. Therefore, in the process of step ST16, the first hierarchy window is the lowest hierarchy. Window.

  The discriminator group 34 discriminates the input partial image B using the discriminator group corresponding to the window size among the 12 types of discriminators described above, and the detection control unit 31 determines the discrimination result R. Is obtained (step ST18), and it is determined whether or not the determination result R is that the partial image B is a face (step ST19). When the determination result R is that the partial image B is not a face (No in step ST19), the detection control unit 31 determines that the currently extracted partial image B is the partial image located at the last pixel of interest, that is, the last portion. It is determined whether or not the image is an image (step ST20). If it is determined that the partial image B is not the last partial image, the position where the window is set is set to the position of the next pixel of interest (that is, the next position). After setting (step ST21), the process returns to step ST16, and the window setting unit 33 cuts out a new partial image B. Thus, as shown in FIG. 12, the face image is detected from the resolution image S1′_i while scanning the resolution image S1′_i by the window.

  Here, the setting of the window position in step ST21 is performed by moving the third layer window W3_3 by one pixel on the resolution image of the layer in which the third layer window W3_3 is set. As a result, the window W3_2 in the second hierarchy is set by moving two pixels on the resolution image of the hierarchy in which the window W3_2 in the second hierarchy is set. The window W3_1 in the first hierarchy is set in the first hierarchy. The window W3_1 is set by moving four pixels on the resolution image of the set hierarchy.

  When it is determined that the partial image B is the last partial image, the detection control unit 31 determines the image for which the currently selected resolution image S1′_i is determined last, that is, the last resolution image S1. It is determined whether it is' _n (step ST22). If it is determined that the image is the last resolution image, the detection process is terminated and the detection result is output (step ST23). On the other hand, if it is determined that the resolution image is not the last resolution image, the process returns to step ST14, and the resolution image selection unit 32 generates a resolution image S1′_i + 1 that is one step smaller than the currently selected resolution image S1′_i. Then, the face image is detected.

  The detection result is output when a face cannot be detected from the detection target image S1, and when a face is detected in the detection target image S1, the face on the detection target image S0 is output. The coordinates of the position of the detected partial image are output.

  On the other hand, if the determination result R is that the partial image B is a face, the detection control unit 31 performs more detailed detection processing (step ST24). FIG. 14 is a flowchart of detailed detection processing. In the detailed detection process, the partial image cut out from the resolution image (here, the third resolution image S1′_3) in which the lowest hierarchy window (here, the third hierarchy window W3_3) is set. Since it is determined that B is a face, it is determined whether or not the number of layers above the lowest layer window is greater than 2 in order to perform more detailed determination (step ST31). If step ST31 is affirmed, the window setting unit 33 cuts out the partial image B from the resolution image of the hierarchy in which the hierarchy one level higher than the window of the lowest hierarchy, that is, the window having a large size of one stage is set (step ST32). The partial image B is input to the classifier group 34 (step ST33).

  The discriminator group 34 discriminates the input partial image B using the discriminator group corresponding to the window size among the 12 types of discriminators described above, and the detection control unit 31 determines the discrimination result R. Is acquired (step ST34), and it is determined whether or not the determination result R is that the partial image B is a face (step ST35). If the determination result R is that the partial image B is not a face, the process proceeds to step ST20. If the determination result R is that the partial image B is a face, the process returns to step ST31, and the processes from step ST31 to step ST35 are repeated.

  On the other hand, if step ST31 is negative, the detection control unit 31 determines whether or not the number of layers above the lowest layer window is 0 (step ST36). If step ST36 is affirmed, the process proceeds to step ST20. If step ST36 is negative, the detection control unit 31 determines whether or not the number of layers higher than the lowest layer window is 1 (step ST37). If step ST37 is negative, since the number of layers higher than the window of the lowest hierarchy is 2, the window setting unit 33 sets the windows W1_2 and W2_2 of the second hierarchy of all the window groups W1 to W4. W3_2 and W4_2 are set to the resolution image of the hierarchy one level higher than the hierarchy in which the window of the third hierarchy is set (here, the resolution image S1′_2 of the second hierarchy) (the second of all the window groups). Setting of the window of the hierarchy, step ST38). Note that the positions at which the second-level windows W1_2, W2_2, W3_2, and W4_2 are set are the positions of the current pixel of interest in the resolution image S1′_i + 1 in the next layer after the currently selected resolution image S1′_i. Is a position corresponding to.

  Next, the window setting unit 33 sequentially cuts out the partial images B1_2, P2_2, P3_2, and P4_2 from each of the set four windows (step ST39), and sequentially sets the partial images B1_2, P2_2, P3_2, and P4_2 to the discriminator group 34. Input (step ST40). In addition, the process of step ST40 may be performed in parallel and may be performed continuously. The discriminator group 34 discriminates the input partial images B1_2, P2_2, P3_2, and P4_2 using the discriminator group corresponding to the window size among the 12 types of discriminators described above, and the detection control unit 31 acquires cumulative scores K1_2, K2_2, K3_2, and K4_2 of each classifier group (step ST41), and determines the maximum cumulative score among the cumulative scores K1_2, K2_2, K3_2, and K4_2 (step ST42).

  Next, the window setting unit 33 sets the first layer window Wi_1 corresponding to the window for which the maximum cumulative score is obtained as the currently selected resolution image S1′_i (step ST43). For example, when the accumulated score acquired by the window W2_2 is the largest, the window W2_1 of the first hierarchy is set as the resolution image S1′_i. The window setting unit 33 cuts out the partial image B from the resolution image S1′_i using the set window (step ST44), and inputs the partial image B to the classifier group 34 (step ST45).

  The discriminator group 34 discriminates the input partial image B using the discriminator group corresponding to the window size among the 12 types of discriminators described above, and the detection control unit 31 determines the discrimination result R. Is acquired (step ST46), and the process proceeds to step ST20. On the other hand, when step ST37 is affirmed, the window setting unit 33 cuts out the partial image B from the resolution image of the hierarchy in which the window of the hierarchy one level higher than the window of the lowest hierarchy (here, the first hierarchy) is set. (Step ST47), the partial image B is input to the classifier group 34 (step ST48), and the process proceeds to step ST18. By performing the above processing, face images of various sizes can be detected from the detection target image S0.

  Next, a learning method (generation method) of the discriminator will be described. Note that learning is performed for each type of discriminator, that is, for each face size to be discriminated.

  A plurality of sample images to be learned are a plurality of faces that are standardized by all sizes of the window groups W1 to W4, that is, 12 sizes of windows included in the window groups W1 to W4. Sample images (face sample image group) and a plurality of sample images (non-face sample image group) known not to be faces.

  A sample image known to be a face is scaled in steps of 0.1 times in the range of 0.7 to 1.2 times in length and / or width for each sample image. For each sample image to be obtained, a plurality of deformation variations obtained by stepwise rotation in units of 3 degrees within a range of ± 15 degrees on the plane are used. At this time, in the face sample image, the size and position of the face are standardized so that the eye position is at a predetermined position, and the above-described rotation and enlargement / reduction on the plane are performed based on the eye position. . For example, in the case of a sample image of a front face of d × d size, as shown in FIG. 15, the positions of both eyes are 1/4 d inward from the upper left vertex and the upper right vertex of the sample image, respectively. The face size and position are standardized so as to come to each position shifted by ¼d downward, and the above-mentioned rotation and enlargement / reduction on the plane are performed around the middle point of both eyes.

  Twelve types of face sample image groups having different sizes according to the window size are prepared. Each of these 12 types of face sample image groups and the non-face sample image group is used to learn a classifier for each type, thereby generating 12 types of classifiers. The specific learning method will be described below.

  FIG. 16 is a flowchart showing a learning method of the classifier. It is assumed that each sample image constituting the face sample image group and the non-face sample image group has been subjected to illumination correction in advance by the same local normalization process as the illumination correction process by the illumination correction unit 20.

  Each of these sample images is assigned a weight or importance. First, the initial value of the weight of all sample images is set equal to 1 (step ST51). Next, when a plurality of pairs of groups consisting of a plurality of pairs are set with a predetermined two points set in the plane of the sample image and the reduced image as one pair, each of the plurality of types of pairs is weak. A separate device is created (step ST52). Here, each weak discriminator sets one pair group consisting of a plurality of pairs, with a predetermined two points set in the plane of the partial image cut out by the window W and the reduced image as one pair. This provides a reference for discriminating between a face image and a non-face image using a combination of difference values of pixel values (luminance) between two points in each pair constituting this one pair group. . In the present embodiment, a histogram for a combination of pixel value difference values between two points in each pair constituting one pair group is used as the basis of the score table of the weak classifier.

  The creation of a classifier will be described with reference to FIG. As shown in the sample image on the left side of FIG. 17, two points of each pair constituting the pair group for creating this discriminator are a plurality of sample images that are known to be faces. The right eye on the reduced image of 16 × 16 pixel size in which the point in the center of the right eye is P1, the point in the right cheek part is P2, the point in the part between the eyebrows is P3, and the sample image is reduced by an average of four neighboring pixels The point at the center of P4, the point at the cheek on the right side is P5, and the point at the forehead part on the reduced image of 8 × 8 pixel size reduced by the average of 4 neighboring pixels is P6, the mouth part A certain point is P7, and there are five pairs of P1-P2, P1-P3, P4-P5, P4-P6, and P6-P7. Note that the coordinate positions of the two points of each pair constituting one pair group for creating a certain classifier are the same in all sample images. For all sample images that are known to be faces, combinations of pixel value difference values between two points of each of the five pairs are obtained, and a histogram thereof is created. Here, the value that can be taken as a combination of the difference values of the pixel values depends on the number of luminance gradations of the image, but if it is a 16-bit gradation, there are 65536 kinds of difference values of one pixel value, As a whole, the number of gradations is (the number of pairs), that is, 65536 to the fifth power, and a large number of samples, time, and memory are required for learning and detection. For this reason, in the present embodiment, the difference value of the pixel value is divided by an appropriate numerical value width and quantized to be n-valued (for example, n = 100).

  As a result, the number of combinations of difference values of pixel values is n to the fifth power, so that the number of data representing combinations of difference values of pixel values can be reduced.

  Similarly, histograms are created for a plurality of sample images that are known not to be faces. For sample images that are known not to be faces, positions corresponding to the positions of the two predetermined points of each pair on the sample image that is known to be a face (similarly, reference numerals P1 to P7 are used). ) Is used. A histogram obtained by taking logarithm values of the ratios of the frequency values indicated by these two histograms and representing the histogram is used as the basis of the weak discriminator score table shown on the rightmost side of FIG. The value of each vertical axis indicated by the histogram of the weak classifier is hereinafter referred to as a discrimination point. According to this weak discriminator, an image showing the distribution of combinations of pixel value difference values corresponding to positive discrimination points is highly likely to be a face, and the possibility increases as the absolute value of the discrimination point increases. It can be said. Conversely, an image showing a distribution of combinations of difference values of pixel values corresponding to negative discrimination points is highly likely not to be a face, and the possibility increases as the absolute value of the discrimination point increases. In step ST52, a plurality of weak discriminators in the above-described histogram format are created for combinations of pixel value difference values between two predetermined points of each pair constituting a plurality of types of pair groups that can be used for discrimination.

  Subsequently, the most effective weak discriminator for discriminating whether or not the image is a face is selected from the plurality of weak semi-divided devices created in step ST52. The most effective weak classifier is selected in consideration of the weight of each sample image. In this example, the weighted correct answer rates of the weak discriminators are compared, and the weak discriminator showing the highest weighted correct answer rate is selected (step ST53). That is, in the first step ST53, since the weight of each sample image is equal to 1, the one with the largest number of sample images in which it is simply determined correctly whether or not the image is a face by the weak classifier, Selected as the most effective weak classifier. On the other hand, in the second step ST53 after the weight of each sample image is updated in step ST55, which will be described later, a sample image with a weight of 1, a sample image with a weight greater than 1, and a sample image with a weight less than 1 The sample images having a weight greater than 1 are counted more in the evaluation of the correct answer rate because the weight is larger than the sample images having a weight of 1. Thereby, in step ST53 after the second time, more emphasis is placed on correctly identifying a sample image having a large weight than a sample image having a small weight.

  Next, the correct answer rate of the combination of the weak classifiers selected so far, that is, using the weak classifiers selected so far (in the learning stage, the weak classifiers do not necessarily have to be linearly combined) ) It is ascertained whether the result of determining whether or not each sample image is a face image has exceeded a predetermined threshold value at a rate that matches the answer of whether or not it is actually a face image (step) ST54). Here, the current weighted sample image group or the sample image group with equal weight may be used for evaluating the correct answer rate of the combination of weak classifiers. When the predetermined threshold value is exceeded, learning is terminated because it is possible to determine whether the image is a face with a sufficiently high probability by using the weak classifier selected so far. If it is equal to or less than the predetermined threshold value, the process proceeds to step ST56 in order to select an additional weak classifier to be used in combination with the weak classifier selected so far.

  In step ST56, the weak classifier selected in the most recent step ST53 is excluded so as not to be selected again.

  Next, the weight of the sample image that could not correctly determine whether or not it is a face in the weak classifier selected in the most recent step ST53 is increased, and the sample image that can correctly determine whether or not the image is a face. Is reduced (step ST55). The reason for increasing or decreasing the weight in this way is that in the selection of the next weak classifier, importance is placed on images that could not be correctly determined by the already selected weak classifier, and whether or not those images are faces is correct. This is because a weak discriminator that can be discriminated is selected to enhance the effect of the combination of the weak discriminators.

  Subsequently, the process returns to step ST53, and the next effective weak classifier is selected based on the weighted correct answer rate as described above.

  As a weak discriminator suitable for discriminating whether or not it is a face by repeating the above steps ST53 to S56, the difference value of the pixel value between two predetermined points of each pair constituting a specific pair group If the weak discriminator corresponding to the combination is selected and the correct answer rate confirmed in step ST54 exceeds the threshold, the type of the weak discriminator used for discriminating whether or not it is a face and the discrimination condition are determined. (Step ST57), thereby completing the learning. The selected weak classifiers are linearly combined in descending order of the weighted correct answer rate to constitute one classifier. For each weak classifier, a score table for calculating a score according to a combination of pixel value difference values is generated based on the obtained histogram. Note that the histogram itself can also be used as a score table. In this case, the discrimination point of the histogram is directly used as a score.

  In this way, by learning for each face sample image group, the 12 types of discriminators described above are generated.

  In the case of adopting the above learning method, the weak classifier uses a combination of difference values of pixel values between two predetermined points of each pair constituting a specific pair group, and a face image and a non-face image. Is not limited to the above-described histogram format, and may be anything, for example, binary data, a threshold value, a function, or the like. Moreover, even in the same histogram format, a histogram or the like indicating the distribution of difference values between the two histograms shown in the center of FIG. 17 may be used.

  Further, the learning method is not limited to the above method, and other machine learning methods such as a neural network can be used.

  As described above, according to the face detection system of the present embodiment, when the detection target image is converted to multi-resolution, it is necessary to make the difference in resolution between the multi-resolution images fine as in the technique described in Patent Document 1. Therefore, it is possible to reduce the amount of calculation when the detection target image is multi-resolution, and thus it is possible to detect the face at high speed.

  Further, while making the difference in resolution when the detection target image S0 is multi-resolution larger than the main described in Patent Document 1, a plurality of windows W for generating partial images are interpolated with magnifications between the resolution images. Since it has a size, faces of various sizes included in the detection target image S0 can be detected with high accuracy.

  In addition, since the number of resolution images is reduced compared to the method described in Patent Document 1, the amount of calculation when performing illumination correction on each resolution image is reduced compared to the method described in Patent Document 1. can do.

  In addition, according to the present embodiment, when learning the discriminator group 34, a plurality of sample images that have been subjected to the same illumination correction as the illumination correction unit 20 are used. It is not necessary to perform illumination correction for each different partial image. Thereby, the amount of calculation for face detection can be reduced, and as a result, a face can be detected at high speed.

  Further, unlike the method described in Non-Patent Document 1, it is not necessary to perform a calculation for changing the scale of the weak classifiers constituting the classifier group 34, so that face detection can be performed at high speed and with high accuracy. .

  Note that when the face detection system according to the present embodiment is applied to face detection from a moving image, the following processing is possible. FIG. 18 is a flowchart showing processing when a face is detected from a moving image in the present embodiment. In the following description, it is assumed that processing is performed using a frame to be processed (current frame) ft in a moving image and a frame (previous frame) ft-1 immediately before the processing target frame ft.

  First, the detection control unit 31 determines whether or not a face is detected in the previous frame ft-1 (step ST61). When the face is not detected in the previous frame ft-1, the process of step ST11 in FIG. 11 is performed in order to perform the same process as the flowchart shown in FIGS. 11 and 14 with the current frame ft as the detection target image. move on. When a face is detected in the previous frame ft-1, between corresponding pixels of the previous frame ft-1 and the current frame ft in a window unit having a size used for generating the partial image B determined to be a face. Is calculated between the resolution images of the layers corresponding to each other in the frames ft-1 and ft (step ST62), and the previous frame ft-1 and the current frame in units of windows. It is determined whether or not there is an object moved between ft, that is, a moving object (step ST63).

  FIG. 19 is a diagram for explaining calculation of a difference. As shown in FIG. 19, when the window used for face detection in the previous frame ft-1 is the window W2_1, the window W2_1 is scanned between the previous frame ft-1 and the current frame ft, and the corresponding position is scanned. It is determined whether or not a moving object exists by calculating the difference between the windows W2_1 (t-1) and W2_1 (t). Thus, by calculating the difference in window units, the calculation time can be shortened compared to the case of detecting the moving object by calculating the difference in pixel units.

  Note that when calculating the difference between the windows W2_1 (t-1) and W2_1 (t), the difference between all the pixels in the windows W2_1 (t-1) and W2_1 (t) may be calculated. Among weak discriminators constituting each discriminator group 34_j (j = 1 to 12) corresponding to the size, a difference is obtained by using only pixel positions used for calculating a feature amount input to a specific weak discriminator. May be calculated. For example, when the feature amount input to a weak discriminator of the discriminator group 34_4 corresponding to the window W2_1 is the pixel values of the three pixel positions P11 to P13 shown in FIG. 19, the window W2_1 (t−1), You may make it detect a moving body by calculating the difference between each pixel position P11-P13 in W2_1 (t). Thereby, the calculation for a moving body detection can be reduced significantly and a moving body can be detected at high speed.

  If step ST63 is negative, the detection control unit 31 proceeds to the process of step ST11 in FIG. 11 to perform the same process as the flowchart shown in FIGS. 11 and 14 using the current frame ft as the detection target image. If step ST63 is affirmed, the detection control unit 31 performs the same detection process as that in FIG. 11 and FIG. 14 only on the moving object region and its neighboring region in the current frame ft (step ST64). The result is output (step ST65), the detection target frame is changed to the next frame (step ST66), and the process returns to step ST61.

  As described above, in the case of a moving image, first, by performing the moving object detection process, it is possible to estimate the position of the moving object, and to detect the face only in the estimated moving object and the area in the vicinity thereof. The face can be detected efficiently. In particular, by performing a moving object detection process in units of windows, it is possible to reduce the amount of calculation and detect a moving object at high speed.

In the above embodiment, the detection target is a human face, but other objects such as a human hand may be detected. In this case, the discriminator may perform learning using the sample image group including the object and the sample image group not including the object. In the above embodiment, the illumination correction unit 20 performs the illumination correction process. The illumination correction is not essential to the present invention.

  Although the face detection system according to the embodiment of the present invention has been described above, a program for causing a computer to execute each process in a portion corresponding to the face detection device of the present invention in the face detection system is also included in the present invention. This is one of the embodiments. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

DESCRIPTION OF SYMBOLS 1 Face detection system 10 Multi-resolution part 20 Illumination correction part 30 Face detection part 31 Detection control part 32 Resolution image selection part 33 Window setting part 34 Classifier group

Claims (6)

Multi-resolution processing means for obtaining a plurality of resolution images having different resolutions by multi-resolution a detection target image for detecting a specific type of object at a predetermined magnification;
A plurality of windows having a size for interpolating the predetermined magnification between the resolution images on the plurality of resolution images , each of which is multi-resolutiond by the predetermined magnification and having at least one sub-window having different resolutions. Partial image generation means for generating a partial image by scanning a plurality of windows ;
A determination unit that calculates a feature amount related to a distribution of pixel values of the partial image based on the partial image and determines whether the partial image is an image of the object using the feature amount; ,
The determination unit is configured to learn, for each window size, a feature amount related to the object distribution by using a plurality of sample images having different sizes corresponding to the sizes of the plurality of windows and subjected to the same illumination correction. A discriminator for discriminating whether or not the partial image is an image of the object using the feature amount of the partial image, the discriminator having different sizes corresponding to the size of the sub-window, and the same An object detection apparatus comprising: a discriminator that further learns a feature amount related to the distribution of the object for each size of the sub-window using a plurality of sample images subjected to the illumination correction . A window having a predetermined size among the plurality of windows is set for the target pixel of the resolution image to be detected, and the predetermined pixel is set to a pixel corresponding to the target pixel of the resolution image having a resolution lower than that of the target resolution image. Set the sub-windows of the size window in order of resolution,
Performing a first determination as to whether or not the partial image generated by the relatively low resolution sub-window is the image of the object;
Only when the first determination is affirmative, the generation of the partial image by the plurality of windows and / or a sub-window having a resolution higher than the relatively low resolution of the plurality of windows and the partial image is the object. as for judging whether or not the image, the object detection apparatus according to claim 1, further comprising a control means for controlling the partial image generating means and said determining means. The control means, when the determination on the partial image generated by the sub-window having the second highest resolution among the sub-windows of the predetermined size window is affirmed, the highest resolution for all of the plurality of windows. Generate multiple partial images by subwindow,
Performing a second determination as to whether the plurality of partial images are images of the object;
By the second determination, the partial image is generated only by a window corresponding to the sub-window that generated the partial image having the highest probability of being the image of the object,
3. The object detection device according to claim 2 , wherein the partial image generation unit and the determination unit are controlled to perform a third determination as to whether or not the partial image is an image of the object. . Wherein for a plurality of resolution images, the object detection apparatus according to any one of claims 1 to 3, further comprising a lighting correction means for performing the illumination correction. The detection target image for detecting a specific type of object is multi-resolutiond at a predetermined magnification to obtain a plurality of resolution images having different resolutions,
A plurality of windows having a size for interpolating the predetermined magnification between the resolution images on the plurality of resolution images , each of which is multi-resolutiond by the predetermined magnification and having at least one sub-window having different resolutions. Generating a partial image by scanning a plurality of windows ,
The partial image having a different size corresponding to the size of the plurality of windows and learning a feature amount related to the distribution of the object for each size of the window by using a plurality of sample images subjected to the same illumination correction. A discriminator for discriminating whether or not the partial image is an image of the object using the feature amount according to the above, and having a different size corresponding to the size of the sub-window, and performing the same illumination correction. In addition, a plurality of sample images is used to determine the distribution of pixel values of the partial image based on the partial image by a determination unit including a discriminator that further learns the feature amount related to the distribution of the object for each size of the subwindow. Calculating the feature amount, and determining whether or not the partial image is an image of the object using the feature amount. Object detection method that. A procedure for obtaining a plurality of resolution images having different resolutions by converting a detection target image for detecting a specific type of object into a multi-resolution with a predetermined magnification,
A plurality of windows having a size for interpolating the predetermined magnification between the resolution images on the plurality of resolution images , each of which is multi-resolutiond by the predetermined magnification and having at least one sub-window having different resolutions. A procedure for generating a partial image by scanning a plurality of windows ,
The partial image having a different size corresponding to the size of the plurality of windows and learning a feature amount related to the distribution of the object for each size of the window by using a plurality of sample images subjected to the same illumination correction. A discriminator for discriminating whether or not the partial image is an image of the object using the feature amount according to the above, and having a different size corresponding to the size of the sub-window, and performing the same illumination correction. In addition, a plurality of sample images is used to determine the distribution of pixel values of the partial image based on the partial image by a determination unit including a discriminator that further learns the feature amount related to the distribution of the object for each size of the subwindow. And calculating a feature amount and determining whether the partial image is an image of the object using the feature amount. Program for executing the object detection method characterized in the computer in that.

Images