JP2009026326A - Group study device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detector and method, and a group study device and method with which arithmetic processing in study and in detection is accelerated, a target object with optional size is detected, and which has extremely high discrimination capability when the target object is detected by group study. <P>SOLUTION: A target object detector 1 has: a scaling part 3 which reduces a gray image input from an image output part 2 to generate a scaling image; a scanning part 4 which sequentially operates and cuts off window images from the scaling image; and a discrimination unit 5 which discriminates whether or not the respective window images are target objects. The discrimination unit 5 is comprised of a plurality of weak discrimination units of which the group study is performed by boosting and an adder which performs majority with weight to output of the weak discrimination units, and each weak discrimination unit uses difference in luminance values of two pixels as feature quantity to output an estimation value for estimating whether or not the respective window images are the target objects. In addition, the discrimination unit 5 discontinues calculation of the estimation value about window images determined as not being the target objects by a pre-studied discontinuation threshold. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object detection apparatus and method for detecting an object such as a face image in real time, and a group learning apparatus and method for group learning of the object detection apparatus.

  Conventionally, many face detection methods have been proposed that use only a light / dark pattern of an image signal without using a motion from a complicated image scene. For example, the face detector described in Patent Document 1 below uses an AdaBoost that uses a filter such as a Haar basis for a weak learner (weak learner), which will be described later. By using an image called an integral image and a rectangle feature, a weak hypothesis can be calculated at high speed.

  FIG. 15 is a schematic diagram showing the rectangular feature described in Patent Document 1. As shown in FIG. As shown in FIG. 15, in the technique described in Patent Document 1, in input images 142A to 142D, the sum of luminance values of adjacent rectangular regions of the same size is obtained, and the sum of luminance values of one rectangular region and the other A plurality of filters (weak hypotheses) that output the difference from the sum of the luminance values of the rectangular area are prepared. For example, in the input image 142A, a filter 154A that subtracts the sum of the luminance values of the rectangular region 154A-2 shown with a shadow from the sum of the luminance values of the rectangular region 154A-1. Such a filter composed of two rectangular regions is called a two rectangle feature. In addition, the input image 142C includes three rectangular areas 154C-1 to 154C-3 obtained by dividing one rectangular area into three, and a shadow is added from the sum of luminance values of the rectangular areas 154C-1 and 154C-3. A filter 154C that subtracts the sum of the luminance values of the central rectangular area 154C-2 shown is shown. Such a filter composed of three rectangular areas is called a three rectangle feature. Further, in the input image 142D, one rectangular area is composed of four rectangular areas 154D-1 to 154D-4 divided into upper, lower, left and right, and a shadow is added from the sum of luminance values of the rectangular areas 154D-1, 154D-3. A filter 154D that subtracts the sum of the luminance values of the rectangular regions 154D-2 and 154D-4 shown. Such a filter composed of four rectangular regions is called a four rectangle feature.

  For example, a case will be described in which the face image shown in FIG. 16 is determined to be a face using, for example, the rectangular feature 154B shown in FIG. The two rectangular feature 154B is composed of two rectangular areas 154B-1 and 154B-2 in which one rectangular area is divided into two in the vertical direction (vertical direction), and a shadow is added from the sum of the luminance values of the rectangular area 154B-2. The sum of the luminance values of the rectangular area 154B-1 shown is subtracted. When the human face (object) 138 uses the fact that the eye area has a lower luminance value than the cheek area, it is determined from the output value of the rectangular feature 154B whether the input image is a face (correct answer or incorrect answer). It can be estimated with a certain probability. This is used as one of weak classifiers in AdaBoost.

  Here, at the time of detection, in order to detect face areas of various sizes included in the input image, it is necessary to determine whether or not the face is by cutting out areas of various sizes (hereinafter referred to as search windows). is there. However, for example, an input image composed of 320 × 240 pixels includes face regions (search windows) of about 50,000 types of sizes, and it takes a very long time to perform calculations for all these window sizes. Therefore, in Patent Document 1, an image called an integral image is used. As shown in FIG. 17, the integral image is the sum of the luminance values of the pixels at the upper left of the pixel 162 in the input image 144 as shown in the following equation (1) in the input image 144. It is an image that is. That is, the value of the pixel 162 is the sum of the luminance values of the pixels included in the upper left rectangular area 160 of the pixel 162. Hereinafter, an image in which each pixel value is a value represented by the following formula (1) is referred to as an integral image.

  If this integral image is used, a rectangular region of an arbitrary size can be calculated at high speed. That is, as shown in FIG. 18, when the rectangular area 170 at the upper left and the rectangular areas 172, 174, and 176 at the right side, the lower, and the lower right of the rectangular area 170 are set, the four vertices of the rectangular area 176 are rotated clockwise from the upper left. P1, P2, P3, and P4. Here, P1 is the sum A of the luminance values of the rectangular area 170 (P1 = A), P2 is A + the total sum B of the luminance values of the rectangular area 172 (P2 = A + B), and P3 is the sum C of the luminance values of the A + rectangular area 174 (P3 = A + C), P4 is the sum D of luminance values of A + B + C + rectangular area 176 (P4 = A + B + C + D). At this time, the sum D of the luminance values of the rectangular area D can be calculated as P4- (P2 + P3) -P1, and the sum of the luminance values of the rectangular area can be increased at high speed by adding and subtracting the pixel values at the four corners of the rectangular area. Can be calculated. Usually, it is possible to search for a search window of a different size by scaling the input image and cutting out a window (search window) of the same size as the learning sample used for learning from each scale-converted image. . However, as described above, if the input image is scale-converted so that search windows of all sizes can be set, the amount of calculation becomes extremely large. Therefore, in the technique described in Patent Document 1, the amount of calculation is reduced by using an integral image that can calculate the sum of luminance values in a rectangular area at high speed and using a rectangular feature.

US Patent Application Publication No. 2002/0102024

  However, the face detector described in Patent Document 1 can only detect a target object that is an integer multiple of the size of the learning sample used during learning. This is because the above-mentioned Patent Document 1 does not change the size of the search window by scaling the input image, but converts the input image to an integral image and uses this to detect the face area of a different search window It is to do. That is, since the integral image is discretized in units of pixels, for example, when a window size of 20 × 20 is used, a size of 30 × 30 cannot be set as a search window. Face detection cannot be performed.

  Further, as the rectangular feature, only the difference in luminance value between adjacent rectangular areas is used for speeding up the calculation. For this reason, a change in luminance between distant rectangular areas cannot be captured, and the performance of object detection is limited.

  Note that, for example, if the integral image is scale-converted, it is possible to search for a window of an arbitrary size, and it is also possible to use the difference in luminance value between the rectangular regions at distant positions, but the integral image is scale-converted. As a result, the amount of calculation increases, and the effect of speeding up the processing using the integral image is canceled. Also, if the difference of the luminance value between the separated rectangular areas is included, the types of filters become enormous. Similarly, the amount of processing increases.

  The present invention has been proposed in view of such a conventional situation. When detecting a target object by collective learning, the arithmetic processing at the time of learning and detection is accelerated, and an arbitrary size is provided. It is an object of the present invention to provide an object detection apparatus and method, and a collective learning apparatus and method that can detect the target object and have extremely high discrimination ability.

  In order to achieve the above-described object, an object detection apparatus according to the present invention is a pixel detection apparatus that detects whether a given grayscale image is an object or not. A plurality of weak discriminating means for calculating an estimated value indicating whether or not the grayscale image is an object based on a feature amount consisting of a difference in luminance value, and the estimation calculated by at least one of the plurality of weak discriminating means. And determining means for determining whether the grayscale image is an object based on the value.

  In the present invention, the weak determination means uses a very simple feature amount that is a difference between the luminance values of the pixels at two positions, and the given grayscale image is an object to be detected or a non-object. Since the weak determination is performed, the detection process can be speeded up.

  The discriminating means calculates a weighted majority value obtained by multiplying the estimated value by the reliability of each weak discriminating means obtained by the learning and adding the value, and based on the majority value, the grayscale image is calculated. It is possible to determine whether or not the object is an object, and it is possible to determine whether or not the object is an object using a majority result obtained by combining the estimated values of a plurality of weak discriminating means.

  Further, the plurality of weak discriminating means sequentially calculate the estimated value, and the discriminating means sequentially updates the weighted majority value every time the estimated value is calculated, and the updated weighted majority vote. Whether or not to stop the calculation of the estimated value based on the value of the value, and by sequentially calculating the estimated value by the weak classifier and evaluating the value of the weighted majority vote, by all weak determining means By suspending the processing without waiting for the calculation, the detection speed can be further increased.

  Furthermore, the determining means aborts the calculation of the estimated value depending on whether the value of the weighted majority vote is smaller than an abort threshold, and each weak distinguishing means is an object or a non-object. Are generated sequentially by collective learning using a learning sample consisting of a plurality of grayscale images to which the correct answer is given, and the truncation threshold value is generated each time weak discriminating means is generated during the learning. And adding the value obtained by weighting the reliability to the estimated value for the learning sample that is the object calculated by the generated weak discriminating means to obtain the minimum value of the weighted majority value that is updated. In addition, by learning the minimum value that can be taken by the correct object grayscale image as the abort threshold, the process of the weak discriminating means can be aborted accurately and efficiently.

  In addition, when the minimum value of the weighted majority vote at the time of learning is positive, 0 can be set as the truncation threshold value, and it is determined whether the output of the weak discrimination means is positive or negative like AdaBoost. When learning by a collective learning algorithm, a minimum value of 0 or more can be set as an abort threshold.

  Further, the weak discriminating means estimates by calculating the binary estimated value indicating whether or not the object is an object according to whether or not the feature amount of the grayscale image is equal to or greater than a predetermined threshold. A value may be output deterministically, or an estimated value may be output probabilistically by calculating a probability that the grayscale image is an object based on the feature amount as the estimated value.

  The object detection method according to the present invention is the object detection method for detecting whether or not a given grayscale image is an object, and is a feature amount comprising a difference between luminance values of pixels at two previously learned positions. And a weak discrimination step for calculating an estimated value indicating whether or not the gray image is an object by a plurality of weak discriminating means, and the gray image based on the estimated value calculated by at least one of a plurality of weak discriminators. And a determination step of determining whether or not the object is an object.

  The group learning device according to the present invention is a group learning device that performs group learning using a learning sample composed of a plurality of grayscale images that are correctly identified as being an object or a non-object. Learning means for performing collective learning using a plurality of weak discriminators that output an estimated value indicating whether or not a grayscale image given as an input is a difference between luminance values of two pixels at an arbitrary position as a feature amount It is characterized by having.

  In the present invention, a weak discriminator using a very simple feature amount, which is a difference between luminance values of two pixels at arbitrary positions in a learning sample, is generated by collective learning, so that the discrimination result of the generated weak discriminator is obtained. When a detection apparatus that detects a target object by using a large number is configured, the detection process can be made extremely fast.

  The learning means calculates the feature quantity of each learning sample, and generates a weak classifier based on each feature quantity, and a weak classifier generated by the weak classifier generation means. An error rate calculating means for calculating an error rate for discriminating the learning sample based on the data weight set for each learning sample, and a reliability calculating means for calculating a reliability for the weak classifier based on the error rate; And a data weight calculating means for updating the data weight so that the weight of the learning sample that the weak discriminator makes an incorrect answer relatively increases, and the weak discriminator generating means updates the data weight. A new weak classifier can be generated, a weak classifier is generated, its error rate and reliability are calculated, data weight is updated, and a series of processes of generating a weak classifier again is repeated. Study It can be carried out.

  Further, the weak discriminator generation means repeats the process of calculating the feature quantity a plurality of times to calculate a plurality of types of feature quantities, generates weak discriminator candidates for each feature quantity, and generates each of the generated plurality of feature quantities. For the weak discriminator candidate, the error rate obtained by discriminating the learning sample is calculated based on the data weight set for each learning sample, and the one having the smallest error rate can be used as the weak discriminator. Each time is updated, a large number of weak discriminator candidates are generated, and one weak discriminator can be generated (learned) by selecting the one having the smallest error rate from among them.

  Further, each time the weak classifier generating means generates the weak classifier, the weak classifier calculates the estimated value for each learning sample that is the object, and the estimated value is weighted with the reliability. It is possible to have an abort threshold value storage means for calculating the added weighted majority value and storing the minimum value, and learning the minimum value as the abort threshold value, thereby generating a plurality of weak classifiers. It is possible to further speed up the detection process in the detection apparatus.

  The group learning method according to the present invention is a group learning method in which group learning is performed using a learning sample composed of a plurality of grayscale images that are correctly identified as being an object or a non-object. A learning step of using a plurality of weak discriminators to output an estimated value indicating whether or not a grayscale image given as an input is a difference between luminance values of two pixels at arbitrary positions. It is characterized by having.

  An object detection apparatus according to the present invention cuts out a fixed-size window image from a grayscale image, and expands or reduces the size of the input grayscale image in the object detection apparatus that detects whether the window image is an object. Scale conversion means for generating the scale image, window image scanning means for scanning the window of the fixed size from the scale image and cutting out the window image, and an object for detecting whether the given window image is an object Detecting means, and the object detecting means calculates an estimated value for estimating whether or not the window image is an object based on a feature amount comprising a difference between luminance values of pixels at two positions learned in advance. A window image based on the estimated value calculated by at least one of the plurality of weak discrimination means and at least one of the plurality of weak discrimination means. And having a discriminating means for discriminating whether the whether ones.

  In the present invention, a grayscale image is scaled and a window image is cut out to detect an object of an arbitrary size, and the weak discriminating means is a very simple difference between the luminance values of two pixels. Since an estimated value indicating whether or not the window image is an object is calculated based on the feature amount, the detection process can be performed at a very high speed.

  In the object detection method according to the present invention, a window image having a fixed size is cut out from a grayscale image, and the size of the input grayscale image is enlarged or reduced in the object detection method for detecting whether the window image is an object. A scale conversion step for generating a scale image, a window image scanning step for scanning the window of the fixed size from the scale image and cutting out the window image, and an object for detecting whether the given window image is an object A detection step, wherein the object detection step has a plurality of estimated values indicating whether or not the grayscale image is an object based on a feature amount that is a difference between luminance values of pixels at two positions learned in advance. Based on the weak discriminating step calculated by the weak discriminator and the estimated value calculated by at least one of a plurality of weak discriminators, the grayscale image is an object. Characterized in that it has a determination step of determining whether there.

  According to the object detection device of the present invention, in the object detection device that detects whether or not a given grayscale image is an object, the difference between the luminance values of pixels at two previously learned positions is included. A plurality of weak discriminating means for calculating an estimated value indicating whether the grayscale image is an object based on a feature amount, and the grayscale image is based on the estimated value calculated by at least one of the plurality of weak discriminating means. Since it has a discriminating means for discriminating whether or not it is an object, it is very easy to weakly determine whether or not a grayscale image is an object, the detection process can be made extremely fast, and face detection can be performed in real time. it can.

  In addition, according to the object detection method of the present invention, it is possible to detect at high speed whether or not a given grayscale image is an object.

  According to the group learning device according to the present invention, in the group learning device that performs group learning using a learning sample composed of a plurality of grayscale images that are correctly identified as being an object or a non-object, Learning that uses a sample and collectively learns a plurality of weak classifiers that output the estimated value indicating whether the grayscale image given as an input is the target value using the difference between the luminance values of two pixels at an arbitrary position As a result, it is possible to generate a weak discriminator using a very simple feature value, which is the difference between the luminance values of two pixels at an arbitrary position, by collective learning. And the detection process when the object detection device having a large number of generated weak discriminators is configured can be extremely accelerated.

  Further, according to the means learning method according to the present invention, by performing collective learning using a learning sample composed of a plurality of grayscale images to which a correct answer as to whether it is an object or a non-object, It is possible to learn a weak classifier that constitutes an object detection device capable of detecting an object.

  According to the object detection device of the present invention, a window image having a fixed size is cut out from the grayscale image, and the size of the input grayscale image is enlarged in the object detection device that detects whether or not the window image is the target object. Alternatively, a scale conversion unit that generates a reduced scale image, a window image scanning unit that scans the window of the fixed size from the scale image and cuts out the window image, and detects whether the given window image is an object. An object detection means, and the object detection means estimates whether the window image is an object based on a feature amount consisting of a difference between luminance values of pixels at two positions learned in advance. A window image based on the estimated value calculated by at least one of a plurality of weak discriminating means and a plurality of weak discriminating means. Since it has a discriminating means for discriminating whether or not the object is an object, it is possible to detect an object of any size by scaling the grayscale image of the input image and cutting out the window image, Since the weak discriminating means detects whether the window image is an object by using an extremely simple feature amount that is the difference between the luminance values of the two pixels, the detection process can be performed at a very high speed.

  In addition, according to the object detection method of the present invention, it is possible to cut out a fixed-size window image from a grayscale image and to detect at high speed whether the window image is an object.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is applied to an object detection apparatus that detects an object from an image using ensemble learning (group learning).

  A learning machine obtained by group learning is composed of a number of weak hypotheses and a combiner that combines them. Here, boosting is an example of a combiner that integrates weak hypothesis outputs with fixed weights regardless of input. Boosting uses the previously generated weak hypothesis learning result to process the distribution that the learning sample follows to increase the weight of the learning sample (example) that is not prone to mistakes, and then creates a new weakness based on this distribution. Learn hypotheses. As a result, the weights of learning samples that have many incorrect answers and are difficult to discriminate as objects are relatively increased, and as a result, weak classifiers that correctly answer learning samples that are large in weight, that is, difficult to discriminate are sequentially selected. That is, the generation of weak hypotheses in learning is performed sequentially, and the weak hypotheses generated later depend on the weak hypotheses generated before that.

  When detecting an object, the discrimination results of a number of weak hypotheses sequentially generated by learning as described above are used. For example, in the case of AdaBoost, all the weak hypotheses (hereinafter referred to as weak discriminators) generated by this learning are supplied to the coupling machine (1 for an object and -1 for a non-object). The combiner weights and adds the reliability calculated at the time of learning for each weak discriminator to all the discrimination results, outputs the result of the weighted majority vote, and evaluates the output value of the combiner This selects whether or not the image input in is an object.

  The weak classifier determines whether the object is an object or a non-object by using some feature amount. As will be described later, whether or not the output of the weak discriminator may be deterministically output, or the likelihood of the object may be output probabilistically by a probability density or the like. Here, in this embodiment, a weak discriminator that discriminates whether or not the object is an object using a very simple feature quantity called a difference in luminance value between two pixels (hereinafter referred to as an inter-pixel difference feature). By using a group learning device that uses, the object detection process is speeded up.

(1) Object Detection Device FIG. 1 is a functional block diagram showing processing functions of the object detection device in the present embodiment. As shown in FIG. 1, the object detection apparatus 1 is scaled by an image output unit 2 that outputs a grayscale image (luminance image) as an input image, a scaling unit 3 that scales the input image by enlarging or reducing it. In the input image, for example, a scanning unit 4 that sequentially scans from the upper left with a window image of a predetermined size, and a discriminator that determines whether each window image sequentially scanned by the scanning unit 4 is an object or a non-object 5 and outputs the position and size of the target object indicating the area of the target object from the given image (input image). That is, the scaling unit 3 enlarges or reduces the input image to all designated scales and outputs a scaled image. For each scaled image, the scanning unit 3 sequentially scans the window having the size of the object to be detected to cut out the window image, and the discriminator 5 determines whether each window image is a face.

  Here, the discriminator 5 refers to the learning result of the collective learning machine 6 that performs collective learning of a plurality of weak discriminators constituting the discriminator 5 by collective learning, and the current window image is, for example, a face image or the like. It is determined whether the object is an object or a non-object.

  Moreover, the target object detection apparatus 1 outputs a plurality of region information when a plurality of target objects are detected from the input image. Furthermore, when there is an overlapping area among a plurality of pieces of area information, it is possible to perform a process of selecting an area having the highest evaluation as a target by a method described later.

  The image (grayscale image) output from the image output unit 2 first enters the scaling unit 3. The scaling unit 3 performs image reduction using bilinear interpolation. In the present embodiment, instead of first generating a plurality of reduced images, a required image is output to the scanning unit 4, and after the processing of the image is finished, the next smaller reduction is performed. The process of generating an image is repeated.

  That is, first, as shown in FIG. 2, the input image 10A is output to the scanning unit 4 as it is. Then, after waiting for the processing of the scanning unit 4 and the discriminator 5 to end for the input image 10A, an input image 10B in which the size of the input image 10A is reduced is generated. Then, after the processing of the scanning unit 4 and the discriminator 5 in the input image 10B is completed, the input image 10C obtained by further reducing the size of the input image 10B is output to the scanning unit 4 so as to be sequentially reduced. 10D, 10E, etc. are generated, and the process is terminated when the image size of the reduced image becomes smaller than the window size scanned by the scanning unit 4. After the end of this process, the image input unit 2 outputs the next input image to the scaling unit 3.

  As shown in FIG. 3, the scanning unit 4 sequentially applies a window 11 having a window size S received by the subsequent discriminator 5 to the entire image (screen) for the given image 10 </ b> A, for example. The image at each position (hereinafter referred to as a cut image) is output to the discriminator 5. Here, while the window size S is constant, as described above, the input image is sequentially reduced by the scaling unit 3, and the image size of the input image is converted into various scales. Can be detected.

The discriminator 5 determines whether the cut image given from the previous stage is a target object such as a face. As shown in FIG. 4, the discriminator 5 includes a plurality of weak discriminators 21 _n (21 _{1 to} 21 _N ) obtained by ensemble learning, and weights W _n (W ₁ to W _N ), and an adder 22 for obtaining a weighted majority. The discriminator 6 sequentially outputs an estimated value as to whether or not each weak discriminator 21 _n (21 _{1 to} 21 _N ) is an object for the input window image, and the adder 22 calculates a weighted majority vote. And output. In accordance with the weighted majority value, it is determined whether or not a determination means (not shown) is an object.

The group learning machine 6 learns the weak classifier 21 _n and weights for multiplying their outputs (estimated values) in advance by group learning using a method described later. As group learning, any method can be applied as long as it can obtain the results of a plurality of discriminators by majority vote. For example, it is possible to apply group learning using boosting such as AdaBoost, which performs weighted majority by weighting data.

Each weak discriminator 21 _n constituting the discriminator 5 uses a difference in luminance value between two pixels (inter-pixel difference feature) as a feature amount for discrimination. Then, for the discrimination, a feature amount learned in advance by a learning sample made up of a plurality of grayscale images labeled as objects or non-objects is compared with the feature amount of the window image. Then, an estimated value for estimating whether or not the window image is an object is output deterministically or probabilistically.

Here, the adder 22, the estimated value of the weak discriminator 21 _n, multiplied by the weight as a reliability for each weak discriminator 21 _n, and outputs the added value (the value of the weighted majority decision) this. Here, in AdaBoost, the plurality of weak classifiers 21 _n sequentially calculate estimated values, and the weighted majority values are sequentially updated accordingly. The plurality of weak classifiers are sequentially generated by group learning using the above-described learning samples according to an algorithm described later by the group learning machine 6, and for example, the estimated values are calculated in the order of generation. Further, the weight (reliability) of the weighted majority decision is learned in a learning process described later for generating a weak classifier.

When the weak discriminator 21 _n should perform binary output, such as AdaBoost, for example, the weak discriminator 21 _n divides the inter-pixel difference feature by a threshold value to determine whether the object is a target object. Further, a plurality of threshold values may be used as the determination method based on the threshold values. Further, the weak classifier may probabilistically output a continuous value indicating the degree of whether the target object is based on the inter-pixel difference feature, for example, Real-AdaBoost. The feature quantity (threshold value) for discrimination required by these weak discriminators 21 _n is also learned according to the above algorithm at the time of learning.

  In addition, the weighted majority decision does not wait for the calculation results of all weak classifiers, and even during the calculation, depending on the value, it is determined that the object is not the target object, so the calculation is aborted. To do. With this abort process, the amount of calculation in the detection process can be significantly reduced. As a result, it is possible to shift to the next window image discrimination process during the calculation without waiting for the calculation results of all weak discriminators.

  In this way, the discriminator 5 calculates a weighted majority vote as an evaluation value for determining whether or not the window image is an object, and determination means for determining whether or not the window image is an object based on the evaluation value. Function as. Further, the discriminator 5 sequentially calculates and outputs an estimated value by a plurality of weak discriminators generated by learning in advance, and each weak discriminant obtained by learning for the estimated value every time the estimated value is calculated. Whether or not to abort the calculation of the estimated value by using the above-mentioned truncation threshold each time the weighted majority vote value multiplied by the weight for the unit is updated and the weighted majority vote value (evaluation value) is updated Can also be controlled.

  The discriminator 5 is generated by the group learning machine 6 using the learning sample and performing group learning according to a predetermined algorithm. Here, a group learning method in the group learning machine 6 will be described first, and then a method for determining an object from an input image using the discriminator 5 obtained by the group learning will be described.

(2) Group learning machine The group learning machine 6 that performs group learning using a boosting algorithm combines a plurality of weak discriminators as described above, and learns to obtain a strong determination result as a result. . Each weak discriminator has a very simple configuration, and one has a low ability to discriminate whether it is a face or not a face, but by combining several hundred to several thousand, for example, a high discrimination Can have ability. The group learning machine 6 uses, for example, a sample image consisting of a target object and a non-target object, for example, a face image and a non-face image, which are pre-corrected (labeled), for example, thousands of learning samples. A weak classifier is generated by selecting (learning) one hypothesis from a learning model (hypothesis combination) according to a predetermined learning algorithm, and the combination of the weak classifiers is determined. Although the weak discriminator itself has a low discrimination performance, a classifier having a high discrimination ability can be obtained as a result of the selection and combination of the weak discriminators. On the other hand, the weak classifiers are selected and the weights when the weighted majority of the output values are determined are learned.

  Next, a learning method of the collective learning machine 6 for obtaining a discriminator in which a number of appropriate weak discriminators are combined according to a learning algorithm will be described. Prior to the description of the learning method of the collective learning machine 6, learning is performed by collective learning. Among the learning data to be learned, the learning data that is the feature in the present embodiment, specifically, the inter-pixel difference feature for constituting the weak classifier, and the detection process (detection process) to be interrupted in the middle The truncation threshold value will be described.

(3) Configuration of Weak Discriminator The discriminator 5 according to the present embodiment is configured such that the weak discriminators constituting the discriminator 5 are the luminance values of two pixels selected in all pixels included in an image input to the weak discriminator. By using a very simple configuration for discriminating whether or not the face is based on the difference (difference feature between pixels), the calculation of the discrimination result of the weak discriminator is accelerated in the discrimination process. The image input to the weak classifier is a learning sample in the learning step, and is a window image cut out from the scaling image in the determination step.

FIG. 5 is a schematic diagram showing an image for explaining the inter-pixel difference feature. In this embodiment, in the present embodiment, the difference between the luminance values of any two pixels, for example, the difference between the luminance value I ₁ of the pixel 31 and the luminance value I ₂ of the pixel 32, that is, the following equation (2) is obtained. It is defined as an inter-pixel difference feature.

  Here, which inter-pixel difference feature is used for face detection is the capability of the weak classifier. Therefore, it is necessary to select a set of pixel positions used for the weak classifier from a combination of arbitrary two pixels included in the cut-out image (also referred to as a filter or a weak hypothesis).

  For example, in AdaBoost, the weak classifier is requested to have a deterministic output indicating whether it is +1 (target object) or -1 (non-target object). Therefore, in AdaBoost, a weak discriminator can be obtained by dividing a difference feature between pixels at one arbitrary pixel position into two (+1 or −1) using one or a plurality of thresholds.

  In addition, in the case of a boosting algorithm such as Real-AdaBoost or Gentle Boost that probabilistically outputs a continuous value (real value) indicating the probability distribution of the learning sample instead of such binary output, a weak classifier Outputs the probability (probability) that the input image is an object. The output of the weak classifier may be deterministic or stochastic as described above. First, these two types of weak classifiers will be described.

(3-1) Binary Output Weak Discriminator A weak discriminator that performs deterministic output performs 2-class discrimination of whether or not an object is present according to the value of the inter-pixel difference feature. Whether the luminance values of two pixels in the target image area are I _{1 and} I _2, and the threshold for determining whether or not the target object is based on the inter-pixel difference feature is Th, whether the following expression (3) is satisfied: It can be determined which class it belongs to.

  Here, in order to construct a weak classifier, it is necessary to determine two pixel positions and their threshold values. The determination method will be described later. The threshold determination of the above equation (3) is the simplest case. For threshold determination, the two thresholds shown in the following formula (4) or formula (5) can also be used.

6 (a) to 6 (c), the frequency is plotted on the vertical axis and the inter-pixel difference feature is plotted on the horizontal axis, and the three discrimination methods shown in the above formulas (3) to (5) are respectively used as data. It is a schematic diagram shown according to the characteristic case of frequency distribution. Here, in FIGS. 6A to 6C, y _i indicates the output of the weak discriminator, and the output of all learning samples in which the data indicated by the broken line is y _i = −1 (non-object). The values shown are the output values of all the learning samples in which the data indicated by the solid line is y _i = 1 (target object). When the frequency with respect to the same difference feature between pixels is taken with respect to a learning sample made up of a large number of face images and non-face images, histograms shown in FIGS. 6A to 6C are obtained.

  As shown in FIG. 6A, the histogram has a distribution like a normal curve in which the broken line data indicating the non-object and the solid line data indicating the object have the same distribution, and the peak position is deviated. In such a case, the boundary is set as the threshold Th, and it can be determined whether or not it is an object by the above equation (3). For example, in AdaBoost, when the output of the weak classifier is f (x), the output f (x) = 1 (target object) or −1 (non-target object). FIG. 6A shows an example in which when the inter-pixel difference feature is larger than the threshold value Th, the object is determined to be an object, and the output of the weak discriminator is f (x) = 1.

  Further, when the peak positions are at the same position and the distribution widths are different, the above formula (4) or formula is used with the vicinity of the upper limit value and the lower limit value of the inter-pixel difference feature having the narrower distribution as a threshold value. Whether the object is an object can be determined by (5). In FIG. 6B, an example in which a narrower distribution is determined as an object is determined, and in FIG. 6C, an object obtained by excluding a narrower distribution width from a wider distribution is determined as an object. In this example, the output of the weak classifier is f (x) = 1.

  The weak discriminator is configured by determining a certain inter-pixel difference feature and its threshold value, but it is necessary to select an inter-pixel difference feature that makes the error rate as small as possible by the determination, that is, has a high discrimination rate. is there. For example, the threshold value is determined so that two pixel positions are determined, the histogram shown in FIG. 6 is obtained with respect to the correctly-acquired learning sample, and the threshold value is such that the highest correct answer rate and the lowest incorrect answer rate (error rate) are obtained. It can be obtained by searching. For the two pixel positions, the one with the smallest error rate obtained together with the threshold may be selected. However, in AdaBoost, each learning sample is given a weight (data weight) that reflects the difficulty of discrimination, and an appropriate inter-pixel difference feature (whether the luminance value of two pixels at which position is used as a feature value) ) Is learned to minimize the weighted error rate described later.

(3-2) Weak discriminator of continuous value output As described above, as a weak discriminator that performs stochastic output, for example, a weak discriminator that outputs a continuous value, such as Real-AdaBoost or Gentle Boost, is used. is there. In this case, the degree to which the input image is an object is different from the above-described case where the discrimination problem is solved by a certain fixed value (threshold value) and binary output (f (x) = 1 or −1) is performed. Is output as a probability density function, for example.

Such a stochastic output indicating the degree (probability) of the target object is obtained by using P _p (x) as the probability density function of the target of the learning sample and P _n ( If x) is the probability density function of the non-object of the learning sample, the function f (x) shown in the following equation (6) can be obtained.

FIG. 7A is a diagram showing a characteristic case of the frequency distribution of data, with probability density on the vertical axis and inter-pixel difference features on the horizontal axis, and FIG. 7B is a function on the vertical axis. FIG. 8 is a graph showing a function f (x) in the data distribution shown in FIG. 7A, where f (x) is taken and the inter-pixel difference feature is taken on the horizontal axis. In FIG. 7A, the probability density indicating that the broken line is a non-target object and the probability density indicating that the solid line is a target object are shown. When the function f (x) is obtained from the above equation (6), a graph shown in FIG. 7B is obtained. In the discrimination process, the weak discriminator outputs a function f (x) corresponding to the inter-pixel difference feature d shown in the above formula (2) obtained from the input window image. This function f (x) indicates the degree of object-likeness. For example, when the non-object is -1 and the object is 1, the function f (x) takes continuous values from -1 to 1. Can do. For example, a table composed of the inter-pixel difference feature d and the corresponding f (x) is stored, and f (x) is read out from the table according to the input and output. Accordingly, although the storage amount is slightly larger than the threshold value Th or Th ₁ or Th ₂ which is a constant value, the discrimination performance is improved.

  These multiple estimation methods (discrimination methods) can be expected to improve discrimination performance when used in combination during ensemble learning. Moreover, if only any one of the determination methods is used, the execution speed performance can be extracted.

  The weak discriminator used in the present embodiment is characterized in that it can discriminate an object at a very high speed as described above because the feature amount (inter-pixel difference feature) used is very simple. It is. In this way, when detecting a face as an object, an extremely good discrimination result can be obtained by the threshold judgment shown in the simplest formula (3) of the above-described discrimination methods for the inter-pixel difference feature. Whether the weak classifier functions effectively depending on the method depends on the target problem, and the threshold setting method or the like may be selected as appropriate. Further, depending on the problem, instead of the difference between the luminance values of two pixels, the difference between the luminance values of a plurality of pixels may be used as a feature amount, or a feature amount combining them may be used.

(4) Abort threshold Next, the abort threshold will be described. In a group learning machine using boosting, normally, as described above, it is determined whether or not the window image is an object by a weighted majority vote of the outputs of all weak classifiers constituting the classifier 5. The weighted majority vote is calculated by sequentially adding the discrimination results (estimated values) of the weak classifiers. For example, the number of weak classifiers is t (= 1,..., K), the majority decision weight (reliability) corresponding to each weak classifier is α _t , and the output of each weak classifier is f _t (x). Then, the weighted majority value F (x) in AdaBoost can be obtained by the following equation (7).

  In FIG. 8, the horizontal axis represents the number of weak classifiers and the vertical axis represents the weighted majority value F (x) shown in the above equation (7), depending on whether the input image is an object or not. It is a graph which shows the change of the value F (x) of the weighted majority vote. In FIG. 8, data D1 to D4 indicated by solid lines sequentially calculate an estimated value f (x) calculated by a weak discriminator using an image (learning sample) labeled as an object as an input, and the weighted majority vote. The value F (x) is obtained sequentially. As shown in the data D1 to D4, when an object is an input image, the weighted majority value F (x) is positive due to a certain number of weak classifiers.

  Here, in the present embodiment, a method different from a normal boosting algorithm is introduced. That is, in the process of sequentially adding the discrimination results of the weak classifiers, even before obtaining the results of all weak classifiers, the discrimination is canceled for window images that can clearly be determined not to be objects. Is. At this time, a threshold value for determining whether to stop the discrimination is learned in the learning step. Hereinafter, the threshold used for determining whether or not to stop the determination is referred to as an abort threshold.

  When it is possible to reliably estimate the non-target object for all window images without using the output result of all weak classifiers, the calculation of the estimated value f (x) of the weak classifier is in progress. As a result, the amount of calculation can be significantly reduced as compared to performing weighted majority using all weak classifiers.

  The abort threshold can be a minimum value that can be taken by the weighted majority value of the discrimination result of the learning sample indicating the detection target object among the labeled learning samples. In the discrimination step, the result of the weak discriminator of the window image is sequentially weighted and output, that is, the weighted majority value is sequentially updated. Every time update is performed, that is, each time one weak discriminator outputs a discrimination result, a comparison is made. If the updated weighted majority value falls below the abort threshold, the window image is not an object and the computation is aborted. Thus, it is possible to further speed up the discrimination process by omitting useless calculations.

That is, the cutoff threshold R _K of the output f _K (x) of the K-th weak discriminator is the learning sample x _j (= x ₁ to x ₁ ) that is the object among the learning samples x _i (= x _{1 to} x _N ). x _J ) is used as the minimum value of the weighted majority vote, and is defined as the following equation (8).

As shown in this equation (8), 0 is set in the termination threshold R _K if the minimum value of the weighted majority value of the learning samples x ₁ ~x _J as an object is greater than 0. It should be noted that not exceeding 0 is the case of AdaBoost in which discrimination is performed using 0 as a threshold, and this may differ depending on the group learning technique. In the case of AdaBoost, as shown by the bold line in FIG. 8, the abort threshold is set to the minimum value that can be taken out of all data D1 to D4 when an object is input as an input image, and all data D1 to D4 If the minimum value exceeds 0, the abort threshold is set to 0.

In the present embodiment, by learning the truncation threshold value R _t (R _{1 to} R _k ) each time a weak classifier is generated, in a discrimination process described later, for example, a plurality of data D5 The estimated value is sequentially output by the weak discriminator, and the weighted majority value is successively updated. When this value falls below the abort threshold, the process of discriminating by the subsequent weak discriminator ends. That is, by learning this truncation threshold value _Rt , it is possible to determine whether or not to calculate the next weak classifier every time the estimated value of the weak classifier is calculated. In this case, it can be determined that the object is a non-object without waiting for the determination results of all weak classifiers, and the detection process can be speeded up by aborting the calculation.

(5) Learning Method Next, a learning method of the group learning machine 6 will be described. An image (training data) that is a learning sample that has been manually labeled (corrected) in advance as a premise of a general pattern recognition problem for two-class classification, such as a problem of determining whether the given data is a face, for example. Prepare. The learning sample includes an image group obtained by cutting out a region of the target object to be detected, and a random image group obtained by cutting out a landscape image or the like that has nothing to do with it.

A learning algorithm is applied based on these learning samples to generate learning data used for discrimination. In this embodiment, the learning data used at the time of discrimination is the following four learning data including the learning data described above. That is,
(A) A set of two pixel positions (K)
(B) Weak classifier threshold (K)
(C) Weight of majority vote (reliability of weak classifier) (K)
(D) Abort threshold (K)

(5-1) Generation of Discriminator An algorithm for learning the four types of learning data shown in (A) to (D) above from a large number of learning samples as described above will be described. FIG. 9 is a flowchart showing a learning method of the group learning machine 6. Here, learning according to an algorithm (AdaBoost) that uses a constant value as a threshold value for weak discrimination will be described as a learning algorithm, but a continuous value indicating the probability (probability) of the correct answer is used as the threshold value. For example, a learning algorithm is not limited to AdaBoost as long as it performs collective learning to combine a plurality of weak classifiers, such as Real-AdaBoost.

(Step S0) Labeling of Learning Sample As described above, a learning sample (x _i , y _i ) that has been previously labeled as an object or a non-object is prepared.
here,
Learning sample (x _i , y _i ): (x ₁ , y ₁ ), ..., (x _N , y _N )
x _i εX, y _i ε {−1,1}
X: Learning sample data Y: Learning sample label (correct answer)
N: Indicates the number of learning samples. That is, x _i represents a feature vector made up of all luminance values of the learning sample image. Further, y _i = −1 indicates a case where the learning sample is labeled as a non-object, and y _i = 1 indicates that the learning sample is labeled as an object.

(Step S1) Initialization of data weight In boosting, the weight (data weight) of each learning sample is varied to relatively increase the data weight for the learning sample that is difficult to discriminate. The discriminant result is used to calculate the error rate (error) for evaluating the weak discriminator, but by multiplying the discriminant result by the data weight, the evaluation of the weak discriminator that erroneously discriminates the more difficult learning sample is actually performed. Will fall below the discrimination rate. Although the data weight is sequentially updated by a method to be described later, first, the data weight of this learning sample is initialized. The initialization of the data weights of the learning samples is performed by making the weights of all the learning samples constant, and is defined as the following equation (9).

Here, the data weight D _{1, i} of the learning sample indicates the data weight of the learning sample x _i (= x _{1 to} x _N ) for the _first iteration t = 1. N is the number of learning samples.

(Steps S2 to S7) Repetitive Processing Next, the discriminator 5 is generated by repeating the processing of steps S2 to S7 shown below. Here, it is assumed that the number of repetition processes is t = 1, 2,. Each time iterative processing is performed, one weak discriminator, that is, one set of pixels and the inter-pixel difference feature at that position are learned. Accordingly, weak discriminators are generated for the number of times of repeated processing (K times). Thus, the discriminator 5 composed of K weak discriminators is generated. Usually, hundreds to thousands of iterative processes generate hundreds to thousands of weak classifiers, but the number of iterations (number of weak classifiers) t is the required discrimination performance, What is necessary is just to set suitably according to the problem (object) to identify.

(Step S2) Weak classifier learning In step S2, weak classifier learning (generation) is performed. This learning method will be described later. In the present embodiment, one weak discriminator is generated according to a method described later for each repetition process.

(Step S3) Calculation of Weighted Error Rate e _t Next, the weighted error rate of the weak discriminator generated in Step S2 is calculated by the following equation (10).

As shown in the equation (10), the weighted error ratio e _t, among the learning samples, which is incorrect determination result of the weak discriminator _{(f t (x i) ≠} y i) of a is learning samples Only the data weights are added, and as described above, the weighted error rate _et is calculated to be large if the data sample Dt _{, i} has a large (difficult to distinguish) learning sample. The weighted error rate _et is less than 0.5, and the reason will be described later.

Calculation of (Step S4) of weighted majority decision weight (weak discriminator reliability) Next, based on the weighted error ratio e _t shown in equation (10) described above, the weight of weighted majority decision (hereinafter, referred reliability .) The reliability α _t is calculated by the following equation (11). The weight of the weighted majority vote indicates the reliability α _t of the weak classifier generated at the t-th iteration.

As shown in the equation (10), the reliability alpha _t of the weak classifier increases as the ones weighted error ratio e _t is small.

(Step S5) Data Weight Update of Learning Sample Next, the data weight D _{t, i} of the learning sample is updated by the following equation (12) using the reliability α _t obtained by the above equation (11). . The data weights D _{t, i} are normally normalized to be 1 when all are added, and the following equation (13) is for normalizing the data weights D _{t, i} .

(Step S6) calculates the abort threshold value R _t Next, as described above, calculates the abort threshold value R _t for aborting the determination at determination step. As the censoring threshold value R _t , the smallest value among the weighted majority values of learning samples (positive learning samples) x _{1 to} x _J or 0 is selected according to the above equation (8). As described above, the minimum value or 0 is set as the abort threshold in the case of AdaBoost in which discrimination is performed using 0 as the threshold. In any case, the truncation threshold value _Rt is set to a maximum value that allows at least all positive learning samples to pass.

  In step S7, it is determined whether or not a predetermined number of times (= K times) of boosting has been performed. If not, the processes in steps S2 to S7 are repeated. When the predetermined number of times of learning is finished, the learning process is finished. This iterative process ends when a number of weak discriminators that can sufficiently discriminate an object to be detected are learned from a given image such as a learning sample.

(5-2) Generation of weak classifier Next, the learning method (generation method) of the weak classifier in step S2 described above will be described. The weak discriminator is generated differently when the weak discriminator outputs a binary value and when a continuous value is output as the function f (x) shown in the above equation (6). Even in the case of binary output, the processing is slightly different between the case of determination using one threshold Th and the case of determination using two thresholds Th ₁ and Th ₂ as shown in the above equation (2). Here, a learning method (generation method) of a weak classifier that outputs a binary value with one threshold Th will be described. FIG. 10 is a flowchart showing a learning method (generation method) of a weak classifier that outputs a binary value with one threshold Th.

(Step S11) Selection of Pixels Here, arbitrary two are selected from all the pixels in the learning sample. For example, when a learning sample of 20 × 20 pixels is used, there are 400 × 399 selection methods for two pixels, and one of them is selected. Here, it is assumed that the positions of the two pixels are S ₁ and S ₂ , and the luminance values thereof are I ₁ and I ₂ , respectively.

(Step S12) Frequency Distribution Creation Next, for all learning samples, an inter-pixel difference feature d, which is a difference (I ₁ -I ₂ ) between the luminance values of the two pixels selected in step S11, is obtained. A histogram (frequency distribution) as shown in FIG.

(Step S13) Calculation of the threshold _{Th min} Then, from the frequency distribution obtained in step S12, obtains the threshold _{Th min} that the weighted error ratio _{e t} shown in the equation (10) to the minimum _{(e min).}

(Step S14) the threshold _{Th max} calculated yet, it obtains the threshold _{Th max} that maximizes _{(e max)} The weighted error ratio _{e t} shown in the equation (10), inverts the threshold value by the following method (14) To do. That is, the weak discriminator outputs two values of correct answer and incorrect answer depending on whether or not it is larger than one threshold value Th. Therefore, when the weighted error rate _et is less than 0.5, it is inverted. By doing so, the error rate can be made 0.5 or more.

(Step S15) Parameter Determination Finally, each parameter constituting the weak classifier, that is, the positions S ₁ and S ₂ of the _two pixels, and the threshold Th thereof are determined from the above-described e _min and e _max ′. That is,
When e _min <e _max ′: S ₁ , S ₂ , Th _min
When e _min > e _max ′: S ₁ ′ (= S ₂ ), S ₂ ′ (= S ₁ ), Th _min
Then, in step S16, it is determined whether or not the predetermined number of times has been repeated. If the predetermined number of times has been repeated, the process proceeds to step S17, and the error rate _et is the highest among the weak discriminators generated by the M number of repetition processes. The smaller one is used as a weak discriminator, and the process proceeds to step S3 shown in FIG. On the other hand, if the predetermined number of times has not been reached in step S16, the processes in steps S11 to S16 are repeated. As described above, m (= 1, 2,..., M) iterations are performed to generate one weak classifier. For convenience of explanation, it has been described that the weighted error rate _et is calculated in step S3 shown in FIG. 9. However, in step S17, when the weak discriminator having the smallest error rate _et is selected, step S3 is performed. error ratio e _t shown in the automatically obtained.

In the present embodiment, the data weights D _{t, i} obtained in step S5 in the previous iterative process are used to learn the feature quantities of a plurality of weak classifiers, and these weak classifiers (weak classifiers). The case where one weak classifier is generated by selecting the one with the smallest error rate shown in the above formula (10) from among the candidate generators has been described. In step S2, for example, preparation or learning is performed in advance. The weak discriminator may be generated by selecting an arbitrary pixel position from the plurality of pixel positions. Moreover, you may produce | generate a weak discriminator using the learning sample different from the learning sample used for the iterative process to the above-mentioned step S2-step S7. In addition, the weak classifier and classifier generated by preparing a sample different from the learning sample, such as the evaluation of the cross-validation method or jack-knife method, should be evaluated. Also good. Here, the cross-validation means that the learning sample is equally divided into I pieces, learning is performed using one of them, and the learning result is evaluated using the one for I times. This is a method for evaluating learning results.

On the other hand, when the weak discriminator has two threshold values Th ₁ and Th ₂ as shown in the above formula (4) or formula (5), the processing in steps S13 to S15 shown in FIG. 10 is slightly different. As shown in the above equation (3), when the threshold value Th is one, the error rate can be reversed when the error rate is larger than 0.5 by reversing. As shown, when the difference feature between pixels is larger than the threshold Th _{2 and} smaller than the threshold Th is a correct discrimination result, when this is reversed, as shown in the equation (5), the difference feature is smaller than the threshold Th ₂ or the threshold Th When Th _{1 is} greater, the correct answer is determined. That is, the inversion of Expression (4) becomes Expression (5), and the inversion of Expression (5) becomes Expression (4).

When the weak discriminator has two threshold values Th ₁ and Th ₂ and outputs a discrimination result, in step S12 shown in FIG. 10, the frequency distribution in the inter-pixel difference feature is obtained, and the error rate _et is minimized. Threshold values Th ₁ and Th ₂ are obtained. Then, similarly to step S16, it is determined whether or not it has been repeated a predetermined number of times, and after repeating the predetermined number of times, the weak discriminator having the smallest error rate among the generated weak discriminators is adopted.

  Further, as shown in the above equation (6), in the case of a weak discriminator that outputs a continuous value, first, two pixels are randomly selected as in step S11 shown in FIG. And the frequency distribution in all the learning samples is calculated | required similarly to step S12. Then, based on the obtained frequency distribution, a function f (x) shown in the above equation (6) is obtained. Then, a series of processes of calculating an error rate according to a predetermined learning algorithm that outputs the degree of being an object (degree of correct answer) as the output of the weak classifier is repeated a predetermined number of times, and the error rate is the smallest (correct answer) A weak classifier is generated by selecting a parameter having a high rate.

  Here, in the generation of any weak discriminator, for example, when a learning sample of 20 × 20 pixels is used, there are 159000 selection methods for two pixels, and the above-described iterative processing is performed at maximum M = 159000 times. Among them, the one with the smallest error rate can be adopted as the weak classifier. In this way, the maximum number of weak discriminators that can be generated is generated by repeating the maximum number of iterations, and a weak discriminator with high performance is generated when the one with the smallest error rate is adopted as the weak discriminator. However, it is also possible to repeat the processing less than the maximum number of times, for example, several hundred times, and adopt the one with the smallest error rate.

(6) Object detection method Next, the object detection method of the object detection apparatus shown in FIG. 1 is demonstrated. FIG. 11 is a flowchart illustrating the object detection method. At the time of detection (discrimination step), the discriminator 5 using the weak discriminator group generated as described above is used to detect the target object from the image according to a predetermined algorithm.

(Step S21) Scaling Image Generation First, the scaling unit 3 shown in FIG. 1 scales the grayscale image supplied from the image output unit 2 at a certain rate. The image output unit 2 may receive a grayscale image as an input image, and the image output unit 2 may convert the input image into a grayscale image. The image supplied from the image output unit 2 is output to the scaling unit 3 without scaling, and the scaled image scaled down after the next timing is output. The images output from the scaling unit 3 are collectively referred to as a scaled image. Here, the timing for generating the scaled image is the time when the face detection of the entire area of the scaled image output previously is completed, and when the scaled image becomes smaller than the window image, the process proceeds to the processing of the input image of the next frame.

(Step S22)
The scanning unit 4 shown in FIG. 1 scans the position of the search window vertically and horizontally with respect to the scaled image, and outputs a window image.

(Steps S23 and 24) Calculation of Evaluation Value s Then, it is determined whether or not the window image output by the scanning unit 4 is an object. The discriminator 5 calculates, as the evaluation value s, a value obtained by sequentially adding the estimated values f (x) of the plurality of weak discriminators described above to the window image with weighting (update value of the weighted majority decision value). Then, based on the evaluation value s, it is determined whether or not the window image is an object and whether or not the determination is terminated.

First, when a window image is input, the evaluation value s = 0 is initialized. Weak classifiers 21 ₁ of the first-stage determination unit 5 calculates the inter-pixel difference feature _{d t} (step S23). And an estimation value output from the the weak discriminator 21 ₁ is reflected in the evaluation value s (Step S24).

  Here, the weak discriminator that outputs the binary estimated value and the weak discriminator that outputs the function f (x) shown in the equation (6) as the estimated value according to the above-described equations (3) to (5). The method of reflecting the estimated value on the evaluation value s is different.

  First, when the above equation (2) is used as a weak classifier and a binary value is output as an estimated value, the evaluation value s is as shown in (15) below.

  When the above equation (3) is used as a weak classifier and a binary value is output as an estimated value, the evaluation value s is expressed by the following equation (16).

  When the above equation (4) is used as a weak classifier and a binary value is output as an estimated value, the evaluation value s is expressed by the following equation (17).

  When the above equation (5) is used for the weak classifier and the function f is output as an estimated value, the evaluation value s is expressed by the following equation (18).

(Steps S25, S26) Abort Determination Then, the discriminator 5 determines whether or not the (updated) evaluation value s obtained by any one of the four methods described above is larger than the abort threshold R _t. . When the evaluation value s is larger than the abort threshold value R _t is a predetermined number of times (= K times) repeated or not (step S26), if not repeat the process is repeated from step S23.

On the other hand, a predetermined number of times (= K times) if repeatedly are, and when the evaluation value s is smaller than the abort threshold value R _t, the process proceeds to step S27, resulting in that the evaluation value s is the zero or not greater than, the object It is determined whether or not. If it is an object, the current window position is stored, and it is determined whether or not there is a next search window (step S27). If there is a next search window, the processing from step S22 is repeated. If the search window has been scanned for all the next areas, the process proceeds to step S28 to determine whether or not there is a next scaled image. If not, the process proceeds to step S29 to execute the overlapping area deletion process. If there is a scaled image, the processing from step S21 is repeated. The scaling process in step S21 ends when the scaled image becomes smaller than the window image.

(Steps S29 to S31) Deletion of Overlapping Region When processing of all the scaled images is completed for one input image, the process proceeds to step S29. In the processing after step S29, when the areas determined to be the target objects overlap in one input image, the overlapping areas are removed. First, it is determined whether or not there are areas overlapping each other. If there are a plurality of areas stored in step S26 and there are overlapping areas, the process proceeds to step S30. Then, two regions that overlap each other are extracted, and the region having a small evaluation value s is regarded as having low reliability, and the region having a large evaluation value s is selected from the two regions (step S29). Then, the processing from step S29 is repeated again. As a result, only one region having the highest evaluation value s is selected from the regions that are extracted multiple times. If two or more object areas do not overlap or there is no object area, the process for one input image is terminated, and the process proceeds to the next frame process.

As described above, according to the object detection method in the present embodiment, a weak discriminator that weakly discriminates based on an inter-pixel difference feature is used to detect an object using a discriminator learned by collective learning. By simply reading the luminance values of the two corresponding pixels and calculating the difference between them, the calculation process of the feature amount of the object in step S23 is completed, and the face detection processing can be performed at a very high speed. Face detection is possible. Further, each time the evaluation value s is sequentially updated by adding a value obtained by multiplying the discrimination result (estimated value) discriminated from the feature amount and the reliability of the weak discriminator used for discrimination, it is compared with the cutoff threshold R _t. Then, it is determined whether or not to continue the calculation of the estimated value of the weak classifier. Then, abort the operation of the weak discriminator when it falls below the evaluation value s of the abort threshold value R _t, by moving to the next window image processing, further face detection at a high speed dramatically recommendations wasteful operations It becomes possible. That is, when a window image is cut out by scanning the input image and all areas of the scaled image obtained by reducing and scaling the input image, the probability that the window image is an object is small, and most of the window images are non-objects. By discriminating the window image that is the non-object, the discrimination process can be made highly efficient. On the other hand, if there are a lot of objects to be detected, a threshold value may be provided so that the calculation of the window image that is clearly the object is interrupted by the same method as the above-described threshold value. Good. Further, by scaling the input image by the scaling unit, a search window having an arbitrary size can be set and an object having an arbitrary size can be detected.

(7) Example Next, an example of the present invention in which a face is actually detected as an object will be described. Note that the object is not limited to a face, but has features on a two-dimensional plane such as a logotype, a pattern, or an object image other than a human face, and is determined to some extent by the above-described inter-pixel difference feature. It is needless to say that any object can be detected as long as it can perform (a weak classifier can be configured).

  FIG. 12A and FIG. 12B are diagrams showing a part of the learning samples of this embodiment. The learning sample uses a face image group shown in FIG. 12A labeled as an object and a non-face image group shown in FIG. 12B labeled as a non-object. FIGS. 12A and 12B show a part of an image used as a learning sample. Examples of the learning sample include several thousand face images and tens of thousands of non-face images. Is used. The image size is, for example, 20 × 20 pixels.

In this embodiment, a face discrimination problem using only the above equation (3) is learned from these learning samples according to the algorithm shown in FIGS. FIGS. 13A to 13F show the first to sixth weak classifiers generated first by such learning. These are considered to represent facial features well. Qualitatively, the weak discriminator f ₁ in FIG. 13A indicates that the forehead (S ₁ ) is brighter than the eye (S ₂ ) (threshold value: 18.5), and the weak discrimination in FIG. 13B. The vessel f ₂ indicates that the cheek (S ₁ ) is brighter than the eyes (S ₂ ) (threshold value: 17.5). Furthermore, the weak discriminator _{f 3} in FIG. 13 (c), the amount _{(S 1)} is lighter than the hair _{(S 2)} (threshold value: 26.5) shows that, weak classifier _{f 4} shown in FIG. 13 (d) Indicates that the bottom of the nose (S ₁ ) is brighter than the nasal cavity (S ₂ ) (threshold: 5.5). Further, the weak discriminator f ₅ in FIG. 13 (e) indicates that the cheek (S ₁ ) is brighter (threshold: 22.5) than the hair (S ₂ ), and the weak discriminator f _{6 in} FIG. 13 (F). Indicates that the chin (S ₁ ) is brighter than the lips (S ₂ ) (threshold value: 4.5).

In the present embodiment, 70% correct rate by the first one of the weak classifier f ₁ can be obtained (performance on learning sample), 80% by utilizing all the weak classifiers f ₁ ~f ₆ Reach the correct answer rate. By combining 40 weak classifiers, a correct answer rate of 90% was achieved, and by combining 765 weak classifiers, a correct answer rate of 99% could be reached.

  FIG. 14 is a diagram showing a face detection result detected from one input image, and (a) and (b) are diagrams showing before and after removing the overlapping region, respectively. A plurality of frames shown in FIG. 14 (a) are detected faces (objects), and a plurality of faces (regions) are detected by processing from one image to steps S21 to S28 shown in FIG. Is done. This can be detected as one face by performing the overlapping area removal processing shown in steps S29 to S31. If there are a plurality of faces in the image, the plurality of faces can be detected simultaneously. As described above, the face detection process according to the present embodiment can be processed at a very high speed, and can detect a face from about 30 input images per second even if a normal PC or the like is used. It is also possible to detect a face from a moving image.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a functional block diagram which shows the processing function of the target object detection apparatus in embodiment of this invention. It is a schematic diagram which shows the image scale-converted by the scaling part in the said target object detection apparatus. It is a figure which shows a mode that the scanning part in the said target object detection apparatus scans a search window. It is a schematic diagram which shows the structure of the discriminator in the said target object detection apparatus. It is a schematic diagram which shows the image for demonstrating the difference feature between pixels. In (a) to (c), the frequency is plotted on the vertical axis, and the inter-pixel difference feature is plotted on the horizontal axis. It is a schematic diagram shown according to a characteristic case. (A) is a diagram showing a characteristic case of frequency distribution of data, with probability density on the vertical axis and inter-pixel difference features on the horizontal axis, and (b) is a function f (x) on the vertical axis. FIG. 7 is a graph showing a function f (x) in the data distribution shown in FIG. Taking the number of weak classifiers on the horizontal axis and the weighted majority value F (x) on the vertical axis, the weighted majority value F (x) according to whether the input image is an object or not. It is a graph which shows a change. It is a flowchart which shows the learning method of the group learning machine for obtaining the discriminator in the said target object detection apparatus. It is a flowchart which shows the learning method (generation method) of the weak discriminator which outputs a binary value with one threshold value Th. It is a flowchart which shows the target object detection method in the said target object detection apparatus. (A) And (b) is a figure which shows a part of learning sample used for the Example of this invention, Comprising: The facial image group labeled as a target object, and the non-facial image group labeled as a non-target object, respectively FIG. (A) thru | or (f) is a figure explaining the 1st-6th weak discriminator each produced | generated initially by the learning in the said group learning machine. It is a figure which shows the face detection result detected from one sheet of input images, Comprising: (a) And (b) is a figure which shows before and after removing an overlapping area | region, respectively. It is a schematic diagram which shows the rectangular feature (rectangle feature) of patent document 1. FIG. It is a figure explaining the method to discriminate | determine in a face image using the rectangular feature of patent document 1. FIG. It is a schematic diagram which shows the integral image (integral image) described in patent document 1. FIG. It is a figure explaining the method of calculating the sum total of the luminance value of a rectangular area using the integral image of patent document 1. FIG.

Explanation of symbols

1 object detecting device, second image input unit, 3 a scaling unit, 4 a scanning unit, 5 discriminator 6 group learner, 21 _n weak classifiers, 22 an _adder, f 1 ~f ₆ weak classifier

Claims (28)

In an object detection device for detecting whether a given grayscale image is an object,
A plurality of weak discriminating means for calculating an estimated value indicating whether or not the grayscale image is an object based on a feature amount consisting of a difference between luminance values of pixels at two positions learned in advance;
An object detection apparatus comprising: a determination unit that determines whether the grayscale image is an object based on the estimated value calculated by at least one of the plurality of weak determination units. The discriminating means calculates a weighted majority value obtained by multiplying the estimated value by the reliability for each weak discriminating means obtained by the learning and adding the result, and based on the majority value, the gray image is targeted. It is discriminate | determined whether it is an object. The target object detection apparatus of Claim 1 characterized by the above-mentioned. The plurality of weak discrimination means sequentially calculate the estimated value,
The discriminating means sequentially updates the weighted majority value every time the estimated value is calculated, and controls whether or not to stop the calculation of the estimated value based on the updated weighted majority value. The object detection device according to claim 2. The discriminating means aborts the calculation of the estimated value depending on whether the value of the weighted majority vote is smaller than an abort threshold,
Each of the weak discriminating means is sequentially generated by group learning using a learning sample composed of a plurality of grayscale images that are correctly identified as being an object or a non-object,
The truncation threshold value is a value obtained by weighting the reliability to the estimated value for the learning sample that is the object calculated by the generated weak discriminating means every time the weak discriminating means is generated during the learning. The object detection device according to claim 3, comprising a minimum value of weighted majority values updated by addition. The object detection device according to claim 4, wherein when the minimum value of the weighted majority vote at the time of learning is positive, 0 is set as the truncation threshold value. The weak discriminating means calculates the binary estimated value indicating whether or not the object is an object in accordance with whether or not the feature amount of the grayscale image is equal to or greater than a predetermined threshold value. Item 1. The object detection apparatus according to Item 1. The object detection apparatus according to claim 1, wherein the weak determination unit calculates, as the estimated value, a probability that the grayscale image is an object based on the feature amount. In an object detection method for detecting whether a given grayscale image is an object,
A weak discrimination step of calculating an estimated value indicating whether or not the grayscale image is an object based on a feature amount consisting of a difference between luminance values of pixels at two positions learned in advance by a plurality of weak discriminators;
And a discrimination step of discriminating whether or not the grayscale image is an object based on the estimated value calculated by at least one of a plurality of weak classifiers. In the determining step, a weighted majority value obtained by multiplying the estimated value by the reliability for each weak classifier obtained by the learning and adding the calculated value is calculated, and the grayscale image is processed based on the majority value. It is discriminate | determined whether it is an object. The target object detection method of Claim 8 characterized by the above-mentioned. In the weak discrimination step, the estimated value is sequentially calculated by the plurality of weak discriminators,
In the determining step, each time the estimated value is calculated, the weighted majority value is sequentially updated,
The object detection method according to claim 9, further comprising: an abort control step of controlling whether or not to cancel the calculation of the estimated value based on the updated weighted majority value in the determination step. In a group learning device that performs group learning using a learning sample consisting of a plurality of grayscale images that are correctly identified as being an object or a non-object,
Collectively learning multiple weak classifiers that use the above learning samples and output an estimated value indicating whether the grayscale image given as an input is a target value using the difference between the luminance values of two pixels at an arbitrary position as a feature quantity A group learning apparatus characterized by comprising learning means for performing The learning means is
Weak classifier generating means for calculating the feature quantity of each learning sample and generating the weak classifier based on each feature quantity;
For the weak discriminator generated by the weak discriminator generating means, an error rate calculating means for calculating an error rate for discriminating the learning sample based on the data weight set for each learning sample;
A reliability calculation means for calculating a reliability for the weak classifier based on the error rate;
Data weight calculating means for updating the data weight so that the weight of the learning sample that the weak discriminator makes an incorrect answer relatively increases,
The group learning device according to claim 11, wherein the weak classifier generation unit generates a new weak classifier when the data weight is updated. The weak discriminator generating means repeats the process of calculating the feature quantity a plurality of times to calculate a plurality of types of feature quantities, generates weak discriminator candidates for each feature quantity, and generates a plurality of weak discriminators. 13. An error rate obtained by discriminating the learning sample is calculated based on the data weight set for each learning sample, and the one having the smallest error rate is used as the weak discriminator. The group learning device described. The weak discriminator generating means generates the weak discriminant candidate for determining whether or not the object is an object in accordance with whether or not the feature amount of the grayscale image is a predetermined threshold value or more. The group learning device according to claim 12. The group learning device according to claim 12, wherein the weak discriminator generating unit generates the weak discriminant candidate that outputs a probability that the grayscale image is an object based on the feature amount. Each time the weak discriminator generating means generates the weak discriminator, the weak discriminator calculates the estimated value for each learning sample that is the object, and weights and adds the reliability to the estimated value. The group learning device according to claim 12, further comprising: an abort threshold value storage unit that calculates a weighted majority value and stores the minimum value. In a group learning method for performing group learning using a learning sample consisting of a plurality of grayscale images with correct answers as to whether they are objects or non-objects,
Collectively learning multiple weak classifiers that use the above learning samples and output an estimated value indicating whether the grayscale image given as an input is a target value using the difference between the luminance values of two pixels at an arbitrary position as a feature quantity A collective learning method characterized by having a learning step. In the learning step, the feature amount of each learning sample is calculated, and the weak discriminator generation step that generates the weak discriminator based on each feature amount, and the weak discriminator generated by the weak discriminator generation step, An error rate calculating step of calculating an error rate by discriminating the learning sample based on the data weight set for each learning sample; a reliability calculating step of calculating a reliability for the weak discriminator based on the error rate; The group learning method according to claim 17, wherein a series of steps including the data weight calculation step of updating the data weight is repeated so that the weight of the learning sample that the weak discriminator makes an incorrect answer is relatively increased. In the weak discriminator generation step, a plurality of types of feature quantities are calculated by repeating the process of calculating the feature quantities a plurality of times, weak discriminator candidates are generated for each feature quantity, and each of the generated weak discriminators is generated. An error rate obtained by discriminating the learning sample is calculated based on the data weight set for each learning sample, and the weak error discriminator is used as the weak discriminator. 18. The group learning method according to 18. In the weak discriminator generating step, the weak discriminant candidate for determining whether or not the object is an object according to whether or not the feature amount of the grayscale image is equal to or greater than a predetermined threshold is generated. The group learning method according to claim 18. The group learning method according to claim 18, wherein, in the weak discriminator generation step, the weak discriminant candidate that outputs a probability that the grayscale image is an object is generated based on the feature amount. Each time the weak discriminator is generated in the weak discriminator generation step, the weak discriminator calculates the estimated value for each learning sample that is the object, and adds the reliability to the estimated value by weighting. The group learning method according to claim 18, further comprising: an abort threshold value storing step of calculating a weighted majority value and storing the minimum value. In an object detection device that cuts out a fixed-size window image from a grayscale image and detects whether the window image is an object,
Scale conversion means for generating a scale image obtained by enlarging or reducing the size of the input grayscale image;
Window image scanning means for scanning the window of the fixed size from the scale image and cutting out the window image;
Object detection means for detecting whether a given window image is an object or not,
The object detection means includes a plurality of weak discrimination means for calculating an estimated value for estimating whether or not the window image is an object based on a feature amount consisting of a difference between luminance values of pixels at two positions learned in advance. An object detection apparatus comprising: determination means for determining whether or not the window image is an object based on the estimated value calculated by at least one of a plurality of weak determination means. The discriminating means calculates a weighted majority value obtained by multiplying the estimated value by the reliability for each weak discriminating means obtained by the learning and adding the result, and based on the majority value, the gray image is targeted. It is discriminate | determined whether it is an object. The target object detection apparatus of Claim 23 characterized by the above-mentioned. The plurality of weak discrimination means sequentially calculate the estimated value,
The discriminating means sequentially updates the weighted majority value every time the estimated value is calculated, and controls whether or not to stop the calculation of the estimated value based on the updated weighted majority value. 25. The object detection device according to claim 24. In an object detection method for cutting out a fixed-size window image from a grayscale image and detecting whether the window image is an object,
A scale conversion step for generating a scale image obtained by enlarging or reducing the size of the input grayscale image;
A window image scanning step of scanning the fixed-size window from the scale image and cutting out the window image;
An object detection step of detecting whether a given window image is an object, or
The object detection step includes
A weak discrimination step of calculating an estimated value indicating whether or not the grayscale image is an object based on a feature amount consisting of a difference between luminance values of pixels at two positions learned in advance by a plurality of weak discriminators;
An object detection method comprising: a determination step of determining whether the grayscale image is an object based on the estimated value calculated by at least one of a plurality of weak classifiers. In the determining step, a weighted majority value obtained by multiplying the estimated value by the reliability for each weak classifier obtained by the learning and adding the calculated value is calculated, and the grayscale image is processed based on the majority value. It is discriminate | determined whether it is an object. The target object detection method of Claim 26 characterized by the above-mentioned. In the weak discrimination step, the estimated value is sequentially calculated by the plurality of weak discriminators,
In the determining step, each time the estimated value is calculated, the weighted majority value is sequentially updated,
28. The object detection method according to claim 27, further comprising an abort control step of controlling whether or not to cancel the calculation of the estimated value based on the updated weighted majority value in the discrimination step.

Images