JP2006318341A - Detection object image determination device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for efficiently detecting and determining a detection object image such as a human image part. <P>SOLUTION: This detection object image determination device determines whether a given grayscale image is a detection object image or not. The detection object image determination device is provided with a plurality of weak determination means, which is arranged for each pair in a plurality of adjacent or proximity pixel pairs in two positions previously decided by learning among pixels constituting the grayscale image for finding a difference between luminance values of two pixels in a pixel pair as characteristic quantity and calculating an estimation value showing whether a pair of pixel is an outline part of the detection object image or not based on the found characteristic quantity. The determination means determines whether the given grayscale image is the detection object image or not based on the estimation value calculated by a plurality of weak determination means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a detection target image determination apparatus, method, and program for detecting a detection target image such as a person image from a given grayscale image.

  Conventionally, as a method of automatically finding a person image portion from an image by a computer, a method of detecting a moving portion as a person image portion, a skin color of a face or a head representing a person image, There is a method in which a human image portion is detected by detecting with a facial part.

  As an example of the former, there is a method of detecting a changed portion as a person image portion by taking a difference between a background image created in advance and an input image (for example, Japanese Patent Application Laid-Open No. 11-311682). )reference).

  As an example of the latter, there is a method in which a face is modeled as an ellipse and face detection is performed by detecting an ellipse in a skin color area (see, for example, Patent Document 2 (Japanese Patent Laid-Open No. 11-185026)).

  In addition to this, for example, Patent Document 3 (Japanese Patent Laid-Open No. 2001-222719) discloses a method of detecting a head by modeling a head as a circle, creating a template, and detecting a circular shape by voting. Is disclosed.

  Further, in Patent Document 4 (Japanese Patent Application Laid-Open No. 2004-178229), similar to Patent Document 3, it is approximated by a shape model surrounded by a boundary curve convex to the head and the outside, and the shape is detected by voting, A method of performing head detection is disclosed.

The above-mentioned patent documents are as follows.
JP-A-11-311682 Japanese Patent Laid-Open No. 11-185026 JP 2001-222719 A JP 2004-178229 A

  However, as in the invention of Patent Document 1, in the method of finding the changed portion, an area where the brightness on the image is greatly changed is used as the person image portion. Even when doing so, there is a problem of detecting as a person.

  Further, in the case of the invention of Patent Document 2, it is difficult to stably detect the skin color of the person image portion in various lighting environments. Elliptical fitting is an effective means for a face in which a person is facing the front, but is difficult to use for detection when the person is facing obliquely. There is also a problem that the head cannot be detected when the person is facing backward.

  Further, in the invention of Patent Document 3, since the shape of the template used for voting used for detection is a concentric circle, there are many votes that do not contribute to detection, processing time is wasted, and false voting peaks are generated. There is a problem that false detection occurs.

  Furthermore, in the invention of Patent Document 4, since the template is created on the assumption that the head is convex outward, the assumption does not hold depending on the shooting direction and the like, and the person image portion cannot be detected. There's a problem.

  An object of the present invention is to provide an apparatus and method that can reduce the above-described problems and efficiently detect and determine a detection target image such as a person image portion.

In order to solve the above problems, the invention of claim 1
A detection target image determination device that determines whether or not a given grayscale image is a detection target image,
Among the pixels constituting the grayscale image, the pixel is provided for each of a plurality of sets of adjacent or adjacent two positions determined by learning, and between the two pixels of the set of pixels. A plurality of weak discriminating means for calculating an estimated value indicating whether or not the set of pixels is a contour portion of the detection target image based on the feature amount and obtaining a difference between luminance values as a feature amount;
A detection target image comprising: a determination unit that determines whether the given grayscale image is the detection target image based on the estimated values calculated by the plurality of weak determination units. A determination device is provided.

  According to the first aspect of the present invention, a difference in luminance value between two adjacent or adjacent pixels is used as a feature amount for determining a detection target image. This feature amount corresponds to object outline information in a broad sense. Therefore, according to the first aspect of the present invention, it is possible to preferentially detect a contour portion or an edge portion having a rough luminance change such as a human head or shoulder, and a given grayscale image is a person image (human type). It is possible to efficiently detect and determine whether the image is a detection target image such as an image.

The invention of claim 3
A detection target image determination device that detects and detects a detection target image from grayscale images,
Image reduction means for reducing the grayscale image and generating a plurality of images of different sizes;
Scanning means for scanning each of the plurality of different size reduced images from the image reducing means in window units of a fixed size;
Of the pixels constituting the grayscale image of the window unit obtained from the scanning means, provided for each of a plurality of sets of adjacent or adjacent two positions of pixels determined by learning in advance, A weakness for calculating a difference between luminance values of two pixels of the pixel set as a feature amount, and calculating an estimated value indicating whether the pixel set is a contour portion of the detection target image based on the feature amount A plurality of discrimination means;
A detection target image, comprising: a determination unit that determines whether or not the grayscale image in window units is the detection target image based on the estimated values calculated by the plurality of weak determination units. A determination device is provided.

  In the third aspect of the invention, when the detection target image is included in the grayscale image, the detection determination process for the detection target image similar to that of the first aspect is performed in units of a predetermined fixed size window. Do. In this case, the detection target image in the grayscale image is likely to be included in various sizes. In the invention of claim 3, the grayscale image is reduced to generate reduced images of various sizes. Then, each of the reduced images is scanned in the window unit of the fixed size, and it is determined whether or not the window unit image is a detection target image.

  Therefore, according to the invention of claim 3, it is possible to efficiently detect and determine detection target images included in various sizes in the grayscale image.

  According to the present invention, it is possible to preferentially detect a contour portion or an edge portion having a rough luminance change such as a human head or shoulder, and a given grayscale image is detected as a human image (humanoid image) or the like. It is possible to efficiently detect and determine whether the image is a target image. Further, it is possible to efficiently detect and determine a detection target image included in various sizes in the grayscale image.

  Hereinafter, embodiments of a detection target image determination apparatus and method according to the present invention will be described with reference to the drawings. The embodiment described below is a case where a detection target image is detected from an input image using ensemble learning (group learning). In the following description, the case of processing a still image will be described, but the same processing can be performed for a moving image. In the case of moving image processing, a plurality of detection target image determination devices described below may be provided, and the plurality of detection target image determination devices may be operated in parallel.

  FIG. 1 is a block diagram illustrating a configuration example of an object detection system including a detection target image determination device according to an embodiment. The input image providing device 1, the detection target image determination device 2 according to the embodiment, and a detection target image It consists of a result output unit 3.

  The input image supply device unit 1 outputs a grayscale image as the input image to the detection target image determination device unit 2. For example, the input image providing device unit 1 scans and reads an image recorded on a recording sheet and outputs the scanned image as grayscale image data, or captures image data input through an input terminal to obtain a grayscale image. It has a function to output as image data. When the image data of the input image is not grayscale image data, a function of converting the input image image data into grayscale image data is also provided.

  The detection target image determination device unit 2 includes a scaling unit 21, a scanning unit 22, a determination unit 23, and a processing control unit 20, and detects a detection target from the given image (input image). Information on the detection target image position indicating the image area and the size of the detection target image is output.

  The result output unit 3 receives the detection target image position and the size of the detection target image from the detection target image determination unit 2 and notifies the user of the detection target image position and the size of the detection target image.

  In this example, the scaling unit 21, the scanning unit 22, the determination unit 23, and the processing control unit 20 of the detection target image determination device unit 2 are function blocks, and the detection target image determination device 2 is configured by a computer. Yes. That is, in the case of this example, the scaling unit 21, the scanning unit 22, the determination unit 23, and the processing control unit 20 are software function means realized by executing a program stored in a memory included in the computer. It is configured.

  However, each of the scaling unit 21, the scanning unit 22, the determination unit 23, and the processing control unit 20 may be configured in hardware by a DSP (Digital Signal Processor) or the like. In that case, the process control unit 20 may be configured as a microcomputer that controls the entire detection target image determination device 2.

  Based on an instruction from the processing control unit 20, the scaling unit 21 reduces or enlarges the input image to all of a plurality of sizes (sizes) set in advance, and outputs the scaled image of the processing result to the scanning unit 22. To do.

  The size of the input image from the input image supply unit 1 may be a fixed size, but in this example, it is not necessarily constant. If the size of the input image is equal to or larger than the maximum size among the preset sizes, the scaling unit 21 performs only the reduction processing to obtain the scaled images of all the preset sizes. Can do. However, when the size of the input image is smaller than the maximum size among the preset sizes, an enlargement process of the image size is required. Here, as the image reduction process, an image reduction process using bilinear interpolation is performed.

  The scanning unit 22 cuts out the window image at each scanning position while sequentially scanning the scaled image of each size in units of windows of the size of the object to be detected, and supplies the cut-out window image to the determination unit 23. .

  The determination unit 23 determines whether each window image is a detection target image, and outputs the determination result to the processing control unit 20. The process control unit 20 outputs to the result output unit 3 information on the size of the scaled image and the window image position (window position) in the scaled image when it is determined to be the detection target image.

  Here, when a plurality of detection target images are detected from the input image, the processing control unit 20 of the detection target image determination device unit 2 outputs a plurality of region information to the result output device unit 3. Furthermore, when there is an overlapping area among the plurality of area information, it is possible to perform a process of selecting an area having the highest evaluation as a detection target image by a method described later.

  As described above, the scaling unit 21 generates scaling images having a plurality of sizes set in advance, and outputs the generated scaling images to the scanning unit 22. In this embodiment, the scaling unit 21 first generates all the plurality of scaled images and outputs them to the scanning unit 22 instead of outputting them to the scanning unit 22. The generated scaled image is output to the scanning unit 4, and after the scanning process and the determination process for the scaled image are finished, a scaled image of the next size is generated and passed to the scanning unit 22. Repeat until a scaled image is output.

  The detection target image determination process including this repeated control is executed under the control of the process control unit 20. An outline of the detection target image determination processing control operation in the processing control unit 20 will be described with reference to the flowchart of FIG.

  First, the process control unit 20 captures an input image from the input image supply device unit 1 into the detection target image determination device unit 2 (step S1). Then, in this example, the processing control unit 20 instructs the scaling unit 21 to generate a scaled image having the largest image size (step S2). Based on this instruction, the scaling unit 21 generates a scaled image of the instructed size, for example, the scaled image 10A of FIG. 3 and outputs it to the scanning unit 22. Here, for example, when the scaled image 10A having the largest image size is the size of the input image itself, the scaling unit 21 outputs the input image as it is to the scanning unit 22 as the scaled image 10A.

  Next, the processing control unit 20 instructs the scanning unit 22 to receive the scaled image 10A and scan the window and cut out the window image based on, for example, the generation completion notification of the scaled image 10A from the scaling unit 21. (Step S3). Based on the instruction from the processing control unit 20, the scanning unit 22 scans the received scaled image and cuts out the window image.

  In this case, the scanning unit 22 prepares a window WD having a fixed size as shown in FIG. 4, for example, 24 pixels × 24 pixels, and this window WD is arranged in the horizontal direction of the scaled image as shown in FIG. N pixels (N ≧ 1) are moved and scanned, and when the horizontal scanning is completed, the horizontal scanning is repeated by moving M pixels (M ≧ 1) in the vertical direction of the scaled image. Window scan. That is, in this example, the scanning unit 22 performs a so-called raster scan type scan.

  Then, the scanning unit 22 cuts out an image of an area surrounded by the window WD at each window scanning position on the schedule image, and outputs it to the determination unit 23 as a window image.

  Here, if the values of N and M are set to one pixel, very fine image scanning is possible, but the number of window images to be processed increases, resulting in a decrease in processing speed. If the values of N and M are too large, the image scanning becomes rough and the reliability of the determination result is lowered. Therefore, in this example, the values of N and M are set in consideration of the processing speed and the reliability of the determination result.

  The window WD does not need to have the same vertical and horizontal sizes as in this example, and does not have to be rectangular. For example, depending on the processing, the window WD may have a complicated shape, for example, a shape designated by a diamond shape or freehand.

  In addition, as a scanning method of the window WD, in the above-described example, raster scanning is performed at a constant pixel interval, but it is not necessary to have a constant pixel interval. For example, when the input image is a series of moving images, scanning may be performed with a narrow interval in the vicinity where the detection target image was previously detected, and with a wide interval in other locations. Further, the scanning method may be a method in which the vertical direction is scanned first and the scanning position is shifted in the horizontal direction. Further, instead of raster scanning, for example, scanning may be performed from the peripheral part in a spiral shape toward the center.

  Next, the processing control unit 20 receives the clipped window image at the determination unit 23 for each scan completion notification from the scanning unit 22, and determines whether the window image is a detection target image. (Step S4).

  When the determination unit 23 receives a determination result (information indicating whether the image is a detection target image) about the window image, the processing control unit 20 uses the determination result as the size of the scaling image and the window position at that time. After temporarily holding the information together with the information, it is determined whether or not all the scanning by the window WD in the scaled image is completed (step S5). At this time, in this example, the process control unit 20 recognizes the image size of the scaled image being processed, and thus the process control unit 20 also recognizes the number of scans in the scaled image by the window WD. Therefore, in step S, it is possible to determine whether or not all scanning by the window WD in the scaled image has been completed.

  If it is determined in step S5 that the full scan by the window WD in the scaled image has not been completed, the process control unit 20 returns to step S3 and moves the window WD to the next scan position with respect to the scan unit 22. And instructing to cut out the window image and output it to the determination unit 23. And the process control part 20 repeats the process of step S3, step S4, and step S5.

  As described above, the process control unit 20 controls the scanning unit 22 and the determination unit 23 so as to repeat Steps S3 to S5 for one scaling image for all window scanning positions.

  When it is determined in step S5 that all the scans by the window WD in the scaled image have been completed, the process control unit 20 determines whether or not the detection target image detection determination process for all the scaled images has been completed (step S5). S6) When it is determined that the process has not been completed, the process returns to step S2 to instruct the scaling unit 21 to generate a scaled image of the next image size, for example, the image 10B of FIG. . And the process control part 20 repeats the process after this step S2 mentioned above.

  Here, in the example of FIG. 3, the image size is sequentially reduced from the image 10A having the largest size in the order of image 10A → image 10B → image 10C → image 10D → image 10E. For example, the image 10A is multiplied by 0.875 to generate the image 10B, the image 10B is multiplied by 0.875 to generate the image 10C, the image 10C is multiplied by 0.875, and the image 10D is generated ... Thus, a scaled image of each image size is generated.

  In this example, the size of the window WD is constant, and the determination unit 23 determines whether or not this fixed-size window image is a detection target image. Since the scaling unit 21 generates a schedule image of each image size and converts the image size of the input image into various sizes, it is possible to determine a detection target image having an arbitrary size. .

  Note that the processing result similar to that described above can be obtained by changing the size of the window WD instead of changing the size of the input image. In this case, the determination unit 23 has various sizes. It is necessary to make a determination on the window image, which is not preferable.

  Further, as will be described later, in this example, the determination unit 23 uses a determination method that reflects the learning result of group learning. In this case, if the window image has various sizes, a plurality of the window images are used. There is also a problem that learning corresponding to each of the window images is performed, and a determination process using the learning results for each of the plurality of window images is required. In this respect, according to this embodiment, only one type of window image is required only by performing an image size reduction process (in some cases, an image size increase process), so that the overall configuration can be simplified.

  As described above, when the detection target image determination process for the scaled images of all image sizes is completed, the process proceeds from step S6 to step S7, and the process control unit 20 detects from the determination unit 23 temporarily held. With reference to the determination result for the target image (including information on the size of the scaling image and the window position), the size output of the scaling image determined to be the detection target image and the information on the window position are output to the result output unit 3 To do.

  Note that the determination result output in the result output device unit 3 is not limited to the information on the size of the scaled image and the position of the window, as in the example described above. You may make it display together with said determination result or independently.

  In the above description, the determination result is output from the detection target image determination device unit 2 to the result output device unit 3 after the determination process for one input image is completed. However, the detection target image is detected. The determination result may be output from the detection target image determination device unit 2 to the result output device unit 3 each time. Further, when there is an overlap in the determination region regarding the determination result of the detection target image at the time of output, a process for removing the overlap is added to the detection target image determination device unit 2 so that the overlap portion is eliminated and output. You can also.

[Configuration Example of Determination Unit 23]
The determination unit 23 in this embodiment uses ensemble learning (group learning) to determine whether the input image (window image) is a detection target image. The determination unit 23 constitutes an embodiment of the invention of claim 1.

  A learning machine obtained by group learning is composed of a number of weak hypotheses and a combiner that combines them. Here, boosting is an example of a combiner that integrates weak hypothesis outputs with fixed weights regardless of input. Boosting uses the previously generated weak hypothesis learning result to process the distribution that the learning sample follows to increase the weight of the learning sample (example) that is not good at mistakes, and based on this distribution a new Learn weak hypotheses.

  As a result, the weights of learning samples that have many incorrect answers and are difficult to discriminate as the detection target image are relatively increased. As a result, weak discriminators that correctly correct the learning samples that have large weights, that is, difficult to discriminate, are sequentially selected. Is done. That is, generation of weak hypotheses in learning is performed sequentially, and weak hypotheses generated later depend on weak hypotheses generated before that.

  When the detection target image is detected, the determination results of a number of weak hypotheses sequentially generated by learning are used as described above. For example, in the case of AdaBoost, all discrimination results of the weak hypothesis (hereinafter referred to as weak discriminator) generated by this learning (1 for a detection target image, -1 for a non-detection target image) Is supplied to the combiner, and the combiner weights and adds the reliability calculated at the time of learning for each corresponding weak discriminator to all the discrimination results, and outputs the result of the weighted majority vote. By evaluating the output value, it is determined whether or not the input image is a detection target image.

  The weak discriminator determines whether the image is a detection target image or a non-detection target image using some feature amount. As will be described later, whether or not the weak classifier output is a detection target image may be deterministically output, or the detection target image likelihood may be output probabilistically by a probability density or the like.

  Here, in this embodiment, it is determined whether or not the image is a detection target image by using a very simple feature amount called a difference in luminance value between two pixels (pixels) (hereinafter referred to as an inter-pixel difference feature amount). By using a group learning apparatus that uses a weak classifier for discrimination, the detection processing of the detection target image is speeded up. In addition, in this embodiment, instead of using the inter-pixel difference feature amount between all two pixels in the window image, for example, a contour portion having a rough luminance change in the head or the person's shoulder as the detection target image or Edge portions are preferentially detected to enable more efficient detection determination of the detection target image.

  That is, in this embodiment, the difference between all two pixels in the window image is not used, but as shown in FIG. 5, the two adjacent or close to each other as the contour portion or the edge portion of the detection target image. Only the difference between the luminance values I1 and I2 between the pixels P1 and P2 is used as a feature amount for determination (hereinafter referred to as a restricted inter-pixel difference feature amount).

  FIG. 6 is a block diagram illustrating a configuration example of the determination unit 23 of this embodiment. That is, the determination unit 23 assigns a weighting coefficient Wt (i) to each of the plurality of weak classifiers 20t (t = 1 to T) obtained by ensemble learning described later and the outputs of the plurality of weak classifiers 20t. = 1 to n), a plurality of coefficient multipliers 21t (t = 1 to T), and an adder 220 for receiving a weighted decision output from the coefficient multiplier 21t and obtaining a weighted majority decision, The determination output unit 230 determines whether the image is a detection target image according to the weighted majority value from the adder 220. The determination output from the determination output unit 230 is supplied to the processing control unit 20.

  In this embodiment, in order to obtain a plurality of weak classifiers 20t and a weighting coefficient Wt, the process control unit 20 is provided with a collective learning unit 24 configured as a processing function unit. In this example, the group learning machine unit 24 obtains the weak discriminator 20t and the weighting coefficient Wt by group learning. In this case, as group learning, any method can be applied as long as it can obtain the results of a plurality of discriminators by majority vote. For example, it is possible to apply group learning using boosting such as the above-mentioned Adaboost that weights data and makes a weighted majority decision.

  As described above, each weak classifier 20t uses the restricted inter-pixel difference feature quantity as the feature quantity for discrimination. Then, for the determination, the feature amount learned in advance by a learning sample composed of a plurality of grayscale images labeled as detection target images or non-detection target images learned in advance, and features of the input window image The estimated value for estimating whether or not the window image is the detection target image is output deterministically or probabilistically.

  Here, in Adaboost, the plurality of weak discriminators 20t sequentially calculate estimated values, and the value of the weighted coefficient Wt is sequentially updated accordingly. The plurality of weak classifiers 20t are sequentially generated by group learning using the learning samples described above according to the algorithm described later by the group learning machine unit 24. For example, the estimated values are generated in the order of generation. calculate. Further, the weighting factor Wt (reliability) of the weighted majority decision is learned in a learning process described later that generates the weak discriminator 20t.

  When the weak classifier 20t should perform binary output, such as Adaboost, for example, the weak classifier 20t divides the constrained pixel difference feature quantity by a threshold value to determine whether the image is a detection target image. To do. A plurality of threshold values may be used as the determination method based on the threshold value.

  Further, the weak discriminator 20t may probabilistically output a continuous value indicating the degree of whether or not the target object is based on the inter-constraint pixel difference feature quantity, for example, Real-AdaBoost. The feature amount (threshold value) for discrimination required by these weak discriminators 20t is also learned according to the above algorithm at the time of learning.

  Furthermore, in this embodiment, when performing weighted majority processing, the calculation result of all weak classifiers 20t is not waited, and even during the calculation, it is determined that the image is not a detection target image and the calculation is aborted. To do. The truncation threshold for this is also learned during learning. With this abort process, the amount of calculation in the detection process can be significantly reduced. As a result, it is possible to shift to the next window image discrimination process during the calculation without waiting for the calculation results of all the weak discriminators 20t.

  Thus, the determination unit 23 calculates a weighted majority value as an evaluation value for determining whether or not the window image is a detection target image, and the window image is determined based on the evaluation value (weighted majority value). It functions as a determination unit that determines whether the image is a detection target image. Moreover, the determination part 23 is made into embodiment of the detection target determination apparatus of Claim 1 as mentioned above.

  Furthermore, the determination unit 23 sequentially calculates and outputs estimated values by a plurality of weak classifiers 20t generated in advance by learning, and each time the estimated value is calculated, the estimated value is obtained by learning. The weighted majority vote value multiplied by the weighting coefficient Wt for each weak discriminator 20t is updated, and each time the weighted majority vote value (evaluation value) is updated, the value is estimated using the truncation threshold value. It is also possible to control whether or not to stop the calculation of the value.

  The determination unit 23 is generated by the group learning machine unit 24 using the learning sample and performing group learning according to a predetermined algorithm. Here, a group learning method in the group learning unit 24 will be described first, and then a method for determining a detection target image from an input image using the determination unit 23 obtained by learning through the group learning will be described. .

[Group learning unit 24]
The group learning machine unit 24 that performs group learning using the boosting algorithm combines a plurality of weak classifiers as described above, and learns so that a strong determination result can be obtained as a result.

  Each weak discriminator has a very simple configuration, and one weak discriminator has a low ability to discriminate whether it is a detection target image, for example, a face or a face. However, a high discrimination capability can be provided by combining, for example, several hundred to several thousand weak discriminators.

  The group learning machine unit 24 uses, for example, a sample image composed of a detection target image and a non-detection target image, for example, a face image and a non-face image, which are correctly labeled (labeled), which are referred to as thousands of learning samples. A weak discriminator is generated by selecting (learning) one hypothesis from a large number of learning models (hypothesis combinations) according to a predetermined learning algorithm, and a combination of the generated weak discriminators is determined.

  As described above, the weak discriminator itself has a low discrimination performance, but as a result, it is possible to obtain a discriminator having a high discrimination ability by the selection and combination of these, so that the group learning machine unit 23 Then, how to combine weak classifiers, that is, selection of weak classifiers and learning of weighting coefficients Wt for weighted majority processing of their output values are learned.

  Next, a learning method of the collective learning machine unit 24 for obtaining a determination unit 23 in which a large number of appropriate weak classifiers are combined according to a learning algorithm will be described. Here, prior to the description of the learning method of the group learning machine unit 24, among the learning data to be learned by group learning, learning data that is a feature amount in this embodiment, specifically, a weak classifier is configured. An explanation will be given of the restricted inter-pixel difference feature amount and the abort threshold for aborting detection in the determination step (detection step).

[Configuration of weak classifier]
In the determination unit 23 in this embodiment, the plurality of weak classifiers 20t constituting the same are adjacent or close to each other selected in all pixels included in the image input to the plurality of weak classifiers 20t. By using a very simple configuration for discriminating whether or not the image is a detection target image, for example, a face, based on the difference between the luminance values of the two pixels (constraint pixel difference feature quantity), the discrimination result of the weak discriminator 20t in the discrimination process The calculation is speeded up. The image input to the weak discriminator 20t is a learning sample in the learning step, and is a window image cut out from the scaling image in the discrimination step.

As described above, in this embodiment, as shown in FIG. 5, the difference between the luminance values of any two adjacent or adjacent pixels, in the example of FIG. 5, the luminance value I1 of the pixel P1 and the pixel P2 The difference from the luminance value I2 is defined as a constrained inter-pixel difference feature quantity as shown in the following (formula a). That is,
Restricted inter-pixel difference feature: d = I1-I2 (Expression a)
It is defined as

  Here, the ability of the weak discriminator determines which restricted inter-pixel difference feature quantity is used for detection target image detection. Therefore, it is necessary to select a set of pixel positions to be used for the weak classifier from a combination of two adjacent pixels (also referred to as a filter or a weak hypothesis) included in the clipped image by the window WD.

  For example, in AdaBoost, the weak classifier is requested to have a definitive output as to whether it is +1 (is a detection target image) or -1 (non-detection target image). Therefore, in Adaboost, weak discrimination is performed by dividing the difference feature quantity between restricted pixels into two (+1 or −1) using one or a plurality of thresholds at an arbitrary adjacent pixel position. Can be a container.

  Further, instead of such binary output, a continuous value (real value) indicating the probability distribution of the learning sample is output stochastically, for example, Real-AdaBoost or Gentle Boost. In the case of the boosting algorithm, the weak classifier outputs the probability (probability) that the input image is the target. The output of the weak classifier may be deterministic or stochastic as described above. First, these two types of weak classifiers will be described.

<Weak classifier with binary output>
The weak discriminator that performs deterministic binary output performs two-class discrimination as to whether or not the image is a detection target image according to the value of the inter-constraint pixel difference feature quantity. When the luminance values of two adjacent or adjacent pixels in the target image region (window image) are I1 and I2, and the threshold for determining whether or not the image is a detection target image based on the difference feature quantity between the restricted pixels is Th. ,
I1-I2> Th (Formula b)
Which class it belongs to can be determined depending on whether or not it satisfies.

Here, in order to configure a weak classifier, it is necessary to determine two adjacent or adjacent pixel positions and their threshold values. The determination method will be described later. The threshold determination in the above (formula b) is the simplest case. In addition, for threshold judgment,
Th1>I1-I2> Th2 (Formula c)
I1-I2> Th1 and Th2> I1-I2 (formula d)
It is also possible to use two threshold values represented by (Expression c) or (Expression d), respectively.

  7A, FIG. 7B, and FIG. 7C, the frequency is plotted on the vertical axis and the difference feature quantity between the restricted pixels is plotted on the horizontal axis. FIG. 4 is a schematic diagram for explaining the three determination methods shown in (Expression d) in accordance with characteristic cases of frequency distribution of detection target image data and non-detection target image data.

  Here, in FIG. 7A, FIG. 7B, and FIG. 7C, yi indicates the output of the weak discriminator, and each curve indicated by a broken line is yi = −1 (non-detection target image). In the case of (2), the frequency distribution of all learning samples is shown, and the curves shown by solid lines respectively show the frequency distribution of all learning samples where yi = 1 (in the case of the detection target image).

  When the detection target image is, for example, a face image, when the frequency for the same restricted pixel difference feature quantity is determined for a learning sample made up of a large number of face images and non-face images, FIGS. 7A and 7B. A histogram distribution shown in FIG. 7C is obtained.

  As shown in FIG. 7A, the histogram distribution is a distribution like a normal distribution curve having the same shape, for example, in the case of a non-detection target image indicated by a broken line and in the case of a detection target image indicated by a solid line. In the case where the peak position of the normal distribution curve is deviated, the restricted inter-pixel difference feature quantity at the boundary between the two normal distribution curves is set as a threshold Th, and whether or not the image is a detection target image according to the above (formula b). Can be determined.

  For example, in Adaboost, when the output of the weak discriminator is f (x), if it is determined that the input window image is the detection target image, the output f (x) = 1, and the input window image is not detected. If it is determined to be an image, the output f (x) = − 1. FIG. 7A shows an example in which when the difference feature quantity between restricted pixels is larger than the threshold Th, it is determined that the image is a detection target image, and the output of the weak classifier is f (x) = 1.

  Further, as shown in FIG. 7B or FIG. 7C, the peak positions of the respective normal distribution curves are the same between the non-detection target image indicated by the broken line and the detection target image indicated by the solid line. If the width of the histogram distribution is different in such a position, the value Th1 in the vicinity of the lower limit value and the value Th2 in the vicinity of the upper limit value of the constrained pixel difference feature quantity with the narrower distribution are used as threshold values. Whether the image is a detection target image can be determined by c) or (expression d).

  FIG. 7B shows an example in which the narrower distribution is determined as the detection target image, and FIG. 7C shows the detection target excluding the wider distribution from the narrower distribution. In this example, the image is determined to be an image and the output of the weak classifier is f (x) = 1.

  The weak discriminator is configured by determining a certain inter-constraint pixel difference feature amount and its threshold value, but the error rate is minimized by the judgment, that is, the constrained inter-pixel difference feature amount is high. It is necessary to select.

  For example, the threshold value is determined as two adjacent or adjacent pixel positions, the histogram shown in FIG. 7 is obtained for the correctly-acquired learning sample, the highest accuracy rate is the highest, and the inaccuracy rate (error rate) is the highest. It can be obtained by searching for a threshold value that decreases. In addition, for two adjacent or adjacent pixel positions, the one with the smallest error rate obtained together with the threshold value may be selected.

  However, in Adaboost, a weight (data weight) reflecting the difficulty of discrimination is attached to each learning sample, and an appropriate restricted inter-pixel difference feature (the brightness of two adjacent pixels at or near which position) Whether the value is a feature value) is learned so as to minimize a weighted error rate described later.

<Weak discriminator for continuous value output>
As described above, as weak classifiers that perform stochastic output, there are those in which weak classifiers output continuous values such as Real-AdaBoost and Gentle Boost, as described above. In this case, the discrimination problem is solved by a certain fixed value (threshold value), and the degree that the input image is the detection target image, not the binary output (f (x) = 1 or −1), For example, it is output as a probability density function.

  Such a stochastic output indicating the degree (probability) of the detection target image is obtained by using Pp (x) as a probability density function of the detection target image of the learning sample, when the inter-constraint pixel difference feature quantity d is input. When Pn (x) is a probability density function of the non-detection target image of the learning sample, a function f (x) shown in (Equation e) in FIG. A can be obtained.

  FIG. 8A is a diagram showing a characteristic case of the data frequency distribution, with the probability density on the vertical axis and the difference feature quantity between restricted pixels on the horizontal axis. FIG. 8B shows the data distribution shown in FIG. 8A with the vertical axis representing the value of the function f (x) of the above (formula e) and the horizontal axis representing the constrained pixel difference feature quantity. It is a figure which shows the characteristic of the function f (x) value in.

  In FIG. 8A, the probability density indicating that the broken line is a non-detection target image and the probability density indicating that the solid line is a detection target image are shown. When the function f (x) is obtained from the (formula e), a graph shown in FIG. 8B is obtained.

  In this case, the weak discriminator outputs a function f (x) corresponding to the restricted inter-pixel difference feature quantity d shown in (Formula a) obtained from the input window image in the discrimination step. This function f (x) indicates the degree of likelihood of the detection target image. For example, when the non-detection target image is -1 and the detection target image is 1, the function f (x) takes a continuous value from −1 to 1. It can be.

  For example, a table composed of the inter-restricted pixel difference feature quantity d and the function f (x) corresponding thereto is stored, and the function value f (x) is read out from the table according to the input and output. Therefore, although the storage amount is slightly larger than the case where the threshold value Th or Th1, Th2, which is a constant value, is stored, the discrimination performance is improved.

  These multiple estimation methods (discrimination methods) can be expected to improve discrimination performance when used in combination during ensemble learning. Moreover, if only any one of the determination methods is used, the execution speed performance can be extracted.

  Since the weak discriminator used in this embodiment has a very simple feature amount (difference feature amount between constrained pixels), the detection target image can be discriminated extremely quickly as described above. The point is a feature.

  For example, when a face is detected as the detection target image, an extremely good discrimination result can be obtained even if the threshold value judgment shown in the simplest (formula b) of the above-described discrimination methods is used for the difference feature amount between restricted pixels. The discrimination method by which the weak discriminator functions effectively depends on the target problem, and the threshold setting method and the like may be appropriately selected.

  In addition, depending on the problem, the difference between the luminance values of two or more adjacent or adjacent pixels is used as a feature amount or a combination of the luminance values of two or more adjacent or adjacent pixels. A feature amount may be used.

<Cutoff threshold>
Next, the abort threshold will be described. In a group learning machine using boosting, it is usually determined whether the window image is a detection target image by the weighted majority of the outputs of all weak classifiers constituting the determination unit 23 as described above. The weighted majority vote is calculated by sequentially adding the discrimination results (estimated values) of the weak classifiers. For example, when the weight (reliability) of the majority decision corresponding to each weak discriminator 20t is Wt as described above and the output of each weak discriminator is ft (x), the weighted majority decision value F (x in Adaboost ) Can be obtained by (Formula f) in FIG.

  In FIG. 9, the horizontal axis represents the number of weak classifiers constituting the determination unit 23, and the vertical axis represents the weighted majority value F (x) shown in (Formula f). It is a graph which shows the change of the value F (x) of the weighted majority according to whether it is a detection target image.

  In FIG. 9, data D1 to D4 indicated by broken lines sequentially calculate an estimated value f (x) calculated by a weak discriminator using an image (learning sample) labeled as a detection target image as an input, and weighted. The majority value F (x) is obtained sequentially. As shown in the data D1 to D4, when the detection target image is an input image, the weighted majority value F (x) becomes positive by the determination of a certain number of weak classifiers.

  Here, in this embodiment, a method different from a normal boosting algorithm is introduced. That is, in the process of sequentially adding the discrimination results of the weak classifiers, even before obtaining the results of all the weak classifiers, the window image that can clearly be determined not to be a detection target image is subjected to the discrimination. It will be canceled. At this time, a threshold value for determining whether to stop the discrimination is learned in the learning step. Hereinafter, the threshold used for determining whether or not to stop the determination is referred to as an abort threshold.

  If the determination unit 23 can reliably estimate that all the window images are non-detection target images without using the output result of all the weak discriminators, the threshold value f ( The calculation of x) can be stopped in the middle, and the amount of calculation can be significantly reduced as compared with the case where a weighted majority decision using all weak classifiers is performed.

  As the censoring threshold, a minimum value that can be taken by the weighted majority value of the discrimination result of the learning sample indicating the detection detection target image among the labeled learning samples can be used.

  In the discrimination step, the result of the weak discriminator of the window image is sequentially weighted and output, i.e., the weighted majority value is sequentially updated. Every time update is performed, that is, each time a weak classifier outputs a discrimination result, a comparison is made. If the updated weighted majority value falls below the abort threshold, the window image is not a detection target image, and calculation is performed. As a result, it is possible to eliminate unnecessary computations and further speed up the discrimination process.

That is, the abort threshold value _{R K} output _f K (x) of the K-th weak discriminator is a learning sample xi; Of (= x1~xN i = 1~N), a detection target image learning samples xj (= x1 to xJ; j = 1 to J) is used as a minimum value of the weighted majority value, and is defined as (formula g) in FIG.

As shown in (Formula g), if the minimum value of the weighted majority value of a detection target image learning sample x1~xJ exceeds 0, the termination threshold R _K 0 is set. It should be noted that not exceeding 0 is the case of Adaboost that performs determination using 0 as a threshold, and this may differ depending on the group learning method.

  In the case of Adaboost, the abort threshold is set to the minimum possible value among all data D1 to D4 when a detection target image is input as an input image, as shown by a thick line in FIG. 9, and all data D1 When the minimum value of .about.D4 exceeds 0, the abort threshold is set to 0.

  In this embodiment, each time a weak discriminator is generated, the censoring threshold value Rt is learned, so that an estimated value is sequentially generated by a plurality of weak discriminators as in data D5, for example, in a later-described discrimination step. The weighted majority value that is output is sequentially updated, and when this value falls below the abort threshold value, the process of determining by the weak classifier at the subsequent stage is terminated.

  That is, by learning this truncation threshold value Rt, it is possible to determine whether or not to calculate the next weak discriminator every time the estimated value of the weak discriminator is calculated. In this case, it is possible to determine that the image is a non-detection target image without waiting for the determination results of all the weak classifiers, and it is possible to speed up the detection process by aborting the calculation halfway.

[How to learn]
Next, a learning method in the group learning machine unit 24 will be described. An image (training data) that is a learning sample that has been manually labeled (corrected) in advance as a premise of a general pattern recognition problem for two-class classification, such as a problem of determining whether the given data is a face, for example. Prepare. The learning sample consists of an image group (detection target image group) obtained by cutting out a region of the detection target image desired to be detected, and a random image group (non-detection target) obtained by cutting out, for example, a landscape image that has no relation to the detection target image desired to be detected Image group).

A learning algorithm is applied based on these learning samples to generate learning data used for discrimination. In this embodiment, the learning data used at the time of discrimination is the following four learning data including the learning data described above. That is,
(A) A set of two adjacent pixel positions (T)
(B) Weak classifier threshold (T)
(C) Weight of majority vote (reliability of weak classifier) (T)
(D) Abort threshold (T)
It is.

  Next, an algorithm for learning the four types of learning data shown in the above (A) to (D) from a large number of learning samples as described above will be described.

  FIG. 10 is a flowchart showing a learning method in the group learning machine unit 24. Here, learning according to an algorithm (Adaboost (AdaBoost)) that uses a constant value as a threshold value in weak discrimination will be described as a learning algorithm. However, a continuous value indicating the probability (probability) of a correct answer as a threshold value will be described. The learning algorithm is not limited to AdaBoost as long as the group learning is performed in order to combine a plurality of weak discriminators such as Real-AdaBoost using values.

(Preparation Process) Labeling of Learning Sample Prior to the processing flow of FIG. 10, as described above, a learning sample (xi, yi) that is previously labeled as a detection target image or a non-detection target image is prepared.

here,
Learning sample (xi, yi): (x1, y1), ..., (xN, yN)
xiεX, yiε {-1,1}
X: Learning sample data Y: Learning sample label (correct answer)
N: Indicates the number of learning samples.

  That is, xi represents a feature vector composed of all the luminance values of the learning sample image. In addition, yi = −1 indicates that the learning sample is labeled as a non-detection target image, and yi = 1 indicates that the learning sample is labeled as a detection target image.

(Step S11) Initialization of data weight In boosting, the weight (data weight) of each learning sample is made different to relatively increase the data weight for the learning sample that is difficult to discriminate. The discriminant result is used to calculate the error rate (error) for evaluating the weak discriminator, but by multiplying the discriminant result by the data weight, the evaluation of the weak discriminator that erroneously discriminates the more difficult learning sample is actually performed. Will fall below the discrimination rate. The data weight is sequentially updated by a method described later. First, the data weight of this learning sample is initialized. The data weights of the learning samples are initialized by making the weights of all the learning samples constant and are defined as shown in (Equation 1) in FIG.

Here, the data weight D _{1, i} of the learning sample in (Equation 1) indicates the data weight of the learning sample xi (= x 1 to xN) of the first iteration t = 1. N is the number of learning samples.

(Steps S12 to S17) Repetition Processing Next, the determination unit 23 is generated by repeating the processing of steps S12 to S17 shown below. Here, it is assumed that the number of repetition processes is t = 1, 2,. Each time iterative processing is performed, one weak discriminator, that is, a set of adjacent or adjacent pixels and a difference feature quantity between the constrained pixels of the set are learned, and accordingly, the number of iterations (T times), A weak classifier is generated, and a determination unit 23 including T weak classifiers is generated.

  In general, hundreds to thousands of iterative processes generate hundreds to thousands of weak classifiers. The number of iterations (number of weak classifiers) depends on the required discrimination performance and discrimination. What is necessary is just to set suitably according to the problem (detection object image) to do.

(Step S12) Weak Classifier Learning In step S12, weak classifier learning (generation) is performed. This learning method will be described later. In this embodiment, one weak discriminator is generated according to a method described later for each iteration.

(Step S13) Calculation of the weighted error ratio _{e t} Then, the weighted error ratio _{e t} of the weak classifier generated in step S12, is calculated by (Equation 2) in FIG. B.

As shown in (Equation 2), the weighted error ratio e _t, among the learning samples, only the data weights of learning samples is what is wrong discrimination result of the weak discriminator (ft (xi) ≠ yi) As described above, if the data sample Dt _{, i} is large (difficult to distinguish) and the learning sample is mistakenly determined, the weighted error rate _et is calculated to be large. The weighted error rate _et is less than 0.5, and the reason will be described later.

Calculation of (step S14) of weighted majority decision weight (weak discriminator reliability) Next, based on the weighted error ratio e _t shown in the above equation (2), the weight Wt of the weighted majority decision, in Figure B Calculated by (Equation 3). The weight Wt of the weighted majority vote indicates the reliability of the weak discriminator generated at the number of repetitions t. Hereinafter, the weight Wt of the weighted majority decision is referred to as reliability Wt.

As shown in the above equation (3), as those weighted error ratio e _t is small, the reliability Wt of the weak classifier increases.

(Step S15) Data Weight Update of Learning Sample Next, using the reliability Wt obtained in (Equation 3), the data weight D _{t, i} of the learning sample is updated according to (Equation 4) in FIG. To do. The data weights D _{t, i} are normally normalized to be 1 when all are added together, and (Equation 5) in FIG. B is for normalizing the data weights D _{t, i} .

(Step S16) Calculation of abort threshold value R _t Next, as described above, at determination step calculates the abort threshold value R _t for aborting the discrimination at the stage of the weak classifiers 20t. Abort threshold value R _t in accordance with equation (g) of Figure A described above, the smallest value of the values or 0 of weighted majority of the detection target image in which the learning sample (Pojidibu learning samples) X1～xJ is selected . Note that, as described above, the minimum value or 0 is set as the cutoff threshold value in the case of Adaboost that performs determination using 0 as the threshold value. In any case, the truncation threshold value _Rt is set to a maximum value that allows at least all positive learning samples to pass.

(Step S17) Repetition Processing In step S17, it is determined whether or not a predetermined number of times (= T) of boosting has been performed. If it is determined that the boosting has not been performed, the process returns from step S17 to step S12 to Steps S12 to S17 are repeated. If it is determined that the predetermined number of times of learning has been completed, the learning process in FIG. 10 is ended. In this embodiment, it is assumed that the learning is completed when a number of weak discriminators that can sufficiently discriminate a detection target image as a detection target are learned from a given image such as a learning sample.

[Generate weak classifiers]
Next, the weak classifier learning method (generation method) in step S12 described above will be described. The weak discriminator is generated differently when the weak discriminator outputs a binary value and when a continuous value is output as the function f (x) shown in (Equation e) of FIG. Also in the case of binary output, a case where the determination is made with one threshold Th as shown in the above (Formula b) and two thresholds as shown in the above (Formula c) and (Formula d). The process is slightly different depending on whether Th1 or Th2 is used for discrimination.

  Here, a learning method (generation method) of a weak classifier that outputs a binary value with one threshold Th will be described. FIG. 11 is a flowchart for explaining a learning method (generation method) of a weak classifier that outputs a binary value with one threshold Th, and corresponds to a parameter determination procedure of the weak classifier.

(Step S21) Selection of Pixels Here, arbitrary two pixels that are adjacent or close to each other from all the pixels in the learning sample are selected. For example, when a learning sample of 24 × 24 pixels is used, one of two adjacent or adjacent pixel sets is randomly selected instead of all the two pixel sets.

  In this embodiment, since the difference feature amount, that is, the edge strength is used as a feature for discrimination, such a pair of adjacent or adjacent two pixels is used. As shown in FIG. 5 described above, when the two selected pixels are P1 and P2, and the luminance values are I1 and I2, respectively, the two pixels P1 and P2 for obtaining the difference feature amount are the window WD. For example, the selection is made in accordance with (Equation 6) in FIG. In (Expression 6), (x1, y1) represents the position of the pixel P1, (x2, y2) represents the position of the pixel P2, and θ represents the threshold value.

  By this pixel selection, for example, so-called four neighbors adjacent to one pixel in the vertical and horizontal directions, or eight neighbors including eight neighboring pixels including pixels adjacent in the oblique direction in addition to the four neighboring pixels, Two adjacent pixels are selected.

  In this step S21, a group (M) of two adjacent pixel groups is selected from all the two pixel groups of the learning sample, and one random group is selected from the group of the pixel groups. Two adjacent or adjacent pixels are selected. Here, the reason for selecting at random is to increase the effect of learning.

  Here, for example, FIG. 12 shows a combination example of two pixels when used for detecting a human type of a human head and shoulders. This example is a case where the window size is 24 pixels × 24 pixels (represented by 12 pixels × 12 pixels for simplification in the figure), and those indicated by black circles are two pixels used by the selected weak classifier. is there. FIG. 12 shows a combination of adjacent or adjacent two pixels used in the top eight weak classifiers. In actual ensemble learning, more weak classifiers are used.

  For reference, an example of the edge shape of the head and shoulders of a person who is a detection target is shown in FIG. FIG. 13 shows an average of edge detection processing performed on a plurality of head images. Since edge detection processing is not performed in advance in the actual learning process, it is only referred to here, but if the set of adjacent or adjacent two pixels selected as described above is accumulated, this average image and It becomes the same shape.

(Step S22) Frequency Distribution Creation Next, the inter-restricted pixel difference feature quantity d is set as the difference (I1-I2) between the luminance values of two adjacent or adjacent pixels selected in step S21 for all learning samples. And a histogram (frequency distribution) as shown in FIG.

(Step S23) Calculation of threshold Thmin Next, the frequency distribution obtained in step S22, the weighted error ratio _{e t} shown in equation (2) of FIG. 17, calculate a threshold Thmin that its minimum emin .

(Step S24) Calculation of threshold Thmax Then, from the frequency distribution obtained in step S22, the weighted error ratio _{e t} shown in equation (2) of FIG. 17, obtains a threshold Thmax that its maximum value emax The threshold value is inverted by the method shown in (Equation 7) of FIG. That is, the weak discriminator outputs two values of correct answer and incorrect answer depending on whether or not it is larger than one threshold value Th. Therefore, when the weighted error rate _et is less than 0.5, By reversing the threshold, the weighted error rate can be made 0.5 or more.

(Step S25) Parameter Determination Next, each parameter constituting the weak classifier, that is, the position of two adjacent or adjacent pixels P1 and P2, and the threshold Th thereof are determined from the above-described emin and emax ′. . That is,
When emin <emax ': P1, P2, Thmin,
When emin> emax ′: P1 ′ (= P2), P2 ′ (= P1), Thmin,
And

(Step S26) Repetition Processing Then, in step S26, it is determined whether or not the processing of steps S21 to 25 has been repeated for all the number M of two sets of adjacent or adjacent pixels for the learning sample, and all pixels When it is determined that steps S21 to 25 have not been repeated yet, the process returns to step S21, and the processes of steps S21 to S26 are repeated. As described above, m (= 1, 2,..., M) iterations are performed to generate one weak classifier.

(Step S27) Selection of weak discriminator When it is determined in step S26 that steps S21 to 25 have been repeated for the number M of sets of all pixels, the process proceeds to step S27, and the weak generated by M iterations. Among the classifiers, the parameter candidate of the weak classifier having the smallest error rate _et is adopted as the final weak classifier parameter. Then, the process of FIG. 11 is terminated, and the process proceeds to step S13 shown in FIG.

Here, in step S27, the weak discriminator is discriminated based on (Equation 8) in FIG. In (Equation 8), k ₁ and k ₂ are the positions of the pixels I 1 and I 2, x is the luminance value of the window image, and θ _t is a threshold value.

For convenience of explanation, at the time has been described as calculating the weighted error ratio e _t in step S13 shown in FIG. 10, in step S27, and selects the error rate e _t is the smallest weak classifier, error ratio _{e t} shown in step S13 automatically obtained.

In this embodiment, the data weights D _{t, i} obtained in step S15 in the previous repetitive process are used to learn the feature quantities of a plurality of weak classifiers, and these weak classifiers (weak classifiers). by weighted error ratio e _t shown in equation (2) of FIG. 17 from the vessel candidates) selects the smallest has described the case of generating one weak discriminator, the above step In S12, for example, a weak discriminator may be generated by selecting an arbitrary pixel position from a plurality of pixel positions prepared or learned in advance.

  Moreover, you may produce | generate a weak discriminator using the learning sample different from the learning sample used for the above-mentioned repetition process from step S12 to step S17. Further, a sample other than the learning sample is prepared, such as an evaluation such as a cross-validation method or a jack-knife method, and the generated weak discriminator and the determination unit 23 An evaluation may be performed.

  Here, the cross-validation method means that the learning sample is equally divided into L pieces, learning is performed using one of them, and the learning result is evaluated using the one by L times. This is a method of repeatedly evaluating learning results.

  The above is a case where the weak discriminator has one threshold Th, but as shown in the above-described (Equation c) or (Equation d), the weak discriminator has two thresholds Th1 and Th2. Is slightly different from step S23 to step S25 shown in FIG.

  That is, as shown in the above (formula b), in the case of one threshold Th, the error rate can be inverted when the weighted error rate is larger than 0.5 by inverting. As shown in (Equation c), when the difference feature amount between the restricted pixels is larger than the threshold Th2 and smaller than the threshold Th1 is a correct discrimination result, when this is inverted, (Equation d) is obtained. As shown, a correct determination result is obtained when the threshold value is smaller than the threshold value Th2 or larger than the threshold value Th1. That is, the inversion of (expression c) becomes (expression d), and the inversion of (expression d) becomes (expression c).

When the weak discriminator has two threshold values Th1 and Th2 and outputs a discrimination result, in step S22 shown in FIG. 11, the frequency distribution in the constrained pixel difference feature quantity is obtained, and the weighted error rate e _t. Threshold values Th1 and Th2 that minimize the threshold are obtained. Then, after determining that it has been repeated a predetermined number of times M in step S26, in step S27, the weak classifier having the smallest weighted error rate _et is adopted among the generated weak classifiers.

  Further, as shown in the above (formula e), in the case of a weak discriminator that outputs a continuous value instead of a binary output, first, as in step S21 in FIG. Then, two pixels are selected at random. Then, the frequency distribution in all the learning samples is obtained in the same manner as in step S22. Then, based on the obtained frequency distribution, the function f (x) shown in (Expression e) is obtained.

  Then, a series of processes of calculating an error rate according to a predetermined learning algorithm that outputs the degree of detection target image (degree of correct answer) as an output of the weak classifier is repeated a predetermined number of times M, and the error rate is the highest. A weak classifier is generated by selecting a small parameter (high correct answer rate).

  As described above, when the maximum number of iterations is repeated, that is, the maximum number of weak discriminators that can be generated is generated, and the one with the smallest error rate is adopted as the weak discriminator, the performance is weak. Although the discriminator can be generated, it is possible to repeat the processing less than the maximum number of times, for example, several hundred times, and adopt the one with the smallest error rate.

  In the above description, the window WD has a rectangular shape as shown in FIG. 14A of, for example, 24 pixels × 24 pixels. However, as described above, the shape and size of the window WD are not limited thereto. For example, the window shape may be combined with the contour shape feature of the detection target image. For example, in the case where the above-described human shape of the head and shoulders of the person is to be detected, as shown in FIG. 14B, an area filled in black is used as a mask area, and the area is not used and effective. Two pixels adjacent or close to each other may be extracted and used for learning.

[Detection method for detection target image]
Next, an embodiment of a detection target image detection method in the detection target image detection apparatus shown in FIG. 1 will be described. FIG. 15 is a flowchart showing an embodiment of the detection target image detection method, and is a more detailed process explanatory diagram than the flowchart of the process shown in FIG.

  At the time of detection (determination step), the determination unit 23 using the weak classifier groups 201 to 20T generated as described above is used, and the input image from the input image supply device unit is detected according to a predetermined algorithm. A detection target image is detected. Note that the example of FIG. 15 is a case where the input image from the input image supply unit is a scaling image of the maximum size.

(Step S31) Scaling Image Generation First, in the detection target image determination device unit 2 shown in FIG. 1, the scaling unit 21 reduces the grayscale image (input image) given from the input image supply device unit 1 at a constant rate. Scale. In this case, the input image supply device unit 1 may output the input grayscale image as an input image to the detection target image determination device unit 2 as it is. After converting the input image into a grayscale image, the input image may be output to the detection target image determination device unit 2.

  The scaling unit 21 first outputs an image supplied from the input image supply device unit 1 without performing scale conversion, and outputs a scaled image reduced and scaled after the next timing. Here, the timing for generating the next scaled image is the time when the detection determination of the detection target image is completed for all the areas of the previously output scaled image.

  In this example, when the scaled image becomes smaller than the window image, the detection determination process for the detection target image for one input image is completed, and the next input image (in the case of a moving image, The process proceeds to the next frame image). However, FIG. 15 shows detection detection processing for a detection target image for one input image, and the processing in FIG. 15 is performed for each input image.

(Step S32)
The scanning unit 22 receives the information of the scaling image from the scaling unit 22, scans the position of the window WD vertically and horizontally on the scaled image, and outputs the window image at each scanning position to the determination unit 23.

(Steps S33 and 34) Calculation of Evaluation Value s The determination unit 23 determines whether or not the window image output from the scanning unit 22 is a detection target image. The determination unit 23 sequentially adds the weighted addition values (weighted majority decision) of the estimated values ft (x) of the plurality of weak classifiers 20t (= 201 to 20T) described above to the window image. Value update value) is calculated as the evaluation value s. Then, based on the evaluation value s, it is determined whether or not the window image is a detection target image and whether or not the determination is terminated.

  Actually, the determination unit 23 outputs the evaluation value s to the processing control unit 20, and the processing control unit 20 determines whether the window image is a detection target image and whether to abort the determination. Like that.

  First, when a window image is input, the determination unit 23 initializes the evaluation value s = 0. The weak discriminator 201 at the first stage of the determination unit 23 calculates the difference feature amount d between restricted pixels (step S33). Then, the estimated value output by the weak classifier 201 is reflected in the evaluation value s (step S34).

  Here, the weak discriminator that outputs a binary estimated value and the function f (x) shown in (Expression e) are output as estimated values by the above-described (Expression b), (Expression c), and (Expression d). The method of reflecting the estimated value on the evaluation value s differs from that of the weak classifier.

  First, when (Formula b) is used for the weak classifier 20t and a binary value is output as an estimated value, the evaluation value s is as shown in (Formula 9) of FIG.

  When the (expression c) is used for the weak discriminator 20t and a binary value is output as an estimated value, the evaluation value s is as shown in (expression 10) in FIG.

  When the (expression d) is used for the weak discriminator 20t and a binary value is output as an estimated value, the evaluation value s is as shown in (expression 11) in FIG.

  Further, when the (formula e) is used for the weak discriminator 20t and the function f is output as an estimated value, the evaluation value s is as shown in (formula 12) of FIG.

(Steps S35, S36, S37, S38) Detection Determination and Abort Determination Then, the determination unit 23 (or the process control unit 20) is obtained (updated) by any one of the four methods described above, for example. value s is determined whether greater than the abort threshold value _{R t} (step S35). If the evaluation value s in this step S35 is judged to be larger than the abort threshold value R _t, the predetermined number of times (= T times) repeated whether determined (step S36), when it is judged that no repeated T times, Returning to step S33, the processing from step S33 to step S36 is repeated.

  If it is determined in step S36 that the predetermined number of times (= T) has been repeated, the determination unit 23 determines whether the obtained evaluation value s is greater than 0 or not, and the window image is the detection target image. If it is determined whether the image is a detection target image, the process control unit 20 stores the current scaled image size and window position (step S37). Then, after step S37, the process proceeds to step S38.

Further, in step S35, even when the evaluation value s is judged to be smaller than the abort threshold value _{R t,} the process proceeds to step S38. In step S38, the process control unit 20 determines whether or not there is a next search window. If it is determined that there is a next search window, the process returns to step S32 and repeats the processes from step S32.

  If it is determined in step S38 that there is no next search window, the process control unit 20 proceeds to step S39, determines whether there is a next scaling image, and determines that there is a next scaling image. In that case, the process returns to step S31 and the process from step S31 is repeated. As described above, the scaling process in step S21 ends when the scaled image becomes smaller than the window image.

(Steps S40 to S42) Deletion of Overlap Area When it is determined in step S39 that there is no next scaled image, it is determined whether or not there is an overlap area for the window image area that has been detected as a detection target image. If there is, the deletion process of the overlap area is executed.

  That is, if it is determined in step S39 that the processing of all the scaled images has been completed for one input image, there is an overlap in the detection target image area (window image area) that has been detected and determined. (Step S40).

  If it is determined in step S40 that there are overlapping areas, two overlapping window areas are extracted (step S41). Of these two window areas, the area with the smaller evaluation value s is the reliability. Is determined to be low, and the region having the large evaluation value s is selected as the region of the true detection target image (step S42).

  Then, the process returns from step S42 to step S40, and the processes from step S40 to step S42 are repeated until there is no overlapping area. When it is determined in step S40 that there are no overlapping areas, the processing routine of FIG. 15 is terminated. Thereby, even if a plurality of overlapping areas are detected, only one area having the highest evaluation value s is selected.

  As described above, according to the detection target image detection method in this embodiment, the detection target image is detected using the determination unit that learns the weak classifier that performs weak determination based on the difference feature quantity between restricted pixels by collective learning. Therefore, the process of calculating the feature amount of the detection target image in step S33 is completed by reading the luminance values of the corresponding two pixels in the window image and calculating the difference between them, and the detection target image is detected extremely quickly. Since it can be processed, it is possible to detect a human type in real time.

  Each time the evaluation value s is sequentially updated by adding a value obtained by multiplying the discrimination result (estimated value) discriminated from the inter-constraint pixel difference feature quantity and the reliability of the weak discriminator used for discrimination, the truncation threshold Rt And determine whether or not to continue the calculation of the estimated value of the weak classifier. Then, when the evaluation value s falls below the truncation threshold value Rt, the weak discriminator's computation is aborted, and the processing of the next window image is performed, thereby drastically reducing wasteful computation and further increasing the detection target image. Can be detected.

  In other words, when a window image is cut out by scanning the entire input image and the scaled image, or the entire scaled image, the probability of being a detection target image is small, and most of the window images are non-detection target images. It is. By discriminating the window image that is the non-detection target image in the middle, the discrimination process can be made extremely efficient.

  On the other hand, when there are many detection target images to be detected, a threshold value that causes the calculation of the window image that is clearly detected as the detection target image to be interrupted in the same way as described above. May also be provided. Furthermore, even if a fixed-size window is used by scaling the input image to various sizes by the scaling unit, it is equivalent to setting a search window of an arbitrary size. Can be detected.

  In addition, for example, when a face is a detection target image, when eyes, nose, mouth, and the like are detected and discriminated in detail, an inter-pixel difference feature amount is obtained for all combinations of two pixels in the window WD. This can be detected and discriminated, but the amount of computation increases as much as it is necessary to find all the difference features between two pixels.

  On the other hand, in the case of this embodiment, not all of the difference feature amounts between two pixels are obtained, but the above-mentioned restricted feature difference amount between pixels using only two adjacent or adjacent pixels is used. In particular, for example, it is suitable and effective when performing human-type detection in which detailed portions such as the eyes, nose and mouth of the face need not be detected. This is because the contour shape on the head and shoulders of a person is generally a characteristic Ω type, and looks like an Ω type regardless of the viewing direction, so it is more efficient than using all the features between two pixels. And effective detection discrimination becomes possible.

  That is, according to this embodiment, it is possible to detect a detection target object based on the similarity of outlines, rather than the comparison of detailed patterns such as image matching that is generally performed. Can be detected.

  As described above, the contour shape of the person's head and shoulder is generally a characteristic Ω type. However, it is difficult to detect a person by expressing the Ω shape using a specific shape template due to the difference in characteristics including the individual body shape and hairstyle. On the other hand, the method of this embodiment is advantageous for detecting an Ω type like a person because the constrained inter-pixel difference feature quantity representing the contour shape is used as the discrimination feature quantity.

  For example, when the Ω type is used as a feature amount, it is possible to detect a facial expression, wearing glasses, wearing a mask, and detecting without depending on the face direction. Furthermore, detection from a direction impossible for face detection, such as detection from the back of the head, is possible.

  Furthermore, since the ensemble learning method type detection / determination device using the inter-constraint pixel difference feature quantity of this embodiment is a determination device that terminates the weak discriminator, it is more personal than a discrimination method consisting of a single discriminator. Robust detection can be performed for differences in individual contour shapes. In addition, since a template is created by learning an actual human form, detection can be performed more robustly than other means using a specific shape template that is arbitrarily defined.

It is a functional block diagram which shows the processing function in embodiment of the detection target image detection apparatus by this invention. 3 is a flowchart for explaining a detection determination processing operation of a detection target image in the embodiment of FIG. 1. It is a schematic diagram which shows the image scale-converted by the scaling part 21 in embodiment of FIG. It is a figure which shows a mode that the scanning part 22 in embodiment of FIG. 1 scans a search window. It is a schematic diagram which shows the image for demonstrating the difference feature quantity between restrictions pixels. It is a figure which shows the structural example of the determination part 23 in embodiment of FIG. It is a figure for demonstrating the relationship between the detection determination method of the detection target image in embodiment of FIG. 1, and a threshold value. It is a figure for demonstrating the relationship between the other detection determination method of the detection target image in embodiment of FIG. 1, and a threshold value. Taking the number of weak classifiers on the horizontal axis and the weighted majority value F (x) on the vertical axis, the weighted majority value F (x) depending on whether the input image is a detection target image or not. It is a figure which shows the change of. It is a flowchart which shows an example of the learning method of the group learning machine for obtaining the some weak discriminator which comprises the determination part 23 in embodiment of FIG. It is a flowchart which shows an example of the learning method (generation method) of the weak discriminator which outputs a binary value with one threshold value Th. It is a figure which shows the example of the group of 2 pixels used with the weak discriminator produced | generated by the learning method of FIG. It is a figure for demonstrating an example of the learning sample used by embodiment of this invention. It is a figure for demonstrating the other example of the window used by the scanning part 22 in embodiment of this invention. It is a flowchart which shows an example of the detection target image detection method in embodiment of this invention. It is a figure which shows the formula used for description of embodiment of this invention. It is a figure which shows the formula used for description of embodiment of this invention. It is a figure which shows the formula used for description of embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Input image provision apparatus part, 2 ... Detection target image detection apparatus part, 3 ... Result output apparatus part, 20 ... Processing control part, 21 ... Scaling part, 22 ... Scanning part, 23 ... Determination part, 24 ... Collective learning machine Part, 201-20T ... weak discriminator, 210 ... adder

Claims (10)

A detection target image determination device that determines whether or not a given grayscale image is a detection target image,
Among the pixels constituting the grayscale image, the pixel is provided for each of a plurality of sets of adjacent or adjacent two positions determined by learning, and between the two pixels of the set of pixels. A plurality of weak discriminating means for calculating an estimated value indicating whether or not the set of pixels is a contour portion of the detection target image based on the feature amount and obtaining a difference between luminance values as a feature amount;
A detection target image comprising: a determination unit that determines whether the given grayscale image is the detection target image based on the estimated values calculated by the plurality of weak determination units. Judgment device. The detection target image determination device according to claim 1,
Weighting assigning means for multiplying the estimated value from each of the plurality of weak discriminating means by a weighting coefficient obtained by the learning;
The determination unit determines whether or not the given grayscale image is the detection target image based on the estimated value to which the weighting is given from the weighting provision unit. Image determination device. A detection target image determination device that detects and detects a detection target image from grayscale images,
Image reduction means for reducing the grayscale image and generating a plurality of images of different sizes;
Scanning means for scanning each of the plurality of different size reduced images from the image reducing means in window units of a fixed size;
Of the pixels constituting the grayscale image of the window unit obtained from the scanning means, provided for each of a plurality of sets of adjacent or adjacent two positions of pixels determined by learning in advance, A weakness for calculating a difference between luminance values of two pixels of the pixel set as a feature amount, and calculating an estimated value indicating whether the pixel set is a contour portion of the detection target image based on the feature amount A plurality of discrimination means;
A detection target image, comprising: a determination unit that determines whether or not the grayscale image in window units is the detection target image based on the estimated values calculated by the plurality of weak determination units. Judgment device. The detection target image determination device according to claim 3,
Weighting assigning means for multiplying the estimated value from each of the plurality of weak discriminating means by a weighting coefficient obtained by the learning;
The determination unit determines whether or not the given grayscale image is the detection target image based on the estimated value to which the weighting is given from the weighting provision unit. Image determination device. A detection target image determination method for determining whether a given grayscale image is a detection target image,
Of the pixels constituting the grayscale image, this is performed for each of a plurality of sets of adjacent or adjacent two positions determined by learning in advance, and 2 of the set of pixels. A difference between luminance values between pixels is obtained as a feature amount, and an estimated value indicating whether or not the pixel set is a contour portion of the detection target image based on the feature amount is calculated for the plurality of pixel sets. A weak discrimination step to calculate for each,
And a determination step of determining whether or not the given grayscale image is the detection target image based on the plurality of estimated values calculated in the weak determination step. Judgment method. The detection target image determination method according to claim 5,
A weighting step of multiplying each of the plurality of estimated values calculated in the weak discrimination step by a weighting coefficient obtained by the learning;
In the determination step, it is determined whether the grayscale image in window units is the detection target image based on the estimated value to which the weighting is applied in the weighting step. Judgment method. A detection target image determination method for detecting and detecting a detection target image from a grayscale image,
An image reduction step of reducing the grayscale image to generate a plurality of reduced images of different sizes;
A scanning step of scanning each of the reduced images of different sizes generated in the image reduction step in units of a fixed size window;
Among the pixels constituting the grayscale image in the window unit obtained in the scanning step, this is performed for each of a plurality of pairs of adjacent or adjacent two positions determined by learning. Then, a difference between luminance values between two pixels of the pixel set is obtained as a feature amount, and an estimated value indicating whether the pixel set is a contour portion of the detection target image based on the feature amount is obtained. A weak discrimination step for calculating each of the plurality of pixel sets;
And a determination step of determining whether or not the grayscale image in window units is the detection target image based on the plurality of estimated values calculated in the weak determination step. Judgment method. The detection target image determination method according to claim 7,
A weighting step of multiplying each of the plurality of estimated values calculated in the weak discrimination step by a weighting coefficient obtained by the learning;
In the determination step, it is determined whether or not the grayscale image in the window unit is the detection target image based on the estimated value to which the weighting is applied in the weighting step. Judgment method. In order to determine whether the given grayscale image is a detection target image,
Among the pixels constituting the grayscale image, the pixel is provided for each of a plurality of sets of adjacent or adjacent two positions determined by learning, and between the two pixels of the set of pixels. A plurality of weak discriminating means for calculating an estimated value indicating whether or not the set of pixels is a contour portion of the detection target image based on the feature amount, and obtaining a difference between luminance values as a feature amount; and
A detection target image determination program for functioning as a determination unit for determining whether or not the given grayscale image is the detection target image based on the estimated values calculated by the plurality of weak determination units. In order to detect and determine the detection target image from the grayscale image,
Image reduction means for reducing the grayscale image and generating a plurality of images of different sizes;
Scanning means for scanning each of the plurality of reduced images of different sizes from the image reducing means in units of a fixed size window;
Of the pixels constituting the grayscale image of the window unit obtained from the scanning means, provided for each of a plurality of sets of adjacent or adjacent two positions of pixels determined by learning in advance, A weakness for calculating a difference between luminance values of two pixels of the pixel set as a feature amount, and calculating an estimated value indicating whether the pixel set is a contour portion of the detection target image based on the feature amount A plurality of discrimination means, and
A detection target image determination program for functioning as a determination unit for determining whether or not the grayscale image in window units is the detection target image based on the estimated values calculated by the plurality of weak determination units.

Images