JP2011034581A - Object detection device and learning device for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a high precision object detection device that can suppress an increase in a processing load, and a learning device for the same. <P>SOLUTION: The object detection device includes a characteristic extracting part 390 which sets a rectangular template 1010 constituted of a plurality of rectangular blocks in an image window 1001, calculates a reference luminance value based on the luminance value of the rectangular blocks included in the rectangular template 1010 and an individual luminance value about each of the rectangular blocks, calculates a pattern 1110 as spatial distribution information to identify the rectangular blocks from magnitude relation between the reference luminance value and the individual luminance value of each of the rectangular blocks, and uses the spatial distribution information to determine whether an image in the rectangular template 1010 includes an object. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an object detection device that detects an object such as a face from an image and a learning device thereof.

  In computer vision, the object detection technique is a problem of determining whether or not a specific object is reflected in an image. Examples of the object include a car, a pedestrian, and a human face. In many applications, object detection is a very difficult problem. For example, when the object is a human face, its appearance changes greatly due to the face direction, lighting, partial hiding by sunglasses, a mask, and the like. In addition, in an application or the like used for a monitoring device or the like, detection becomes more difficult when the image quality is poor and noise is present or when the face shown in the image is small.

  As a general method for solving the problem of object detection, there is a pattern recognition technique based on statistical learning, and the parameters of the discriminator are determined based on learning samples given in advance. As a general method for face detection, there is a method using a neural network, a support vector machine, Bayesian estimation, or the like. These methods usually include a feature selection technique for extracting a feature amount used for identification from an input image, and a discriminator construction technique for constructing a discriminator for determining whether or not the selected feature amount is an object. This technology consists of a technique for determining the presence of a face in an image window using a constructed discriminator. The “image window” means a partial area in the input image, and a large number of windows in which the position and size of the partial area are changed can be cut out from the input image.

  As a construction method of the discriminator, there is adaptive boosting or adaptive boost known in Non-Patent Document 1. This is hereinafter referred to as “AdaBoost learning method”. This is applied in many object detection systems, and there is Non-Patent Document 2 as a face detection method from an image using this. In the AdaBoost learning method, it is said that the discriminator may have a high error rate that the discrimination error should be 50% or less, and this is called a weak discriminator. In the AdaBoost learning method, several weak classifiers are selected from a large number of prepared weak classifiers and ensembles are selected to construct a strong classifier having a low discrimination error rate.

  As a front face detection method in real time using the Adaboost learning method, there are methods shown in Non-Patent Document 2 and Patent Document 1. The face discriminator, that is, the face detector in Non-Patent Document 2 and Patent Document 1, has a cascade structure in which a plurality of strong discriminators are connected in a row. In the cascade structure, connected classifiers are referred to as stages, and the first stage closer to the input is referred to as a first-stage strong classifier or first-stage stage identifier. The classifier at each stage learns by the Adaboost learning method, and connects and constructs a number of weak classifiers based on the feature amount extracted from the learning input image. Each stage discriminator trains the learning sample so that the discrimination is correct at approximately 100%, while training is performed so that the discrimination is correct at about 50% for the learning sample of the non-face image. . In the case of the first stage discriminator, the face is applied to the input image, and in the case of the second and subsequent stage discriminators, the face is applied to the input image determined by the first stage discriminator as the face. / Determine non-face. What is determined to be non-face at the nth stage is determined as non-face without further processing, so it can be processed efficiently and operates at a processing speed of about 15 frames per second It is known to do.

  In addition, there is a method of improving the identification accuracy by constructing a plurality of face detectors using different learning samples and combining the identification results. As an example thereof, Non-Patent Document 2 shows a majority voting method. Viola et al., The author of Non-Patent Document 2, prepare three cascade structure classifiers (classifiers having a cascade structure), and show that the classification error is reduced by the majority of the output results. In another application shown in Non-Patent Document 3, Rowley et al., Author of Non-Patent Document 3, trained numerous neural networks to build a face detector. As a method for combining the results of a plurality of detectors, a method using a neural network trained to output final results from a number of neural network detectors has been proposed instead of the majority voting method.

  As a feature quantity extraction method for face detection, a feature called a rectangle feature is proposed by Viola et al. The rectangular feature of the image window is extracted by measuring the luminance difference between the rectangular partial areas defined by the rectangular filter.

  As another feature amount extraction method, “Modified Census Transform” of Non-Patent Document 4 has been proposed. The feature amount is extracted by converting a 3 × 3 pixel block in the input image into a binary image. The luminance value of the pixel in the block is compared with the average luminance value in the block. If the luminance value of the pixel is higher than the average value, 1 is labeled, otherwise 0 is labeled. If the labels of all the pixels in the block are arranged in order, it becomes 9-bit information, which is used as a feature value.

US Patent Application Publication No. 2002/0102024

Yoav Freund, Robert E. Schapire, `` A decision-theoretic generalization of on-line learning and an application to boosting '', Computational Learning Theory: Eurocolt '95, Springer-Verlag, 1995, p. 23-37 Paul Viola, Michael Jones, `` Rapid Object Detection Using a Boosted Cascade of Simple Features '', IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), December 2001, ISSN: 1063-6919, Vol. 1, p. 511-518 H. Rowley, S. Baluja, T. Kanade, `` Neural Network-Based Face Detection '', IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 20, No. 1, January 1998, p.23- 28 Bernhard Froba, Andreas Ernst, `` Face Detection with the Modified Census Transform '', Proceedings for Sixth IEEE International Conference on Automatic Face and Gesture Recognition (AFGR), May 2004, p.91-96

  However, the above detection technique has the following problems.

  The first problem is that in the cascade classifiers proposed in Non-Patent Document 2 and Patent Document 1, the weak classifier is a linear classifier that processes the entire input space. Weak classifiers are added and trained to compensate for the previous weak classifier identification error. However, adding a newly trained weak discriminator reduces the error in the overall discriminator formed by concatenating them, but improves the error in one subregion of the input space, while another subregion Causes an identification error. Therefore, if the strong discriminator in the subsequent stage in the cascade discriminator is similar to the image including the object (for example, face) and the image not including the object as image features, the data is linearly separated on the feature section. There are many things that cannot be done, and the rate of causing the discrimination error becomes high. Therefore, a very large number of weak classifiers are required, and the number of discrimination processes to be processed at the time of identification increases.

  Further, when learning a stage discriminator, an image that does not include an object (hereinafter referred to as a non-object image) is learned with a different image sample for each stage, and is best suited to the learning non-object image ( The weak discriminator (which can be discriminated best) is selected, but an image including an object (hereinafter referred to as an object image) must be recognized as it at all stages and is learned by the same learning sample. However, for example, when the object is a face, the characteristics of the face image vary greatly depending on the orientation / tilt of the face, lighting conditions, personal characteristics and race, concealment by sunglasses / masks and hair, and the like. It is easy to discriminate a face that faces directly in front and clearly shows eyes, nose, mouth, etc. On the other hand, it is difficult to discriminate when the facial features are reduced by orientation, lighting, or the like. In spite of such circumstances, the conventional classifiers process those facial features with the same weak classifier, and there is no mechanism for processing a face image that can be easily distinguished from a non-face. In addition, since identification according to the feature of each face is not realized, it is difficult to detect a complex face feature with high accuracy.

  The second problem is that in the cascade structure in Non-Patent Document 2 and Patent Document 1, no information is transmitted from a strong discriminator of one stage to another stage. In the conventional method, the weak classifier of a certain stage does not know the output value of the strong classifier of the preceding stage. Therefore, for example, when the object is a face image, the construction of a detector for a face that is easy to discriminate in the front direction can be realized by a cascade structure, but the input space is complicated and the classifier has a high dimension. Realization of a detector for such a complex face (for example, face images in various directions) is difficult in a cascade structure. When learning a new strong classifier without knowing the result of the strong classifier in the previous stage, the weak classifier used in the strong classifier is the weakest class that best separates the learning face and non-face samples as a whole. Selected from the classifier. On the other hand, as a result of the strong classifier in the previous stage, information about whether the strong classifier in the previous stage was difficult to identify (that is, a sample that exists near the boundary between the face and non-face), and was a sample that was easy to distinguish Can be incorporated from weak discriminators that use the information to identify the vicinity of the boundary between the face and the non-face, making it possible to quickly find the boundary between the face and the non-face in the feature space Become. Therefore, it is considered that the information of the strong classifier in the previous stage is indispensable for the construction of a detector that can identify a complex face such as a face in various directions, which becomes a complex identification boundary.

  A third problem is that in the majority voting method shown in Non-Patent Document 2, a plurality of detectors operate in parallel, and thus the processing load is large. In addition, since cascade discriminators are trained independently, it is unclear whether they are operating in a complementary manner. No information is shared between classifiers. The majority method is not the best way to combine the results of multiple detectors. For example, if a completely different classifier is trained and configured to output the best results from multiple classifiers as shown in Non-Patent Document 4, the processing time will be faster.

  The fourth problem is that the rectangular feature based on the luminance value proposed by Viola et al. In Non-Patent Document 2 and Patent Document 1 is sensitive to the lighting environment. For example, FIG. 15 is a diagram for explaining the problem of the rectangular feature in the prior art, and the rectangular feature is a value of a difference in luminance value between the rectangle 1220 and the rectangle 1221 indicated by the oblique lines in FIG. For example, in the face image 1202 in FIG. 15B, the illumination effect appears strongly on a part of the face, and the difference in luminance value in such an image is similar to that of the non-face image 1212 in FIG. Value. Further, since the feature amount only measures the luminance information in the rectangular block, important arrangement information is not acquired. For example, the non-face image 1211 shown in FIG. 15C and the face image 1201 shown in FIG. 15A have approximately the same number of low luminance value pixels, and therefore have the same luminance difference value. Although the spatial distribution of pixels with high and low luminance values is very important in identifying images, they are not considered in feature extraction. In the latter classifier in which the non-face image is more similar to the face image, the rectangular feature becomes less effective in separating the face and the non-face and increases the identification error. This is a factor that greatly increases the number of feature quantities in the stage discriminator.

  As a different approach, for example, there is a “modified census transform” feature amount based on a pattern, which is shown in Non-Patent Document 4, for example, but this is sensitive to the influence of noise because only the local feature amount is extracted. . For example, FIG. 16 is a diagram for explaining the problem of this conventional feature amount. The conversion template takes a feature amount for a block 1250 of 3 × 3 pixels. When the block 1250 is converted to binary as indicated by the binary block 1251, the non-face image 1231 with noise is also converted to the same binary block value as the face image 1201. This is because luminance information is completely ignored in the “modified census transform” feature. Since the “modified census transform” feature value is in units of pixels, it is too localized in the block, and a global feature value cannot be acquired.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an object detection device and a learning device thereof that can suppress an increase in processing load with high accuracy.

  1st of this invention is an object detection apparatus, Comprising: The image window extraction means which extracts multiple image windows which are the partial area | regions of the said image from the input image, and the discriminator which identifies a target object Object detection means for detecting the presence of an object for each of the extracted image windows using a node network in which nodes are connected in a network form.

  With this configuration, an image window is identified using a node network and the presence of an object is detected, so that an increase in processing load can be suppressed with high accuracy.

  A second aspect of the present invention is the object detection apparatus according to the first aspect, wherein the node includes a plurality of discriminators.

  Thirdly, the object detection apparatus according to the first or second aspect described above, wherein the object detection unit includes a route generation unit that generates at least one route in the node network; And identifying means for performing an object identification process for each of the generated paths and outputting an identification result as to whether or not the image window includes the object.

  Fourthly, the present invention provides the object detection apparatus according to the third aspect, wherein the object detection means has a new path that has not yet been evaluated in the node network with respect to the image window. Whether or not to perform the determination is determined, and based on the determination result, the route generation processing by the route generation means and the identification result output processing by the identification means are repeated.

  The fifth aspect of the present invention is the object detection apparatus according to the fourth aspect, wherein the route generation processing and the identification result output processing are repeated until the number of repetitions reaches a predetermined number of times or when a new route is detected. Repeated until it can no longer be generated.

  Sixthly, the present invention provides the object detection apparatus according to any one of the third to fifth aspects, wherein the path generation unit does not exceed a predetermined number of paths and at least one path And a function of dividing each route on the network to generate a new route.

  Seventhly, the present invention provides the object detection apparatus according to any one of the third to sixth aspects, wherein the identification unit obtains an identification result and an identification error in each of the generated paths. An evaluation unit that performs the selection, a selection unit that selects an identification result on a route with the lowest identification error as an identification result of the node network, and an identification process in the evaluation unit when the lowest identification error becomes smaller than a predetermined value And a stop determining unit that determines to stop.

  Eighth, the present invention is the object detection device according to the seventh aspect, wherein the evaluation unit extracts a feature value of the image window for each node of the generated route, and a route An application unit that applies the feature value to the discriminator in order to obtain an evaluation value of each node included in the node, and a coupling unit that combines the evaluation values of the nodes to generate a cumulative evaluation value of the generated path. In order to obtain the identification result of the generated route, an identification unit for identifying whether the image window includes the target object using a cumulative evaluation value, and an identification error with respect to the generated route identification result An error estimation unit for estimation.

  Ninthly, the present invention provides the object detection apparatus according to any one of the first to eighth aspects, wherein whether or not the identification result by the classifier of each of the nodes is used for determining a final result, It is changed according to the feature in the image window.

  The tenth aspect of the present invention is the object detection apparatus according to any one of the first to eighth aspects, wherein a ratio of using a discrimination result by a discriminator of each node for determination of a final result is the image window. It is changed according to the characteristics in

  Eleventhly, in the object detection apparatus according to the tenth aspect described above, the ratio of the identification result obtained by the classifier of each node used for the determination of the final result is the characteristics in the image window, It is determined based on a parameter based on the difference from the image feature used for learning of the classifier.

  The twelfth aspect of the present invention is the object detection apparatus according to any one of the first to eighth aspects, wherein when the discriminator is based on a boosting method, a result of a discriminator is determined as a final result. The ratio that contributes to: the cumulative evaluation value by all classifiers before the classifier for the image window, and the cumulative evaluation value by all classifiers before the classifier when the classifier is constructed based on the learning image Is determined based on the difference from the cumulative evaluation value having the highest identification error.

  According to a thirteenth aspect of the present invention, in the object detection device according to any one of the first to eighth aspects, in the discriminator at each node, a ratio that contributes to a cumulative evaluation value is a parent of the node. The difference between the cumulative evaluation value of the image window at the node and the cumulative evaluation value with the highest identification error among the cumulative evaluation values at the parent node of the node when the feature classifier of the node is constructed based on the learning image To be determined.

  The fourteenth aspect of the present invention is the object detection device according to any one of the first to thirteenth aspects, wherein the object detection means has a feature of a node when there are a plurality of parent nodes of the node. When learning the quantity discriminator, the accumulated evaluation value that has caused the most identification error in the accumulated evaluation value at the parent node of the node in the learning image is searched in the range of the accumulated evaluation value at the parent node.

  15thly, this invention is a learning apparatus of a target object detection apparatus, Comprising: The said target object detection apparatus uses the node network to which the node which has the discriminator which identifies a target object was connected in network form, Node network learning means for dynamically learning the classifier of the node network structure, wherein the node network learning means includes empty node generation means for generating at least one empty node in the node network, and the empty network Learning image collecting means for collecting a plurality of images for node learning, and node learning means for learning an empty node from the collected images.

  The sixteenth aspect of the present invention is the learning device according to the fifteenth aspect described above, wherein the node network learning unit determines whether a new empty node can be created, and generates the empty node based on the determination result. The processing by the means, the learning image collection means and the node learning means is repeated.

  Seventeenth, the present invention is the learning device according to the fifteenth or sixteenth aspect, wherein the empty node generation unit generates a reference node when the node network is empty and already exists in the node network. A new empty node is created by integrating and splitting the nodes to be created.

  The eighteenth aspect of the present invention is the learning device according to any one of the fifteenth to seventeenth aspects, wherein the empty node generation means has an insufficient number of images collected by the learning image collection means. Then remove the node.

  Nineteenth, the present invention is the learning device according to any one of the fifteenth to eighteenth aspects, wherein the learning image collection means gives the image as an input of a node network under construction, propagates the image, Images are shared among the empty nodes, and images reaching each empty node are collected.

  The twentieth aspect of the present invention is the learning device according to any one of the fifteenth to nineteenth aspects, wherein a boosting algorithm is applied to the node learning unit.

  The invention according to the twenty-first aspect is the learning device according to the twentieth aspect, wherein the node learning unit determines a subset from the collected images by the boosting algorithm, and uses the subset. The collected image is weighted, and the classifier of the empty node is determined using the weighted image.

  According to a twenty-second aspect of the present invention, in the object detection device according to any one of the first to fourteenth aspects, the object detection unit includes the spatial distribution information of the pixels of the image window and the intensity of the pixels. Detect using both information.

  According to a twenty-third aspect of the present invention, in the object detection device according to the twenty-second aspect, the spatial distribution information of the pixels is a pixel block composed of one or more pixels. This represents the magnitude relationship between the feature value of each pixel block and the arrangement relationship of the pixel block on the image.

  According to a twenty-fourth aspect of the present invention, in the object detection apparatus according to the twenty-third aspect, the strength information of the pixel is a strength of a magnitude relation of a feature value of each pixel block with respect to the spatial distribution information. It expresses.

  According to a twenty-fifth aspect of the present invention, in the object detection device according to the twenty-third or twenty-fourth, a rectangular template is applied to the pixel block.

  ADVANTAGE OF THE INVENTION According to this invention, the target object detection apparatus which can suppress the increase in processing load with high precision, and its learning apparatus can be provided.

The block diagram which shows schematic structure of the target object detection apparatus which concerns on embodiment of this invention. The conceptual diagram which shows the node network which concerns on embodiment of this invention The flowchart which shows the process sequence of the path | pass production | generation part which concerns on embodiment of this invention. The flowchart which shows the process sequence of the image window detection process part which concerns on embodiment of this invention. The block diagram which shows schematic structure of the target object detection learning apparatus which concerns on embodiment of this invention. The flowchart which shows the process sequence of the empty node production | generation part which concerns on embodiment of this invention. The conceptual diagram explaining the empty node production | generation which concerns on embodiment of this invention The flowchart which shows the process sequence of the image sample collection part which concerns on embodiment of this invention. The flowchart which shows the process sequence of the node learning part which concerns on embodiment of this invention. The block diagram which shows schematic structure of the feature extraction part which concerns on embodiment of this invention. The figure which shows an example of the rectangular template for the feature extraction which concerns on embodiment of this invention The flowchart which shows the process sequence of the feature extraction part which concerns on embodiment of this invention. Explanatory drawing for demonstrating the process sequence in the feature extraction part which concerns on embodiment of this invention The figure which shows the other example of the rectangular template for the feature extraction which concerns on embodiment of this invention The figure for demonstrating the subject of the rectangular feature in a prior art The figure for demonstrating the subject of the conventional feature-value

  Next, the target object detection apparatus which concerns on embodiment of this invention is demonstrated. In the following description, a human face will be described as a specific example as an object.

<Structure of detection device>
FIG. 1 is a block diagram showing a schematic configuration of an object detection apparatus according to an embodiment of the present invention. As shown in FIG. 1, the object detection apparatus of the present embodiment includes an input unit 201, an image window extraction unit 210, a storage unit 502, a network identifier 590, and an output unit 202.

  The image window extraction unit 210 extracts a plurality of image windows from the image input to the input unit 201. The “image window” means a partial area in the input image, and a large number of windows in which the position and size of the partial area are changed can be cut out from the input image.

  The storage unit 502 stores a node network. FIG. 2 is a conceptual diagram showing a node network according to the embodiment of the present invention. The network 100 has a plurality of nodes arranged on the network.

  The figure shows the basic unit 110 of the network. The basic unit 110 includes one node “node N” 111, a joint unit 117 that integrates connections from at most M nodes, and a spirit unit 118 that divides into at most N nodes. For example, FIG. 2 shows a case where M = N = 2. The M nodes are referred to as parent nodes of the node N, and the N nodes are referred to as child nodes of the node N.

  In the case of FIG. 2, the parent nodes of the node N are the nodes 112 and 113, and the child nodes are the nodes 114 and 115. A node having no parent node is referred to as a root node (101 in FIG. 2). The input image 170 is input to the root node 101.

Each node 111 has a plurality of discriminator, the discriminator has feature classifier h _n, the object identifier H _n, and the identification error function E _n. For example, Jin, R., Hauptmann, A., Carbonel, J., Si, L., Liu, Y., `` A New Boosting Algorithm Using Input Dependent Regularizer '', 20th International Conference on Machine Learning (ICML ' 03), Washington, DC, August 21-24, 2003 (hereinafter referred to as Document A), and can be constructed by using boosting locally. The feature quantity classifier may be a weak classifier in the boosting learning method, and the object classifier may be a strong classifier in the boosting learning method.

  Returning to the description of the object detection apparatus of FIG. The network classifier 590 is a classifier having a network structure, acquires an image window from the image window extraction unit 210, and includes an object in each image window using a node network stored in the storage unit 502. Or not.

  The network identifier 590 includes a path generation unit 520 and an image window detection processing unit 530. The path generation unit 520 reads the node network information stored in the storage unit 502 and generates at least one path (route). A path is a sequence of nodes selected to process an image window.

The image window detection processing unit 530 acquires an image window from the image window extraction unit 210 and processes the image window using the path generated by the path generation unit 520. In each pass, calculation is performed by the classifier (feature quantity classifier h _n , target object classifier H _n , identification error function E _n ) at each node described above, and whether or not the image window includes the target object is determined. An identification result is generated. As the identification result to be output, the identification result with the lowest identification error is selected and stored in the output unit 202.

  Subsequently, the network discriminator 590 instructs the path generation unit 520 to generate a new path, and repeats the above identification process until no new paths are generated or until a predetermined number of times is reached.

  FIG. 3 is a flowchart showing a processing procedure of the path generation unit according to the embodiment of the present invention. First, the path generation unit 520 determines whether there is a path generated in the network (step 621). If there is no path, a path including only the root node is generated (step 622), and the path generation process is terminated.

  If there is a path, it is determined whether or not the number of paths generated in the network is larger than a preset number K (step 623). For example, 1, 2, 3, etc. are set as the value of K. If the number of paths exceeds K, the path with a high identification error is terminated so that the number of paths is at most K (step 624). Here, the determination is made based on what has already been obtained by the image window detection processing unit 530. “End a pass” refers to a case where no further processing is performed on an input image for a certain pass.

More paths are generated by dividing each current path (step 625). For example, consider the case where a path includes {node ₀ , node _a , node _b }. At this time, as a child node of the example node _{b, node} and _{b, child1, node b, child2} is generated, a new _{path, {node 0, node a,} node b, node b, child1} and _{node 0, node _a , Node _b , node _{b, child2} } are generated.

  FIG. 4 is a flowchart showing a processing procedure of the image window detection processing unit according to the embodiment of the present invention.

  For each generated path, the image window detection processing unit 530 evaluates an identification result indicating whether or not the image window includes an object. The identification result may be obtained from local boosting as in Document A above, for example. The output identification result is selected from all the identification results of the generated path.

Each step of the flowchart of FIG. 4 will be described. First, the image window detection processing unit 530 extracts a feature amount from the input image window for each node of the generated path (step 631). Let X be the input image window, and let f _n (X) be the feature quantity extracted for node N. Note that the feature extraction is performed by, for example, the feature extraction unit 390 shown in FIG. 10, and details will be described later.

Next, the extracted feature quantity f _n (X) is given to the discriminator, and the score h _n (X) for the node of the generated path is obtained (step 632). The score h _n (X) is obtained from the feature quantity discriminator h _n and is calculated based on the following equation (1).

  Prob (k) in the above equation (1) indicates the probability that the event k will occur. Y = + 1 and Y = −1 mean that the input image includes and does not include the object, respectively.

Next, the score of each node is combined to evaluate the accumulated score S _n (X) for the generated path (step 633). The cumulative score S _n (X) may be obtained by the following equation (2).

The regularization function exp (− | S _{n, parent} (X) −α _n |) localizes the effect of the feature quantity classifier h _n (X) according to the input image. Thus, only when S _{n, parent} (X) takes a value sufficiently close to the regularization parameter α _n , it is added to the cumulative score S _n (X). Therefore, in some cases, exp (− | S _{n, parent} (X) −α _n |) is almost 0, and the new value S _n (X) is relatively close to the old value S _{n, parent} (X). The value may not change.

Next, it is determined whether or not the image window includes an object based on the cumulative score S _n (X) and the identification result H _n (X) of each path (step 634). Here, the discrimination result H _n (X) may be obtained from the object discriminator H _n or may be obtained by the following equation (3).

H _n (X) = + 1 is a determination result that the image window includes the object, and H _n (X) = − 1 means a determination result that the object does not include the object.

Next, an identification error E _n (X) for the path identification result is estimated (step 635). The identification error can be obtained from the error function E _n (error function) or may be obtained by the following equation (4).

Moreover, in order to reduce the processing load concerning calculation, the function h _n may be implemented as a lookup table as shown in the following equation (5). The functions E _n , H _n and S _n may also be implemented as a two-dimensional lookup table as shown in the following equation (6).

  The image window detection processing unit 530 repeats the above steps 631 to 635 to obtain the identification result and identification error (error function) of each path (step 636). Then, the identification result of the lowest identification error value is selected as the output result from the obtained path identification results (step 637).

  If the output result identification error is smaller than a predetermined value, the identification process is terminated (step 638). When the output result identification error is not smaller than a predetermined value, the path generation unit 520 is instructed to generate a new path, and the path generation unit 520 generates a path.

  The output unit 202 thus provides information such as the position (for example, the coordinate value on the input image) and the size of the image window that the image window detection processing unit 530 of the network classifier 590 has identified as including an object. Is output.

  In this way, the network is integrated and divided by the path generation unit 520, so that a large number of paths can be constructed. Each path is used to evaluate the identification result, and the network is a pool of many detectors. By being a detector group, it is possible to produce a more reliable output result than a single discriminator.

  Furthermore, the number of passes is limited by K to ensure a fast identification process. In order to guarantee the best identification result, paths are generated dynamically and paths with high identification errors are terminated during the identification process. Therefore, only paths with a low identification error for the input are used.

  Therefore, in the conventional parallel, the parallel detector is static and cannot be changed during the identification process, whereas in the object detection apparatus of the present embodiment, the detector to be used is dynamically changed, and is not necessary. Processing can be suppressed.

Further, in the image window detection processing unit 530, by using the boosting algorithm, it is guaranteed that the identification error in the newly generated path is statistically smaller than that in the old path. Furthermore, the above equation (2) performs boosting locally and ensures that the classifier h _n (X) acts only on a subset of the input image.

  In addition, the speed of the identification process is realized by this method of stopping the process with sufficiently low identification error. This is because an input image that can be easily discriminated can be identified at an early stage, regardless of whether the object is included, and the number of nodes necessary for the discrimination can be reduced.

  In addition, this method is more efficient than the conventional cascade structure type in which only the input image window that can be identified as not containing an object is recognized at an early stage and then processed by a fixed number of discriminators. .

<Learning Method and Device for Object Detection>
FIG. 5 is a block diagram showing a schematic configuration of the object detection learning device according to the embodiment of the present invention. Parts that are the same as those in the object detection device shown in FIG. As illustrated in FIG. 5, an input unit 701, a network learning unit 790, and a storage unit 502 are included. This object detection learning device is configured to provide a plurality of image samples with determination result information (presence / absence of an object) and to learn a classifier of a node of a node network used in the object detection device. is there. An image sample including the target object is referred to as a positive sample, and an image sample not including the target object is referred to as a negative sample.

  Given a plurality of image samples, the network learning unit 790 determines a node network learned to identify the plurality of image samples. The node network determined here is stored in the storage unit 502 and is used by the network identifier 590 to identify whether the image window has an object instance in the identification process.

  As illustrated in FIG. 7, the network learning unit 790 includes an empty node generation unit 710, an image sample collection unit 720, and a node learning unit 730. An empty node refers to a node for which a discriminator has not been determined, and a learned node refers to a node for which a discriminator has been determined. For example, the local boosting algorithm described in the above-mentioned document A may be used to determine the classifier of the network node.

  First, the empty node generation unit 710 reads the current node network stored in the storage unit 502 and generates an empty node for learning. Then, the empty node generation unit 710 uses the image sample collection unit 720 to collect a predetermined number of image samples from a plurality of input image samples. If the number of collected image samples is less than a predetermined number, learning cannot be performed, and therefore, an empty node is deleted.

  Next, the node learning unit 730 finally determines a discriminator corresponding to each empty node generated using the image samples collected from the image sample collection unit 720. The node network is updated by storing learned nodes in the storage unit 502.

  These learning processes are repeated until no empty node is generated by the empty node generation unit 710.

FIG. 6 is a flowchart showing a processing procedure of the empty node generation unit according to the embodiment of the present invention. In step 811, it is determined whether the node network is empty. The network is empty when there are no nodes. If the network is empty, the network is started by generating an empty root node (node ₀ ) (step 812).

  On the other hand, if the network is not empty, that is, if there is at least one node, a new empty node is generated by combining and dividing the nodes so as to become N or less child nodes (step 813). Here, the generation process of this empty node will be described.

  FIG. 7 is a conceptual diagram illustrating empty node generation according to the embodiment of the present invention. From the current node network 901, empty nodes 950, 951, 952, and 953 are formed by dividing the nodes 960, 961, 962, and 963 so that adjacent empty nodes share a common parent. For example, node 950 and the node have one common parent node.

  Returning to FIG. 6, the procedure for generating an empty node will be described. In step 814, the image sample collection unit 720 collects a predetermined number T (for example, 10,000) of image sample groups in each empty node.

  In step 815, empty nodes with insufficient collected image samples are removed. For example, when the number of image samples is less than T.

  FIG. 8 is a flowchart showing a processing procedure of the image sample collection unit according to the embodiment of the present invention.

  In step 821, the input image sample group 701 is transmitted using the path used for identifying the node network stored in the storage unit 502. For example, in step S821, the image sample collection unit 720 performs the same processing as that of the network identifier 590, and generates one or more paths for identifying one image sample with the same operation as when detecting an object. A copy of the image sample is generated, propagates over the network over the generated path, and reaches the node at the end of the path. At that time, multiple paths generated in the network may exist, and therefore, the copy of the image sample may reach the same node in multiple ways.

  The operation of the image sample collection unit 720 is the same as that when the network identifier 590 detects an object. Therefore, if the identification error of image identification is sufficiently small at a certain node, the path for that image is the end point, and the number of paths is limited to a certain number or less.

  In step 822, the image sample group that has reached the parent node is shared by its empty child nodes. (For example, if there are multiple parent nodes for a child node, the child node uses a merged set of image samples of each parent node as an image sample. If there are multiple child nodes for the parent node, each child node is Take over the same set of image samples from a common parent node). That is, a copy of the image that has reached the parent node in step 821 is generated at each child node.

A positive sample is expressed as (X, Y = + 1), and a negative sample is expressed as (X, Y = -1). For one empty node n, the cumulative score S _{n, parent} (X) of the image sample at the parent node is determined using equation (2). The image sample group at the empty node is expressed as (X, Y _{, Sn, parent} (X)).

  In step 823, image samples that reach each empty node are collected until a maximum of T images are reached. For example, T / 2 positive samples and T / 2 negative samples are randomly selected from all image samples arriving at the empty node.

  FIG. 9 is a flowchart showing a processing procedure of the node learning unit according to the embodiment of the present invention.

  The node learning unit 730 uses the image sample group collected for learning of the empty node n, and determines a subset so that an identification error becomes large at the parent node. Next, a weight function is determined so as to generate a localized and specialized classifier focusing on learning in the subset. Explain the work in order.

  In step 831, a subset of image samples for specializing the classifier of the node is determined by changing the weight when considering the cumulative score according to the image samples. The subset of the image samples is the subset that gives the highest identification error at the parent node. The procedure is as follows.

First, let the image sample group collected from the image sample collection unit 720 be (X, Y _{, Sn, parent} (X)). Next, the current identification error E _{n, parent} (X) for the image sample group is determined by equation (4). Using this identification error E _{n, parent} (X), the value of the regularization parameter α _n is determined by Equation (7).

In Expression (7), α _{n, parent1} is set so that the image sample group having the cumulative score S _{n, parent} (X) of _{the parent} close to α _n has the largest identification error _{En , parent} (X) at the parent node. The value α _n is selected between α _{n and parent2} . In other words, selection is made to intensively learn images that could not be identified well in the previous stage. Therefore, it can be expected that the classifier of node n learns to reduce the identification error for a subset of the image samples chosen to maximize the identification error at the parent node.

Using the regularization function exp (− | S _{n, parent} (X) −α |), α _n is determined by the following equation (7). Both the regularization function and the accumulated score are the same as those in the object detection apparatus.

That is, in α _satisfying α _{n, parent1} <α <α _{n, parent2} , α that maximizes the sum of all the collected image samples of the product of the identification error and the regularization function in the parent node is expressed as α _{Let n} . If there are three or more parent nodes, the selection range of α is min (α _{n, parent} ) <α <max (α _{n, parent} ). At this time, when there is only one parent node, the value shown in the following equation (8) is used.

  Thus, in step 831, the subset of image samples for specializing the classifier is determined as the set of image samples that maximizes the identification error at the parent node.

  Next, using the subset determined in step 831, in step 832, weights are determined for all collected image samples. The weighted learning sample group is denoted as (X, Y, w (X)). Here, the weighting function w (X) is mathematically expressed by the following equation (9).

Here, W _sum is a normalization constant when w (X) is a distribution. In this _{equation, | S n, parent (X} ) -α | large weights in situations where the _{≒ 0, | S n, parent} (X) -α | weighting function is small in situations where a large.

Further, in step 833, various classifiers of the node n are determined using the weighted image sample group. Determining classifier, respectively formula (1), (3), which is a feature classifier shown in (4) _{h n,} the object identifier _{H n,} the identification error function _{E n.} Here, as the feature quantity, for example, one feature quantity that gives the best discrimination is selected from the feature quantities found by the boosting algorithm shown in the above-mentioned document A and used.

The feature classifier h _n may be determined by the following equation 10.

In this equation, P _w (j) represents a weighted probability that event j occurs in distribution w. f _n (X) is the feature quantity extracted from the image sample X using the best choice (= giving the best discrimination result), for example, using a feature extraction unit 390 (described later) in FIG. The extracted feature amount. Σw ₊ and Σw ₋ are sums of weights of positive samples and negative samples each having a feature quantity f _n (X).

Using the regularization parameter α _n and the value of the feature classifier h _n (X), the cumulative score S _n (X) is determined by Equation (2). As a result, the conditional probability is determined by the following equations (11) and (12).

Here, C ₊ and C ₋ are respectively the counts (numbers) of positive samples and negative samples having a cumulative score S _n (X). Using the conditional probability, the object discriminator _Hn is determined by Equation (3).

Furthermore, the identification error E _n (X) of the object identifier is expressed by the following equation (13).

Here, min {a, b} indicates the minimum value of a and b. In this manner, the determining feature classifier h _n for each _node, the object identifier H _n, the identification error function E _n. The node learning process is completed by the series of procedures described above.

  As described above, the network learning unit 790 according to the embodiment of the present invention has the following advantages.

  First, since the empty node generation unit 710 can determine an object using the identification results of a plurality of routes, it has an advantage that it can be determined more effectively than the prior art with a cascade structure.

  In this embodiment, since integration and division of nodes are used, there are one or more upper layer nodes connected to one lower layer node via a connection path. The feature classifiers learned by the boosting algorithm are used for the feature classifiers of each node, and the results of those classifiers are transmitted to one or more lower classifiers. The object can be determined by combining the information of the classifiers of the plurality of paths.

  This is combined with the information of classifiers of a plurality of paths, so that the weak classifier in one stage has a cascade structure that does not have the information of weak classifiers in another stage. It is an advantage over technology. On the other hand, in the prior art, the information of the weak classifier is only transmitted to the next weak classifier in the same stage. This advantage corresponds to the third feature described later.

  Further, the empty node generation unit 710 has the following advantages from the viewpoint of learning. In an object detector having a cascade structure in the prior art, a weak discriminator in one stage does not use information of a weak discriminator in another stage. On the other hand, in the apparatus of the present embodiment, it is possible to learn an empty node using the information of a plurality of weak discriminators belonging to a plurality of stage discriminators by the above-described empty node generation flow.

  In addition, since this method uses node integration and division, there are paths from the root node to newly generated empty nodes, and each path from these root nodes to the empty nodes corresponds to a stage discriminator. is doing. Each node in the upper layer is a feature classifier learned by a boosting algorithm, and a structure that can learn an empty node is realized by using the result of the feature classifier belonging to a plurality of stage classifiers.

  Even in the majority method known as a conventional example, the determination is used in a plurality of detectors operating in parallel. However, the embodiment of the present invention has the following advantages.

  First, since a limited number of paths are generated for the node network, the amount of calculation does not increase as much as the number of paths compared to the majority method. In addition, since the criteria for generating and learning a plurality of empty nodes for the node network are clear, learning by a plurality of paths is effectively performed, and each cascade discriminator is used in the conventional majority voting method. Overcoming the third issue, which is unclear whether or not is operating in a complementary manner. In this way, ensemble learning is more efficient than before.

  Next, the following advantages are obtained by the image sample collection unit. First, in the embodiment of the present invention, since the image sample group is shared by a plurality of empty nodes, it is possible to simultaneously learn one or more nodes of a path by one learning image.

  Therefore, as compared with the case where one image sample is identified by a single classifier, it can be effectively identified by a network classifier configured by a plurality of paths from the same empty node and using identification results from the plurality of paths. This advantage corresponds to the second feature described later. This is made possible by a mechanism in which the image sample collection unit shares an image sample group with a plurality of empty nodes.

  Second, in the embodiment of the present invention, even if the number of classifiers is increased, the learning data is not excessively fragmented, and stable learning is possible even in the latter classifier. Is possible.

  In the conventional method, as the number of stages of the classifier increases, the number of learning samples reaching the classifier decreases (excessive fragmentation occurs), and the learning result depends on the learning sample (overlearning). ) Was strongly indicated. In this method, the images are shared by integrating and dividing the nodes so that the determination by multiple paths is possible, so there is an overlap in the priority area of learning, and the learning sample is excessively fragmented. In addition, since the learning result is less likely to indicate overlearning, there is an advantage that the learning is stable.

  In general, when learning with a high degree of freedom is performed, the statistical characteristics of the original data are not reproduced, but tend to be excessively dependent on the data set used for learning. For example, AIC (Akaike's Information Criteria) is known as an index for determining whether a model is good or bad.

  In multivariate analysis, a model having a degree of freedom that minimizes AIC is used to prevent excessive dependence on the data set used for learning. Similarly, in the network type discriminator of the present invention, increasing the degree of freedom unnecessarily is a good learning result only for the data set used for learning, regardless of the original model. Contains the potential danger that it may be. However, in the method of the present embodiment, the learning sample is not subdivided because the nodes are also integrated, and when the prescribed number of images are not collected in the empty node, the learning is not performed. In this method, the learning result is less likely to indicate over-learning because the restriction that the determination is performed using the discriminator is added and the risk of unnecessarily increasing the degree of freedom is prevented.

  Further, the node learning unit 730 of the present embodiment provides the following advantages.

First, the node learning unit 730 creates a specialized classifier for a subset of image samples that could not be identified by the parent node. This is because the node learning unit uses the weight function of Equation (9) (because the feature discriminator h _n is trained according to Equation (10)), and thus an image that causes a large discrimination error in the parent discriminator. A large weight is given to the sample group.

Second, the node learning unit creates a discriminator h _n whose influence is localized. That is, only when a large identification error occurs in the parent node, the contribution to the accumulated score by this node is made. Actually, when a large identification error occurs in the parent classifier, | S _{n, parent} (X) −α _n | is a small value in the expression (2), so exp (− The term | S _{n, parent} (X) −α _n |) * h _n (X) becomes large, and the value of the discriminator h _n (X) at this node is sufficiently reflected in the accumulated score.

On the other hand, when a large identification error does not occur in the parent classifier, | S _{n, parent} (X) −α _n | is increased in Expression (2), so exp (− | S _{n, parent} ( The term X) −α _n |) * h _n (X) is small, and the value of the discriminator h _n (X) at this node has little effect on the cumulative score.

  For this reason, the influence on the cumulative score is limited to the case where the identification error at the parent node is large. It can be paraphrased as a classifier with localized effects.

  In this way, in each classifier of each node of one path, each classifier has a high contribution and the input image that is effective on the cumulative score is different. Therefore, each classifier has operations for all other classifiers. There is a priority area that operates preferentially without affecting.

  On the other hand, in the cascade structure classifier given as a conventional example, as shown in the first problem, the weak classifier is a linear classifier that processes all input spaces, and all weak classifiers are all Must be identified as a face. On the other hand, in the node classifier according to the embodiment of the present invention, a face and a non-face are identified only for a set of images in which the preceding node classifier is likely to be erroneously identified. Therefore, the input image space in which the node classifier operates is limited, and the face / non-face discrimination with this node classifier is much simpler than the cascade classifier. realizable. This corresponds to the first feature described later.

  As described above, in the embodiment of the present invention, since there is a preferential space of the input image in which the classifier of each node operates preferentially, the operation of one classifier affects the operation of the other classifier. There is hardly anything. This is a significant advantage compared to the prior art, which has the problem that weak classifiers operate on the entire input space and can affect other weak classifier operations.

<Feature extraction>
FIG. 10 is a block diagram showing a schematic configuration of the feature extraction unit according to the embodiment of the present invention. This feature extraction unit is used, for example, in the image window detection processing unit 530 shown in FIG. 1 or the node learning unit 730 shown in FIG.

  As illustrated in FIG. 10, the feature extraction unit 390 extracts a feature amount 309 from the image window input to the input unit 501. This feature quantity 309 is used to identify whether the image window includes an object.

  For example, step 631 shown in FIG. 4 executed by the image window detection processing unit 530 is executed by the feature extraction unit 390.

  FIG. 11 is a diagram showing an example of a rectangular template for feature extraction according to the embodiment of the present invention. For a given image window 1001, the rectangular template includes L rectangular blocks such as rectangles of the same size inside the image window 1001. L may take any value between 2 and 9, for example. Each rectangular block can be specified by the upper right coordinate, the width w, and the height h.

  For example, a rectangular template 1010 shown in FIG. 11A, a rectangular template 1020 shown in FIG. 11B, and a rectangular template 1030 shown in FIG. 11C each include 6, 9, and 7 rectangular blocks. .

  FIG. 12 is a flowchart showing a processing procedure of the feature extraction unit according to the embodiment of the present invention. Moreover, FIG. 13 is explanatory drawing for demonstrating the process sequence in the feature extraction part which concerns on embodiment of this invention.

  In step 491, a rectangular template f that defines a rectangular block in the image window is applied to the image window X.

  In step 492, spatial distribution information is measured from the image window. The spatial distribution information is shown as a pattern 1110 indicating which rectangular block of the pattern has a higher luminance value than the others.

  The pattern 1110 is calculated by comparing the luminance values of the rectangular blocks. First, an average luminance value of all rectangular blocks is calculated and used as a reference luminance value. Next, the average luminance value of each rectangular block is calculated. The rectangular block is labeled 1 if the average luminance value of the rectangular block is smaller than the reference luminance value, and 0 if it is larger. These intensities can be calculated quickly and efficiently using integral images. By collecting the labels of the rectangular blocks, a pattern 1110 is obtained.

  As an example, a rectangular template 1010 is used in FIG. For the input face image 1105, the feature extraction unit 390 generates a pattern 1151 with pattern = 101000. For another non-face image 1106, the feature extraction unit 390 generates a pattern 1161 with pattern = 01000.

  In step 493, spatial luminance value information is measured from the image window. This spatial luminance value information is assumed to be strength, which indicates how much intensity is different between different blocks.

  The intensity 1120 is calculated by subtracting the average luminance value of all rectangular blocks labeled 0 from the average luminance value of all rectangular blocks labeled 1.

  For example, in FIG. 13, the feature extraction unit 390 generates an intensity 1152 of strength = 35 for the input face image 1105. For another non-face image 1106, the feature extraction unit 390 generates an intensity 1152 of strength = 37.

  The feature f (X) extracted by the feature extraction unit 390 is a two-dimensional quantity and can be expressed as the following equation (14).

  The extracted feature quantity uses two attributes of pattern and intensity in order to improve the feature quantity identification capability. In many cases, both spatial distribution information and luminance value information are necessary to distinguish images.

  For example, in FIG. 13, the input face image 1105 and the non-face image 1106 have similar strength. Prior art based on strength information cannot distinguish between the two. However, since the face image 1105 and the non-face image 1106 have different pattern values, they can be distinguished by the pattern values.

  FIG. 14 is a diagram showing another example of the rectangular template for feature extraction according to the embodiment of the present invention. Given an image window 1101, the rectangular template includes L rectangular blocks inside the window 1101.

  Like the rectangular block 1091 shown in FIG. 14A, the rectangular blocks may have different sizes. Further, like the rectangular blocks 1092 and 1093 shown in FIGS. 14B and 14C, the rectangular blocks may not intersect or be adjacent to each other. Further, as illustrated in a rectangular block 1094 illustrated in FIGS. 14D and 14E, the blocks may overlap completely or partially.

  According to the object detection device and the learning device thereof according to the embodiment of the present invention, the following configuration is provided.

  First, in the object detection method, the node includes a plurality of discriminators trained by a boosting learning method. The network learning process takes a large number of image samples as input, divides these image sample groups into several sets, and trains a classifier based on each set of images. In order to determine the classifiers to be placed at a node, the building first collects training samples that reach the node in the identification process. Next, a region having the highest identification error and a large number of learning samples is determined. Then, the weak classifier discriminates in the restricted area. Weak classifiers do not affect the entire input space equally. Each newly added weak classifier is characterized and only improves on the result of the previous weak classifier in the determined region. In the network classifier according to the embodiment of the present invention constructed as described above, the classifier of each path identifies a partial space of an input image composed of an image propagated through the path. The non-face determination is easier to separate than the determination in the entire input image space, and there is an effect that a large number of weak classifiers are not required unlike the strong classifier in the latter stage of the conventional example. Thereby, the first problem can be solved.

  In the node network according to the embodiment of the present invention, (1) a strong discriminator that discriminates between a face and a non-face based on a cumulative evaluation value of a node processed until each node reaches the node, (2) At the time of learning, the classifier of each node is trained based on the learning sample that reaches the node and the result of the cumulative evaluation value at the preceding node. In other words, with this method, each node functions as a strong discriminator consisting of nodes existing on the path up to that point, and when it is not sufficient to determine the presence or absence of an object, it is learned according to the discrimination result. It takes a structure in which processing continues at a child node. Therefore, since the identification results so far are always effectively used, the identification process can be realized optimally at high speed. In addition, the first solving means selects classifiers in which each node specializes in a partial region of the input space, while node integration (parent node) which is one of the characteristic structures of this node network. In the presence of multiple child nodes), this integrated structure that takes over the samples of all parent nodes and learns the optimal discriminator in the partial area that integrates the parent node's identification target areas, Identification processing can be performed. Thereby, one point image on the input space is processed by the weak discriminator of a plurality of nodes, and discrimination based on the cumulative value of these results is realized. Thereby, the second problem can be solved.

  Further, in the object detection device from an image, in the identification process, input data can be processed through a plurality of paths on the network. A path is a path of a node used to process and identify input data. The classifier of each node on the path includes not only a weak classifier that performs identification on an input image but also a classifier that accumulates identification results of weak classifiers of nodes up to the previous stage. Therefore, the discrimination result by the strong discriminator or the discriminator can be obtained by using the cumulative score from the weak discriminator of each node on the path. Further, a plurality of paths are further generated from the current path during the identification process. To prevent the number of unnecessary paths from increasing, the path with the highest identification error is terminated. Since nodes are divided and integrated, the network can be viewed as a combination of different strong classifiers sharing a common weak classifier. Creating a strong classifier using nodes on multiple paths is equivalent to running parallel strong classifiers trained to work cooperatively. Among the path identification results, the one with the best identification error value is selected as the output result. In this way, each path operates in a coordinated manner while having a discrimination capability equivalent to or better than that of a detection device consisting of a plurality of cascade discriminators (= a majority discriminating cascade discriminator), and is more computationally intensive than the majority voting method. A discriminator with less can be achieved. Thereby, the third problem can be solved.

  In the object detection apparatus from an image, the feature amount extraction method of the present embodiment measures both spatial distribution information and luminance value information with respect to an input image window. First, the target area in the image window is determined in the feature template. First, as an attribute described as “pattern”, a value representing whether the luminance value of a region is higher or lower than other regions is calculated. This attribute represents a spatial distribution of brightness values. Subsequently, as one attribute denoted as “strength”, a value representing a difference in magnitude between the luminance values of the image window is calculated. That is, the feature selection method proposed in the present embodiment uses two attribute values to obtain useful information from an image. Thereby, the fourth problem can be solved.

  And the target object detection apparatus which concerns on embodiment of this invention, and its learning apparatus contain the following characteristics. The first feature is that the discriminator is localized so as to discriminate in the partial space of the input image. Unlike the conventional method where weak classifiers are trained to identify the entire input space by introducing a network structure, the present embodiment trains weak classifiers that process a partial region of the input space. That is, each classifier performs a specific identification process on a specific area regardless of the identification process of other classifiers. As a result, accuracy is improved and identification errors are reduced, and more complex nonlinear classifiers can be constructed.

  In this embodiment, since the classifier corresponding to the input image is applied, it is not necessary to evaluate all the classifiers. Therefore, the calculation processing cost in the identification process can be reduced. Furthermore, a network in which an input image is processed early in order to guarantee early identification with respect to a simple (easy to identify) input image, whether or not the object is included. The weak classifiers at the upper and outer positions are specialized to process a simple input image.

  The second feature is that the identification result is shared more effectively. In this embodiment, a network structure is used instead of the cascade structure in which weak classifiers are connected in the conventional method. The discriminator is arranged at a node on the network connected from a plurality of nodes to a plurality of nodes. The network partition structure localizes the discriminator to specific area operations, and the integrated structure ensures that the network is tightly connected. In the construction of the connection node, the input space is not over-divided by integrating and using the parent node learning samples. The identification information can be effectively shared by the method of the present embodiment in which the target problem is obtained by dividing and combining, and the total number of classifiers used in the entire object detector can be reduced.

  A third feature is that a plurality of identification results are generated from one classifier. In the apparatus of the present embodiment, the object detection apparatus selects a plurality of weak classifiers in the classifier network for the input image, and performs a classification process using a plurality of paths. Each path is a set of classifiers for identifying an input image. The discriminator dynamically selects to reduce discrimination errors in the discrimination process. When selecting a classifier path to a node in a certain layer, an old path up to the previous stage is used, and a new path is generated while performing node division and integration. Each pass gives a strong discrimination result that determines whether there is an object in the input image. That is, the network itself is actually a set of strong classifiers sharing a plurality of weak classifiers. Compared to the conventional majority voting method in which independent strong classifiers are used, the plurality of strong classifiers of this embodiment are trained collectively and operate in an emphasized manner. Furthermore, by terminating the path of the highest identification error, it is possible to suppress an increase in processing load when a plurality of discriminators that have occurred in the conventional majority method are operated in parallel.

  The fourth feature is the use of a new feature extraction method with higher discrimination power. In the present embodiment, spatial pattern information and non-patent literature 2 and patent literature 1 are compared with the case where a rectangular feature amount in the input image window measures strength information. A rectangular pattern feature that can measure both intensity information is proposed. Spatial pattern information represents how the high and low luminance pixels are arranged in an area. The strength information represents how strong the difference in luminance value is between the pixel regions with high and low luminance. Compared to the method described in Non-Patent Document 4, the rectangular pattern feature of this embodiment is robust to noise images by incorporating strength information, and the pattern feature of this embodiment is the pixel feature described in Non-Patent Document 4. It is possible to acquire a global feature compared to the comparison. Accordingly, the discriminating power becomes higher, and the number of features used to find the target object used in the detector can be greatly reduced.

  The object detection device and the learning device thereof according to the present invention have an effect capable of suppressing an increase in processing load with high accuracy, and are useful for detecting an object photographed by a monitoring camera or the like.

DESCRIPTION OF SYMBOLS 201 Input part 202 Output part 210 Image window extraction part 390 Feature extraction part 502 Storage part 520 Path generation part 530 Image window detection processing part 590 Network identifier 710 Empty node generation part 720 Image sample collection part 730 Node learning part 790 Network learning part

Claims (7)

Set a rectangular template consisting of multiple rectangular blocks for the image,
Obtaining a reference luminance value based on a luminance value of a rectangular block included in the rectangular template and an individual luminance value for each of the rectangular blocks;
From the magnitude relationship between the reference luminance value and the individual luminance value for each of the rectangular blocks, obtain spatial distribution information for identifying the rectangular block,
An object detection device comprising: a feature extraction unit that determines whether an image in the rectangular template includes an object using the spatial distribution information. The object detection device according to claim 1,
The plurality of rectangular blocks are object detection devices having different sizes. The object detection device according to claim 1 or 2,
The feature extraction unit includes:
When the individual luminance value of each rectangular block is equal to or greater than the reference luminance value, a first label is assigned to the rectangular block,
When the individual luminance value of each rectangular block is less than the reference luminance value, a second label is assigned to the rectangular block,
The spatial distribution information is an object detection device configured by the first label or the second label assigned to each of the rectangular blocks. The object detection device according to claim 3,
The feature extraction unit includes:
The intensity is obtained based on the difference between the average luminance value of the individual luminance values of all rectangular blocks to which the first label is assigned and the average luminance value of the individual luminance values of all rectangular blocks to which the second label is assigned. ,
An object detection apparatus for determining whether an image in which the rectangular template is set using the spatial distribution information and the intensity includes an object. The object detection device according to any one of claims 1 to 4,
The rectangular block is an object detection device separated from each other. The object detection device according to any one of claims 1 to 4,
The object detection device in which the rectangular blocks overlap each other. By the feature extraction unit
Setting a rectangular template composed of a plurality of rectangular blocks for the image;
Obtaining a reference luminance value from the luminance values of all the rectangular blocks and an individual luminance value from each of the rectangular blocks;
Obtaining spatial distribution information for identifying the rectangular block from the magnitude relationship between the reference luminance value and the individual luminance value for each of the rectangular blocks;
Using the spatial distribution information to determine whether an image in the rectangular template includes an object;
An object detection method comprising:

Images