JP2012128623A - Object discrimination device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid duplicate difference calculation in an object discrimination device.SOLUTION: A discriminator 13 includes plural weak discriminators each calculating a difference between at least one pair of two points in an input image, and calculating a score related to existence of a detection target object based on the calculated difference. In the discriminator 13, the plural weak discriminators are cascade-connected. Difference image generation means 15 sets a displacement amount corresponding to a positional relationship between two points in an image between which a difference is to be calculated by a weak discriminator, and generates a difference image between the input image and an image obtained by displacing the input image by the set displacement amount. At least a part of the plural weak discriminators refer to the difference image to acquire the difference, and calculate the score.

Description

  The present invention relates to an object determination device, method, and program, and more particularly, to an object determination device, method, and program for determining whether or not an object to be detected is included in an image.

  Various methods for detecting a predetermined object (object) such as a face from a digital image such as a photographic image using a computer such as a computer have been proposed. As a method for detecting an object from an image, for example, a template matching method that has been used for a relatively long time is known. In recent years, a classifier is configured using a machine learning technique called boosting, and a technique for detecting an object from an image using the classifier has attracted attention. Learning of a discriminator using boosting and object detection using the discriminator are described in Patent Document 1 and Patent Document 2, for example.

  In general, a discriminator generated by boosting learning has a plurality of, for example, hundreds to thousands of weak discriminators. A plurality of weak classifiers are connected in series (cascade connection) to form one classifier (strong classifier). In general, weak classifiers are defined as classifiers that have some correlation with the true classification. Each weak discriminator calculates a feature amount and obtains a score based on the feature amount. The strong classifier performs threshold processing on the total score obtained by all the cascaded weak classifiers with a predetermined threshold value. It is determined that the object of appears.

  The feature amount calculation in the weak classifier is based on the difference in pixel value between two points (two regions). Each weak discriminator performs a difference calculation using one of a plurality of basic feature types related to the difference calculation, and obtains a score related to the presence of the detection target from the input image. The basic feature types related to the difference calculation are, for example, a difference between two points in the template such as a difference between two points arranged in the horizontal direction, a difference between two points arranged in the vertical direction, and a difference between two points arranged in the diagonal direction. It can be defined by positional relationship. In some cases, the basic feature type has a positional relationship between two points in a plurality of pairs (2 pairs, 3 pairs,...), And the weak classifier calculates a feature amount according to the combination. In the case of 2 pairs, 4 points are referenced, and in the case of 3 pairs, 6 points are referenced.

  The object detection device, for example, performs a raster scan of a 32 × 32 pixel template (window) in units of one pixel or two or more pixels on an image to be detected of 640 × 480 pixels, and is cut out at each position of the template. The partial image is given to the strong classifier. The strong discriminator performs discrimination by the weak discriminator sequentially from the first stage side, and performs threshold processing on the total score of each weak discriminator when the final stage is reached. When the total score is equal to or greater than the threshold value, the strong discriminator outputs that a detection target object appears at a position of 32 × 32 pixels cut out by the template.

JP 2007-47965 A JP 2007-128127 A

  Normally, in the strong classifier, each weak classifier performs threshold processing on the score up to that stage, and when the score is lower than the threshold, an early reject that ends the process without performing the subsequent weak classifier processing. A decision (early reject decision) is made. By performing early rejection (early termination), processing is completed at a relatively early stage among thousands of weak classifiers connected in series for images that clearly show that objects to be detected are not included. Therefore, the processing speed can be increased as compared with the case where the processing is performed up to the weak classifier at the final stage. As described in Patent Documents 1 and 2, in general, weak classifiers generated by learning are linearly combined in descending order of weighted correct answer rate to constitute one strong classifier. In other words, a strong discriminator is configured by connecting a plurality of discriminators generated by learning in series in order effective for discrimination.

  By the way, in recent years, a technique for estimating the approximate position and size of an object to be detected at high speed has been developed. Consider that this technique is used as pre-processing for a strong classifier, and an image of an area extracted in the pre-processing is used as a processing target image for the strong classifier. In that case, since most of the areas extracted in the pre-processing are considered to be the areas of the detection target object, it is unlikely that the weak classifier ends early, and in most cases the weak classifier It is thought that the process will proceed to near the last stage. Therefore, even if the early termination is performed, the effect of increasing the processing speed is not significant. Rather, if each weak discriminator determines early termination (conditional branch processing), the pipeline processing is disturbed (hazard), which hinders the processing speed.

  The idea of early termination is based on a large gap between the detection target object and the background area ratio. That is, prior knowledge (assuming) that most of the image is the background area and the number of detection target objects is small. In a general processing system, processing can be speeded up by early termination determination. However, the present inventor has found that, when the strong classifier process is performed on a portion where the probability that an object exists is high, the early termination determination is an impediment to speedup as described above.

  If the early termination is not performed, it is necessary to perform difference calculation every time, for example, in thousands of weak classifiers cascaded. Considering that the template is scanned in the image, two points of pixels (pixels in the image around the portion where the object is estimated to be present) calculated by the difference in the weak classifier at the position of the previous template are In some cases, the difference calculation is performed by another weak classifier at different template positions. For example, one weak classifier A calculates the difference between the position (x, y) and the position (x + 1, y) in relative coordinates in the template, and another weak classifier B is (x + 3, y + 3). Consider a case where the difference between the position and the position of (x + 4, y + 3) is calculated. If the template moves by +3 in the x direction and +3 in the y direction, the relative coordinate position (x, y) after the movement overlaps with the relative coordinate position (x + 3, y + 3) at the template position before the movement. Although the weak classifier B has already calculated the difference between the position (x + 3, y + 3) and the position (x + 4, y + 3) at the position of the template before the movement, the weak classifier A The same difference calculation must be repeated. As described above, when the template is moved, it is necessary to repeatedly perform the same difference calculation in individual weak classifiers, and the processing time is unnecessarily increased by the amount that the same difference calculation must be repeated.

  In view of the above, an object of the present invention is to provide an object discriminating apparatus, method, and program capable of avoiding overlapping difference calculations and accelerating the process accordingly.

  In order to achieve the above object, the present invention obtains a plurality of weak discriminations, each of which obtains a difference between at least one pair of two points in an input image and obtains a score related to the presence of a detection object based on the obtained difference. A shift amount corresponding to the positional relationship between two points of the strong classifier in which the units are cascade-connected and the image whose difference is to be obtained by the weak classifier, and the set shift amount between the input image and the input image Difference image generation means for generating a difference image with the image shifted by at least one of the plurality of weak classifiers, the difference between the at least one set of two points with reference to the difference image Is obtained, and the score is obtained.

  The strong discriminator may scan the template in the input image in a predetermined scanning order, and execute processing in the plurality of weak discriminators for each position of the scanned template.

  In the present invention, the difference image generation means sequentially sets a shift amount according to a positional relationship between a plurality of two points from which a difference is to be obtained by the plurality of weak discriminators, and a plurality of shift amounts corresponding to the shift amounts. A difference image can be generated.

  In the present invention, the difference image generation means sequentially sets a shift amount according to the positional relationship between all the two points for which differences should be obtained by the plurality of weak classifiers, and each of the plurality of weak classifiers. However, the structure which acquires the difference between said at least 1 set of 2 points | pieces with reference to the said difference image is employable.

  The strong discriminator may execute processing in each weak discriminator without early termination from the first stage to the final stage of the cascaded weak discriminators.

  Each of the plurality of weak classifiers obtains a difference between the at least one set of two points in any of a plurality of basic feature types related to difference calculation, and the strong classifier has the same basic feature type as the weak classifier. A configuration may be adopted in which are continuously arranged.

  In the strong discriminator, when there are a plurality of weak discriminators having the same basic feature type, the plurality of weak discriminators having the same basic feature type are arranged according to the reference position of the image in the difference calculation in each weak discriminator. It can be set as the structure arranged in order.

  Instead of the above, the strong discriminator may adopt a configuration in which the weak discriminators are arranged in the arrangement order according to the reference position of the image at the time of difference calculation in each weak discriminator.

  In the object discriminating apparatus of the present invention, the position of the object is estimated from the processing target image, an image around the estimated object position is cut out, and the cut out images are respectively sent to the strong discriminator and the difference image generating unit. It is possible to employ a configuration further comprising object candidate point detection means provided as the input image.

  A smoothing process in which the object candidate point detection unit repeatedly performs a process of convolving a smoothing filter having a filter characteristic corresponding to the contour shape of the object into an image, and generates a plurality of smoothed images having different scales from the frame image Means, a difference image generating means for generating a plurality of difference images between two smoothed images having different scales among the plurality of smoothed images, while changing the scale, and the plurality of difference images. Summing means for summing and generating a summed image, position estimating means for estimating the position of an object based on pixel values in the summed image, and a partial image for cutting out an image of a region around the estimated position from the frame image It is good also as a structure containing a production | generation means.

The smoothing processing means performs a × k smoothed images L (x, y, σ _i ) (i = ₁ to _{a ×)} from the scale σ ₁ to σ _{a × k} (a and k are integers of 2 or more). k), and the difference image generation means smoothes k difference images G (x, y, σ _j ) (j = 1 to k) from the scales σ ₁ to σ _k , respectively, on the scale σ _j . of the image _{L (x, y, σ j} ) the scale sigma _{j × a} smoothed image _{L (x, y, σ j} × a) may be generated based on the difference between.

Instead of the above, the smoothing processing means outputs r smoothed images L (x, y, σ _i ) (i = ₁ to _r ) from the scale σ ₁ to σ _r (r is an integer of 3 or more). The difference image generation means generates kp difference images G (x, y, σ _j ) (j = 1 to k) from the scale σ ₁ to σ _k−p (p is an integer of 1 or more). the -p), the smoothed image L (x, respectively scale σ _j, y, σ _j) the scale sigma _{j + p} of the smoothed image L (x, y, even be generated based on the difference between the σ _{j + p)} Good.

  The present invention also includes a step of cascading a plurality of weak discriminations, each of which obtains a score related to the presence of a detection object based on a difference between at least one set of two points in the input image, and the input image; Generating a difference image between the input image and an image shifted by a shift amount corresponding to a positional relationship between two points of an image whose difference should be obtained by the weak discrimination, and the plurality of weak discriminations are cascaded. In at least a part of the steps to be executed, an object discrimination method is provided, wherein a difference between the at least one set of two points is obtained with reference to the difference image and the score is obtained.

  Furthermore, the present invention provides a procedure for executing, in cascade, a plurality of weak discriminations, each of which obtains a score related to the presence of a detection object based on a difference between at least one set of two points in an input image, and the input image. And a step of generating a difference image between the input image and an image shifted by a shift amount according to a positional relationship between two points of the image from which the difference is to be obtained by the weak discrimination. Provided is a program for acquiring a difference between the at least one set of two points with reference to the difference image and executing the procedure for obtaining the score in at least a part of the procedure of performing a plurality of weak discriminations in cascade To do.

  The object discriminating apparatus, method, and program according to the present invention generate a difference image between an input image and an image obtained by shifting the input image by a shift amount corresponding to the positional relationship between two points of an image whose difference should be obtained by weak discrimination. In the weak discrimination, the difference is acquired with reference to the difference image. When the difference calculation is performed individually in each weak discrimination, the difference at the same position of the input image may be calculated before and after the template movement. In the present invention, since the difference can be acquired by referring to the difference image, it is possible to avoid that the difference calculation is repeatedly executed in a plurality of weak discriminations, and the processing speed can be increased.

1 is a block diagram illustrating an object determination device according to a first embodiment of the present invention. The block diagram which shows the structure of a discriminator. The flowchart which shows the operation | movement procedure of an object discrimination device. The flowchart which shows the procedure of a difference image generation. The block diagram which shows the structural example of an object candidate point detection means. The flowchart which shows the operation | movement procedure of an object candidate point detection means. (A) And (b) is a figure which illustrates a basic feature type. The block diagram which shows the discriminator used with the object discrimination device of 2nd Embodiment of this invention. The block diagram which shows the discriminator structure apparatus used for the structure of a discriminator. (A) is a block diagram showing a discriminator after learning, (b) is a block diagram showing a discriminator after rearrangement. (A) is a block diagram showing the order of weak classifiers in basic feature type 1, and (b) is a diagram showing the reference positions of the images of the weak classifiers in the template. The block diagram which shows the discriminator structure apparatus used for the structure of the discriminator in 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an object discriminating apparatus according to a first embodiment of the present invention. The object discrimination device 10 includes an image input unit 11, an object candidate point detection unit 12, a discriminator 13, a lookup table 14, and a difference image generation unit 15. The function of each unit in the object determination device 10 can be realized by, for example, a computer (processor) executing processing according to a predetermined program. The object determination device 10 is incorporated in, for example, a camera and determines whether or not an object to be detected exists in an image to be photographed by the camera.

  The image input unit 11 inputs an image to be processed. For example, the image input unit 11 inputs an image of 640 × 480 pixels as a processing target image. For example, the image input unit 11 may sequentially input each image (image of each frame) constituting the moving image at a predetermined rate. The object candidate point detection means 12 estimates the approximate position of the detection target object from the processing target image using a predetermined algorithm. Further, the object candidate point detection means 12 estimates the size of the object. The object candidate point detection unit 12 cuts out an image around a position where an object is estimated to exist from the processing target image, and enlarges / reduces the cut-out image according to the estimated size. The image input unit 11 performs predetermined image processing such as noise removal and suppression of luminance fluctuation between frames on the input processing target image, and the processed image is subjected to object candidate point detection unit 12. You may make it input into.

  The discriminator 13 inputs from the object candidate point detection means 12 an image around the position where the object is estimated to be present, which is extracted by the object candidate point detection means 12. Each of the discriminators 13 includes a plurality of weak discriminators that obtain a score related to the presence of the detection target from the input image. A classifier (strong classifier) 13 is configured by cascading a plurality of weak classifiers. The discriminator 13 performs threshold processing on the total score obtained by each weak discriminator, and discriminates whether or not an object to be detected exists in the input image.

  When the size of the input image is larger than the size of the template, the discriminator 13 raster-scans the template in the input image, cuts out an image corresponding to the size of the template from the input image, and sequentially weakens the cut-out images. Give to the classifier. The discriminator 13 executes processing in a plurality of weak discriminators for each position of the scanned template. The discriminator 13 executes processing in each weak discriminator without early termination from the first stage to the final stage of the cascaded weak discriminators.

  FIG. 2 shows the configuration of the discriminator 13. The discriminator 13 includes a plurality of weak discriminators 16 connected in cascade. Each weak discriminator 16 obtains a difference between at least one set of two points in the input image, and obtains a score related to the presence of the detection target based on the obtained difference. The discriminator 13 is generated using machine learning using an image when an object to be detected exists in an image having a template size, for example, a size of 32 × 32 pixels, and an image when the object to be detected does not exist. The Each weak discriminator 16 obtains a difference between at least one set of two points in any of a plurality of basic feature types related to difference calculation. It is determined in the learning process by which basic feature type each weak discriminator 16 obtains the difference.

  The difference obtained by the weak discriminator 16 may be a pixel value difference between two pixel positions or a pixel value difference between two regions. The difference between the pixel values between the regions may be a total difference between the pixel values within the region, or may be a difference between the average values of the pixel values within the region. Each weak discriminator 16 obtains a score based on the difference. Each weak discriminator 16 adds the score obtained by itself to the total score up to the weak discriminator 16 in the previous stage and passes it to the weak discriminator 16 in the next stage. This process is performed up to the weak discriminator 16 at the final stage, and the finally obtained score becomes a score relating to the presence of the detection target object in the discriminator 13.

  Returning to FIG. 1, the difference image generation means 15 determines the input image to the discriminator 13 and the input image according to the positional relationship between two points of the image whose difference is to be obtained by the weak discriminator 16 (FIG. 2). A difference image from the image shifted by the shift amount is generated. The difference image generation means 15 sequentially sets a plurality of shift amounts corresponding to the positional relationship between two points at which differences are to be obtained by several weak classifiers 16, and generates a plurality of difference images corresponding to each shift amount. To do. At least a part of the plurality of weak classifiers 16 connected in cascade acquires a difference by referring to the difference image, and obtains a score based on the acquired difference.

  The look-up table 14 holds the relationship between the feature space obtained by the difference calculation in the weak discriminator 16 and the score regarding the presence of the object to be detected. The lookup table 14 is generated for each basic feature type, for example, when the discriminator 13 learns. Each weak classifier 16 refers to a lookup table prepared for its basic feature type based on the obtained difference, and obtains a score from the obtained difference. For example, when the weak discriminator 16 calculates three sets of differences (see 6 points), the weak discriminator 16 sets the first set of difference values to α, the second set of difference values to β, and the third set. And (α, β, γ) is obtained as a feature space. The weak classifier 16 refers to the array element [α] [β] [γ] of the lookup table, and acquires the value stored in the array element as a score.

  FIG. 3 shows an operation procedure of the object discrimination device 10. The image input unit 11 inputs an image to be processed (step A1). The object candidate point detection means 12 generates an image around the position where the object in the processing target image is estimated to exist as a partial image (step A2). A specific configuration example and procedure of the object candidate point detection unit 12 will be described later. The object candidate point detection means 12 generates, for example, an image having a size larger than the template size as a partial image.

  The difference image generation means 15 generates a difference image based on the partial image generated by the object candidate point detection means 12 (step A3). FIG. 4 shows a difference image generation procedure. The difference image generation means 15 inputs from the object candidate point detection means 12 a partial image generated by the object candidate point detection means 12, that is, an image around a position where an object is estimated to exist (step B1).

  The difference image generation means 15 sets the relative shift amount of the image (step B2). In step B2, the difference image generation means 15 sets the difference pitch to be calculated by the weak discriminator 16 as a deviation amount. In other words, the weak discriminator 16 sets a shift amount corresponding to the positional relationship between two points of the image whose difference is to be obtained. The difference image generation means 15 obtains a difference between each pixel of the image input in step B1 and each pixel of the image obtained by shifting the input image by the shift amount set in step B2, and uses the difference value as a pixel value. A difference image is generated (step B3).

  The difference image generation means 15 determines whether or not the deviation amount corresponding to the positional relationship between all two points for which the difference is to be obtained by the weak classifier 16 constituting the classifier 13 is set in step B2 (step B4). ). When the difference image generation unit 15 determines that all the shift amounts have been set, the generation of the difference image ends. When the difference image generating means 15 determines that there is an unset deviation amount, the difference image generating means 15 returns to step B2 and sets one of the unset deviation amounts. The difference image generation means 15 repeatedly executes steps B2 to B4, so that the weak discriminator 16 sequentially sets a shift amount according to the positional relationship between a plurality of two points for which a difference is to be obtained. A plurality of difference images corresponding to are generated.

  Returning to FIG. 3, the discriminator 13 sets the initial position of the template (step A4). The discriminator 13 determines, for example, the origin (0, 0) of the partial image generated by the object candidate point detection means 12 that is an input image as the initial position of the template. The discriminator 13 cuts out an image of the template size from the input image (step A5), gives the cut-out image to the first stage of the cascade-connected weak discriminator 16, and starts processing by the weak discriminator 16.

  The weak classifier 16 acquires at least one set of differences between two points from the difference image generated in step A3 (step A6). In step A6, the weak discriminator 16 refers to the difference image generated with the shift amount corresponding to the positional relationship between the two points of the image for which the difference is to be obtained among the plurality of difference images generated in step A3. Get the difference. The weak discriminator 16 obtains a score by referring to the lookup table 14 based on the obtained difference (step A7). The weak classifier 16 adds the score obtained in step A7 to the score up to the weak classifier 16 in the previous stage (step A8).

  The discriminator 13 determines whether or not processing has been performed up to the final stage of the cascade-connected weak discriminator 16 (step A9). If the final stage has not been reached, the process proceeds to the next stage (step A10), the process returns to step A6, and the process by the weak classifier 16 at the next stage is continued. In the discriminator 13, the weak discriminator 16 at each stage executes Step A6 to Step A8, and the process proceeds to the next stage. When the discriminator 13 determines that the processing has been completed up to the weak discriminator 16 in the final stage, the total of the scores obtained in the final stage is compared with a predetermined threshold (step A11). If the total score is equal to or greater than the threshold value, the discriminator 13 determines that the object to be detected appears in the image portion cut out in step A5, and outputs the position of the object (step A12). Exit.

  If the discriminator 13 determines in step A11 that the total score is smaller than the threshold value, it determines that no object exists at the current template position, and moves the position of the template (step A13). After moving the template, the process returns to step A5, and the image is cut out from the input image at the position of the moved template. The discriminator 13 gives the clipped image to the first stage of the cascade-connected weak discriminator 16, and executes steps A6 to A10 to determine whether or not the object to be detected exists at the position of the template after movement. Perform the determination. The discriminator 13 repeatedly executes Step A5 to Step A11 and Step A13 until the template reaches the final scan position, and discriminates whether or not the object appears in the partial image generated by the object candidate point detection means 12. .

  In the procedure shown in FIG. 4, the difference image generation unit 15 generates a difference image with a deviation amount corresponding to the positional relationship between all two points for which the difference should be obtained by the weak classifier 16 constituting the classifier 13. Although described as a thing, it is not limited to this. For example, a difference image does not need to be generated for a low frequency of use according to the frequency of use of the positional relationship between two points. The weak discriminator 16 may calculate the difference value from the input image instead of referring to the difference image when there is no difference image corresponding to the positional relationship between the two points for which the difference is to be obtained.

  Next, a specific configuration example of the object candidate point detection unit 12 will be described. FIG. 5 shows a configuration example of the object candidate point detection unit 12. The object candidate point detection unit 12 includes a preprocessing unit 21, a smoothing processing unit 22, a difference image generation unit 23, a summation unit 24, a position estimation unit 25, a size estimation unit 26, and a partial image generation unit 27. The object candidate point detection unit 12 cuts out a specific pattern in the image, for example, an image around a position where a human head is estimated to exist as a partial image. In the following description, it is assumed that the object candidate point detection means 12 estimates one position where an object is estimated to exist and cuts out the surrounding image as a partial image.

  The preprocessing unit 21 includes a resolution conversion unit 51 and a motion region extraction unit 52. The resolution conversion means 51 lowers the frame image constituting the moving image to a predetermined resolution. The resolution conversion means 51 converts, for example, the resolution of an image to 1/8 times in the vertical and horizontal directions.

  The motion region extraction unit 52 extracts a motion region from the frame image constituting the motion image and generates a motion region extraction image. For extracting the motion region, for example, an arbitrary method such as calculating a difference between the background image and the inter-frame image can be used. The motion region extraction means 52 is a grayscale image in which the region with motion is white (the tone value is high) and the region with less motion is black (the tone value is low) according to the amount of motion extracted. Is generated as a motion region extraction image. For example, the motion region extraction unit 52 may perform a contrast reduction process for converting the gray scale according to a predetermined function on a grayscale image having 256 gray scales and reducing the gray scale from white to black. The motion area extraction unit 52 may generate a binarized image in which the motion area is white and the background area is black instead of the grayscale image as the motion area extraction image.

The smoothing processing unit 22 is input with the image P (x, y) pre-processed by the pre-processing unit 21, that is, the image whose resolution is reduced and the motion region is extracted. The smoothing processing means 22 generates a plurality of smoothed images L (x, y, σ _i ) having different scales by repeatedly performing a process of convolving the smoothing filter into the image.

The smoothing processing means 22 first generates a smoothed image L (x, y, σ ₁ ) by convolving a smoothing filter with the image P (x, y), and the smoothed image L (x, y, σ). ₁ ) is further convolved with a smoothing filter to generate a smoothed image + (x, y, σ ₂ ) of scale σ ₂ . The smoothing processing means 22 repeats the convolution of the smoothing filter in the same manner thereafter, and from the smoothed image L (x, y, σ _q ) of an arbitrary scale σ _{q to} the smoothed image L (x of the next scale σ _{q + 1} , Y, σ _{q + 1} ).

The scale number i in the smoothed image L (x, y, σ _i ) corresponds to the number of times the smoothing filter is convoluted. The smoothing processing means 22 is, for example, a × k images (a and k are integers of 2 or more) of different scales L (x, y, σ ₁ ) to L (x, y, σ _{a × k).} ) Is generated. For example, if a = 2 and k = 30, the smoothing processing unit 22 generates 2 × 30 = 60 smoothed images L (x, y, σ ₁ ) to (x, y, σ ₆₀ ).

  As the smoothing filter, for example, a Gaussian filter can be used. The smoothing filter is composed of, for example, a 3 × 3 operator having a filter characteristic that matches the contour shape of the object to be detected. For example, if the object to be detected by the discriminator 13 (FIG. 1) is a human head, the smoothing filter has a characteristic in which the lower filter coefficient decreases along the contour shape of the human head (omega shape). ) Is used. By using such a smoothing filter, it is possible to realize a smoothing process in which a region having a contour shape of a person's head is emphasized and other regions are suppressed.

  The shape of the filter is not limited to the omega shape, and other known techniques such as those described in Japanese Patent Application Laid-Open No. 2003-248824 can be applied. For example, when the object to be detected has a circular shape, a triangular shape, a quadrangular shape, or the like, the smoothing process may be performed using a smoothing filter having a filter characteristic matched to each object shape.

The difference image generation means 23 receives a plurality of smoothed images L (x, y, σ _i ) generated by the smoothing processing means 22 and a difference image G (x between two smoothed images having different scales. , Y, σ _j ) are generated while changing the scale. Here, the maximum value of the scale number j in the difference image G (x, y, σ _j ) is smaller than the maximum value (for example, a × k) of the scale σ _i in the smoothed image L. The difference image generation unit 23 generates a difference image between smoothed images separated by a scale corresponding to the scale number j, for example. Specifically, the difference image generation means 23 can generate the difference image G (x, y, σ _j ) using, for example, the following formula 1.
G (x, y, σ _j ) = L (x, y, σ _j ) −L (x, y, σ _{j × a} ) (1)
The difference image may be an absolute value of the difference value.

As can be seen from the definition of formula 1 above, the difference image _{G (x, y, σ j} ) has a smoothed image of the scale sigma _j, it is defined as the difference between the smoothed image of the scale σ _{j × a.} For example, the a = 2, k = 30, the difference image generation unit 23, the scale sigma ₁ and sigma _2, scale sigma ₂ and sigma _4, scale sigma ₃ and sigma _6, · · ·, scale sigma ₃₀ and sigma ₆₀ of Thirty differential images G (x, y, σ ₁ ) to (x, y, σ ₃₀ ) composed of combinations are generated. When the difference image G (x, y, σ _j ) is generated according to Equation 1, j takes a value from 1 to k. That is, the difference image generation unit 23 generates k difference images G (x, y, σ ₁ ) to (x, y, σ _k ).

Instead of the above, the difference image generation means 23 may generate a difference between smoothed images separated by a certain scale as a difference image. The difference image generating means 23 generates, for example, a difference between a smoothed image of scale σ _{j and} a smoothed image of scale σ _{j + p} (p is an integer of 1 or more) as a difference image (x, y, σ _j ). Also good. Specifically, the difference image generation means 23 may generate the difference image G (x, y, σ _j ) using the following formula 2.
G (x, y, σ _j ) = L (x, y, σ _j ) −L (x, y, σ _{j + p} ) (2)
In this case, if the number of smoothed images is r (r: an integer of 3 or more), j takes a value of 1 to rp. That is, the difference image generation unit 23 generates rp difference images G (x, y, σ ₁ ) to (x, y, σ _r−p ). Specifically, in the case of r = 60 and p = 30, the difference image generation unit 23 determines that the scales σ ₁ and σ ₃₁ , the scales σ ₂ and σ ₃₂ , the scales σ ₃ and σ ₃₃ ,. ₃₀ differential images G (x, y, σ ₁ ) to (x, y, σ ₃₀ ) composed of combinations of ₃₀ and σ ₆₀ are generated.

The summing unit 24 adds the plurality of difference images G (x, y, σ _j ) generated by the difference image generating unit 23 to generate a combined image AP (x, y). The position estimation unit 25 estimates the position of the object based on the pixel value in the combined image AP (x, y). For example, the position estimating unit 25 checks the position where the pixel value (the sum of the difference values) is the largest in the combined image AP (x, y), and estimates the position as the position of the object.

The size estimation means 26 compares the pixel values of a plurality of difference images G (x, y, σ _j ), and estimates the size of the object to be detected based on the scale of the difference image having the maximum pixel value. . For example, the size estimating unit 26 determines the object based on the scale in the smoothed image having the smaller scale of the two smoothed images that are the generation sources of the difference image having the maximum pixel value (difference value). Estimate the size. That is, the size estimation unit 26 obtains the scale σ _j having the maximum difference value among the plurality of difference images G (x, y, σ _j ) generated according to the expression 1 or 2, and the obtained scale σ The position of the object is estimated based on _j .

The estimation of the position and size of the object will be described. The smoothing processing means 22 generates a smoothed image L (x, y, σ _i ) using a smoothing filter having a filter characteristic matched to the object shape, and this smoothed image L (x, y, σ _i ) is an image in which a region having a specific shape is emphasized and other regions are suppressed. For example, even when the smoothing process is performed several tens of times, the contour component of the object remains in the smoothed image, but as the scale σ _i increases, the area of the object blurs and expands.

It is assumed that the shape and size of the object in the smoothed image L (x, y, σ _i ) match the shape and size of the object in the input image, respectively. In order to calculate the saliency of the object shape and size in the smoothed image L (x, y, σ _i ), a smoothed image having a scale larger than that scale is used as a background for the smoothed image of a certain scale. Set as. That scale sigma _j of the smoothed image _{L (x, y, σ j} ) for sets smoothed image L of formula 1 in the scale _{σ j × a (x, y} , σ j × a) as the background image, In Equation 2, a smoothed image L (x, y, σ _{j + p} ) of scale σ _{j + p} is set as the background. Then, according to Equation 1 or Equation 2, the scale sigma difference image G of the smoothed image to be set as _j of the smoothed image and the background image (x, y, σ _j) are scaled sigma _j of the smoothed image L (x , Y, σ _j ) as the saliency of the object. In this way, the saliency of the object is digitized by the difference image generation means 23, and the position estimation means 25 and the size estimation means 26 based on the saliency of the object quantified by the difference image generation means 23. Estimate each size.

Here, the difference image in which the object has an ideal shape in the image, that is, the shape that most closely matches the filter characteristics and has no noise in the background, has the maximum signal compared to the other difference images. In other words, when the component of each pixel constituting the object in the preprocessed image P (x, y) spreads to be approximately equal to the object region, the difference image G (x, y, σ _j ) The difference value is the maximum. For example, when an object in the image P (x, y) is composed of circular pixels having a diameter of 10 pixels, among the plurality of difference images, a difference image G (x, y, σ ₁₀ ) (equation 1) where j = 10. In L (x, y, σ ₁₀ ) −L (x, y, σ _{a × 10} ), and in Equation 2, the difference value in L (x, y, σ ₁₀ ) −L (x, y, σ _{10 + p} )) is Therefore, it has a larger value than the difference value in other difference images.

On the other hand, the object actually reflected in the image differs in the way it is reflected according to the positional relationship between the camera and the object, individual differences, and the like, and the contour shape and size of the object are not necessarily ideal. That is, the contour shape and size of the object vary. Therefore, the position estimation unit 25 estimates the position of the object using the combined image AP (x, y) obtained by adding a plurality of difference images G (x, y, σ _j ). By doing so, it is possible to estimate the position of the object while absorbing the variation of the object. That is, by detecting the maximum value from the summed image AP (x, y) obtained by adding the smoothed image to the object having various outline shape fluctuations included in the large object from the small object, Position estimation can be performed while absorbing fluctuations.

As described above, the scale number j in the expressions 1 and 2 is a parameter corresponding to the size of the object to be detected in the image P (x, y). When the object size is small, the maximum value is detected from the difference image G (x, y, σ _j ) having a small scale number j. When the object size is large, the difference image G (x, The maximum value is detected from y, σ _j ). Using this property, the size estimation means 26 compares the difference values among a plurality of difference images, and calculates the size of the object from the scale number of the difference image that becomes the maximum difference value, that is, the number of repetitions of the smoothing process. presume.

  The partial image generation unit 27 inputs the position of the object estimated from the position estimation unit 25 and inputs the size of the object estimated from the size estimation unit 26. The partial image generating means 27 cuts out an image around a position where an object is estimated to exist from the input image (frame image) as a partial image. Further, the partial image generation means 27 enlarges / reduces the cut out partial image at a magnification according to the estimated size. By enlarging / reducing by a magnification according to the estimated size, a change in the size of the object can be absorbed.

  FIG. 6 shows an operation procedure of the object candidate point detection means 12. The preprocessing unit 21 receives the frame image from the image input unit 11 (FIG. 1) and performs preprocessing on the frame image (step C1). That is, the resolution conversion means 51 lowers the frame image to a predetermined resolution, and the motion area extraction means 52 extracts a motion area from the reduced resolution frame image. The pre-processing means 21 smoothes the pre-processed image, that is, the image P (x, y) gray-scaled so that the resolution is reduced and the motion area is white and the background area is black. 22 is input. Note that either one or both of resolution conversion and motion region extraction in the preprocessing unit 21 may be omitted. When both are omitted, the frame image may be input to the smoothing processing means 22.

The smoothing processing means 22 receives the image P (x, y), and repeats the process of convolving the smoothing filter with the image P (x, y), whereby a plurality of smoothed images L (x, y with different scales) are obtained. , Σ _i ) (step C2). The smoothing processing unit 22 may convolve a smoothing filter with the frame image itself. The difference image generation unit 23 calculates a difference between two smoothed images having different scales, and generates a difference image G (x, y, σ _j ) (step C3). The difference image generation means 23 uses, for example, Equation 1 to calculate k difference images G (x, y, σ) having scale numbers 1 to k from a × k smoothed images L (x, y, σ _i ). ₁ ) to (x, y, σ _k ) are generated. Alternatively, the difference image generation unit 23 uses the equation 2 to calculate rp difference images G (x, y) having scale numbers 1 to rp from r smoothed images L (x, y, σ _i ). , Σ ₁ ) to (x, y, σ _rp ).

The summing unit 24 adds the plurality of difference images generated by the difference image generating unit 23 to generate a summed image AP (x, y) (step C4). For example, the summing unit 24 adds all the pixel values of the _k difference images G (x, y, σ ₁ ) to (x, y, σ _k ) generated by the difference image generating unit 23. The position estimation means 25 estimates the position where the object exists based on the combined image AP (x, y) (step C5). The position estimation unit 25 compares, for example, pixel values (summation values of differences) of pixel positions constituting the summed image AP (x, y), and uses the pixel position having the maximum pixel value in the summed image as the position of the object. presume.

Note that the summing unit 24 does not have to sum all the difference images. For example, the summing unit 24 may sum any number of difference images of all k pieces of difference images and any number of difference images. For example, the summing unit 24 may change the number of difference images (the scale of the difference image to be summed) used for the addition processing in accordance with the size fluctuation range to be absorbed. For example, the size fluctuation range to be absorbed is set according to the type of the object to be detected, and for a certain object, the difference number G (x, y, Even if the difference images G (x, y, σ ₃ ) to (x, y, σ _k ) of scale numbers ₃ to _k are added together by excluding σ ₁ ) and (x, y, σ ₂ ) from the addition. Good. Further, the summing unit 24 may sum the difference images (x, y, σ _j ) from the scale number 1 to an arbitrary scale number smaller than k.

The size estimation means 26 estimates the size of the object based on the plurality of difference images G (x, y, σ _j ) (step C6). The size estimation unit 26 compares pixel values (difference values) of pixels around the position of the object estimated by the position estimation unit 25 between, for example, k difference images. The size estimation means 26 specifies the scale of the difference image that gives the maximum pixel value. Alternatively, the size estimation unit 26 may compare not only the vicinity of the estimated position of the object but also the pixel values of all the pixels of the difference image, and specify the scale of the difference image that gives the maximum pixel value. Since it is known how much the image in the image spreads out by performing the smoothing process, if the scale that gives the maximum difference is found, the size of the object can be estimated based on the scale number. Since the object to be detected varies as described above, the size estimation unit 26 estimates the size ± α (α is a predetermined value) estimated from the difference image having the largest difference value as the object size. You may do it.

  The partial image generating unit 27 generates an image around the position where the object in the frame image is estimated to exist as a partial image using the estimated position and size of the object (step S7). For example, the partial image generation unit 27 cuts out an image around a position where an object is estimated to exist from a frame image, and enlarges / reduces the cut-out image according to the estimated size of the object. By enlarging / reducing according to the estimated size of the object, the size of the object in the partial image can be adapted to the size of the object in the template used in the discriminator 13. The partial image generating means 27 outputs the generated partial image to the discriminator 13. The discriminator 13 performs detailed discrimination processing on the presence of the detection target object on the partial image generated by the partial image generation unit 27.

  Considering the position estimation of an object using a DOG (Differential Of Gaussian) image as a comparative example, the position estimation using a DOG image needs to obtain all the differences between smoothed images of adjacent scales, and needs to be generated. The number of certain difference images increases. When the object candidate point detection means 12 shown in FIG. 5 is used, a difference between a smoothed image having a certain scale and a smoothed image having a scale separated from the scale by a predetermined scale may be generated as a difference image. Compared to the position estimation used, the number of generated difference images can be reduced. For this reason, the position of the object can be estimated efficiently and accurately. Further, the object candidate point detection means 12 having the configuration shown in FIG. 5 can estimate the size of an object without generating a multi-resolution image, and can efficiently estimate the size of the object.

In particular, the smoothing processing unit 22 generates a × k smoothed images L (x, y, σ ₁ ) to (x, y, σ _{a × k} ), and the difference image generating unit 23 uses Expression 1. Te, scale sigma _j of the smoothed image _{L (x, y, σ j} ) the scale sigma smoothed image L of _{a × j (x, y,} σ a × j) the difference between the difference image G (x, y , Σ _j ), the position of the object can be accurately estimated in accordance with various changes in the size of the object. Also, the object size can be estimated with high accuracy.

  In the above description, the motion region extraction unit 52 has been described as performing gray scale processing or binarization processing in which the motion region (object) is white and the background region is black. The operation of 52 is not limited to this. For example, the motion region extraction unit 52 may perform gray scale processing or binarization processing in which the motion region is black and the background region is white. In this case, the position estimation unit 25 may estimate the pixel position where the pixel value is minimum in the summed image AP (x, y) as the position of the object. The size estimation unit 26 may estimate the size of the object based on the scale of the difference image that gives the minimum pixel value (difference value) among the plurality of difference images.

  In the above description, an example has been described in which the object candidate point detection unit 12 estimates only one position where an object is estimated to exist from a moving image. However, the present invention is not limited to this. The object candidate point detection means 12 may estimate the presence of a plurality of objects, and cut out partial images of images around a plurality of positions where the objects are estimated to exist. For example, let M be the number of objects whose positions should be estimated by the object candidate point detection means 12. In that case, the position estimation means 25 arranges the pixel values of the summed image AP (x, y) in descending order, estimates the top M pixel positions as the positions of the respective objects, and uses the peripheral image of each position as a partial image. Cut it out. That is, it is only necessary to estimate M pixel positions as object positions in descending order of pixel values in the combined image AP (x, y). The size estimation means 26 may estimate the size of each object based on the scale of the difference image that gives the maximum pixel value around the estimated positions of the M objects.

  Next, effects in the present embodiment will be described. In the present embodiment, prior to the processing in the discriminator 13, a difference between two points corresponding to the positional relationship of the differences to be obtained by the weak discriminator 16 is generated as a difference image. Each weak discriminator 16 obtains a difference value from the difference image by referring to the difference image generated in advance, instead of performing the difference calculation individually. When each weak discriminator 16 performs the difference calculation individually, in the plurality of weak discriminators 16 that calculate the difference between two points having the same positional relationship, the difference at the same position of the input image is individually obtained before and after the template is moved. Calculated redundantly. In this embodiment, since the weak discriminator 16 can obtain a difference value by referring to the difference image, the difference calculation between two points having the same positional relationship is executed repeatedly when the template is moved. Can be avoided. In the present embodiment, since it is possible to avoid duplicate difference calculation, the number of difference processing calculations can be reduced, and the discrimination processing can be speeded up.

  Hereinafter, a specific example will be described. FIGS. 7A and 7B show basic feature types related to difference calculation in the weak classifier 16. Basic feature types A and B shown in (a) and (b) are each defined as a difference between a pair of three pixels. The basic feature type A shown in (a) and the basic feature type B shown in (b) differ in the positional relationship of the pixels between the pairs, but the difference between the pixels separated by one pixel in the horizontal direction in each pair. They are common in that they perform calculations. As a result of learning, the discriminator 13 includes a weak discriminator 16 that performs difference calculation with the basic feature type A shown in (a), and a weak discriminator 16 that performs difference calculation with the basic feature type B shown in (b). Suppose that each contains 20 pieces.

  If the difference image is not used and the template size is 32 pixels × 32 pixels, the basic feature type A weak discriminator 16 and the basic feature type B weak discriminator 16 respectively in this 32 × 32 region. Difference calculation is performed at 20 locations. Since the basic feature types A and B each perform difference calculation for three pairs, (20 + 20) × 3 = 120 difference calculations are performed in the template. When this template is scanned by shifting one pixel at a time in the range of 32 pixels in the vertical and horizontal directions, difference calculation is performed 120 times at each position of the template, so the total number of difference calculations is 120 × 32 ×. 32 = 122880 times.

  On the other hand, when using the difference image, assuming that the scanning range of the template is the same as the above, the area that needs to be processed is 64 pixels × 64 pixels (template size 32 pixels + scanning width 32 pixels = 64 pixels). When the difference image is generated, 64 × 64 = 4096 difference calculations are required. Since the basic feature type A and the basic feature type B both calculate a difference between pixel pairs that are one pixel apart in the horizontal direction, the basic feature type A weak classifier 16 and the basic feature type B weak classifier 16 The common difference image may be referred to. When a difference image is used, processing in the weak discriminator 16 of the basic feature type A and the weak discriminator 16 of the basic feature type B can be realized by performing the difference calculation 4096 times. The number of difference calculations can be greatly reduced as compared with the case where the calculation is performed. Further, in the generation of the difference image, for example, by performing the difference calculation by raster scanning the position of the pixel of interest, the reference location of the image is localized as compared with the case where the difference calculation is individually performed in each weak discriminator 16. This can be expected to improve cache hits. When the cache hits, the difference image can be generated efficiently.

  Further, in the present embodiment, the object candidate point detection unit 12 is used, and an image portion having a high possibility that an object exists is input to the discriminator 13. In the present embodiment, since the classifier 13 processes an image portion having a high probability that an object exists, the weak classifiers 16 are collectively processed up to the final stage without performing the early termination determination by each weak classifier 16. It is preferable to carry out. If the early termination is not performed, no branch determination occurs in each weak discriminator 16, so that the pipeline is not disturbed. Furthermore, there is an effect that the processing time in the discriminator 13 can be maintained at a constant time by not performing the early termination.

  Next, a second embodiment of the present invention will be described. The configuration of the object discrimination device in the second embodiment is the same as that of the object discrimination device 10 in the first embodiment shown in FIG. In this embodiment, in the discriminator 13, a plurality of weak discriminators 16 (FIG. 2) having the same basic feature type are connected in cascade. Other points are the same as in the first embodiment.

  FIG. 8 shows a discriminator used in the object discriminating apparatus of the second embodiment. In the discriminator 13a, for example, as shown in FIG. 8, the weak discriminators 16 of the basic feature type 1, the basic feature type 2, and the basic feature type 3 are grouped and cascaded in succession. Further, in the classifier 13a, the group of weak classifiers of basic feature type 2 is arranged next to the group of weak classifiers 16 of basic feature type 1, and the group of weak classifiers 16 of basic feature type 2 is next to basic. A group of weak classifiers 16 of feature type 3 is arranged.

  FIG. 9 shows a discriminator constituting apparatus 30 used for the construction of the discriminator 13a. The learning result input means 31 inputs a plurality of weak discriminators 16 learned using machine learning. The grouping means 32 groups a plurality of weak classifiers 16 obtained by learning into a plurality of groups according to the basic feature type. The grouping means 32 groups a plurality of weak classifiers 16 for each basic feature type, for example. The rearrangement means 33 configures a discriminator 13a by cascading a plurality of weak discriminators so that the weak discriminators 16 belonging to the same group are continuously arranged. The function of each part of the discriminator constituting apparatus 30 can be realized by, for example, a computer executing processing according to a predetermined program.

  FIG. 10A shows the discriminator after learning, and FIG. 10B shows the discriminator after rearrangement. Generally, the weak classifiers obtained by learning are arranged in descending order of the weighted correct answer rate, that is, in an order effective for discrimination. FIG. 10A shows a state in which a plurality of weak classifiers are cascade-connected in the order effective for discrimination. As shown in FIG. 10B, the rearrangement unit 33 rearranges the learned weak classifiers so that weak classifiers having the same basic feature type are continuously arranged in the classifier 13a. By performing the rearrangement, for example, the weak discriminator constituting the first stage in the discriminator after learning (FIG. 10A) is arranged in the middle stage of the discriminator 13a after rearrangement (FIG. 10B). Then, the weak discriminator constituting the middle stage in the discriminator after learning can be arranged in the first stage of the discriminator 13a after rearrangement.

  Here, the lookup table 14 shown in FIG. 1 stores a lookup table generated for each basic feature type, and weak classifiers 16 having the same basic feature type refer to the same lookup table. Find the score. Usually, the processor that implements the processing of the discriminator 13a is provided with a cache memory, and the cache memory stores a portion close to the reference location of the lookup table referred to by the weak discriminator 16. .

  In a general classifier (strong classifier) in which weak classifiers are cascade-connected in the order effective for discrimination, the basic feature type of a weak classifier in one stage and the basic feature type of the weak classifier in the next stage are Often different. In that case, even if a part of the lookup table referenced by the weak classifier is stored in the cache memory in the process of the weak classifier at a certain stage, the cache is hit in the process of the weak classifier at the next stage. I can't expect much to do. On the other hand, when weak classifiers having the same basic feature type are successively arranged, the same look-up table is referred to while the weak classifiers 16 having the same basic feature type are continuously processed. Can increase the probability of hit

  In this embodiment, the object discriminating apparatus 10 discriminates whether or not an object to be detected exists in an image using a discriminator 13a in which weak discriminators 16 having the same basic feature type are continuously arranged. . By doing in this way, compared with the case where the weak discriminators 16 of the same basic feature type are not continuously arranged, the reference can be localized in the lookup table reference, and the probability of a cache hit is increased. be able to. In the present embodiment, in addition to the effects obtained in the first embodiment, the processing speed can be increased by the amount corresponding to the cache hit. In particular, in a low-power processing system used mainly in an embedded system, the presence or absence of a cache hit has a large influence on the processing time, and the processing time can be greatly shortened by hitting the cache.

  Next, a third embodiment of the present invention will be described. The configuration of the object discrimination device in the present embodiment is the same as the configuration of the object discrimination device 10 in the first embodiment shown in FIG. In the present embodiment, in the strong discriminator 13a (FIG. 8), a plurality of weak discriminators 16 having the same basic feature type are arranged in the order of arrangement according to the reference position of the image at the time of difference calculation in each weak discriminator 16. It is done. Other points are the same as in the second embodiment.

  FIG. 11A shows the arrangement order of the weak classifiers in the basic feature type 1, and FIG. 11B shows the reference positions of the images of the weak classifiers in the template. The basic feature type 1 is a difference between two pixels arranged in the horizontal direction (x direction). FIG. 11B shows image reference positions in some of the plurality of weak classifiers 16 having the basic feature type 1 as the difference calculation type. As shown in FIG. 11A, the plurality of weak classifiers 16 of the basic feature type 1 are cascade-connected in the order according to the reference position of the image at the time of difference calculation in each weak classifier 16.

  For example, the plurality of weak classifiers 16 of the basic feature type 1 are arranged so that the reference positions of the images in the difference calculation in each weak classifier 16 appear in the raster scan scanning order. In the discriminator 13 shown in FIG. 8, the weak discriminators 16 of the basic feature type 2 and the weak discriminators 16 of the basic feature type 3 are different from each other in the weak discriminator 16 as in the basic feature type 1. The reference positions of the images at the time of calculation are arranged so as to appear in the raster scan scanning order.

  FIG. 12 shows a discriminator configuration apparatus 30a used for the configuration of the discriminator in the present embodiment. The learning result input means 31 inputs a plurality of weak discriminators 16 learned using machine learning. The grouping means 32 groups a plurality of weak classifiers 16 obtained by learning into a plurality of groups according to the basic feature type. The grouping means 32 groups a plurality of weak classifiers 16 for each basic feature type, for example. The sorting unit 34 sorts the weak classifiers 16 belonging to the same group according to the reference position of the image at the time of difference calculation. The rearrangement means 33 forms a discriminator by cascading a plurality of weak classifiers for each group according to the order sorted by the sorting means 34.

  For example, the sorting unit 34 selects a reference position closest to the origin (upper left of the image) among a plurality of reference positions that the weak discriminator 16 refers to when calculating the difference, and the weak discriminator 16 performs the difference calculation. Sort as the reference position of the image to be referenced. Specifically, when the weak discriminator 16 obtains three sets of differences (see 6 points), the sorting unit 34 determines the position of the reference point that is most different from the origin among the 6 points, and the weak discriminator 16. Sorting can be performed as the reference position of the image in the difference calculation in. Alternatively, any one of the six reference points may be sorted as the image reference position in the difference calculation. Further, the centroid positions of a plurality of reference points in the weak discriminator 16, for example, the centroid positions of six reference points may be sorted as the image reference positions in the difference calculation.

  Here, if the grouping is simply performed according to the basic feature type, even if the basic feature type is the same, the reference position of the image in the difference calculation in the weak classifier at a certain stage and the weak level at the next stage. It is considered that there are many cases where the reference location of the image at the time of difference calculation in the discriminator is far away. In that case, in the processing of the weak classifier at a certain stage, even if an image near the position referred to when the weak classifier calculates the difference is stored in the cache memory, the weak classifier at the next stage The image cache never hits when seeking.

  In the present embodiment, a discriminator 13 is used in which a plurality of weak discriminators 16 are cascade-connected in the arrangement order according to the reference position of the image at the time of difference calculation. When the weak discriminators 16 are arranged in an order according to the reference location of the image, the subsequent weak discriminator 16 obtains a difference by referring to a portion close to the reference location of the weak discriminator 16 in the previous stage, and There is a possibility of hitting the cache. In addition to localizing the reference of the lookup table, it is possible to localize the reference for the image, and it is possible to efficiently perform the image reference when obtaining the difference.

  In the second embodiment, grouping is performed for each basic feature type, and the weak classifiers 16 of the same basic feature type are continuously cascaded for all the basic feature types. Is not limited. The weak classifiers 16 of the same basic feature type do not necessarily have to be continuously arranged for all basic feature types. For example, depending on the frequency of use of the basic feature types, some basic feature types may be excluded from grouping targets, and the weak classifiers 16 of the excluded basic feature types may not be cascaded continuously. is there.

  In the third embodiment, the example in which the weak classifiers 16 are arranged according to the reference positions of the images in the difference calculation after grouping by the basic feature type has been described. However, the present invention is not limited to this. For example, the weak classifiers 16 may be arranged according to the reference position of the image at the time of difference calculation without grouping by basic feature type. That is, the discriminator 13 may be configured by cascading a plurality of weak discriminators 16 in the arrangement order according to the reference position of the image at the time of difference calculation in each weak discriminator 16. Even in this case, it is possible to expect an improvement in cache hit when referring to the pixel value, and the processing speed can be increased.

  In each of the above-described embodiments, it has been described that the discriminator 13 does not perform early termination. However, the discriminator 13 may perform early termination. For example, thousands of weak classifiers may be divided into blocks of several hundred weak classifiers, and early termination may be determined for each block. In that case, a plurality of weak classifiers may be cascade-connected so that weak classifiers having the same basic feature type are continuously arranged in the same block. Alternatively, the weak classifiers may be arranged for each block in the arrangement order according to the reference location of the image at the time of difference calculation. In this case, the reference can be localized in the processing in the block, and the processing time can be shortened as compared with the case where the weak classifiers are arranged in the order effective for the determination in the block. It is also possible to construct a strong discriminator consisting of multiple blocks by learning by changing the basic feature type population for each block. In that case, an early termination decision is made only once at the end of each block. Also good.

  As described above, the present invention has been described based on the preferred embodiments. However, the object discriminating apparatus, method, and program of the present invention are not limited to the above-described embodiments, and various configurations are possible from the configuration of the above-described embodiments. Modifications and changes are also included in the scope of the present invention.

10: Object discrimination device 11: Image input means 12: Object candidate point detection means 13: Discriminator (strong discriminator)
14: Lookup table 15: Difference image generation means 16: Weak discriminator 21: Preprocessing means 22: Smoothing processing means 23: Difference image generation means 24: Summation means 25: Position estimation means 26: Size estimation means 27: Part Image generating means 30: discriminator configuration device 31: learning result input means 32: grouping means 33: rearrangement means 34: sorting means 51: resolution conversion means 52: motion region extraction means

Claims (14)

A strong discriminator in which a plurality of weak discriminators each of which obtains a difference between at least one set of two points in the input image and obtains a score relating to the presence of the detection target based on the obtained difference are cascaded;
The weak discriminator sets a deviation amount according to the positional relationship between two points of the image whose difference is to be obtained, and generates a difference image between the input image and an image obtained by shifting the input image by the set deviation amount. Difference image generation means,
An object discriminating apparatus, wherein at least a part of the plurality of weak discriminators obtains the score by obtaining a difference between the at least one set of two points with reference to the difference image. .   The strong discriminator scans a template in the input image in a predetermined scanning order, and executes processing in the plurality of weak discriminators for each position of the scanned template. Item 4. The object discrimination device according to Item 1.   The difference image generation means sequentially sets a shift amount corresponding to a positional relationship between a plurality of two points for which a difference is to be obtained by the plurality of weak classifiers, and generates a plurality of difference images corresponding to each shift amount. The object discriminating apparatus according to claim 1, wherein the object discriminating apparatus is a thing.   The difference image generating means sequentially sets a shift amount according to the positional relationship between all two points for which a difference is to be obtained by the plurality of weak discriminators, and each of the plurality of weak discriminators, The object discriminating apparatus according to claim 3, wherein a difference between the at least one set of two points is acquired with reference to the difference image.   5. The process according to claim 1, wherein the strong classifier executes processing in each weak classifier without performing early termination from the first stage to the last stage of the cascade-connected weak classifier. The object discrimination device according to any one of the above.   Each of the plurality of weak classifiers obtains a difference between the at least one set of two points by any one of a plurality of basic feature types related to difference calculation. In the strong classifier, the basic feature types are the same. 6. The object discriminating apparatus according to claim 1, wherein the weak discriminators are continuously arranged.   In the strong discriminator, when there are a plurality of weak discriminators having the same basic feature type, the plurality of weak discriminators having the same basic feature type are arranged according to the reference position of the image in the difference calculation in each weak discriminator. The object discrimination device according to claim 6, wherein the object discrimination devices are arranged in order.   6. The weak classifiers according to claim 1, wherein the weak classifiers are arranged in an arrangement order according to a reference position of an image in the difference calculation in each weak classifier. Object determination device.   Object candidate inspection that estimates the position of an object from an image to be processed, cuts out an image around the estimated object position, and supplies the cut-out image as the input image to the strong discriminator and the difference image generation unit, respectively 9. The object discriminating apparatus according to claim 1, further comprising an output unit. The object candidate point detection means
Smoothing processing means for repeatedly performing a process of convolving a smoothing filter having a filter characteristic corresponding to the contour shape of an object into an image, and generating a plurality of smoothed images having different scales from the frame image;
A difference image generating means for generating a plurality of difference images between two smoothed images having different scales among the plurality of smoothed images, while changing the scale;
A summing means for summing the plurality of difference images to generate a summed image;
Position estimation means for estimating the position of the object based on the pixel value in the combined image;
The object discriminating apparatus according to claim 9, further comprising: a partial image generating unit that cuts out an image of a region around the estimated position from the frame image. The smoothing processing means performs a × k smoothed images L (x, y, σ _i ) (i = ₁ to _{a ×)} from the scale σ ₁ to σ _{a × k} (a and k are integers of 2 or more). k), and the difference image generation means smoothes k difference images G (x, y, σ _j ) (j = 1 to k) from the scales σ ₁ to σ _k , respectively, on the scale σ _j . And generating _a smoothed image L (x, y, σ _{j × a} ) having _a scale σ _{j × a} based on a difference between the converted image L (x, y, σ _j ) and the smoothed image L (x, y, σ _{j × a} ) having _a scale σ _{j × a.} 10. The object discrimination device according to 10. The smoothing processing means generates r smoothed images L (x, y, σ _i ) (i = ₁ to _r ) from scale σ ₁ to σ _r (r is an integer of 3 or more), and the difference The image generating means outputs kp differential images G (x, y, σ _j ) (j = 1 to k−p) from the scale σ ₁ to σ _k−p (p is an integer of 1 or more), smoothed image L of each scale _{σ j (x, y, σ} j) is characterized in that to produce on the basis of the difference between the scale sigma _{j + p} of the smoothed image _{L (x, y, σ j} + p) and The object discrimination device according to claim 10. Each performing a plurality of weak discriminations in a cascade to obtain a score relating to the presence of the detection object based on a difference between at least one set of two points in the input image; and
Generating a difference image between the input image and an image obtained by shifting the input image by a shift amount according to a positional relationship between two points of an image whose difference should be obtained by the weak discrimination;
In at least a part of the steps of performing the plurality of weak discriminations in cascade, the difference between the at least one set of two points is acquired with reference to the difference image, and the score is obtained. . On the computer,
A procedure for performing in cascade a plurality of weak discriminations, each of which obtains a score relating to the presence of a detection object based on a difference between at least one set of two points in the input image;
A step of generating a difference image between the input image and an image obtained by shifting the input image by a shift amount corresponding to a positional relationship between two points of the image whose difference should be obtained by the weak discrimination;
In order to cause a computer to execute a procedure for obtaining the score by acquiring a difference between the at least one set of two points with reference to the difference image in at least a part of the procedure of performing the plurality of weak discriminations in cascade. Program.