JP2011248525A - Object detection device and detection method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device and a detection method thereof, by which an object image from an image frame is accurately detected in a short time and also the trend of the object image is predicted.SOLUTION: The object detection device and the detection method thereof are the ones for detecting an object image 11 to be a detection object from an image frame 10 obtained by image input means. The method includes: a model generation step for allowing model generation means to previously generate an image model of the object image 11; a feature amount calculation step for allowing feature amount calculation means to calculate the luminance gradients of each pixel in the image frame 10 as feature amounts with respect to an area in the image frame 10, which corresponds to the generated image model of the object image 11; and an identification step for allowing identification means to determine whether or not the object image 11 exists in the image frame through the use of a calculation outcome calculated by the feature amount calculation means and a learning outcome of the previously obtained feature amount of the object image 11.

Description

The present invention relates to an object detection apparatus and a detection method for detecting an object image to be detected from an obtained image frame.

In today's growing demand for security cameras and in-vehicle cameras, research to automatically recognize and understand objects on images (image frames) with computers has been actively conducted in Japan and overseas due to improved performance of personal computers and digital cameras. Has been done. Among them, automatic recognition of human movements and actions by a computer is highly expected to be applied to robot vision, ITS (Intelligent Transport Systems) and the like.

As a technique for detecting a person from an image, for example, a technique using HOG feature amount and SVM proposed by Dalal et al. This is a method in which a small rectangular area is sequentially moved on an image, an HOG feature amount is calculated therein, and it is used to determine whether the small rectangular area includes a human image.
As another method, Non-Patent Document 2 describes a method using a Joint feature that expresses a co-occurrence between a plurality of HOG feature values, and Non-Patent Document 3 describes an HOG feature with various block sizes. Each method of obtaining quantities is disclosed.
Note that the HOG (histograms of oriented gradients) feature amount described above uses luminance gradient information obtained from each pixel (pixel) in a small rectangular area called a cell, and the magnitude of the luminance gradient with respect to the direction of the luminance gradient. This is a histogram feature amount obtained by accumulating the length.

N. Dalal, B.M. Triggs, “Histograms of Oriented Gradients for Human Detection”, IEEE CVPR, p. 886-893, 2005 Hiroyoshi Fujiyoshi, Object Detection by Joint Feature Focusing on Relevance of Local Features, IEICE Technical Committee, 2009 Q. Zhu, S .; Aviden, M.M. Yeh, K .; Cheng, “First Human Detection Using a Cascade of Histograms of Oriented Gradients”, ICE CV E (E

However, the conventional method still has the following problems to be solved.
Since the method disclosed in Non-Patent Document 1 moves the small rectangular region over the entire image, the HOG feature amount (that is, the feature amount obtained by histogramating the gradient direction in the local region) is obtained up to a background having a shape different from that of the human image. The target was to detect human images.
The method disclosed in Non-Patent Document 2 selects effective feature amounts as a result of boosting learning, but selects feature amounts up to almost the background.
Note that in both methods of Non-Patent Documents 1 and 2, the cells at the time of detection are fixed by grids, and there is no degree of freedom when performing calculations.
The method disclosed in Non-Patent Document 3 is also a target for obtaining the HOG feature amount up to a background having a shape different from that of the human image, as in Non-Patent Documents 1 and 2.

As described above, in the methods disclosed in Non-Patent Documents 1 to 3, since a background having a shape different from that of a human image is a target for human image detection, for example, a portion that can be recognized as a background is erroneously detected as a human image. In some cases, the human image cannot be detected depending on the background, and the human image detection accuracy is likely to be lowered. In addition, since it is necessary to calculate the HOG feature value for the entire image including the background, the calculation time is long and the calculation speed is slow.
Note that the methods disclosed in Non-Patent Documents 1 to 3 are intended only to detect a person from an image, and thus have not been considered to predict human behavior.

The present invention has been made in view of such circumstances, and an object detection apparatus and a detection method thereof that can detect an object from an image frame with high accuracy and in a short time, and further can predict the trend of the object. The purpose is to provide.

An object detection apparatus according to a first invention that meets the above-described object is an object detection apparatus for detecting an object image to be detected from an image frame obtained by an image input means,
Model creation means for creating an image model of the object image in advance;
Feature amount calculating means for calculating a luminance gradient of each pixel of the image frame as a feature amount for an area in the image frame corresponding to the image model of the object image created by the model creating means;
An identification unit that determines whether or not the object image exists in the image frame using a calculation result calculated by the feature amount calculation unit and a learning result of the feature amount of the object image obtained in advance; Have

In the object detection device according to the first invention, the learning result of the feature amount of the object image includes learning data for each direction of the object image, and it is determined that the object image exists in the image frame. On the condition that it is, it is preferable that the direction of the object image is further identified by the identification means.
Here, the object image may be a human image.

An object detection method according to the second invention that meets the above object is an object detection method for detecting an object image to be detected from an image frame obtained by an image input means,
A model creation step of creating an image model of the object image in advance;
A feature amount calculating step of calculating a luminance gradient of each pixel of the image frame as a feature amount for a region in the image frame corresponding to the image model of the object image created in the model creating step;
An identification step for determining whether or not the object image exists in the image frame using the calculation result calculated in the feature amount calculation step and the learning result of the feature amount of the object image obtained in advance; Have

In the object detection method according to the second invention, the learning result of the feature amount of the object image includes learning data for each direction of the object image, and it is determined that the object image exists in the image frame. On the condition that it is, it is preferable to further identify the orientation of the object image in the identification step.
Here, the object image may be a human image.

The object detection apparatus and the detection method according to the present invention use an image model of an object image that is a detection target created in advance, and each pixel of the image frame is compared with an area in the image frame corresponding to the image model. Since the brightness gradient is calculated as a feature amount, and the calculation result and the learning result of the feature amount of the object image obtained in advance are used to determine whether or not the object image exists in the image frame, the object image A portion different from the shape of the object image, for example, the background of the image frame can be removed in advance from the object image detection target. As a result, it is possible to reduce the case where a part known as the background is erroneously detected as an object or the case where the object cannot be detected based on the background, and it is not necessary to calculate the feature amount of the entire image including the background.
Therefore, the detection of the object image from the image frame can be performed accurately and in a short time.

In addition, the learning result of the feature amount of the object image includes learning data for each direction of the object image. When the direction of the object image is identified, if the object image is a human image, the human behavior is predicted. It becomes possible. Thus, for example, if the image frame is a video from an in-vehicle camera, a human image facing the road side can be detected, so that the driver can be informed of the possibility of a person jumping out on the road.

It is explanatory drawing of the detection method of the object which concerns on one embodiment of this invention. It is a flowchart of the process performed at the model creation process of the detection method of the same object. (A)-(E) is explanatory drawing which shows the creation process of the person's image model created at the model creation process of the detection method of the same object, respectively. It is explanatory drawing of the process performed at the feature-value calculation process of the detection method of the same object. It is explanatory drawing of the process performed at the identification process of the detection method of the same object. It is a flowchart of the detection method of the same object. (A), (B) is explanatory drawing of the person learning image used by person detection experiment, respectively, and explanatory drawing of a non-human learning image. (A)-(C) are explanatory drawing of the person evaluation image used in the same person detection experiment, explanatory drawing of a non-human evaluation image, and explanatory drawing of the image model of a human image, respectively. It is a graph which shows the result of a coterie detection experiment. (A) is explanatory drawing of the learning image used in the body direction detection experiment, (B) is explanatory drawing of the evaluation image of environment 1, (C) is explanatory drawing of the evaluation image of environment 2. (A)-(E) is explanatory drawing of the image model of the human image used by the body direction detection experiment, respectively.

Next, embodiments of the present invention will be described with reference to the accompanying drawings for understanding of the present invention.
As shown in FIG. 1, an object detection apparatus (hereinafter also simply referred to as a detection apparatus) according to an embodiment of the present invention includes an image frame 10 obtained by a single video camera (an example of image input means). This is a device for detecting a human image (an example of an object image) 11 to be detected from (for example, a video or a photograph). This detection device can be composed of a RAM, CPU, ROM, I / O, and a conventionally known arithmetic unit (that is, a computer) provided with a bus connecting these elements, but is not limited to this. is not. This will be described in detail below.

The detection apparatus includes a storage unit, a model creation unit, a feature amount calculation unit, and an identification unit.
The storage means is composed of a memory (RAM or ROM), and stores images captured by the video camera. As the video camera, for example, a fixed or movable camera such as a CCD camera, a high-speed camera, a handy type camera, a digital VTR, or a digital video camera can be used.
In addition, the model creation unit, the feature amount calculation unit, and the identification unit are realized by the CPU executing a predetermined program.

First, the model creation means will be described with reference to FIGS.
In the model creation means, in step 11 (ST11), the brightness gradient on the image is calculated from the human image (image including the human image) stored in the database in advance. This luminance gradient indicates the degree of luminance change in the vicinity of a target pixel (hereinafter also referred to as a pixel), and has a large value in the boundary region (contour) of the object image within the image frame. Note that all images in the database can be used as the human image, but only a part of the image may be used.
This luminance gradient is calculated by Expression (1), where I (x, y) is the luminance value of the pixel (x, y) and fx and fy are the luminance gradients in the x and y directions, respectively.

Next, in step 12 (ST12), the average value of the above-described luminance gradient is calculated for all pixels on the image, and the image shown in FIG. 3A, that is, an average luminance gradient image is created.
In step 13 (ST13), the contour (edge) portion shown in FIG. 3B is extracted from the created average luminance gradient image, and a contour image (edge image) of a human image is created.
Further, in step 14 (ST14), a shadow image of the person image shown in FIG. 3C is created from the created average luminance gradient image, and the skeleton image of the person image shown in FIG. 3D is created from the shadow image. Create Here, in order to create a skeleton image from a shadow image of a human image, first, distance image conversion is performed from the shadow image. This distance image conversion is a method of calculating the distance from the boundary between the white image and the black image for the white pixel portion of the shadow image. Then, a skeleton image can be created by searching for only the maximum point of the image obtained by the distance image conversion.

Finally, in step 15 (ST15), the outline image of the human image shown in FIG. 3B and the skeleton image of the human image shown in FIG. 3D are combined (an image that becomes a logical sum). An image model of a human image shown in FIG.
Thereby, an image model of a human image used for detecting the human image 11 from the image frame 10 can be created in advance. Note that the image model creation of the human image by the model creation means is not limited to this, and for example, it can be created by taking a background difference from continuously captured images.

Next, the feature amount calculation unit will be described.
As shown in FIG. 1, the feature amount calculating means moves the rectangular area 12 set around the image model of the human image created by the model creating means from the left (one) to the right (the other) of the image frame 10. Further, scanning is performed on the image frame 10 sequentially from the top to the bottom of the image frame 10. Here, the image model of the human image is a mask composed of white portions in FIG. The rectangular area 12 is an area including an image model of a human image, and here, the area shown in FIG. 3E is used as it is.
As a scanning method, a conventionally known raster scan can be used, but other methods may be used. Further, the scanning direction is not limited to the above-described direction.

Then, each region of the image frame 10 at the coordinate position of a pixel (white portion in FIG. 3E) that constitutes the region in the image frame 10 corresponding to the image model of the human image being scanned, that is, the image model of the human image. For each pixel, the luminance gradient is sequentially calculated as a feature amount. Specifically, as shown in FIG. 4, an image model of a human image is used as a mask, a pixel in the image frame 10 at the coordinate position of the mask is the center, and 5 × 5 pixels are one cell in the image frame 10. The obtained rectangular area is taken, and the HOG feature amount of this area is calculated (a luminance gradient direction histogram is created).

The calculation of the HOG feature amount is performed sequentially from the left (one) to the right (the other) of the region in the image frame 10 using the human image model as a mask and from the top to the bottom of the image frame 10. (In the direction of the arrow), it is not limited to this. Note that the HOG feature amount can be obtained using, for example, the same method as in Non-Patent Document 1 or the like.
Next, the calculated HOG feature value is normalized by Expression (2).

Here, a is an element of the HOG feature value, f is a feature value vector in which HOG features calculated in each cell are arranged, v is a feature value after normalization, and ε is to avoid the denominator being “0”. Is a small positive real value.
The HOG described above is characterized in that the gradient of the neighboring pixels is histogrammed by a local region, so that it is not easily affected by illumination and shadows and is robust to local geometric changes.
According to the above method, only the pixel in the image frame 10 located at the coordinate position of the image model of the human image can be set as the target for human image detection, so that the feature amount can be calculated with the minimum size region. In addition, unnecessary feature amounts such as the background can be removed.

Note that the feature quantity calculation means can obtain a HOG feature quantity of a human image in advance and construct a database of learning results (create a human detector) by the same method as described above.
Specifically, using the above-described feature vector V having v as an element, a human image as positive (positive) data and an image other than a human image as negative (negative) data, a conventionally known SVM (Support Vector Machine: (Support vector machine). This SVM is a method of configuring a pattern discriminator using a linear input element, and is based on a standard of margin maximization for obtaining a separation plane (hyperplane) that maximizes the distance from each data point from a learning sample. It learns the parameters of a linear input element.

If a human image can be distinguished from an image other than a human image, other learning methods, for example, boosting such as AdaBoost (LP Boost), LPBoost, BrownBoost, Log Boost, or a neural network may be used. it can.

Then, it is preferable to construct a database of learning results (create a body direction discriminator) by further including learning data for each orientation of the human image in the learning results described above.
Here, as shown in FIG. 5, for example, a feature vector V having v as an element is used by a method similar to the above-described method using an image facing a front, right, or left as a learning image. Is used to perform conventionally known SVM learning. Specifically, using one-to-other method and one-to-one method, using a front image, right-facing image, and left-facing image of a person, three (plural) identifications in two classes of the first and second layers Construct a device (direction discriminator).

Here, the one-to-other method is front-to-other, right-to-other, and left-to-other, and the one-to-one method is front-to-right and front-to-left. Note that the number of classifiers and the conditions of the one-to-other method and the one-to-one method are not limited to those described above, and can be variously changed according to the detection target.
Thus, the orientation of the human image can be identified by voting by a classifier constructed by a one-to-other method. When the number of votes in each direction is equal, a final determination is made using a discriminator created by the one-to-one method.

Finally, the identification means will be described.
The identification unit calculates the HOG feature amount of the image frame 10 as the feature amount for the region in the image frame 10 using the human image model as a mask by the above-described feature amount calculation unit. Whether or not the human image 11 exists in the image frame 10 is determined by the above-described human detector using the learning result of the feature amount of the human image obtained in advance. That is, it is determined whether the calculation result is included in the region of the human image learned by SVM or the region of the image other than the human image.

Further, it is preferable that the identification unit further identifies the orientation of the human image 11 on the condition that it is determined that the human image 11 exists in the image frame 10.
In this case, first, as shown in FIG. 1, a plurality of (here, three) regions 13 in which the human image 11 is detected are integrated into one region 14. Specifically, among the plurality of regions 13, overlapping portions having the same feature amount are overlapped, and one region 14 centered on the human image 11 is formed by using an integration method together. As this integration method, an existing method called MeanShift method is used, but ROI information of the overlapping portion (the position (x, y) of the upper left point of the rectangular area and the vertical and horizontal sizes of the rectangular area) It can also be determined by the average of.
Then, using the calculation result obtained by calculating the HOG feature value of the image frame 10 as the feature value and the learning result of the feature value of the human image obtained in advance, the human image in the image frame 10 is obtained by the body direction discriminator described above. 11 orientations are identified.

Next, an object detection method according to an embodiment of the present invention will be described with reference to FIG.
First, a learning result of a feature amount of a human image or an image other than a human image obtained in advance by the feature amount calculation unit is input to the storage unit. Here, the human image includes, for example, an image of an adult or a child, a front direction, a right direction, or a left direction. Examples of images other than human images include vehicles, buildings, trees, pillars, and signboards.
Using these feature amounts, the feature amount calculating means constructs the human detector and the body direction discriminator described above (the preparation step).

Next, in step 1 (ST1), a video (unknown image) is captured by the video camera, and this image frame 10 is input to the storage means.
In step 2 (ST2), the above-described model creation means performs ST11 to ST15 shown in FIG. 2 in which an image model of the human image 11 is created in advance, and a rectangular area 12 including the human image 11 is set. Here, the rectangular area 12 is an area smaller than the image frame 10. Further, the image model of the human image 11 may be set in advance before the video is captured by the video camera. (End of model creation process)

In step 3 (ST3), as shown in FIG. 1, the set rectangular area 12 is scanned on the image frame 10, and the HOG is applied to the area in the image frame 10 using the created image model of the human image 11 as a mask. The feature amount is calculated as a feature amount. In step 4 (ST4), the obtained feature quantity is normalized using the above-described equation (2) (the feature quantity calculation step).

In step 5 (ST5), the human image 11 is present in the image frame 10 by the above-described human detector using the calculation result calculated in ST4 and the learning result of the feature amount of the human image 11 obtained in advance. Judge whether or not. If it is determined that the human image 11 is present in the image frame 10, the orientation of the human image can be further identified by the above-described body direction discriminator on the condition (the identification process).

Next, examples carried out for confirming the effects of the present invention will be described.
First, a human image detection experiment will be described.
Here, 1602 human images as learning images and 3235 non-human images were used. An example of a human image (human learning image) is shown in FIG. 7A, and an example of a non-human image (non-human learning image) is shown in FIG. 7B.
In addition, 1000 human images and 1000 non-human images were used as evaluation images for the experiment. An example of a person image (person evaluation image) is shown in FIG. 8A, and an example of an image other than a person (non-person evaluation image) is shown in FIG. 8B.

Using the above-mentioned images, the human image detection accuracy was compared. Here, Example 1 uses an image model of a human image shown in FIG. 8C, and Conventional Example 1 uses the method described in Non-Patent Document 1 that does not use an image model.
In addition, the comparison of the discrimination experiment result was evaluated by DET (Detection Error Tradeoff). In the DET, the horizontal axis represents a false detection rate (probability of recognizing a person other than a person) and the horizontal axis represents a non-detection rate (probability of recognizing a person other than a person) by a log-log graph. By changing the threshold value of the discriminator, it is possible to compare the undetected rate against the false detection rate.

The DET curve obtained in this experiment is shown in FIG.
As shown in FIG. 9, when the non-detection rate is 0.5%, the false detection rate is 19.1% in Conventional Example 1 (♦) and 7.5% in Example 1 (■). Was reduced by 11.6%. The processing time per sheet was 29.1 milliseconds in Example 1 and 54.4 milliseconds in Conventional Example 1, and the processing speed could be reduced to about half.

Next, a body direction detection experiment will be described.
Here, 217 human images as learning images are used in each direction (front direction, right direction, left direction). An example of this person image is shown in FIG.
In addition, 200 human images, which are evaluation images for experiments, were used for each direction (front direction, right direction, left direction) in the set environment 1 (an environment in which no people overlap) and the actual environment 2. FIG. 10B shows an example of the human image in the set environment 1 (data set 1), and FIG. 10C shows an example of the human image in the real environment 2 (data set 2).

Using each image described above, the detection accuracy of the body direction was compared. Here, Example 2 uses an image model of a human image shown in FIGS. 11A to 11E, and Conventional Example 2 uses a method described in Non-Patent Document 1 that does not use an image model. 11A to 11C are human images of left-facing SVM (L), front-facing SVM (F), and right-facing SVM (R), respectively, and FIGS. 11A and 11B are used. A human image SVM (FL) facing left front shown in FIG. 11D and a human image SVM (FR) facing right front shown in FIG. 11E using FIGS. 11B and 11C. , Each created.
The results obtained in this experiment are shown in Table 1.

As is clear from Table 1, it was found that the correct answer rate was higher in Example 2 than in Conventional Example 2 regardless of the environmental conditions. The processing time was 26.7 milliseconds for Example 2 and 60.6 milliseconds for Conventional Example 2, and the processing speed could be reduced to half or less.
In the above embodiment, Microsoft Windows XP Professional Intel (R) Celeron (R) 2.80 GHz is used as a computer for detecting human images.

From the above results, it was found that by using the object detection apparatus and the detection method of the present invention, both human detection and body direction detection can obtain better results than the conventional method. This is considered to be because the influence of the background was reduced by using only the feature amount on the image model of the human image.

As described above, the present invention has been described with reference to the embodiment. However, the present invention is not limited to the configuration described in the above embodiment, and the matters described in the scope of claims. Other embodiments and modifications conceivable within the scope are also included. For example, the case where the object detection apparatus and the detection method thereof according to the present invention are configured by combining some or all of the above-described embodiments and modifications are also included in the scope of the right of the present invention.
In the above-described embodiment, the case where a human image that is an example of an object image is set as a detection target has been described, but an animal image may be used, for example. In addition, depending on the installation position of the object detection device, for example, a vehicle (train) image such as a car, a train, a ship, or an airplane may be a detection target, thereby creating a driving (operation) environment that emphasizes safety. it can.

Further, in the above-described embodiment, the case where the feature amount is obtained for all pixels corresponding to the image model of the human image has been described. However, based on the idea of AdaBoost, one or more blocks that are effective for recognition among them May be selected and used.
In the above embodiment, the case where the image frame is obtained by one video camera has been described. For example, the image frame may be obtained by a plurality of video cameras (for example, two or three) having different installation locations. it can. In this case, the processing shown in the above embodiment is performed for each obtained image frame.

10: Image frame, 11: Human image (object image), 12: Rectangular area, 13, 14: Area

Claims (6)

An object detection device for detecting an object image to be detected from an image frame obtained by an image input means,
Model creation means for creating an image model of the object image in advance;
Feature amount calculating means for calculating a luminance gradient of each pixel of the image frame as a feature amount for an area in the image frame corresponding to the image model of the object image created by the model creating means;
An identification unit that determines whether or not the object image exists in the image frame using a calculation result calculated by the feature amount calculation unit and a learning result of the feature amount of the object image obtained in advance; An apparatus for detecting an object characterized by comprising: The object detection apparatus according to claim 1, wherein the learning result of the feature amount of the object image includes learning data for each direction of the object image, and it is determined that the object image exists in the image frame. On the condition, the identification unit further identifies the direction of the object image. 3. The object detection apparatus according to claim 1, wherein the object image is a human image. An object detection method for detecting an object image to be detected from an image frame obtained by an image input means,
A model creation step of creating an image model of the object image in advance;
A feature amount calculating step of calculating a luminance gradient of each pixel of the image frame as a feature amount for a region in the image frame corresponding to the image model of the object image created in the model creating step;
An identification step for determining whether or not the object image exists in the image frame using the calculation result calculated in the feature amount calculation step and the learning result of the feature amount of the object image obtained in advance; An object detection method characterized by comprising: The object detection method according to claim 4, wherein the learning result of the feature amount of the object image includes learning data for each direction of the object image, and it is determined that the object image exists in the image frame. On the condition that, in the identification step, the direction of the object image is further identified. 6. The object detection method according to claim 4, wherein the object image is a human image.