JP4779851B2 - Object detection device - Google Patents

Description

The present invention relates to an object detection device, an object detection method, and a control program, and more particularly, to an object detection device, an object detection method, and a control program for detecting an object such as a human face image from a detection target image.

Various techniques for detecting an object (for example, a face image) included in an image are known.
In such an object detection technique, generally, the size of a face image is normalized to a certain size and detected.
This is because it is not possible to know in advance the size and position of the face image included in the image, so after setting the size of the image to various sizes, set a detection window of a predetermined size,
This is because it is necessary to determine whether the image is a face image.
JP-A-9-114976

For example, if the normalized detection window size is 20 × 20 pixels and a face image is detected from an image of VGA (640 × 480 pixels) size by the above method, about 300,000 detection windows are set. Then, it is determined whether each detection window is a face image.
By the way, in a general photograph, the number of face images included in one photograph is only several to several tens.
In such a situation, as described above, performing the determination on 300,000 detection windows increases the number of useless determination processes, increases the processing time, and causes a problem that it is difficult to detect in real time. .
The technique described in Patent Document 1 discloses a configuration in which an edge image is calculated and an object is detected by a template that detects the position and orientation of the edge.
However, the technique described in Patent Document 1 has a problem that high-speed detection cannot be performed because matching with a template is performed for each detection window.
SUMMARY OF THE INVENTION An object of the present invention is to provide an object detection apparatus, an object detection method, and a control program that can reduce an operation processing burden, recognize an object at high speed, and detect an object image.

In order to solve the above-described problem, the object detection apparatus extracts an image feature amount for each pixel constituting the detection target image, and determines whether each pixel is a pixel constituting the object based on the image feature amount. An image conversion unit that determines and converts the detection target image into an n-value (n is an integer of 2 or more) image, and an image of a predetermined pixel range that constitutes the detection target image based on the n-value image is an object An integrated image generating unit that generates an integrated image for determining the certainty of whether or not to constitute an image, and an image corresponding to a detection window set based on the integrated image is an object image constituting the object And an object discriminating unit for discriminating whether or not.

According to the above configuration, the image conversion unit extracts an image feature amount for each pixel constituting the detection target image, determines whether each pixel is a pixel constituting the object based on the image feature amount, The detection target image is converted into an n-value (n is an integer of 2 or more) image.
The integral image generation unit generates an integral image for determining the certainty of whether or not an image in a predetermined pixel range constituting the detection target image constitutes an object image based on the n-value image.
Thereby, the object determination unit determines whether or not the image corresponding to the detection window set based on the integral image is an object image constituting the object.
Here, since the integral image need only be generated once for the detection target image, high-speed object recognition is possible.

In this case, the integral image generation unit integrates the values of all pixels in a rectangle having the predetermined reference position pixel and the target pixel as vertices on a diagonal line to obtain an integral value corresponding to the target pixel, and the detection The integral image may be generated by obtaining integral values corresponding to all pixels constituting the target image.
According to the above configuration, the calculation process necessary for generating the integral image is substantially only the addition process, and the integral image can be generated easily and at high speed.

A resizing image generating unit that resizes the detection target image to generate a resizing image; and the image converting unit extracts an image feature amount for each pixel constituting the resized image, and the image feature amount And determining whether or not each pixel is a pixel constituting the object, converting the detection target image into an n-value (n is an integer of 2 or more) image, and the integral image generating unit Based on the value image, an integrated image for determining the certainty of whether or not an image of a predetermined pixel range constituting the resized image constitutes an object image, and the object discriminating unit is configured to detect the detection target image Whether the image corresponding to the detection window set based on the integral image corresponding to the image and the integral image corresponding to the resized image is an object image constituting the object It may be determined.

According to the above configuration, the object determination unit determines whether or not the image corresponding to the detection window set based on the integral image corresponding to the detection target image and the integral image corresponding to the resized image is an object image constituting the object. Therefore, the integral image is generated only once for the detection target image and the resized image, so that high-speed object recognition is possible.

In addition, the detection window is resized, and the object determination unit determines whether an image corresponding to the detection window set based on the integral image corresponding to the detection target image is an object image constituting the object. You may make it discriminate | determine.
According to the above configuration, since the integral image needs to be generated only once for the detection target image, high-speed object recognition is possible.
Furthermore, the image conversion unit may convert the image feature amount into the n-value image based on at least one of a color gamut, luminance, and edge.
According to the above configuration, the object image can be reliably detected.
Furthermore, the object image may be a face image, and the color gamut may correspond to the face image.
According to the above configuration, a face image can be reliably detected.

Further, the object detection method extracts an image feature amount for each pixel constituting the detection target image, determines whether each pixel is a pixel constituting the object based on the image feature amount, and detects the detection An image conversion process for converting the target image into an n-value (n is an integer equal to or greater than 2) image, and whether an image in a predetermined pixel range constituting the detection target image based on the n-value image forms an object image An integral image generation process for generating an integral image for determining the certainty of whether or not, and whether or not an image corresponding to a detection window set based on the integral image is an object image constituting the object And an object discriminating process.
According to the above configuration, since the integral image needs to be generated only once for the detection target image, high-speed object recognition is possible.

Further, in a control program for controlling the object detection apparatus by a computer, an image feature amount is extracted for each pixel constituting the detection target image, and each pixel constitutes an object based on the image feature amount. The detection target image is converted into an n-value (n is an integer of 2 or more) image, and an image in a predetermined pixel range constituting the detection target image based on the n-value image is an object. An integrated image for determining the certainty of whether or not to constitute an image is generated, and it is determined whether or not an image corresponding to a detection window set based on the integrated image is an object image constituting the object It is characterized by that.
According to the above configuration, since the integral image needs to be generated only once for the detection target image, high-speed object recognition is possible.

Next, preferred embodiments of the present invention will be described with reference to the drawings.
In the following description, a case will be described in which the size of the detection window is constant, the detection target image is resized (resized image), and a human face image as an object image is detected.
FIG. 1 is a schematic configuration block diagram of an object detection apparatus according to an embodiment.
The object detection device 10 is configured as a personal computer, and includes a detection device main body 11 that performs detection processing, an imaging camera 12 as an image input device, an input device 13 such as a keyboard and a mouse for performing various operations, A display device 14 such as a liquid crystal display that performs various displays, an output device 15 such as a printer that performs printing, and an external storage device 16 such as a hard disk device that stores various data are provided.
The detection device main body 11 performs an interface operation between a microprocessor unit (MPU) 21 that controls the entire detection device main body 11, a ROM 22 that stores various data, and an external communication network 23 such as a LAN or the Internet. And a communication interface unit 24 for performing.

FIG. 2 is an object detection function block diagram of the object detection apparatus.
The object detection device 10 is roughly divided into a luminance image generation unit 31, an edge image generation unit 32, a resize image generation unit 33, a color gamut determination unit 34, a binary image generation unit 35, and an integral image generation unit. 36, a threshold value changing unit 37, a certainty factor determining unit 38, and an identifying unit 39.
Here, the luminance image generation unit 31, the edge image generation unit 32, the color gamut determination unit 34, and the binary image generation unit 35 function as an image conversion unit.
Further, the threshold changing unit 37, the certainty degree determining unit 38, and the identifying unit 39 function as an object determining unit.
About these object detection functions, the detection apparatus main body 11 is actually realized by a predetermined control program.

Here, an outline operation of each unit will be described.
The luminance image generation unit 31 generates a luminance image of the detection target image and outputs it to the edge image generation unit.
The edge image generation unit 32 generates an edge image from the input luminance image to be detected.
The resized image generation unit 33 resizes the detection target image to generate a resize detection target image.
The color gamut determining unit 34 determines whether each pixel constituting the detection target image or the resizing detection target image belongs to a predetermined color gamut corresponding to the detection target object, and the determination result is 2
Output to the value image generator 35.
Based on the determination result of the color gamut determination unit 34, the binary image generation unit 35 generates a binary image for object detection from the detection target image or the resizing detection target image.

The integral image generation unit 36 generates an integral image based on the binary image generated by the binary image generation unit 35.
In parallel with these, the threshold value changing unit 37 sets various threshold values for determining the certainty factor (in-gamut pixel number threshold value, edge strength threshold value, luminance range threshold value, etc.).
The certainty factor determination unit 38 determines the certainty factor that is an object image based on various set threshold values.
The identifying unit 39 determines whether the image is an object image based on the determination result of the certainty factor determining unit 38.

FIG. 3 is an operation flowchart of the embodiment.
First, the detection device main body 11 functions as a luminance image generation unit 31 and is input via an image captured by the imaging camera 12, an image stored in advance in the external storage device 16, or the communication interface unit 24. A luminance image is generated from the detection target image such as the detected image (step S1).
Specifically, reference documents “Yuichi Araki, Nobutaka Shimada, Yoshiaki Shirai,“ Face Detection and Face Direction Independent of Background and Face Direction ”, IEICE Technical Technique PRMU2001-217, pp.87- 94 ”as an example, the color component in each pixel is decomposed into a red component, a green component, and a blue component,
When the red component in each pixel is R, the green component is G, and the blue component is B, the luminance LP of each pixel is calculated by the following equation.
Luminance LP = 0.30 × R + 0.59 × G + 0.11B

Next, the detection apparatus main body 11 functions as the edge image generation unit 32, detects an edge from the luminance image corresponding to the detection target image, and generates an edge image (step S2).
Here, generation of an edge image will be described in detail.
Since the edge is a portion where the luminance value changes abruptly, a differential operation for extracting a change in the function can be used for edge detection. As the differential operation used for edge detection, there are a primary differentiation and a secondary differentiation. In this embodiment, a primary differentiation using a Sobel operator is used for edge detection.
The value of the first derivative using the Sobel operator has a magnitude and direction,
Vector quantity G (x, y) = (fx, fy)
Can be expressed as

Here, fx represents a differential value in the x direction, and fy represents a differential value in the y direction.

The Sobel operator in the x-axis direction is expressed by the following equation.

さらに、ｙ軸方向のSobelオペレータは次式により表される。

Further, the Sobel operator in the y-axis direction is expressed by the following equation.

これらにより、エッジの強度Ｅは、次式により算出することができる。
Ｅ＝√（ｆｘ²＋ｆｙ²）
続いて、検出装置本体１１は、検出対象画像の確信度を表す積分画像を生成する（ステ
ップＳ３）。

Thus, the edge strength E can be calculated by the following equation.
E = √ (fx ² + fy ² )
Subsequently, the detection apparatus main body 11 generates an integral image representing the certainty factor of the detection target image (step S3).

Specifically, first, the detection apparatus main body 11 functions as the color gamut determination unit 34, and for each pixel (for example, 640 × 480 pixels) constituting the detection target image, a predetermined corresponding to the detection target object for each pixel. It is determined whether or not it belongs to the color gamut, and the determination result is output to the binary image generation unit 35.
Here, the detection device main body 11 functioning as the color gamut determination unit 34 has, for example, the color of each pixel in accordance with the condition described in the above-mentioned reference, that is, the color gamut condition corresponding to the color of human skin. It is determined whether it belongs to the color gamut of human skin color.

More specifically, if the color component in each pixel is decomposed into a red component, a green component, and a blue component, and the red component in each pixel is R, the green component is G, and the blue component is B, the skin of the face that is the object Are in the color gamut range that satisfies the following conditions.
0.333 <r <0.664
r> g
0.246 <g <0.398
g ≧ 0.5-0.5r
here,
r = R / (R + G + B)
g = G / (R + G + B)
It has become.

In this case, the above conditions are not limited to the Japanese skin color, and can be applied to various skin colors.
Subsequently, the detection apparatus main body 11 functions as the binary image generation unit 35 and generates a binary image of the detection target image based on the determination result of the color gamut determination.
Specifically, the detection apparatus main body 11 sets a pixel satisfying the above condition as a pixel having a high certainty of being a pixel constituting the face image, sets the value of the pixel to “1”, and sets a pixel that does not satisfy the above condition as a face A binary image is generated by setting a pixel having a high certainty as a pixel that does not constitute an image and setting the value of the pixel to “0”.

FIG. 4 is a schematic explanatory diagram of integrated image generation.
FIG. 4 shows a case where the detection target image is 640 × 480 pixels.
Subsequently, the detection device main body 11 functions as the integral image generation unit 36, and the reference position pixels PX0,0
And all the pixels PX0,0 to PX in the rectangle RA having the target pixel PXm, n as a vertex on the diagonal line
The value of m, n is integrated to obtain an integrated value corresponding to the target pixel PXm, n.

FIG. 5 is an explanatory diagram of a binary image for specific description of integral image generation.
Specifically, as shown in FIG. 5, when the target pixel PXm, n = PX3,3, the reference position pixel PX0,0 and the target pixel PX3,3 are in the rectangle RA1 having the diagonal vertices. An integration value ΣPX3,3 corresponding to the target pixel PX3,3 is calculated by integrating (in practice, simple addition) the values of all the pixels PX0,0 to PX3,3.

FIG. 6 is an explanatory diagram of the generated integral image.
Specifically, it is calculated by the following formula.
ΣPX3,3 = 0 + 0 + 0 + 0 + 0 + 0 + 1 + 1 + 0 + 1 + 1 + 1 + 0 + 1 + 1 + 1
= 8
Similarly, an integrated image in the rectangle RA1 having the reference position pixel PX0,0 and the target pixel PX3,3 as vertices on a diagonal line is as shown in FIG.
Next, the detection apparatus main body 11 functions as the resized image generation unit 33, resizes the detection target image, and generates a resized image (step S4).

More specifically, the detection target image (640 × 480 pixels in the case of the above example) is set to, for example, 1
Reduced by /1.1 to generate a resized image (581 × 436 pixels after the first resize).
Next, the detection apparatus main body 11 performs the same process as the process of step S1 on the resized image, and generates an integrated image representing the certainty factor of the resized image (step S5).

FIG. 7 is an explanatory diagram of a detection window setting state.
Next, the detection apparatus body 11 sets the detection window DW while sequentially shifting the detection window DW in the x-axis and y-axis directions of the resized image.
More specifically, when the size of the detection window DW is 20 × 20 pixels, in the initial state (first time), the detection window DW = DW1, and the start pixel PX0,0 and the end pixel PX19,19 are on the diagonal line. It corresponds to a rectangular area with the vertex of.

The number of pixels of the detection target image is 640 × 480 pixels, and the end pixel is PXP, Q (0 ≦ P
≦ 639, 0 ≦ Q ≦ 479), Q until P = 620 (= 639−19)
The detection window DW is set with the same value. Thereafter, when P = 620, 1 is added to the value of Q, the value of P is set to 0 again, and the next detection window DW is set.
Similarly, the setting of the detection window DW is repeated until P = 620 (= 639-19) and Q = 460 (= 479-19).

Subsequently, the detection device main body 11 functions as the certainty factor determination unit 38 and calculates the certainty factor from the integrated image generated from the resized image. Similarly, the detection device main body 11 also applies to the integrated image generated from the detection target image. A certainty factor is calculated (step S5). At this time, the detection window D described above
Since W is set for the resized image, the detection window for the integral image generated from the detection target image corresponds to the ratio of the detection window DW to the size of the resized image and the position of the detection window DW in the resized image. Based on the position of the target image, a detection window for the detection target image is separately set and the certainty factor is calculated.
Here, the principle of the certainty factor calculation will be described.
The integral image II (PXx, y) when the image feature amount in the pixel PXx, y is C is calculated by the following equation.

算出された積分画像を用いれば、画像ブロック（特に検出窓ＤＷ内の画像）の特徴量の
和を迅速に求めることができる。
画像ブロックの範囲を（ｘ₀≦ｘ≦ｘ₁、ｙ₀≦ｙ≦ｙ₁）とすれば、特徴量の和である積
分値は、次式により表される。

By using the calculated integral image, the sum of the feature quantities of the image block (particularly the image in the detection window DW) can be quickly obtained.
Assuming that the range of the image block is (x ₀ ≦ x ≦ x ₁ , y ₀ ≦ y ≦ y ₁ ), the integral value that is the sum of the feature amounts is expressed by the following equation.

この場合において、積分画像を生成する処理は、検出対象画像またはリサイズ画像に対
して高々一回行えばよいので、領域内の画素数によらず高速な演算が可能になるのである
。

In this case, the process for generating the integral image may be performed at most once on the detection target image or the resized image, so that high-speed calculation is possible regardless of the number of pixels in the region.

Next, the calculation of the certainty factor will be described in detail.
FIG. 8 is an explanatory diagram for calculating the certainty factor.
In calculating the certainty factor, the detection apparatus main body 11 performs the calculation of the certainty factor of the detection window DW using the integral value for each rectangle of the rectangles RA11 to RA14.
Specifically, if the size of the detection window DW is 20 × 20 pixels, if the start pixel of the detection window DW is PXS, T, the end pixel is PXS + 19, T + 19.
At this time, the rectangle RA11 is a rectangle having the start pixel PX0,0 and the end pixel PXS-1, T-1 as vertices on a diagonal line. Therefore, the integral value ΣRA11 of the rectangle RA11 is expressed as ΣPXS-1, T−
1

Similarly, the rectangle RA12 is a rectangle having the start pixel PX0,0 and the end pixel PXS + 19, T-1 as vertices on a diagonal line. Therefore, the integral value ΣRA12 of the rectangle RA12 is ΣPXS + 19, T−1.
It becomes.
The rectangle RA13 is a rectangle having the start pixel PX0,0 and the end pixel PXS-1, T + 19 as vertices on a diagonal line. Therefore, the integral value ΣRA13 of the rectangle RA13 is ΣPXS−1, T + 19.
It becomes.
The rectangle RA14 is a rectangle having the start pixel PX0,0 and the end pixel PXS + 19, T + 19 as vertices on a diagonal line. Therefore, the integral value ΣRA14 of the rectangle RA13 is ΣPX + 19, T + 19.
It becomes.

As a result, the certainty factor of the pixel corresponding to the detection window DW, that is, the integral value ΣDW corresponding to the detection window DW is expressed by the following equation.
ΣDW = ΣRA14 + ΣRA11−ΣRA12−ΣRA13
Subsequently, the detection apparatus main body 11 determines whether or not the certainty that is the object image is equal to or greater than a predetermined threshold value (step S6).

More specifically, when the following three conditions are satisfied, the detection apparatus main body 11 determines that the certainty factor that the image in the detection window of the resized image and the detection target image is an object image is high.
(1) The number of pixels in the set color gamut range is greater than or equal to the threshold value.
(2) The number of pixels whose edge intensity is a certain value or more is greater than or equal to a threshold value.
(3) The number of pixels having a certain luminance range is equal to or greater than a threshold value.

Subsequently, the detection device main body 11 functions as the identification unit 39 and determines whether the image is an object image based on the high certainty factor (the determination result of the certainty factor determination unit 38) (step S7).
.
Specifically, the detection apparatus main body 11 determines whether or not the image is an object image by using a machine learning technique represented by AdaBoost or SVM (support vector machine). That is, the detection apparatus main body 11 of the present embodiment functions as the identification unit 39 by preparing in advance a large number of object images and non-object images as sample images and learning them according to the mechanical learning method described above. As a result, it is possible to more reliably determine whether the image is an object image.

Subsequently, the detection apparatus main body 11 determines whether or not it is determined whether or not the setting of the detection window DW for the resized image has been completed (step S10).
If it is determined in step S10 that detection window settings have not been completed in all areas of the resized image, and therefore detection window settings have not been completed in all areas of the detection target image (step S10; No) ), The process again proceeds to step S, and the same process is performed thereafter.
In the determination of step S10, when the setting of the detection window is completed in all areas of the resized image, and when the setting of the detection window is completed in all areas of the detection target image (step S10; Yes), a predetermined value is set. It is determined whether or not the generation of the resized image has been completed (step S11).

Here, examples of the case where the generation of the predetermined resized image is completed include a case where the size of the resized image becomes equal to the detection window DW.
If it is determined in step S11 that the generation of the predetermined resized image has not been completed (step S11; No), the process proceeds to step S4 again, and the same process is continued thereafter.
If it is determined in step S11 that the generation of the predetermined resized image has been completed (step S11; Yes), the process is terminated.

FIG. 9 is an explanatory diagram of an example of a face image detection result as an object image.
As a result of the above-described processing, two face image areas F1 and F2 (indicated by rectangular areas in FIG. 9) are detected as shown in FIG. 9B with respect to the detection target image G shown in FIG. 9A. Will be.
As described above, according to the present embodiment, the certainty factor of the object image is calculated using the integral image, and the integral image need only be generated once for the detection target image or the resized image. Therefore, high-speed object recognition is possible.

In the above description, when determining the certainty factor, the threshold value corresponding to the number of pixels in the set color gamut range, the threshold value corresponding to the number of pixels whose edge intensity is equal to or greater than a certain value, and the luminance in a certain range. Although the threshold value and each range corresponding to the number of pixels have not been described in detail, the threshold value and the range may be determined from a sample image composed of a large number of object images.

In the above description, the size of the detection window DW is made constant and the size of the detection target image is resized. However, the detection target image is not resized, and the size of the detection window is detected. It is also possible to adopt a configuration in which resizing (enlargement or reduction) is performed within a range not exceeding the size of the image.
In the above description, the case where the binary image is generated from the detection target image has been described.
An n-value image having three or more values may be generated and processed in the same manner.

It is a general | schematic block diagram of the object detection apparatus of embodiment. It is an object detection functional block diagram of an object detection apparatus. It is an operation | movement flowchart of embodiment. It is an outline explanatory view of integral image generation. It is explanatory drawing of the binary image for the concrete description of integral image generation. It is explanatory drawing of the produced | generated integral image. It is explanatory drawing of the setting state of a detection window. It is explanatory drawing of calculation of reliability. It is explanatory drawing of an example of the face image detection result as an object image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Object detection apparatus, 11 ... Detection apparatus main body, 12 ... Imaging camera, 13 ... Input device, 14 ... Display apparatus, 15 ... Output device, 16 ... External storage device, 22 ... ROM, 23 ... Communication network, 24 ... Communication Interface unit 31... Luminance image generation unit 32. Edge image generation unit 33. Resize image generation unit 34. Color gamut determination unit 35 35 Binary image generation unit 36.
Integrated image generation unit 37... Threshold change unit 38 38 Certainty determination unit 39 39 Discrimination unit F1, F2.
Face image.

Claims (6)

For each pixel constituting the detection target image extracting an image feature amount, the image feature amount is equal to or a pixel each pixel constituting the object based on whether or not a predetermined condition is satisfied, An image conversion unit that converts the detection target image into an n-value (n is an integer of 2 or more) image based on the determination result ;
An integrated image generating unit that generates an integrated image of the detected image for determining whether or not a certain pixel range image forming the detection target image forms an object image from the n-value image;
By integrating the values of pixels included in the detection window set for the integrated image of the detection target image, it calculates the certainty factor of the detection window, whether crab said confidence is a predetermined threshold value or more Based on an object determination unit for determining whether an image in the detection window is an object image constituting the object;
Including an object detection device. The object detection device according to claim 1,
The integrated image generation unit integrates the values of all pixels in a rectangle having a predetermined reference position pixel and the target pixel as vertices on a diagonal line to obtain an integrated value corresponding to the target pixel, and configures the detection target image The integral image corresponding to all the pixels to be calculated is generated,
Object detection device. In the object detection device according to claim 1 or 2,
Further comprising a resized image generation unit for generating a resized image by performing a resizing of the detection target image,
The image conversion unit extracts an image feature amount for each pixel constituting the resized image,
Wherein each pixel image characteristic amount based on whether a predetermined condition is satisfied, it is determined whether or not the pixels constituting the object, based on the determination result, the detection target image n values (n Is an integer greater than or equal to 2)
The integral image generator generates an integral image of the resized image to the image of the predetermined pixel range that constitutes the resized image from the n value image to determine whether the confidence factor forming the object image,
The object determination unit integrates the values of the pixels included in the detection window set for the integral image before Symbol resized image, and calculates the certainty factor of the detection window, a threshold value or more in which the certainty factor is predetermined Based on whether or not the image in the detection window is an object image constituting the object,
Object detection device. In the object detection device according to claim 1 or 2,
Resizing the detection window,
The object determination unit determines whether an image corresponding to a detection window set based on an integral image corresponding to the detection target image is an object image constituting the object;
Object detection device. In the object detection apparatus according to any one of claims 1 to 4,
The image conversion unit converts the image feature amount into the n-value image based on at least one of a color gamut, luminance, and edge;
Object detection device. The object detection apparatus according to claim 5, wherein
The object image is a face image;
The color gamut corresponds to the face image.
Object detection device.

Images