JP2009301104A - Object detecting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy of detection of an object in an object detecting apparatus in which feature amount based on intensity gradient is used. <P>SOLUTION: In the object detecting apparatus which calculates first feature amount of an area based on the intensity gradient of the area which is divided into a plurality of pieces within an input image, calculates second feature amount which combines the first feature amounts of a plurality of areas whose positions are different with each other, and identifies the object within the input image based on the second feature amount, the second feature amount is made to be input, then a weak identifier for identifying whether it is the object or not by the fact whether or not conditional probability of being the object to be calculated from weight of a learning sample is higher than conditional probability of being one except for the object. Then a first strong identifier for selecting the second feature amount useful for identifying the object is configured in each of all areas of the input image by a first AdaBoost using a plurality of weak identifiers, then combination of areas useful for identifying the object is selected from the second feature amount corresponding to all combination of the plurality of areas by second AdaBoost using the first strong identifier. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection apparatus that detects an object from an input image.

In recent years, the realization of human detection is expected in the fields of security, ITS (Intelligent Transport System), marketing, etc., and many methods have been proposed. Conventionally, as an appearance feature used for human detection, a method using HOG (Histograms of Oriented Gradients), which is a feature vector in which the gradient direction of luminance in a local region is histogrammed, has been proposed (see Non-Patent Document 1). Since this HOG is not easily affected by illumination fluctuations and shadows and is robust against local geometric changes, many human detection methods using HOG have been proposed (Non-Patent Documents 2 to 4).
N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection”, IEEE Computer Vision and Pattern Recognition, pp. 886-893, 2005 F. Suard and A. Broggi, “Pedestrian Detection using Infrared images and Histograms of Oriented Gradients”, IEEE Symposium on Intelligent Vehicule, pp. 206-212, Jun, 2006 Q.Zhu, S.Avidan, M.Yeh and K.Cheng, “Fast Human Detection Using a Cascade of Histograms of Oriented Gradients”, IEEE Computer Vision and Pattern Recognition, Vol. 2, pp. 1491-1498, Jun, 2006 Year Takuya Kobayashi, Akinori Hidaka, Takio Kurita, “Feature Selection for Object Identification Using Histograms of Oriented Gradients”, IEICE Technical Report, Vol. 106, pp. 119-124, Mar, 2007

  However, a low-level feature such as an HOG feature may have a pattern that is difficult to identify a detection target using only a single feature.

  Therefore, in view of the above points, the present invention has an object of improving the object detection accuracy in an object detection apparatus using a feature amount based on a luminance gradient.

In order to achieve the above object, according to the first aspect of the present invention, the first feature value for calculating the first feature value of the region based on the luminance gradient of the region obtained by dividing the input image into a plurality of regions. A calculating means; a second feature quantity calculating means for calculating a second feature quantity by combining the first feature quantities of the plurality of regions having different positions; and the input based on the second feature quantity. An identification unit that detects an object to be detected from an image; and a learning unit that constructs the identification unit by learning using a learning sample as the input image,
The learning means receives the second feature amount and determines whether a conditional probability that is the object calculated from a weight of the learning sample set in advance is higher than a conditional probability that is other than the object. In each of the regions of the input image, a weak classifier construction unit that constructs a weak classifier that identifies the object or other than the object, and a first Adaboost using a plurality of the weak classifiers, A first strong classifier constructing unit that constructs a first strong classifier that selects the second feature quantity useful for identifying the object, and a second AdaBoost using the first strong classifier, so that a plurality of A second strong classifier construction unit that selects combinations of the regions useful for identifying the object from the second feature values corresponding to all combinations of the regions; and
The discriminating unit detects the object from the input image based on the second feature amount corresponding to the combination of the regions selected by the second strong discriminator constructing unit.

  In this way, by using the second feature value obtained by combining the first feature values of a plurality of regions having different positions, the first feature value can be captured at a plurality of locations at the same time. It becomes easy to capture the feature on the appearance of the object, and the identification accuracy can be improved. Further, by using two-stage Adaboost, only the second feature amount effective for identifying the detection target can be selected, and high-precision identification is possible.

  The invention according to claim 2 further comprises third feature amount calculation means for calculating a state of the region based on a luminance change of each region in the image frame in the input image as a third feature amount, The second feature quantity calculating means combines the first feature quantities of the plurality of regions having different positions, the third feature quantities, or the combination of the first feature quantity and the third feature quantity. The second feature amount is configured to be calculated. In this way, by using the first feature quantity based on the luminance gradient and the third feature quantity based on the luminance change in combination, other objects having a shape similar to the detection target that is difficult to identify only by the first feature quantity, etc. Can be prevented from being erroneously detected, and the object identification rate can be improved.

  In the third aspect of the invention, the third feature amount calculation unit includes a third feature amount detection unit that detects a change amount of luminance of the region in the image frame, and the change amount is determined in advance. A first state determination unit that determines a moving state when the amount of change is larger than a predetermined change amount; and a variance calculation unit that calculates a variance of luminance in a region corresponding to the region in a plurality of image frames after the image frame And a second state determination unit that determines a background state or a static state when the first state determination unit determines that the state is a moving state and the variance is smaller than a predetermined value that is set in advance. Third state determination means for determining that the background of the image frame region belongs to the background luminance set in advance and determining that the background is still or the still state when not belonging to the background. And For example, the background the area is characterized by being configured to calculate a dynamic state, the third feature quantity the result of determination in either static state. In this way, by determining each pixel as one of the background, the moving state, and the static state, the identification rate can be improved even when the object is in the static state.

  According to a fourth aspect of the present invention, the first feature amount calculating unit is configured to multi-resolution the input image, and the co-occurrence feature calculating unit includes a plurality of the input images having different resolutions. The second feature value is calculated by combining the first feature values. Thus, for example, if the detection target is a person, the second feature value is obtained by combining the feature value calculated from the resolution image that allows easy selection of facial features and the feature value calculated from the resolution image that allows easy selection of upper body features. It can be calculated, the input image can be observed in various ways, and highly accurate detection can be performed.

  In addition, the code | symbol in the bracket | parenthesis of each said means shows the correspondence with the specific means as described in embodiment mentioned later.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. The object detection device according to the present embodiment analyzes an image acquired by an imaging unit and detects an object included in the image. The object detection device of this embodiment is configured as a human detection device that detects a person from an input image. As the imaging means, a fixed camera often used for a surveillance camera, a mobile camera such as an in-vehicle camera, or the like can be used.

  FIG. 1 shows the configuration of the object detection apparatus of this embodiment. The object detection device can be configured by a known computer including a CPU, a memory, and the like. As shown in FIG. 1, the object detection apparatus includes a feature amount calculation unit 10 and an identification unit 20. Each of these components 10 and 20 is realized by the CPU executing a predetermined program.

  The feature amount calculation unit 10 calculates a HOG (Histograms of Oriented Gradients) feature amount as a feature amount (first feature amount) based on the luminance gradient of the input image. The identification unit 20 calculates a joint HOG feature amount (second feature amount) obtained by combining HOG feature amounts of a plurality of different cells (regions) in the input image, and detects an input image based on the joint HOG feature amount. Whether or not a person is included is identified. The discriminating unit 20 is configured as a cascade type discriminator based on the well-known AdaBoost. The discriminator constructed by Adaboost performs re-learning with emphasis on data in which the discriminant function has caused erroneous recognition. After this process is repeated T rounds, a final discriminant function is generated by ensemble of discriminant functions of the generated discriminator group. A learning sample is used for learning by the identification unit 20. The learning sample is made up of images of detection target images (human images) and non-detection target images (non-human images). The feature amount calculation unit corresponds to the first feature amount calculation unit of the present invention, and the identification unit 20 includes the identification unit, the learning unit, the weak classifier construction unit, the first strong classifier construction unit, the second of the present invention. This corresponds to strong classifier construction means.

  In the AdaBoost classifier, all the discrimination results of the weak classifiers (“1” for the detection target image and “0” for the non-detection target image) are supplied to the coupling machine. The input image is detected by weighting and adding the reliability calculated at the time of learning for each corresponding weak classifier, outputting the weighted majority result, and evaluating the output value of the combiner. It is determined whether or not it is a target. By constructing the discriminator in a cascade type, it is possible to suppress the false detection rate without reducing the detection rate of the detection target.

  As shown in FIG. 1, the identification unit 20 of the present embodiment is an Adaboost classifier constructed in two stages. First, a joint HOG feature pool that is an aggregate of joint HOG features obtained by combining two low-level features of HOG features with different positions is created by the first-stage Adaboost classifier. Next, a final discriminator is constructed by a second-stage Adaboost discriminator using the joint HOG feature as an input. The second-stage Adaboost classifier automatically selects joint HOG features that are optimal for human detection from the joint HOG feature pool.

  Hereinafter, a learning method based on Adaboost constructed in two stages in the identification unit 20 will be described.

  First, the HOG feature amount based on the luminance gradient of each pixel (pixel) in the input image is calculated. The feature amount calculation unit 10 calculates the HOG feature amount. The feature amount calculation unit 10 uses HOG (Histograms of Oriented Gradients) based on appearance (image appearance) as low-level features. “HOG” is a feature vector in which the gradient direction of luminance in the local region is histogrammed, can obtain the contour information of the object, and can represent the shape (edge) of the object. Since the gradient of the neighboring pixels is made into a histogram by the local region, it is not easily affected by illumination or shadow, and is robust to local geometric changes.

  FIG. 2 shows the structure of the input image, FIG. 2 (a) shows the input image, FIG. 2 (b) shows the state in which the input image is divided into a plurality of cells, and FIG. This shows the state where normalization is performed. Note that the input image of the present embodiment is composed of 30 × 60 pixels.

  First, the feature amount calculation unit 10 calculates the luminance gradient of each pixel. The luminance gradient indicates the degree of luminance change in the vicinity of the target pixel, and has a large value in the boundary region (contour) of the object in the input image. Here, with respect to all the pixels included in the target input image, the luminance gradient strength m and the gradient direction θ are calculated from the luminance L of each pixel using Equation 1.

ここで、ｆｘ（ｘ，ｙ）はｘ方向（図２の左右方向）の輝度の差分であり、ｆｙ（ｘ，ｙ）はｙ方向（図２の上下方向）の輝度の差分であり、これらは数式２により算出することができる。

Here, fx (x, y) is a luminance difference in the x direction (left and right direction in FIG. 2), and fy (x, y) is a luminance difference in the y direction (up and down direction in FIG. 2). Can be calculated by Equation 2.

次に、特徴量算出部１０は、数式１で算出された勾配強度ｍと勾配方向θを用いて、勾配方向ヒストグラムの作成を行う。勾配方向ヒストグラム作成は、複数のピクセルからなるセル単位で行う。図２（ｂ）に示すように、本実施形態では、５×５ピクセルを１セルとした領域において、輝度の勾配方向ヒストグラムを作成する。ただし、算出された勾配方向は０°〜３６０°となるが、ここでは勾配方向の向きを考慮する必要がないため、０°〜１８０°の範囲とする。本実施形態では、勾配方向を２０°ずつに分割し、各方向毎に１セルに含まれる各ピクセルの勾配強度ｍを加算して９方向の勾配方向ヒストグラムを作成する。このため、１セル当たり９次元の特徴量が存在する。本実施形態では入力画像に６×１２個のセルが存在し、７２個のセルそれぞれに対して勾配方向ヒストグラム（Ｖ＝［ｖ₁，ｖ₂，ｖ₃，…，ｖ₉］）を作成する。

Next, the feature quantity calculation unit 10 creates a gradient direction histogram using the gradient strength m and the gradient direction θ calculated by Equation 1. Gradient direction histogram creation is performed in units of cells composed of a plurality of pixels. As shown in FIG. 2B, in this embodiment, a luminance gradient direction histogram is created in a region where 5 × 5 pixels are one cell. However, although the calculated gradient direction is 0 ° to 360 °, it is not necessary to consider the direction of the gradient direction here, so the range is 0 ° to 180 °. In the present embodiment, the gradient direction is divided by 20 °, and the gradient intensity histogram of nine directions is created by adding the gradient strength m of each pixel included in one cell for each direction. For this reason, there is a nine-dimensional feature amount per cell. In this embodiment, 6 × 12 cells exist in the input image, and a gradient direction histogram (V = [v ₁ , v ₂ , v ₃ ,..., V ₉ ]) is created for each of 72 cells. .

  Next, the feature amount calculation unit 10 normalizes the luminance gradient histogram created in each cell. In other words, since there is a case where luminance deviation is included in each cell, normalization is performed in units of blocks including adjacent cells and averaged. In this embodiment, normalization is performed with 3 × 3 cells as one block. Since each cell has 9-dimensional feature values, each block (= 3 × 3 cells) has 81-dimensional feature values. Finally, the HOG feature value is normalized by the following Equation 3.

ここで、ｖは正規化後のＨＯＧ特徴量、ｋはブロック内のＨＯＧ特徴量の数、εは分母が０の場合に計算不能になることを防ぐ係数である。

Here, v is the normalized HOG feature value, k is the number of HOG feature values in the block, and ε is a coefficient that prevents the calculation from becoming impossible when the denominator is 0.

  Normalization is performed by moving the block one cell at a time as shown in FIG. For this reason, the feature amount is normalized many times by different block regions. When the input image is 30 × 60 pixels, it can move 4 blocks in the x direction and 10 blocks in the y direction, so normalization is performed for a total of 40 blocks. The HOG feature vector normalized for each block is 3240 dimensions (= 40 blocks × 81 dimensions).

Through the above process, nine HOG feature values (HOGv _{1 to} HOGv ₉ ) of the cells c _{1 to} c ₇₂ in the input image are obtained, and the low-level HOG feature pool shown at the bottom of FIG. (A collection of HOG features) is created.

  Next, joint HOG features are calculated. The calculation of the joint HOG feature is performed by the identification unit 20. First, in order to generate a joint HOG feature, a co-occurrence is expressed by a plurality of HOG features. In this embodiment, the co-occurrence expression method proposed in “T. Mita, T. Kaneko and O. Hori:“ Joint Haar-like Features for Face Detection ”, ICCV, pp. 1619-1626, 2005.” Use. First, for each HOG feature of each cell, a binary code s representing “person (1)” or “non-person (0)” is calculated from Equation 4 below.

ここで、θは閾値、ｐは不等号の向きを決定する符号であり、「＋１」または「−１」をとる。Ｖ（＝［ｖ₁，ｖ₂，ｖ₃，…，ｖ₉］）は、１のセルから算出される特徴量、ｏは勾配の方向を表わす。これにより、得られた２値化符号を２つ組み合わせることで共起を表現した特徴ｊを得ることができる。

Here, θ is a threshold value, p is a code for determining the direction of the inequality sign, and takes “+1” or “−1”. V (= [v ₁ , v ₂ , v ₃ ,..., V ₉ ]) is a feature amount calculated from one cell, and o represents the direction of the gradient. Thereby, the characteristic j expressing co-occurrence can be obtained by combining two obtained binarized codes.

FIG. 3 is a diagram for explaining co-occurrence of HOG features. For example, in the input image as shown in FIG. 3, when s1 = 1 and s2 = 1, which are binarized HOG features v of two different cells cm and cn, are observed, a feature j = (representing co-occurrence 11) ₂ = 3. The feature j expressing the co-occurrence is an index number of a combination with the feature expressed in binary number, and is a combination of two feature amounts in the present embodiment, so (00) ₂ = 0, (01) ₂ = There are four values: 1, (10) ₂ = 2 and (11) ₂ = 3.

  Next, feature values expressing the co-occurrence of the HOG feature values calculated so far are combined and expressed as a mid-lebel joint HOG feature value. Using the co-occurrence of the HOG feature calculated by the above equation 4, a joint HOG feature is generated by the feature amount expressing the co-occurrence calculated from the low-level HOG feature of two cells and the first-stage AdaBoost. By using joint HOG feature values that combine HOG feature values of a plurality of different cells, it becomes possible to capture not only object shape symmetry and edge continuity, but also the relationship between cells at different positions. .

First, in the two cells cm and cn at different positions, a feature effective for identifying the detection target is selected by learning from features expressing co-occurrence. When the feature value J _t (x) = j from the input image x is observed, the first-stage Adaboost weak classifier h _t (x) is expressed by the following Equation 5. In addition, the process of Numerical formula 5 which the discrimination | determination part 20 performs corresponds to the weak discriminator construction means of this invention.

ここで、ｙは正解ラベルｙ∈｛＋１，−１｝を表わし、Ｐｔ（ｙ＝＋１｜ｊ）及びＰｔ（ｙ＝−１｜ｊ）は、それぞれＨＯＧ特徴の共起を表現したときの条件付き確率である。条件付き確率は、予め設定された学習サンプルｉの重みＤ_t（ｉ）に基づいて、次の数式６により算出することにより、誤認識した学習サンプルを重視した学習が可能となる。

Here, y represents the correct label yε {+ 1, −1}, and Pt (y = + 1 | j) and Pt (y = −1 | j) are the conditions for expressing the co-occurrence of HOG features, respectively. It is a probability. The conditional probability is calculated by the following equation 6 based on the preset weight D _t (i) of the learning sample i, so that learning with an emphasis on the erroneously recognized learning sample can be performed.

次に、１段階目の強識別器であるジョイントＨＯＧ特徴Ｈ_cm,cn（ｘ）を次の数式７により構築する。なお、識別部２０が行う数式７の処理が本発明の第１強識別器構築手段に相当している。

Next, the joint HOG feature H _{cm, cn} (x) _, which is the strong discriminator at the first stage, is constructed by the following Expression 7. Note that the processing of Equation 7 performed by the identification unit 20 corresponds to the first strong classifier construction unit of the present invention.

ここで、Ｔは学習回数、α_t ^cm,cnは１段目の弱識別器ｈｔ^cm,cn（数式５）の重みを表わしている。重みα_t ^cm,cnは学習により設定される。

Here, T represents the number of learning times, and α _t ^{cm, cn} represents the weight of the first-stage weak classifier ht ^{cm, cn} (Formula 5). The weight α _t ^{cm, cn} is set by learning.

  The process of Formula 7 is performed for all combinations of cells of the input image. In the present embodiment, since there are 72 cells for the detection window (30 × 60 pixels), there are 2556 combinations of the two cells, and a joint HOG feature pool (joint HOG feature consisting of 2556 joint HOG features). Is created). As a result, a mid-lebel joint HOG feature pool shown in the middle of FIG. 2 is created.

Next, the second stage Adaboost classifier will be described. In the second-stage Adaboost classifier, the final classifier is constructed with the joint HOG feature pool generated by the first-stage Adaboost classifier (Formula 7) as an input. In the second-stage AdaBoost, learning is performed by selecting an effective classifier from the pool of first-stage strong classifiers H _{cm, cn} (x) constructed with the joint HOG feature, and performing the second-level strong boost. Construct classifier G (c). When the strong classifier selected from the pool of strong classifiers H _{cm, cn} (x) is g _t (c), the final strong classifier G (c) obtained by the second-stage Adaboost Is obtained by the following equation (8). Note that the processing of Equation 8 performed by the identification unit 20 corresponds to the first strong classifier construction unit of the present invention.

λは識別器の閾値であり、α_tは１段階目の強識別器ｇ_t（ｃ）の重みを表わしている。重みα_tは学習により設定される。ｃｍとｃｎはセルの組み合わせを表わし、ｃ＝｛ｃｍ，ｃｎ：１≦ｍ≦７２，１≦ｎ≦７２｝である。

λ is a discriminator threshold, and α _t represents the weight of the strong discriminator g _t (c) at the first stage. The weight α _t is set by learning. cm and cn represent a combination of cells, and c = {cm, cn: 1 ≦ m ≦ 72, 1 ≦ n ≦ 72}.

  FIG. 4 is a diagram for explaining the joint HOG feature selected by the second-stage strong classifier G (c). As shown in FIG. 4, the final classifier is constructed by selecting only the feature quantity effective for discrimination from the joint HOG feature pool by the second stage of boosting.

  The learning of the identification unit 20 is completed through the above process. The identification unit 20 constructed by learning identifies whether or not a person to be detected is included in the input image using the joint HOG feature corresponding to the combination of cells selected by Equation 8.

  Next, the joint HOG feature value selected by the two-stage AdaBoost of the object detection device of the present embodiment will be described. First, in the first stage AdaBoost (Formula 7), joint HOG features of all cell regions in the input image are selected, and among them, many joint HOG features corresponding to the gradient along the human shape are selected. The weight increases. Next, in the second stage AdaBoost (Formula 8), even the joint HOG feature selected in the first stage AdaBoost tends to be difficult to select other than the human contour. This is because, in the second-stage AdaBoost feature selection, it is determined that the joint HOG feature value corresponding to other than the contour of the person is not effective for identifying the person and the person other than the person. From the above, it can be seen that by building Adaboost in two stages, features effective for human identification are automatically selected.

  Next, the results of an evaluation experiment performed with the object detection apparatus of this embodiment will be described. The images used in the experiment are images taken at a plurality of different locations such as various illuminations, backgrounds, and walking directions taken by a fixed camera. 2054 positive samples for learning, 6258 negative samples for learning, 1000 positive samples for evaluation, and 1234 negative samples for evaluation were used. In the evaluation experiment, the case where the joint HOG feature by the object detection device of the present embodiment is used is compared with the case where the low-level (low-lebel) HOG feature calculated by the feature calculator 20 is used.

  The comparison of the discrimination experiment results was evaluated by DET (Detection Error Tradeoff). In the DET, the horizontal axis represents a false detection rate (probability of recognizing a person other than a person) and the vertical axis represents a non-detection rate (probability of recognizing a person other than a person) by a log-log graph. By changing the threshold value of the discriminator, it is possible to compare the undetected rate against the false detection rate.

  FIG. 5 shows the results of the evaluation experiment. When the joint HOG feature of this embodiment is used, it can be seen that the detection accuracy is improved as compared with the case where the HOG feature is used. When the false detection rate was 5.0%, the detection rate could be improved by about 24.6%. This is because even a pattern that is difficult to identify with only a single HOG feature amount can be identified by using a joint HOG feature in which HOG features of a plurality of cells at different positions are combined. Thus, by using the joint HOG feature, it is possible to simultaneously capture the HOG feature quantity at a plurality of locations, so that it becomes easy to capture the feature on the appearance of the detection target automatically, and the identification accuracy can be improved.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  FIG. 6 shows the configuration of the object detection apparatus of the second embodiment. In this embodiment, spatio-temporal features are used as low-level features in addition to HOG features based on appearance. In this embodiment, in order to handle spatio-temporal features, a fixed camera that is often used as a monitoring camera is used as an imaging unit, and a continuous video composed of a plurality of image frames arranged in time series is used as an input image. . It should be noted that the feature quantity calculation unit 10 of the present embodiment has a first feature quantity calculation unit, a third feature quantity detection unit, a first state determination unit, a variance calculation unit, a second state determination unit, and a third state of the present invention. This corresponds to the determination means.

  In the feature amount calculation unit 10 of the present embodiment, in addition to the calculation of the HOG feature, a pixel state analysis (PSA) is performed based on the luminance change of the pixel of the input image, the state of each pixel is determined, and the PSA The feature amount is calculated. “Pixel state analysis” is a method of modeling temporal changes in the brightness of each pixel included in a frame, distinguishing each pixel into a background and foreground based on background differences, and further changing the brightness value of each pixel in the foreground. In other words, each pixel is identified as a background, a moving state, or a static state as a moving state and a static state indicating the movement of the object from the stability. The PSA feature by the pixel state analysis is a feature including both spatial information and motion information (temporal information).

  FIG. 7 shows the state transition of the pixel state analysis. As shown in FIG. 7, the initial state of each pixel is set to the background (BG), and the transition from the background (BG) to the moving state (TR) can be made, and the moving state (TR) to the background (BG). Or it can change to a static state (ST) and can change to a dynamic state (TR) from a static state (ST).

  As a basic principle for determining these three states, the point that the luminance of the pixel changes as in (1) to (3) depending on the situation is used. (1) When an object passes over a pixel, the luminance value of the pixel is accompanied by a rapid change. Thereafter, the state of anxiety temporarily continues, and after a sudden change again, the original luminance value as the background is restored. (2) When the object stops on the pixel, the luminance value of the pixel is suddenly changed and then temporarily in an unstable state, and finally becomes stable at the luminance value of the object. (3) When an environmental change occurs, such as when the sun is hidden behind a cloud, the luminance value changes gradually.

  FIG. 8 shows a time-series relationship between frames in which pixel state analysis is performed. In order to capture the state transition of a pixel, a sudden change (Motion Trigger) in the luminance value of each pixel and the stability (Stability Measure) of the luminance value are detected. Detection of a sudden change in the luminance value is performed by paying attention to a plurality of frames (5 frames in the example of FIG. 8) before the current frame t, which is an object of pixel state analysis, and detection of luminance value stability. Is performed by paying attention to a plurality of frames (5 frames in the example of FIG. 8) after the current frame t.

First, a sudden change in luminance value is detected. Here, it is assumed that the luminance value before k frames from the current frame t which is the object of the pixel state analysis is It. In order to obtain the change amount T of the luminance value, the absolute value of the difference (the maximum change amount of the luminance value) between It and _Itj of each frame from frame t to k frames before is calculated. When a sudden luminance value change occurs on the pixel, the luminance value change amount T increases. The change amount T of the luminance value can be calculated by Equation 9.

次に、ピクセルの輝度値の安定度について考える。輝度値の安定状態の検出は、現在のフレームｔより後のｋフレームに着目し、フレームｔからフレームｔ＋ｋまでの輝度値の分散の逆数として算出する。安定度Ｓは、輝度値が安定した状態では大きい値となる。安定度Ｓは、数式１０により算出することができる。

Next, the stability of the luminance value of the pixel will be considered. The detection of the stable state of the luminance value is calculated as the reciprocal of the variance of the luminance values from the frame t to the frame t + k, focusing on the k frames after the current frame t. The stability S is a large value when the luminance value is stable. The stability S can be calculated by Equation 10.

ここで、フレームの各ピクセルの判別方法を図９のフローチャートに基づいて説明する。

Here, a method of discriminating each pixel of the frame will be described based on the flowchart of FIG.

  First, it is determined whether or not the pixel state is the background or the static state, and the change amount T of the luminance value calculated by Equation 4 exceeds a predetermined change amount (S10). The predetermined amount of change is a threshold value for discriminating abrupt changes, and may be a fixed value set in advance, but in the case of a fixed value, the object moves in an area that is behind the building. However, since there is no great difference between the luminance values of the shadow portion that is the object and the background, the amount of change in the luminance value may not exceed the threshold value. However, when the object moves in the shadow area, a change larger than the average value of the past luminance of this pixel occurs. For this reason, an appropriate threshold value can be obtained by determining the predetermined change amount based on the dispersion of the luminance values of the corresponding pixels in a plurality of frames past the detection target frame t.

  If an affirmative determination is made as a result of the determination process in S10, the pixel state is set to a moving state (S11). On the other hand, if a negative determination is made, the pixel state remains the background or the static state.

  Next, it is determined whether or not the pixel state is a moving state and the stability S of the luminance value calculated by Expression 5 exceeds a predetermined stability (S12). The predetermined stability is a threshold for determining stability. As a result, if a negative determination is made, the pixel state remains in the moving state. On the other hand, if a positive determination is made, it is determined whether the luminance value of the pixel is the luminance value of the background image (S13). The background image is prepared in advance prior to the determination processing in S13, and may be updated as appropriate so as to be able to cope with environmental changes using an IIR filter or the like.

  As a result, when an affirmative determination is made, the pixel state is set to the background (S14), and when a negative determination is made, the pixel state is set to a static state (S15). By performing the above processing on each pixel included in the frame, each pixel can be classified into one of a background, a moving state, and a static state. Note that the processing of S10 and S11 in the flowchart of FIG. 9 corresponds to the first state determination means of the present invention, the processing of S12, S13 and S15 corresponds to the second state determination means of the present invention, and S12, S13 and S14. This process corresponds to the third state determination means of the present invention.

  In the pixel state analysis, since the background difference is used in addition to the inter-frame difference, even if the same frame includes a pedestrian and a stationary person, the pedestrian is in a moving state, The person who is present can be determined as a static state. Although it is difficult to obtain information on an object in a stationary state with a feature quantity indicating the movement of the object such as an optical flow, information on an object in a stationary state can be obtained by performing pixel state analysis.

  Next, the pixel state analysis result is converted into a histogram by the cell region, and a feature vector to be a PSA feature is calculated. First, a pixel state histogram is created in the same manner as HOG, using the structure of the cell region (see FIG. 2B) used in the above-described HOG feature vector histogram. Since each pixel is classified into three states (background, static state, and moving state), three feature vectors are calculated from one histogram (one cell).

  Finally, normalization by the block (see FIG. 2C) is performed in the same procedure as the HOG described in the above-described normalization of the HOG feature vector. Since one block is 3 × 3 cells, the feature vector has 27 dimensions (= 3 × 3 × 9 dimensions) per block. When the input image is 30 × 60 pixels, 40 frames per frame, so the feature vector obtained from the PSA feature is 1080 dimensions (= 40 blocks × 27 dimensions).

Through the above process, in addition to the _nine HOG feature values (HOGv _{1 to} HOGv ₉ ) of the cells c _{1 to} c ₇₂ in the input image, three PSA feature values (PSAv _{10 to} PSAv ₁₂ ) are obtained. A low-lebel feature pool is created as shown at the bottom. Further, the same processing as Expressions 4 to 10 described in the first embodiment is performed, a joint feature pool is created by two-stage Adaboost, and a final discriminator is constructed. There are three types of joint features of the present embodiment: combinations of HOG features, combinations of PSA features, and combinations of HOG features and PSA features.

  FIG. 10 shows the ratio of features selected during learning by the object detection apparatus of the present embodiment. As shown in FIG. 10, many PSA features tend to be selected in the early stage of learning, and many HOG features tend to be selected in the latter half of learning. This is because, at the time of identification, first, a PSA feature that can represent the movement of an object is used to roughly discriminate between a person and a person, and then an HOG feature amount that is appearance information to form an identification boundary in more detail. Is considered to have been selected. Furthermore, when the PSA features selected at the initial stage of learning are examined in detail, many PSA features obtained from the background of pixel state analysis are selected. This indicates that the person is discriminated based on the number of pixels occupying the background state.

  Next, the results of an evaluation experiment performed with the object detection apparatus of this embodiment will be described. In the evaluation experiment, the same learning sample image and evaluation sample image as those in the first embodiment were used. In the evaluation experiment, when the joint feature is created from the HOG feature and the PSA of the present embodiment, when the joint feature is used only from the HOG feature of the first embodiment, the lower level ( Low-lebel) HOG features were compared.

  FIG. 11 shows the results of the evaluation experiment. In the object detection device of the second embodiment, the detection rate is improved by about 9% when the false detection rate is 5%, compared with the object detection device of the first embodiment, and the detection rate is about 99%. Could get. In other words, it is possible to identify with higher accuracy when using a joint feature that uses a PSA feature based on a spatio-temporal feature in addition to a HOG feature based on appearance than when using a joint feature consisting only of HOG features. I understand.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  In the present embodiment, the HOG feature obtained from the multi-resolution image is used as the low-level feature. Specifically, the input image is multi-resolution, and joint HOG features are created by combining the HOG features of cells of images with different resolutions.

  FIG. 12 shows an input image handled by the object detection apparatus of this embodiment. The human head region and the upper body region do not necessarily have the same resolution that best represents the appearance features. For this reason, in the object detection apparatus according to the present embodiment, as shown in FIG. 12, the head image and the upper body image cut out from the input image are each converted to multiple resolutions.

  In the example of FIG. 12, the resolution of the upper body image cut out from the input image is 128 × 128 pixels, and the magnification is reduced by 0.125 to a half size. Therefore, the number of pixels of the five upper body images is 128 × 128, 112 × 112, 96 × 96, 80 × 80, and 64 × 64, respectively. Similarly, the resolution of the face image cut out from the input image is 64 × 64 pixels, and the magnification is reduced by 0.125 to a half size. For this reason, the number of pixels of the five face images is 64 × 64, 56 × 56, 48 × 48, 40 × 40, and 32 × 32, respectively. In the upper body image, the cell size is 16 × 16 pixels, and the block size is 2 × 2 cells. In the face image, the cell size is 8 × 8 pixels, and the block size is 2 × 2 cells.

  Next, calculation of co-occurrence features using multi-resolution HOG feature amounts will be described. First, the HOG feature value is calculated by fixing the cell size from the multi-resolution image. Accordingly, the HOG feature amount of each cell is calculated for a plurality of images having different resolutions, and an HOG feature pool is created. Next, co-occurrence features are calculated by using one HOG feature amount calculated from the head image and the upper body image. As a result, it is possible to express the co-occurrence between HOG feature quantities having different positions and resolutions. Then, a two-stage AdaBoost is performed to construct a final discriminator.

  Next, features selected during learning of the object detection device of the present embodiment will be described. At the initial stage of learning that tends to cause feature selection, a high-resolution HOG feature is selected for the head, and a low-resolution HOG feature is selected for the upper body. From this, it is considered that the head is easier to express the features when the resolution is higher, and the upper body is easier to express the features when the resolution is lower. If the gradient is small as in the head, the head line can be captured even in the local region, so it is considered that a high-resolution HOG feature is selected. On the other hand, the upper body has a tendency that the gradient of the shoulder line varies depending on gender, clothes, age, and the like. Since the low resolution HOG feature collects histograms over a wide range, it can absorb gradient variations and is easily selected in the upper body.

  In the object detection device according to the present embodiment described above, the HOG feature amount calculated from the resolution image that allows easy selection of the facial feature and the upper body feature, for example, by using the HOG feature amount calculated while changing the resolution of the input image. The joint HOG feature can be calculated by combining with the HOG feature amount calculated from the resolution image that is easy to select, the input image can be observed in various ways, and high-precision detection can be performed.

(Other embodiments)
In each of the embodiments described above, the object to be detected by the object detection device is a person. Furthermore, the detection target of the object detection device does not necessarily have to be the entire object, and may be a part of the object such as a human face, for example.

  In each of the above embodiments, the joint HOG feature value is calculated by combining the HOG feature values of two cells, but the number of cells to be combined is not limited to two and may be three or more.

  Further, the feature amount calculation unit 10 of the second embodiment is configured to calculate the PSA feature amount obtained by determining the state of each pixel by the pixel state analysis as the third feature amount. The third feature amount may be calculated based on the luminance change by the inter-frame difference or the background difference.

  The inter-frame difference (TD) calculates the difference between the currently input image frame and the previously input image frame, and detects a region having a large difference value as an object. It can be determined other than the moving state. Difference between backgrounds (BS) is to prepare a background image frame in which no object to be detected exists in advance and calculate the difference between the currently input image frame and the background image frame to detect the foreground. The state of each pixel can be discriminated other than the background and the background. Even when the inter-frame difference and the inter-background difference are used, similarly to the second embodiment, histogram formation by cells and normalization by blocks are performed.

  In the third embodiment, different images (a face image and an upper body image) cut out from the input image are used with multiple resolutions. However, the present invention is not limited to this, and the same image (for example, only a face image or only an upper body image) is used. May be used with multiple resolutions.

It is a figure which shows the structure of the object detection apparatus of 1st Embodiment. It is a figure which shows the structure of an input image, (a) shows an input image, (b) shows the state which divided | segmented the input image into several cells, (c) shows the state which normalizes by a block Yes. It is a figure for demonstrating the co-occurrence of a HOG feature. It is a figure for demonstrating the joint HOG characteristic selected with the strong discriminator of a 2nd step. It is a figure which shows the evaluation experiment result of the object detection apparatus of 1st Embodiment. It is a figure which shows the structure of the object detection apparatus of 2nd Embodiment. It is a figure which shows the state transition of a pixel state analysis. It is a figure which shows the time-sequential relationship of the frame which performs a pixel state analysis. It is a flowchart which shows the discrimination method of the pixel by pixel state analysis. It is a figure which shows the ratio of the feature selected at the time of the learning of the object detection apparatus of 2nd Embodiment. It is a figure which shows the evaluation experiment result of the object detection apparatus of 2nd Embodiment. It is an input image handled with the object detection apparatus of 3rd Embodiment.

Explanation of symbols

10 feature amount calculation unit 20 classifier

Claims (4)

First feature amount calculating means for calculating a first feature amount of the region based on a luminance gradient of the region obtained by dividing the input image into a plurality of regions;
A second feature amount calculating means for calculating a second feature amount by combining the first feature amounts of the plurality of regions having different positions;
Identification means for detecting an object to be detected from the input image based on the second feature amount;
Learning means that uses the learning sample as the input image and constructs the identification means by learning;
The learning means includes
Depending on whether the conditional probability that is the object calculated from the weight of the learning sample set in advance is higher than the conditional probability that is other than the object, the object or the Weak classifier construction means for constructing a weak classifier for discriminating objects other than objects;
A first strong classifier that selects the second feature quantity useful for classifying the object in each of all the regions of the input image is constructed by first Adaboost using a plurality of the weak classifiers. First strong classifier construction means;
The second Adaboost using the first strong discriminator selects a combination of the regions useful for identifying the object from the second feature amount corresponding to all the combinations of the plurality of regions. With strong classifier construction means,
The object detection device, wherein the identification unit detects the object from the input image based on the second feature amount corresponding to the combination of the regions selected by the second strong classifier construction unit. . A third feature amount calculating means for calculating a state of the region based on a luminance change of each region in the image frame in the input image as a third feature amount;
The second feature quantity calculation means combines the first feature quantities, the third feature quantities, or the first feature quantity and the third feature quantity in a plurality of regions having different positions. The object detection device according to claim 1, wherein the second feature amount is calculated.   The third feature amount calculation means is configured to detect a change amount of luminance of the region in the image frame, and detect when the change amount is larger than a predetermined change amount. A first state determination unit that determines a state, a variance calculation unit that calculates a variance of luminance in a region corresponding to the region in a plurality of image frames after the image frame, and a moving state by the first state determination unit And when the variance is smaller than a predetermined value set in advance, a second state determination unit that determines a background or a static state and a second state determination unit that determines a background or a static state And a third state determining means for determining that the region of the image frame belongs to a background when the luminance belongs to a preset background luminance, and determines that the region is a static state when not belonging to the background luminance. , Static Object detection apparatus according to claim 2, characterized in that it is configured to calculate the result of the determination in any one of states as the third feature amount. The first feature quantity calculating means is configured to multi-resolution the input image,
4. The co-occurrence feature calculating unit calculates the second feature amount by combining the first feature amounts of a plurality of the input images having different resolutions. The object detection apparatus described in 1.

Images