JP6350018B2 - Object detection device and element selection device - Google Patents

Description

  The present invention relates to an object detection device and an element selection device.

  Conventionally, a technique for identifying an object such as a pedestrian outside the vehicle or another vehicle from information (observation data) acquired by a sensor such as a camera or Lidar mounted on the vehicle has been proposed. As an example of a technique for identifying such an object, there is a method of matching observation data with a classifier (model) created in advance.

  The discriminator can be created by learning in advance using Pos data (data to be discriminated) and Neg data (data other than the discriminating target) as learning data, and is generally expressed in the form of feature values. Is.

  The feature amount is a parameter representing the feature of an object obtained by converting input data, and representative features include HoG and SIFT. The inner product calculation (high-dimensional vector calculation) between the input data converted into the feature quantity and the discriminator is performed while changing the target position of the image, and the similarity is calculated at each position. Thereafter, the target object can be identified by performing threshold processing on the similarity calculated at each target pixel position.

  By using a discriminator, it is possible to identify an object with high accuracy. However, since it is necessary to perform a high-dimensional inner product calculation, the processing load increases and the processing time tends to increase. In order to solve such a problem, it is conceivable to compress the feature dimension for performing the inner product calculation.

  For example, the feature quantity of the input pattern is decomposed into its element vector, and a discrimination matrix obtained by discriminant analysis for each feature vector is prepared in advance. After projecting each feature vector generated from the observation data into the discriminant space defined by the discriminant matrix and compressing the dimensions, the obtained feature vectors are combined and projected again using the discriminant matrix. A method of compressing dimensions has been proposed (see Patent Document 1). Thereby, the dimension of the feature amount can be compressed from the S dimension to the T dimension (S> T).

JP 2009-140513 A

  In the technique of Patent Document 1, after calculating an S-dimensional feature quantity once in advance, the dimension compression is performed to the T dimension. Therefore, although it is possible to increase the speed of the inner product calculation between the dimensionally compressed feature quantity and the classifier, there is a problem that the memory usage increases and the processing time increases at the stage of calculating the dimensionally compressed feature quantity in the first place. Resulting in.

  An object of the present invention is to propose an object detection device and an element selection device that can reduce the processing load.

The first aspect of the present disclosure includes feature amount acquisition means (11, S42) and binarization means (11, S
45) and a detection means (11, S46, S47).
The feature quantity acquisition unit acquires, from the observation data, a plurality of elements of the feature quantity that can be acquired from the observation data that is acquired by the observation unit (3) and varies depending on the presence of the object.

  Based on a plurality of feature quantity elements acquired by the feature quantity acquisition means, the binarization means binarizes each element of the feature quantity with a predetermined threshold as a reference (for example, “0” or “1” The binarized feature value converted into a value shown in FIG.

The detecting means detects the object based on the degree of similarity between the plurality of binarized feature amounts calculated by the binarizing means and the object model prepared in advance.
The binarizing means binarizes the elements of the feature amount using a predetermined threshold for each element.

  The object detection apparatus configured as described above executes the object detection process using a binarized value of the feature amount acquired from the observation data acquired by the observation unit. Therefore, compared to the case of detecting an object based on the degree of similarity between a non-binarized feature quantity acquired from the entire observation data and the object model, the inner product calculation is faster and the feature quantity is not a floating point. The processing load can be reduced, such as a reduction in memory usage during processing by being expressed in binary.

The 2nd mode of this indication is an element selection device provided with element selection means (11, S3-S8). The element selection means is data obtained by the observation means (3), and uses any one or more of the plurality of elements of the feature quantity that can be obtained from the observation data that changes depending on the presence of the target object. The plurality of elements used in the detecting means (11, S25, S26) for detecting the object from the observation data based on the similarity between the object model and the object model prepared in advance.

  For each of the plurality of elements, the element selection means selects an element determined to be useful for identifying whether or not the observation data is observation data in which the object is present, It is characterized by selecting from.

  The element selection device configured as described above selects an element that is determined to be useful for identifying whether or not the observation data is observation data in which the object exists. A detection means performs the detection process of a target object using only the element. Therefore, when all elements of feature quantities are obtained from observation data and compared with the case of detecting an object based on the similarity between the feature quantity and the object model, the elements that are useful are used with high accuracy. In addition to enabling detection processing, it is possible to increase the processing speed and reduce the amount of memory used in the processing.

  In addition, the code | symbol in the parenthesis described in this column and a claim shows the correspondence with the specific means as described in embodiment mentioned later as one aspect, Comprising: The technical scope of this invention is shown. It is not limited.

It is a block diagram which shows the whole structure of an obstruction detection system. It is a flowchart explaining the process sequence of a feature-value selection process. It is a figure explaining the calculation method of the feature-value. It is an example of a histogram of HoG features. It is an example of the vote number histogram based on Pos data and Neg data. It is a figure which shows the state which took the difference of the vote number histogram. It is an example of the vote number histogram based on Pos data and Neg data. It is a flowchart explaining the process sequence of an object identification process. It is a flowchart explaining the process sequence of a threshold value calculation process. It is a figure explaining the threshold value determination method based on the vote number histogram of Pos data and Neg data, and the element of the feature-value after binarization of each element. It is a flowchart explaining the process sequence of the object identification process by binarization. It is a figure explaining the feature-value selection method of the modification.

Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
(1-1) Configuration of Obstacle Detection System The obstacle detection system 1 of the present embodiment is a system that is used by being mounted on a vehicle such as an automobile, and as shown in FIG. 5 and an object detection device 7. The obstacle detection system 1 detects an object to be detected (hereinafter simply referred to as an object) such as a person or an object around the vehicle from a captured image taken by the in-vehicle camera 3, and the detected object An object can be displayed on the display means 5.

  The in-vehicle camera 3 is a photographing device that photographs the periphery of the vehicle, and for example, a known CCD image sensor or CMOS image sensor can be used. The in-vehicle camera 3 captures the vehicle periphery centered on the traveling direction of the vehicle at a predetermined time interval (100 ms as an example), and outputs the captured image to the object detection device 7.

  The display unit 5 includes a liquid crystal display or a head-up display that displays an image, and displays an image according to a signal input from the object detection device 7. For example, a warning screen is displayed when an object that is highly likely to collide with a vehicle is detected.

The object detection device 7 acquires a captured image captured by the in-vehicle camera 3 and detects a predetermined object by image processing. In addition, it is predicted whether or not the detected object and the vehicle collide.
The object detection device 7 includes a CPU 11, a ROM 13 that stores a program executed by the CPU 11, a RAM 15 that is used as a work area when the CPU 11 executes a program, a nonvolatile memory 17 such as a flash memory and an EEPROM, and the like. The computer system is configured to execute a predetermined process by executing a program.

  The CPU 11 performs processing for detecting an object from a captured image using a feature amount calculated based on the gradient strength and gradient direction of an edge, which is known as HoG (Histogram of Gradient). This feature amount is calculated for each predetermined unit region (unit image 53 described later) in the image, and can be expressed as a feature amount including a plurality of elements for each unit region. Hereinafter, a set of a plurality of elements calculated for each unit area is referred to as a HoG feature.

  The non-volatile memory 17 is set according to the information of the classifier that is a model used for the matching process executed when detecting the target object from the captured image captured by the in-vehicle camera 3 and the target image. Information on feature amounts to be acquired from the image is stored.

  The discriminator is data representing a typical HoG characteristic of an object, and is created based on captured images of a large number of objects photographed in advance. The nonvolatile memory 17 stores various data used for processing by the CPU 11.

(1-2) Process by CPU11 CPU11 of the target object detection apparatus 7 of this embodiment performs a feature-value selection process and an object identification process.

(1-2-1) Feature Quantity Selection Process The feature quantity selection process executed by the CPU 11 of the object detection device 7 will be described based on the flowchart shown in FIG. This process is a process of selecting an element to be used for detecting an object in an object identification process to be described later from HoG features including a plurality of elements. Since the HoG feature is multidimensional information including a plurality of elements, the processing load is reduced by reducing the number of dimensions.

  In S1, the CPU 11 inputs learning data. The learning data is a plurality of image data, Pos (Positive) data that is captured image data obtained by capturing a common object, and Neg (Negative) that is captured image data other than the common object. Data. The learning data includes many different data for each of Pos data and Neg data.

  In S <b> 2, the CPU 11 calculates a feature amount for each input learning data. In the present embodiment, k-dimensional (9-dimensional in this embodiment) information is calculated as the HoG feature. In the following description, k is the number of dimensions of the HoG feature and indicates the number of feature amounts included in the HoG feature. Hereinafter, a method for calculating the feature amount will be described.

  As shown in FIGS. 3A and 3B, the learning data input image is a unit obtained by dividing the basic image 51 into units of a rectangular area (Q and R are natural numbers) composed of a plurality of Q × R pixels. This is an image 53. In the present embodiment, a rectangular area composed of 4 × 4 pixels is used as the unit image 53.

  For such a unit image 53, the gradient intensity E and the gradient direction θ are calculated for each pixel constituting the image according to the following equations (1) to (4). In the following expression, I (x, y) represents the luminance value at the pixel (x, y) where x is the horizontal coordinate of the image and y is the vertical coordinate.

For each unit image 53, the gradient intensity E (x, y) obtained from each pixel of the image is divided into a predetermined gradient direction (0 ° to 180 ° in k (k = 9) directions every 20 °. Each direction) is binned, and a histogram is formed as shown in FIG. This histogram shows the HoG feature, and this HoG feature has k feature amounts H (a) (a is an integer of 1 to 9). In the present embodiment, the voting value of the histogram is normalized for each unit image 53, and a normalized feature amount in which the range that the voting value can take is 0 to 1 is used.

  Note that the above HoG feature calculation method is merely an example, and the HoG feature can be obtained by various calculation methods such as setting the vote value to 1 instead of E. Moreover, although said E and (theta) were computed from the brightness | luminance, it is not restricted to this. For example, let I be R, G, and B of a pixel, and execute formulas (1) to (4), respectively, and use the value that maximizes E as the final E and θ values for that pixel. It may be adopted.

  When the calculation of HoG features is completed for all learning data, the process proceeds to S3. From one learning data, the number of HoG features of the unit images 53 included in the basic image 51 is calculated.

  In subsequent S3 to S8, for any one of the k types of elements H (a) constituting the feature amount obtained from each of the plurality of learning data calculated in S2, the element H (a) Is an element to be used for detection of an object (that is, an element useful for identifying whether or not it is data of a captured image in which the object exists). Then, S3 to S8 are repeated until the determination of the k types of elements is completed.

  In S3, the CPU 11 creates a vote count histogram. Here, based on the HoG features of the plurality of learning data calculated in S2, attention is paid to any one type of element H (a) among the elements of the k types of feature values, and the abscissa indicates the element vote value. (Size) H (a), the vertical axis represents the number of data (data appearance frequency), and histograms D (Pos) and D (Neg) are created from Pos data and Neg data, respectively. FIGS. 5A and 5B are examples of the vote count histograms of the elements H (1) and H (2), respectively.

The horizontal axis and the vertical axis described above are examples of one axis and the other axis in the present invention. The vertical axis and the horizontal axis may be interchanged. The same applies to histograms appearing in the following description.
In S4, the CPU 11 calculates a difference D (Diff) between the vote count histograms D (Pos) and D (Neg) created in S3 by the following formula.

D (Diff) = | D (Pos) −D (Neg) |
An example of the calculated D (Diff) is shown in FIGS. 6 (a) and 6 (b). After calculating D (Diff), the process proceeds to S5.

In S5, the CPU 11 calculates a threshold value for the number of votes. Here, as shown in FIGS. 6A and 6B, the number of votes Th _ij (i: feature number (i = 1,...) Where D (Diff) calculated in S4 is equal to or less than a predetermined first threshold value. , K), j = 1,..., N), the threshold number N is acquired. In the present embodiment, the first threshold value is 0, and the number of D (Diff) that is equal to or lower than the first threshold value, that is, the number of votes Th _ij of D (Diff) whose value is 0 is counted.

  Note that the intersection of the two histograms does not necessarily occur on the bin, and there is not always a bin that is zero, so the first threshold may not be zero. However, it is preferable that the value be as close to 0 as possible.

  In S6, the CPU 11 determines whether or not the threshold number N calculated in S5 is equal to or less than a predetermined second threshold. It is considered that the smaller the threshold number N, the more useful the element H (a) is in identifying whether or not the captured image data is observation data in which an object exists. The reason will be described below.

  As shown in FIG. 7, the vote number histogram of elements that are not common in representing Pos data (for example, “clothing pattern”, “background”, etc. in pedestrian identification) has a distribution of vote numbers as Pos data and Neg data. Regardless of being scattered.

  That is, since the distribution of the number of votes of Pos data and Neg data does not converge and is biased and not characteristic, it is not an effective element for detecting an object. In this case, if the difference between D (Pos) and D (Neg) is taken, the threshold number N becomes a large value because there are many points where D (Pos) and D (Neg) intersect. Therefore, it can be determined that the element H (a) is more useful as the threshold number N is smaller.

  If the first threshold value is not 0, the threshold number N increases if D (Pos) and D (Neg) approach the first threshold value or less. If the number of votes (threshold number N) in which D (Pos) and D (Neg) are close to each other is large, it can be determined that there is little difference in the characteristics of the vote number histogram between Pos data and Neg data, and it is not useful.

  On the other hand, as shown in FIGS. 5A and 5B, when both or one of the Pos data and Neg data converge, the threshold number N = 1 in FIG. 7A and the threshold value in FIG. 7B. The threshold number N of the element H (a) becomes small so that the number N = 2.

  For this reason, if the threshold number N is smaller than the predetermined second threshold, it can be determined that the element H (a) is effective for identifying Pos and Neg and is suitable for detecting an object. On the other hand, if the threshold number N is larger than the predetermined threshold, it can be determined that the element is not suitable for detection of an object (identification of whether image data is image data in which the object exists).

  Therefore, in S6, if the threshold number N is equal to or smaller than the predetermined second threshold (S6: YES), the process proceeds to S7, and the element H (a) is set as an element used for detecting the object. . Thereafter, the process proceeds to S9.

  On the other hand, if the threshold number N exceeds the predetermined second threshold (S6: NO), the process proceeds to S8, and the element H (a) is detected as an object among the elements constituting the feature amount. Set as an element not used for. Thereafter, the process proceeds to S9.

  In S9, the CPU 11 determines whether or not the setting of S7 or S8 has been completed for all the elements H (a). If all the elements H (a) have not been completed (S9: NO), the process returns to S3, and the element H (a) for which the use / nonuse determination is not performed by the threshold determination of the threshold number N is performed. Then, the processing of S3 to S8 is executed. On the other hand, if all the elements H (a) are finished (S9: YES), this process is finished.

  With the above processing, elements used in the object identification processing described later can be determined in advance before the object identification processing. Hereinafter, the determined one or more elements are also referred to as selection feature amounts. In addition, the discriminator is also expressed by the selection feature amount.

  By the way, in this embodiment, the object to be detected is not limited to one such as a pedestrian or a vehicle, but may be a plurality of types. In this case, this process is executed for each object, and a different selection feature amount is set for each object.

(1-2-2) Object Identification Processing Object identification processing executed by the CPU 11 of the object detection device 7 will be described based on the flowchart shown in FIG. In this process, a vehicle or a pedestrian is detected from a captured image captured by the in-vehicle camera 3. This process is executed at a predetermined cycle (for example, 100 ms) while the object detection device 7 is activated.

In S <b> 21, the CPU 11 acquires data of a captured image captured by the in-vehicle camera 3.
In S22, the CPU 11 selects an object to be detected. The criteria for performing the selection are not particularly limited. For example, the selection target may be automatically switched every time this process is repeated or at a predetermined cycle. The configuration may be such that one or more types of objects are selected based on the input operation.

In S <b> 23, the CPU 11 acquires information on the selection feature amount corresponding to the selected target obtained in advance in S <b> 22.
In S24, the CPU 11 calculates a selection feature amount from the captured image acquired in S21. Here, for the entire captured image, an element determined as an element to be used in the feature amount selection process is calculated.

  The feature amount calculation method is the same as the feature amount calculation method executed in S2 of FIG. 2, but elements other than the selected feature amount are not calculated. If the object is a pedestrian, if only the elements H (1), H (5), and H (9) are set to be calculated, the other elements are not calculated and HoG The feature is three-dimensional data instead of the conventional nine-dimensional (maximum obtainable from image data).

  In S25, the CPU 11 performs inner product calculation. Here, a region having the same size as the region of the discriminator corresponding to the selected object is cut out from the captured image, and the selection feature amount of the cut-out image and the selection feature amount of the discriminator are shifted while gradually shifting the extraction position. And calculate the inner product.

  In S <b> 26, the CPU 11 detects the object by performing threshold processing on the result of the inner product calculation. That is, in S25 and S26, the object discriminator and the captured image are matched by inner product calculation, and when the degree of similarity is large, it is determined that the object is the object.

  In S27, the CPU 11 outputs a detection result. Specifically, it is conceivable to display information on the detected object on the display means 5. The display content is not particularly limited, and it is conceivable to display a bird's eye view showing the positional relationship between the vehicle and the object, or to notify when there is a danger of a collision. After S27, this process ends.

(1-3) Effect In the obstacle detection system 1 according to the present embodiment, the CPU 11 of the object detection device 7 (an example of the element selection unit and the detection unit in the present invention) A feature quantity that can be acquired from data (an example of observation data in the present invention), and a selected feature quantity including one or more elements among HoG features including a plurality of elements is acquired from the data of the captured image. At this time, the CPU 11 acquires, from the captured image data, only the elements predetermined according to the object among the elements of the plurality of feature amounts as the selected feature amounts.

  Then, the CPU 11 detects the object based on the degree of similarity (matching degree) between the acquired selection feature quantity and a classifier (an example of the object model in the present invention) prepared in advance. Here, the selected feature amount is an element that is determined to be useful for identifying whether or not it is data of a captured image in which an object exists.

  The object detection device 7 configured in this way performs object detection processing using a predetermined selection feature amount from among feature amounts that can be acquired from a captured image, and therefore performs matching by inner product calculation. When performing or calculating a feature value from a captured image, it is possible to increase the processing speed and reduce the amount of memory used in the processing.

  In the present embodiment, the feature quantity acquired as the selected feature quantity is the voting value (size) of the element on the horizontal axis and the number of data (frequency of data appearance) on the vertical axis. D (Pos) (an example of the first histogram in the present invention) which is a histogram of Pos data acquired from a plurality of different learning data, and a plurality of different learning data, each of which is observed except for the object. The number of bins whose difference between D (Neg) which is a histogram of Neg data (an example of the second histogram in the present invention) and a predetermined first threshold value or less is a predetermined second threshold value or less. .

That is, the object detection apparatus 7 of the present embodiment uses only elements that are considered to have high object identification performance for each process, and thus can detect the object from the captured image with high accuracy.
In the present embodiment, the configuration of creating a histogram for each of Pos data and Neg data has been exemplified. However, the distribution of the number of data with respect to the element of the feature amount acquired from the Pos data, and the feature amount acquired from the Neg data. As long as it is determined whether or not the target object is an element of a feature quantity useful for identification based on the difference between the distribution of the number of data with respect to the element, the determination is not necessarily performed using the histogram. If there is a bias or convergence in the distribution of elements of at least one of Pos data and Neg data, it can be determined that the feature quantity has high identification performance.

Moreover, the target object detection apparatus 7 of this embodiment can detect a several target object with a several discriminator, and the element which CPU11 acquires as a selection feature-value is set for every target object.
Therefore, the target object detection device 7 of the present embodiment can detect a plurality of target objects, and can use an appropriate selection feature amount according to the target object.

[Second Embodiment]
(2-1) Configuration of Obstacle Detection System The obstacle detection system of the present embodiment has the same hardware configuration as that of the obstacle detection system 1 of the first embodiment, but is based on the CPU 11 of the object detection device 7. Since the processing is different, this point will be described.

  A problem to be solved by the obstacle detection system of the present embodiment will be described. In a method of using a discriminator (model) for detecting pedestrians and vehicles in observation data, the feature quantity generated from the discriminator and the observation data may be mainly represented by a floating point value. At this time, in the inner product calculation performed in the identification process, it is necessary to perform the process of “floating point × floating point” very many times, and there is a problem that this process requires a lot of processing time.

  Therefore, "binarization" that approximates each discriminator and feature quantity by "0" or "1" is performed to reduce memory and to calculate inner product by performing inner product operation by bit operation. Speed up processing. However, if the threshold value for binarizing the feature value is uniformly set for each element of the feature value, there is a possibility that the threshold value is not appropriate. In the present embodiment, a value suitable for identification is obtained as a threshold value when the feature value is binarized.

(2-2) Process by CPU11 CPU11 of the target object detection apparatus 7 performs a threshold value calculation process and an object identification process.
(2-2-1) Threshold Calculation Processing The threshold calculation processing executed by the CPU 11 of the object detection device 7 will be described based on the flowchart shown in FIG. This process is a process for calculating an appropriate threshold number and value for each element of the HoG feature constituted by a plurality of elements.

  In S31, the CPU 11 inputs learning data, and in S32, calculates a feature amount for the input learning data. The processes of S31 and S32 are the same as the processes of S1 and S2 described above.

In S33 and S34, the CPU 11 creates a vote-number histogram and calculates a difference in the same manner as the processes in S3 and S4 described above. FIGS. 10A and 10B show examples of the vote count histograms of the elements H (1) and H (2). The point that becomes 0 when the difference is taken is one point of Th ₁₁ in FIG. 10A and two points of Th ₂₁ and Th ₂₂ in FIG. 10B. The number of votes (size) Th _{ij of} this element becomes a threshold value in the binarization process.

  In S35, the CPU 11 calculates a threshold value for the number of votes. Here, when the difference calculated in S34 is equal to or less than a predetermined threshold (0 in the present embodiment), it is calculated as the vote (magnitude) threshold of the element at that point.

  In S <b> 36, the CPU 11 determines whether or not the vote number threshold value calculation processing has been completed for all the elements H (a). If the calculation process has not been completed for all the elements (S36: NO), the process returns to S33, and the processes of S33 to S35 are executed for the feature quantity for which the threshold value process for the number of votes is not performed. On the other hand, if the calculation process has been completed for all elements (S36: YES), this process ends.

(2-2-2) Object Identification Processing Object identification processing executed by the CPU 11 of the object detection device 7 will be described based on the flowchart shown in FIG. In this process, a vehicle or a pedestrian is detected from a captured image captured by the in-vehicle camera 3. This process is executed at a predetermined cycle (for example, 100 ms) while the object detection device 7 is activated.

In S <b> 41, the CPU 11 acquires captured image data captured by the in-vehicle camera 3.
In S42, the CPU 11 performs the same feature amount calculation as in S2 on the captured image acquired in S41.

In S43, the CPU 11 selects an object to be detected. This process is the same as S22.
In S44, the CPU 11 acquires threshold value information (number of threshold values, threshold value) at the time of the binarization process corresponding to the object selected in S43. This threshold information is information obtained by a threshold calculation process performed in advance.

In S45, the CPU 11 performs a binarization process on the element calculated in S42 based on the threshold information acquired in S44.
A method of binarization processing will be described. In FIG. 10A, when the element H (1) has a vote count less than Th ₁₁ , the similarity to Neg data is high, and when the vote count exceeds Th ₁₁ , the similarity to Pos data is high. it can. Here, the threshold value is one of Th ₁₁ as described above.

  When threshold processing is performed on the element H (1) with this threshold as a reference, it can be binarized as “0” if it is less than the above threshold, and “1” if it is above the threshold. In the example of FIG. 10A, H (1) is equal to or greater than the threshold value, and H (1) after binarization is “1”, which is represented by a one-dimensional binary feature amount.

On the other hand, in FIG. 10B, when the element H (2) has a vote count less than Th ₂₁ , similarity to Neg data is high, and in the case of a vote count greater than Th ₂₁ and less than Th _22, Pos data and In the case of Th ₂₂ or more, the similarity to Neg data is high.

In this case, since the threshold value is two, Th ₂₁ and Th ₂₂ , binarization can be realized by two threshold processes. For each threshold, if the feature amount H (2) is less than the threshold value set to "0", expressed in the order as "1" if above the threshold value, feature quantity H (2) is less than Th ₂₁ " “00”, “10” if Th ₂₁ or more and less than Th ₂₂ , and “11” if Th ₂₂ or more. In the example of FIG. 10B, H (2) after binarization is “10”. Similarly, when there are three or more thresholds, the number of dimensions of the binarized features according to the number of thresholds. Will increase.

  In S46, the CPU 11 performs inner product calculation. Here, a region having the same size as the region of the discriminator corresponding to the selected object is cut out from the captured image, and the cut-out image is binarized, the feature amount obtained by binarizing the cut-out image, the discriminator, Calculate the inner product of. The discriminator used here is a discriminator generated by learning using a binarized feature amount based on threshold information calculated in the threshold calculation processing.

  In S47, the CPU 11 detects the object by performing threshold processing on the result of the inner product calculation. In S48, the CPU 11 outputs a detection result. The processing of S47 and S48 is the same processing as S26 and S27. After S48, this process ends.

(2-3) Effects In the obstacle detection system 1 of the present embodiment, the CPU 11 of the object detection device 7 (an example of the feature amount acquisition unit, binarization unit, and detection unit in the present invention) is the in-vehicle camera 3. The HoG features that can be acquired from the captured image data are acquired from the captured image data. Then, based on the acquired HoG feature, the CPU 11 calculates an element (an example of the binarized feature value in the present invention) binarized with respect to a plurality of elements with a predetermined threshold as a reference. At this time, the CPU 11 binarizes the feature amount using a predetermined threshold for a plurality of elements.

Then, the CPU 11 detects an object based on the degree of similarity between a plurality of binarized feature amounts and a classifier prepared in advance.
Therefore, the object detection device 7 of the present embodiment executes the object detection process using the binarized value of the feature amount acquired from the captured image, so that the inner product calculation can be processed by a bit operation. Speed and memory usage can be reduced.

  In the present embodiment, the threshold values used for binarizing the feature values are each in a histogram in which the horizontal axis is the element vote value (size) and the vertical axis is the number of data (data appearance frequency). D (Pos) (an example of the first histogram in the present invention) that is a histogram of Pos data acquired from a plurality of different learning data obtained by observing a common object, and a plurality of objects each of which is observed except for the object This is a value based on a feature amount corresponding to a bin whose difference is D (Neg) (an example of the second histogram in the present invention) which is a histogram of Neg data acquired from different learning data and which is equal to or less than a predetermined threshold .

That is, the target object detection apparatus 7 of the present embodiment can detect the target object from the captured image with high accuracy because the threshold value for the binarization process is set to a value that increases the target identification performance.
Further, the object detection device 7 of the present embodiment can detect a plurality of objects by a plurality of discriminators, and the threshold value of each element used as a reference when the CPU 11 binarizes the feature amount is set for each object. Is set. Each element also has a different number of dimensions.

Therefore, the target object detection apparatus 7 of the present embodiment can detect a plurality of target objects, and can use an appropriate threshold value that exhibits good discrimination performance in each element according to the target object.
[Modification]
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and it goes without saying that various forms can be taken as long as they belong to the technical scope of the present invention.

  For example, in each of the above-described embodiments, the feature amount selection process described with reference to FIG. 2 and the threshold value calculation process described with reference to FIG. It may not be executed by the object detection device 7 but may be executed in advance by another information processing device. That is, the target object detection apparatus 7 may have a selection feature amount or threshold information in advance, or may be acquired from the outside and execute the object identification process.

  In each of the above embodiments, the configuration using captured image data as the observation data is exemplified. However, data (images) that is obtained by observation means that can observe the outside of the apparatus and whose information changes depending on the presence of the object. Or other distance measurement points), other data can be used. For example, data that can be measured by a sensor such as a laser radar (Lidar) or a radar (Radar) that can measure the distance to an object outside the vehicle (a set data of ranging points) can be used.

  Moreover, it is good also as a structure which combined 1st Embodiment and 2nd Embodiment. That is, a selection feature amount is set for each target object to be detected, the selection feature amount is acquired from the captured image, and each of the acquired feature amounts is binarized to detect the target object. May be.

  Further, in each of the above embodiments, the configuration using the elements constituting the HoG feature as the feature amount is exemplified, but the feature amount may be obtained by a method other than the HoG feature. For example, it is conceivable to use the SIFT feature.

In the first embodiment, the feature quantity selection is illustrated using the vote histogram of Pos data and Neg data. However, the vote histogram is approximated (formulated) by a normal distribution or the like, and the average value is obtained. Based on the relationship between μ _Pos , μ _Neg and the variance values σ ² _Pos , σ ² _Neg , the feature quantity discrimination performance may be determined.

  Specifically, as shown in FIGS. 12A to 12C, when the vote count histogram is approximated to a normal distribution represented by the following equation (5), condition 1 represented by the following equation (6), and When both of the conditions 2 shown by the following equation (7) are not satisfied, it is determined that the feature amount has low identification performance, and is determined as a feature amount that is not used in the detection of the object.

  Condition 1 is that the difference between the average values is greater than a predetermined threshold, and condition 2 is that both the distribution values of the distribution of Pos data and Neg data are large. Α and β in the following formula are predetermined threshold values.

In the case of elements H (1) and H (2) of the feature quantity shown in FIGS. 12A and 12B, both conditions are satisfied, and it is determined that the feature quantity has high discrimination performance. On the other hand, in the case of the element H (3) shown in FIG. 12C, it is determined that the element is not used because the condition 2 is not satisfied.

DESCRIPTION OF SYMBOLS 1 ... Obstacle detection system, 3 ... Car-mounted camera, 7 ... Object detection apparatus, 11 ... CPU.

Claims (9)

Feature quantity acquisition means (11, S42) for acquiring, from the observation data, a plurality of elements of the feature quantity that can be acquired from the observation data that is acquired by the observation means (3) and varies depending on the presence of the object. ,
Based on the plurality of elements acquired by the feature quantity acquisition means, binarization means (11, 11) that calculates a binarized feature quantity binarized with reference to a predetermined threshold for each element of the feature quantity S45)
Detecting means (11, S46, S47) for detecting the object based on the degree of similarity between the plurality of binarized feature amounts calculated by the binarizing means and an object model prepared in advance; With
The object detection apparatus characterized in that the binarization means binarizes the feature value using a predetermined threshold for each element of the feature value. The threshold used by the binarization means to binarize the elements of the feature amount is a histogram in which one axis is the voting value of the element and the other axis is the number of data. The difference between the first histogram acquired from a plurality of different observed data observed and the second histogram acquired from a plurality of different observed data, each of which is other than the object, is less than or equal to a predetermined threshold value The object detection device according to claim 1, wherein the object value is a value of an element of the feature amount corresponding to a bin. The object detection apparatus according to claim 2, wherein in the binarization unit, a threshold value of a histogram difference is different for each element of the feature amount. In the binarization means, the number of dimensions of the binarized feature quantity differs for each feature quantity element according to the number of threshold values used for binarizing each element of the feature quantity. The object detection device according to claim 2 or 3. In the binarization means, when the number of thresholds used for binarizing each element of the feature quantity is 0, the element is not used for detection of the object by the detection means. The target object detection apparatus according to any one of claims 2 to 4. The detection means can detect a plurality of the objects by a plurality of the object models,
The threshold value of each of the plurality of elements to be used as a reference when the binarization means binarizes is set for each of the objects. 6. The object detection device according to item. Prepared in advance with the observation data by using any one or more of the plurality of elements of the feature quantity that can be acquired from the observation data that is acquired by the observation means (3) and changes depending on the presence of the target object One or more elements selected from the plurality of elements used in the detection means (11, S25, S26) for detecting the object from the observation data based on the degree of similarity with the object model thus selected Element selection means (11, S3-S8) for
The element selected by the element selection means is a histogram with the abscissa indicating the vote value of the element and the ordinate indicating the number of data, each acquired from a plurality of different observation data obtained by observing the common object. The difference between the first histogram created from the generated element and the second histogram created from the element obtained from a plurality of different observation data, each of which is observed except for the object, is a predetermined first An element selection device , wherein the number of bins equal to or less than a threshold is determined to be useful for identifying the object by being equal to or less than a predetermined second threshold . The observation means is configured to be able to acquire an image,
The feature amount is indicated by a feature amount histogram which is a histogram generated by voting the luminance gradient intensity of the unit region with the luminance gradient direction of the unit region included in the image as a bin. ,
The element selection apparatus according to claim 7, wherein the plurality of elements are each bin of the feature amount histogram . The element selection device according to claim 7 or claim 8,
Among the plurality of elements to a data acquired by the observation means (3) constituting the feature quantity that can be obtained from the observation data which varies by the presence of the object selected by the element selection means provided in the said element selection device using one or more elements comprises a detecting means for detecting (11, S25, S26) the object from the observation data on the basis of the degree of similarity between the observed data and the previously prepared object model ,
The detection means can detect a plurality of the objects by a plurality of the object models,
The element selected by the element selection means is selected for each of the objects.