JP2011048485A - Device and method for detecting target - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a target detection device and a target detection method with both high recognition precision and high real time property. <P>SOLUTION: Featured values (positive data, negative data) for reference for training to be used for the recognition of an object for a plurality of image sizes are stored in a storage part 4 with consideration for the size of ROI in an image. The featured values corresponding to the distance (size) of the featured values are selected among a plurality of featured values for reference, and matching is performed under the control of a control part 4. Thus, it is possible to prevent the distance (size) of the two-dimensional image of the ROI obtained by a stereo camera 2 from becoming largely different from the distance (size) of preliminarily prepared featured values for reference. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a target detection apparatus and a target detection method for detecting a target such as a pedestrian.

  In recent years, ITS (Intelligent Transport System) has been attracting attention for the purpose of solving road traffic problems such as traffic accidents, traffic jams, and environmental pollution. With ITS, people, roads and vehicles are connected via a network using state-of-the-art information and communication technology to achieve improvements in safety, transportation efficiency and comfort, and to realize road traffic that contributes to environmental conservation. Is a social system. Within the ITS field, problems related to traffic accidents are particularly important. Although the number of traffic fatalities has been decreasing in recent years, the percentage of pedestrians among traffic fatalities is increasing. A traffic accident in a pedestrian is likely to lead to a fatal accident, and a pedestrian is a traffic weak person in a road traffic environment. Therefore, there is an urgent need to take measures to prevent traffic accidents, giving priority to pedestrians.

  As an approach from the vehicle side for preventing traffic accidents, there is active safety technology. This technology is a technology that warns the driver of obstacles around the vehicle by recognizing the surrounding environment of the vehicle, or that the vehicle itself takes an accident avoidance action. The recognition of the surrounding environment includes recognition by a distance measuring device such as a laser radar and recognition by an image processing technique by analyzing an image from a vehicle-mounted camera. Attention has been focused on environment recognition technology by sensor fusion using both technologies.

  Much research has been conducted on pedestrian recognition techniques, and various techniques have been proposed. Examples of pedestrian recognition methods that have been used so far include Neural Network (Non-patent Document 1) and SVM (Support Vector Machine) using HOG (Histograms of Oriented Gradients) features (Non-patent Document 2). .

L.Zhao, C.E.Thorpe, "Stereo and Neural Network Based Pedestrian Detection", IEEE Transaction on Intelligent Transportation System, Vol.1, No.3, September 2000, pp.148-154. F.Suard, A.Rakotomamonjy, A.Bensrhair, A.Broggi, "Pedestrian Detection using Infrared image and Histograms of Oriented Gradients", Intelligent Vehicle Symposium 2006, June 13-15, 2006, Tokyo, Japan, pp.206-212 .

  By the way, in the conventional classifier, when the training image or reference image is compared with the classification target image, it is necessary to perform scaling conversion on the classification target image. However, for example, when there is an object in the distance, the feature information included in the identification target image is greatly changed by the scaling conversion, which causes a problem that the identification accuracy is lowered. In addition, there is a problem that the time required for recognition increases as the identification accuracy decreases.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a target detection apparatus and a target detection method with high recognition accuracy and a short calculation time required for recognition.

  A first target detection apparatus according to the present invention includes storage means, feature amount extraction means, and identification processing means. The storage means is associated with the plurality of first feature quantities, the plurality of second feature quantities, the first distance information associated with each first feature quantity, and each second feature quantity. The second distance information is stored. The plurality of first feature amounts are obtained from a plurality of first two-dimensional images cut out from a plurality of first front images obtained from the first imaging means. Each first two-dimensional image includes one object having a different distance. The plurality of second feature amounts are obtained from a plurality of second two-dimensional images cut out from the plurality of first front images obtained from the second imaging unit. Each second two-dimensional image includes one non-object having a different distance. The feature amount extraction unit uses the third distance information obtained from the distance acquisition unit and the third front image obtained from the third imaging unit to determine the target candidate region included in the third front image. After the third two-dimensional image is cut out, a third feature amount is extracted from the third two-dimensional image, and distance information corresponding to the third feature amount is extracted from the third distance information. . The identification processing means includes a first feature amount associated with the first distance information that is the same as or close to the distance information corresponding to the third feature amount among the plurality of first feature amounts, and the plurality of second feature amounts. Based on the second feature amount associated with the second distance information that is the same as or close to the distance information corresponding to the third feature amount of the feature amount, and the third feature amount, The target candidate included is identified.

  A second target detection apparatus according to the present invention includes a storage unit, a feature amount extraction unit, and an identification processing unit. The storage means is associated with the plurality of first feature quantities, the plurality of second feature quantities, the first size information associated with each first feature quantity, and each second feature quantity. The second size information is stored. The plurality of first feature amounts are obtained from a plurality of first two-dimensional images cut out from a plurality of first front images obtained from the first imaging means. Each first two-dimensional image includes one object having a different size. The plurality of second feature amounts are obtained from a plurality of second two-dimensional images cut out from the plurality of first front images obtained from the second imaging unit. Each second two-dimensional image includes one non-object having a different size. The feature amount extraction unit uses the third distance information obtained from the distance acquisition unit and the third front image obtained from the third imaging unit to determine the target candidate region included in the third front image. After cutting out the third two-dimensional image, the third feature amount is extracted from the third two-dimensional image, and the distance information corresponding to the third feature amount is extracted from the third distance information and extracted. The distance information is converted into third size information. The identification processing means includes a first feature amount associated with the first size information that is the same as or close to the third size information among the plurality of first feature amounts, and the first feature amount among the plurality of second feature amounts. The target candidate included in the target candidate area is identified based on the second feature amount associated with the second size information that is the same as or close to the size information 3 and the third feature amount. It is.

The first target detection method according to the present invention includes the following three steps.
(A) A plurality of first feature amounts, a plurality of second feature amounts, first distance information associated with each first feature amount, and each second feature amount Preparation Step for Preparing Second Distance Information The plurality of first feature quantities are the same as the plurality of first feature quantities in the first target detection device. The plurality of second feature amounts are the same as the plurality of second feature amounts in the first target detection apparatus.
(B) From the third distance information obtained from the distance acquisition unit and the third front image obtained from the third imaging unit, the third candidate target area included in the third front image is obtained. After extracting the two-dimensional image, a third feature amount is extracted from the third two-dimensional image, and distance information corresponding to the third feature amount is extracted from the third distance information (C ) Among the plurality of first feature amounts, the first feature amount associated with the first distance information that is the same as or close to the distance information corresponding to the third feature amount, and among the plurality of second feature amounts The target included in the target candidate region based on the second feature amount associated with the second distance information that is the same as or close to the distance information corresponding to the third feature amount, and the third feature amount. Identification processing step for identifying candidates

The second target detection method according to the present invention includes the following three steps.
(A) A plurality of first feature quantities, a plurality of second feature quantities, first size information associated with each first feature quantity, and associated with each second feature quantity Preparation Step for Preparing Second Size Information The plurality of first feature amounts are the same as the plurality of first feature amounts in the second target detection apparatus. The plurality of second feature quantities are the same as the plurality of second feature quantities in the second target detection device.
(B) From the third distance information obtained from the distance acquisition unit and the third front image obtained from the third imaging unit, the third candidate target area included in the third front image is obtained. After cutting out the two-dimensional image, the third feature amount is extracted from the third two-dimensional image, distance information corresponding to the third feature amount is extracted from the third distance information, and the extracted distance information is obtained. Feature amount extraction step for converting to third size information (C) The first feature amount associated with the first size information that is the same as or close to the third size information among the plurality of first feature amounts, Based on the second feature amount associated with the second size information that is the same as or close to the third size information among the plurality of second feature amounts, and the third feature amount, Identification processing step for identifying included target candidates

  Here, in the identification processing means of the first and second target detection devices according to the present invention, or the identification processing step of the first and second target detection methods according to the present invention, the first feature amount and the third While matching with the feature amount, matching between the second feature amount and the third feature amount may be performed. At this time, the first correlation score is derived from the result of matching between the first feature quantity and the third feature quantity, and the second correlation score is obtained from the result of matching between the second feature quantity and the third feature quantity. The target candidates included in the target candidate area may be identified based on the first correlation score and the second correlation score.

  In the first and second target detection apparatuses of the present invention and the first and second target detection methods of the present invention, identification processing is performed based on the above-described three types of feature amounts. At this time, a first feature amount having a distance (size) that is the same as or close to the distance (size) of the third feature amount is selected from among the plurality of first feature amounts, and the first feature amount is selected from among the plurality of second feature amounts. The second feature amount having the same or close distance (size) as the feature amount distance (size) of 3 is selected. This eliminates the need to resize the feature amount obtained from the second two-dimensional image. In the identification process, for example, a classifier such as Neural Network, SVM, or AdaBoost may be used.

  According to the first and second target detection devices of the present invention and the first and second target detection methods of the present invention, the identification process is performed based on the three types of feature amounts selected according to the distance. I did it. Thereby, it is not necessary to resize the feature amount obtained from the second two-dimensional image, so that loss or addition of information due to resizing can be reduced. As a result, the identification accuracy can be increased. Also, as the identification accuracy increases, the time required for recognition can be shortened. Therefore, a target detection apparatus with high recognition accuracy and a short calculation time required for recognition can be realized.

It is a schematic block diagram of the sensor fusion which concerns on one embodiment of this invention. It is a flowchart showing an example of the detection procedure of the target in the sensor fusion of FIG. 3 shows an example of a training data and verification feature data set in the method of the present embodiment. It shows a two-dimensional image before resizing and a two-dimensional image after resizing in a conventional method. It represents an example of a feature data set for training and verification in a conventional method. An example of an ROC (receiver operating characteristic) curve in the method of the present embodiment and an example of an ROC curve in a conventional method are shown.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic configuration of a sensor fusion (target detection apparatus) according to an embodiment of the present invention. This sensor fusion is a system for detecting a target, and is mounted on the automobile C, for example. Here, the target refers to, for example, an automobile, a motorcycle, a pedestrian, an obstacle, a moving body, and the like.

  This sensor fusion includes, for example, a laser radar 1, a stereo camera 2, a control unit 3, and a storage unit 4.

  The control unit 3 is configured by, for example, a DSP (Digital Signal Processor) or the like, and processes information obtained by the laser radar 1 and the stereo camera 2 to calculate the position and type of a target ahead of the car C, for example. Is to be specified. The storage unit 4 includes, for example, a RAM (Random Access Memory) and an HD (hard disk). The storage unit 4 stores a program for calibrating sensor fusion, a program for detecting a target, a LUT (Look up Table), and the like, and the calculation results obtained by the control unit 3 are stored as needed. Stored.

  The LUT in the storage unit 4 includes information used for detecting a target and training described later. The LUT includes, for example, a plurality of types of feature amounts having different distances and distance information associated with each feature amount. Note that the LUT may include, for example, a plurality of types of feature amounts having different sizes and size information associated with each feature amount. Here, the above-mentioned size information is information about the size of the two-dimensional image used for deriving the feature amount or the size corresponding thereto, and imaging with the object or non-object included in the two-dimensional image. It has a predetermined correlation with the distance to the device. Therefore, when the size information associated with the first feature quantity and the size information associated with the second feature quantity are the same, they are used when deriving the first feature quantity. The distance between the object or non-object included in the two-dimensional image and the imaging device, and the distance between the object or non-object included in the two-dimensional image used to derive the second feature amount and the imaging device And are equal to each other.

  The feature amount included in the LUT is used for training and reference, for example. The feature amount includes a plurality of types of positive data having different distances (or sizes) and a plurality of types of negative data having different distances (or sizes). Here, positive means an object, for example, a pedestrian. Further, negative means a non-object, and means, for example, an automobile, a motorcycle, an obstacle, and the like. Further, the positive data refers to a region 2 including an object from a front image (for example, a stereo image or a single image) including the object, which is previously captured by the stereo camera 2, another stereo camera, or another monocular camera. It refers to a feature vector (feature amount) of an object obtained by cutting out a two-dimensional image by a predetermined method and extracting it from the cut out two-dimensional image. Negative data refers to a region 2 including a non-object from a front image (for example, a stereo image or a single image) including a non-object captured in advance by the stereo camera 2, another stereo camera, or another monocular camera. It refers to a feature vector (feature amount) of a non-object obtained by cutting out a two-dimensional image by a predetermined method and extracting it from the cut out two-dimensional image.

  The distance information associated with the feature amount is a distance list associated with the feature amount. For example, the distance type of the target included in the original image (two-dimensional image) from which the feature amount is derived. Is a list. The distance list includes, for example, three types of distances of 5 m, 20 m, and 35 m. The size information associated with the feature quantity is a size list associated with the feature quantity. For example, the size information associated with the feature quantity is a list of sizes of the original image (two-dimensional image) from which the feature quantity is derived. It is. The size list includes, for example, three types of sizes of 24 × 38, 40 × 80, and 56 × 112.

  The laser radar 1 is attached to, for example, a central portion of a bumper of the automobile C. The laser radar 1 emits laser light in front of the automobile C, detects reflected light of the laser light emitted in front of the automobile C, and further detects a target existing in front of the automobile C from the detected reflected light. An azimuth θ, a distance d, a relative speed v, and the like are measured.

  In general, the laser radar has a higher detection capability than the millimeter wave radar, and can measure a target such as a pedestrian that hardly reflects radio waves. In general, the laser radar is superior to the millimeter wave radar in terms of viewing angle, resolution, and followability. In terms of cost, laser radar is generally cheaper than millimeter wave radar. Therefore, in the present embodiment, the laser radar 1 is used with emphasis on detection of pedestrians and cost.

  The stereo camera 2 includes a right camera 21 and a left camera 22. The right camera 21 and the left camera 22 are composed of, for example, a charge coupled device (CCD), and are, for example, the inner walls of the windshield of the automobile C, are spaced from each other by a predetermined distance, and are mounted at the same height from the road surface. ing. Accordingly, the stereo camera 2 captures the front of the car C from different viewpoints, and the two-dimensional image (right image) captured by the right camera 21 and the two-dimensional image (left image) captured by the left camera 22. The stereo image which consists of is acquired. In addition, it is preferable that the optical axes of the right camera 21 and the left camera 22 are parallel to each other, and the respective image planes are on the same plane.

  Next, an example of a target detection method in the sensor fusion according to the present embodiment will be described with reference to FIG. In the following, (A) detection of a target candidate region (ROI: Region of Interest), (B) extraction of features, (C) detection of an object, (D) target tracking step, (E) training These will be described in order.

(A) Detection of ROI The control unit 3 first detects an ROI on a two-dimensional image (step S1).

  The ROI is detected as follows, for example. First, the control unit 3 requests the laser radar 1 to scan the laser and requests the stereo camera 2 to acquire a stereo image. Then, the laser is scanned by the laser radar 1, radar information (distance d and angle θ) is output from the laser radar 1, and a stereo image (right image, left image) is output from the stereo camera 2. Next, the control unit 3 determines whether a target candidate exists in the stereo image based on the radar information and the stereo image. As a result, when it is determined that the target candidate exists, the control unit 3 cuts out a two-dimensional image of the ROI including the target candidate determined to exist from the stereo image (for example, the left image). The size of the two-dimensional ROI image differs depending on the position cut out from the stereo image, the type of the target, and the like. Therefore, for example, the control unit 3 extracts the ROI distance from the radar information, and compares the extracted distance d1 with the distance information included in the LUT stored in the storage unit 4. In addition, the range which distance d1 can take is normally limited, for example, is 5 m or more and 35 m or less.

  The control unit 3 compares the distance d1 with the distance list read from the storage unit 4, and extracts a distance equal to or close to the distance d1 from the distance list. When the distance d1 is equal to the distance d2 extracted from the distance list, the control unit 3 maintains the size of the ROI two-dimensional image without enlarging or reducing the size. On the other hand, when the distance d1 is different from the distance d2, the control unit 3 changes (enlarges or reduces) the size of the ROI two-dimensional image to the size when the distance is d2. Here, the degree of enlargement or reduction depends on the type of distance included in the distance list.

  For example, when the distance d1 is 30 m and the distance list includes, for example, three types of distances of 5 m, 20 m, and 35 m, the control unit 3 determines the size of the ROI two-dimensional image as follows: Reduce to a distance of 35m. Since the difference between the distance d1 and the distance d2 at this time is 5 m, the degree of reduction is small. When the distance list includes three types of distances, for example, 5 m, 20 m, and 35 m, the difference between the distance d1 and the distance d2 is the largest because the distance d1 is 17.5 m or 27, for example. The distance between the distance d1 and the distance d2 is 7.5 m.

  On the other hand, for example, when the distance d1 is 30 m and the distance list includes only one type of distance of 20 m, for example, the control unit 3 determines the size of the ROI two-dimensional image as the distance. Will be enlarged to 20 m. Since the difference between the distance d1 and the distance d2 at this time is 10 m, the degree of enlargement is large. When the distance list includes only one type of distance of 20 m, the difference between the distance d1 and the distance d2 is greatest when the distance d1 is 5 m or 35 m, for example. The difference between the distance d1 and the distance d2 is 15 m.

  When the LUT stored in the storage unit 4 includes size information in which the size of the original image (two-dimensional image) from which the reference feature amount is derived is associated with the reference feature amount. For example, instead of comparing the distance d1 and the distance d2, the control unit 3 compares the size of the two-dimensional image of the ROI (the size obtained from the distance d1) with the size information included in the LUT. May be.

(B) Feature Quantity Extraction Next, the control unit 3 uses the ROI two-dimensional image after the distance (or size) correction (hereinafter, simply referred to as “corrected ROI two-dimensional image”). Then, the feature quantity of the target is extracted (step S2). Here, the feature amount of the target refers to information regarding the shape of the target.

(C) Object Detection Next, the control unit 3 identifies target candidates included in the ROI two-dimensional image using the target feature amount extracted from the ROI two-dimensional image. Specifically, the control unit 3 detects (identifies) an object (for example, a pedestrian) using the feature amount of the target extracted from the two-dimensional image of the ROI (step S3).

  The detection of the object is performed as follows, for example. For example, the control unit 3 performs matching between the feature amount of the target extracted from the ROI two-dimensional image and the feature amount stored in the LUT. For example, first, the control unit 3 has the same distance (or size) as the distance (or size) of the target feature amount extracted from the two-dimensional image of the ROI among the plurality of reference positive data stored in the LUT. Size) data. At this time, for example, the control unit 3 uses the distance information or the size information associated with the positive data stored in the LUT, and uses the ROI from among the plurality of reference positive data stored in the LUT. Data that matches the distance or size of the feature amount of the target extracted from the two-dimensional image is selected. Next, the control unit 3 performs matching between the feature amount of the target extracted from the two-dimensional image of the ROI and the selected positive data for reference, and derives a first correlation score from the result. Further, the control unit 3 performs matching between the target feature amount extracted from the two-dimensional image of the ROI and the selected negative data for reference, and derives a second correlation score from the result.

  Next, the control unit 3 identifies target candidates included in the two-dimensional image of the ROI based on the first correlation score and the second correlation score. For example, if the control unit 3 determines that the correlation between the feature amounts is greater in the first correlation score than the second correlation score, the feature amount of the target extracted from the two-dimensional image of the ROI is positive. That is, it is determined that the target candidate is an object, and the process proceeds to a target tracking step (described later). At this time, for example, the control unit 3 adds the target candidate to the tracking list in association with the position information and the correlation score (first correlation score, second correlation score), or when the target candidate has already been added. Updates the position information and correlation score of the target candidate.

  For example, when the control unit 3 determines that the second correlation score has a larger correlation between the feature amounts than the first correlation score, the feature amount of the target extracted from the two-dimensional image of the ROI is It is negative, that is, the target candidate is determined to be a non-target object, and the process does not proceed to the target tracking step (described later). At this time, for example, the control unit 3 does not add the target candidate to the tracking list, or deletes the target candidate from the tracking list if it has already been added.

  Note that it may be difficult to determine which one has the higher correlation between the feature amounts from the first correlation score and the second correlation score. In such a case, for example, the control unit 3 determines that the feature amount of the target extracted from the two-dimensional image of the ROI is indefinite, that is, the target candidate may be a target object, The process proceeds to a target tracking step (described later). At this time, for example, the control unit 3 adds the target candidate to the tracking list in association with the position information and the correlation score (first correlation score, second correlation score), or when the target candidate has already been added. Updates the position information and correlation score of the target candidate.

  In this way, the control unit 3 performs the classification matching processing on the feature amount of the target extracted from the two-dimensional image of the ROI, and then proceeds to the following target tracking step. Note that the control unit 3 can use various classifiers such as Neural Network, SVM, and AdaBoost in the above-described classification matching process.

(D) Target Tracking Step Subsequently, the control unit 3 repeatedly executes the above steps (ROI detection, feature amount extraction, object detection) to update the correlation score (step S4). As a result, in a target candidate, when the first correlation score is larger than the second correlation score in each target candidate, the feature amount correlation is larger each time it is updated, for example, the target The candidate is determined to be an object. Further, the control unit 3 determines that the correlation score does not fluctuate in a certain target candidate, or the correlation of the feature amount is larger each time the second correlation score is updated than the first correlation score. For example, it is determined that the target candidate is a non-target object.

  The phenomenon in which the correlation of the feature amount becomes larger every time the first correlation score is updated than the second correlation score is, in particular, the movement of the object approaching the car C as time passes, such as a pedestrian. Occurs when the body. On the other hand, when the object is a moving object traveling in front of the car C, such as an automobile or a motorcycle, the correlation score does not vary greatly with time, and the first correlation score is the second correlation score. In other words, the correlation between the feature amounts does not increase with each update. Therefore, it is possible to determine whether or not the target candidate is an object by observing changes in the first correlation score and the second correlation score over time.

  Note that the training of the discriminator used for detecting the object is performed as follows, for example.

(E) Training First, the trainer prepares a data set as shown in FIG. 3 as a feature vector for training. The data set includes a plurality of features obtained from a plurality of two-dimensional images including a pedestrian as a target, and a plurality of two including a target other than a pedestrian (for example, an automobile, a motorcycle, an obstacle) as a target. And feature quantities obtained from the dimensional image. Note that “pedestrian” in the figure refers to a feature amount obtained from a plurality of two-dimensional images including a pedestrian as a target. Further, “non-pedestrian” in the figure refers to a feature amount obtained from a plurality of two-dimensional images including objects other than pedestrians as targets. In this embodiment, a plurality of types of training feature values are provided. For example, as shown in FIG. 3, three types of 24 × 38, 40 × 80, and 56 × 112 are provided. Yes. Note that numbers (for example, 2939 and 1261) in the figure indicate the number of feature quantities for training.

  Next, the control unit 3 compares the feature quantity for training with the feature quantity stored in the LUT, and determines whether or not the feature quantity for training corresponds to the object. Then, the control part 3 sets the numerical value according to the determination result to a discriminator. In this way, training is performed.

  Next, the effect of the sensor fusion of this embodiment will be described.

  In the sensor fusion of the present embodiment, a laser radar 1 and a stereo camera 2 are used to detect a target. This makes it possible to detect and recognize targets such as pedestrians that are difficult to reflect millimeter-wave radar signals, so it is possible to select targets such as vehicles, pedestrians, and obstacles that are far away from the vehicle with a simple configuration. Can be detected.

  By the way, in the conventional object recognition technique, the training and reference feature values used for object recognition are normalized to a single image size regardless of the size of the ROI in the image. It was. For this reason, the size of the two-dimensional image of the ROI obtained from the stereo camera 2 often differs greatly from the size of the reference feature amount prepared in advance. As a result, it is necessary to perform large resizing on the feature amount obtained from the two-dimensional image of ROI. For example, as shown in FIGS. 4A and 4B, information loss due to resizing And addition will occur. Therefore, the conventional method has not been easy to recognize with high accuracy.

  On the other hand, in the present embodiment, the storage unit 4 includes a plurality of training and reference feature quantities (positive data, negative data) used for object recognition in consideration of the size of the ROI in the image. The image size is stored. Further, the control unit 4 performs matching after selecting a feature amount corresponding to a feature amount distance (size) from among a plurality of reference feature amounts. Thereby, the distance (size) of the two-dimensional image of the ROI obtained from the stereo camera 2 does not greatly differ from the distance (size) of the reference feature amount prepared in advance. As a result, since it is not necessary to perform large resizing on the feature amount obtained from the two-dimensional image of ROI, information loss and addition due to resizing can be reduced as compared with the conventional method. As a result, the identification accuracy can be increased. Also, as the identification accuracy increases, the time required for recognition can be shortened. Therefore, in this embodiment, it is possible to achieve both high recognition accuracy and a short calculation time.

  For example, in the conventional technique, the trainer prepares a data set as shown in FIG. 5 as a feature amount for training. This data set includes feature quantities obtained from a plurality of two-dimensional images including pedestrians as targets, and feature quantities obtained from a plurality of two-dimensional images including objects other than pedestrians as targets. It is included. Furthermore, as the size of the feature amount for training, for example, one type of 32 × 64 is provided as shown in FIG. Then, the trainer uses this data set to cause sensor fusion to perform training. Thereafter, the trainer prepares a data set as shown in FIG. 5 as a feature amount for verification. This data set also includes feature amounts obtained from a plurality of two-dimensional images including pedestrians as targets, and feature amounts obtained from a plurality of two-dimensional images including items other than pedestrians as targets. It is included. Further, a plurality of types of verification feature sizes are provided. For example, as shown in FIG. 5, three types of 24 × 38, 40 × 80, and 56 × 112 are provided. Then, the trainer uses the data set to cause the sensor fusion to detect the target. The result is shown in the ROC (receiver operating characteristic) curve of FIG.

  On the other hand, for example, in the present embodiment, the trainer prepares a data set as shown in FIG. 3 as a training feature amount, and then uses this data set to cause the sensor fusion to perform training. This data set includes feature quantities obtained from a plurality of two-dimensional images including pedestrians as targets, and feature quantities obtained from a plurality of two-dimensional images including objects other than pedestrians as targets. It is included. As the size of the feature quantity for training, for example, as shown in FIG. 3, three types of 24 × 38, 40 × 80, and 56 × 112 are provided. Thereafter, the trainer prepares a data set as shown in FIG. 3 as a feature amount for verification. This data set also includes feature amounts obtained from a plurality of two-dimensional images including pedestrians as targets, and feature amounts obtained from a plurality of two-dimensional images including items other than pedestrians as targets. It is included. Furthermore, as the size of the feature quantity for verification, for example, as shown in FIG. 3, three types of size of 24 × 38, 40 × 80, and 56 × 112 are provided. Then, the trainer uses the data set to cause the sensor fusion to detect the target. The results are shown in the ROC curve of FIG.

  FIG. 6 shows that the method of the present embodiment has a higher recognition rate and a lower recognition rate than the conventional method. From this, it can be said that by providing a plurality of types of training and verification feature vector sizes, a high recognition rate can be obtained while keeping the erroneous recognition rate low.

  While the present invention has been described with reference to the embodiment and examples, the present invention is not limited to these, and various modifications are possible.

  For example, in the above-described embodiment, the stereo camera 2 is used, but instead, for example, it is also possible to use a time-of-flight distance sensor that acquires a single image in front of the vehicle and a distance to the subject. . Further, a monocular camera can be used instead of the stereo camera 2. However, in these cases, it is necessary to replace “stereo image” with “single image” in the above embodiment.

  In the above embodiment, the laser radar 1 is used. However, the laser radar 1 can be eliminated. However, in these cases, in the above embodiment, it is necessary to obtain distance information from the parallax image obtained from the stereo camera 2.

  Moreover, in the said embodiment, although the case where the sensor fusion (target detection apparatus) was installed in the motor vehicle C was illustrated, you may install in the place other than that. In the above embodiment, a pedestrian is mentioned as an example of the target (target). However, it is possible to select a target, an animal, a plant, or the like according to the purpose. For example, the sensor fusion may be installed above a belt conveyor on which miscellaneous fish are shed in a fresh fish factory, and a specific fish may be selected as an object.

  DESCRIPTION OF SYMBOLS 1 ... Laser radar, 2 ... Stereo camera, 21 ... Right camera, 22 ... Left camera, 3 ... Control part, 4 ... Memory | storage part.

Claims (6)

A plurality of first two-dimensional images including one object each having a different distance are cut out from the plurality of first front images obtained from the first imaging means, and the plurality of first two cut out are obtained. A plurality of second features each including one non-object having a different distance from the plurality of first feature amounts obtained from the dimensional image and the plurality of second front images obtained from the second imaging means. A two-dimensional image is cut out, a plurality of second feature amounts obtained from the plurality of cut-out second two-dimensional images, and first distance information associated with each of the first feature amounts; Storage means for storing second distance information associated with each second feature amount;
From the third distance information obtained from the distance acquisition means and the third front image obtained from the third imaging means, the third two-dimensional of the target candidate area included in the third front image Feature amount extraction means for extracting a third feature amount from the third two-dimensional image after extracting an image and extracting distance information corresponding to the third feature amount from the third distance information; ,
The first feature amount associated with the first distance information that is the same as or close to the distance information corresponding to the third feature amount among the plurality of first feature amounts, and the plurality of second feature amounts Based on the second feature amount associated with the second distance information that is the same as or close to the distance information corresponding to the third feature amount, and the third feature amount, the target candidate region A target detection apparatus comprising: identification processing means for identifying target candidates included in the target. The identification processing means performs matching between the first feature value and the third feature value, derives a first correlation score from the result, and also calculates the second feature value and the third feature value. The second correlation score is derived from the result of the matching, and then the target candidates included in the target candidate region are identified based on the first correlation score and the second correlation score. The target detection apparatus according to 1. A plurality of first two-dimensional images including one object each having a different size are cut out from the plurality of first front images obtained from the first imaging means, and the plurality of first two cut out. A plurality of second features each including one non-object having a different size from the plurality of first feature values obtained from the dimensional image and the plurality of second front images obtained from the second imaging means. A two-dimensional image is cut out, a plurality of second feature amounts obtained from the plurality of cut-out second two-dimensional images, and first size information associated with each of the first feature amounts; Storage means for storing second size information associated with each second feature amount;
From the third distance information obtained from the distance acquisition means and the third front image obtained from the third imaging means, the third two-dimensional of the target candidate area included in the third front image After the image is cut out, a third feature amount is extracted from the third two-dimensional image, and distance information corresponding to the third feature amount is extracted from the third distance information. A feature amount extraction means for converting the information into third size information;
Of the plurality of first feature amounts, the first feature amount associated with the first size information that is the same as or close to the third size information, and the third feature amount among the plurality of second feature amounts. A target candidate included in the target candidate area is identified based on the second feature amount associated with the second size information that is the same as or close to the size information of the target feature and the third feature amount. A target detection apparatus comprising: an identification processing unit. A plurality of first two-dimensional images including one object each having a different distance are cut out from the plurality of first front images obtained from the first imaging means, and the plurality of first two cut out are obtained. A plurality of second features each including one non-object having a different distance from the plurality of first feature amounts obtained from the dimensional image and the plurality of second front images obtained from the second imaging means. A two-dimensional image is cut out, a plurality of second feature amounts obtained from the plurality of cut-out second two-dimensional images, and first distance information associated with each of the first feature amounts; A preparation step of preparing second distance information associated with each second feature amount;
From the third distance information obtained from the distance acquisition means and the third front image obtained from the third imaging means, the third two-dimensional of the target candidate area included in the third front image A feature amount extracting step of extracting a third feature amount from the third two-dimensional image after extracting the image and extracting distance information corresponding to the third feature amount from the third distance information; ,
The first feature amount associated with the first distance information that is the same as or close to the distance information corresponding to the third feature amount among the plurality of first feature amounts, and the plurality of second feature amounts Based on the second feature amount associated with the second distance information that is the same as or close to the distance information corresponding to the third feature amount, and the third feature amount, the target candidate region A target detection method comprising: an identification processing step for identifying target candidates included in the target. In the identification processing step, the first feature value and the third feature value are matched, a first correlation score is derived from the result, and the second feature value and the third feature value are derived. The second correlation score is derived from the result of the matching, and then the target candidates included in the target candidate region are identified based on the first correlation score and the second correlation score. 4. The target detection method according to 4. A plurality of first two-dimensional images including one object each having a different size are cut out from the plurality of first front images obtained from the first imaging means, and the plurality of first two cut out. A plurality of second features each including one non-object having a different size from the plurality of first feature values obtained from the dimensional image and the plurality of second front images obtained from the second imaging means. A two-dimensional image is cut out, a plurality of second feature amounts obtained from the plurality of cut-out second two-dimensional images, and first size information associated with each of the first feature amounts; A preparation step of preparing second size information associated with each second feature amount;
From the third distance information obtained from the distance acquisition means and the third front image obtained from the third imaging means, the third two-dimensional of the target candidate area included in the third front image After the image is cut out, a third feature amount is extracted from the third two-dimensional image, and distance information corresponding to the third feature amount is extracted from the third distance information. A feature amount extraction step for converting into a third size information;
Of the plurality of first feature amounts, the first feature amount associated with the first size information that is the same as or close to the third size information, and the third feature amount among the plurality of second feature amounts. A target candidate included in the target candidate area is identified based on the second feature amount associated with the second size information that is the same as or close to the size information of the target feature and the third feature amount. A target detection method comprising: an identification processing step.

Images