JP2012064026A - Vehicular object detection device and vehicular object detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicular object detection device and a vehicular object detection method capable of accurately acquiring detail information of an object even when a plurality of objects exist in the same direction viewed from the subject vehicle.SOLUTION: A vehicular object detection device includes: radar information acquisition means for acquiring radar information of an object; image pickup means for picking up an image; first image processing zone setup means for setting a first image processing zone that is an object acquiring image information from the pickup image picked up by the image pickup means; second image processing zone setup means for setting a second image processing zone different from the first image processing zone when radar information of a plurality of objects is acquired in the same direction viewed from the subject vehicle; image information acquisition means for acquiring image information within the second image processing zone; and detail information acquisition means for acquiring detail information of the object within the second image processing zone based on the radar information and image information.

Description

  The present invention relates to a technical field of an object detection device for detecting an object such as a vehicle mounted on a vehicle and present in the vicinity of the host vehicle, and a method thereof.

  In recent years, driving support devices such as collision mitigation devices and inter-vehicle distance control devices have been developed in order to improve safety during vehicle travel. In such a driving support device, it is important to accurately acquire detailed information on a target object such as a vehicle existing around the host vehicle. Therefore, for example, in the object detection device described in Patent Document 1, information acquired from a radar sensor or the like (hereinafter referred to as radar information) and information acquired from an image captured by the image sensor or the like (hereinafter referred to as image information). (Hereinafter referred to as fusion processing) to acquire detailed information on the object.

  However, when there is a passenger car in front of the host vehicle and a two-wheeled vehicle exists between the host vehicle and the passenger vehicle, and the passenger car and the two-wheeled vehicle overlap on the image indicated by the image information, An image sensor or the like may recognize the passenger car and the motorcycle, which are actually two objects, as one object. In reality, when two objects are recognized as one object by an image sensor or the like, if radar information and image information are fusion-processed, there is a problem that detailed information on the object cannot be obtained accurately.

  In order to solve this problem, in the vehicle obstacle detection device described in Patent Document 2, when a passenger car and a two-wheeled vehicle overlap on the image indicated by the image information, the image information is not effective. Assuming that there is no obstacle, the obstacle is determined only by the radar information.

JP 2007-132748 A JP 2008-276689 A

  However, in the vehicle obstacle detection device described in Patent Document 2, when the passenger vehicle and the two-wheeled vehicle overlap on the image indicated by the image information, that is, a plurality of objects in the same direction as viewed from the own vehicle. If there is an object, the image information is not valid and the image information is not used for the determination of the obstacle, so that detailed information on the object cannot be obtained accurately.

  In view of the above-described facts, the present invention provides a vehicle object that enables accurate acquisition of detailed information on an object by fusion processing even when a plurality of objects exist in the same direction as viewed from the host vehicle. An object is to provide an object detection device.

In order to achieve the above object, the present invention has the following features.
A first aspect of the present invention is a vehicle object detection device for detecting an object around a host vehicle, the radar information acquisition means for acquiring radar information of the object around the host vehicle, and an image around the host vehicle. Image capturing means, first image processing area setting means for setting a first image processing area for acquiring image information from an image captured by the image capturing means, and the radar information acquiring means A second image processing area setting means for setting a second image processing area different from the first image processing area when radar information of a plurality of objects is acquired in the same direction as viewed from the host vehicle; Image information acquisition means for acquiring image information in the second image processing area set by the second image processing area setting means, radar information acquired by the radar information acquisition means, and image information On the basis of the acquired image information by the acquisition unit, and a detailed information obtaining unit configured to obtain detailed information about the object of the second image processing region.

  A second invention is an invention subordinate to the first invention, wherein the second image processing area setting means sets the object having the shortest distance from the host vehicle among the plurality of objects. On the other hand, the second image processing area is set.

  A third invention is an invention dependent on the second invention, wherein the object having the shortest distance from the host vehicle is a stationary object.

  A fourth invention is an invention subordinate to the third invention, wherein the second image processing area setting means sets a third image processing area for the stationary object for the stationary object, The second image processing area is set.

  A fifth invention is an invention subordinate to any one of the first to fourth inventions, wherein the second image processing region setting means sets the second image processing region to the first image. The setting is narrower than the image processing area.

  A sixth invention is an invention dependent on any one of the first to fifth inventions, wherein distances between a plurality of objects detected in the same direction as viewed from the host vehicle are predetermined. The second image processing area is set when the value is less than the first threshold value.

  A seventh invention is an invention subordinate to any one of the first to sixth inventions, wherein the second image processing region setting means includes the host vehicle and the plurality of objects. A second image processing area is set when the distance from the vehicle closest to the subject vehicle is less than a predetermined second threshold value.

  An eighth invention is an invention subordinate to any one of the first to seventh inventions, wherein the image information acquisition means is a first set by the first image processing area setting means. Image information in the image processing area is further acquired, and the detailed information acquisition means performs first information based on the radar information acquired by the radar information acquisition means and the image information in the first image processing area. Detailed information about the object in the image processing area is acquired.

  A ninth invention is an invention subordinate to any one of the first to eighth inventions, wherein the second image processing region setting means uses the radar information acquired by the radar information acquisition means. When it is determined that there are a plurality of objects in the vicinity of the host vehicle and the plurality of objects are present in the first image processing area, the inside of the first image processing area is determined. It is determined that the plurality of objects existing in the vehicle exist in the same direction as viewed from the host vehicle.

  A tenth invention is an invention according to any one of the first to ninth inventions, wherein the detailed information acquisition means is an object indicated by radar information acquired by the radar information acquisition means. Is acquired within the region of the object indicated by the image information acquired by the image information acquisition means, and detailed information about the object is acquired based on the radar information and the image information. It is characterized by doing.

  A vehicle object detection method for detecting an object around the host vehicle, the radar information acquiring step for acquiring radar information of the object around the host vehicle, and an image capturing step for capturing an image around the host vehicle; Same as the first image processing region setting step for setting the first image processing region for obtaining image information from the image captured in the image capturing step, as viewed from the own vehicle by the radar information acquisition step. A second image processing region setting step for setting a second image processing region different from the first image processing region when radar information of a plurality of objects is acquired in the direction; and the second image processing An image information acquisition step for acquiring image information in the second image processing region set by the region setting step, and a record acquired by the radar information acquisition step. Comprising a header information, based on the image information acquired by the image information acquisition step, and a detailed information obtaining step of obtaining detailed information about the object of the second image processing region.

  According to the present invention, a vehicle object detection device that can accurately acquire detailed information of an object by fusion processing even when a plurality of objects exist in the same direction as viewed from the host vehicle, And a method thereof.

1 is a block diagram showing a schematic configuration of a vehicle object detection device according to a first embodiment. The figure which showed an example of the image imaged by the image acquisition part The figure which showed an example of the image which set the 1st image processing area The figure which showed an example of the image which set the 2nd image processing area | region The figure which showed an example of the image when the distance between each target object is apart from a fixed distance The flowchart which shows the flow of a process of the process part which concerns on 1st Embodiment. The flowchart which shows the flow of a process of the process part which concerns on 2nd Embodiment. The figure which showed an example of the scene where a radar detection part detects the preceding vehicle of a preceding vehicle The figure which showed an example of the image which imaged the situation of FIG. 5A by the image acquisition part The figure which showed an example of the image which set the 1st image process area | region to the image of FIG. 5B The flowchart which shows the flow of a process of the process part which concerns on 3rd Embodiment. The figure which showed an example of the scene where a radar detection part detects an object which is not an obstacle The figure which showed an example of the image which imaged the situation of FIG. 7A by the image acquisition part The figure which showed an example of the image which set the 1st image process area | region to the image of FIG. 7B The figure which showed an example of the image which set the 2nd image processing area | region to the image of FIG. 7C The figure which showed an example of the scene where the preceding vehicle approaches the own vehicle The figure which showed an example of the optical flow of the preceding vehicle in the situation of Drawing 8A A graph showing the relationship between feature points on the screen and optical flow The flowchart which shows the flow of a process of the process part which concerns on 4th Embodiment. The figure which showed the relationship between the horizontal position of the target object shown by radar information and the horizontal width of the target object shown by image information

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of a vehicle object detection device 1 according to a first embodiment of the present invention. The vehicle object detection device 1 according to the present embodiment includes a radar detection unit 11, an image acquisition unit 12, and a processing unit 13. The processing unit 13 includes an image processing unit 131 and a determination unit 132.

  The radar detection unit 11 is connected to the determination unit 132. As the radar detection unit 11, for example, a laser radar, a microwave radar, a millimeter wave radar, an ultrasonic radar, or the like can be used. The radar detection unit 11 transmits electromagnetic waves or the like to the periphery of the host vehicle and receives a reflected wave or the like reflected by an object existing around the host vehicle. The speed and the direction in which the object exists are detected. Then, the radar detection unit 11 is a signal (hereinafter referred to as a radar signal) indicating radar information, which is information obtained as a result of detecting the distance, relative speed, the direction in which the target exists, and the like. Is output to the determination unit 132. In the present embodiment, the radar detection unit 11 is described as being installed so as to detect radar information of an object ahead of the host vehicle, but the gist of the present invention is not limited to this. The present invention can also be applied to other directions such as the rear and side of the host vehicle.

  The image acquisition unit 12 is connected to the image processing unit 131. The image acquisition unit 12 typically processes a camera that captures an image around the host vehicle using a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD), and an image captured by the camera. And the processing circuit to be configured. The image acquisition unit 12 captures an image around the host vehicle with the camera, and outputs a signal indicating the captured image (hereinafter referred to as an image signal) to the image processing unit 131. In the present embodiment, the image acquisition unit 12 is described as being installed so as to acquire an image ahead of the host vehicle. However, the gist of the present invention is not limited to this, and The present invention can also be applied to other directions such as rearward and lateral directions.

  The image processing unit 131 is connected to the image acquisition unit 12 and the determination unit 132. The image processing unit 131 receives the image signal output from the image acquisition unit 12. The image processing unit 131 sets a region (hereinafter referred to as a first image processing region) for performing image processing on the image indicated by the image signal received from the image acquisition unit 12 using a known method. An image in which the first image processing area is set is hereinafter referred to as an image processing area setting image. As an example of a processing technique for setting the first image processing area, there is a technique for setting the first image processing area by performing image processing such as edge processing on the image.

  The image processing unit 131 acquires image information indicating the horizontal width and vertical width of the object in the first image processing area in the image processing area setting image, the horizontal position of the object viewed from the host vehicle, and the like. Further, for example, the type of the object may be specified from the image by pattern matching or the like, and information on the specified type of the object may be included in the image information. The image processing unit 131 outputs a signal indicating image information (hereinafter referred to as an image information signal) to the determination unit 132.

  The determination unit 132 is typically an ECU (Electronic Control Unit) mainly composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The determination unit 132 is connected to the radar detection unit 11 and the image processing unit 131. The determination unit 132 receives the radar signal output from the radar detection unit 11 and the image information signal output from the image processing unit 131.

  The determination unit 132 acquires detailed information of the target object by performing fusion processing on the radar information indicated by the received radar signal and the image information indicated by the received image information signal.

  However, in some cases, it is not possible to accurately acquire detailed information on the target object simply by performing fusion processing on radar information and image information in the first image processing area. A case where the detailed information of the object cannot be accurately acquired simply by performing the fusion processing of the radar information and the image information of the first image processing area will be described with reference to FIGS. 2A and 2B.

  FIG. 2A is a diagram schematically showing an image captured by the image acquisition unit 12 in a situation where a plurality of objects exist in the same direction as viewed from the host vehicle. The situation according to the image in FIG. 2A is a situation in which the object 22 is present in front of the host vehicle and the object 21 having a width or the like larger than that of the object 22 is present in front. In the situation relating to the image in FIG. 2A, the object 22 having a smaller width than the object 21 is typically a two-wheeled vehicle, and the object 21 having a larger width or the like than the object 22 is typically a passenger car. is there.

  In the situation related to the image of FIG. 2A, the radar detection unit 11 transmits electromagnetic waves or the like to the periphery of the own vehicle and receives reflected waves or the like reflected by an object existing around the own vehicle. The radar information of the existing object 22 and the radar information of the object 21 existing around the host vehicle are acquired. Then, the radar detection unit 11 outputs a radar signal related to the radar information for each of the object 22 and the object 21 to the determination unit 132.

  Similarly, in the situation relating to the image in FIG. 2A, the image acquisition unit 12 captures an image around the host vehicle with the camera, and outputs an image signal indicating the captured image (that is, FIG. 2A) to the image processing unit 131. The image processing unit 131 sets a first image processing area in the image shown in FIG. 2A indicated by the image signal received from the image acquisition unit 12. At this time, since the object 22 and the object 21 exist as the objects on the image, it is necessary to set a first image processing region for each of the objects 22 and 21 if originally intended. . However, since the object 22 and the object 21 are integrated on the image shown in FIG. 2A, the image processing unit 131 applies the object 22 and the object 21 integrated on the image to one object. It may be recognized as a thing. When the image processing unit 131 recognizes the target object 22 and the target object 21 integrated on the image as one target object, the image processing unit 131 performs the processing on the target object 22 and the target object 21 integrated on the image. One first image processing area is set.

  FIG. 2B is a diagram schematically showing an image processing area setting image when one first image processing area 23 is set for the object 22 and the object 21 that are integrated on the image. . As described above, the image processing unit 131 acquires image information of the object in the first image processing area 23 in the image processing area setting image. However, the acquired image information is image information acquired in a state where the object 22 and the object 21 are recognized as one object. The image processing unit 131 outputs an image information signal related to the image information acquired after recognizing the object 22 and the object 21 as one object to the determination unit 132. On the other hand, even when the image processing unit 131 sets one first image processing region 23 for the target object 22 and the target object 21 integrated on the image, the radar detection unit 11 does not operate as described above. Radar information is acquired for each of the object 22 and the object 21, and a radar signal related to the acquired radar information is output to the determination unit 132.

  The determination unit 132 recognizes the radar signal for each of the target object 22 and the target object 21 output from the radar detection unit 11 and the target object 22 and the target object 21 output from the image processing unit 131 as one target object. Each of the state image information signals is received.

  However, since the image information indicated by the received image information signal is image information in a state where the object 22 and the object 21 are recognized as one object, the determination unit 132 receives the image information and the received radar signal. Even if the radar information about each of the object 22 and the object 21 shown in FIG. 6 is simply fusion-processed, detailed information about each of the object 22 and the object 21 cannot be obtained accurately.

  Therefore, the determination unit 132 according to the present embodiment determines whether or not there are a plurality of objects in the same direction as viewed from the host vehicle based on the radar information, and a plurality of objects in the same direction as viewed from the host vehicle. If it is determined that there is an object, a new image processing area (hereinafter referred to as a second image processing area) is set for the object having the shortest distance from the host vehicle calculated from the radar information. When the determination unit 132 according to the present embodiment determines that there are a plurality of objects in the same direction as viewed from the host vehicle, the determination unit 132 determines the first object with the shortest distance from the host vehicle calculated from the radar information. By setting two image processing areas, image information for each object can be acquired. And the determination part 132 can acquire the detailed information of a target object by carrying out the fusion process of the acquired image information for every target object, and the radar information for every target object. A specific flow of processing for setting the second image processing area will be described with reference to FIGS. 2B and 2C.

  The determination unit 132 determines whether there are a plurality of objects in the same direction as viewed from the host vehicle, based on the radar information about each of the objects 22 and 21 indicated by the received radar signal. As an example of a method for determining whether or not there are a plurality of objects in the same direction as viewed from the host vehicle, a plurality of objects are referred to in the first image processing region with reference to image information and radar information. There is a method of determining by determining whether or not there is. That is, when the determination unit 132 refers to the radar information and determines that there are a plurality of objects in the first image processing region, the determination unit 132 determines that “a plurality of objects in the same direction as viewed from the host vehicle”. Is determined to exist. On the other hand, when the determination unit 132 refers to the radar information and determines that there are not a plurality of objects in the first image processing region, the determination unit 132 determines that “a plurality of objects in the same direction as viewed from the host vehicle”. It is determined that there is no object. An example of a method for determining whether or not there are a plurality of objects is a method of determining based on the relative speeds of the subject vehicle and each object indicated by the radar information being different.

  When the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the own vehicle, the second image processing region is calculated for the object having the shortest distance from the own vehicle calculated from the radar information. Set. As an example of a technique for determining the size of the second image processing area when setting the second image processing area, the following technique can be considered. The size of the object on the image is determined by the size of the object itself, the distance between the vehicle and the object, the distance between the objects, and the like. Thus, first, a situation is assumed in which the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the host vehicle. That is, the situation in which the processing unit 13 determines that there are a plurality of objects in the same direction when viewed from the host vehicle is that the object 22 exists in front of the host vehicle, such as the situation according to the image in FIG. Further, it is assumed in advance that a vehicle corresponding to the target object 22 having the shortest distance from the host vehicle is a two-wheeled vehicle in a situation where the target object 21 having a width or the like larger than the target object 22 exists in front. Next, information on the relationship between the size of the object itself, the distance between the host vehicle and the object, and the size of the object on the image is stored in advance in a storage device (not shown). deep. When the determination unit 132 determines that “a plurality of objects exist in the same direction as viewed from the host vehicle”, the distance between the host vehicle and the object indicated by the radar information, the direction viewed from the host vehicle, and the like A method of setting the size that the two-wheeled vehicle occupies on the image as the second image processing area, which is stored in advance, can be considered.

  In FIG. 2C, when the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the host vehicle, the second image processing region 24 with respect to the object 22 having the shortest distance from the host vehicle. FIG. 6 is a diagram schematically illustrating an image processing area setting image in which is set. The determination unit 132 acquires image information of the target object 22 in the second image processing region, and performs fusion processing on the acquired image information of the target object 22 and radar information of the target object 22 to thereby detect the target object 22. Get information accurately. In addition, the determination unit 132 acquires image information of the target object 21 in the first image processing region, and performs fusion processing on the acquired image information of the target object 21 and radar information of the target object 21. 21 information may be obtained accurately.

  On the other hand, if the determination unit 132 determines that a plurality of objects do not exist in the same direction as viewed from the host vehicle, the object in the first image processing area is set without setting the second image processing area. The detailed information of the target object is acquired by performing fusion processing on the image information and the radar information of the target object.

  The above is the description of the schematic configuration of the vehicle object detection device 1 according to the first embodiment.

  FIG. 3 is a flowchart showing a processing flow of the processing unit 13 according to the present embodiment. The processing flow of the processing unit 13 will be described below with reference to the flowchart shown in FIG.

  The image processing unit 131 sets a first image processing area in the image indicated by the image signal output from the image acquisition unit 12 (step S301). The determination unit 132 refers to the radar information indicated by the radar signal output from the radar detection unit 11 (step S302). The determination unit 132 determines whether there are a plurality of objects in the same direction as viewed from the host vehicle (step S303). When it is determined that there are a plurality of objects in the same direction as viewed from the host vehicle (YES in step S303), the determination unit 132 sets the second image processing area for the object having the shortest distance from the host vehicle. (Step S304), the image information of the second image processing area and the radar information of the object in the second image processing area are fusion-processed to obtain detailed information of the object in the second image processing area. Obtain (step S305) and finish the process. When it is determined that there are not a plurality of objects in the same direction as viewed from the host vehicle (NO in step S303), the determination unit 132 determines the image information of the first image processing area and the object related to the first image processing area. The radar information is fusion-processed to obtain detailed information on the object (step S305), and the process ends.

  The above is the description of the vehicle object detection device 1 according to the present embodiment. According to the vehicle object detection device 1 according to the first embodiment, by setting the second image processing region, even when there are a plurality of objects in the same direction as viewed from the host vehicle, each object is detected. Image information can be acquired. Then, according to the vehicle object detection device 1 according to the first embodiment, the details of each object are obtained by performing fusion processing on the acquired image information for each object and radar information for each object. Information can be acquired.

  In the above, as an example of a method for determining whether or not a plurality of objects exist in the same direction as viewed from the host vehicle, the image information and the radar information are referred to in the first image processing area. The method of determining by determining whether there are a plurality of objects is given. However, another method may be used as a method for determining whether or not there are a plurality of objects in the same direction as viewed from the host vehicle. Another example of a method for determining whether or not there are a plurality of objects in the same direction as viewed from the host vehicle is information included in radar information indicating the direction viewed from the host vehicle in which the target exists. A method of determining based on this is given. That is, if there are two objects (hereinafter referred to as the object T1 and the object T2 respectively) around the own vehicle, the object is viewed from the own vehicle with reference to the radar information of each object. The angle formed by the direction in which the target object T1 exists and the direction in which the target object T2 is viewed from the host vehicle is detected, and the absolute value of the detected angle exceeds a predetermined direction determination threshold value. If not, the determination unit 132 determines that “a plurality of objects exist in the same direction as viewed from the host vehicle”. On the other hand, when the absolute value of the angle that is the detection result exceeds a predetermined direction determination threshold value, the determination unit 132 states that “a plurality of objects do not exist in the same direction as viewed from the host vehicle”. judge.

  In addition, processing when the object 22 and the object 21 are separated from each other by a certain distance will be described. FIG. 2D is a diagram schematically showing an image captured by the image acquisition unit 12 when the object 22 and the object 21 are apart from each other by a certain distance. The situation according to the image in FIG. 2D is a situation in which the object 22 is present in front of the host vehicle, and the object 21 having a width or the like larger than the object 22 is present ahead. Is a situation where the distance is more than a certain distance. As shown in the image of FIG. 2D, when the object 22 and the object 21 are separated from each other by a certain distance, the object 21 is hidden by the object 22 when viewed from the own vehicle. Therefore, the image processing unit 131 sets one first image processing region 23 for the object 22, and the determination unit 132 acquires image information of the object 22. On the other hand, as shown in the image of FIG. 2D, since the object 22 and the object 21 are separated by a certain distance or more, even if the object 21 is hidden by the object 22 on the image of FIG. The unit 11 can acquire radar information for each of the object 22 and the object 21. This situation is a situation in which the determination unit 132 determines that “a plurality of objects are present in the same direction as viewed from the host vehicle”, and thus the processing from step S303 to step S304 according to the first embodiment is applied as it is. For example, the process of setting the second image processing area (step S304) is performed. However, unlike the situation shown in FIG. 2A to FIG. 2C described above, the determination unit 132 acquires only the image information about the object 22 with the shortest distance from the host vehicle. By subjecting the image information of the object 22 to fusion processing, the information of the object 22 can be accurately acquired. Therefore, when the target object 22 and the target object 21 are separated from each other by a certain distance, the process of step S304 is not necessary. Therefore, when the distance between the object 22 and the object 21 is equal to or greater than the predetermined value, the process of step S304 is not performed, and when the distance between the object 22 and the object 21 is less than the predetermined value, the process of step S304 is performed. May be performed. Specifically, a threshold value Lth is provided in advance for the distance Lv between the object 22 and the object 21 indicated by the radar information, and the distance Lv between the object 22 and the object 21 indicated by the radar information is the threshold value. If the distance Lv is less than Lth, the process of step S304 is performed. If the distance Lv is equal to or greater than the threshold value Lth, the process of step S304 may not be performed. Note that the threshold value Lth may be determined in advance in consideration of not only the distance between the objects but also the distance between the host vehicle and each object. Further, the size may be determined in consideration of the size of the object.

(Second Embodiment)
Next, an embodiment in which the distance between the host vehicle and an object existing ahead of the host vehicle is considered when the determination unit 132 determines whether or not to set the second image processing region will be described. In addition, the description about this embodiment uses the code | symbol attached | subjected in description of 1st Embodiment, and demonstrates only a different part from embodiment mentioned above.

  FIG. 4 is a flowchart illustrating an example of a processing flow of the processing unit 13 according to the present embodiment. In the present embodiment, between the own vehicle and the shortest distance from the own vehicle between step S303 and step S304 in the flowchart showing the processing flow of the processing unit 13 in the first embodiment shown in FIG. 3 described above. Step S401 for determining whether or not the distance Lm to the object is less than a predetermined threshold value Lo is provided.

  If the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the own vehicle (YES in step S303), the radar unit refers to the radar information and the object having the shortest distance from the own vehicle is determined. It is determined whether the distance Lm to 22 is less than a predetermined threshold Lo (step S401).

  When the distance Lm between the host vehicle and the object having the shortest distance from the host vehicle is less than a predetermined threshold Lo (YES in step S401), the determination unit 132 has the shortest distance from the host vehicle. A second image processing area is set for the object (step S304). When the distance Lm between the host vehicle and the object having the shortest distance from the host vehicle is equal to or greater than a predetermined threshold Lo (NO in step S401), the determination unit 132 sets the second image processing region. do not do.

  The reason why the above processing is performed will be described with reference to FIGS. 5A to 5C. FIG. 5A simply shows a situation in which an object 53 (for example, a passenger car) exists in front of the host vehicle 51 and an object 52 (for example, a passenger car) exists between the host vehicle 51 and the object 53. FIG. FIG. 5B is a diagram simply showing an image obtained by capturing the situation shown in FIG. 5A by the image acquisition unit 12. FIG. 5B shows that the object 53 is hidden behind the object 52, and only the object 52 can be recognized from the image captured by the image acquisition unit 12. The image processing unit 131 sets a first image processing area for the object 52 with respect to the image illustrated in FIG. 5B and acquires image information of the object 52. FIG. 5C is a diagram schematically showing an image processing area setting image in which the image processing unit 131 sets the first image processing area 56 for the object 52 with respect to the image shown in FIG. 5B. On the other hand, as shown in FIG. 5A, the determination unit 132 receives the reflected wave 54 such as an electromagnetic wave transmitted by the radar detection unit 11 to acquire the radar information of the object 52. Further, when the distance Lm between the host vehicle 51 and the target object 52 is longer than a certain distance, the radar detection unit 11 receives the reflected wave 55 passing through the gap between the vehicle body of the target object 52 and the road surface, etc. 53 radar information can be acquired. That is, the determination unit 132 acquires image information about the target object 52 and also acquires radar information of each of the target object 52 and the target object 53. It is determined that there are a plurality of objects. In this case, if the processing from step S303 to step S304 according to the first embodiment is applied as it is, there is no target for which the second image processing region should be set before the target 52 as viewed from the host vehicle. However, the determination unit 132 sets the second image processing area. However, according to the determination unit 132 according to the present embodiment, when the distance Lm between the host vehicle 51 and the object 52 is longer than a certain value, the process for setting the second image processing region is not performed, and thus the radar detection unit Even if 11 detects the reflected wave 55 or the like and acquires radar information of the object 53, it is possible to prevent the unnecessary second image processing area from being set.

  The above is the description of the vehicle object detection device 1 according to the present embodiment. According to the vehicle object detection device 1 according to the present embodiment, even if the radar detection unit 11 detects the reflected wave 55 or the like and acquires the radar information of the object 53, the vehicle 51 and the object 52 are not detected. When the distance Lm is longer than a certain length, detailed information on the object having the shortest distance from the host vehicle can be accurately acquired without performing the process of setting the second image processing area.

(Third embodiment)
Next, an embodiment in which the determination unit 132 considers whether or not the object is a stationary object when determining whether or not to set the second image processing area will be described. In addition, the description about this embodiment uses the code | symbol attached | subjected in description of embodiment mentioned above, and demonstrates only a different part from embodiment mentioned above.

  FIG. 6 is a flowchart illustrating an example of a processing flow of the processing unit 13 according to the present embodiment. In the flowchart according to the present embodiment, whether or not the object is stationary with respect to the road surface between step S303 and step S305 in the flowchart of FIG. 3 showing the processing flow of the processing unit 13 in the first embodiment. And a step S602 for setting a new image processing area (hereinafter referred to as a third image processing area) for an object that is a stationary object (hereinafter referred to as a stationary object) on the road surface. ing.

  If the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the host vehicle (YES in step S303), the determination unit 132 refers to the speed information of the radar information of the object to determine whether the object is a stationary object. Is determined (step S601). If the determination unit 132 determines that the object is a stationary object (YES in step S601), the determination unit 132 sets a third image processing region for the object (step S602). When the determination unit 132 determines that the object is not a stationary object (NO in step S601), the third image processing area is not set for the object. The determination unit 132 acquires detailed information of the stationary object by performing fusion processing on the radar information of the stationary object and the image information of the third image processing area set in step S602. Further, the determination unit 132 acquires detailed information of the target object by subjecting the target object of the first image processing area to fusion processing of the image information and the like of the first image processing area.

  The reason why the above processing is performed will be described with reference to FIGS. 7A to 7D. 7A shows an object 73 (for example, a passenger car) in front of the host vehicle 71 and an object 73 (not an obstacle for the host vehicle 71) on the road surface between the host vehicle 71 and the target object 72 or the like. For example, it is a diagram schematically showing a situation in which there is an iron plate or a manhole on a road. FIG. 7B is a diagram simply showing an image obtained by capturing the situation according to FIG. 7A by the image acquisition unit 12. FIG. 7B shows that the object 72 and the object 73 are integrated on the image depending on the positional relationship between the host vehicle 71 and the object 72. When the object 72 and the object 73 are integrated on the image, the image processing unit 131 performs first image processing using the object 72 and the object 73 integrated on the image as one object. There is a possibility to set the area. FIG. 7C is a diagram schematically showing an image processing area setting image in which the first image processing area 74 is set in the image according to FIG. 7B. FIG. 7C shows that the image processing unit 131 sets one first image processing area 74 for the object 72 and the object 73 that are integrated on the image. However, even if the determination unit 132 simply performs fusion processing on the image information in a state where the object 72 and the object 73 are recognized as one object and the radar information about each of the object 72 and the object 73, the object Detailed information about each of the object 72 and the object 73 cannot be obtained accurately. Therefore, in the determination unit 132 according to the present embodiment, when the object 73 with the shortest distance from the host vehicle is a stationary object, the third image processing area is set for the object 73. . FIG. 7D is a diagram schematically showing an image processing area setting image in which the third image processing area 75 is set in the image according to FIG. 7C. When the object 73 with the shortest distance from the host vehicle is a stationary object, the third image processing area 75 is set for the object 73 if the object 73 is a stationary object. This is because 73 is considered not to be an obstacle for the host vehicle 71. According to the determination unit 132 according to the present embodiment, when the object 73 with the shortest distance from the host vehicle is a stationary object, the third image processing area 75 is set for the object 73. By subjecting the object 73 to fusion processing of image information and the like of the object 73 in the third image processing region 75, detailed information about the object 73 can be accurately acquired. In addition, by subjecting the object 72 to fusion processing of the image information and the like of the object 72 in the first image processing area 74, detailed information about the object 72 can be accurately acquired.

  In the present embodiment, when determining whether or not the object is an obstacle for the host vehicle, it is determined whether or not the object is a stationary object. However, in consideration of a motorcycle or the like stopped on the road, whether or not the object is an obstacle is determined by determining whether or not the object is a three-dimensional object in addition to determining whether or not the object is a stationary object. You may make it determine. That is, when the target object is a stationary object and a non-three-dimensional object, it is determined that the target object is not an obstacle for the host vehicle, and when the target object is not a stationary object or when the target object is a three-dimensional object, the host vehicle has an obstacle. You may make it determine with it being a thing.

  As a method for determining whether or not the object is a stationary object, the relative speed Vr between the host vehicle and the object is subtracted from the vehicle speed Vs of the host vehicle acquired by a vehicle speed measuring unit (not shown) of the host vehicle. When the absolute value of the value is smaller than a predetermined constant ΔV, a method for determining that the object is a stationary object can be used.

  As an example of a method for determining whether or not the object is a non-three-dimensional object, the optical flow of the object in the third image processing area is calculated by processing the image in the third image processing area, and the calculation is performed. There is a method for determining whether an object is a non-three-dimensional object by determining whether the optical flow is a three-dimensional optical flow or a non-three-dimensional optical flow. The following method is mentioned as an example of the optical flow calculation method.

  FIG. 8A is a diagram illustrating a scene in which the preceding vehicle 82 approaches the host vehicle 81. A preceding vehicle 82 ′ (represented by a dotted line) in FIG. 8A represents a state before approaching the host vehicle 81, and a preceding vehicle 82 (represented by a solid line) in FIG. 8A represents a state after approaching the host vehicle 81. (A) of FIG. 8A is a diagram showing a case where the preceding vehicle 82 approaches the host vehicle 81 as viewed from the side to the vertical direction with respect to the traveling direction of the vehicle. FIG. 8A (B) is a diagram illustrating a case in which the preceding vehicle 82 ′ approaches the host vehicle 81 as viewed from above. The distance between the host vehicle 81 and the preceding vehicle 82 'is represented by L, and the distance that the preceding vehicle 82' has approached the host vehicle 81 is represented by ΔL. The feature point 83 represents one of arbitrary feature points of the preceding vehicle 82. x represents the lateral position of the feature point 83 ′ with reference to the center line of the host vehicle 81 extended in the traveling direction of the host vehicle 81. Δx represents a movement vector on the image of the feature point 83 ′, that is, an optical flow when the preceding vehicle 82 ′ approaches the host vehicle by the distance ΔL. FIG. 8B is a diagram illustrating an optical flow when the preceding vehicle 82 ′ approaches the host vehicle 81. A preceding vehicle 82 ′ (represented by a dotted line) in FIG. 8B represents the appearance on the image when the distance between the preceding vehicle 82 ′ and the host vehicle 81 is L, and a preceding vehicle 82 (represented by a solid line) in FIG. The appearance on the image when the preceding vehicle 82 'approaches the host vehicle 81 by a distance ΔL is shown. Each arrow in FIG. 8B represents an optical flow indicating the movement vectors of the feature points of the preceding vehicle 82 ′ and the preceding vehicle 82. Note that L and ΔL are calculated from distance information of the radar information and relative speed information.

Here, the optical flow Δx is expressed by Expression (1).
Δx = (x / L) × ΔL (1)

  FIG. 8C is a graph showing a relationship between Δx and x in Equation (1). When the relationship between Δx and x in Equation (1) is represented by a graph in which Δx is the vertical axis and x is the horizontal axis, as shown in FIG.

  If the area on the image for which the optical flow is to be calculated is set to a predetermined height of the distance where the object exists, the object detected by the radar detection unit 11 is a non-three-dimensional object such as a manhole. Since the optical flow of the object that does not exist at the predetermined height is calculated, the linear relationship as shown in FIG. 8C is not obtained. Therefore, it can be determined whether the object is a three-dimensional object or a non-three-dimensional object by determining whether the optical flow has the relationship shown in FIG. 8C.

  In the foregoing, an example of a method for determining whether a target object is a three-dimensional object or a non-three-dimensional object using the principle of optical flow has been described. To apply the method more reliably, the optical flow graph of FIG. 8C is used. It is necessary to appropriately determine an image processing area for obtaining the above. When there are a plurality of objects in the same direction as viewed from the host vehicle, the image processing area is set as a second image processing area set for the object having the shortest distance from the host vehicle. The reliability of the method can be further increased. The method for determining whether or not the object is a three-dimensional object by the optical flow is not limited to the third embodiment, and may be applied to other embodiments.

  Moreover, the process of the determination part 132 which concerns on this embodiment shown in FIG. 6 is performed with respect to the said object in the flow when the determination part 132 determines with a target object not being a stationary object (it is NO at step S601). A step of setting a second image processing region assuming a two-wheeled vehicle (corresponding to step S304 in FIG. 3 and the like) is provided, and the radar information of the target object and the image information of the set second image processing region are fusion processed. And you may make it acquire the detailed information of the said target object.

  The above is the description of the vehicle object detection device 1 according to the present embodiment. According to the vehicle object detection device 1 according to the present embodiment, a plurality of objects are present in the same direction as viewed from the own vehicle, and the object having the shortest distance from the own vehicle is a stationary object. In some cases, in order to set the third image processing area for the stationary object, the image information about the third image processing area and the radar information of the object in the third image processing area are fusion-processed, Detailed information on the stationary object can be obtained accurately. In addition, by subjecting the object in the first image processing area to the fusion processing of the image information in the first image processing area, detailed information on the object can be accurately acquired.

(Fourth embodiment)
Next, an embodiment will be described in which the determination unit 132 considers the lateral position information of the object indicated by the radar information and the lateral width information of the object indicated by the image information when determining whether or not to perform the fusion process. In addition, the description about this embodiment uses the code | symbol attached | subjected in description of embodiment mentioned above, and demonstrates only a different part from embodiment mentioned above.

  FIG. 9 is a flowchart illustrating an example of a processing flow of the processing unit 13 according to the present embodiment. The processing flow of the processing unit 13 will be described below with reference to the flowchart shown in FIG.

  The determination unit 132 refers to the image information and the radar information (Step S901 and Step S302). Then, it is determined whether or not there are a plurality of objects in the same direction as viewed from the own vehicle based on the referenced radar information (step S303). If the determination unit 132 determines that there are a plurality of objects in the same direction as viewed from the own vehicle (YES in step S303), the lateral position of the object having the shortest distance from the own vehicle indicated by the radar information is It is determined whether or not the object is within the horizontal width indicated by the image information of the object (step S902). If the determination unit 132 determines that the lateral position of the target object indicated by the radar information with the shortest distance from the host vehicle does not fall within the horizontal width of the target object indicated by the image information (NO in step S902). ) No fusion treatment. If the determination unit 132 determines that the lateral position of the target object indicated by the radar information with the shortest distance from the host vehicle is within the horizontal width of the target object indicated by the image information (YES in step S902). ), Or when it is determined that there are not a plurality of objects in the same direction when viewed from the host vehicle (NO in step S303), the image information and the like are subjected to fusion processing (step S305).

  The reason why the above processing is performed will be described with reference to FIG. FIG. 10 is a diagram simply showing the relationship between the horizontal position of the object indicated by the radar information and the horizontal width of the object indicated by the image information. The situation shown in FIG. 10 is a situation in which the object 101 exists in front of the host vehicle (not shown).

  The lateral position 102 in FIGS. 10A to 10C indicates the lateral position of the object 101 viewed from the host vehicle calculated from the radar information. The widths 103A to 103C in FIGS. 10A to 10C respectively indicate the widths of the object 101 calculated from the image information in FIGS. A horizontal width 103A in FIG. 10A indicates a state in which the horizontal width of the object 101 is appropriately represented by image information. However, there are cases where the image information does not properly represent the width of the object due to the influence of the color of the vehicle, the way of sunlight, and the like. A horizontal width 103B in FIG. 10B indicates a state where one object 101 is actually divided into two objects in the image information. A horizontal width 103C in FIG. 10C indicates a state in which the horizontal width of the object 101 indicated by the image information is recognized to be narrower than the actual horizontal width (that is, the horizontal width 103A).

  When the width of the object 101 is actually the width 103A but is recognized as a width different from the actual width, such as the width 103B or the width 103C, the processing unit 13 displays the image information of the object. Since it has not been acquired correctly, even if the image information or the like is fusion-processed, detailed information on the object cannot be acquired accurately.

  Therefore, in the determination unit 132 according to the present embodiment, when there are a plurality of objects in the same direction as viewed from the host vehicle, the width information of the object indicated by the image information accurately determines the width of the object. Since the fusion process is performed only in the case where the object 101 is represented, detailed information of the object 101 can be obtained accurately.

  The above is the description of the vehicle object detection device 1 according to the present embodiment. According to the vehicle object detection device 1 according to the present embodiment, since the fusion process is performed only when the base image information is accurate information, it is possible to accurately acquire detailed information on the object. .

  In the first to fourth embodiments, when the image processing or the like performed by the processing unit 13 involves a calculation delay, the acquired image information or the like may not reflect the real-time state of the target object. If the acquired image information does not reflect the real-time status of the target object, the timing at which the image information is acquired and the timing at which the radar information is acquired, which is the basis of the fusion process, will be different. Detailed information about the object cannot be obtained accurately. Thus, when image processing or the like performed by the processing unit 13 involves calculation delay, it is conceivable to correct radar information, image information, or the like in consideration of the calculation delay. As a technique for correcting radar information, image information, and the like, for example, radar information is multiplied by a value obtained by multiplying a lateral position X after correction of a target object by a lateral speed Vx of the target object and a constant Δt indicating a calculation delay time. A method of adding the lateral position x of the object indicated by Note that the corrected lateral position X of the object may be determined based on the map data by previously setting information on the relationship between the distance between the host vehicle and the object, the relative speed, and the like as map data.

  In the description of the first to fourth embodiments, for convenience of description, a situation where two objects exist is described. However, the present invention is not applied only to the situation where there are two objects. For example, the present invention can be applied even when three or more objects exist and the three or more objects are integrated on the image. That is, the determination unit 132 can accurately acquire detailed information of each of the three or more objects by setting the second image processing region for each of the three or more objects. it can.

  As mentioned above, although embodiment of this invention was described, the above-mentioned description is only an example of this invention in all the points, and does not intend to limit the range. It goes without saying that the above embodiments can be appropriately combined and various improvements can be made without departing from the scope of the present invention. For example, the processing of the first to third embodiments may be performed after the processing of the fourth embodiment is performed in order to determine the validity of the image information.

  The present invention provides an object detection device for a vehicle, and a method thereof, which makes it possible to accurately acquire detailed information of an object even when a plurality of objects exist in the same direction as viewed from the host vehicle. For example, it can be used for a driving support device mounted on a moving body such as an automobile.

DESCRIPTION OF SYMBOLS 1 Vehicle target detection apparatus 11 Radar detection part 12 Image acquisition part 13 Processing part 131 Image processing part 132 Determination part 21, 22, 52, 53, 72, 73, 82, 82 ', 101 Target object 23, 24, 56 , 74, 75 Image processing area 51, 71, 81 Own vehicle 54, 55 Reflected wave, etc. 83, 83 ′ Feature point of target object 102 Horizontal position of target object 103A to C Width information of target object

Claims (11)

A vehicle object detection device for detecting an object around a host vehicle,
Radar information acquisition means for acquiring radar information of an object around the vehicle by radar;
Image capturing means for capturing an image of the surroundings of the host vehicle;
First image processing area setting means for setting a first image processing area as a target for acquiring image information from an image captured by the image capturing means;
When radar information of a plurality of objects is acquired in the same direction as viewed from the host vehicle by the radar information acquisition means, a second image processing area that is different from the first image processing area is set. Image processing area setting means;
Image information acquisition means for acquiring image information in the second image processing area set by the second image processing area setting means;
Detailed information for acquiring detailed information about the object in the second image processing area based on the radar information acquired by the radar information acquisition unit and the image information acquired by the image information acquisition unit An object detection device for a vehicle, comprising an acquisition unit.   The second image processing area setting means sets the second image processing area for an object having the shortest distance from the host vehicle among the plurality of objects. Item 2. The vehicle object detection device according to Item 1.   The vehicle object detection device according to claim 2, wherein the object having the shortest distance from the host vehicle is a stationary object.   4. The second image processing area setting means sets, for the stationary object, a third image processing area for the stationary object as the second image processing area. The vehicle object detection device according to claim 1.   The vehicle according to any one of claims 1 to 4, wherein the second image processing region setting means sets the second image processing region to be narrower than the first image processing region. Object detection device.   The second image processing area setting means is configured to output a second image when a distance between a plurality of objects detected in the same direction as viewed from the own vehicle is less than a predetermined first threshold value. The vehicle object detection device according to claim 1, wherein a processing region is set.   The second image processing area setting means is configured such that a distance between the own vehicle and an object closest to the own vehicle among the plurality of objects is less than a predetermined second threshold value. The vehicle object detection device according to any one of claims 1 to 6, wherein a second image processing region is set.   The image information acquisition unit further acquires image information in the first image processing region set by the first image processing region setting unit, and the detailed information acquisition unit is acquired by the radar information acquisition unit. The detailed information about the object in the first image processing area is acquired based on the radar information obtained and the image information in the first image processing area. The vehicle object detection device according to any one of the above.   The second image processing region setting means determines that there are a plurality of objects around the vehicle based on the radar information acquired by the radar information acquisition means, and the plurality of objects are the When it is determined that the object exists in the first image processing area, it is determined that the plurality of objects existing in the first image processing area exist in the same direction as viewed from the host vehicle. The vehicle object detection device according to any one of claims 1 to 8.   The detailed information acquisition unit is configured such that the position of the object indicated by the radar information acquired by the radar information acquisition unit is within the area of the object indicated by the image information acquired by the image information acquisition unit. The vehicle object detection device according to claim 1, wherein detailed information about the object is acquired based on the radar information and the image information. An object detection method for a vehicle for detecting an object around the own vehicle,
A radar information acquisition step of acquiring radar information of an object around the host vehicle;
An image capturing step for capturing an image around the host vehicle;
A first image processing region setting step for setting a first image processing region that is a target for acquiring image information from the image captured by the image capturing step;
When radar information of a plurality of objects is acquired in the same direction as viewed from the host vehicle in the radar information acquisition step, a second image processing area that is different from the first image processing area is set. An image processing area setting step;
An image information acquisition step of acquiring image information in the second image processing region set by the second image processing region setting step;
Detailed information for acquiring detailed information about the object in the second image processing area based on the radar information acquired by the radar information acquisition step and the image information acquired by the image information acquisition step. A vehicle object detection method comprising: an acquisition step.

Images