JP2009174900A - Object detecting device, object detection method, and object detecting system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect even a weakly reflected object, such as a pedestrian, by a first radar search. <P>SOLUTION: An object detection device or an object detection system is constituted of an object kind determining part determines the kind of an object, based on an image imaged by a camera loaded on a vehicle; and a threshold selection part selects a prescribed threshold correlated with the kind determined, and detects an object, on condition that intensities of the reflected waves relative to an irradiated wave irradiated in a range corresponding to the image be the selected threshold or higher. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an object detection device, an object detection method, and an object detection system for detecting an object present in the vicinity of a vehicle, and in particular, even a weakly reflective object such as a pedestrian can be detected by an initial radar search. The present invention relates to an object detection device, an object detection method, and an object detection system.

  Conventionally, vehicles that perform damage-reducing automatic braking and brake assist have been equipped with radar capable of ranging, such as laser radar and millimeter wave radar, and such radar can capture objects such as vehicles and pedestrians in front of the vehicle. Has been done. When an object is captured by such a radar, it is common that the intensity of a reflected wave from the object is not less than a predetermined threshold.

  There is also known a technique for capturing an object using an image sensor configured by a camera mounted on a vehicle and an analysis unit that analyzes an image captured by the camera. Some of such image sensors perform object recognition by pattern matching and distance measurement by a stereo camera.

  And the object detection technique which combined the above-mentioned radar and image sensor is also proposed. For example, in Patent Document 1, when a radar search is performed after setting a threshold value for a strong reflection object (vehicle) with high reflection intensity, and a weak reflection object (pedestrian) with low reflection intensity is detected, A technique is disclosed in which a radar search is re-executed after changing to a threshold value for a weakly reflecting object lower than a threshold value for a strongly reflecting object, and the detected weakly reflecting object is tracked by an image sensor.

JP 2006-284293 A

  However, in order to detect a weakly reflective object such as a pedestrian using the technique of Patent Document 1, at least two radar searches are required, which increases the time required for the radar search. In addition, when a weakly reflecting object such as a pedestrian cannot be captured by the first radar search, there is a problem that the threshold value itself cannot be lowered.

  For these reasons, how to realize an object detection device, an object detection method, or an object detection system that can detect a weakly reflecting object such as a pedestrian in the first radar search is a major issue. ing.

  The present invention has been made to solve the above-described problems caused by the prior art, and an object detection method and an object detection method that can detect even a weakly reflecting object such as a pedestrian by an initial radar search. An object is to provide an apparatus and an object detection system.

In order to solve the above-described problems and achieve the object, the present invention provides an object detection device that detects an object existing in the vicinity of a vehicle, and is based on an image captured by a camera mounted on the vehicle. Object type determination means for determining the type of object, threshold selection means for selecting a predetermined threshold value associated with the type determined by the object type determination means, and an irradiation wave irradiated on a range corresponding to the image And an object detection determination unit that detects the object on condition that the intensity of the reflected wave is greater than or equal to the threshold selected by the threshold determination unit.

  Further, the present invention is an object detection method for detecting an object present in the vicinity of a vehicle, the object type determination step for determining the type of the object based on an image captured by a camera mounted on the vehicle. , A threshold selection step for selecting a predetermined threshold associated with the type determined by the object type determination step, and an intensity of a reflected wave related to an irradiation wave irradiated on a range corresponding to the image is the threshold determination step And an object detection determination step of detecting the object on condition that it is equal to or more than the threshold value selected by (1).

  Further, the present invention is an object detection system for detecting an object existing in the vicinity of a vehicle with an image search device and a radar search device mounted on the vehicle, wherein the image search device detects an image captured by a camera. An object type determining unit configured to determine the type of the object based on the radar search device, wherein the radar search device selects a predetermined threshold associated with the type determined by the object type determining unit of the imaging device; Means, and an object detection determination means for detecting the object on condition that the intensity of the reflected wave related to the irradiation wave irradiated on the range corresponding to the image is equal to or greater than the threshold selected by the threshold determination means. It is characterized by that.

  According to the present invention, the type of an object is determined based on an image captured by a camera mounted on a vehicle, a predetermined threshold value associated with the determined type is selected, and irradiation is performed for a range corresponding to the image. Since the object is detected on the condition that the intensity of the reflected wave related to the irradiated wave is equal to or higher than the selected threshold value, the radar search is performed after changing to the threshold value corresponding to the object type obtained from the image. As a result, even a weakly reflecting object such as a pedestrian can be detected by the first radar search.

  Exemplary embodiments of an object detection device, an object detection method, and an object detection system according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, after describing the outline of the object detection technique according to the prior art and the outline of the object detection technique according to the present invention, examples of the object detection apparatus and the object detection system to which the object detection technique according to the present invention is applied. Will be explained.

  First, the outline of the object detection method according to the prior art will be described with reference to FIG. FIG. 6 is a flowchart showing the processing procedure of the object detection method according to the prior art. In the figure, a general method (prior art) for performing a radar search using a millimeter wave radar or the like is shown.

  As shown in the figure, a radar search threshold value α is set (step S1001), the first radar search is executed (step S1002), and it is determined whether or not the object is a low reflection object such as a pedestrian. (Step S1003). Here, the determination as to whether or not the object is a low reflection object is made based on whether or not the threshold value α has been exceeded even once, for example.

  If the object is a low reflection object such as a pedestrian (step S1003, Yes), it is lowered to a threshold value β smaller than the threshold value α set in step S1001 (step S1004), and a radar search is performed using this threshold value β. It executes (step S1005). If the determination condition of step S1003 is not satisfied (step S1003, No), the processing after step S1002 is repeated.

  Subsequently, it is determined whether or not a low-reflection object such as a pedestrian is being tracked (step S1006). If a low-reflection object such as a pedestrian is being tracked (step S1006, Yes), step S1005 is performed. ) Repeat the subsequent processing. On the other hand, when the determination condition of step S1006 is not satisfied (step S1006, No), the processing after step S1001 is repeated.

  As described above, in the object detection method according to the related art, in the initial radar search using the large threshold β (see step S1002), it is determined whether or not the object is a low reflection object, and it is determined that the object is a low reflection object. In this case, the radar search using the small threshold value α has to be executed again (step S1005). Therefore, there is a problem that the time required for radar search is increased.

  Further, in the object detection method according to the related art, if the threshold value α is never exceeded in the first radar search (see step S1002), the low reflection object cannot be detected and the threshold value is lowered ( There is also a problem that the process itself cannot be performed (see step S1004).

  Next, an outline of the object detection method according to the present invention will be described with reference to FIG. FIG. 1 is a diagram showing an outline of an object detection method according to the present invention. As shown in the figure, the object detection method according to the present invention uses both radar search and image search using a camera in order to solve the problems in the object detection method described with reference to FIG. did.

  Specifically, in the object detection method according to the present invention, first, the object type is determined based on the camera image (see (1) in FIG. 1). Here, the object type refers to a pedestrian and a non-pedestrian (for example, a vehicle). The determination of the object type is performed using, for example, a pattern matching method.

  Subsequently, a radar search threshold associated with the object type is selected (see (2) in the figure). For example, when the object type is a pedestrian, the threshold A is selected, and when the object type is a vehicle (non-pedestrian), a threshold B that is a threshold larger than the threshold A is selected. Subsequently, a radar search is executed using the selected threshold (see (3) in the figure).

  As described above, in the object detection method according to the present invention, the object type is determined based on the camera image, and the radar search is executed using the threshold value determined for each determined object type. Therefore, even a low reflection object such as a pedestrian can be reliably detected by the first radar search. In addition, in the case of a highly reflective object such as a vehicle, since a threshold value for a highly reflective object (threshold value larger than a threshold value for a low reflective object) is used, reliable detection is possible by eliminating the influence of noise and the like. It becomes.

  Below, the Example which concerns on the object detection apparatus and object detection system to which the object detection method based on this invention shown in FIG. 1 is applied is described. The object detection device will be described with reference to FIGS. 2 to 4, and the object detection system will be described with reference to FIG.

  FIG. 2 is a block diagram illustrating a configuration of the object detection apparatus 10 according to the present embodiment. As shown in the figure, the object detection device 10 is connected to a radar 21, a camera 22, a vehicle sensor 31, and a vehicle control device 32, and includes an object recognition processing unit 11 and a collision determination processing unit 14. ing. In this embodiment, the object detection apparatus 10 including the collision determination processing unit 14 will be described. However, an object detection apparatus that does not include the collision determination processing unit 14 may be configured.

  The object recognition processing unit 11 further includes a radar search unit 12 connected to the radar 21 and an image search unit 13 connected to the camera 22. The object recognition processing unit 12 further includes a threshold selection unit 12a, an object detection determination unit 12b, and an object motion calculation unit 12c. The image search unit 13 includes an object detection unit 13a and an object type determination unit. 13b. The collision determination processing unit 14 further includes an object motion acquisition unit 14a, a vehicle motion acquisition unit 14b, a collision determination unit 14c, and a collision avoidance control unit 14d.

  The radar 21 is a device that performs processing for irradiating a target with a millimeter wave (laser) and acquiring a reflected wave from the target. The radar 21 also performs a process of outputting radar data regarding the irradiation wave and the reflected wave to the object detection determination unit 12b of the radar search unit 12.

  The camera 22 is a camera such as a CCD (Charge Coupled Devices) camera, for example. The camera 22 also performs a process of outputting the captured image data to the object detection unit 13a of the image search unit 13. Note that a stereo camera in which a plurality of cameras are combined may be used as the camera 22.

  The vehicle sensor 31 is a sensor that acquires the motion state of the vehicle, such as a vehicle speed sensor or a steering angle sensor, and is mounted on the vehicle together with the object detection device 10. The vehicle sensor 31 also performs a process of outputting the acquired motion state to the vehicle motion acquisition unit 14b of the collision determination processing unit 14.

  The vehicle control device 32 is a device that performs vehicle control such as brake control, steering control, and warning control in accordance with an instruction from the collision avoidance control unit 14d of the collision determination processing unit 14. Here, the brake control refers to control that performs automatic braking or brake assist for avoiding a collision. Steering control refers to control that performs automatic steering and steering assist to avoid collisions, and warning control refers to control that provides warnings to the driver by voice, vibration, and seat belt operation. Point to.

  The object recognition processing unit 11 determines the object type based on the image data from the camera 22 and performs object detection on the radar data from the radar 21 using a threshold corresponding to the previously determined object type, It is a processing unit that performs a process of calculating the motion of the detected object and outputting it to the collision determination processing unit 14.

  The radar search unit 12 performs object detection based on the radar data from the radar 21 and the object type from the image search unit 13, calculates the motion of the detected object, and performs processing to output to the collision determination processing unit 14. Part.

  The threshold selection unit 12a is a processing unit that performs processing to select a threshold corresponding to the object type received from the object type determination unit 13b of the image search unit 13 and pass the selected threshold to the object detection determination unit 12b. Specifically, the threshold selection unit 12a sets a threshold A when the object type is a pedestrian, and a threshold B that is a threshold larger than the threshold A when the object type is a vehicle (non-pedestrian). Select each.

  In the present embodiment, a case where a corresponding threshold value is selected from two threshold values according to whether the object type is a pedestrian or a non-pedestrian will be described. The corresponding threshold value may be selected from the same number of threshold values associated with each object type.

  The object detection determination unit 12b is a processing unit that performs detection determination of an object included in the radar data received from the radar 21 using the threshold selected by the threshold selection unit 12a. The object detection determination unit 12b also performs a process of passing radar data on the detected object to the object motion calculation unit 12c.

  Here, an outline of the object detection determination performed by the object detection determination unit 12b will be described with reference to FIG. FIG. 3 is a diagram showing an outline of object detection determination. As shown in “(1) When the object type is a pedestrian” in the figure, there is a pedestrian (see 103b in the figure) in the image (103a in the figure), and the object type is selected from the threshold selection unit 12a. When receiving a notification that is a pedestrian, the object detection determination unit 12b selects the threshold value A as indicated by 103c in FIG. In this case, it is possible to detect a low-reflection object such as a pedestrian if 103d shown in FIG.

  Further, as shown in “(2) When the object type is a vehicle” in the figure, there is a vehicle (see 103f in the figure) in the image (see 103e in the figure), and the object is selected from the threshold selection unit 12a. When the notification that the type is a vehicle (non-pedestrian) is received, the object detection determination unit 12b selects the threshold value B as indicated by 103g in FIG. In this case, if 103h shown in the figure is the intensity of the reflected wave from the object, a highly reflective object such as a vehicle can be reliably detected without being affected by noise.

  Moreover, in the same figure, although it has shown about the case where only a pedestrian and only a vehicle exist in the image, the threshold value selection part 12a selects the threshold value A when a pedestrian and a vehicle are mixed. It will be. In addition, while distinguishing the area | region where a pedestrian exists, and the area | region where a vehicle (non-pedestrian) exists, while performing the object detection determination using the threshold value A about the area | region where a pedestrian exists, a vehicle (non-pedestrian) The object detection determination may be performed using the threshold value B for the region where the is present.

  Returning to the description of FIG. 2, the object motion calculation unit 12c will be described. The object motion calculation unit 12c is a processing unit that performs processing for calculating object motion data such as the position, orientation, and relative velocity of an object from radar data related to the object detected by the object detection determination unit 12b. The object motion calculation unit 12c also performs a process of outputting the calculated object motion data to the object motion acquisition unit 14a of the collision determination processing unit 14.

  The image search unit 13 is a processing unit that performs processing for determining the object type by performing image analysis on the image data received from the camera 22 and outputting the determined object type to the threshold selection unit 12 a of the radar search unit 12. .

  The object detection unit 13a is a processing unit that detects an object by performing image analysis on the image data received from the camera 22. Note that a technique such as a saliency algorithm is used to detect an object. The object type determination unit 13b is a processing unit that performs a process of determining the type of the detected object by using a method using pattern matching or a neural network when the object detection unit 13a detects an object. For example, the object type determination unit 13b determines two object types, a pedestrian and a non-pedestrian (vehicle or the like), and outputs the determined object type to the threshold selection unit 12a of the radar search unit 12.

  The collision determination processing unit 14 may collide between the host vehicle and the object based on the object motion data received from the object motion calculation unit 12 c of the radar search unit 12 and the vehicle motion data received from the vehicle sensor 31. And a processing unit that performs a process of outputting a collision avoidance instruction to the vehicle control device 32 when a collision is predicted.

  The object motion acquisition unit 14a receives the object motion data such as the position, orientation, and relative velocity of the object from the object motion calculation unit 12c of the radar search unit 12, and performs processing to output the received object motion data to the collision determination unit 14c. Part. The vehicle motion acquisition unit 14b is a processing unit that performs processing for receiving vehicle motion data such as a vehicle speed and a steering angle relating to the host vehicle from the vehicle sensor 31 and outputting the received vehicle motion data to the collision determination unit 14c.

  The collision determination unit 14c determines whether the detected object collides with the host vehicle based on the object motion data received from the object motion calculation unit 12c of the radar search unit 12 and the vehicle motion data received from the vehicle motion acquisition unit 14b. It is a processing unit that performs a process of determining whether or not. The collision determination unit 14c also performs a process of outputting the determination result to the collision avoidance control unit 14d.

  The collision avoidance control unit 14d performs a process of outputting an instruction for avoiding a collision to the vehicle control device 32 when receiving a notification that the detected object collides with the own vehicle from the collision determination unit 14c. Part. For example, the collision determination unit 14c instructs the vehicle control device 32 to operate the assist brake in order to avoid a collision.

  Next, a processing procedure executed by the object detection apparatus 10 will be described with reference to FIG. FIG. 4 is a flowchart showing a processing procedure executed by the object detection apparatus 10. As shown in the figure, the image search unit 13 executes an image search process (step S101). Then, the object type determination unit 13b of the image search unit 13 outputs whether the object type is a pedestrian or a non-pedestrian (such as a vehicle) to the threshold selection unit 12a of the radar search unit 12b.

  Subsequently, the threshold selection unit 12a determines whether or not the object type is a pedestrian (step S102), and when the object type is a pedestrian (Yes in step S102), a low reflection such as a pedestrian. A threshold value A (see FIG. 3) that is a threshold value for an object is selected (step S103). On the other hand, when the object type is not a pedestrian (No in step S102), a threshold B (see FIG. 3) that is a threshold for a highly reflective object such as a vehicle is selected (step S104). The threshold value B is larger than the threshold value A.

  Then, the radar search unit 12 executes a radar search process for the image range using the threshold selected in step S103 or step S104 (step S105), and the object detection determination unit 12b has a reflected wave intensity equal to or greater than the threshold. It is determined whether or not there is (step S106). If the reflected wave intensity is greater than or equal to the threshold (step S106), the object motion calculation unit 12c calculates the distance, angle (azimuth), and relative speed of the object (step S107).

  Subsequently, the collision determination processing unit 14 acquires the object motion and the vehicle motion (step S108), and determines whether or not the object collides with the host vehicle (step S109). And when it determines with an object colliding with the own vehicle (step S109, Yes), a collision avoidance instruction | indication is performed with respect to the vehicle control apparatus 32 (step S110), and a process is complete | finished.

  If the determination condition in step S106 is not satisfied (No in step S106), the process ends without performing the processes in steps S107 to S110. If the determination condition in step S109 is not satisfied (No in step S109), the process ends without performing the process in step S110.

  By the way, although the object detection apparatus 10 to which the object detection method according to the present invention is applied has been described so far, an object detection system including a radar search apparatus, an image search apparatus, and a collision determination apparatus may be configured. FIG. 5 is a diagram illustrating a configuration of the object detection system 100. In the following description, the same components as those of the object detection apparatus 10 illustrated in FIG. 2 are denoted by the same reference numerals, and the description thereof will be omitted or only a brief description.

  As shown in FIG. 5, the object detection system 100 includes a radar search device 200, an image search device 300, and a collision determination device 400. The radar search device 200 corresponds to the radar 21 and the object recognition processing unit 12 shown in FIG. 2, and the image search device 300 corresponds to the camera 22 and the image search unit 13 in the same manner. The collision determination device 14 also corresponds to the collision determination processing unit 14.

  In this way, by configuring the radar search device 200 that performs radar search and the image search device 300 that performs image search as independent devices, for example, each device can be developed by different manufacturers. In FIG. 5, the object detection system 100 including the collision determination device 400 is shown, but an object detection system including the radar search device 200 and the image search device 300 may be configured.

  As described above, according to the present embodiment, the object type determination unit determines the type of the object based on the image captured by the camera mounted on the vehicle, and the threshold selection unit determines the determined type. The object detection device or the object detection device is configured to select an associated predetermined threshold and detect an object on the condition that the intensity of the reflected wave related to the irradiation wave irradiated on the range corresponding to the image is equal to or greater than the selected threshold. An object detection system was constructed.

  Therefore, by performing a radar search after changing to a threshold corresponding to the object type obtained from the image, even a weakly reflective object such as a pedestrian can be detected by the first radar search.

  In the above-described embodiment, threshold values corresponding to each object type are prepared, and the selected threshold values are used as they are (statically). However, these threshold values are obtained using the image search result. May be corrected. For example, when a factor that hinders the radar search such as rain or snow is detected from the image search result, the threshold values may be dynamically corrected to low values.

  As described above, the object detection device, the object detection method, and the object detection system according to the present invention are useful for detecting an object that is an obstacle to a traveling vehicle, and in particular, detection of a low-reflection object such as a pedestrian and a vehicle. This is suitable for the case where it is desired to detect a highly reflective object at high speed and reliably.

It is a figure which shows the outline | summary of the object detection method based on this invention. It is a block diagram which shows the structure of the object detection apparatus which concerns on a present Example. It is a figure which shows the outline | summary of an object detection determination. It is a flowchart which shows the process sequence which an object detection apparatus performs. It is a block diagram which shows the structure of an object detection system. It is a flowchart which shows the process sequence of the object detection method based on a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection apparatus 11 Object recognition process part 12 Radar search part 12a Threshold selection part 12b Object detection determination part 12c Object motion calculation part 13 Image search part 13a Object detection part 13b Object type determination part 14 Collision determination process part 14a Object motion acquisition part 14b Vehicle motion acquisition unit 14c Collision determination unit 14d Collision avoidance control unit 21 Radar 22 Camera 31 Vehicle sensor 32 Vehicle control device 100 Object detection system 200 Radar search device 300 Image search device 400 Collision determination device

Claims (8)

An object detection device for detecting an object in the vicinity of a vehicle,
Object type determination means for determining the type of the object based on an image captured by a camera mounted on the vehicle;
Threshold selection means for selecting a predetermined threshold associated with the type determined by the object type determination means;
An object detection determination unit configured to detect the object on condition that the intensity of the reflected wave related to the irradiation wave irradiated in the range corresponding to the image is equal to or greater than the threshold selected by the threshold determination unit. A featured object detection device. The object type determination means includes
Determining whether the type is a vehicle or a pedestrian,
The threshold value determining means includes
The object detection apparatus according to claim 1, wherein a threshold value corresponding to the pedestrian is determined as a value smaller than a threshold value corresponding to the vehicle. The object detection determination means is
3. The object detection apparatus according to claim 1, wherein when the plurality of different threshold values are selected by the threshold selection unit, the object is detected using only the smallest threshold value. The object type determination means includes
The position information of the object for which the type is determined is acquired in association with the type,
The object detection determination means is
3. The object according to claim 1, wherein, when a plurality of different threshold values are selected by the threshold selection unit, the object is detected using threshold values corresponding to the type based on the position information. The object detection apparatus described.   The object detection apparatus according to claim 1, further comprising a threshold correction unit that corrects the threshold selected by the threshold selection unit based on the image. Object motion acquisition means for calculating a distance to the object, a direction in which the object is located, and a relative speed between the object and the vehicle based on the reflected wave related to the object detected by the object detection determination means;
Vehicle motion acquisition means for acquiring the speed and traveling direction of the vehicle;
A collision determination unit for determining whether or not the object collides with the vehicle based on the information acquired by the object motion acquisition unit and the information acquired by the vehicle motion acquisition unit. The object detection device according to any one of claims 1 to 5. An object detection method for detecting an object existing in the vicinity of a vehicle,
An object type determination step of determining the type of the object based on an image captured by a camera mounted on the vehicle;
A threshold selection step of selecting a predetermined threshold associated with the type determined by the object type determination step;
And an object detection determination step for detecting the object on condition that the intensity of the reflected wave related to the irradiation wave irradiated with respect to the range corresponding to the image is not less than the threshold selected by the threshold determination step. Characteristic object detection method. An object detection system for detecting an object existing in the vicinity of a vehicle with an image search device and a radar search device mounted on the vehicle,
The image search device includes:
An object type determining means for determining the type of the object based on an image captured by a camera;
The radar search device
Threshold selection means for selecting a predetermined threshold associated with the type determined by the object type determination means of the imaging device;
An object detection determination unit configured to detect the object on condition that an intensity of a reflected wave related to an irradiation wave irradiated on a range corresponding to the image is equal to or greater than the threshold selected by the threshold determination unit. A featured object detection system.

Images