JP2009236623A - Object detector, periphery monitoring apparatus, driving support system, and object detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object detector and an object detection method for accurately recognizing various objects, and a periphery monitoring apparatus and a driving support system using an associated object detection. <P>SOLUTION: An object detection wave section 10 recognizes an image captured by a camera 32, identifies an object candidate area, and creates a threshold map for setting a threshold of a radar detection based on a type, a direction, a distance and an angle of the potential object existing in the area. A detecting/processing section 16 compares a reflection wave strength of a radar 31 with the threshold indicated in the threshold map, and detects the existence of the object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection device that detects the presence of an object using an input image captured by a camera and a reflected wave of a radar, an object detection method, and a peripheral monitoring device and a driving support system using such object detection. is there.

  Conventionally, a technique for detecting an object using an image captured by a camera or a reflected wave of a radar has been used. In particular, in recent years, it has been considered that a camera or radar is mounted on a vehicle to detect nearby vehicles or pedestrians to support driving operations and prevent accidents.

  In such on-vehicle object recognition, high-speed and high-precision object detection is required. Therefore, a combination of image recognition by a camera and radar detection has been used. For example, in Patent Document 1, in order to accurately recognize a preceding vehicle traveling in front of the host vehicle by radar, radar recognition is performed after eliminating reflections from vehicles other than the preceding vehicle and roadside objects using image recognition. The technology is disclosed.

JP 2003-84064 A

  However, with the conventional technology, objects other than the vehicle ahead cannot be recognized with high accuracy. A vehicle that is located at a relatively short distance from the host vehicle and is facing the host vehicle can obtain a radar reflected wave with a wide range and high intensity. This is because the intensity of the reflected wave is low for objects that are not. In addition, if the threshold value of the reflection intensity is lowered, it is possible to detect the presence of distant objects and objects that are not facing each other, but the number of false detections increases, and even if you know that an object exists, what is that object? Cannot be identified.

  Therefore, it is possible to detect the presence of an object directly, and more effectively use radar that can accurately measure the distance to the detected object, and can detect various objects around the vehicle with high accuracy. To improve the accuracy of vehicle periphery monitoring and driving support.

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems. An object detection apparatus, an object detection method, and such an object that can recognize various objects with high accuracy are provided. An object of the present invention is to provide a periphery monitoring device and a driving support system using detection.

  In order to solve the above-described problems and achieve the object, the object detection device, the object detection method, the periphery monitoring device using the object detection, and the driving support system according to the present invention include an input image captured by a camera, a radar When the presence of an object is detected using the reflected wave of the image, image recognition is performed on the input image, an object candidate area where the object may exist is detected from the input image, and the reflected wave of the radar is detected. A determination threshold value for determining the presence / absence of an object in comparison with intensity is set based on the recognition result by the object candidate area recognition unit, and the determination threshold value set by the threshold setting unit is compared with the reflection intensity of the radar. Is detected.

  According to the present invention, an object detection apparatus capable of accurately recognizing various objects such as an object at a long distance, an object of a type with low radar reflection intensity, and an object with low radar reflection intensity. The object detection method and the periphery monitoring device and the driving support system using the object detection can be obtained.

  Hereinafter, an object detection device, a periphery monitoring device, a driving support system, and an object detection method according to the present invention will be described with reference to the drawings. In addition, although an Example demonstrates a vehicle object detection apparatus as an example, it is applicable not only to this but to the general object detection using an image and a radar.

  FIG. 1 is a schematic configuration diagram showing a schematic configuration of an in-vehicle periphery monitoring device 20 according to an embodiment of the present invention. As shown in the figure, the periphery monitoring device 20 mounted on the vehicle is connected to a radar 31, a camera 32, a navigation device 33, a yaw rate sensor 34, a vehicle speed sensor 35, and a control ECU (Electronic Control Unit) 40.

  For example, the radar 31 scans and emits a millimeter wave in front of the host vehicle, acquires the reflected wave, and inputs the reflected wave to the periphery monitoring device 20. In addition, the camera 32 inputs an image obtained by photographing the front of the host vehicle to the periphery monitoring device 20. Here, the search range of the radar 31 and the imaging range of the camera 32 are arranged so that at least a part (preferably the entire range) overlaps. The object search by the radar 31 and the imaging by the camera 32 are repeatedly executed at a predetermined timing (more preferably, the timing at which the radar 31 and the camera 32 are synchronized).

  The navigation device 33 is an in-vehicle device that sets and guides a travel route using the position of the host vehicle specified by communicating with a GPS (Global Positioning System) artificial satellite and map data 33a stored in advance. In addition, the navigation device 33 provides the location monitoring device 20 with position information of the host vehicle, surrounding map information, information about roads, and the like.

  The yaw rate sensor 34 and the vehicle speed sensor 35 acquire the yaw rate and travel speed of the host vehicle, respectively, and input them to the periphery monitoring device 20.

  The control ECU 40 protects the occupant by notifying the passenger through the display 43 and the speaker 44, controlling the behavior of the vehicle by the brake 41 and the engine control device (EFI) 42, operating support based on the monitoring result of the periphery monitoring unit 20. Operation control of the airbag 45, which is a device that reduces collision damage, is performed. In addition, the control ECU 40 acquires information on the behavior of the vehicle from the brake 41 and the EFI 42, a steering control ECU and a transmission (shift) control ECU (not shown), and provides the information to the periphery monitoring device 20.

  The periphery monitoring device 20 includes an object detection unit 10, a collision determination unit 21, and a lane recognition processing unit 22 therein. The object detection unit 10 is a processing unit that detects an object around the host vehicle using the reflected wave of the radar 31 and the input image captured by the camera 32. The lane recognition processing unit 22 performs a process of recognizing a lane by identifying a white line or the like by image recognition on the input image.

  The collision determination unit 21 is a unit that determines the possibility of collision between the object detected by the object detection unit 10 and the host vehicle. In this determination, in addition to the detection result by the object detection unit 10, the recognition result by the lane recognition processing unit 22, the map information acquired from the navigation device 33, the information related to the planned travel route, the yaw rate sensor 34 and the vehicle speed sensor 35 are output. Information regarding the yaw rate and traveling speed of the host vehicle and the behavior of the vehicle acquired from the control ECU 40 is used.

  That is, the collision determination unit 21 acquires the type, position, and movement state of an object existing around the host vehicle from the object detection unit 10, and the host vehicle collides with the object from the traveling lane, planned route, map information, etc. of the host vehicle. Whether or not there is a possibility of the determination is comprehensively determined, and the determination result is output to the control ECU 40.

  The control ECU 40 receives this determination result, and notifies the driver using the display 43 or the speaker 44 when there is a possibility of a collision, and reduces the play of the brake 41 if necessary. Execute support control of the operation, control of the vehicle behavior such as applying the brake 41 and lowering the engine speed, and when the possibility of occurrence of a collision is very high, preparation for deployment of the airbag 45 and hoisting of the seat belt Control the operation of the collision damage mitigation device.

  The object detection unit 10 includes therein a reflected wave acquisition unit 11, an image acquisition unit 12, an object candidate region recognition processing unit 13, a threshold map creation unit 14, a reflection intensity database 15, and a detection processing unit 16.

  The reflected wave acquisition unit 11 performs a process of acquiring a reflected wave from the radar 31 and outputting it to the detection processing unit 16. Similarly, the image acquisition unit 12 acquires an image captured by the camera 12 as an input image, and provides it to the object candidate region recognition processing unit 13 and the lane recognition processing unit 22.

  The object candidate area recognition processing unit 13 executes image recognition processing using pattern matching or the like on the input image, and recognizes an area where an object may exist as an object candidate area. At this time, the object candidate area recognition processing unit 13 recognizes the type of object estimated to exist in the object candidate area, in other words, the type of object that may exist in the object candidate area, the existing direction, the distance, and the angle. . The types of objects to be recognized are objects that are involved in vehicle travel and may become obstacles, such as other vehicles, pedestrians, and motorcycles. Note that the possibility of existing in the object candidate area is determined as “possibility” when the degree of matching is 80% or more by pattern matching processing, for example. The coincidence threshold may be changed according to the environment (time, season, region, etc.) in which the vehicle is placed.

  The threshold map creating unit 14 sets a determination threshold for object detection based on the reflected wave intensity of the radar based on the recognition result of the object candidate area. Specifically, the threshold map creation unit 14 creates a threshold map in which regions having different determination thresholds are set with respect to the radar search range based on the position of the object candidate region. Here, the threshold value is determined with reference to the reflection intensity database 15 in which the determination threshold value is determined for each type of object, existing direction, distance, and angle.

  The detection processing unit 16 detects the object by comparing the intensity of the reflected wave acquired by the reflected wave acquisition unit 11 with the threshold defined by the threshold map, and outputs the detection result to the collision determination unit 21.

  The processing of the object detection unit 10 will be described with a specific example. FIG. 2 is an explanatory diagram for explaining threshold map creation from the image recognition result. In the example shown in the figure, the object candidate area recognition processing unit 13 recognizes three object candidate areas in the shooting range of the camera, that is, in the input image.

  In each object candidate area, the horizontal position in the image indicates the direction of the object with respect to the own vehicle, and the vertical position (height) at the lower end of the object candidate area indicates the distance from the own vehicle to the object. ing. Further, the type of the object and the angle of the object with respect to the host vehicle can be identified by image recognition.

  In the example shown in FIG. 2, a candidate area for a pedestrian who is facing the host vehicle in the left direction from the host vehicle (angle 0 °), is facing the host vehicle in front of the host vehicle ( A candidate area of a vehicle (vehicle type 1) with an angle of 0 ° and a candidate area of a vehicle (vehicle type 1) with an angle of 30 ° with respect to the host vehicle are recognized in the right direction of the host vehicle.

  The threshold map creation unit 14 converts the object candidate area map, which is the image recognition result, into a threshold map for reflection intensity. Specifically, based on the relationship between the shooting range of the camera and the search range of the radar, the position of each object candidate region in the object candidate region map is replaced with a corresponding position in the radar search range, and the object type of the object candidate region The reflection intensity database is searched based on the direction, distance, and angle, and the threshold value is determined.

  In the figure, for example, a threshold of 30% is set at a position corresponding to the left object candidate area, a threshold of 60% is set at a position corresponding to the center object candidate area, and a threshold of 40% is set at a position corresponding to the right object candidate area. A threshold of 80% is set in a place other than the object candidate area.

  If the object candidate area in the front direction is another vehicle that is located at a short distance in front of the host vehicle and is facing the host vehicle as shown in the image recognition, a strong reflected wave should be obtained from this area. It is. Therefore, the threshold is set to a relatively high value of 60%, and if the reflection intensity is 60% or more, it is determined that an object as recognized by the image, that is, another vehicle exists.

  In addition, when the reflected wave returns from the direction of the right object candidate region, it is considered that the vehicle is another vehicle having an angle of 30 ° with respect to the host vehicle in consideration of the image recognition result. Here, the intensity of the reflected wave is lower in a vehicle that is not directly facing the host vehicle than in a vehicle that is facing the vehicle. Therefore, the threshold is set to a relatively low value of 40%, and if the reflection intensity is 40% or more, it is determined that an object as recognized by the image, that is, another vehicle exists.

  Similarly, when a reflected wave returns from the direction of the object candidate area on the left side, it is considered that the person is a pedestrian facing the own vehicle in consideration of the result of image recognition. Here, the intensity of the reflected wave from the pedestrian is small even if the pedestrian faces the host vehicle. On the other hand, prevention of accidents with pedestrians during vehicle travel is very important. Therefore, for this object candidate region, the determination threshold is set to a relatively low value of 30%, and if the reflection intensity is 30% or more, it is determined that there is a pedestrian as recognized by the image.

  For a search range in which no object candidate area is set, the determination threshold is 80%, and when the reflection intensity is 80%, it is recognized that some object exists. This is a threshold value for determining that an object is present only by radar detection even if the presence of the object cannot be identified by image recognition. For example, a threshold value for detecting an object using a conventional radar alone may be used.

  Next, various objects and reflection intensity will be described. FIG. 3 is an explanatory diagram for explaining the reflection intensity from the vehicle. As shown in FIG. 3, the vehicle C1 faces the host vehicle C0, that is, the surface of the vehicle C1 that receives the radar search wave emitted from the host vehicle C0 to the vehicle C1 is perpendicular to the emission direction. In this case, the search wave returns straight in the direction of the host vehicle, and the intensity of the reflected wave increases as shown as the state (a) in FIG.

  On the other hand, like the vehicle C2 shown in FIG. 3, the surface of the vehicle C1 that has an angle with respect to the own vehicle C0, that is, receives the radar search wave emitted from the own vehicle C0 to the vehicle C2, is relative to the emission direction. In the case of not being vertical, as shown in FIG. 4 as the state (b), the component of the search wave that scatters in the lateral direction increases, and the intensity of the reflected wave received on the own vehicle side decreases. Therefore, in order to detect a vehicle having an angle with respect to the own vehicle by the radar, it is necessary to lower the threshold value as compared with the case of detecting a vehicle facing the own vehicle.

  Further, when the object to be detected is a two-wheeled vehicle, as shown in FIG. 5, the reflection intensity is small when the two-wheeled vehicle is directly facing the own vehicle, and the angle is 90 ° (sideways) with respect to the own vehicle. In some cases, the reflection intensity increases. Therefore, when detecting a two-wheeled vehicle, it is necessary to decrease the threshold when facing the host vehicle and increase the threshold when the vehicle is close to the side.

  Similarly, when the object to be detected is a pedestrian, as shown in FIG. 6, the reflected wave has a large scattered component regardless of the direction (angle) of the pedestrian, and the vehicle side The received reflection intensity is low. Therefore, when detecting a pedestrian, the threshold is set small to increase the sensitivity of radar detection.

  FIG. 7 is an explanatory diagram illustrating a specific example of the reflection intensity database. As shown in the figure, the reflection intensity database 15 defines a reflection intensity determination threshold in association with items of object type, direction, distance, and angle.

  Specifically, when the object type is “vehicle type 1”, the direction is “0 °”, the distance is “−10 m”, and the object angle is “0 °”, the reflection intensity threshold is set to “60%”. . And when the direction and distance are the same and the angle is changed, the angle “90 °” at which the vehicle is directly beside can be expected to have a strong reflection intensity as in the case of the angle “0 °” facing directly. The intensity threshold is set to “60%”. At an angle of 30 ° or an angle of 60 °, the scattering component of the reflected wave from the vehicle increases, so the threshold value is reduced. At the angle of 30 °, the reflection intensity threshold value is “40%”, and at the angle 60 °, the reflection intensity threshold value is “ 50% ".

  Even if the direction from the host vehicle is “0 °”, the intensity of the reflected wave decreases as the distance from the host vehicle increases. Therefore, in the example shown in the figure, in the case of “vehicle type 1”, direction “0 °”, distance “˜20 m”, and object angle “0 °”, the reflection intensity threshold is “55%”. When the angle is “30 °”, the reflection intensity threshold is “45%”.

  Further, since the reflection intensity changes depending on the direction from the host vehicle, the reflection intensity threshold is set in the case of “vehicle type 1”, direction “15 °”, distance “−10 m”, and object angle “0 °”. In the case of “55%” and the angle of the object “30 °”, the reflection intensity threshold is “45%”.

  Such reflection intensity threshold data is set for each type of vehicle (for example, ordinary vehicles (sedans, hatchbacks, wagons, RV vehicles, etc.), trucks, buses, light vehicles, etc.). Similarly, a reflection intensity threshold value is set for two-wheeled vehicles and pedestrians according to the type, direction, distance, and angle. In the case of a vehicle, the reflection intensity is high in a state of facing up (angle 0 °) and in a sideways state (angle of 90 °). Is low, and the reflection intensity increases as the state becomes horizontal (angle 90 °). In addition, in the case of a pedestrian, the reflection intensity is low regardless of whether it is a face-to-face state (angle 0 °) or a sideways state (angle 90 °). It is the same as that of the vehicle in that a reflection intensity threshold is set for each type of motorcycle and pedestrian.

  Here, the unit of the reflection intensity threshold is “%”, but the threshold may be set in a different unit such as dB.

  Next, the processing operation of the object detection unit 10 will be described with reference to the flowchart of FIG. The processing operation shown in the figure is processing that is repeatedly executed each time the camera 32 performs imaging.

  As shown in the figure, first, the image processing unit 12 acquires an image from the camera 32 (step S101), and the object candidate amount region recognition processing unit 13 executes image recognition (step S102). As a result, when an object candidate area exists in the image (step S103, Yes), the object type, position (direction and distance), and angle are recognized for each object candidate area (step S104).

  Thereafter (No at Step S103, or after Step S104), the threshold map creation unit 14 creates a threshold map by referring to the reflection intensity database 15 based on the recognition result of the object candidate region (Step S105).

  The reflected wave acquisition unit 11 acquires the reflected wave of the radar 31 (step S106), and the detection processing unit 16 compares the intensity of the reflected wave with a threshold map and executes an object detection process (step S107). .

  As a result, when an object is detected (the reflection intensity is greater than or equal to the threshold value) (step S108, Yes), the position information based on the radar detection result and the object type based on the image recognition are output for each detected object (step S109). ), The process is terminated. If no object is detected (No at step S108), the process is terminated.

  As described above, the object detection wave unit 10 according to the present embodiment identifies an object candidate area by image recognition, and performs radar based on the type, direction, distance, and angle of an object that may exist in the area. Create a threshold map with detection thresholds. Using this threshold map, it is determined from the reflected wave intensity of the radar 31 whether or not an object exists.

  For this reason, it is possible to accurately recognize objects that are difficult to accurately detect with radar alone, such as objects at long distances, vehicles that are not directly facing the host vehicle, and motorcycles and pedestrians with low original reflection strength. can do.

  In addition to the accurate distance obtained by the radar for the detected object, it is possible to obtain information on the type, information on the state, etc. by image recognition.

  Therefore, the periphery monitoring device 20 using the detection result of the object detection unit 10 can accurately perform the collision determination with the object, and the control ECU 40 can obtain the collision determination result with high accuracy, so that the operation of driving assistance is accurately performed. Can be done.

  The configuration and operation shown in this embodiment are merely examples, and the present invention is not limited to this. For example, in the present embodiment, the configuration in which the periphery monitoring device 20 includes the object detection unit 10 has been described as an example. However, the detection result of the object detection device is determined by another device as an object detection device that performs object detection. It is good also as a structure to use.

  Needless to say, the information used for collision determination and the method of using the collision determination result can be modified as appropriate. For example, information acquired by VICS (Vehicle Information and Communication System) may be used for collision determination, and the state of the transmission can be further controlled.

  Furthermore, in the above embodiment, the type and angle information of the object are acquired by the camera, but not limited thereto, the type, position, and angle of the object are determined based on the inter-vehicle communication, the road-to-vehicle communication, or the position information of the mobile phone. The object may be detected by estimating and changing the threshold value of the radar.

  As described above, the object detection device, the periphery monitoring device, the driving support system, and the object detection method according to the present invention are useful for improving the object detection accuracy, and are particularly suitable for object detection and driving support in a vehicle.

It is a schematic block diagram which shows schematic structure of the periphery monitoring apparatus concerning the Example of this invention. It is explanatory drawing explaining the threshold value map creation from an image recognition result. It is explanatory drawing explaining the direction from the own vehicle to an object, and the angle with respect to the own vehicle. It is explanatory drawing explaining the angle and reflection intensity of a vehicle. It is explanatory drawing explaining the angle and reflection intensity of a two-wheeled vehicle. It is explanatory drawing explaining a pedestrian's angle and reflection intensity. It is explanatory drawing explaining a reflection intensity database. It is a flowchart explaining the processing operation | movement of an object detection process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection part 11 Reflected wave acquisition part 12 Image acquisition part 13 Object candidate area | region recognition process part 14 Threshold map creation part 15 Reflection intensity database 16 Detection process part 20 Perimeter monitoring apparatus 21 Collision determination part 22 Lane recognition process part 31 Radar 32 Camera 33 Navigation device 33a Map data 34 Yaw rate sensor 35 Vehicle speed sensor 40 Control ECU
41 Brake 42 EFI
43 Display 44 Speaker 45 Airbag

Claims (11)

An object detection device that detects the presence of an object using a reflected wave of a radar,
Object candidate area recognition means for detecting an object candidate area where the presence of an object is estimated using means other than the radar;
A threshold value setting means for setting a determination threshold value for determining the presence or absence of an object in comparison with the reflected wave intensity of the radar, based on a recognition result by the object candidate area recognition means;
Detection processing means for detecting an object by comparing the determination threshold set by the threshold setting means with the reflection intensity of the radar;
An object detection device comprising:   The threshold setting means creates a threshold map in which areas having different determination thresholds are set for the radar search range based on the position of the object candidate area, and the detection processing means is based on a radar search direction. The object detection apparatus according to claim 1, wherein a determination threshold is selected with reference to the threshold map.   3. The object detection device according to claim 1, wherein the object candidate area recognition unit performs image recognition on an input image captured by a camera and detects the object candidate area from the input image. .   The object candidate area recognizing means recognizes at least one of a type, an existing direction, a distance, and an angle of an object that may exist in the object candidate area, and the threshold setting means determines the result of the identification. The object detection device according to claim 1, wherein a threshold value is set based on the threshold value.   The object detection apparatus according to claim 1, wherein the object candidate area recognition unit recognizes an object candidate area in which an obstacle related to vehicle travel may exist.   A database for determining a determination threshold value for each type of object, existing direction, distance, and angle; and the threshold setting unit refers to the database based on a recognition result of the object candidate area, and the object candidate area The object detection device according to claim 4, wherein a determination threshold value to be set in a radar detection range corresponding to is obtained. A periphery monitoring device that monitors an object around the host vehicle using an input image captured by a camera and a reflected wave of a radar,
Object candidate area recognition means for performing image recognition on the input image and detecting an object candidate area where an object may exist from the input image;
A threshold value setting means for setting a determination threshold value for determining the presence or absence of an object in comparison with the reflected wave intensity of the radar, based on a recognition result by the object candidate area recognition means;
Detection processing means for detecting an object by comparing the determination threshold set by the threshold setting means with the reflection intensity of the radar;
A collision determination means for determining the possibility of collision between the object detected by the detection processing means and the host vehicle;
A peripheral monitoring device comprising:   The peripheral according to claim 7, further comprising lane recognition means for recognizing information indicating a lane from the input image, wherein the collision determination means further makes a collision determination using a recognition result by the lane recognition means. Monitoring device.   The periphery monitoring device according to claim 7 or 8, wherein the collision determination unit performs the collision determination by further using map information, planned traveling route information of the own vehicle, and behavior information indicating the behavior of the own vehicle. . A driving support system that monitors an object around the host vehicle using an input image captured by a camera and a reflected wave of a radar, and supports driving operation of the vehicle based on a monitoring result,
Object candidate area recognition means for performing image recognition on the input image and detecting an object candidate area where an object may exist from the input image;
A threshold value setting means for setting a determination threshold value for determining the presence or absence of an object in comparison with the reflected wave intensity of the radar, based on a recognition result by the object candidate area recognition means;
Detection processing means for detecting an object by comparing the determination threshold set by the threshold setting means with the reflection intensity of the radar;
A collision determination means for determining the possibility of collision between the object detected by the detection processing means and the host vehicle;
Control means for executing at least one of notification control for the driver, driving operation support control, vehicle behavior control, and collision damage mitigation device operation control based on the determination result by the collision determination means;
A driving support system characterized by comprising: An object detection method for detecting the presence of an object using an input image captured by a camera and a reflected wave of a radar,
An object candidate region recognition step of performing image recognition on the input image and detecting an object candidate region in which an object may exist from the input image;
A threshold setting step for setting a determination threshold for determining the presence or absence of an object in comparison with the reflected wave intensity of the radar based on a recognition result by the object candidate region recognition step;
A detection processing step of detecting an object by comparing the determination threshold set by the threshold setting step and the reflection intensity of the radar;
The object detection method characterized by including.

Images