JP2011027457A - Object detecting device, information processing method and information processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for reducing a processing load on an electronic control device in determining the type of an object in an image photographed by a camera. <P>SOLUTION: An object detecting device predicts the type of an object based on radar information detected by a radar device, and determines the type of the object by an electronic control device by cutting out a predetermined image range in a single frame of an image photographed by a camera based on the predicted type of the object. With this configuration, the processing load for determining the type of an object by an electronic control device using an image is reduced. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for performing vehicle control using object information and image information detected by a radar apparatus.

  In general, in a radar apparatus, a signal such as a radio wave transmitted from an antenna rebounds from an object outside the vehicle, and the antenna receives the rebounded signal. Based on the time difference between the signal transmitted from the antenna and the signal received by the antenna and the Doppler shift information, the distance between the vehicle and the object outside the vehicle, the speed of the object viewed from the vehicle, the angle of the object outside the vehicle viewed from the vehicle, and The reflection intensity of the received wave received from the object outside the vehicle can be detected.

  However, when detecting an object using only a radar device, a predetermined threshold is set to distinguish received wave information bounced from the object and noise information, and a signal having a reflection intensity exceeding the threshold is set as the object. To detect. As a result, it is possible to detect an object such as a vehicle or a guardrail with high reflection intensity, but there is a problem that it is difficult to detect a person with low reflection intensity. Therefore, in order to detect a person with low reflection intensity, it is necessary to lower the threshold for detecting an object. However, if the threshold is lowered, noise information is also detected as an object, and erroneous vehicle control is performed. There was a problem of doing.

  In addition, there is a technique in which an image processing apparatus including a camera is mounted on a vehicle, and image information for each frame captured by the camera is scanned to determine the type of object in the image.

  When an object is detected only by an image processing device equipped with a camera, even if the type of object can be determined, the relative speed cannot be detected, so the time until the vehicle and the object collide is detected. It was difficult to perform vehicle control to avoid a collision. In addition, since it is necessary to perform scanning processing for detecting an object to be subjected to vehicle control for the entire area of the image for each frame, an electronic control device that is mounted on a vehicle and determines the type of an object in the image. There has been a problem that the processing load of the CPU becomes large and quick vehicle control cannot be performed.

  In order to solve the problem in the case of using only such a radar device or in the case of using only a camera, the distance, angle, and distance of an object outside the vehicle are combined with a radar device and an image processing device equipped with a camera. There is a technique for performing vehicle control that discriminates the relative speed and the type of an object and avoids a collision between the vehicle and the object.

  In addition, when image processing is performed by combining information of a radar apparatus and an image processing apparatus, a technique for limiting an image processing range captured by a camera using distance information of an object detected by the radar apparatus is disclosed. (For example, refer to Patent Document 1).

JP-A-9-264554

  However, even when an object is detected by combining a radar apparatus and an image processing apparatus using a camera, it is necessary to perform a scanning process for detecting an object to be subjected to vehicle control for every region of the image for each frame. For this reason, there has been a problem that the processing load on the CPU of the electronic control device is increased and rapid vehicle control cannot be performed.

  In the technology described in Patent Document 1, when a white line and a vehicle are used as objects, the image processing load is reduced by setting a predetermined range including an image of the object in one frame as an object of image processing. Is described, however, image processing of different ranges depending on the size of the target object is not considered for each type of target object, and the image processing load is increased.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technology OLE_LINK1 that reduces the image processing load of the OLE_LINK1 electronic control device and performs vehicle control quickly.

  In order to solve the above-described problem, the invention of claim 1 is directed to radar information from a radar device that is provided in a vehicle and detects the position and speed of an object outside the vehicle, and a detection range of the radar device that is provided in the vehicle. An object detection device that detects a detection target outside the vehicle based on an image obtained by a camera that captures an overlapping region, and a prediction unit that predicts the type of the candidate object based on the radar information; Deriving means for deriving the coordinates in the image of the candidate object to be detected based on the radar information, and the predicted type of the candidate object in the vicinity of the derived coordinates of the candidate object in the image An object comprising: setting means for setting a region having a size corresponding to the processing target region; and detecting means for detecting the detection target only for the processing target region in the image. Detection device.

  The invention according to claim 2 is the object detection apparatus according to claim 1, wherein when the detection unit detects the detection target from the processing target region, the prediction of the candidate object corresponding to the processing target region is performed. An object detection apparatus using a determination method according to the type of the object.

  According to a third aspect of the present invention, in the object detection device according to the first or second aspect, the acquisition unit that acquires the traveling information indicating the traveling state of the vehicle, and the radar information based on the traveling information. An object detection apparatus further comprising means for selecting a part of a candidate object as a processing target.

  According to a fourth aspect of the present invention, in the object detection apparatus according to any one of the first to third aspects, at least one of the derivation unit, the prediction unit, the setting unit, and the detection unit is When there are a plurality of candidate objects based on radar information, the object detection apparatus is characterized in that a processing target is selected in order from the closest distance to the vehicle among the plurality of candidate objects.

  According to a fifth aspect of the present invention, there is provided a radar information from a radar device that detects a position and a relative speed of an object outside the vehicle provided in the vehicle, and a region that overlaps a detection range of the radar device provided in the vehicle. An object detection method for detecting a detection target outside the vehicle based on an image obtained by an imaging camera, the step of predicting a type of the candidate object based on the radar information, and the radar information Deriving the coordinates in the image of the candidate object with the detection target based on, and the size of the candidate object in the size around the derived coordinates of the candidate object in the image according to the predicted type of the candidate object An object detection method comprising: setting a region as a processing target region; and detecting the detection target only for the processing target region in the image.

  According to a sixth aspect of the present invention, there is provided an object detection system comprising: a radar device provided in a vehicle for detecting a position and speed of an object outside the vehicle; and a region provided in the vehicle and overlapping a detection range of the radar device. The object detection device according to any one of claims 1 to 4, wherein a detection target outside the vehicle is detected based on a camera that captures the image, radar information from the radar device, and an image obtained by the camera. An object detection system comprising:

  According to the first to sixth aspects of the present invention, the image processing performed by the electronic control device is limited to the image region where the object to be detected exists, and the processing load of the radar device is reduced while reducing the image processing load for detecting the object. The processing load of the entire object detection system can be reduced. Moreover, the vehicle control with respect to the object can be quickly performed by reducing the processing load of the electronic control device.

  In particular, according to the second aspect of the present invention, it is not necessary to repeat the object type determination by the electronic control unit by predicting the object type in advance based on information from the radar device, so that the object type determination can be performed quickly. It can be done. Furthermore, the vehicle control on the object can be quickly performed by reducing the processing load of the electronic control unit.

  In particular, according to the invention of claim 3, by using the traveling information indicating the traveling state of the vehicle together with the information of the radar device, a candidate object to be processed can be selected according to the traveling state of the vehicle. . As a result, the processing load on the electronic control device is reduced, and vehicle control according to the running state of the vehicle can be performed quickly.

  In particular, according to the invention of claim 4, it is possible to detect in order from an object that is most likely to be a target of vehicle control, and it is possible to quickly ensure the safety of a vehicle occupant such as a driver.

FIG. 1 is a diagram showing the entire vehicle. FIG. 2 is a block diagram of the object detection system. FIG. 3 is a diagram illustrating an object detection range of an image processing apparatus including a radar apparatus and a camera. FIG. 4 is a diagram showing object information in terms of frequency and power. FIG. 5 is a diagram illustrating an image of object detection by an image processing apparatus including a camera. FIG. 6 is a process flowchart of the electronic control device. FIG. 7 is a block diagram of an object detection system including a navigation device and various sensors. FIG. 8 is a diagram showing an object detection range by an image processing device including a radar device and a camera at a curve. FIG. 9 is a diagram illustrating an image of object detection by an image processing apparatus including a camera on a curve. FIG. 10 is a process flowchart of the electronic control device using the travel information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Embodiment 1]
<1-1. Configuration>
FIG. 1 is an overall view of the vehicle 1. The vehicle 1 is provided with a radar device 2, an image processing device 3, and an electronic control device 4 as elements provided in the object detection system of the present embodiment. The radar device 2 is provided at a front portion in front of the vehicle. The radar device 2 is a vehicle having a distance between the vehicle 1 and an object outside the vehicle, an angle of the object outside the vehicle as viewed from the vehicle 1 (hereinafter referred to as “object position”), and a speed of the object outside the vehicle as viewed from the vehicle 1. The relative speed between 1 and an object outside the vehicle, or the absolute speed of an object outside the vehicle (hereinafter referred to as “speed information”) is detected. The mounting position of the radar device 2 on the vehicle 1 is not limited to the front portion in front of the vehicle, and may be provided at the rear or side of the vehicle 1.

  The radar device 2 repels a signal such as a radio wave transmitted from an antenna (hereinafter referred to as “transmission wave”) and strikes an object outside the vehicle, and receives the rebound signal (hereinafter referred to as “reception wave”). Based on the time difference between the transmission wave and the reception wave and information on the Doppler shift, the radar apparatus 2 detects the position and speed information of the object. The object detection range of the radar device 2 is the range of R1.

  The image processing device 3 is provided in the OLE_LINK 2 near the vehicle interior ceiling of the OLE_LINK 2 windshield of the vehicle 1. The image processing apparatus 3 includes a camera that captures the detection range C1 and electronically acquires an image, and detects the position and size of an object in an image obtained by capturing these ranges. The mounting position of the image processing apparatus 3 provided with the camera on the vehicle 1 is not limited to the vicinity of the ceiling of the vehicle interior of the windshield, and may be provided on the rear or side of the vehicle inside and outside.

  The electronic control unit 4 is provided inside the vehicle body of the vehicle 1. The electronic control unit 4 receives information on an object outside the vehicle obtained from the radar device 2 and the image processing device 3 (object position, velocity information, angle information, and a received wave received by the antenna of the radar device 2 by bouncing off the object). The type of the object outside the vehicle is determined using the reflection intensity, the position of the object on the image, the size of the object, etc., and the vehicle is controlled according to the possibility of the collision between the object outside the vehicle and the vehicle 1.

  Vehicle control notifies the driver and other passengers of the danger of a collision between the vehicle 1 and an object outside the vehicle, automatically intervenes in the driving operation before the collision, applies the brakes, or steers the steering in a predetermined direction. To avoid collisions with objects outside the vehicle. Also, the seat belt 5 is automatically pulled in the direction of the arrow to fix the body of the driver or other occupant during the collision, or the headrest 6 is moved in the direction of the arrow during the collision to alleviate the impact of the collision. There are also vehicle controls that ensure safety.

  FIG. 2 is a block diagram of the object detection system 10. The radar apparatus 2 outputs a transmission wave from the antenna 12 and receives a reception wave bounced off an object outside the vehicle. The radar control unit 22 generates a transmission wave or mixes the reception wave and the transmission wave received by the antenna 21 with the time difference between the transmission wave and the reception wave and information on the Doppler shift as a beat signal. calculate. The radar controller 22 performs fast Fourier transform on the calculated beat signal to calculate distance information and speed information between the vehicle 1 and an object outside the vehicle.

  Here, a specific example of object detection by the radar apparatus 2 will be described with reference to FIGS. FIG. 3 is a diagram illustrating an object detection range of the image processing apparatus 3 including a radar device and a camera, and FIG. 4 is a diagram illustrating object information in terms of frequency and power.

  As shown in FIG. 3, the manhole 101, the person 102, the oncoming vehicle 103, and the distant vehicle 104 are included in the detection range R <b> 1 of the radar device 2. The distance from the vehicle 1 is the manhole 101, the person 102, the oncoming vehicle 103, and the distant vehicle 104 in order from the closest. FIG. 4 shows information of these objects 101 to 104 expressed by frequency and power. The object information of FIG. 4 represented by frequency and power shows the detection result of the object in the table with the horizontal axis representing frequency and the vertical axis representing power. The frequency on the horizontal axis indicates the distance between the vehicle 1 and the object, and the distance between the vehicle 1 and the object increases as the frequency increases.

  Further, the power on the vertical axis indicates the reflection intensity of the received wave from the object. Therefore, the power increases when the reflected area is large and a signal such as a radio wave is likely to rebound (the reflectance of the surface signal is high), and the power increases as the distance from the vehicle 1 decreases. Further, the power is weaker as the reflection area is smaller and the surface reflectance is lower, and the power is weaker as the distance from the vehicle 1 is farther. In the table of FIG. 4, signals having various values are called peaks.

  Looking at which of the objects in FIG. 3 corresponds to this peak, the information of the object in the manhole 101 exceeds a predetermined threshold value h among the plurality of peaks shown in FIG. Corresponds to peak A at the closest distance. Here, the predetermined threshold is a value that can detect a person whose reflection intensity is the weakest among the detected objects, and a peak that does not exceed the threshold is regarded as noise and is not a detection target as an object.

  Next, the object information of the person 102 at the closest distance from the vehicle 1 after the manhole 101 corresponds to the peak B in FIG. 4, the oncoming vehicle 103 corresponds to the peak C, and the far vehicle 104 corresponds to the peak D. Here, the peak N exceeding the threshold value h and having a reflection intensity similar to that of the person 102 does not correspond to any object in FIG. In order to detect an object with low reflection intensity such as a person, it is necessary to lower the threshold value h as much as possible and extract a peak exceeding the power threshold value h for detecting the object as object information. As a result of lowering, there is a possibility that even a non-existing object is extracted as a peak having object information. The processing in the case of extracting information on an object that does not originally exist as the presence of the object will be described in detail by processing image information by the electronic control unit 4 described later.

  In FIG. 4, regarding the peak C corresponding to the oncoming vehicle 103 and the peak D corresponding to the distant vehicle 104, both peaks exceed the power threshold value h, and the vehicle is in the order of the peak C and the peak D. The distance from 1 is in a state of being separated.

  In addition, regarding the angle of each object viewed from the vehicle 1, in the case of scanning with the antenna 12 swung to a predetermined angle (for example, a maximum swing angle of 30 degrees), the antenna swing angle and the received wave are associated with each other. Is calculated. When scanning is performed with the antenna fixed in a predetermined direction without shaking the antenna 12, the angle is calculated from the phase difference information of the received wave for each peak in FIG.

  Next, returning to FIG. 2, the image processing device 3 and the electronic control device 4 will be described. The image processing device 3 uses the camera 13 to photograph the range C1 that is the range that overlaps the object detection range R1 of the radar device 2. The camera control unit 23 electrically connected to the camera 13 generates an image related to only a predetermined image range from one framed image based on information of the electronic control device 4 to be described later. Send to.

  The electronic control device 4 includes a CPU 14 that receives information from the radar device 2 and the image processing device 3, and a memory 24 that is electrically connected to the CPU 14. The CPU 14 receives information such as the object position, speed information, and object type from the radar device 2 and the image processing device 3, and performs vehicle control by operating an alarm device 17, a brake 27, and a steering wheel 37, which will be described later. . The memory 24 stores information for determining the type of object. Examples of information for determining the type of an object include template information for various objects for determining the size of an object in an image.

  Specific examples of images processed by the electronic control device 4 and the image processing device 3 will be described with reference to FIGS. 3 and 5. FIG. 5 is a diagram illustrating an image acquired by the image processing apparatus 3 including a camera. In FIG. 3, there are a manhole 101, a person 102, an oncoming vehicle 103, and a distant vehicle 104 within a range of C <b> 1 that is a shooting range of the image processing apparatus 3. Note that the noise detected by the radar device 2 and described as the peak N in FIG. 4 does not exist as an object in the detection range.

  As shown in FIG. 5, the manhole 101, the person 102, the oncoming vehicle 103, and the distant vehicle 104 exist in the image captured by the image processing device 3. The position of the image of each object is marked with x. These X marks are attached corresponding to the information of the object detected by the radar device 2, and each X mark has distance information, speed information, angle information, and reflection intensity information with respect to the vehicle 1. The electronic control unit 4 determines which position on the image of one frame each object corresponds to from the object information of the radar device 2 based on the distance and angle information. Further, the electronic control unit 4 also determines whether or not the X mark, which is information on each object, is combined to form one object from the speed information and the reflection intensity information, and the size of the combined range, etc. The type of object is predicted from the information. In the image, the information on the noise peak N shown in FIG. 4 is also detected on the image as the object 105.

  As a specific example of object type prediction, for example, the manhole 101 has no reflection point in the periphery having the same property that the absolute velocity is zero and the reflection intensity is relatively strong, and the absolute velocity is zero and the reflection intensity is relatively strong. Therefore, it can be determined that the object is not an object having a plurality of reflection points. The electronic control unit 4 predicts the object type of the manhole 101 as a manhole from the object type determination information stored in advance in the memory 24 in the electronic control unit 4.

  In addition, since there is no reflection point in the vicinity with a reflection point having a relatively low relative velocity and a relatively low reflection intensity, the electronic control unit 4 determines the type of the object of the person 102 based on the information in the memory 24. Predict. Since there are reflection points having a relatively high relative velocity and a relatively strong reflection intensity, and there are a plurality of reflection points having the same properties in the vicinity region, the electronic control unit 4 predicts the object type of the oncoming vehicle 103 as a vehicle. To do. In addition, since the relative speed is constant, the reflection intensity is relatively strong, and there are a plurality of reflection points having the same property in the vicinity region, so the electronic control unit 4 predicts the type of the object of the distant vehicle 104 as a vehicle.

  In addition, since there is no reflection point in the periphery with a reflection point having a relatively low relative velocity and a relatively low reflection intensity, the electronic control unit 4 determines the object type of the object 105 based on the information in the memory 24. Predict.

  In this way, the electronic control unit 4 estimates the object type from the radar information, and sets an image range of a size according to the object type determination information stored in advance in the memory 24 according to the object type. Then, the image processing apparatus 3 is instructed to cut out the selected image range. For example, if the estimated object type is a pedestrian, a region having a relatively small vertical size is selected. In addition, if the estimated object type is a vehicle, a relatively large horizontally long region is selected. In response to the cut instruction, the image processing apparatus 3 cuts the designated image range. In addition to the person and vehicle described above, the object type determination pattern can be stored in the memory 24 according to various object types such as two-wheeled vehicles, power poles, pedestrian bridges, and iron bridges. It is possible to select regions according to the sizes of different objects.

  The cut image is transmitted as a processing target area from the image processing apparatus 3 to the electronic control apparatus 4, and image recognition for determining the type of the object is performed by the electronic control apparatus 4 only for the processing target area. At this time, a determination method according to the type of object predicted when the processing target area is cut out is used. For example, as an example of an image recognition method when the type of the predicted object is a person, the image size α of the person is cut out from the prediction of the candidate object based on the radar information, and the brightness of the image is scanned by scanning the range. Detect changes. If the luminance changes with a predetermined width, an object corresponding to the size of the width can be determined as a person. For example, in the case of the object 102 shown in FIG. 5, since it has a predetermined width symmetrically in the X-axis direction from the central axis L, the type of the object can be determined as a person. On the other hand, in the case of the object 105 shown in the figure, since the change in the luminance of the image corresponding to the change in the luminance of the person is not detected even if the image obtained by cutting the image range β is scanned, Not determined as noise.

  In addition, as a determination method when the predicted object type is a vehicle, the brightness of the image is detected in a different direction from the edge portion where the change in the brightness of the image is detected, and the width of the change in the brightness in the detected direction is large. There is a method for discriminating an object. For example, in FIG. 5, when a change in the brightness of the image at the point P is detected by scanning the image range γ in the X-axis direction, the change in the brightness of the image is detected from the point P in the Y-axis direction instead of the X-axis direction. Thus, the object type of the oncoming vehicle 103 is determined to be a vehicle.

  Further, as a technique that can be used when the type of the predicted object is both a person and a vehicle, the range of luminance change of the scanned image is detected and stored in the memory 24 in advance. There is also a technique for discriminating the type of an object by applying a template of the size and shape of the object. For example, in FIG. 5, the object type of the far vehicle 104 is determined to be a vehicle by applying a vehicle template to the far vehicle 104 in the image range θ.

<1-2. Operation>
The processing of the electronic control device 4 will be described with reference to the flowchart shown in FIG. The electronic control device 4 receives radar information from the radar device 2 (step S101). The radar information is information held by a candidate object to be detected (hereinafter referred to as “candidate object”), and is information on distance between the vehicle 1 and an object outside the vehicle, speed information, angle information, and information on reflection intensity. .

  Based on the distance information between the vehicle and the object included in the radar information, the electronic control unit 4 prioritizes the processes in order of increasing distance from the vehicle 1 to a plurality of candidate objects (step S102). Since it is considered that the possibility of a collision with the vehicle 1 is high if the distance to the vehicle 1 is short, an early warning is given to an occupant such as a driver of the vehicle 1 or the vehicle 1 is intervened. This is because such vehicle control needs to be performed quickly. The prioritization according to the present embodiment is an example in which the priority of an object with a relatively short distance is set high. However, for example, the priority is set in descending order of relative speed as an element of prioritization. The priority order of candidate objects that are relatively close to the vehicle 1 and have a large relative speed may be increased.

  Next, the electronic control unit 4 predicts the type of the object from candidate objects having a high processing priority (step S103). Specifically, it predicts the type of object from the radar information that each candidate object has. For example, it has radar information with a relatively close distance, a relatively large relative velocity, and a relatively strong reflection intensity. The candidate object is predicted to be a vehicle. Since the vehicle rebounds the transmission wave of the radar device 2 at a plurality of locations in the vehicle body, when there are a plurality of radar information having almost the same distance, relative speed, and reflection intensity in the radar information, Radar information is treated as a plurality of reflection points that constitute the same object, and the type of object is predicted as a larger object as the number of reflection points forming one object increases.

  Even if the distance is short, the relative velocity is relatively small, the reflection intensity is weak, and if the number of reflection points is small, the person is predicted. When the reflection intensity is strong and the absolute speed is zero, if the number of reflection points is small, it is predicted that the object is a stationary object on the road (for example, a manhole or a guardrail).

  Based on the radar information of the candidate object and the information on the type of the object, the electronic control unit 4 calculates which coordinate in the one-frame image in the image processing device 3 the corresponding candidate object corresponds to (step). S104). Specifically, the coordinate position in the image of one frame photographed by the image processing device 3 is associated with the candidate object based on the information on the distance and angle of the radar information of the candidate object having a high processing priority. In addition, a predetermined range in one frame image based on the coordinate position is calculated according to the predicted object type.

  The electronic control unit 4 transmits image coordinate information corresponding to the predicted object type to the image processing unit 3 (step S105). The image processing device 3 that has received the coordinate information of the image range cuts out an image range of a size corresponding to the type of object of one frame captured using the camera 13 and transmits it to the electronic control device 4. For example, if the predicted object type is a pedestrian, a relatively small vertically long region is cut out. If the predicted object type is a vehicle, a relatively large horizontally long region is cut out. In addition to the objects such as people and vehicles described so far, the object 24 can be stored in the memory 24 according to various object types such as two-wheeled vehicles, utility poles, pedestrian bridges, and iron bridges. It is possible to select and cut out regions according to the sizes of these different types of objects.

  The electronic control device 4 receives the image information transmitted by the image processing device 3 as a processing target area (step S106), and performs object type determination only on the received image information (step S107). The object type is determined by scanning the image, detecting a range where the luminance of the target image is changed, and determining the object type. As the object type determination method, a determination method according to the predicted type of the object corresponding to the processing target region is used. For example, there is a method of determining the type of an object by taking the center of a range with a change and observing the symmetry of the range of the change. Further, there is a method of scanning in a predetermined direction, changing the scanning direction from a point where there is a change to a direction intersecting, and determining the type of the object based on the luminance change range. Furthermore, there is a method of determining the type of an object by applying a template stored in advance in the memory 24 to an image range in which the luminance changes.

  In addition, the S / N ratio of the signal detected by the image sensor may deteriorate due to changes in temperature and humidity, and it may be difficult to detect weak light luminance. Therefore, it is possible to set a threshold value for detecting the brightness of an image based on changes in temperature and humidity to a predetermined value.

  As a result of the object type determination, if the object is an object for vehicle control (step S108: Yes), it is determined whether or not notification by the alarm from the alarm device 17 is performed (step S109). If the object is not a vehicle control target, the process ends (No in step S108).

  When notification by an alarm is necessary (step S109 is Yes), an alarm is output from the alarm device 17 (step S110). As for the alarm, an audio warning is given from a navigation device provided in the passenger compartment, or a video warning is given.

  After outputting the alarm or when notification by the alarm is not necessary (No in Step S109), it is determined whether or not driving intervention is necessary (Step S111). When driving intervention is necessary (Yes in step S111), driving intervention is performed by automatically operating the brake 27 and the steering wheel 37 (step S112). As an example of driving intervention, when there is a possibility that the vehicle 1 collides with an object outside the vehicle, the brake 27 is automatically operated to stop the vehicle, or the steering 37 is automatically operated in a direction in which no object exists. There is something to operate. Further, by fixing the seat belt 5 or the headrest 6 before the collision, it is possible to ensure the safety of a passenger such as a driver of the vehicle 1 in preparation for the collision.

  If no driving intervention is required (No at step S111), the process is terminated. For example, a collision with an object outside the vehicle can be avoided by a driver's operation.

  As described above, in the object detection system 10 according to the present embodiment, the electronic control unit 4 derives the coordinates in the image of the candidate object to be detected based on the radar information, and the type of candidate object based on the radar information. Predict. Then, the electronic control unit 4 designates a region having a size corresponding to the predicted type of the candidate object in the vicinity of the coordinates derived from the candidate object in the image as the processing target region, and the image processing device 3 for the image region. To cut out. An object detection process is performed only for the cut-out image region. Thereby, the image processing performed by the CPU 14 of the electronic control device 4 is limited to the image area where the object to be detected exists, and the image processing load for determining the type of the object is reduced. In addition, the use of the radar device places a load on itself, but the image processing load by limiting the image processing area is greatly reduced, so that the processing load of the entire object detection system can be reduced. . Furthermore, by reducing the processing load of the CPU 14 of the electronic control unit 4, vehicle control for an object outside the vehicle can be performed quickly.

  Further, the type of the object is predicted in advance based on the information from the radar device 2, and a determination method according to the type predicted in the type determination of the object by the CPU 14 of the electronic control device 4 is used. Therefore, it is not necessary to repeat the object type determination many times, and the object type determination can be performed quickly. Furthermore, the vehicle control of the object can be quickly performed by reducing the processing load of the CPU 14 of the electronic control unit 4.

  In addition, when there are a plurality of candidate objects, the processing objects are processed in order from the shortest distance to the vehicle among the plurality of candidate objects, so it is possible to detect from the object that is most likely to be the target of vehicle control, such as a driver. It is possible to ensure the safety of passengers quickly.

[Embodiment 2]
<2-1. Configuration>
FIG. 7 is a block diagram of an object detection system including a navigation device and various sensors. The main difference from the block diagram shown in FIG. 2 of the first embodiment is that the navigation device 8, the vehicle speed sensor 19, and the steering sensor 20. In addition, various sensors of the yaw rate sensor 21 are newly provided. Hereinafter, the operation content described in the first embodiment will be omitted, and different operation content will be described.

  The navigation device 8 includes a display unit 18 and a navigation control unit 28, and the display unit 18 notifies navigation information such as a map or the like to a driver or other occupant by video or voice, or audio such as music or digital TV. Provide content information. The navigation control unit performs control for providing such information to an occupant such as a driver.

  The vehicle speed sensor 19 transmits vehicle speed information to the electronic control unit 4. The steering sensor 29 detects the steering angle of the vehicle steering and transmits it to the electronic control unit 4. The yaw rate sensor 39 detects the change in the direction of the vehicle body of the vehicle 1 by the angular velocity and transmits it to the electronic control unit 4. The electronic control unit 4 detects candidate objects to be subjected to vehicle control in accordance with the traveling state of these vehicles.

  Next, for a specific example of candidate object detection performed using the navigation device 8 and various sensors, a diagram showing an object detection range by an image processing device including a radar device and a camera in the curve of FIG. 8, and FIG. A description will be given with reference to a diagram showing an image of object detection by an image processing apparatus equipped with a camera on a curve.

  In FIG. 8, the object detection range R1 of the radar device 2 provided in the vehicle 1 and the object detection ranges of C1 and C2 of the image processing device 3 provided with a camera overlap. A person 201 and a guard rail 202 are included as candidate objects in the object detection ranges of the radar apparatus 2 and the image processing apparatus 3. Here, the navigation device 4 provided in the vehicle 1 detects that the road in the traveling direction of the vehicle 1 is curved when traveling a predetermined distance from the current position of the vehicle 1. Here, the curve radius of the present embodiment is a curve of 130 m or less. However, the present invention is not limited to this, and the curve radius may be equal to or smaller than the curve radius value of the present embodiment if the effects described in the present embodiment are provided. .

  The radar information of the radar apparatus 2 has a predetermined distance such that the distance from the vehicle 1 is, for example, 20 m or more, the absolute velocity is zero, the angle is substantially in front of the traveling direction, and the reflection intensity is strong. If there is (a peak having a power exceeding the threshold for detecting the peak), this radar information is not selected as a candidate object for cutting out a predetermined range from the electronic control unit 4 to the image processing unit 3, or a candidate Do not prioritize processing as objects.

  A stationary object such as the card rail 202 just before the entrance of the curve in the own lane can be avoided by the driver's steering operation without being controlled by the vehicle 1. By performing automatic driving intervention, there is a risk of erroneous vehicle control such that the steering is operated more than the steering operation along the angle of the curve. Therefore, a stationary object such as a guard rail in the own lane detected before the vehicle 1 reaches the curve is not subject to vehicle control.

  As a condition for not selecting radar information of a stationary object having a zero absolute speed such as the guard rail 202 as a candidate object for vehicle control, the object is on its own lane according to the information of the navigation device 4 and the vehicle 1 It is also included that the steering sensor 29 or the yaw rate sensor 39 provided in the above indicates that the vehicle 1 is not traveling along a curve. Here, on the own lane, in the present embodiment, as an example, when the vehicle is traveling in a predetermined lane, the vehicle 1 exists within a distance of about 2.5 m from the white line to the white line at both ends in the traveling direction. However, the present invention is not limited to this and includes a range having the effects of the present embodiment.

  Further, when the vehicle 1 reaches the entrance of the curve and the driver operates the steering along the angle of the curve, the steering sensor 29 and the yaw rate sensor 39 are in the curve running state of the vehicle 1 according to the degree of turning of the vehicle. Indicates. When the vehicle 1 travels along the curve, the detection range R2 of the radar device 2 and the detection range C1 of the image processing device also move along the angle of the curve. As a result, if the candidate object is detected based on the radar information, prioritization is performed according to the distance to the vehicle 1 and information on the relative speed, and the image processing apparatus 3 cuts out the target range image as the candidate object, and the electronic If the type of the object is determined by the control device 4 and the object is a vehicle control object, it is possible to perform the vehicle control according to the state of the object.

  As a modification of the present embodiment, when the vehicle 1 acquires speed information of the vehicle 1 from the vehicle speed sensor 19 while traveling on the own lane, the object in the traveling direction of the vehicle 1 is determined from the relationship between the speed and the braking distance. If there is a possibility of collision with the object, it is possible to determine the type of the object by increasing the priority of the candidate object processing, and to perform quick vehicle control.

Specifically, when the road surface is frozen from the information of the navigation device 4, the relationship between the speed information of the vehicle 1 by the vehicle speed sensor 19 and the braking distance is calculated. In particular, the distance to the vehicle 1 is lower than the other candidate objects, such that the priority is lower on the dry road surface, including that the braking distance is extended two to three times or more on the frozen road surface than the dry road surface. The priority of processing of candidate objects of existing radar information is increased.
<2-2. Operation>
FIG. 10 is a process flowchart of the electronic control device using the travel information. The electronic control device 4 receives radar information from the radar device 2 (step S201).

  Next, the travel information of the vehicle 1 is received from various sensors such as the navigation device 4, the vehicle speed sensor 19, the steering sensor 29, or the yaw rate sensor 39 (step S202). Here, the traveling information includes the number of road lanes, road gradient, road surface temperature, temperature, humidity and the like in addition to the road curve and the frozen state obtained from the navigation device 4 as described above. Further, the traveling state of the vehicle 1 includes information such as the vehicle speed and the steering angle.

  The electronic control device 4 prioritizes the plurality of candidate objects based on the distance information included in the radar information together with the navigation device 4 and information on prioritization of the candidate objects. (Step S203).

  Specifically, as described above, when there is a curve at a predetermined distance on the traveling path on the own lane, radar information such as a guardrail on the own lane is not subject to the object type determination of the electronic control unit 4. As such, it is not selected as a candidate object. Alternatively, the priority of processing as a candidate object is set lower than other candidate objects.

  Also, if the road on the lane is frozen, the radar information that the distance from the vehicle is more than a predetermined distance compared to other candidate objects is also considered as a candidate object in consideration of the speed and braking distance of the vehicle As a result, the priority of processing is raised over other candidate objects.

  Next, the electronic control unit 4 predicts the type of the object from the candidate objects having a high processing priority (step S204), and the candidate object is imaged based on the radar information of the candidate object and the information of the object type. The coordinates corresponding to one frame image in the processing device 3 are calculated (step S205).

  The electronic control unit 4 transmits the coordinate information of the image range corresponding to the predicted object type to the image processing device 3 (step S206). The image processing device 3 that has received the information of the coordinates of the image range cuts out an image range corresponding to the type of object of one frame imaged using the camera 13 and transmits the image range to the electronic control device 4.

  The electronic control device 4 receives the image information transmitted by the image processing device 3 (step S207), and performs object type determination (step S208).

  As a result of the object type determination, if the object is an object for vehicle control (step S209 is Yes), it is determined whether or not notification by the alarm from the alarm device 17 is performed (step S210). If the object is not a target for vehicle control (No in step S209), the process ends.

  When notification by an alarm is necessary (step S210 is Yes), an alarm is output from the alarm device 17 (step S211). The alarm outputs a warning by voice from a vehicle interior navigation device (not shown) or gives a warning by video.

  After outputting the alarm or when notification by the alarm is not necessary (No in Step S210), it is determined whether or not driving intervention is necessary (Step S212). If driving intervention is necessary (Yes in step S212), the driving intervention is performed by automatically operating the brake 27 and the steering wheel 37 (step S213).

  If driving intervention is not necessary (No in step S212), the process ends. For example, a collision with an object outside the vehicle can be avoided by a driver's operation.

  As described above, in the present embodiment, the traveling information indicating the traveling state of the vehicle obtained from the navigation device 8 or various sensors is used together with the information of the radar device, so that the vehicle 1 travels. A processing target area where an object to be detected exists can be selected. As a result, the processing load on the CPU 14 of the electronic control unit 4 is reduced, and vehicle control according to the traveling state of the vehicle 1 can be performed quickly.

  Further, it is possible to detect in order from an object that is likely to be a target of vehicle control, and it is possible to quickly ensure the safety of a vehicle occupant such as a driver.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 2 ... Radar device 3 ... Image processing device 5 ... Seat belt 6 ... Headrest 12 ... Antenna 13 ... Camera 14 ... CPU
17 .... Alarm 22 ... Radar control unit 23 ... Camera control unit 27 ... Brake 37 ... Steering

Claims (6)

Based on radar information from a radar device that detects the position and speed of an object outside the vehicle provided in a vehicle, and an image obtained by a camera that images a region that is provided in the vehicle and overlaps the detection range of the radar device. An object detection device for detecting a detection target outside the vehicle,
Derivation means for deriving coordinates in the image of the candidate object to be detected based on the radar information;
Prediction means for predicting the type of the candidate object based on the radar information;
Setting means for setting a region around the derived coordinates of the candidate object in the image and having a size corresponding to the predicted type of the candidate object as a processing target region;
Detecting means for detecting the detection target only for the processing target region in the image;
An object detection apparatus comprising: The object detection device according to claim 1,
The detection unit uses a determination method according to a predicted type of the candidate object corresponding to the processing target region when detecting the detection target from the processing target region. The object detection apparatus according to claim 1 or 2,
Obtaining means for obtaining traveling information indicating a traveling state of the vehicle;
Means for selecting a part of the candidate object based on the radar information as a processing target based on the traveling information;
An object detection device further comprising: In the object detection device according to any one of claims 1 to 3,
At least one of the derivation means, the prediction means, the setting means, and the detection means is:
When there are a plurality of candidate objects based on the radar information, the object detection device is set as a processing target in order of a short distance from the vehicle among the plurality of candidate objects. Radar information from a radar device that detects the position and speed of an object outside the vehicle provided in the vehicle, and an image obtained by a camera that images a region provided in the vehicle and overlapping the detection range of the radar device. Based on the object detection method for detecting a detection target outside the vehicle,
Deriving coordinates in the image of candidate objects with the detection target based on the radar information;
Predicting the type of the candidate object based on the radar information;
Setting a region around the derived coordinates of the candidate object in the image with a size according to the predicted type of the candidate object as a processing target region;
Detecting the detection target for only the processing target region in the image;
An object detection method characterized by comprising: An object detection system,
A radar device provided in a vehicle for detecting the position and speed of an object outside the vehicle;
A camera that is provided in the vehicle and images a region that overlaps a detection range of the radar device;
The object detection device according to any one of claims 1 to 4, wherein a detection target outside the vehicle is detected based on radar information from a radar device and an image obtained by a camera.
An object detection system comprising:

Images