JP4289210B2 - Vehicle object detection device - Google Patents

Description

  The present invention relates to a vehicle object detection device capable of detecting an object such as an obstacle around a vehicle.

  2. Description of the Related Art Conventionally, for example, an in-vehicle radar device (ranging sensor: radar) that detects a distance from an obstacle using radio waves is known as a vehicle object detection device that detects an obstacle ahead of the vehicle. . Various methods have been proposed in order to increase the detection accuracy of this type of device.

  In particular, by combining an image processing device (image sensor: camera) using a camera with the radar, the type of object (target) that cannot be detected by the radar alone, the exact center position, and the width are calculated, and the detection accuracy is increased. A method has been proposed for enhancing the above.

The image processing performed by this camera has a high processing load, and it is not realistic in terms of cost to perform all target detection by image processing, so the image to be processed using the position of the target detected by the radar A method for limiting the processing range is also proposed (see Patent Document 1).
Japanese Patent No. 3123303 (page 7, FIG. 13)

  However, an apparatus for detecting an object such as a radar or a camera generally does not detect a target by relying on a detection result at a certain moment in order to maintain its detection performance. It is necessary to determine whether or not the movement is reliable in time and detect the target. Therefore, this target detection process requires a certain amount of time.

  For example, as shown in FIG. 16A, in the case where the result detected by the radar is further processed by the camera to detect the attribute, position, width, etc. of the target, each processing of the radar and the camera takes time. Is required. That is, there is a problem that it takes a lot of time to accurately detect (determine) a target.

  In order to shorten the detection time, as shown in FIG. 16B, when the radar detects the target, the target detection result may be temporarily output regardless of whether the detection by the camera is completed. Conceivable.

However, in this case, there is a possibility of erroneous detection that the target exists even though the target does not actually exist.
FIG. 16C is an explanatory diagram for explaining symbols in FIGS. 16A and 16B.

  The present invention solves the above-described problems, and an object of the present invention is to provide an object detection apparatus for a vehicle that can make a highly accurate determination in a short time and reliably detect an object.

  (1) The invention of claim 1 includes information from a radar device that irradiates a detection wave to the surroundings and detects an object by the reflected wave, information from a camera device that detects the object from an image captured by a camera, In the vehicle object detection device for determining the object based on the information, the radar device determines the object (based on the radar device) based on predetermined uncertain information about the object obtained by the radar device. The gist of the present invention is an object detection device for a vehicle, which has a configuration for confirming and transmits uncertain information on the object obtained by the radar device to the camera device.

  In the present invention, the information that is determined to be an object by the radar device is not transmitted to the camera device as in the prior art, but the uncertain information is transmitted to the camera device. For example, by preferentially checking an image area corresponding to an undetermined object based on this information, it is possible to quickly determine whether the undetermined object is an object instead of noise or the like.

Accordingly, it is possible to quickly determine the object in the camera device, the object in the radar device, and finally the final object based on the both determination results.
(2) The invention of claim 2 is characterized in that the undetermined information of the object obtained by the radar device is sequentially transmitted to the camera device every time undetermined information of the object is obtained. The gist of the vehicle object detection device according to claim 1 is described.

In the present invention, the undetermined information of the object obtained by the radar device is sequentially transmitted to the camera device, so that the camera device can quickly perform necessary processing.
(3) According to the invention of claim 3, in the camera device, the undetermined object is actually an object from an image captured by the camera based on the undetermined information of the object transmitted from the radar device. The gist of the vehicle object detection device according to claim 1 or 2 is characterized in that it is determined whether or not.

  The present invention exemplifies processing on the camera device side. Here, based on the undetermined information of the object transmitted from the radar device, the undetermined object is actually the object from the image corresponding to the undetermined object (for example, the image of the predetermined area corresponding to the position of the object). It can be judged efficiently whether it is.

(4) The gist of the invention of claim 4 is that the vehicle object detection device according to claim 3 is characterized in that a determination result in the camera device is transmitted to the radar device.
In the present invention, the determination result as to whether or not the undetermined object is actually an object is transmitted to the radar apparatus. Therefore, the radar apparatus uses the determination result to check the accuracy of information on the undetermined object. It can be performed. It is also possible to quickly determine an object in the radar apparatus.

  (5) The invention according to claim 5 is characterized in that, in the radar apparatus, a condition for determining an undetermined object is changed based on a determination result transmitted from the camera apparatus. The vehicle object detection device described in 1 is used as a gist.

  In the present invention, since the condition for determining an undetermined object is changed based on the determination result transmitted from the camera device, an optimal determination can be made according to the actual object detection status.

  For example, when the determination result is “object”, there is a high possibility that the object is actually an object, so the number of determinations for determining an object in the radar apparatus can be reduced. As a result, the time for determining the object can be shortened without reducing the accuracy of determining the object.

  Also, for example, when the determination result is “not an object”, the number of determinations for determining the object in the radar apparatus is increased or the initial setting value is not changed, thereby determining the object. Accuracy can be maintained.

  (6) In the invention of claim 6, when changing a condition for determining the undetermined object, when using a counter that is incremented every detection of each undetermined object, The gist of the vehicle object detection device according to claim 5, wherein means for changing a threshold value serving as a determination reference or means for changing a counter value is used.

  For example, in the case where the threshold value for determining an object is set to 5 and the counter value is incremented by 1 every time an undefined object is detected, the threshold value is set to 1 when the camera device determines that the object is an object. Reduce and set to 4. As a result, the counter reaches the threshold value early, so that the object can be determined early. Alternatively, the same effect can be obtained by changing the increasing counter value without changing the threshold value.

(7) In the invention of claim 7, the information of the object obtained by the radar device is handled as undetermined information, and is determined to be an object by the determination by the camera device based on the undetermined information. The gist of the vehicle object detection device according to claim 1, wherein a threshold value for determining object information in the radar device is reduced.
As a result, the number of cycles (number of cycles) for determining objects for several cycles in the radar device can be reduced, so that the time for determining objects can be reduced without degrading the accuracy of object determination. Can do.
( 8 ) In the invention of claim 8 , even if it is determined that the object is an object by the radar apparatus, if the camera apparatus does not determine that the object is an object, the object determination in the radar apparatus is determined. The gist of the object detection device for a vehicle according to any one of claims 4 to 7 , wherein the object is cancelled.

  In general, since it is considered that the accuracy of recognition of an object of a radar device is lower than that of a camera device, in the present invention, even if it is determined that the object is an object by the radar device, the object is not determined by the camera device. Sometimes, the determination of object determination in the radar apparatus is canceled. As a result, the possibility of erroneous detection of noise or the like as an object is reduced, so that an object with higher accuracy can be determined.

( 9 ) According to the ninth aspect of the present invention, when extrapolation processing is performed on the assumption that the radar device has detected this time with respect to an object detected in the past but not detected this time, The gist of the vehicular object detection device according to any one of claims 4 to 8 , wherein an indeterminate object extrapolated by the radar device is confirmed based on a determination result.

  Since the information of the undetermined object subjected to so-called extrapolation processing is virtual information, it is confirmed by the camera device whether or not the object actually exists. Thereby, the accuracy of the determination of the object can be increased.

  For example, when the number of extrapolation processes is counted (for example, with an extrapolation counter), when the camera apparatus determines that the data is an object, the extrapolation counter is reset so that the radar apparatus extrapolates. By limiting the number of times, it is possible to prevent the object from being lost.

( 10 ) In the invention of claim 10, when the extrapolated undetermined object is confirmed based on the determination result by the camera device , information on the extrapolated undetermined object is stored in the outer area. and gist a vehicle object detecting apparatus according to claim 9, characterized in that said radar apparatus thus obtained undetermined object information rather than interpolation process information.

If the extrapolation process is performed many times, the reliability of the information is reduced. However, once the camera apparatus determines that the information is an object, the reliability is increased. Therefore, in this case, the information that has been confirmed is not the information that has been extrapolation is treat as actual information detected by the radar apparatus (information-doubt object).

(11) The invention of claim 11, the information that has been determined to be an object in the radar device, based on the information that has been determined to be an object in the camera device, to determine the overall object The gist of the vehicle object detection device according to any one of claims 1 to 10 is characterized.

  In the present invention, since the object is comprehensively determined (confirmed by fusion) based on the confirmed information in the radar device and the confirmed information in the camera device, there is an effect that the accuracy of determining the object is high.

  Next, an example (example) of the best mode of the present invention will be described.

a) First, the system configuration of a vehicle equipped with the vehicle object detection device of this embodiment will be described with reference to FIG.
As shown in FIG. 1, a vehicle (automobile) includes an FMCW radar device 1 that detects an object (target) such as an obstacle in front of the vehicle using an FMCW radar, and a camera that detects a target from an image captured by the camera. A vehicle object detection device 5 including the device 3 is mounted.

  As is well known, the radar apparatus 1 includes a transmitting antenna 7 that transmits a radar wave (millimeter wave) that is a frequency-modulated continuous wave, and a receiving antenna 9 that receives a reflected wave reflected by the transmitted radar wave when it hits an object. A mixer 11, a filter 13, an A / D converter 15, and a radar signal processing unit 17 whose main part is a microcomputer.

Further, as is well known, the camera device 3 includes a CCD camera 19, an A / D converter 21, and an image processing unit 23 whose main part is a microcomputer.
Then, as will be described in detail later, each time an uncertain target is detected by the radar apparatus 1, the data of the uncertain target is sequentially transmitted from the radar signal processing unit 17 to the image processing unit 23. From the image processing unit 23 to the radar signal processing unit 17, data obtained as a result of checking the presence / absence of the target is transmitted by the camera device 3 (based on the data of the undetermined target). Then, the radar signal processing unit 17 determines the target based on the data transmitted from the camera device 3.

  The radar signal processing unit 17 and the image processing unit 23 can communicate with each other via a LAN or the like, and the target data determined by the radar signal processing unit 17 and the image processing unit 23 are determined. Based on the target data, the finally determined target information is output from the radar signal processing unit 17 to the outside (for example, a vehicle control device or the like).

The radar signal processing unit 17 and the image processing unit 23 determine (determine) that the target is a target when the target is detected a plurality of times.
b) Next, data processed by the radar device 1 and the camera device 3 will be described.

  As shown in FIG. 2, the radar apparatus 1 indicates a target ID for distinguishing each radar target candidate, a vertical position indicating the forward distance of the target candidate (see FIG. 3), and a horizontal distance of the target candidate. Horizontal position (see FIG. 3), speed indicating the speed of the target candidate, width value indicating the width of the target candidate, detection value indicating the number of points for new detection, recognition flag indicating whether or not the target candidate is confirmed, Extrapolation flag indicating whether the target candidate is extrapolated, extrapolation counter indicating the number of extrapolation, history connection flag indicating identity with the target candidate before the cycle, whether to output to the outside The external output flag indicating (processing such as setting and storage) is performed.

The extrapolation is a process of extrapolating and assuming that a target candidate is detected when a target candidate detected one cycle before is not detected in a subsequent cycle.
As shown in FIG. 4, the camera device 3 has a target ID for distinguishing each camera target candidate, an image range indicating a range of an area considered to be an object, a target candidate (from the own vehicle). ) Horizontal position indicating the distance in the horizontal direction, width value indicating the width of the target candidate, attribute value indicating whether it is solid or non-solid, detection value indicating the number of points for new detection, whether detection has been confirmed in the camera The confirmation flag indicating (processing such as setting and storage) is performed.

  The target candidate is a target that has been detected by the radar device 1 or the camera device 3, that is, target data that may be determined as a target (thus, data of an undefined target). That is.

  Then, as shown in FIG. 5A, communication is performed from the radar apparatus 1 to the camera apparatus 3 in a predetermined communication data format A. That is, the target ID, the vertical position, the horizontal position, and the speed are transmitted from the radar apparatus 1 to the camera apparatus 3.

  Further, as shown in FIG. 5B, communication is performed from the camera device 3 to the radar device 1 with a predetermined communication data format B. That is, the target ID, the attribute value, the confirmation flag, the lateral position, and the width value are transmitted from the camera device 3 to the radar device 1.

c) Next, processing performed by the radar apparatus 1 will be described.
(1) First, the main routine showing the main operation procedure will be described.
As shown in the flowchart of FIG. 6, in step 100, a transmission wave is output from the transmission antenna 7 while performing frequency modulation.

In step 110, the beat signal received by the receiving antenna 9 is taken into a memory (not shown) while performing AD conversion.
In step 120, the modulation wave output is stopped at a predetermined time, and the process is started from the data fetched in the memory.

In step 130, a frequency analysis is performed using the values taken into the memory. In practice, frequency analysis is performed using FFT to create the frequency spectrum.
In step 140, a frequency that is considered to be reflection of the target is extracted from the frequency spectrum. In this case, the maximum point (peak) of the frequency spectrum above a certain value is identified as reflection from the target, and this peak is extracted.

  In step 150, the distance and azimuth | direction of what seems to be a target are calculated from the extracted peak information, and a target candidate is produced. Here, since the FMCW method is used, the distance and the relative speed are calculated by combining the peaks of the frequency rising section and the falling section. Also, the orientation of the target is calculated at the same time. Any method can be used for calculating the direction, but actually, a method called beam forming processing using an array antenna is used. If the distance and direction are aligned, the vertical and horizontal positions of the target candidate are determined, and the speed is determined by the relative speed.

In step 160, the calculation result by the camera apparatus 3 (described later) regarding the target candidate data transmitted one control cycle (one cycle) is received.
In step 170, an output target data creation process described later is performed (see FIG. 7). This process combines the reception result from the camera device 3 with the target data detected by the radar device 1 calculated up to the target extrapolation processing (step 250) one cycle before the final output. This is a process for creating target data.

In step 180, the final output target data is transmitted to the outside. That is, the data finally determined as the target is output.
In step 190, target detection value adjustment processing described later is performed (see FIG. 8). This process is a process of raising and lowering the target detection threshold for detecting a target based on the calculation result of the target candidate obtained from the camera device 3 and looking at the temporal accuracy. That is, when the value (detection value) of whether or not the target is likely in terms of time reaches the threshold value, the target candidate is determined as the target. Based on the result, the target detection performance can be changed.

  In step 200, an extrapolation data update process before one cycle, which will be described later, is performed (see FIG. 9). This processing is performed for target candidates obtained from the camera device 3 with respect to targets that have already reached detection and for which the radar device 1 cannot detect one cycle before and has extrapolated predicted values. The calculation result is applied to rewrite the extrapolated position in the prediction and clear the extrapolation number limit counter (extrapolation counter) to change it as if it was detected.

  Even if the radar apparatus 1 cannot be detected, if the target is detected at a probable location by the camera apparatus 3, the camera apparatus 3 is used to complement the target as if it was detected as the radar apparatus 1. To do. As a result, the radar detection result of the previous cycle becomes accurate, and the accuracy in the next history connection process increases.

  In step 210, a history connection process described later is performed (see FIG. 10). This process is a process of comparing the target candidates one cycle before or the target being detected with the target candidates of the current cycle, and linking the ones that are likely to work in terms of time. Target IDs are assigned to the items linked here in this processing.

  In step 220, history ID assignment processing is performed. This process is a process of assigning a target ID to a target candidate for which a target ID has not been assigned in the previous history connection process. Here, the target ID to be assigned is assigned a number other than the number assigned to the target candidate one cycle before the current cycle. Thereby, on the camera device 3 side, it can be confirmed whether there is temporal continuity according to the received target ID value.

  In step 230, processing for transmitting the target candidate to the camera device 3 is performed. That is, the “(radar → camera) transmission data format A” shown in FIG. 5A is created from all the target candidate data obtained in this cycle, and the data is sent from the radar device 1 to the camera device 3 side. Send to.

  In step 240, a target recognition process described later is performed (see FIG. 11). This process is a process of registering a target candidate as a target, assuming that the detected value calculated in the previous history connection process is equal to or greater than a certain threshold value, and that the target is likely to move. is there. As a result, the target is determined.

  In step 250, a target extrapolation process, which will be described later, is performed (see FIG. 12), and this process is temporarily terminated. This process calculates a predicted value from the current position and speed when no new target candidate is connected in the history connection process of the current cycle. This is a process of extrapolating a target. This processing is performed because it is considered that the radar device 1 happens to lose sight of the target for a moment due to the influence of noise or the like.

(2) Next, the output target data creation process in step 170 will be described.
This process is a process of creating a final output by combining the target reached by the radar apparatus 1 and the reception result received from the camera apparatus 3.

  Here, the target detected by the radar apparatus 1 is expressed by a variable called a recognition flag, and the arbitration between the output result from the camera apparatus 3 and the target detected by the radar apparatus 1 is an ID number. The point is to process using. This will be specifically described below.

  As shown in the flowchart of FIG. 7, in step 300, it is determined whether or not the number of target candidates by the radar apparatus 1 one cycle before is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 310.

In step 310, the target candidate Ai by the radar apparatus 1 one cycle before is extracted.
In the following step 320, it is determined whether or not the recognition flag of Ai is 1 (that is, whether or not the target is fixed). If an affirmative determination is made here, the process proceeds to step 330, while if a negative determination is made, the process returns to step 310.

In step 330, the detection result Bj obtained by the camera device 3 corresponding to the target IDrid of Ai is taken out.
In the subsequent step 340, it is determined whether or not the Bj object-marked result (attribute value) cr is a solid (= 1) and the confirmation flag cd is 1. If an affirmative determination is made here, the process proceeds to step 350, whereas if a negative determination is made, the process proceeds to step 370.

In step 350, the center value and width value of Ai are rewritten. That is, the horizontal position crx of Bj is substituted for the horizontal position rrx of Ai, and the width value of Bj is substituted for the width value rw of Ai.
In the following step 360, the external output flag rout of Ai is set to 1.

  In the subsequent step 370, it is determined whether or not all target candidates by the radar apparatus 1 in the previous cycle have been searched. If an affirmative determination is made here, the present process is temporarily terminated. If a negative determination is made, the process returns to step 310, and the same process is repeated until the search for all target candidates is completed.

(3) Next, the target detection value adjustment processing in step 190 will be described.
This process is a process for raising and lowering the target detection value based on the calculation result of the target candidate obtained from the camera device 3.

  Here, since the result of the target candidate one cycle before is returned, the detection value of the target ID corresponding to the result is increased or decreased. In the case of a target that has already been detected, the threshold value is not increased or decreased. This will be specifically described below.

  As shown in the flowchart of FIG. 8, in step 400, it is determined whether or not the number of target candidates obtained from the calculation result of the camera device 3 is zero. If an affirmative determination is made here, the present process is temporarily terminated. If a negative determination is made, the process proceeds to step 410.

In step 410, a target candidate Bj that is a calculation result in the camera device 3 is extracted.
In the following step 420, the target candidate Ai by the radar apparatus 1 in the previous cycle corresponding to the target ID cid of Bj is extracted.

In the following step 430, it is determined whether or not the recognition flag rd in Ai is 0. If an affirmative determination is made here, the process proceeds to step 440, whereas if a negative determination is made, the present process is temporarily terminated.
In step 440, it is determined whether or not the object identification result (attribute value) cr of Bj is a solid. If an affirmative determination is made here, the process proceeds to step 460, whereas if a negative determination is made, the process proceeds to step 450.

In step 450, it is determined whether or not the confirmation flag cf of Bj is 1. If an affirmative determination is made here, the process proceeds to step 470, while if a negative determination is made, the process proceeds to step 480.
In step 470, the detection value rs of Ai is set to “rs + SP1”, and the process proceeds to step 510.

On the other hand, in step 470, the detection value rs of Ai is set to “rs + SP2”, and the process proceeds to step 510.
In step 460, where the determination is affirmative in step 440, the process proceeds to step 460, in which it is determined whether the determination flag cf for Bj is 1. If an affirmative determination is made here, the process proceeds to step 500, whereas if a negative determination is made, the process proceeds to step 490.

In step 490, the detection value rs of Ai is set to “rs + SP3”, and the process proceeds to step 510.
On the other hand, in step 500, the detection value rs of Ai is set to “rs + SP4”, and the process proceeds to step 510.

SP1 is a value that greatly decreases the detection value, SP2 is a value that decreases the detection value, SP3 is a value that increases the detection value, and SP4 is a value that greatly increases the detection value.
In step 510, it is determined whether or not all the calculation results in the camera device 3 have been used. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 410 to repeat the same process.

(4) Next, the extrapolated data update process before one cycle in step 200 will be described.
This processing is performed for target candidates obtained from the camera device 3 with respect to targets that have already reached detection and for which the radar device 1 cannot detect one cycle before and has extrapolated predicted values. This is a process of applying the calculation result and rewriting the position extrapolated in the prediction.

  The reason for performing this process is that the value extrapolated by the radar apparatus 1 is a value estimated by the radar apparatus 1 and not an actual value. In other words, if the camera device 3 can calculate an actual value, the position can be detected more accurately by using the value.

  In this example, since the output of the camera device 3 is received in the next cycle, it is possible to accurately perform the history connection in the history connection process as the next process by rewriting the radar detection extrapolation result of the previous cycle. it can.

  The point in the process is a part that resets an extrapolation counter that limits the number of extrapolations when it can be rewritten well. As a result, even if the situation in which the radar apparatus 1 is not good continues for a long time, as long as the camera apparatus 3 is able to detect, the detection of the target as a whole is not interrupted, and the target is really not in the detection range. If this happens, extrapolation can be stopped immediately. That is, it is possible to achieve both continuous target detection and appropriate extrapolation stop. This will be specifically described below.

  As shown in the flowchart of FIG. 9, in step 600, it is determined whether or not the number of target candidates obtained from the calculation result of the camera device 3 is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 610.

In step 610, a target candidate Bj which is a calculation result in the camera device 3 is extracted.
In the following step 620, the target candidate Ai by the radar apparatus 1 one cycle before corresponding to the target ID cid of Bj is extracted.

  In the subsequent step 630, it is determined whether the recognition flag rd in Ai is 1 and the extrapolation flag rx is 1. If an affirmative determination is made here, the process proceeds to step 640, whereas if a negative determination is made, the process proceeds to step 680.

  In step 640, it is determined whether or not the object identification result (attribute value) cr of Bj is a solid. If an affirmative determination is made here, the process proceeds to step 650, whereas if a negative determination is made, the process proceeds to step 680.

In step 650, the center position crx of Bj is substituted for the center position rrx of Ai.
In the following step 660, it is determined whether or not the confirmation flag cf of Bj is 1. If an affirmative determination is made here, the process proceeds to step 670, whereas if a negative determination is made, the process proceeds to step 680.

In step 670, the Ai extrapolation counter rxc is set to 0, the Ai width value rw is set to the Bj width value cw, and the process proceeds to step 680.
In step 680, it is determined whether or not all the calculation results in the camera device 3 have been used. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 610 to repeat similar processing.

(5) Next, the history connection process in step 210 will be described.
This process is a process of comparing the target candidates of the previous cycle with the target candidates of the current cycle and linking those that are likely to move in time.

  That is, the targets of the previous cycle and the current cycle are combined, and the position difference and relative speed difference between the position predicted for the current cycle and the target candidate for the current cycle are determined to be within a certain range. When there are a plurality of items that fall within this range, only one with the best evaluation value is selected.

  If there is such a target connection, it is called a history connection. In this case, the target data of the previous cycle is copied and the value is taken over. In particular, the detected value is inherited and further added. And when this detection value exceeds a certain threshold value, it is a mechanism detected as a target.

  Further, the target candidate one cycle before the history connection is set to 1 in the history connection flag in order to store that there is a connection. This value is used when extrapolating the target. This will be specifically described below.

  As shown in the flowchart of FIG. 10, in step 700, it is determined whether or not the number of target candidates by the radar apparatus 1 in the current cycle is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 710.

  In step 710, a target candidate Ai by the radar apparatus 1 in the current cycle is extracted. Then, DRTSD (history connection threshold in distance) is set in drmin, and DVTSD (history connection threshold in speed) is set in dvmin.

  In the following step 720, it is determined whether or not the number of target candidates by the radar apparatus 1 one cycle before is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 730.

In step 730, the target candidate Ck by the radar apparatus 1 one cycle before is extracted.
In the subsequent step 740, the predicted position and predicted speed of the current cycle of Ck are calculated.
In the following step 750, the difference (position difference dr, speed difference dv) between the predicted position and predicted speed of the current cycle of Ck and the position and speed of Ai is calculated.

  In the following step 760, it is determined whether or not the position difference dr <DRTSD and the speed difference dv <DVTSD. If an affirmative determination is made here, the process proceeds to step 770, whereas if a negative determination is made, the process proceeds to step 790.

  In step 770, it is determined whether or not position difference dr × speed difference dv <drmin × dvmin. If an affirmative determination is made here, the process proceeds to step 780, while if a negative determination is made, the process proceeds to step 790.

In step 780, the connection target candidate number min is set to k, drmin is set to dr, dvmin is set to dv, and the process proceeds to step 790.
In step 790, it is determined whether all the target candidates by the radar apparatus 1 one cycle before have been searched. If an affirmative determination is made here, the process proceeds to step 800, whereas if a negative determination is made, the process returns to step 730.

In step 800, it is determined whether drmin is DRTSD. If an affirmative determination is made here, the process proceeds to step 810, whereas if a negative determination is made, the process proceeds to 830.
In step 810, “Cmin detection value rs + 6PS” is set as the detection value rs of the target candidate Si by the radar apparatus 1.

  Note that PS is a detection value addition point by history connection, and Cmin is the most temporal among candidate candidates by the radar apparatus 1 one cycle before the target candidate Ai detected in the current cycle. It seems to have a strong connection (good evaluation value).

Also, the Cmin width value rw is set as the width value rw, the Cmin target IDrid is set as the target IDrid, and the Cmin recognition flag rd is set as the recognition flag rd.
In the subsequent step 820, the history connection flag rc of the target candidate Cmin by the radar apparatus 1 one cycle before is set.

  In the subsequent step 830, it is determined whether or not all the target candidates have been searched by the radar apparatus 1 one cycle before. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 710 to repeat the same process.

(6) Next, the target recognition process in step 240 will be described.
This process registers target candidates that have been detected in the current cycle and that have already been detected in connection with a target that has already been detected, or target candidates whose detection value has newly exceeded the threshold. It is processing to do. Whether or not it is a target is determined by substituting 1 for the recognition flag. This will be specifically described below.

  As shown in the flowchart of FIG. 11, in step 900, it is determined whether or not the number of target candidates by the radar apparatus 1 in the current cycle is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 910.

In step 910, a target candidate Ai by the radar apparatus 1 in the current cycle is extracted.
In the following step 920, it is determined whether the recognition flag rd of Ai is 1. If an affirmative determination is made here, the process proceeds to step 940, while if a negative determination is made, the process proceeds to step 930.

  In step 930, it is determined whether or not the detection value rs of Ai exceeds DETTSD (a threshold value for setting a detection state when the detection value exceeds this value). If an affirmative determination is made here, the process proceeds to step 940, while if a negative determination is made, the process proceeds to step 950.

  In the subsequent step 950, it is determined whether or not all target candidates have been searched. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 910 to repeat the same process.

(7) Next, the target extrapolation process in step 250 will be described.
In this process, if no new target candidate is connected in the history connection process of the current cycle, the target candidate is calculated by calculating the predicted value from the current position and speed. Is newly created and extrapolated.

  Here, an extrapolation counter is used to limit the number of extrapolations, and extrapolation cannot be performed continuously more than a certain number of times. This extrapolation counter decrements the value by 1 when there is extrapolation, and when it becomes 0, the next extrapolation cannot be performed. This will be specifically described below.

  As shown in the flowchart of FIG. 12, in step 1000, it is determined whether or not the number of target candidates by the radar apparatus 1 one cycle before is zero. If an affirmative determination is made here, the present process is temporarily terminated, whereas if a negative determination is made, the process proceeds to step 1010.

In step 1010, the target candidate Ck by the radar apparatus 1 one cycle before is extracted.
In the following step 1020, it is determined whether or not the Ck recognition flag rd is 1. If an affirmative determination is made here, the process proceeds to step 1030, while if a negative determination is made, the process proceeds to step 1090.

In step 1030, it is determined whether the history connection flag rxc of Ck is 1. If an affirmative determination is made here, the process proceeds to step 1070, while if a negative determination is made, the process proceeds to step 1040.
In step 1070, the target candidate Ai of the current cycle is searched for the target ID of Ck.

  In the subsequent step 1080, EXMAX is set to the extrapolation counter rxc of Ai (this EXMAX is the maximum number of remaining extrapolations and means that the extrapolation count is reset). The extrapolation flag rx is set to 0, and the process proceeds to step 1090.

  On the other hand, in step 1040, which is determined to be negative in step 1030, it is determined whether or not the extrapolation counter rxc of Ck is less than zero. If an affirmative determination is made here, the process proceeds to step 1050, while if a negative determination is made, the process proceeds to step 1090.

In step 1050, the position and speed of Ck are estimated, and a target candidate Ai for the current cycle is created.
In the subsequent step 1060, “Ck extrapolation counter rxc−1” is set in the extrapolation counter rxc of the target candidate Ai. Further, 1 is set to the recognition flag rd of Ai, 1 is set to the extrapolation flag rx of Ai, the width value cw (by the camera device 3) is set to the width value rw of Ai, and the process proceeds to Step 1090.

  In step 1090, it is determined whether all target candidates have been searched by the radar apparatus 1 one cycle before. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 1010 to repeat the same process.

d) Next, processing performed by the camera device 3 will be described.
(1) First, the main routine will be described.
As shown in the flowchart of FIG. 13, in step 1100, the image data captured by the camera 19 is taken into the memory.

In subsequent step 1110, the target candidate data transmitted from the radar signal processing unit 17 of the radar apparatus 1 is received and taken into a memory (not shown) of the image processing unit 23.
In the following step 1120, a target determination process described later is performed (see FIG. 14). This process is a process in which image processing is performed on each of the image and the target candidate, the position and width of the target are calculated, and attribute determination (discrimination between solid and non-solid) is performed.

  In the following step 1130, the result of the target determination is created as communication data in the communication data format B (camera → radar) shown in FIG. 5B and transmitted to the radar signal processing unit 17 of the radar apparatus 1. This processing is once finished. This completes one cycle of processing.

(2) Next, the target determination process in step 1120 will be described.
This process is a process in which image processing is performed on each of the image and the target candidate, the position and width of the target are calculated, and attribute determination (discrimination between solid and non-solid) is performed. This will be specifically described below.

  As shown in the flowchart of FIG. 14, in step 1200, it is determined whether or not the number of target candidates received from the radar apparatus 1 in the current cycle is zero. If an affirmative determination is made here, the present process is once terminated, whereas if a negative determination is made, the process proceeds to step 1210.

In step 1210, the target candidate transmitted from the radar apparatus 1 is extracted.
In the following step 1220, an image processing range is determined. Here, for each target candidate, which position on the image corresponds to is calculated from the vertical position and the horizontal position.

  In the following step 1230, the image is cut out and normalized. Here, the image is cut into a certain size, and normalization is performed to stabilize the processing time. In an actual system, the purpose is to calculate the width of the vehicle, so that, for example, the cut-out width is 4 m and the height is 3 m so that one lane is sufficiently included.

  In the following step 1240, attribute identification processing is performed. Here, in order to distinguish whether the cut out image is solid or non-solid, the cut out image is subjected to a process called identification. This identification is to determine whether the input image is close to a three-dimensional image or a non-three-dimensional image. Generally, as an effective process for performing such identification, there is a process by a learning type classifier represented by a neural network, and the same process is performed here.

  In the subsequent step 1250, center / width calculation processing is performed. In other words, in order to extract a part of an object that has been determined to be a solid by identification, vertical and horizontal edge processing is performed to calculate a boundary part between the object and the road. By the processing so far, the width and center position of the object are calculated.

  In the following step 1260, a determination flag determination process by image tracking is performed. Here, when there is a target candidate by the camera device 3 of the previous cycle and a target candidate by the camera device 3 of the current cycle having the same target ID, the patterns of the two images are compared. Determine if they are the same. Note that processing for determining whether or not they are the same is image tracking processing.

  If it is determined that the process is the same, the determination flag is set to 1 to determine a probable result when several cycles are continued, as in the radar apparatus 1. As a result, the target is confirmed by the camera device 3.

  In the subsequent step 1270, it is determined whether or not all target candidates output from the radar apparatus 1 have been searched one cycle before. If an affirmative determination is made here, the present process is temporarily terminated. On the other hand, if a negative determination is made, the process returns to step 1210 to repeat the same process.

e) Next, effects of the present embodiment will be described.
(1) In the present embodiment, when the radar device 1 generates a target (target candidate) that may be a target, the data is immediately sent from the radar signal processing unit 17 to the image processing unit 23 of the camera device 3. Sent.

  In the image processing unit 23, a predetermined image area including undetermined target data (target candidate data) is determined, and whether or not it is a target and whether an accurate position and width can be detected by known image processing. Judge whether. The determination result is transmitted from the image processing unit 23 to the radar signal processing unit 17.

The radar signal processing unit 17 changes a threshold value for determining whether or not the target is likely in time according to the determination result transmitted from the image processing unit 23.
If it is determined that the target is likely to be detected, early detection is possible by changing the threshold value to shorten the time required for detection check. In addition, if it is transmitted from the camera device 3 that it is not a target, erroneous detection can be prevented by extending the time required for the detection check (or keeping the initial setting value).

  At this time, since the target candidate data that can be detected is transmitted from the radar apparatus 1 to the camera apparatus 3 as well, values such as width, position, and attribute are determined from the data until the radar apparatus 1 determines the target. Sufficient accuracy can be ensured for the calculation.

  In summary, in this embodiment, while using the fact that the processing range of the image in the camera device 3 is suppressed based on the result detected by the radar device 1, detection time delay or erroneous detection that occurs at that time is used. Can be reduced, and accurate detection is possible.

(2) Further, the operation of the present embodiment described above will be described in more detail with reference to FIG.
As shown in FIG. 15 (a), when a target is detected by the radar apparatus 1, in order to increase the determination accuracy, it is not immediately determined to be a target (determined as a target), but the first number For the cycle, the target is undefined.

  At this time, the data of the undetermined target is immediately transmitted to the camera device 3. The camera device 3 performs image processing of the uncertain target transmitted from the radar device 1 and determines whether the target has an attribute (such as solid or non-solid) as the target. Then, the determination result is transmitted to the radar device 1.

  In the radar apparatus 1, the threshold value for determining the target is changed according to the determination result transmitted from the camera apparatus 3. For example, when it is determined that a target candidate is an object (target) instead of noise or the like, the threshold value is decreased. As a result, the number of cycles (number of cycles) for determining the target for several cycles in the radar apparatus 1 can be reduced, so that the time for determining the target can be reduced without degrading the accuracy of target determination. can do. For example, when a certain uncertain target is determined not to be a target by the camera device 3, the threshold value is not changed, so that the detection accuracy does not decrease.

  On the other hand, in the camera device 3 as well, it can be determined that the target is indeed a target (confirmation in the camera device 3) from the determination results of several times (for the same target) in the camera device 3 alone.

Therefore, the target can be determined as a result of sensor fusion comprehensively by determining the target in the radar device 1 and determining the target in the camera device 3.
Further, as shown in FIG. 15B, when the target is detected by the radar apparatus 1, the data of the undetermined target is transmitted to the camera apparatus 3. If it is determined that it is not, the determination result is transmitted to the radar apparatus 1.

  Here, even if the radar apparatus 1 detects a predetermined cycle target for determining the target, if the camera apparatus 1 does not determine that the target is a target, the radar apparatus 1 determines the target. Cancel. As a result, the accuracy of target determination can be increased.

FIG. 15C is an explanatory diagram for explaining the symbols in FIGS. 15A and 15B.
In addition, this invention is not limited to the said Example at all, and it cannot be overemphasized that it can implement with a various aspect in the range which does not deviate from the summary of this invention.

  (1) In the above embodiment, the radar signal processing unit is described as an electronic control device disposed in a radar device, and the image processing unit is described as an electronic control device disposed in a camera device. The functions of the radar signal processing unit and the image processing unit may be realized. In this case, the communication between the radar signal processing unit and the image processing unit can be regarded as data exchange in the arithmetic unit that performs processing corresponding to the radar signal processing unit and the image processing unit.

  (2) Further, according to the present invention, the functions of the radar signal processing unit and the image processing unit of the vehicle object detection device are realized by processing executed by a computer program, that is, a program for realizing the functions. Is also applicable.

  That is, the functions of the control processing in the radar signal processing unit and the image processing unit of the vehicle object detection device described above can be realized by processing executed by a computer program.

  In the case of such a program, for example, it is recorded on a computer-readable recording medium such as FD, MO, DVD-ROM, CD-ROM, hard disk, etc., and is used by being loaded into a computer and started up as necessary. it can. In addition, the ROM or backup RAM may be recorded as a computer-readable recording medium, and the ROM or backup RAM may be incorporated into a computer and used.

It is explanatory drawing which shows the system configuration | structure of the vehicle object detection apparatus of Example 1. FIG. It is explanatory drawing which shows the data of the radar target candidate used with a radar apparatus. It is explanatory drawing which shows the relationship between the vertical position of the own vehicle and a target, and a horizontal position. It is explanatory drawing which shows the data of the camera target candidate used with a camera apparatus. (A) It is explanatory drawing which shows the communication data format from a radar to a camera, (b) It is explanatory drawing which shows the communication data format from a camera to a radar. It is a main flowchart which shows the process sequence implemented in a radar apparatus. It is a flowchart which shows the output target data creation process of a radar apparatus. It is a flowchart which shows the target detection value adjustment process of a radar apparatus. It is a flowchart which shows the extrapolation data update process 1 cycle before. It is a flowchart which shows a log | history connection process. It is a flowchart which shows a target recognition process. It is a flowchart which shows a target extrapolation process. It is a main flowchart which shows the process sequence implemented with a camera apparatus. It is a flowchart which shows the target determination process of a camera apparatus. It is explanatory drawing which shows operation | movement of the vehicle object detection apparatus. It is explanatory drawing which shows operation | movement of the vehicle object detection apparatus of a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... FMCW radar apparatus 3 ... Camera apparatus 5 ... Vehicle object detection apparatus 7 ... Transmission antenna 9 ... Reception antenna 17 ... Radar signal processing part 19 ... CCD camera 21 ... Image processing part

Claims (11)

A vehicle that determines an object based on information from a radar device that irradiates a detection wave to the surroundings and detects an object by the reflected wave, and information from a camera device that detects the object from an image captured by a camera In the object detection device for
The radar apparatus has a configuration for confirming the object based on predetermined uncertain information about the object obtained by the radar apparatus, and
An object detection device for a vehicle, which transmits uncertain information on the object obtained by the radar device to the camera device.   2. The vehicular vehicle according to claim 1, wherein the undetermined information of the object obtained by the radar device is sequentially transmitted to the camera device every time the undetermined information of the object is obtained. Object detection device.   In the camera device, based on the undetermined information of the object transmitted from the radar device, it is determined whether or not the undetermined object is actually an object from an image captured by the camera. The vehicle object detection device according to claim 1, wherein the vehicle object detection device is a vehicle object detection device.   The vehicle object detection device according to claim 3, wherein a determination result in the camera device is transmitted to the radar device.   5. The vehicle object detection device according to claim 4, wherein the radar device changes a condition for determining an undetermined object based on a determination result transmitted from the camera device.   When changing a condition for determining the undetermined object, and using a counter that is incremented for each detection of the undetermined object, means for changing a threshold value used as a criterion for determining the object 6. The vehicle object detection device according to claim 5, wherein means for changing the counter value is used. Information on the object obtained by the radar device is handled as undetermined information, and when it is determined by the camera device based on the undetermined information that the object is an object, The vehicle object detection device according to claim 1, wherein a threshold for determining information is reduced. The determination of an object in the radar device is canceled when the camera device does not determine that the object is an object even if the radar device determines that the object is an object. The vehicle object detection device according to any one of 4 to 7 . In the radar device, when performing extrapolation processing assuming that the object detected in the past but not detected this time is detected this time, the radar device performs the extrapolation processing based on the object determination result in the camera device. The vehicle object detection device according to any one of claims 4 to 8 , wherein an indeterminate object extrapolated by the operation is confirmed. When the extrapolated undetermined object is confirmed based on the determination result by the camera device, the information on the extrapolated undetermined object is not the extrapolated information but the radar device. the vehicle object detection apparatus according to claim 9, characterized in that the the thus obtained undetermined object information. And information that is determined to be an object in the radar device, on the basis of by the camera device and the information that has been determined to be an object, claim 1, characterized in that to determine the overall object the vehicle object detection apparatus according to any one of 10.

Images