JP2007188354A - Forward solid object recognition system for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely recognize a forward solid object using a forward recognition system which spreads generally without using a highly precise forward recognition means. <P>SOLUTION: A solid object detection processing part 9 obtains a featured value comprising positional coordinates in a body width direction, distance coordinates in a body longitudinal direction and a relative speed in the body longitudinal direction on the basis of forward solid objects Mi, Ii and Li detected by respective forward recognition means of a millimeter wave radar system 3, a stereoscopic camera apparatus 2 and a laser radar apparatus 4, and obtains the same probability of each featured value from all the combinations of the respective forward solid objects Mi, Ii and Li on the basis of Gaussian distribution regarding the error of each featured value. Then, it is decided whether the respective forward solid objects Mi, Ii and Li include the same solid objects on the basis of the same probability. Regarding the forward solid object decided to be the same, a fusion solid object is generated on the basis of each of the featured value, and regarding the forward solid object which is not decided to be the same, the fusion solid object is generated on the basis of only a featured value specifying the forward solid object. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a vehicle front three-dimensional object recognition device that determines whether or not front three-dimensional objects detected by a plurality of front recognition means are the same based on a Gaussian distribution relating to an error in a feature amount that identifies each front three-dimensional object.

  In recent years, in vehicles such as automobiles, a technique has been proposed in which a front three-dimensional object is recognized using a front recognition unit such as a camera or a millimeter wave radar mounted on the vehicle, and the recognition result is used for various vehicle controls. .

  By the way, various forward recognition means represented by a camera device and a radar device have variations in detection accuracy depending on the traveling environment. For this reason, recently, a technique called sensor fusion has been proposed in which a plurality of forward recognition means such as a camera device and a radar apparatus are mounted and the results detected by the respective forward recognition means are combined to recognize a forward three-dimensional object.

  For example, in Patent Document 1 (Japanese Patent Laid-Open No. 2005-165421), a probability distribution indicating the certainty of each data is obtained based on each data detected by an image recognition apparatus including a millimeter wave radar and a camera, and the same data value is obtained. A technique is disclosed in which data is fused by taking a product of a plurality of probability distributions and the type of the front three-dimensional object is examined based on the data value indicating the highest probability.

According to the technique disclosed in this document, even if the probability of the probability distribution obtained based on the data detected by a certain forward recognition means is reduced, based on the data detected by other forward recognition means If the obtained probability distribution indicates a high probability, the type of the front three-dimensional object can be recognized using the data value with the high probability.
JP 2005-165421 A

  By the way, for example, the image recognition apparatus needs to improve the detection accuracy by correcting the probability distribution of the data value because the detection accuracy at night, during rain, and snow falls. In this respect, for example, the millimeter wave radar is not easily affected by sunlight, and can perform highly accurate detection, but the detection accuracy of a pedestrian is low.

  Therefore, for example, even when trying to recognize a front three-dimensional object using two front recognition means of a camera and a millimeter wave radar, the detection accuracy of the front three-dimensional object is not good in a driving environment such as at night, during rainfall, or during snowfall. There is a problem that decreases.

  Therefore, in Patent Document 1, the detection accuracy is improved by correcting the probability distribution of the data value detected by the image recognition device based on the data value relating to the darkness detected by the illuminance sensor.

  However, in the technique disclosed in Patent Document 1, auxiliary sensors such as an illuminance sensor must be provided in order to increase the detection accuracy of the front recognition means such as the image recognition device under the influence of the traveling environment. However, there is a problem that the number of parts increases and the manufacturing cost increases.

  On the other hand, it is conceivable to reduce the number of auxiliary sensors to be used by using the forward recognition means with high detection accuracy, but there is a problem that the forward recognition means becomes expensive and the product cost increases.

  In view of the above circumstances, the present invention can detect a front three-dimensional object with high accuracy using a generally recognized front recognition device without using high-precision front recognition means, and has a large number of parts. An object of the present invention is to provide a vehicle front three-dimensional object recognition device that can suppress a significant increase and a rise in product cost and obtain high reliability.

  In order to achieve the above object, the present invention provides at least three types of front recognition means having different detection characteristics and mounted on a vehicle, and the same front three-dimensional object from each front three-dimensional object detected by each front recognition means. In a vehicle front three-dimensional object recognition device including a three-dimensional object detection processing unit to detect, the three-dimensional object detection processing unit obtains a feature amount that identifies each front three-dimensional object based on each front three-dimensional object, and The same probability of the feature quantity of each front three-dimensional object is obtained from all combinations of the front three-dimensional objects detected by the forward recognition means based on the Gaussian distribution related to the error of each feature quantity, and based on the obtained same probability It is determined whether or not each front three-dimensional object has the same three-dimensional object, and if it is determined to be the same, a feature amount of the fusion three-dimensional object is generated based on the feature amount of each front three-dimensional object, If it is determined that the same and having a fusion solid objects generator only by the feature amount generating a feature amount of the fusion solid objects of the three-dimensional objects ahead it is determined that non-identical.

  According to the present invention, the front three-dimensional object is detected by at least three types of front recognition means, and the same probability of each front three-dimensional object detected by each of the front recognition means is used as a Gauss relating to an error in a feature amount that identifies each front three-dimensional object. Since the determination is made based on the distribution, it is possible to detect the front three-dimensional object with high accuracy using a generally recognized front recognition device without using high-precision front recognition means.

  In addition, since the front three-dimensional object is detected using at least three types of front recognition means, for example, even if the detection accuracy of one front recognition means is lowered under a specific traveling environment, Therefore, there is no need to provide auxiliary sensors, and a large increase in the number of parts and a rise in product cost are suppressed, and high reliability can be obtained.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a plan view of the vehicle, and FIG. 2 is a schematic configuration diagram of a front three-dimensional object recognition device mounted on the vehicle.

  As shown in FIG. 1, a vehicle (own vehicle) 1 such as an automobile includes three types of stereo camera devices 2, millimeter wave radar devices 3, and laser radar devices 4 having different detection characteristics as forward recognition means. Yes.

  The stereo camera device 2 has a pair of left and right cameras (CCD camera and CMOS camera) on which a solid-state image pickup device is mounted as a stereo optical system, and both the cameras are mounted at a certain distance above the room mirror. Stereo imaging is performed in front of the vehicle 1 from different viewpoints. Further, the millimeter wave radar device 3 is attached to the front end portion of the vehicle 1 substantially at the center in the vehicle width direction. Since the millimeter wave emitted from the millimeter wave radar device 3 is difficult to narrow down compared to a laser to be described later, a plurality of millimeter wave beams are transmitted radially to the front of the vehicle 1 and from the front three-dimensional object. Receive the reflected wave.

  The laser radar device 4 is attached at substantially the same position as the millimeter wave radar device 3. Since the laser beam emitted from the laser radar device 4 can be narrowed down, as shown in FIG. 1, the narrow laser beam transmitted to the front of the vehicle 1 is changed to the vehicle width direction (or the vehicle width direction). Pulse irradiation is performed while scanning with a predetermined viewing angle in the height direction, and reflected light from the front three-dimensional object is received.

  A stereo image captured by the stereo camera device 2, a millimeter wave signal detected by the millimeter wave radar device 3, and a laser signal detected by the laser radar device 4 are input to the front three-dimensional object recognition device 5 shown in FIG. The front three-dimensional object recognition device 5 is mainly configured by a computer such as a microcomputer, and includes a known CPU, ROM, RAM, and the like.

  The front three-dimensional object recognition device 5 includes a stereo image processing unit 6, a millimeter wave signal processing unit 7, a laser signal processing unit 8, and a three-dimensional object detection processing unit 9 as functions for detecting a front three-dimensional object. Further, a vehicle control unit 21 is connected to the front three-dimensional object recognition device 5.

  The stereo image processing unit 6 detects a front three-dimensional object (hereinafter referred to as “image three-dimensional object”) Ii (i is a variable, i = 1, 2 in FIG. 1) based on a stereo image captured by the stereo camera device 2. , 3) Generate distance data based on the principle of triangulation from the amount of position deviation, and based on this distance data, compare with well-known grouping processing and pre-stored three-dimensional object data to compare vehicles and pedestrians. Each of the three-dimensional image objects is extracted. Further, a feature amount (image feature amount) for specifying each image solid object Ii is obtained. In the present embodiment, the image feature amount includes distance coordinates (hereinafter referred to as “z coordinates”) Iz in the vehicle longitudinal direction (hereinafter referred to as “z direction”) with the vehicle 1 as the origin, and vehicle width directions (hereinafter referred to as “z direction”). Position coordinate (hereinafter referred to as “x coordinate”) Ix of the “x direction”) and relative speed in the z direction (hereinafter referred to as “Vz speed”) obtained based on the temporal change of the z coordinate Iz. Iv.

  The millimeter wave signal processing unit 7 identifies each front three-dimensional object (hereinafter referred to as “millimeter wave three-dimensional object”) Mi (in FIG. 1, i = 1, 2, 3) based on the detection signal from the millimeter wave radar device 3. The feature quantity (millimeter wave feature quantity) to be obtained is obtained. In this embodiment, the millimeter wave feature amount is obtained based on the time from the millimeter wave transmission to reception of the millimeter wave by the millimeter wave radar device 3, the z coordinate Mz, and the transmission direction of the beam from which the reflected wave is received among the plurality of beams. X coordinate Mx obtained based on Vz, and Vz velocity Mv obtained based on the Doppler principle.

  The laser signal processing unit 8 is a feature quantity (laser) for identifying each front three-dimensional object (hereinafter referred to as “laser three-dimensional object”) Li (i = 1, 2 in FIG. 1) based on the detection signal from the laser radar device 4. (Feature). In this embodiment, the laser feature amount is obtained based on the z-coordinate Lz based on the time from laser light transmission to light reception by the laser radar device 4, and the x-coordinate is obtained based on the laser light transmission direction when the reflected light is received. Lx and Vz velocity Lv obtained from a temporal change of the z coordinate Lz are represented.

  The coordinate system representing each feature quantity of the three-dimensional object Ii, Mi, Li detected by each of the processing units 6 to 8 described above is a reference coordinate system (for example, a coordinate system that recognizes the image three-dimensional object I). Aligned and set.

Table 1 shows the detection characteristics of the front three-dimensional object by the millimeter wave radar device 3, the stereo camera device 2, and the laser radar device 4 in relation to the traveling environment. Here, ◎, ○, Δ, and X indicate detection accuracy, ◎ is good, ◯ is slightly good, Δ is slightly inferior, and × is inferior.

  As described above, since the millimeter wave output from the millimeter wave radar device 3 is difficult to narrow down compared to the laser beam, the resolution in the x direction (lateral position) is lower than that of the laser radar device 4. Although the accuracy of detecting the width of the front three-dimensional object is low, the front three-dimensional object can be detected without being affected by the weather.

  On the other hand, the laser light emitted from the laser radar device 4 can be narrowed down and can be continuously scanned in the x direction, so that the lateral position resolution is excellent. Therefore, although the lateral position can be detected with high accuracy, the laser light is refracted or reflected by raindrops, and is therefore easily affected by the weather. On the other hand, when the front three-dimensional object is a preceding vehicle, the laser light is strongly reflected from the left and right reflectors provided on the rear surface of the vehicle body, so that the lateral position and the left and right width of the front three-dimensional object are excellent in detection accuracy. However, it is easy to mistakenly judge two motorcycles running side by side on the left and right reflectors of one four-wheeled vehicle, or two motorcycles running side by side as one four-wheeled vehicle. The ability to recognize the shape of an object is low.

  On the other hand, since the stereo camera device 2 recognizes the front three-dimensional object with the light contrast, the lateral position and the shape resolution are excellent, but the driving environment in which the contrast is difficult to recognize (nighttime, during rain, etc.) The resolution at is low.

  As shown in FIG. 3, the three-dimensional object detection processing unit 9 includes a primary fusion three-dimensional object generation unit 11 and a secondary fusion three-dimensional object generation unit 12.

  The primary fusion three-dimensional object generation unit 11 obtains the image feature amount (x coordinate Ix, z coordinate Iz, Vz velocity Iv) of the image solid object I of interest obtained by the stereo image processing unit 6, and the millimeter wave signal processing unit. 7. Read the millimeter wave feature quantity (x coordinate Mx, z coordinate Mz, Vz speed Mv) of the millimeter wave solid object M of interest that was obtained in step 7, and the feature quantities (x coordinate Mx) of the two forward solid objects I and M. , Ix, z coordinates Mz, Iz, Vz speeds Mv, Iv) are calculated as probabilities (identical probabilities) Pz, Px, PV, respectively, and the three-dimensional object is identical from the product of the same probabilities Pz, Px, Pv. A probability P is calculated, and it is checked whether or not both forward three-dimensional objects I and M are identical based on the three-dimensional object probability P.

  When it is determined that both the front three-dimensional objects I and M are the same, the front three-dimensional object by primary fusion (from the combination of the feature amounts (x coordinates Mx, Ix, z coordinates Mz, Iz, Vz speed Mv, Iv)) ( Hereinafter, a primary fusion feature value (x coordinate Fx, z coordinate Fz, Vz velocity Fv) that is a feature value of F) (referred to as “primary fusion solid object”) is generated.

  Further, the secondary fusion three-dimensional object generation unit 12 obtains the secondary fusion feature amount (x coordinate Fx, z coordinate Fz, Vz velocity Fv) that is the feature amount of the primary fusion three-dimensional object F and the laser signal processing unit 8. , The laser feature amount (x coordinate Lx, z coordinate Lz, Vz speed Lv) of the laser solid object L of interest is read, and the feature amounts (x coordinate Fx, Lx, z coordinate Fz) of both front solid objects F, L are read. , Lz, Vz velocities Fv, Lv) are calculated as probabilities (identity probabilities) Pz, Px, PV, respectively, and a three-dimensional object probability P is calculated from the product of the same probabilities Pz, Px, Pv. Based on this three-dimensional object identity probability P, it is checked whether or not both front three-dimensional objects F and L are identical. When it is determined that both the front three-dimensional objects F and L are the same, the front three-dimensional object by the secondary fusion (hereinafter referred to as “two” from the combination of the x coordinate Fx, Lx, the z coordinate Fz, Lz, and the Vz speed Fv, Lv). A feature quantity (x-coordinate Sx, z-coordinate Sz, Vz velocity Sv) of S) (referred to as “next fusion solid object”) is generated.

  The vehicle control unit 21 recognizes and monitors the front three-dimensional object based on the feature amount (x coordinate Sx, z coordinate Sz, Vz speed Sv) of the secondary fusion three-dimensional object S generated by the front three-dimensional object recognition device 5. If necessary, the entire vehicle including the alarm device is controlled. For example, when a front three-dimensional object is recognized as a pedestrian, the distance to the pedestrian is shortened, and in a driving situation where a warning to the driver is required, an alarm device is activated and a voice prompting attention from a speaker is emitted. The driver is warned by sounding or displaying warning characters on the monitor. Furthermore, if necessary, collision avoidance control such as operating an automatic brake or the like to decelerate the vehicle 1 is performed.

  Specifically, the three-dimensional object recognition process executed by the front three-dimensional object recognition device 5 described above is processed according to a three-dimensional object recognition process routine shown in FIG. This routine is executed every set cycle after turning on the ignition switch. First, a stereo image captured by the stereo camera device 2 is read in step S1, and in step S2, each stereo image Ii is read from the read stereo image. The x-coordinate Ix, the z-coordinate Iz, and the Vz speed Iv are respectively calculated and temporarily stored in the memory (stereo image processing unit 6).

  Next, the process proceeds to step S3, and the millimeter wave signal detected by the millimeter wave radar device 3 is read. In step S4, the x-coordinate Mx, z-coordinate Mz, and Vz velocity Mv of each millimeter-wave solid object Mi are read from the read millimeter-wave signal. Are calculated and temporarily stored in the memory (millimeter wave signal processing unit 7).

  Thereafter, the process proceeds to step S5, and the laser signal recognized by the laser radar device 4 is read. In step S6, the x-coordinate Lx, the z-coordinate Lz, and the Vz velocity Lv of the laser three-dimensional object Li are calculated from the read laser signal. Temporary storage in the memory (laser signal processing unit 8).

  Then, the process proceeds to step S7, and based on the x-coordinates Ix, Mx, Lx, z-coordinates Iz, Mz, Lz, and Vz velocities Iv, Mv, Lv of the respective front three-dimensional objects Ii, Mi, Li, A fusion three-dimensional object is generated (three-dimensional object detection processing unit 9), and the routine is exited.

  The fusion three-dimensional object generation process executed in step S7 is performed according to a fusion three-dimensional object generation subroutine shown in FIGS. In addition, the process performed by the primary fusion solid object production | generation part 11 is performed by step S11-S21 of this subroutine, and the process performed by the secondary fusion solid object production | generation part 12 is performed by subsequent step S22-S32. .

  First, in steps S11 to S22, the three-dimensional object detected by the millimeter wave radar device 3 is compared with the three-dimensional image captured by the stereo camera device 2, and the same three-dimensional object is detected. In this case, when the millimeter wave and the image are compared, the millimeter wave three-dimensional object Mi is set as the reference three-dimensional object because the Vz velocity and the distance (x coordinate) in the z direction have high millimeter wave detection accuracy.

  That is, in step S11, a reference n-th millimeter wave solid object (hereinafter referred to as “reference millimeter wave solid object”) Mn is selected from each millimeter wave solid object Mi detected by the millimeter wave signal processing unit 7. . Note that n is an integer and the initial value is 1.

  Next, the process proceeds to step S12, and one image solid object Ii is selected. Thereafter, in step S13, the standard deviation σx in the x direction, the standard deviation σz in the z direction, and the standard deviation σv in the Vz velocity between the millimeter wave radar device 3 and the stereo camera device 2 are calculated. By the way, as described above, when millimeter wave and image detection accuracy are compared, millimeter wave detection accuracy is excellent with respect to the distance and speed in the z direction. However, the image detection accuracy is excellent for the distance in the x direction. However, the image detection accuracy decreases as the distance in the x direction increases.

  Therefore, each standard deviation σx, σz, σv is calculated from the following equation based on the z coordinate Mz of the reference millimeter-wave solid object Mn with relatively high detection accuracy.

σx = ηx [m] + k1 · Mz (1)
σz = ηz [m] + k2 · Mz (2)
σv = (ηv [m] / t [sec]) + k3 · Mz (3)
Here, ηx, ηz, and ηv are values for correcting variations in detection accuracy of the millimeter wave and the image in the x direction, the z direction, and the Vz speed, respectively.
ηx = (α · kmx) + (β · kix)
ηz = (α · kmz) + (β · kiz)
ηv = (α · kmv) + (β · kiv)
Is required. Here, α and β are coefficients, which are set in advance by experiments. Further, kmx, kmz, and kmv are errors in the x direction, z direction, and Vz velocity of the millimeter wave radar device 3, and kix, kiz, and kiv are errors in the x direction, z direction, and Vz velocity of the stereo camera device 2, and It is set by seeking from experiments.

  Thus, in this embodiment, the standard deviations σx, σz, σv are based on the z coordinate (distance coordinate in the z direction) Mz of the reference millimeter wave solid object Mn detected by the millimeter wave radar device 3 having excellent detection accuracy. Since it is variably set, a more accurate Gaussian distribution can be obtained.

Next, the process proceeds to step S14, where the same probability Px, Pz, Pv of the x coordinate Mx, Ix, z coordinate Mz, Iz, Vz velocity Mv, Iv between the reference millimeter wave solid object Mn and the image solid object Ii is Each is set based on the probability density function shown in the equation. However, in the following equation, the x coordinate, z coordinate, and Vz velocity of the reference millimeter wave solid object Mn and the image solid object Ii are Mn = (x1, z1, v1) and Ii = (x2, z2.v2), respectively. Represents.

  The meanings of the equations (4) to (6) mean that a Gaussian distribution relating to errors in the x-coordinates x1, x2, z-coordinates z1, z2, and Vz velocities v1, v2 between the reference millimeter-wave solid object Mn and the image solid object Ii. , X-coordinate x1, z-coordinate z1, and Vz velocity v1 are obtained as references, and from the difference (x2-x1), (z2-z1), (v2-v1), the reference millimeter-wave solid object Mn and the image solid object Ii The same probability is to be calculated. The maximum value of each of the same probabilities Px, Pz, Pv is 1 (100 [%]). Therefore, as shown in FIG. 7, for example, the relationship between the x coordinates x1, x2 is x2> x1, that is, a difference occurs. Px <1. The same applies to the z coordinates z1 and z2 and the Vz coordinates v1 and v2.

  In this embodiment, the same probabilities Px, Pz, and Pv are set by map search. Each map (x coordinate identical probability map, z coordinate identical probability map, Vz velocity identical probability map) includes an x coordinate Mx, z coordinate Mz, Vz velocity Mv and standard deviations σx, σz of the reference millimeter wave solid object Mn in advance. , Σv and the respective Gaussian distribution tables are stored, and based on the respective Gaussian distribution tables, the same probabilities Px, Pz, Pv are respectively set by using the x-coordinate Ix, the z-coordinate Iz, and the Vz velocity Iv as variables. The

Subsequently, it progresses to step S15 and the solid thing identity probability Pi is calculated from following Formula.
Pi ← Px · Pz · Pv (7)
As described above, since the maximum value of each of the same probabilities Px, Pz, and Pv is 1, P = 1 (100 [%]) when they are completely the same, and when there is a difference in each coordinate system Is P <1.

  Thereafter, the process proceeds to step S16, where the three-dimensional object identity probability Pi is compared with a preset threshold value SL1. This threshold value SL1 is obtained by setting a range that can be regarded as the same in advance through experiments or the like. In this embodiment, the threshold value SL1 is set to about 40 to 70 [%].

  When the three-dimensional object identity probability Pi is less than the threshold value SL1 (Pi <SL1), it is determined that the three-dimensional object identity probability Pi is invalid, and the process proceeds to step S17, where the three-dimensional object identity probability Pi is cleared (Pi ← 0), go to step S18. If the three-dimensional object identity probability Pi is greater than or equal to the threshold value SL1 (P ≧ SL1), it is determined that the three-dimensional object identity probability Pi is valid, and the process proceeds to step S18. In this case, the three-dimensional object identity probability Pi determined to be valid is not limited to one for one reference millimeter wave three-dimensional object Mn. There is also a possibility to do.

  Then, when proceeding to step S18, the three-dimensional object identity probability Pi set this time is stored. As shown in FIG. 1, for example, the millimeter wave solid object M1 is selected as the reference millimeter wave solid object, and the same probability P with all the image solid objects I1, I2, and I3 based on the reference millimeter wave solid object M1. As a result, when it is determined that the reference millimeter wave solid object M1 and the image solid object I1 are the same, but the other image solid objects I2 and I3 are not identical, the reference millimeter wave solid object M1 The three-dimensional object identity probability P with the image three-dimensional object I1 is determined to be valid and stored as it is. The three-dimensional object identity probability P of the reference millimeter wave solid object M1 and the image solid object I2 and the three-dimensional object identity probability P of the reference millimeter wave solid object M1 and the image solid object I3 are stored as 0, respectively. Similarly, for the other millimeter wave solid objects M2 and M3, the solid object probability P with all the image solid objects I1, I2 and I3 is calculated.

  As a result, the solid object probability P is calculated for all combinations of the millimeter wave solid objects M1, M2, and M3 and the image solid objects I1, I2, and I3. Therefore, even when the same millimeter wave solid object is not detected as in the image solid object I3, the solid object probability P is calculated. In this case, the solid object probability P is set to zero.

  Of course, even when an image solid object is not detected like the millimeter wave solid object M3, the solid object probability P is calculated. In this case also, the three-dimensional object probability P is set to zero. In this case, for example, even the front three-dimensional object that is not detected by each of the millimeter wave solid objects M1, M2, and M3, such as the image solid object I3 of FIG. 1, is connected to each millimeter wave solid object M1, M2, and M3. Although the same probability P is respectively set, in the generation of the primary fusion three-dimensional object Fi described later, they are integrated as one front three-dimensional object.

  Thereafter, the process proceeds to step S19, where it is checked whether or not all of the image solid object Ii has been selected. If there is an unselected image solid object Ii, the process returns to step S12 to select the next image solid object Ii. To do. If all of the image solid object Ii is selected, the process proceeds to step S20.

  In step S20, it is checked whether or not all the millimeter wave solid objects Mi are selected. If there is an unselected millimeter wave solid object Mi, the process returns to step S11, and the next millimeter wave solid object Mi is used as a reference. The millimeter wave solid object Mn is set, and the solid object probability Pi with the image solid object Ii is obtained with the reference millimeter wave solid object Mn as a reference. If all the millimeter wave solid objects Mi are selected, the process proceeds to step S21.

  In step S21, a primary fusion solid object Fi is generated. When generating the primary fusion three-dimensional object Fi, first, the millimeter wave three-dimensional object Mi and the image three-dimensional object Ii determined to be the same three-dimensional object are extracted based on all the calculated three-dimensional object identity probabilities Pi. Determination conditions for extracting the same three-dimensional object are as follows.

Condition 1: Pi> 0 Condition 2: The image solid object Ii has the maximum same probability with respect to the millimeter wave solid object Mi Condition 3: The millimeter wave solid object Mi has the maximum same probability with respect to the image solid object Ii When all of the conditions 1 to 3 are satisfied, it is determined that the millimeter wave solid object Mi and the image solid object Ii are the same.

  Based on the combination of the millimeter wave solid object Mi and the image solid object Ii determined to be the same, and the millimeter wave solid object Mi and the image solid object Ii whose solid object probability Pi is set to 0, the primary fusion solid is obtained. Each of the objects Fi is generated. In this case, in the case of only the image solid object Mi (no identical millimeter wave solid object Mi) or only the millimeter wave solid object (no identical image solid object Ii), the feature amount of the image solid object Ii ( The primary fusion solid object Fi is generated with the x-coordinate Ix, the z-coordinate Iz, the Vz velocity Iv), or the feature quantity of the millimeter-wave solid object Mi (x-coordinate Mx, z-coordinate Mz, Vz velocity Mv). When it is determined that the object Mi and the image solid object Ii are the same, the primary fusion solid object Fi is generated based on the feature amount on the side where the millimeter wave radar device 3 and the stereo camera device 2 have high detection accuracy.

Specifically, the feature amount of the primary fusion three-dimensional object Fi is generated according to the following criteria.
Pi = 0 (However, there is no millimeter wave solid object Mi that is the same, only the image solid object Ii)
x-coordinate ← x-coordinate Ix of image solid object Ii
z coordinate ← z coordinate Iz of the three-dimensional image Ii
Vz speed ← Vz speed Iv of the three-dimensional object Ii
Therefore, the feature amount of the primary fusion solid object Fi generated only by the image solid object Ii is represented by the x coordinate Ix, the z coordinate Iz, and the Vz speed Iv.

Pi = 0 (However, there is no image solid object Ii that is the same, only the millimeter wave solid object Mi)
x coordinate ← x coordinate Mx of millimeter wave solid object Mi
z-coordinate ← z-coordinate Mz of solid object Mi
Vz velocity ← Vz velocity Mv of millimeter wave solid object Mi
Therefore, the feature quantity of the primary fusion three-dimensional object Fi generated only by the millimeter wave three-dimensional object Mi is represented by the x coordinate Mx, the z coordinate Mz, and the Vz velocity Mv.

Pi> 0 (the millimeter wave solid object Mi and the image solid object Ii are the same)
x-coordinate ← weighted average of x-coordinates Ii and Mi of image solid object Ii and millimeter-wave solid object Mi z-coordinate ← z-coordinate Mz of millimeter-wave solid image Mi
Vz velocity ← Vz velocity Mv of millimeter wave solid object Mi
Accordingly, the primary fusion three-dimensional object Fi generated when it is determined that the millimeter wave three-dimensional object Mi and the image three-dimensional object Ii are the same is represented by an x coordinate (weighted average of Ii and Mi), a z coordinate Mz, and a Vz velocity Mv. Is done. As described above, when comparing the millimeter wave and image detection accuracy, the millimeter wave detection accuracy is excellent for the distance and speed in the z direction, and the image detection accuracy is excellent for the x direction distance. . However, the image detection accuracy decreases as the distance in the x direction increases.

  Therefore, the feature quantity representing the primary fusion solid object Fi when the millimeter wave solid object Mi and the image solid object Ii are recognized to be the same is the z coordinate, and the Vz velocity is the z coordinate Mz of the millimeter wave solid image Mi. The Vz velocity Mv is preferentially adopted, and the x-coordinate is calculated from the weighted average of both x-coordinates Ii and Mi, thereby improving the detection accuracy.

  For example, in FIG. 1, when it is determined that the millimeter wave solid object M1 and the image solid object I1, and the millimeter wave solid object M2 and the image solid object I2 are the same, they are respectively fused, and the middle line in FIG. Primary fusion solids F1 and F2 as shown by the frame are generated. Further, the image solid object I3 does not have the same millimeter wave image, and the millimeter wave solid object M3 does not have the same image solid object. Therefore, the primary fusion three-dimensional objects F3 and F4 are generated with the feature quantities of the image solid object I3 and the image solid object M3.

  When all the primary fusion three-dimensional objects Fi are generated in step S21 described above, the program proceeds to step S22, and in step S22 and subsequent steps, the secondary fusion is obtained from the combination of the primary fusion three-dimensional object Fi and the laser three-dimensional object Li. A three-dimensional object Si is generated.

  First, in step S22, an n-th primary fusion solid object (hereinafter referred to as “reference primary fusion solid object”) Fn serving as a reference is selected from the primary fusion solid object Fi. Note that n is an integer and the initial value is 1.

  Next, the process proceeds to step S23, and one laser solid object Li is selected. Thereafter, in step S24, the standard deviation σx in the x direction, the standard deviation σz in the z direction, and the standard deviation σv in the Vz velocity of the millimeter wave radar device 3, the stereo camera device 2, and the laser radar device 4 are calculated. Each standard deviation σx, σz, σv is calculated from the following equation based on the z coordinate Fz of the reference primary fusion solid object Fn.

σx = ηx [m] + k1 · Fz (1 ′)
σz = ηz [m] + k2 · Fz (2 ′)
σv = (ηv [m] / t [sec]) + k3 · Fz (3 ′)
Here, ηx, ηz, and ηv are values for correcting variations in detection accuracy of the millimeter wave, the image, and the laser beam in the x direction, the z direction, and the Vz speed, respectively.
ηx = (α · kmx) + (β · kix) + (γ · klx)
ηz = (α · kmz) + (β · kiz) + (γ · klz)
ηv = (α · kmv) + (β · kiv) + (γ · klv)
Is required. Here, α, β, and γ are coefficients, which are set in advance by experiments. Further, kmx, kmz, kmv are errors in the x direction, z direction, and Vz velocity of the millimeter wave radar device 3, kix, kiz, kiv are errors in the x direction, z direction, Vz velocity of the stereo camera device 2, and klx, klz. , Klv are errors in the x-direction, z-direction, and Vz speed of the laser radar device 4, and are set in advance by experiments.

  As described above, in the present embodiment, the standard deviations σx, σz, and σv are based on the front three-dimensional objects Mi and Ii detected by the millimeter wave radar device 3 or the stereo camera device 2 having better detection accuracy than the laser radar device 4. Since it is variably set based on the z coordinate (distance coordinate in the z direction) Fz of the generated reference primary fusion solid object Fn, a more accurate Gaussian distribution can be obtained.

  Next, the process proceeds to step S25, where the same probabilities Px, Pz, Pv of the x coordinate Fx, Lx, z coordinate Fz, Lz, Vz velocity Fv, Lv of the reference primary fusion solid object Fn and the laser solid object Li are described above. Each is set based on the probability density function shown in equations (4) to (5). In this case, the x coordinate, the z coordinate, and the Vz velocity of the reference primary fusion three-dimensional object Fn and the laser three-dimensional object Li are set to Fn = (x1, z1, v1) and Ii = (x2, z2.v2), respectively. In this embodiment, the same probabilities Px, Pz, and Pv are set by map search. In each map (x coordinate identical probability map, z coordinate identical probability map, Vz velocity identical probability map), the x-coordinate Fx, z-coordinate Fz, Vz velocity Fv and standard deviations σx, σz of the reference primary fusion three-dimensional object Fn in advance. , Σv and the respective Gaussian distribution tables are stored, and based on the Gaussian distribution tables, the respective probabilities Px, Pz, Pv are set by using the x-coordinate Lx, the z-coordinate Lz, and the Vz velocity Lv as variables. Is done.

  Subsequently, it progresses to step S26 and the solid thing identity probability Pi is calculated from (7) Formula mentioned above.

  Thereafter, the process proceeds to step S27, where the three-dimensional object identity probability Pi is compared with a preset threshold value SL2. This threshold value SL2 is obtained by setting a range that can be regarded as the same in advance through experiments or the like, and is set to about 40 to 70 [%] in this embodiment.

  If the three-dimensional object identity probability Pi is less than the threshold value SL2 (Pi <SL2), the process proceeds to step S28, the three-dimensional object identity probability Pi is cleared (Pi ← 0), and the process proceeds to step S29. If the three-dimensional object identity probability Pi is greater than or equal to the threshold value SL2 (P ≧ SL2), the process proceeds directly to step S29. If it progresses to step S29, the solid object probability Pi set this time will be memorize | stored.

  For example, in FIG. 1, the primary fusion three-dimensional object F1 is selected as the reference primary fusion three-dimensional object, and the same probability P with all the laser three-dimensional objects L1, L2 is calculated based on the reference primary fusion three-dimensional object F1, When it is determined that the reference primary fusion three-dimensional object F1 and the laser three-dimensional object L1 are the same, but the other laser three-dimensional object L2 is not identical, the three-dimensional structure of the reference primary fusion three-dimensional object F1 and the laser three-dimensional object L1. The object identity probability P is determined to be valid, and the value of the three-dimensional object identity probability P is stored as it is. The three-dimensional object probability P of the reference primary fusion three-dimensional object F1 and the other laser three-dimensional object L2 is stored as zero. Similarly, for the other fusion solid objects F2 to F4, the solid object probability P with all the laser solid objects L1 and L2 is calculated. As a result, for example, when it is determined that the primary fusion three-dimensional object F3 and the laser three-dimensional object L2 are the same, the value of the three-dimensional object identity probability P is stored as it is. On the other hand, since the primary fusion solid object F4 does not have the same laser solid object, the solid object identity probability P is set to zero.

  Then, it progresses to step S30, it is investigated whether all the laser solid objects Li were selected, and when the unselected laser solid object Li exists, it returns to step S23 and selects the next laser solid object Li To do. On the other hand, when all of the laser solid objects Li are selected, the process proceeds to step S31.

  In step S31, it is checked whether or not all of the fusion three-dimensional object Fi has been selected. If there is an unselected fusion three-dimensional object Fi, the process returns to step S22, and the next fusion three-dimensional object Fi is used as the reference primary fusion three-dimensional object. The object Fn is set, and the three-dimensional object probability Pi with the laser three-dimensional object Li is obtained with the reference primary fusion three-dimensional object Fn as a reference. If all of the fusion solid objects Fi are selected, the process proceeds to step S32.

  If it progresses to step S32, the secondary fusion solid object Si will be produced | generated. When generating the secondary fusion solid object Si, first, the primary fusion solid object Fi and the laser solid object Li, which are determined as the same solid object, are extracted based on all the calculated three-dimensional object probabilities Pi. Determination conditions for extracting the same three-dimensional object are as follows.

Condition 1: Pi> 0 Condition 2: The solid three-dimensional object Li has the highest probability of being the same as the primary fusion solid object Fi Condition 3: The primary fusion solid object Fi has the highest probability of being the same as the laser three-dimensional object Li And when these conditions 1-3 are all satisfied, it determines with the primary fusion solid object Fi and the laser solid object Li being the same.

  Based on the combination of the primary fusion solid object Fi and the laser solid object Li determined to be the same, and the primary fusion solid object Fi and the laser solid object Li in which the solid object probability Pi is set to 0, the secondary fusion is performed. A three-dimensional object Si is generated.

  In this case, in the case of only the laser three-dimensional object Li (no identical primary fusion three-dimensional object Fi) or only the primary fusion three-dimensional object Fi (no identical laser three-dimensional object Li), the feature amount of the laser three-dimensional object Li A secondary fusion three-dimensional object Si is generated with (x-coordinate Lx, z-coordinate Lz, Vz velocity Lv) or a feature quantity (x-coordinate Mx, z-coordinate Mz, Vz velocity Mv) of the primary fusion three-dimensional object Fi. When it is determined that the fusion three-dimensional object Fi and the laser three-dimensional object Li are the same, the secondary fusion three-dimensional object Si is generated based on the feature amount having a high detection accuracy.

Specifically, the feature quantity of the secondary fusion solid object Si is generated according to the following criteria.
Pi = 0 (However, there is no primary fusion solid object Fi that is the same, only the laser solid object Li)
x coordinate ← x coordinate Lx of laser solid object Li
z-coordinate ← z-coordinate Lz of laser solid object Li
Vz velocity ← Vz velocity Lv of laser solid object Li
Therefore, the feature quantity of the secondary fusion solid object Si generated only by the laser solid object Li is represented by the x coordinate Lx, the z coordinate Lz, and the Vz speed Lv.

Pi = 0 (However, there is no laser solid object Li that is the same, only the primary fusion solid object Fi)
x-coordinate ← x-coordinate Fx of primary fusion solid object Fi
z-coordinate ← z-coordinate Fz of primary fusion solid Fi
Vz velocity ← Vz velocity Fv of primary fusion solid object Fi
Therefore, the feature quantity of the secondary fusion solid object Si generated only by the primary fusion solid object Fi is represented by the x coordinate Fx, the z coordinate Fz, and the Vz velocity Fv.

Pi> 0 (primary fusion three-dimensional object Fi and laser three-dimensional object Li are the same) and primary fusion three-dimensional object Fi is generated only from image three-dimensional object Ii x-coordinate ← each of primary fusion three-dimensional object Fi and laser three-dimensional object Li Weighted average of x-coordinates Fx and Lx z-coordinate ← z-coordinate Lz of laser solid object Li
Vz velocity ← Vz velocity Lv of laser solid object Li
Therefore, the feature quantity of the secondary fusion solid object Si generated by the primary fusion solid object Fi generated only by the image solid object Ii and the laser solid object Li is an x coordinate (weighted average of Fi and Li), The z coordinate Lz and the Vz speed Lv are obtained.

  When the detection accuracy of the stereo camera device 2 and the laser radar device 4 is compared, the detection accuracy in the x direction is about the same, so the x coordinate is obtained by a weighted average of the x coordinates Fx and Lx. On the other hand, since the z-direction detection accuracy is higher in the laser radar device 4, the z-coordinate Lz and the Vz velocity Lv detected by the laser radar device 4 are employed.

Pi> 0 (the primary fusion solid object Fi and the laser solid object Li are the same), and the primary fusion solid object Fi is generated only by the millimeter wave solid object Mi x-coordinate ← of the primary fusion solid object Fi and the laser solid object Li Weighted average of each x-coordinate Fx, Lx z-coordinate ← z-coordinate Mz of millimeter wave solid object Mi
Vz velocity ← Vz velocity Mv of millimeter wave solid object Mi
Therefore, the feature quantity of the secondary fusion solid object Si generated by the primary fusion solid object Fi generated only by the millimeter wave solid object Mi and the laser solid object Li is the x coordinate (weighted average of Fi and Li). , Z coordinate Mz, and Vz velocity Mv.

  When the detection accuracy of the millimeter wave radar device 3 and the laser radar device 4 is compared, the detection accuracy in the x direction is comparable, and therefore the x coordinate is obtained by a weighted average of the x coordinates Fx and Lx. On the other hand, since the millimeter wave radar device 3 has higher z-direction detection accuracy, the z coordinate Mz and the Vz velocity Mv detected by the millimeter wave radar device 3 are employed.

Pi> 0 (the primary fusion three-dimensional object Fi and the laser three-dimensional object Li are the same), and the primary fusion three-dimensional object Fi is generated by the millimeter wave three-dimensional object Mi and the image three-dimensional object Ii x-coordinate ← primary fusion three-dimensional object Fi and laser Weighted average of each x-coordinate Fx, Lx of the three-dimensional object Li z-coordinate ← z-coordinate Mz of the three-dimensional object Mi
Vz velocity ← Vz velocity Mv of millimeter wave solid object Mi
Accordingly, the feature quantity of the secondary fusion solid object Si generated by the primary fusion solid object Fi generated by the millimeter wave solid object Mi and the image solid object Ii and the laser solid object Li is expressed by the x coordinate (Fi and Li ), Z coordinate Mz, and Vz velocity Mv. The reason for selecting the combination is as described above.

  And after all the secondary fusion solid object Si is produced | generated by step S32 mentioned above, a routine is exited.

  Next, the secondary fusion solid object Si generated by the solid object detection processing unit 9 shown in FIG. 2 is output to the vehicle control unit 21. The vehicle control unit 21 recognizes and monitors the front three-dimensional object based on the feature quantities (x coordinate Sx, z coordinate Sz, Vz speed Sv) of the secondary fusion three-dimensional object Si. When it is determined that a collision with the secondary fusion three-dimensional object Si is unavoidable, collision avoidance control such as operating an automatic brake or the like to decelerate the vehicle 1 is performed.

  As described above, in this embodiment, the front three-dimensional object of the vehicle 1 is combined with the stereo camera device 2, the millimeter wave radar device 3, and the laser radar device 4 each having different detection characteristics. ) Since Si is generated, the detection accuracy of one or two of the stereo camera device 2, the millimeter wave radar device 3, and the laser radar device 4 is reduced in a specific traveling environment. Even if it exists, since it is interpolated by other devices, it is not necessary to interpolate the decrease in detection accuracy by providing auxiliary sensors as in the past, which greatly increases the number of parts and increases the product cost. It is suppressed and high reliability can be obtained. For example, a front three-dimensional object that is not detected by the stereo camera device 2 or the millimeter wave radar device 3 can be detected by the laser radar device 4 in a traveling environment such as at night, during rainfall, or during snowfall. Therefore, higher detection accuracy of the front three-dimensional object can be obtained.

  In addition, since only three types of the stereo camera device 2, the millimeter wave radar device 3, and the laser radar device 4 are used, a significant increase in the number of parts is suppressed. In addition, since these devices are generally used, the rise in component costs is suppressed.

  By the way, it is also possible to apply the front three-dimensional object recognition technique according to this embodiment to the pedestrian recognition process. That is, the feature amount (x coordinate Sx, z coordinate Sz, Vz speed Sv) of the secondary fusion solid object Si, the width information of the secondary fusion solid object Si detected by the stereo camera device 2, and the moving speed in the x direction. Based on the above, it is roughly determined whether or not the detected secondary fusion three-dimensional object Si is a pedestrian. And when it determines with a pedestrian, the said secondary fusion solid object imaged with the stereo camera apparatus 2 is image-processed, and it is investigated in more detail whether it is a pedestrian.

  By roughly determining whether or not a person is a pedestrian based on the feature quantity of the secondary fusion solid object Si, the computational load required for pedestrian recognition is greatly reduced compared to when processing all solid objects. can do.

  Note that the present embodiment is not limited to the above-described embodiment. For example, the forward recognition means has a detection characteristic different from these in addition to the stereo camera device 2, the millimeter wave radar device 3, and the laser radar device 4 described above. There may be four or more types by adding other front recognition lines such as an infrared laser radar device.

Plan view of the vehicle Schematic configuration diagram of a front three-dimensional object recognition device mounted on a vehicle Functional block diagram of the three-dimensional object detection processing unit Flow chart showing a three-dimensional object recognition processing routine Flow chart showing a fusion three-dimensional object generation routine (part 1) Flow chart showing a fusion three-dimensional object generation routine (part 2) Chart showing the same probability of the horizontal position of the front three-dimensional object

Explanation of symbols

1 ... vehicle,
2 ... Stereo camera device,
3 Millimeter wave radar device,
4 ... Laser radar device,
5 ... Front three-dimensional object recognition device,
6 ... Stereo image processing unit,
7: Millimeter wave signal processing unit,
8: Laser signal processing unit,
9 ... Solid object detection processing unit,
11 ... Primary fusion three-dimensional object generation unit,
12 ... Secondary fusion three-dimensional object generator,
Fi ... primary fusion solid,
Fn: Standard primary fusion solid,
Ii, Mi, Li ... solid objects,
Iv, Mv, Lv, Sv ... Vz velocity,
Ix, Mx, Lx, Sx ... x coordinate,
Iz, Mz, Lz, Sz ... z coordinate,
Li: Laser solid object,
Mi ... Millimeter wave solid object,
Mn: Standard millimeter wave solid object,
Pi: solid object probability,
Px, Pz, Pv ... same probability,
Si: Secondary fusion solid,
σx, σz, σv ... standard deviation

Claims (5)

At least three types of forward recognition means having different detection characteristics and mounted on a vehicle, and a three-dimensional object detection processing unit for detecting the same forward three-dimensional object from the respective front three-dimensional objects detected by the respective front recognition means. In the vehicle front three-dimensional object recognition device,
The three-dimensional object detection processing unit is
Based on each of the front three-dimensional objects, a feature amount that specifies each of the front three-dimensional objects is obtained, and the same amount of the feature amount of each of the front three-dimensional objects is obtained from all combinations of the front three-dimensional objects detected by the front recognition means. The probability is calculated based on the Gaussian distribution relating to the error of each feature value, and it is determined whether or not each front three-dimensional object has the same three-dimensional object based on the obtained same probability. A feature amount of the fusion three-dimensional object is generated based on the feature amount of each front three-dimensional object, and when it is determined as non-identical, only the feature amount of each front three-dimensional object determined as non-identical is the feature of the fusion three-dimensional object A vehicle front three-dimensional object recognition device comprising a fusion three-dimensional object generation unit that generates a quantity. The fusion three-dimensional object generation unit variably sets the standard deviation of the Gaussian distribution according to the detection accuracy of the front recognition means that detects the front three-dimensional object that is the setting target of the Gaussian distribution. Item 3. A vehicle front three-dimensional object recognition device according to Item 1. The fusion three-dimensional object generation unit sets the feature amount of the fusion three-dimensional object according to detection characteristics of the front recognition means that detects the front three-dimensional objects that are the basis of the fusion three-dimensional object. The vehicle front three-dimensional object recognition device according to claim 1 or 2. The feature amount includes a distance coordinate in a vehicle longitudinal direction with respect to the vehicle as an origin, a position coordinate in a vehicle width direction, and a relative speed obtained based on a temporal change in the distance coordinate in the vehicle longitudinal direction. The vehicle front three-dimensional object recognition device according to any one of claims 1 to 3. The vehicle front three-dimensional object recognition device according to any one of claims 1 to 4, wherein the front recognition means is at least a millimeter wave radar device, a stereo camera device, and a laser radar device.

Images