JP4753765B2 - Obstacle recognition method - Google Patents

Description

The present invention is an obstacle recognition method in which a radar and a camera provided in a vehicle repeatedly search in front of the host vehicle, and an obstacle such as a preceding vehicle in front of the host vehicle is recognized by a sensor fusion recognition process between the radar and the camera. it relates to the law.

  In general, in a vehicle equipped with a vehicle driving support system (Adaptive Cruise Control) called ACC, in order to realize a so-called damage reduction automatic brake function, an obstacle such as a preceding vehicle or a sign in front of the host vehicle may be recognized. Required.

  However, in a conventional ACC-equipped vehicle, in many cases, a sensor that obtains information in front of the ACC vehicle is composed of a single sensor such as a laser radar or a millimeter wave radar. There is an inconvenience that can only be done in scenes that are relatively easy to recognize.

  In addition, the sensor fusion recognition process using multiple sensors improves recognition performance and eliminates the above-mentioned disadvantages. When various obstacles are detected in front of the vehicle, Does not sample data at exactly the same time, but there is a difference of about several tens of milliseconds (ms) in the search and detection of each sensor, and the influence of the difference in the coordinates of radar search and camera shooting cannot be avoided. There was a problem that the recognition accuracy could not be improved.

  Therefore, the applicant of the present invention is able to recognize the preceding vehicle in front of the subject vehicle by the sensor fusion recognition process based on the radar distance measurement result and the image captured by the camera without being affected by the difference in coordinates between the radar exploration and the camera image capturing. An obstacle recognizing method and an obstacle recognizing apparatus capable of accurately recognizing other obstacles have already been filed (for example, see Patent Document 1).

  The obstacle recognition method and obstacle recognition apparatus of the already-applied application are generally an obstacle including a preceding vehicle in front of the host vehicle based on the radar search result by repeatedly searching the front of the host vehicle with a radar and a camera provided in the vehicle. The radar-side detection magnification is obtained from the change in the ranging distance up to, etc., and the image of the magnification of the horizontal and vertical image histogram of the image captured in front of the camera's own vehicle is processed by image processing of the image taken in front of the camera's own vehicle. The camera side detection magnification is obtained based on the minimum value of the residual sum with the histogram, and the horizontal and vertical magnifications at which the residual sum becomes the minimum value are equalized by the sensor fusion recognition process between the radar and the camera. It is determined whether or not the image of the obstacle area (candidate area) in the captured image is a road surface vertical object such as a preceding vehicle, and a road surface vertical object in which the radar side detection magnification and the camera side detection magnification are equal is determined as an obstacle. To identify.

  In this case, if the object is an obstacle, the image is enlarged / reduced at the same magnification in both the horizontal and vertical directions on the captured image, and the residual sum is minimized at the same horizontal and vertical magnification. On the other hand, if it is not an obstacle but a manhole cover on the road surface, a cat's eye (鋲), etc., the horizontal and vertical residual sums do not change at the same magnification in the horizontal and vertical directions on the photographed image. The minimum magnification is different.

  Therefore, it is determined whether or not the candidate target of the candidate area is a road surface vertical object such as an obstacle existing on the road surface by determining whether or not the magnification of the residual sum minimum value of the horizontal and vertical image histograms is equal. Thus, it is possible to determine whether or not the road surface vertical object is an obstacle from the coincidence or mismatch between the camera side detection magnification and the radar side detection magnification, and to recognize the obstacle from the candidate target image in the candidate area.

  Then, it is only necessary to calculate the horizontal and vertical histograms of the image edges that are simple and require a small amount of calculation. The calculation of the image processing is simple and requires a very small amount of calculation. There are advantages such as being considered and recognized.

Japanese Patent Laying-Open No. 2005-329779 (abstract, claims 1, 13, paragraphs [0055]-[0123], FIG. 1, etc.)

  In the case of the obstacle recognition method and obstacle recognition device of the above-mentioned application, the camera-side detection magnification and the radar-side detection magnification are compared to determine whether or not the road surface vertical obstacle is an obstacle from the coincidence or mismatch. However, the detection method is different between the camera and the radar, and the camera can obtain the result immediately, but since the radar scans the front of the vehicle and obtains the exploration result, the camera and the radar can simultaneously Even if the detection is started, the radar detection is delayed from the camera detection.

  Therefore, it is difficult to compare the radar-side detection magnification and the camera-side detection magnification based on the detection results of the radar and the camera at exactly the same time.

  Therefore, there is a possibility that the radar side detection magnification and the camera side detection magnification do not coincide with each other even though the radar and the camera firmly grasp the obstacle such as the preceding vehicle, and it is determined that the obstacle is not an obstacle. .

  The present invention avoids misrecognition (misrecognition that it is not an obstacle) based on a mismatch between the radar side detection magnification and the camera side detection magnification caused by the time difference between the detection of the radar and the camera as much as possible. The purpose is to improve the recognition accuracy of obstacles. Another object of the present invention is to reliably prevent erroneous recognition when the recognition target is obviously not a road surface vertical object.

In order to achieve the above-described object, the obstacle recognition method of the present invention repeatedly searches the front of the host vehicle with a radar and a camera provided in the vehicle, and includes a preceding vehicle ahead of the host vehicle based on the search result of the radar. Radar-side detection magnification is obtained from a change in distance measurement to an obstacle, etc., and each magnification of horizontal and vertical image histograms of the captured image of the camera in front of the vehicle by image processing of the imaging result of the camera in front of the vehicle The camera side detection magnification is obtained based on the minimum value of the residual sum with the reference image histogram, and the horizontal and vertical values at which the residual sum becomes the minimum value by the sensor fusion recognition process between the radar and the camera. It is determined whether or not the image of the obstacle area of the captured image is a road surface vertical object such as a preceding vehicle based on whether or not the magnification is equal, and the radar side detection magnification and the camera side detection magnification are not equal. In recognizing the obstacle recognition method the road vertical product as the obstacle, the horizontal, horizontal the residual sum of vertical image histogram is minimized value, on condition that vertical magnification is substantially equal, the radar side An error within a predetermined range between the detection magnification and the camera-side detection magnification is allowed to determine that the radar-side detection magnification and the camera-side detection magnification are equal, and the road surface vertical object is recognized as the obstacle. It is characterized (claim 1).

  The obstacle recognizing method according to the present invention is the obstacle recognizing method according to claim 1, wherein horizontal and vertical magnifications at which the residual sum is a minimum value are substantially equal, and both of the magnifications are on the radar side. The range of the error for determining that the camera-side detection magnification and the radar-side detection magnification are equal when the near-distance approach of the obstacle is detected to be larger than the detection magnification is increased (claim). Item 2).

  Further, according to the obstacle recognition method of the present invention, the radar side detection magnification is larger than both horizontal and vertical magnifications where the residual sum is a minimum value, and the radar side detection magnification and the residual are the same. When the difference between the horizontal and vertical magnifications at which the sum is minimum is greater than a set threshold value and there is a high possibility that the image of the obstacle area is not the road surface vertical object, the camera The range of the error for judging that the side detection magnification is equal to the radar side detection magnification is narrowed (Claim 3).

First, according to the configuration of claim 1 , the radar side detection magnification and the camera side detection magnification are set on condition that the horizontal and vertical magnifications at which the residual sum of the horizontal and vertical image histograms is the minimum value are substantially equal . Because the difference is allowed within a predetermined range and obstacles ahead of the vehicle are recognized, even if there is a time difference between the detection of the radar and the camera, the time difference is allowed and caused by the time difference between the detection of the radar and the camera Misrecognition (misrecognition that the object is not an obstacle) based on the mismatch between the radar side detection magnification and the camera side detection magnification can be avoided as much as possible.

  In this case, the difference between the radar side detection magnification and the camera side detection magnification is allowed within a predetermined range on condition that the magnification at which the residual sum of the horizontal and vertical image histograms is the minimum value is substantially equal. It is avoided as much as possible that an object that is not a road surface reflector such as a manhole cover or a cat's eye (鋲) is erroneously recognized as an obstacle by permission.

  Therefore, it is not an obstacle when the radar and the camera firmly grasp an obstacle such as a preceding vehicle without erroneously recognizing an obstacle that is not a road surface reflector such as a manhole cover on the road surface or a cat's eye (鋲). Can be prevented from being erroneously recognized, and obstacle recognition accuracy can be improved.

  Next, according to the configuration of the second aspect, when the obstacle approaching the detection time difference between the radar and the camera becomes larger, the horizontal and vertical magnifications on the latest camera side are both higher than the latest radar side detection magnification. Therefore, the error detection range for determining that the camera-side detection magnification and the radar-side detection magnification are equal is expanded, and the erroneous recognition due to the time lag between the detection of the radar and the camera is more reliably avoided. And the obstacle recognition accuracy can be further improved.

  Further, according to the configuration of claim 3, the latest radar side detection magnification is larger than both the latest camera side horizontal and vertical magnifications, and the radar side detection magnification and the camera side horizontal and vertical magnifications. If any of the differences is larger than the set threshold value and there is a high possibility that the recognition target is not a vertical road object, the error range is narrowed to the error of an obstacle. Recognition can be reliably prevented, and obstacle recognition accuracy can be further improved.

  Next, in order to describe the present invention in more detail, an embodiment thereof will be described in detail with reference to FIGS.

  FIG. 1 is a block diagram of an obstacle recognition device mounted on a vehicle (own vehicle) 1, FIG. 2 is a flowchart for explaining operations, FIG. 3 is a detailed flowchart of a part thereof, and FIG. 4 is a distance measurement of a laser radar 2. FIG. 5 is an explanatory diagram of edge image processing of a captured image of the monocular camera 3, and FIG. 6 is an explanatory diagram of an example of an edge histogram obtained from the edge image of FIG.

  FIG. 7 is an explanatory diagram of image processing, FIGS. 8 and 9 are schematic diagrams of traveling for explanation of magnification calculation, and schematic diagrams of camera photographing magnification. FIG. 10 is a horizontal and vertical residual sum (individual residual sum). ) And a characteristic diagram of one example of a residual sum (integrated residual sum) obtained by integrating them, FIG. 11 is a detailed flowchart of another part of FIG. 2, and FIG. 12 is a characteristic diagram of an example of a calculated magnification. It is.

  Further, FIG. 13 is an explanatory diagram of the influence of the detection time difference between the radar side detection magnification and the camera side detection magnification when the obstacle is at a relatively far distance in front of the host vehicle, and FIG. 14 is a diagram when the obstacle approaches. It is explanatory drawing of the influence of the detection time shift | offset | difference of a radar side detection magnification and a camera side detection magnification.

(Constitution)
The obstacle recognition apparatus of FIG. 1 uses a general-purpose scanning laser radar (hereinafter simply referred to as “laser radar”) 2 that is less expensive than a radio wave radar as a ranging radar that searches the vehicle 1 in front of the vehicle. A monocular camera 3 having a small and inexpensive monochrome CCD camera configuration is provided as a camera (image sensor) for photographing the front of the vehicle. Of course, a radio wave radar such as a millimeter wave radar may be provided instead of the laser radar 2.

  Then, after the engine of the own vehicle 1 is started by the ignition key, the laser radar 2 sweeps and irradiates the laser pulse and repeatedly searches the front of the own vehicle, and the distance measurement result signal in front of the own vehicle has a microcomputer configuration of the own vehicle 1. The monocular camera 3 continuously captures the front of the vehicle and outputs, for example, 8-bit luminance data signals per pixel of the captured image to the control ECU 4.

  The control ECU 4 forms a recognition calculation unit 6 as a recognition processing unit described later together with a memory unit 5 that holds various information (data) in a readable and writable manner, and performs a sensor fusion recognition process between the laser radar 2 and the monocular camera 3. Radar side detection by determining whether the image of the obstacle area in the captured image is a road surface vertical object such as a preceding vehicle based on whether the horizontal and vertical magnifications at which the residual sum (to be described later) is the minimum are equal. In order to recognize a road surface vertical object whose magnification and camera-side detection magnification are equal as an obstacle such as a preceding vehicle, the obstacle recognition program in steps S1 to S7 in FIG. 2 set in advance is executed based on the engine start. (In step S4, steps S41 to S45 in FIG. 3 are executed), and the following means (a) to (h) of the software configuration are provided.

(A) Candidate area setting means This means performs a clustering process on the latest distance measurement result of the laser radar 2 and sets an obstacle candidate area in front of the vehicle. Specifically, based on the distance measurement data of a large number of reflection points of the laser radar 2, the reflection points that are close to each other are clustered and shown in the screen view of the captured image P of the monocular camera 3 in FIG. As described above, one or a plurality of cluster regions surrounded by a frame line are formed in the search range of the laser radar 2, and a cluster region suitable for the feature of the obstacle is selected as a candidate region for an obstacle such as a preceding vehicle. Set to Cc.

  At this time, as shown in the schematic diagram of the radar ranging area in FIG. 4B, normally, an obstacle ahead of the host vehicle such as a preceding vehicle surrounded by a frame in the figure is captured by the candidate area Cc. .

(B) Relative Distance Measuring Means This means measures the relative distance Z every moment between the vehicle 1 and the candidate object α of the obstacle in the candidate area Cc from the distance measurement result of the radar search in the candidate area Cc.

(C) Histogram calculation means This means calculates the histogram of the horizontal and vertical binarized image edges of the candidate area Cc as the target feature quantity of the latest photographed image P of the monocular camera 3, and obtains the latest horizontal and vertical histograms. Calculated as edge histograms Y and X, and further stores previously stored horizontal and vertical binarized image edge histograms Ym and Xm stored as updateable candidate feature quantities for respective magnifications K and ( 0 <Kmin ≦ K ≦ Kmax, Kmin: minimum value, Kmax: maximum value), and the horizontal and vertical reference edge histograms Yr (= K · Ym), Xr ( = K · Xm).

  At this time, the horizontal and vertical latest edge histograms Y and X are obtained by differentiating and binarizing the latest captured image P of the camera in order to reduce the calculation processing load of the control ECU 4 and increase the speed. It is formed by adding the respective image edge information.

Specifically, for each frame in which the captured image P is obtained, the luminance data of 8 bits / pixel of the candidate area Cc of the latest captured image P is subjected to differential binarization processing in both horizontal and vertical directions, as shown in FIG. In addition, the 8-bit / pixel original image of the candidate area Cc is converted into the edge image shown in the figure (the gray part is the horizontal edge and the white part is the vertical edge), and the edge image is binarized to obstruct obstacles such as the preceding vehicle Image edge information of 1 bit / pixel in the horizontal and vertical directions is obtained with the candidate object α.

  Further, as shown in FIG. 6, the horizontal and vertical image edge information is added in the horizontal direction for the horizontal image edge and in the vertical direction for the vertical image edge, respectively, and the horizontal axis is in the vertical direction and the horizontal direction, respectively. The horizontal and vertical latest edge histograms Y and X having the pixel positions of are formed.

  Next, as shown in FIG. 7, histograms of already stored horizontal and vertical binarized image edges (stored edge histograms) Ym and Xm as candidate target feature values are obtained by using both histograms Ym and Xm respectively. In order to suppress the influence of sudden fluctuations in the reference edge histograms Yr and Xr of the horizontal and vertical magnifications K formed by multiplying the horizontal axis direction (position direction) by K, Formed by weighted average of vertical edge histograms.

  Incidentally, the magnification K and the relative distance Z have the following relationship.

  That is, as shown in the travel schematic diagram of FIG. 8, the relative distance Z of the preceding vehicle 7 as an obstacle becomes Z at time t and Z + ΔZ at time t + Δt, and the actual vehicle width in the photographing schematic diagram of FIG. If W is the vehicle width K · ω, ω at time t, t + Δt on the imaging surface of the monocular camera 3, the relative distances Z, Z + ΔZ are distances from the lens focal point of the monocular camera 3 to the preceding vehicle 7, respectively. When the distance from the lens focus of the monocular camera 3 to the CCD light receiving surface (image surface) is Cf, the following equations (1) and (2) are established, and the relationship between the relative distance Z and the magnification K is (3 ) Expression.

  Cf: ω = (Z + ΔZ): W Formula (1)

  Cf: K · ω = Z: W Formula (2)

  Z = ΔZ / (K−1) (3) Formula

  Kmin and Kmax are set as shown in the following equations (4) and (5). In these equations, Z is the relative distance, ΔZ_max is the maximum change amount of the relative distance Z (the change amount when the host vehicle 1 is at maximum acceleration and the preceding vehicle 7 is at the maximum deceleration state), and ΔZ_min is the minimum change amount of the relative distance Z. (The amount of change when the host vehicle 1 is at the maximum deceleration and the preceding vehicle 7 is at the maximum acceleration), and the maximum acceleration and the maximum deceleration are set based on experiments or the like.

  Kmax = 1 + ΔZ_max / Z Equation (4)

  Kmin = 1 + ΔZ_min / Z Equation (5)

(D) Residual sum calculation means This means converts the horizontal and vertical reference edge histograms Yr and Xr of each magnification K into, for example, the respective horizontal axes y and x in the predicted movement ranges My and Mx of FIG. The horizontal and vertical individual residual sums Vy and Vx consisting of the absolute value sum of the differences between the horizontal and vertical latest edge histograms Y and X and the reference edge histograms Yr and Xr for each magnification K, and both residuals The integrated residual sum V obtained by adding the difference sums Vy and Vx is calculated, and the minimum values Vy_min, Vx_min, and Vmin shown in (a), (b), and (c) of FIG. 10 are obtained.

  Note that Ky0, Kx0, and K0 in FIG. 10 are magnifications K that give the minimum values Vy_min, Vx_min, and Vmin.

  By the way, the horizontal and vertical reference edge histograms Yr and Xr of each magnification K are translated in the horizontal y and x directions within the predicted movement ranges My and Mx for the following reason.

  That is, if the own vehicle 1 does not have a pitching motion due to vibration, and if an accurate position can always be detected from the output of the laser radar 2 without a time delay, the reference edge histograms Yr and Xr need not be translated in parallel. Actually, it is difficult to make the distance measurement time of the laser radar 2 and the photographing time of the monocular camera 3 exactly coincide with each other. Further, since the own vehicle 1 pitches in the vertical direction due to vibrations or the like, To accurately detect the magnifications Ky0, Kx0, K0, etc. by absorbing the resulting coordinate axis shift between the laser radar 2 and the monocular camera 3, the reference edge histograms Yr, Xr are translated in a predicted range. This is because it is necessary to search for positions such as the magnifications Ky0 and Kx0.

  The predicted movement ranges My and Mx are set based on experiments and the like in consideration of the above-described pitching motion due to vibrations, and the reference edge histograms Yr and Xr have their respective horizontal axes y and y within the predicted movement ranges My and Mx. Translated in the x direction.

  The predicted movement range Mx is specifically determined by the pitching angle range of the host vehicle 1 and the relative distance Z, and varies depending on the relative distance Z. The predicted movement range My is the yawing angle range of the host vehicle 1; It is determined by the expected lateral movement range of the candidate object α and the relative distance Z, and changes depending on the relative distance Z.

  If the candidate area Cc does not mistakenly surround a road surface object (not a road surface vertical object) such as a manhole cover or fence, a road marking, etc., but if it surrounds a road vertical object such as a preceding vehicle, the radar The search plane and the camera imaging plane coincide, and at this time, the graph of the magnification characteristics of the residual sums Vy, Vx, and V has the hyperbolic shape shown in FIGS. 10A to 10C, that is, the minimum value. If the shape is a downwardly convex quadratic curve and the magnifications Ky0 and Kx0 are equal (Ky0 = Kx0), but if they accidentally enclose manhole covers, ridges, road markings, etc., the candidate area Cc is correct. In addition, since the radar search plane and the camera imaging plane do not belong to the same plane, the graph of the magnification characteristics of the residual sums Vy, Vx, V does not have the downward convex quadratic curve, and the magnification Ky0, Kx0 is Ky0 ≠ Kx0.

  Therefore, by this means, the minimum values Vy_min, Vx_min, Vmin of the residual sums Vy, Vx, V are obtained, and the magnifications Ky0, Kx0, K0c (= K0) at those times are detected.

  Actually, the values of the substantially minimum values Vy_min, Vx_min, and Vmin of the residual sums Vy, Vx, and V (the values of the error ranges set for the minimum values Vy_min, Vx_min, and Vmin) are set as the minimum values Vy_min, Vx_min, and Vmin. The magnifications at these values are detected as minimum values Vy_min, Vx_min, and magnifications Ky0, Kx0, and K0c (= K0) of Vmin.

(E) First object discriminating means This means forms road surface vertical object discriminating means according to claim 3, and at least magnifications Ky0 and Kx0 at which horizontal and vertical individual residual sums Vy and Vx become minimum values Vy_min and Vx_min. Are equal to each other (Ky0 = Kx0), it is determined whether or not the candidate object α is a road surface vertical object on the road surface.

  Actually, as described above, the values of the approximate sums Vy_min, Vx_min, Vmin of the residual sums Vy, Vx, V detected as the magnifications Ky0, Kx0, K0c (= K0) of the minimum values Vy_min, Vx_min, Vmin. Whether or not the candidate object α is a road surface vertical object on the road surface is determined based on whether or not the horizontal and vertical magnifications Ky0 and Kx0 are substantially equal.

(F) Magnification Detection Unit This unit detects the magnification K0 at which the integrated residual sum V becomes the minimum value Vmin as the obstacle camera side detection magnification K0c.

  If the roadside object is not an obstacle, the stored information and the observation information change greatly, and the minimum value Vmin of the integrated residual sum V becomes equal to or greater than a predetermined roadside object detection threshold Va. In order to prevent recognition errors and improve recognition accuracy, it is preferable to detect only the minimum value Vmin smaller than the roadside object threshold value Va as the camera-side detection magnification K0c.

(G) Second object discrimination means This means is based on whether or not the camera side detection magnification K0c is equal to the obstacle radar side detection magnification K0r calculated from the change in the relative distance Z (K0c = K0r). It is determined whether the road surface vertical object of the candidate object α is an effective detection object such as a preceding vehicle, or an invalid detection object such as splashing of the preceding vehicle or the like, or dust adhered to the laser radar 2.

  That is, if the candidate object α is a detection invalid object such as splashing of a preceding vehicle or the like, or attached dust of the laser radar 2, the candidate object α is detected from the search of the laser radar 2 even though no obstacle actually exists. Is detected as an obstacle, the radar side detection magnification K0r observed from the laser radar 2 and the camera side detection magnification K0c measured from the captured image do not have the same value (if K0c ≠ K0r), The vertical object of the candidate object α is determined to be a detection invalid object, and only when K0c = K0r, the candidate object α is determined to be a detection effective object to prevent erroneous recognition.

(H) Recognition processing means This means basically recognizes the horizontal and vertical individual by performing the sensor fusion recognition process between the laser radar 2 and the monocular camera 3 based on the discrimination between the vertical object on the road surface and the effective detection object. The image of the candidate area Cc (obstacle area) of the photographed image P is preceded by whether or not the magnifications Ky0 and Kx0 at which the residual sums Vy and Vx become the minimum values Vy_min and Vx_min are equal (Ky0 = Kx0). It is determined whether the object is a road surface vertical object such as a car, and a candidate object α for a road surface vertical object in which the radar side detection magnification and the camera side detection magnification are equal is recognized as an obstacle.

  That is, since Kx0 = Ky0 and K0c = K0r, it is determined that the distance measurement result based on the search of the laser radar 2 is correct, and the candidate area Cc is definitely an obstacle area, and the image of the candidate area Cc To recognize the candidate object α as an obstacle.

  However, by comparing the radar-side detection magnification K0r and the camera-side detection magnification K0c and determining whether or not the road surface vertical object is an obstacle based on the coincidence or mismatch, as described above, The detection method is different between the laser radar 2 and the monocular camera 3, and the detection of the laser radar 2 is delayed from the detection of the monocular camera 3. Therefore, when the obstacle approaches, it is actually a laser radar. Although the 2 and the monocular camera 3 firmly capture an obstacle such as a preceding vehicle, the radar-side detection magnification K0r and the camera-side detection magnification K0c are inconsistent, and there is a high possibility that the object is not an obstacle. Become.

  Therefore, the recognition processing means is such that the magnifications Ky0 and Kx0 at which the horizontal and vertical individual residual sums Vy and Vx become the minimum values Vy_min and Vx_min are substantially equal, and the candidate object α is a road surface vertical object (such as a preceding vehicle). An obstacle), it is determined that the radar-side detection magnification K0r and the camera-side detection magnification K0c are equal to each other by allowing an error within a predetermined range between the radar-side detection magnification K0r and the camera-side detection magnification K0c. It has a function of recognizing a vertical object as an obstacle such as a preceding vehicle.

  At this time, if the magnifications Ky0 and Kx0 are substantially equal, and both the magnifications Ky0 and Kx0 are both greater than the radar-side detection magnification K0r, the detection of the laser radar 2 is delayed from the detection of the monocular camera 3 and relatively close. Since the latest detection result of the monocular camera 3 is obtained closer to the obstacle coming from the detection result of the laser radar 2, it is determined that the obstacle is approaching a short distance, and the camera side detection magnification The error range for determining that K0c is equal to the radar-side detection magnification K0r is widened and expanded.

  Specifically, in the case of this embodiment, the image certainty factor GCF (%) of the following equation (6) is calculated, and this image reliability GCF is used as the obstacle recognition accuracy, so that A predetermined range is variably set, and an obstacle is recognized by allowing an error between the radar side detection magnification K0r and the camera side detection magnification K0c.

  At the same time, when there is a high possibility that the recognition target is obviously a road surface reflector such as a manhole or cat's eye (鋲) or a ghost target rather than a vertical object, the image confidence GCF is actively lowered to In particular, the predetermined range is made extremely narrow to reliably prevent erroneous recognition as an obstacle.

  GCF = 100− (Gx · | Kx−Kc | + Gy · | Ky−Kc | + Gxy · | Kx−Ky |) (6)

  In equation (6), Kx represents the camera side (image side) vertical magnification Kx0, Ky represents the camera side (image side) horizontal magnification Ky0, Kr represents the radar side detection magnification K0r, and Gx, Gy and Gxy represent coefficients.

  The closer the image certainty factor GCF is to 100 (%), the more likely it is to be “vehicle-like”, in other words, the “like road vertical object” that is an obstacle, and the closer the image certainty factor GCF is to 0, the manhole and the cats. The possibility of being a “road surface reflector” such as an eye (鋲) or a “ghost target” increases.

  By the way, the coefficients Gx, Gy, and Gxy are variably set as follows according to the magnitude relationship between the magnifications Kx, Ky, and Kxy by the processing in steps S41 to S45 in FIG.

  (A) Normally, Gx = m1, Gy = n1, and Gxy = mn1 are set based on the initial setting in step S41. Note that m1, n1, and mn1 are all constants set through experiments, for example, m1 = n1, and m1, n1 << mn1. The reason why Gxy is larger than Gx and Gy is a recognition target whose horizontal and vertical magnifications Kx and Ky do not match is a “road surface vertical object” such as a preceding vehicle even if Kx and Ky are larger than Kr. This is because the possibility is extremely low, and the reliability of recognition of an obstacle such as a preceding vehicle from the radar side detection magnification K0r and the camera side detection magnification K0c is lowered.

  (B) When the horizontal and vertical magnifications Ky and Kx match within the set error range and become larger than the radar-side detection magnification Kr, the recognition target (obstacle) is approaching as described above. Therefore, in order to allow the time delay of the laser radar side detection magnification K0r to be larger by equivalently widening the predetermined range, Kx> Kr, Ky> Kr, | Kx−Kr is performed in steps S42 and S43. | <Δa, | Ky−Kr | <Δb, | Kx−Ky | <Δc, Kr> Kr (near), Gx = m2 (<m1), Gy = n2 (<n1), Gxy = mn2 (<< mn1). Note that m2, n2, mn2, Δa, Δb, Δc, and Kr (near) are threshold constants set by experiments or the like. By setting the constants m2, n2, and mn2, the image certainty factor GCF is difficult to decrease.

  (C) On the other hand, the radar-side detection magnification Kr is larger than both the image-side magnifications Kx and Ky, (Kr> Kx, Kr> Ky), and the difference | Kr−Kx | or | Kr−Ky | However, if it is larger than the set threshold constant Δd, it is highly likely that the recognition target is not a “road surface vertical object” but a “road surface reflection object” or a “ghost target”. Gx = m3 (> m1), Gy = n3 (> n1), Gxy = mn3 (≧ mn1), and recognition of an obstacle such as a preceding vehicle from the radar side detection magnification K0r and the camera side detection magnification K0c The reliability is reduced, and equivalently, the predetermined range is extremely narrowed. Note that m3, n3, and mn3 are also constants set by experiments or the like.

(Operation)
Next, the operation of the recognition calculation unit 6 will be described mainly with reference to FIGS.

  After the engine of the host vehicle 1 is started, the candidate area setting means executes a clustering process according to step S1 of FIG. 2 to set a candidate area Cc which is regarded as an obstacle area such as the preceding vehicle 7 every moment, The relative distance measuring means measures the relative distance Z every moment.

  Further, at step S2 in FIG. 2, the histogram calculation means obtains image edge information by performing image differential binarization processing on the captured image P every moment (for each field), and in the subsequent calculation processing, 8 bits / pixel is obtained. The amount of information is reduced to 1/8 and the amount of calculation is reduced by using edge information of 1 bit / pixel instead of using the luminance value as it is.

  Then, in step S3 of FIG. 2, the histogram calculation means calculates the horizontal and vertical latest edge histograms Y and X and the horizontal and vertical reference edge histograms Yr and Xr of each magnification K for each field, and the clustering process described above. The feature amount (vector) on the image of the candidate target α for the region Cc set by is calculated.

  At this time, the magnification K changes in the range of the maximum value Kmax to the minimum value Kmin set based on the calculations of the above expressions (4) and (5), and the unit change width (calculation step width K_step). Is set to, for example, 0.04 in consideration of the processing load of the control ECU 4 and the like.

  Also, the latest vertical edge histogram X at times t, t + 1,... Is the current candidate target feature amount (vector) Xt, Xt + 1,..., And is held in the memory unit 5 at each time t, t + 1,. If the stored edge histogram Xm is the feature amounts (vectors) Pt, Pt + 1,... Up to the previous time, the feature amounts Pt, Pt + 1,... Take into account the temporal continuity (correlation) of the captured image P and the like. By calculating the weighted average of the vertical edge histograms of the existing frames that have been calculated previously, it is calculated while suppressing the influence of sudden fluctuations, etc., and is rewritten and held up-to-date so as to be updatable.

  Specifically, for example, the feature amount Pt + 1 of the stored edge histogram Xm at the time t + 1 is calculated by the weighted average of the following equation (7) based on the latest edge histogram Xt and the feature amount Pt at the time t until the time t + 1. Retained. H in the equation is a His coefficient set to an appropriate value of 0 <H <1.

  Pt + 1 = H × Xt + (1−H) × Pt (7)

  The horizontal stored edge histogram Ym is also maintained while being updated with a temporal hysteresis in the same manner as described above.

  The horizontal and vertical edge histograms (Yt, Xt), (Yt + 1, Xt + 1),... As the candidate target feature values at each time are stored as the horizontal and vertical stored edge histograms as the candidate target feature values. When it is significantly different from Pt (= Ymt, Xmt), Pt + 1 (= Ymt + 1, Xmt + 1),..., The candidate object α is recognized as being different from the previous one, or for some reason the feature on the image of the candidate object α is local Can be determined to have changed.

  Next, the residual sum calculation means calculates the residual sums Vy, Vx, V and the minimum values Vy_min, Vx_min, Vmin based on the candidate target feature amounts Pt, Pt + 1,.

  Specifically, this calculation process includes, for example, steps Q1 to Q11 in FIG. 11. In step Q1, the magnification K that is gradually increased from the initial value Kmin is set in the search range to Kmin ≦ K ≦ Kmax. In step Q2, the vertical individual residual sum Vx at the magnification K is calculated. In step Q3, the calculated individual residual sum Vx is set to the minimum value Vx_min set to the initial value at the start of calculation. If Vx <Vx_min, the process proceeds to step Q4, where the minimum value Vx_min is updated to the calculated individual residual sum Vx, and the vertical detection magnification Kx0 is updated to the magnification K.

  Further, in step Q5, the horizontal individual residual sum Vy at the magnification K is calculated, and in step Q6, is the calculated individual residual sum Vy smaller than the minimum value Vy_min set to the initial value at the start of the calculation? If Vy <Vy_min, the process proceeds to step Q7 where the minimum value Vy_min is updated to the calculated individual residual sum Vy, and the horizontal detection magnification Ky0 is also updated to that magnification K.

  Next, in step Q8, an integrated residual sum V (= Vx + Vy) is calculated based on the individual residual sums Vx and Vy calculated at the magnification K. In step Q9, the calculated integrated residual sum V is calculated. It is determined whether or not it is smaller than the minimum value Vmin set to the initial value, and if Vy <Vmin, the process proceeds to step Q10, where the minimum value Vmin is updated to the calculated integrated residual sum V, and the detection magnification K0 Is updated to the magnification K.

  In step Q11, the magnification K is stepped and gradually increased to K + K_step. In this state, the process returns to step Q1 and the process is repeated from step Q1, and in step Q3, Vx ≧ Vx_min is changed and individual residual sums are changed. When the minimum value Vx0 of Vx is detected, step Q4 is passed and the process proceeds to step Q5. Similarly, when the minimum value Vy0 of the individual residual sum Vy is detected by changing to Vy ≧ Vy_min in step Q6, step Q7 is detected. Is passed to step Q8.

  Further, when the minimum value V0 of the integrated residual sum V is detected by changing to V ≧ Vmin in step Q9, the residual sums Vy, Vx, V and the minimum values Vy_min, Vx_min, Vmin at that time are determined as the candidate target α. While detecting as a feature quantity, the magnifications Ky0, Kx0, and K0 are detected, and the process of FIG. 11 ends.

  In steps Q3, Q6, and Q9, the vertical and horizontal reference edge histograms Xr and Yr of each magnification K are translated in the horizontal axis x and y directions within the predicted movement ranges Mx and My, respectively. Each residual sum Vx, Vy, V is calculated.

  Then, based on the calculation results and the like, when the candidate region Cc is an appropriate region, for example, a quadratic curve characteristic shown in FIG. 12 is obtained as an actual measurement characteristic of the magnification K of the residual sums Vy, Vx, and V.

  When the processing of FIG. 11 is completed, the first target determination unit determines whether or not the magnifications Ky0 and Kx0 of the minimum values Vy_min and Vx_min obtained in step S3 are equal (Ky0 = Kx0). It is determined whether α is a road surface vertical object, and the magnification K0 obtained in step S3 is detected as the camera-side detection magnification K0c by the magnification detection means.

  Further, the magnification K obtained from the above equation (3) is detected as the radar-side detection magnification K0r based on the change of the relative distance Z from the previous time by the second object discrimination means, and whether or not K0c = K0r is satisfied. Then, it is determined whether the candidate object α is an effective detection object such as a preceding vehicle, or an invalid detection object such as water splashing on the preceding vehicle or adhering dust of the laser radar 2.

  Then, the process proceeds from step S3 to step S4, the image certainty factor GCF is calculated in steps 41 to 45 in FIG. 3, the process proceeds to step S5, and the road surface vertical object considering the image certainty factor GCF (Ky0 = Kx0) and detection. Based on the discrimination of the effective object (K0c = K0r), the recognition processing means detects that the candidate area Cc is an appropriate area, and determines the candidate object α from the image of the candidate area Cc as an obstacle (in FIG. )).

  Next, the process proceeds to step S6, and in this embodiment, in order to further improve the recognition accuracy, the next observation using the Kalman filter is performed on the candidate region Cc of the obstacle such as the preceding vehicle based on the current observation position. The position is predicted and held in the memory unit 5 in a rewritable manner. For example, in the next process of step S1, the validity of the candidate region Cc is examined from the error between the predicted position and the actually observed position. Is too large, the candidate area Cc is invalidated to make radar cluster detection robust.

  Further, in order to compare each feature quantity obtained this time with the feature quantity calculated next time, each feature quantity obtained this time is stored in the memory unit 5 in a rewritable manner, for example, calculated in step S3. The validity of the calculated feature value is examined based on a comparison between the stored feature value and the stored previous feature value. If the error is too large, the calculated feature value is invalidated to improve recognition accuracy. .

  Then, the process returns from step S6 to step S7 through step S7 to step S1 and step S2 until the recognition process is terminated by switching the control mode by the driver's operation or by stopping the engine of the host vehicle 1, and each step S1 to S7. This process is repeated to recognize obstacles such as the preceding vehicle 7 every moment.

  In this way, simple image calculation processing can prevent false recognition of obstacles due to road surface reflections such as cat's eyes and manholes, and ghost articles, and adaptation to changes (changes) in recognition targets It is possible to obtain the same effects as those of the above-mentioned applications such as no need for knowledge to open the vehicle image, no need to switch the algorithm day and night, and of course, the time lag between the detection of the laser radar 2 and the monocular camera 3 Even in such a case, the deviation is allowed on the condition that the horizontal and vertical magnifications Kx and Ky substantially coincide with each other, and the radar-side detection magnification K0r and the camera-side detection caused by the detection time lag between the laser radar 2 and the monocular camera 3 are detected. It is possible to avoid misrecognition that the object is not an obstacle based on the mismatch with the magnification K0c as much as possible, and in particular, an obstacle in which the time lag of detection between the laser radar 2 and the monocular camera 3 becomes larger. It can be very well avoid erroneous recognition when approaching.

  Further, as in the present embodiment, when an obstacle such as a preceding vehicle is recognized based on the image certainty factor GCF, as described above, the threshold constants of the calculation formulas for the image certainty factor GCF are set. Therefore, the recognition degree of obstacles such as a preceding vehicle can be easily improved.

  That is, for example, when the obstacle candidate target α in the candidate area Cc of the photographed image P appears relatively small away from the host vehicle as shown in FIG. 13A, the detection time of the laser radar 2 and the monocular camera 3 Even if there is a deviation, the influence of the deviation is small, and the error between the radar-side detection magnification K0r and the camera-side detection magnification K0c due to the detection time deviation between the laser radar 2 and the monocular camera 3 is small. ), The difference between the recognition accuracy from the radar side detection magnification K0r and the recognition accuracy from the camera side detection magnification K0c due to the time lag is small. However, as shown in FIG. When the obstacle candidate target α in the region Cc is relatively close to the host vehicle and is photographed largely, the influence of the time difference between the detection of the laser radar 2 and the monocular camera 3 is large, and the detection of the laser radar 2 and the monocular camera 3 is detected. The error between the radar-side detection magnification K0r and the camera-side detection magnification K0c due to the time lag of is increased from the recognition accuracy from the radar-side detection magnification K0r and the camera-side detection magnification K0c, as shown in FIG. The difference due to the time lag with respect to the recognition accuracy increases. Note that lr, lx, and ly in FIGS. 13B and 14B indicate characteristics of recognition accuracy from the radar-side detection magnification K0r and recognition accuracy from the camera-side magnifications K0x and Ky0.

  In such a case, the radar-side detection magnification K0r is allowed by allowing the radar-side detection magnification K0r and the camera-side detection magnification K0c and the error caused by the time lag between the detections of the laser radar 2 and the monocular camera 3 under predetermined conditions. Recognition as non-obstacle based on discrepancy between camera and camera-side detection magnification K0c as much as possible without increasing the mis-recognition of obstacles due to cat-eye (ゴ ー), manhole, etc. In particular, misrecognition at the time of the approach of an obstacle in which the time lag of detection between the laser radar 2 and the monocular camera 3 becomes larger can be avoided very well.

  In addition, the term “Gxy · | Kx−Ky |” in equation (6) further reliably prevents erroneous recognition of a manhole cover, a cat's eye (鋲), or the like on the road surface as an obstacle.

  Furthermore, when there is a high possibility that the recognition target is not a road surface vertical object, Gx and Gy in the equation (6) are increased to positively lower the image certainty factor GCF, and conversely, the error range is narrowed. Thus, the recognition accuracy of the obstacle can be further improved by reliably preventing the erroneous recognition of the obstacle.

  Therefore, it is erroneous that the laser radar 2 and the monocular camera 3 are not obstacles when the obstacle such as the preceding vehicle is firmly recognized without erroneously recognizing the manhole cover or cat's eye (鋲) on the road surface as an obstacle. Occurrence of a situation of being recognized can be prevented, and obstacle recognition accuracy can be significantly improved.

  Then, for example, by controlling the automatic brake system of the vehicle 1 based on the recognition result, a significant reduction in traffic accidents can be expected.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit thereof. For example, a radar as a distance measuring sensor Instead of a scanning laser radar, a similar scanning-type inexpensive ultrasonic radar may be used. In some cases, a radio wave radar such as a millimeter wave radar may be used. Moreover, the camera as an imaging sensor is not limited to a monocular camera.

  Furthermore, the obstacle recognition processing program of the control ECU 4 may be different from that shown in FIGS. 2, 3, and 10, and the horizontal and vertical image histograms Ym and Xm are the reference image histograms for the respective magnifications. Any method may be used for calculating the camera side detection magnification K0c based on the minimum value of the residual sum of Yr and Xr, and within a predetermined range between the radar side detection magnification K0r and the camera side detection magnification K0c. The method for allowing the error is not limited to the method for calculating the image certainty factor GCF. For example, the method using the certainty factor in which the term Gxy · | Kx−Ky | in the equation (6) is omitted. There may be a method of actually setting an allowable threshold value appropriately.

  By the way, in order to reduce the number of equipped parts of the own vehicle 1, the present invention can be applied to the case where the laser radar 2 and the monocular camera 3 of FIG. 1 are also used as sensors for other controls such as follow-up running control and brake control. it can.

It is a block diagram of one embodiment of this invention. It is a flowchart for operation | movement description of FIG. 3 is a detailed flowchart of a part of FIG. 2. It is explanatory drawing of the candidate area | region detection based on the radar ranging result of FIG. It is explanatory drawing of the edge image process of the camera picked-up image of FIG. It is explanatory drawing of an example of the edge histogram obtained from the edge image of FIG. It is explanatory drawing of the image processing of FIG. It is a driving | running | working schematic diagram for magnification calculation description of FIG. It is a schematic diagram for camera side detection magnification description of FIG. It is a characteristic view of an example of the individual residual sum of FIG. 1, and an integrated residual sum. 3 is a detailed flowchart of another part of FIG. 2. FIG. 2 is a characteristic diagram of an example of a calculated magnification of FIG. 1. It is explanatory drawing of the influence of the detection time shift | offset | difference of a radar side detection magnification and a camera side detection magnification when an obstruction exists in the distance comparatively far ahead of the own vehicle. It is explanatory drawing of the influence of the detection time shift | offset | difference of a radar side detection magnification and a camera side detection magnification when an obstacle approaches.

Explanation of symbols

1 Vehicle 2 Scanning laser radar 3 Monocular camera 4 Control ECU
5 Memory unit 6 Recognition calculation unit

Claims (3)

Repeated exploration in front of the vehicle with the radar and camera installed in the vehicle,
Obtain the radar side detection magnification from the change in distance measurement distance to obstacles including the preceding vehicle ahead of the vehicle based on the radar search results,
The camera-side detection magnification is obtained based on the minimum value of the residual sum with the reference image histogram of each magnification of the photographed image of the camera front of the camera in front of the camera of the camera,
According to the sensor fusion recognition process between the radar and the camera, the image of the obstacle area in the photographed image is the preceding vehicle or the like based on whether the horizontal and vertical magnifications at which the residual sum becomes a minimum value are equal. In the obstacle recognition method for recognizing the road surface vertical object having the radar side detection magnification equal to the camera side detection magnification as the obstacle,
Allow an error within a predetermined range between the radar-side detection magnification and the camera-side detection magnification, provided that the horizontal and vertical magnifications at which the residual sum of the horizontal and vertical image histograms is the minimum value are substantially equal. Then, it is determined that the radar side detection magnification is equal to the camera side detection magnification, and the obstacle vertical recognition method is recognized as the obstacle. The obstacle recognition method according to claim 1,
When the horizontal and vertical magnifications at which the residual sum is minimum are substantially equal, both magnifications are greater than the radar-side detection magnification, and the near-side approach of the obstacle is detected. An obstacle recognition method characterized by enlarging the range of the error for judging that the magnification and the radar side detection magnification are equal.   The radar-side detection magnification is larger than either the horizontal or vertical magnification at which the residual sum is a minimum value, and the radar-side detection magnification and the residual sum are at a minimum value. When any of the differences is larger than the set threshold value and there is a high possibility that the image of the obstacle area is not the road surface vertical object, the camera side detection magnification and the radar side detection magnification are equal. The obstacle recognition method according to claim 1, wherein the range of the error to be determined is narrowed.