JP4791086B2 - Collision avoidance system - Google Patents

Description

The present invention is a collision avoidance system provided in a vehicle and capable of avoiding a collision with at least one of another vehicle and an obstacle .

      The incidence of traffic accidents in Japan is still high, and there is a high percentage of collisions with vehicles ahead due to driver's carelessness. As a countermeasure, collision avoidance systems using various sensors have been proposed.

As an ITS system that can be applied at intersections where traffic accidents occur most frequently, a device has been patented that can quickly recognize the presence of a partner vehicle even at intersections with poor visibility. (
reference) Vehicle driving support system Japanese Patent Application Laid-Open No. 2005-35497

In the collision avoidance system, it is possible to recognize the predicted trajectory of the own vehicle and the danger of a collision with another vehicle at an early stage by detecting the light beam irradiated forward. in this case,
The light beam is not affected by noise and requires a stable detection rate. Further, even if the shape of the light beam point sequence changes due to obstacles or road surface irregularities due to depressions, the ID of each light beam must be recognized. In addition, the light beams of the own vehicle and other vehicles have the same wavelength band, but these should be detected separately from the own vehicle and other vehicles in consideration of the overall shape of the light beam.

          However, an apparatus for removing the influence of the perspective difference and noise when noise occurs in the collision avoidance system has not been studied. In addition, since it has not been studied to determine the ID of the light beam, it is difficult to automatically recognize the change in the beam point sequence due to the unevenness or the intersection of the light beam with another vehicle.

      Therefore, the present invention solves the above problems, accurately detects the light beam irradiated on the road surface, accurately detects changes in the overall shape of the beam point sequence due to obstacles ahead, and It is an object of the present invention to provide a collision avoidance system using an image sensor and a laser, to which a function of automatically recognizing the danger of a vehicle collision by detecting the intersection of light beam point sequences is added.

In order to solve the above problems, the collision avoidance system of the present invention is provided on the vehicle, among other vehicle and the obstacle, a collision avoidance system that avoid collision with at least one, the running direction of the vehicle A laser projection apparatus for projecting a laser dot sequence composed of a plurality of laser points onto the ground in a shape in which a line segment connecting the plurality of laser points forms a substantially trapezoidal shape by irradiating the ground on the side with a laser And an image processing device that images the laser point sequence, and an image processing device that acquires the coordinates of the plurality of laser points based on an image captured by the image capturing device, and is captured by the image capturing device. Further, when the laser point sequence is defined as a detection laser point sequence and the laser point sequence when the ground is a plane is defined as an estimated laser point sequence, the image processing apparatus is configured to detect the detection level. DP matching is performed by associating a plurality of detection laser points constituting the laser point sequence with a plurality of estimated laser points constituting the estimated laser point sequence, and the DP matching is formed by the detection laser point sequence. Coordinates of corners on both sides of the upper base of the substantially trapezoid, coordinates of corners on one side of the upper base of the substantially trapezoid formed by the detection laser point sequence, or the upper base of the substantially trapezoid formed by the estimated laser point sequence Among the coordinates of the corners on one side of each of the reference points, the distances and directions of the plurality of detection laser points with respect to the reference points, the distances of the plurality of estimated laser points with respect to the reference points, and By using each direction as the DP matching element, the plurality of detection laser points and the plurality of estimated laser points are associated with each other. Further, the image processing apparatus, of the detection laser points constituting the detection laser point sequence, if it can not obtain the coordinates of at least one point, the coordinates of the estimated laser point corresponding to the one point to the coordinates of the single point And a region surrounded by the coordinates of the plurality of detection laser points or the coordinates of the plurality of detection laser points to which the coordinates of the estimated laser points are applied is specified as a vehicle region when the vehicle travels in the traveling direction. And determining that there is a risk of collision with another vehicle when a laser point that is not similar to the estimated laser point sequence is detected in the vehicle region, and the amount of time-series change in the coordinates of each detected laser point It is characterized in that it is determined that an obstacle exists when the value exceeds a threshold value.

The forward collision avoidance system according to the image processing acquires an image erasing light beam other light beams images simultaneously, it is possible to remove noise other than light beams by comparing the two images It is possible to stably recognize the situation ahead of the vehicle.

The forward collision avoidance system according to image processing by performing a calculation using the characteristic DP paths in the DP matching, it detects a missing-insertion of the light beam, recognize exactly the state of the light beam point sequence It is possible.

The forward collision avoidance system according to image processing is to automatically recognize the photographed image through a special engineering filter drivers do not need to check the filter images visually.

According to the present invention, it is possible to detect a collision even if the machine approaches from any angle by vigilance of the drawing area of the laser dot sequence, and it is possible to realize a general-purpose collision avoidance system compared to the conventional technology. . In addition, collisions at intersections that were considered problematic in traffic accidents
The effect can be expected in all scenes with a high accident rate, such as a collision with the vehicle ahead. Japanese traffic accidents occur at intersections, accounting for 46.5%, and are considered effective because they have a direct effect on the most problematic situations.

In scenes such as intersections, lane changes, overtaking, and rear-end collisions due to carelessness, it is possible to prevent intersections and lane changes, overtaking, and rear-end collisions by preventing intrusion of obstacles and other vehicles in the point-description area. Accidents can be prevented and a safe transportation society can be realized. As a driver's advantage, it is possible to confirm the expected trajectory of the vehicle traveling in the actual scene, so that a sense of security can be obtained. Since it can be judged whether it can pass a narrow road and where the own vehicle is running in the lane, it is thought to lead to improvement of driving technology.
In the present invention, a monocular camera and a laser are combined to accurately obstruct the obstacle.
It is possible to estimate the distance, the size of the obstacle, etc.

          The road surface condition can be determined by the shape of the laser. By controlling the drive unit according to the road surface condition, it will lead to comfortable driving support for the driver.

        The present invention will be described below with reference to the drawings. FIG. 1 is a prototype vehicle showing a configuration of an embodiment of the present invention, and shows a case where one camera used as a distance measuring device and a special laser device are used. In FIG. 1, reference numeral 1 denotes an electronic camera, which is set to face the front of the host vehicle, and is set in a fixed state as specified by the camera installation position and installation angle. Image acquisition is performed by acquiring two images through a special filter. One is an image photographed with emphasis on the vicinity of the laser frequency band, and the other is an image photographed by cutting the laser frequency band. Since this image is obtained by synchronizing the shutter timing, it is acquired at the same time. Reference numeral 2 denotes a laser projection device that irradiates the front of the vehicle, and is installed facing the front at the front of the vehicle. The prototype vehicle uses a green YAG laser that projects an elliptical point sequence laser. In an embodiment, the laser is a visible laser. The laser is used with its output reduced to the extent that it does not affect the human body in the daily environment. Even if the laser of 2 is not a YAG laser, as long as it can shoot with the camera of 1 even in a daytime environment, the laser device can constitute the present invention. A system control device 3 sends a stop command to the brake ECU 4 based on the result of image processing. 4 provides vehicle data (driver operation amount and vehicle state amount) to 3, and automatically activates a brake in response to an instruction from 3. Reference numeral 5 denotes an image processing apparatus which processes two images obtained from 1 to grasp the situation ahead of the vehicle. When a danger or road surface condition is detected ahead, a warning is given to the driver and a control signal is sent to 3. FIG. 2 shows only the device portion of FIG.

        Hereinafter, first, a number of calculation means and methods used in this embodiment will be described, and then the overall calculation flow will be described based on a flowchart. FIG. 3 is an example of a laser point sequence irradiated by a laser projection apparatus (corresponding to the laser projection apparatus 2). Equation 1 represents the world coordinates (x, y) of the laser projection position in consideration of the installation state of the laser projection apparatus. 3 shows an expression for converting from camera coordinates (x ′, y ′). The laser shape changes depending on the turning angle of the steering wheel and the vehicle speed, and draws a vehicle area where the own vehicle exists about 1 second later. In addition, when there is an obstacle ahead, the laser shape changes arbitrarily.

Formula 1

        The laser point sequence projected from the laser device is photographed with a special camera (corresponding to 1 in FIG. 1). A simplified diagram of the laser acquisition format is shown in FIG. As shown in FIG. 4, one of the images is photographed through a filter that enhances the frequency band of the laser (hereinafter referred to as a laser enhanced image) so that the laser can be acquired even in a daytime high luminance state. On the other hand, imaging is performed through a filter that erases the frequency band of the laser (hereinafter referred to as a laser erased image), and only the laser projected by the laser apparatus of the present invention is obtained by taking the difference from the laser enhanced image. The shutter timing of the camera is synchronized, and an image at the same time can be acquired.

      First, a method for detecting the laser shape from the image will be described.

        The flow of the laser detection process is shown in FIG. The laser-enhanced image is relatively dark because the specific image is taken only. Therefore, the laser shape can be clearly captured by taking the difference from the laser erased image after performing the feature extraction process. In the embodiment, first, image normalization is performed for the purpose of increasing the contrast of the image. Since the laser-enhanced image often has a bias in the luminance of the image, normalization is performed using an equation that takes that tendency into account.

      Normalization is expressed by Equation 2 when changing the shade range of an image from [c, d] to [a, b] where Xin is input luminance and Xout is output luminance. The average Ave / standard deviation σ of the histogram is obtained, the range of Ave ± 2σ is set as the normalization range, the luminance below it is set to 0, and the luminance above it is set to 255. Since the normalization range covers 90% or more of the entire pixel, the contrast of the image can be increased efficiently. Image normalization is similarly performed for laser-erased images.

Formula 2

        Next, consideration is given to the overlay of images. Due to vibration during running, the arrangement in the apparatus is displaced, and there is a concern that the laser-enhanced image and the laser-erased image cannot capture the same scene. Therefore, in the embodiment, a feature point extraction filter is applied to both images, and a feature conversion matrix for optimally superimposing the images on one side is obtained by associating the feature points of both images. In the embodiment, image transformation is performed on the laser-erased image using this matrix, and the laser-enhanced image is superimposed in an optimum state.

Next, a smoothing filter is applied to the image. On roads with many cracks or rough surfaces, many noises may be detected in addition to the laser. Therefore,
A smoothing filter is applied to the image to roughly change the luminance of the image. By this processing, fine noise in the image can be eliminated.

Next, edge detection of both images is performed. As a result, it is possible to extract the outline of the laser having a characteristic shape photographed in the image. In the edge image of the laser-enhanced image, it is conceivable to detect white lines and characters on the road, asphalt cracks, etc. in addition to the shape of the laser. In order to erase edges other than the laser shape, the edge of the laser erased image is used. Since edges other than the laser shape are extracted from the edges of the laser erased image, only the laser shape can be extracted by taking the difference between these edge images. However, due to the deviation of the image coordinates due to the vibration of the vehicle body or the like, there may be a case where something other than the laser shape remains even if the difference is taken. In order to remove edge noise due to such a slight shift, the edge region of the laser erased image is expanded. Thereby, even if a slight shift occurs, an edge such as a white line can be erased. (Hereinafter, an image obtained by this processing is called a difference image.)
The laser shape is detected from the difference image. As a detection method, template matching or the like can be considered as a method considering the characteristics of the laser. As shown in FIG. 6, a shape similar to the template is searched from the difference image. As another method, it is possible to make a determination in consideration of the contour of the shape, the size of the laser, the luminance distribution, and the like. In this way, the laser is detected from the laser enhanced image and the laser erased image. Note that the present invention is less dependent on the illumination conditions because the laser can be detected in a high ambient luminance such as daytime or at night.

By automatically recognizing the state of this in-vehicle laser with an in-vehicle camera, vehicle danger can be avoided. As an example, if there is an obstacle in front of the vehicle, the laser depicted on the road surface will appear distorted on the on-board camera, and this will be automatically recognized by the image processing system of the present invention to notify the driver of the danger and brake control. And so on. In addition, if the vehicle's laser and the other vehicle's laser intersect, there is a risk of a vehicle collision. Therefore, the driver will be notified and the brakes and steering wheel will be controlled to perform the danger avoidance operation. FIG. 7 shows images of laser shapes at the time of obstacle collision and when approaching another vehicle.

        After detecting the laser, the vehicle area needs to be detected. The reason and detection method are shown below.

        In the present invention, a collision at an intersection or the like with a vehicle equipped with a similar system can be detected in advance. In the collision determination, it is necessary to detect the intersection of the own vehicle laser and the other vehicle laser. Therefore, it is important to always clearly distinguish the own vehicle area. Hereinafter, the detection method of the own vehicle area will be described.

        In the laser projector, when irradiating a laser, the image coordinates of the irradiated laser are calculated. In the embodiment, since the laser is irradiated with a sequence of 20 points, coordinates for 20 points can be obtained. However, since these coordinates are image coordinates when the front is a plane, this image is hereinafter referred to as estimated coordinates. It can be determined that the laser point sequence having coordinates and arrangement similar to the estimated coordinates is the own vehicle laser, and the others are the other vehicle lasers.

        In order to obtain the own vehicle area, it is necessary to associate the estimated laser coordinates with the detected coordinates. Therefore, DP matching is used in the embodiment. DP matching is a technique for associating a reference with an input. In the present invention, the reference is an estimated coordinate and the input is a detected coordinate. What is important here is the setting of elements and DP paths used for DP matching. In the present invention, “distance and direction from the reference point” is used as a DP matching element. First of all, it is necessary to obtain a reference point as a procedure. The overall shape of the laser obtained from the image shows a trapezoidal shape. A characteristic part in this point sequence is around the corner of the upper trapezoid. Thus, the trapezoidal corner is searched from the detected coordinates in comparison with the estimated coordinate arrangement. This is shown in FIG. Pay attention to the distance and angle between points as similar elements. Laser detection points detected by this processing are set as trapezoidal corners, that is, reference points.

        In a natural luminance environment or a state where the laser is hidden by an obstacle (occlusion), the angle may not be detected by the angle detection. Normally, the left and right corners can be detected, but conditions are determined in preparation for a state where only one side can be detected or when both corners cannot be detected. When both corners are detected, DP matching is performed separately on the right and left sides of the laser shape with both corners as reference points. When corners can be detected on only one side, DP matching is performed on the entire laser shape using only one side as a reference point. If both corners cannot be detected, one side of the estimated point is set as a reference point, and calculation is performed on the entire laser shape. Hereinafter, a detection point detected by corner detection is referred to as a reference point.

        The distance and direction from the detected reference point are determined for each detection laser. The elements used are shown in FIG. If the distance from the reference point in the x direction and y direction is dx, dy, the angle is θ, the distance from the reference point of the estimation point is dx ', dy', and the angle is θ ', the detection point and the estimation point The distance D is calculated as shown in Equation 3.

Formula 3

        Calculate D at all points of the detected and estimated points. In Expression 3, the value calculated using the detection point i and the estimation point j is set to Dij, and Dij is calculated for all patterns to create a matrix of i rows and j columns.

DP matching is inherently elastic and often uses a DP path as shown in FIG. However, since an estimated point and a detected point are associated with each other so that overall correspondence is established, a general path is not suitable for the present invention. However, in the DP of the present invention, a DP path as shown in FIG. 11 is used in order to make the one-to-one correspondence relationship between the estimation point and the detection point valuable. Thereby, the correspondence between the estimated point and the detected point can be performed almost perfectly. Fig. 11
The global distance is calculated using the DP path as shown above, and the estimated coordinates are associated with the detected coordinates by back-tracing.

        If the estimated points and the detected points are associated by DP matching, it is possible to recognize which laser is in what state. For example, when a laser is depicted on a white line, the laser shape may not be detected, and the laser may be lost, causing a false detection of the laser shape. However, if the position of the point that is lost is understood, the entire shape of the laser can be maintained by supplementing that portion with the estimated coordinates.

        In addition, since the estimated coordinates and the detected coordinates can be compared and used for obstacle detection, it is effective to associate the estimated coordinates with the detected coordinates by DP matching.

        Next, obstacle detection will be described. Obstacles ahead are detected by observing time-series changes in the coordinate changes of the vehicle laser. The flow of processing is shown in FIG. FIG. 12A shows the flow of obstacle detection. The laser ID of the vehicle is accurately detected by DP matching, and changes in the past several frames are observed to determine the time series change of each ID laser. The change is observed in the horizontal line part of FIG. The average (Ave) and standard deviation (σ) are obtained. As for the calculated change, the past several frames are stored in the ring buffer of FIG. 12 (c) so that the past frame change can always be calculated. When the image coordinate change is seen from Ave and σ and a large change exceeding the allowable value of the laser change is shown by (d) in FIG. 12, it is determined as an obstacle. It is also possible to determine the unevenness according to the moving direction. In addition, it is possible to determine how many meters away from the vehicle body the obstacle is based on the changed laser ID. Further, the size of the obstacle can be estimated from the amount of change.

      When the other vehicle laser is detected inside the own vehicle area, risk avoidance control is applied because there is a risk of collision with another vehicle. The own vehicle region can be clarified by DP matching, and when an unexpected laser is detected in the region, it is determined as a laser of another vehicle.

    Depending on the position of the laser, etc., the brake strength can be controlled to avoid accidents.

      The above-described invention can be used as a system for avoiding a vehicle collision as an ITS system. Also, it can be attached to other moving objects and used as various obstacle recognition and collision avoidance systems.

Prototype vehicle equipped with a laser projector Laser projector internal configuration diagram Laser shape examples Laser enhanced image and laser erased image Difference image acquisition process flow Laser detection by template matching Laser shape at the time of obstacle collision and crossing Corner detection Element extraction and local distance definition for DP matching Commonly used symmetric DP path DP path used in the present invention Obstacle detection algorithm

Claims (1)

Provided in the vehicle, among other vehicle and the obstacle, a collision avoidance system that avoid collision with at least one,
By irradiating the ground on the traveling direction side of the vehicle with a laser, a laser dot sequence composed of a plurality of laser points is formed on the ground in a shape in which a segment connecting the plurality of laser points forms a substantially trapezoidal shape. A laser projection device for projecting;
An imaging device for imaging the laser dot sequence;
An image processing device that acquires each coordinate of the plurality of laser points based on an image photographed by the photographing device;
When the laser point sequence photographed by the imaging device is defined as a detection laser point sequence, and the laser point sequence when the ground is a plane is defined as an estimated laser point sequence,
In the image processing device, DP matching is performed in which a plurality of detection laser points constituting the detection laser point sequence and a plurality of estimation laser points constituting the estimation laser point sequence are associated with each other,
In the DP matching,
Coordinates of corners on both sides of the upper base of the substantially trapezoid formed by the detection laser point sequence, coordinates of corners on one side of the upper base of the substantially trapezoid formed by the detection laser point sequence, or the estimated laser point sequence Among the coordinates of the corners on one side of the upper base of the substantially trapezoid formed by
By using each distance and each direction of the plurality of detection laser points with respect to the reference point and each distance and each direction of the plurality of estimated laser points with respect to the reference point as elements of the DP matching, the plurality of detection lasers A point and each of the plurality of estimated laser points ,
Further, in the image processing apparatus, when at least one coordinate of the detection laser points constituting the detection laser point sequence cannot be acquired, the coordinate of the estimated laser point corresponding to the one point is set as the coordinate of the one point. Apply,
A region surrounded by the coordinates of the plurality of detection laser points or the coordinates of the plurality of detection laser points to which the coordinates of the estimated laser points are applied is specified as a vehicle region when the vehicle travels in the traveling direction,
When a laser point that is not similar to the estimated laser point sequence is detected in the vehicle area, it is determined that there is a risk of collision with another vehicle,
It is determined that there is an obstacle when the time series change amount of the coordinates of each detection laser point exceeds a threshold value ,
A collision avoidance system characterized by that.