JP2005156199A - Vehicle detection method and vehicle detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely specify a position and a direction ranging over from a vehicle in the vicinity thereof to a vehicle in the long distance, by a single eye camera or combination of the single eye camera and a radar. <P>SOLUTION: This method/detector uses a voting system provided with an imaging means for acquiring an image of a front side of own vehicle, an edge extraction means for processing the image acquired by the imaging means to extract a brightness change portion, a voting means for executing voting on a voting space of a two-dimensional coordinate system, based on information of an edge point, and a peak detecting means for detecting a peak of a voting value in the voting space. Inexpensive realization is possible thereby because the vehicle is detected by the single eye camera, and various types of vehicles are detected because the edges of vehicular both ends are used and because not using a pattern of vehicular back face. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a vehicle detection method and a vehicle detection device using image processing.

  There are many methods for detecting the position and direction of a vehicle based on an image obtained from an in-vehicle camera. One is a method of detecting an edge and analyzing the edge, and the other is a method of performing pattern matching. Among them, a number of methods using edges have been proposed because they can follow changes in the size of the vehicle and can detect the left and right ends of the vehicle. In particular, the stereo camera system detects parallax by analyzing edges. For example, a vehicle detection method that detects a pair of vertical edges with two cameras and estimates that the parallax of the pair of vertical edges is at both ends of the vehicle when the parallax of the pair of vertical edges is almost equal. Separation is relatively easy and reliability is high. However, the vehicle detection method requires two cameras, which leads to an increase in weight and cost. Also, calibrating the two cameras leads to an increase in production costs.

  Therefore, many vehicle detection methods for monocular cameras have been proposed. For example, in a lane, a horizontal edge dense area is used as a vehicle candidate, and in an adjacent lane, the movement of a vertical edge dense area is analyzed to determine whether it is a vehicle candidate. Has been proposed (for example, Patent Citation 1). However, the situation where it can be detected that the horizontal edges are dense is limited to the case where the preceding vehicle is at a short distance, and if the vehicle becomes small in the distance, the horizontal edges do not always appear dense due to the resolution. . Further, since the rear part of the vehicle becomes black with shadows during backlighting, the horizontal edge is hardly observed.

  In addition, after setting the initial vehicle edge contour model, after that, a vehicle detection method for matching with the model, a horizontal edge present in the shadow part of the preceding vehicle in the traveling lane of the own vehicle, There has been proposed a vehicle detection method in which a pair of vertical edges representing both ends of a preceding vehicle is obtained and then continuously calculated based on the position of the horizontal edge and the width of the vertical edge at both ends. In either case, the initial vehicle detection is performed accurately, and the effect is exhibited. However, if the initial vehicle detection fails, it leads to a subsequent erroneous recognition. A detection method has been proposed in which vehicle candidates are obtained from features in which peaks appear at the upper and lower ends and left and right ends of a vehicle from projection distributions of the vertical and horizontal axes of edges. In the above method, there is a difference in how the peaks appear depending on how the windows are set. If a noise area other than the vehicle is included, a peak appears in the other part and the analysis becomes complicated. Therefore, it is necessary to set a window that does not include an area other than the vehicle as much as possible. However, it is difficult to set an optimal window when the vehicle position is unknown.

JP 2002-334330 A

  Provided is a system capable of specifying from a nearby vehicle to a distant vehicle with high accuracy by using a monocular camera or a combination of a monocular camera and a radar.

  A vehicle characterized by detecting another vehicle traveling in front of the own vehicle using a voting method for voting in a voting space of a two-dimensional coordinate system based on an image obtained by photographing the front of the own vehicle. The above problem is solved by using a detection method.

  This vehicle detection method can be realized at low cost because the vehicle detection is performed with a monocular camera, and various types of vehicles can be detected because the vehicle is detected using edges at both ends of the vehicle.

  FIG. 1 is a processing flow of a vehicle detection method according to the present invention. In the present embodiment, the vehicle front image acquisition step (S1) for inputting an image from a camera installed in the vehicle, and the luminance change portion of the input image input in the vehicle front image acquisition step S1 as an edge, the position of the edge , An edge detection step (S2) for detecting the intensity and direction components, and whether or not there is a vertical edge from the direction component of the edge point, and if there is a vertical edge, search for whether there is another vertical edge in the horizontal direction. The edge pair search step (S3), the vertical edge pair presence / absence determination step (S4) for determining whether a vertical edge pair is found and changing the processing, and if the vertical edge pair is found, the center position is determined from the respective x coordinates of the vertical edge pair. xw space voting step (S5) for calculating x and width w and voting to xw space, and peak coordinate detection for detecting peak coordinates in xw space A step (S6), the vehicle is detected by the vehicle information calculating step of calculating the distance direction information from xw information peak coordinates to the vehicle (S7).

A configuration example of the inter-vehicle distance control system (hereinafter referred to as ACC) of the present embodiment is shown in FIG. In the vehicle detection unit 301, the vehicle front image obtained from the camera 304 directed forward in the traveling direction of the vehicle is transferred to the image processing unit 305. The image processing means 305 recognizes the position of the preceding vehicle on the image by the vehicle detection method of the present invention, calculates the positional relationship between the host vehicle and the preceding vehicle, and transfers it to the ACC control unit 302. The ACC control unit 302 instructs the ACC execution unit 303 to perform accelerator control by the accelerator control unit 306 and brake control by the brake control unit 307 as necessary so as to keep the inter-vehicle distance constant from the positional relationship with the preceding vehicle. Send a signal to control the speed of the vehicle. Hereinafter, the vehicle detection method will be described in detail according to FIG.

  Details of each step in the vehicle detection method will be described with reference to FIG.

  First, in the vehicle front image acquisition step S <b> 1, the vehicle front image captured by the camera 304 installed so as to capture the front of the vehicle is transferred to the image processing unit 305. The vehicle front image is a grayscale image as shown in FIG. 2A, and FIG. 2A is a picture showing an example in which the tail portion of the preceding vehicle 201 and the background are shown. The processing from the edge detection step S2 to the vehicle information calculation step S7 after the vehicle front image acquisition step S1 is performed by the image processing means 305.

  In the edge detection step S2, the luminance change portion of the input image stored in the image memory (not shown) of the image processing means 305 is extracted as an edge, and is calculated as data with the edge direction information added thereto. The data includes position information x, y and direction information θ for each edge point. Here, the position information is a value on the screen, the unit is a pixel, and the direction information θ represents the edge direction on the screen, and takes a value of 0 to 360 degrees. In this embodiment, the edge direction is defined to be inclined 90 degrees to the right with respect to the high luminance direction as indicated by the arrow in FIG. The reason for tilting 90 degrees with respect to the direction of high brightness is to match the direction of the edge with the direction of the contour line, and the tilting direction may be left, but it is unified to either.

  In the vertical edge pair search step S3 to the xw space voting step S5, the vehicle width and center position candidates are obtained using the edge information detected in the edge detection step S2. Since the brightness difference from the background, that is, the vertical edge tends to appear on the left and right of the preceding vehicle, attention is paid to the vertical edge. Since the vertical edges at the left and right ends of the preceding vehicle often appear in left and right pairs, they are considered as a set of left and right information.

  In the vertical edge pair search step S3, attention is paid only to the vertical edge, and a pair of vertical edges arranged in the horizontal direction is searched. In the example of FIG. 2, the two arrows surrounded by the detection area 202 of FIG. FIG. 2D is an enlarged view of the detection area 202 portion of FIG. The direction of the arrow represents the direction of the edge. In the case of edges at both ends of the preceding vehicle, there is a high possibility that the direction of the edges is the opposite direction as shown in FIG. Therefore, by ignoring vertical edge pairs in the same direction on the left and right sides and searching for pairs of vertical edges having opposite directions, it is possible to select as few combinations as possible that are not edge pairs of both ends of the vehicle.

  In the vertical edge pair presence / absence determination step S4, it is determined whether the vertical edge pair still remains in the image. If the search has not been completed, the process goes to the xw space voting step S5. If all the searches have been completed, the peak coordinate detection is performed. The process is switched to step S6.

  In the xw space voting step S5, the center position xp and the width wp are calculated as shown in FIG. 2D from the respective x coordinates of the vertical edge pair searched in the vertical edge pair searching step S3, and are shown in FIG. Vote on coordinates (xp, wp) 203 on xw sky. Voting on the coordinates (xp, wp) 203 on the xw space means adding an arbitrary value (for example, 1) to the value of the coordinates (xp, wp) 203 on the xw space. It is assumed that the xw space is secured in the memory in advance before the vertical edge pair search step S3, and all coordinates are cleared to zero and initialized. That is, a two-dimensional frequency distribution of (x, w) calculated from the vertical edge pair is taken. As shown in FIG. 2D, when the left coordinate of the vertical edge is (x1, y1) and the right coordinate is (x2, y1), xp and wp are calculated by the following equations.

  Here, x2> x1. Since the preceding vehicle on the image is captured from behind and can be approximated to a rectangle, the vertical edge pair at both ends of the preceding vehicle converges to a value in the vicinity of (xp, wp) even if the y coordinate changes. That is, by repeating the voting of the vertical edge pairs at both ends of the preceding vehicle, the value of the coordinate (xp, wp) 203 or its neighboring coordinates increases, and a value peak appears near the coordinate (xp, wp) 203.

  In the present embodiment, the horizontal axis of the voting space is defined as the vehicle center position x, and the vertical axis is defined as the vehicle width w. However, it may be defined as long as the vehicle center position and the vehicle width are finally obtained. . For example, the voting space may be formed with the horizontal axis as the x coordinate on the left side of the vertical edge and the vertical axis as the x coordinate on the right side of the vertical edge, and the vehicle center position and the vehicle width may be calculated after the peak position is obtained. As described above, the vertical axis and the horizontal axis of the voting space may be defined in any way as long as the vehicle center position and the vehicle width are finally obtained.

  In the peak coordinate detection step S6, a peak coordinate in the xw space is searched to obtain a combination (xp, wp) of the vehicle center position and the vehicle width. Here, the pair of vertical edges is not limited to both ends of the preceding vehicle, but may be noise from the number plate, tail lamp, guardrail, utility pole, etc. of the preceding vehicle, and voting is performed accordingly. In addition, since there may be a plurality of preceding vehicles in the adjacent lane or the like, a plurality of peaks actually appear. Therefore, it must be determined which peak is that of the preceding vehicle.

  One of the determination methods is a method using image processing. Since the left and right ends of the area can be calculated backward from the detected peak, it is determined whether or not the preceding vehicle seems to be within the calculated area. For example, since a shadow is formed at the lower end of the vehicle, the information is used to search for a horizontal line in which horizontal edges are arranged in the area, or search for a black horizontal line that is sufficiently lower than the average density. It can be determined whether the vehicle is a vehicle.

  Another method for determining whether the vehicle is a vehicle is to perform sensor fusion using radar information. That is, the radar 401 for detecting the distance, direction, etc. to the preceding vehicle is incorporated as shown in FIG. The radar 401 only needs to be able to detect the distance and direction, and may be any of millimeter wave radar, laser radar, and the like. When radar information is used, the range for searching for peaks in the xw space can be limited from the distance and direction information obtained by the radar 401, so that noise peaks can be efficiently eliminated.

  Hereinafter, a method for limiting the range for searching for a peak in the xw space from the distance and direction information obtained by the radar 401 will be described. The vehicle center position information x and the vehicle width information w are values on the image. If the actual vehicle width (for example, assumed to be 1.7 m) and camera parameters (CCD size, focal length, etc.) are known, x, The distance and direction from w to the actual vehicle can be determined uniquely. That is, the xw information and the actual distance direction information are in one-to-one correspondence and can be converted into each other. That is, since it is possible to predict where the peak appears in the xw space from the radar information, if the distance and direction information of the radar information is converted into xw and the peak is searched only in the vicinity thereof. The noise peak can be eliminated from the beginning.

Here, the merit of sensor fusion viewed from the radar 401 side will be described. One of the problems of the radar 401 is the poor lateral accuracy. This is because, since the preceding vehicle has a width, it cannot be determined in which part of the preceding vehicle the reflected wave is reflected. However, in the case of this vehicle detection method, since both ends of the vehicle are detected on the image, the lateral accuracy is high, and the merit of fusion is great. In addition, there is a problem that the radar 401 may erroneously detect a reflected wave such as a guardrail. In this regard, when distance and direction information, which is false detection information, is converted to xw and a peak is searched for only in the vicinity thereof, the possibility that a peak exists is low because it is not a vehicle. Therefore, it is possible to determine that something other than the vehicle is erroneous detection.

  In the vehicle information calculation step S7, the peak coordinate value xw detected in the peak coordinate detection step S6 is converted into actual distance direction information, and the process ends. As described above, when the actual vehicle width (for example, assumed to be 1.7 m) and camera parameters (CCD size, focal length, etc.) are known, the distance and direction from x and w to the actual vehicle are unique. I want to.

  Next, an edge detection method will be described. There are various methods for detecting only the position of the edge. Examples of the calculation method including the edge direction component include a method using a Sobel filter and a two-dimensional zero-cross method. In this embodiment, a method using a 3 × 3 Sobel filter will be described.

  In the 3 × 3 Sobel filter, the edge intensity is calculated by multiplying the nine pixel values of the top, bottom, left, and right centered on a target pixel by the coefficients shown in FIG. 5 and summing the results. To do. This process is performed using a vertical coefficient matrix 402 and a horizontal coefficient matrix 403.

  When the total value in the vertical direction is Gx and the total value in the horizontal direction is Gy, the pixel value G of the target pixel is expressed by Equation (3).

  This pixel value G represents the intensity of the edge, and the larger the value, the greater the difference in brightness near that point. If the value of G is larger than a preset edge extraction threshold, the pixel in that portion is extracted as an edge point. In order to prevent extraction omission, it is necessary to reduce the edge extraction threshold sufficiently, but if it is too low, the number of edges will become enormous, so it is necessary to set it to an appropriate value in consideration of processing time and misrecognition. is there.

  The edge direction θ is obtained by the following equation (4) from the respective direction components Gx and Gy of the edge strength.

  As described above, the direction θ represents the edge direction on the screen, and takes a value of 0 degree or more and less than 360 degree. The edge direction is converted into a direction shifted to the right by 90 degrees with respect to the direction with high luminance.

It is a processing flow of the vehicle detection method which is one Example of this invention. It is a flowchart of the image processing of the vehicle detection method which is one Example of this invention. It is a system block diagram of ACC which is one Example of this invention. It is a system block diagram of ACC which is the other Example of this invention. 3 is a coefficient matrix of a 3 × 3 Sobel filter.

Explanation of symbols

201 ... preceding vehicle, 202 ... detection area, 203 ... coordinate (xp, wp), 301 ... vehicle detection unit, 302 ... ACC control unit, 303 ... ACC execution unit, 304 ... camera, 305 ... image processing means, 306 ... accelerator Control means, 307 ... brake control means, 401 ... radar.

Claims (5)

Take an image of the front of the vehicle with the imaging means,
Extract the brightness change part of the image taken by the edge point extraction means as an edge point,
Voting is performed on a voting space in a two-dimensional coordinate system based on the edge point information by voting means,
Detecting a peak of the voting value in the voting space by a peak detecting means;
A vehicle detection method, wherein the coordinates at the peak are detected as a vehicle. The vehicle detection method according to claim 1,
Information for voting on the voting space is information obtained from horizontal positions of two vertical edges positioned in a horizontal direction on an image. The vehicle detection method according to claim 1 or 2,
Ranging means for detecting the distance and direction to the preceding vehicle, and based on the distance and direction data measured by the distance measuring means, the peak detecting means detects a peak of the voting value in the voting space. The vehicle detection method characterized by specifying.   A vehicle detection method for detecting another vehicle traveling in front of the host vehicle by a voting method for voting in a voting space of a two-dimensional coordinate system based on an image of the front of the host vehicle. . An imaging means for acquiring an image ahead of the host vehicle, an edge point extracting means for processing the image acquired by the imaging means and extracting a luminance change portion;
A vehicle comprising: voting means for voting on a voting space of a two-dimensional coordinate system based on the edge point information; and a peak detecting means for detecting a peak of the voting value in the voting space. Detection device.

Images