JP2002175534A - Method for detecting road white line - Google Patents

Abstract

PROBLEM TO BE SOLVED: To securely detect a position where a white line is actually present by algorithm which is as simple as possible and inexpensive equipment constitution. SOLUTION: Luminance space differential values of pixels in an image photographed by an on-vehicle camera are computed and the edge of the white line is detected according to the positions having the extreme values thereof, and pixels having close luminance values of the detected edge are selected as white candidates 1 and 2, which are grouped according to their position relation to detect the white line. Consequently, the pixel positions where the luminance space differential values (difference between adjacent luminance values) show the extreme values, are adopted, thus the edge can stably be detected without being affected by luminance values and contrast. A part which varies in luminance owing to a shadow, a blur, etc., is not forcibly detected together, but detected as a white line candidate having become narrow to break and then integrated according to the position relation (direction and position) among white line candidates, and thus the white line can be detected without being affected by disturbance even if the white line is a broken line or a blurred line.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention analyzes a forward image captured by an on-vehicle camera and analyzes various types of road lane boundaries (white lines, yellow lines, solid lines, broken lines, etc.). The present invention relates to a road white line detection method for recognizing a white line collectively in the specification.

[0002]

2. Description of the Related Art In realizing a driving support system for an automobile, recognition of a road environment using images is expected. As one of road environment recognitions, there is automatic white line recognition of a driving lane, and various methods have been developed. Oike "Road White Line Recognition by Model-Based Recognition Method" (IEICE Technical Report)
PRMU99-211 (January 2000)) proposes white line recognition using a string model that does not require binarization processing.

[0003] Ninomiya, Takahashi, and Ota, "High-speed pattern matching method and application to lane detection" (3rd next-generation runway display / recognition technology workshop material, Shoken 3-4-1, November 1999) We propose a method to perform road structure collation by applying a template.

[0004]

The former "model-based ‥‥" does not require binarization and simplifies the hardware, but uses a continuous active contour model. Cannot be distinguished, and two models are defined so as to converge on the left and right white lines, respectively, and it is not assumed that the model crosses the white line.

The latter "high-speed pattern matching method ‥‥"
Although it is resistant to fading white lines and bad weather, and its computational cost is low, it has a drawback that it requires a large number of road shape templates and requires a huge amount of memory. Therefore, the present invention can be realized with the simplest algorithm and inexpensive device configuration as much as possible, can reliably detect the position where the white line actually exists, and can detect the white line on the road which can be applied to advanced vehicle control in the future. The aim is to realize the method.

[0006]

A method for detecting a white line on a road according to the present invention calculates a luminance spatial differential value of a pixel in an image photographed by a vehicle-mounted camera, and determines a white line based on a position indicating the extreme value. Edges are extracted, the detected edges having similar luminance values are grouped as white line candidates, the grouped white line candidates are grouped based on their positional relationship, and one white line is separated into left and right sides separately. This is a method for detecting (claim 1).

According to the above method, the edge is detected by
Since the pixel position at which the luminance space differential value (difference between adjacent luminance values) indicates an extreme value is adopted without using a commonly used binarization method, the pixel position is not affected by the luminance value or the contrast. Thus, the edge can be stably detected. Then, by detecting those with similar luminance values of the detected edges as white line candidates, the portions having different luminances due to the influence of shadows, blurring, etc. are detected as thin white line candidates without being forcibly detected together. I do.

Furthermore, since the integration is performed based on the positional relationship (direction and position) of these white line candidates, white lines can be detected even with broken lines or faint white lines without being affected by disturbance. In particular,
Unlike the technique of detecting white lines by applying to a template,
Since the detection is performed based on the actual white line position, the white line position can be correctly recognized. Finally, one white line is extracted for each of the left and right sides. When obtaining the extremum, it is preferable to determine the edge extraction range using the average value and the standard deviation of the luminance space differential values (claim 2).

As a result, the edge extraction range is not fixed, but is determined dynamically for each image, so that the influence of disturbances (road surface stains, ruts, etc.) mainly present on the road surface is eliminated as much as possible. Can be. In addition, it becomes strong (robust) against a change in brightness. It is preferable to use a lateral shift direction and a joint angle between the white line candidates as a reference when grouping the white line candidates. Since it integrates using the shape features (information on the image plane) drawn on the white line road surface,
Less susceptible to road and background disturbances.

The procedure for detecting one white line separately for the left and right is as follows:
Based on the Hough transformation parameter, it is preferable to obtain the position similarity of the integrated group between the images before and after temporally continuous and extract one white line group having a large similarity (claim 4). As a result, one continuous white line can be determined. If the similarity is small, it can be determined that the white line has been lost. In addition, since the Hough transform parameter is employed, a simple same evaluation expression can be used regardless of how the white line looks, and uniformity of parameter sensitivity can be secured. (For example, if a and b of the primary expression y = ax + b are adopted as parameters, the weights of the parameters must be changed in the vicinity of a = ∞ and in the vicinity of a = 0.) It is preferable to set an overlap area at the center of the range on the image (claim 5).

Thus, when changing lanes, the transition from the left white line to the right white line or vice versa can be smoothly performed.

[0012]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. 1. Definition of Image Data (Refer to FIG. 1) The coordinate system of an image captured by the on-vehicle camera has an origin (0, 0) at the upper left, an x axis to the right, and a y axis to the lower. The image size is 640 pixels horizontally (x direction) and 48 pixels vertically (y direction).
0 pixels.

The pixel value is represented by 256 levels of 8 bits, 0 to 255. 0 is black and 255 is white. The pixel value at the position (x, y) is described as Im [y] [x]. FIG. 2 is an overall flowchart showing the white line detection processing. This process is performed every time one image is captured. Capture of images is performed 30 sheets per second (step S1). Hereinafter, each step will be described separately. 2. Preprocessing 2.1 Reduction Sampling is performed at every other pixel in both the x and y directions.
As a result, the processing amount is reduced, and the processing time can be shortened.

2.2 Smoothing Smoothing is performed by convolving the pixel values with the following operators over the entire image.

[0015]

(Equation 1)

3. Edge detection The range where the white line exists is limited to the lower half area of the screen. The following processing is executed only for the lower half area of the screen. 3.1 Calculation of Pixel Edge Intensity Based on the spatial derivative (difference), the edge intensity (Intensity) at the position (x, y) is defined as follows, and the edge is detected using this.

Edge strength = (Im [y] [x + 3] + Im [y] [x + 2] −
Im [y] [x-2] -Im [y] [x-3]) / 10 A space representing the edge strength is called an “edge space”. In addition,
This equation is an equation equivalent to the one-dimensional Prewitt type spatial derivative (-1 0 1) obtained by averaging five pixels around the target point. 3.2 Edge Detection An edge is detected by selecting a pixel showing an extreme value in the edge space. The selected edges each have a signed value of ±, and are grouped in the subsequent processing based on the values (described later).

However, in order to eliminate the influence of road surface noise and the like, the invention is devised so that the range for extremum search can be limited. Specifically, the average value of the edge strength of the below-screen judgment,
The standard deviation is calculated, and the outside of the following range is set as the extreme value search target range. Upper limit: Average value of edge intensity + 1.5 × standard deviation Lower limit: Average value of edge intensity−1.5 × standard deviation Assuming that the edge intensity distribution in the edge space follows a normal distribution, the above range includes approximately all pixels. 86.6% are included, and as a result, an extreme value pixel is extracted from about 6.7% of pixels included in the upper extreme value search target range and the lower extreme value search target range, respectively. Become. This can speed up the processing.

4. Extraction of White Line Candidates The edges extracted in the preceding section are grouped according to the following procedure to extract white line candidates (a portion of a single white line). (1) The lowermost edge of the image is registered as the starting point G0 of the white line candidate. (2) An edge near the white line candidate start point G0 and having an intensity similar to that of the white line candidate start point G0 is registered as a new white line candidate G. Hereinafter, this process is repeated. In this way, a white line candidate is constituted by the white line candidate start point G0 and the white line candidate G registered in relation thereto.

Neighboring pixels are searched from a line up to three levels up. If it exceeds three lines, it is registered as another white line start point. The search range in the horizontal direction is (distance to search line) × (± 3 pixels). For example, when searching for the second line without an edge on one line, ± 6 pixels are searched. In terms of the expression, if the position is on two lines with reference to the (x, y) position, the search is performed from (x-6, y-2) to (x + 6, y-2).

The intensity similarity determination is performed by determining the white line candidate start point G0.
Is performed within ± 50% of the edge strength. 5. Grouping white line candidates The white line candidates extracted in the previous section are grouped according to the following procedure. (1) For a white line candidate having a certain number of constituent pixels or more, a left white line and a right white line are selected. Note that the search areas for the left and right white lines are overlapped so that the left and right white lines can be followed as much as possible when changing lanes where the existing positions of the left and right white lines are switched.

Specifically, the lower limit of the number of pixels constituting the white line candidate is three. If the x-coordinate of the white line candidate start point G0 is equal to or less than 160 + α and the edge strength is negative, it is determined that the x-coordinate of the white line candidate start point G0 is 161-α or more.
If the edge strength is positive, it is determined to be the left end of the right white line. Here, the coordinate "160" is set at a half position on the assumption that the image size after reduction in the x direction is 640/2 = 320 pixels.

Α is a value in consideration of the overlap.
Approximately 10% (32 pixels) of the image size in the x direction. (2) White line candidates are grouped in order from the bottom (larger y coordinate). The conditions are as follows: (a) the relationship between the straight line extending the lower reference white line candidate and the starting point of the white line candidate to be inspected is within the coupling range, and (b) the difference between the slopes of the two lines. θ is within a predetermined angle range.

As shown in FIG. 3A, the "joining range" is a line segment L drawn in the horizontal direction from the starting point G0 of the white line candidate 2 to be inspected and a reference below. Assuming that a point at which the straight line extending the white line candidate intersects is P, the length G0
P is a range of ± a predetermined number of pixels. Where P is G0
Positive when it is further to the right and negative when P is to the left of G0. In FIG. 3A, the predetermined number is 10. The “predetermined angle range” is, as shown in FIG. 3B, when the point P is to the right of the starting point G0, −5 ° to + 20 °, and as shown in FIG. -20 ° when on the left
To + 5 °.

6. Three-dimensional coordinate conversion The starting point G0 of the white line group is three-dimensionally converted into real space coordinates using the following equation on the screen. Z (vehicle travel direction distance) = Dividend (η, sinψ, cosco) /
Divider (η, sinψ, cosψ) X (vehicle lateral distance) = Conv2X (ξ, Z) Conv2X (ξ, Z) = (ξ / F) [(Z cosψ) + (H sin
ψ)] Divider (η, sinψ, cosψ) = sinψ− (η / F) cosψ Dividend (η, sinψ, cosψ) = H [cosψ + (η / F) sin
ψ] H: Camera installation height (m) F: Lens focal length (m) ψ: Depression angle (rad) of camera optical axis ξ, η represents distance (m) from the center of the imaging surface (origin) ( The y-axis is positive upward and the x-axis is positive rightward).

FIGS. 4A and 4B are diagrams showing images before and after the three-dimensional coordinate conversion. FIG. 4A is a diagram showing an edge image captured by a camera, and FIG. 4B is a diagram showing an image after the three-dimensional coordinate conversion. . 7. Extraction of the most reliable left and right white lines From the grouped white line candidates, the left and right white lines are extracted one by one. FIG. 5 is a diagram showing a plurality of white line groups in the real space. FIG. 6 is a flowchart for explaining the procedure in this section.

(1) White line candidate starting point G0 in the case of three-dimensional coordinate conversion is within ± 4 m in the horizontal direction and within 30 m in front (see the thick frame in FIG. 5). Are extracted with the maximum length L1, L2,... (Step S66). (2) On the screen before the three-dimensional coordinate conversion, a white line candidate having a certain number of pixels (for example, 5) or more is selected and a Hough conversion parameter is calculated (step S6).
7).

Here, the Hough transform means transforming a straight line xcosθ + ysinθ = r expressed in an xy orthogonal coordinate system into an (r, θ) plane. The origin of the polar coordinates is the upper left of the screen as shown in FIG. 7A for the left white line candidate, and the upper right of the screen as shown in FIG. 7B for the right white line candidate. The Hough transform parameters are r and θ. The dotted line is a white line extracted from the screen shot last time, and the solid line is a white line extracted from the screen shot this time. In FIG. 7, two white lines having different acquisition times are simultaneously drawn.

(3) Based on the Hough transform parameters, the similarity with the previously detected white line candidate is obtained, and the one with the highest similarity is separately extracted as the current white line candidate (step S62). To determine the similarity, a determination value 100 | θ0−θ1 | + | r0−r1 | is adopted. r0 and θ0 are Hough transformation parameters of the previous white line candidate, and r1 and θ1 are Hough transformation parameters of the current white line candidate. "100" is a weighting factor (constant)
It is. The smaller this determination value is, the higher the similarity is.

If the determination value exceeds 30, it is determined that the position of the white line has been lost (NO in step S63). In this case, the process jumps to step S66, and the optimum candidate is searched again by the method (1). When the position of the white line is lost in FIG. 6, the process may not proceed to step S66, but may discard the Hough transform parameter and start from step S66 in the next cycle.

The Hough transform parameters of the left and right white lines thus extracted are updated (step S64), and the left and right white lines are specified (step S65). 8. Road Shape / White Line Position Estimation The following is a procedure for converting white line position information into data. FIG.
This will be described with reference to FIG. (1) The left and right white lines representing the left and right white lines are regarded as circles, and the center and radius R are calculated.

(2) Estimation of Left and Right White Line Positions The coordinates of two points on the near side of the left and right white lines are extended linearly, and the left and right white line positions S immediately below the camera are estimated.

[0033]

As described above, according to the road white line detection method of the present invention, the position where the white line actually exists can be reliably detected with the simplest algorithm and the cheapest equipment configuration. Therefore, it is possible to provide a road white line detection method optimal for future advanced vehicle traveling control.

[Brief description of the drawings]

FIG. 1 is a definition diagram of image data.

FIG. 2 is an overall flowchart illustrating a white line detection process.

3A and 3B are diagrams for explaining integration conditions of white line candidates, and FIG. 3A is a diagram illustrating a positional relationship between a white line candidate serving as a reference below and a white line candidate to be inspected; ) And (c)
FIG. 3 is a diagram showing an angle relationship between both straight lines.

FIG. 4 is a diagram showing images before and after three-dimensional coordinate conversion;
(a) is a diagram showing an edge image captured by a camera, and (b) is a diagram showing an image after three-dimensional coordinate conversion.

FIG. 5 is a diagram illustrating a frame for extracting an initial value of a white line candidate when three-dimensional coordinate conversion is performed.

FIG. 6 is a flowchart illustrating a procedure for extracting left and right white lines one by one.

7A and 7B are diagrams for explaining the Hough transformation parameters r and θ, wherein FIG. 7A shows a case of a left white line candidate, and FIG. 7B shows a case of a right white line candidate.

FIG. 8 is a diagram showing a procedure for converting white line position information into data.

[Explanation of symbols]

 S Position of left and right white line immediately below camera R Radius of left and right white line r, θ Hough transformation parameter L1, L2 Length of white line candidate when three-dimensional coordinate transformation is performed

Claims (5)

[Claims] 1. A method for calculating a luminance spatial differential value of a pixel in an image photographed by a vehicle-mounted camera, extracting an edge of a white line based on a position indicating the extreme value, and detecting a luminance value of the detected edge is close. A method for detecting white lines on a road, comprising: combining the white line candidates as white line candidates; grouping the collected white line candidates based on their positional relationship; and detecting one white line separately for the left and right sides. 2. The method for detecting white lines on a road according to claim 1, wherein an edge extraction range is determined using an average value and a standard deviation of the luminance space differential values when obtaining the extremum. 3. A criterion for grouping white lines,
2. The road white line detection method according to claim 1, wherein a lateral shift direction and a joint angle between the white line candidates are used. 4. The procedure for detecting one white line separately for the left and right sides is as follows.
2. The white line detection of a road according to claim 1, wherein a position similarity of an integrated group is obtained between images before and after the temporally continuous image based on the Hough transformation parameter, and a white line group having a large similarity is selected. Method. 5. The road white line detection method according to claim 1, wherein an overlap area is set at the center of the range on the image for obtaining lines as left and right white line candidates.