JP2005157723A - Traffic lane recognition device and traffic lane recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To increase opportunity to recognize a white line without deteriorating recognition accuracy of the white line. <P>SOLUTION: When a luminance change between adjacent pixels among images picked up by a camera 14 picking up an image in front of one's own vehicle 12 is a first prescribed value or above, a first white line candidate point is extracted, and a first white line candidate line on which a largest number of the first white line candidate points are placed is extracted from straight lines each including the first white line candidate point, and a second prescribed value smaller than the first prescribed value is set when the number of the first white line candidate points included in the first white line candidate line is less than the prescribed value. When the luminance change between the adjacent pixels among the images is the second prescribed value or above, a second white line candidate point is extracted, and a second white line candidate line on which a largest number of second white line candidate points are placed is extracted from straight lines each including the second white line candidate point. A straight line on which a largest number of the first white line candidate points are placed is extracted from straight lines having the same inclination as the second white line candidate line, as a third white line candidate line, and it is recognized as the white line. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a lane recognition device and a lane recognition method that recognize a lane by detecting a white line on a road ahead of the host vehicle.

2. Description of the Related Art Conventionally, a technique for detecting a white line on a road on which the host vehicle travels from an image captured by an electronic camera provided on the host vehicle and recognizing the lane (traveling path) of the host vehicle is known.
For example, in the prior art disclosed in Patent Document 1 below, a window is set in an image captured by a camera, a plurality of white line candidate points are extracted based on luminance changes in the window, and the number of extracted white line candidate points is the largest. The included straight line is recognized as a white line.

JP-A-7-78234

  However, in the prior art in Patent Document 1, it is determined that a white line could not be recognized when the number of extracted white line candidate points is less than a predetermined value. For this reason, there are fewer opportunities to recognize white lines, and there is a problem in that white line recognition accuracy is low.

  An object of the present invention is to provide a lane recognition device and a lane recognition method that can increase the chance of recognizing a white line without reducing the recognition accuracy of the white line.

  In order to solve the above-described problem, the present invention provides a first white line when the luminance change of an adjacent pixel in an image captured by an image acquisition unit that captures the front of the host vehicle is greater than or equal to a first predetermined value. First white line candidate point extracting means for extracting candidate points, and a first white line candidate line on which most first white line candidate points are placed among straight lines including the extracted first white line candidate points White line candidate line extraction means and a second predetermined value smaller than the first predetermined value when the number of first white line candidate points included in the extracted first white line candidate line is less than a predetermined value And second white line candidate point extracting means for extracting a second white line candidate point when the luminance change of an adjacent pixel in the image is equal to or greater than a second predetermined value. And the second white line candidate point is the largest among the extracted straight lines including the second white line candidate point. A second white line candidate line extracting means for extracting the second white line candidate line and a third white line candidate line based on the second white line candidate line and the first white line candidate point or the first white line candidate line. And white line recognition means for recognizing the third white line candidate line as a white line.

  According to the present invention, the chance of recognizing a white line can be increased without reducing the recognition accuracy of the white line.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings described below, components having the same function are denoted by the same reference numerals, and repeated description thereof is omitted.
FIG. 2 is a diagram showing a vehicle equipped with a lane recognition device according to an embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of the lane recognition device according to the present embodiment.
In FIG. 2, 12 is the own vehicle, 10 is the lane recognition device (road white line recognition device) of the present embodiment, 14 is an electronic camera, 16 is a processor constituting the main part of the lane recognition device 10, and 12a is the own vehicle. An interior ceiling of the vehicle 12, 12b is a windshield of the host vehicle 12, and R is a road.
As shown in FIG. 2, a camera 14 that acquires a road image ahead of the host vehicle 12 and a processor 16 that processes an image signal acquired by the camera 14 (hereinafter simply referred to as an image) are installed in the host vehicle 12. Yes. The camera 14 is preferably a charge coupled device (CCD) camera or a complementary metal-oxide semiconductor (CMOS) camera capable of high-sensitivity imaging. The camera 14 is installed in the front center portion of the indoor ceiling 12a of the host vehicle 12 so as to face the front lower side, and acquires an image of the road R in front of the host vehicle 12 through the windshield 12b. The processor 16 is installed at an appropriate location of the host vehicle 12 that is not affected by heat or wind and rain, recognizes the white line of the road R from the image of the road R acquired by the camera 14, and forms the road R and the host vehicle with respect to the road R. Control for estimating 12 postures is executed.

In FIG. 1, 15 is an image memory for storing an image of a road R ahead of the host vehicle 12 acquired by the camera 14 mounted on the host vehicle 12, 101 is a window setting unit, 102 is a white line candidate point extracting unit, and 103 is a white line. Candidate line calculation means, 104 is a white line candidate point re-extraction means, 105 is a white line candidate line re-calculation means, and 106 is a road parameter estimation means.
As shown in FIG. 1, the lane recognition device 10 includes a processor 16 that uses, for example, a microprocessor to estimate road parameters based on an image of a road R stored in an image memory 15.
The processor 16 includes a window setting unit 101, a white line candidate point extracting unit 102, a white line candidate line calculating unit 103, a white line candidate point reextracting unit 104, a white line candidate line recalculating unit 105, and a road parameter estimating unit 106. It has. And these means 101-106 are implement | achieved by program control. Needless to say, these means 101 to 106 indicate control functions executed by the processor 16 in accordance with the following flowcharts (FIGS. 3 and 4).
The window setting means 101 provides a plurality of windows on the road R image stored in the image memory 15 along the extending direction of the white line so that each window includes the white line of the road R. The white line candidate point extraction means 102 processes the image for each window and extracts white line candidate points. The white line candidate line calculation unit 103 calculates the image coordinates and inclination of the white line candidate line having the highest possibility of a white line, based on the white line candidate points extracted for each of these windows. The white line candidate point re-extracting means 104 processes these images for each window to re-extract white line candidate points. The white line candidate line recalculation means 105 recalculates the image coordinates and inclination of the white line candidate line having the highest possibility of a white line based on the white line candidate points extracted for each window. The road parameter estimation means 106 estimates a road parameter including at least a lateral displacement of the host vehicle 12 with respect to the lane using the image coordinates and inclination of the white line candidate line calculated for each window and the road model formula.
As the road model formula, the following formula 1 is used to estimate the shape of the road R and the posture of the host vehicle 12 with respect to the road R as described above.

式１において、道路パラメータＡ，Ｂ，Ｃ，Ｄ，Ｈは、それぞれ、自車両１２の車線に対する横変位（Ａ）、道路曲率（Ｂ）、自車両１２の車線に対するヨー角（Ｃ）、自車両１２のピッチ角（Ｄ）、および道路Ｒの路面からのカメラ１４の高さ（Ｈ）である。
また、Ｅ_０は車線幅（左右の白線の内側間の距離）を示す定数、ｆはカメラ１４のカメラ透視変換定数、ｊは左右の白線を区別するパラメータとし、左白線のとき、ｊ＝０、右白線のときｊ＝１とする。また、（ｘ，ｙ）は、左白線または右白線内端上の任意の点の画像上の座標であり、画像左上を原点に取り、右方向がｘ軸正方向、下方向がｙ軸正方向とする。
なお、式１は、道路モデルを表す一手法を示したものであり、例えば、Ｅ_０を変数にしたもの、Ｈを固定したもの、車線幅を前述のように左右の白線の内側間の距離ではなく、左白線中央と右白線中央との間の距離とし、（ｘ，ｙ）を白線幅の中央位置の画像座標としても定義することができる。

In Equation 1, road parameters A, B, C, D, and H are respectively the lateral displacement (A) with respect to the lane of the own vehicle 12, the road curvature (B), the yaw angle (C) with respect to the lane of the own vehicle 12, The pitch angle (D) of the vehicle 12 and the height (H) of the camera 14 from the road surface of the road R.
E ₀ is a constant indicating the lane width (the distance between the insides of the left and right white lines), f is a camera perspective conversion constant of the camera 14, and j is a parameter for distinguishing the left and right white lines. When the right white line, j = 1. Also, (x, y) is the coordinates on the image of an arbitrary point on the inner edge of the left white line or right white line, taking the upper left of the image as the origin, the right direction is the x-axis positive direction, and the lower direction is the y-axis positive The direction.
Note that Equation 1, which shows one technique for representing the road model, for example, those of E ₀ to the variable, which is fixed to H, the distance between the inner white line of the right and left lane width as described above Instead, it can be defined as the distance between the center of the left white line and the center of the right white line, and (x, y) can also be defined as the image coordinates of the center position of the white line width.

FIG. 5 is a diagram showing an arrangement state of a plurality of windows set on the screen, in which 18 is a road white line and 20 is a window.
FIG. 6 is a diagram showing a plot state of white line candidate points in the window 20, where 20a is the upper side of the window 20, 20b is the lower side of the window 20, and 21 is a white line candidate point.
FIG. 7 is a diagram showing a selection state of white line candidate lines in the window 20, and 22 is a white line candidate line.

3 and 4 are flowcharts of a program for estimating the road parameters by the processor 16. For example, it is assumed that the process is repeated every 30 [msec] while the host vehicle 12 is traveling.
Next, an outline process for each step in FIG. 3 will be described.
First, in step S1, a front image of the host vehicle 12 is taken into the image memory 15 from the camera 14 of FIG.
In step S2, a plurality (m) of white line detection windows 20 (see FIG. 5) are set on the captured image.
In step S3, the window processing number i is set to an initial value of 1.
Next, in step S4, a white line candidate point 21 (for detecting the luminance change in the horizontal direction from the image data of the i-th window 20 (hereinafter referred to as window i) and recognizing the end position of the white line 18 ( (See FIG. 6).
Next, in step S5, based on the coordinate position of the white line candidate point 21 in the window 20, the white line candidate line 22 (see FIG. 7) having the highest possibility of the white line 18 is calculated from these white line candidate points 21.
Next, in step S6, it is determined whether or not the white line 18 is detected.
If it is determined in step S6 that the white line 18 is detected, the process proceeds to step S9. If it is determined in step S6 that the white line 18 is not detected, a white line candidate for recognizing the position of the end of the white line 18 by detecting a luminance change in the horizontal direction from the image data of the window i in step S7. Re-extract the point 21 (see FIG. 6).
Next, in step S8, based on the coordinate position of the white line candidate point 21 in the window 20, the white line candidate line 22 (see FIG. 7) having the highest possibility of the white line 18 from these white line candidate points 21 is recalculated.

Next, in step S9, the window process number i is incremented by one.
Next, in step S10, it is determined whether i exceeds the number of windows (m).
If it is determined in step S10 that i does not exceed the number of windows (m), the process returns to step S4 to process the next window 20.
If it is determined in step S10 that i exceeds the number of windows (m), the pre-processing for all the windows has been completed. In step S20, road parameter estimation processing is performed.

Hereinafter, the processing of the program will be described in more detail for each step.
(1) Capture forward image (S1)
An image in front of the host vehicle 12 captured by the camera 14 is taken into the image memory 15 every predetermined time for which the processor 16 processes the program, for example, every 30 [msec]. The image is represented as luminance data for each pixel.
(2) Setting of a plurality of windows 20 (S2)
As shown in FIG. 5, a plurality of windows 20 are provided along the extending direction of the white line 18 on the road R (in the example of FIG. 5, five for the left white line and five for the right white line). Based on the parameters of the road model obtained from the previous image processing result, the position of each window 20 is set so that the white line 18 is captured in each window 20, and the white line 18 is detected in these windows 20. In FIG. 5, the vertical direction indicates the image coordinate y-axis direction, and the lower side in the figure is y plus. The horizontal direction indicates the image coordinate x-axis direction, and the right side in the figure is x plus.
Here, the y coordinate of the upper side 20a (FIG. 6) of the nth window (i = n) is yn, and the previous parameter estimation results are {A (-1), B (-1), C (-1 ), D (−1), H (−1)}, the central coordinate x _{est in} the horizontal direction of the current window 20 is expressed by Equation 2 from Equation 1.

ウインドウ２０のｘ方向の幅は固定値としてもよく、また、特開平８−２６１７５６号公報に示されるようにパラメータの分散から合理的に設定することもできる。

The width of the window 20 in the x direction may be a fixed value, or may be set rationally from the dispersion of parameters as disclosed in JP-A-8-261756.

(3) Initialization of window processing number i (S3)
A loop control parameter (i.e., window processing number) i for performing the processing from step S4 to step S7 on all windows 20 is initialized to 1.
(4) Extraction of white line candidate points 21 (S4)
A white line candidate point 21 (FIG. 6) is obtained by using edge detection (brightness change detection) processing such as a sobel filter processing from the image of the range of each window 20 set as described above in the image captured from the camera 14. ) Is selected. In this edge detection, as shown in FIG. 6, when the luminance of the pixel on the left (x value of the image coordinate is small) is larger than the luminance of the pixel on the right (x value of the image coordinate is large), the output of the filter is positive. Suppose there is. Since the inner end of the white line 18 having a certain width is the position of the white line 18, in the case of the left white line, the point where the output of the filter exceeds the positive discrimination value is set as the white line candidate point 21, and the right white line In this case, a point where the output of the filter falls below the negative discrimination value is set as the white line candidate point 21. All the image data in the window 20 is scanned, and the white line candidate point 21 is extracted by using the above discriminant value by the output of the filter. The discriminant value may be set as appropriate depending on the entire image or the average brightness, contrast, etc. of each window 20.
In this edge detection, the white line candidate point 21 is extracted from the change in luminance of the pixels adjacent to the left and right, but the white line candidate point 21 is determined from the change in luminance of the pixels adjacent in the vertical direction in the captured image. May be extracted. However, in this case, since the white line candidate point 21 is extracted from the stop line existing at the intersection in addition to the white line 18 present on the side of the road, the left and right adjacent pixels described in the former are extracted. It is preferable to extract the white line candidate point 21 from the luminance change.

(5) Calculation of white line candidate line 22 (S5)
A white line candidate line 22 (FIG. 7) is searched from a set of white line candidate points 21 extracted from the window 20. For this search, a Hough transform, a least square method, or the like can be used. In this embodiment, the Hough transform is used. In the Hough transform straight line approximation, as shown in FIG. 7, among the straight lines passing through the window 20, the white line candidate line 22 that penetrates the white line candidate point 21 most frequently, that is, a white line candidate. The white line candidate line 22 on which the most points 21 are placed is the white line candidate line 22 with the highest possibility of the white line 18.
At this time, a straight line passing through the coordinates (x, y) can be expressed as the following Expression 3 using the parameters a and b. Here, a is the slope (∂x / ∂y) of the white line candidate line 22, and b is its x intercept.

図８は、白線候補線２２を探索する際のハフ変換による配列表を示す図、図９は、ハフ変換を配列要素で示す図である。
すなわち、白線候補点２１の座標を（ｘ，ｙ）とすると、ａを決めるとｂが式３より計算でき、これにより図８に示すハフ変換による配列が得られる。この配列の横方向は、一定刻みのａの値であり、縦方向は一定刻みのｂの値である。ここで、配列の空欄は零を意味する。また、「１」が立っている（ａ，ｂ）の組み合わせの中に真値が含まれる。この配列を白線候補点２１の全てに対してそれぞれ作成し、作成した配列を重ね合わせて、各配列要素毎の数を合計することにより図９に示す配列が得られる。図９において、配列要素（ａ_ｒ，ｂ_ｒ）の数ｚｒは、式４が貫く白線候補点２１の数を表している。

FIG. 8 is a diagram showing an array table by Hough transform when searching for the white line candidate line 22, and FIG. 9 is a diagram showing Hough transform by array elements.
That is, assuming that the coordinates of the white line candidate point 21 are (x, y), b can be calculated from Equation 3 when a is determined, thereby obtaining an array by the Hough transform shown in FIG. The horizontal direction of this array is the value of a in constant increments, and the vertical direction is the value of b in constant increments. Here, the blank in the array means zero. In addition, a true value is included in the combination of (a, b) where “1” stands. The array shown in FIG. 9 is obtained by creating this array for all the white line candidate points 21 and superimposing the created arrays and summing the numbers for each array element. In FIG. 9, the number zr of array elements (a _r , b _r ) represents the number of white line candidate points 21 that Formula 4 penetrates.

したがって、白線候補点２１を最も多く貫く白線候補線２２が最も白線１８の可能性が高い白線候補線２２であり、そのａ，ｂを求めることが白線候補線２２を算出することになる。すなわち、配列要素の値として最大のｚ（以下、ｚ_ｍａｘと記す）を持つ配列要素（ａ，ｂ）が白線候補線２２を示すことになる。ｚ_ｍａｘは、続く（６）白線１８の検出判定にて使用する。
図１０は、ウインドウ２０の上辺２０ａと白線候補線２２との交点との道路座標を示す図である。
こうして、白線候補線２２が算出されると、次いで、図１０に示すように、ウインドウ処理番号ｎにおける白線候補線２２とウインドウ２０の上辺２０ａとの交点の画像上の座標である道路座標（ｘｎ，ｙｎ）を算出する。ウインドウ２０の上辺２０ａのｙ座標ｙｎは、ウインドウ処理番号ｎに対する固定値である。道路座標のｘ成分ｘｎは、式５で算出される。

Therefore, the white line candidate line 22 that penetrates most of the white line candidate points 21 is the white line candidate line 22 that has the highest possibility of the white line 18, and obtaining the a and b will calculate the white line candidate line 22. That is, the array element (a, b) having the maximum z (hereinafter referred to as “z _max” ) as the value of the array element indicates the white line candidate line 22. z _max is used in the subsequent (6) white line 18 detection determination.
FIG. 10 is a diagram illustrating road coordinates between the upper side 20 a of the window 20 and the intersection of the white line candidate line 22.
When the white line candidate line 22 is calculated in this way, next, as shown in FIG. 10, road coordinates (xn) which are coordinates on the image of the intersection of the white line candidate line 22 and the upper side 20a of the window 20 in the window processing number n. , Yn). The y coordinate yn of the upper side 20a of the window 20 is a fixed value for the window processing number n. The x component xn of the road coordinates is calculated by Equation 5.

図１１は、ウインドウ２０内のかすれた白線１８’の場合の第１の判別値による白線候補点２１のプロット状態を示す図、図１２は、ウインドウ２０内のかすれた白線１８’の場合の第２の判別値による白線候補点２１のプロット状態を示す図である。
（６）白線１８の検出判定（Ｓ６）
ｚ_ｍａｘは、上記で算出された白線候補線２２がハフ変換において白線候補点２１を何点貫いているかを表しているが、このｚ_ｍａｘが所定値に満たない場合は、白線１８を検出していない可能性が高い。例えば、今回、問題にしている図１１のようなかすれた白線１８’もそうであるが、路面の局所的汚れを誤検出している可能性がある。よって、このｚ_ｍａｘは所定値以上を持つものを白線１８とすべきである。この所定値は、設定しているウインドウ２０の大きさ、カメラ１４の特性をもとに実験値で設定するものである。よって、ｚ_ｍａｘが所定値に達していない場合は、このウインドウｉでは白線を検出していないとする。
（７）白線候補点２１の再抽出（Ｓ７）
上記（６）白線１８の検出判定で、白線１８が検出できなかったウインドウｉに対して再抽出を行う。使用する画像は同じ画像を用いる。この抽出の際に、上記（４）白線候補点２１の抽出で用いた第１の判別値より低い第２の判別値を用いて、上記（４）白線候補点２１の抽出と同様なエッジ検知（輝度変化検知）処理により白線候補点２１の再抽出を行う。白線１８を対象とした場合は、図１２のような白線候補点２１’が再抽出される。なお、上記第２の判別値は、実験により最適な値を選定する。
（８）白線候補線２２の再算出（Ｓ８）
ウインドウ２０内から抽出された白線候補点２１および白線候補点２１’の集合から白線候補線２２を探索する。ハフ変換直線近似は、前述したのと同様な方法をとる。手順としては、
ａ）白線候補点２１’に対してハフ変換直線近似を適用し、白線候補点２１’を最も多く貫く仮白線候補線を算出する。
ｂ）得られた仮白線候補線の傾きを用いて白線候補点２１を最も多く貫く白線候補線２２を算出する。そして、この白線候補線２２を白線１８として認識する。この場合、最も多く白線候補点２１を貫いた候補点数は、上記の（６）白線１８の検出判定で用いた所定値より少ない値でも白線を検出しているとするが、候補点数が所定以下の場合は白線１８を検出できなかったと処理する。

FIG. 11 is a diagram showing a plot state of the white line candidate point 21 based on the first discriminant value in the case of the faint white line 18 ′ in the window 20, and FIG. It is a figure which shows the plot state of the white line candidate point 21 by the discriminant value of 2. FIG.
(6) White line 18 detection determination (S6)
z _max represents the number of white line candidate points 21 that the white line candidate line 22 calculated above passes through the white line candidate points 21 in the Hough transform. If this z _max is less than a predetermined value, the white line 18 is detected. Most likely not. For example, this is the case with the faint white line 18 ′ as shown in FIG. 11, but there is a possibility that local soiling on the road surface is erroneously detected. Therefore, the white line 18 should have z _max having a predetermined value or more. This predetermined value is set as an experimental value based on the size of the set window 20 and the characteristics of the camera 14. Therefore, when z _max does not reach the predetermined value, it is assumed that no white line is detected in this window i.
(7) Re-extraction of white line candidate points 21 (S7)
In the above (6) white line 18 detection determination, re-extraction is performed on the window i in which the white line 18 could not be detected. The same image is used as the image to be used. At the time of this extraction, the same edge detection as in the extraction of the (4) white line candidate point 21 is performed using the second determination value lower than the first determination value used in the extraction of the (4) white line candidate point 21. The white line candidate point 21 is re-extracted by the (luminance change detection) process. When the white line 18 is targeted, a white line candidate point 21 ′ as shown in FIG. 12 is re-extracted. Note that an optimal value is selected as the second discriminant value by experiment.
(8) Recalculation of white line candidate line 22 (S8)
A white line candidate line 22 is searched from a set of white line candidate points 21 and white line candidate points 21 ′ extracted from the window 20. The Hough transform linear approximation takes the same method as described above. As a procedure,
a) Hough transform straight line approximation is applied to the white line candidate point 21 'to calculate a provisional white line candidate line that penetrates most of the white line candidate point 21'.
b) A white line candidate line 22 penetrating most of the white line candidate points 21 is calculated using the inclination of the obtained provisional white line candidate line. The white line candidate line 22 is recognized as the white line 18. In this case, it is assumed that white lines are detected even if the number of candidate points that penetrate the white line candidate points 21 is the smaller than the predetermined value used in the detection determination of (6) white line 18 described above. In the case of, it is processed that the white line 18 could not be detected.

(9) Road parameter estimation (S20)
Equation 6 is differentiated by y to obtain Equation 6.

そして、式１と式６からＡを消去すると、式７を得る。

Then, when A is eliminated from Equations 1 and 6, Equation 7 is obtained.

このとき、Ａは自車両１２の車線に対する横位置のパラメータであり、このＡを消去すると同時に、車線幅を表すＥ_０も消去されることになる。つまり、道路幅がいかなるものであれ、式７が成立することになる。
次に、カルマンフィルタを用いて道路パラメータを推定する場合、式８に示す方程式が与えられることにより、パラメータｚを推定することができる。ただし、同式中のパラメータｙｙの値およびｆの構造はそれぞれ既知とし、一般にｙｙはベクトルとなる。

At this time, A is a parameter of the lateral position with respect to the lane of the host vehicle 12, and at the same time as A is deleted, E ₀ indicating the lane width is also deleted. That is, Expression 7 is established regardless of the road width.
Next, when the road parameter is estimated using the Kalman filter, the equation z shown in Expression 8 is given, whereby the parameter z can be estimated. However, the value of the parameter yy and the structure of f in the equation are known, and generally yy is a vector.

ここで、この式８に相当する観測方程式（仮称）は、前記式１が式９として成立する。

Here, the observation equation (tentative name) corresponding to Equation 8 is established as Equation 9 above.

この式９は、ｘ＝ｆｚ（Ａ，Ｂ，Ｃ，Ｄ，Ｈ）…（式９’）と表すこともできる。
式９’は一般に複数列より構成され、これらを線型近似した上で並べることにより次の行列式１０が得られる。

Expression 9 can also be expressed as x = fz (A, B, C, D, H) (Expression 9 ′).
Expression 9 ′ is generally composed of a plurality of columns, and the following determinant 10 is obtained by arranging these after linear approximation.

この行列式１０中、ｍは、１画面中に設定されたウインドウ２０の数である。この場合、ｚｎは、ｚｎ＝ｘｎとなる。

In this determinant 10, m is the number of windows 20 set in one screen. In this case, zn is zn = xn.

Next, with reference to FIG. 4, the subroutine P3 of the road parameter estimation process will be described.
From the road parameter estimation (S20) also shown in FIG. 3, first, the window processing number parameter i is initialized to 1 in step S21.
Next, in step S22, zi in Expression 10 is set to xi, and the right side of Expression 9 ′ is linearly approximated, so that Mi1, Mi2, Mi3, Mi4, and Mi5 are dfz / dA, dfz / dB, dfz / dC, dfz / dD, and dfz / dH.
Next, in step S23, the window process number i is incremented by one.
Next, in step S24, it is determined whether or not the window process number i exceeds the window number m.

If it is determined in step S24 that i does not exceed the number m of windows, the process returns to step S22 to process the next window 20.
If it is determined in step S24 that i exceeds the number of windows m, parameters (A, B, C, D, H) are estimated by Kalman filter processing in step S25, and the process returns to the main routine.

As described above, in the lane recognition device 10 and the lane recognition method according to the present embodiment, the luminance change of the image captured by the camera 14 that images the front of the host vehicle 12 is the first predetermined value (the first determination above). Value)), the first white line candidate point 21 is extracted. Moreover, the 1st white line candidate line (refer 22 of FIG. 7) in which the 1st white line candidate point 21 mounts most is extracted among the straight lines containing the extracted 1st white line candidate point 21. FIG. In addition, when the number of first white line candidate points 21 included in the extracted first white line candidate line is less than a predetermined value, a second predetermined value smaller than the first predetermined value (the second second above) Set the discriminant value. Further, when the luminance change of the image is equal to or greater than the second predetermined value, the second white line candidate point 21 ′ is extracted. In addition, a second white line candidate line on which the second white line candidate points 21 ′ are most frequently extracted is extracted from straight lines including the extracted second white line candidate points 21 ′. A third white line candidate line is extracted based on the second white line candidate line and the first white line candidate point 21. The third white line candidate line is recognized as the white line 18. Further, in the present embodiment, a straight line having the largest number of first white line candidate points 21 is extracted as a third white line candidate line out of straight lines having the same inclination as the second white line candidate line.
For example, when a black and white image is captured by the camera 14 and the white line 18 is recognized by edge detection, in the case of the white line 18 ′ blurred as described above, even if there is a sufficient luminance difference, the white line 18 is recognized. Since there are not enough white line candidate points 21, the white line 18 may not be recognized. The same applies to a road surface such as a white line drawn on a concrete road surface and a road surface having a small luminance difference from the white line 18. In such a case, the recognition rate of the white line 18 is reduced. Further, when the present invention is applied to a device that prevents the vehicle from deviating from between the two white lines 18 based on the recognized white line 18, the lane recognition system does not operate on a part of the road. Then, the setting operation of the lane recognition system is required at the point where the white line 18 can be recognized again, which is complicated.

In the present embodiment, when the number of first white line candidate points 21 extracted first as described above is less than a predetermined value, the first predetermined value which is a threshold value for luminance change is once decreased. Then, a second white line candidate point 21 ′ is extracted. Then, the inclination of the second white line candidate line is calculated. A third white line candidate line including the first white line candidate point 21 extracted with a first predetermined value that is a high threshold is extracted with respect to the inclination of the second white line candidate line, and this is extracted as a white line 18. Recognize as Thus, when the number of white line candidate points 21 is insufficient with the normal threshold value and the white line 18 cannot be detected, it is determined as a white line with a low degree of shading, and the normal threshold value and a lower threshold value are determined. Two types of threshold values are set. In the Hough conversion at a low threshold, the directionality of the white line is determined. In the normal threshold Hough conversion, the boundary line of the white line 18 is searched using the Hough conversion direction at a low setting. This can increase the chances of recognizing the white line 18 without reducing the recognition accuracy of the white line 18. In the case of other than the faint white line 18 ', a normal signal is applied in the direction of the second white line candidate line found by the low threshold only by increasing noise elements such as variations in road luminance even if the threshold is lowered. It is unlikely that the Hough transform using the threshold value satisfies the predetermined condition. Therefore, only the faint white line 18 ′ can be detected stably, the recognition rate of the white line 18 increases, and deviation from the lane can be prevented for more roads. In addition, the portion of the road where the lane recognition system does not operate is reduced, and the user's useless operations can be reduced.
The first white line candidate lines in which the number of first white line candidate points 21 extracted first is less than a predetermined value and the second white line candidate lines extracted again are basically the same. . In the case of a faint white line 18 ', as the threshold value is lowered, the white line candidate points 21' that can be extracted increase on the white line 18 '. Therefore, by temporarily lowering the threshold value, the white line candidate point 21 ′ is increased and the inclination of the second white line candidate line is confirmed. Then, again, although the number may be small, it is confirmed whether or not the white line candidate point 21 has been extracted on the second white line candidate line with the initial threshold value. As a result, if both are satisfied, it is determined that the first extracted white line candidate line may be recognized as the white line 18 although the number of white line candidate points 21 is smaller than the original number.
Furthermore, a plurality of windows 20 are set in the image captured by the camera 14, and the first white line candidate point 21 and the second white line candidate point 21 ′ are detected based on the luminance change in the window 20. Yes. With such a configuration, the white line 18 can be detected easily and satisfactorily.

In the present embodiment, among the straight lines having the same inclination as the second white line candidate lines, the straight line on which the first white line candidate points 21 are most frequently extracted is extracted as the third white line candidate line. It is also possible to take the following configuration. That is, the extracted first white line candidate line is stored in the storage device, and it is determined whether or not the stored first white line candidate line matches the extracted second white line candidate line. In the case of a match, this matched straight line is extracted as a third white line candidate line, and this third white line candidate line is recognized as a white line 18. The storage device is provided in the processor 16, for example.
In addition, the camera 14 of FIG. 1, FIG. 2, and step S1 of FIG. 3 in this Embodiment correspond to the image acquisition means of a claim. The processor 16 in FIG. 1, the white line candidate point extracting unit 102 in FIG. 1, and step S4 in FIG. 3 correspond to the first white line candidate point extracting unit in the claims. The processor 16 in FIG. 1, the white line candidate line calculation unit 103 in FIG. 1, and step S5 in FIG. 3 correspond to the first white line candidate line extraction unit in the claims. The processor 16 in FIG. 1 and step S6 in FIG. 3 in the present embodiment correspond to second predetermined value setting means in the claims. The processor 16 in FIG. 1, the white line candidate line point re-extracting means 104 in FIG. 1, and step S7 in FIG. 3 correspond to the second white line candidate point extracting means in the claims. The processor 16 in FIG. 1, the white line candidate line recalculating unit 105 in FIG. 1, and step S8 in FIG. 3 correspond to the second white line candidate line extracting unit in the claims. The processor 16 in FIG. 1 and the procedures a) and b) in step S8 in FIG. 3 correspond to the third white line candidate line extracting means in the claims. The white line candidate line 22 that penetrates most of the white line candidate points 21 by using the inclination of the temporary white line candidate line of b) corresponds to the third white line candidate line in the claims. The processor 16 in FIG. 1 and step S8 in FIG. 3 in the present embodiment correspond to the white line recognition means in the claims.
Further, the embodiments described above are described for facilitating the understanding of the present invention, and are not described for limiting the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.

It is a block diagram which shows the basic composition of the lane recognition apparatus of embodiment of this invention. It is a figure which shows the vehicle carrying the lane recognition apparatus of embodiment of this invention. It is a flowchart for performing the image processing program by embodiment of this invention. It is a flowchart which shows the estimation subroutine of the road parameter in the image processing program by embodiment of this invention. It is a figure which shows the arrangement | sequence state of the some window set on the screen in embodiment of this invention. It is a figure which shows the plot state of the white line candidate point in the window set in embodiment of this invention. It is a figure which shows the selection state of the white line candidate line in the window set in embodiment of this invention. It is a figure which shows the arrangement | sequence table by the Hough transform at the time of searching for the white line candidate line performed in embodiment of this invention. It is a figure which shows the Hough transformation performed in embodiment of this invention by an array element. It is a figure which shows the road coordinate of the intersection of the upper side of the window set in embodiment of this invention, and a white line candidate line. It is a figure which shows the plot state of the white line candidate point by the 1st discriminant value in the case of the faint white line in the window set in embodiment of this invention. It is a figure which shows the plot state of the white line candidate point by the 2nd discriminant value in the case of the faint white line in the window set in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Lane recognition apparatus 12 ... Own vehicle 12a ... Indoor ceiling 12b ... Windshield 14 ... Camera 16 ... Processor R ... Road 18 ... White line 20 ... Window 20a ... Upper side 20b ... Lower side 21 ... White line candidate point 22 ... White line candidate line 101 ... Window setting means 102 ... White line candidate point extraction means 103 ... White line candidate line calculation means 104 ... White line candidate point re-extraction means 105 ... White line candidate line re-calculation means 106 ... Road parameter estimation means

Claims (5)

Image acquisition means for imaging the front of the host vehicle;
A first white line candidate point extracting unit that extracts a first white line candidate point when the luminance change of an adjacent pixel in the image captured by the image acquisition unit is equal to or greater than a first predetermined value;
The first white line candidate for extracting the first white line candidate line on which the first white line candidate points are most frequently included among the straight lines including the first white line candidate points extracted by the first white line candidate point extraction unit. Line extraction means;
When the number of the first white line candidate points included in the first white line candidate line extracted by the first white line candidate line extraction unit is less than a predetermined value, it is smaller than the first predetermined value. Second predetermined value setting means for setting a second predetermined value;
Second white line candidate point extraction for extracting a second white line candidate point when the luminance change of adjacent pixels in the image is equal to or greater than the second predetermined value set by the second predetermined value setting means Means,
A second white line candidate for extracting a second white line candidate line on which the second white line candidate points are most frequently included among straight lines including the second white line candidate points extracted by the second white line candidate point extraction unit. Line extraction means;
The second white line candidate line extracted by the second white line candidate line extracting means, and the first white line candidate point or the first white line candidate line extracting means extracted by the first white line candidate point extracting means. Third white line candidate line extraction means for extracting a third white line candidate line based on the first white line candidate line extracted by
A lane recognition device comprising: white line recognition means for recognizing the third white line candidate line as a white line.   The third white line candidate line extraction unit extracts, as the third white line candidate line, a straight line on which the first white line candidate points are most frequently placed among straight lines having the same inclination as the second white line candidate line. The lane recognition device according to claim 1. A first white line candidate line storage unit for storing the first white line candidate line extracted by the first white line candidate line extraction unit;
The third white line candidate line extraction unit is configured to store the first white line candidate line stored in the first white line candidate line storage unit and the second white line candidate extracted by the second white line candidate line extraction unit. 2. The lane recognition device according to claim 1, wherein it is determined whether or not the line matches, and the matched straight line is extracted as the third white line candidate line.   A plurality of windows are set in the image picked up by the image acquisition means, and at least one of the first white line candidate point and the second white line candidate point based on a luminance change of an adjacent pixel in the window. The lane recognition device according to claim 1, wherein: An image acquisition step of imaging the front of the host vehicle;
A first white line candidate point extracting step of extracting a first white line candidate point when the luminance change of an adjacent pixel in the image captured by the image acquisition step is equal to or greater than a first predetermined value;
The first white line candidate for extracting the first white line candidate line on which the first white line candidate points are most frequently included among the straight lines including the first white line candidate points extracted in the first white line candidate point extracting step. A line extraction process;
When the number of the first white line candidate points included in the first white line candidate line extracted by the first white line candidate line extraction step is less than a predetermined value, it is smaller than the first predetermined value. A second predetermined value setting step of setting a second predetermined value;
Second white line candidate point extraction for extracting a second white line candidate point when a luminance change of an adjacent pixel in the image is greater than or equal to the second predetermined value set by the second predetermined value setting step Process,
A second white line candidate for extracting a second white line candidate line on which the second white line candidate points are most frequently included among straight lines including the second white line candidate points extracted in the second white line candidate point extraction step. A line extraction process;
The second white line candidate line extracted by the second white line candidate line extraction step, and the first white line candidate point or the first white line candidate line extraction step extracted by the first white line candidate point extraction step. A third white line candidate line extracting step for extracting a third white line candidate line based on the first white line candidate line extracted by:
A white line recognition step of recognizing the third white line candidate line as a white line.

Images