CN103839264A - Detection method of lane line - Google Patents

Abstract

The invention discloses a detection method of a lane line. At first, edge information of a road image is extracted by means of a self-adaption threshold value edge extraction algorithm, cumulative probability Hough transformation in which an edge gradient direction constraint is brought is utilized so that straight lines can be quickly detected from an edge image, then the accurate positions of blanking points are estimated between successive frames by means of the detected straight lines through an iterative calculation method, a part of the straight lines not deriving from the edge of the lane line are filtered out by means of blanking point constraints, a tracking door is built so that noise can be further filtered out, candidate straight lines are obtained, and ultimately a pair of optimal straight lines are selected from the candidate straight lines to represent a current lane line through use of color information of candidate straight line neighborhoods and lane line information detected by a previous frame. According to the method, robustness on road environment change is achieved, the calculated quantity is small and real-time performance is achieved.

Description

A kind of detection method of lane line

Technical field

The present invention relates to vehicle assistant drive technical field, particularly a kind of lane line method for quick for real-time system.

Background technology

The invention of automobile has changed the mankind's trip mode, and it effectively promotes flowing of commodity and personnel with its convenience and high efficiency, and economical and social development are played an important role.By 2010, approximately there was the various automobile that adds up to 1,000,000,000 in the whole world, and this numeral is still in rapid growth.But, many cities are all faced with the problems of being brought by the excessive use of personal automobile in the world at present, and such as various serious traffic hazards, the vexed series of problems such as road congestion, huge energy resource consumption, noise pollution and environmental pollution have just seriously reduced townie quality.Along with scientific and technological progress and social development, people are more and more higher in the requirement of safety, energy consumption, the aspect such as convenient, comfortable to automobile.And the technology such as sensor technology, control automatically, artificial intelligence, machine vision are increasingly mature and application more and more in Communication and Transportation Engineering, this background has expedited the emergence of a new developing direction---the intelligent vehicle of automobile industry.

At present, every country has been carried out the research of intelligent automobile in succession, but is limited to the development of technology, and machine replaces the mankind completely to be completed vehicle drive and also need a period of time.The intelligent vehicle of this life often adopts DAS (Driver Assistant System) to reduce driving workload, guarantees the safety of human pilot.The automatic parking backup system that such as can see in high-end vehicle, Brake Assist, reversing aid system, driving backup system, lane keeping backup system etc.

Wherein the gordian technique of lane keeping backup system is lane detection, can obtain the relative position relation of vehicle and lane line by it, and then can remind the travel conditions of human pilot vehicle, the deviation problem that can effectively solve fatigue driving or human negligence and produce, and then improve travel safety.

The key issue of lane detection is how to extract the feature of lane line, and which kind of model to simulate lane line with.Because the residing environment of road is subject to the impact of the many factors such as weather, illumination and road conditions, the detection algorithm that complexity is low is difficult to find the feature and the matching lane line accurately that conform and change, and the high detection algorithm of complexity cannot guarantee the real-time of system conventionally, be difficult to be applied to the vehicle of running at high speed.

Therefore, be necessary to propose a kind of real-time, reliable and stable method for detecting lane lines.

Summary of the invention

It is that current method for detecting lane lines is difficult to meet the real-time of lane detection and the problem of robustness that the present invention wants technical solution problem.

For solving the problems of the technologies described above, the present invention proposes a kind of detection method of the lane line based on vanishing point constraint, and the method comprises the following steps:

Step 1, obtains road image I by vehicle-mounted imageing sensor _t, wherein t is frame number;

Step 2, carries out pre-service to described road image, obtains through pretreated road image I _p,t;

Step 3, detects pretreated road image I _p,tin marginal information, obtain edge image I _e,tand the gradient direction of each marginal point;

Step 4, from described edge image I _emiddle detection of straight lines, obtains straight line set { l _t,i| l _t,i=(k _t,i, b _t,i), i=1,2 ... N}, wherein, k and b are respectively slope and the intercept of straight line, and N is the quantity of the straight line that detects;

Step 5, utilizes the straight line obtaining in step 4 to estimate the position of vanishing point in described road image;

Step 6, utilizes vanishing point to retrain the partial noise in the straight line obtaining in step 4 described in filtering;

Step 7, by building the noise in the further filtering straight line of tracking gate, obtains candidate's straight line collection of the right and left;

Step 8, calculates described candidate's straight line and concentrates the posterior probability of every candidate's straight line, then concentrates from left and right candidate's straight line left side bearing and the right side bearing of selecting the straight line of a pair of posterior probability maximum to come matching track, finally obtains the lane line detecting.

The obtained beneficial effect of the present invention has:

The present invention is by using the Boundary extracting algorithm of adaptive threshold, the impact that can alleviate illumination variation;

The present invention is by using PPHT, and detection of straight lines from edge image rapidly, guarantees the real-time of straight-line detection, and by introducing edge gradient direction, improve the accuracy of straight-line detection;

The present invention passes through proposed between successive frame, the straight line that utilization detects is estimated the method for the position of vanishing point, can obtain fast the accurate location of vanishing point, and utilize vanishing point to retrain the partial noise in filtering straight line, improve the robustness of lane detection;

The present invention is by using tracking gate to obtain candidate's straight line, further filtering noise;

The present invention is by using colouring information in candidate straight line neighborhood and the historical information of lane line, selects optimum straight line to represent current track, thereby improves the accuracy of lane detection.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of method for detecting lane lines of the present invention;

Fig. 2 is the duality schematic diagram between the straight line in vanishing point and the parameter space in image space of the present invention;

Fig. 3 is the process flow diagram of vanishing point location estimation of the present invention;

Fig. 4 is tracking gate schematic diagram of the present invention;

Fig. 5 is candidate's straight line neighborhood gray scale check schematic diagram of the present invention.

Embodiment

For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in more detail.

The present invention proposes a kind of detection method of lane line, as shown in Figure 1, said method comprising the steps of:

Step 1, obtains road image I by vehicle-mounted imageing sensor _t, wherein t is frame number;

Step 2, carries out pre-service to described road image, obtains through pretreated road image I _p,t;

Image pre-service is mainly to prepare in order to carry out the operations such as feature extraction below, mainly comprises gray processing, smoothing denoising, yardstick convergent-divergent or the image cropping of road image, and wherein, the object of graphical rule convergent-divergent is to reduce calculated amount; Image cropping also can reach similar object, particularly include region, non-road surface in image in.

The object of follow-up all operations is the image I after convergent-divergent _p,t, wherein t is frame number.

Step 3, detects pretreated road image I _p,tin marginal information, obtain edge image I _e,tand the gradient direction of each marginal point;

For edge extracting, in an embodiment of the present invention, use the method that is similar to Canny algorithm to extract marginal information, different, what adopt here is adaptive threshold value.Adaptive threshold value is determined by histogram of gradients, supposes that in image, only having the pixel of P% is marginal point, can release high threshold T _h, wherein, P is rule of thumb set by user, and certainly, the span of P is between 0-100, same, and the certainly not pixel ratio at edge that arranges in image can be released low threshold value T _l.With respect to fixed threshold is set, the method that this adaptive threshold calculates can successfully manage the impact of illumination variation.

Step 4, from described edge image I _emiddle detection of straight lines, obtains straight line set { l _t,i| l _t,i=(k _t,i, b _t,i), i=1,2 ... N}, wherein, k and b are respectively slope and the intercept of straight line, and N is the quantity of the straight line that detects;

For straight-line detection, in an embodiment of the present invention, use cumulative probability Hough transformation (PPHT) from edge image I _e,tthe straight line that middle detection exists, compared with standard Hough transformation (SHT), PPHT only uses part point to vote, thereby has improved the speed of straight-line detection.And in order to eliminate the impact of noise, improve the accuracy of straight-line detection, in the process of marginal point ballot, each marginal point is only vertical with its gradient direction or approximately perpendicular straight line ballot, and gradient information calculated in the process of edge extracting, here without double counting.

Step 5, utilizes the straight line obtaining in step 4 to estimate the position of vanishing point in described road image;

Estimate that the position of vanishing point is that to detect in the straight line obtaining be not the straight line that comes from lane line edge in order to remove.Due to perspective effect, in image, parallel lines converges on vanishing point, because lane line is parallel lines at real world, so lane line also converges on vanishing point.Utilize the duality between the point in image space X-Y cathetus and parameter space K-B, can be by the straight line l obtaining _t,ibe converted into the point (k in parameter space K-B _t,i, b _t,i), because vanishing point is the intersection point of parallel lines under perspective effect, the location estimation problem of vanishing point can be converted into the fitting a straight line problem in parameter space K-B so, as shown in Figure 2.In Fig. 2, if the position of vanishing point is (x in image space X-Y _v, y _v), so in parameter space K-B, (k _t,i, b _t,i) the straight line l at place can be expressed as b=-x _vk+y _v.Owing to there being part exterior point in parameter space K-B, i.e. noise spot, so in an embodiment of the present invention, uses weight w _t,iweigh a point (k _t,i, b _t,i) be the probability of interior point.Because which point is that exterior point is unknown, to regard weight w as hidden variable here, and adopt expectation-maximization algorithm (EM algorithm) to come the position of iterative estimate vanishing point, the number of times of iteration represents with j.Fig. 3 is the process flow diagram of vanishing point location estimation of the present invention, and as shown in Figure 3, the location estimation of vanishing point comprises the following steps:

Step 51, if t>1, the initial position of vanishing point is set to previous frame estimates the position of the vanishing point obtaining, i.e. (x _v,t ⁽⁰⁾, y _v,t ⁽⁰⁾)=(x _{v, t-1}, y _{v, t-1}); If t=1, does not arrange the initial position of vanishing point; Iterations j is initialized as to 1, enters step 52 to step 54 and carry out iterative computation;

Step 52, calculates the weight of each point in the j time iteration parameter space K-B: if j=1 and t=1 distribute equal initial weight w for each point _{1, i} ⁽¹⁾=1/N _t, otherwise utilize following formula to calculate w _t,i ^j:

${w_{t,i}}^{\left(j\right)}=\frac{\exp [-{(b_{t,i}+{x_{v,t}}^{(j-1)}\·k_{t,i}-{y_{v,t}}^{(j-1)})}^2]}{\underset{n=1}{\overset{N_t}{\mathrm{\Σ}}}\exp [-{(b_{t,n}+{x_{v,t}}^{(j-1)}\·k_{t,n}-{y_{v,n}}^{(j-1)})}^2]}---\left(1\right)$

Namely: if j=1 and t>1, utilize previous frame t-1 frame estimate that the position of the vanishing point obtaining calculates initial weight w as initial position _t,i ⁽¹⁾; If j>1, the position (x of the vanishing point obtaining by the j-1 time iteration _v,t ^(j-1), y _v,t ^(j-1)) each point (k in the K-B of undated parameter space _t,i, b _t,i) weight w _t,i ^(j).If point (k _t,i, b _t,i) from (x in K-B space _v,t ^(j-1), y _v,t ^(j-1)) nearer as the straight line of parameter, weight w _t,i ^(j)larger.

Step 53, according to the weight w after upgrading _t,i ^(j)calculate the parameter of the j time iteration K-B space cathetus, namely position (the x of vanishing point _v,t ^(j), y _v,t ^(j)), make the residual sum of squares (RSS) minimum with weight, and (x _v,t ^(j), y _v,t ^(j)) position (x of the vanishing point that calculates with previous frame _{v, t-1}, y _{v, t-1}) variation as much as possible little, minimize following objective function:

$\begin{array}{c}f({x_{v,t}}^{\left(j\right)},{y_{v,t}}^{\left(j\right)})=[1-\mathrm{\δ}(t-1)]\·[{\mathrm{\λ}}_1{(x_{v,t}-x_{v,t-1})}^2+{\mathrm{\λ}}_2{(y_{v,t}-y_{v,t-1})}^2]\\ +\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w}_{t,i}\·{(b_i+x_{v,t}\·k_{t,i}-y_{v,t})}^2\end{array}---\left(2\right)$

Wherein λ ₁, λ ₂with α be given coefficient, δ is unit sequence of impacts:

Try to achieve the j time iteration time vanishing point position (x _v,t ^(j), y _v,t ^(j)) be:

$\left[\begin{array}{c}{x_{v,t}}^{\left(j\right)}\\ {y_{v,t}}^{\left(j\right)}\end{array}\right]=A^{-1}\left[\begin{array}{c}{\mathrm{\λ}}_1(1-\mathrm{\δ}(t-1))x_{v,t-1}-\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w_{t,i}}^{\left(j\right)}{b}_{t,i}{k}_{t,i}\\ {\mathrm{\λ}}_2(1-\mathrm{\δ}(t-1))y_{v,t-1}-\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w_{t,i}}^{\left(j\right)}{b}_{t,i}\end{array}\right]---\left(4\right)$

Wherein, $A=\left[\begin{array}{cc}\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w_{t,i}}^{\left(j\right)}{k_{t,i}}^2+{\mathrm{\λ}}_1(1-\mathrm{\δ}(t-1))& -\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w_{t,i}}^{\left(j\right)}{k}_{t,i}\\ -\mathrm{\α}\underset{i=1}{\overset{N_t}{\mathrm{\Σ}}}{w_{t,i}}^{\left(j\right)}{k}_{t,i}& \mathrm{\α}+{\mathrm{\λ}}_2(1-\mathrm{\δ}(t-1))\end{array}\right].$

Step 54, if the estimated result of vanishing point position restrained, | x _v,t ^(j)-x _v,t ^(j-1)| and | y _v,t ^(j)-y _v,t ^(j-1)| in given error range, or reach given maximum iteration time N _max, termination of iterations and the position of vanishing point that obtains t frame are for (x _v,t, y _v,t)=(x _v,t ^(j), y _v,t ^(j)), proceed iteration otherwise return to step 52.

Step 6, utilizes vanishing point to retrain the partial noise in the straight line obtaining in step 4 described in filtering;

Utilizing vanishing point constraint to remove in the process of noise, first calculate vanishing point (x _v,t, y _v,t) and every straight line l _t,ibetween distance d _t,i, consider certain error, a distance threshold r need to be set _eif, vanishing point (x _v,t, y _v,t) to straight line l _t,ibetween distance be greater than given distance threshold r _e: d _t,i>r _e, think that this straight line is noise.

Step 7, by building the noise in the further filtering straight line of tracking gate, obtains the right and left candidate straight line collection;

Utilize tracking gate to determine that the step of candidate's straight line is further comprising the steps:

Step 71, builds left and right tracking gate, as shown in Figure 4;

Described left and right tracking gate is trapezoidal, and on it, the position of base mid point P1 is the position at the vanishing point place that obtains, and the slope of the straight line that the mid point P2 of left tracking gate bottom and P1 form equals the slope of the left side bearing in the track that t-1 frame detects.

The width w on the high h of given tracking gate, upper base _uwidth w with bottom _d, can determine the coordinate of the each end points of left tracking gate, in like manner can calculate the coordinate of the each end points of right tracking gate.

Step 72, if the intersection point on the upper and lower base of straight line and corresponding tracking gate all between two end points on base, place, this straight line is counted as candidate's straight line.

Utilize said method can determine respectively candidate's straight line collection of the right and left.

Step 8, calculates described candidate's straight line and concentrates the posterior probability of every candidate's straight line, then concentrates from left and right candidate's straight line left side bearing and the right side bearing of selecting the straight line of a pair of posterior probability maximum to come matching track, finally obtains the lane line detecting.

The calculating of the posterior probability of candidate's straight line is further comprising the steps:

Step 81, for every candidate's straight line, utilizes the half-tone information in its neighborhood to calculate its likelihood probability;

In order to reduce calculated amount, at candidate's straight line l _t,iupper random uniform sampling, calculate each sample point average gray of the block of pixels of four direction around, as shown in Figure 5, in Fig. 5, the edge pixel point that oblique line is filled is the current sample point being verified, block of pixels in overstriking square frame is the block of pixels that need to be verified, and wherein the block of pixels in the square frame of bottom right has higher gray scale.If maximum average gray is in four block of pixels that sample point m is around verified

and

be greater than given threshold value

this point is considered to an evidence point.By the sample point set S gathering _t,ithe ratio of middle evidence point is as the likelihood probability of this candidate's straight line, that is:

$P\left(S_{t,i}\right|l_{t,i})=\frac{1}{N_s}\underset{m=1}{\overset{N_s}{\mathrm{\Σ}}}\mathrm{II}({\stackrel{\‾}{I}}_m>{\stackrel{\‾}{I}}_{\min ,t})---\left(5\right)$

Wherein, II is indicator function, N _sfor the number of the concentrated sample point of sample point.

Step 82, utilizes the lane line L detecting in t-1 frame _t-1calculated candidate straight line l _t,iprior probability:

$P\left(l_{t,i}\right|L_{t-1})=\frac{1}{2\mathrm{\π}{\mathrm{\γ}}_k{\mathrm{\γ}}_b}\exp [-\frac{{(k_{t,i}-k_{L,t-1})}^2}{{{\mathrm{\γ}}_k}^2}-\frac{{(b_{t,i}-b_{L,t-1})}^2}{{{\mathrm{\γ}}_b}^2}]---\left(6\right)$

Wherein, γ _kand γ _bbe respectively the standard deviation of slope and intercept.

Step 83, calculates candidate's straight line l according to the likelihood probability obtaining and prior probability _t,iposterior probability.

Candidate's straight line l _t,iposterior probability be proportional to the product of prior probability and likelihood probability:

P(l _t,i|L _t-1,S _t,i)∝P(S _t,i|l _t,i)·P(l _t,i|L _t-1) （7）

The detection that above-mentioned steps circular flow can continue lane line.

Above-described specific embodiment; object of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the foregoing is only specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (10)

1. a detection method for lane line, is characterized in that, the method comprises the following steps:

Step 1, obtains road image I by vehicle-mounted imageing sensor _t, wherein t is frame number;

Step 2, carries out pre-service to described road image, obtains through pretreated road image I _p,t;

Step 3, detects pretreated road image I _p,tin marginal information, obtain edge image I _e,tand the gradient direction of each marginal point;

Step 4, from described edge image I _emiddle detection of straight lines, obtains straight line set { l _t,i| l _t,i=(k _t,i, b _t,i), i=1,2 ... N}, wherein, k and b are respectively slope and the intercept of straight line, and N is the quantity of the straight line that detects;

Step 5, utilizes the straight line obtaining in step 4 to estimate the position of vanishing point in described road image;

Step 6, utilizes vanishing point to retrain the partial noise in the straight line obtaining in step 4 described in filtering;

Step 7, by building the noise in the further filtering straight line of tracking gate, obtains candidate's straight line collection of the right and left;

Step 8, calculates described candidate's straight line and concentrates the posterior probability of every candidate's straight line, then concentrates from left and right candidate's straight line left side bearing and the right side bearing of selecting the straight line of a pair of posterior probability maximum to come matching track, finally obtains the lane line detecting.

2. method according to claim 1, is characterized in that, in described step 4, uses cumulative probability Hough transformation from described edge image I _e,tthe straight line that middle detection exists.

3. method according to claim 2, is characterized in that, in the process of linear edge point ballot, each marginal point is only vertical with its gradient direction or approximately perpendicular straight line ballot.

4. method according to claim 1, it is characterized in that, in described step 5, utilize the duality between image space X-Y cathetus intersection point and parameter space K-B cathetus, the vanishing point estimation problem in image space X-Y is converted to the Parameter Estimation Problem in parameter space K-B.

5. method according to claim 4, is characterized in that, utilizes the duality between the point in image space X-Y cathetus and parameter space K-B, by the straight line l obtaining _t,ibe converted into the point (k in parameter space K-B _t,i, b _t,i), if the position of vanishing point is (x in image space X-Y _v, y _v), so in parameter space K-B, (k _t,i, b _t,i) the straight line l at place can be expressed as b=-x _vk+y _v.

6. method according to claim 4, is characterized in that, in the straight line that use detects simultaneously and previous frame, the position of vanishing point is as constraint.

7. method according to claim 4, is characterized in that, uses weight w _t,iweigh the point (k in a parameter space K-B _t,i, b _t,i) be the probability of interior point, and regard weight as hidden variable, then adopt expectation-maximization algorithm to come the position of iterative estimate vanishing point.

8. method according to claim 1, is characterized in that, in described step 6, first calculates vanishing point (x _v,t, y _v,t) and every straight line l _t,ibetween distance d _t,iif this distance is greater than given distance threshold r _e, think that this straight line is noise.

9. method according to claim 1, is characterized in that, in described step 7, utilizes tracking gate to determine that the step of candidate's straight line is further comprising the steps:

Step 71, builds left and right tracking gate;

Step 72, if the intersection point on the upper and lower base of straight line and corresponding tracking gate all between two end points on base, place, this straight line is counted as candidate's straight line.

10. method according to claim 1, is characterized in that, in described step 8, the calculating of the posterior probability of candidate's straight line is further comprising the steps:

Step 81, for every candidate's straight line, utilizes the half-tone information in its neighborhood to calculate its likelihood probability;

Step 82, utilizes the roadmarking L detecting in t-1 frame _t-1calculated candidate straight line l _t,iprior probability;

Step 83, calculates candidate's straight line l according to the likelihood probability obtaining and prior probability _t,iposterior probability.

Images