CN103738243A - Early warning method for lane departure - Google Patents

Abstract

The invention provides an early warning method for lane departure. The early warning method for lane departure comprises the video capture device installation step, the parameter calibration step, the image preprocessing step, the edge detection step, the lane line recognition step and the lane departure early warning time determining step. The lane departure early warning time determining step comprises the vehicle off-course distance calculation step, the lane pressing time estimation step and the lane departure judgment step. In the vehicle off-course distance calculation step, the vehicle off-course distance is calculated according to the external parameters, the internal parameters and the coordinate transformation change calibrated by a video capture device. In the lane departure judgment step, the lane departure time threshold value is set, and when the lane pressing estimation time is smaller than the lane departure time threshold value, lane departure early warning is carried out. According to the early warning method, the anti-noise capacity and the anti-interference capacity of lane departure waning can be improved, the reliability of early warning for lane departure can be improved, the false alarm rate and the missed alarm rate can be lowered, the accuracy of lane pressing estimation time is improved, and therefore effective early warning for lane departure can be provided.

Description

A kind of lane departure warning method

Technical field

The present invention relates to traffic safety ancillary technique field, be specifically related to a kind of lane departure warning method and precompressed track time estimation method.

Background technology

Deviation is reported to the police, and (Lane Departure Warning is called for short: LDW) system is that (Advanced Driver Assistance Systems is called for short: wherein one ADAS) is an automobile driving safe ancillary system senior drive assist system.When sensitive member detects automotive run-off-road, if driver does not lay the indicator signal of crossover lane because of tired or carelessness, system can be sent alerting to remind driver to return to track.Deviation is reported to the police (LDW) system at present take camera as main sensors, based on machine vision technique, survey and follow the tracks of lane mark, in conjunction with driver characteristics, judge whether vehicle has the danger that departs from track, when dangerous, for chaufeur provides sound, light and vibrations, warn.

Because road environment is complicated and changeable, the existing lane departure warning method based on image processing techniques still faces many challenges, also Shortcomings and area for improvement: different weather conditions, illumination day and night difference, and the factor such as shade affects light and change, to image, processing is added to the difficulties; The interference of insulation strip, guardrail, track arrow and vehicle, affects the accuracy of lane detection.Therefore, lane departure warning system works reliability has much room for improvement, and needs to reduce false alarm rate and false dismissed rate, and needs to improve the accuracy of estimating the precompressed track time, to provide actv. lane departure warning.

And, deviation time (Time to Lane Crossing at present, be called for short: method of calculating TLC) is generally first to calculate vehicle yaw distance and yaw angle, by yaw angle and the speed of a motor vehicle, calculate vehicle side to yawing velocity, by cross track distance and side direction yawing velocity, estimate the vehicle precompressed track time again, but in engineering application, vehicle can not keep definitely steadily in the process of moving, vehicle-mounted camera there will be jitter phenomenon unavoidably, this directly can affect the calculating of yaw angle, because vehicle is run at high speed, the yaw angle error of two or three degree all may cause the calculation error of TLC to exceed allowed band, therefore, this method is not too reliable in actual applications.

Summary of the invention

For solving the problems of the technologies described above, the invention provides a kind of lane departure warning method, can improve antinoise and jamproof ability that deviation is reported to the police, lane departure warning reliability is improved, reduce false alarm rate and false dismissed rate, and improve the accuracy of estimating the precompressed track time, to provide actv. lane departure warning.

The invention provides a kind of lane departure warning method, comprising:

I. video acquisition device is installed and parameter calibration step, specifically comprises:

(i) camera is installed in automobile lateral center position, the ambient parameter of recording of video harvester: camera and level ground height D, camera are to vertical distance dl and dr, the camera angle of depression θ ° of vehicle right and left both sides;

(ii) demarcate video acquisition device inner parameter, determine the mapping relations of image coordinate system and world coordinate system;

II. image pre-treatment step, comprise acquisition testing region coloured image, determine surveyed area, image carried out to gray processing processing and medium filtering denoising;

It is characterized in that, also comprise:

III. edge detecting step, comprises and adopts local maximal difference extraction Image edge gradient, employing maximum variance between clusters edge image to carry out binary conversion treatment, the edge image after binaryzation is carried out to refinement, noise Transformatin;

IV. lane mark identification step, goes out many straight lines by Hough change detection, selected satisfactory lane mark;

V. lane departure warning time determining step, specifically comprises:

(i) vehicle yaw is apart from calculation procedure: ambient parameter, inner parameter and the coordinate transform relation of according to video acquisition device, demarcating calculate vehicle yaw distance;

(ii) precompressed track time estimation step, specifically comprises:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence trend model, the estimation precompressed track time;

(iii) deviation determining step: set deviation time threshold, when the described precompressed track time is less than deviation time threshold, carry out lane departure warning.

As improvement project of the present invention, described I. video acquisition device is installed and parameter calibration step is defined as the mapping relations of image coordinate system and world coordinate system:

$X_w=\frac{b(x-x_c)(Z_w-D)}{a\×\cos \mathrm{\θ}+(y_c-y)\sin \mathrm{\θ}}$

$Y_w=\frac{(Z_w-D)[(y-y_c)\sin \mathrm{\θ}+a\×\sin \mathrm{\θ}]}{a\×\cos \mathrm{\θ}+(y_c-y)\sin \mathrm{\θ}}$

Wherein, definition intrinsic parameters of the camera a=f × Ny, b=Ny ÷ Nx, establishes Nx, and Ny represents respectively the proportionality coefficient of two coordinate directions; F is camera focal length; (xc, yc) is image coordinate system, and (Xw, Yw, Zw) is world coordinate system;

(i) vehicle yaw in described v. lane departure warning time determining step is specially apart from calculation procedure:

Lane mark equation in coordinates under image coordinate system is:

x=ky+b’

Lane mark equation in coordinates under world coordinate system is:

X _w=KY _w+B,Z _w=0

According to the mapping relations of image coordinate system and world coordinate system, calculate:

$K=\frac{\mathrm{kba}\cos \mathrm{\θ}+(b^{\′}-x_c+0.5\mathrm{Wk})b\sin \mathrm{\θ}}{a}$

$B=\frac{D[\mathrm{kba}\sin \mathrm{\θ}+(0.5L-b^{\′}-0.5\mathrm{Wk})b\cos \mathrm{\θ}]}{a}$

In world coordinate system, K and cos θ trend 0, camera is to lane mark distance: d ≈ B=Dkb;

L, W are the horizontal and vertical size of computer picture, k, and b ' is equation of straight line slope and the intercept under image coordinate system;

Thus, calculate the distance L l=d-dl of car body left side to left-hand lane line, right side Lr=d-dr.

As improvement project of the present invention, in described III. edge detecting step, adopt local maximal difference to extract Image edge gradient, be specially: adopt the border of texture filter detected image, template size is set, in output template neighborhood, maxim and minimum value is poor.

As improvement project of the present invention, the concrete steps of in described III. edge detecting step, the edge image after binaryzation being carried out to refinement are: take image median vertical line as boundary, detected image is divided into left half-image and right half-image; Left half-image is carried out by picture element scan from horizontal and vertical direction, when running into white point, if rear, be some white point just by this point deletion, otherwise reservation, last pixel of every row does not make a decision, pixel value is directly made as 0; Right half-image is carried out by picture element scan from horizontal and vertical direction, if run into white point, if more front, be also white point just by this point deletion, otherwise retain, first pixel of every row is directly made as stain.

As improvement project of the present invention, in described III. edge detecting step, the concrete steps of image noise Transformatin are: by being communicated with scale of notation, remove noise, line scanning is from bottom to top carried out in image border after refinement, when finding white point, take this point as starting point, grow, and record is communicated with white point number, if be communicated with white point number, be less than 3, think shot noise, remove white point, reservation connected pixel is counted and is more than or equal to 3 white point connection edge.

The present invention also provides a kind of precompressed track time estimation method based on cross track distance, it is characterized in that, comprising:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence trend model, the estimation precompressed track time.

The present invention is by adopting local maximal difference extraction Image edge gradient, edge image to carry out binary conversion treatment, the edge image after binaryzation is carried out to refinement, noise Transformatin, improve antinoise and jamproof ability that deviation is reported to the police, lane departure warning reliability is improved, reduce false alarm rate and false dismissed rate, and by precompressed track time estimation method, improve the accuracy of estimating the precompressed track time, to provide actv. lane departure warning.

Accompanying drawing explanation

Fig. 1 is the schematic flow sheet of lane departure warning method of the present invention;

Fig. 2 is the schematic flow sheet of the squeeze time estimation steps in lane departure warning method of the present invention;

Fig. 3 is the schematic diagram in image detection region definite in the present invention;

Fig. 4 is the computation process schematic diagram that the local maximal difference of the employing in the present invention extracts Image edge gradient;

Fig. 5 is that frame data schematic diagram is read in the circulation of the precompressed track time estimation method in the present invention;

Fig. 6 is the cross track distance point sequence trend model schematic diagram of the precompressed track time estimation method in the present invention;

Fig. 7 shows the whole implementation schematic flow sheet of lane departure warning method provided by the invention;

Fig. 8-1 to Fig. 8-3 show under different road conditions environment, use the effect schematic diagram of lane departure warning method inspection vehicle diatom provided by the invention.

The specific embodiment

Below in conjunction with accompanying drawing, specifically illustrate embodiments of the present invention, accompanying drawing is only for reference and use is described, does not form the restriction to scope of patent protection of the present invention.

As shown in Figure 1, 2, the embodiment of the present invention provides a kind of lane departure warning method, comprising:

I. video acquisition device is installed and parameter calibration step, specifically comprises:

(i) camera is installed in automobile lateral center position, the ambient parameter of recording of video harvester: camera and level ground height D, camera are to vertical distance dl and dr, the camera angle of depression θ ° of vehicle right and left both sides; In the present embodiment, video acquisition camera is arranged on after windshield, near vehicle mirrors, guarantees that camera is in automobile lateral center position as far as possible, and shooting direction and car body parallel longitudinal, so that the calculating of video acquisition and deviation;

(ii) demarcate video acquisition device inner parameter, determine the mapping relations of image coordinate system and world coordinate system;

II. image pre-treatment step, comprise acquisition testing region coloured image, determine surveyed area, image carried out to gray processing processing and medium filtering denoising;

Fig. 3 shows image detection region.Image-region more than automobile engine cover and between below the visual end horizon of road is as lane detection region, to reduce sky, branch, high mountain and the interference of traffic indicating image to lane detection.Determining of region can, when device normalization, arrange in the enterprising pedestrian's work of software interface.

The lane departure warning method that the embodiment of the present invention provides also comprises:

III. edge detecting step, comprises and adopts local maximal difference extraction Image edge gradient, employing maximum variance between clusters edge image to carry out binary conversion treatment, the edge image after binaryzation is carried out to refinement, noise Transformatin; In an embodiment of the present invention, by maximum variance between clusters (OTSU) binary conversion treatment, white point value 255, stain value 0.

IV. lane mark identification step, goes out many straight lines by Hough change detection, selected satisfactory lane mark;

V. lane departure warning time determining step, specifically comprises:

(i) vehicle yaw is apart from calculation procedure: ambient parameter, inner parameter and the coordinate transform relation of according to video acquisition device, demarcating calculate vehicle yaw distance;

(ii) precompressed track time estimation step, specifically comprises:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence trend model, the estimation precompressed track time;

(iii) deviation determining step: set deviation time threshold, when the described precompressed track time is less than deviation time threshold, carry out lane departure warning.

As improvement project of the present invention, video camera imaging is that real three-dimensional world is projected to two-dimensional camera plane, obtain the movable information of vehicle at real world, need to determine the coordinate transform relation of real world and image.

Described I. video acquisition device is installed and parameter calibration step is defined as the mapping relations of image coordinate system and world coordinate system:

$X_w=\frac{b(x-x_c)(Z_w-D)}{a\×\cos \mathrm{\θ}+(y_c-y)\sin \mathrm{\θ}}$

$Y_w=\frac{(Z_w-D)[(y-y_c)\sin \mathrm{\θ}+a\×\sin \mathrm{\θ}]}{a\×\cos \mathrm{\θ}+(y_c-y)\sin \mathrm{\θ}}$

Wherein, definition intrinsic parameters of the camera a=f × Ny, b=Ny ÷ Nx, establishes Nx, and Ny represents respectively the proportionality coefficient of two coordinate directions, is the uncertain graphical rule factor by camera, and CCD sensitization is apart from waiting inner parameter jointly to determine; A, the value of b can by the pavement marker priori value of background image and coordinate transform relation is counter be released.F is camera focal length; Computer picture resolution is L × W, and (xc, yc) is image coordinate system, and (Xw, Yw, Zw) is world coordinate system;

(i) vehicle yaw in described v. lane departure warning time determining step is specially apart from calculation procedure:

Lane mark equation in coordinates under image coordinate system is:

x=ky+b’

Lane mark equation in coordinates under world coordinate system is:

X _w=KY _w+B,Z _w=0

According to the mapping relations of image coordinate system and world coordinate system, calculate:

$K=\frac{\mathrm{kba}\cos \mathrm{\θ}+(b^{\′}-x_c+0.5\mathrm{Wk})b\sin \mathrm{\θ}}{a}$

$B=\frac{D[\mathrm{kba}\sin \mathrm{\θ}+(0.5L-b^{\′}-0.5\mathrm{Wk})b\cos \mathrm{\θ}]}{a}$

In world coordinate system, K and cos θ trend 0, camera is to lane mark distance: d ≈ B=Dkb;

L, W are the horizontal and vertical size of computer picture, k, and b ' is equation of straight line slope and the intercept under image coordinate system;

Thus, calculate the distance L l=d-dl of car body left side to left-hand lane line, right side Lr=d-dr.

In the present embodiment, in described III. edge detecting step, adopt local maximal difference to extract Image edge gradient, be specially: adopt the border of texture filter detected image, template size is set, in output template neighborhood, maxim and minimum value is poor.As shown in Figure 4, acquiescence template size is 3 × 3 to the computation process of local maximal difference, and each numerical value is got the poor of maxim Nmax in 8 numerals of neighborhood inside circumference and minimum value Nmin.

The binary image splitting from edge gradient image, the edge obtaining is thicker, and noise is also many, is unfavorable for next step straight-line detection, and the present invention respectively edge has carried out refinement and denoising.

The concrete steps of in an embodiment of the present invention, in described III. edge detecting step, the edge image after binaryzation being carried out to refinement are: take image median vertical line as boundary, detected image is divided into left half-image Img_Left and right half-image Img_Right; To left half-image Img_Left from bottom to top, from left to right carry out by picture element scan, when running into white point, if rear, be some white point just by this point deletion, otherwise reservation, last pixel of every row does not make a decision, pixel value is directly made as 0; To right half-image Img_Right from bottom to top, from left to right carry out by picture element scan, if run into white point, if more front, be also white point just by this point deletion, otherwise retain, first pixel of every row is directly made as stain.

In an embodiment of the present invention, in described III. edge detecting step, the concrete steps of image noise Transformatin are: by being communicated with scale of notation, remove noise, line scanning is from bottom to top carried out in image border after refinement, when finding white point, take this point as starting point, grow, and record is communicated with white point number, if be communicated with white point number, be less than 3, think shot noise, remove white point, reservation connected pixel is counted and is more than or equal to 3 white point connection edge.In order to prevent double counting, the white point value that is judged as connection edge is made as to 100, to distinguish untreated edge, when having traveled through after a frame detected image, can by being communicated with the edge value of establishing, be 255 again.

The present invention also provides a kind of precompressed track time estimation method based on cross track distance, without knowing yaw angle, comprising:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence; Fig. 5 for circulation read frame data schematic diagram (frame data comprise time t and camera to lane mark apart from d), when data amount check is less than n, successively data are deposited in stack, when data are greater than n, displacement is upgraded successively, organizes data complete the estimation to yaw angle with n, realizes in real time and detecting;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence

Trend model, the estimation precompressed track time.Fig. 6 is the coordinate relation of vehicle avris to distance d and the time t of lane mark, point sequence is carried out to matching with straight line.Also can carry out matching with the higher curve model of precision, but in practical engineering application, straight line model can meet the demands, and calculates, realizes fairly simple.

Drawn range points Sequence Trend model, just can be in the hope of t coordinate axle intersection point t ', setting threshold thr, when the time t ' <thr of precompressed track, provides lane departure warning.

Fig. 7 shows the whole implementation schematic flow sheet of lane departure warning method provided by the invention.

Precompressed track time estimation step shown in Fig. 2 is the schematic flow sheet of the precompressed track time estimation method based on cross track distance provided by the invention.

Fig. 8-1 to Fig. 8-3 show under different road conditions environment, use the effect schematic diagram of lane departure warning method inspection vehicle diatom provided by the invention.Fig. 8-1 to Fig. 8-3 show respectively three kinds of lane detection design sketchs under road conditions environment, and road conditions environment is: daytime-front have car, evening-have street lamp, evening-without street lamp.

Above disclosed is only preferred embodiment of the present invention, can not limit the scope of the present invention with this, and the equivalent variations of therefore doing according to the present patent application the scope of the claims, still belongs to the scope that the present invention is contained.

Claims (5)

1. a lane departure warning method, comprising:

I. video acquisition device is installed and parameter calibration step, specifically comprises:

(i) camera is installed in automobile lateral center position, the ambient parameter of recording of video harvester: camera and level ground height D, camera are to vertical distance dl and dr, the camera angle of depression θ ° of vehicle right and left both sides;

(ii) demarcate video acquisition device inner parameter, determine the mapping relations of image coordinate system and world coordinate system;

II. image pre-treatment step, comprise acquisition testing region coloured image, determine surveyed area, image carried out to gray processing processing and medium filtering denoising;

It is characterized in that, also comprise:

III. edge detecting step, comprises and adopts local maximal difference extraction Image edge gradient, employing maximum variance between clusters edge image to carry out binary conversion treatment, the edge image after binaryzation is carried out to refinement, noise Transformatin;

IV. lane mark identification step, goes out many straight lines by Hough change detection, selected satisfactory lane mark;

V. lane departure warning time determining step, specifically comprises:

(i) vehicle yaw is apart from calculation procedure: ambient parameter, inner parameter and the coordinate transform relation of according to video acquisition device, demarcating calculate vehicle yaw distance;

(ii) precompressed track time estimation step, specifically comprises:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence trend model, the estimation precompressed track time;

(iii) deviation determining step: set deviation time threshold, when the described precompressed track time is less than deviation time threshold, carry out lane departure warning.

2. lane departure warning method according to claim 1, is characterized in that:

(i) vehicle yaw in described v. lane departure warning time determining step is specially apart from calculation procedure:

Calculate the distance L l=d-dl of car body left side to left-hand lane line, right side Lr=d-dr;

Wherein, camera is to lane mark distance: d ≈ B=Dkb; The expression formula of straight line intercept B under world coordinate system is:

$B=\frac{D[\mathrm{kba}\sin \mathrm{\&theta;}+(0.5L-b^{\&prime;}-0.5\mathrm{Wk})b\cos \mathrm{\&theta;}]}{a}$

Wherein, L, W are the horizontal and vertical size of computer picture, and k, b ' are equation of straight line slope and the intercept under image coordinate system, and a, b are intrinsic parameters of the camera.

3. lane departure warning method according to claim 1, is characterized in that:

The concrete steps of in described III. edge detecting step, the edge image after binaryzation being carried out to refinement are: take image median vertical line as boundary, detected image is divided into left half-image and right half-image; Left half-image is carried out by picture element scan from horizontal and vertical direction, when running into white point, if rear, be some white point just by this point deletion, otherwise reservation, last pixel of every row does not make a decision, pixel value is directly made as 0; Right half-image is carried out by picture element scan from horizontal and vertical direction, if run into white point, if more front, be also white point just by this point deletion, otherwise retain, first pixel of every row is directly made as stain.

4. lane departure warning method according to claim 1, is characterized in that:

In described III. edge detecting step, the concrete steps of image noise Transformatin are:

By being communicated with scale of notation, remove noise, line scanning is from bottom to top carried out in image border after refinement, when finding white point, take this point as starting point, grow, and record connection white point number, if be communicated with white point number, be less than 3, think shot noise, remove white point, reservation connected pixel is counted and is more than or equal to 3 white point connection edge.

5. the precompressed track time estimation method based on cross track distance, is characterized in that, comprising:

1. obtain vehicle yaw range points sequence: when vehicle starts to depart from non-zone of alarm, every processing one two field picture, obtains the vehicle yaw distance in this moment;

2. analyze vehicle yaw range points sequence, in its short time, some distribution near linear, when vehicle yaw distance reaches default driftage threshold value, starts circulation and obtains vehicle yaw range points sequence;

3. according to the cross track distance point sequence in the setting-up time section of obtaining, set up cross track distance point sequence trend model, the estimation precompressed track time.

Images