CN101894271A - Visual Calculation and Early Warning Method of Vehicle Deviation Lane Angle and Distance - Google Patents

Abstract

The invention relates to a visual calculation and early warning method for the angle and distance of a vehicle deviating from a lane line, which uses image processing and computer vision technology to calculate the angle and distance of a vehicle deviating from a lane line in real time according to the road surface image obtained by a vehicle-mounted camera, and estimate the time of crossing the line Used for security warnings. The implementation steps are as follows: first, detect the lane line on the road surface image to obtain the linear equation of the local lane line; establish a three-dimensional coordinate system with the camera as the origin, and record the installation height and depression angle of the camera; Calibrate its focal length; calculate the deflection angle and vertical distance of the car relative to the lane line according to the pinhole camera model; estimate the time to leave the lane according to the real-time driving speed of the car, and obtain the safety warning or intelligent control information of the driving car.

Description

Visual Calculation and Early Warning Method of Vehicle Deviation Lane Angle and Distance

technical field

The invention relates to a fast calculation and safety warning method for real-time calculation of the angle and distance of a vehicle deviating from a lane line based on road surface images acquired by a vehicle-mounted camera by applying image processing and computer vision technology. First, lane line detection is performed on the road surface image to obtain the linear equation of the local lane line; a three-dimensional coordinate system is established with the camera as the origin, and the installation height and depression angle of the camera are recorded; the focal length is calibrated, and then the vehicle relative The deflection angle and vertical distance of the lane line, the estimated time to leave the lane, and provide effective information for safety warning or intelligent control during driving.

Background technique

In recent years, the autonomous driving technology of automobiles has developed rapidly and has gradually made progress. The autonomous driving systems developed by various research institutions can already drive autonomously at high speed on structured roads (expressways), and have various intelligent functions. In the huge and complex modern traffic system, ensuring driving safety is the primary goal. Automotive active safety technology is to achieve the purpose of assisting driving by monitoring, adjusting and controlling various parameters of vehicle operation, and the lane departure warning system is It is one of the main technologies.

The purpose of researching the lane departure warning system is to give an alarm to the dangerous situation that the vehicle is about to leave the lane. This dangerous situation is mostly caused by the driver's lack of concentration, sleepiness, fatigue and other reasons, and belongs to unconscious lane departure. Therefore, the lane departure warning system fundamentally warns the driver of the bad driving state, and the computer vision measurement method has become a mainstream research direction because of its intuition, ease of use and reliability. This method integrates the three systems of people, vehicles and roads, and obtains the state of people by studying the relationship between vehicles and roads.

Lane departure warning systems are directly dependent on the vehicle's speed, direction and distance from the lane markings. Among them, the driving speed is easy to be directly extracted from the electronic system on the car, but the driving direction and the distance from the lane line are not easy to obtain. The single-point preview optimal curvature model proposed by Professor Guo Konghui of Jilin University uses the Ackerman geometric relationship when the vehicle is turning and the balance of the horizontal and vertical internal forces during steady steering to determine the target steering angle and target roll angle respectively, and uses ADAMS software to establish the driving The driver-vehicle closed-loop dynamics model is simulated according to the two typical driving conditions of double lane change and snaking. The established driver model is suitable for the dynamics simulation research of the monorail vehicle human-vehicle closed-loop control model. the

As the current lane detection technology based on image analysis is very mature, scholars have proposed a variety of lane detection algorithms, even in complex environments such as urban roads, lane detection has good results. Since the distribution of lanes can be approximated as a straight line in a local area, the present invention uses image analysis technology to detect lanes, obtains the lane line equation, and establishes a three-dimensional coordinate system with the camera as the origin; combined with the spatial information of the camera installation, according to the pinhole camera model Calculating the deflection angle and vertical distance of the car relative to the lane line, and obtaining the early warning information of the car's deviation based on the car-lane relationship has become an important technical link in the intelligent transportation system and autonomous driving to ensure driving safety.

Contents of the invention

Aiming at the shortcomings of the existing early warning method for vehicle deviation, the model is complex, the amount of calculation is large, and the environment is highly dependent, the purpose of the present invention is to provide a visual calculation and early warning method for the angle and distance of the vehicle deviation from the lane line. The method utilizes image analysis technology Complete the lane detection, and use the visual calculation method to obtain the spatial position and driving direction of the car relative to the road lane line, as the early warning information of the car's deviation in intelligent driving.

The present invention comprises the following steps

a) Install the CCD video camera in the car or on the roof along the front direction of the car, adjust the depression angle and focal length to make a clear image of the road within 50m ahead, and record the installation height h and depression angle θ of the camera;

b) The road surface image sequence collected by continuous shooting of the road surface by the camera, and the data acquisition is realized through the digital video high-speed channel of the DSP;

c) Detect the lane lines in the road image, including:

图像边沿检测。利用两个5×5模板分别对图像进行乘—加运算，得到对应于水平和垂直方向的两幅梯度图像，由这两幅图像求出原图的边沿图像。边沿图像将凸显图像的轮廓特征，尤其是车道线的边沿信息；

Image edge detection. Using two 5×5 templates to multiply and add the image respectively, two gradient images corresponding to the horizontal and vertical directions are obtained, and the edge image of the original image is obtained from these two images. The edge image will highlight the contour features of the image, especially the edge information of the lane line;

图像的二值化。对边沿图像采用Otsu算法计算出自适应阈值，依据该阈值对图像进行黑白两色二值化处理；

Image binarization. For the edge image, the Otsu algorithm is used to calculate the adaptive threshold, and the image is binarized in black and white according to the threshold;

③Remove the horizontal outline: In order to reduce the number of white points and reduce the calculation amount of subsequent processing, according to the characteristics that the lane line does not appear horizontal in the picture, the horizontal edge points are merged, that is, only the most white points that appear continuously in the horizontal direction are kept. The first white point on the left, so as to delete the horizontal edge and reduce the number of white points without affecting the lane detection effect;

④ Apply constraints to points. Set constraint conditions according to the brightness, width and continuity characteristics of the actual lane line distribution to further remove interference points;

⑤ Obtain the lane line equation. After the above series of treatments, the number of white spots is effectively reduced. Hough transform is a widely used straight line detection method, which determines the most likely straight line position by "voting", and is used to realize lane line detection in the present invention. Reintroduce constraint conditions in the detection process, and set the minimum number of votes. When there are more than 4 straight lines that meet the number of votes, keep the 4 straight lines with the highest number of votes as lane candidate lines, and finally get the straight line equation of 0 to 4 lane lines;

d) Calibrate the focal length parameter f of the camera. Park the vehicle with the camera installed in a direction at a known angle to the lane line, and calculate the focal length f based on the detected lane line equation, which is used for the calculation of the deflection angle and vertical distance during subsequent vehicle driving;

e) Calculate the deflection angle β and vertical distance d of the car relative to the lane line: During the driving process of the car, the lane line of each frame image is detected in real time and the straight line equation is obtained, based on the height h and depression angle θ recorded in step a) and step d) Calculate the deflection angle β and the vertical distance d from the focal length f obtained by calibration;

f) According to the instant driving speed of the car and the deflection angle β and distance parameter d obtained in step e), calculate the time required for the car to cross the lane line, set the alarm threshold, if the calculated time for crossing the line is less than the threshold, then give a warning message to remind The driver handled it in a timely manner.

Description of the operating results of the method mentioned in the patent of the present invention:

(1) The current lane detection methods almost all use the straight line detection method of Hough transform. Due to the large amount of calculation of Hough transform, it affects the real-time performance. Due to the adoption of various effective constraint conditions in this method, the points participating in Hough transform voting are greatly reduced, generally only dozens, and the speed is improved. Experiments have proved that it can fully meet the requirements of real-time detection;

(2) Since the effective constraints remove most of the interference of non-lane image information, the method has strong anti-interference ability. The experiment proves that the agreement has a satisfactory reliability in the complex environment of urban roads;

(3) The method of on-site calibration reduces the influence caused by the error of the camera itself and improves the accuracy;

(4) The angle calculation method adopts the geometric calculation form, which improves the calculation speed.

In short, compared with the current similar research results, this method has its own characteristics in the system's environmental adaptability, calculation speed, and reliability, and is more conducive to meeting the actual application requirements.

Description of drawings

Figure 1 is a real picture of road lane lines captured by the CCD camera during the experiment.

Figure 2 is the lane detection result obtained in step c).

Figure 3 is a schematic top view of the road surface.

Figure 4 is a schematic side view of the road surface.

Figure 5 is a model diagram of the space coordinate system used in the algorithm.

Detailed ways

The implementation process of the present invention will be further described below in conjunction with a non-limiting example, see Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5.

The present invention focuses on the description of the method. The experimental device used in the implementation, including a camera and a DSP image processing board, is realized by general-purpose equipment on the market. Image acquisition and size scaling are also popular technologies, and will not be described in detail. In order to ensure the integrity of the description of the implementation process, some common technologies will be involved, which do not have the nature of patent protection, and will be clearly stated in the claims.

Embodiments of the present invention are as follows:

(a) Install a CCD video camera along the front of the car, adjust the depression angle and focal length to make a clear image of the road within 50m ahead, and record the installation height h and depression angle θ of the camera, as shown in Figure 4.

(b) Lane line detection in road images, including:

图像边沿检测：利用式(1)、(2)的两个5×5模板分别对图像进行乘—加运算，得到对应于水平和垂直方向的两幅梯度图像

和

,然后由公式得到边沿图像

；边沿图像保留了图像的轮廓信息，尤其是车道线的边沿信息：

Image edge detection: use the two 5×5 templates of formulas (1) and (2) to multiply and add the image respectively, and obtain two gradient images corresponding to the horizontal and vertical directions

and

, then by the formula get edge image

;The edge image retains the contour information of the image, especially the edge information of the lane line:

            (1)

(1)

            (2)

(2)

对边沿图像的二值化处理。对边沿图像采用Otsu算法计算出自适应阈值，并对图像进行二值化处理;

Binarization of edge images. The Otsu algorithm is used to calculate the adaptive threshold for the edge image, and the image is binarized;

③ Remove the horizontal outline. Scan the binary image in the horizontal direction and examine two consecutive points: if a point is a white point and the previous adjacent point is a black point, then keep the white point, and turn all points that do not meet this condition into black points. That is, its gray value is cleared to 0. In this way, the horizontal outline can be removed, the number of white points can be greatly reduced, and the interference of other traffic signs can be eliminated without affecting the detection effect of lane lines;

④ Apply constraints to further eliminate white spots. Set constraints according to the characteristics of the actual lane to further remove interference points. Three constraint conditions are used in the present invention: the first, the lane line is brighter than the road surface, and the difference between the gray value of the pixel on the lane line and the gray value on the side road is not less than 20; second, the width constraint of the lane line , generally between 2 and 20 pixels, and there is at least one empty (no white point) area of more than 40 pixels on the left and right sides; third, the continuity limit, the distribution of white points should meet or approximately meet (left and right Deviation does not exceed one pixel) straight line features. Delete the white dots that do not meet the above constraints;

。其中为横坐标，

为纵坐标，

为斜率，

为截距。见图1、图2。⑤ Obtain the lane line equation. After the above series of processing, the number of white points is effectively reduced, and then the Hough transform is used for straight line detection. In this process, constraints are introduced: first, the number of lane lines in the image is limited to no more than 4; second, for visual reasons, the angle difference between two lane lines cannot be less than 5--; third, the lane line Intersections cannot be allowed at the bottom of the screen; Fourth, the number of votes for Hough transformation is limited, and less than 15 votes are considered as interference information. After the above constraints, select the 0-4 straight lines with the most votes as the lane lines. With the center of the image as the origin, calculate the straight line equation of each lane line in the image plane

. in is the abscissa,

is the vertical coordinate,

is the slope,

is the intercept. See Figure 1 and Figure 2.

，连同步骤a)记录的θ值，依据式(3)计算出焦距f（说明：式(3)由β=45--导出）,用于后续车辆行驶过程中对偏转角和距离的计算：(c) Calibrate the focal length parameters of the camera: see Figure 3, park the vehicle with the camera installed in a direction where the angle β with the lane line is known, according to the detected lane line equation

, together with the θ value recorded in step a), calculate the focal length f according to formula (3) (Note: formula (3) is derived from β=45--), which is used for the calculation of deflection angle and distance during subsequent vehicle driving:

                      (3)。

(3).

，依据步骤a)记录的高度h和俯角θ以及步骤d)定标得到的焦距f，借助图5所示的空间坐标系，采用针孔摄像机模型可以推导出偏转角β和垂直距离d的计算公式，见式(4)、(5)：(d) Calculate the deflection angle β and vertical distance d of the car relative to the lane line, see Figure 3. During the driving process of the car, the lane line of each frame image is detected in real time and the straight line equation is obtained

, according to the height h and depression angle θ recorded in step a) and the focal length f obtained by calibration in step d), with the help of the space coordinate system shown in Figure 5, the calculation of deflection angle β and vertical distance d can be deduced by using the pinhole camera model Formula, see formula (4), (5):

(4)

                 (5)

(5)

If multiple lane lines appear, the deflection angle and vertical distance of the car relative to each lane line can be calculated.

以及步骤e)得到的β和d计算汽车超越车道线所需时间t,见式(6)(e) According to the real-time driving speed of the car

And the β and d obtained in step e) calculate the time t required for the car to surpass the lane line, see formula (6)

                       (6)

(6)

Set T as the alarm time. T is related to the driver's reaction speed and the braking effect of the car. When t<T, a warning message will be given to ensure the driving safety of the car.

Claims (4)

1. the vision of an automobile run-off-road line angle and distance is calculated and method for early warning, may further comprise the steps:

A) with the CCD video frequency pick-up head along the automobile dead ahead to being installed in Che Nei or roof, adjust the angle of depression and focal length and make it, the setting height(from bottom) h and the angle of depression θ of record camera road surface blur-free imaging in the 50m of the place ahead;

B) by the camera road pavement carry out continuous shooting, collecting to the pavement image sequence, the digital video high-speed channel by DSP is realized data acquisition;

C) lane line in the road pavement image detects, and comprising:

The image edge detects: utilize two 5 * 5 templates respectively computing is taken advantage of-added to image, obtain two width of cloth gradient images corresponding to level and vertical direction, obtained the side information of edge image, the especially lane line of former figure by this two width of cloth image;

The binaryzation of image: adopt the Otsu algorithm computation to go out adaptive threshold to the edge image, image is carried out the black-and-white two color binary conversion treatment according to this threshold value;

3. remove horizontal profile: thus the subsequent treatment calculated amount reduced in order to reduce white point quantity, the characteristics that in picture, can not occur the level trend according to lane line, the horizontal edge point is merged, promptly the white point that horizontal direction is occurred continuously only keeps first white point of Far Left, thereby deletion horizontal edge, reduce white point quantity, the lane detection effect is not exerted an influence again simultaneously;

4. the application constraint condition is gone a little: set constraint condition according to brightness, width and continuity Characteristics that actual lane line distributes, further remove noise spot;

5. obtain the lane line equation: adopt the Hough conversion to determine most probable linear position, finally obtain the straight-line equation of 0～4 lane line;

D) the focal length parameter f of camera is demarcated: the vehicle that will install camera is parked on the direction that becomes known angle with lane line, discharge of the coke apart from f according to detected lane line Equation for Calculating, be used for of the calculating of follow-up vehicle ' process deflection angle and vertical range;

E) the deflection angle β and the vertical range d of the relative lane line of calculating automobile: in vehicle traveling process, lane line to every two field picture detects and obtains straight-line equation in real time, calculates deflection angle β and vertical range d according to the focal distance f that the height h of step a) record and angle of depression θ and step d) calibration obtain according to following formula:

、

；

F) deflection angle β and the distance parameter d that obtains according to the instant travel speed and the step e) of automobile is according to formula Calculate automobile and surmount lane line required time t, set alarm threshold value, if calculate to such an extent that the line time then provides information warning less than threshold value more.

2. the vision of the described automobile run-off-road of claim 1 line angle and distance is calculated and the pre-police, it is characterized in that: step c) 3. in for reducing white point quantity, the edge bianry image has been removed horizontal profile, bianry image is carried out horizontal direction scanning, investigate continuous two points: if certain point is a white point, and the front consecutive point are stain, then keep this white point, and the institute that does not satisfy this condition is become stain a little.

3. the vision of the described automobile run-off-road of claim 1 line angle and distance is calculated and the pre-police, it is characterized in that: step c) 4. in for reducing white point quantity, adopt constraint condition to remove white point, adopted three constraint conditions: the first, on the lane line on gray values of pixel points and the next door road difference of gray-scale value be not less than 20; The second, the pixel of lane line along continuous straight runs is between 2～20, and there is a stain zone that is no less than 40 pixels at least in the outer the right and left of line; The 3rd, white point should meet on distributing or approximate meeting is the linear feature that left-right deviation is no more than a pixel; The white point deletion of above-mentioned constraint condition will do not met.

4. the vision of the described automobile run-off-road of claim 1 line angle and distance is calculated and the pre-police, it is characterized in that: step c) retrains the quantity and the distribution of lane line in 5.: the first, limit in the image no more than 4 of lane line quantity; The second, according to the vision reason, the differential seat angle of two lane lines can not be less than 5 degree; The 3rd, lane line can not allow to occur intersecting below picture; The 4th, the ballot quantity of Hough conversion is limited, abandon being less than the testing result of 15 tickets; Calculate the straight-line equation of every lane line in the plane of delineation at last.

Images