CN101294801A - Vehicle distance measurement method based on binocular vision - Google Patents

Abstract

The vehicle distance measurement method based on binocular vision provided by the present invention adopts the self-adaptive front vehicle distance measurement system of stereo vision to obtain the parameters required for measurement, and the stereo vision self-adaptive front vehicle distance measurement system includes A camera unit for information on the road ahead, the camera unit includes a first camera C1 and a second camera C2; the coordinate system of any camera in the camera unit is selected as the world coordinate system of the entire measurement process, and the optical center of the camera is used as The origin of the world coordinate system is recorded as 0, the positional relationship between the cameras in the camera unit is calibrated, the size of the lane area where the vehicle is located and the lane line information are acquired, and the acquired lane area size and lane line information of the vehicle is The information is recorded as S1; thereby improving the accuracy of measurement and ensuring the safety of driving.

Description

Vehicle distance measurement method based on binocular vision

technical field

The invention relates to a vehicle distance measurement method between a front vehicle and a rear vehicle.

Background technique

As an important part of the Intelligent Transportation System (ITS), the safety system of the vehicle has always been an important research field, and the distance measurement technology of the vehicle ahead is one of the most critical technologies in the safety system of the vehicle. The vehicle-to-vehicle distance measurement technology in front is not only the frontier and hot spot in the field of current vehicle safety system research, but also a high-tech combination of computer vision technology and electronic technology. The accuracy of distance measurement to vehicles ahead largely determines the performance of vehicle safety systems.

In the current vehicle distance measurement system, the vision-based distance measurement method is recognized as the most promising method. Most of the current visual distance measurement systems in the existing technology use the front vehicle based on the principle of monocular vision. The method of distance measurement, this method installs the camera on the vehicle in a certain fixed way, measures the height and pitch angle of the installed camera in advance, and realizes the measurement of the distance to the vehicle ahead through a series of formula calculations. Although adopting this kind of method can make the design of the system simple, the cost is low and the calculation complexity of the vehicle distance is reduced, but there are still some problems. First of all, since the measurement results of this method are strongly dependent on the camera installation height and pitch angle measured in advance, as the vehicle travels, the camera position will inevitably change relative to the original installation position due to the shaking effect, thus changing the camera parameters. , especially the pitch angle, which will bias the measurement results. Secondly, this method of monocular vehicle distance measurement is more accurate for the measurement of the measured point near the ground. If the measured point on the vehicle in front is at a certain height from the ground, there will be a large error in the measurement result at this time. When the vehicle distance is relatively short, the error is more obvious, which will seriously affect the safety of the vehicle.

On the other hand, as an advanced visual measurement method, binocular stereo vision has been widely used. Stereo vision can directly restore the three-dimensional coordinates of the measured point, so that the spatial distance information can be obtained accurately. Therefore, the use of stereo vision to measure the distance of the vehicle ahead has better accuracy and reliability.

Contents of the invention

The purpose of the present invention is to provide a vehicle distance measurement method based on binocular vision, which solves the problem of large measurement errors in the prior art. The present invention can automatically measure the distance between the vehicle and the vehicle in front according to the vehicle outline information and road information It can automatically compensate the camera parameter changes caused by vehicle shaking, and improve the accuracy and stability of the system measurement.

For achieving above-mentioned technical purpose, the technical scheme that the present invention adopts is:

The parameters required for the measurement are obtained by using the adaptive front vehicle distance measurement system with stereo vision, the adaptive front vehicle distance measurement system with stereo vision includes a camera unit for photographing information on the road ahead, and the camera unit includes the first A camera C1 and a second camera C2; the first camera C1 and the second camera C2 are installed behind the windshield of the vehicle, and the coordinate system of any camera in the camera unit is selected as the world coordinate system of the entire measurement process, the The optical center of the camera is used as the origin of the world coordinate system, denoted as O, the positional relationship between the cameras in the camera unit is calibrated, the size of the lane area and the lane line information of the vehicle are obtained, and the acquired lane of the vehicle is obtained The area of the rectangular area formed by the area size and lane line information is recorded as S1;

Step 1: Detect and verify the partial rectangular area of the vehicle outline of the vehicle in front, and denote the area of the partial rectangular area of the vehicle outline of the obtained vehicle in front as S2;

Step 2: Compare the acquired rectangular area area S1 formed by the size of the lane area where the vehicle is located and lane line information with the obtained partial rectangular area area S2 of the vehicle outline of the vehicle in front, if S1/S2<= Threshold T, execute step 3, if S1/S2>threshold T, execute step 4, threshold T=2;

Step 3: Calculate the centroid of the rectangle of the partial rectangular area of the vehicle outline of the vehicle in front, denoted as P1, and perform step 5;

Step 4: Extract the point of any corner of the rectangular area of the license plate outline of the vehicle in front, denoted as P2;

Step 5: recovering the three-dimensional coordinates P of P1 or P2;

Step 6: Calculate the distance between OPs to get the distance between the front and rear cars.

In a preferred embodiment of the present invention, in the process of calculating the distance between front and rear vehicles, the external parameters of the camera in the camera unit can be dynamically adjusted at the same time, the method is as follows:

Step 11: Calculate the three-dimensional coordinates of points on the two lane lines of the lane where the vehicle in front is located, denoted as Pi;

Step 12: fitting the spatial plane V according to the three-dimensional coordinates Pi of points on the two lanes; the method for fitting the spatial plane is as follows: set the coordinates of the three-dimensional point P _i as (X _i , Y _i , Z _i ), and the plane equation AX+BY+CZ+1=0, then the purpose of fitting is to obtain the plane equation parameters A, B and C, and substitute the coordinates of the three-dimensional point _Pi into AX+BY+CZ+1=0 to form the following equations:

$\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; A</span>}\\ \mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}$

By solving the equations shown above, the plane equation of the space plane V can be obtained;

Step 13: Project the three-dimensional coordinates Pi of the points on the two lanes to the space plane V to obtain the plane two-dimensional coordinates Qi of the three-dimensional coordinates Pi on the space plane V;

Step 14: Carry out straight line fitting to the two-dimensional coordinates Qi of the plane to obtain two straight line vectors a and b; the method of straight line fitting is as follows: set the coordinates of the two-dimensional point Qi as ( _xi , y _i ), and there are m points, the straight line is y=a ₀ +a ₁ x, then the parameters a ₀ and a ₁ of the straight line equation can be obtained by using the formula of the least square method:

$\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; m</span>}& \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}\right]<span\; class="notranslate">\; ;</span>$

Step 15: judging whether the straight line vectors a and b are parallel, if not, recalibrating the positional relationship between the cameras in the camera unit to obtain new camera calibration parameters;

In a preferred embodiment of the present invention, the method steps of the obtained partial rectangular area of the vehicle outline of the vehicle in front are as follows:

1) On the standard road surface, it can be considered that in the lane area obtained above, the gray level change of the road between the own party and the vehicle in front is gentle in the image, but at the intersection of the road surface and the vehicle, due to the existence of shadows, it forms The edge of the image gray scale from bright to dark is considered to be the lower edge of the vehicle. On the road ahead of the vehicle, scan the profile of the vehicle in front according to the horizontal line to obtain an image of the profile of the vehicle in front, and calculate the average gray value RowAvr(r) of each line in the image, as shown in the following formula:

$\mathrm{RowAvr}\left(r\right)=\frac{1}{\mathrm{Right}\left(r\right)-\mathrm{Left}\left(r\right)}\underset{c=\mathrm{Left}\left(r\right)}{\overset{\mathrm{Right}\left(r\right)}{\mathrm{\&Sigma;}}}I(r,c),$ In the above formula, Right(r) represents the rightmost coordinate of the rth row of the lane area in the image; Left(r) represents the leftmost coordinate of the rth row of the lane area in the image; I(r, c) is the image (r, c) grayscale; RowAvr(r) represents the grayscale average value of the rth row in the lane area, and the r with the largest change in RowAvr(r) and RowAvr(r-1) is used as the lower edge of the vehicle in front, which is recorded as LowerPos;

2) Perform Sobel (SOBEL) edge extraction algorithm in the lane area to obtain the edge image of the vehicle in front of the lane area, and judge the position of the leftmost edge point and the rightmost edge in the [LowerPos-Tr, LowerPos] line of the edge image Point position, and consider it as the left edge and right edge of the vehicle in front, which are respectively recorded as LeftPos and RightPos; where Tr is 2~6, LeftPos and RightPos represent the leftmost and rightmost of the vehicle area respectively;

3) Assuming that the aspect ratio of the vehicle rectangle is Ta, then according to step 2), it can be known that the wide side of the partial rectangular area of the vehicle outline is RightPos-LeftPos, then the long side of the obtained rectangular area is (RightPos-LeftPos)*Ta, Ta The value is generally 0.5-1.0;

In a preferred embodiment of the present invention, said step 3 includes the following steps:

Step 31: Extract all edge points Pi in the partial rectangular area S2 of the vehicle outline of the vehicle in front, and put them into the edge sequence;

Step 32: Use the average value of the coordinates of the edge sequence Pi as the centroid coordinates, denoted as P1, and use P1 as the measurement point.

In a preferred embodiment of the present invention, said step 4 includes the following steps:

Step 41: using a Sobel edge operator to extract an edge image in the rectangular area of the license plate of the vehicle in front;

Step 42: Binarize the extracted edge image according to the image segmentation algorithm;

Step 43: Perform grayscale projection on the extracted edge image to obtain the upper limit a, lower limit b, left limit c and right limit d of the license plate;

Step 44: According to the obtained upper limit a, lower limit b, left limit c and right limit d of the license plate, obtain the upper left, lower left, upper right and lower right corners of the rectangle in the image. The coordinates of are (c, a), (c, b), (d, a) and (d, b);

Step 45: Locate a point on any corner in the license plate area, denoted as P2, and take P2 as a measurement point.

In a preferred embodiment of the present invention, while step 1 is being performed, the vehicle in front is tracked in combination with a Kalman filter algorithm.

In a preferred embodiment of the present invention, the extrinsic parameters of the camera include a rotation matrix R and a translation matrix T.

In a preferred embodiment of the present invention, in step 15, adopt Levenberg-Marquardt algorithm (Levenberg-Marquardt) whether parallel to vector a and vector b;

In a preferred embodiment of the present invention, performing grayscale projection on the extracted edge image in step 43 includes simultaneously performing horizontal grayscale projection and vertical grayscale projection on the extracted edge image. degree projection, and look for two maximum values from bottom to top for the horizontal grayscale projection map, as the upper limit and lower limit value of the front car license plate; look for two maximum values for the vertical grayscale projection map from bottom to top value, as the left limit value and right limit value of the license plate of the preceding vehicle.

In a preferred embodiment of the present invention, the projection method is: denote the obtained edge image as I, I(x, y) represents the gray value at the point (x, y) in the image, and the horizontal Grayscale projection is to fix the y coordinate for grayscale accumulation, the formula is as follows:

$h_j=\underset{i=0}{\overset{\mathrm{width}}{\mathrm{\&Sigma;}}}I(x_i,{\mathrm{the\; y}}_j),$ Where width represents the width of the image, and the _{y j} coordinates are fixed;

In the same way, the vertical grayscale projection is to fix the x coordinate for grayscale accumulation. The formula is as follows:

$V_j=\underset{i=0}{\overset{\mathrm{height}}{\mathrm{\&Sigma;}}}I(x_j,{\mathrm{the\; y}}_i),$ where height represents the height of the image, and x _j coordinates are fixed; for the license plate area, there are two maximum values in the horizontal grayscale projection _Hj , corresponding to the upper limit and lower limit of the license plate, and there are two maximum values in the vertical grayscale projection _Vj A maximum value, corresponding to the left limit value and right limit value of the license plate.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

Fig. 1 is the structural device schematic diagram that two cameras of the present invention are installed;

Fig. 2 is the flowchart of the vehicle distance measurement method based on binocular vision provided by the present invention;

3 is a flowchart of a method for dynamically adjusting external parameters of a camera in a camera unit provided by the present invention;

Fig. 4 is a flow chart of the method for obtaining the partial rectangular area of the vehicle profile of the vehicle in front provided by the present invention;

Fig. 5 is a flow chart of a method for calculating the centroid of a rectangle of a partial rectangular area of the vehicle outline of the vehicle in front provided by the present invention;

Fig. 6 is a flow chart of the method for extracting points at any corner of the rectangular area of the license plate outline of the preceding vehicle provided by the present invention.

Detailed ways

As shown in Figure 1, two cameras are installed behind the windshield of the vehicle, and the placement of the two cameras is adjusted so that the two cameras have a common view, and the common view can contain most of the information on the road ahead.

The high accuracy of stereo vision measurement depends on two prerequisites, one is that the measurement points of the front vehicle are correct, and the other is that the camera calibration parameters are correct.

Since the vehicle is a man-made rigid body, its tail image has the following characteristics:

(1) Compared with the natural scenery on the road, the edge of the vehicle has the characteristics of obvious and continuous edges, and the direction is approximately vertical or horizontal;

(2) The outline of the vehicle, when viewed from the rear, generally conforms to a rectangle;

(3) There is a large difference between the texture of the back of the vehicle and the texture of the road;

(4) There is a shadow at the bottom of the vehicle, which has obvious grayscale features, and is generally the darkest place on the road.

Therefore, according to the characteristics of the vehicle rear image, the measurement point of the vehicle in front can be determined according to different situations. As shown in Figure 2, a vehicle distance measurement method based on binocular vision, using stereo vision to measure the vehicle distance in front of the vehicle The system acquires the parameters required for measurement, and the self-adaptive vehicle distance measurement system in front of the stereo vision includes a camera unit for photographing information on the road ahead, and the camera unit includes a first camera C1 and a second camera C2; The first camera C1 and the second camera C2 are installed behind the windshield of the vehicle, and the coordinate system of any camera in the camera unit is selected as the world coordinate system of the entire measurement process, and the optical center of the camera is used as the world coordinate system. The origin, denoted as 0, calibrates the positional relationship between the cameras in the camera unit, (the positional relationship between the calibration cameras in the present invention utilizes the high-precision camera calibration algorithm based on the plane template proposed by Zhang Zhengyou ("A Flexible New Technique for Camera Calibration") ", Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2000, 11 (22): 1330-1334)), obtain the lane area size and lane line information where the car is located, (the present invention utilizes the lane area and lane line proposed in the following article Line acquisition algorithm (Zhu Wennan; Chen Qiang; Wang Hong "lane detection in some complex conditions", Intelligent Robots and Systems, 2006IEEE/RSJ International Conference on, 2006, 117-122), the lane area where the vehicle is located will be acquired The area of the rectangular area formed by the size and lane line information is denoted as S1;

Step 1: Detect and verify the partial rectangular area of the vehicle outline of the vehicle in front, and denote the area of the partial rectangular area of the vehicle outline of the obtained vehicle in front as S2;

The steps of the method for obtaining the partial rectangular area of the vehicle outline of the vehicle in front are as follows:

1) On the standard road surface, it can be considered that in the lane area obtained above, the gray level change of the road between the own party and the vehicle in front is gentle in the image, but at the intersection of the road surface and the vehicle, due to the existence of shadows, it forms The edge of the image gray scale from bright to dark is considered to be the lower edge of the vehicle. On the road ahead of the vehicle, scan the profile of the vehicle in front according to the horizontal line to obtain an image of the profile of the vehicle in front, and calculate the average gray value RowAvr(r) of each line in the image, as shown in the following formula:

$\mathrm{RowAvr}\left(r\right)=\frac{1}{\mathrm{Right}\left(r\right)-\mathrm{Left}\left(r\right)}\underset{c=\mathrm{Left}\left(r\right)}{\overset{\mathrm{Right}\left(r\right)}{\mathrm{\&Sigma;}}}I(r,c),$ In the above formula, Right(r) represents the rightmost coordinate of the rth row of the lane area in the image; Left(r) represents the leftmost coordinate of the rth row of the lane area in the image; I(r, c) is the image (r, c) grayscale; RowAvr(r) represents the grayscale average value of the rth row in the lane area, and the r with the largest change in RowAvr(r) and RowAvr(r-1) is used as the lower edge of the vehicle in front, which is recorded as LowerPos;

2) Perform Sobel (SOBEL) edge extraction algorithm in the lane area to obtain the edge image of the vehicle in front of the lane area, and judge the position of the leftmost edge point and the rightmost edge in the [LowerPos-Tr, LowerPos] line of the edge image Point position, and consider it as the left edge and right edge of the vehicle in front, which are respectively recorded as LeftPos and RightPos; where Tr is 2~6, LeftPos and RightPos represent the leftmost and rightmost of the vehicle area respectively;

3) Assuming that the aspect ratio of the vehicle rectangle is Ta, then according to step 2), it can be known that the wide side of the partial rectangular area of the vehicle outline is RightPos-LeftPos, then the long side of the obtained rectangular area is (RightPos-LeftPos)*Ta, Ta The value is generally 0.5-1.0; for the case where the aspect ratio of the truck is greater than 1, the vehicle rectangular area obtained at this time is only a part of the rectangle that actually contains the vehicle, but it does not affect the subsequent processing results.

Step 2: Compare the acquired rectangular area area S1 formed by the size of the lane area where the vehicle is located and lane line information with the obtained partial rectangular area area S2 of the vehicle outline of the vehicle in front, if S1/S2<= Threshold T, execute step 3, if S1/S2>threshold T, execute step 4, threshold T=2;

Step 3: Calculate the centroid of the rectangle of the partial rectangular area of the vehicle outline of the vehicle in front, denoted as P1, and perform step 5;

Calculate the centroid of the rectangle of the partial rectangle area of the vehicle outline of the vehicle in front, the method is as follows:

Step 31: Extract all edge points Pi in the partial rectangular area S2 of the vehicle outline of the vehicle in front, and put them into the edge sequence;

Step 32: Use the average value of the coordinates of the edge sequence Pi as the centroid coordinates, denoted as P1, and use P1 as the measurement point.

Step 4: Extract the point of any corner of the rectangular area of the license plate outline of the vehicle in front, denoted as P2;

The method for extracting the point of any corner of the rectangular area of the vehicle license plate outline in front is:

Step 41: using a Sobel edge operator to extract an edge image in the rectangular area of the license plate of the vehicle in front;

Step 42: Binarize the extracted edge image according to the image segmentation algorithm; (for the image segmentation algorithm, see: Kong Ming, Sun Xiping, Wang Yongji, "An Improved Threshold Segmentation Method Based on Variance Between Classes", Huazhong Science and Technology University Journal, 2007, 7];

Step 43: Perform grayscale projection on the extracted edge image to obtain the upper limit a, lower limit b, left limit c and right limit d of the license plate;

Step 44: According to the obtained upper limit a, lower limit b, left limit c and right limit d of the license plate, obtain the upper left, lower left, upper right and lower right corners of the rectangle in the image. The coordinates of are (c, a), (c, b), (d, a) and (d, b);

Step 45: Locate a point on any corner in the license plate area and record it as P2, and use P2 as a measurement point. Since the license plate area is considered as a rectangular area, any corner point (intersection point) of the rectangular area can be taken. But it must be noted that for the images captured by the left and right cameras, the corner positions must be unified, that is, if the upper left corner is selected in the license plate area of the left image, the upper left corner must also be selected in the license plate area of the right image.

Step 5: recovering the three-dimensional coordinates P of P1 or P2;

Step 6: Calculate the distance between OPs to get the distance between the front and rear cars.

In the process of calculating the distance between front and rear vehicles, the extrinsic parameters of the camera in the camera unit can be dynamically adjusted at the same time, the method is as follows:

Step 11: Calculate the three-dimensional coordinates of points on the two lane lines of the lane where the vehicle in front is located, and record it as Pi; according to (Zhu Wennan; Chen Qiang; Wang Hong "lane detection in some complex conditions", Intelligent Robots and Systems, 2006 IEEE /RSJ InternationalConference on, 2006, 117-122) algorithm to obtain the approximate straight line equations of the two lane lines of the lane where the vehicle in front is located, which are marked as L1 and L2 respectively, using the stereo matching algorithm of straight lines and the 3D reconstruction algorithm of points (stereo matching For the algorithm and 3D reconstruction algorithm, please refer to Ma Songde, Zhang Zhengyou. Computer Vision - Computational Theory and Algorithm Basis [C]. Beijing: Science Press, 1997). Obtain the 3D coordinates of the points on the two lane lines L1 and L2, and record them as Pi; for a binocular camera that has been calibrated, if the corresponding points in the left and right images are obtained, the 3D coordinates of the corresponding points can be restored using a 3D reconstruction algorithm. For the specific algorithm, see: Ma Songde, Zhang Zhengyou. Computer Vision - Computational Theory and Algorithm Basis [C]. Beijing: Science Press, 1997

Step 12: fitting the spatial plane V according to the three-dimensional coordinates Pi of points on the two lanes; the method for fitting the spatial plane is as follows: set the coordinates of the three-dimensional point P _i as (X _i , Y _i , Z _i ), and the plane equation AX+BY+CZ+1=0, then the purpose of fitting is to obtain the plane equation parameters A, B and C, and substitute the coordinates of the three-dimensional point _Pi into AX+BY+CZ+1=0 to form the following equations:

$\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&CenterDot;</span>\\ <span\; class="notranslate">\; \&CenterDot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>& <span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; A</span>}\\ \mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}$

The plane equation of the space plane V can be obtained by solving the equation system shown above by using the singular value decomposition method;

Step 13: Project the three-dimensional coordinates Pi of the points on the two lanes to the space plane V to obtain the plane two-dimensional coordinates Qi of the three-dimensional coordinates Pi on the space plane V;

Step 14: Carry out straight line fitting to the two-dimensional coordinates Qi of the plane to obtain two straight line vectors a and b; the method of straight line fitting is as follows: set the coordinates of the two-dimensional point Qi as ( _xi , y _i ), and there are m points, the straight line is y=a ₀ +a ₁ x, then the parameters a ₀ and a ₁ of the straight line equation can be obtained by using the formula of the least square method:

$\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; m</span>}& \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}\right]<span\; class="notranslate">\; ;</span>$

Step 15: Determine whether the straight line vectors a and b are parallel, if not parallel, then recalibrate the positional relationship between the cameras in the camera unit to obtain new camera calibration parameters; if not parallel, it means that the extrinsic parameters of the camera have occurred at this time At this time, the target image is taken again, and the high-precision camera calibration algorithm based on the plane template proposed by Zhang Zhengyou ("A Flexible New Technique for Camera Calibration", Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2000, 11(22) : 1330-1334) to re-calibrate the camera.

While step 1 is being carried out, the vehicle in front is tracked in combination with the Kalman filter algorithm. Kalman filter algorithm is an existing method, the references are: Zhao Li, Chen Quanlin, "Realization of Vehicle Detection and Tracking System Based on Kalman Filter", Electronic Measurement Technology, 2007, 30(2): 165-168.

The extrinsic parameters of the camera include the rotation matrix R and the translation matrix T.

Use the Levenberg-Marquardt algorithm (Levenberg-Marquardt) for whether the vector a and the vector b are parallel;

The grayscale projection of the extracted edge image described in step 43 includes performing horizontal grayscale projection and vertical grayscale projection on the extracted edge image at the same time, and the horizontal grayscale projection image from below Find two maximum values from top to top, as the upper limit and lower limit of the front vehicle license plate; look for two maximum values from bottom to top for the vertical grayscale projection map, as the left limit of the front vehicle license plate value and right limit.

The projection method is: the obtained edge image is denoted as I, I (x, y) represents the gray value at the point (x, y) in the image, and the horizontal gray projection is to fix the y coordinate for gray degree accumulation, the formula is as follows:

$h_j=\underset{i=0}{\overset{\mathrm{width}}{\mathrm{\&Sigma;}}}I(x_i,{\mathrm{the\; y}}_j),$ Where width represents the width of the image, and the _{y j} coordinates are fixed;

In the same way, the vertical grayscale projection is to fix the x coordinate for grayscale accumulation. The formula is as follows:

$V_j=\underset{i=0}{\overset{\mathrm{height}}{\mathrm{\&Sigma;}}}I(x_j,{\mathrm{the\; y}}_i),$ where height represents the height of the image, and x _j coordinates are fixed; for the license plate area, there are two maximum values in the horizontal grayscale projection _Hj , corresponding to the upper limit and lower limit of the license plate, and there are two maximum values in the vertical grayscale projection _Vj A maximum value, corresponding to the left limit value and right limit value of the license plate.

Claims (10)

1, a kind of distance measurement method based on binocular vision, it is characterized in that: adopt the self-adaptive front vehicle distance measurement system of stereo vision to obtain the parameter required for measurement, the self-adaptive front vehicle distance measurement system of described stereo vision Including a camera unit for photographing information on the road ahead, the camera unit includes a first camera C1 and a second camera C2; the first camera C1 and the second camera C2 are installed behind the windshield of the vehicle, and the camera is selected The coordinate system of any camera in the unit is used as the world coordinate system of the entire measurement process, and the optical center of the camera is used as the origin of the world coordinate system, which is recorded as 0. The positional relationship between the cameras in the camera unit is calibrated to obtain the vehicle The size of the lane area and the lane line information, the area of the rectangular area formed by the obtained lane area size and lane line information of the vehicle is recorded as S1; Step 1: Detect and verify the partial rectangular area of the vehicle outline of the vehicle in front, and denote the area of the partial rectangular area of the vehicle outline of the obtained vehicle in front as S2; Step 2: Compare the obtained area S1 of the rectangular area formed by the size of the lane area where the vehicle is located and the lane line information with the obtained area S2 of the partial rectangular area of the vehicle outline of the vehicle in front, if S1/S2< =Threshold T, execute step 3, if S1/S2>threshold T, execute step 4, threshold T=2; Step 3: Calculate the centroid of the rectangle of the partial rectangular area of the vehicle outline of the vehicle in front, denoted as P1, and perform step 5; Step 4: Extract the point of any corner of the rectangular area of the license plate outline of the vehicle in front, denoted as P2; Step 5: recovering the three-dimensional coordinates P of P1 or P2; Step 6: Calculate the distance between OPs to get the distance between the front and rear cars. 2. The vehicle distance measurement method based on binocular vision as claimed in claim 1, characterized in that, in the process of calculating the front and rear vehicle distances, the external parameters of the camera in the camera unit can be dynamically adjusted simultaneously, the method as follows: Step 11: Calculate the three-dimensional coordinates of points on the two lane lines of the lane where the vehicle in front is located, denoted as Pi; Step 12: fitting the spatial plane V according to the three-dimensional coordinates Pi of points on the two lanes; the method for fitting the spatial plane is as follows: set the coordinates of the three-dimensional point P _i as (X _i , Y _i , Z _i ), and the plane equation AX+BY+CZ+1=0, then the purpose of fitting is to obtain the plane equation parameters A, B and C, and substitute the coordinates of the three-dimensional point _Pi into AX+BY+CZ+1=0 to form the following equations: $\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>\\ <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>& <span\; class="notranslate">\; .</span>\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; A</span>}\\ \mathrm{<span\; class="notranslate">\; B</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}$ By solving the equations shown above, the plane equation of the space plane V can be obtained; Step 13: Project the three-dimensional coordinates Pi of the points on the two lanes to the space plane V to obtain the plane two-dimensional coordinates Qi of the three-dimensional coordinates Pi on the space plane V; Step 14: Carry out straight line fitting to the two-dimensional coordinates Qi of the plane to obtain two straight line vectors a and b; the method of straight line fitting is as follows: set the coordinates of the two-dimensional point Qi as ( _xi , y _i ), and there are m points, the straight line is y=a ₀ +a ₁ x, then the parameters a ₀ and a ₁ of the straight line equation can be obtained by using the formula of the least square method: $\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; m</span>}& {\mathrm{<span\; class="notranslate">\; \&Sigma;x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ {\mathrm{<span\; class="notranslate">\; \&Sigma;x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}& {\mathrm{<span\; class="notranslate">\; \&Sigma;x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\\ \mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; \&Sigma;y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ {\mathrm{<span\; class="notranslate">\; \&Sigma;x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}\right]<span\; class="notranslate">\; ;</span>$ Step 15: If the straight line vectors a and b are not parallel, recalibrate the positional relationship between the cameras in the camera unit to obtain new camera calibration parameters. 3. The vehicle distance measurement method based on binocular vision as claimed in claim 1, wherein in said step 1, the steps of the method for obtaining the partial rectangular area of the vehicle profile of the vehicle in front are as follows: 1) On the road ahead of the vehicle, scan the outline of the vehicle in front according to the horizontal line to obtain an image of the outline of the vehicle in front, and calculate the average gray value RowAvr(r) of each line in the image, as shown in the following formula: $\mathrm{RowAvr}\left(r\right)=\frac{1}{\mathrm{Right}\left(r\right)-\mathrm{Left}\left(r\right)}\underset{c=\mathrm{Left}\left(r\right)}{\overset{\mathrm{Right}\left(r\right)}{\mathrm{\&Sigma;}}}I(r,c),$ In the above formula, Right(r) represents the rightmost coordinate of the rth row of the lane area in the image; Left(r) represents the leftmost coordinate of the rth row of the lane area in the image; I(r, c) is the image (r, c) grayscale; RowAvr(r) represents the grayscale average value of the rth row in the lane area, and the r with the largest change in RowAvr(r) and RowAvr(r-1) is used as the lower edge of the vehicle in front, which is recorded as LowerPos; 2) Perform Sobel (SOBEL) edge extraction algorithm in the lane area to obtain the edge image of the vehicle in front of the lane area, and judge the position of the leftmost edge point and the rightmost edge in the [LowerPos-Tr, LowerPos] line of the edge image Point position, and consider it as the left edge and right edge of the vehicle in front, which are respectively recorded as LeftPos and RightPos; where Tr is 2~6, LeftPos and RightPos represent the leftmost and rightmost of the vehicle area respectively; 3) Assuming that the aspect ratio of the vehicle rectangle is Ta, then according to step 2), it can be known that the wide side of the partial rectangular area of the vehicle outline is RightPos-LeftPos, then the long side of the obtained rectangular area is (RightPos-LeftPos)*Ta, Ta The value is generally 0.5-1.0. 4. The method for measuring distance between vehicles based on binocular vision as claimed in claim 1, wherein said step 3 comprises the following steps: Step 31: Extract all edge points Pi in the partial rectangular area S2 of the vehicle outline of the vehicle in front, and put them into the edge sequence; Step 32: Use the average value of the coordinates of the edge sequence Pi as the centroid coordinates, denoted as P1, and use P1 as the measurement point. 5. The method for measuring vehicle distance based on binocular vision as claimed in claim 1, wherein said step 4 comprises the following steps: Step 41: using a Sobel edge operator to extract an edge image in the rectangular area of the license plate of the vehicle in front; Step 42: Binarize the extracted edge image according to the image segmentation algorithm; Step 43: Perform grayscale projection on the extracted edge image to obtain the upper limit a, lower limit b, left limit c and right limit d of the license plate; Step 44: According to the obtained upper limit value a, lower limit value b, left limit value c and right limit value d of the license plate, obtain the upper left, lower left, upper right and lower right four corner points of the rectangle representing the license plate area in the image The coordinates in are (c,a), (c,b), (d,a) and (d,b); Step 45: Locate a point on any corner in the license plate area, denoted as P2, and take P2 as a measurement point. 6. The vehicle distance measurement method based on binocular vision according to claim 1, characterized in that, while step 1 is being performed, the vehicle in front is tracked in combination with a Kalman filter algorithm. 7. The method for measuring vehicle distance based on binocular vision according to claim 2, characterized in that the extrinsic parameters of the camera include a rotation matrix R and a translation matrix T. 8. The vehicle distance measuring method based on binocular vision as claimed in claim 2, characterized in that, in step 15, whether the vector a and the vector b are parallel to adopt the Levenberg-Marquardt algorithm (Levenberg-Marquardt ). 9. The vehicle distance measurement method based on binocular vision according to claim 5, characterized in that, performing grayscale projection on the extracted edge image in the step 43 includes: Perform horizontal grayscale projection and vertical grayscale projection on the edge image at the same time, and search for two maximum values from bottom to top for the horizontal grayscale projection map, as the upper limit and lower limit of the front car license plate; The straight grayscale projection map looks for two maximum values from bottom to top, which are used as the left limit value and the right limit value of the license plate of the vehicle in front. 10. The method for measuring distance between vehicles based on binocular vision as claimed in claim 9, wherein the projection method is: the obtained edge image is denoted as I, and I(x, y) represents the distance in the image The gray value at the point (x, y), the horizontal gray projection, that is, the fixed y coordinate for gray accumulation, the formula is as follows: $h_j=\underset{i=0}{\overset{\mathrm{width}}{\mathrm{\&Sigma;}}}I(x_i,{\mathrm{the\; y}}_j),$ Where width represents the width of the image, and the _{y j} coordinates are fixed; In the same way, the vertical grayscale projection is to fix the x coordinate for grayscale accumulation. The formula is as follows: $V_j=\underset{i=0}{\overset{\mathrm{height}}{\mathrm{\&Sigma;}}}I(x_j,{\mathrm{the\; y}}_i),$ where height represents the height of the image, and x _j coordinates are fixed; for the license plate area, there are two maximum values in the horizontal grayscale projection _Hj , corresponding to the upper limit and lower limit of the license plate, and there are two maximum values in the vertical grayscale projection _Vj A maximum value, corresponding to the left limit value and right limit value of the license plate.

Images