CN104504680A - Inverse perspective calibration method used for intelligent vehicle - Google Patents

Abstract

The invention discloses an inverse perspective calibration method used for an intelligent vehicle, and belongs to the self-driving technology field of the intelligent vehicle. The inverse perspective calibration method comprises the following steps: firstly, marking a rectangle on a road surface in front of the intelligent vehicle; recording the width and the length of the rectangle; capturing an image by a camera which needs inverse perspective calibration; then, finding image coordinates corresponding to four vertex angle points of the rectangle from the image; and finally, calculating a homography matrix which is required by inverse perspective through four coordinate points and the length and the width of the rectangle. The inverse perspective calibration method has the advantages of being simple and easy in operation, few in input parameters and high in calibration precision, and overcomes the defects of a traditional calibration method.

Description

A kind of inverse perspective scaling method for intelligent vehicle

Technical field:

The present invention is a kind of inverse perspective scaling method for intelligent vehicle, belongs to the unmanned technical field of intelligent vehicle.

Background technology:

Safe driving, intelligent transportation are more and more paid close attention to by people and are paid attention to, and the research of unmanned intelligent vehicle also becomes focus.The recognition technology of the traffic marking that lane line, zebra stripes, stop line etc. are important is the important composition of the unmanned technology of intelligent vehicle, and the identification of these traffic marking often needs to carry out inverse perspective mapping to original image.Because some changes occur in the position that intelligent vehicle is tested for a long time, in driving process can be video camera, therefore often need to carry out inverse perspective and demarcate.

Now delivered about the paper of inverse perspective mapping method in all research emphasis has been placed on inverse perspective mapping method on and have ignored the scaling method of inverse perspective, do not have researchist to study inverse perspective scaling method in intelligent vehicle.Due to the singularity of unmanned intelligent automobile, require that the inverse perspective scaling method being used for intelligent vehicle has the advantages that operation is simple and easy, error is little, precision is high.

Summary of the invention:

The object of the invention is to solve unmanned intelligent automobile frequent difficult problem of video camera being carried out to run into complicated operation in inverse perspective calibration process, error is large, precision is low, the nominal time is long in practical study and experiment, propose a kind of inverse perspective scaling method for intelligent vehicle.

To achieve these goals, this invention takes following technical scheme:

Step 1: selecting of rectangle;

Step 1-1: choose a rectangle on the within the vision smooth ground of video camera, requires the longitudinal axis of rectangle and the longitudinal axes parallel of intelligent vehicle;

Step 1-2: mark four drift angles of selected rectangle, requires to be marked in the image frame of video camera and can clearly to show;

Step 1-3: measure the length of side of rectangle, horizontal length of side W, longitudinal length of side H, unit is centimetre; In order to reduce calibrated error, require the W of rectangle that selectes meet 100<W<1000, H meets 300<H<1500;

Step 2: catch image;

Step 2-1: use and treat that an image caught by the video camera that inverse perspective is demarcated, what require four of rectangle on ground angles is marked in image;

Step 3: set up world's plane coordinate system; Described world's plane refers to the two dimensional surface that the ground at intelligent vehicle place forms; Described coordinate system take transverse axis as X-axis, and the longitudinal axis is Y-axis, and wherein the left side of initial point is the positive axis of X-axis, and the right is that X-axis bears axle, and top is that Y-axis bears axle, is the positive axis of Y-axis below;

Step 4: set up image coordinate system; Described image coordinate system is with the image upper left corner for initial point, and transverse axis is X-axis, and the longitudinal axis is Y-axis;

Step 5: the world coordinates calculating rectangle four angles; The coordinate points at four angles is expressed as P1 (a/k, b/k), P2 ((a+W)/k, b/k), P3 (a/k, (b+H)/k), P4 ((a+W)/k, (b+H)/k); Its mid point P1, P2, P3, P4 are rectangle upper left corner, upper right corner, lower-left corner, bottom right corner respectively; Wherein k is a ratio, representative be the actual range of a pixel representative in inverse fluoroscopy images be k centimetre; What wherein a represented is X-axis offset distance, and unit is centimetre, a on the occasion of time to turn right skew against fluoroscopy images, to turn left skew against fluoroscopy images when a is negative value; What wherein b represented is Y-axis offset distance, and unit is centimetre, b on the occasion of time down offset against fluoroscopy images, up offset against fluoroscopy images when b is negative value; P1 (a/k, b/k) is the coordinate points of rectangle upper left angle point relative to world coordinates initial point, namely when a, b are zero, P1 (a/k, b/k)=0,0) be the initial point of world coordinate system; The upper left corner (0,0) in inverse fluoroscopy images is namely the initial point of world coordinate system;

Step 6: calculate rectangle four angle image coordinate in the picture; The coordinate points at four angles is expressed as p1 (x1, y1), p2 (x2, y2), p3 (x3, y3), p4 (x4, y4); Wherein its mid point p1, p2, p3, p4 are rectangle upper left corner, upper right corner, lower-left corner, bottom right corner coordinate in the picture respectively;

Step 7: calculate homography matrix H; Described homography matrix H is the transformational relation of image coordinate system and world coordinate system, is one 3 matrix taking advantage of 3; Utilization is increased income the function cvGetPerspectiveTransform (P, p, H) in computer vision storehouse (OpenCV); Wherein P, p are coordinate points arrays, save the P1 in world coordinate system respectively, the p1 in P2, P3, P4 tetra-coordinate points and image coordinate system, p2, p3, p4 tetra-coordinate points; Wherein H is homography matrix, and this function is by calculating homography matrix to the coordinate points in P, p and being kept in H;

Step 8: according to the H of step 7 gained, inverse perspective mapping is carried out to image and obtain inverse fluoroscopy images.

Such scheme yes needs special instruction:

1. the rectangular selection in step 1 is most important to demarcation, determines the effect of inverse fluoroscopy images; Three steps are wherein had to need to further illustrate: in step 1-1, the position of rectangle needs and intelligent vehicle remains on same ground and requires that the longitudinal axis of intelligence keeps on the same line with the longitudinal axis of rectangle frame, its objective is the symmetry of guarantee against fluoroscopy images; Because program cannot identify four angles of rectangle automatically, and needing the artificial image coordinate of these four some correspondences being found out, in order to reduce artificial interference error, therefore requiring in step 1-2 to mark rectangle four angles; In step 1-3, the selection of the rectangle length of side is also the success or failure determining whole calibration process, wherein choosing of W, H will be determined according to the distance of vehicle in the field range of video camera and rectangle frame, if field range is less and close to vehicle, W, H can be less, anti-then can be larger, its objective is and ensure rectangle frame at video camera within sweep of the eye;

2. step 5 calculates in the world coordinates at rectangle four angles has three parameters extremely important, and the selection of its value will directly have influence on demarcation effect, wherein k representative be the actual range of a pixel representative in inverse fluoroscopy images is k centimetre, therefore arrange according to specific needs, can by W but be set to as far as possible, the integer that H can be divided exactly, citing is as W=300, H=600, then k can be 3, 5, 6, 10 etc., but k should not arrange too much or too small, otherwise all effect is demarcated in impact, because if k value is too little, inverse fluoroscopy images size can be caused very large and add image carrying cost, if k value is too large, inverse fluoroscopy images can be caused small-sized and be unfavorable for lane line, stop line, the identification of zebra stripes, image 5 and Fig. 6 are a, the design sketch that the identical k of b is different, wherein the setting of a, b is also extremely important, which determine the field range of inverse fluoroscopy images, because inverse fluoroscopy images is that world's plane carries out projecting in the ratio of k centimetre of every pixel, so namely the initial point of inverse perspective is the initial point of world coordinate system, and a, b are exactly in the position against fluoroscopy images or world coordinates plane for the control rectangle upper left corner, therefore a, b determine the field range of inverse fluoroscopy images, Fig. 5 with Fig. 7 is the design sketch that k identical a, b are different, a, b can be positive and negative values, representative implication is also different, but in the application of reality, generally can not select negative value, because negative value means the field range reducing inverse fluoroscopy images, but this needs according to practical application is main, arranging of a, b should not be too large, and its absolute value remains within 600 as far as possible, and as far as possible ensure a, b value aliquot k, thus the error reduced because of decimal band,

The invention has the beneficial effects as follows:

The present invention, by feasible technical scheme, has some beneficial effect following:

1. solve the problem that during tradition is demarcated, calibrated error is large because needing the measurement information such as camera height, the angle of pitch;

2. solve the undersized problem of traditional scaling board; Because generally use black and white chessboard case marker to determine plate in traditional scaling method to demarcate, and vehicle volume is comparatively large, video camera is difficult to recognize scaling board onboard;

3. the scaling method in the present invention is simple to operation, whole inverse perspective calibration process can be completed in 5 minutes;

4. can meet application request according to adjustment member parameter, strong adaptability;

Accompanying drawing illustrates:

Fig. 1 schematic flow sheet of the present invention

The original image that Fig. 2 obtains from video camera

The parameter of Fig. 3 input

Fig. 4 rectangle four angle coordinates in the picture

The result of Fig. 5 parameter 1

The result of Fig. 6 parameter 2

The result of Fig. 7 parameter 3

Embodiment:

Adopt method of the present invention, provide the example of an indefiniteness, composition graphs 1 is described specific embodiment of the invention process further.The present invention realizes at intelligent vehicle platform, intelligent vehicle test site, and in order to ensure unmanned intelligent automobile and personal security, platform used and place are unmanned technology specialty experiment porch and test site.Some current techiques used such as Image Acquisition, coordinate points acquisition etc. are not describing in detail.

Embodiments of the present invention are as follows:

1. require to choose a rectangle in intelligent vehicle front according to step 1, the choosing and the mark at four angles rectangle by lane line and cone bucket; Concrete operations are: the middle first vehicle being parked in two tracks, then use tape measure to have chosen the rectangle that a 610cm takes advantage of 610cm vehicle front 4 meters, and mark four of rectangle angles with the base of cone bucket; Require to obtain a kind of image from video camera according to step 2, see Fig. 2;

2. according to step 3,4,5,6,7, the detailed step of 8 realizes, wherein involved parameter is as follows: parameter 1 is k=5, W=610, H=610, a=150, b=300; Parameter 2 is k=10, W=610, H=610, a=150, b=300; Parameter 3 is k=5, W=610, H=610, a=0, b=0; Original image size is 1000X270; The coordinate at rectangle four angles p1 (304,103), p2 (677,104), p3 (100,234), p4 (880,238) respectively in image;

3. inverse perspective calibration result and effect:

According to 2 parameter 1 gained calibration results

$H=\left|\begin{array}{ccc}2.40115142& -1.66877472& 2.69854919e+2\\ 1.86429778e-3& 6.07412830e-2& 7.82127075e+1\\ -4.47532038e-5& -3.38974083e-3& 1\end{array}\right|;$ Design sketch is shown in Fig. 5;

According to 2 parameter 2 gained calibration results

$H=\left|\begin{array}{ccc}4.73403549& -3.30037975& 2.68011566e+2\\ -1.91638749e-2& 1.27451822e-1& 7.83218613e+1\\ -1.86057048e-4& -6.74905721e-3& 1\end{array}\right|;$ Design sketch is shown in Fig. 6;

According to 2 parameter 3 gained calibration results $H=\left|\begin{array}{ccc}6.04704428& -2.08066559& 302\\ 5.34602022e-3& 7.94393867e-2& 103\\ -2.74105860e-5& 4.24927846e-3& 1\end{array}\right|;$ Design sketch is shown in Fig. 7.

Claims (1)

1., for an inverse perspective scaling method for intelligent vehicle, it is characterized in that, comprise the following steps:

Step 1: selecting of rectangle;

Step 1-1: choose a rectangle on the within the vision smooth ground of video camera, requires the longitudinal axis of rectangle and the longitudinal axes parallel of intelligent vehicle;

Step 1-2: mark four drift angles of selected rectangle, requires to be marked in the image frame of video camera and can clearly to show;

Step 1-3: measure the length of side of rectangle, horizontal length of side W, longitudinal length of side H, unit is centimetre; In order to reduce calibrated error, require the W of rectangle that selectes meet 100<W<1000, H meets 300<H<1500;

Step 2: catch image;

Step 2-1: use and treat that an image caught by the video camera that inverse perspective is demarcated, what require four of rectangle on ground angles is marked in image;

Step 3: set up world's plane coordinate system; Described world's plane refers to the two dimensional surface that the ground at intelligent vehicle place forms; Described coordinate system take transverse axis as X-axis, and the longitudinal axis is Y-axis, and wherein the left side of initial point is the positive axis of X-axis, and the right is that X-axis bears axle, and top is that Y-axis bears axle, is the positive axis of Y-axis below;

Step 4: set up image coordinate system; Described image coordinate system is with the image upper left corner for initial point, and transverse axis is X-axis, and the longitudinal axis is Y-axis;

Step 5: the world coordinates calculating rectangle four angles; The coordinate points at four angles is expressed as P1 (a/k, b/k), P2 ((a+W)/k, b/k), P3 (a/k, (b+H)/k), P4 ((a+W)/k, (b+H)/k); Its mid point P1, P2, P3, P4 are rectangle upper left corner, upper right corner, lower-left corner, bottom right corner respectively; Wherein k is a ratio, representative be the actual range of a pixel representative in inverse fluoroscopy images be k centimetre; What wherein a represented is X-axis offset distance, and unit is centimetre, a on the occasion of time to turn right skew against fluoroscopy images, to turn left skew against fluoroscopy images when a is negative value; What wherein b represented is Y-axis offset distance, and unit is centimetre, b on the occasion of time down offset against fluoroscopy images, up offset against fluoroscopy images when b is negative value; P1 (a/k, b/k) is the coordinate points of rectangle upper left angle point relative to world coordinates initial point, namely when a, b are zero, P1 (a/k, b/k)=0,0) be the initial point of world coordinate system; The upper left corner (0,0) in inverse fluoroscopy images is namely the initial point of world coordinate system;

Step 6: calculate rectangle four angle image coordinate in the picture; The coordinate points at four angles is expressed as p1 (x1, y1), p2 (x2, y2), p3 (x3, y3), p4 (x4, y4); Wherein its mid point p1, p2, p3, p4 are rectangle upper left corner, upper right corner, lower-left corner, bottom right corner coordinate in the picture respectively;

Step 7: calculate homography matrix H; Described homography matrix H is the transformational relation of image coordinate system and world coordinate system, is one 3 matrix taking advantage of 3; Utilization is increased income the function cvGetPerspectiveTransform (P, p, H) in computer vision storehouse (OpenCV); Wherein P, p are coordinate points arrays, save the P1 in world coordinate system respectively, the p1 in P2, P3, P4 tetra-coordinate points and image coordinate system, p2, p3, p4 tetra-coordinate points; Wherein H is homography matrix, and this function is by calculating homography matrix to the coordinate points in P, p and being kept in H;

Step 8: according to the H of step 7 gained, inverse perspective mapping is carried out to image and obtain inverse fluoroscopy images.