CN102221358B - Monocular visual positioning method based on inverse perspective projection transformation - Google Patents

Abstract

The invention provides a monocular visual positioning method based on inverse perspective projection transformation. The technical scheme employed in the invention is that: an attitude sensor and a camera are fixed together and arranged on a wheeled vehicle; and images shoot in driving are subjected to the following processes: step 1, carrying out inverse perspective projection transformation on an image sequence; step 2, obtaining transformation matrixes between adjacent images through calculating; and step 3, determining a driving locus curve of the wheeled vehicle. The method allows high accurate positioning results to be obtained by assisting to position the wheeled vehicle through real time attitude information obtained by the attitude sensor, and the perspective effect to be eliminated through carrying out inverse perspective projection transformation on the images, so the precision of wheeled vehicle positioning can be further improved.

Description

Monocular vision localization method based on contrary perspective projection transformation

Technical field

The present invention relates to videographic measurment and technical field of image processing, further relate to a kind of method of utilizing contrary perspective projection transformation algorithm motion platform to be carried out the monocular vision location.

Background technology

Automobile autonomous driving technology is the hot issue of studying both at home and abroad in recent years, and gordian technique wherein is how to realize self-align accurately in real time to wheeled vehicle.At present; Targeting scheme commonly used is to utilize GPS (Global Position System; GPS) add the integrated navigation system of IMU (Inertial Measurement Unit, inertial measuring unit), but price general charged is comparatively expensive; And when environment of living in made that motion platform can't receive gps signal, this scheme can't be accomplished positioning function.

Existing vision positioning method mainly is divided into stereoscopic vision location and monocular vision location two big classes.The stereoscopic vision localization method is the testing environment unique point in three dimensions, and estimates the motion conditions of wheeled vehicle on this basis.There is following shortcoming in it: the one, and complex algorithm, comparatively consuming time, be difficult to requirement of real time; The 2nd, when environmental background did not have obvious textural characteristics, the unique point limited amount of extraction can cause algorithm bigger measuring error to occur.Therefore, the stereoscopic vision localization method will reach real through engineering approaches application level and still takes day.

By comparison, the monocular vision localization method is comparatively smooth with the hypothesis road surface to be prerequisite, realizes resolving of wheeled vehicle driving trace through finding the solution between image sequence simple displacement relation.Its algorithm is simple, and is ageing strong, and installs easily.But traditional monocular vision localization method need satisfy the application conditions such as variation with the wheeled vehicle motion of the smooth and camera attitude in road surface.Lv Qiang etc. have carried out studying (sensing technology journal to the monocular vision location technology in " based on the realization of monocular vision odometer in navigational system of SIFT feature extraction " literary composition; The 20th volume; The 5th phase; The 1148-1152 page or leaf), the fluoroscopy images that it is taken camera directly carries out the SIFT characteristic matching, calculates the wheeled vehicle motion state according to the theoretical model of deriving by match point.Because fluoroscopy images has true scenery is deformed into nearly far away little effect, the point of the unique point correspondence of then extracting on the image under world coordinate system is far away more apart from wheeled vehicle, and its image mapped that causes concerns that resolution error will be big more.If single apart from wheeled vehicle closer environment textural characteristics, unique point is difficult to extract, and then bigger measuring error can appear in algorithm, even loses efficacy.Because the theoretical model of this article has carried out being similar to a certain degree; Do not consider that simultaneously camera is when moving along with wheeled vehicle; Can occur the problem that attitude changes unavoidably, and directly fluoroscopy images is carried out characteristic matching also exists certain limitation, causes final bearing accuracy not ideal enough.

Summary of the invention

The technical matters that the present invention solves is: to the deficiency of existing vision localization technology, a kind of monocular vision localization method based on contrary perspective projection transformation is proposed, and simple than stereoscopic vision localization method algorithm of the present invention, be easy to realize; Compare with existing monocular vision localization method, the present invention has high orientation precision.

The concrete technical scheme that the present invention adopted is following:

With attitude sensor and camera fixing together, and the placement location of camera make camera can photograph the road surface on any one direction around the vehicle body.The frame frequency and the resolution of camera are set as required.Being provided with of camera frame frequency need guarantee that in wheeled vehicle cruising process the corresponding field of view of adjacent two width of cloth images that camera is taken has overlapping part.The initial moment of hypothetical record wheeled vehicle driving trace curve is t ₁, the image that this moment camera is taken is P ₁At moment t _i(i=1,2..., n, wherein n is a total number of images), camera photographs i width of cloth image P _i, the camera attitude angle that attitude sensor obtains is a _iImplement following step:

The first step is carried out contrary perspective projection transformation to image sequence.

At moment t _i, by camera attitude angle a _iObtain Camera extrinsic matrix number A _i, combining camera intrinsic parameter matrix obtains the contrary perspective projection transformation matrix B of image _iAccording to the contrary perspective projection transformation matrix B that obtains _i, at moment t _iThe image P that takes _iCarry out contrary perspective projection transformation, obtain the road surface and just playing view P ' _i

Suppose that all road surfaces are constantly just playing view P ' _iForm and just descending view sequence P ' ₁, P ' ₂..., P ' _n

In second step, calculate the transformation matrix between adjacent image.

Align down view sequence P ' ₁, P ' ₂..., P ' _nAny adjacent image P ' _qAnd P ' _Q+1, q=1,2 ..., n-1, carry out following processing:

(1) step, extract minutiae.

Use SURF feature description operator to adjacent image P ' _qAnd P ' _Q+1Carry out feature extraction, image P ' after the feature extraction _qFeature point set be F _q, image P ' _Q+1Feature point set be F _Q+1It is relevant that the unique point number of every width of cloth characteristics of image point set and the parameter of the resolution of image, texture complexity and SURF operator are provided with, and can confirm according to the use needs.

(2) goes on foot, and obtains the parameter of rigid body translation model between image.

To adjacent image P ' _qAnd P ' _Q+1, set up the transformation relation between rigid body translation model description two width of cloth images, the rigid body translation model is following:

$\left[\begin{array}{c}{x}_j\\ y_j\end{array}\right]=\left[\begin{array}{cc}\cos {\mathrm{\θ}}_q& -\sin {\mathrm{\θ}}_q\\ \sin {\mathrm{\θ}}_q& {\cos \mathrm{\θ}}_q\end{array}\right]\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]+\left[\begin{array}{c}{\mathrm{dx}}_q\\ {\mathrm{dy}}_q\end{array}\right]$ q＝1，2...，n-1

Wherein, suppose image P ' _qFeature point set F _qCorresponding characteristics of image pixel coordinate set is { (x _j, y _j), j=1...m, image P ' _Q+1Feature point set F _Q+1Corresponding characteristics of image pixel coordinate set be (x ' _k, y ' _k), k=1...m ', wherein m, m ' are respectively feature point set F _qAnd F _Q+1The unique point quantity that comprises, θ _qPresentation video P ' _Q+1With respect to P ' _qThe angle of rotation, (dx _q, dy _q) be image P ' _Q+1With respect to P ' _qThe pixel coordinate translational movement.

Utilize image P ' _qAnd P ' _Q+1Corresponding characteristics of image pixel coordinate set adopts the RANSAC algorithm for estimating to find the solution the rigid body translation model, just can calculate the parameter θ of rigid body translation model _q(dx _q, dy _q).

In the 3rd step, confirm wheeled vehicle driving trace curve.

According to the rigid body translation model parameter θ that obtains _q(dx _q, dy _q), calculate adjacent moment t _qAnd t _Q+1Between the displacement dL of wheeled vehicle _q, Q=1,2..., n-1, wherein, M is a longitudinal frame of just playing view, is just descending the longitudinal frame of each width of cloth image in the view sequence identical; D is vertical practical field of view scope corresponding when just playing the view longitudinal frame to be M, and the value of M and D can be confirmed as required.In conjunction with course information θ _q, can infer t constantly _Q+1Wheeled vehicle is present position coordinate Loc under world coordinate system _Q+1, Loc wherein ₁Be positioned at true origin, promptly All the other Loc _Q+1(q=1,2 ..., horizontal ordinate n-1)

Can analogize as follows:

$\left\{\begin{array}{c}{X}_{{\mathrm{Loc}}_{q+1}}=X_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\cos \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\\ Y_{{\mathrm{Loc}}_{q+1}}=Y_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\sin \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\end{array}\right.$

With each moment correspondence position coordinate points Loc _i(i=1,2 ..., n) connect successively, just can obtain the driving trace curve of wheeled vehicle.

Beneficial effect of the present invention is:

1, on the basis of traditional monocular vision localization method, the real-time attitude information that utilizes attitude sensor to obtain is assisted the location of wheeled vehicle, has obtained more high-precision location result.

2, image is carried out contrary perspective projection transformation, eliminated transparent effect, carry out feature extraction, model parameter calculating on this basis, avoided classic method to cause image mapped to concern the problem that resolution error is big, improved the wheeled vehicle bearing accuracy.

3, the present invention only uses a camera and an attitude sensor, just can calculate the driving trace information of wheeled vehicle, so structure of the present invention is comparatively simple, and simple installation need not complicated operations such as demarcation.

Description of drawings

Fig. 1 is the process flow diagram of practical implementation of the present invention;

Fig. 2 is the calculating principle schematic of wheeled vehicle geometric locus;

Fig. 3 is monocular vision location principle of work synoptic diagram;

The 34th width of cloth pavement image that Fig. 4 takes for camera in the experiment;

Fig. 5 is that view is just being played on the road surface of corresponding the 34th width of cloth image of Fig. 4 after contrary perspective transform;

The wheeled vehicle driving trace curve map of Fig. 6 for obtaining in the experiment.

Embodiment

Fig. 1 has provided the process flow diagram of practical implementation of the present invention.The first step is carried out contrary perspective projection transformation to image sequence.In this step, find the solution camera intrinsic parameter matrix and Camera extrinsic matrix number method referring to the 22nd page of " videographic measurment learn principle and applied research " book (Science Press publishes, Yu Qifeng/Shangyang work) to 33 pages.In second step, calculate the transformation matrix between adjacent image.Wherein, (1) step was an extract minutiae.In this step; The feature detection operator that uses is the SURF operator, about the character of SURF characteristic operator and method of application referring to document " Distinctive image features from scale invariant keypoints " (periodical: International Journal of Computer Vision, 2004; 60 (2); Page number 91-110, author David G. Lowe) and document " SURF:Speeded up robust features " (meeting: Proceedings ofthe 9th European Conference on Computer Vision, 2006; 3951 (1); Page number 404-417, author Herbert Bay, Tinne Tuytelaars and Luc Van Gool).(2) step was a parameter of obtaining rigid body translation model between image.In this step; Adopt the RANSAC algorithm for estimating to find the solution the rigid body translation model; The RANSAC algorithm speed is fast, and the parameter accuracy that estimates is high, is algorithm for estimating relatively more commonly used at present; The detailed introduction of RANSAC algorithm principle is referring to document " PreemptiveRANSAC for Live Structure and Motion Estimation " (meeting: Proceedings of theNinth IEEE International Conference on Computer Vision; ICCV 2003, author DavidNist ' er, Sarnoff Corporation and Princeton).The 3rd goes on foot, and confirms the driving trace curve of wheeled vehicle.Fig. 2 has provided the schematic diagram calculation of wheeled vehicle geometric locus.XY coordinate axis among the figure is represented world coordinate system, the initial point of coordinate system the is corresponding position coordinates Loc of wheeled vehicle in world coordinate system when camera takes the 1st width of cloth image ₁, this position the is corresponding initial moment t of location ₁, θ ₁Be the angle rotation amount between the 1st width of cloth image and the 2nd width of cloth image, dL ₁Be the translation distance between the 1st width of cloth image and the 2nd width of cloth image, Loc ₂The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken the 2nd width of cloth image.θ ₂Be the angle rotation amount between the 2nd width of cloth image and the 3rd width of cloth image, dL ₂Be the translation distance between the 2nd width of cloth image and the 3rd width of cloth image, Loc ₃The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken the 3rd width of cloth image.θ _N-1Be the angle rotation amount between n-1 width of cloth image and the n width of cloth image, wherein n is the total number of images that camera is taken, dL _N-1Be the translation distance between n-1 width of cloth image and the n width of cloth image, Loc _N-1The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken n-1 width of cloth image, Loc _nThe position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken n width of cloth image.With each position coordinates Loc under the world coordinate system ₁, Loc ₂... Loc _nBe connected, the curve of acquisition is the driving trace curve of wheeled vehicle.

Utilize embodiment of the present invention to test; Experimental selection is carried out in outdoor smooth place; Camera frame is located at wheeled vehicle the place ahead, also can be set up in any side of wheeled vehicle or rear in the practical application, includes only the road surface as long as guarantee the visual field that camera photographs.Camera is gathered 93 in picture altogether with the speed acquisition pictures of 3 frame/seconds in this experiment, i.e. n=93, and attitude sensor (model MTI) and camera (model FLE2-14S3) synchronous acquisition data, the principle of work of experiment is as shown in Figure 3.If digital 1 indication is shown t constantly among Fig. 3 _q(q=1,2 ... n-1, the total number of images of n for taking) the wheeled vehicle present position, the field range of this corresponding camera of moment is shown in trapezoid area 3, and the image that this moment, camera was taken is P _q, digital 2 indications are shown t _Q+1Moment wheeled vehicle present position, the field range of this corresponding camera of moment are shown in trapezoid area 4, and the image that this moment, camera was taken is P _Q+1, black arrow represents wheeled vehicle at moment t _qAnd t _Q+1Between travel distance, image P _qAnd P _Q+1The overlapping region, visual field as the numeral 5 shown in.The obtaining just overlapping region through two visual fields relative position relation obtain in each visual field of wheeled vehicle displacement and direction.

In the experiment, dolly is by serpentine fashion curve driving one segment distance.Fig. 4 is the 34th P in the image sequence of camera shooting in the experiment ₃₄, Fig. 5 is image P ₃₄View P ' is just being played on road surface after contrary perspective transform ₃₄Fig. 6 has provided the wheeled vehicle driving trace curve that utilizes the present invention to obtain in the experiment, and XY represents world coordinate system among the figure, and unit is a rice, and each green square point is represented the corresponding wheeled vehicle position coordinates Loc constantly of each sub-picture among the figure _i, all some connections are just obtained wheeled vehicle driving trace curve.By the visible operating range obtained by the method for the present invention of Fig. 6 is 22.76 meters, is 22.63 meters and get the actual travel distance by the tape measure field survey, error about 6 ‰.

Claims (1)

1. the monocular vision localization method based on contrary perspective projection transformation is characterized in that, with attitude sensor and camera fixing together, and the placement location of camera makes camera can photograph the vehicle body road surface on any one direction on every side; The frame frequency and the resolution of camera are set as required; Being provided with of camera frame frequency need guarantee that in wheeled vehicle cruising process the corresponding field of view of adjacent two width of cloth images that camera is taken has overlapping part; The initial moment of hypothetical record wheeled vehicle driving trace curve is t ₁, the image that this moment camera is taken is P ₁At moment t _i, camera photographs i width of cloth image P _i, the camera attitude angle that attitude sensor obtains is a _i, i=1,2..., n, wherein n is a total number of images; Implement following step:

The first step is carried out contrary perspective projection transformation to image sequence;

At moment t _i, by camera attitude angle a _iObtain Camera extrinsic matrix number A _i, combining camera intrinsic parameter matrix obtains the contrary perspective projection transformation matrix B of image _iAccording to the contrary perspective projection transformation matrix B that obtains _i, at moment t _iThe image P that takes _iCarry out contrary perspective projection transformation, obtain the road surface and just playing view

Assuming that all the roads are always under the view

view sequence composition is under

In second step, calculate the transformation matrix between adjacent image;

Align down any adjacent image

and

q=1 of view sequence

; 2; N-1, carry out following processing:

(1) step, extract minutiae;

Use SURF feature description operator to adjacent image

With

Carry out feature extraction, image after the feature extraction

Feature point set be F _q, image

Feature point set be F _Q+1It is relevant that the unique point number of every width of cloth characteristics of image point set and the parameter of the resolution of image, texture complexity and SURF operator are provided with, and can confirm according to the use needs;

(2) goes on foot, and obtains the parameter of rigid body translation model between image;

Adjacent image

and

are set up the transformation relation between rigid body translation model description two width of cloth images, and the rigid body translation model is following:

$\left[\begin{array}{c}{x}_j\\ y_j\end{array}\right]=\left[\begin{array}{cc}\cos {\mathrm{\θ}}_q& -\sin {\mathrm{\θ}}_q\\ \sin {\mathrm{\θ}}_q& \cos {\mathrm{\θ}}_q\end{array}\right]\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]+\left[\begin{array}{c}{\mathrm{dx}}_q\\ {\mathrm{dy}}_q\end{array}\right],q=\mathrm{1,2}...,n-1$

Wherein, suppose image Feature point set F _qCorresponding characteristics of image pixel coordinate set is { (x _j, y _j), j=1...m, image

Feature point set F _Q+1Corresponding characteristics of image pixel coordinate set does Wherein m, m ' are respectively feature point set F _qAnd F _Q+1The unique point quantity that comprises, θ _qPresentation video

With respect to

The angle of rotation, (dx _q, dy _q) be image With respect to The pixel coordinate translational movement;

Utilize image

With

Corresponding characteristics of image pixel coordinate set adopts the RANSAC algorithm for estimating to find the solution the rigid body translation model, just can calculate the parameter θ of rigid body translation model _q(dx _q, dy _q);

In the 3rd step, confirm wheeled vehicle driving trace curve;

According to the rigid body translation model parameter θ that obtains _q(dx _q, dy _q), calculate adjacent moment t _qAnd t _Q+1Between the displacement dL of wheeled vehicle _q,

Wherein, M is a longitudinal frame of just playing view; D is vertical practical field of view scope corresponding when just playing the view longitudinal frame to be M, and the value of M and D can be confirmed as required; Wheeled vehicle is present position coordinate Loc under world coordinate system _Q+1, Loc wherein ₁Be positioned at true origin, promptly All the other Loc _Q+1Horizontal ordinate and ordinate

For, q=1,2 ..., n-1:

$\left\{\begin{array}{c}{X}_{{\mathrm{Loc}}_{q+1}}=X_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\cos \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\\ Y_{{\mathrm{Loc}}_{q+1}}=Y_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\sin \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\end{array}\right.$

With each moment correspondence position coordinate points Loc _iConnect successively, just can obtain the driving trace curve of wheeled vehicle.

Images