CN102221358A - Monocular visual positioning method based on inverse perspective projection transformation - Google Patents

Abstract

The invention provides a monocular visual positioning method based on inverse perspective projection transformation. The technical scheme employed in the invention is that: an attitude sensor and a camera are fixed together and arranged on a wheeled vehicle; and images shoot in driving are subjected to the following processes: step 1, carrying out inverse perspective projection transformation on an image sequence; step 2, obtaining transformation matrixes between adjacent images through calculating; and step 3, determining a driving locus curve of the wheeled vehicle. The method allows high accurate positioning results to be obtained by assisting to position the wheeled vehicle through real time attitude information obtained by the attitude sensor, and the perspective effect to be eliminated through carrying out inverse perspective projection transformation on the images, so the precision of wheeled vehicle positioning can be further improved.

Description

Monocular vision localization method based on contrary perspective projection transformation

Technical field

The present invention relates to videographic measurment and technical field of image processing, further relate to a kind of method of utilizing contrary perspective projection transformation algorithm motion platform to be carried out the monocular vision location.

Background technology

Automobile autonomous driving technology is the hot issue of studying both at home and abroad in recent years, and gordian technique wherein is how to realize self-align accurately in real time to wheeled vehicle.At present, targeting scheme commonly used is to utilize GPS (Global Position System, GPS) adds IMU (Inertial Measurement Unit, inertial measuring unit) integrated navigation system, but price general charged is comparatively expensive, and when environment of living in made that motion platform can't receive gps signal, this scheme can't be finished positioning function.

Existing vision positioning method mainly is divided into stereoscopic vision location and monocular vision location two big classes.The stereoscopic vision localization method is the testing environment unique point in three dimensions, and estimates the motion conditions of wheeled vehicle on this basis.There is following shortcoming in it: the one, and the algorithm complexity, comparatively consuming time, be difficult to requirement of real time; The 2nd, when environmental background did not have obvious textural characteristics, the unique point limited amount of extraction can cause algorithm bigger measuring error to occur.Therefore, the stereoscopic vision localization method will reach real through engineering approaches application level and still takes day.

By comparison, the monocular vision localization method is comparatively smooth with the hypothesis road surface to be prerequisite, realizes resolving of wheeled vehicle driving trace by finding the solution between image sequence simple displacement relation.Its algorithm is simple, and is ageing strong, and installs easily.But traditional monocular vision localization method need satisfy the application conditions such as variation with the wheeled vehicle motion of the smooth and camera attitude in road surface.Lv Qiang etc. have carried out studying (sensing technology journal to the monocular vision location technology in " based on the realization of monocular vision odometer in navigational system of SIFT feature extraction " literary composition, the 20th volume, the 5th phase, the 1148-1152 page or leaf), the fluoroscopy images that it is taken camera directly carries out the SIFT characteristic matching, calculates the wheeled vehicle motion state according to the theoretical model of deriving by match point.Because fluoroscopy images has true scenery is deformed near big and far smaller effect, the point of the unique point correspondence of then extracting on the image under world coordinate system is far away more apart from wheeled vehicle, and its image mapped that causes concerns that resolution error will be big more.If the environment textural characteristics is single more nearby at the distance wheeled vehicle, unique point is difficult to extract, and then bigger measuring error can appear in algorithm, even loses efficacy.Because the theoretical model of this article has carried out being similar to a certain degree, do not consider that simultaneously camera is when moving along with wheeled vehicle, can occur the problem that attitude changes unavoidably, and directly fluoroscopy images is carried out characteristic matching also exists certain limitation, causes final bearing accuracy not ideal enough.

Summary of the invention

The technical matters that the present invention solves is: at the deficiency of existing vision localization technology, a kind of monocular vision localization method based on contrary perspective projection transformation is proposed, and simple than stereoscopic vision localization method algorithm of the present invention, be easy to realize; Compare with existing monocular vision localization method, the present invention has high orientation precision.

Concrete technical scheme of the present invention is as follows:

Attitude sensor and camera are fixed together, and the placement location of camera makes camera can photograph the road surface on any one direction around the vehicle body.The frame frequency and the resolution of camera are set as required.Being provided with of camera frame frequency need guarantee that in wheeled vehicle cruising process the field of view of adjacent two width of cloth image correspondences that camera is taken has overlapping part.The initial moment of hypothetical record wheeled vehicle driving trace curve is t ₁, the image that this moment camera is taken is P ₁At moment t _i(i=1,2..., n, wherein n is a total number of images), camera photographs i width of cloth image P _i, the camera attitude angle that attitude sensor obtains is a _iImplement following step:

The first step is carried out contrary perspective projection transformation to image sequence.

At moment t _i, by camera attitude angle a _iObtain Camera extrinsic matrix number A _i, combining camera intrinsic parameter matrix obtains the contrary perspective projection transformation matrix B of image _iAccording to the contrary perspective projection transformation matrix B that obtains _i, at moment t _iThe image P that takes _iCarry out contrary perspective projection transformation, obtain the road surface and just playing view P ' _i

Suppose that all road surfaces are constantly just playing view P ' _iForm and just descending view sequence P ' ₁, P ' ₂, L, P ' _n

In second step, calculate the transformation matrix between adjacent image.

Align down view sequence P ' ₁, P ' ₂, L, P ' _nAny adjacent image P ' _qAnd P ' _Q+1, q=1,2, L, n-1, carry out following processing:

(1) step, extract minutiae.

Use SURF feature description operator to adjacent image P ' _qAnd P ' _Q+1Carry out feature extraction, image P ' after the feature extraction _qFeature point set be F _q, image P ' _Q+1Feature point set be F _Q+1It is relevant that the unique point number of every width of cloth characteristics of image point set and the parameter of the resolution of image, texture complexity and SURF operator are provided with, can be according to using needs to determine.

(2) goes on foot, and obtains the parameter of rigid body translation model between image.

To adjacent image P ' _qAnd P ' _Q+1, set up the transformation relation between rigid body translation model description two width of cloth images, the rigid body translation model is as follows:

$\left[\begin{array}{c}{x}_j\\ y_j\end{array}\right]=\left[\begin{array}{cc}{\cos \mathrm{\θ}}_q& -\sin {\mathrm{\θ}}_q\\ {\sin \mathrm{\θ}}_q& {\cos \mathrm{\θ}}_q\end{array}\right]\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]+\left[\begin{array}{c}{\mathrm{dx}}_q\\ {\mathrm{dy}}_q\end{array}\right]$ q＝1，2...，n-1

Wherein, suppose image P ' _qFeature point set F _qCorresponding characteristics of image pixel coordinate set is { (x _j, y _j), j=1...m, image P ' _Q+1Feature point set F _Q+1Corresponding characteristics of image pixel coordinate set be (x ' _k, y ' _k), k=1...m ', wherein m, m ' are respectively feature point set F _qAnd F _Q+1The unique point quantity that comprises, θ _qPresentation video P ' _Q+1With respect to P ' _qThe angle of rotation, (dx _q, dy _q) be image P ' _Q+1With respect to P ' _qThe pixel coordinate translational movement.

Utilize image P ' _qAnd P ' _Q+1Corresponding characteristics of image pixel coordinate set adopts the RANSAC algorithm for estimating to find the solution the rigid body translation model, just can calculate the parameter θ of rigid body translation model _q(dx _q, dy _q).

In the 3rd step, determine wheeled vehicle driving trace curve.

According to the rigid body translation model parameter θ that obtains _q(dx _q, dy _q), calculate adjacent moment t _qAnd t _Q+1Between the displacement dL of wheeled vehicle _q,

Q=1,2..., n-1, wherein, M is a longitudinal frame of just playing view, is just descending the longitudinal frame of each width of cloth image in the view sequence identical; D is vertical practical field of view scope of correspondence when just playing the view longitudinal frame to be M, and the value of M and D can be determined as required.In conjunction with course information θ _q, can infer t constantly _Q+1Wheeled vehicle is present position coordinate Loc under world coordinate system _Q+1, Loc wherein ₁Be positioned at true origin, promptly

All the other Loc _Q+1(q=1,2, L, horizontal ordinate n-1)

Can analogize as follows:

$\left\{\begin{array}{c}{X}_{{\mathrm{Loc}}_{q+1}}=X_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\cos \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\\ Y_{{\mathrm{Loc}}_{q+1}}=Y_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\sin \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\end{array}\right.$

With each moment correspondence position coordinate points Loc _i(i=1,2, L n) connects successively, just can obtain the driving trace curve of wheeled vehicle.

Beneficial effect of the present invention is:

1, on the basis of traditional monocular vision localization method, the real-time attitude information that utilizes attitude sensor to obtain is assisted the location of wheeled vehicle, has obtained more high-precision positioning result.

2, image is carried out contrary perspective projection transformation, eliminated transparent effect, carry out feature extraction, model parameter calculating on this basis, avoided classic method to cause image mapped to concern the problem that resolution error is big, improved the wheeled vehicle bearing accuracy.

3, the present invention only uses a camera and an attitude sensor, just can calculate the driving trace information of wheeled vehicle, so structure of the present invention is comparatively simple, and simple installation need not complicated operations such as demarcation.

Description of drawings

Fig. 1 is the concrete process flow diagram of implementing of the present invention;

Fig. 2 is the calculating principle schematic of wheeled vehicle geometric locus;

Fig. 3 is monocular vision location principle of work synoptic diagram;

The 34th width of cloth pavement image that Fig. 4 takes for camera in the experiment;

Fig. 5 is that view is just being played on the road surface of corresponding the 34th width of cloth image of Fig. 4 after contrary perspective transform;

The wheeled vehicle driving trace curve map of Fig. 6 for obtaining in the experiment.

Embodiment

Fig. 1 has provided the concrete process flow diagram of implementing of the present invention.The first step is carried out contrary perspective projection transformation to image sequence.In this step, find the solution camera intrinsic parameter matrix and Camera extrinsic matrix number method referring to the 22nd page of " videographic measurment learn principle and applied research " book (Science Press publishes, Yu Qifeng/Shangyang work) to 33 pages.In second step, calculate the transformation matrix between adjacent image.Wherein, (1) step was an extract minutiae.In this step, the feature detection operator that uses is the SURF operator, about the character of SURF feature operator and using method referring to document " Distinctive image features from scale invariant key points " (periodical: International Journal of Computer Vision, 2004,60 (2), page number 91-110, author David G.Lowe) and document " SURF:Speeded up robust features " (meeting: Proceedings of the 9th European Conference on Computer Vision, 2006,3951 (1), page number 404-417, author Herbert Bay, Tinne Tuytelaars and Luc Van Gool).(2) step was a parameter of obtaining rigid body translation model between image.In this step, adopt the RANSAC algorithm for estimating to find the solution the rigid body translation model, the RANSAC algorithm speed is fast, the parameter accuracy height that estimates, it is algorithm for estimating relatively more commonly used at present, the detailed introduction of RANSAC algorithm principle is referring to document " Preemptive RANSAC for Live Structure and Motion Estimation " (meeting: Proceedings of the Ninth IEEE International Conference on Computer Vision, ICCV 2003, author David Nist ' er, Sarnoff Corporation and Princeton).The 3rd goes on foot, and determines the driving trace curve of wheeled vehicle.Fig. 2 has provided the schematic diagram calculation of wheeled vehicle geometric locus.XY coordinate axis among the figure is represented world coordinate system, the initial point correspondence of coordinate system camera wheeled vehicle position coordinates Loc in world coordinate system when taking the 1st width of cloth image ₁, this position correspondence the location initial moment t ₁, θ ₁Be the angle rotation amount between the 1st width of cloth image and the 2nd width of cloth image, dL ₁Be the translation distance between the 1st width of cloth image and the 2nd width of cloth image, Loc ₂The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken the 2nd width of cloth image.θ ₂Be the angle rotation amount between the 2nd width of cloth image and the 3rd width of cloth image, dL ₂Be the translation distance between the 2nd width of cloth image and the 3rd width of cloth image, Loc ₃The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken the 3rd width of cloth image.θ _N-1Be the angle rotation amount between n-1 width of cloth image and the n width of cloth image, wherein n is the total number of images that camera is taken, dL _N-1Be the translation distance between n-1 width of cloth image and the n width of cloth image, Loc _N-1The position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken n-1 width of cloth image, Loc _nThe position coordinates of wheeled vehicle under world coordinate system when corresponding camera is taken n width of cloth image.With each position coordinates Loc under the world coordinate system ₁, Loc ₂... Loc _nBe connected, the curve of acquisition is the driving trace curve of wheeled vehicle.

Utilize the specific embodiment of the present invention to test, experimental selection is carried out in outdoor smooth place, camera frame is located at wheeled vehicle the place ahead, also can be set up in any side of wheeled vehicle or rear in the practical application, includes only the road surface as long as guarantee the visual field that camera photographs.Camera is gathered 93 in picture altogether with the speed acquisition pictures of 3 frame/seconds in this experiment, i.e. n=93, and attitude sensor (model MTI) and camera (model FLE2-14S3) synchronous acquisition data, the principle of work of experiment is as shown in Figure 3.If digital 1 indication is shown t constantly among Fig. 3 _q(q=1,2 ... n-1, the total number of images of n for taking) the wheeled vehicle present position, the field range of this corresponding camera of moment is shown in trapezoid area 3, and the image that this moment, camera was taken is P _q, digital 2 indications are shown t _Q+1Moment wheeled vehicle present position, the field range of this corresponding camera of moment are shown in trapezoid area 4, and the image that this moment, camera was taken is P _Q+1, black arrow represents wheeled vehicle at moment t _qAnd t _Q+1Between travel distance, image P _qAnd P _Q+1The overlapping region, visual field as the numeral 5 shown in.The obtaining just overlapping region by two visual fields relative position relation obtain in each visual field of wheeled vehicle displacement and direction.

In the experiment, dolly is by serpentine fashion curve driving one segment distance.Fig. 4 is the 34th P in the image sequence of camera shooting in the experiment ₃₄, Fig. 5 is image P ₃₄View P ' is just being played on road surface after contrary perspective transform ₃₄Fig. 6 has provided the wheeled vehicle driving trace curve that utilizes the present invention to obtain in the experiment, and XY represents world coordinate system among the figure, and unit is a rice, and each green square point is represented the corresponding wheeled vehicle position coordinates Loc constantly of each sub-picture among the figure _i, all some connections are just obtained wheeled vehicle driving trace curve.Operating range obtained by the method for the present invention as seen from Figure 6 is 22.76 meters, is 22.63 meters and get the actual travel distance by the tape measure field survey, error about 6 ‰.

Claims (1)

1. the monocular vision localization method based on contrary perspective projection transformation is characterized in that, attitude sensor and camera are fixed together, and the placement location of camera makes camera can photograph the vehicle body road surface on any one direction on every side; The frame frequency and the resolution of camera are set as required; Being provided with of camera frame frequency need guarantee that in wheeled vehicle cruising process the field of view of adjacent two width of cloth image correspondences that camera is taken has overlapping part; The initial moment of hypothetical record wheeled vehicle driving trace curve is t ₁, the image that this moment camera is taken is P ₁At moment t _i(i=1,2.., n, wherein n is a total number of images), camera photographs i width of cloth image P _i, the camera attitude angle that attitude sensor obtains is a _iImplement following step:

The first step is carried out contrary perspective projection transformation to image sequence;

At moment t _i, by camera attitude angle a _iObtain Camera extrinsic matrix number A _i, combining camera intrinsic parameter matrix obtains the contrary perspective projection transformation matrix B of image _iAccording to the contrary perspective projection transformation matrix B that obtains _i, at moment t _iThe image P that takes _iCarry out contrary perspective projection transformation, obtain the road surface and just playing view P ' _i

Suppose that all road surfaces are constantly just playing view P ' _iForm and just descending view sequence P ' ₁, P ' ₂, L, P ' _n

In second step, calculate the transformation matrix between adjacent image;

Align down view sequence P ' ₁, P ' ₂, L, P ' _nAny adjacent image P ' _qAnd P ' _Q+1, q=1,2, L, n-1, carry out following processing:

(1) step, extract minutiae;

Use SURF feature description operator to adjacent image P ' _qAnd P ' _Q+1Carry out feature extraction, image P ' after the feature extraction _qFeature point set be F _q, image P ' _Q+1Feature point set be F _Q+1It is relevant that the unique point number of every width of cloth characteristics of image point set and the parameter of the resolution of image, texture complexity and SURF operator are provided with, can be according to using needs to determine;

(2) goes on foot, and obtains the parameter of rigid body translation model between image;

To adjacent image P ' _qAnd P ' _Q+1, set up the transformation relation between rigid body translation model description two width of cloth images, the rigid body translation model is as follows:

$\left[\begin{array}{c}{x}_j\\ y_j\end{array}\right]=\left[\begin{array}{cc}{\cos \mathrm{\θ}}_q& -\sin {\mathrm{\θ}}_q\\ {\sin \mathrm{\θ}}_q& {\cos \mathrm{\θ}}_q\end{array}\right]\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]+\left[\begin{array}{c}{\mathrm{dx}}_q\\ {\mathrm{dy}}_q\end{array}\right]$ q＝1，2...，n-1

Wherein, suppose image P ' _qFeature point set F _qCorresponding characteristics of image pixel coordinate set is { (x _j, y _j), j=1...m, image P ' _Q+1Feature point set F _Q+1Corresponding characteristics of image pixel coordinate set be (x ' _k, y ' _k), k=1...m ', wherein m, m ' are respectively feature point set F _qAnd F _Q+1The unique point quantity that comprises, θ _qPresentation video P ' _Q+1With respect to P ' _qThe angle of rotation, (dx _q, dy _q) be image P ' _Q+1With respect to P ' _qThe pixel coordinate translational movement;

Utilize image P ' _qAnd P ' _Q+1Corresponding characteristics of image pixel coordinate set adopts the RANSAC algorithm for estimating to find the solution the rigid body translation model, just can calculate the parameter θ of rigid body translation model _q(dx _q, dy _q);

In the 3rd step, determine wheeled vehicle driving trace curve;

According to the rigid body translation model parameter θ that obtains _q(dx _q, dy _q), calculate adjacent moment t _qAnd t _Q+1Between the displacement dL of wheeled vehicle _q,

Q=1,2..., n-1, wherein, M is a longitudinal frame of just playing view; D is vertical practical field of view scope of correspondence when just playing the view longitudinal frame to be M, and the value of M and D can be determined as required; Wheeled vehicle is present position coordinate Loc under world coordinate system _Q+1, Loc wherein ₁Be positioned at true origin, promptly All the other Loc _Q+1(q=1,2, L, horizontal ordinate n-1) and ordinate

For:

$\left\{\begin{array}{c}{X}_{{\mathrm{Loc}}_{q+1}}=X_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\cos \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\\ Y_{{\mathrm{Loc}}_{q+1}}=Y_{{\mathrm{Loc}}_q}+{\mathrm{dL}}_q\×\sin \left(\underset{l=1}{\overset{q}{\mathrm{\Σ}}}{\mathrm{\θ}}_l\right)\end{array}\right.$

With each moment correspondence position coordinate points Loc _i(i=1,2, L n) connects successively, just can obtain the driving trace curve of wheeled vehicle.

Images