CN103310464B - A kind of method of the direct estimation camera self moving parameter based on normal direction stream - Google Patents

Abstract

Based on a direct method for normal direction flow field estimation camera self moving parameter, be based upon on the basis of geometric model, the parameter of camera autokinesis can be estimated.The method comprises six large steps, step one: the normal direction flow field calculating input sequential frame image; Step 2: ballot obtains possible FOE point, selects the pure rotation amount of its correspondence and tentatively rejects FOE distracter; Step 3: use RANSAC algorithm to estimate rotation parameter ω, the reliability of inspection estimation result also rejects FOE distracter further; Step 4: utilize rotation parameter to choose pure translational movement, and again reject FOE distracter according to half-plane constraint; Step 5: the reliability that back forecasting is estimated; Step 6: K means clustering algorithm finds out the optimum solution of estimation.Instant invention overcomes the aperture problem in existing self moving parameter method of estimation, and need imaging scene to meet the shortcomings such as some constraints, estimate the camera self moving parameter good reliability obtained, be applicable to all monocular system, there is wide application background.

Description

A kind of method of the direct estimation camera self moving parameter based on normal direction stream

(1) technical field

The present invention relates to a kind of method of the direct estimation camera self moving parameter based on normal direction stream, to imaging scene without any constraint, be namely applicable to the action reference variable of any monocular system, belong to computer vision field.

(2) background technology

Namely camera ego-motion estimation is estimate relative motion between camera and scenery from the sequential frame image before and after camera motion, and this is computer vision field, one of gordian technique of especially vision guided navigation aspect application.In addition, the self moving parameter of camera estimates there is important application in a lot of field such as automatic Pilot, mobile robot's self-navigation and electronic image stabilizing, is the hot research problem of computer vision field in decades.At present, along with the raising of computer processor arithmetic capability, carry out real-time estimation become possibility to the self moving parameter of camera, researchist has taken up to be applied in autonomous vision guided navigation technology, such as, application etc. in moon exploration.

(scene is such as needed to comprise specific, Special complex background etc.) when not having artificial restraint, how accurately and estimate that the self moving parameter of camera is a difficult problem for computer vision field always efficiently.Although in the past few decades, research work personnel have carried out a large amount of correlative study work, and the estimation of camera autokinesis remains a very challenging technical barrier.

The traditional implementation of the self moving parameter of estimation camera has two schemes.The first, characteristic matching method, namely sets up the relation of characteristic matching from sequential frame image, and in then setting up according to matching relationship, epipolar geom etry constraint realizes action reference variable.But accurately and fast matching characteristic amount is a difficult problem for computer vision field all the time, not yet solves so far.And, feature choose with matching process in often need to comprise a large amount of notable feature in scene, this has just had very large constraint to imaging scene.The second, optical flow method.Its realization means utilizes the optical flow field of each pixel motion in token image to estimate, but due to the existence of aperture problem, light stream cannot directly be obtained by sequential frame image.Usually through artificial restraint conditions such as introducing imaging scene are level and smooth, continuous in traditional implementation method, calculate optical flow field.But these additional constraint conditions are difficult to meet in actual applications, and therefore classic method is very limited in the application of reality.

Different from traditional implementation, first the eighties Horn in latter stage proposes direct method (DirectMethod)---and directly utilize normal direction stream to solve the self moving parameter of camera.The feature of direct method directly utilizes sequential frame image to calculate normal direction flow field, and without any need for context restrictions condition, thus avoid the interference of the error of calculation and aperture problem in classic method.The more classical vector field model being Ferm ü ller and proposing in direct method, this model utilizes the direction of normal direction stream to be divided into "+" and " ﹣ " two parts, estimates kinematic parameter according to this two-part critical line (being called " zero line ").But the accurate location of " zero line " is very difficult in algorithm realization process, this also brings larger error directly to the estimation result of algorithm.Sinclair directly locates FOE(FocusofExpansion by normal direction stream) estimate the translational motion information of camera, but this method can not estimate rotary motion, does not have versatility.Silva proposes by setting up linear search subspace to locate the kinematic parameter finding out camera, but this method is easily subject to the interference of external condition, and very responsive to error.At present, the technological achievement of direct method is also little, and Existing methods is also difficult to realize self moving parameter comparatively accurately and estimates.

(3) summary of the invention

1, object: a kind of method that the object of this invention is to provide direct estimation camera self moving parameter based on normal direction stream, which overcome the deficiencies in the prior art, estimation error is less, and to imaging scene without any constraint, can be applied to any monocular system.

2, technical scheme:

The present invention is intended to the relative movement parameters estimated between camera and scene.Its input quantity is the normal direction flow field directly calculated by sequential frame image, exports the estimated value into camera self moving parameter.First the motion model that the present invention relates to is introduced.

Normal direction flow field is input quantity of the present invention, but include translational component and the rotational component of motion in normal direction flow vector, the scenery degree of depth and gradient information, and these information are all mixed in together, therefore flow into method inherently technological difficulties of hand estimation camera self moving parameter from normal direction.The present invention flows into hand from normal direction, is estimated the self moving parameter of camera by the vector carrying out deep analysis to filter out meeting particular geometric characteristic to its geometrical property.First motion model is set up as follows:

Suppose that scene is motionless, and camera carries out general motion, namely comprise translation motion t=[UVW] ^twith rotary motion ω=[ω ₁ω ₂ω ₃] ^t, then normal direction stream size Vn (x) at each pixel place can be expressed as vector form:

$\mathrm{Vn}\left(x\right)={\mathrm{Vn}}_{\mathrm{trans}}\left(x\right)+{\mathrm{Vn}}_{\mathrm{rot}}\left(x\right)$

$=\frac{W}{Z\left(x\right)}n\left(x\right)(x-\mathrm{FOE})+n^T\left(x\right)R\left(x\right)\mathrm{\ω}---\left(1\right)$

Wherein, Vn _trans(x) and Vn _rotx () is translational component and the rotational component of normal direction stream respectively; Z (x) is pixel x=(x, y) ^tthe scenery degree of depth at place; N (x) is its gradient direction; FOE (FocusofExpansion) is the translation motion axle of camera and the intersection point of imaging plane, can represent the direction of translatory motion of camera; The coefficients R (x) of rotary motion can be expressed as:

$R\left(x\right)=\left[\begin{array}{ccc}\mathrm{xy}/f& -(x^2/f+f)& y\\ (y^2/f+f)& -(\mathrm{xy}/f)& -x\end{array}\right]---\left(2\right)$

Wherein, f is the focal length of imaging system, (x, y) ^tit is the coordinate of arbitrary pixel in picture plane.

The present invention mainly utilizes the geometrical property of normal direction flow vector in formula (1), by choosing the normal direction stream of particular orientation, some blending constituents in normal direction flow vector are made to be zero, namely the normal direction stream characterizing certain peculair motion is found out, such as, pure flatly to move or pure rotary process Xiang Liu etc., then utilize these jus singulars to estimate the self moving parameter of camera to stream.The present invention that Here it is realizes the motion model that self moving parameter is estimated.

Based on above-mentioned motion model, introduce the concrete technical scheme that the present invention estimates camera self moving parameter below.The method of a kind of direct estimation camera self moving parameter based on normal direction stream of the present invention, its step is as follows:

Step one: read in sequential frame image I ₁(x), I ₂x (), computing method is to flow field u _n

Normal direction flow field is input quantity of the present invention, first introduces the acquisition methods in normal direction flow field.Be different from optical flow field, normal direction flow field directly to be calculated by the gray scale-temporal information analyzing sequential frame image before and after motion, and its computation process does not need there is any artificial constraint condition to imaging scene, and therefore the present invention has certain reliability and versatility.

From the angle of practical application, the normal direction stream size of each pixel is all the projection of light stream on this gradient direction.Light stream representative as the motion of each pixel in plane, but due to the existence of aperture problem, is difficult to accurately to calculate when not introducing artificial restraint condition the light stream vector that each pixel is pointed out.And normal direction stream is only the component of light stream on gradient direction, it can directly be calculated by sequential frame image, and it comprises a large amount of movable information equally.

First light stream represents the motion of pixel, and its equation of constraint is:

E _xu+E _yv+E _t=0(3)

Wherein, [uv] is the light stream vector at pixel (x, y) place, [E _xe _y] be shade of gray direction at pixel (x, y) place, E _tit is the gray scale-time rate of change at point (x, y) place.E _x, E _yand E _tdirectly can be calculated by the half-tone information of image sequence:

$\left\{\begin{array}{c}{E}_x=I_2(i+1,j)-I_2(i,j)\\ E_y=I_2(i,j+1)-I_2(i,j)\\ E_t=I_2(i,j)-I_1(i,j)\end{array}\right.---\left(4\right)$

Wherein I ₁, I ₂be the gray matrix of front and back two two field picture, I (i, j) represents the gray-scale value of the i-th row jth row pixel.Normal direction stream is the projection of light stream on gradient direction, and therefore, normal direction flow vector may be defined as:

$u_n=({\left[\begin{array}{cc}u& v\end{array}\right]}^T\·n)\frac{n}{\left|\right|n\left|\right|}={\left[\frac{-E_x{E}_t}{{E_x}^2+{E_y}^2}\frac{-E_y{E}_t}{{E_x}^2+{E_y}^2}\right]}^T---\left(5\right)$

Wherein, vector u _nmethod of representatives to flow vector, it is the unit gradient direction at pixel (x, y) place.Known by formula (4) and formula (5), by means of any artificial constraint condition, need directly can not calculate the normal direction flow vector of any pixel from the image sequence before and after motion.

Step 2: " ballot " obtains possible FOE point, finds out corresponding pure rotation amount and preliminary rejecting FOE distracter

Started with by the normal direction flow field obtained, the present invention adopts the mode of " ballot " to filter out jus singular to flow vector, then by analyzing these jus singulars to the geometric relationship between stream and camera motion parameter, estimates the self moving parameter of camera.

Known by formula (1), the translational component Vn in normal direction stream _transx () can be considered the inner product of vector n (x) and (x-FOE).And if at certain pixel place two vector normal, then their inner product is zero.If but want to choose the point meeting this orthogonal property, determine that the particular location of FOE is condition precedent.Therefore, adopt the mode of " ballot " to select possible FOE point in the present invention, its constraint condition is:

n(x)(x-FOE)=0(6)

Wherein, n (x) is pixel x=(x, y) ^tthe gradient direction at place, FOE represents the direction of camera translation motion.Meeting translational component in the normal direction stream at the pixel place of formula (6) pure rotation condition is zero, and therefore this point is " the pure point of rotation ", and its normal direction stream is " pure rotation amount ".Known by pure rotation condition, the judgement of each pure point of rotation is based on a specific FOE, if this point is the pure point of rotation, then FOE must be positioned at through this point and on straight line perpendicular to its normal direction stream, and this pixel is also this FOE " backer ", straight line is " ballot line ".Also be like this for other pure point of rotation, therefore FOE should be positioned at the point of intersection of all ballot lines.The principle of ballot that Here it is.

The ballot adopted in the present invention carries out when pure rotation amount and FOE are all unknown.Therefore, first suppose that all pixels are the pure point of rotation, so after voting in each pixel place, the FOE that " backer " is maximum has the pure rotation number of maximum quantity, and reliability is relatively high.Therefore, the present invention chooses possible FOE according to " backer " order from many to few and comes further to check.First for selected FOE, the pure rotation amount satisfied condition is chosen according to the pure rotation condition in formula (6).Compare with formula (1), translational component Vn in the normal direction stream at pure point of rotation place _trans(x)=0, therefore pure rotation amount meets:

Vn(x)=n ^T(x)R(x)ω(7)

That is, there is a kind of linear relationship between pure rotation amount and the pure point of rotation, and the rotary motion ω of camera is the coefficient of this linear relationship.Wherein, Vn (x) is the size of this x place normal direction stream, and n (x) is its gradient direction, and definition in the same formula of R (x) (2), ω is the rotational motion parameter of camera.Therefore, if the pure point of rotation and the normal direction stream thereof of more than three can be found, then the rotation parameter of camera can be estimated according to above formula.Otherwise if the pure rotation amount chosen according to constraint formulations (6) is less than three, this FOE will be regarded as distracter and disallowable.

Step 3: estimate rotation parameter ω with RANSAC algorithm, inspection estimation result reliability go forward side by side one step card FOE

The present invention adopts consistent (RANdomSAmpleConsensus, the RANSAC) algorithm of random sampling to carry out estimation equation (7) neutral line model, namely simulates the rotation parameter of camera.Because FOE in the present invention is obtained by the mode of ballot, therefore portion interfering terms may be included in this FOE and pure rotation amount thereof, and RANSAC algorithm can exclusive PCR effectively.Therefore the present invention adopts RANSAC algorithm to estimate the rotation parameter of camera, and whether reliably the possibility P that this algorithm can be obtained useful consequence as this FOE of measurement standard.P can be expressed as

P=1-(1-ρ ⁿ) ^m(8)

Wherein, m is iterations, and n is the minimum number that appraising model needs random selecting data.If certain pixel is applicable to the model estimated, think that it is " intra-office point ".ρ is that intra-office point accounts for all number percent corresponding to the pure point of rotation of this FOE.By P<P in the present invention _thresthe FOE of (threshold value) is considered as distracter and rejects.

Now, the kinematic parameter FOE-ω one_to_one corresponding of camera, and most FOE distracter is disallowable.Whether kinematic parameter reliably will be verified by pure translational constraints further to FOE-ω.

Step 4: choose pure translational movement and again reject FOE distracter according to half-plane constraint

Choose pure translational movement according to the rotation parameter ω obtained in the present invention and verify the reliability that kinematic parameter FOE-ω estimates.Pure rotation amount choose based on specific FOE, similarly, by analyzing based on the motion model of rotation parameter ω, can filter out rotational component be zero pure flatly move normal direction stream, the geometrical property then by analyzing these pure translational movements rejects the distracter of estimation further.

First, the motion model based on rotation parameter ω is gang take photocentre as summit, and ω is the centrum of axle, and centrum is projected as gang's quafric curve on imaging plane:

$F(x,y)={(\frac{{\mathrm{\&omega;}}_1}{{\mathrm{\&omega;}}_3}x+\frac{{\mathrm{\&omega;}}_2}{{\mathrm{\&omega;}}_3}y+f)}^2/(x^2+y^2+f^2)---\left(9\right)$

$=C$

Wherein, f is the focal length of camera, ω=[ω ₁ω ₂ω ₃] ^tbe the rotational motion parameter of camera, C is 0 to (1+ (ω ₁/ ω ₃) ²+ (ω ₂/ ω ₃) ²) between constant.This is that gang is by turning axle (ω ₁/ ω ₃, ω ₂/ ω ₃) curve determined, any point place on curve, if its normal direction flow vector is perpendicular to this tangential direction, then wherein rotation information is zero, is " pure translational movement ".

The geometrical property analyzing pure translational movement is known, if camera (namely near scene) motion forward, then FOE (FocusofExpansion) should be positioned at the contrary half-plane of translational movement pure with this; Otherwise if camera (namely away from scene) motion backward, then FOC (FocusofContraction) should be positioned at the consistent half-plane of translational movement pure with this.Wherein FOE and FOC represents the different directions of camera translation motion, the half-plane constraint of Here it is pure translational movement.

Choose pure translational movement according to the rotation parameter calculated, then check this FOE(or FOC) whether meet half-plane constraint, if do not met, be considered as distracter and reject, the present invention checks by setting a very little number of threshold values δ in proof procedure.

Step 5: back forecasting

Rejected by three screenings, most of FOE distracter is excluded.If this camera self moving parameter is estimated reliable, then the FOE point in inter-trust domain should be centered around near true value, presents state of aggregation.Therefore, the present invention adopts the method for back forecasting, is traced back on original image by the FOE finally screening " credible " that obtain, if FOE presents state of aggregation, then thinks that this estimates reliable; Otherwise the threshold value of setting is finely tuned when suitably the pure rotation amount of amendment or pure translational movement are chosen, finally make to estimate reliably.

Step 6: K means clustering algorithm finds out the optimum solution of estimation

After guaranteed this estimation reliable results by back forecasting, the present invention adopts K-means clustering algorithm to find out optimum kinematic parameter to FOE-ω as final estimation result of the present invention.

3, beneficial effect:

The present invention relates to a kind of method of the direct estimation camera self moving parameter based on normal direction stream, its advantage is:

1) half-tone information of two dimensional image is directly utilized to calculate input quantity of the present invention---normal direction flow field, effectively prevent extraction and the tracking error of the characteristic quantity introduced in " aperture " problem common in optical flow method and characteristic matching process.

2) to imaging scene without any artificial restraint, such as: imaging scene is level and smooth, continuously, include a large amount of notable features etc.Therefore the present invention is applicable to all monocular system.

3) RANSAC algorithm effectively prevent the interference of " point not in the know ".

4) back forecasting checks the present invention to estimate the reliability of result further.

5) inspection of quadruple closed loop can go out believable kinematic parameter by Effective selection, thus ensure that reliability and feasibility of the present invention.

(4) accompanying drawing explanation

Fig. 1 the method for the invention FB(flow block).

In figure: FOE is the translation motion axle of camera and the intersection point of imaging plane;

RANSAC(RANdomSAmpleConsensus) be random sampling unification algorism;

P represents that RANSAC algorithm can obtain the possibility of useful consequence;

P _thresrepresent threshold value;

FOE-ω represents the estimates of parameters of one group of camera autokinesis.

(5) embodiment

In order to understand technical scheme of the present invention better, below the specific embodiment of the present invention is further described:

The present invention realizes under MatlabR2011a language environment.First computing machine reads in the normal direction flow field that sequential frame image calculates characterizing motility, then possible FOE point is found out by " ballot ", and then select corresponding pure rotation amount and calculate the rotation parameter ω of camera, select pure translational movement according to turning axle again and verify that whether FOE is reliable, find out optimal estimation value finally by back forecasting and clustering algorithm.

The present invention is a kind of method of the direct estimation camera self moving parameter based on normal direction stream, and the flow process of the method as shown in Figure 1.The method comprises the following steps:

Step one: read in sequential frame image I ₁(x), I ₂x (), computing method is to flow field u _n

(1) under MatlabR2011a language environment, sequential frame image I is read in ₁(x), I ₂(x);

(2) pre-service is carried out to sequential frame image, such as: Gaussian smoothing filter, mean filter etc., Gaussian function chooses 5 × 5, the template of σ=1.4, mean filter chooses the template of 3 × 3;

(3) the Sobel operator of 3 × 3 is selected to find out gradient direction n (x) of each pixel;

(4) according to the Computing Principle of normal direction stream in formula (5), input quantity---the normal direction flow field u of present system is directly obtained _n(x).Because normal direction stream is the projection of light stream on gradient direction, therefore normal direction stream u _n(x) and n (x) conllinear, and the large I of normal direction stream just can born and can be zero.

Step 2: " ballot " obtains possible FOE point, finds out corresponding pure rotation amount and preliminary rejecting FOE distracter

(1) in a manifold larger than imaging plane size, carry out " ballot " and find out possible FOE point.Manifold there is a gray scale zero setting, then to do at each pixel place through this point and perpendicular to the straight line (" ballot line ") of this original image normal direction stream, and the pixel gray scale of this ballot line process is increased a certain amount of τ, τ and be size according to pretreatment image and specifically set.The true dimension of picture processed in the present invention is 574 × 652, τ value is 0.00001.

First judge whether certain pixel votes on line at certain, adopts the relation between two vectors to weigh in this subject study.Such as, judge pixel (x, y) whether on straight line Ax+By+C=0, can be regarded as and judge whether vector (A, B, C) and (x, y, 1) is vertical.And by ε ₁be set to the angle threshold value between two vectors, namely

By threshold epsilon ₁the angle system of being set to, whether voting on line Ax+By+C=0, generally can get ε by judging point (x, y) more accurately ₁=0 ~ 0.001.If point on this line, then think " backer " of corresponding FOE.After poll closing, the brightest pixel can think the FOE point that " backer " is maximum.

(2) the FOE point that selected " backer " is maximum, then finds out the pure rotation amount corresponding to this FOE according to rotation condition condition pure in formula (6).Need in specific operation process to adopt threshold epsilon, namely meet constraint

n(x)(x-FOE)<ε(11)

Wherein, n (x) is the gradient direction at pixel x place, and FOE represents the direction of camera translation motion.If meet pure rotation condition in above formula, then think this some place normal direction stream u _nx (), for pure rotary process is to stream, generally gets threshold epsilon <0.001.

(3) known by formula (7), there is a kind of linear relationship between pure rotation amount and the pure point of rotation, and the rotational motion parameter ω of camera is its linear coefficient.As long as find more than three the pure rotation amounts of non-zero, the rotation parameter ω of camera just can be estimated.If the pure rotation amount corresponding to certain FOE point less than three, then thinks that this FOE is distracter and is rejected.If the brightest FOE point is disallowable, then sequentially chooses residual pixel point the brightest heavy point and be FOE and repeat aforesaid operations.

Step 3: estimate rotation parameter ω with RANSAC algorithm, inspection estimation result reliability go forward side by side one step card FOE

(1) because RANSAC algorithm can avoid interference the impact of item on fitting effect effectively, therefore the present invention's linear model of adopting RANSAC algorithm to come in estimation equation (7).First from the pure rotation amount selected, random selecting n=3 pure rotation amount carries out initial fitting, then other pure rotation amounts are checked whether to be applicable to this model with the linear model that initial fitting goes out, if be applicable to, then be labeled as " intra-office point ", and then again estimate linear model with all " intra-office points ".Iteration is until " the intra-office point " of twice mark in front and back only becomes hardly so like this, and the coefficient of the linear model that now matching obtains is the camera rotation parameter ω corresponding to FOE.

(2) first the number of times m providing circulation random selecting sample is needed in RANSAC algorithm.The probability P that can produce useful consequence according to this algorithm can estimate the number of times m of random sampling:

$m=\frac{\log (1-P)}{\log (1-{\mathrm{\&rho;}}^n)}---\left(12\right)$

Wherein, the number of the pure rotation amount of ρ=intra-office point number/all correspondences is the number percent of intra-office point.N is the minimum number that appraising model needs random selecting data, estimates three parameters of camera rotation parameter in advance, therefore choose minimum number n=3 in the present invention.The threshold value P of Effective Probability is supposed in the present invention _thres=80%, get P=P _thresestimate and obtain random sampling number of times m.

(3) carrying out in each random sampling in the process of iteration all will according to formula (8) calculating probability P, if P>P _thresthen think this reliable results stop random sampling iteration.

(4) if still do not find reliable estimation model when frequency in sampling is greater than m, then this FOE is considered as distracter and rejects.After RANSAC round-robin algorithm is complete, most of FOE is identified as distracter and disallowable, the rotation parameter ω that minority FOE is corresponding with it define one to one action reference variable to FOE-ω.

(5) rotation parameter ω is directly calculated by the pure rotation amount corresponding to a certain FOE, and therefore it is believable for FOE.So the key of estimation is that whether FOE is reliable in the present invention.In order to determine to estimate result further, the present invention checks action reference variable to the reliability of FOE-ω by by the pure flat half-plane constraint moved again.

Step 4: choose pure translational movement and again reject FOE distracter according to half-plane constraint

(1) gang's quafric curve can be determined by rotation parameter ω:

$F(x,y)={(\frac{{\mathrm{\&omega;}}_1}{{\mathrm{\&omega;}}_3}x+\frac{{\mathrm{\&omega;}}_2}{{\mathrm{\&omega;}}_3}y+f)}^2/(x^2+y^2+f^2)---\left(13\right)$

$=C$

Wherein f is the focal length of camera, ω=[ω ₁ω ₂ω ₃] ^tbe the rotational motion parameter of camera, C is 0 to (1+ (ω ₁/ ω ₃) ²+ (ω ₂/ ω ₃) ²) between constant.The normal direction stream analyzing pixel on this race's curve is known, is zero, does not namely comprise rotation information, be therefore referred to as " pure translational movement " perpendicular to rotational component in the normal direction stream of this pixel place tangent line.

(2) the choosing of pure translational movement.Known according to the relevant geometrical constraint of above-mentioned quafric curve, meet

u _n(x,y)//(F _x(x,y),F _y(x,y))(14)

Normal direction flow vector be " pure translational movement ".Wherein, F _x(x, y), F _y(x, y) be F (x, y) at point (x, y) place along the partial derivative on X-and Y-direction, u _n(x, y) is the direction flow vector at this some place.

(3) known by pure flat geometrical property of moving normal direction stream, if the translation of camera is " close ", scenery travels forward, then FOE should be positioned at the half-plane region contrary with pure translational movement; Otherwise FOC is positioned at the half-plane region consistent with pure translational movement.The half-plane constraint of Here it is pure translational movement, only needs checking in implementation procedure

u _n(x)·(x-FOE)>0(15)

Or

u _n(x)·(x-FOC)<0(16)

Wherein u _nx () is the normal direction flow vector at pixel x place, FOE(or FOC) represent the direction of camera translation motion.The present invention checks by setting a very little threshold value δ in proof procedure.

(4) the present invention verifies often couple of FOE-ω.First the quafric curve determined according to rotation parameter ω is chosen " pure translational movement ", then checks FOE (or FOC) whether to meet half-plane constraint with these " pure translational movements ".If FOE (or FOC) meets the half-plane constraint of " the pure translational movement " of more than 85%, then think that this parameter estimation is credible to FOE-ω.Otherwise, be considered as distracter and reject.

(5) through the checking of triple closed loop, FOE distracter is close to all rejects.The result of this estimation is close to be determined, but whether this estimation is reliable, and the present invention will be determined by back forecasting.

Step 5: back forecasting

Adopt back forecasting to check the reliability of this estimation in the present invention.Through inspection and the rejecting of triple closed loop, FOE distracter is close to all to be got rid of.Therefore, if this estimation is reliable, then most remaining parameter should near its true value to FOE-ω, therefore FOE point traces back on original image and tests by the present invention.If remaining FOE presents the state of " gathering ", then think that this estimates reliable; Otherwise, make to estimate reliably by suitably adjusting the threshold epsilon of screening pure rotation amount.

Step 6: K means clustering algorithm finds out the optimum solution of estimation

After this estimation of back forecasting is reliable, the present invention adopts K means clustering algorithm to find out the center of most kinematic parameter as optimal estimation value, the estimation of namely final camera self moving parameter.

Validity of the present invention and accuracy are verified by synthesising picture and true picture, achieve good estimated result.Sharpest edges of the present invention are that the Gray Level-Gradient information directly utilizing image is estimated, effectively prevent the aperture problem existed in classic method, and to imaging scene without any artificial constraint, the camera self moving parameter being applicable to all monocular system is estimated.

From the experimental results, method in the present invention can estimate the parameter (translation direction FOE and rotation parameter ω) of camera autokinesis effectively from sequential frame image, reliability is high, and the self moving parameter that can be applicable in all monocular system is estimated, has broad application prospects and is worth.

Claims (1)

1., based on a method for the direct estimation camera self moving parameter of normal direction stream, the method is carried out under following motion model, suppose that scene is motionless, and camera carries out general motion, namely comprises translation motion t=[UVW] ^twith rotary motion ω=[ω ₁ω ₂ω ₃] ^t, then normal direction stream size Vn (x) at each pixel place is expressed as vector form:

$\begin{array}{c}Vn\left(x\right)={\mathrm{Vn}}_{trans}\left(x\right)+{\mathrm{Vn}}_{rot}\left(x\right)\\ =\frac{W}{Z\left(x\right)}n\left(x\right)(x-FOE)+n^T\left(x\right)R\left(x\right)\mathrm{\&omega;}\end{array}---\left(1\right)$

Wherein, Vn _trans(x) and Vn _rotx () is translational component and the rotational component of normal direction stream respectively; Z (x) is pixel x=(x, y) ^tthe scenery degree of depth at place; N (x) is its gradient direction; FOE is the translation motion axle of camera and the intersection point of imaging plane, represents the direction of translatory motion of camera; The coefficients R (x) of rotary motion is expressed as:

$R\left(x\right)=\left[\begin{array}{ccc}xy/f& -(x^2/f+f)& y\\ (y^2/f+f)& -(xy/f)& -x\end{array}\right]---\left(2\right)$

Wherein, f is the focal length of imaging system, (x, y) ^tit is the coordinate of arbitrary pixel in picture plane;

Utilize the geometrical property of normal direction flow vector in formula (1), by choosing the normal direction stream of particular orientation, some blending constituents in normal direction flow vector are made to be zero, namely the normal direction stream characterizing certain peculair motion is found out, such as, pure flat move or pure rotary process to stream, then utilize these jus singulars to stream the self moving parameter of camera is estimated, Here it is realize self moving parameter estimate motion model;

It is characterized in that: the step of the method is as follows

Step one: read in sequential frame image I ₁(x), I ₂x (), computing method is to flow field u _n;

Normal direction flow field is input quantity, is directly to be calculated by the gray scale-temporal information analyzing sequential frame image before and after motion, and its computation process does not need there is any artificial constraint condition to imaging scene;

From the angle of practical application, the normal direction stream size of each pixel is all the projection of light stream on this gradient direction; Light stream representative as the motion of each pixel in plane, but due to the existence of aperture problem, is difficult to accurately to calculate when not introducing artificial restraint condition the light stream vector that each pixel is pointed out; And normal direction stream is only the component of light stream on gradient direction, it is directly calculated by sequential frame image, and it comprises a large amount of movable information equally;

First light stream represents the motion of pixel, and its equation of constraint is:

E _xu+E _yv+E _t＝0(3)

Wherein, [u, v] is the light stream vector at pixel (x, y) place, [E _x, E _y] be shade of gray direction at pixel (x, y) place, E _tit is the gray scale-time rate of change at point (x, y) place; E _x, E _yand E _tdirectly calculated by the half-tone information of image sequence:

$\left\{\begin{array}{c}{E}_x=I_2(i+1,j)-I_2(i,j)\\ E_y=I_2(i,j+1)-I_2(i,j)\\ E_t=I_2(i,j)-I_1(i,j)\end{array}\right.---\left(4\right)$

Wherein I ₁, I ₂be the gray matrix of front and back two two field picture, I (i, j) represents the gray-scale value of the i-th row jth row pixel; Normal direction stream is the projection of light stream on gradient direction, and therefore, normal direction flow vector is defined as:

$u_n=({\left[\begin{array}{cc}u& v\end{array}\right]}^T\&CenterDot;n)\frac{n}{\left|\right|n\left|\right|}={\left[\begin{array}{cc}\frac{-E_x{E}_t}{{E_x}^2+{E_y}^2}& \frac{-E_y{E}_t}{{E_x}^2+{E_y}^2}\end{array}\right]}^T---\left(5\right)$

Wherein, vector u _nmethod of representatives to flow vector, it is the unit gradient direction at pixel (x, y) place; Known by formula (4) and formula (5), by means of any artificial constraint condition, need directly can not calculate the normal direction flow vector of any pixel from the image sequence before and after motion;

Step 2: " ballot " obtains possible FOE point, finds out corresponding pure rotation amount and preliminary rejecting FOE distracter;

Being started with by the normal direction flow field obtained, adopt the mode of " ballot " to filter out jus singular to flow vector, then by analyzing these jus singulars to the geometric relationship between stream and camera motion parameter, estimating the self moving parameter of camera;

Known by above-mentioned formula (1), the translational component Vn in normal direction stream _transx () is considered as the inner product of vector n (x) and (x-FOE); And if at certain pixel place two vector normal, then their inner product is zero; If but want to choose the point meeting this orthogonal property, determine that the particular location of FOE is condition precedent; Therefore, adopt the mode of " ballot " to select possible FOE point, its constraint condition is:

n(x)(x-FOE)＝0(6)

Wherein, n (x) is pixel x=(x, y) ^tthe gradient direction at place, FOE represents the direction of camera translation motion; Meeting translational component in the normal direction stream at the pixel place of formula (6) pure rotation condition is zero, and therefore this point is " the pure point of rotation ", and its normal direction stream is " pure rotation amount "; Known by pure rotation condition, the judgement of each pure point of rotation is based on a specific FOE, if this point is the pure point of rotation, then FOE must be positioned at through this point and on straight line perpendicular to its normal direction stream, and this pixel is also this FOE " backer ", straight line is " ballot line "; Also be like this for other pure point of rotation, therefore FOE is positioned at the point of intersection of all ballot lines;

The ballot adopted is carry out when pure rotation amount and FOE are all unknown, therefore, first suppose that all pixels are the pure point of rotation, so after voting in each pixel place, the FOE that " backer " is maximum has the pure rotation number of maximum quantity, and reliability is relatively high; Therefore, choose possible FOE according to " backer " order from many to few to come further to check; First for selected FOE, the pure rotation amount satisfied condition is chosen according to the pure rotation condition in formula (6); Compare with formula (1), translational component Vn in the normal direction stream at pure point of rotation place _trans(x)=0, therefore pure rotation amount meets:

Vn(x)＝n ^T(x)R(x)ω(7)

That is, there is a kind of linear relationship between pure rotation amount and the pure point of rotation, and the rotary motion ω of camera is the coefficient of this linear relationship; Wherein, Vn (x) is the size of this x place normal direction stream, and n (x) is its gradient direction, the coefficient that R (x) is rotary motion, represents same formula (2); ω in formula (7) is the rotational motion parameter of camera; Therefore, if find the pure point of rotation and the normal direction stream thereof of more than three, then the rotation parameter of camera is estimated according to above formula; Otherwise if the pure rotation amount chosen according to constraint formulations (6) is less than three, this FOE will be regarded as distracter and disallowable;

Step 3: estimate rotation parameter ω with RANSAC algorithm, inspection estimation result reliability go forward side by side one step card FOE;

Adopt random sampling to be unanimously that RANSAC algorithm carrys out estimation equation (7) neutral line model, namely simulate the rotation parameter of camera; Because FOE is obtained by the mode of ballot, therefore portion interfering terms may be included in this FOE and pure rotation amount thereof, and RANSAC algorithm can exclusive PCR effectively, therefore adopt the rotation parameter of RANSAC algorithm estimation camera, and whether reliably the possibility P that this algorithm can be obtained useful consequence as this FOE of measurement standard; P is expressed as

P＝1-(1-ρ ⁿ) ^m(8)

Wherein, m is the number of times of circulation random selecting sample, and n is the minimum number that appraising model needs random selecting data; If certain pixel is applicable to the model estimated, think that it is " intra-office point "; ρ is that intra-office point accounts for all number percent corresponding to the pure point of rotation of this FOE; P is less than threshold value P _thresfOE be considered as distracter and reject;

Now, the kinematic parameter FOE-ω one_to_one corresponding of camera, and most FOE distracter is disallowable; Whether kinematic parameter reliably will be verified by pure translational constraints further to FOE-ω;

Step 4: choose pure translational movement and again reject FOE distracter according to half-plane constraint;

Choose pure translational movement according to the rotation parameter ω obtained and verify the reliability that kinematic parameter FOE-ω estimates; Pure rotation amount choose based on specific FOE, similarly, by analyzing based on the motion model of rotation parameter ω, filter out rotational component be zero pure flatly move normal direction stream, the geometrical property then by analyzing these pure translational movements rejects the distracter of estimation further;

First, the motion model based on rotation parameter ω is gang take photocentre as summit, and ω is the centrum of axle, and centrum is projected as gang's quafric curve on imaging plane:

$\begin{array}{c}F(x,y)={(\frac{{\mathrm{\&omega;}}_1}{{\mathrm{\&omega;}}_3}x+\frac{{\mathrm{\&omega;}}_2}{{\mathrm{\&omega;}}_3}y+f)}^2/(x^2+y^2+f^2)\\ =C\end{array}---\left(9\right)$

Wherein, f is the focal length of camera, ω=[ω ₁ω ₂ω ₃] ^tbe the rotational motion parameter of camera, C is 0 to (1+ (ω ₁/ ω ₃) ²+ (ω ₂/ ω ₃) ²) between constant; This is that gang is by turning axle (ω ₁/ ω ₃, ω ₂/ ω ₃) curve determined, any point place on curve, if its normal direction flow vector is perpendicular to this tangential direction, then wherein rotation information is zero, is " pure translational movement ";

The geometrical property analyzing pure translational movement is known, if camera is forward namely near scene motion, then FOE should be positioned at the contrary half-plane of translational movement pure with this; Otherwise if camera is backward namely away from scene motion, then FOC should be positioned at the consistent half-plane of translational movement pure with this; Wherein FOE and FOC represents the different directions of camera translation motion, the half-plane constraint of Here it is pure translational movement;

Choose pure translational movement according to the rotation parameter calculated, then check this FOE or FOC whether to meet half-plane constraint, if do not met, be considered as distracter and reject; Check by setting a very little threshold value δ in proof procedure;

Wherein, the threshold value δ that setting one is very little in the proof procedure described in step 4 checks, this threshold value δ=0.001 ~ 0.005;

Step 5: back forecasting;

Rejected by three screenings, most of FOE distracter is excluded; If this camera self moving parameter is estimated reliable, then the FOE point in inter-trust domain should be centered around near true value, presents state of aggregation; Therefore, adopt the method for back forecasting, the FOE finally screening " credible " that obtain is traced back on original image, if FOE presents state of aggregation, then think that this estimates reliable; Otherwise the threshold value of setting is finely tuned when suitably the pure rotation amount of amendment or pure translational movement are chosen, finally make to estimate reliably;

Step 6: K means clustering algorithm finds out the optimum solution of estimation;

After guaranteed this estimation reliable results by back forecasting, K-means clustering algorithm is adopted to find out optimum kinematic parameter to FOE-ω as finally estimating result.