CN103985154A - Three-dimensional model reestablishment method based on global linear method - Google Patents

Abstract

The invention discloses a three-dimensional model reestablishment method based on a global linear method. According to the method, firstly, the orientation array of all cameras is obtained by solving the relative rotating relations between every two cameras; secondly, stable camera triad reestablishment is obtained through the nonlinear optimization technology, and a linear relation in a triad is solved; thirdly, a system of linear equations is formed according to equations provided by the triad, accurate camera postures can be obtained by solution through a numerical method, and finally, final optimization is carried out on reestablishment results through the technology of concentration of beam adjustment. The three-dimensional model reestablishment method based on the global linear method is fast, robust and accurate through experimental verification.

Description

A kind of method for reconstructing three-dimensional model based on overall linear method

Technical field

The present invention relates to a kind of method for reconstructing three-dimensional model based on overall linear method, belong to computer software technical field.

Background technology

Have in recent years a lot of people at the three-dimensional model of a large amount of unordered incompatible reconstruction scenery of online photograph collection of research and utilization, and exercise recovery structure (sfm) is the committed step in these reconstruction flow processs.The flow process of three-dimensional reconstruction comprises: 1. pair all pictures are done image characteristics extraction (generally adopting SIFT feature); 2. pair unique point is mated (finding the common point of seeing between camera); 3. the attitude of an initialization phase group of planes and the three-dimensional coordinate of unique point; 4. adjust camera attitude and scene structure (bundle adjustment).The 3rd step wherein, traditional method [Snavely06], [Li08], [Agarwal09] all adopts increment bundle adjustment (incremental bundle adjustment) in preliminary reconstruction camera attitude and unique point three-dimensional coordinate, be two the closest pictures of initial selected contact, attitude and the characteristic point position of two cameras of reduction, then adjust and remove error; Add again a pictures, reduce the position of attitude and unique point of this camera, so continuous repetition until camera attitude corresponding to all picture and the three-dimensional coordinate of unique point be reduced out.This method for the reconstruction effect of a lot of scenes well.

The method [David Crandall2011] that David adopts is regarded exercise recovery structure (sfm) Markov model is asked to energy minimization as, the disposable three-dimensional coordinate that solves attitude and the unique point of all cameras.It is all fast that David claims that his method is adjusted (IBA) method than current any increment boundling, and more have robustness for the mistake in rebuilding.First the method for David utilizes all pictures to calculate initial all camera attitudes, then with boundling adjustment, does once last scene structure correction.But the algorithm of David is better for the reconstruction effect of large-scale view data, because people are not high for the accuracy requirement of camera attitude in this case, only pay close attention to whole structure.But the algorithm of David has fatal defect for solving accurate camera attitude, algorithm is supposed all cameras all at grade, and this causes the attitude of camera impossible completely accurately.

Some had proposed the algorithm that the new overall situation solves exercise recovery structure in recent years, [Sinha] designed the linear algorithm of the multipaths of a robust, utilize two cameras between rebuilding camera towards the linear relationship of situation, the overall situation solve camera attitude and unique point coordinate.[Jiang] designs a kind of linear algorithm of novel robust equally, known camera towards in the situation that, the geometric error of distance between algorithmic minimizing image center, overall linearity solves a good original reconstruction.It is high that but above-mentioned two kinds of methods all rely on the accuracy to utmost point geometry that camera is rebuild between two, if can accurately not screen out wrong diagonal angle how much, algorithm will collapse.

Classic method: a lot of famous Sfm systems are all that first camera between two calculates relative attitude, adopts in order or the method for layering is added camera in whole reconstruction to one by one.Their common defects is exactly, and 1. each camera that adds need to be done a boundling adjustment, if this initial seed that has a strong impact on the arithmetic speed 2. increment boundling adjustment of algorithm does not choose, the effect of rebuilding so can be affected; 3. for the scene of large wide-angle, increment adds camera and unavoidably brings cumulative errors, causes the camera posture deforming of rear interpolation, characteristic point position out of true.

The method of the overall situation: some overall algorithms are computing camera attitudes in world coordinate system in two steps, their first computing camera towards, and then the position of computing camera photocentre.[Crandall2011] and [Jiang2013] [sinha2010] all belongs to this method, the GPS information that [Crandall2011] utilizes camera to carry, set up MRF model, camera towards regarding hidden variable as with photocentre position, adopt belief propagation algorithm to solve energy minimization and can rebuild other SFM of City-level.[sinha2010] and [Jiang2013] first uses [Martinec2007] to solve linearly to obtain all cameras towards, the method, to be verified proof result quite stable and accurately in world coordinate system.But the position in the alive boundary of camera photocentre coordinate system is also not easy to solve, because the yardstick difference that camera is rebuild between [Nister05] is between two unknown.[sinha2010] utilizes two cameras to rebuild (I, j), (j, k) between, there is linear relationship (comprising yardstick and skew), first solve all legal linear relationships between reconstruction between two, then utilize system of linear equations to solve globally all yardstick and skews of rebuilding between two relatively overall world coordinates, thereby the position of the camera photocentre in rebuilding between two also can easily obtain.This method has been avoided boundling adjustment repeatedly, and algorithm is greatly improved on the time, but according to [Arie-Nachimson2012], the result accuracy that this method produces cannot with traditional method comparison.[Jiang2013] investigates the three-view diagram of each sharing feature point, utilizes linear equation to optimize the geometric distance relation between three cameras, can solve the photocentre position that obtains all cameras by all three-view diagrams of simultaneous equally.But this method is too paid attention to the geometrical constraint (comprising the anglec of rotation and baseline ratio between camera) between camera, the projection error of having ignored unique point, herein through facts have proved that the gained camera reconstructed results of averaging projection's error [jiang2013] method is optimized to(for) tlv triple does not reach 30-200 not etc., the reconstruction projection error of camera tlv triple is just so big, the reconstruction finally solving for algorithm, projection error still exists, for some data set, because projection error is too large, even if do boundling adjustment, also cannot obtain relatively good result.

Summary of the invention

For the technical matters existing in prior art, the object of the present invention is to provide a kind of method for reconstructing three-dimensional model based on overall linear method.

The present invention is according to the three-dimensional model method of overall linear method reconstruction of stability from unordered, extensive view data.Traditional exercise recovery structural approach, adds the view of camera one by one in order, and this method may cause the error accumulation of pilot process, affects reconstruction quality.On the contrary, the linear method of the overall situation can be shared these mistakes from the overall situation, thereby guarantees whole reconstruction effect.Propose a kind of method of new nonlinear optimization herein, the view that can obtain stable camera tlv triple is rebuild, and then by the method for overall situation linearity, solves the three-dimensional coordinate of unique point in the attitude of camera and photograph again.First, we by the relative rotation relationship between camera between two solve obtain all cameras towards matrix; Secondly, our technology by nonlinear optimization obtains stable camera tlv triple and rebuilds, and solves the linear relationship in tlv triple; Again, the equation that abundant tlv triple provides, can form a system of linear equations, by numerical method, can be solved and be obtained camera attitude accurately, and the technology of finally adjusting by boundling is carried out last optimization to reconstructed results.Method in this paper through experimental verification be fast, robust and accurately.

Compared with prior art, good effect of the present invention is:

Proposed a kind of linear method of new robust herein, it has been done the method for [Sinha] and [Jiang] comprehensively, makes can access more accurate camera tlv triple when the overall situation solves.Just as most method, we first computing camera towards, the method for using [Martinec] to describe.Then, we utilize the method for [Jiang], solve the position of camera by minimizing geometric error between image center.But facts have proved this method and unlike said so accurate in its paper, find that, after the optimization of its algorithm for tlv triple, the result after its unique point trigonometric ratio is unsatisfactory herein, average error is unacceptable.Therefore,, after this step, the initial solution of the tlv triple that the linear relationship about camera exists between rebuilding between two of our utilization [Sinha] obtains Jiang is done further optimization.Then utilize linear relationship between rebuilding between two to obtain linear equation and solve, obtain preliminary reconstructed results, finally do a boundling adjustment.Our algorithm quite effectively and robust, and is easy to walk abreast.

Accompanying drawing explanation

Fig. 1 is that Rbc is to the linear transformation relation of the 4DOF of Rab;

Fig. 2 is that the optimal value of Ck solves figure.

Embodiment

In this paper is a kind of method that overall situation solves SFM, and step is similar with [sinha2010] [Jiang2013].Linear relationship between rebuilding between two in the method attention camera tlv triple of Sinha, this linear relationship is solved by maximal possibility estimation, and the principle solving is to pay attention to projection error to minimize; The method of Jiang is paid attention to the geometric relationship between camera photocentre, comprises angle and geometric proportion; Method in this paper is by the advantages of the two together, avoids the shortcoming of the two.First utilize the method for [martinec2007] solve fast and accurately all cameras towards.Construct suitable tlv triple, these tlv triple are shared some unique points, then use nonlinear least square method (LM) to estimate the position of three camera photocentres in tlv triple, make to make the averaging projection's error in sinha method less on the one hand, simultaneously the tlv triple camera photocentre geometric error in Jiang method is less, thereby obtains the linear relationship between rebuilding between two of camera accurately.The linear relationship of the camera of finally obtaining by previous step between rebuilding between two, sets up system of linear equations, and the overall situation solves all overall yardstick and skews of rebuilding between two, thereby solves the photocentre position that obtains all cameras.

1) solve camera towards

Attitude Rij relative between a pair of camera can calculate by 5 methods, solves finding towards set Ri towards being regarded as of all cameras, i=1, and 2 ..., n, the relation between them is given by Rij,

In fact, formula 1 has comprised three little equations, and the least square method that last camera can provide by Eigen towards the solution of matrix solves system of linear equations and obtains.

As long as to searching out a spanning tree in camera match map, so all cameras towards obtaining, in order to make last result more accurate, we need to add more believable EG limit and form the set of credible limit on the basis of spanning tree.First, we set up the maximum spanning tree of match map according to the number of the Feature Points Matching number on ijEG limit; Then adding new credible limit in maximum spanning tree, its standard is if a camera tlv triple (I, j, k), wherein (I, j) and (j, k) all in the set of credible limit, limit (I, k) also can be added into credible limit and gather so.The credible limit set forming is like this considered to reliably, and can solve obtain accurate camera towards.

Because accurate camera is towards extremely important, we reject inaccurate EG in match map by [zach2010] about the algorithm of match map loop restriction.

2) solve the linear relationship in tlv triple

First suppose that all EG limits all do not have error, on the one hand, known camera towards, each rebuilds Rab (comprising camera a, b) between two, rebuilds and to differ 4 degree of freedom with the overall situation, comprises unknown yardstick s and three-D migration vector t.Suppose that Rbc and Rab share camera b and some unique points, we can arrive the linear transformation relation of the 4DOF of Rab in the hope of Rbc so, as shown in Figure 1, optimize unique point Xj by the projection error after converting, can obtain s, the value of t, the projection error after conversion is:

${\mathrm{\Σ}}_{k}d(x_{\mathrm{kj}},f_k^{\mathrm{ab}}\left(S^{-1}{X}_j^{\mathrm{bc}}\right))+{\mathrm{\Σ}}_{k}d(x_{\mathrm{kj}},f_k^{\mathrm{bc}}\left({\mathrm{SX}}_j^{\mathrm{ab}}\right))---\left(1\right)$

Wherein, function two dimensional surface by 3 dimensional feature spot projections to the camera in the reconstruction coordinate system of camera two tuples (a, b), k ∈ a, b}, definition with similar; D projects to position in camera plane and the original position x of unique point place camera plane about unique point _kjdistance function.

On the other hand, known camera towards, investigate camera tlv triple, the relative offset vector cij between camera photocentre, cjk, cik is known.We want to estimate camera photocentre coordinate ci, cj, ck.Under ideal state, cij, cik and cjk should be coplanar.It is not coplanar that but various noises make in True Data.We first consider that cij is correct and minimizes ck to two line l (ci, cik) and the Euclidean distance of l (cj, cjk).Here l (p, q) represent a line with q towards crossing point p.The solution of so optimized Ck should be the mid point of two common ground AB, as shown in Figure 2.

C _koptimized solution can be calculated by following formula:

$C_k\≈\frac{1}{2}((C_i+s_{\mathrm{ij}}^{\mathrm{ik}}||C_i-C_j|\left|C_{\mathrm{ik}}\right)+(C_j+s_{\mathrm{ij}}^{\mathrm{jk}}||C_i-C_j|\left|C_{\mathrm{jk}}\right))---\left(2\right)$

Wherein, || C _i-C _j|| be C _ito C _jdistance, they are ratios of stable camera baseline, and the relation at angle as can be seen from Figure 2.

Formula (2) is not linear, and for linearity solves, we have:

||C _i-C _j||C _ik=||C _i-C _j||R _i(θ)C _ij=R(θ)(C _j-C _i) (3)

Here R _i(θ) be Cij to the rotation matrix (counterclockwise) of Cjk, we can obtain as follows about camera photocentre Ci like this, cj, ck linear equation, as follows:

${2c}_k-c_i-c_j=R_i\left({\mathrm{\θ}}_i^{\′}\right)s_{\mathrm{ij}}^{\mathrm{ik}}(c_j-c_i)+R_j\left({-\mathrm{\θ}}_j^{\′}\right)s_{\mathrm{ij}}^{\mathrm{jk}}(c_i-c_j).$

Same, we can obtain by same method following two system of equations

${2c}_j-c_i-c_k=R_i\left({\mathrm{\θ}}_i^{\′}\right)s_{\mathrm{ik}}^{\mathrm{ij}}(c_k-c_i)+R_k\left({-\mathrm{\θ}}_k^{\′}\right)s_{\mathrm{ik}}^{\mathrm{jk}}(c_i-c_k),$

${2c}_i-c_j-c_k=R_j\left({\mathrm{\θ}}_j^{\′}\right)s_{\mathrm{jk}}^{\mathrm{ij}}(c_k-c_j)+R_k\left({-\mathrm{\θ}}_k^{\′}\right)s_{\mathrm{jk}}^{\mathrm{ik}}(c_j-c_k).$

The position of three camera photocentres after solving system of equations obtained above and can being optimized.

As foreword, mention, in order, in conjunction with the advantage of these two kinds of methods, to avoid defect separately.We need to find such one to separate ci, cj, and ck, makes individual features point transformation bring projection average error as far as possible little, ci simultaneously, cj, ck must meet the restriction of above-mentioned equation as far as possible, makes to keep between the camera light heart geometric relationship more accurately.

By nonlinear least square method (LM), solve and obtain Ci herein, Cj, Ck, and utilize yardstick and the offset relationship of formula between can being rebuild between two.The global offset that this linear relationship will be rebuild for solving camera between two at next step.

3) estimation of overall yardstick and skew

Once obtain the abundant transformation relation (s, t) between reconstruction between two, they are yardstick and the skew of the overall situation relatively, is expressed as (s, t), just can obtain by their relation.As follows

$s^{\mathrm{bc}}X+t^{\mathrm{bc}}=s_{\mathrm{ab}}^{\mathrm{bc}}(s^{\mathrm{ab}}X+t^{\mathrm{ab}})+t_{\mathrm{ab}}^{\mathrm{bc}},$

Because X is any 3d unique point, formula both sides should be correspondent equals.Can obtain

$w_{\mathrm{ab}}^{\mathrm{bc}}(s^{\mathrm{bc}}-s_{\mathrm{ab}}^{\mathrm{bc}}{s}^{\mathrm{ab}})=0,$

$w_{\mathrm{ab}}^{\mathrm{bc}}\left(s^{\mathrm{bc}}{t}^{\mathrm{bc}}\right)=w_{\mathrm{ab}}^{\mathrm{bc}}(s_{\mathrm{ab}}^{\mathrm{bc}}{t}^{\mathrm{ab}}+t_{\mathrm{ab}}^{\mathrm{bc}}).$

Wherein w is the number of unique point number, and at this moment, as long as the overall yardstick that any one camera is rebuild is between two made as 1, skew is made as 0, and this system of linear equations can solve so.

The scale of system of linear equations, depends in match map G the number of legal tlv triple.If cannot solve by the method for sinha, obtain preferably preliminary linear relationship s, t, its unique point is at two coordinate system ab, and the projection error converting between bc surpasses 10 pixels, thinks that this tlv triple is illegal.If the projection error after optimizing by nonlinear least square method surpasses 10 pixels, or camera photocentre form leg-of-mutton angle before and after difference surpass 5 degree, think that equally this tlv triple is illegal.Remaining tlv triple is considered to robust accurately, and we find largest connected figure to it, and this largest connected figure is exactly all camera scales that can reduce.

Claims (3)

1. the method for reconstructing three-dimensional model based on overall linear method, the steps include:

1) by the relative rotation relationship between camera between two solve obtain all cameras towards matrix;

2) technology by nonlinear optimization obtains stable camera tlv triple and rebuilds, and solves the linear relationship in tlv triple;

3) equation providing according to tlv triple forms a system of linear equations, by Numerical Methods Solve, obtains camera attitude accurately;

4) technology of adjusting by boundling is carried out last optimization to reconstructed results.

2. the method for claim 1, it is characterized in that camera towards method for solving be: first, according to the number of the Feature Points Matching number on ijEG limit, set up the maximum spanning tree of match map; Then adding new credible limit in maximum spanning tree, its standard is if a camera tlv triple (I, j, k), wherein (I, j) and (j, k) all in the set of credible limit, limit (I, k) is also added into credible limit and gathers so; Then according to the set of credible limit solve obtain accurate camera towards.

3. the method for claim 1, is characterized in that being solved and being obtained three camera photocentre position Ci by nonlinear least square method, Cj, and Ck, and utilize yardstick and the offset relationship of formula between can being rebuild between two.