CN103985154A - Three-dimensional model reestablishment method based on global linear method - Google Patents

Three-dimensional model reestablishment method based on global linear method Download PDF

Info

Publication number
CN103985154A
CN103985154A CN201410171960.4A CN201410171960A CN103985154A CN 103985154 A CN103985154 A CN 103985154A CN 201410171960 A CN201410171960 A CN 201410171960A CN 103985154 A CN103985154 A CN 103985154A
Authority
CN
China
Prior art keywords
camera
linear
tlv triple
limit
reestablishment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201410171960.4A
Other languages
Chinese (zh)
Inventor
林嘉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to CN201410171960.4A priority Critical patent/CN103985154A/en
Publication of CN103985154A publication Critical patent/CN103985154A/en
Pending legal-status Critical Current

Links

Landscapes

  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a three-dimensional model reestablishment method based on a global linear method. According to the method, firstly, the orientation array of all cameras is obtained by solving the relative rotating relations between every two cameras; secondly, stable camera triad reestablishment is obtained through the nonlinear optimization technology, and a linear relation in a triad is solved; thirdly, a system of linear equations is formed according to equations provided by the triad, accurate camera postures can be obtained by solution through a numerical method, and finally, final optimization is carried out on reestablishment results through the technology of concentration of beam adjustment. The three-dimensional model reestablishment method based on the global linear method is fast, robust and accurate through experimental verification.

Description

A kind of method for reconstructing three-dimensional model based on overall linear method
Technical field
The present invention relates to a kind of method for reconstructing three-dimensional model based on overall linear method, belong to computer software technical field.
Background technology
Have in recent years a lot of people at the three-dimensional model of a large amount of unordered incompatible reconstruction scenery of online photograph collection of research and utilization, and exercise recovery structure (sfm) is the committed step in these reconstruction flow processs.The flow process of three-dimensional reconstruction comprises: 1. pair all pictures are done image characteristics extraction (generally adopting SIFT feature); 2. pair unique point is mated (finding the common point of seeing between camera); 3. the attitude of an initialization phase group of planes and the three-dimensional coordinate of unique point; 4. adjust camera attitude and scene structure (bundle adjustment).The 3rd step wherein, traditional method [Snavely06], [Li08], [Agarwal09] all adopts increment bundle adjustment (incremental bundle adjustment) in preliminary reconstruction camera attitude and unique point three-dimensional coordinate, be two the closest pictures of initial selected contact, attitude and the characteristic point position of two cameras of reduction, then adjust and remove error; Add again a pictures, reduce the position of attitude and unique point of this camera, so continuous repetition until camera attitude corresponding to all picture and the three-dimensional coordinate of unique point be reduced out.This method for the reconstruction effect of a lot of scenes well.
The method [David Crandall2011] that David adopts is regarded exercise recovery structure (sfm) Markov model is asked to energy minimization as, the disposable three-dimensional coordinate that solves attitude and the unique point of all cameras.It is all fast that David claims that his method is adjusted (IBA) method than current any increment boundling, and more have robustness for the mistake in rebuilding.First the method for David utilizes all pictures to calculate initial all camera attitudes, then with boundling adjustment, does once last scene structure correction.But the algorithm of David is better for the reconstruction effect of large-scale view data, because people are not high for the accuracy requirement of camera attitude in this case, only pay close attention to whole structure.But the algorithm of David has fatal defect for solving accurate camera attitude, algorithm is supposed all cameras all at grade, and this causes the attitude of camera impossible completely accurately.
Some had proposed the algorithm that the new overall situation solves exercise recovery structure in recent years, [Sinha] designed the linear algorithm of the multipaths of a robust, utilize two cameras between rebuilding camera towards the linear relationship of situation, the overall situation solve camera attitude and unique point coordinate.[Jiang] designs a kind of linear algorithm of novel robust equally, known camera towards in the situation that, the geometric error of distance between algorithmic minimizing image center, overall linearity solves a good original reconstruction.It is high that but above-mentioned two kinds of methods all rely on the accuracy to utmost point geometry that camera is rebuild between two, if can accurately not screen out wrong diagonal angle how much, algorithm will collapse.
Classic method: a lot of famous Sfm systems are all that first camera between two calculates relative attitude, adopts in order or the method for layering is added camera in whole reconstruction to one by one.Their common defects is exactly, and 1. each camera that adds need to be done a boundling adjustment, if this initial seed that has a strong impact on the arithmetic speed 2. increment boundling adjustment of algorithm does not choose, the effect of rebuilding so can be affected; 3. for the scene of large wide-angle, increment adds camera and unavoidably brings cumulative errors, causes the camera posture deforming of rear interpolation, characteristic point position out of true.
The method of the overall situation: some overall algorithms are computing camera attitudes in world coordinate system in two steps, their first computing camera towards, and then the position of computing camera photocentre.[Crandall2011] and [Jiang2013] [sinha2010] all belongs to this method, the GPS information that [Crandall2011] utilizes camera to carry, set up MRF model, camera towards regarding hidden variable as with photocentre position, adopt belief propagation algorithm to solve energy minimization and can rebuild other SFM of City-level.[sinha2010] and [Jiang2013] first uses [Martinec2007] to solve linearly to obtain all cameras towards, the method, to be verified proof result quite stable and accurately in world coordinate system.But the position in the alive boundary of camera photocentre coordinate system is also not easy to solve, because the yardstick difference that camera is rebuild between [Nister05] is between two unknown.[sinha2010] utilizes two cameras to rebuild (I, j), (j, k) between, there is linear relationship (comprising yardstick and skew), first solve all legal linear relationships between reconstruction between two, then utilize system of linear equations to solve globally all yardstick and skews of rebuilding between two relatively overall world coordinates, thereby the position of the camera photocentre in rebuilding between two also can easily obtain.This method has been avoided boundling adjustment repeatedly, and algorithm is greatly improved on the time, but according to [Arie-Nachimson2012], the result accuracy that this method produces cannot with traditional method comparison.[Jiang2013] investigates the three-view diagram of each sharing feature point, utilizes linear equation to optimize the geometric distance relation between three cameras, can solve the photocentre position that obtains all cameras by all three-view diagrams of simultaneous equally.But this method is too paid attention to the geometrical constraint (comprising the anglec of rotation and baseline ratio between camera) between camera, the projection error of having ignored unique point, herein through facts have proved that the gained camera reconstructed results of averaging projection's error [jiang2013] method is optimized to(for) tlv triple does not reach 30-200 not etc., the reconstruction projection error of camera tlv triple is just so big, the reconstruction finally solving for algorithm, projection error still exists, for some data set, because projection error is too large, even if do boundling adjustment, also cannot obtain relatively good result.
Summary of the invention
For the technical matters existing in prior art, the object of the present invention is to provide a kind of method for reconstructing three-dimensional model based on overall linear method.
The present invention is according to the three-dimensional model method of overall linear method reconstruction of stability from unordered, extensive view data.Traditional exercise recovery structural approach, adds the view of camera one by one in order, and this method may cause the error accumulation of pilot process, affects reconstruction quality.On the contrary, the linear method of the overall situation can be shared these mistakes from the overall situation, thereby guarantees whole reconstruction effect.Propose a kind of method of new nonlinear optimization herein, the view that can obtain stable camera tlv triple is rebuild, and then by the method for overall situation linearity, solves the three-dimensional coordinate of unique point in the attitude of camera and photograph again.First, we by the relative rotation relationship between camera between two solve obtain all cameras towards matrix; Secondly, our technology by nonlinear optimization obtains stable camera tlv triple and rebuilds, and solves the linear relationship in tlv triple; Again, the equation that abundant tlv triple provides, can form a system of linear equations, by numerical method, can be solved and be obtained camera attitude accurately, and the technology of finally adjusting by boundling is carried out last optimization to reconstructed results.Method in this paper through experimental verification be fast, robust and accurately.
Compared with prior art, good effect of the present invention is:
Proposed a kind of linear method of new robust herein, it has been done the method for [Sinha] and [Jiang] comprehensively, makes can access more accurate camera tlv triple when the overall situation solves.Just as most method, we first computing camera towards, the method for using [Martinec] to describe.Then, we utilize the method for [Jiang], solve the position of camera by minimizing geometric error between image center.But facts have proved this method and unlike said so accurate in its paper, find that, after the optimization of its algorithm for tlv triple, the result after its unique point trigonometric ratio is unsatisfactory herein, average error is unacceptable.Therefore,, after this step, the initial solution of the tlv triple that the linear relationship about camera exists between rebuilding between two of our utilization [Sinha] obtains Jiang is done further optimization.Then utilize linear relationship between rebuilding between two to obtain linear equation and solve, obtain preliminary reconstructed results, finally do a boundling adjustment.Our algorithm quite effectively and robust, and is easy to walk abreast.
Accompanying drawing explanation
Fig. 1 is that Rbc is to the linear transformation relation of the 4DOF of Rab;
Fig. 2 is that the optimal value of Ck solves figure.
Embodiment
In this paper is a kind of method that overall situation solves SFM, and step is similar with [sinha2010] [Jiang2013].Linear relationship between rebuilding between two in the method attention camera tlv triple of Sinha, this linear relationship is solved by maximal possibility estimation, and the principle solving is to pay attention to projection error to minimize; The method of Jiang is paid attention to the geometric relationship between camera photocentre, comprises angle and geometric proportion; Method in this paper is by the advantages of the two together, avoids the shortcoming of the two.First utilize the method for [martinec2007] solve fast and accurately all cameras towards.Construct suitable tlv triple, these tlv triple are shared some unique points, then use nonlinear least square method (LM) to estimate the position of three camera photocentres in tlv triple, make to make the averaging projection's error in sinha method less on the one hand, simultaneously the tlv triple camera photocentre geometric error in Jiang method is less, thereby obtains the linear relationship between rebuilding between two of camera accurately.The linear relationship of the camera of finally obtaining by previous step between rebuilding between two, sets up system of linear equations, and the overall situation solves all overall yardstick and skews of rebuilding between two, thereby solves the photocentre position that obtains all cameras.
1) solve camera towards
Attitude Rij relative between a pair of camera can calculate by 5 methods, solves finding towards set Ri towards being regarded as of all cameras, i=1, and 2 ..., n, the relation between them is given by Rij,
In fact, formula 1 has comprised three little equations, and the least square method that last camera can provide by Eigen towards the solution of matrix solves system of linear equations and obtains.
As long as to searching out a spanning tree in camera match map, so all cameras towards obtaining, in order to make last result more accurate, we need to add more believable EG limit and form the set of credible limit on the basis of spanning tree.First, we set up the maximum spanning tree of match map according to the number of the Feature Points Matching number on ijEG limit; Then adding new credible limit in maximum spanning tree, its standard is if a camera tlv triple (I, j, k), wherein (I, j) and (j, k) all in the set of credible limit, limit (I, k) also can be added into credible limit and gather so.The credible limit set forming is like this considered to reliably, and can solve obtain accurate camera towards.
Because accurate camera is towards extremely important, we reject inaccurate EG in match map by [zach2010] about the algorithm of match map loop restriction.
2) solve the linear relationship in tlv triple
First suppose that all EG limits all do not have error, on the one hand, known camera towards, each rebuilds Rab (comprising camera a, b) between two, rebuilds and to differ 4 degree of freedom with the overall situation, comprises unknown yardstick s and three-D migration vector t.Suppose that Rbc and Rab share camera b and some unique points, we can arrive the linear transformation relation of the 4DOF of Rab in the hope of Rbc so, as shown in Figure 1, optimize unique point Xj by the projection error after converting, can obtain s, the value of t, the projection error after conversion is:
Σ k d ( x kj , f k ab ( S - 1 X j bc ) ) + Σ k d ( x kj , f k bc ( SX j ab ) ) - - - ( 1 )
Wherein, function two dimensional surface by 3 dimensional feature spot projections to the camera in the reconstruction coordinate system of camera two tuples (a, b), k ∈ a, b}, definition with similar; D projects to position in camera plane and the original position x of unique point place camera plane about unique point kjdistance function.
On the other hand, known camera towards, investigate camera tlv triple, the relative offset vector cij between camera photocentre, cjk, cik is known.We want to estimate camera photocentre coordinate ci, cj, ck.Under ideal state, cij, cik and cjk should be coplanar.It is not coplanar that but various noises make in True Data.We first consider that cij is correct and minimizes ck to two line l (ci, cik) and the Euclidean distance of l (cj, cjk).Here l (p, q) represent a line with q towards crossing point p.The solution of so optimized Ck should be the mid point of two common ground AB, as shown in Figure 2.
C koptimized solution can be calculated by following formula:
C k ≈ 1 2 ( ( C i + s ij ik | | C i - C j | | C ik ) + ( C j + s ij jk | | C i - C j | | C jk ) ) - - - ( 2 )
Wherein, || C i-C j|| be C ito C jdistance, they are ratios of stable camera baseline, and the relation at angle as can be seen from Figure 2.
Formula (2) is not linear, and for linearity solves, we have:
||C i-C j||C ik=||C i-C j||R i(θ)C ij=R(θ)(C j-C i) (3)
Here R i(θ) be Cij to the rotation matrix (counterclockwise) of Cjk, we can obtain as follows about camera photocentre Ci like this, cj, ck linear equation, as follows:
2 c k - c i - c j = R i ( θ i ′ ) s ij ik ( c j - c i ) + R j ( - θ j ′ ) s ij jk ( c i - c j ) .
Same, we can obtain by same method following two system of equations
2 c j - c i - c k = R i ( θ i ′ ) s ik ij ( c k - c i ) + R k ( - θ k ′ ) s ik jk ( c i - c k ) ,
2 c i - c j - c k = R j ( θ j ′ ) s jk ij ( c k - c j ) + R k ( - θ k ′ ) s jk ik ( c j - c k ) .
The position of three camera photocentres after solving system of equations obtained above and can being optimized.
As foreword, mention, in order, in conjunction with the advantage of these two kinds of methods, to avoid defect separately.We need to find such one to separate ci, cj, and ck, makes individual features point transformation bring projection average error as far as possible little, ci simultaneously, cj, ck must meet the restriction of above-mentioned equation as far as possible, makes to keep between the camera light heart geometric relationship more accurately.
By nonlinear least square method (LM), solve and obtain Ci herein, Cj, Ck, and utilize yardstick and the offset relationship of formula between can being rebuild between two.The global offset that this linear relationship will be rebuild for solving camera between two at next step.
3) estimation of overall yardstick and skew
Once obtain the abundant transformation relation (s, t) between reconstruction between two, they are yardstick and the skew of the overall situation relatively, is expressed as (s, t), just can obtain by their relation.As follows
s bc X + t bc = s ab bc ( s ab X + t ab ) + t ab bc ,
Because X is any 3d unique point, formula both sides should be correspondent equals.Can obtain
w ab bc ( s bc - s ab bc s ab ) = 0 ,
w ab bc ( s bc t bc ) = w ab bc ( s ab bc t ab + t ab bc ) .
Wherein w is the number of unique point number, and at this moment, as long as the overall yardstick that any one camera is rebuild is between two made as 1, skew is made as 0, and this system of linear equations can solve so.
The scale of system of linear equations, depends in match map G the number of legal tlv triple.If cannot solve by the method for sinha, obtain preferably preliminary linear relationship s, t, its unique point is at two coordinate system ab, and the projection error converting between bc surpasses 10 pixels, thinks that this tlv triple is illegal.If the projection error after optimizing by nonlinear least square method surpasses 10 pixels, or camera photocentre form leg-of-mutton angle before and after difference surpass 5 degree, think that equally this tlv triple is illegal.Remaining tlv triple is considered to robust accurately, and we find largest connected figure to it, and this largest connected figure is exactly all camera scales that can reduce.

Claims (3)

1. the method for reconstructing three-dimensional model based on overall linear method, the steps include:
1) by the relative rotation relationship between camera between two solve obtain all cameras towards matrix;
2) technology by nonlinear optimization obtains stable camera tlv triple and rebuilds, and solves the linear relationship in tlv triple;
3) equation providing according to tlv triple forms a system of linear equations, by Numerical Methods Solve, obtains camera attitude accurately;
4) technology of adjusting by boundling is carried out last optimization to reconstructed results.
2. the method for claim 1, it is characterized in that camera towards method for solving be: first, according to the number of the Feature Points Matching number on ijEG limit, set up the maximum spanning tree of match map; Then adding new credible limit in maximum spanning tree, its standard is if a camera tlv triple (I, j, k), wherein (I, j) and (j, k) all in the set of credible limit, limit (I, k) is also added into credible limit and gathers so; Then according to the set of credible limit solve obtain accurate camera towards.
3. the method for claim 1, is characterized in that being solved and being obtained three camera photocentre position Ci by nonlinear least square method, Cj, and Ck, and utilize yardstick and the offset relationship of formula between can being rebuild between two.
CN201410171960.4A 2014-04-25 2014-04-25 Three-dimensional model reestablishment method based on global linear method Pending CN103985154A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410171960.4A CN103985154A (en) 2014-04-25 2014-04-25 Three-dimensional model reestablishment method based on global linear method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410171960.4A CN103985154A (en) 2014-04-25 2014-04-25 Three-dimensional model reestablishment method based on global linear method

Publications (1)

Publication Number Publication Date
CN103985154A true CN103985154A (en) 2014-08-13

Family

ID=51277111

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410171960.4A Pending CN103985154A (en) 2014-04-25 2014-04-25 Three-dimensional model reestablishment method based on global linear method

Country Status (1)

Country Link
CN (1) CN103985154A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200523A (en) * 2014-09-11 2014-12-10 中国科学院自动化研究所 Large-scale scene three-dimensional reconstruction method for fusion of additional information
CN106846467A (en) * 2017-01-23 2017-06-13 阿依瓦(北京)技术有限公司 Entity scene modeling method and system based on the optimization of each camera position
CN107507277A (en) * 2017-07-31 2017-12-22 北京康邦科技有限公司 Three-dimensional point cloud reconstructing method and device, server and readable storage medium storing program for executing
CN108280858A (en) * 2018-01-29 2018-07-13 重庆邮电大学 A kind of linear global camera motion method for parameter estimation in multiple view reconstruction
WO2018214179A1 (en) * 2017-05-23 2018-11-29 上海交通大学 Low-dimensional bundle adjustment calculation method and system
CN109493415A (en) * 2018-09-20 2019-03-19 北京大学 A kind of the global motion initial method and system of aerial images three-dimensional reconstruction
CN109741403A (en) * 2018-12-29 2019-05-10 重庆邮电大学 It is a kind of that scaling method is translated based on global linear camera
CN113177999A (en) * 2021-03-25 2021-07-27 杭州易现先进科技有限公司 Visual three-dimensional reconstruction method, system, electronic device and storage medium

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200523A (en) * 2014-09-11 2014-12-10 中国科学院自动化研究所 Large-scale scene three-dimensional reconstruction method for fusion of additional information
CN104200523B (en) * 2014-09-11 2017-09-26 中国科学院自动化研究所 A kind of large scene three-dimensional rebuilding method for merging additional information
CN106846467A (en) * 2017-01-23 2017-06-13 阿依瓦(北京)技术有限公司 Entity scene modeling method and system based on the optimization of each camera position
WO2018214179A1 (en) * 2017-05-23 2018-11-29 上海交通大学 Low-dimensional bundle adjustment calculation method and system
CN107507277A (en) * 2017-07-31 2017-12-22 北京康邦科技有限公司 Three-dimensional point cloud reconstructing method and device, server and readable storage medium storing program for executing
CN107507277B (en) * 2017-07-31 2021-04-06 北京康邦科技有限公司 Three-dimensional point cloud reconstruction method and device, server and readable storage medium
CN108280858A (en) * 2018-01-29 2018-07-13 重庆邮电大学 A kind of linear global camera motion method for parameter estimation in multiple view reconstruction
CN108280858B (en) * 2018-01-29 2022-02-01 重庆邮电大学 Linear global camera motion parameter estimation method in multi-view reconstruction
CN109493415A (en) * 2018-09-20 2019-03-19 北京大学 A kind of the global motion initial method and system of aerial images three-dimensional reconstruction
CN109741403A (en) * 2018-12-29 2019-05-10 重庆邮电大学 It is a kind of that scaling method is translated based on global linear camera
CN109741403B (en) * 2018-12-29 2023-04-07 重庆邮电大学 Camera translation calibration method based on global linearity
CN113177999A (en) * 2021-03-25 2021-07-27 杭州易现先进科技有限公司 Visual three-dimensional reconstruction method, system, electronic device and storage medium

Similar Documents

Publication Publication Date Title
CN103985154A (en) Three-dimensional model reestablishment method based on global linear method
CN103106688B (en) Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering
WO2018127007A1 (en) Depth image acquisition method and system
CN104156968B (en) Large-area complex-terrain-region unmanned plane sequence image rapid seamless splicing method
CN101887589B (en) Stereoscopic vision-based real low-texture image reconstruction method
CN104661010B (en) Method and device for establishing three-dimensional model
WO2021138989A1 (en) Depth estimation acceleration method for multiband stereo camera
CN103440653A (en) Binocular vision stereo matching method
CN103413352A (en) Scene three-dimensional reconstruction method based on RGBD multi-sensor fusion
CN107862744A (en) Aviation image three-dimensional modeling method and Related product
CN103426165A (en) Precise registration method of ground laser-point clouds and unmanned aerial vehicle image reconstruction point clouds
Honegger et al. Embedded real-time multi-baseline stereo
CN104361628A (en) Three-dimensional real scene modeling system based on aviation oblique photograph measurement
CN103822616A (en) Remote-sensing image matching method with combination of characteristic segmentation with topographic inequality constraint
CN104820991A (en) Multi-soft-constraint stereo matching method based on cost matrix
CN102589530A (en) Method for measuring position and gesture of non-cooperative target based on fusion of two dimension camera and three dimension camera
CN109215118B (en) Incremental motion structure recovery optimization method based on image sequence
CN110488871B (en) Unmanned aerial vehicle path dynamic planning method based on two-dimensional and three-dimensional integration
CN104240229B (en) A kind of adaptive method for correcting polar line of infrared binocular camera
CN105913488A (en) Three-dimensional-mapping-table-based three-dimensional point cloud rapid reconstruction method
CN103345765B (en) Based on moving object detection devices and methods therefor under the mobile platform of DSP+FPGA
CN104463953A (en) Three-dimensional reconstruction method based on inertial measurement unit and RGB-D sensor
CN112734839A (en) Monocular vision SLAM initialization method for improving robustness
Stucker et al. ResDepth: Learned residual stereo reconstruction
CN106408531A (en) GPU acceleration-based hierarchical adaptive three-dimensional reconstruction method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20140813

WD01 Invention patent application deemed withdrawn after publication