CN102496184A - Increment three-dimensional reconstruction method based on bayes and facial model - Google Patents

Abstract

The invention discloses an increment three-dimensional reconstruction method based on the bayes and a facial model, which includes the following steps: step1 obtaining projection matrixes of two-dimensional images corresponding to each view angle; step 2 establishing a ball model for all the two-dimensional images and sampling a group of two-dimensional images corresponding to the key view angle; step 3 conducting three-dimensional reconstruction based on a surface element on the two-dimensional images corresponding to the key view angle and obtaining surface element cloud; step 4 locating a two-dimensional image corresponding to a new view angle on the ball model and updating the ball model; step 5 selecting a surface element subset from the surface element cloud; step 6 comparing the average value of the surface element density on the concentrated local three-dimensional surface of the surface element and the average value of the surface element density of the three-dimensional surface of the surface element cloud, and step 7 conducting modeling through the bayes to achieve increment three-dimensional reconstruction. The increment three-dimensional reconstruction method achieves increment reconstruction, and is capable of being used in future real-time three-dimensional reconstruction and multiple resolution ratio reconstruction and updating existing relative three-dimensional models at any point in time.

Description

A kind of increment three-dimensional rebuilding method based on Bayes and bin model

Technical field

The present invention relates to Computerized 3 D visual and rebuild the field, specifically is a kind of increment three-dimensional rebuilding method based on Bayes and bin model, can increment upgrade three-dimensional model with two dimensional image, thereby finally generate three-dimensional model accurately.

Background technology

The development of Along with computer technology, computing machine have related to and the application of human natural interaction with intelligence, and therefore computer vision should meet and give birth to.The scene perception is a huge challenge to computing machine.The basis of scene perception is the three-dimensional information that obtains scene, and then can be used for realizing identification and analysis.In present scene cognition technology; The technology of getting up early mainly is partial to obtain instrument with three-dimensional information and is directly obtained three-dimensional scenic; But these can only handle small-sized object (corresponding to large scene) as scanner owing to obtain instrument, and cost an arm and a leg, and come so can not use widely.In all environment sensing instruments, video camera and camera utilize with cheap quilt in a large number, but how to let video camera can from perception data, obtain three-dimensional scene information as human eyes, are the only way which must be passed that computing machine moves towards intelligence.

At present three-dimensional reconstruction mainly is based upon all pictorial informations is realized that comprehensively three-dimensional information obtains, however these algorithm practical application, because 1) Data Source in the reality all is asynchronous, various mostly.The online picture of picture is clapped from different video cameras mostly; Has no rule; And illumination condition differs; And picture uploads more that new capital is asynchronous, how from new picture mined information to upgrade the three-dimensional model that existed (rebuild in the past or scan) be a urgent problem.2) can not realize real-time application.Under many environment,, in the application of robot, need carry out in time comprehensive and analysis to the real time environment data as real-time monitoring.

How to realize that it must be the challenge of next three-dimensional reconstruction aspect that increment type is rebuild.

Summary of the invention

Goal of the invention: technical matters to be solved by this invention is the deficiency to prior art, and a kind of increment three-dimensional rebuilding method based on Bayes and bin model is provided.

For remedying the deficiency of prior art, the invention discloses a kind of three-dimensional increment method for reconstructing based on Bayes and bin model, comprise following steps:

Step 1 is carried out camera parameter to the two dimensional image under one group of different visual angles of input and is demarcated, and obtains the projection matrix of the corresponding two dimensional image in each visual angle;

Step 2 is set up spherical model to all two dimensional images, the one group of two dimensional image that crucial visual angle is corresponding of sampling, and the trigonometric ratio corresponding three-dimensional point of two dimensional image of sampling; Said three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that the visual angle is corresponding;

Step 3 carries out obtaining bin cloud S based on the three-dimensional reconstruction of bin to the corresponding two dimensional image in said crucial visual angle _Initial

Step 4, the corresponding two dimensional image i in new visual angle, location on said spherical model _NewAnd spherical model upgraded;

Step 5 is according to two dimensional image i _NewPosition on spherical model is from bin cloud S _InitialIn choose a bin subclass P _Update

Step 6, relatively bin subclass P _UpdateMiddle partial 3 d surface bin density and bin cloud S _InitialThree-dimensional surface bin density mean value, use synthetic few samples method expansion bin subclass P _Update

Step 7 is carried out modeling through Bayes, according to the maximum a posteriori method to bin subclass P _UpdateUpgrade, thereby realize the increment three-dimensional reconstruction.

In the step 1 of the present invention, adopt sparse bundle adjustment method that two dimensional image is carried out camera parameter and demarcate, obtain the corresponding projection matrix P of two dimensional image under each visual angle,

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method sees Manolis I.A.Lourakis for details, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.

In the step 2 of the present invention, set up spherical model and be: for the corresponding two dimensional image in any visual angle, the coordinate that makes its corresponding point on spherical model is the normalized vector of optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^T, p ₃₁, p ₃₂, p ₃₃Correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of the corresponding two dimensional image in one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v ₁, v ₂, v ₃), on spherical model, seek the three-dimensional point of once sampling and obtaining with the nearest some conduct of said reference point point Euclidean distance, the corresponding X-Y scheme of said three-dimensional point becomes the corresponding two dimensional image in crucial visual angle; The corresponding three-dimensional point of two dimensional image through the Delaunay triangulation is corresponding with crucial visual angle on the spherical model is carried out trigonometric ratio.

In the step 3 of the present invention, adopt three-dimensional rebuilding method that the corresponding two-dimension image rebuild in crucial visual angle that step 2 obtains is obtained bin cloud S based on the bin model _Initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robust multiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, three-dimensional surface is to be covered by a series of bins, and each local tangential plane is a bin, in model, representes with the three-dimensional rectangle of rule.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is a bin p centre coordinate; N (p) is a normal vector, and direction is pointed to observation point, is used to weigh surface local curvature; R (p) is a corresponding two dimensional image of bin p; It has following attribute: two dimensional image R (p) is a two dimensional image among the image collection V (p); V (p) is the two dimensional image set of a bin p decision, and every two dimensional image in the said set can both unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction that the direction on two limits of three-dimensional rectangle is accomplished a limit wherein as far as possible as far as possible with camera coordinate system in the x direction of principal axis parallel, the rectangle topology size is that its projection in R (p) is no more than the u*u pixel of arranging by axle, is made as 5*5 in the present invention.

Pass through following formula in the step 4 of the present invention and confirm the corresponding two dimensional image i in new visual angle of input _NewThe triangle T of correspondence on spherical model:

$T\←\underset{T}{\arg \max }{\mathrm{\Σ}}_{v\∈T}\left|x_{i_{\mathrm{new}}}^v\right|,$

Wherein v is a summit of a triangle T in the middle of the spherical model,

Be two dimensional image i _NewThe two dimensional image corresponding with vertex v obtains the match point set through the conversion of yardstick invariant features.That is: T is one and two dimensional image i _NewThe triangle that has the maximum match amount;

With after the center-of-mass coordinate normalization of triangle T as two dimensional image i _NewThree-dimensional point coordinate on spherical model, (z), (x, y z) are connected the spherical model after obtaining upgrading with three summits of triangle T in twos with point for x, y to be designated as point.

In the step 5 of the present invention, from initialization bin S set _InitialIn choose and two dimensional image i _NewThe bin subclass that correlativity is the highest.This correlativity is embodied in: two dimensional image i _NewIn can see this bin and bin section and two dimensional image i _NewPlane of delineation angle less relatively.Bin subclass P _UpdateObtain according to following formula:

$P_{\mathrm{update}}=\underset{v\∈T}{\∪}\left\{p\right|p\∈S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\}.$

In the step 6 of the present invention, to bin subclass P _UpdateSet is expanded, and spread step makes full use of two dimensional image i _NewPixel Information and the geological information of three-dimensional model, and can let the three-dimensional surface bin distribute as far as possible evenly, accomplish that the information that makes full use of new input picture goes out some new bins in the three-dimensional surface area extension of low resolution.Spread step is following: to bin cloud S _InitialIn any one bin p calculate local density, gather the neighbours' bin quantity D among the N (p) with neighbours' bin of bin p _pIts local density, the neighbours' bin quantity D of replacing of equal value _pAccount form following:

N(p)＝{p′|p′∈S _initial，|(c(p)-c(p′))·n(p)|+|(c(p)-c(p′))·n(p′)|＜ρ}，

D _p＝|N(p)|，

Wherein ρ is a threshold values; ρ determines through the depth distance that the center of calculating bin p and bin p ' corresponds to the number of pixels β among the two dimensional image R (p) automatically, is that its depth distance that is the center of bin p and bin p ' corresponds to 2 pixels among the two dimensional image R (p) multiply by 2 in the present invention.

Through to all at bin cloud S _InitialIn the D of local density of bin _pAsk arithmetic mean to calculate bin cloud S _InitialThree-dimensional surface bin density mean value D _gTo bin subclass P _UpdateIn arbitrary bin p, if the D of local density _pThree-dimensional surface bin density mean value D less than 1/2nd _g, adopt synthetic minority oversampler method between bin p and neighbours' bin p, to expand the bin k that makes new advances.Wherein synthetic minority oversampler method sees Nitesh V.Chawla for details; Kevin W.Bowyer, Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR, vol.16; Pp.321-357,2002.

In the step 7 of the present invention, through following formula to bin subclass P _UpdateUpgrade and realize Bayes's increment three-dimensional modeling:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\∈\mathrm{\Ω},$

Wherein S is real three-dimensional model, i _NewBe the two dimensional image in the step 4, p (S|i _New) be that three-dimensional model S is at two dimensional image i _NewUnder posterior probability, Z is a normaliztion constant, Ω is the probability space of three-dimensional model, with the bin subclass P of probability space Ω dimensionality reduction in the step 5 _UpdateOn.Probability p (S) is the level and smooth priori of three-dimensional model; P (i _New| S) be two dimensional image i _NewLikelihood probability, be used to weigh three-dimensional model S and two dimensional image i _NewThe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|V\left(p\right)\right|-1}\underset{i\&Element;V\left(p\right)/i_{\mathrm{new}}}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pBe energy function, be used for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have its indirect measurement reflection on image.Said accuracy by arbitrary bin p at two dimensional image i _NewAnd the correlativity h between the projection among the two dimensional image i (p, i _New, i) decision, wherein i is the two dimensional image among the image collection V (p), and η is a control variable, and the h calculation procedure is following: on bin p, cover the grid of a u*u, sizing grid is 5*5 among the present invention; Through each is put at two dimensional image i in the bilinear interpolation computing grid _NewWith the projection among the two dimensional image i; With 1 deduct grid projection in two width of cloth two dimensional images normalization positive correlation amount.This shows that during entirely accurate, the h between arbitrary two pictures is 0, last E _pAlso be 0 to reach minimum, but because the recovery of measuring error and bin p is a reverse problem, so E _pAlways, work as E greater than 0 _pWhen smaller, explain that under existing measurement environment, bin p is accurate more.η is a control variable, is 0.7 among the present invention.

Priori p (S) weighs the level and smooth degree of three-dimensional surface, has showed the geological information of three-dimensional surface to a certain extent.Priori is through energy function E ₁With energy function E ₂Be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

Energy function E wherein ₁Be used to weigh the flatness of three-dimensional surface, weigh local slickness with the local curved transition of bin among the present invention.Energy function E ₂Weigh the divorce degree of bin, because at E at whole three-dimensional surface ₁In the usage vector level and smooth degree of coming presentation surface only; But for level and smooth between some normal vectors; But coordinate drops on the outside bin of three-dimensional surface a threshold values control to be arranged, can play the purpose of filtering unusual bin like this, and accomplish truly level and smooth.Energy function E ₁With energy function E ₂Computing method are:

${\mathrm{<figure-callout\; id="1"\; label="E"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">E</figure-callout>}}_1=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

d(p，v)＝|n(p)·(c(v)-c(p))|，

Wherein f (p, n) be bin p and bin n normal vector between Euclidean distance, (p v) is bin p and the bin v absolute value distance on normal vector n (p) to d, and λ, ζ are two controlled variable, are respectively 0.3,0.2 among the present invention.

Maximization posteriority p (S|i _New) obtain the maximum likelihood three-dimensional model, promptly for P _UpdateIn bin parameter upgrade; Said parameter is the (P at probability space Ω _Update) in the three-dimensional coordinate and the normal vector of each bin.Finally obtain a convergent and separate, be equivalent to:

c(p)，n(p)←arg?max(exp(-{λE ₁+ζE ₂+ηE _p})，p∈P _update。

Get negative logarithm operation, it is: c (p), n (p) ← arg min (λ E ₁+ ζ ₂+ η E _p), p ∈ P _Update, through bin is gathered P _UpdateThe bin cloud of renewal after finally obtaining upgrading.

Adopt conjugate gradient to be optimized problem solving among the present invention.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.When importing new two dimensional image, the present invention turns back to step 4 again and carries out.

Beneficial effect: the present invention has effectively integrated three-dimensional geometric information and the initiate two-dimensional image information that has reconstructed through Bayesian frame; Break through existing three-dimensional reconstruction algorithm and only paid attention to image information; And can not realize the leak that increment is rebuild; For the three-dimensional applications in the reality such as in the future real-time, asynchronous reconstruction has been opened a good head, for computer realization intelligence contributes.The present invention can be used as the core technology of a system of exploitation; Between unmanned car steering, three-dimensional fitting, in the Smart Home, large-scale city modeling, play the part of the renewal process that transfers data to three-dimensional model from video camera; So that next step analysis, application intelligent provide reliable data.Since to view data source require low, so increased robustness to a great extent.

Description of drawings

Below in conjunction with accompanying drawing and embodiment the present invention is done specifying further, above-mentioned and/or otherwise advantage of the present invention will become apparent.

Fig. 1 is a spherical model synoptic diagram of the present invention.

Fig. 2 is the synoptic diagram of bin model modeling three-dimensional surface of the present invention.

The synoptic diagram of Fig. 3 new bin for the present invention expands.

Fig. 4 is the present invention and E ₁The synoptic diagram that priori is relevant.

Fig. 5 is the present invention and E ₂Relevant synoptic diagram.

Fig. 6 a1～Fig. 6 c5 is the experimental result synoptic diagram under three embodiment data sets of the present invention.

Fig. 7 carries out schematic flow sheet for the present invention

Embodiment

As shown in Figure 7, the present invention comprises following steps: step 1, and the two dimensional image under one group of different visual angles of input is carried out camera parameter demarcate, obtain the projection matrix of the corresponding two dimensional image in each visual angle; Step 2 is set up a spherical model to all two dimensional images, the one group of two dimensional image that crucial visual angle is corresponding of sampling, and the trigonometric ratio corresponding three-dimensional point of two dimensional image of sampling; Said three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that the visual angle is corresponding; Step 3 carries out obtaining bin cloud S based on the three-dimensional reconstruction of bin to the corresponding two dimensional image in said crucial visual angle _InitialStep 4, corresponding two dimensional image i in new visual angle in location on spherical model _NewAnd spherical model upgraded; Step 5 is according to two dimensional image i _NewPosition on spherical model is from bin cloud S _InitialIn choose a bin subclass P _UpdateStep 6, relatively bin subclass P _UpdateMiddle partial 3 d surface bin density and bin cloud S _InitialThree-dimensional surface bin density mean value, use synthetic few samples method expansion bin subclass P _UpdateStep 7 is carried out modeling through Bayes, according to the maximum a posteriori method to bin subclass P _UpdateUpgrade, thereby realize the increment three-dimensional reconstruction.

Below in conjunction with accompanying drawing the present invention is done detailed introduction.

In the step 1, adopt sparse bundle adjustment method that two dimensional image is carried out camera parameter and demarcate, obtain the corresponding projection matrix P of two dimensional image under each visual angle,

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method sees Manolis I.A.Lourakis for details, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.

In the step 2, set up spherical model and be: for the corresponding two dimensional image in any visual angle, the coordinate that makes its corresponding point on spherical model is the normalized vector of optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^T, p ₃₁, p ₃₂, p ₃₃Correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of the corresponding two dimensional image in one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v _l, v ₂, v ₃), on spherical model, seek the three-dimensional point of once sampling and obtaining with the nearest some conduct of said reference point Euclidean distance, the corresponding X-Y scheme of said three-dimensional point becomes the corresponding two dimensional image in crucial visual angle; The quantity of the three-dimensional point of sampling generally is 1/3rd of data set size; The corresponding three-dimensional point of two dimensional image through the Delaunay triangulation is corresponding with crucial visual angle on the spherical model is carried out trigonometric ratio.Fig. 1 is the spherical model of trigonometric ratio, from figure to seeing that a three-dimensional point represents a two dimensional image; The corresponding two dimensional image in crucial visual angle forms triangle through the trigonometric ratio method and covers sphere.

In the step 3, adopt three-dimensional rebuilding method that the image reconstruction that step 2 obtains crucial visual angle correspondence is obtained bin cloud S based on the bin model _Initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robustmultiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, what three-dimensional surface was approximate covers with local tangential plane, and as shown in Figure 2, each local tangential plane is a bin, in model, representes with the three-dimensional rectangle of rule.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is a bin p centre coordinate; N (p) is a normal vector, and direction is pointed to observation point, is used to weigh surface local curvature; R (p) is a corresponding two dimensional image of bin p; It has following attribute: two dimensional image R (p) is a two dimensional image among the image collection V (p); V (p) is the visual angle image collection of a bin p decision, and every two dimensional image in the said set can both unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction that the direction on two limits of three-dimensional rectangle is accomplished a limit wherein as far as possible as far as possible with camera coordinate system in the x direction of principal axis parallel, the rectangle topology size is that its projection in picture is no more than the u*u pixel of arranging by axle, is made as 5*5 in the present invention.

Step 4 is confirmed the corresponding two dimensional image i in new visual angle of input through following formula _NewThe triangle T of correspondence on spherical model:

Wherein v is a summit of a triangle T in the middle of the spherical model, Be two dimensional image i _NewThe two dimensional image corresponding with vertex v obtains the match point set through the conversion of yardstick invariant features.T is one and two dimensional image i in fact _NewThe triangle that has the maximum match amount; With after the center-of-mass coordinate normalization of triangle T as two dimensional image i _NewThree-dimensional point coordinate on spherical model, (z), (x, y z) are connected the spherical model after obtaining upgrading with three summits of triangle T in twos with point for x, y to be designated as point.

Can see new visual angle i among Fig. 1 _NewInterconnect with leg-of-mutton three summits, obtained the spherical model after the renewal.

In the step 5, from initialization bin S set _InitialIn choose and two dimensional image i _NewThe bin subclass that correlativity is the highest.This correlativity is embodied in: two dimensional image i _NewIn can see this bin and bin section and two dimensional image i _NewPlane of delineation angle less relatively.Bin subclass P _UpdateObtain according to following formula:

$P_{\mathrm{update}}=\underset{v\&Element;T}{\&cup;}\left\{p\right|p\&Element;S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\}.$

In the step 6 of the present invention, to bin subclass P _UpdateSet is expanded, and spread step makes full use of two dimensional image i _NewPixel Information and the geological information of three-dimensional model, and can let the three-dimensional surface bin distribute as far as possible evenly, accomplish to make full use of pictorial information and go out some new bins in the three-dimensional surface area extension of low resolution.Spread step is following: to bin cloud S _InitialIn any one bin p calculate local density, gather the neighbours' bin quantity D among the N (p) with neighbours' bin of bin p _pIts local density, the neighbours' bin quantity D of replacing of equal value _pAccount form following:

N(p)＝{p′|p′∈S _initial，|(c(p)-c(p′))·n(p)|+|(c(p)-c(p′))·n(p′)|＜ρ}，

D _p＝|N(p)|，

Wherein ρ determines through the depth distance that the center of calculating bin p and bin p ' corresponds to the number of pixels β among the two dimensional image R (p) automatically, is that its depth distance that is the center of bin p and bin p ' corresponds to 2 pixels among the two dimensional image R (p) multiply by 2 in the present invention; Through to all at bin cloud S _InitialIn the D of local density of bin _pAsk arithmetic mean to calculate bin cloud S _InitialThree-dimensional surface bin density mean value D _pTo bin subclass P _UpdateIn arbitrary bin p, if the D of local density _pThree-dimensional surface bin density mean value D less than 1/2nd _g, adopt synthetic minority oversampler method between bin p and neighbours' bin p, to expand the bin k that makes new advances.As can be seen from Figure 3, p ₀And p ₁Between newly-generated at random new bin p on the linear space of line _New, its coordinate and normal vector are according to p _NewPosition in line segment is to p ₀And p ₁Consistent attribute weight on average obtain.Wherein synthetic minority oversampler method sees Nitesh V.Chawla for details; Kevin W.Bowyer; Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR; Vol.16, pp.321-357,2002..

In the step 7 of the present invention, replacement problem is carried out Bayes Modeling through following formula:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\&Element;\mathrm{\&Omega;},$

Wherein S is real three-dimensional scenic, i _NewBe the two dimensional image in the step 4, p (S|i _New) be that three-dimensional model S is at two dimensional image i _NewUnder posterior probability, Z is a normaliztion constant, Ω is the probability space of three-dimensional model, we are on its dimensionality reduction to one bin subclass, i.e. bin subclass P in step 5 _UpdateProbability p (S) is the level and smooth priori of model; P (i _New| S) be two dimensional image i _NewLikelihood probability, be used to weigh three-dimensional model S and two dimensional image i _NewThe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|V\left(p\right)\right|}\underset{i\&Element;V\left(p\right)}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pBe energy function, be used for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have its indirect measurement reflection on image.Said accuracy by arbitrary bin p at two dimensional image i _NewAnd the correlativity h between the projection among the two dimensional image i (p, i _New, i) decision, wherein i is the two dimensional image among the image collection V (p), and η is a control variable, and the h calculation procedure is following: on bin p, cover the grid of a u*u, sizing grid is 5*5 among the present invention; Through each is put at two dimensional image i in the bilinear interpolation computing grid _NewWith the projection among the two dimensional image i; With 1 deduct grid projection in two width of cloth two dimensional images normalization positive correlation amount.This shows that during entirely accurate, the h between arbitrary two pictures is 0, last E _pAlso be 0 to reach minimum, but because the recovery of measuring error and bin p is a reverse problem, so E _pAlways, work as E greater than 0 _pWhen smaller, explain that under existing measurement environment, bin p is accurate more.η is a control variable, is 0.5 among the present invention.

Priori P (S) weighs the level and smooth degree on 3D surface, has excavated the geological information of three-dimensional surface to a certain extent.It passes through energy function E priori ₁With energy function E ₂Be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

E wherein ₁Be used to weigh the flatness of three-dimensional surface, weigh local slickness with the local curved transition of bin among the present invention.E ₁Can not accurately weigh its surface smoothing property, among the present Fig. 4 of its defect body, be curvature though bin p and neighbours on every side have level and smooth normal vector, because it has broken away from original fit surface, so it is a divorce bin.So propose E among the present invention ₂Weigh the divorce degree of bin p at whole three-dimensional surface, coordinate is dropped on the outside bin of three-dimensional surface has a threshold values control, can play the purpose of filtering singular point like this, and accomplishes truly level and smooth.Energy function E ₁With energy function E ₂Computing method are:

${\mathrm{<figure-callout\; id="1"\; label="E"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">E</figure-callout>}}_1=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

d(p，v)＝|n(p)·(c(v)-c(p))|，

Wherein f (p, n) be bin p and bin n normal vector between Euclidean distance, from Fig. 5, mark and can see, (p v) is bin p and the bin v absolute value distance on normal vector n (p) to d; λ, ζ are two controlled variable, are respectively 0.3,0.2 among the present invention.

Maximization posteriority p (S|i _New) obtain the maximum likelihood three-dimensional model, promptly for P _UpdateMiddle bin parameter upgrades; Said parameter is at probability space P _UpdateIn the three-dimensional coordinate and the normal vector of each bin.Finally obtain a convergent and separate, be equivalent to:

C (p), n (p) ← arg max (exp ({ λ E ₁+ ζ E ₂+ η E _p), p ∈ P _UpdateGet negative logarithm operation, that is:

c(p)，n(p)←arg?min(λE ₁+ζE ₂+ηE _p)，p∈P _update

Like this through bin is gathered P _UpdateThe bin cloud of renewal after finally obtaining upgrading.

Adopt conjugate gradient to be optimized problem solving among the present invention.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.

When importing new picture, algorithm turns back to step 4 again and carries out.

Embodiment

As shown in Figure 7; The step of present embodiment comprises: the two dimensional image to all inputs carries out the camera parameter demarcation with sparse bundle adjustment method; Calculate the optical axis vector of the corresponding two dimensional image in each visual angle then; And it is carried out normalization obtain three-dimensional point, set up spherical model according to this, wherein the three-dimensional point on the spherical model is represented the corresponding two dimensional image in a visual angle; Adopt random algorithm to select the corresponding two dimensional image in crucial visual angle at random, the selection that repeats is disregarded; The three-dimensional point that the two dimensional image of selecting to obtain is corresponding adopts triangle gridding subdivision algorithm that they are carried out trigonometric ratio; The two dimensional image corresponding to crucial visual angle carries out the three-dimensional reconstruction based on bin.Initial work is accomplished, and then gets into the increment link.Read in the corresponding two dimensional image in new visual angle, with yardstick invariant features conversion matching algorithm and spherical model, find a triangle, and select bin subclass P according to this _UpdateSpherical model is upgraded; Then with synthetic minority oversampler method expansion P _Update, the bin that makes new advances in sparse area extension; At last through under Bayesian frame, finding the solution an optimization problem to bin subclass P _UpdateUpgrade the renewal that reaches the general three model.Along with constantly reading in of new images, the step in the middle of the increment link is constantly carried out.

Fig. 6 a1～Fig. 6 c5 is the experimental result of having enumerated out under three data sets (owing to be picture, can only adopt the performance of gray scale form).Fig. 6 a1 is the dinosaur image data set; Fig. 6 a2 uses from the corresponding two dimensional image in crucial visual angle that dinosaur image data set sampling obtains and carries out the bin cloud design sketch that the dinosaur 3-dimensional reconstruction based on the bin model obtains, and Fig. 6 a3～Fig. 6 a5 is for constantly reading in the bin cloud design sketch of incremental update behind the corresponding two-dimentional dinosaur image in new visual angle.Fig. 6 b1 is the skull image data set; Fig. 6 b2 uses from the corresponding two dimensional image in crucial visual angle that skull image data set sampling obtains and carries out the bin cloud design sketch that the skull three-dimensional reconstruction based on the bin model obtains, and Fig. 6 b3～Fig. 6 b5 is for constantly reading in the bin cloud design sketch of incremental update behind the corresponding two-dimentional skull image in new visual angle.Fig. 6 c1 is the temple image data set; Fig. 6 c2 uses from the corresponding two dimensional image in crucial visual angle that temple image data set sampling obtains and carries out the bin cloud design sketch that the temple three-dimensional reconstruction based on the bin model obtains, and Fig. 6 c3～Fig. 6 c5 is for constantly reading in the bin cloud design sketch of incremental update behind the corresponding temple two dimensional image in new visual angle.Can find out that bin milks up, and more and more accurate.The present invention has obtained a reasonable result in these three embodiment.

The invention provides a kind of increment three-dimensional rebuilding method based on Bayes and bin model; The method and the approach of concrete this technical scheme of realization are a lot, and the above only is a preferred implementation of the present invention, should be understood that; For those skilled in the art; Under the prerequisite that does not break away from the principle of the invention, can also make some improvement and retouching, these improvement and retouching also should be regarded as protection scope of the present invention.The all available prior art of each ingredient not clear and definite in the present embodiment realizes.

Claims (8)

1. the increment three-dimensional rebuilding method based on Bayes and bin model is characterized in that, may further comprise the steps:

Step 1 is carried out camera parameter to the two dimensional image under one group of different visual angles of input and is demarcated, and obtains the projection matrix of the corresponding two dimensional image in each visual angle;

Step 2 is set up a spherical model to all two dimensional images, the one group of two dimensional image that crucial visual angle is corresponding of sampling, and the trigonometric ratio corresponding three-dimensional point of two dimensional image of sampling; Said three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that the visual angle is corresponding;

Step 3 carries out obtaining bin cloud S based on the three-dimensional reconstruction of bin to the corresponding two dimensional image in said crucial visual angle _Initial

Step 4, corresponding two dimensional image i in new visual angle in location on spherical model _NewAnd spherical model upgraded;

Step 5 is according to two dimensional image i _NewPosition on spherical model is from bin cloud S _InitialIn choose a bin subclass P _Update

Step 6, relatively bin subclass P _UpdateMiddle partial 3 d surface bin density and bin cloud S _InitialThree-dimensional surface bin density mean value, use synthetic few samples method expansion bin subclass P _Update

Step 7 is carried out modeling through Bayes, according to the maximum a posteriori method to bin subclass P _UpdateUpgrade, thereby realize the increment three-dimensional reconstruction.

2. a kind of increment three-dimensional rebuilding method according to claim 1 based on Bayes and bin model; It is characterized in that, in the step 1, adopt sparse bundle adjustment method that two dimensional image is carried out camera parameter and demarcate; Obtain the corresponding projection matrix P of two dimensional image under each visual angle

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.

3. a kind of increment three-dimensional rebuilding method according to claim 2 based on Bayes and bin model; It is characterized in that; In the step 2; Setting up spherical model is: for the corresponding two dimensional image in any visual angle, the coordinate that makes its corresponding point on spherical model is the normalized vector of optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^T, p ₃₁, p ₃₂, p ₃₃Correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of the corresponding two dimensional image in one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v ₁, v ₂, v ₃), on spherical model, seek the three-dimensional point of once sampling and obtaining with the nearest some conduct of said reference point point Euclidean distance, the corresponding X-Y scheme of said three-dimensional point becomes the corresponding two dimensional image in crucial visual angle;

The corresponding three-dimensional point of two dimensional image through triangulation is corresponding with crucial visual angle on the spherical model is carried out trigonometric ratio.

4. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 3 is characterized in that, in the step 4, confirms the corresponding two dimensional image i in new visual angle of input through following formula _NewThe triangle T of correspondence on spherical model:

$T\&LeftArrow;\underset{T}{\arg \max }{\mathrm{\&Sigma;}}_{v\&Element;T}\left|x_{i_{\mathrm{new}}}^v\right|,$

Wherein v is a summit of a triangle T in the middle of the spherical model,

Be two dimensional image i _NewThe two dimensional image corresponding with vertex v obtains the match point set through the conversion of yardstick invariant features;

With after the center-of-mass coordinate normalization of triangle T as two dimensional image i _NewThree-dimensional point coordinate on spherical model, (z), (x, y z) are connected the spherical model after obtaining upgrading with three summits of triangle T in twos with point for x, y to be designated as point.

5. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 4 is characterized in that, the bin subclass P in the step 5 _UpdateObtain according to following formula:

$P_{\mathrm{update}}=\underset{v\&Element;T}{\&cup;}\left\{p\right|p\&Element;S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\}.$

Wherein R (p) is a corresponding two dimensional image of bin p; It has following attribute: two dimensional image R (p) is a two dimensional image among the image collection V (p); V (p) is the two dimensional image set of bin p decision, and every two dimensional image in the said set can both unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.

6. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 5 is characterized in that, in the step 6 to bin subclass P _UpdateSpread step following:

To bin cloud S _InitialIn any one bin p calculate local density, gather the neighbours' bin quantity D among the N (p) with neighbours' bin of bin p _pIts local density, the neighbours' bin quantity D of replacing of equal value _pAccount form following:

N(p)＝{p′|p′∈S _initial，|(c(p)-c(p′))·n(p)|+|(c(p)-c(p′))·n(p′)|＜ρ}，

D _p＝|N(p)|，

C (p) is the 3 dimension geometric center coordinates of bin p, and n (p) is the normal vector of bin p, and wherein the direction of normal vector is pointed to viewpoint direction, and ρ is a threshold values;

Through to all at bin cloud S _InitialIn the D of local density of bin _pAsk arithmetic mean to calculate bin cloud S _InitialThree-dimensional surface bin density mean value D _g

To bin subclass P _UpdateIn arbitrary bin p, if the D of local density _pThree-dimensional surface bin density mean value D less than 1/2nd _g, adopt synthetic minority oversampler method between bin p and neighbours' bin p, to expand the bin k that makes new advances.

7. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 6 is characterized in that, passes through following formula in the step 7 to bin subclass P _UpdateUpgrade and realize Bayes's increment three-dimensional modeling:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\&Element;\mathrm{\&Omega;},$

Wherein S is real three-dimensional model, i _NewBe the two dimensional image in the step 4, p (S|i _New) be that three-dimensional model S is at two dimensional image i _NewUnder posterior probability, Z is a normaliztion constant, Ω is the probability space of three-dimensional model, with the bin subclass P of probability space Ω dimensionality reduction in the step 5 _Update, p (S) is the level and smooth prior probability of three-dimensional model S; P (i _New| S) be two dimensional image i _NewLikelihood probability, be used to weigh three-dimensional model S and two dimensional image i _NewThe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{\left|V\left(p\right)\right|-1}\underset{i\&Element;V\left(p\right)/i_{\mathrm{new}}}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pBe energy function, be used for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, said accuracy by arbitrary bin p at two dimensional image i _NewAnd the correlativity h between the projection among the two dimensional image i (p, i _New, i) decision, wherein i is the two dimensional image among the image collection V (p), η is a control variable, h (p, i _New, i) calculation procedure is following:

On bin p, cover the grid of a u*u; Through each is put at two dimensional image i in the bilinear interpolation computing grid _NewWith the projection among the two dimensional image i; With 1 deduct grid projection in two width of cloth two dimensional images normalization positive correlation amount;

Priori p (S) is through energy function E ₁With energy function E ₂Be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

Energy function E wherein ₁Be used to weigh the flatness of three-dimensional surface, energy function E ₂Replenish and weigh the divorce degree of bin at whole three-dimensional surface, energy function E ₁With energy function E ₂Computing method are:

$E_1=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

d(p，v)＝|n(p)·(c(v)-c(p))|，

N (p) wherein, n (normal vector of v) corresponding bin p and bin v, c (p); C (the geometric center coordinate of v) corresponding bin p and bin v, f (p, n) be bin p and bin n normal vector between Euclidean distance; D (p; V) be bin p and the bin v absolute value distance on normal vector n (p), λ, ζ are two controlled variable.

8. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 7 is characterized in that, maximization posterior probability p (S|i _New), obtain the maximum likelihood surface model, promptly upgrade P _UpdateObtain new three-dimensional model, that is: c (p), n (p) ← arg max (exp ({ λ E ₁+ ζ E ₂+ η E _p), p ∈ P _Update, get negative logarithm operation, that is: c (p), n (p) ← arg min (λ E ₁+ ζ E ₂+ η E _p), p ∈ P _Update, through bin is gathered P _UpdateThe bin cloud of renewal after finally obtaining upgrading.

Images