CN102496184B - Increment three-dimensional reconstruction method based on bayes and facial model - Google Patents

Abstract

The invention discloses an increment three-dimensional reconstruction method based on the bayes and a facial model, which includes the following steps: step1 obtaining projection matrixes of two-dimensional images corresponding to each view angle; step 2 establishing a ball model for all the two-dimensional images and sampling a group of two-dimensional images corresponding to the key view angle; step 3 conducting three-dimensional reconstruction based on a surface element on the two-dimensional images corresponding to the key view angle and obtaining surface element cloud; step 4 locating a two-dimensional image corresponding to a new view angle on the ball model and updating the ball model; step 5 selecting a surface element subset from the surface element cloud; step 6 comparing the average value of the surface element density on the concentrated local three-dimensional surface of the surface element and the average value of the surface element density of the three-dimensional surface of the surface element cloud, and step 7 conducting modeling through the bayes to achieve increment three-dimensional reconstruction. The increment three-dimensional reconstruction method achieves increment reconstruction, and is capable of being used in future real-time three-dimensional reconstruction and multiple resolution ratio reconstruction and updating existing relative three-dimensional models at any point in time.

Description

A kind of increment three-dimensional rebuilding method based on Bayes and bin model

Technical field

The present invention relates to Computerized 3 D visual and rebuild field, a kind of increment three-dimensional rebuilding method based on Bayes and bin model specifically, can increment upgrade three-dimensional model with two dimensional image, thereby finally generate three-dimensional model accurately.

Background technology

Along with the development of computer technology, computing machine has related to and mankind's natural interaction and intelligent application, and therefore computer vision should meet and give birth to.The scene perception is a huge challenge to computing machine.The basis of scene perception is the three-dimensional information that obtains scene, and then can be used for realizing identification and analyze.In current scene cognition technology, the technology of getting up early mainly is partial to obtain instrument with three-dimensional information and is directly obtained three-dimensional scenic, but due to the instrument that obtains, as scanner, these can only process small-sized object (corresponding to large scene), and expensive, so can not apply and come widely.In all environment sensing instruments, video camera and camera utilize in a large number with cheap quilt, but how to allow video camera can obtain three-dimensional scene information from perception data as mankind's eyes, are that computing machine moves towards intelligent the only way which must be passed.

At present three-dimensional reconstruction mainly is based upon all pictorial informations is comprehensively realized to three-dimensional information obtains, however these algorithm practical application, because 1) Data Source in reality is all asynchronous, various mostly.The online picture of picture is clapped from different video cameras mostly, without any rule, and illumination condition differs, and picture uploading more new capital is asynchronous, how from new picture mined information to upgrade the three-dimensional model that existed (rebuild in the past or scan) be a urgent problem.2) can not realize real-time application.Under many environment, as real-time monitoring, in the application of robot, need to carry out comprehensive and analysis in time to the real time environment data.

How to realize that it must be the challenge of next three-dimensional reconstruction aspect that increment type is rebuild.

Summary of the invention

Goal of the invention: technical matters to be solved by this invention is for the deficiencies in the prior art, and a kind of increment three-dimensional rebuilding method based on Bayes and bin model is provided.

For making up the deficiencies in the prior art, the invention discloses a kind of three-dimensional increment method for reconstructing based on Bayes and bin model, comprise following steps:

Step 1, carry out the camera parameter demarcation to the two dimensional image under one group of different visual angles of input, obtains the projection matrix of the corresponding two dimensional image in each visual angle;

Step 2, set up spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding;

Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S _initial;

Step 4, location two dimensional image i corresponding to new visual angle on described spherical model _newand spherical model is upgraded;

Step 5, according to two dimensional image i _newposition on spherical model, from bin cloud S _initialin choose a bin subset P _update;

Step 6, relatively bin subset P _updatemiddle partial 3 d surface bin density and bin cloud S _initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P _update;

Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P _updateupgraded, thereby realized the increment three-dimensional reconstruction.

In step 1 of the present invention, adopt sparse bundle to adjust method two dimensional image carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle,

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method refers to Manolis I.A.Lourakis, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.

In step 2 of the present invention, set up spherical model and be: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^t, p ₃₁, p ₃₂, p ₃₃correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v ₁, v ₂, v ₃), find the point nearest with described reference point point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle; Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by the Delaunay triangulation carried out trigonometric ratio.

In step 3 of the present invention, two-dimension image rebuild corresponding to crucial visual angle that adopts the three-dimensional rebuilding method based on the bin model to obtain step 2 obtains bin cloud S _initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robust multiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, three-dimensional surface is to be covered by a series of bins, and each local tangential plane is a bin, in model, by regular three-dimensional rectangle, means.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is bin p centre coordinate; N (p) is normal vector, and direction is pointed to observation point, for weighing surface local curvature; R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the two dimensional image set that a bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction on two limits of three-dimensional rectangle accomplishes that wherein the direction on a limit is as far as possible parallel with the x direction of principal axis in camera coordinate system as far as possible, and the rectangle topology size is that its projection in R (p) is no more than the u*u pixel of pressing the axle arrangement, is made as in the present invention 5*5.

Determine the two dimensional image i corresponding to new visual angle of input in step 4 of the present invention by following formula _newthe triangle T of correspondence on spherical model:

$T\←\underset{T}{\arg \max }{\mathrm{\Σ}}_{v\∈T}\left|x_{i_{\mathrm{new}}}^v\right|,$

The summit that wherein v is a triangle T in the middle of spherical model,

two dimensional image i _newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features.That is: T is one and two dimensional image i _newthe triangle that has maximum coupling amount;

Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i _newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.

In step 5 of the present invention, from initialization bin S set _initialin choose and two dimensional image i _newthe bin subset that correlativity is the highest.This correlativity is embodied in: two dimensional image i _newin can see this bin and bin section and two dimensional image i _newplane of delineation angle less.Bin subset P _updateaccording to following formula, obtain:

$P_{\mathrm{update}}=\underset{v\∈T}{\∪}\left\{p\right|p\∈S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\}.$

In step 6 of the present invention, to bin subset P _updateset is expanded, and spread step takes full advantage of two dimensional image i _newpixel Information and the geological information of three-dimensional model, and can allow the three-dimensional surface bin distribute as far as possible evenly, accomplish that the information that takes full advantage of new input picture goes out some new bins in the three-dimensional surface area extension of low resolution.Spread step is as follows: to bin cloud S _initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p _pits local density, the neighbours' bin quantity D of replacing of equal value _paccount form as follows:

N(p)={p′|p′∈S _initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ}，

D _p=|N(p)|，

Wherein ρ is threshold values; ρ corresponds to the number of pixels β in two dimensional image R (p) depth distance by calculating bin p and bin p ′ center automatically determines, is that its depth distance that is 2 pixels during bin p and bin p ′ center correspond to two dimensional image R (p) is multiplied by 2 in the present invention.

By to all at bin cloud S _initialin the D of local density of bin _pask arithmetic mean to calculate bin cloud S _initialthree-dimensional surface bin density mean value D _g; To bin subset P _updatein arbitrary bin p, if the D of local density _pbe less than 1/2nd three-dimensional surface bin density mean value D _g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin p.Wherein synthetic minority oversampler method refers to Nitesh V.Chawla, Kevin W.Bowyer, Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR, vol.16, pp.321-357,2002.

In step 7 of the present invention, by following formula to bin subset P _updaterenewal realizes Bayes's increment three-dimensional modeling:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\&Element;\mathrm{\&Omega;},$

Wherein S is real three-dimensional model, i _newthe two dimensional image in step 4, p (S|i _new) be that three-dimensional model S is at two dimensional image i _newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, by probability space Ω dimensionality reduction to the bin subset P in step 5 _updateon.The level and smooth priori that Probability p (S) is three-dimensional model; p(i _new| S) be two dimensional image i _newlikelihood probability, for weighing three-dimensional model S and two dimensional image i _newthe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png,HDA0000118628770000021.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|V\left(p\right)\right|-1}\underset{i\&Element;V\left(p\right)/i_{\mathrm{new}}}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have it indirectly to measure reflection on image.Described accuracy by arbitrary bin p at two dimensional image i _newand the correlativity h between the projection in two dimensional image i (p, i _new, i) determine, wherein i is the two dimensional image in image collection V (p), and η is control variable, and the h calculation procedure is as follows: cover the grid of a u*u on bin p, in the present invention, sizing grid is 5*5; By in the bilinear interpolation computing grid, each is put at two dimensional image i _newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images.As can be seen here, during entirely accurate, the h between arbitrary two pictures is 0, last E _pbe also 0 to reach minimum, but because the recovery of measuring error and bin p is a Reverse Problem, so E _palways be greater than 0, work as E _pwhen smaller, illustrate that, under existing measurement environment, bin p is more accurate.η is control variable, in the present invention, is 0.5.

Priori p (S) weighs the level and smooth degree of three-dimensional surface, has showed to a certain extent the geological information of three-dimensional surface.Priori is by energy function E ₁with energy function E ₂be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

Energy function E wherein ₁for weighing the flatness of three-dimensional surface, the curvature with the bin part in the present invention changes to weigh local slickness.Energy function E ₂weigh the divorced degree of bin at whole three-dimensional surface, because at E ₁in only usage vector carry out the level and smooth degree of presentation surface, but for level and smooth between some normal vectors, but coordinate drops on the bin of three-dimensional surface outside, to there is threshold values to control, can play like this purpose of filtering unusual bin, and accomplish truly level and smooth.Energy function E ₁with energy function E ₂computing method are:

${\mathrm{<figure-callout\; id="1"\; label="E"\; filenames="HDA0000118628770000014.png,HDA0000118628770000021.png"\; state="\{\{state\}\}">E</figure-callout>}}_1=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png,HDA0000118628770000021.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

$d(p,v)=|n\left(p\right)\&CenterDot;(c\left(v\right)-c\left(p\right))|,$

Wherein f (p, v) be bin p and bin v normal vector between Euclidean distance, d (p, v) is bin p and the absolute value distance of bin v on normal vector n (p), λ, ζ is two control parameters, is respectively 0.3,0.2 in the present invention.

Maximize posteriority p (S|i _new) obtain the maximum likelihood three-dimensional model, for P _updatein bin parameter upgraded; Described parameter is the (P at probability space Ω _update) in three-dimensional coordinate and the normal vector of each bin.Finally obtain the solution of a convergence, be equivalent to:

c(p),n(p)←argmax（exp(-{λE ₁+ζE ₂+ηE _p}),p∈P _update。

Get negative logarithm operation, it is: c (p), n (p) ← argmin (λ E ₁+ ζ E ₂+ η E _p), p ∈ P _update, by the set P of opposite unit _updatethe bin cloud of renewal after finally being upgraded.

In the present invention, adopt conjugate gradient to be optimized problem solving.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.When inputting new two dimensional image, the present invention turns back to again step 4 and carries out.

Beneficial effect: the present invention has effectively integrated the three-dimensional geometric information reconstructed and the two-dimensional image information newly added by Bayesian frame, break through existing three-dimensional reconstruction algorithm and only focused on image information, and can not realize the leak that increment is rebuild, for the three-dimensional applications in the reality such as in the future real-time, asynchronous reconstruction has been opened a good head, for computer realization intelligence contributes.The present invention can be used as the core technology of a system of exploitation, play the part of the renewal process that transfers data to three-dimensional model from video camera between unmanned car steering, three-dimensional fitting, in Smart Home, large-scale city modeling, so that next step analysis, application intelligence provides reliable data.Due to view data source require low, so increased to a great extent robustness.

The accompanying drawing explanation

Below in conjunction with the drawings and specific embodiments, the present invention is done further and illustrates, above-mentioned and/or otherwise advantage of the present invention will become apparent.

Fig. 1 is spherical model schematic diagram of the present invention.

The schematic diagram that Fig. 2 is bin model modeling three-dimensional surface of the present invention.

Fig. 3 is the schematic diagram that the present invention expands new bin.

Fig. 4 is the present invention and E ₁the schematic diagram that priori is relevant.

Fig. 5 is the present invention and E ₂relevant schematic diagram.

Fig. 6 a1～Fig. 6 c5 is the experimental result schematic diagram under three embodiment data sets of the present invention.

Fig. 7 is that the present invention carries out schematic flow sheet

Embodiment

As shown in Figure 7, the present invention comprises following steps: step 1, the two dimensional image under one group of different visual angles of input is carried out to the camera parameter demarcation, and obtain the projection matrix of the corresponding two dimensional image in each visual angle; Step 2, set up a spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding; Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S _initial; Step 4, the two dimensional image i that new visual angle is corresponding in location on spherical model _newand spherical model is upgraded; Step 5, according to two dimensional image i _newposition on spherical model, from bin cloud S _initialin choose a bin subset P _update; Step 6, relatively bin subset P _updatemiddle partial 3 d surface bin density and bin cloud S _initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P _update; Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P _updateupgraded, thereby realized the increment three-dimensional reconstruction.

Below in conjunction with accompanying drawing, the present invention is done to detailed introduction.

In step 1, adopt sparse bundle to adjust method two dimensional image carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle,

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method refers to Manolis I.A.Lourakis, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.

In step 2, set up spherical model and be: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^t, p ₃₁, p ₃₂, p ₃₃correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v ₁, v ₂, v ₃), find the point nearest with described reference point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle; The quantity of the three-dimensional point of sampling is generally 1/3rd of data set size; Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by the Delaunay triangulation carried out trigonometric ratio.The spherical model that Fig. 1 is trigonometric ratio, from figure to seeing that a three-dimensional point represents a two dimensional image; Two dimensional image corresponding to crucial visual angle forms triangle by Triangulation Algorithm and covers sphere.

In step 3, the three-dimensional rebuilding method of employing based on the bin model obtains image reconstruction corresponding to crucial visual angle to step 2 and obtains bin cloud S _initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robust multiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, what three-dimensional surface was approximate covers with local tangential plane, and as shown in Figure 2, each local tangential plane is a bin, in model, by regular three-dimensional rectangle, means.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is bin p centre coordinate; N (p) is normal vector, and direction is pointed to observation point, for weighing surface local curvature; R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the visual angle image collection that a bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction on two limits of three-dimensional rectangle accomplishes that wherein the direction on a limit is as far as possible parallel with the x direction of principal axis in camera coordinate system as far as possible, and the rectangle topology size is that its projection in picture is no more than the u*u pixel of pressing the axle arrangement, is made as in the present invention 5*5.

Step 4 is determined the two dimensional image i corresponding to new visual angle of input by following formula _newthe triangle T of correspondence on spherical model: the summit that wherein v is a triangle T in the middle of spherical model, two dimensional image i _newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features.T is one and two dimensional image i in fact _newthe triangle that has maximum coupling amount; Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i _newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.

Can see new visual angle i in Fig. 1 _newinterconnect with leg-of-mutton three summits, obtained the spherical model after a renewal.

In step 5, from initialization bin S set _initialin choose and two dimensional image i _newthe bin subset that correlativity is the highest.This correlativity is embodied in: two dimensional image i _newin can see this bin and bin section and two dimensional image i _newplane of delineation angle less.Bin subset P _updateaccording to following formula, obtain:

$P_{\mathrm{update}}=\underset{v\&Element;T}{\&cup;}\left\{p\right|\&Element;S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\}.$

In step 6 of the present invention, to bin subset P _updateset is expanded, and spread step takes full advantage of two dimensional image i _newpixel Information and the geological information of three-dimensional model, and can allow the three-dimensional surface bin distribute as far as possible evenly, accomplish to take full advantage of pictorial information and go out some new bins in the three-dimensional surface area extension of low resolution.Spread step is as follows: to bin cloud S _initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p _pits local density, the neighbours' bin quantity D of replacing of equal value _paccount form as follows:

N(p)={p′|p′∈S _initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ}，

D _p=|N(p)|，

The depth distance that wherein ρ corresponds to the number of pixels β in two dimensional image R (p) by calculating bin p and bin p ′ center automatically determines, is that its depth distance that is 2 pixels during bin p and bin p ′ center correspond to two dimensional image R (p) is multiplied by 2 in the present invention; By to all at bin cloud S _initialin the D of local density of bin _pask arithmetic mean to calculate bin cloud S _initialthree-dimensional surface bin density mean value D _g; To bin subset P _updatein arbitrary bin p, if the D of local density _pbe less than 1/2nd three-dimensional surface bin density mean value D _g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin p.As can be seen from Figure 3, p ₀and p ₁between a random newly-generated new bin p on the linear space of line _new, its coordinate and normal vector are according to p _newposition in line segment is to p ₀and p ₁consistent attribute weight on average obtain.Wherein synthetic minority oversampler method refers to Nitesh V.Chawla, Kevin W.Bowyer, Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR, vol.16, pp.321-357,2002..

In step 7 of the present invention, by following formula, replacement problem is carried out to Bayes Modeling:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\&Element;\mathrm{\&Omega;},$

Wherein S is real three-dimensional scenic, i _newthe two dimensional image in step 4, p (S|i _new) be that three-dimensional model S is at two dimensional image i _newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, we are on its dimensionality reduction to one bin subset, i.e. bin subset P in step 5 _update.The level and smooth priori that Probability p (S) is model; p(i _new| S) be two dimensional image i _newlikelihood probability, for weighing three-dimensional model S and two dimensional image i _newthe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{V\left(p\right)}\underset{i\&Element;V\left(p\right)}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have it indirectly to measure reflection on image.Described accuracy by arbitrary bin p at two dimensional image i _newand the correlativity h between the projection in two dimensional image i (p, i _new, i) determine, wherein i is the two dimensional image in image collection V (p), and η is control variable, and the h calculation procedure is as follows: cover the grid of a u*u on bin p, in the present invention, sizing grid is 5*5; By in the bilinear interpolation computing grid, each is put at two dimensional image i _newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images.As can be seen here, during entirely accurate, the h between arbitrary two pictures is 0, last E _pbe also 0 to reach minimum, but because the recovery of measuring error and bin p is a Reverse Problem, so E _palways be greater than 0, work as E _pwhen smaller, illustrate that, under existing measurement environment, bin p is more accurate.η is control variable, in the present invention, is 0.5.

Priori P (S) weighs the level and smooth degree on 3D surface, has excavated to a certain extent the geological information of three-dimensional surface.It passes through energy function E priori ₁with energy function E ₂be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

E wherein ₁for weighing the flatness of three-dimensional surface, the curvature with the bin part in the present invention changes to weigh local slickness.E ₁can not accurately weigh its surface smoothness, in the present Fig. 4 of its defect body, be curvature although bin p and neighbours on every side have level and smooth normal vector, because it has broken away from original fit surface, so it is a divorced bin.So propose E in the present invention ₂weigh the divorced degree of bin p at whole three-dimensional surface, the bin that coordinate is dropped on to the three-dimensional surface outside has a threshold values to control, and can play like this purpose of filtering singular point, and accomplishes truly level and smooth.Energy function E ₁with energy function E ₂computing method are:

${\mathrm{<figure-callout\; id="1"\; label="E"\; filenames="HDA0000118628770000014.png,HDA0000118628770000021.png"\; state="\{\{state\}\}">E</figure-callout>}}_1=\frac{1}{\left|S\right|}\underset{p\&Element;\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="HDA0000118628770000014.png,HDA0000118628770000021.png"\; state="\{\{state\}\}">S</figure-callout>}}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

d(p,v)=|n(p)·(c(v)-c(p))|，

Wherein f (p, n) be bin p and bin n normal vector between Euclidean distance, mark and can see from Fig. 5, d (p, v) is bin p and the absolute value distance of bin v on normal vector n (p); λ, ζ is two control parameters, is respectively 0.3,0.2 in the present invention.

Maximize posteriority p (S|i _new) obtain the maximum likelihood three-dimensional model, for P _updatemiddle bin parameter is upgraded; Described parameter is at probability space P _updatein three-dimensional coordinate and the normal vector of each bin.Finally obtain the solution of a convergence, be equivalent to:

c(p),n(p)←argmax（exp(-{λE ₁+ζE ₂+ηE _p}),p∈P _update

Get negative logarithm operation, that is:

c(p),n(p)←argmin(λE ₁+ζE ₂+ηE _p),p∈P _update

Like this by the set P of opposite unit _updatethe bin cloud of renewal after finally being upgraded.

In the present invention, adopt conjugate gradient to be optimized problem solving.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.

When inputting new picture, algorithm turns back to again step 4 and carries out.

Embodiment

As shown in Figure 7, the step of the present embodiment comprises: the two dimensional image to all inputs carries out the camera parameter demarcation by sparse bundle adjustment method, then calculate the optical axis vector of two dimensional image corresponding to each visual angle, and it is carried out to normalization obtain three-dimensional point, set up according to this spherical model, wherein the three-dimensional point on spherical model represents a two dimensional image that visual angle is corresponding; Adopt random algorithm to select at random two dimensional image corresponding to crucial visual angle, the selection repeated is disregarded; Adopt triangle gridding subdivision algorithm to carry out trigonometric ratio to them three-dimensional point corresponding to two dimensional image of selecting to obtain; The two dimensional image corresponding to crucial visual angle carries out the three-dimensional reconstruction based on bin.Initial work completes, and then enters the increment link.Read in two dimensional image corresponding to new visual angle, with yardstick invariant features Transformation Matching algorithm and spherical model, find a triangle, and select according to this bin subset P _update; Spherical model is upgraded; Then with synthetic minority oversampler method expansion P _update, the bin made new advances in sparse area extension; Finally by solving an optimization problem to bin subset P under Bayesian frame _updateupgrade the renewal that reaches the general three model.Along with constantly reading in of new images, the step in the middle of the increment link is constantly carried out.

Fig. 6 a1～Fig. 6 c5 is the experimental result (owing to being picture, can only adopt the performance of gray scale form) listed under three data sets.Fig. 6 a1 is the dinosaur image data set, Fig. 6 a2 uses two dimensional image corresponding to crucial visual angle obtained from dinosaur image data set sampling to carry out the bin cloud design sketch that the dinosaur 3-dimensional reconstruction based on the bin model obtains, and Fig. 6 a3～Fig. 6 a5 is for constantly reading in the bin cloud design sketch of incremental update after two-dimentional dinosaur image corresponding to new visual angle.Fig. 6 b1 is the skull image data set, Fig. 6 b2 uses two dimensional image corresponding to crucial visual angle obtained from skull image data set sampling to carry out the bin cloud design sketch that the skull three-dimensional reconstruction based on the bin model obtains, and Fig. 6 b3～Fig. 6 b5 is for constantly reading in the bin cloud design sketch of incremental update after two-dimentional skull image corresponding to new visual angle.Fig. 6 c1 is the temple image data set, Fig. 6 c2 uses two dimensional image corresponding to crucial visual angle obtained from temple image data set sampling to carry out the bin cloud design sketch that the temple three-dimensional reconstruction based on the bin model obtains, and Fig. 6 c3～Fig. 6 c5 is for constantly reading in the bin cloud design sketch of incremental update after temple two dimensional image corresponding to new visual angle.Can find out, bin milks up, and more and more accurate.In these three embodiment, the present invention has obtained a reasonable result.

The invention provides a kind of increment three-dimensional rebuilding method based on Bayes and bin model; method and the approach of this technical scheme of specific implementation are a lot; the above is only the preferred embodiment of the present invention; should be understood that; for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.In the present embodiment not clear and definite each ingredient all available prior art realized.

Claims (5)

1. the increment three-dimensional rebuilding method based on Bayes and bin model, is characterized in that, comprises the following steps:

Step 1, carry out the camera parameter demarcation to the two dimensional image under one group of different visual angles of input, obtains the projection matrix of the corresponding two dimensional image in each visual angle;

Step 2, set up a spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding;

Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S _initial;

Step 4, the two dimensional image i that new visual angle is corresponding in location on spherical model _newand spherical model is upgraded;

Step 5, according to two dimensional image i _newposition on spherical model, from bin cloud S _initialin choose a bin subset P _update;

Step 6, relatively bin subset P _updatemiddle partial 3 d surface bin density and bin cloud S _initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P _update;

Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P _updateupgraded, thereby realized the increment three-dimensional reconstruction;

In step 6 to bin subset P _updatespread step as follows:

To bin cloud S _initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p _pits local density, the neighbours' bin quantity D of replacing of equal value _paccount form as follows:

N(p)={p′|p′∈S _initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ}，

D _p=|N(p)|，

The 3 dimension Geometric center coordinates that c (p) is bin p, the normal vector that n (p) is bin p, wherein the direction of normal vector is pointed to viewpoint direction, and ρ is threshold values;

By to all at bin cloud S _initialin the D of local density of bin _pask arithmetic mean to calculate bin cloud S _initialthree-dimensional surface bin density mean value D _g;

To bin subset P _updatein arbitrary bin p, if the D of local density _pbe less than 1/2nd three-dimensional surface bin density mean value D _g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin;

In step 7 by following formula to bin subset P _updaterenewal realizes Bayes's increment three-dimensional modeling:

$p\left(S\right|i_{\mathrm{new}})=\frac{1}{Z}p\left(i_{\mathrm{new}}\right|S)p\left(S\right),S\&Element;\mathrm{\&Omega;},$

Wherein S is real three-dimensional model, i _newthe two dimensional image in step 4, p (S|i _new) be that three-dimensional model S is at two dimensional image i _newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, by probability space Ω dimensionality reduction to the bin subset P in step 5 _update, the level and smooth prior probability that p (S) is three-dimensional model S; p(i _new| S) be two dimensional image i _newlikelihood probability, for weighing three-dimensional model S and two dimensional image i _newthe likelihood degree, be expressed as:

p(i _new|S)∝exp(-ηE _p)，

$E_p=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{\left|V\left(p\right)\right|-1}\underset{i\&Element;V\left(p\right)/i_{\mathrm{new}}}{\mathrm{\&Sigma;}}h(p,i_{\mathrm{new}},i),$

E wherein _pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, described accuracy by arbitrary bin p at two dimensional image i _newand the correlativity between the projection in two dimensional image i

H (p, i _new, i) determine, wherein i is the two dimensional image in image collection V (p), η is control variable, h (p, i _new, i) calculation procedure is as follows:

Cover the grid of a u*u on bin p; By in the bilinear interpolation computing grid, each is put at two dimensional image i _newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images;

Priori p (S) is by energy function E ₁with energy function E ₂be expressed as:

p(S)∝exp(-{λE ₁+ζE ₂})，

Energy function E wherein ₁for weighing the flatness of three-dimensional surface, energy function E ₂supplement and weigh the divorced degree of bin at whole three-dimensional surface, energy function E ₁with energy function E ₂computing method are:

$E_1=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{\left|N\left(p\right)\right|}\underset{n\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}f(p,v),$

$f(p,v)=\sqrt{{(n\left(p\right)-n\left(v\right))}^T(n\left(p\right)-n\left(v\right))},$

$E_2=\frac{1}{\left|S\right|}\underset{p\&Element;S}{\mathrm{\&Sigma;}}\frac{1}{N\left(p\right)}\underset{v\&Element;N\left(p\right)}{\mathrm{\&Sigma;}}d(p,v),$

d(p,v)=|n(p)·(c(v)-c(p))|，

N (p) wherein, the normal vector of the corresponding bin p of n (v) and bin v, c (p), the Geometric center coordinates of the corresponding bin p of c (v) and bin v, f (p, v) be bin p and bin v normal vector between Euclidean distance, d (p, v) be bin p and the bin v absolute value distance on normal vector n (p), λ, ζ is two control parameters;

Maximize posterior probability p (S|i _new), obtain the maximum likelihood surface model, upgrade P _updateobtain new three-dimensional model, that is: c (p), n (p) ← argmax (exp ({ λ E ₁+ ζ E ₂+ η E _p), p ∈ P _update, get negative logarithm operation, that is: c (p), n (p) ← argmin (λ E ₁+ ζ E ₂+ η E _p), p ∈ P _update, by the set P of opposite unit _updatethe bin cloud of renewal after finally being upgraded.

2. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 1, it is characterized in that, in step 1, adopt sparse bundle to adjust method two dimensional image is carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle

$P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& p_{13}& p_{14}\\ p_{21}& p_{22}& p_{23}& p_{24}\\ p_{31}& p_{32}& p_{33}& p_{34}\end{array}\right],$

Wherein projection matrix P is the real matrix of 3*4.

3. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 2, it is characterized in that, in step 2, setting up spherical model is: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p ₃₁p ₃₂p ₃₃) ^t, p ₃₁, p ₃₂, p ₃₃correspond respectively to the first three columns element of its corresponding projection matrix P the third line;

The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v ₁, v ₂, v ₃), find the point nearest with described reference point point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle;

Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by triangulation carried out trigonometric ratio.

4. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 3, is characterized in that, in step 4, determines the two dimensional image i corresponding to new visual angle of input by following formula _newthe triangle T of correspondence on spherical model:

$T\&LeftArrow;\underset{T}{\arg \max }{\mathrm{\&Sigma;}}_{v\&Element;T}\left|x_{i_{\mathrm{new}}}^v\right|,$

The summit that wherein v is a triangle T in the middle of spherical model,

two dimensional image i _newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features;

Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i _newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.

5. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 4, is characterized in that, the bin subset P in step 5 _updateaccording to following formula, obtain:

$P_{\mathrm{update}}=\underset{v\&Element;T}{\&cup;}\left\{p\right|p\&Element;S_{\mathrm{initial}},\mathrm{visR}\left(p\right)\},$

Wherein R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the two dimensional image set that bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.

Images