CN102496184B - Increment three-dimensional reconstruction method based on bayes and facial model - Google Patents
Increment three-dimensional reconstruction method based on bayes and facial model Download PDFInfo
- Publication number
- CN102496184B CN102496184B CN201110411429.6A CN201110411429A CN102496184B CN 102496184 B CN102496184 B CN 102496184B CN 201110411429 A CN201110411429 A CN 201110411429A CN 102496184 B CN102496184 B CN 102496184B
- Authority
- CN
- China
- Prior art keywords
- bin
- dimensional
- dimensional image
- new
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000001815 facial effect Effects 0.000 title abstract 2
- 238000005070 sampling Methods 0.000 claims abstract description 22
- 230000000007 visual effect Effects 0.000 claims description 53
- 239000013598 vector Substances 0.000 claims description 31
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000005303 weighing Methods 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000007476 Maximum Likelihood Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 239000013589 supplement Substances 0.000 claims 1
- 230000006870 function Effects 0.000 description 12
- 238000013461 design Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 241000086550 Dinosauria Species 0.000 description 4
- 210000003625 skull Anatomy 0.000 description 4
- 230000008447 perception Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000004513 sizing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 210000004080 milk Anatomy 0.000 description 1
- 235000013336 milk Nutrition 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Landscapes
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an increment three-dimensional reconstruction method based on the bayes and a facial model, which includes the following steps: step1 obtaining projection matrixes of two-dimensional images corresponding to each view angle; step 2 establishing a ball model for all the two-dimensional images and sampling a group of two-dimensional images corresponding to the key view angle; step 3 conducting three-dimensional reconstruction based on a surface element on the two-dimensional images corresponding to the key view angle and obtaining surface element cloud; step 4 locating a two-dimensional image corresponding to a new view angle on the ball model and updating the ball model; step 5 selecting a surface element subset from the surface element cloud; step 6 comparing the average value of the surface element density on the concentrated local three-dimensional surface of the surface element and the average value of the surface element density of the three-dimensional surface of the surface element cloud, and step 7 conducting modeling through the bayes to achieve increment three-dimensional reconstruction. The increment three-dimensional reconstruction method achieves increment reconstruction, and is capable of being used in future real-time three-dimensional reconstruction and multiple resolution ratio reconstruction and updating existing relative three-dimensional models at any point in time.
Description
Technical field
The present invention relates to Computerized 3 D visual and rebuild field, a kind of increment three-dimensional rebuilding method based on Bayes and bin model specifically, can increment upgrade three-dimensional model with two dimensional image, thereby finally generate three-dimensional model accurately.
Background technology
Along with the development of computer technology, computing machine has related to and mankind's natural interaction and intelligent application, and therefore computer vision should meet and give birth to.The scene perception is a huge challenge to computing machine.The basis of scene perception is the three-dimensional information that obtains scene, and then can be used for realizing identification and analyze.In current scene cognition technology, the technology of getting up early mainly is partial to obtain instrument with three-dimensional information and is directly obtained three-dimensional scenic, but due to the instrument that obtains, as scanner, these can only process small-sized object (corresponding to large scene), and expensive, so can not apply and come widely.In all environment sensing instruments, video camera and camera utilize in a large number with cheap quilt, but how to allow video camera can obtain three-dimensional scene information from perception data as mankind's eyes, are that computing machine moves towards intelligent the only way which must be passed.
At present three-dimensional reconstruction mainly is based upon all pictorial informations is comprehensively realized to three-dimensional information obtains, however these algorithm practical application, because 1) Data Source in reality is all asynchronous, various mostly.The online picture of picture is clapped from different video cameras mostly, without any rule, and illumination condition differs, and picture uploading more new capital is asynchronous, how from new picture mined information to upgrade the three-dimensional model that existed (rebuild in the past or scan) be a urgent problem.2) can not realize real-time application.Under many environment, as real-time monitoring, in the application of robot, need to carry out comprehensive and analysis in time to the real time environment data.
How to realize that it must be the challenge of next three-dimensional reconstruction aspect that increment type is rebuild.
Summary of the invention
Goal of the invention: technical matters to be solved by this invention is for the deficiencies in the prior art, and a kind of increment three-dimensional rebuilding method based on Bayes and bin model is provided.
For making up the deficiencies in the prior art, the invention discloses a kind of three-dimensional increment method for reconstructing based on Bayes and bin model, comprise following steps:
Step 2, set up spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding;
Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S
initial;
Step 4, location two dimensional image i corresponding to new visual angle on described spherical model
newand spherical model is upgraded;
Step 5, according to two dimensional image i
newposition on spherical model, from bin cloud S
initialin choose a bin subset P
update;
Step 6, relatively bin subset P
updatemiddle partial 3 d surface bin density and bin cloud S
initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P
update;
Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P
updateupgraded, thereby realized the increment three-dimensional reconstruction.
In step 1 of the present invention, adopt sparse bundle to adjust method two dimensional image carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle,
Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method refers to Manolis I.A.Lourakis, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.
In step 2 of the present invention, set up spherical model and be: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p
31p
32p
33)
t, p
31, p
32, p
33correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v
1, v
2, v
3), find the point nearest with described reference point point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle; Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by the Delaunay triangulation carried out trigonometric ratio.
In step 3 of the present invention, two-dimension image rebuild corresponding to crucial visual angle that adopts the three-dimensional rebuilding method based on the bin model to obtain step 2 obtains bin cloud S
initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robust multiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, three-dimensional surface is to be covered by a series of bins, and each local tangential plane is a bin, in model, by regular three-dimensional rectangle, means.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is bin p centre coordinate; N (p) is normal vector, and direction is pointed to observation point, for weighing surface local curvature; R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the two dimensional image set that a bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction on two limits of three-dimensional rectangle accomplishes that wherein the direction on a limit is as far as possible parallel with the x direction of principal axis in camera coordinate system as far as possible, and the rectangle topology size is that its projection in R (p) is no more than the u*u pixel of pressing the axle arrangement, is made as in the present invention 5*5.
Determine the two dimensional image i corresponding to new visual angle of input in step 4 of the present invention by following formula
newthe triangle T of correspondence on spherical model:
The summit that wherein v is a triangle T in the middle of spherical model,
two dimensional image i
newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features.That is: T is one and two dimensional image i
newthe triangle that has maximum coupling amount;
Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i
newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.
In step 5 of the present invention, from initialization bin S set
initialin choose and two dimensional image i
newthe bin subset that correlativity is the highest.This correlativity is embodied in: two dimensional image i
newin can see this bin and bin section and two dimensional image i
newplane of delineation angle less.Bin subset P
updateaccording to following formula, obtain:
In step 6 of the present invention, to bin subset P
updateset is expanded, and spread step takes full advantage of two dimensional image i
newpixel Information and the geological information of three-dimensional model, and can allow the three-dimensional surface bin distribute as far as possible evenly, accomplish that the information that takes full advantage of new input picture goes out some new bins in the three-dimensional surface area extension of low resolution.Spread step is as follows: to bin cloud S
initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p
pits local density, the neighbours' bin quantity D of replacing of equal value
paccount form as follows:
N(p)={p′|p′∈S
initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ},
D
p=|N(p)|,
Wherein ρ is threshold values; ρ corresponds to the number of pixels β in two dimensional image R (p) depth distance by calculating bin p and bin p ′ center automatically determines, is that its depth distance that is 2 pixels during bin p and bin p ′ center correspond to two dimensional image R (p) is multiplied by 2 in the present invention.
By to all at bin cloud S
initialin the D of local density of bin
pask arithmetic mean to calculate bin cloud S
initialthree-dimensional surface bin density mean value D
g; To bin subset P
updatein arbitrary bin p, if the D of local density
pbe less than 1/2nd three-dimensional surface bin density mean value D
g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin p.Wherein synthetic minority oversampler method refers to Nitesh V.Chawla, Kevin W.Bowyer, Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR, vol.16, pp.321-357,2002.
In step 7 of the present invention, by following formula to bin subset P
updaterenewal realizes Bayes's increment three-dimensional modeling:
Wherein S is real three-dimensional model, i
newthe two dimensional image in step 4, p (S|i
new) be that three-dimensional model S is at two dimensional image i
newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, by probability space Ω dimensionality reduction to the bin subset P in step 5
updateon.The level and smooth priori that Probability p (S) is three-dimensional model; p(i
new| S) be two dimensional image i
newlikelihood probability, for weighing three-dimensional model S and two dimensional image i
newthe likelihood degree, be expressed as:
p(i
new|S)∝exp(-ηE
p),
E wherein
pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have it indirectly to measure reflection on image.Described accuracy by arbitrary bin p at two dimensional image i
newand the correlativity h between the projection in two dimensional image i (p, i
new, i) determine, wherein i is the two dimensional image in image collection V (p), and η is control variable, and the h calculation procedure is as follows: cover the grid of a u*u on bin p, in the present invention, sizing grid is 5*5; By in the bilinear interpolation computing grid, each is put at two dimensional image i
newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images.As can be seen here, during entirely accurate, the h between arbitrary two pictures is 0, last E
pbe also 0 to reach minimum, but because the recovery of measuring error and bin p is a Reverse Problem, so E
palways be greater than 0, work as E
pwhen smaller, illustrate that, under existing measurement environment, bin p is more accurate.η is control variable, in the present invention, is 0.5.
Priori p (S) weighs the level and smooth degree of three-dimensional surface, has showed to a certain extent the geological information of three-dimensional surface.Priori is by energy function E
1with energy function E
2be expressed as:
p(S)∝exp(-{λE
1+ζE
2}),
Energy function E wherein
1for weighing the flatness of three-dimensional surface, the curvature with the bin part in the present invention changes to weigh local slickness.Energy function E
2weigh the divorced degree of bin at whole three-dimensional surface, because at E
1in only usage vector carry out the level and smooth degree of presentation surface, but for level and smooth between some normal vectors, but coordinate drops on the bin of three-dimensional surface outside, to there is threshold values to control, can play like this purpose of filtering unusual bin, and accomplish truly level and smooth.Energy function E
1with energy function E
2computing method are:
Wherein f (p, v) be bin p and bin v normal vector between Euclidean distance, d (p, v) is bin p and the absolute value distance of bin v on normal vector n (p), λ, ζ is two control parameters, is respectively 0.3,0.2 in the present invention.
Maximize posteriority p (S|i
new) obtain the maximum likelihood three-dimensional model, for P
updatein bin parameter upgraded; Described parameter is the (P at probability space Ω
update) in three-dimensional coordinate and the normal vector of each bin.Finally obtain the solution of a convergence, be equivalent to:
c(p),n(p)←argmax(exp(-{λE
1+ζE
2+ηE
p}),p∈P
update。
Get negative logarithm operation, it is: c (p), n (p) ← argmin (λ E
1+ ζ E
2+ η E
p), p ∈ P
update, by the set P of opposite unit
updatethe bin cloud of renewal after finally being upgraded.
In the present invention, adopt conjugate gradient to be optimized problem solving.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.When inputting new two dimensional image, the present invention turns back to again step 4 and carries out.
Beneficial effect: the present invention has effectively integrated the three-dimensional geometric information reconstructed and the two-dimensional image information newly added by Bayesian frame, break through existing three-dimensional reconstruction algorithm and only focused on image information, and can not realize the leak that increment is rebuild, for the three-dimensional applications in the reality such as in the future real-time, asynchronous reconstruction has been opened a good head, for computer realization intelligence contributes.The present invention can be used as the core technology of a system of exploitation, play the part of the renewal process that transfers data to three-dimensional model from video camera between unmanned car steering, three-dimensional fitting, in Smart Home, large-scale city modeling, so that next step analysis, application intelligence provides reliable data.Due to view data source require low, so increased to a great extent robustness.
The accompanying drawing explanation
Below in conjunction with the drawings and specific embodiments, the present invention is done further and illustrates, above-mentioned and/or otherwise advantage of the present invention will become apparent.
Fig. 1 is spherical model schematic diagram of the present invention.
The schematic diagram that Fig. 2 is bin model modeling three-dimensional surface of the present invention.
Fig. 3 is the schematic diagram that the present invention expands new bin.
Fig. 4 is the present invention and E
1the schematic diagram that priori is relevant.
Fig. 5 is the present invention and E
2relevant schematic diagram.
Fig. 6 a1~Fig. 6 c5 is the experimental result schematic diagram under three embodiment data sets of the present invention.
Fig. 7 is that the present invention carries out schematic flow sheet
Embodiment
As shown in Figure 7, the present invention comprises following steps: step 1, the two dimensional image under one group of different visual angles of input is carried out to the camera parameter demarcation, and obtain the projection matrix of the corresponding two dimensional image in each visual angle; Step 2, set up a spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding; Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S
initial; Step 4, the two dimensional image i that new visual angle is corresponding in location on spherical model
newand spherical model is upgraded; Step 5, according to two dimensional image i
newposition on spherical model, from bin cloud S
initialin choose a bin subset P
update; Step 6, relatively bin subset P
updatemiddle partial 3 d surface bin density and bin cloud S
initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P
update; Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P
updateupgraded, thereby realized the increment three-dimensional reconstruction.
Below in conjunction with accompanying drawing, the present invention is done to detailed introduction.
In step 1, adopt sparse bundle to adjust method two dimensional image carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle,
Wherein projection matrix P is the real matrix of 3*4.Wherein sparse bundle adjustment method refers to Manolis I.A.Lourakis, Antonis A.Argyros. " SBA:A software package for generic sparse bundle adjustment " .TOMS, vol.36, no.1, pp.1-30,2009.
In step 2, set up spherical model and be: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p
31p
32p
33)
t, p
31, p
32, p
33correspond respectively to the first three columns element of its corresponding projection matrix P the third line; The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v
1, v
2, v
3), find the point nearest with described reference point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle; The quantity of the three-dimensional point of sampling is generally 1/3rd of data set size; Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by the Delaunay triangulation carried out trigonometric ratio.The spherical model that Fig. 1 is trigonometric ratio, from figure to seeing that a three-dimensional point represents a two dimensional image; Two dimensional image corresponding to crucial visual angle forms triangle by Triangulation Algorithm and covers sphere.
In step 3, the three-dimensional rebuilding method of employing based on the bin model obtains image reconstruction corresponding to crucial visual angle to step 2 and obtains bin cloud S
initial, method for reconstructing is referring to Y.Furukawa and J.Ponce. " Accurate, dense, and robust multiview stereopsis " .PAMI, vol.32, no.8, pp.1362-1376,2010.In the bin model, what three-dimensional surface was approximate covers with local tangential plane, and as shown in Figure 2, each local tangential plane is a bin, in model, by regular three-dimensional rectangle, means.To one of them bin p, it includes three attribute: c (p), n (p), R (p).C (p) is bin p centre coordinate; N (p) is normal vector, and direction is pointed to observation point, for weighing surface local curvature; R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the visual angle image collection that a bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.The direction on two limits of three-dimensional rectangle accomplishes that wherein the direction on a limit is as far as possible parallel with the x direction of principal axis in camera coordinate system as far as possible, and the rectangle topology size is that its projection in picture is no more than the u*u pixel of pressing the axle arrangement, is made as in the present invention 5*5.
Step 4 is determined the two dimensional image i corresponding to new visual angle of input by following formula
newthe triangle T of correspondence on spherical model:
the summit that wherein v is a triangle T in the middle of spherical model,
two dimensional image i
newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features.T is one and two dimensional image i in fact
newthe triangle that has maximum coupling amount; Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i
newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.
Can see new visual angle i in Fig. 1
newinterconnect with leg-of-mutton three summits, obtained the spherical model after a renewal.
In step 5, from initialization bin S set
initialin choose and two dimensional image i
newthe bin subset that correlativity is the highest.This correlativity is embodied in: two dimensional image i
newin can see this bin and bin section and two dimensional image i
newplane of delineation angle less.Bin subset P
updateaccording to following formula, obtain:
In step 6 of the present invention, to bin subset P
updateset is expanded, and spread step takes full advantage of two dimensional image i
newpixel Information and the geological information of three-dimensional model, and can allow the three-dimensional surface bin distribute as far as possible evenly, accomplish to take full advantage of pictorial information and go out some new bins in the three-dimensional surface area extension of low resolution.Spread step is as follows: to bin cloud S
initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p
pits local density, the neighbours' bin quantity D of replacing of equal value
paccount form as follows:
N(p)={p′|p′∈S
initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ},
D
p=|N(p)|,
The depth distance that wherein ρ corresponds to the number of pixels β in two dimensional image R (p) by calculating bin p and bin p ′ center automatically determines, is that its depth distance that is 2 pixels during bin p and bin p ′ center correspond to two dimensional image R (p) is multiplied by 2 in the present invention; By to all at bin cloud S
initialin the D of local density of bin
pask arithmetic mean to calculate bin cloud S
initialthree-dimensional surface bin density mean value D
g; To bin subset P
updatein arbitrary bin p, if the D of local density
pbe less than 1/2nd three-dimensional surface bin density mean value D
g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin p.As can be seen from Figure 3, p
0and p
1between a random newly-generated new bin p on the linear space of line
new, its coordinate and normal vector are according to p
newposition in line segment is to p
0and p
1consistent attribute weight on average obtain.Wherein synthetic minority oversampler method refers to Nitesh V.Chawla, Kevin W.Bowyer, Lawrence O.Hall and W.Philip Kegelmeyer. " SMOTE:Synthetic Minority Over-sampling Technique " .JAIR, vol.16, pp.321-357,2002..
In step 7 of the present invention, by following formula, replacement problem is carried out to Bayes Modeling:
Wherein S is real three-dimensional scenic, i
newthe two dimensional image in step 4, p (S|i
new) be that three-dimensional model S is at two dimensional image i
newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, we are on its dimensionality reduction to one bin subset, i.e. bin subset P in step 5
update.The level and smooth priori that Probability p (S) is model; p(i
new| S) be two dimensional image i
newlikelihood probability, for weighing three-dimensional model S and two dimensional image i
newthe likelihood degree, be expressed as:
p(i
new|S)∝exp(-ηE
p),
E wherein
pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, the variation of the variation of the normal vector of arbitrary bin and bin topology information all can have it indirectly to measure reflection on image.Described accuracy by arbitrary bin p at two dimensional image i
newand the correlativity h between the projection in two dimensional image i (p, i
new, i) determine, wherein i is the two dimensional image in image collection V (p), and η is control variable, and the h calculation procedure is as follows: cover the grid of a u*u on bin p, in the present invention, sizing grid is 5*5; By in the bilinear interpolation computing grid, each is put at two dimensional image i
newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images.As can be seen here, during entirely accurate, the h between arbitrary two pictures is 0, last E
pbe also 0 to reach minimum, but because the recovery of measuring error and bin p is a Reverse Problem, so E
palways be greater than 0, work as E
pwhen smaller, illustrate that, under existing measurement environment, bin p is more accurate.η is control variable, in the present invention, is 0.5.
Priori P (S) weighs the level and smooth degree on 3D surface, has excavated to a certain extent the geological information of three-dimensional surface.It passes through energy function E priori
1with energy function E
2be expressed as:
p(S)∝exp(-{λE
1+ζE
2}),
E wherein
1for weighing the flatness of three-dimensional surface, the curvature with the bin part in the present invention changes to weigh local slickness.E
1can not accurately weigh its surface smoothness, in the present Fig. 4 of its defect body, be curvature although bin p and neighbours on every side have level and smooth normal vector, because it has broken away from original fit surface, so it is a divorced bin.So propose E in the present invention
2weigh the divorced degree of bin p at whole three-dimensional surface, the bin that coordinate is dropped on to the three-dimensional surface outside has a threshold values to control, and can play like this purpose of filtering singular point, and accomplishes truly level and smooth.Energy function E
1with energy function E
2computing method are:
d(p,v)=|n(p)·(c(v)-c(p))|,
Wherein f (p, n) be bin p and bin n normal vector between Euclidean distance, mark and can see from Fig. 5, d (p, v) is bin p and the absolute value distance of bin v on normal vector n (p); λ, ζ is two control parameters, is respectively 0.3,0.2 in the present invention.
Maximize posteriority p (S|i
new) obtain the maximum likelihood three-dimensional model, for P
updatemiddle bin parameter is upgraded; Described parameter is at probability space P
updatein three-dimensional coordinate and the normal vector of each bin.Finally obtain the solution of a convergence, be equivalent to:
c(p),n(p)←argmax(exp(-{λE
1+ζE
2+ηE
p}),p∈P
update
Get negative logarithm operation, that is:
c(p),n(p)←argmin(λE
1+ζE
2+ηE
p),p∈P
update
Like this by the set P of opposite unit
updatethe bin cloud of renewal after finally being upgraded.
In the present invention, adopt conjugate gradient to be optimized problem solving.In order further to reduce dimension, in optimization problem, bin p centre coordinate c (p) only moves on the line of initial point and projection centre, like this three-dimensional coordinate space has been reduced to one dimension, has reduced degree of freedom in the present invention; Normal vector n (p) replaces with two Eulerian angle are approximate simultaneously.So each bin p only uses the three degree of freedom modeling, has increased optimization speed.
When inputting new picture, algorithm turns back to again step 4 and carries out.
Embodiment
As shown in Figure 7, the step of the present embodiment comprises: the two dimensional image to all inputs carries out the camera parameter demarcation by sparse bundle adjustment method, then calculate the optical axis vector of two dimensional image corresponding to each visual angle, and it is carried out to normalization obtain three-dimensional point, set up according to this spherical model, wherein the three-dimensional point on spherical model represents a two dimensional image that visual angle is corresponding; Adopt random algorithm to select at random two dimensional image corresponding to crucial visual angle, the selection repeated is disregarded; Adopt triangle gridding subdivision algorithm to carry out trigonometric ratio to them three-dimensional point corresponding to two dimensional image of selecting to obtain; The two dimensional image corresponding to crucial visual angle carries out the three-dimensional reconstruction based on bin.Initial work completes, and then enters the increment link.Read in two dimensional image corresponding to new visual angle, with yardstick invariant features Transformation Matching algorithm and spherical model, find a triangle, and select according to this bin subset P
update; Spherical model is upgraded; Then with synthetic minority oversampler method expansion P
update, the bin made new advances in sparse area extension; Finally by solving an optimization problem to bin subset P under Bayesian frame
updateupgrade the renewal that reaches the general three model.Along with constantly reading in of new images, the step in the middle of the increment link is constantly carried out.
Fig. 6 a1~Fig. 6 c5 is the experimental result (owing to being picture, can only adopt the performance of gray scale form) listed under three data sets.Fig. 6 a1 is the dinosaur image data set, Fig. 6 a2 uses two dimensional image corresponding to crucial visual angle obtained from dinosaur image data set sampling to carry out the bin cloud design sketch that the dinosaur 3-dimensional reconstruction based on the bin model obtains, and Fig. 6 a3~Fig. 6 a5 is for constantly reading in the bin cloud design sketch of incremental update after two-dimentional dinosaur image corresponding to new visual angle.Fig. 6 b1 is the skull image data set, Fig. 6 b2 uses two dimensional image corresponding to crucial visual angle obtained from skull image data set sampling to carry out the bin cloud design sketch that the skull three-dimensional reconstruction based on the bin model obtains, and Fig. 6 b3~Fig. 6 b5 is for constantly reading in the bin cloud design sketch of incremental update after two-dimentional skull image corresponding to new visual angle.Fig. 6 c1 is the temple image data set, Fig. 6 c2 uses two dimensional image corresponding to crucial visual angle obtained from temple image data set sampling to carry out the bin cloud design sketch that the temple three-dimensional reconstruction based on the bin model obtains, and Fig. 6 c3~Fig. 6 c5 is for constantly reading in the bin cloud design sketch of incremental update after temple two dimensional image corresponding to new visual angle.Can find out, bin milks up, and more and more accurate.In these three embodiment, the present invention has obtained a reasonable result.
The invention provides a kind of increment three-dimensional rebuilding method based on Bayes and bin model; method and the approach of this technical scheme of specific implementation are a lot; the above is only the preferred embodiment of the present invention; should be understood that; for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.In the present embodiment not clear and definite each ingredient all available prior art realized.
Claims (5)
1. the increment three-dimensional rebuilding method based on Bayes and bin model, is characterized in that, comprises the following steps:
Step 1, carry out the camera parameter demarcation to the two dimensional image under one group of different visual angles of input, obtains the projection matrix of the corresponding two dimensional image in each visual angle;
Step 2, set up a spherical model to all two dimensional images, one group of two dimensional image corresponding to crucial visual angle of sampling, and the trigonometric ratio three-dimensional point corresponding to two dimensional image of sampling; Described three-dimensional point i.e. the corresponding point of two dimensional image on spherical model that visual angle is corresponding;
Step 3, the three-dimensional reconstruction that corresponding two dimensional image carries out based on bin to described crucial visual angle obtains bin cloud S
initial;
Step 4, the two dimensional image i that new visual angle is corresponding in location on spherical model
newand spherical model is upgraded;
Step 5, according to two dimensional image i
newposition on spherical model, from bin cloud S
initialin choose a bin subset P
update;
Step 6, relatively bin subset P
updatemiddle partial 3 d surface bin density and bin cloud S
initialthree-dimensional surface bin density mean value, use synthetic a small amount of samples method expansion bin subset P
update;
Step 7, carry out modeling by Bayes, according to maximum a posteriori to bin subset P
updateupgraded, thereby realized the increment three-dimensional reconstruction;
In step 6 to bin subset P
updatespread step as follows:
To bin cloud S
initialin any one bin p calculate local density, by the neighbours' bin quantity D in neighbours' bin set N (p) of bin p
pits local density, the neighbours' bin quantity D of replacing of equal value
paccount form as follows:
N(p)={p′|p′∈S
initial,|(c(p)-c(p'))·n(p)|+|(c(p)-c(p′))·n(p′)|<ρ},
D
p=|N(p)|,
The 3 dimension Geometric center coordinates that c (p) is bin p, the normal vector that n (p) is bin p, wherein the direction of normal vector is pointed to viewpoint direction, and ρ is threshold values;
By to all at bin cloud S
initialin the D of local density of bin
pask arithmetic mean to calculate bin cloud S
initialthree-dimensional surface bin density mean value D
g;
To bin subset P
updatein arbitrary bin p, if the D of local density
pbe less than 1/2nd three-dimensional surface bin density mean value D
g, adopt synthetic minority oversampler method to expand the bin k that makes new advances between bin p and neighbours' bin;
In step 7 by following formula to bin subset P
updaterenewal realizes Bayes's increment three-dimensional modeling:
Wherein S is real three-dimensional model, i
newthe two dimensional image in step 4, p (S|i
new) be that three-dimensional model S is at two dimensional image i
newunder posterior probability, Z is normaliztion constant, the probability space that Ω is three-dimensional model, by probability space Ω dimensionality reduction to the bin subset P in step 5
update, the level and smooth prior probability that p (S) is three-dimensional model S; p(i
new| S) be two dimensional image i
newlikelihood probability, for weighing three-dimensional model S and two dimensional image i
newthe likelihood degree, be expressed as:
p(i
new|S)∝exp(-ηE
p),
E wherein
pfor energy function, for weighing the accuracy of the arbitrary bin p of three-dimensional model S in its visible two dimensional image, described accuracy by arbitrary bin p at two dimensional image i
newand the correlativity between the projection in two dimensional image i
H (p, i
new, i) determine, wherein i is the two dimensional image in image collection V (p), η is control variable, h (p, i
new, i) calculation procedure is as follows:
Cover the grid of a u*u on bin p; By in the bilinear interpolation computing grid, each is put at two dimensional image i
newwith the projection in two dimensional image i; By the 1 normalization positive correlation amount that deducts grid projection in two width two dimensional images;
Priori p (S) is by energy function E
1with energy function E
2be expressed as:
p(S)∝exp(-{λE
1+ζE
2}),
Energy function E wherein
1for weighing the flatness of three-dimensional surface, energy function E
2supplement and weigh the divorced degree of bin at whole three-dimensional surface, energy function E
1with energy function E
2computing method are:
d(p,v)=|n(p)·(c(v)-c(p))|,
N (p) wherein, the normal vector of the corresponding bin p of n (v) and bin v, c (p), the Geometric center coordinates of the corresponding bin p of c (v) and bin v, f (p, v) be bin p and bin v normal vector between Euclidean distance, d (p, v) be bin p and the bin v absolute value distance on normal vector n (p), λ, ζ is two control parameters;
Maximize posterior probability p (S|i
new), obtain the maximum likelihood surface model, upgrade P
updateobtain new three-dimensional model, that is: c (p), n (p) ← argmax (exp ({ λ E
1+ ζ E
2+ η E
p), p ∈ P
update, get negative logarithm operation, that is: c (p), n (p) ← argmin (λ E
1+ ζ E
2+ η E
p), p ∈ P
update, by the set P of opposite unit
updatethe bin cloud of renewal after finally being upgraded.
2. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 1, it is characterized in that, in step 1, adopt sparse bundle to adjust method two dimensional image is carried out to the camera parameter demarcation, obtain projection matrix P corresponding to two dimensional image under each visual angle
Wherein projection matrix P is the real matrix of 3*4.
3. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 2, it is characterized in that, in step 2, setting up spherical model is: the corresponding two dimensional image for any one visual angle, the normalized vector that the coordinate that makes its corresponding point on spherical model is optical axis vector N, wherein optical axis vector N=(p
31p
32p
33)
t, p
31, p
32, p
33correspond respectively to the first three columns element of its corresponding projection matrix P the third line;
The method of two dimensional image corresponding to one group of crucial visual angle of sampling is: in three values of interval [0,1] stochastic sampling as reference point (v
1, v
2, v
3), find the point nearest with described reference point point Euclidean distance as the three-dimensional point that once sampling obtains on spherical model, X-Y scheme corresponding to described three-dimensional point becomes two dimensional image corresponding to crucial visual angle;
Three-dimensional point corresponding to two dimensional image crucial visual angle on spherical model is corresponding by triangulation carried out trigonometric ratio.
4. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 3, is characterized in that, in step 4, determines the two dimensional image i corresponding to new visual angle of input by following formula
newthe triangle T of correspondence on spherical model:
The summit that wherein v is a triangle T in the middle of spherical model,
two dimensional image i
newthe two dimensional image corresponding with vertex v obtains the match point set by the conversion of yardstick invariant features;
Using after the center-of-mass coordinate normalization of triangle T as two dimensional image i
newthree-dimensional point coordinate on spherical model, be designated as point (x, y, z), point (x, y, z) is connected in twos to the spherical model after being upgraded with three summits of triangle T.
5. a kind of increment three-dimensional rebuilding method based on Bayes and bin model according to claim 4, is characterized in that, the bin subset P in step 5
updateaccording to following formula, obtain:
Wherein R (p) is the two dimensional image that bin p is corresponding, it has following attribute: two dimensional image R (p) is a two dimensional image in image collection V (p), V (p) is the two dimensional image set that bin p determines, every two dimensional image in described set can unobstructedly show the projection of bin p fully; The plane of delineation of the correspondence of two dimensional image R (p) is parallel with the section of bin p.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110411429.6A CN102496184B (en) | 2011-12-12 | 2011-12-12 | Increment three-dimensional reconstruction method based on bayes and facial model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201110411429.6A CN102496184B (en) | 2011-12-12 | 2011-12-12 | Increment three-dimensional reconstruction method based on bayes and facial model |
Publications (2)
Publication Number | Publication Date |
---|---|
CN102496184A CN102496184A (en) | 2012-06-13 |
CN102496184B true CN102496184B (en) | 2013-07-31 |
Family
ID=46188006
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201110411429.6A Expired - Fee Related CN102496184B (en) | 2011-12-12 | 2011-12-12 | Increment three-dimensional reconstruction method based on bayes and facial model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN102496184B (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103198521A (en) * | 2013-04-16 | 2013-07-10 | 大连理工大学 | Caput femoris surface three-dimensional model reconstruction method for individualized design |
CN104657985B (en) * | 2015-02-02 | 2018-07-03 | 燕山大学 | Static vision target occlusion bypassing method based on depth image block information |
CN104807465B (en) * | 2015-04-27 | 2018-03-13 | 安徽工程大学 | Robot synchronously positions and map creating method and device |
CN110021041B (en) * | 2019-03-01 | 2021-02-12 | 浙江大学 | Unmanned scene incremental gridding structure reconstruction method based on binocular camera |
CN110599593B (en) * | 2019-09-12 | 2021-03-23 | 北京三快在线科技有限公司 | Data synthesis method, device, equipment and storage medium |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2457839C (en) * | 2001-08-14 | 2010-04-27 | Pulse Entertainment, Inc. | Automatic 3d modeling system and method |
CN1271572C (en) * | 2004-08-05 | 2006-08-23 | 上海交通大学 | Three dimension re-set-up method for two dimension image sequence |
US8670606B2 (en) * | 2010-01-18 | 2014-03-11 | Disney Enterprises, Inc. | System and method for calculating an optimization for a facial reconstruction based on photometric and surface consistency |
-
2011
- 2011-12-12 CN CN201110411429.6A patent/CN102496184B/en not_active Expired - Fee Related
Also Published As
Publication number | Publication date |
---|---|
CN102496184A (en) | 2012-06-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Chen et al. | Model-based 3d hand reconstruction via self-supervised learning | |
CN109859296B (en) | Training method of SMPL parameter prediction model, server and storage medium | |
US10360718B2 (en) | Method and apparatus for constructing three dimensional model of object | |
Newcombe et al. | Kinectfusion: Real-time dense surface mapping and tracking | |
US20180211399A1 (en) | Modeling method and apparatus using three-dimensional (3d) point cloud | |
KR20200087757A (en) | Full convolutional point of interest detection and description through homographic adaptation | |
WO2015139574A1 (en) | Static object reconstruction method and system | |
US11568601B2 (en) | Real-time hand modeling and tracking using convolution models | |
CN102496184B (en) | Increment three-dimensional reconstruction method based on bayes and facial model | |
WO2021063271A1 (en) | Human body model reconstruction method and reconstruction system, and storage medium | |
US11922562B2 (en) | Methods and systems for rendering view-dependent images using 2D images | |
Chun et al. | Learnable human mesh triangulation for 3d human pose and shape estimation | |
Tu et al. | Consistent 3d hand reconstruction in video via self-supervised learning | |
CN114429555A (en) | Image density matching method, system, equipment and storage medium from coarse to fine | |
CN108537887A (en) | Sketch based on 3D printing and model library 3-D view matching process | |
CN114723884A (en) | Three-dimensional face reconstruction method and device, computer equipment and storage medium | |
CN103544492A (en) | Method and device for identifying targets on basis of geometric features of three-dimensional curved surfaces of depth images | |
CN115272608A (en) | Human hand reconstruction method and equipment | |
CN110030979A (en) | A kind of space non-cooperative target relative pose measurement method based on sequence image | |
Esteban et al. | Fit3d toolbox: multiple view geometry and 3d reconstruction for matlab | |
Correia et al. | 3D reconstruction of human bodies from single-view and multi-view images: A systematic review | |
Zhao et al. | Three-dimensional face reconstruction of static images and computer standardization issues | |
Deng et al. | Cerberus: A multi-headed derenderer | |
CN115082636B (en) | Single image three-dimensional reconstruction method and device based on mixed Gaussian network | |
CN115731345A (en) | Human body three-dimensional reconstruction method based on binocular vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20130731 Termination date: 20151212 |
|
EXPY | Termination of patent right or utility model |