CN101038678A - Smooth symmetrical surface rebuilding method based on single image - Google Patents

Abstract

A three-dimensional entity model restructuring method of the invention comprises following steps: inputting an original view of a slippy symmetrical curve surface, and mirror-reversing the original view to obtain a mirror reversing view; determining matched characteristic point pairs in the original view and the mirror reversing view; calculating a camera inner parameter matrix used by an original image; calculating an essence matrix according to the characteristic points between the original image and a mirror reversing image obtained by the inner parameter matrix and the matched characteristic point pairs, decomposing the obtained essence matrix into a rotary matrix and a translation matrix; calculating a three-dimensional coordinate of a corresponding point based on a coordinate of a two-dimensional point of the original image to generate a plane model of an object; obtaining a final slippy curve surface model by minimizing a slippy target function; extracting texture of the original image and mapping the texture on the generated slippy curve surface to obtain the three-dimensional entity model. The method of the invention can be used for restructuring the three-dimensional entity model of stacked objects and curve surfaces from a network, a digital camera or a scanner.

Description

Smooth symmetroid method for reconstructing based on single image

Technical field

The invention belongs to computer application field, particularly based on the three-dimensional entity model reconstructing system of image.Method of rebuilding based on the smooth symmetroid of single image of the present invention can be applicable to automatic network, the reconstruction of the digital camera three-dimensional model that piles object and curved surface that take or scanner, these models can be used for fields such as computer vision and virtual reality provides various three-dimensional entity models with complicated symmetrical smooth surface true to nature.

Background technology

In the past few decades, single-view is rebuild (Single View Reconstruction:SVR) technology and has been caused computer vision and field of Computer Graphics researchist's very big interest.Utilize the single image that only contains two-dimensional signal to rebuild, obviously can't find the solution the spatial point position according to principle of triangulation, geometrical constraint that therefore can only be by scene itself (as conllinear, coplane, parallel, quadrature and symmetry etc.) or the method realization three-dimensional reconstruction by user interactions input depth information.

Owing to the quantity of information that single image is contained is less, so the single-view reconstruction is a difficult point problem in the three-dimensional reconstruction.For buildings isostructuralism scene, the less and geometric configuration description difficulty of the geometrical constraint of curved surface, so the single-view curved surface modeling is again the difficult point during single-view is rebuild.

Present newest research results ginseng is shown in Table 1, proposition all be the three-dimensional rebuilding method and the device of multiple image basically.Secondly, in the newest fruits of having delivered about the method for reconstructing of single image, the single image method for reconstructing is limited to the parametric surface method for expressing that adopts a kind of standard, mostly with the silhouettes line as main constraints, obtained comparatively real three-dimensional model.But because supposition whole profile line is all as seen, and the image of under common viewpoint position, taking, the user is difficult to estimate depth value accurately, therefore rebuilds and only limits to the simple object of topological structure; Secondly, these methods have adopted the rectangular projection model, and be not suitable for the obviously reconstruction (M.Prasad of perspective distortion image, A.Zisserman and A.Fitzgibbon.Single ViewReconstruction of Curved Surfaces[C] .Proceedings of the IEEE Conference onComputer Vision and Parrern Recognition.New York, 2006:1345-1354.).

In table 1 prior art based on the method for reconstructing patent of image

Sequence number	Application number	Patent name
				1	200410053489.5	The two-dimensional image sequence three-dimensional rebuilding method
2	200510042734.7	The rapid progressive direct volume drawing three-dimensional rebuilding method of CT image
			3	200510092287.6	The three-dimensional rebuilding method of image and system
4	200610050797.1	Small-sized scene three-dimensional rebuilding method and device thereof based on double camera

Summary of the invention

The invention provides a kind of symmetrical smooth curve reestablishing method, comprise the steps: based on single image

(1), and it is carried out mirror face turning obtains horizontal mirror face turning view with the input of smooth symmetroid original view;

(2) determine that the match point in original view and the horizontal mirror face turning view is right;

(3) the intrinsic parameter matrix of the used camera of calculating original image;

(4) right according at least 8 Feature Points Matching between intrinsic parameter matrix and original image and the horizontal mirror reversing image, solve essential matrix and it is decomposed into rotation matrix and translation matrix;

(5) according to the coordinate of original image two-dimensional points, try to achieve the three-dimensional coordinate of corresponding point, generate the areal model of object;

(6) obtain final smooth surface model by minimizing smooth objective function;

(7) extract the original image texture, and be mapped on the smooth surface of generation, obtain three-dimensional entity model.

The invention provides a kind of symmetrical smooth curve reestablishing method based on single image, the original image that is used to rebuild is the image of single width symmetric objects.

The smooth curve reestablishing method of symmetry provided by the invention, the formation of smooth surface model is wherein finished as follows, and promptly linear restriction is set up in the expression of the smooth surface of curved surface, and the optimization of linear restriction lower surface camber objective function.

The smooth curve reestablishing method of symmetry provided by the invention adopts center of gravity texture extracting method to extract the original image texture, and is mapped on the smooth surface of generation, obtains three-dimensional surface model true to nature.

The smooth curve reestablishing method of symmetry provided by the invention, the linear restriction kind of being set up are at least a of position constraint, perspective constraint, topological constraints, plane restriction or line constraint.

Description of drawings

Fig. 1 symmetric position is taken the symmetric objects synoptic diagram;

Fig. 2 is based on single image symmetroid method for reconstructing schematic diagram;

Relation between Fig. 3 focal length and the image size;

Fig. 4 principle of triangulation;

Fig. 5 two-dimensional map schematic three dimensional views;

Fig. 6 extracts schematic diagram based on the weighting texture texture of barycentric coordinates;

Fig. 7 single-view symmetroid is rebuild flow process.

Embodiment

With reference to Fig. 1, to theory: under the perspective imaging model after the simplification, the image of the minute surface symmetric objects behind the flip horizontal just is equivalent to the image of taking from the another one symmetric position, proves.This theory is the theoretical foundation of subsequent step.Detailed process is as follows: adopt traditional pinhole camera model, for any 1 M=[X under the world coordinate system, Y, Z] ^TIn image, obtain m=[u, v through after the perspective projection] ^TCan be expressed as:

λ[u，v，1] ^T＝K[R|t][X，Y，Z，1] ^T (1-1)

Wherein λ is a non-zero zoom factor, (R is rotation and the translation of world coordinate system with respect to camera coordinate system t), and K is the camera intrinsic parameter matrix,

$K=\left[\begin{array}{ccc}f& 0& I_w/2\\ 0& f& I_h/2\\ 0& 0& 1\end{array}\right],---(1-2)$

I wherein _wAnd I _hBe respectively the width and the height of image.

Situation as shown in Figure 1, C and C ' they are respectively the photocentres of first and second camera, OXYZ is a world coordinate system.Behind the image of a given width of cloth symmetric objects, first camera towards just having determined with the position.Determine world coordinate system according to following two principles: (1) YOZ overlaps with the object symmetrical plane; The photocentre of (2) first cameras is on the OX axle.The world coordinate system here is equivalent to first coordinate system in the coordinate transform, and camera coordinates system is equivalent to second coordinate system.If with world coordinate system around OX axle anglec of rotation α, then around OY axle rotation-β, at last along former OX axle translation-t _XThe back overlaps with first camera coordinates system, and therefore the rotation and the translation of first camera can be expressed as follows:

$R=\left[\begin{array}{ccc}\cos \mathrm{\β}& -\sin \mathrm{\α}\sin \mathrm{\β}& -\cos \mathrm{\α}\sin \mathrm{\β}\\ 0& \cos \mathrm{\α}& -\sin \mathrm{\α}\\ \sin \mathrm{\β}& \sin \mathrm{\α}\cos \mathrm{\β}& \cos \mathrm{\α}\cos \mathrm{\β}\end{array}\right]---(1-3)$

t＝-R[-t _X 0 0] ^T (1-4)

In like manner, the coordinate system of second camera can be regarded as by around the OX rotation alpha, rotates β around OY, at last along former OX translation t _XObtain, therefore the rotation and the translation of second camera can be expressed as follows:

$R^{\′}=\left[\begin{array}{ccc}\cos \mathrm{\β}& \sin \mathrm{\α}\sin \mathrm{\β}& \cos \mathrm{\α}\sin \mathrm{\β}\\ 0& \cos \mathrm{\α}& -\sin \mathrm{\α}\\ -\sin \mathrm{\β}& \sin \mathrm{\α}\cos \mathrm{\β}& \cos \mathrm{\α}\cos \mathrm{\β}\end{array}\right]---(1-5)$

T '=-R ' [t _X0 0] ^T(1-6) given a pair of space symmetric points M ₁(X, Y, Z) and M ₂(Z), their subpoints in first and second width of cloth images are expressed as m respectively for X, Y ₁(u ₁, v ₁), m ₂(u ₂, v ₂), m ' ₁(u ' ₁, v ' ₁), m ' ₂(u ' ₂, v ' ₂), according to formula (1-1), following projection relation is arranged:

$\left\{\begin{array}{c}{\mathrm{\λ}}_1{[u_1,v_1,1]}^T=K\left[R\right|t]{[-X,Y,Z,1]}^T\\ {\mathrm{\λ}}_2{[u_2,v_2,1]}^T=K\left[R\right|t]{[X,Y,Z,1]}^T\\ {{\mathrm{\λ}}^{\′}}_1{[{u^{\′}}_1,{v^{\′}}_1,1]}^T=K\left[R^{\′}\right|t^{\′}]{[-X,Y,Z,1]}^T\\ {{\mathrm{\λ}}^{\′}}_2{[{u^{\′}}_2,{v^{\′}}_2,1]}^T=K\left[R^{\′}\right|t^{\′}]{[X,Y,Z,1]}^T\end{array}\right.---(1-7)$

Can obtain following according to formula (1-7) about m ' ₁And m ₂, m ' ₂And m ₁Between relation:

$\left\{\begin{array}{c}({u^{\&prime;}}_1,{v^{\&prime;}}_1)=(I_w-u_2,v_2)\\ ({u^{\&prime;}}_2,{v^{\&prime;}}_2)=(I_w-{\mathrm{<figure-callout\; id="1"\; label="u"\; filenames="A20071009850500183.png"\; state="\{\{state\}\}">u</figure-callout>}}_1,v_1)\end{array}\right.---(1-8)$

From formula (1-8) as can be seen, the image point locations in second width of cloth image is equivalent to the image coordinate of symmetric space point corresponding in first width of cloth image is carried out flip horizontal.Can reach a conclusion thus: under the perspective imaging model after the simplification, the image of the minute surface symmetric objects behind the flip horizontal just is equivalent to the image of taking from the another one symmetric position.

With reference to Fig. 2, in the process of reconstruction based on single width symmetric graph picture, be input picture and flip horizontal image thereof in 201, the object in this image is for carrying out the image of the object of three-dimensional reconstruction.Pass through the coupling of at least 8 unique points between two width of cloth images in 202 realizations 201 then.Demarcate by 203 pairs of images, obtain the intrinsic parameter matrix of camera.202 and 203 output can be found the solution essential matrix by 204.204 essential matrixs that obtain can be decomposed into rotation matrix and translation matrix by 205.By 206, utilize 205 result, can carry out the reconstruction of three-dimensional point and the generation of areal model.Obtain the smooth surface model by 207 then.In 208, adopt texture as shown in Figure 5 to extract principle, realize that texture extracts, and is mapped on the surface model at last based on center of gravity.The final three-dimensional model true to nature of 209 outputs, its output format is Openflight form (* .flt).

Model with reference to the camera calibration among Fig. 3 the present invention adopts simple perspective imaging model.The intrinsic parameter of camera has only unknown parameter of focal length to determine.The invention provides two kinds of methods,, then can attempt single-view camera based on scene corner structure constraint and parallel structural constraint from calibration algorithm if contain the sets of parallel of mutually orthogonal in the photo.If the scene in the photo can not provide geometrical constraint to realize camera from demarcating, then utilize the relation (as shown in Figure 3) between focal length and the image size to carry out the focal length estimation, concrete formula is 1-9.

$f=\frac{I_h}{2\tan (\mathrm{fov}/2)}---(1-9)$

Known camera intrinsic parameter, matching characteristic point between two width of cloth images utilizes traditional based drive reconstruction technique then, can at first find the solution the camera motion that utilizes R and t to describe, and then according to principle of triangulation, recovers the position of spatial point.Essential matrix is found the solution and is adopted 8 algorithms.Its argument is: be provided with n three-dimensional point M _i=(x _i, y _i, z _i) ^T, these spot projections are in first width of cloth image, and its picture point is m _i=(u _i, v _i) ^TProject in second width of cloth image, its picture point is m ' _i=(u ' _i, v ' _i) ^THere suppose picture point m _iAnd m ' _iUnder normalized coordinate system, measure, so satisfy following relation between them:

${{\tilde{m}}_i}^{\&prime;T}E{\tilde{m}}_i=0---(1-10)$

Wherein ${{\tilde{m}}_i}^{\&prime;}={(u_i,v_i,1)}^T,{{\tilde{m}}_i}^{\&prime;}={({u_i}^{\&prime;},{v_i}^{\&prime;},1)}^T,$ E=t * R=[t] _*R is essential matrix (essentialmatrix), and t and R are respectively translation vector and rotation matrix.Created symbol [t] _*Express the antisymmetric matrix of tri-vector t definition, if t=(t _x, t _y, t _z) ^T, so

${\left[t\right]}_{\&times;}=\left[\begin{array}{ccc}0& -t_x& t_y\\ t_z& 0& -t_x\\ {-t}_z& t_x& 0\end{array}\right]---(1-11)$

By this symbol, t * x=[t is arranged] _*X,  x.

Prove that easily essential matrix is for using the fundamental matrix of normalized image coordinate time, therefore finding the solution essential matrix can use 8 the algorithm derivation algorithms the same with basis matrix.

The decomposition of essential matrix is about to the calculating that essential matrix is decomposed into rotation matrix R and translation matrix t.The hypothetical world coordinate system overlaps with the camera coordinates system of first camera, makes t and R be respectively translation and the rotation of second camera with respect to first camera.

Essential matrix E is carried out svd:

E＝USV ^T (1-12)

U wherein, V is respectively 3 * 3 orthogonal matrix, $S=\left[\begin{array}{ccc}{\mathrm{\&sigma;}}_1& 0& 0\\ 0& {\mathrm{\&sigma;}}_2& 0\\ 0& 0& 0\end{array}\right],{\mathrm{\&sigma;}}_1\&ap;{\mathrm{\&sigma;}}_2>0.$

U wherein ₃Be last row of U, $W=\left[\begin{array}{ccc}0& -1& 0\\ 1& 0& 0\\ 0& 0& 1\end{array}\right].$

From formula (1-13) as can be known, t and R there are 4 kinds of combinations, in order to determine correct separating, can be by selecting a pair of match point, utilize this 4 kinds of locus that this point is calculated in combination respectively respectively, and determine this spatial point at two magazine depth values, be chosen in depth value in two cameras and all be on the occasion of t and R be correct separating.The foundation of areal model need recover spatial point by triangulation, calculate t and R after, then can determine the projection matrix of first width of cloth and second width of cloth image respectively,

$\left\{\begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="A20071009850500183.png"\; state="\{\{state\}\}">P</figure-callout>}}_1=K\left[I\right|0]\\ P_2=K\left[R\right|t]\end{array}\right.---(1-14)$

Wherein K is a camera intrinsic parameter matrix, and I is 3 * 3 unit matrix, shows that world coordinate system overlaps with the camera coordinates system of first camera.

Suppose that spatial point M is at C ₁Subpoint on the video camera is m ₁, at C ₂Subpoint on the video camera is m ₂By the pin-hole imaging principle as can be known, spatial point M must pass through first camera photocentre C ₁With m ₁Straight line on, also by second photocentre and m ₂On the straight line of point, therefore the intersection point of two straight lines is exactly the position of spatial point M, and this process is called as triangulation visually.

But owing to the existence of experimental error and noise, two space lines often can not meet at a bit in the operating process of reality.In this case, can utilize the principle of least square to find the solution the position of spatial point.Make m ₁=(u ₁, v ₁, 1) ^T, m ₂=(u ₂, v ₂, 1) ^T, M=(X, Y, Z, 1) ^T, the projection matrix of first camera and second camera is:

${\mathrm{<figure-callout\; id="1"\; label="P"\; filenames="A20071009850500183.png"\; state="\{\{state\}\}">P</figure-callout>}}_1=\left[\begin{array}{cccc}{p}_{11}^1& p_{12}^1& p_{13}^1& p_{14}^1\\ p_{21}^1& p_{22}^1& p_{23}^1& p_{24}^1\\ p_{31}^1& p_{32}^1& p_{33}^1& p_{34}^1\end{array}\right]---(1-15)$

$P_2=\left[\begin{array}{cccc}{p}_{11}^2& p_{12}^2& p_{13}^2& p_{14}^2\\ p_{21}^2& p_{22}^2& p_{23}^2& p_{24}^2\\ p_{31}^2& p_{32}^2& p_{33}^2& p_{34}^2\end{array}\right]---(1-16)$

According to projection relation, can get:

${\mathrm{\&lambda;}}_1\left[\begin{array}{c}{u}_1\\ {\mathrm{<figure-callout\; id="1"\; label="v"\; filenames="A20071009850500183.png"\; state="\{\{state\}\}">v</figure-callout>}}_1\\ 1\end{array}\right]=\left[\begin{array}{cccc}{p}_{11}^1& p_{12}^1& p_{13}^1& p_{14}^1\\ p_{21}^1& p_{22}^1& p_{23}^1& p_{24}^1\\ p_{31}^1& p_{32}^1& p_{33}^1& p_{34}^1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]---(1-17)$

${\mathrm{\&lambda;}}_2\left[\begin{array}{c}{u}_2\\ v_2\\ 1\end{array}\right]=\left[\begin{array}{cccc}{p}_{11}^2& p_{12}^2& p_{13}^2& p_{14}^2\\ p_{21}^2& p_{22}^2& p_{23}^2& p_{24}^2\\ p_{31}^2& p_{32}^2& p_{33}^2& p_{34}^2\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]---(1-18)$

Cancellation λ ₁And λ ₂After:

$\left\{\begin{array}{c}(u_1{p}_{31}^1-p_{11}^1)X+(u_1{p}_{32}^1-p_{12}^1)Y+(u_1{p}_{33}^1-p_{13}^1)Z=p_{14}^1-u_1{p}_{34}^1\\ (v_1{p}_{31}^1-p_{21}^1)X+(v_1{p}_{32}^1-p_{22}^1)Y+(v_1{p}_{33}^1-p_{23}^1)Z=p_{24}^1-v_1{p}_{34}^1\end{array}\right.---(1-19)$

$\left\{\begin{array}{c}(u_2{p}_{31}^2-p_{11}^2)X+(u_2{p}_{32}^2-p_{12}^2)Y+(u_2{p}_{33}^2-p_{13}^2)Z=p_{14}^2-u_2{p}_{34}^2\\ (v_2{p}_{31}^2-p_{21}^2)X+(v_2{p}_{32}^2-p_{22}^2)Y+(v_2{p}_{33}^2-p_{23}^2)Z=p_{24}^2-v_2{p}_{34}^2\end{array}\right.---(1-20)$

Write formula (1-19) and formula (1-20) form of matrix as,

$\left[\begin{array}{ccc}(u_1{p}_{31}^1-p_{11}^1)& (u_1{p}_{32}^1-p_{12}^1)& (u_1{p}_{33}^1-p_{13}^1)\\ (v_1{p}_{31}^1-p_{21}^1)& (v_1{p}_{32}^1-p_{22}^1)& (v_1{p}_{33}^1-p_{23}^1)\\ (u_2{p}_{31}^2-p_{11}^2)& (u_2{p}_{32}^2-p_{12}^2)& (u_2{p}_{33}^2-p_{13}^2)\\ (v_2{p}_{31}^2-p_{21}^2)& (v_2{p}_{32}^2-p_{22}^2)& (v_2{p}_{33}^2-p_{23}^2)\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[\begin{array}{c}{p}_{14}^1-u_1{p}_{34}^1\\ p_{24}^1-v_1{p}_{34}^1\\ p_{14}^2-u_2{p}_{34}^2\\ p_{24}^2-v_2{p}_{34}^2\end{array}\right]---(1-21)$

Formula (1-21) is write the form shown in the accepted way of doing sth (1-22):

AM＝c (1-22)

Utilize the principle of least square, the volume coordinate that can ask M to order,

M＝(A ^TA) ^-1A ^Tc (1-23)

After finishing the finding the solution of unique point locus, the task of flip horizontal image has just been finished, and ensuing curve reestablishing carries out in single image.

Be mapped to the synoptic diagram of three-dimensional model for the two-dimensional grid that will disperse with reference to Fig. 5.The formation of curved surface mainly comprises three steps, i.e. the expression of the smooth surface of curved surface, and the linear restriction that add, and three steps of the optimization of linear restriction lower surface camber objective function are finished.

(1) expression of the smooth surface of curved surface

The smooth surface of curved surface utilizes the parametric surface of standard Represent real three-dimensional model.

Continuous curved surface r (u, v)=[x (u, v), Y (u, v), Z (u, v)] ^TExpression, the calculating of curved surface is to realize by minimizing smooth objective function E (r).

$E\left(r\right)=\underset{0}{\overset{1}{\&Integral;}}\underset{0}{\overset{1}{\&Integral;}}({\left|\right|r_{\mathrm{uu}}\left|\right|}^2+{\left|\right|r_{\mathrm{uv}}\left|\right|}^2+{\left|\right|r_{\mathrm{vv}}\left|\right|}^2)\mathrm{dudv}---(1-24)$

Adopt the grid of discretize to optimize this function.Curved surface r is the matrix X by three m * n, Y, and Z represents.In the solution procedure of curved surface, net point coordinates all on the curved surface is formed a column vector g=[x ^T, y ^T, z ^T] ^T, x wherein, y, z represent respectively matrix X, Y, the arrangement of elements among the Z becomes column vector.In addition, to the single order of net point coordinate and the approximate central difference method of employing of finding the solution of second derivative, promptly more arbitrarily (i, first order derivative discretize j) is expressed as:

$\left\{\begin{array}{c}{X}_u(i,j)=X(i+1,j)-X(i-1,j)\\ X_v(i,j)=X(i,j+1)-X(i,j-1)\end{array}\right.---(1-25)$

In like manner, but the second derivative discretize be expressed as:

$\left\{\begin{array}{c}{X}_{\mathrm{uu}}(i,j)=X(i+1,j)-2X(i,j)+X(i-1,j)\\ X_{\mathrm{uv}}(i,j)=X(i+1,j+1)-X(i,j+1)-X(i+1,j)+X(i,j)\\ X_{\mathrm{vv}}(i,j)=X(i,j+1)-2X(i,j)+X(i,j-1)\end{array}\right.---(1-26)$

These derivatives can utilize the constant matrices C of a mn * mn to represent that easily then first order derivative can be expressed as: x _u=C _uX, x _v=C _vX.Similarly, C _Uu, C _UvAnd C _VvCan be used for representing second-order partial differential coefficient.Thus, formula (1-24) can be expressed as a secondary discrete function about g.

$\left\{\begin{array}{c}\mathrm{\&epsiv;}\left(x\right)=x^T(C_{\mathrm{uu}}^T{C}_{\mathrm{uu}}+{2C}_{\mathrm{uv}}^T{C}_{\mathrm{uv}}+C_{\mathrm{vv}}^T{C}_{\mathrm{vv}})x\\ E\left(g\right)=\mathrm{\&epsiv;}\left(x\right)+\mathrm{\&epsiv;}\left(y\right)+\mathrm{\&epsiv;}\left(z\right)={\mathrm{<figure-callout\; id="3"\; label="g\; T\; C"\; filenames="A20071009850500173.png"\; state="\{\{state\}\}">g</figure-callout>}}^T{C}_{}{}_{3\mathrm{mn}\&times;3\mathrm{mn}}g\end{array}\right.---(1-27)$

(2) linear restriction

After the discretize of finishing curved surface is represented, need increase linear restriction on curved surface, constraint is represented with the form of Ag=b.If do not increase constraint, find the solution formula (1-24) so and will obtain singular solution, promptly r (u, v)=0.

Position constraint: position constraint be based on r (u, v)=[X (and u, v), Y (u, v), Z (u, v)] ^TForm.Certainly, also the part position constraint be can adopt, (X, Y, Z) one or two parameter in promptly only retrained.Employing rectangular projection hypothesis, shown in (1-28):

$\left[\begin{array}{c}u\\ v\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]---(1-28)$

Consider now to calculate under the discrete state, the size of supposing discrete grid block is m * n, corresponding to (u, discrete grid block coordinate v) be (i, j).Be that i=0 is the discrete grid block coordinate corresponding to u=0, i=m is the discrete grid block coordinate corresponding to u=1.In like manner, i=0, j=n corresponds respectively to v=0, v=1.With article one silhouettes line is the interpolation that example illustrates position constraint.At first need the silhouettes line is divided into the n five equilibrium, so just can obtain n+1 summit, remember that their coordinate is

Point on known this curve is corresponding to the point on the u=1/4, so position constraint is:

$\left\{\begin{array}{c}X(m/4,j)=s_j\\ Y(m/4,j)=t_j\end{array}\right.j=0\&CenterDot;\&CenterDot;n---(1-29)$

Using the same method is added to the part position constraint with remaining silhouettes line, adds the discrete grid block after retraining.Strictly speaking, in order to add more accurate position constraint, need be with the top and the bottom complete matching of two views, and make that the identical summit of j value has identical Y coordinate on the every silhouettes line.The method that this chapter provides itself is exactly a kind of approximate method for reconstructing (as perspective projection is assumed to be rectangular projection), and these are similar to and make modeling work to realize quickly and easily, and can reconstruct three-dimensional model very true to nature,

Topological constraints: except position constraint above-mentioned, the constraint that also has the object topological structure itself to bring, promptly the curve at u=0 place overlaps with the curve at u=1 place, and their single order also must be identical with second derivative.

r(0，v)＝r(1，v)v (1-30)

$\left\{\begin{array}{c}{r}_u(0,v)=r_u(1,v)\\ r_v(0,v)=r_v(1,v)\end{array}\right.\&ForAll;v---(1-31)$

$\left\{\begin{array}{c}{r}_{\mathrm{uu}}(0,v)=r_{\mathrm{uu}}(1,v)\\ r_{\mathrm{uv}}(0,v)=r_{\mathrm{uv}}(1,v)\\ r_{\mathrm{vv}}(0,v)=r_{\mathrm{vv}}(1,v)\end{array}\right.\&ForAll;v---(1-32)$

These can be expressed as linear restriction.When finding the solution derivative, in order to simplify calculating, all is usually when calculating the C matrix, X (1, j) and X (m j) calculates as abutment points, thus the simplification constraint matrix.In addition, can also increase Y (u, 0)=Y (0,0)  u and Y (u, 1)=Y (0,1)  u makes curved surface top and bottom smooth.

The perspective projection constraint: in the locus that utilizes principle of triangulation calculated characteristics point, in order to simplify calculating, the camera coordinates system with first camera overlaps with world coordinate system usually, so rotation and translation just need not have been considered.The perspective projection model can be used formula (1-33) expression:

$\mathrm{\&lambda;}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& u_0& 0\\ 0& f& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}\mathrm{<figure-callout\; id="1"\; label="x\; y\; z"\; filenames="A20071009850500183.png"\; state="\{\{state\}\}">x</figure-callout>}& \\ y\\ z\\ 1\end{array}\right],---(1-33)$

Wherein f is a camera focus, (u ₀, v ₀) be the image coordinate of principal point, (x, y z) are the coordinate under the world coordinate system, and (u v) is an image coordinate, and λ is a non-zero zoom factor.According to formula (1-33), can obtain two linear restrictions:

$\left\{\begin{array}{c}\mathrm{fx}+(u_0-u)z=0\\ \mathrm{fy}+(v_0-v)z=0\end{array}\right.---(1-34)$

Plane restriction:, then can use formula (1-35) expression constraint if the point on the curved surface is positioned on certain plane.

Ax(i，j)+By(i，j)+C(i，j)+D＝0 (1-35)

Wherein (C D) is the parameter on plane for A, B.

Line constraint: the space line equation can be described by formula (1-36).

$\frac{{x-x}_0}{m}=\frac{{y-y}_0}{n}=\frac{{z-z}_0}{p}---(1-36)$

(x wherein ₀, y ₀, z ₀) be a bit on the straight line, (m, n p) are the direction vector of straight line.According to formula (1-36), can obtain two linear restrictions:

$\left\{\begin{array}{c}{\mathrm{nx}-\mathrm{my}=\mathrm{nx}}_0-m{y}_0\\ {\mathrm{px}-\mathrm{mz}=\mathrm{px}}_0-{\mathrm{mz}}_0\end{array}\right.---(1-37)$

(3) optimization of secondary objective function under the linear restriction

In summary, seeking the smooth surface satisfy these constraints is exactly double optimization problem under these linear restriction conditions.All linear restrictions can be expressed as the form of Ag=b, and all are linear restrictions.E (g) is a secondary, can be expressed as g ^TCg, wherein C is Hessian matrix (Hessian matrix).Therefore, the linear restriction double optimization can be defined as follows:

Formula (1-38) can be converted into linear system by lagrange's method of multipliers:

$\left[\begin{array}{cc}C& A^T\\ A& 0\end{array}\right]\left[\begin{array}{c}g\\ \mathrm{\&lambda;}\end{array}\right]=\left[\begin{array}{c}0\\ b\end{array}\right]---(1-39)$

Hessian matrix C is a banded sparse matrix, and for the grid of a m * n, the size of C is 3mn * 3mn, nearly 13 nonzero elements of every row.Find the solution formula (1-39), and separating of being asked is globally optimal solution.

With reference to Fig. 6, in the drafting of virtual scene, three-dimensional model has not only comprised geometric data, has equally also comprised texture image, could make that like this drafting effect is more true to nature.Among the present invention, texture image all is to store and represent according to the mode of rectangle.Just in time the discrete grid block with curved surface is corresponding for this, and the discrete grid block in the three-dimensional model shown in 601 is 32 * 32,

Can the spot projection on the discrete grid block in image, extract the colouring information of image, thereby be arranged in texture according to the form of discrete grid block.But the easy like this texture sampling that causes is too sparse, therefore grid need be carried out refinement, and the net point after each refinement is corresponding to a pixel in the texture image.Shown in 602, heavy line is represented original geometric grid, and size is 4 * 3.In order to extract texture, each geometric grid evenly is refined as 4 * 4 grid again, the whole sizing grid after the refinement is 16 * 12 like this.With the corresponding pixel in each summit on the grid, each summit projected in the image again (can all project to the longitudinal network ruling and the transverse grid line of net point correspondence in the image in the actual solution procedure, find intersection in image), and obtain this colouring information in image, thereby formation texture image, its size are 16 * 12.

The coordinate figure of the picture point after the process refinement is floating number often, if directly get the color value of the color value of its some neighborhood pixels as current point, then the texture of Ti Quing will become comparatively fuzzy.Therefore, this chapter adopts the weighting scheme based on barycentric coordinates to extract colouring information according to the far and near degree of four neighborhood pixels around the residing position of current point and its.Shown in 602, (x y) is the pixel coordinate of present image point, (x ₀, y ₁), (x ₁, y ₁), (x ₀, y ₀), (x ₁, y ₀) be the coordinate (coordinate figure is an integer) of four neighborhood pixels.Make f (x, y) be point (x, the colouring information of y) locating extract and can represent with following formula based on the texture of barycentric coordinates so:

$f(x,y)=\frac{V_{00}}{V}f(x_0,y_0)+\frac{V_{01}}{V}f(x_0,y_1)+\frac{V_{10}}{V}f(x_1,y_0)+\frac{V_{11}}{V}f(x_1,y_1)---(1-40)$

V wherein ₀₀=(x ₁-x) (y ₁-y), V ₀₁=(x ₁-x) (y-y ₀), V ₁₀=(x-x ₀) (y ₁-y), V ₁₁=(x-x ₀) (y-y ₀), V=V ₀₀+ V ₀₁+ V ₁₀+ V ₁₁

Fig. 7 shows the flow process that whole single width symmetroid is rebuild, the step of rebuilding is: after (1) obtains the image of symmetroid, need utilize Flame Image Process instrument (as Photoshop) to obtain the flip horizontal image of this image, and with original image and flip horizontal image input modeling; (2) matching characteristic point in this two width of cloth image, at least 8 pairs of the matching characteristic points of input calculate the polar curve restriction relation between this two width of cloth image, and new Feature Points Matching can be carried out under the guiding of polar curve, thereby improves the correctness and the precision of coupling; (3) after the work of Feature Points Matching is finished, the focal length value that input has been demarcated, or utilize the constraint of scene itself to realize the demarcation certainly of camera intrinsic parameter, or directly carry out focal length and estimate; (4) calculate essential matrix between two width of cloth images; (5) utilize based drive reconstruction technique to calculate the locus of these unique points; (6) utilize these spatial point positions to set up the areal model of curved face object; (7) be at last with these spatial point as reference, under the hypothesis of perspective projection, reconstruct the surface model of object, and extract texture, thereby finally obtain the three-dimensional model of object.

Claims (5)

1. the smooth symmetroid three-dimensional entity model method for reconstructing based on single image is characterized in that comprising the steps:

(1), and it is carried out mirror face turning obtains horizontal mirror face turning view with the input of smooth symmetroid original view;

(2) determine that the match point in original view and the horizontal mirror face turning view is right;

(3) the intrinsic parameter matrix of the used camera of calculating original image;

(4) right according to the Feature Points Matching between intrinsic parameter matrix and original image and the horizontal mirror reversing image, solve essential matrix and the essential matrix that obtains is decomposed into rotation matrix and translation matrix;

(5) according to the coordinate of original image two-dimensional points, try to achieve the three-dimensional coordinate of corresponding point, generate the areal model of object;

(6) obtain final smooth surface model by minimizing smooth objective function;

(7) extract the original image texture, and be mapped on the smooth surface of generation, obtain three-dimensional entity model.

2. three-dimensional entity model method for reconstructing according to claim 1 is characterized in that: the original image that is used to rebuild is the image of single width symmetric objects.

3. three-dimensional entity model method for reconstructing according to claim 1 and 2, it is characterized in that: the formation of the smooth surface model in the step (6) is finished as follows, be the expression of the smooth surface of curved surface, set up linear restriction, and the optimization of linear restriction lower surface camber objective function.

4. three-dimensional entity model method for reconstructing according to claim 1 is characterized in that: adopt center of gravity texture extracting method to extract the original image texture, and be mapped on the smooth surface of generation, obtain three-dimensional surface model true to nature.

5. three-dimensional entity model method for reconstructing according to claim 3 is characterized in that: the linear restriction kind of being set up is at least a of position constraint, perspective constraint, topological constraints, plane restriction or line constraint.

Images