CN103955960B - Image viewpoint transformation method based on single input image - Google Patents

Abstract

The invention discloses an image viewpoint transformation method based on a single input image. The method comprises the following steps: step one, interactively deducting a cube structure contained in an image and marking a typical line in the image; step two, reconstructing a three-dimensional scene through combination with the space characteristics of a cube object by use of the imaging principle of a pinhole camera, the three-dimensional scene being capable of completely simulating a real three-dimensional scene during viewpoint transformation; step three, under the condition that a focal length is not changed, through the viewpoint transformation, projecting the reconstructed three-dimensional scene to an image plane corresponding to a new viewpoint; step four, according to interaction constraints, performing Delaunay triangle gridding with constraints; step five, by taking a scene theme structure constraint obtained through reprojection in step three as deformation driving, solving a grid deformation energy equation formed in step four by use of a method for solving an energy equation; and step six, by use of a target grid obtained through solving, mapping grid textures corresponding to the target grid in an original image one by one to generate a target image.

Description

A kind of image viewpoint change method based on single width input picture

Technical field

The present invention relates to a kind of image viewpoint change method based on single width input picture, belong to Computer Image Processing andMultimedia information technology process field.

Background technology

Along with the hardware improvement of digital camera and the development of image editing software, people can obtain height more easilyThe image frame of quality. But the captured photo of a large amount of shutterbugs is but often because choosing of visual angle caused whole pictureThe not nature of face. For instance, cameraman may be oblique originally clapping askew bat perpendicular to the building on ground, make shooting effect fromSo, how to proofread and correct the photo that such agent structure tilts and there is good using value. In this case, people begin one's studyImage viewpoint transfer problem. Traditional mode is first to utilize the image of multiple Same Scene to carry out registration, then carries out three-dimensionalRebuild, recycle existing three-dimensional structure and under new viewpoint, carry out re-projection and play up the image frame obtaining under new viewpoint. ThisThe remarkable shortcoming of kind method is to need multiple input pictures or a lot of artificial mutual to carry out the accurate weight of three-dimensional scenic of needsBuild, all infeasible for complicated a little three-dimensional scenic, therefore the applicability of these class methods and efficiency are all very low, and effect is also owedGood.

Summary of the invention

Goal of the invention: the invention provides a kind of image viewpoint change method based on single width input picture, make userBy seldom and easily obtaining the image after viewpoint change with simple calculating alternately.

Technical scheme: the invention discloses a kind of image viewpoint change method based on single width input picture, mainly compriseFollowing steps:

Step 1, image is mutual: by the cube structure in manual interactively pick-up image, specifically pass through its of mark side's bodyIn six corresponding image projection pixels of point, be designated as (p₀,p₁,p₂,p₃,p₄,p₅), wherein (p₀,p₁,p₂,p₃) corresponding threeDimensional plane is perpendicular to (p₀,p₁,p₅,p₄) corresponding three-dimensional planar, p₀p₁Corresponding space line is the intersection of two planes, mutualTo the characteristic curve in image, comprise the characteristic curve that affects picture material, horizontal and vertical constraint line segment and specify image borderLine;

Step 2, utilizes pinhole camera image-forming principle to rebuild a three-dimensional scenic in conjunction with the spatial character of cube structure object,In the time of viewpoint change, utilize this scene structure three-dimensional scene structure that is virtually reality like reality, cube structure in hypothesis three-dimensional scenicOn the basis that the picture point on image overlaps with it, one of them summit, the geometrical constraint that the side's of utilization body comprises, calculates (p₀, p₁,p₂,p₃,p₄,p₅) square body space point coordinates (P in corresponding three-dimensional scenic₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T)；

Step 3, utilizes the square body corresponding to space coordinates of six points that under new viewpoint, re-projection step 2 calculates, and comesThe three-dimensional scenic that representative is rebuild: as the centre of sphere, arrive the distance of camera with simulated scenario center at the initial point O taking place, simulated scenario centerFrom | in the ball plane that OF| is radius, define a new viewpoint O', by the three-dimensional cube structure (P rebuilding₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T, P₅ ^T) again project to picture plane corresponding to new viewpoint;

Step 4, image triangle gridding: according to the interactive structure of step 1, utilize the Delaunay trigonometric ratio pair of belt restrainingImage carries out triangle gridding; The algorithm of Delaunay trigonometric ratio can be referring to " computational geometry (the 3rd edition) Burger work, world bookThe 2013-10-1 of publishing house publishes, ISBN:9787510061776 ";

Step 5, mesh parameterization: utilize conformal projection and various feature to keep constraint principles to set up mesh parameterizationEquation, utilizes scene side's body framework that step 4 re-projection obtains to build the hard constraint of projection mapping, solves parametrization energy sideJourney, obtains target gridding;

Step 6, target image generates: solve the grid obtaining according to step 5, by corresponding with target gridding in original imageGrid texture utilize bilinear interpolation method to be mapped to one by one corresponding target gridding, generate target image.

In step 1, by manually obtaining alternately the image implicit cube structure that accounts for image subject structure and characteristic curve.

Step 1 comprises following steps:

Step 11, described single width input picture, implicit cube structure in its scene, manually the party's body in interactively pick-up imageThe projection of structure, with wherein six the corresponding image projection pixel of point (p of square body₀,p₁,p₂,p₃,p₄,p₅) represent thisCube structure, wherein (p₀,p₁,p₂,p₃) corresponding space three-dimensional planar quadrature is in (p₀,p₁,p₅,p₄) corresponding three-dimensional is flatFace, p₀p₁Corresponding space line is the intersection of two planes;

Step 12, the cube structure (p obtaining according to step 11₀,p₁,p₂,p₃,p₄,p₅), by cube structure (p in image₀, p₁,p₂,p₃,p₄,p₅) between the line that comprises be all labeled as hard constraint characteristic curve, on these hard constraint characteristic curves, extract equablyTriangle gridding summit, concrete grammar is as follows: to a specific hard constraint characteristic curve, starts to get a little from one end, every along rectilinear directionGet the next point on this straight line every 25 pixel distances, until the other end of straight line, for institute on all hard constraint characteristic curvesThe grid vertex of obtaining, is designated as { v_c1,v_c2,...,v_ck, the grid that wherein ck representative is extracted is here determined summit sum;

Step 13, obtains the characteristic straight line in image alternately, comprises the characteristic curve that affects picture material, and horizontal and vertical approximatelyEvenly get a little on these lines bunch section and specify image boundary line, and concrete grammar is as follows: to a special characteristic line constraintStraight line, starts to get a little from one end, gets the next point on this straight line along rectilinear direction every 25 pixel distances, until straight lineThe other end;

Step 14, for the indicatrix in image, for example, is the circular sun originally, extracts successively triangle on curveMesh point, concrete grammar is as follows: from one end of indicatrix, along indicatrix prolonging direction, with about 25 pixelsDistance is as getting an interval, until arrive the other end of indicatrix. .

In step 2, utilize the space geometry structure of pinhole camera image-forming principle and square body object to set up the three-dimensional of simulationCube structure model.

Step 2 specifically comprises the following steps:

Step 21, taking picture centre as initial point O (0,0,0), O to camera loca F (0,0, f) be Z axis forward, image is flatThe transverse axis of face and y direction are that XY direction is set up three-dimensional world coordinate system, and wherein f is camera focus, specifically with reference to Fig. 3;

Step 22, calculates straight line p₀p₃With straight line p₁p₂Intersection point c₁, straight line p₀p₄With straight line p₁p₅Intersection point c₂, aobviousSo in three dimensions, straight line Fc₁With straight line P₀ ^TP₃ ^T，P₁ ^TP₂ ^TIntersect at infinite point, obviously have Fc₁Put down and straight line P₀ ^TP₃ ^T, withSample straight line Fc₂With straight line P₀ ^TP₄ ^T，P₁ ^TP₅ ^TIntersect at infinite point, obviously have Fc₂Put down and straight line P₀ ^TP₄ ^T, according to P₀ ^TP₃ ^T⊥ P₀ ^TP₅ ^T, obtainObtain focal distance f so solve;

Step 23, in note real scene on square body corresponding to cube structure (p in image₀,p₁,p₂,p₃,p₄,p₅) sixPoint is (Q₀,Q₁,Q₂,Q₃,Q₄,Q₅), have according to pin-hole imaging model:K=0,1 ..., 5, whereinRepresentative toAmountWithBetween ratio, same, in simulated scenario with (p₀,p₁,p₂,p₃,p₄,p₅) corresponding (P₀ ^T, P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T), must have equallyK=0,1 ..., 5 set up, wherein rk representation vectorWithBetween ratio, fixing r₀Value be 1, utilizeWithSolve r₁,r₂,r₃,r₄,r₅; WhenTime, P_iWith Q_iOverlap completely, now we can count completelyCalculate in real scene cube structure corresponding to (p₀,p₁,p₂,p₃,p₄,p₅) true coordinate, otherwise have P_i ^TP_j ^T//Q_iQ_jSet up,Both (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) form model be (Q₀,Q₁,Q₂,Q₃,Q₄,Q₅) convergent-divergent of component model;

Step 24, the ratio r obtaining according to step 23₀,r₁,r₂,r₃,r₄,r₅, calculate in simulated scenario corresponding to square bodyStructure (p₀,p₁,p₂,p₃,p₄,p₅) cube structure (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T), set P₀ ^TWith p₀Overlap, definition sceneCenter was summit P in cube structure in scene₀ ^TSubpoint on primary optical axis is initial point O, in simulated scenario herein justThe heart drops on initial point, corresponding to Q in real scene₀Subpoint on primary optical axis, real scene center O_Q。

In step 3, the three-dimensional cube structure of shadow simulation again under new viewpoint.

Step 3 specifically comprises the following steps:

Step 31, the cube structure (P obtaining according to step 24₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) we are taking O as the center of circle |OF| gets a new viewpoint O' on the sphere of radius at random, whereinWithAngle be no more than 45 degree, through simulation yardScape center O structure perpendicular to the planar I of OO' ', try to achieve point (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) with the line of O' and planar I 'Intersection point (p'₀,p₁',p'₂,p'₃,p'₄,p'₅), complete the projection of cube structure under new viewpoint in simulated scenario;

Step 32, by normal vectorObtain unit vector x' with the multiplication cross of vertical vector (0,1,0), by x' withForkTake advantage of and obtain unit vector y', on plane of delineation I' taking O as initial point, vector x ' direction is abscissa forward, vectorial y' direction isOrdinate forward is set up image two-dimensional coordinate system, willI=0,1,2 ..., 5 project to respectively vector x ', y' both directionUpper, obtain intersection point (p'₀,p₁',p'₂,p'₃,p'₄,p'₅) two-dimensional coordinate on plane of delineation I', be designated as (q₀,q₁,q₂,q₃,q₄, q₅), we have obtained in original image on cube structure the two dimension that six points are projected on target image under new viewpoint and have sat like thisMark.

In step 4, utilize the grid vertex extracting to carry out Delaunay trigonometric ratio to image.

Step 4 specifically comprises the following steps:

Step 41, on the basis of the feature grid point obtaining in step 1, evenly adopts a little at random for the remaining area of imageAs grid vertex;

Step 42, in the process of trigonometric ratio, with the mutual characteristics of image line of step 1, gets adjacent top on same characteristic curvePoint line, as triangle gridding limit, utilizes the Delaunay trigonometric ratio of belt restraining to carry out triangle gridding to image, DelaunayThe algorithm of trigonometric ratio can referring to " computational geometry (the 3rd edition) the Burger work world book 2013-10-1 of publishing house publishes, ISBN: 9787510061776”。

In step 5, the energy constraint of construct image distortion.

Step 5 specifically comprises the following steps:

Step 51, (the p that utilizes step 32 to calculate₀,p₁,p₂,p₃,p₄,p₅) under corresponding new viewpoint in picture planePixel (q₀,q₁,q₂,q₃,q₄,q₅), utilizeCalculate a v'_ciCorresponding some v'_ci, wherein v_ciForGrid vertex on original image hard constraint characteristic straight line, p_s，p_eFor v_ciTwo end points on the hard constraint characteristic straight line of place, q_s， q_eFor being respectively and p_s，p_eCorresponding point, i.e. { v_c1,v_c2,...,v_ckUnder new viewpoint O', be mapped as { v'_c1,v'_c2,..., v'_ck, remember that it is F that its corresponding hard constraint closes_C(v'_c1,v'_c2,...,v'_ck)=0, shows v in the time that anamorphose is shone upon_iReflectedBe mapped to v_i'；

Step 52, definition shape constraining: two-dimentional triangle gridding affine transformation is designated as to M:(x, y) → (x', y'), according to KeXi ?Riemann (Cauchy – Riemann) equation have:

$\frac{\∂M}{\∂x}+i\frac{\∂M}{\∂y}=0$

Wherein i is imaginary unit, about Ke Xi ?Riemann equation, and can be referring to " LSCM:Least squares conformal maps for automatic texture atlas generation,ACM Transactions on Graphics,21(3): 362 ?372. ", mapping Jacobian matrix corresponding to M has following form:

$J=\left[\begin{array}{cc}a& b\\ b& -a\end{array}\right]---\left(1\right)$

Wherein a, b value is any, in order to equate and the relation of opposite number each other between each element in representing matrix J;

Corresponding former grid is to the conversion of target gridding, and we adopt radiation conversion; Remember that former triangle is T (v_ti,v_tj,v_tk)， v_ti,v_tj,v_tkFor its three summits, after its corresponding viewpoint change, deforming triangle to be solved is T'(v'_ti,v'_tj,v'_tk)， v'_ti,v'_tj,v'_tkFor with v_ti,v_tj,v_tkThree summits one to one, A=T'T^-1The affine transformation matrix from T to T', T^-1There is following form:

$T^{-1}=\left[\begin{array}{ccc}{a}_1& b_1& d_1\\ a_2& b_2& d_2\\ a_3& b_3& d_3\end{array}\right]={\left[\begin{array}{ccc}{x}_{\mathrm{ti}}& \stackrel{}{x_{\mathrm{tj}}}& x_{\mathrm{tk}}\\ y_{\mathrm{ti}}& y_{\mathrm{tj}}& y_{\mathrm{tk}}\\ z_{\mathrm{ti}}& z_{\mathrm{tj}}& z_{\mathrm{tk}}\end{array}\right]}^{-1}---\left(2\right)$

Wherein, x_ti,x_tj,x_tkFor a v_ti,v_tj,v_tkAbscissa on image, y_ti,y_tj,y_tkFor a v_ti,v_tj,v_tk?Ordinate on image, z_ti,z_tj,z_tkFor for representing v_ti,v_tj,v_tkOrdinate, unified value is 1 here; a₁,a₂,a₃, b₁,b₂,b₃,c₁,c₂,c₃It is matrix T^-1In the Parametric Representation of each element, value is all the constant calculating, by formula (1) and (2)Obtain the following energy equation of conformal projection:

E_TJ1＝a₁x'_ti+a₂x'_tj+a₃x'_tk+(b₁y'_ti+b₂y'_tj+b₃y'_tk)（3）

E_TJ2＝b₁x'_ti+b₂x'_tj+b₃x'_tk-(a₁y'_ti+a₂y'_tj+a₃y'_tk)

Wherein x'_ti,x'_tj,x'_tkBe respectively v'_ti,v'_tj,v'_tkAbscissa value, y'_ti,y'_tj,y'_tkBe respectively v'_ti, v'_tj,v'_tkOrdinate value, their gangs are obtained:

$E_S=\underset{T}{\mathrm{\Σ}}(E_{\mathrm{TJ}1}^2+E_{\mathrm{TJ}2}^2)---\left(4\right)$

Defined feature line constraint: note (v_i,v_j,v_k) be continuous three points on a characteristic curve, keepWithItBetween ratio r_ljAnd anglec of rotation θ_lj, be defined as follows target equation:

$E_L=\underset{(v_{\mathrm{li}},v_{\mathrm{lj}},v_{\mathrm{lk}})}{\mathrm{\Σ}}{\left|\right|(v_{\mathrm{lk}}^{\′}-v_{\mathrm{lj}}^{\′})-r_{\mathrm{lj}}{R}_{\mathrm{lj}}(v_{\mathrm{lj}}^{\′}-v_{\mathrm{li}}^{\′})\left|\right|}^2---\left(5\right)$

Wherein $R_{\mathrm{lj}}=\left(\begin{array}{cc}\cos {\mathrm{\θ}}_{\mathrm{lj}}& -\sin {\mathrm{\θ}}_{\mathrm{lj}}\\ \sin {\mathrm{\θ}}_{\mathrm{lj}}& \cos {\mathrm{\θ}}_{\mathrm{lj}}\end{array}\right),$

Definition vertical constraint: remember that the mesh point on vertical line segment lv is designated as { v_lv1,v_lv2,...,v_lvm, vertical constraint representsFor:

$E_V=\underset{\mathrm{lv}}{\mathrm{\Σ}}\underset{\mathrm{nv}=1}{\overset{\mathrm{lvm}}{\mathrm{\Σ}}}{(x_{\mathrm{nv}}^{\′}-x_{\mathrm{lv}1}^{\′})}^2---\left(6\right),$

Wherein x'_nvRepresent some v'_nvAbscissa, x'_lv1Represent some v'_lv1Abscissa.

Definition horizontal restraint: the mesh point on note horizontal line section lh is designated as { v_lh1,v_lh2,...,v_lhm, vertical constraint representsFor: $E_H=\underset{\mathrm{lh}}{\mathrm{\Σ}}\underset{\mathrm{nh}=1}{\overset{\mathrm{lhm}}{\mathrm{\Σ}}}{(x_{\mathrm{nh}}^{\′}-x_{\mathrm{lh}1}^{\′})}^2---\left(7\right),$

Wherein y'_nvRepresent some v'_nvAbscissa, y_l'_h1Represent some v'_lh1Abscissa.

Specify according to user interactions, non-rigid Characteristic constraint line is appointed as in part or all of original image border, note (v_bi,v_bj,v_bk) be continuous three the feature summits on boundary characteristic line, keepWithBetween ratio r_bjDefinition figureThe boundary constraint E of picture_BAs follows;

$E_B=\underset{(v_{\mathrm{bi}},v_{\mathrm{bj}},v_{\mathrm{bk}})}{\mathrm{\Σ}}{\left|\right|(v_{\mathrm{bk}}^{\′}-v_{\mathrm{bj}}^{\′})-r_{\mathrm{bj}}(v_{\mathrm{bj}}^{\′}-v_{\mathrm{bi}}^{\′})\left|\right|}^2---\left(8\right),$

Wherein r_bjForWithBetween ratio.

Step 53, comprehensive step 51 and step 52, represent that the energy equation of mesh parameterization is as follows:

argmaxλ_SE_S+λ_LE_L+λ_VE_V+λ_HE_H+λ_BE_B（9）

s.t.F_C(v₁',v'₂,...,v'_k)＝0

Wherein λ_S,λ_L,λ_V,λ_H,λ_BFor weights corresponding to each energy term, in actual running for different picturesParameter has certain adjustment, provides one group of empirical parameter value as follows: λ through too much group experiment picture_SGet 1, λ_LWith λ_BGet 100, λ_VWith λ_HGet 10.

In step 6, utilize grid pinup picture to generate target image.

Step 6 specifically comprises the following steps:

Step 61, according to equation (9), solves and the target triangle gridding that in original image, each triangle gridding is corresponding,Adopt the method for bilinear interpolation to be mapped to corresponding target triangle gridding the picture texture in former triangle gridding, form targetImage.

Beneficial effect: the present invention comprises following advantage:

(1) can process situation and the more shirtsleeve operation flow process of only having single width scene graph, without several scene graphAs inputting and avoided complicated image registration operation.

(2) user interactions easily comparatively on a small quantity. Than adopting pure craft to simulate the method for viewpoint change, thisBright method interworking amount reduces greatly, only needs manually to mark out alternately cube structure and a small amount of characteristic curve of main body, protectsDemonstrate,prove the ease for use of method.

(3) authenticity of effect. Due to taking individual real shooting photo as input, generate result images by anamorphose, knotFruit image effect is good, and the present invention supports the manual mark of user need to keep the characteristic curve of shape in addition, and therefore result images is not allowedBe prone to the distortion flaw highlighting.

(4) processing speed and robustness fast. Remove user interactions, the inventive method can obtain about 0.1 secondResult images.

Brief description of the drawings

Below in conjunction with the drawings and specific embodiments, the present invention is done further and is illustrated, the present invention above-mentioned and/Or otherwise advantage will become apparent.

Fig. 1 is the basic flow sheet of the inventive method.

Fig. 2 is the implicit cube structure schematic diagram to image interactively pick-up that the present invention describes.

Fig. 3 is simulation three-dimensional structure and the true three-dimension re-projection uniformity schematic diagram that the present invention describes.

Fig. 4 is the example flow chart that the inventive method is implemented.

Fig. 5 is the inventive method result and Lightroom software institute of Adobe Systems Inc of the U.S. (Adobe Systems)The comparative effectiveness figure of the method realizing.

Fig. 6 is that the inventive method simulation ken ball is observed the generation design sketch of image scene.

Detailed description of the invention:

The flow process of this method as shown in Figure 1. First take off alternately in the cube structure and mark image comprising in imageIndicatrix; Then utilize the spatial character of pinhole camera image-forming principle and square body object to recover the three-dimensional structure of simulation;And then change by viewpoint, the three-dimensional structure re-projection recovering is arrived to picture plane corresponding to new viewpoint; Retrain figure according to mutualPicture carries out triangle gridding; Use the result that step 3 re-projection obtains to drive as distortion, utilize the method that solves energy equationGrid is carried out to parametrization to be solved; Finally generate target image by texture mapping interpolation. The present invention can be certain spherical lookingWithin the scope of territory, the object of individual input picture is carried out to three-dimensional simulation observation, form continuous real scene simulation, specifically can be referring to Fig. 4Example flow chart.

Specifically, as shown in Figure 1, a kind of image viewpoint change method based on single width input picture:

Step 1, image is mutual: by the cube structure in manual interactively pick-up image, specifically pass through its of mark side's bodyIn six the corresponding image projection pixel of point (p₀,p₁,p₂,p₃,p₄,p₅), wherein (p₀,p₁,p₂,p₃) corresponding three-dimensional is flatFace is perpendicular to (p₀,p₁,p₅,p₄) corresponding three-dimensional planar, p₀p₁Be the intersection of two planes, specifically as shown in Figure 2, obtain alternately figureCharacteristic curve in picture, comprises the characteristic curve that affects picture material, horizontal and vertical line segment and specify image boundary line;

Step 2, utilizes pinhole camera image-forming principle to rebuild a three-dimensional scenic in conjunction with the spatial character of cube structure object,In the time of viewpoint change, utilize this scene structure three-dimensional scene structure that is virtually reality like reality, cube structure in hypothesis three-dimensional scenicOn the basis that the picture point on image overlaps with it, one of them summit, the geometrical constraint that the side's of utilization body comprises, calculates (p₀, p₁,p₂,p₃,p₄,p₅) square body space point coordinates (P in corresponding three-dimensional scenic₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T)；

Step 3, utilizes the square body corresponding to space coordinates of six points that under new viewpoint, re-projection step 2 calculates, and comesThe three-dimensional scenic that representative is rebuild: with simulated scenario center, initial point O is the centre of sphere, the distance with simulated scenario center to camera |OF| defines a new viewpoint O' in the ball plane of radius, by reconstruction of three-dimensional cube structure (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) heavyNewly project to picture plane corresponding to new viewpoint;

Step 4, image triangle gridding: according to the interactive structure of step 1, utilize the Delaunay trigonometric ratio pair of belt restrainingImage carries out triangle gridding; The algorithm of Delaunay trigonometric ratio can be referring to " computational geometry (the 3rd edition) Burger work, world bookThe 2013-10-1 of publishing house publishes, ISBN:9787510061776 ";

Step 5, mesh parameterization: utilize conformal projection principle and various feature to keep constraint to set up mesh parameterizationEquation, utilizes scene side's body framework that step 4 re-projection obtains to build the hard constraint of projection mapping, solves parametrization energy sideJourney, obtains target gridding;

Step 6, target image generates: solve the grid obtaining according to step 5, by corresponding with target gridding in original imageGrid texture utilize bilinear interpolation method to be mapped to one by one corresponding target gridding, generate target image.

In step 1, by manually obtaining alternately the image implicit cube structure that accounts for image subject structure and characteristic curve.

Step 1 comprises following steps:

Step 11, as shown in Figure 2, by the projection structure of cube structure implicit in manual interactively pick-up image, weGenerally choose 6 such point (p₀,p₁,p₂,p₃,p₄,p₅), it is corresponding to 6 point (Q on square body in real scene₀,Q₁,Q₂, Q₃,Q₄,Q₅), wherein (Q₀,Q₁,Q₂,Q₃) on same side surface, (Q₀,Q₁,Q₃,Q₄) coexist on a square surface, andThere is Q₀Q₃//Q₁Q₂，Q₀Q₄//Q₁Q₅，Q₀Q₄⊥Q₀Q₁，Q₀Q₃⊥Q₀Q₁；

Step 12, the cube structure (p obtaining according to step 11₀,p₁,p₂,p₃,p₄,p₅), by cube structure (p in image₀, p₁,p₂,p₃,p₄,p₅) between the line that comprises be all labeled as hard constraint characteristic curve, on these hard constraint characteristic curves, extract equablyTriangle gridding summit, concrete grammar is as follows: to a specific hard constraint characteristic curve, starts to get a little from one end, every along rectilinear directionGet the next point on this straight line every 25 pixel distances, until the other end of straight line, for institute on all hard constraint characteristic curvesThe grid vertex of obtaining, is designated as { v_c1,v_c2,...,v_ck, the hard constraint grid that wherein ck representative is extracted is determined summit sum;

Step 13, obtains the characteristic straight line in image alternately, comprises the Eigenvector that affects picture material, horizontal and verticalEvenly get a little on these line segments constraint line segment and specify image boundary line, and concrete grammar is as follows: straight to a special characteristicLine, we start to get a little from its one end, get the next point on this straight line along rectilinear direction every 25 pixel distances, until straightThe other end of line;

Step 14, for the indicatrix in image, for example, is the circular sun originally, extracts successively triangle on curveMesh point, concrete grammar is as follows: since one end of an indicatrix, along indicatrix prolonging direction, with about 25Pixel distance is as getting an interval, until arrive the other end of indicatrix.

In step 2, utilize the space geometry structure of pinhole camera image-forming principle and square body object to set up the three-dimensional of simulationCube structure model.

Step 2 specifically comprises the following steps:

Step 21, taking picture centre as initial point O (0,0,0), O to camera loca F (0,0, f) be Z axis forward, image is flatThe transverse axis of face and y direction are that XY direction is set up three-dimensional world coordinate system, and wherein f is camera focus, specifically with reference to Fig. 3;

Step 22, calculates straight line p₀p₃With straight line p₁p₂Intersection point c₁, straight line p₀p₄With straight line p₁p₅Intersection point c₂, aobviousSo in three dimensions, straight line Fc₁With straight line P₀ ^TP₃ ^T，P₁ ^TP₂ ^TIntersect at infinite point, obviously have Fc₁Put down and straight line P₀ ^TP₃ ^T, withSample straight line Fc₂With straight line P₀ ^TP₄ ^T，P₁ ^TP₅ ^TIntersect at infinite point, obviously have Fc₂Put down and straight line P₀ ^TP₄ ^T, according to P₀ ^TP₃ ^T⊥ P₀ ^TP₅ ^T, obtainObtain focal distance f so solve;

Step 23, in note real scene on square body corresponding to cube structure (p in original image₀,p₁,p₂,p₃,p₄,p₅)Six points are cube structure (Q₀,Q₁,Q₂,Q₃,Q₄,Q₅), obviously we haveK=0,1 ..., 5,, whereinGenerationTable vectorWithBetween ratio, in simulated scenario and cube structure (p six points₀,p₁,p₂,p₃,p₄,p₅)Corresponding cube structure (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T), must have equallyK=0,1 ..., 5 r wherein_kRepresentation vectorWithBetween ratio, fixing r₀Value be 1, utilize Solve r₁,r₂,r₃,r₄,r₅; WhenTime, P_iWith Q_iCompletely overlap, now weCan calculate in real scene cube structure completely corresponding to (p₀,p₁,p₂,p₃,p₄,p₅) true coordinate, otherwise have P_iP_j// Q_iQ_jSet up both (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) form model be (Q₀,Q₁,Q₂,Q₃,Q₄,Q₅) contracting of component modelPut;

Step 24, the ratio r obtaining according to step 23₀,r₁,r₂,r₃,r₄,r₅Calculate in simulated scenario corresponding to square bodyStructure (p₀,p₁,p₂,p₃,p₄,p₅) cube structure (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T), set P₀ ^TWith p₀Overlap, definition sceneCenter was summit P in cube structure in scene₀ ^TSubpoint on primary optical axis is initial point O herein, and simulated scenario center fallsOn initial point, corresponding to Q in real scene₀Subpoint on primary optical axis, real scene center O_Q。

In step 3, the three-dimensional cube structure of shadow simulation again under new viewpoint.

Step 3 specifically comprises the following steps:

Step 31, the simulation cube structure (P obtaining according to step 2₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T), we are taking O as roundThe heart | on the sphere that OF| is radius, get at random a new viewpoint O', whereinWithAngle be no more than 45 degree, cross simulation yardScape center O structure perpendicular to the planar I of OO' ', try to achieve point (P₀ ^T,P₁ ^T,P₂ ^T,P₃ ^T,P₄ ^T,P₅ ^T) with the line of O' and planar I 'New intersection point (p'₀,p₁',p'₂,p'₃,p'₄,p'₅), complete the projection of cube structure under new viewpoint in simulated scenario;

Step 32, by normal vectorObtain unit vector x' with the multiplication cross of vertical vector (0,1,0), by x' withForkTake advantage of and obtain unit vector y', on plane of delineation I' taking O as initial point, vector x ' direction is abscissa forward, vectorial y' direction isOrdinate forward is set up image two-dimensional coordinate system, willK=0,1,2 ..., 5 project to respectively vector x ', y' both directionUpper, obtain intersection point (p'₀,p₁',p'₂,p'₃,p'₄,p'₅) two-dimensional coordinate on plane of delineation I', be designated as point (q₀,q₁,q₂,q₃, q₄,q₅), we have obtained in original image six points on cube structure and under new viewpoint, have been projected in the two dimension on target image like thisCoordinate.

In step 4, utilize the grid vertex extracting to carry out Delaunay trigonometric ratio to image.

Step 4 specifically comprises the following steps:

Step 41, on the basis of the feature grid point obtaining in step 1, evenly adopts a little at random for the remaining area of imageAs grid vertex;

Step 42, in the process of trigonometric ratio, with the mutual characteristics of image line of step 1, gets adjacent top on same characteristic curvePoint line, as triangle gridding limit, utilizes the Delaunay trigonometric ratio of belt restraining to carry out triangle gridding to image, DelaunayThe algorithm of trigonometric ratio can referring to " computational geometry (the 3rd edition) the Burger work world book 2013-10-1 of publishing house publishes, ISBN: 9787510061776”。

In step 5, the energy constraint of construct image distortion.

Step 5 specifically comprises the following steps:

Step 51, (the p that utilizes step 32 to calculate₀,p₁,p₂,p₃,p₄,p₅) under corresponding new viewpoint in picture planePixel (q₀,q₁,q₂,q₃,q₄,q₅), utilizeCalculate a v_ciCorresponding some v'_ci, wherein v_ciFor formerGrid vertex on image hard constraint characteristic straight line, p_s，p_eFor v_ciTwo end points on the hard constraint characteristic straight line of place, q_s，q_eForBe respectively and p_s，p_eCorresponding point, i.e. { v_c1,v_c2,...,v_ckUnder new viewpoint O', be mapped as { v'_c1,v'_c2,...,v'_ck}，Remember that it is F that its corresponding hard constraint closes_C(v'_c1,v'_c2,...,v'_ck)=0, shows v in the time that anamorphose is shone upon_ciBe mapped to v'_ci；

Step 52, definition shape constraining: two-dimentional triangle gridding affine transformation is designated as to M:(x, y) → (x', y'), according to KeXi ?Riemann (Cauchy – Riemann) equation have:

$\frac{\∂M}{\∂x}+i\frac{\∂M}{\∂y}=0$

Wherein i is imaginary unit, about Ke Xi ?Riemann equation, and can be referring to " LSCM:Least squares conformal maps for automatic texture atlas generation,ACM Transactions on Graphics,21(3): 362 ?372. ", mapping Jacobian matrix corresponding to M has following form:

Wherein a, b value is any, in order to equate and the relation of opposite number each other between each element in representing matrix J;

Corresponding former grid is to the conversion of target gridding, and we adopt radiation conversion; Remember that former triangle is T (v_ti,v_tj,v_tk)， v_ti,v_tj,v_tkFor its three summits, after its corresponding viewpoint change, deforming triangle to be solved is T'(v'_ti,v'_tj,v'_tk)， v'_ti,v'_tj,v'_tkFor with v_ti,v_tj,v_tkThree summits one to one, A=T'T^-1The affine transformation matrix from T to T', T^-1There is following form:

$T^{-1}=\left[\begin{array}{ccc}{a}_1& b_1& d_1\\ a_2& b_2& d_2\\ a_3& b_3& d_3\end{array}\right]={\left[\begin{array}{ccc}{x}_{\mathrm{ti}}& \stackrel{}{x_{\mathrm{tj}}}& x_{\mathrm{tk}}\\ y_{\mathrm{ti}}& y_{\mathrm{tj}}& y_{\mathrm{tk}}\\ z_{\mathrm{ti}}& z_{\mathrm{tj}}& z_{\mathrm{tk}}\end{array}\right]}^{-1}---\left(2\right)$

x_ti,x_tj,x_tkFor a v_ti,v_tj,v_tkAbscissa on image, y_ti,y_tj,y_tkFor a v_ti,v_tj,v_tkOn imageOrdinate, z_ti,z_tj,z_tkFor for representing v_ti,v_tj,v_tkOrdinate, unified value is 1 here; a₁,a₂,a₃,b₁,b₂, b₃,c₁,c₂,c₃Be the Parametric Representation of each element in matrix T-1, value is all the constant calculating, pushed away by formula (1) and (2)The following energy equation of conformal projection:

E_TJ1＝a₁x'_ti+a₂x'_tj+a₃x'_tk+(b₁y'_ti+b₂y'_tj+b₃y'_tk) （3）

E_TJ2＝b₁x'_ti+b₂x'_tj+b₃x'_tk-(a₁y'_ti+a₂y'_tj+a₃y'_tk)

Wherein x'_ti,x'_tj,x'_tkBe respectively v'_ti,v'_tj,v'_tkAbscissa value, y'_ti,y'_tj,y'_tkBe respectively v'_ti, v'_tj,v'_tkOrdinate value, their gangs are obtained:

$E_S=\underset{T}{\mathrm{\Σ}}(E_{\mathrm{TJ}1}^2+E_{\mathrm{TJ}2}^2)---\left(4\right)$

Defined feature line constraint: note (v_i,v_j,v_k) be continuous three points on a characteristic curve, keepWithBetween ratio r_ljAnd anglec of rotation θ_lj, be defined as follows target equation:

$E_L=\underset{(v_{\mathrm{li}},v_{\mathrm{lj}},v_{\mathrm{lk}})}{\mathrm{\Σ}}{\left|\right|(v_{\mathrm{lk}}^{\′}-v_{\mathrm{lj}}^{\′})-r_{\mathrm{lj}}{R}_{\mathrm{lj}}(v_{\mathrm{lj}}^{\′}-v_{\mathrm{li}}^{\′})\left|\right|}^2---\left(5\right)$

Wherein $R_{\mathrm{lj}}=\left(\begin{array}{cc}\cos {\mathrm{\θ}}_{\mathrm{lj}}& -\sin {\mathrm{\θ}}_{\mathrm{lj}}\\ \sin {\mathrm{\θ}}_{\mathrm{lj}}& \cos {\mathrm{\θ}}_{\mathrm{lj}}\end{array}\right),$

Define vertical and horizontal restraint: in target image, should strive the straight line that maintenance is originally horizontal or vertical, not lose oneAs property, suppose that the mesh point on vertical line segment lv is designated as { v_lv1,v_lv2,...,v_lvm, vertical constraint can be expressed as:

$E_V=\underset{\mathrm{lv}}{\mathrm{\Σ}}\underset{\mathrm{nv}=1}{\overset{\mathrm{lvm}}{\mathrm{\Σ}}}{(x_{\mathrm{nv}}^{\′}-x_{\mathrm{lv}1}^{\′})}^2---\left(6\right),$

Wherein x'_nvRepresent some v'_nvAbscissa, x'_lv1Represent some v'_lv1Abscissa. Definition horizontal restraint: note horizontal line sectionMesh point on lh is designated as { v_lh1,v_lh2,...,v_lhm, vertical constraint is expressed as:

$E_H=\underset{\mathrm{lh}}{\mathrm{\Σ}}\underset{\mathrm{nh}=1}{\overset{\mathrm{lhm}}{\mathrm{\Σ}}}{(x_{\mathrm{nh}}^{\′}-x_{\mathrm{lh}1}^{\′})}^2---\left(7\right),$

Wherein y'_nvRepresent some v'_nvAbscissa, y_l'_h1Represent some v'_lh1Abscissa.

Specify according to user interactions, non-rigid Characteristic constraint line is appointed as in part or all of original image border, note (v_bi,v_bj,v_bk) be continuous three the feature summits on boundary characteristic line, keepWithBetween ratio r_bjDefinitionThe boundary constraint E of image_BAs follows;

$E_B=\underset{(v_{\mathrm{bi}},v_{\mathrm{bj}},v_{\mathrm{bk}})}{\mathrm{\Σ}}{\left|\right|(v_{\mathrm{bk}}^{\′}-v_{\mathrm{bj}}^{\′})-r_{\mathrm{bj}}(v_{\mathrm{bj}}^{\′}-v_{\mathrm{bi}}^{\′})\left|\right|}^2$

(8) r wherein_bjForWithBetween ratio.

Step 53, comprehensive step 51 and step 52, represent that the energy equation of mesh parameterization is as follows:

argmaxλ_SE_S+λ_LE_L+λ_VE_V+λ_HE_H+λ_BE_B （9）

s.t.F_C(v₁',v'₂,...,v'_k)＝0

Wherein λ_S,λ_L,λ_V,λ_H,λ_BFor weights corresponding to each energy term, in actual running for different picturesParameter has certain adjustment, provides one group of empirical parameter value as follows: λ through too much group experiment picture_SGet 1, λ_LWith λ_BGet 100, λ_VWith λ_HGet 10.

In step 6, utilize grid pinup picture to generate target image.

Step 6 specifically comprises the following steps:

Step 61, according to equation (9), solves and the corresponding target triangle gridding of each former triangle gridding in original image, threePicture texture in the grid of angle adopts the method for bilinear interpolation to be mapped to corresponding triangle gridding, forms target image.

Embodiment

The present embodiment for the hardware environment of testing is: Intel Core i3-2100CPU3.1GHz dominant frequency, 4G internal memory.Software environment is Microsoft Visual Studio software 2008 versions, Matlab software 2012a version and MicrosoftWindows7 operating system Ultimate. Concrete process is referring to the embodiment of Fig. 4.

This example produces the difference of mesh-density according to input picture, calculate and solve the time from 100 milliseconds to hundreds of millisecondNot etc. As shown in Figure 4, first method of the present invention carries out alternately, adding heavy line and having marked obtain alternately hidden to input pictureContaining square phosphor bodies structure, all the other dotted lines are the characteristic curve of the manual mark of user and the image boundary of appointment; At interaction figure baseOn plinth, adopt the Delaunay method of belt restraining to carry out triangle gridding to image on the one hand, simulate by foundation on the other handThreedimensional model then re-projection obtains the square phosphor bodies structure of projection under new viewpoint; In conjunction with mesh parameterization energy term and heavily throwingThe object construction constraint solving that shadow obtains goes out target gridding image; The grid image generating is carried out border editor and newly lookedResult images under point. Fig. 5 has shown that the result and the Lightroom of Adobe Systems Inc. that adopt method of the present invention to generate are soft in additionThe effect contrast figure of part uses circle to enclose the flaw of mark in Lightroom result, and the inventive method has been avoided this wellThe generation of class situation; Fig. 6 has shown the result that the inventive method is observed scene with multiple new viewpoints in the sphere ken, centerFigure be input picture, and remaining 12 width scenes are to play up the result images of generation by each self-corresponding viewpoint, pass through viewpointInterpolation and then obtain grid vertex position interpolation, having calculated after crucial viewpoint structure, can utilize grid interpolation sideMethod generates the scene graph of different points of view under the current sphere ken in real time.

Method of the present invention is using a real scene shooting image as input, by the simple mutual main body with calculating Recovery imageLine three-dimensional structure, adopts the method for anamorphose to generate the brand-new image after viewpoint change, has ensured the true of result imagesProperty has ensured the physics reasonability of result simultaneously, in addition because the inventive method does not need to carry out accurate 3 D scene rebuilding,The efficiency of method is high.

Claims (7)

1. the image viewpoint change method based on single width input picture, is characterized in that, comprises the following steps:

Step 1, image is mutual: extract the cube structure in image, wherein six corresponding image projection of point of square body of markPixel, and obtain alternately the characteristic curve in image, characteristic curve comprises the characteristic curve that affects picture material, horizontal and vertical is approximatelyBunch section and specify image boundary line;

Step 2, utilizes pinhole camera image-forming principle to rebuild a three-dimensional scenic in conjunction with the spatial character of cube structure object, is lookingWhen point transformation, by this three-dimensional scene structure Reality simulation three-dimensional scene structure, in hypothesis three-dimensional scenic, cube structure whereinOn the basis that the picture point on image overlaps with it, a summit, the geometrical constraint that the side's of utilization body comprises, calculates in step 1The three dimensions point coordinates of six some correspondences;

Step 3, utilizes the square body corresponding to space coordinates of six points that under new viewpoint, re-projection step 2 calculates, and representsThe three-dimensional scenic of rebuilding: using the central point of image as step 2 reconstruction of three-dimensional scene center, be designated as O; Taking O as the centre of sphere, this sceneCenter is to the distance of camera | in the ball plane that OF| is radius, define a new viewpoint O', in the three-dimensional scenic that step 2 is rebuildCube structureAgain project to picture plane corresponding to new viewpoint;

Step 4, image triangle gridding: according to the result of step 1, utilize the Delaunay trigonometric ratio of belt restraining to carry out imageTriangle gridding;

Step 5, mesh parameterization: utilize conformal projection principle and feature to keep constraint to set up the equation of mesh parameterization, profitThe hard constraint that obtains scene side's body framework structure projection mapping with step 3 re-projection, solves parametrization energy equation, obtains netLattice;

Step 6, target image generates: solve the grid obtaining according to step 5, by net corresponding with target gridding in original imageCheck reason utilizes bilinear interpolation method to be mapped to one by one target gridding, generates target image.

2. a kind of image viewpoint change method based on single width input picture according to claim 1, described step 1 toolBody comprises the following steps:

Step 11, described single width input picture, implicit cube structure in its scene, mark out alternately this cube structure at imageIn projection, with six point (p₀,p₁,p₂,p₃,p₄,p₅) expression cube structure, wherein (p₀,p₁,p₂,p₃) four points are correspondingSpace three-dimensional planar quadrature is in (p₀,p₁,p₅,p₄) four corresponding three-dimensional planars of point, p₀p₁Corresponding space line is two flatThe intersection of face;

Step 12, the cube structure (p obtaining according to step 11₀,p₁,p₂,p₃,p₄,p₅), by cube structure (p in image₀,p₁, p₂,p₃,p₄,p₅) between the line that comprises be all labeled as hard constraint characteristic curve, on these hard constraint characteristic curves, extract equably threeAngle grid vertex, concrete grammar is as follows: to a specific hard constraint characteristic curve, start to get a little from one end, along rectilinear direction every25 pixel distances are got the next point on this straight line, until the other end of straight line, for getting on all hard constraint characteristic curvesThe grid vertex obtaining, is designated as { v_c1,v_c2,...,v_ck, the hard constraint grid vertex sum that wherein ck representative is extracted;

Step 13, obtains the characteristic straight line in image alternately, comprises the Eigenvector that affects picture material, horizontal and vertical constraintEvenly get a little on described characteristic straight line line segment and specify image boundary line, and concrete grammar is as follows: straight to a special characteristicLine, starts to get a little from one end, gets the next point on this straight line along rectilinear direction every 25 pixel distances, until straight line is anotherOne end;

Step 14 for the indicatrix in image, is extracted triangulation network lattice point successively on curve, and concrete grammar is as follows: from spyThe one end of levying curve starts, along indicatrix prolonging direction, using 25 pixel distances as getting an interval, until arrive featureThe other end of curve.

3. a kind of image viewpoint change method based on single width input picture according to claim 2, described step 2 toolBody comprises the following steps:

Step 21, taking picture centre as initial point O (0,0,0), O to camera loca F (0,0, f) be Z axis forward, the plane of delineationTransverse axis and y direction are that XY direction of principal axis is set up three-dimensional world coordinate system, and wherein f is camera focus;

Step 22, finds out respectively plane (p₀,p₁,p₂,p₃) and plane (p₀,p₁,p₅,p₄) the vanishing point c that forms on as plane₁With c₂, obtain according to vanishing point scaling algorithmObtain focal distance f so solve;

Step 23, the cube structure in note true three-dimension scene is corresponding to the cube structure (p of six points₀,p₁,p₂,p₃,p₄,p₅)Point be cube structure (Q₀,Q₁,Q₂,Q₃,Q₄,Q₅), have according to pin-hole imaging model:WhereinRepresentation vectorWithBetween ratio; For in simulated scenario and cube structure (p six points₀,p₁,p₂,p₃, p₄,p₅) corresponding cube structureFormulaSet up wherein r_kGenerationTable vectorWithBetween ratio, fixing r₀Value be 1, utilize WithSolve r₁,r₂,r₃,r₄,r₅；

Step 24, the r obtaining according to step 23₀,r₁,r₂,r₃,r₄,r₅, calculate in simulated scenario corresponding to cube structure (p₀, p₁,p₂,p₃,p₄,p₅) cube structureSetWith p₀Overlap, definition scene center is interludeSummit in cube structure in scapeSubpoint on primary optical axis is initial point O herein, and simulated scenario center is dropped on initial point,Corresponding to Q in real scene₀Subpoint on primary optical axis, i.e. real scene center O_Q。

4. a kind of image viewpoint change method based on single width input picture according to claim 3, described step 3 toolBody comprises the following steps:

Step 31, obtains cube structure according to step 24Taking O as the center of circle | OF| is as radiusOn sphere, get at random a new viewpoint O', whereinWithAngle be no more than 45 degree, through simulated scenario center O structurePerpendicular to the planar I of OO' ', try to achieve cube structureWith the line of new viewpoint O', remember itself and planeThe new intersection point of I' is (p'₀,p′₁,p'₂,p'₃,p'₄,p'₅), complete the throwing of cube structure under new viewpoint O' in simulated scenarioShadow;

Step 32, by normal vectorObtain unit vector x' with the multiplication cross of vertical vector (0,1,0), by x' withMultiplication cross obtainsTo unit vector y', on plane of delineation I' taking O as initial point, vector x ' direction is abscissa forward, vectorial y' direction is sat for verticalMark forward is set up image two-dimensional coordinate system, willProject to respectively vector x ', on y' both direction, obtainIntersection point (p'₀,p′₁,p'₂,p'₃,p'₄,p'₅) two-dimensional coordinate on plane of delineation I', be designated as point (q₀,q₁,q₂,q₃,q₄,q₅)，Obtain in original image six points on cube structure and under new viewpoint, be projected in the two-dimensional coordinate on target image.

5. a kind of image viewpoint change method based on single width input picture according to claim 4, described step 4 toolBody comprises the following steps:

Step 41, on the basis of the triangulation network lattice point obtaining in step 1, evenly adopts a conduct at random for the remaining area of imageGrid vertex;

Step 42, in the process of trigonometric ratio, with the mutual characteristics of image line of step 1, gets adjacent vertex on same characteristic curve and connectsLine, as triangle gridding limit, utilizes the Delaunay trigonometric ratio of belt restraining to carry out triangle gridding to image.

6. a kind of image viewpoint change method based on single width input picture according to claim 5, described step 5 toolBody comprises the following steps:

Step 51, the two-dimensional coordinate point (q that utilizes step 32 to obtain₀,q₁,q₂,q₃,q₄,q₅), according to formulaCalculate a v_ciCorresponding some v'_ci, wherein v_ciFor the grid vertex on original image hard constraint characteristic straight line, p_s，p_eFor v_ciTwo end points on the hard constraint characteristic straight line of place, q_s，q_eFor being respectively and p_s，p_eCorresponding point, i.e. { v_c1,v_c2,...,v_ck}Under new viewpoint O', be mapped as { v'_c1,v'_c2,...,v'_ck, remember that it is F that its corresponding hard constraint closes_C(v'_c1,v'_c2,..., v'_ck)=0, shows v in the time that anamorphose is shone upon_ciBe mapped to v'_ci；

Step 52, definition shape constraining: two-dimentional triangle gridding affine transformation is designated as to M:(x, y) → (x', y'), according to Cauchy-Riemann's equation:

$\frac{\∂M}{\∂x}+i\frac{\∂M}{\∂y}=0,$

Wherein i is imaginary unit, and Jacobian matrix corresponding to mapping M has following form:

$J=\left[\begin{array}{cc}a& b\\ b& -a\end{array}\right]---\left(1\right),$

Wherein a, b value is any, in order to equate and the relation of opposite number each other between each element in representing matrix J;

Corresponding former grid, to the conversion of target gridding, adopts affine transformation; Remember that former triangle is T (v_ti,v_tj,v_tk)，v_ti,v_tj, v_tkFor its three summits, after its corresponding viewpoint change, deforming triangle to be solved is T'(v'_ti,v'_tj,v′_tk)，v'_ti,v'_tj, v′_tkFor with v_ti,v_tj,v_tkThree summits one to one, A=T'T^-1The affine transformation matrix from T to T', T^-1Have as followsForm:

$T^{-1}=\left[\begin{array}{ccc}{a}_1& b_1& d_1\\ a_2& b_2& d_2\\ a_3& b_3& d_3\end{array}\right]={\left[\begin{array}{ccc}{x}_{ti}& x_{tj}& x_{tk}\\ y_{ti}& y_{tj}& y_{tk}\\ z_{ti}& z_{tj}& z_{tk}\end{array}\right]}^{-1}---\left(2\right),$

Wherein, x_ti,x_tj,x_tkFor a v_ti,v_tj,v_tkAbscissa on image, y_ti,y_tj,y_tkFor a v_ti,v_tj,v_tkAt imageOn ordinate, z_ti,z_tj,z_tkFor for representing v_ti,v_tj,v_tkOrdinate, unified value is 1 here; a₁,a₂,a₃,b₁,b₂, b₃,c₁,c₂,c₃It is matrix T^-1In the Parametric Representation of each element, obtained the following energy side of conformal projection by formula (1) and formula (2)Journey:

$\begin{array}{c}{E}_{TJ1}=a_1{x}_{ti}^{\′}+a_2{x}_{tj}^{\′}+a_3{x}_{tk}^{\′}+(b_1{y}_{ti}^{\′}+b_2{y}_{tj}^{\′}+b_3{y}_{tk}^{\′})\\ E_{TJ2}=b_1{x}_{ti}^{\′}+b_2{x}_{tj}^{\′}+b_3{x}_{tk}^{\′}-(a_1{y}_{ti}^{\′}+a_2{y}_{tj}^{\′}+a_3{y}_{tk}^{\′})\end{array}---\left(3\right),$

Wherein x'_ti,x'_tj,x'_tkBe respectively v'_ti,v'_tj,v′_tkAbscissa value, y'_ti,y'_tj,y'_tkBe respectively v'_ti,v'_tj, v′_tkOrdinate value, their gangs are obtained:

$E_S=\underset{T}{\Σ}(E_{TJ1}^2+E_{TJ2}^2)---\left(4\right)$

Defined feature line constraint: note (v_li,v_lj,v_lk) be continuous three points on a characteristic curve, keepWithBetweenRatio r_ljAnd anglec of rotation θ_lj, be defined as follows target equation:

$E_L=\underset{(v_{li},v_{lj},v_{lk})}{\Σ}\left|\right|(v_{lk}^{\′}-v_{lj}^{\′})-r_{lj}{R}_{lj}(v_{lj}^{\′}-v_{li}^{\′})|{|}^2---\left(5\right),$

Wherein

Definition vertical constraint: remember that the mesh point on vertical line segment lv is designated as { v_lv1,v_lv2,...,v_lvm, vertical constraint is expressed as:

$E_V=\underset{lv}{\Σ}\underset{nv=1}{\overset{lvm}{\Σ}}{(x_{nv}^{\′}-x_{lv1}^{\′})}^2---\left(6\right),$

Wherein x'_nvRepresent some v'_nvAbscissa, x'_lv1Represent some v'_lv1Abscissa;

Definition horizontal restraint: the mesh point on note horizontal line section lh is designated as { v_lh1,v_lh2,...,v_lhm, vertical constraint is expressed as:

$E_H=\underset{lh}{\Σ}\underset{nh=1}{\overset{lhm}{\Σ}}{(y_{nh}^{\′}-y_{lh1}^{\′})}^2---\left(7\right),$

Wherein y'_nvRepresent some v'_nvAbscissa, y'_lh1Represent some v'_lh1Abscissa;

Specify according to user interactions, non-rigid Characteristic constraint line is appointed as in part or all of original image border, note (v_bi,v_bj, v_bk) be continuous three the feature summits on boundary characteristic line, keepWithBetween ratio r_bjThe limit of definition imageBound constrained E_BAs follows;

$E_B=\underset{(v_{bi},v_{bj},v_{bk})}{\Σ}\left|\right|(v_{bk}^{\′}-v_{bj}^{\′})-r_{bj}(v_{bj}^{\′}-v_{bi}^{\′})|{|}^2---\left(8\right),$

Wherein r_bjForWithBetween ratio;

Step 53, according to step 51 and step 52, represents that the energy equation of mesh parameterization is as follows:

$\begin{array}{c}\arg {\mathrm{max\λ}}_S{E}_S+{\mathrm{\λ}}_L{E}_L+{\mathrm{\λ}}_V{E}_V+{\mathrm{\λ}}_H{E}_H+{\mathrm{\λ}}_B{E}_B\\ s.t.F_C(v_{c1}^{\′},v_{c2}^{\′},\mathrm{...},v_{ck}^{\′})=0\end{array}---\left(9\right),$

Wherein λ_S,λ_L,λ_V,λ_H,λ_BFor weights corresponding to each energy term, λ_SGet 1, λ_LWith λ_BGet 100, λ_VAnd λ_HGet 10.

7. a kind of image viewpoint change method based on single width input picture according to claim 6, described step 6 toolBody comprises the following steps:

Step 61, according to formula (9), solves and the target triangle gridding that in original image, each triangle gridding is corresponding, by former trianglePicture texture in grid adopts the method for bilinear interpolation to be mapped to corresponding target triangle gridding, forms target image.