CN1917658B - Method for generating sequence of stereo images from monocular image sequence - Google Patents

Abstract

The method comprises: first, using monocular image array to build its corresponding monocular camera parameter array, and then to build a binocular camera array, and its corresponding parameter array and index array; then, adjusting the values of the binocular parameter array and the values of index value, and controlling the error in allowable range; finally, according to the index array, building a binocular image array, and then according to the parameters array of binocular camera, making view transformation for the binocular source image to get 3D image array.

Description

A kind of method that generates sequence of stereoscopic images from monocular image sequence

Technical field

The present invention relates to general video data three-dimensional method, relate in particular to a kind of method that generates sequence of stereoscopic images from monocular image sequence.

Technical background

From the monocular video of no three-dimensional geometric information structure binocular tri-dimensional video, three kinds of main method are arranged: a kind ofly be to use synchronous stereographic hardware; A kind ofly be based on dense three-dimensional geometry image restored method for drafting; A kind of image rendering technique that is based on dense sampling.This several method all has its limitation, and highly automated, accurate dense three-dimensional reconstruction remains the difficult problem in the computer vision.

Drafting based on video utilizes a plurality of synchronization videos to generate 3 D video, needs special hardware device or reconstruction of three-dimensional model.

Early stage work mainly is the three-dimensional geometric information that utilizes scene.Yet the threedimensional model of reality scene is difficult to obtain usually.Also the someone has proposed the three-dimensional rebuilding method based on two width of cloth or multiple image, yet these class methods automatically still can not obtain ideal results, needs manually to get involved the cost height.

Some methods of drawing based on image are avoided three-dimensional reconstruction by dense sampling, also can synthetic stereo image.As Light-field, can be with reference to M.Levoy and P.Hanrahan.Lightfieldrendering.In SIGGRAPH ' 96:Proceedings of the 23rd annual conference onComputer graphics and interactive techniques, pages 31.42, New York, NY, USA, 1996.ACM Press; Lumigraph, can be with reference to S.J.Gortler, R.Grzeszczuk, R.Szeliski, and M.F.Cohen.The lumigraph.In SIGGRAPH ' 96:Proceedingsof the 23rd annual conference on Computer graphics and interactivetechniques, pages 43.54, New York, NY, USA, 1996.ACM Press.The method that adopts all is to use a plurality of video cameras from each angle scene to be carried out intensive sampling, synthetic stereo image under without any the condition of scene three-dimensional geometric information.This method desired data amount is big, complex disposal process.

Summary of the invention

The objective of the invention is at the deficiencies in the prior art, propose a kind of method from monocular image sequence generation sequence of stereoscopic images, this method does not need hardware to assist, need not to recover the degree of depth and three-dimensional geometry just can construct sequence of stereoscopic images from the monocular video image sequence.

In order to achieve the above object, the technical solution used in the present invention is as follows:

A kind of method from monocular image sequence generation sequence of stereoscopic images may further comprise the steps:

1) by monocular image sequence P _Mono={ p _iThe corresponding monocular-camera argument sequence (Q of establishment _B, V _B): monocular-camera sequence B={ b _iArgument sequence comprise rotating vector sequence Q _B={ q _iAnd locus sequence V _B={ v _i, obtain each frame monocular image p by the video camera track algorithm _iCorresponding camera parameters (q _Bi, v _Bi);

2) by monocular-camera argument sequence (Q _B, V _B) and monocular image sequence P _MonoStructure binocular camera sequence S={s _iAnd corresponding parameters sequence (Q _S, V _S) and index sequence (∏ ^L, ∏ ^R): make left eye index sequence ∏ ^L=i is then to choosing monocular-camera b _iDistance near binocular apart from d _EyeMonocular-camera b _k, make ∏ ^R[i]=k; Binocular camera rotating vector q (s _i)=(q (b _i)+q (b _k)/2, center c (s _i)=(v (b _i)+v (b _k))/2, can determine binocular camera right and left eyes parameter (q (s thus _i), v (s _i));

3) with known (Q _S, V _S) and (∏ ^L, ∏ ^R) the calculation control error E, if E within the error allowed band, then directly carries out next step, otherwise, (Q adjusted _S, V _S) and (∏ ^L, ∏ ^R) value E is decreased within the error allowed band; Method of adjustment is in two steps: at first, and fixing (∏ ^L, ∏ ^R), adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S), then, fixing (Q _S, V _S), adjust (∏ ^L, ∏ ^R), obtain making (the ∏ of E minimum ^L, ∏ ^R); Repeating these two steps reduces within the error allowed band until E;

4) according to index sequence (∏ ^L, ∏ ^R) structure binocular sequence of source images (P _Lwarp, P _Rwarp), left eye sequence of source images wherein $P_{\mathrm{lwarp}}=\left\{p_{{\mathrm{\Π}}^L\left[i\right]}\right\},$ The right eye sequence of source images $P_{\mathrm{rwarp}}=\left\{p_{{\mathrm{\Π}}^R\left[i\right]}\right\},$ According to binocular camera argument sequence (Q _S, V _S), to binocular sequence of source images (P _Lwarp, P _Rwarp) carry out view transformation, obtain sequence of stereoscopic images (P _Lstereo, P _Rstereo).

Described departure E (S, ∏ ^L, ∏ ^R)=E _S+ E _Q+ (E _CV+ E _CO), comprise following components:

1) third dimension error E _S: L (s) refers to the left-eye camera of binocular camera s, and R (s) refers to the right-eye camera of binocular camera s:

δ(s，l，r)＝||v(b _l)-v(L(s))|| ²+||v(b _r)-v(R(s)|| ²

$E_S(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\left(\mathrm{\δ}\left(S\right[i],{\mathrm{\Π}}^L[i],{\mathrm{\Π}}^R[i\left]\right)\right)$

2) similitude error E _Q:

γ(，s，l，r)＝||q(s)-q(b _l)|| ²+||q(s)-q(b _r)|| ²

$E_Q(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\mathrm{\γ}\left(S\right[i],{\mathrm{\Π}}^l[i],{\mathrm{\Π}}^r[i\left]\right)$

3) direction continuity error E _CO:

$E_{\mathrm{CQ}}(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}\left|\right|2q\left(s_i\right)-q\left(s_{i+1}\right)-q\left(s_{i-1}\right){\left|\right|}^2$

4) position continuity error E _CV:

$E_{\mathrm{CV}}=(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(s_i\right)-v\left(s_{i+1}\right)-v\left(s_{i-1}\right)\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^L\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^L[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^L[i-1]}\right)\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^R\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^R[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^R[i-1]}\right)\left|\right|}^2$

Described fixing (∏ ^L, ∏ ^R), adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S) method can be: fixing (∏ ^L, ∏ ^R), be target function with E, use the nonlinear optimization method to adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S);

Described fixing (Q _S, V _S), adjust (∏ ^L, ∏ ^R), obtain making (the ∏ of E minimum ^L, ∏ ^R) method can be: fixing (Q _S, V _S), with (∏ ^L, ∏ ^R) segmentation, each all successive frames in adjusting wherein a certain section simultaneously, each ∏ simultaneously ^L[i] ∏ ^R[i] all enumerates pairing in the some frames that close on separately.All situations are recomputated departure E, find out a group of making the E minimum as adjusting the result.Each segmentation is adjusted one by one, finished until whole adjustment.

The present invention compares with background technology, and the beneficial effect that has is:

The present invention is directed to three-dimensional video-frequency manufacture method costliness in the past, complexity, data volume is big, and shortcomings such as processing time length have proposed a kind of new approaches that common monocular video image sequence are converted into sequence of stereoscopic images.

The present invention does not need special hardware, or based on the intensive sampling and the three-dimensional modeling of image, do not need the recovery of depth information, only need one section monocular video image sequence that comprises the motion camera lens, can construct and make the required sequence of stereoscopic images of three-dimensional video-frequency.

Computational efficiency height of the present invention, the result is stable, and the sequence of stereoscopic images that is produced by this method can be used for making purposes such as three-dimensional video-frequency.Because the monocular video material is abundant, therefore be suitable for occasion with the monocular video three-dimensional.

Description of drawings

The invention will be further described below in conjunction with drawings and embodiments.

Fig. 1 is the flow chart of four steps of the inventive method;

Fig. 2 is the schematic diagram of monocular-camera b and monocular-camera sequence B in the step 1;

Fig. 3 is the structural representation of binocular camera s in the step 2;

Fig. 4 is index sequence ∏ in the step 2 ^L, ∏ ^RThe purposes schematic diagram;

Fig. 5 is the binocular camera sequence in the step 2 and the initial method of index camera sequence.

Embodiment

The present invention proposes a kind of method, comprise four steps: at first, create corresponding monocular-camera argument sequence by monocular image sequence from monocular image sequence generation sequence of stereoscopic images; Then construct binocular camera sequence, corresponding parameters sequence and index sequence; Then, adjust the value of binocular camera argument sequence and index sequence, make departure within the error allowed band; At last,, the binocular sequence of source images is carried out view transformation, obtain sequence of stereoscopic images according to the binocular camera argument sequence according to index sequence structure binocular sequence of source images.

Idiographic flow such as Fig. 1 are represented, the existing realization details of specifically introducing each step:

1) monocular image sequence P _Mono={ p _iThe corresponding monocular-camera argument sequence (Q of establishment _B, V _B) this paper indication camera parameters comprises two the external parameter locus v and the rotating vector q of video camera sometime, rotating vector q Eulerian angles formal representation wherein is as (q _x, q _y, q _z), q _x, q _y, q _zBe respectively around x y, the Eulerian angles of z axle rotation.Video camera parameter in a period of time constitutes camera parameters sequence space position sequence V and rotating vector sequence Q.

As Fig. 2, monocular-camera b represents the video camera of certain frame monocular video image correspondence.B refers to the sequence of monocular-camera b, B[i] refer to i monocular-camera, i.e. b _iV (b) is the locus of monocular-camera b, and q (b) is the rotating vector of monocular-camera b.

Monocular-camera sequence B={ b _iArgument sequence comprise rotating vector sequence Q _B={ q _BiAnd locus sequence V _B={ v _Bi.

Obtain each frame monocular image p by the video camera track algorithm _iCorresponding camera parameters (q _Bi, v _Bi).

There are many kinds to can be used for the algorithm that video camera is followed the tracks of, general employing is non real-time based on structure and exercise recovery (Structure And Motion Recovery, abbreviation SMR) video camera tracking method can recover camera motion accurately to most sequence of video images.May be used to video camera as the boujou of 2D3 company, the matchmover of REALVIZ company or the WecamTrack of ranovae company etc. follows the tracks of.

This step obtains (Q _B, V _B) and the three dimensional depth of some sparse points, in step 4, can get the degree of depth in the middle of these sparse points as the parameter of view transformation.

2) by monocular-camera argument sequence (Q _B, V _B) and monocular image sequence P _MonoStructure binocular camera sequence S={s _iAnd corresponding parameters sequence (Q _S, V _S) and index sequence (∏ ^L, ∏ ^R):

As Fig. 3, binocular camera s refers to be used to generate the system that two video cameras of stereo-picture are formed, and be made of identical rotating vector and the constant synchronous right and left eyes video camera of distance, and two video camera rotating vectors is vertical with the video camera line.V (L (s)) wherein, v (R (s)) is the locus of the right and left eyes video camera of stereo camera; L (s) and the mid point c (s) of R (s) are called the binocular camera center; The right and left eyes video camera is called video camera binocular distance apart from d; And the rotating vector of definition L (s) and R (s) is the rotating vector q (s) of s; L (s) is a binocular camera x axle with R (s) line, e _xBe the unit vector of x axle, q (s) and e _xVertically.S refers to the sequence of binocular camera s, S[i] refer to i binocular camera, i.e. s _i

∏ refers to index sequence, ∏ ^L, ∏ ^RBe divided into left eye index sequence and right eye index sequence.A certain ∏ of left eye index sequence ^L[i] refers to the pairing monocular-camera B[k of each left-eye camera L (S[i]) of binocular camera sequence S] index k in the monocular-camera sequence; A certain ∏ of right eye index camera sequence ^R[i] refers to the pairing monocular-camera B[m of each right-eye camera R (S[i]) of binocular camera sequence] index m in the monocular-camera sequence.

As Fig. 4, that choose on request from the monocular-camera sequence is b ₂₇, b ₁, b ₃₀, b ₉Four video cameras, wherein b ₂₇, b ₁Corresponding monocular video frame f ₂₇, f ₁Will be with generating binocular camera S[i] the right and left eyes image, b ₃₀, b ₉Corresponding monocular video frame f ₃₀, f ₉Be used to generate binocular camera S[i+1] the right and left eyes image.∏ then ^L=27,30}, ∏ ^R=1,9}, B ^L={ b ₂₇, b ₃₀, B ^R={ b ₁, b ₉.

Structure right and left eyes index sequence ∏ ^L, ∏ ^R: as Fig. 5, make ∏ ^L[i]=i, promptly the left eye index sequence is the sequence number of monocular-camera sequence, and then is right eye index sequence selection initial value.Consider i left-eye camera, its corresponding monocular-camera is b _i, choose b _iDistance near binocular apart from d _EyeMonocular-camera b _k, with the index of k as corresponding right-eye camera, i.e. ∏ ^R[i]=k.

The parameter of binocular camera sequence S: binocular camera rotating vector q (s _i) initial value is exactly b _iAnd b _kThe mean value q (s of rotating vector _i)=(q (b _i)+q (b _k)/2; Center c (s _i) be b _iAnd b _kThe mid point of locus, c (s _i)=(v (b _i)+v (b _k)/2.Then the locus of corresponding binocular camera right and left eyes video camera is respectively c (s _i) ± 0.5d _EyeE _x, promptly on the x axle with mid point at a distance of d _EyeTwo spatial point of/2, rotating vector is q (s _i).e _xBe the unit vector of x axle (we the front-left of video camera to as x axle positive direction).

Binocular camera s _iRemove q, other inner parameter that v is outer such as focal distance f etc. are with its corresponding monocular-camera B of structure ^L[i] or B ^R[i] identical (supposing not zoom of video camera here).

3) adjust binocular camera argument sequence (Q _S, V _S) and index sequence (∏ ^L, ∏ ^R) value, make departure E within the error allowed band: with known (Q _S, V _S) and (∏ ^L, ∏ ^R) the calculation control error E, if E within the error allowed band, then directly carries out next step; Otherwise, adjust (Q _S, V _S) and (∏ ^L, ∏ ^R) value E is decreased within the error allowed band.

Departure E (S, ∏ ^L, ∏ ^R)=E _S+ E _Q+ (E _CV+E _CO);

The third dimension error E _S:

δ(s，l，r)＝||v(b _l)-v(L(s))|| ²+||v(b _r)-v(R(s))|| ²

$E_S(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\left(\mathrm{\δ}\left(S\right[i],{\mathrm{\Π}}^L[i],{\mathrm{\Π}}^R[i\left]\right)\right)$

The similitude error E _Q:

γ(s，l，r)＝||q(s)-q(b _l)|| ²+||q(s)-q(b _r)|| ²

$E_Q(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\mathrm{\Σ}}\mathrm{\γ}\left(S\right[i],{\mathrm{\Π}}^l[i],{\mathrm{\Π}}^r[i\left]\right)$

The continuity error E _C, comprise direction continuity target function E _CQ, spatial continuity target function E _CV:

$E_{\mathrm{CQ}}=(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}\left|\right|2q\left(s_i\right)-q\left(s_{i+1}\right)-q\left(s_{i-1}\right){\left|\right|}^2$

$E_{\mathrm{CV}}=(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}\left|\right|2v\left(s_i\right)-v\left(s_{i+1}\right)-v\left(s_{i-1}\right){\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^L\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^L[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^L[i-1]}\right)\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^R\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^R[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^R[i-1]}\right)\left|\right|}^2$

The allowed band of departure E is an empirical, is used to control the effect of last output stereo-picture, if the output result is accurate inadequately, and can be by dwindling this scope, by adjusting (Q _S, V _S) and (∏ ^L, ∏ ^R) value the result is optimized refinement.

Adjust (Q _S, V _S) and (∏ ^L, ∏ ^R) method have a lot, the simplest strategy is that to enumerate institute possible, but because the combination that enumerates is too much, has increased computation complexity greatly.Therefore, adjustment can be divided into two parts: at first, fixing (∏ ^L, ∏ ^R), adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S); Then, fixing (Q _S, V _S), adjust (∏ ^L, ∏ ^R), obtain making (the ∏ of E minimum ^L, ∏ ^R); Repeat these two steps and change not quite, as (E until E _i-E _I-1)/E _iCan stop in＜0.0001 o'clock.

It is very big to adopt the amount of calculation of dividing two parts to enumerate all combinations to be still, and therefore, the actual strategy that adopts optimization algorithm and local enumerative technique to combine reduces respectively and adjusts (Q _S, V _S) and (∏ ^L, ∏ ^R) computation complexity.

For example, at first, fixing (∏ ^L, ∏ ^R), be target function with E, use nonlinear optimization method such as Levenberg-Marquardt method to adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S).

The local enumerative technique adjustment of segmentation (∏ then ^L, ∏ ^R): fixing (Q _S, V _S), with (∏ ^L, ∏ ^R) segmentation, the front and back section has repeat element, adjusts (∏ simultaneously at every turn ^L, ∏ ^R) in all successive frames in a certain section, each ∏ simultaneously ^L[i], ∏ ^R[i] all enumerates pairing in several frames that close on separately.All situations are recomputated departure E, find out a group of making the E minimum as adjusting the result.Each segmentation is adjusted one by one, finished until whole adjustment.

The local enumerative technique of segmentation is exemplified below: suppose that employing adjusts continuous three frames simultaneously, the adjusting range of each frame is the strategy that comprises oneself 5 frame.For first group of controlled three frame S[0], S[1], S[2] and, its corresponding right and left eyes index is (∏ ^l[0], ∏ ^r[0]), (∏ ^l[1], ∏ ^r[1]), (∏ ^l[2], ∏ ^r[2]), suppose that wherein the corresponding right and left eyes index of binocular camera of the 0th frame is ∏ ^L[0]=31, ∏ ^R[0]=9, the 0th frame left eye index adjusting range ∏ so ^L[0]=29,30,31,32,33}, right eye index adjusting range ∏ ^R[0]=7,8,9,10,11}, other frames are in like manner.For each frame totally 25 groups of (∏ ^l[i], ∏ ^r[i]) combination, continuous three frame (∏ ^l[0], ∏ ^r[0]), (∏ ^l[1], ∏ ^r[1]), (∏ ^l[2], ∏ ^r[2]) always have 25 ³Plant combination.If find have certain combination can make general objective function E minimum, then get this value therein.Adjust so successively, adjust 0,1,2 frames simultaneously earlier, then adjust 1,2,3 frames again, finish until whole adjustment.

4) according to index sequence (∏ ^L, ∏ ^R) structure binocular sequence of source images (P _Lwarp, P _Rwarp), according to binocular camera argument sequence (Q _S, V _S), to binocular sequence of source images (P _Lwarp, P _Rwarp) carry out view transformation, obtain sequence of stereoscopic images (P _Lstereo, P _Rstero).

The method of view transformation is to give a normal degree of depth Z for each pixel on the image _c, each pixel is projected to earlier on the three-dimensional tram, and then project on the plane of delineation of binocular camera parameter appointment, thus the right and left eyes image of generation stereo-picture.The z here _c=2 (1/z _Min+ 1/z _Max) ^-1, [z wherein _Min, z _Max] be the depth bounds of scene, in the video camera tracking step, when recovering camera motion, can obtain the three dimensional depth of some sparse points, can get the central minimum-depth of these sparse points as z _Min, depth capacity is as z _Max

Claims (3)

1. one kind generates the method for sequence of stereoscopic images from monocular image sequence, it is characterized in that may further comprise the steps:

1) by monocular image sequence P _Mono={ p _iThe corresponding monocular-camera argument sequence (Q of establishment _B, V _B): monocular-camera sequence B={ b _iArgument sequence comprise rotating vector sequence Q _B={ q _iAnd locus sequence V _B={ v _i, obtain each frame monocular image p by the video camera track algorithm _iCorresponding camera parameters (q (b _i), v (b _i));

2) by monocular-camera argument sequence (Q _B, V _B) and monocular image sequence P _MonoStructure binocular camera sequence S={s _iAnd corresponding parameters sequence (Q _S, V _s) and index sequence (∏ ^L, ∏ ^R): make left eye index sequence ∏ ^L=i is then to choosing monocular-camera b _iDistance near binocular apart from d _EyeMonocular-camera b _k, make ∏ ^R[i]=k; Binocular camera rotating vector q (s _i)=(q (b _i)+q (b _k))/2, center c (s _i)=(v (b _i)+v (b _k))/2, can determine binocular camera right and left eyes parameter (q (s thus _i), c (s _i));

Described binocular camera: refer to be used to generate the system that two video cameras of stereo-picture are formed, constitute by identical rotating vector and the constant synchronous right and left eyes video camera of distance, and two video camera rotating vectors are vertical with the video camera line; Described binocular distance: refer to right and left eyes video camera distance;

3) with known (Q _S, V _S) and (∏ ^L, ∏ ^R) the calculation control error E, if E within the error allowed band, then directly carries out next step, otherwise, (Q adjusted _S, V _S(∏ ^L, ∏ ^R) value E is decreased within the error allowed band; Method of adjustment is in two steps: at first, and fixing (∏ ^L, ∏ ^R), adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S), then, fixing (Q _S, V _S), adjust (∏ ^L, ∏ ^R), obtain making (the ∏ of E minimum ^L, ∏ ^R); Repeating these two steps reduces within the error allowed band until E;

4) according to index sequence (∏ ^L, ∏ ^R) structure binocular sequence of source images (P _Lwarp, P _Rwarp), left eye sequence of source images wherein The right eye sequence of source images According to binocular camera argument sequence (Q _S, V _S), to binocular sequence of source images (P _Lwarp, P _Rwarp) carry out view transformation, obtain sequence of stereoscopic images (P _Lstereo, P _Rstereo); The method of view transformation is to give a normal degree of depth Z for each pixel on the image _c, each pixel is projected to earlier on the three-dimensional tram, and then project on the plane of delineation of binocular camera parameter appointment, thus the right and left eyes image of generation stereo-picture; The z here _c=2 (1/z _Mim+ 1/z _Max) ^-1, [z wherein _Min, z _Max] be the depth bounds of scene, in the video camera tracking step, when recovering camera motion, can obtain the three dimensional depth of some sparse points, get the central minimum-depth of these sparse points as z _Min, depth capacity is as z _Max

Described departure E (S, ∏ ^L, ∏ ^R)=E _S+ E _Q+ (E _CV+ E _CQ), comprise following components:

(1) third dimension error E _S: L (s) refers to the left-eye camera of binocular camera s, and R (s) refers to the right-eye camera of binocular camera s:

$\begin{array}{c}\mathrm{\δ}(s,l,r)={\left|\right|v\left(b_1\right)-v\left(L\left(s\right)\right)\left|\right|}^2+{\left|\right|v\left(b_r\right)-v\left(R\left(s\right)\right)\left|\right|}^2\\ E_S(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\left(\mathrm{\δ}\left(S\right[i],{\mathrm{\Π}}^L[i],{\mathrm{\Π}}^R[i\left]\right)\right)\end{array}$

(2) similitude error E _Q:

γ(s，l，r)＝||q(s)-q(b ₁)|| ²+||q(s)-q(b _r)|| ²

$E_Q(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\mathrm{\γ}\left(S\right[i],{\mathrm{\Π}}^l[i],{\mathrm{\Π}}^r[i\left]\right)$

(3) direction continuity error E _CO:

$E_{\mathrm{CQ}}(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2q\left(s_i\right)-q\left(s_{i+1}\right)-q\left(s_{i-1}\right)\left|\right|}^2$

(4) position continuity error E _CV:

$E_{\mathrm{CV}}(S,{\mathrm{\Π}}^L,{\mathrm{\Π}}^R)=\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(s_i\right)-v\left(s_{i+1}\right)-v\left(s_{i-1}\right)\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^L\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^L[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^L[i-1]}\right)\left|\right|}^2$

$+\underset{i=2}{\overset{N-1}{\mathrm{\Σ}}}{\left|\right|2v\left(b_{{\mathrm{\Π}}^R\left[i\right]}\right)-v\left(b_{{\mathrm{\Π}}^R[i+1]}\right)-v\left(b_{{\mathrm{\Π}}^R[i-1]}\right)\left|\right|}^2.$

2. a kind of method from monocular image sequence generation sequence of stereoscopic images according to claim 1 is characterized in that: described fixing (∏ ^L, ∏ ^R), adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S) method can be: fixing (∏ ^L, ∏ ^R), be target function with E, use the nonlinear optimization method to adjust (Q _S, V _S), obtain making (the Q of E minimum _S, V _S).

3. a kind of method from monocular image sequence generation sequence of stereoscopic images according to claim 1 is characterized in that: described fixing (Q _S, V _S), adjust (∏ ^L, ∏ ^R), obtain making (the ∏ of E minimum ^L, ∏ ^R) method can be: fixing (Q _S, V _S), with (∏ ^L, ∏ ^R) segmentation, each all successive frames in adjusting wherein a certain section simultaneously, each ∏ simultaneously ^L[i], ∏ ^R[i] all enumerates pairing in the some frames that close on separately, all situations are recomputated departure E, finds out a group of making the E minimum as adjusting the result, and each segmentation is adjusted one by one, finishes until whole adjustment.

Images