CN101325724B - Method for correcting polar line of stereoscopic picture pair - Google Patents

Abstract

This invention discloses a polar regulating method for vertical image pair, comprising following steps: utilizing the matched point to calculate out a basic matrix and respective polar points of two images, then calculating the coordinate of the depth rotary shaft of the second image, converting proportion and translational magnitude, all of which are used as the initial value with the coordinate of the polar point of the image pair, adopting the nonlinear least square to optimize so as to obtain the optimum value; calculating out the respective projection change matrixes of two images, then converting so as to obtain the regulated vertical image pair. The optimum parameters of this invention are reduced to at least seven from at least ten of the traditional method so as to save the calculated amount. At the same time, the initial value of parameters is calculated out by the method completely based on the matrix, which ensures the precision of the initial value so as to conquer the sensitive defect to the initial value when optimized. Because the nonlinearity optimization is executed after the initial value of parameters is obtained, compared with the method completely based on the matrix, this invention does not excessively depend on the precision of the matrix.

Description

The method for correcting polar line that a kind of stereo-picture is right

Technical field

The present invention relates to the computer vision research field, specifically, relate to the right method for correcting polar line of a kind of stereo-picture.

Background technology

In the computer vision research field, stereo-picture is a key issue to the characteristic matching point of (stereo pairs) always.Its purpose is to find out the characteristic of correspondence point from two right width of cloth images of stereo-picture.Present most of matching algorithm all is based on such hypothesis prerequisite: right 2 width of cloth image characteristic of correspondence points of stereo-picture are on same scan line.So just the hunting zone with coupling has dropped to one dimension from two dimension, has accelerated matching speed, has improved matching precision.Some scholar will satisfy this hypothesis prerequisite and be called and satisfy scan line characteristic (scanline property).But, the most of stereo-pictures that obtain in the reality are not to satisfying the scan line characteristic, often there is vertical parallax in characteristic of correspondence point, and the right vertical parallax of great majority points neither an identical value, can not reach the purpose of eliminating vertical parallax by the translation image.At this situation, the method that some scholar proposes to adopt polar curve to proofread and correct (epipolar line rectification) makes stereo-picture, and two right width of cloth images satisfy the scan line characteristic.

Polar curve is proofreaied and correct be otherwise known as image rectification (image rectification), projection correction (projectiverectification), refer to two width of cloth images are respectively carried out a projective transformation, make the polar curve of two width of cloth image correspondences on same scan line, thereby satisfy the scan line characteristic.

1996, concentrated " View morphing " literary composition of SIGRAPH96 meeting paper was introduced two width of cloth images and has been satisfied the necessary and sufficient condition of scan line characteristic, and has provided a kind of method of definite projective transformation matrix in view of the above.This method is a kind of method that relies on fundamental matrix fully, and its shortcoming is the precision that too relies on fundamental matrix.If the precision of fundamental matrix is not high enough, then this method is easy to collapse.And up to the present, by the also comparatively unstable multiprecision arithmetic of match point calculating fundamental matrix of two width of cloth images.

1999, IEEE Conference on Computer Vision and Pattern Recognition meeting paper concentrates " Projective rectification without epipolar geometry " literary composition to disclose a kind of method for correcting polar line that need not fundamental matrix, this method only depends on the coordinate of images match point, utilizes linear least-squares estimation and nonlinear optimization method to calculate projective transformation matrix.When being optimization, the weak point of this method, might be absorbed in local optimum to the initial value sensitivity.

Summary of the invention

The objective of the invention is to overcome the deficiencies in the prior art, a kind of shortcoming to the initial value sensitivity, the right method for correcting polar line of stereo-picture of dependence fundamental matrix computational accuracy within reason again simultaneously can overcome optimization the time are provided.

In order to reach the foregoing invention purpose, the method for correcting polar line that stereo-picture of the present invention is right may further comprise the steps:

(1), utilize the match point of two right width of cloth images of stereo-picture to calculate fundamental matrix and limit separately;

(2), utilize the limit of first width of cloth image to calculate the degree of depth rotating shaft of first width of cloth image, again according to fundamental matrix, calculate the degree of depth rotating shaft of second width of cloth image;

(3), calculate two width of cloth images degree of depth rotation transformation matrix separately according to two width of cloth images degree of depth rotating shaft separately;

(4), calculate two width of cloth images through the degree of depth postrotational new limit, obtain two width of cloth images polar curve rotation transformation matrix separately according to new limit according to two width of cloth images degree of depth rotation transformation matrix and limit separately;

(5), two width of cloth images degree of depth spin matrix and the polar curve rotation transformation matrix separately that obtain of the fundamental matrix that obtains according to step (1), step (3), (4), obtain the fundamental matrix after two width of cloth planes of delineation become parallel plane and polar curve level, go out to become conversion ratio peaceful the move amount of second width of cloth image of two width of cloth images after parallel plane and the polar curve level according to this matrix computations again for first width of cloth image;

(6), two width of cloth images that step (1) is obtained separately the peaceful amount of moving of conversion ratio that obtains of coordinate, coordinate that step (2) obtains second width of cloth picture depth rotating shaft and the step (5) of limit as initial value, adopt nonlinear least square method that initial value is optimized, obtain their optimal value;

(7), according to the optimal value that obtains, calculate two width of cloth images projective transformation matrix separately, then two width of cloth images are carried out conversion, the stereo-picture after obtaining proofreading and correct is right.

The present invention makes when carrying out optimization owing to introduce degree of depth rotation transformation matrix when determining to carry out optimized parameter, and the number that is used for optimized parameter has reduced at least 7 by at least 10 of conventional method, thereby has saved amount of calculation.Simultaneously, the initial value of parameter is by calculating based on the method for fundamental matrix fully, guaranteed the precision of initial value, thus when having overcome optimization to the shortcoming of initial value sensitivity.Owing to obtain also will carrying out nonlinear optimization behind the initial value of parameter to it, therefore relatively complete method, the precision of the present invention's dependence fundamental matrix within reason again based on fundamental matrix.

Description of drawings

Fig. 1 is the particular flow sheet of the reverse mapping method of embodiment of the invention employing.

Embodiment

For understanding the present invention better, the present invention is more described in detail below in conjunction with embodiment.In the following description, when perhaps the detailed description of existing prior art can desalinate subject content of the present invention, these were described in here and will be left in the basket.

In the present embodiment, two right width of cloth images are respectively I to set up volume image ₀And I ₁, F is its corresponding fundamental matrix.H ₀, H ₁Represent I respectively ₀, I ₁Corresponding projective transformation matrix.If I ₀And I ₁Limit be respectively e ₀And e ₁, u _0jAnd u _1jThe homogeneous coordinates of representing a pair of match point, j=1 ..., N.

(1), according to the match point u of two width of cloth images _0jAnd u _1j(j=1 ..., N, altogether N to) adopt improved 8 algorithms to calculate fundamental matrix F, first width of cloth image I ₀Limit

With second width of cloth image I ₁Limit

E wherein ₀ ^x, e ₀ ^y, e ₁ ^x, e ₁ ^yThe expression homogeneous coordinates value of trying to achieve, subscript x, y represent coordinate on the x axle and the coordinate on the y axle respectively, T represents transposition.

(2), according to first width of cloth image I ₀Limit e ₀Coordinate figure, the coordinate that calculates its degree of depth rotating shaft is:

$d_0^x=\frac{e_0^y}{\sqrt{{\left(e_0^x\right)}^2+{\left(e_0^y\right)}^2}},$ $d_0^y=\frac{{-e}_0^x}{\sqrt{{\left(e_0^x\right)}^2+{\left(e_0^y\right)}^2}}$

With calculate d ₀ ^xAnd d ₀ ^yThe substitution formula 1.

$\left[\begin{array}{c}m\\ n\\ 1\end{array}\right]=F{\left[\begin{array}{ccc}{d}_0^x& d_0^y& 0\end{array}\right]}^T$ ①

According to m, the n that 1. formula obtains, 2. the substitution formula calculates second width of cloth image I ₁The coordinate of degree of depth rotating shaft.

$d_1^x=\frac{n}{\sqrt{m^2+n^2}},$ $d_1^y=\frac{-m}{\sqrt{m^2+n^2}}$ ②

In the present embodiment, also further will calculate second width of cloth image I ₁The coordinate d of degree of depth rotating shaft ₁ ^xAnd d ₁ ^y3. the substitution formula obtains angle parameter α:

$\mathrm{\α}=\arctan \frac{d_1^y}{d_1^x}$ ③

(3), according to two width of cloth image I that calculate in limit coordinate that calculates in the step (1) and the step (2) ₀, I ₁The coordinate d of degree of depth rotating shaft separately ₀ ^x, d ₀ ^yAnd d ₁ ^x, d ₁ ^y, 4. according to formula

${\mathrm{\θ}}_i=\arctan \frac{1}{d_i^y{e}_i^x-d_i^x{e}_i^y},(i=\mathrm{0,1})$ ④

Calculate two width of cloth image I ₀, I ₁Degree of depth anglec of rotation θ separately ₀And θ ₁Value.

Then 5., according to formula

$R_{{\mathrm{\θ}}_i}^{d_i}=\left[\begin{array}{ccc}{\left(d_i^x\right)}^2+{\left(d_i^y\right)}^2\cos {\mathrm{\θ}}_i& d_i^x{d}_i^y(1-{\cos \mathrm{\θ}}_i)& d_i^y\sin {\mathrm{\θ}}_i\\ d_i^x{d}_i^y(1-\cos {\mathrm{\θ}}_i)& {\left(d_i^y\right)}^2+{\left(d_i^x\right)}^2\cos {\mathrm{\θ}}_i& -d_i^x\sin {\mathrm{\θ}}_i\\ -d_i^y\sin {\mathrm{\θ}}_i& d_i^x\sin {\mathrm{\θ}}_i& \cos {\mathrm{\θ}}_i\end{array}\right],(i=\mathrm{0,1})$ ⑤

Calculate two width of cloth image I ₀, I ₁Degree of depth rotation transformation matrix separately

With

(4), according to two width of cloth image I ₀, I ₁Degree of depth rotation transformation matrix separately With With limit e ₀And e ₁6. calculate two width of cloth image I according to formula ₀, I ₁Through the postrotational new limit e ' of the degree of depth ₀And e ' ₁

$e_i^{\′}={\left[\begin{array}{ccc}{e}_i^{\′x}& e_i^{\′y}& 0\end{array}\right]}^T=R_{{\mathrm{\θ}}_i}^{d_i}{e}_i,(i=\mathrm{0,1})$ ⑥

Then, 7. according to formula

${\mathrm{\φ}}_i=-\arctan \frac{e_i^{\′y}}{e_i^{\′x}},(i=\mathrm{0,1})$ ⑦

Obtain two width of cloth image I ₀, I ₁The anglec of rotation φ of polar curve ₀, φ ₁, by formula 8.:

$R_{{\mathrm{\φ}}_i}=\left[\begin{array}{ccc}{\cos \mathrm{\φ}}_i& -{\sin \mathrm{\φ}}_i& 0\\ {\sin \mathrm{\φ}}_i& {\cos \mathrm{\φ}}_i& 0\\ 0& 0& 1\end{array}\right],(i=\mathrm{0,1})$ ⑧

Get two width of cloth image I ₀, I ₁Polar curve rotation transformation matrix

(5), two width of cloth images degree of depth spin matrix separately that obtains of the fundamental matrix F that obtains according to step (1), step (3), (4)

With

With polar curve rotation transformation matrix

Obtain the fundamental matrix F ' after two width of cloth planes of delineation become parallel plane and polar curve level:

$F^{\′}=R_{{\mathrm{\φ}}_1}{R}_{{\mathrm{\θ}}_1}^{d_1^{\′}}F{R}_{-{\mathrm{\θ}}_0}^{d_0}{R}_{-{\mathrm{\φ}}_0}$ ⑨

Become second width of cloth image I of two width of cloth images after parallel plane and the polar curve level again according to this matrix F ' calculate ₁For the first sub-picture I ₀Conversion ratio a and translational movement b.Be specially:

It is not 0 scale factor that F ' is multiplied by one, makes the element of its 3rd row the 2nd row equal 1.The opposite number of the element of the 2nd row the 3rd row of the matrix behind the convergent-divergent is conversion ratio a, and the opposite number of the element of the 3rd row the 3rd row is translational movement b.If a＜0 o'clock illustrates that then the order up and down of polar curve is opposite in two width of cloth images, just image has been put upside down, this moment anglec of rotation Ф ₁Needing increases π, promptly uses Ф ₁+ π replaces original Ф ₁Recomputate.

(6), two width of cloth images that step (1) the is obtained homogeneous coordinates e of limit separately ₀ ^x, e ₀ ^y, e ₁ ^x, e ₁ ^yAnd step (2) obtains second width of cloth image I ₁The coordinate d of degree of depth rotating shaft ₁ ^xAnd d ₁ ^yAnd the conversion ratio a that obtains of step (5) and translational movement b adopt nonlinear least square method that initial value is optimized as initial value, obtains their optimal value.

In the present embodiment, angle parameter α is replaced second width of cloth image I ₁The coordinate d of degree of depth rotating shaft ₁ ^xAnd d ₁ ^y

$\mathrm{\α}=\arctan \frac{d_1^y}{d_1^x}$

Like this, finally obtained the initial value of 7 parameters, they are α, a, b, e ₀ ^x, e ₀ ^y, e ₁ ^xAnd e ₁ ^yIf target function is:

$f(H_0,H_1)=\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left[\begin{array}{c}{\left(H_1{u}_{1j}\right)}^{\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="H2008100456220E00011.png"\; state="\{\{state\}\}">T</figure-callout>}}\begin{array}{c}\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& -1\\ 0& 1& 0\end{array}\right]\left(H_0{u}_{0j}\right)\end{array}\end{array}\right]}^2$ ⑩

Wherein:

$H_0=R_{{\mathrm{\&phi;}}_0}{R}_{{\mathrm{\&theta;}}_0}^{{\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="H2008100456220E00011.png"\; state="\{\{state\}\}">d</figure-callout>}}_0}$

$H_1={\mathrm{TR}}_{\&PlusMinus;{\mathrm{\&phi;}}_1}{R}_{{\mathrm{\&theta;}}_1}^{d_1^{\&prime;}}$

$T=\left[\begin{array}{ccc}1& 0& 0\\ 0& a& \mathrm{<figure-callout\; id="0"\; label="b"\; filenames="H2008100456220E00011.png"\; state="\{\{state\}\}">b</figure-callout>}\\ 0& 0& 1\end{array}\right]$

10. adopt the Levenberg-Marquardt algorithm that f is minimized the initial value substitution formula of 7 parameters, obtain the optimal value of these parameters.

(7) according to 7 parameter alpha, a, b, the e that obtain ₀ ^x, e ₀ ^y, e ₁ ^xAnd e ₁ ^yOptimal value, calculate two width of cloth image I ₀, I ₁Projective transformation matrix H separately ₀And H ₁, then to two width of cloth image I ₀, I ₁Carry out conversion, the stereo-picture after obtaining proofreading and correct is to H ₀I ₀, H ₁I ₁

Obtain projective transformation matrix H ₀And H ₁Afterwards, to two width of cloth image I ₀, I ₁Image after the enforcement conversion just can obtain proofreading and correct is to H ₀I ₀, H ₁I ₁Can make new images to H but do like this ₀I ₀, H ₁I ₁In some point can not be at original image I ₀, I ₁In find corresponding point, thereby produce " cavity (holes) " phenomenon.

In the present embodiment, the projective transformation matrix H that obtains in the step (7) ₀And H ₁To two width of cloth image I ₀, I ₁Implement conversion, the stereo-picture after adopting the method for reverse mapping to obtain proofreading and correct is right, thereby eliminates cavitation.

Fig. 1 is the particular flow sheet of reverse mapping method, and step is as follows:

(a), two width of cloth image I ₀, I ₁Respectively with separately projective transformation matrix H ₀, H ₁Multiply each other, obtain stereo-picture H ₀I ₀, H ₁I ₁

(b), find the solution the H of image separately that obtains after multiplying each other ₀I ₀, H ₁I ₁Minimum boundary rectangle Rect ₀, Rect ₁

(c), to minimum boundary rectangle Rect ₀, Rect ₁Carry out translation respectively, make the point in its upper left corner move to the initial point of coordinate system separately; Then, compare two width of cloth image H ₀I ₀, H ₁I ₁Minimum boundary rectangle at the translational movement Δ y of vertical direction ₀, Δ y ₁, the minimum boundary rectangle that translational movement is little in vertical direction translation again, makes the translational movement of the big minimum boundary rectangle of its translational movement and translational movement identical.Δ y even ₀＜Δ y ₁, Rect then ₀At vertical direction translation Δ y again ₁-Δ y ₀Otherwise Rect ₁At vertical direction translation Δ y ₀-Δ y ₁

(d), with from left to right, from top to bottom order, calculate each the some u in the minimum boundary rectangle of two width of cloth images ₀', u ₁' at original image I separately ₀, I ₁Corresponding points u in the plane ₀, u ₁, check corresponding points u ₀, u ₁Whether drop on original image I ₀, I ₁In the scope; If, then with original image I ₀, I ₁Corresponding points u in the plane ₀, u ₁Corresponding pixel copies corresponding points u in the minimum boundary rectangle to ₀', u ₁' the position; Otherwise with corresponding points u in the minimum boundary rectangle ₀', u ₁' become black color dots, it is right to obtain reverse stereo-picture of hinting obliquely at after the correction.

Image H after the correction ₀I ₀, H ₁I ₁With original image I ₀, I ₁Compare, size may change.

Table 1 is the correction accuracy contrast table that method for correcting polar line of the present invention is compared with Francesco method for correcting polar line of the prior art, Mallon method for correcting polar line.

The bearing calibration correction accuracy

Francesco?Method 0.75000

Mallon?Method 0.62692

Method for correcting polar line 0.25424 of the present invention

Table 1

Adopt in the table 1 and proofread and correct back two width of cloth image H ₀I ₀, H ₁I ₁The mean value of the vertical parallax of characteristic of correspondence point is weighed correction accuracy.From table 1 we as can be seen, compare with Francesco method, Mallon method, correction accuracy improves a lot.

Although above the illustrative embodiment of the present invention is described; but should be understood that; the invention is not restricted to the scope of embodiment; to those skilled in the art; as long as various variations appended claim limit and the spirit and scope of the present invention determined in; these variations are conspicuous, and all utilize innovation and creation that the present invention conceives all at the row of protection.

Claims (5)

1. the method for correcting polar line that stereo-picture is right is characterized in that, may further comprise the steps:

(1), utilize the match point of two right width of cloth images of stereo-picture to calculate fundamental matrix and limit separately;

(2), utilize the limit of first width of cloth image to calculate the degree of depth rotating shaft of first width of cloth image, again according to fundamental matrix, calculate the degree of depth rotating shaft of second width of cloth image;

(3), calculate two width of cloth images degree of depth rotation transformation matrix separately according to two width of cloth images degree of depth rotating shaft separately;

(4), calculate two width of cloth images through the degree of depth postrotational new limit, obtain two width of cloth images polar curve rotation transformation matrix separately according to new limit according to two width of cloth images degree of depth rotation transformation matrix and limit separately;

(5), two width of cloth images degree of depth rotation transformation matrix and the polar curve rotation transformation matrix separately that obtain of the fundamental matrix that obtains according to step (1), step (3), (4), obtain the fundamental matrix after two width of cloth planes of delineation become parallel plane and polar curve level, the fundamental matrix that obtains according to this step calculates second width of cloth image that becomes two width of cloth images after parallel plane and the polar curve level peaceful amount of moving of conversion ratio for first width of cloth image again;

(6), two width of cloth images that step (1) is obtained separately the peaceful amount of moving of conversion ratio that obtains of coordinate, coordinate that step (2) obtains second width of cloth picture depth rotating shaft and the step (5) of limit as initial value, adopt nonlinear least square method that initial value is optimized, obtain their optimal value;

(7), according to the optimal value that obtains, calculate two width of cloth images projective transformation matrix separately, then two width of cloth images are carried out conversion, the stereo-picture after obtaining proofreading and correct is right.

2. the right method for correcting polar line of stereo-picture according to claim 1 is characterized in that, what the described match point by two right width of cloth images of stereo-picture of step (1) calculated the fundamental matrix employing is improved 8 algorithms.

3. the right method for correcting polar line of stereo-picture according to claim 1 is characterized in that, the coordinate of described second width of cloth picture depth rotating shaft of step (6) further is calculated as angle parameter as initial value.

4. the right method for correcting polar line of stereo-picture according to claim 1 is characterized in that, the described nonlinear least square method of step (6) is the Levenberg-Marquardt algorithm.

5. the right method for correcting polar line of stereo-picture according to claim 1 is characterized in that, step (7) is described carries out conversion to two width of cloth images, may further comprise the steps:

(a), two width of cloth images are multiplied each other with separately projective transformation matrix respectively;

(b), find the solution the minimum boundary rectangle of the image separately that obtains after multiplying each other;

(c), the minimum boundary rectangle of image is separately carried out translation respectively, make the point in its upper left corner move to the initial point of coordinate system separately; Then, relatively the minimum boundary rectangle of two width of cloth images is at the translational movement of vertical direction, and the minimum boundary rectangle that translational movement is little in vertical direction translation again, makes the translational movement of the big minimum boundary rectangle of its translational movement and translational movement identical;

(d), with from left to right, from top to bottom order, calculate the corresponding points of each point in original image plane separately in the minimum boundary rectangle of two width of cloth images, check whether corresponding points drop in the original image scope; If, then the pixel of the corresponding points correspondence in the original image plane is copied to the position of corresponding points in the minimum boundary rectangle; Otherwise corresponding points in the minimum boundary rectangle are become black color dots, and the stereo-picture after obtaining proofreading and correct is right.

Images